LLM Interactive Agent with memory, reasoning & vision

I’m proud to announce I think I am working on something pretty cool! It uses Qwen’s 32B model and supports any OpenAI-compatible API. Source code will probably be released once it is completed.

It can call itself to process complex thought processes, execute articulate lua code to give precise math results, do facial expressions, emotes, jump, follow, turn, move to, welcome people, and even vision (configurable custom context on objects, tell directions, etc.)




9 Likes

thats cool! did you train the ai yourself or did you just use and customize something like google gemini api or chatgpt api, altough i think it probably is custom trained

1 Like

HI! It uses the Qwen 2.5 weights, but the prompts are handcrafted and it’s divided into modules that each provides actions the agent can make or adds passive context into each time the api is called

1 Like

Is this open-source? I would like to try this into roblox studio.

Hi, source code will be released by this month, we need to finish pathfinding and seat interaction

2 Likes

Alrighty then. I love to be interested into testing these little interactive agents. Privately.

Any updates on this? Would love to take a look.

Hi! The testing game has been moved to

Unfortunately, we have other projects on our backlog (like servers in roblox, web servers, audio mixers, etc.), so it will take a while before development is completely finished

1 Like

Looks cool! Do the agents produce structured outputs or is this all done through prompting and string manipulation?

Any chance at this being open source?

Hi! Everything is done using string manipulation and a Qwen Instruct model. It would be very much possible to structure this in a cleaner way with support for tool calls and strucutred output

The source code will be released when I have time to lol, then there’s also pathfinding and other to add