Feedback on Integrated AI that can manipulate objects in-game

For the past week or so, I’ve been extremely hard at work on this experiment of mine.

Using OpenAI’s API, and my cloud provider, Heroku, I was able to customize and make my own Artificial Intelligence, implement it in-game, and I even made it so that it could manipulate and place objects.

At the time of writing this, it can respond to player input and chat, as well as place any objects from ReplicatedMemory, into the game, just from a single prompt.

In the future, I plan to train my model so that it can put objects together to form a model, such as a house, or a car, or a boat. With enough training, I can even have it build modern architectural structures or your iconic structures such as the Eiffel tower, in a Roblox game, with a single prompt!

I’m incredibly excited to explore and test the limits of generative AI within Roblox, even though it can be extremely difficult, this insatiable hunger of mine will not rest. Only thing I worry about is building and modeling in-game, to train the AI, because I’m not so good within those fields, but maybe there’s a workaround so I don’t need builds!

Are there any interesting ideas you think I should explore? Am I going in a good direction? Any feedback on my prototype would be appreciated!

8 Likes