Code Assist Beta: AI powered code completion

As @BobbieTrooper said, this looks extremely promising, but sometimes, it takes an extremely long time for AI to generate responses, and 45% of the time, it just spams the same line. (Or creates something that doesn’t fit the intended prompt) I definitely assume there are going to be improvements, as it is a beta, but right now, it’s mostly unusable. I’m definitely not disappointed because I didn’t expect anything in the first place, just kind of hoping it gets improved. Thanks hotkey too, very helpful.

Any plans to expand the dataset of the AI?

19 Likes

I’m still not very crazy over it. It’s buggy, and generally not helpful. Over the past few days, I’ve been finding even more bugs with it:

https://devforum.roblox.com/t/ai-code-assist-material-generator-is-here-thoughts/2220430/159?u=darkpixlz



image
just to name a few of my experiences with it so far

66 Likes

In my opinion, this is a rather interesting update that allows for better learning of the Lua programming language. I’m very curious to see how it will improve and what its potential will be in the future.

13 Likes

Please add an option to disable the automatic suggestions when waiting on a line.

47 Likes

It seems to me that Code Assist is good if you give it clear requests. It’s worse when you ask it non-standard questions, or, in other words, questions that are unclear or difficult to implement.
Of course, it is still in beta testing, so there are still many flaws.

5 Likes

From the few days of testing I can clearly say that it only works if you know how to code, meaning that for beginners, this isn’t going to help… Much, at least.

As mentioned on this post, albeit recommending it to new scripters, it’s of no use because the code that the AI generates is not perfect. Additionally, the AI only writes based on what exists, so if you’re not careful, it’ll write on its own mistakes. This meaning, if you don’t have a clear knowledge of the language, what use will a new scripter give to it? They’ll expect it to be perfect, or simply won’t see mistakes.

Another problem is that the AI is extremely slow, at least compared to Copilot. For a single line of code, Copilot suggested it in <0.7 seconds, while Generative took around 3 for the same result. Is this the cooldown between line pause and suggestions, or does it take time to compute solutions?

Since GenerativeAI uses free model code, the outcome is also generally not the best. It cannot typecheck, uses deprecated globals (not looking at wait), and suggests beyond what we can call “good” code. The current version is plagued with bugs and does not promote using more performant and recommended functions (like task). It does, but not out of the box, so another no-no for what you want to achieve as a solution for beginners.

It’s a beta version, yes, it is unstable and has its issues. However, this AI really needs to be trained using a larger data collection. Free model code, as everyone knows, is not the best in the world. Say what you want, though.

Honestly speaking, you can simply enable a feature in your games that allow data (script) collection for the AI to train on. Much better than using free models. At least then we know our data is being sold.

12 Likes

Wow, I’m impressed. Reminds me of Github copilot.

Now I do wonder, does it adapt to my coding style?
Does it suggest variables, functions and whatnot similar to how I would write them?
Does it know optimization techniques?

I do a lot of manual writing and optimizations and I have a bit of an unusual coding style.
I use metatables to make classes and make modular systems, is it aware of other module scripts and whatnot?

Importantly, CAN I train it on my own code to improve it and make it adapt better?

11 Likes

I have a very very important question:
Did you send us the Alpha?
Because if this is Beta I understand why this will not replace Programmers, it’s too dumb to been create a while loop withh Hy out crashing the Game

6 Likes

Even if a stable version is released, it won’t replace scripters dude. You would take more time trying to get the AI to write the thing out than actually doing it yourself.

8 Likes

It clearly says it is in Beta release. :skull:

3 Likes

And what is the daily cap? Is there an exact number orrr?

5 Likes

Regarding FAQ item #1, please work on the safety filter. Otherwise, good job.

1 Like

It’s definitely buggy, but I am excited to see if it becomes comparable with Copilot, GhostWriter, or ChatGPT!

I have come across a few errors and it doesn’t always seem to generate prompts when I ask it to.

If I start writing something and I need to tab it will auto do it for me which is quite frustrating.

6 Likes

To ensure a high-quality experience and manage our total cost of providing Cost Assist to all creators, we are experimenting with a daily cap. It’s subject to changes. As of today, it’s primarily set around 100-200.

15 Likes

I haven’t read the full topic, and I haven’t read many of the replies. But, Currently, roblox is not using any private data to train this AI. This is fine. But when you say currently, are you implying that this is the current state, and in the future, roblox might use non public data to train this AI? Or does roblox not plan on using any private data to train the AI at all? Do you know what the plans on this are? And if roblox ever starts using private data to train the AI, will there be a way to add some type of attribute to the private data to tell roblox not to use this data to train the AI (for personal / privacy reasons, assuming that this private data comes from roblox games)? What if we have some type of secret key used to access some information, that can be detrimental to our game if someone other than ourselves know what the secret key is?

6 Likes

This is pretty cool when it has enough context to work with, I requested when the beta first enabled a few days ago that we could give the AI our own specialised training info.

The ability for it to seek through requires and DOM-aware is probably the killer feature for this. It would be nice if the AI was trained to not use deprecated methods unless there isn’t a suitable difference, for example, wait()s should be converted to task.wait() before spitting it out

Can we also have a toggle to turn it off like you can with Copilot

6 Likes

Maybe it will work like the “noai” tags; so people that don’t want their stuff to be used in/for AI training can choose to easily opt out.

I hope that they allow people to choose whether to opt-in (being opted out by default) and not be forced into a dataset.

1 Like

Turned it into a budget ChatGPT lmao.

7 Likes

I asked the AI several times with prompt engineering and it said it runs GPT-2, which has pretty poor reasoning skills. So it’s probably only going to suggest code it’s seen before and not come up with specific optimizations


(did this several times, even trying to trick it, and it always said gpt 2)

5 Likes

Interesting, I wonder if that is true.
Do keep in mind that AI can be a bit biased sometimes, some information might not be entirely true but it is certainly interesting.
I do wonder if Roblox indeed used some version of GPT-2.