AI-generated Package Diff Summaries

Hey Creators,

We are excited to share that you can generate an AI-powered summary of package changes in the Package Diff window. This should help you collaborate more effectively on packages by providing insight into the key changes before you dive deeper into the details of the package structure.

Getting Started

To use this feature, right-click on a package (in Viewport or Explorer) and select Compare Versions to open the Package Diff Window. You can use this feature for packages you are able to access.

Then inside the package diff window, select two versions of your package,

Next, click on the Change Summary dropdown toggle to generate the summary.

After a few seconds, you should see a summary, generated using our language models based on your selected versions. This summary is generated by ingesting the package structure at the two versions you selected and summarizing their key differences in terms of scripts, properties and other attributes. This way you can get a sense of what has changed without needing to inspect every child of the package.

This is our first step at summarizing changes with AI and we plan to expand this AI functionality to other aspects of Studio in the future. Please give us your feedback about this capability and what you’d like to see next.

Big shoutout to the team that brought this to life: @code4xp, @s1lly_g00se8, @Regal_Corgi, @yohooyohoo & @idevride

Thank you!

187 Likes

This topic was automatically opened after 10 minutes.

This is amazing and really helpful! Question is, are you using the same AI version as the coding assistant? Other then that really useful resource and will definitely help with updating packages.

27 Likes

Very powerful application of this tech. At first glance though I’m a little uncertain about the accuracy of its assumptions regarding the purpose of changes. “Enhancing visual effects” and “indicating a visual overhaul” are fluffy for change notes. Maybe it’s fine in other contexts and I’m picking at it too much.

If this works retroactively for existing package versions, super cool and will take a closer look later. Love to see this AI stuff getting tied more into the datamodel.

35 Likes

I love seeing the integration of Artificial Intelegence into Roblox’s rotation with updates. I’m exited to incorporate AI into usage whilst developing! Thanks Roblox team! :+1:

17 Likes

Could this same feature be coming to the team-create script drafting mode?

24 Likes

This is very cool! I gave it a try in my game. It did a good job summarizing the most important changes. Although I do agree with @PeZsmistic it’s sometimes a bit wordy and vague. I guess it does need a bit of context via code commenting or something to improve the quality of the description. I only tried it on code changes though.

Since this tool detects differences between versions of something, it should be natural to extend it from comparing packages to comparing entire game updates? You’d treat the entire game as a single package. Not sure of the resource demands but it could make writing patch notes a lot easier since all changes are logged and summarized.

24 Likes

Actually good use of AI for once🙏

25 Likes

Definitely a very cool addition to the AI usage here. Would the next application, somehow, be a summary of team create collaborators changing stuff? Would help with tracking who changed what!

16 Likes

Can we get this same type of logs for game updates? It’d be useful.

17 Likes

Am I the only one that doesn’t like this? It’s bloated. I don’t need flourish or good writing when I see differences. I just want to see the differences and a brief explanation if needed. I can literally see in the last screenshot that it’s repeating itself.

25 Likes

Not too keen on the wording here. I don’t need the AI to tell me that something “enhances visual effects” or “received a visual overhaul”. IMO it’d be better if it were more focused on just saying what changed.

16 Likes

Honestly, I just played with this to get an idea of the change summary, and I am rather shocked at it’s discernment. Hat’s off this is pretty handy

11 Likes

Does the AI learn based off of our packages’ content?
I ask this question from a privacy and asset ownership prespective.

13 Likes

Some cool stuff! Will we ever get something like this to compare two script versions using Drafts?

12 Likes

Finally a good use of AI, need more stuff like this, not text prompt generation doohickey.

11 Likes

Currently, everything you upload to Roblox is completely public (minus sounds).

There is no such thing as asset privacy or ownership on Roblox as of now. You still retain the copyrights, but nothing is technologically private. They just had an announcement about this recently, that they will be implementing asset privacy for all kinds of assets (meshes, images, etc), but that’s set to release in 2025.

AI learning off of peoples assets is usually considered a bad thing because it tries to recreate peoples assets. This AI is not doing that, its creating descriptions of assets, not trying to recreate them. This use of AI is one of the better use cases of AI, and actually helps developers.

10 Likes

I’m not too keen on letting AI create summaries of code that I publish to packages under my groups. For example I have a global banlist that is published as a package, the last thing I need is for people to read the updates of my script and for it to show something along the lines of that "The code was updated to kick people more people" and for that to be taken the wrong way by collaborators or others. Is there a way to disable AI summaries from being generated or just write are own summaries of the changes made?

11 Likes

Very appreciated, this makes my life a whole lot less stressful.

10 Likes

Hi,

Thank you for checking this out. We’re currently biased towards briefer summaries versus more words and accuracy on the assumption that for higher accuracy you can open up the diff tree.

The summaries are generated on the fly (another reason to go for brevity).

10 Likes