Script_grep / script_read serve stale script source --- (line numbers and content lag the live editor document)

Bug Reports Studio Bugs
1

Expected behavior

Bug report: script_grep / script_read serve stale script source (line numbers and content lag the live editor document)

Summary

The Studio MCP Proxy tools script_grep and script_read can return an outdated snapshot of a script’s source instead of the current editor document. Line numbers and content are sometimes hundreds of lines behind ScriptEditorService:GetEditorSource() (the ground truth), and in one case the two tools disagreed with each other on the same script. Any client that trusts these results for follow-up edits (e.g. multi_edit with line anchors, or splices computed from grep hits) will corrupt the script.

Environment

  • Roblox Studio 0.730.0.7300790 (Edit mode)
  • Windows 11 Home 10.0.26200
  • Studio MCP Proxy (official Roblox Studio MCP server) driven by Claude Code
  • Scripts affected were large ModuleScripts/LocalScripts (several thousand lines) open in Studio editor tabs and edited during the session via ScriptEditorService:UpdateSourceAsync

Steps to reproduce

  1. Open a place with a large script and open that script in a Studio editor tab.
  2. Insert or remove a few hundred lines — either by typing in the editor or via ScriptEditorService:UpdateSourceAsync from execute_luau — without saving/publishing the place.
  3. Call script_grep for a string near the end of the script.
  4. Compare the returned line numbers/content against execute_luau:
    local src = game:GetService("ScriptEditorService"):GetEditorSource(script)
return select(2, src:gsub("\n", "")) + 1 -- live line count

Expected: script_grep/script_read results match the live editor source.

Actual: results reflect an older revision of the script; line numbers are offset by the size of the unsynced edits.

Observed occurrences

Date Script Symptom
2026-07-07 Large ModuleScript (NPCManager), edited during session Both script_read and script_grep were ~370 lines behind the editor source.
2026-07-10 LocalScript (EditorTray) ~700 lines behind; script_grep and script_read returned different stale revisions of the same script. Other scripts in the same place (recently created ones) were fresh at the same moment — staleness is per-script, not global.
2026-07-14 LocalScript (SelectionController), different place script_grep returned stale line numbers/content while script_read matched GetEditorSource exactly — only the grep index was stale this time.

Pattern / suspected cause

  • Staleness correlates with scripts that are open in an editor tab and have been modified during the session (typed edits or UpdateSourceAsync). It looks like the search index (and sometimes the read path) is built from the saved/committed Source and is not invalidated when the open editor document changes.
  • The 2026-07-10 case where script_grep and script_read disagreed with each other suggests they use separate caches with independent (and independently broken) invalidation.
  • Related but possibly intended: in Play mode the search tools query only the focused (Client) DataModel, so ServerScriptService scripts are invisible to search until Play is stopped. If intended, it would help if the tools said so instead of returning no hits.

Impact

  • Line-anchored follow-up edits (multi_edit, splices computed from grep results) land in the wrong place and corrupt scripts.
  • Agents/automation cannot trust the search tools at all after any edit, and must bypass them entirely.

Current workaround

Skip script_grep/script_read for any script that may have been edited, and search live via execute_luau:

local ses = game:GetService("ScriptEditorService")
local src = ses:GetEditorSource(theScript)
local i = string.find(src, "needle", 1, true) -- plain-text find against ground truth

This works but defeats the purpose of the dedicated search tools and burns tokens re-reading sources manually.

Suggested fixes

  1. Serve script_read and the script_grep index from ScriptEditorService:GetEditorSource() (live document) rather than the saved Source, or invalidate the index on editor-document change.
  2. Whatever the source of truth, make both tools read from the same snapshot so they can never disagree.
  3. Expose a freshness signal (e.g. a document revision/hash in tool output) so clients can detect a stale index instead of silently acting on wrong line numbers.
3

The algorithm does not consider whether your experience was made with a generative model and it should stay that way. There’s not a real way to dictate “quality” as that’s a subjective metric. What Roblox’s algorithms do is boost experiences based on player metrics that indicate user spending, retention, onboarding, etc.

Many people consider that media made with AI is of a significantly lower quality than those made entirely by human input. By that reasoning, they would argue that games made specifically made by AI models should be tagged as such and stifled. Personally, I think we shouldn’t be boosting or deboosting these games and let players decide.

2 Likes
4

they recently had changed their algorithm to focus on certain signals such as bounce rate (if they leave ur game immediately) over CTR and focus on long-term retention and such.

They also made it so that for your game to be considered in the algorithm at all you need to have a active user base and to be “established.” Only then will they actually start recommending your game – basically when they figure out what your games about – and they do so relative to your competition (no longer on independent stats)

The point I’m trying to make is that Roblox has shifted from “We want to see if your game is good” to “We have too many games – which of our existing games is good.” They don’t consider your game in the algorithm unless it knows exactly what your games about and has strong proven signal because it now relies on peer comparison.

So its not whether its being “boosted” or “deboosted” but rather whether its even part of the game mixing pot

Thats a problem because it worsens their cold start problem with games. roblox’s has shifted their mission from creator democratizaton and accessibility to player curation and protection. The end goal is good but democratization and accessibility was the reason most devs started in the first place. Now its the same as steam where the marketting is entirely yourself until you can lift your boostraps high enough to lift off.

By the way the phrase “pull yourself up by your bootstraps” is a 1800s joke from the working class on the realities of wealth inequality because you cant literally pull yourself up by your boostraps.

5

Honestly, I just think its too hard of a metric to define. People use Roblox’s assistant, but also use outside models with their MCP and with this theres really no way to tell which model a game was developed with. I’m not against game’s being tagged with an AI label or their specific model, even if it’s an internal signal, but someone can make garbage with any model just as easily as they could make garbage without AI assistance. I’m not sure something like this would indicate quality.

Cold starts are definitely a problem with games though. Ad campaigns within the system can only get you so far. I wish there was a tutorial about how to gain traction / a fanbase but honestly more than often I feel like this is something that takes time, commitment, and even a little luck. I don’t think it’s a good mindset to expect an overnight success.

6

Roblox shifted their strategy so that currently there is no way to have players play your game without external marketting. To answer your question about a tutorial on getting players the only 3 options now are

  1. Become a content creator / post social media on your game
  2. Run ad campaigns
  3. Know a existing Youtuber or developer or somehow contact them.

The old method was that roblox gave you players naturally as a talent scout. Now its a quality triage that would rather pick a careful curation than to boost underrated / new games.

This is just like how other platforms work like Steam and Apple… except for the fact that roblox takes ~75% of your games earnings.

7

I’m not sure this is entirely true. While Roblox does indicate what they are boosting more and less in posts about their recommendation algorithm, games are still recommended to players based on what their friends play and Roblox’s recommendation algorithm isn’t an entirely known variable. I’ve seen posts of people getting large CCUs for their games without any advertisement since the latest change alongside people who have needed ads both internally and externally to boost their games to the audiences they want.

8

I’m going to research it a bit more to make sure i dont get anything wrong

9

It seems like my previous statements contained some misinformation and some truths.

Roblox states the first step is per user: it curates game specifically that align with that users taste. This comes from signals from basically things the user interacts with to learn their interests. Specifically ads, search, charts, friends, teleports, etc.

Then, they rank games in that category mainly based on:

  1. CTR
  2. bounce rate
  3. How many days they play
  4. playtime

Roblox also states they specifically compare your game to similar games, and if a similar game is significantly more successful they will deboost you.

Honestly though this system has the same flaw I was saying earlier, and my cutting edge Roblox game getting ignored is a live example of its failure.

Both steps 1 and 2 require existing signals about your game – mostly about how people play it. But if you have no players then Roblox has no alternative way to distinguish your game. Thats a existential problem for Roblox Especially if they’re betting MCP created games will recover their stock.

I’m a live example of the creators they’re betting on: I created an extremely high quality cutting edge roblox game using roblox’s MCP, and even with a front page game icon roblox will not even test it. The issue is that the algorithm doesn’t even give your game a chance anymore unless you have significant existing signal.