Native code generation fails with "internal lowering failure"

When using --!native on functions with many local variables combined with read-modify-write patterns, native code generation fails with an internal lowering error and the script falls back to interpreted mode.

Studio Version: 0.700.0.7000935
Affected Architectures: Both AMD and Intel (x64)

Error Message
Native code generation of script =ServerScriptService.Script failed:
Native code generation failed to compile some of the module functions:
Function ‘kMul’ at line 2 encountered an internal lowering failure
. Script will be interpreted.

Root Cause
Based on examining the Luau source code, this appears to be a bug in the dead store elimination (DSE) optimization pass in “OptimizeDeadStore.cpp”.

The DSE incorrectly marks STORE_DOUBLE instructions as dead when a variable is stored, then later updated using its own value. For example, “a00 = a00 + expr” generates the following IR:

STORE_DOUBLE R0, value1    -- first store
...
LOAD_DOUBLE R0             -- reads from R0
ADD_NUM                    -- computes new value  
STORE_DOUBLE R0, result    -- second store

The DSE sees two stores to R0 and incorrectly kills the first store, failing to recognize that the second store’s value depends on reading the first store’s result.

Repo

--!native
local function kMul(smallK)
    local a00, a01, a02, a03, a04, a05, a06, a07, a08, a09, a10, a11 = 1,2,3,4,5,6,7,8,9,10,11,12
    local c00, c01, c02, c03, c04, c05, c06, c07, c08, c09, c10, c11
    a09 = a09 * smallK
    a10 = a10 * smallK
    a11 = a11 * smallK
    c11 = a11 + 3 * 2 ^ 306 - 3 * 2 ^ 306
    a00 = a00 + 19 / 2 ^ 255 * c11
    c00 = a00 + 3 * 2 ^ 73 - 3 * 2 ^ 73
    a01 = a01 + c00
    c01 = a01 + 3 * 2 ^ 94 - 3 * 2 ^ 94
    a02 = a02 + c01
    c02 = a02 + 3 * 2 ^ 115 - 3 * 2 ^ 115
    a03 = a03 + c02
    c03 = a03 + 3 * 2 ^ 136 - 3 * 2 ^ 136
    a04 = a04 + c03
    c04 = a04 + 3 * 2 ^ 158 - 3 * 2 ^ 158
    a05 = a05 + c04
    c05 = a05 + 3 * 2 ^ 179 - 3 * 2 ^ 179
    return {
        a00 - c00 + 19 / 2 ^ 255 * c11,
        a01 - c01,
        a02 - c02,
        a03 - c03,
        a04 - c04,
        a05 - c05,
        a11 - c11,
    }
end
print(kMul(121666))

Additional Context

This info was sent to me (not my own research) and apparently Keccak is also affected, though I was not able to reproduce the issue on my machine.

2 Likes

(additional context: bug was introduced in 0.616 and persists through to latest)

1 Like

Hello and thank you for the report.

Native code generation has some limits on code complexity that it can handle.
In this case you are unfortunately hitting one of those limits (too many live variables that need to be spilled to allocate a table).

This is not a DSO bug and it was not ‘introduced’, there are no issues with the DSO pass here.

Of course we aim to support more complex patterns and for people not to hit surprising failures which degrade performance.
We will be working to solve the limit in this example and it’s great to have a clear repro, thank you.

1 Like

Possibly fixed? → Sync to upstream/release/702 (#2126) · luau-lang/luau@c836feb · GitHub