Website/Engine(?) Bug: Recent Changes to Uploaded Audio Compression/Processing Causing Timing Drift
As of the time of the recent Audio Update (and possibly some time before), the compression/processing of audio on Roblox’s side after being uploaded to the library appears to have changed. This has caused the timing of audio files on the platform to not match their original file equivalents in an inconsistent manner, especially for longer audios.
A background:
Click to expand
Most pianos on Roblox based on the NickPatella implementation use a set of 6 audio files collectively referred to as a “soundfont” in order to play note sounds. Each audio is comprised of 8-second-long notes from a pair of keys. For instance, the first audio in this set contains 8-second-long C/C# notes, such that the audio begins with 8 seconds of a C2 note, followed by 8 seconds of a C#2 note, 8 seconds of a C3 note, so on and so forth until reaching the end of the range of a 61-key keyboard (C7).
This allows for the TimePosition of notes to be easily calculated: C2 is located at Sound.TimePosition = 0, C#2 at Sound.Timeposition = 8, C3 at Sound.Timeposition = 16, etc., and this format is likewise used for the rest of the note pairs in the soundfont (D/D#, E/F, F#/G, G#/A, and A#/B). This format is reliant, however, on the audio being in their assigned TimePositions – any deviation creates clicking, popping, a noticeable delay before any sound, or any combination of the three.
The errors caused by this deviation are referred to as “desync”, as the sound is no longer consistently “synced” with other notes in the same soundfont. Uploading soundfonts to the platform for use in production has always introduced a certain amount of desync due to (what is presumed to be) compression/processing of the audio before being stored on Roblox servers. This desync is definitely added by Roblox, as testing soundfonts in Studio using local files prior to uploading shows no desync whatsoever.
In prior years, desync was handled by adding an additional offset to the Sound.TimePosition, which was possible due to the amount/level of desync to the soundfont being mostly consistent. As of late, however, I and other developers have noticed a change in that desync added is now markedly inconsistent and makes the Sound.TimePosition method nearly impossible to use properly, even when trying to account for it. This has the effect of the sound of a note not matching up with the exact moment it is expected to be played, which becomes extremely obvious when playing chords or notes in consistent succession.
Reproduction steps (see @lexandstuff’s post below for a simpler repro):
-
Download and extract the original audio files from the .zip below:
soundfiles.zip (7.0 MB) -
Place the extracted files (1.mp3 through 6.mp3) into the /content/sounds folder of the Studio version being used. These are the original asset files prior to being uploaded to the platform.
-
Download and open the following reproduction place:
compressionreport.rbxl (42.9 KB) -
Start a Play Here session and open Output. Using the gui, you can toggle what keys to play simultaneously and see their set
TimePositionin the output. Clicking the “Asset:” button will toggle which sound assets to play (between Uploaded and Local). Clicking the “Play Selected” button will play all selected notes simultaneously. -
As an example, select C2 and E2 and hit “Play Selected”. The sounds should play simultaneously for both Uploaded and Local assets. The Output should show that the
TimePositionof both notes are 0.
-
Next, select C2 and F2 and hit “Play Selected”. The sounds should play simultaneously for Local assets, but a delay (desync) should be noticeably present for Uploaded assets. The Output should show that the
TimePositionof F2 is being set to 8 as expected.
-
Testing various combinations should show that Uploaded assets contain varying and inconsistent amounts of timing drift at all nonzero
TimePositionvalues, even for sounds at the sameTimePosition(e.g. C#3 & G3, which are bothTimePosition = 16).
A video demonstration of the steps above can be viewed below (sans Output):
robloxapp-20220329-0124393.wmv (654.0 KB)
Expected Behavior:
No significant delay/desync should occur when settings the TimePosition of sounds to specific values where they are expected to be synced, as with the playing of Local assets in the reproduction place.
Actual Behavior:
Delay/desync is present for all recently uploaded sound assets in varying, inconsistent amounts in both Studio and production games.
Issue Area: Roblox Website(?)
Issue Type: Other
Impact: Medium
Frequency: Constantly
Date First Experienced: 2022-03-23
Possible workarounds:
- Reuploading files in different bitrates and file formats
- Did not work, desync was still present
- Attempting to account for desync by calculating & adding additional offset
- Desync is present in inconsistent amounts all throughout, making this virtually impossible
- Uploading 61/88 individual audio files to avoid setting
TimePositionentirely- Currently untenable: common audio upload limits are 10/100, a disproportionate amount of work is required, and easily affected by wrongful moderation
cc: @Extragold, @Seenall, @angelkireina, & @Disformed_Potato on Roblox for additional info regarding this issue.
