This is honestly somewhat of a confusion for me as I’m not that experience in digital audio.
Before potentially spending time over-engineering something, have you tried just using PlaybackLoudness with a threshold to trigger whatever effect it is you want in time with the music? I mean, is it actually necessary to determine the BPM or sync animations to the music?
The threshold hold trigger is what I made the only issue is that it will need to change very quickly to adjust to the background music. Someone could play a remix and a remix’s background music level can range quickly. Not to mention I need a debounce method to stop “after beats”.
I wish Roblox would add an option for sound effects to affect playbackloudness as low gain has the Hertz where the average “main beat” would be located.
Edit: After experimenting I find it better to divide the playbackloudness by 100 well for me.
Edit2: Here is the script I manage to cook up
repeat wait() until workspace:FindFirstChild("Sound")
local Sound = workspace:FindFirstChild("Sound")
local lastLoudness = 0
game:GetService("RunService").Heartbeat:Connect(function()
local PBS = Sound.PlaybackLoudness/100
if lastLoudness~=0 then
if PBS > lastLoudness+1 then
--[[local newPart = workspace.Part:Clone()
newPart.Parent = workspace
newPart.Anchored = false--]]
print("Detected a beat")
end
end
lastLoudness = PBS
end)
(Planning to make this a open source script so anyone can use it)
6 Likes
Sorry for another ping but is there anyway I could possibly improve this script?
It’s really bad right now and misses off by miles. it has issues with detecting the “beats” and has issues with firing randomly
repeat wait() until workspace:FindFirstChild("Sound")
local Sound = workspace:FindFirstChild("Sound")
local lastLoudness = 0
game:GetService("RunService").Heartbeat:Connect(function()
local PBS = ((Sound.PlaybackLoudness/100)^(Sound.PlaybackLoudness/100))
if lastLoudness~=0 then
local formula = math.abs((lastLoudness*15) + ((lastLoudness+PBS)/1.9))
if (math.abs(PBS) > formula) == true then
local newPart = workspace.Part:Clone()
newPart.Parent = workspace
newPart.Anchored = false
print("Detected a beat")
print(formula, PBS)
end
end
lastLoudness = PBS
end)
1 Like
Did you mean for this to be
local PBS = ((Sound.PlaybackLoudness/100)*(Sound.PlaybackLoudness/100))
Nothing of the form x^x is going to be something you want here.
I was attempting to make PBS have a more bigger gap range in levels of playbackloudness. I hoped it would’ve allowed my script to have a better shot at attempting to identify the audio peaks
But that function has a minimum at 1/e, and will make values near zero and near max loudness behave the same. You want a non-linearity that emphasizes the difference. Something of the form c^x would do that, as would something x^c, both for c>1, but x^x is no good.
I don’t know what to make of the “formula”, that’s not anything I recognized and the constants seem arbitrary?
I started off reasonable then I started adding numbers ontop of it then added more until I had no clue what my formula even done. I am willing to just scrap the formula and just make a new threshold formula at this point.
local Sound = workspace:FindFirstChild("Sound")
local lastLoudness = 0
local debounce = true
game:GetService("RunService").Heartbeat:Connect(function()
local PBS = Sound.PlaybackLoudness*Sound.PlaybackLoudness
if lastLoudness~=0 then
local formula = math.abs( (lastLoudness * (Sound.PlaybackLoudness/140)) )
print(PBS, formula)
if (math.abs(PBS) > formula) == true and debounce then
local newPart = workspace.Part:Clone()
newPart.Parent = workspace
newPart.Anchored = false
print("Detected a beat")
print(formula, PBS)
debounce = false
spawn(function()
wait(.1)
debounce = true
end)
end
end
lastLoudness = PBS
end)
^ New changes
(I kinda want Roblox to add an Equalizer API to the current Sound API. Left and right channels and etc… It might be a lot but Roblox already has an Equalizer effect.)
4 Likes
I recommend picking a sampling of music you might actually use, and as diverse a test set as you can manage, and make something to visualize whatever your applied formulas come up with, and see how it corresponds to the points where you want things to happen.
spawn(function()
wait(.1)
debounce = true
end)
A. You can’t do anything time-critical with spawn(), as it waits a frame or two before even calling the function you give it. B. 0.1 seconds corresponds to a fixed rejection of >= 600 bpm. C. Debounce will only work if/when you have detection that is extremely confident, otherwise, triggering once on a transient that isn’t on the main beat could cause you to miss the beat itself, possibly all beats from that point on in the worse case where there is always a transient just before the beat that isn’t as loud as the beat, but loud enough to trigger your comparison. Just something to keep in mind. Fixed time-constant debounce mechanisms are not usually a part of beat detection.
2 Likes
Removing the debounce helped a lot now. The major issue is now it has a small chance of spamming the event 2 - 30 times. I believe this would be a easy fix my tweaking around the number in the formula.
Alright so I tested it on another audio it doesn’t work 1 bit. I need a method of filtering out the higher pitch frequencies and only accepting in lower pitch frequencies. I can then from the lower pitch frequencies use a formula to check if a “main beat” is detected
What is the “PlaybackLoudness” property measured in? It’s not in Hertz I’m unsure.
A thread said it is PCM amplitude. I need to convert it to DB so I can make a sound peak system
Should I ping spotco?
Update 1:
To get the Hertz I’ll need to get the number of oscillation per second. Frequency = 1 over seconds that makes 1 hertz. I spent the entire 2 days trying to solve this (I really really really really wanna make this) This means I gotta convert the PBS to a Frequency then to a Hertz. How will I do this??
This just isn’t possible, PlaybackLoudness is not an audio signal, it’s a value that only updates once per frame, at 60fps (60hz). There are no high pitches in this signal at all; the highest-frequency that can even be represented at this sample rate (per Nyquist-Shannon Sampling Theorem) is half of this, or 30hz. 30Hz is at the low end of the frequency range of human hearing, in the “sub-bass” part of the audio spectrum. To get what you want, the audio would have to be filtered before the RMS averaging, and that’s just not being done.
It’s not PCM amplitude exactly. It’s the RMS (Root Mean Square) of the last 1024 samples of the uncompressed audio sample data, normalized to the range 0-1000. It’s a unitless value, a type of average signal amplitude.
You don’t necessarily need to convert to decibels to make the system work, but if you need this value, it’s just:
20 * math.log10(0.001*sound.PlaybackLoudness)
or equivalently:
20 * (math.log10(sound.PlaybackLoudness) - 3)
I’m not surprised. I’m not sure you got that my italicization of “little” in “there might be a little bit of special stuff going on” was my way of indicating that this was a huge understatement. If what you’re trying to do is along the lines of the pulsing light orbs in Club RAVEN, there is a lot of signal processing being done on the PlaybackLoudness values to clean things up; there are multiple layers of circular buffers with the raw loudness, various weighted averages (including FIR filters), dynamic range compression (automatic gain control), exponentially-decaying weighted averages of min and max values, peak-over-average calculation, autocorrelation (convolution of the signal with time-delayed copies of itself) etc… I may well take me 2 days just to explain it 
. The orbs aren’t just setting brightness = PlaybackLoudness, that would flicker a lot more erratically, and would not work well for sound files of different loudness (or even different loudness passages within the same song).
If you want to see if the results of just autocorrelation are useful for your songs, you can try the IDs here:
The BPM detection will only be good for a song with a steady beat (not breakbeat DnB, or syncopated), EDM style, that is fairly dominant, or a sample loop that repeats. Strong basslines and dubstep don’t play nice with this kind of algorithm. This particular code is not for the lighting, it’s for an unfinished feature that syncs dance animation speed to the music–unfinished because none of us have taken the time to measure the natural bpm speed of the mocap dance animations in order to complete the job.
What that “lab” is displaying is the PBL waveform in orange, a low-pass filtered copy in green, and on top between the white bars is the output of the autocorrelator. When autocorrelation is weak to non-existent, that top display will flatline. When it has strong peaks, they correspond to BPM. There are no scales because this was meant only for internal use, but the second vertical line from the left is at 120 bpm, and the 3rd is 60 bpm. For a song with a strong 120 bpm beat, the display should end up looking reminiscent of the Golden Gate Bridge. (for example, try id 175242342)
14 Likes
Yea, it’s very confusing by just getting an audio peak by playbackloudness only.
I kinda wish Roblox added more info onto audio noise i.e. Frequency, LowGain, MediumGain, and HighGain, Left and Right Channels, and etc… If those were added to the audio this would’ve made this a bit more “easier”
My only question is how would you make this??? The wikipedia formulas give me no luck as it’s kinda confusing for me. I just know where to replace E with or Replace A with, and etc… I want to give up on this project as it’s too confusing for me but I just dislike giving up.
(I wish I could post on Engine Feature Requests as I would’ve made a audio request thread.)
2 Likes
Just some interesting information:
There was a study by John Clarke, who examined the top 100 list of songs every month for several years (2011-2017). Nearly all of the top hits had a 120BPM. If you use 120BPM as default on your game, most songs would go perfectly with it.
7 Likes
Alright that should help me a lot.
1 Like