Roblox should allow us to perform the Fast Fourier Transform (FFT) algorithm on portions of audios. This API would be very useful to make games that need a API to calculate real time frequency of a sound. Roblox implemented SoundLoudness, however that isn’t as efficient to determine things like an audio visualizer, or how fast something should move depending on the sound.
Sound:GetFrenquencyData(Interval)
Returns an array of numbers containing the frequency of the sound at different times (times that are divisible by Interval)
I think some staff member mentioned that they’d rather implement specific sound effects than to implement full frequency domain manipulation, since the former is easier to use and arguably covers most of the use cases.
easiest (and most likely to be implememented) will be a FFT soundeffect, it’ll basically be a wrapper over this (http://www.fmod.org/docs/content/generated/FMOD_DSP_FFT.html) and you’ll be able to do all the use cases you mentioned with it. coming soon
I don’t think the FFT is the ideal choice, at least from an API level.
First, the FFT imperfectly gives complex data points that are linearly scaled and on with equal bandwidth in terms of Fs. Temporal resolution is also uniform.
Every part detracts or complicates the intended application.
1.) The FFT is imperfect – if you have 1Hz spaced bins in frequency, a 101.1Hz signal will spill into ALL bins, with 100 being most dominant.
2.) complex data points contain more information than needed, requiring an extra lua-based conversion step at a rate similar to the sample rate of the playback. Also you get back a bonus set of redundant data points as the FFT also worked on complex inputs and has to be conjugate symmetric for real inputs.
3.) hearing is closer to log-based than linear in terms of amplitude.
4.) hearing is closer to log-based than linear in terms of bandwidth.
5.) the FFT is also based on sampling frequency, unless this is normalized up front.
6.) hearing is also less based on uniform temporal resolution.
I’m not saying the idea for a visualizer is bad, but I get the feeling that raw FFT data isn’t what most users will actually want.
–edit: The second reason I don’t like raw FFT data for this application is that similar concepts are used in equalizers. It seems odd to have a visualizer based on FFT parameters and an EQ based on practical parameters.
Currently, as a Roblox Developer, it is impossible to create accurate, hertz-based audio visualization or response machines. If I want to design a basic UI that responds to the music playing in a game, or the sound being heard, I have to use PlaybackLoudness, which is not hertz-based and is simply just the cumulative loudness of the audio.
Numerous uses can be made, for example, as seen in the example above, one could make a little visualizer akin to that to represent how loud the player is. One could also make UI that enhances the look and feel of the game, if there’s music playing. It also has uses for games that have clubs or other things that want to have VFX respond to the music’s components, like the bass.