How do AudioAnalyzers work?

Help and Feedback Scripting Support
#1

Hello there.

I am currently working with the new Audio API, and so far I created a working radio where you can change your frequency and talk with other players. I wanna add a feature where the highlighted bars visualize the frequency when the player talks. The AudioAPI tutorial place used AudioAnalyzers, however, I do not really understand how they work. Anybody has experience with them?
image

image

Thanks for any help!

3 Likes
#2

As a heads up, the AudioAnalyzer:GetSpectrum() method that makes it possible to visualize the frequencies has been temporarily disabled for over a month now, so it won’t be possible to use that until it’s re-enabled:

Edit: It turns out that the feature has been re-enabled in some capacity! The example visualizer with the AudioPlayer in the test place has started working again. The original post in the Audio API announcement thread still hasn’t been updated to clarify that it works again, though.

However, it was pointed out to me by another user that it currently doesn’t work with AudioDeviceInputs (as of April 2nd, 2024), so for the time being, AudioAnalyzer:GetSpectrum() currently cannot be used to analyze the frequencies of audio transmitted via player’s input devices (which means that it may be impossible to create the feature that you described in your post, at the moment).

Hoping that the Roblox Staff Members in the Audio API Announcement thread will be able to provide some clarification about the status of AudioAnalyzer:GetSpectrum() and if it’ll be compatible with voice chat audio in the future.

Original Post

Unfortunately, I don’t completely understand how it works so my explanations wouldn’t be of much use. However, for anyone who is more familiar with it and would like to try to explain how this works (and potentially provide insight into how the same thing could be achieved through UI rather than parts in the Workspace, in order to answer OP’s question), here’s the client-sided code included in the Audio API Tutorial test place that makes the spectrogram / frequency visualizer functional:

local analyzer = script.Parent:WaitForChild("AudioAnalyzer")
local spectrogram = script.Parent.Spectrogram

local rng = Random.new()

local function getBins() : {Instance}
	local bins : {Part} = spectrogram:GetChildren()
	table.sort(bins, function(a, b) 
		return a.Position.X > b.Position.X
	end)
	
	return bins
end

local function lerp(lower, upper, fract)
	return upper * fract + lower * (1 - fract)
end

local function getMappedBins(binCount) : {number}	
	local bins = analyzer:GetSpectrum()
	if not bins or #bins == 0 then 
		local empty = {}
		for i = 1, binCount do
			table.insert(empty, 0)
		end
		return empty
	end
	local result = {}
	for i = 1, binCount do
		local j = math.pow(#bins, i / binCount)
		local lower = math.max(1, math.floor(j))
		local upper = math.min(#bins, math.ceil(j))
		local fract = j - math.floor(j)
		result[i] = lerp(bins[lower], bins[upper], fract)
		result[i] = math.sqrt(result[i]) * 2
		result[i] = math.clamp(result[i], 0, 10)
	end
	
	return result
end

local parts : {Part} = getBins()
while true do
	local bins = getMappedBins(#parts)
	for i = 1, #bins do
		local part =  parts[i]
		local size = part.Size
		part.Size = Vector3.new(size.X, math.clamp(10 * bins[i], 0, 10), size.Z)
	end
	task.wait()
end
1 Like
#3

This is how I do it, note, I haven’t included everything and won’t give a tutorial on how to achieve everything. (You’d possibly need to constrain the bars, etc).

But the ‘bars’ are just frames sorted by LayoutOrder within a folder (container), like so:

image
image538×774 32.3 KB

Note: This won’t be correctly optimised for live games, so bare that in mind - it was just used for testing purposes.

local playersService: Players = game:GetService("Players");
local runService: RunService = game:GetService("RunService");

local speakerSet = workspace:WaitForChild("SpeakerSet");
local audioAnalyzer: AudioAnalyzer = speakerSet:WaitForChild("AudioAnalyzer");

local localPlayer = playersService.LocalPlayer;
local playerGui: PlayerGui = localPlayer.PlayerGui;

local hudGui = playerGui:WaitForChild("HUD");
local hudFrame = hudGui:WaitForChild("HUD");
local topBar = hudFrame:WaitForChild("Topbar");
local center = topBar:WaitForChild("Center");
local barsFrame = center:WaitForChild("Bars");
local container = barsFrame:WaitForChild("Container");

-- Retrieves bars (Frames) sorted by their order or X position
local function getSortedGuiBars(): {GuiObject}
	local bars: {GuiObject} = {};
	
	for _, child in pairs(container:GetChildren()) do
		if (child:IsA("Frame")) then -- Only include Frames in the sorting process
			table.insert(bars, child);
		end;
	end;
	
	table.sort(bars, function(a: GuiObject, b: GuiObject)
		return a.LayoutOrder < b.LayoutOrder or a.Position.X.Scale < b.Position.X.Scale
	end);
	
	return bars;
end;

-- Linear interpolation function for more visually consistent adjustments
local function linearInterpolate(min: number, max: number, fraction: number): number
	return max * fraction + min * (1 - fraction);
end;

-- Maps the audio spectrum to bars, adjusting their height based on the spectrum
local function mapSpectrumToGuiBars(targetBarCount): {number}
	local spectrum: {number} = audioAnalyzer:GetSpectrum() or {};
	
	if (#spectrum == 0) then
		return table.create(targetBarCount, 0); -- Returns a table filled with zeroes if the spectrum is empty
	end;

	local mappedBins: {number} = {};
	for i: number = 1, targetBarCount do
		local targetIndex: number = math.pow(#spectrum, i / targetBarCount);
		local lowerIndex: number = math.max(1, math.floor(targetIndex));
		local upperIndex: number = math.min(#spectrum, math.ceil(targetIndex));
		local fraction: number = targetIndex - lowerIndex;
		local interpolatedValue: number = linearInterpolate(spectrum[lowerIndex], spectrum[upperIndex], fraction);
		
		interpolatedValue = math.sqrt(interpolatedValue) * 2; -- Adjusting the amplitude for visual clarity
		mappedBins[i] = math.clamp(interpolatedValue, 0, 1); -- Clamp the values to ensure they fit within the gui's scale
	end;

	return mappedBins;
end;

-- Update function to adjust the bars based on the audio spectrum
local function updateGuiBars()
	local bars: {GuiObject} = getSortedGuiBars();
	local bins: {number} = mapSpectrumToGuiBars(#bars);

	for i: number, bar: GuiObject in ipairs(bars) do
		-- Adjusting the height of each bar according to the spectrum. Scale can be adjusted for desired effect
		bar.Size = UDim2.new(bar.Size.X.Scale, bar.Size.X.Offset, 0, math.clamp(100 * bins[i], 0, 100));
	end;
end;

-- Connect the update function to the RenderStepped event for real-time updates
runService.RenderStepped:Connect(updateGuiBars);
2 Likes