New Audio API [Beta]: Elevate Sound and Voice in Your Experiences

[ReallyLongArms]

chaining three AudioFader instances really isn’t ideal for recreating the old behavior

Since .Volume is multiplicative, if you set AudioPlayer.Volume to 3, you’d only need one AudioFader with .Volume = 3 to boost the asset’s volume 9x

I wonder if it would be more useful to have something like an AudioPlayer.Normalize property – is the desire for a 10x volume boost coming from quiet source-material?

Currently, AudioFader instances cannot serve this use case, as piping multiple AudioPlayer istances through a single AudioFader will stack them all on top of each other

SoundGroups are actually AudioFaders under the hood – so it should be possible to recreate their exact behavior in the new api. The main difference is that Sounds and SoundGroups “hide” the Wires that the audio engine is creating internally, whereas the new API has them out in the open.

When you set the Sound.SoundGroup property, it’s exactly like wiring an AudioPlayer to a (potentially-shared) AudioFader.
I.e. this

is equivalent to

Similarly, when parenting one SoundGroup to another, it’s exactly like wiring one AudioFader to another
I.e. this

is equivalent to

If there are effects parented to a Sound or a SoundGroup, those get applied in sequence before going to the destination, which is exactly like wiring each effect to the next.

Things get more complicated if a Sound is parented to a Part or an Attachment, since this makes the sound 3d – internally, it’s performing the role of an AudioPlayer, an AudioEmitter, and an AudioListener all in one.

So this

is equivalent to

where the Emitter and the Listener have the same AudioInteractionGroup

And if the 3d Sound is being sent to a SoundGroup, then this

is equivalent to

Exposing wires comes with a lot of complexity, but it enables more flexible audio mix graphs, which don’t need to be tree-shaped. I recognize that the spaghetti can get pretty rough to juggle – I used SoundGroups extensively in past projects, and made a plugin with keyboard shortcuts to convert them to the new instances, which is how I’ve been generating the above screenshots. The code isn’t very clean, but I can try to polish it up and post it if that would be helpful!

[ReallyLongArms]

@MonkeyIncorporated

I’m making a voice chat system where if someone is behind a wall, their voice will get muffled. While rewiring one effect isn’t much of a hassle, the connections might start looking like spaghetti once more effects get added.

Hey @MonkeyIncorporated – we recently added a .Bypass property to all the effect instances
When an effect is bypassed, audio streams will flow through it unaltered; so I think you could use an AudioFilter or an AudioEqualizer to muffle voices, and set it to .Bypass = true when there’s no obstructing wall. We have a similar code snippet in the AudioFilter documentation, but we used the .Frequency property instead of .Bypass – I think either would work in this case

Instead of putting everything in a linear order, will it soon be possible to have a standalone effect and connect it to an emitter via a wire?

A trick that I’ve used is having two AudioFaders act as the Input/Output of a more-complicated processing graph, that’s encapsulated somewhere else

here, the inner Processing folder might have a variety of different effects already wired-up to the Input/Output. Then, anything wanting to use the whole bundle of effects only has to connect to the Input/Output fader(s)

Does this help?

[Rocky28447 (Rocky)]

@ReallyLongArms

Since .Volume is multiplicative, if you set AudioPlayer.Volume to 3, you’d only need one AudioFader with .Volume = 3 to boost the asset’s volume 9x

Ah, I suppose that’s a bit better.

@ReallyLongArms

is the desire for a 10x volume boost coming from quiet source-material?

More so it’s just coming from a desire to not need an AudioFader where one wasn’t needed previously. Every additional audio instance in the chain requires setting up wires, so minimizing the need for extra instances where possible is ideal.

@ReallyLongArms

This image helps a lot. I was trying to apply the group fader before the AudioPlayer in the chain, which isn’t allowed. I didn’t think about applying the group fader between the listener and the output and creating a listener for each group using AudioInteractionGroup, but this makes a lot more sense. Thanks for the tip!

[MonkeyIncorporated (MonkeyInc)]

This works well for what I’m doing, thanks for sharing!

[Synceratus (Zaiden)]

Is there any way to get the position of the source that the AudioListener is listening to?

[ReallyLongArms]

AudioEmitters and AudioListeners inherit their position & rotation from their parent, so you can write a helper method like

local function getCFrameFrom(inst: Instance) : CFrame?
    local parent = inst.Parent
    if not parent then
        return nil
    elseif parent:IsA("Model") then
        return parent.WorldPivot
    elseif parent:IsA("BasePart") then
        return parent.CFrame
    elseif parent:IsA("Attachment") then
        return parent.WorldCFrame
    elseif parent:IsA("Camera") then
        return parent.CFrame
    else
        return nil
    end
end

[tacheometrist]

Just to confirm - is it not possible to read the RmsLevel property of an AudioAnalyzer on the server side? It seems to read 0 both in the demo place and in my own testing.

My use case is getting a player’s microphone input volume - right now I’m using an UnreliableRemoteEvent to share this data with the server, but I presume the latency for reading this on the server might be lower if this information is already sent with the voice input (unless it isn’t, I’m just guessing).

Let me know your thoughts

[ReallyLongArms]

Just to confirm - is it not possible to read the RmsLevel property of an AudioAnalyzer on the server side? It seems to read 0 both in the demo place and in my own testing.

Since nobody can hear audio on the server, all audio processing is disabled to conserve CPU & Memory – this also causes AudioAnalyzer to only work client-side.

right now I’m using an UnreliableRemoteEvent to share this data with the server

That sounds like a great solution

I presume the latency for reading this on the server might be lower if this information is already sent with the voice input

Honestly it might be a wash – your own voice (i.e. AudioDeviceInput.Player == LocalPlayer) is ahead of the server and/or any other clients. So if you have each client send only their own RmsLevel to the server, it might arrive at around the same time as the actual voice packets

[tacheometrist]

Ah alright. Thank you for the insightful reply

[bytesleuth]

Is there any way to directly connect player audio input into an audio analyzer instead of using an audio listener?

[ReallyLongArms]

Yes; AudioDeviceInput can be wired directly to an AudioAnalyzer without going through an intermediate AudioEmitter & AudioListener – emitters and listeners are only necessary if you want it to be part of the 3d world.

E.x.

local function findDevice(forPlayer : Player) : AudioDeviceInput?
    return forPlayer:FindFirstChildWhichIsA("AudioDeviceInput")
end

local function wireUp(source : Instance, target : Instance) : Wire
    local wire = Instance.new("Wire")
    wire.SourceInstance = source
    wire.TargetInstance = target
    wire.Parent = target
    return wire
end

local device = findDevice(game:GetService("Players").LocalPlayer)
if device then
    local analyzer = Instance.new("AudioAnalyzer")
    analyzer.Parent = device
    wireUp(device, analyzer)
end

[sussiestwetamongus]

@ReallyLongArms

AudioEmitters and AudioListeners inherit their position & rotation from their parent, so you can write a helper method like

local function getCFrameFrom(inst: Instance) : CFrame?
    local parent = inst.Parent
    if not parent then
        return nil
    elseif parent:IsA("Model") then
        return parent.WorldPivot
    elseif parent:IsA("BasePart") then
        return parent.CFrame
    elseif parent:IsA("Attachment") then
        return parent.WorldCFrame
    elseif parent:IsA("Camera") then
        return parent.CFrame
    else
        return nil
    end
end

This would only get the position of the AudioListener’s parent correct? So is it possible to get the position of where the AudioListener picked up the sound.

For example, let’s assume all sound including the player has an emitter with the same AudioInteractionGroup. If an AudioEmitter parented to a part emits sound and the player emits sound, would I be able to get the position where the audio played from? In this case, the part position as well as the player position.

GetConnectedWires seems to be a way to get a source instance but that doesn’t work if I have my AudioListener set to listen through an AudioInteractionGroup. There isn’t the ability to retrieve where the sound is played from unless I manually wire it.

[TuroIatias (Turolatias)]

Will there be an option to choose which AudioEmitters can be ignored by an AudioListener? It would be useful to avoid feedback loops (for in-game microphones that pick up everything), but not only.

Also, will there be a way to set more than 1 AudioInteractionGroup for the new Instances?

[ReallyLongArms]

Will there be an option to choose which AudioEmitters can be ignored by an AudioListener?

Currently this can be controlled by the AudioInteractionGroup property – listeners will only hear emitters that are in the same group.

will there be a way to set more than 1 AudioInteractionGroup for the new Instances?

This is something we’ve discussed but don’t yet have any firm plans on – for now, in order to have audio emitted and heard by multiple listeners, you’d also need multiple emitters (each one in a different interaction group).

The good news is that those extra wires and emitters are pretty lightweight, but I understand it can contribute to an overall mess of disorganized, wire-spaghetti.

It would be more convenient if these behaved like CollisionGroups, where you can define cross-group interactions; but CollisionGroups had some limitations (e.x. maximum count) that dissuaded us from reusing them directly.

[sussiestwetamongus]

Is there a way to get the position of a sound playing through AudioListener? Not the AudioListener’s parent but the sound’s parent.

[ReallyLongArms]

We actually just added two methods: AudioEmitter:GetInteractingListeners() and AudioListener:GetInteractingEmitters(). These can be used to list all the listeners that would hear a given emitter, or all the emitters that can be heard-by a given listener

Combining these and the helper method from above, you could

1. list all the emitters heard by a listener with listener:GetInteractingEmitters(),
2. for a particular emitter in the list of emitters, use getCFrameFrom(emitter) to get its position

Does that help?

[sussiestwetamongus]

Thank you for the follow up. I played with this a little and it seems very useful in altering listeners and emitters in groups.

I was wondering if you guys were planning on creating an event for when an AudioListener picks up any sound, it provides the AudioEmitter with it possibly, something like AudioListener.SoundDetected:Connect(someAudioEmitter: AudioEmitter)

[ReallyLongArms]

Adding an event like that directly would come with a passive performance penalty, since the stream being produced by the AudioListener is already mixed; we’d have to comb back through the graph to unpack it/determine where it came from.

But, you can implement something similar with AudioAnalyzers if you don’t mind the overhead – a rough idea would be

1. wire up your AudioListener, as well as the inputs of each AudioEmitter to several AudioAnalyzers
2. periodically check the listener-analyzer’s PeakLevel property; whenever it crosses a threshold, get the listener’s interacting emitters
3. for each emitter, check whether its analyzer also has PeakLevel above a threshold

If both the Listener-analyzer and the Emitter-analyzer have crossed a volume threshold at about the same time, this indicates that the emitter is partially-responsible for whatever the listener heard

[sussiestwetamongus]

Ah, performance didn’t really cross my mind there.

@ReallyLongArms

But, you can implement something similar with AudioAnalyzers if you don’t mind the overhead – a rough idea would be

1. wire up your AudioListener, as well as the inputs of each AudioEmitter to several AudioAnalyzers
2. periodically check the listener-analyzer’s PeakLevel property; whenever it crosses a threshold, get the listener’s interacting emitters
3. for each emitter, check whether its analyzer also has PeakLevel above a threshold

If both the Listener-analyzer and the Emitter-analyzer have crossed a volume threshold at about the same time, this indicates that the emitter is partially-responsible for whatever the listener heard

Thanks for giving me a good solution. I’ll be sure to try it.

[JOSIMA33 (JOSIMA)]

I’m encountering an issue where I hear a short pop or click when I create an AudioPlayer, set its AssetId, and immediately call :Play().

This is the code I’m using:

while true do
	local audioPlayer = Instance.new("AudioPlayer", workspace)
	audioPlayer.AssetId = "rbxassetid://7003103879"
	audioPlayer:Play()
	
	audioPlayer.Ended:Connect(function()
		audioPlayer:Destroy()
	end)
	
	task.wait(0.1)
end

Observations

• The issue occurs consistently when I use Bluetooth headphones.
• When I use my laptop speakers, the issue does not seem to occur.
• I haven’t tested it with wired headphones.
• The problem is not limited to this specific AssetId; it also happens with other audio assets.
• The issue persists regardless of whether the AudioPlayer is connected to an emitter or not.