New Audio API [Beta]: Elevate Sound and Voice in Your Experiences

[purpledanx (daniel)]

Can I have a follow up to this? At least some answer to my question. please

New Audio API [Beta]: Elevate Sound and Voice in Your Experiences

I do wish the GetConnectedWires() function is usable in normal scripts. Because currently (I think) there’s no way to know what AudioPlayer an AudioEmitter is connected to without looping through the whole game. Unless all the AudioPlayers are in a single folder.

[StrongBigeMan9]

@EmesOfficial

It seems like the voice chat is functional within the studio, but when testing with multiple players, the audio cannot be heard by others.

If you’re running a “Local Server” playtest with 2 or more clients, the clients won’t be able to hear one another, unfortunately. No matter if you locally enable or server-side enable the “Active” property for each clients’ AudioDeviceInput, and regardless if the IsReady property is replicated as being “true” between the clients in the playtest, AND even if each client is able to simultaneously transmit audio, it’s not picked up by any of the other clients in the local playtest.

I’m not sure if that’s an unintended issue specifically with local playtests, where the Voice Chat UI and the “Input Device” option within the Roblox Settings Menu are not present, whereas it’s available in Team Tests and in live servers.

While it’s still possible to use Voice Chat in 1-player solo playtests, it’s very unfortunate that it doesn’t work properly in multi-client local playtests since that adds extra layers of friction for solo developers wanting to test out Voice Chat features that typically require separate accounts with Voice Chat enabled to test.

Thus far, for all of the multiplayer Voice Chat testing I’ve done with the new Audio API, I’ve had to publish the game, leave Roblox Studio, then join a live server of that game from my computer on one account and from my phone on another account.

---

I had wondered if it would be possible to seamlessly host local playtests with multiple clients for testing out “multiplayer Voice Chat situations” ever since the “Chat with Voice Developer Beta” was announced in 2021:

@StrongBigeMan9

For instance, if you hosted a Local Server playtest with 2 player Instances, you could do the following:

• Mute the mic for one player Instance and keep the game volume enabled
• Unmute the mic for the other player Instance and mute the game audio

Afterward, you could talk through Voice Chat from the Character that has the unmuted microphone setting and be able to hear what it sounds like from the other Character model (that has the game audio enabled) at different distances, varying levels of volume, etc.

…

Although the primary focus of my question back then was wondering if developers would be able to locally test Voice Chat functionality whether or not they verified their ID, it’s really unfortunate that it’s not currently possible to simulate having multiple clients communicate between one another with Voice Chat in Roblox Studio, even if the individual account being used for the multi-client local playtest is ID verified and has Voice Chat enabled.

[XenoDenissboss1]

Right so after playing around with the new audio API’s, i have come to the conclusion that the new systems right now are very hard to work with overall. It feels as if the new audio api was more designed on the idea that you’ll just set up some audio source and never touch it again past just deleting it once its no longer needed. Swapping in and out audio effects is unreasonably inefficient as were pretty much forced to constantly cycle through every wire and whatever else other instances needed just to make the entire process work. One idea i have to potentially fix this issue would be some sort of “Wire Hub” instance. Basically it would be an instance that lets you hook up multiple input wires and a single output wire just so you dont need to constantly rechain many other wire instances.

Overall i think the new system is great, it just feels as if its half complete (or at least half usable).

(also proper pitch shifting update yes?)

[Floo_d]

in my game i had this enabled out of curiosity

I’ve come across a bug where overtime players voices will be heard globally.

It’s a bit of a silly bug and I’m not complaining about it (but I know it can become annoying). I’m not sure if this has been reported yet & the voices can still be heard when the Roblox volume slider is at 0 (like in the video)

[lrogerorrit]

When we enabled this in our game we had users report they couldnt access it because it kept crashing. They said they had no error and it just closed the player window. Im not sure if its like this for all the users that were crashing, but some reported that it was not crashing in the windows store app, but was in the normal pc app.

[vsk_1 (vsk)]

Is push to talk going to be added as said on this post:

Creator Roadmap - AMA with Dan and Manuel [Jan 24, 2024]

We’re diligently working on the beta release of our new Audio API which should be out in the next month or so. It includes highly anticipated controls over sound and voice that were previewed at RDC. You can access all the use cases you mentioned including exercising more control over spatial simulation of voice, modifying voice, walkie-talkies, push-to-talk, and more!

Also, is there going to be a way to disable the default proximity chat, for example if I wanted to make a team voice chat system.

[Lxi099 (lollo)]

Has anyone got AudioAnalyzer to work when connected to an AudioListener?
It seems to always return 0s and never pick up the audio, even if everything is properly wired up and audio is being picked up by the listener. I’ve also tried hooking the listener to a fader and the analyzer to the fader, but still no results

[NobleReign (Noble)]

might just be because audioanalyzers are disabled right now, they’ll likely make an announcement when they’re back up again

@therealmotorbikematt

We will be temporarily disabling AudioAnalyzer:GetSpectrum while we investigate an issue. We’ll re-enable this after reviewing and optimizing its capabilities & performance.

[StrongBigeMan9]

@vsk_1

Also, is there going to be a way to disable the default proximity chat, for example if I wanted to make a team voice chat system.

Turning off the EnableDefaultVoice property of the VoiceChatService is the way to disable proximity-based Voice Chat by default, since that property is what automatically creates the AudioEmitter and AudioListener within each player’s Character model (which are two of the primary instances that make it possible for players to emit / hear proximity-based Voice Chat audio when using the new Audio API).

@vsk_1

Is push to talk going to be added as said on this post:

I had initially thought there was going to be a built-in feature for it, too, but considering how several other features that were mentioned alongside that required scripts to interact directly with the new Audio API (e.g. modifying voice and walkie-talkies) it seems like push-to-talk might be a feature we have to code ourselves.

Fortunately, it appears that push-to-talk is fairly easy to implement Here’s a pretty barebones version I created for enforcing push-to-talk by locally enabling / disabling the Muted property of the player’s AudioDeviceInput depending on whether or not the player is holding down the specified keybind.

(Note that there’s more that would need to be added to this into this in order to make it mobile compatible, along with other quality of life features such as making it possible for players to specify their own keybind).

Push-to-talk Example

-- LocalScript in StarterPlayerScripts
local UserInputService = game:GetService("UserInputService")
local Players = game:GetService("Players")
local player = Players.LocalPlayer

local audioDeviceInput = player:WaitForChild("AudioDeviceInput") -- If you manually created an AudioDeviceInput with a different name, make sure to update this to the new name
audioDeviceInput.Muted = true -- Muting the AudioDeviceInput immediately since it starts out as being unmuted, by default

local pushToTalkKeybind = Enum.KeyCode.C -- Define the key that players have to hold down while talking in order to be heard by other players


UserInputService.InputBegan:Connect(function(inputObject, gameProcessedEvent) -- Function activated when the player interacts with the mouse, keyboard, etc.
	if inputObject.UserInputType == Enum.UserInputType.Keyboard and gameProcessedEvent == false then -- Checks if the type of input was from a keyboard and makes sure that the player wasn't interacting with UI at the time (such as typing in a TextBox)
		
		local keycode = inputObject.KeyCode -- Refers to the KeyCode of the InputObject, which will be used to check which key on the keyboard the player pressed
		
		if keycode == pushToTalkKeybind then -- If the key that the player pressed matches the "pushToTalkKeybind" defined at the top of the LocalScript, then...
			audioDeviceInput.Muted = false -- The LocalScript unmutes the AudioDeviceInput since the player is holding down the push-to-talk key
		end
	end
end)

UserInputService.InputEnded:Connect(function(inputObject, gameProcessedEvent) -- Function activated when the player stops interacting with the mouse, keyboard, etc.
	if inputObject.UserInputType == Enum.UserInputType.Keyboard and gameProcessedEvent == false then
		
		local keycode = inputObject.KeyCode
		
		if keycode == pushToTalkKeybind then -- If the player let go of the push to talk key, then...
			audioDeviceInput.Muted = true -- The LocalScript mutes the AudioDeviceInput since the player is no longer holding down the push-to-talk key
		end
	end
end)

[ReallyLongArms]

Hey purpledanx; we made the GetConnectedWires method accessible to plugins for starters – did you have a use-case in mind for traversing the audio graph at runtime?

[ReallyLongArms]

Hey StrongBigeMan9, we just updated the docs for AudioDeviceInput today – the AccessType property is intended to pair with SetUserIdAccessList

[Laqotaa]

Will there ever be like Nodes? Where you can connect wires to a Node, and then connect to the Node to all the speakers or effects? I find it very bad to connect wires to a Emitter, Use a listener, and then wire all speakers around the map to this listener.

For example. Let’s say your making an intercom system and you want to make a cool sound before relaying the Audio. You could connect an audioplayer to the Node, play the sound, and then connect the players microphone to the Node.

[purpledanx (daniel)]

I have a dynamic reverb system that gets every audio emitter and traces back to the audio modifiers to changes the values of the fader, reverb, gain, etc. So the only way I could trace back would be to find the connected wires.

[StrongBigeMan9]

Thanks for updating it and letting me know! The documentation provided there mostly clears up the questions I had.

I had actually been updating my original post for the past half an hour when I first noticed the documentation was updated but realized it doesn’t appear to answer all the questions I had.

I’ll quote the sections of my original post that describe behavior with the AccessType property that appear unintentional, given what is documented about it at the moment:

@StrongBigeMan9

Edit (March 4th, 2024):

Hooray! There’s now documentation for the properties and methods of the AudioDeviceInput, which clears up some of the questions I had originally outlined below.

However, it doesn’t answer all of my questions, because the most confusing issue I had observed still persists and is not clarified by the provided documentation:

• Issue: If a client locally updates the AccessType from the default value of “Deny” to “Allow”, then swaps it back to “Deny”, the client will no longer be able to hear audio being transmitted from the AudioDeviceInput. This behavior occurs without the client or server ever specifying an AccessList using :SetUserIdAccessList() and without making any other changes.It is mentioned in the documentation for AccessType that: “this property should only be assigned from the server in order to replicate properly.” However, even if the server forcefully sets the AccessType of the AudioDeviceInput to “Deny”, the client still isn’t able to hear audio being transmitted from the specific AudioDeviceInput that they had previously locally swapped back and forth between “Deny” and “Allow”.

For more detail regarding this issue, please refer to “Scenario #2” of the “Locally updating AccessType” section of this post, as well as the entirety of the “Server-side updates to AccessType” section of this post. Although I wrote both of those before there was documentation for the properties / methods of the AudioDeviceInput, the described issues are still relevant, because based on what is currently documented about the AccessType property, AccessType should only have an effect when a list of User IDs is specified.

…

• Scenario #2: Imagine there are 2 players in the server (Player1 and Player2). Player1 is able to hear when Player2 talks. Next, Player2 presses a TextButton that locally updates their own AudioDeviceInput.AccessType from the default value of “Deny” to “Allow”. This makes it so that Player1 cannot hear Player2. However, if Player2 presses the TextButton again to update their own AudioDeviceInput.AccessType back to “Deny”, they still cannot be heard by Player1.
  • I tested Scenario #2 with the following setups and observed the same behavior:
    • With VoiceChatService.EnableDefaultVoice enabled. None of the AudioEmitters/Listeners were modified during this test; only the AccessType property of the AudioDeviceInput was changed.
    • With VoiceChatService.EnableDefaultVoice disabled. For each player, an AudioDeviceInput, a Wire, and an AudioDeviceOutput were all created on the server within the player object and hooked up to one another (Source is the DeviceInput and Target is the DeviceOutput). The Player property of the AudioDeviceOutput was set to the other player in the game. This means that the AudioDeviceOutput within the Player2 object had the Player property set to Player1, and vice versa.

Server-side updates to AccessType

• Assuming an AudioDeviceInput is created on the server side, the server can update its AccessType property, which is replicated to every player in the server.
• Using Scenario #2 from the previous section of this post, here’s another example. Player1 and Player2 have just joined the game and are able to hear one another. The server updates Player2’s AudioDeviceInput.AccessType to Allow. Player1 is still able to hear Player2.A few seconds later, the server sets the AccessType back to “Deny”. Player1 is still able to hear Player2. After that, Player1 presses a TextButton that locally updates the AccessType of Player2’s AudioDeviceInput to “Allow”. Now, Player1 cannot hear Player2. When Player1 presses the TextButton once more to locally update the AccessType of Player2’s AudioDeviceInput back to “Deny”, Player1 is still unable to hear Player2.Even if the server forcefully sets Player2’s AccessType to “Deny”, nothing changes, meaning that Player1 cannot hear Player2 until Player2 rejoins the session or until the existing AudioDeviceInput of Player2 is replaced with a new one.

[ReallyLongArms]

Ahh I see – for the time being you might be able to use CollectionService/tags to speed up lookup, but that’s not as general; I can see how GetConnectedWires makes this nicer.

We’ll discuss and get back

[OnceWasGrand]

In terms of moderation, how will users go about reporting people who say, use a broadcasting system, where their user isn’t shown but they are speaking inappropriate things.

[ReallyLongArms]

Hey XenoDenissBoss1,

Swapping in and out audio effects is unreasonably inefficient as were pretty much forced to constantly cycle through every wire and whatever else other instances needed just to make the entire process work. One idea i have to potentially fix this issue would be some sort of “Wire Hub” instance. Basically it would be an instance that lets you hook up multiple input wires and a single output wire just so you dont need to constantly rechain many other wire instances.

You can use AudioFaders as hub nodes for organization; for example accepting many input streams and producing one output stream. With .Volume set to 1, this does not modify the input streams at all – does that help?

[ReallyLongArms]

Since player voices are networked, changing AccessType or calling SetUserIdAccessList locally is not guaranteed to work well – however, that server-side update behavior does not sound intended; we’ll look into it

[Laqotaa]

We need like a settable table value for players the Emitter will ignore so we don’t hear our own voice.

[Rileyg1974 (RileyLagiEmosi)]

I’m turning on the Audio API stuff, and i’ll share my thoughts once i get a grip of how it works