FMOD fights me when I try to add background music to my game

EDITED TO BE UP TO DATE

Finally got background music playing in my game.

Sometimes my (long running) sound stops playing. I think the FMOD channel is being victimized, probably to play someone’s footstep noises.

I think FMOD victimization priority should be:

  1. All 3D sounds
  2. Only then victimize local player-only ambient sounds

Or

  1. Let me specify that a Sound is high priority and to never victimize it.

I cannot hack around it short of trying to manage the channels myself by creating 32 local sounds for each player and syncing them over the network when I want to play 3D sounds.

I just confirmed that when FMOD victimizes a channel, Sound.IsPlaying is not toggled to represent the new state of the sound. I assume TimePosition and any other internal state is also not updated. So it is not possible to detect the failure mode and restart.

13 Likes

#######ShortSoundsMatter?

I’m not a fan of shortest-first. If I had a relatively short explosion occurring right next to me, I’d want to hear about it.

The simplest solution to your case would be to ensure that non-3D sounds won’t get cut off by 3D sounds. It’s so simple that I assumed it was already a thing. Prioritizing client-only sounds could be another solution.

I’m sure that the engine already prioritizes based on proximity. But, if it doesn’t already, it should take loudness into account. That way the perceptually quietest sounds will be cut first.

In this sense, non-3D sounds would always get priority, because they would always be perceptually closer to the players ears than any other 3D sound ever could be.

1 Like

I think any ordering that is solely based on length is going to work great in one situation and be dreadful in another. I don’t really know how FMOD works but I think some kind of distinction between 3D and non-3D like @Anaminus mentions could work out.

1 Like

Actually I agree. This is much more elegant than what I suggested.

As another alternative, we could give control to devs and add a priority property to the Sound object itself.

1 Like

Seems like something @spotco could explore!

1 Like

Please.

It literally took me all afternoon to figure out how to play background music in SFOTH and it still isn’t perfect.

1 Like

So I’m planning on doing a few sound changes, this stuff is mainly making workspace sound replication work better.

I’m totally open to having some sort of sound priority system (necessary since we don’t expose the separate concepts of FMOD::Sound and FMOD::Channel), any ideas for what kind of API you’d want?

2 Likes

In the same process, can you make it so sounds created on the client with filteringEnabled not error saying only the server can play timeposition?

Specifically referencing this thread
http://devforum.roblox.com/t/only-use-a-server-script-to-set-timeposition/22736

Yeah that’s going to be fixed. That was necessary before since nothing about a Sound was replicated (other than its creation).

Will be doing a post with the updated API within the coming weeks.

1 Like

This is my use case:

I want to play background music for each client in the game. However, I want them to all be playing the same track at roughly the same (+/- 1 second) position. I want to be able to control the current track from a server-side script. This allows me to change the sound track for the main game vs. lobby vs. different regions of my map can have different music.

What is missing/broken right now that makes this hard:

  1. The FMOD channel victimization issue. FMOD will occasionally stop playing my sounds. This is only noticeable when you are trying to play a 2 minute long chunk of background music. To solve this I think the cleanest thing to do is add a priority property to the Sound object and to make all Sounds low priority by default. I would then set my music to high priority. Anaminus’s suggestion is also good. By default local sounds should have higher priority than server sounds. This would fix most people’s issues without them needing to know about this property or do anything.

  2. The server has no way to query the length of a Sound. The sounds are never loaded server-side, which makes sense. But this also makes it impossible to get the length of a Sound asset server-side. The result is I have a giant table on the server that I have to populate with the length of all my sound assets. I need Sound.Length to be populated server-side. Probably the most elegant way to do this is to have the website populate this field on upload when it generates the Sound object xml. The website already knows how to check the length of a sound because it rejects uploads over 2 min. Note that if we do this right we can make Sound.TimePosition work server-side without loading the sound server side.

  3. ROBLOX makes me waste time trying to stitch together 2 min sections of tracks because the website limits uploads to 2 min. Why do we have to play this game. Let me upload tracks that are as long as I want. If we are worried about people using ROBLOX as a rapidshare site for sounds then do some mild XOR encryption on the asset when uploaded so that the bits are unintelligible. If we are worried about moderation cost, let me pay more to upload them. If it is a bandwidth cost issue then add support for streaming (which I would prefer anyways because the results of requesting a 2 min chunk of mp3 is nondeterministic to game play experience - I imagine some players on the iPhone are getting choked out whenever I request the next sound chunk). Note that #1 and #2 above makes stitching even more annoying. Unity doesn’t make me jump through these hoops.

  4. Add support for sequencer formats like MIDI, .mod, .s3 and .it. These don’t need to be moderated and they are much more bandwidth efficient. FMOD already knows how to play them. I just need the website to allow it.

  5. We might want to canonize the idea of music vs. sound effects. Other games let me control the volume of sound effects vs. the main background track. We might want to canonize this notion in our API so that we can provide this functionality as well. The main use case here is for playing games on the PC when you want to listen to your own music but still hear sound effects.

If you play any of the ROBLOX Xbone games I think it is really weird that there is no background music. Every console game since Super Mario has a sound track. Games feel very low quality without music and this is a very high leverage thing to fix in terms of !/$

7 Likes

Your final point is totally spot on. I’ve never thought much of it because I’ve been playing ROBLOX since 2007 and almost no game on ROBLOX has had good sound design…

Ha I added an extra point to the end so now you look like an idiot.

2 Likes

No, you look like an idiot because I was addressing your last paragraph and you didn’t read what I said!

Wait I guess I didn’t really mention what I was referring to…

2 Likes

I must be playing too much Idiot Simulator.

2 Likes

??? This just looks normal to me!

So a bunch of the changes I have down the road are addressing a couple of points you mentioned. Specifically, I’m going to make sounds (and all their relevant properties) replicate similarly to other networked objects like physics/etc. The server is going to be loading the sound assets to know their length, and in “networked sound” mode, the TimePosition and other properties will be server-authoritative (local-only sounds will work more or less as before).

  1. I’ll see about adding some sort of sound prioritization system. It’ll probably be something like a 3-priority (high, normal and low) enum. You also must be playing a bunch of sounds simultaneously in order to run into this I imagine. How many does your game play at once?

3 & 4) It’s a moderation and infrastructure issue thats been discussed internally before, so unfortunately can’t give any guarantees on if/when that kind of thing will happen.

  1. I don’t see a whole lot of gain in the bgm/sfx distinction other than the separate volume controls. Again, volume controls are something that could currently be implemented per-game in lua (and are there any popular plugins out there that make that kind of thing easier?)

Finally, my personal opinion about the state of sound design is that the root cause is the “100 robux per upload” deal. My changes will at the very least allow sound “sheets” (multiple sfx in one sound) to work both locally and on the server (since TimePosition will be accurately replicated).

3 Likes

Let’s fix the upload deal then. I should be able to do it from in Studio like I can with images. If my game is making ROBLOX non-zero dollars I shouldn’t even pay to upload sounds.

Sound sheets sound like an awful dev experience to me. That is making the problem worse instead of fixing it. It’s fine if that is how the assets are managed behind the scenes, but as a dev I should be ignorant to how the sausage is made. Think of how I’m going to be reusing sound effects in my different games. Am I going to load a bunch of sheets where I only care about 1 or 2 of the 100 sound effects I’ve saved there? Am I going to have to maintain a giant table in a lua script with offsets for each effect and their length? This is taking the current problem and making it 100x worse. This would be the worst sound management solution of any game engine that I am aware of.

I am also concerned about syncing the TimePosition across client/server. If done in a naive way, as a property update, it will spam guaranteed ordered/guaranteed delivery packets for every playing sound in the level like the old animation system did. It should only send {soundguid, start, timestamp} and {soundguid, stop, timestamp} tuples over the wire.

3 Likes

The soundsheets stuff is a fact-of-life “you can do this right now” kind of thing (I know Virtual Piano uses that trick, and I imagine a couple other sound-based games do too. I’m also using it for my own rock-band like game I’m making). Virtual piano more or less has a lua implementation of what I’d like to do (it uses remote events to replicate localsounds across different players).

I totally agree with the lowering barrier of entry for sound upload. I feel that right now, the high cost really discourages experimentation and iteration with sound design.

As for the TimePosition sync implementation, my current thought is that the server will be “authoritative” with the properties, but TimePosition will only “resync” when there’s a large-enough difference between server TimePosition and client FMOD TimePosition (the goal here is to avoid as much skipping as possible).

My gut feeling here is that in practice the system can be designed so that resyncing is not required.

Here is the situation I think resync is most applicable:

  1. New player joins my game
  2. Server tells player to fetch Sound Asset ID 4354356 and start playing it 23.05 seconds into the song (right now my script tracks the sound cursor but maybe it can be done with Sound.TimePosition in the future - the issue with this is that all the sounds are local so you actually have N Sounds locally all playing with the same cursor)
  3. New player fetches this sound which takes 10 seconds
  4. New player erronously starts playing at cursor position 23.05 where really it should be 33.05 if we are counting loading time.

This is a bug in my current implementation because I don’t know how to query the length of time between when the asset was requested and when it was loaded. Maybe there should be an event on the Sound object that fires when the asset is ready to play (maybe there is and I don’t know about it), coupled with an IsLoaded flag on the object.

Or maybe asset load state is best queried through ContentProvider, which would be neat because then it would work for decals and maybe models as well.

1 Like