AudioDeviceInput with AudioAnalyzer:GetSpectrum

Feature Requests Engine Features
,
1

Regarding the AudioAnalyzer class GetSpectrum method,

GetSpectrum

Array

Returns the frequency spectrum of the last audio buffer, as an array of numbers. The elements of the array are root-mean-square volume levels, evenly spaced from 0 hertz to 24,000 hertz. If any of the analyzer’s inputs come from an AudioDeviceInput, or this method is used from a server script, it returns an empty array.

Usage

I’ll keep it short so you don’t have to read too much.
AudioDeviceInput with GetSpectrum would invite new types of games.

Concerns

Of course there are also some concerns like, people saving conversations on a webserver and such for malicious intent, but there are positive things like.

Examples

Phasmophobia
There could be a Phasmophobia inspired game, where you would use the spirit box and speak to ghost, or use to ouija board. There are already phasmophobia type games but this adds to the experience (speech recognition model), this could be useful for a bunch of horror games

Voice activated commands
Like probably a game where you say something and something happens, like Simon says.
(would be sent through a speech recognition model, probably either make this in Roblox or have the developers make a webserver then send it through a model like whisper or sum)

Karaoke
yk listening for the spectrum of their voice the pitch and such.

TL;DR
AudioDeviceInput with AudioAnalyzer:GetSpectrum usable upon using ID Verification.

I really want to make these unique type of experiences.

3 Likes
2

I never understood this. This exists (despite already implemented to not record inputs), how is anyone supposed to decypher words from just the spectrum?

1 Like
3

the spectrum contain like frequencies of ur audio, you can save that as a wav file and send it through a machine learning or speech to text model. and it’ll return you the words. usually theres an external server you send the wav file probably using httpservice and it returns the words.

4

Then just, I don’t know, implement a prompt whether an “experience” can use your microphone for audio analysis.

2 Likes
5

yeah probably roblox could add something on their webpage like “This experience records you microphone” or something like that

1 Like
6

A Speech to Text API is already on the roadmap which should cover most use cases where a developer actually wants input from the microphone.

It’s already said that it won’t be allowed to use GetSpectrum() with AudioDeviceInputs anytime in the foreseeable future, mainly because of privacy (but there are MANY more issues than that if you wanted to use it for speech recognition). Even if you add a confirmation prompt, it doesn’t solve the issue of bad actors storing people’s voice recordings.

2 Likes
7

Hi, thanks for the request. We are exploring the feasibility of this feature but do not yet have a timeline.