Page MenuHomePhabricator

Stormsound
Updated 324 Days AgoPublic

Vision

stormsound.png (366×469 px, 121 KB)
Stormsound is a C++ library for playing sound, both as an "event" and as a "stream". It works with the Opus codec from Xiph.Org as well as SDL2 and SDL_Mixer.

Stormsound will be used by both Trailcrest and Anari to play all sounds - music, speech, and Foley - without putting a huge strain on the user's system.

Features

High Performance, Low Overhead (g5)

Stormsound needs to be particularly careful of resources, favoring CPU performance first, and memory limits second.

Memory Limits (g4)

Ideally, it should be possible for the user to define/limit how much memory Stormsound can consume in total, so a developer doesn't inadvertently consume all available memory by preloading too much.

Event Audio Playback (g5)

Plays a sound on cue. Can be started only.

Stream Audio Playback (g5)

Plays a sound as a stream (sequence of packets), which can be synced to an Anari timeline. Can be started, paused, and stopped.

Preloading (g5)

Sounds can be loaded from file into memory in advance, and also unloaded. This allows faster playback. For example, all the audio for a level or a cutscene should be loaded just before the level or cutscene is displayed.

Batches (g4)

Sounds can be grouped in user-defined batches, which are loaded and unloaded on demand.

Load Progress (g3)

To facilitate loading screens, should report when loading is underway, and when it's complete. (Bonus points for percent-loaded updates.)

Volume and Pan Control (g5)

The volume and panning of a particular sound can be adjusted.

Audio Position in 3D (g5)

The volume and pan of an audio clip can be set in the context of the sound's origin position in visual space. This is represented as a set of coordinates (X,Y,Z) on a defined canvas size (set separately.)

  • X is the horizontal position. 0 would be the leftmost visual position, and 1024 would be the rightmost visual position on a 1024px-wide canvas. Values less than zero and greater than the canvas size would represent positions off the screen.
  • Y is the vertical position. This is ignored for now, but it is possible to simulate this with audio; we'll research and implement someday.
  • Z is the distance from the listener. It should be 0 by default, representing it being right next to the player. This subtly affects volume, the same way distance does.
NOTE: In later versions of Stormsound, this feature is where we can really go nuts! We may be able to add effects like reverb that are added on-the-fly based on the Z value and a user-specified room type and room size. Don't attempt this crazy stuff until we've shipped Stormsound 1.0, however. Keep it simple for 1.0; just volume and pan animation.
// Example API call for audio animation. This is entirely theoretical; actual API may look different.
stormsound.canvas(640,480,100);  // defines virtual space as 640x by 480y by 100z
sound mysound = stormsound.stream("dialogue3.opus");
sound.position(200, 500, 10);  // sets the sound source as 200x, 500y, 10z
mysound.start();

Fade In/Fade Out (g3)

Allow fading in a stream (or event?) sound on play start, or fading out on play stop. The option to fade in or fade out would be specified as a parameter on the API call to start or stop the sound, perhaps like this:

sound mysound = stormsound.stream("funky_music.opus");
sound.play(Stormsound::Effect::FadeIn);
NOTE: Although this is only a g3, this would be a good place to start before implementing Audio Animation; it's simpler, but built on the same technical principle of live volume update.

Audio Animation (g4)

The volume and pan of an audio clip can be adjusted live (during playback) to simulate movement of the sound in visual space.

// Example API call for audio animation. This is entirely theoretical; actual API may look different.
stormsound.canvas(640,480,100);  // defines virtual space as 640x by 480y by 100z
sound mysound = stormsound.stream("dialogue3.opus");

// animate so it sounds like the character walks towards the player and to the right
mysound.keyframe(0, 100, 200, 20);  // at 0 seconds, sound source is at 300x200x20
mysound.keyframe(30, 500, 200, 3);  // at 30 seconds, sound source is at 800x200x3
// Stormsound would have to actually calculate and perform the adjustments itself; this is known as "tweening" in animation.

mysound.start();

Volume Normalization (g3)

The volume of sounds are automatically adjusted by default to be the same perceived loudness. This might occur by amplifying a sound to maximum-without-clipping, before applying other volume adjustments. Optionally, this feature can be disabled by the user on a per-sound basis.

Panning Animation (g5)

The pan of an audio clip can be adjusted live.

Optionally, this feature can be disabled at runtime by the player.

Classes

Stormsound Singleton

Last Author
memateo
Last Edited
Sep 24 2021, 8:23 PM

Document Hierarchy

Event Timeline

jcmcdonald edited the content of this document. (Show Details)
atprice added a subscriber: atprice.
jcmcdonald changed the edit policy from "All Users" to "Stormsound [Project] (Project)".Jan 27 2017, 12:01 PM
jcmcdonald edited the content of this document. (Show Details)
jcmcdonald added a subscriber: memateo.
jcmcdonald edited the content of this document. (Show Details)
jcmcdonald edited subscribers, added: tfu; removed: atprice.
jcmcdonald edited the content of this document. (Show Details)