A sound in a world is described, not recorded: a recipe of a few numbers (wind at this strength, water this fast, a footstep on stone) that each machine's mixer turns into sound. What makes a sound belong to the world is what it is bound to: a field asked at the listener's place (the wind), a part and its place (the brook), events a behavior raises (footsteps), an event from a rule (a cry), or state (the burner's roar). A long-lived sound is a part; a sound that starts once and ends by itself is an act, play. Sounds are heard only: a server has none, and nothing about a sound crosses the network beyond the events and state it is bound to. A world that sounds rich downloads no audio at all.

Words used on this page:

Voice

One sound playing in the mixer.

The ear

The listener in a view: a place and a facing. Typically the player's head turned the way the camera faces; the content never says whose.

Gain

Loudness as a multiplier, 0 silent, 1 as recorded.

Attenuation

How a sound fades with distance from the ear.

How you write it

Bound to a field, asked at the ear: the wind

part windSound, lives: client                             // everywhere: not placed
  part breeze: audiomixer.wind(intensity: 0.25, seed: 1, glide: 0.3 s)
  part gale   : audiomixer.wind(intensity: 0.9, seed: 2, glide: 0.3 s)
  field mix: fraction, within: 0 ..= 1                 // derived: not mutable
  derive(mix, clamp(clamp(0.12 + 0.30 * weather.cloudLow + 0.45 * weather.rain - 0.10 * weather.snow, within: 0 ..= 1)
                    + 0.3 * clamp((ear.y - land.height(ear) - 3 m) / 120 m, within: 0 ..= 1), within: 0 ..= 1))   // 0 all breeze, 1 all gale
  derive(breeze.gain, cos(mix * 90 deg) * (0.55 + 0.45 * mix) * (1 - shelter))
  derive(gale.gain,   sin(mix * 90 deg) * (0.55 + 0.45 * mix) * (1 - shelter))
field shelter: fraction = 0, within: 0 ..= 1, mutable: true, lives: client   // 0 in the open, 1 underground
on(created, {
  start(chase(shelter, toward: if occupied(underground, by: ear) { 1 } else { 0 },
              settle: rate(63%, per: sim(0.8 s))));
})
part roomTone: audiomixer.wind(intensity: 0.02, seed: 3, glide: 0.2 s), lives: client
derive(roomTone.gain, 0.35 * shelter)

Two versions of the wind recipe, a gentle breeze and a gale, blended by the wind's strength where the listener is, the cosine and sine split keeping the total loudness even. Underground the wind and brook fade and a low hush rises, easing so that 63% of the change happens in 0.8 s. The wind's strength is not a field of the world's wind: the sound makes it from the weather's numbers, as the line above does: 0.12, plus 0.30 of the low cloud's cover, plus 0.45 of the rain, less 0.10 of the snow, held within 0 and 1 (0.13 in clear weather, 0.25 fair, 0.34 cloudy, 0.42 overcast, 0.87 in rain, 0.32 in snow), and then up to 0.3 more as the listener's height above the ground rises from 3 m to 123 m. Each ground effect states its own wind in the same way (Sky and weather: the wind). audiomixer.wind is the mixer's wind recipe (wind(rate, seed, intensity)).

Bound to a part and its place: the brook

part brookSound, lives: client
  field along     : meters                                    // derived: how far along the course the ear's nearest point is
  field rushShare: fraction, within: 0 ..= 1                  // derived: not mutable
  field bedStones: List[rock] = filter(rocks, r => land.brook.nearest(r.place, within: 1.4 m) != None)
  fn waterSpot(a: meters) place {                             // the water's surface a meters along the course
    let q = land.brook.spotAt(a);
    (q.x, land.brook.waterLevelAt(a), q.z)
  }
  derive(along, if let Some((_, a)) = land.brook.nearest(ear, within: 2000 m) { a } else { 0 m })
  derive(rushShare, clamp(remap(land.brook.waterSpeedAt(along), from: 0.4 m/s ..= 2 m/s, to: 0 ..= 1), within: 0 ..= 1))
  part trickle: audiomixer.water(intensity: 0.2, heardTo: 70 m, seed: 11, rolloff: 1.4), at: waterSpot(along)
  part rush   : audiomixer.water(intensity: 0.8, heardTo: 70 m, seed: 12, rolloff: 1.4), at: waterSpot(along)
  derive(trickle.gain, 0.9 * cos(rushShare * 90 deg) * (1 - shelter))
  derive(rush.gain,    0.9 * sin(rushShare * 90 deg) * (1 - shelter))
  each k in 0 ..< bedStones.count {
    let r = bedStones[k];
    let a = if let Some((_, a)) = land.brook.nearest(r.place, within: 1.4 m) { a } else { 0 m };
    part stone[k]: audiomixer.water(intensity: 0.85, heardTo: 16 m, seed: 41 + k, refDistance: 1.5 m,
                                    attack: 0.2 s, release: 0.5 s),
                   at: waterSpot(a) + (0, 0.2 m, 0)             // a stone rushes 0.2 m over the water beside it
    derive(stone[k].gain, 0.9 * (1 - shelter))
  }
  part arch    : audiomixer.water(intensity: 0.55, heardTo: 35 m, seed: 21, refDistance: 2.5 m), at: bridge.archSpot
  part cascade: audiomixer.water(intensity: 1.0,  heardTo: 90 m, seed: 22, refDistance: 3 m, rolloff: 1.0),
                at: waterSpot(land.brook.alongAt(22))      // where control point 22 begins (the first is 0): the cascade's middle
  derive(arch.gain,    0.6 * (1 - shelter))
  derive(cascade.gain, 0.8 * (1 - shelter))

The brook's water is heard from the point of its course nearest the listener, at the water's level: walk along the bank and the sound walks with you. brook.nearest(ear, within: 2000 m) answers the distance and the distance along the course (Shape tools), and the brook answers waterLevelAt(along) and waterSpeedAt(along) (Terrain). It is a trickle where the water is slow (0.4 m/s) and a rush where it is fast (2 m/s and over); the ramp between is a straight line, clamp(remap(...)). A bed stone is a rock within 1.4 m of the course, found by filter; each rushes loudly to 16 m, 0.2 m over the water beside it, and the mixer plays only the three nearest. The arch under the bridge is 0.3 m over the water at the bridge's middle, a spot field of the bridge (bridge.archSpot: its middle, at that height), and the cascade is the course where control point 22 begins (alongAt(22)), at the water's level. Their gains are 0.6 and 0.8. at: waterSpot(along) is a placement that is different in every view.

A one-shot: play

enum surface { grass, wood, stone, bridge }
fn underfoot(here: place, b: being) surface {               // what the heel lands on
  if b.onPlatform { wood }
  else if occupied(underground, by: b) { stone }
  else if abs(here.x - bridge.place.x) <= 1.2 m && abs(here.z - bridge.place.z) <= 3.9 m { bridge }   // the road: 2.4 m wide, 7.8 m long
  else if planDistance(here, cottage.place) < 8 m && here.y - land.height(here) > 0.15 m { wood }  // the porch and its steps
  else if any(rocks, r => planDistance(here, r.place) < r.radius * 0.8 + 0.1 m) { stone }              // a rock or an edging stone
  else { grass }
}
part footSounds, lives: client                            // its rules run on each visitor's machine, and only there
  on(walkgait.footDown(foot, of: b), {
    let surface = underfoot(foot.heel, b);
    let material = if surface == wood { wood } else if surface == grass { grass } else { stone };
    let gain = if surface == wood { 1.6 } else if surface == grass { 1.4 } else { 1.1 };
    play(audiomixer.step(material: material, weight: clamp(0.25 + 0.15 * b.speed, within: 0.2 ..= 1)),
         at: foot.heel, heardTo: 32 m, gain: gain, priority: 1, refDistance: 1.5 m, rolloff: 1.5);
    if surface == bridge {                                // the arch answers under the stone: a wood knock, gain 0.7
      play(audiomixer.step(material: wood, weight: 1), at: foot.heel, heardTo: 32 m,
           gain: 0.7, priority: 1, refDistance: 1.5 m, rolloff: 1.5);
    }
  })
  on(zapLanded(on: b), {
    let pitch = if b.height < 1.7 m { 200 Hz } else { 120 Hz };
    play(audiomixer.voice(pitch: pitch), at: b.chest, heardTo: 40 m, gain: 1.2, priority: 2, refDistance: 2 m, rolloff: 1.2);
  })
  on(bullBeat(beat: k, at: p), {                          // the Minotaur's six beats, from a world message
    play(audiomixer.voice(pitch: beatPitch(k)), at: p, heardTo: 40 m, gain: 1.2, priority: 2, refDistance: 2 m, rolloff: 1.2);
  })
enum beat { windup, charge, hit, pullup, shove, stand }
fn beatPitch(k: beat) frequency {
  match k {
    case windup { 52 Hz }
    case charge { 44 Hz }
    case hit    { 38 Hz }
    case pullup { 58 Hz }
    case shove  { 46 Hz }
    case stand  { 60 Hz }
  }
}

A step sound at the heel of the kind the surface gives (grass a soft hiss and crunch, stone a short ringing tick, wood a hollow knock), heavier the faster the being moves. underfoot answers a surface, and each surface has its own gain: grass 1.4, stone and bridge 1.1, wood 1.6. The bridge is stone, and under it a hollow note: a second, wooden knock at gain 0.7, written as a second play. Each test in underfoot is the island's: the bridge's road is 1.2 m either side of its middle and 3.9 m along it, the cottage's porch is within 8 m of the cottage on the plan with the heel more than 0.15 m over the ground, and a rock or an edging stone is under the heel when the heel is nearer its middle, on the plan, than its radius times 0.8 plus 0.1 m. A zapped being cries out at its chest, 0.7 of its height up (zapLanded is a message the server sends, Events: a message is its own trigger). The Minotaur's beats are a world message too, bullBeat, sent where the Minotaur is (1.9 m up, 1.6 m for the hit) and sounded in a deep voice, 38 to 60 Hz; the island also squeaks at 640 Hz when the Worm rises, which a world message would carry in the same way, and the Worm's lines and the guards' notes come free from spoke and voice:. The step handler needs no test of distance: heardTo: 32 m already stops a voice beyond it. play is an act: it may appear in an on(...), never in a derive.

footDown(foot, of: b) is raised once for each foot that comes down, and only for a being that is walking (not limp, seated or getting up) and on the ground; a being just set down, whose gait has only begun, raises none. foot.heel is a place, the heel's contact point. A one-shot played without a seed: takes the next number in a count, so no two steps are alike; give a seed: to make one exactly again.

Bound to state: the burner's roar

part burner
  part flame: effect(...), at: nozzle
  part roar : audiomixer.fire(crackle: 0.4, roar: 1, heardTo: 140 m, seed: 31, bus: effects, refDistance: 3 m, rolloff: 1.0,
                              priority: 3, attack: 0.1 s, release: 0.5 s), at: nozzle
  derive(roar.gain, if flame.firing { 1.5 } else { 0 })

The roar is at gain 1.5 while the flame burns and 0 when it does not, with a short attack and a longer release, so a burst swells and dies away (the fire recipe has crackle 0.4, roar 1, and is heard to 140 m). A sound part has two fields, gain and playing. gain is its loudness, a multiplier a formula may define, and it starts at the option's value (1 if none). playing is a switch: a part that is not playing makes no sound and holds no voice. A roar that should be louder or quieter derives gain; playing is for a sound that is simply on or off.

What it means

Parts and one-shots

A sound that lives as long as the world (the wind, the brook) is a part, placed and carried like a light, with a gain a formula may define. A footstep or a cry is played and forgotten: no part, no state. Both are needed, for different lifetimes. Whether long-lived sounds are parts, or bindings the content states on other parts ("the burner roars while it fires") that each view realizes, is open.

Heard, not shared

Every sound runs in each view, on what that view already knows: the weather, the brook's course, the beings' movement, the flame's bursts. So nothing is sent for it. A sound in the server's world (a guard who hears footsteps) would need the server to know a step happened without playing it.

The ear

Content never names a player, yet the wind and the brook are defined by where the listener is. ear names "this view's listener": a place and a facing. It is a place, so it goes where a place does (ear.y, land.height(ear), brook.nearest(ear, ...)), and occupied(underground, by: ear) asks whether that place is in the area. Whether it is the character's head or the camera is the browser's choice; the usual third-person convention hears from the player's head, turned the way the camera faces (setListener).

What content states and what it does not

The options of a sound. A sound part and a play take, beside the recipe's own numbers, heardTo: (the range) and these named options. Left out, each takes the value in the right-hand column, the island's most common one; the code states every sound's own, and the island's sounds are in the listings above.

Option	Meaning	Left out
gain:	loudness, a multiplier	1
seed:	which voice of the recipe: a different seed is a different wind, brook or step	1 for a part; a one-shot takes the next number in a count
bus:	ambience or effects: the group a sound belongs to (the browser's mute works on the master and the groups)	ambience for a part, effects for a play
priority:	when voices run out, the lowest is stolen first (1 a step, 2 most sounds, 3 the burner)	2
refDistance:	the distance inside which the sound is at full loudness	2 m
rolloff:	how steeply it falls beyond that (1 the usual)	1.2
attack:, release:	seconds to swell when a voice starts and to die away when it stops	0.3 s, 0.6 s
glide:	the gain ramp: a changed gain is reached over this long, not at once (the wind 0.3 s, the room tone 0.2 s)	0.12 s

• How far a sound carries (heardTo) is a fact about the world: the brook 70 m, a bed stone 16 m, a footstep 32 m, a cry 40 m. How it fades on the way is the product of two curves. The taper is 1 up to 60% of heardTo, then a smoothstep down to 0 at heardTo, and the voice is stopped beyond it. The inverse model is refDistance / (refDistance + rolloff * (d - refDistance)), with the distance d held between refDistance and heardTo. A sound's loudness at a distance is its gain times both. When a voice is freed is the mixer's.
• Mute and the buses are the browser's. The N key silences the master bus over 80 ms; the person's volume for each bus is theirs. Content names a bus only on a sound (bus:), to say which group it belongs to.
• Loudness is a requirement; the threshold for sending a gain change (2%, a hundredth for the wind) is a mechanism below the content.
• Every bed stone has a voice; that only the three nearest play is the mixer's resource choice, as is the ceiling of 32 voices.
• A sound with a value (the wind at this strength) may be made inside the recipe or by blending two copies; the mixer's recipes fix their numbers when they start, so a world blends. Content says only "the wind sound at this strength".

Shelter

Hushing the weather underground is one value every outdoor sound's formula reads. In the design (to be built), sound's shelter reads the same answer as falling weather blocked like a shadow (blockedByRoofs, Sky and weather), so a porch roof would quiet the rain's patter.

Surfaces

What a foot lands on is worked out after the fact from where things happen to be. It would be more naturally stated by each part (the porch is wood, a rock is stone), and the same answer would serve footprints, dust and grip. A surface may be a mixture (the bridge: stone over a hollow).

The package: audiomixer

Sound for a 3-D world, with no device and no C: a real-time mixer, positional audio and procedural synthesis. Output is the same for the same commands: seeded recipes, no clocks.

Recipe	Numbers	Kind
wind(rate, seed, intensity)	intensity 0 (a breath) to 1 (a gale)	continuous
water(rate, seed, intensity)	0 (a trickle) to 1 (a rushing stream)	continuous
fire(rate, seed, crackle, roar)	a balloon burner is roar 1; a campfire roar 0.1	continuous
step(material: grass | wood | stone, weight:, seed:)	the material; weight 0 (light) to 1 (heavy); a different seed is a different step	one-shot
bird(rate, seed, pitch)	a pitch	one-shot
voice(rate, seed, pitch)	a startled "ah", a man 110, a woman 200, 0.35 s	one-shot

Control: playSynth (not placed) and playSynthAt (placed, with a velocity and an attenuation); setGain(id, gain, seconds), a linear ramp; setPosition; setOcclusion(id, amount), 0 clear to 1 blocked; setListener; bus gains for master, music, effects and ambience. Attenuation models are OpenAL 1.1's: inverse, linear, exponent, custom, none. When voices run out, the lowest priority, then the quietest, then the oldest is stolen. Recorded sound: WAV files decode (decodeWav); Ogg Vorbis and Opus are not decoded, and sounds are kept whole in memory. Not done: head-related filtering for direction, pitch and Doppler for recipes (a synth runs at its own fixed pitch). audiodevice carries the mixer to the speakers.

How it connects

• Effects: the same bindings; the roar follows the flame.
• Actions: play.
• Package behaviors: events in a behavior's contract (a foot down).
• Sky and weather: the wind, shelter.
• Terrain: the brook's course and water speed.
• Overlays: speech shown and voiced.

Open questions

1. Sounds: parts, or the view's bindings? Parts in the tree; bindings stated on other parts; or the browser's own, outside the content.
2. One-shot sounds: play(sound, at:, heardTo:) as an action.
3. Where an event is handled: closed. A rule inside a part with lives: client runs on each visitor's machine and only there (Events).
4. Which clock the shelter's chase counts: simulation or real time, since it is about the person's hearing.
5. Events in a library's contract: a gait should list the events it raises.
6. A voice for a person: pitch from height stands in for a property of the person.
7. Shelter by region or by shape: stated once, read by every engine.
8. Recorded sound: whether a world may carry an audio file, and in which formats.

← Effects | Contents | Images →