An overlay is anything drawn over a visitor's picture of the world: a status panel, a map, a row of action slots, a dialogue with a guard, the words above someone who has just spoken, a red glow at the screen's edge when the visitor is hurt. In ConeScript an overlay is not a kind of part. It is a layout (Layouts) made into an image and composited over the camera's image (Images), in each visitor's own picture. This page covers the overlays a world draws, how speech appears, who owns which overlay (the world or the browser), what the visitor's own preferences change, and how a press of the pointer finds its way back through the layers to the thing pressed.
Terms used on this page. A HUD (heads-up display) is the set of panels that stay over the picture while the visitor plays. A dialogue here is a panel that asks the visitor something and waits for an answer. Modal: while such a panel is up, it takes every press and key, and nothing behind it answers. A scrim is the darkened backdrop laid over the whole picture behind a modal panel. A plate is a small panel with a name or words on it, placed over a point in the world.
Four kinds of interface
What a game shows its player can be sorted by two questions: is it placed in the world's space, and is it part of the world's story (something the characters themselves could see)? The answers give four kinds, and a world uses all of them:
| Kind | In the world's space | In the story | Illustrations |
|---|---|---|---|
| Diegetic | yes | yes | a mirror, a notice board; a ship's terminal; a watch on the wrist |
| Spatial | yes | no | a name plate over a head; a marker over a quest-giver; a path drawn on the ground to a goal |
| Non-diegetic | no | no | a status panel, a minimap, a hotbar; an inventory window; a quest log |
| Meta | no | yes | a red glow at the edges of the picture when hurt; frost creeping in from the corners when cold |
The language does not need four mechanisms for these. A diegetic interface is a layout or a camera's image on a surface in the world (Images, Layouts). The other three are overlays: a spatial one is placed over a point in the world, a non-diegetic one is anchored to the screen, and a meta one is an image composited over the whole picture whose strength follows the visitor's state.
How you write it
An overlay is a part holding a layout's image, and the composite that lays it over the picture:
part hud field picture: image derive(picture, layout(...)) // the layout: see below composite(viewingCamera.picture, over: [layer(hurtEdge.picture, blend: add), layer(hud.picture), layer(talk.picture)], order: [world, hud, browser]) // the browser's own controls always last
A plate over a point in the world is an overlay(layout, above: part, ...) element (Layouts, "Placing a layout over a 3-D anchor"). An overlay shown for a while in answer to an event is the action show(element, for: d): it is shown in each visitor's picture from that moment for the duration d, then removed. Within a layout, you names the being the visitor drives, whose picture this is, and viewingCamera the camera drawing the picture.
An example HUD
What it shows
A HUD for a small world might hold a status panel (the hour and weather, the frame time, whether the visitor is playing alone, the keys worth pressing now), a minimap with a place panel under it, a hotbar of action slots, and a name plate over each other being's head. All its panels share one look, a dark glass. Written as a world's overlay, with example numbers:
part hud
field shown: bool = true, mutable: true, lives: client // a toggle hides it
field map: image = paintedOnce(topDown(land, walkway, brook, vegetation), texel: 2 m)
field picture: image
derive(picture, layout(anchored(List[element]() <-
column([
text(clockText(now(world), weather.state)), // "12:00 fair"
text(frameText(frameTime)), // "6.4 ms 156 fps"
text(if connected { "connected " <- msText(roundTrip) } else { "single player" }),
text(keyHints(you), visible: keyHints(you) != "")], // "S sit", "H hail"
style: styles.hudGlass, gap: 3 px, visible: shown,
anchor: (0, 0), pivot: (0, 0), offset: (14 px, 14 px), size: (172 px, auto)),
minimap(map, size: 176 px, visible: shown,
anchor: (1, 0), pivot: (1, 0), offset: (-14 px, 14 px),
center: onMap(you.place), zoom: 6,
markers: List[marker]() <-
each b in others(you) yield marker(onMap(b.place), rgba(232, 80, 60, 255), 8 px),
marker(onMap(ahead(you, 16 m)), rgba(255, 216, 74, 255), 4 px),
marker(onMap(you.place), rgba(255, 255, 255, 255), 9 px)),
box([text(placeText(you.place))], // "x 12 z -40"
style: styles.hudGlass, padding: (8 px, 3 px), visible: shown,
anchor: (1, 0), pivot: (1, 0), offset: (-14 px, 196 px)),
hotbar([slot(does: paint, key: F1, icon: icons.splat),
slot(does: zap, key: F2, icon: icons.bolt),
slot(does: jig, key: F3, icon: icons.dancer),
slot(), slot(), slot(), slot(), slot()],
anchor: (0.5, 1), pivot: (0.5, 1), offset: (0 px, -16 px), visible: shown),
each b in others(you) yield // the name plates
overlay(box([text(b.name)], style: styles.hudGlass, padding: (6 px, 2 px)),
above: b, by: b.height + 0.45 m, pivot: (0.5, 1), within: 80 m,
whenBlocked: fade(real(0.17 s)), visible: shown))))
on(action(toggleHud, by: you), set(hud.shown, !hud.shown)) // illustration: the action's name
Each line of it is a subscription: the clock text follows world time, the markers follow the beings, and the layout is laid out again only where something changed (Layouts). The key: on each slot is a suggestion; the visitor's own bindings win (Visitors).
Dialogues
An example dialogue
A dialogue with a guard is a modal panel over the picture: the whole picture is darkened behind it by a scrim, and in the middle a panel holds the guard's name, a few lines of words, one button for each question the visitor may ask, and a "Walk away" button. Once a question is chosen the buttons go dim; when the reply comes, it is shown in the words.
part talk
field guard: ?guard = None, mutable: true, lives: client // typed by its design, not a bare being
field words: text = "", mutable: true, lives: client
field picture: image
derive(picture, layout(
box([column(List[element]() <-
text(guardName(guard), style: styles.title),
text(words, wrap: 364 px, lines: 4), // a label that wraps (to be built: layouts.html)
each q in [1, 2, 3, 4] yield
button(questionText(q), key: q, does: ask(q, to: guard), enabled: offered(ask, to: you)),
button("Walk away", style: styles.close, does: walkAway), // illustration: the action's name
style: styles.dialog, gap: 8 px, padding: 18 px, size: (400 px, 392 px),
anchor: (0.5, 0.5), pivot: (0.5, 0.5))],
style: styles.scrim, anchor: (0, 0, 1, 1), modal: true, visible: guard != None)))
on(spoke(g, line, to: you), {
if g == guard { set(words, "The guard says: \"" <- line <- "\" Now choose a door."); }
})
The browser's own Escape closes any world panel, whatever the world has drawn, so content offers its own "Walk away" button beside it, as this dialogue does.
The buttons perform the visitor's action ask, carrying the question; the hall's rule on ask answers with say (Events, Text and structured text). Whether a question can be asked is the action's availability, so the buttons dim themselves when it is no longer offered.
A panel such as "Play with others" (a name, a line to the world's owner, and a request to be let in) is the browser's, not the world's (below, "Who owns which overlay").
Speech
Speech is an act; how it appears is the world's
A speaker (a guard, a door, a radio, a player, a narrator) works out its words as a pure value, then performs the act say(speaker, words, to: v) at a moment. It puts the words nowhere. The world has a rule that answers every speech act and shows it as its designer chose: a bubble above the speaker, a subtitle, the words spoken aloud, a line in a chat log, several at once, or nothing out of earshot or through a wall. Only the act crosses the network; each visitor's machine shows it, and the visitor's own preferences (captions, translation) may apply on top.
The separation means one world shows speech as bubbles and another as subtitles, a door or a radio speaks by the same act as a guard, and a visitor who needs captions gets them in every world.
How you write it
A guard's act, and the world's rule that shows every speech act (the numbers are example numbers):
on(action(ask, q, to: g, by: v), say(g, answer(g, q), to: v)) // the guard's act
// the world's rule for every speech act,
// written at the top of the world's text (the root)
on(spoke(speaker, words, to: v), {
show(overlay(box([text(titleOf(speaker) <- ": " <- words)], // "Guard: ..."
style: styles.hudGlass, padding: (6 px, 2 px)),
above: speaker, by: speaker.height + 0.45 m,
pivot: (0.5, 1), within: 18 m,
whenBlocked: fade(real(0.17 s)),
inPlaceOf: namePlate(speaker)),
for: sim(6 s));
play(audiomixer.voice(pitch: if speaker == westGuard { 118 Hz } else { 98 Hz }),
at: speaker, up: 0.9 * speaker.height);
})
Here the words stay over the speaker's head even with the HUD hidden, and the addressee also gets the line in the dialogue panel (above).
What it means
- The act runs where the speaker's logic runs (an answer that reads a server-only field runs on the server). The rule on
spokeruns on each visitor's machine, once per visitor, and decides what that visitor sees and hears (Distribution). - Earshot is measured from the camera drawing the picture: the rule compares the plate's point with the eye. A line spoken further off leaves the plate showing the speaker's name.
- A bubble is in each visitor's picture, not in the world: it is not in a mirror's reflection, which is drawn before the overlays are laid over the picture. Words placed in the world instead (a spatial label) would be.
- The act carries the words. The speaker computes the words and the act carries them. The engine may still compress lines on the wire, which content never sees.
to: vnames the addressee, not the audience. Everyone near sees the plate, because the act goes to every visitor's machine; the addressee's panel takes the line, because the panel's rule acts only for its own visitor (on(spoke(g, line, to: you), ...)above).say(..., voice: 118 Hz)overrides the speaker's voice for one line.
Speech bubbles over the 3-D world
A bubble is a layout placed over the speaker (Layouts, "Placing a layout over a 3-D anchor"). A bubble may:
- be placed over a point 0.45 m above the head, its bottom middle on the point;
- be drawn only when the point is in front of the eye and on the screen, and otherwise hidden;
- fade out over about 0.17 s when a ray from the eye to the point meets something solid, and back in when it clears.
Not yet supported: holding a bubble inside the screen's edge when the speaker is off it; hiding it pixel by pixel by what stands in front (half a bubble behind a pillar's edge; depth in overlays, Images); keeping several bubbles apart so they do not cover each other.
Meta effects driven by state
A meta effect is an image over the whole picture whose strength follows the visitor's state: a red glow at the edges when hurt, the picture graying as health falls, frost at the corners in the cold. It is an overlay like any other, with a subscription or an animator on its strength:
part hurtEdge // illustration
field strength: fraction = 0, within: 0 ..= 1, mutable: true, lives: client
field picture: image
derive(picture, layout(anchored([
box([], anchor: (0, 0, 0, 1), size: (48 px, auto),
fill: linear((0 px, 0 px), rgba(200, 0, 0, 160 * strength), (48 px, 0 px), rgba(200, 0, 0, 0))),
box([], anchor: (1, 0, 1, 1), pivot: (1, 0), size: (48 px, auto),
fill: linear((48 px, 0 px), rgba(200, 0, 0, 160 * strength), (0 px, 0 px), rgba(200, 0, 0, 0)))])))
// and the top and bottom edges likewise
on(hardLanding(b, speed: s, cost: c), { // the engine's hard-landing trigger
if b == you {
set(strength, min(1, c * 4));
start(transition(strength, to: 0, over: real(0.8 s)));
}
})
The fade counts real time: it is about the person, not the world, which is what real time is for (Values, clocks). Not yet available: a glow that falls off round the picture's middle (a radial gradient), since vector2d paints only solid colors and straight-line gradients; the render package's vignette only darkens the corners, with no color; and gradient fills on boxes come from vector2d, which uilayout does not draw with (Layouts).
Who owns which overlay
| What | Whose | What the other may do |
|---|---|---|
| The browser's own controls: "Play with others" (F9, J), the window's Escape and F11; the reserved actions: leave, portal home, menu, mute, block | the browser | The world cannot hide, cover, reorder, restyle or intercept them, and never receives the reserved actions; they are composited last. |
| The browser's generic action menu, offering every action now available, by name or the world's label (the main way to act on a tablet with no keyboard) | the browser | The world gives the labels and the availability. |
| The HUD, dialogues, bubbles, meta effects, everything else drawn over the picture | the world | The browser composites them in the world's order, below its own controls. |
| Captions, translation and other preferences of the visitor | the visitor, applied by the browser | They apply on top of the world's manifestation. |
The reason for the line: a world that could hide or imitate the browser's controls could pretend to be the browser, and a visitor must always be able to leave, mute or block, whatever a world draws. That is also why the browser's controls keep the browser's style, untouched by a world's theme.
The visitor's preferences
Because only the act of speaking crosses the network and each visitor's machine manifests it, a visitor's own preferences can apply in every world without the world's help:
- Captions: speech that a world shows only as a voice can also be shown as words for a visitor who needs them.
- Translation: the words of an act can be put into the visitor's language on the visitor's machine. A line kept as a number with its words in a table is the easiest to translate.
The same holds for any overlay built from acts and values rather than from pixels: the description reaches the visitor's machine, so it can be adapted there.
Pointer routing
What it is
A visitor's picture is a composition of layers: the browser's controls, the world's overlays, and under them the camera's picture of the world, in which surfaces may show further images. A press of the pointer lands on one pixel of the finished picture. Routing it means working back through the composition to the element that drew that pixel: overlays, not only the 3-D world, receive the pointer, and so do surfaces in the world that show layouts.
What it means
Working back follows from the composition itself: the pointer is offered to the layers in the reverse of the order they were laid down, the top layer first.
- The browser's controls, composited last, are asked first.
- The world's overlays, from the top layer down. Within a layout, the box under the pointer is found the same way: the topmost layer first, the last child (drawn on top) first, and, while a modal box is set, nowhere but in it. Boxes that only arrange others take no presses, so a press on the empty middle of a HUD passes through. A press no box takes goes on: the world gets the press.
- The 3-D picture. The press becomes a ray from the camera's eye through the pixel, and the first thing it meets is found: each shown part's bounding box, then its triangles, nearest first. That part receives
clicked(part)(Events). - Through a surface. If the ray meets a surface showing a layout, the point it meets becomes a pixel of the layout's image and the press goes to the element there: a ray in, a pixel out, so hover, press and drag work as on the screen. If the surface shows a camera's picture (a mirror, a portal), the ray goes on from that camera into the world it shows.
For this to work every compositing step must be invertible (Images), and so it is designed in from the start. A press, once found, is a visitor's act: a control performs its action, a part receives its event; the world never sees the device, only actions and events (Visitors).
Not yet supported: a press through a mirror's picture. Overlays get the pointer while a modal panel is up; keys go to the box with the focus, and a text field with the focus turns on the system's text input. The ground is not pickable.
How it connects
- Images: compositing, channels, the browser's controls last.
- Layouts: boxes, text, controls, styles, placing over a 3-D anchor.
- Visitors and their actions: what controls perform; keys as suggestions; speech as an act.
- Actions:
say,play,show. - Events:
spoke,clicked, the engine's messages. - Distribution: only acts cross the network; each machine manifests them.
- Sounds: voices and other speech heard rather than seen.
- Cameras and views: the camera drawing the picture.
Open questions
- Per camera or per visitor. A speech plate is in each visitor's picture; a mirror's picture is made per camera. The language needs a way to say which (as for rain round each camera).
- Who hears. Measured from the eye, and through walls. "Not through a wall" needs the same blocking question as falling weather.
- Who draws an offered choice. When a guard offers the action
ask, one reading has the browser show the choices its own way (a panel, buttons, a spoken menu, the generic action menu); the other has the world draw its own dialogue layout. Whether both are allowed, and which wins, is not settled. - Where overlays sit in the pointer's order. The browser has its own picking against the 3-D world; overlays should see pointer events first and say whether they took them. Where that sits in the event order is open.
- Accessibility. Whether overlays and parts carry a name and role for screen readers from the start, or later.