author: zep // picotron.net
updated: 2023-10-25 (WIP!)
This is a working technical document for Picotron's audio internals.
2023-10-25: added track Index, Track_Content, Common_Header for envelopes, Special_Parameter_Control, and additional Future_Features
All of Picotron's audio is generated by a built-in synthesizer that takes instrument definitions and pattern data as input. All of the information the synthesizer depends on is placed in regular RAM and can be manipulated (using poke / memcpy etc) during live playback.
Instruments in Picotron are based on nodes: one for generating a raw signal from wavetables (OSC) and additional nodes for modifying the signal (filters, modulating oscillators). The bundled Picotron Tracker can be used both to design instruments and to enter pattern data for creating sound effects and music.
16 channels
128 instrument definitions
44100MHz stereo output
256-entry wavetables
8-channel tracker format
64 active nodes
Each channel can have a single instrument playing on it: either one track (a sequence of note instructions), or a single instrument directly played on that channel. Each instrument can have up to 8 nodes contributing to its output, but there can only be a total of 64 nodes active at one time globally, and only 16 of them can be fx:echo.
Like the graphics pipeline, Picotron's audio system aims to provide a set of primitives that are reasonably flexible while still having some kind of aesthetic center, and low-cost enough to run consistently across target platforms.
By focusing on dynamically generated audio (as opposed to pre-rendered or recorded samples), instrument definitions can also be relatively tiny and suitable for Picotron and Voxatron's 256k cartridge formats.
An instrument is a collection of nodes that are activated each time a note is played using that instrument. Each node either generates, modifies, or mixes audio data.
For example, a bass pluck instrument might have a white noise node that fades out rapidly at the start, and a saw wave that fades out more slowly.
There is only one type of oscillator (OSC), and it is not really an oscillator. It reads data from a table of waveforms (a "wavetable"), each entry in the table representing a short looping waveform. Common waveforms such as sine wave and triangle are all implemented in this way rather than having special dedicated oscillator types.
There are 128 global instruments defined at 0x40000 by default, each 512 bytes.
Nodes each have a parent index, giving them a tree structure that is used to determine the order in which nodes are evaluated, and their effect on each other.
For example, a low pass filter could be attached to the root node (applied to the entire instrument), a particular oscillator, or the modulator of an oscillator.
The root node of an instrument is always of type MIXER and does not produce a signal itself. So an instrument always needs to use at least 2 nodes.
Here is an instrument that has two oscillators -- the first with a filter on it, and the second one modulated by another OSC:
MIXER
OSC
FX (FILTER)
OSC
OSC (FM MOD)
If the low-pass filter were to be applied to the whole instrument, it would look like this:
MIXER
OSC
OSC
OSC (FM MOD)
FX (FILTER)
To generate the output signal for a given instrument:
1. FM modulating oscillators are evaluated first from leaves to root.
2. Carrier oscillators are then evaluated from root to leaves.
To evaluate an oscillator, first the raw signal is generated, and then children of that node are applied to the resulting signal in order:
- OSC nodes are added to the signal
- OSC:RING nodes are multipled with the signal
- FX nodes (filter, echo, gain) are applied to the signal
In all cases a simple limiting function is applied to soften clipping, with the exception of the gain filters that have explicit control over clipping level and hardness.
Nodes are all evaluated as mono signals, except for children of the MIXER node which have their output split into stereo channels based on their panning value.
Each node consists of a collection of parameters, each one controllable with a knob in the tracker.
For example, a filter FX node has knobs for 3 parameters: low pass, high pass, and resonance.
Each parameter can be either a single value, a range of values, and/or a range of values controlled by an envelope. When an envelope is not assigned, the parameter can also operate in random-selection mode, where a random value within the specified range is used each time the instrument is triggered.
Some parameters can be multiplied or added to their parent values. For example, the pitch of an instrument can be adjusted 7 semitones up with +7, and then each child node can further adjust their pitch relative to the parent's evaluated pitch. TUNE also has a special integer ratio mode that can express frequency ratios [1..16]/[1..16] (useful for getting pure harmonics).
The scale of each parameter can be adjusted by *4, /4, *16, /16, *64, /64. Sometimes this produces unpredictable results depending on the parameter type.
In the tracker, parameter values can be controlled with knobs:
- click and drag a knob to turn it (left/down - right/up +)
- mb2 click and drag to alter the second value (to define a range)
- click to the right of the numeric display to toggle an envelope
- drag and drop the envelope on a parameter to assign it
- right click to the right of the numeric display to toggle random selection (when env is off)
- click on the left of the numeric display to cycle through available parent relationships (*, +)
- click below the numberic display to cycle through scaling factors:
*4, /4 (+ctrl for *16, /16, *64, /64)
This is the only node type that generates an audible signal. The wavetable (0..3) can be selected in tracker by clicking on "WT-0" in the waveform scope.
VOL: 64 means full volume
PAN: -128 (left) .. 127 (right)
TUNE: Semitones above C0 (48 is middle C)
BEND: -1..+1 semitone
WAVE: Waveform selection
PHASE: Waveform phase shift (-0.5..0.5)
FM: The output is applied to its parent as waveform position deltas.
RING: Each sample is multiplied by its parent's corresponding value.
LOW: Low pass filter
HIGH: High pass filter
RES: Resonant filter
The resonant filter can be unpredictable especially at high values; when experimenting with changes, consider using speakers instead of headphones and/or turning the volume down to prevent damage to your eardrums.
Add an echo / low-quality reverb to the parent.
DELAY: 0..1 second offset into signal history
VOL: percentage of original signal to add back to itself
Using an envelope on DELAY can cause the sample offset to move wildly, sometimes in retrograde, producing fascinating but often extremely noisey results. Also, scaling x4 can cause the read position to wrap around within one tick, and low delay values can severely alter and enrich the tone of the waveform. Again: distance your headphones from your eardrums and check the signal scope with your eyeballs first!
GAIN: sample multiplier 1..8
ELBOW: 0: no limiting .. 128: limit to max amplitude .. 255: limit to CUT
CUT: amplitudes above CUT (0..1) are limited
MIX: output volume to go back into the mix (64 == x1.0)
Defaults to CUT 64 and ELBOW 128 to implement a simple limiting function.
For distortion, try high gain and CUT at around 0.5 with a high elbow value. (see the scope output in the tracker to visualise how this shapes amplitudes)
Envelopes can be attached to any node parameter (drag and drop the evelope onto a node knob in the tracker to assign it). Each envelope evaluates to a value 0..1 that maps to the parent parameter's range. To adjust the range in the tracker, click and drag for val1 (shown with a white notch), and right-click and drag to change val0 (currently hard to see ~ use the force!)
Some durations used in envelopes are non-linear:
00..15 -> 0..15 ticks
16..32 -> 16..46 ticks
32..63 -> 48..172 ticks
64..127 -> 176..680 ticks
128..255 -> 688..2720 ticks (~22.6 seconds)
The first 8 bytes of envelope are the same for each type:
kind 0 ADSR 1 LFO 2 DATA
flags 0x1 lerp 0x8 rnd_start [0x10 daisy]
spd ticks per time unit T // 0 == track spd
loop0 loop back point T (when loop0 < loop1)
loop1 loop back to loop0 at T until released
start start evaluating from T
unused(2) reserved for future use. should be zero
flag 0x8 means choose a starting time from 0..start inclusive.
The last 16 bytes depend on the envelope type:
Starting from byte 8:
attack: how long to reach 1.0
decay: how long to fall back down to sustain
sustain: sustain level until release (0..1)
release: how long to fall down to 0 from current value
Starting from byte 12:
freq: duration to repeat
phase: phase offset
function: 0 sine, 1 tri, 2 saw, 3 reverse_saw 4 square, 5 pulse
Bytes 8..23 of a data envelope are explicit U8s; one per time unit.
They are useful for creating custom envelope shapes and for sequencing pitch changes.
When lerp flag is set (dat[1] & 0x1) the target value is returned on the first tick of that entry, and subsequent ticks are linearly interpolated to the next value (observing loop points).
A track is a list of note instructions with some metadata about how to play it. Each note instruction can optionally include a pitch, instrument, volume and special effect. In each case except for the effect parameter, 0xff means "unspecified".
len (I16) max 32767 rows (0 == default)
spd (U8) ticks per row (0 == default)
loop0 (U8) loop back to this point
loop1 (U8) loop back to this position
delay (I8) start playing after n rows. use negative values to process -n commands.
flags (U8) 0x1 mute
Track data is arranged in memory column by column (e.g. 64 pitches values, followed by 64 inst values..). This allows track data to compress well as-is stored in pod format.
pitch 0 means c-0, 48 means middle c (c-4)
inst instrument index (0x40000 + n * 0x200 in memory)
vol 64 means 1.0 but can over-amplify with larger values
effect an ascii character indicating effect
effect_p parameters for the effect (often treated as 2 nibbles)
Default track size is 5 * 64 rows + 8 = 328 bytes
A pattern definition is 20 bytes, and points to up to 8 tracks:
pat_len (I16) how many rows to play this pattern for at the default speed [0x8000: in ticks]
flow (U8) 0x1 stop playback 0x2 loop start 0x4 loop end
track_mask (U8) each bit is set when there is a track assigned to that channel
track (I16 * 8) track index
When pat_len is 0, the length of the left-most non-looping track is used.
The following 7 effects map roughly to PICO-8 when used with default parameter 00. (The first 5 can alternatively be expressed with '1'..'5')
g glide to note (spd)
v vibrato (spd, depth)
* drop (spd)
< fade in (spd)
> fade out (spd)
6 fast trill (grouping)
7 slow trill (grouping)
s slide from node (signed spd; 0x80 == 0)
f fade to volume
a arpeggio (rel_pitch1, rel_pitch2)
t tremelo (spd, depth)
p set channel panning offset
r retrigger (every n ticks -- can carry over)
d delayed trigger
c cut after n ticks
[in consideration for future]
x and y are general purpose parameters that can be fed into instrument envelopes to do things like controlling the pitch of individual nodes, or to control a low pass filter directly from track data. They would work with a special envelope type (MAP) that can also respond to other channel state such as volume and pitch (e.g. to implement an extra bass oscillator kicks in at low pitches). Conceptually they can also be though of a way to reduce an otherwise complex instrument down to two salient knobs.
Controlling x and y from track data:
x set x
y set y
q set x,y (one nibble each)
X,Y,Q: same, but for all channels
Default top level layout:
0x030000 index + pattern data // 20 * num_patterns
0x040000 instrument data // 512 * num_instruments
0x050000 track data // 328 * num_tracks
0xf00000 wavetables
An index of resources used by the synthesizer is expected at 0x30000, and normally represents the audio data for a single tracker file. The memory addresses below are always relative to that base address. This will make it easier in future to load in and use multiple tracker files at different locations.
Index (32 bytes)
num_instruments (I16)
num_tracks (I16)
num_patterns (I16)
flags (I16) 0x1 use default track indexing (base+0x20000, increments of 328 bytes)
insts_addr (I32) relative address of instruments
tracks_addr (I32) relative address of track index
patterns_addr (I32) relative address of pattern data
unused (I32) should be 0
tick_len (I16) in 1/16ths of a sample at 44100Hz [0 means 5880 -- 120 ticks / second]
def_len (I16) used by patterns that do not have a default length specified
def_spd (U8) used by patterns that do not have a default speed specified
unused (U8 * 3)
Song title and author name etc can be set in the .pod metadata.
Each instrument is 512 bytes:
NODE (256 bytes) // 8 nodes
ENV (192 bytes) // 8 envelopes
UNUSED (32 bytes) // should be zeroed
WT (16 bytes) // 4 wavetable definitions
NAME (16 bytes) // ascii name
parent (4bits)
operator (4bits) 0:add 1:fm 2:ring [3:xor 4:or]
kind (4bits) 1:mixer 2:osc 3:copy 8:filter 9:echo 10:gain
kind_p (4bits) for OSC nodes, means wavetable index 0..3
flags (U8) 0x2:muted 0x8:continue_waveform_position
unused (U8)
[PVAL*7] (28 bytes)
Parameter value -- one for each of volume, pan, tune etc.
flags (U8)
val0 (U8) signed depending on parameter (e.g. BEND is treated as I8)
val1 (U8) second value to define a range
env (4bits) envelope index
scale (4bits) 0x20: /,* 0x40: x4 0x80: x16
flags 0x1 add to parent 0x2 multiply by parent 0x4 has envelope 0x8 continue envelope tick counter state when note is triggered 0x10 select a random value between val0..val1 inclusive 0x20 quantize to integers (e.g. quantize tune slide to semitones)
See Envelopes section
addr (U16) address of wavetable data in 256-byte increments
w_bits (U8) width of each wavetable entry is 1 << w_bits
height (U8) number of waves in wavetable. 0 == 256.
Each individual track can be placed anywhere in memory, but for the common case that tracks are all <= 64 rows long, a default indexing scheme can be used: track data starts at 0x50000 (+0x20000) and each track uses 328 bytes.
When a process is created, a set of default wavetables is provided at 0xf00000, and the tracker uses these in the default instrument definitions. The memory at 0xf00000 is otherwise not special and can be used for something if the wavetable data is not needed.
0xf00000 wt-0 256 1k entries: sine -> tri -> saw -> square -> pulse)
0xf80000 wt-1 noise: white, brown, pink, low-fi (pitched)
0xfc0000 wt-2 (future) waveform zoo: various waveforms, some with transitions
The following may or may not happen in the future.
- Track instruments (similar to PICO-8 sfx instruments) -- bit-7 could be used for this.
- Sample instruments (similar to traditional MOD instruments). Can already use wide wavetables though.
- Custom instrument waveforms: already supported, but not exposed in editor. I want to doodle waveforms.
- FX nodes that can be attached to entire pattern output and controlled in the effects column.
- MAP envelope type: map channel state (volume, pitch, x, y), to one of 16 [optionally lerped] output values. Allows things like resonance that kicks in only at high volumes, or to play chords on a single channel by setting up 2 extra nodes tuned to +x, +y (ref: comeback tracker). See: Special_Parameter_Control
- General purpose parameters x, y can be set using channel commands and digested by MAP envelopes
- Daisy-chain envelopes (multiply by another envelope, or allow some env knobs to also be enveloped) Not sure how this should work, but seems needed to make MAP envelope types more useful.
- Glide on individual node parameters. e.g. the relative pitch of an FM osc takes a second to catch up to its parent.
