Audio and MIDI #

Budo includes audio and MIDI support with synthesis, decoded audio files, MIDI I/O, RTP-MIDI network sessions. It shares the same design as the rest of the runtime: small handles, project-relative assets, explicit capabilities, and platform-aware fallbacks.

Audio context #

The audio context starts stopped. You can call sys.audio.start() explicitly, or let playback helpers start it when needed.

main.js

// Start the audio context explicitly.
sys.audio.start();
// Set a comfortable master volume.
sys.audio.setMasterGain(0.5);

The master gain is clamped to 0.0 through 1.0.

Oscillators #

Oscillators are represented by integer IDs.

main.js

// Create an oscillator and keep its runtime handle.
const osc = sys.audio.createOscillator();
// Choose a simple sine wave.
sys.audio.setOscillatorType(osc, 'sine');
// Tune the oscillator to A3.
sys.audio.setOscillatorFrequency(osc, 220);
// Set the oscillator gain before playback begins.
sys.audio.setOscillatorGain(osc, 0.25);
// Start continuous playback for this oscillator.
sys.audio.startOscillator(osc);

Wave types may be strings or constants: SINE, SQUARE, SAWTOOTH, TRIANGLE, and NOISE.

ADSR envelopes let the runtime shape notes for you.

main.js

// Configure a short attack and release envelope.
sys.audio.setOscillatorEnvelope(osc, 0.01, 0.12, 0.7, 0.25);
// Trigger the note on the configured oscillator.
sys.audio.noteOn(osc);
// Release the note after 300 milliseconds.
sys.timer.once(300, () => sys.audio.noteOff(osc));

sys.audio.midiToFreq(69) returns 440, which is handy when MIDI, sequencers, and synthesis meet.

Buffers #

Audio buffers store sample data in runtime-managed slots.

main.js

// Choose the sample rate for the generated buffer.
const sampleRate = 44100;
// Allocate one second of mono samples.
const samples = new Array(sampleRate);

// Fill the buffer with a quiet 110 Hz sine wave.
for (let i = 0; i < samples.length; i++) {
  // Convert the sample index into a phase angle.
  samples[i] = Math.sin((i / sampleRate) * Math.PI * 2 * 110) * 0.25;
}

// Create a runtime audio buffer for the samples.
const buffer = sys.audio.createBuffer(sampleRate, 1, samples.length);
// Upload the generated samples starting at offset 0.
sys.audio.setBufferData(buffer, samples, 0);
// Play the buffer once at full gain.
const playback = sys.audio.playBuffer(buffer, false, 1.0);

Stop playback handles with stopBuffer and destroy buffers you no longer need.

Audio files #

For sound effects, music cues, voice clips, and other prepared media, load an audio asset into the same buffer system with sys.audio.loadBuffer(path) or sys.audio.loadBufferFromBuffer(buffer). Budo decodes WAV, MP3, Ogg Vorbis, and FLAC data to floating-point PCM, then returns a buffer ID that can be played, looped, stopped, and destroyed just like a buffer you created manually.

main.js
// Decode a project asset into a runtime audio buffer.
const hit = sys.audio.loadBuffer('sounds/hit.ogg');
const hitFromNetwork = sys.audio.loadBufferFromBuffer(await (await fetch(url)).arrayBuffer());

// Check for a load failure before using the returned handle.
if (hit < 0) {
  // Print the most recent audio error to help diagnose missing or invalid assets.
  console.log(sys.audio.getError());
}

// Play the decoded sound once at a comfortable level.
const hitPlayback = sys.audio.playBuffer(hit, false, 0.8);

Loaded audio paths are project-relative. Use paths such as sounds/hit.ogg or music/theme.flac; absolute paths and . or .. path segments are rejected. The FromBuffer form accepts bytes from local file reads, network responses, or generated data.

Looping works through playBuffer, so a longer ambient file can stay simple too.

main.js
// Load a background loop from the bundled project assets.
const ambience = sys.audio.loadBuffer('audio/ambience.mp3');

// Start the buffer in looping mode at a low gain.
const ambiencePlayback = sys.audio.playBuffer(ambience, true, 0.35);

// Stop the looping playback later when the scene changes.
sys.audio.stopBuffer(ambiencePlayback);

loadBuffer is synchronous and best used while entering a scene, opening a tool, or preparing a small sound set. For large libraries, keep the returned buffer IDs in your own state and release buffers with destroyBuffer when they are no longer needed.

MIDI devices #

sys.midi exposes local MIDI input and output where the platform backend is available. Use onDevicesChanged to keep long-running tools synchronized with device connectivity.

main.js
// Perform the initial scan before showing a picker.
sys.midi.refreshDevices();

// Iterate through the available MIDI inputs.
for (const device of sys.midi.getInputDevices()) {
  // Log the stable device id and human-readable name.
  console.log(device.id + ': ' + device.name);
}

Subscribe once to receive later topology changes. Registration establishes a baseline and does not invoke the callback immediately. A new registration replaces the old listener; pass null to unsubscribe.

main.js
sys.midi.onDevicesChanged((event) => {
  console.log('MIDI topology generation ' + event.generation);
  console.log(event.inputs.length + ' inputs, ' + event.outputs.length + ' outputs');
  // Reconcile selected devices and reopen handles here when needed.
});

Callbacks are delivered on the runtime frame thread, never directly from an OS MIDI thread. The event contains fresh device snapshots and detects replacement or reordering even when the number of devices is unchanged.

Lua exposes the same contract as sys.midi.onDevicesChanged(function(event) ... end); pass nil to unsubscribe.

Open an input with a callback.

main.js
// Open the first MIDI input and receive messages through a callback.
const input = sys.midi.openInput(0, (message) => {
  // Treat NOTE_ON with velocity greater than zero as a played note.
  if (message.type === sys.midi.NOTE_ON && message.data2 > 0) {
    // Log note number and velocity.
    console.log('Note ' + message.data1 + ' velocity ' + message.data2);
  }
});

Open an output and send notes or raw bytes.

main.js
// Open the first MIDI output.
const output = sys.midi.openOutput(0);
// Send middle C on channel 0.
sys.midi.noteOn(output, 0, 60, 100);
// Release middle C on channel 0.
sys.midi.noteOff(output, 0, 60, 0);
// Send channel pressure aftertouch on channel 0.
sys.midi.channelPressure(output, 0, 72);
// Send polyphonic pressure for note 60 on channel 0.
sys.midi.polyPressure(output, 0, 60, 64);
// Send a raw MIDI System Exclusive message.
sys.midi.sendRaw(output, [0xf0, 0x7e, 0x7f, 0x06, 0x01, 0xf7]);

Close input and output handles when done.

RTP-MIDI #

RTP-MIDI sessions let apps exchange MIDI messages over the network using the MIDI namespace.

main.js
// Create an RTP-MIDI session on an automatically selected port.
const session = sys.midi.createSession('Jam', 0);
// Receive MIDI messages from the network session.
sys.midi.onSessionMessage(session, (message) => {
  // Log the raw status byte for quick diagnostics.
  console.log('Network MIDI: ' + message.status);
});

// Connect to another RTP-MIDI peer.
sys.midi.connectSession(session, '192.168.1.20', 5004);
// Send a note through the session.
sys.midi.sessionNoteOn(session, 0, 64, 100);

RTP-MIDI depends on UDP support. It is available on desktop and Android builds that compile the transport, and unavailable on web because browsers do not expose raw UDP sockets.

Platform notes #

Audio, and MIDI depend on platform permissions or linked backends. Use capability checks and clear UI states.