Asynchronous Images #

Native JavaScript/TypeScript and Lua can read, probe, decode, and resize images on a bounded worker pool, then upload pixels on the graphics thread. Rendering never waits for decoding: draw a placeholder until completion. The existing synchronous loadTexture2D and loadTexture2DFromBuffer APIs remain unchanged. Desktop Wasmtime exposes the same worker service through the polling ABI below. Web JavaScript and Lua use the same request/upload scheduler with a separate Worker-backed WebAssembly decoder, described under Browser Backend.

API #

sys.gl.isAsyncTextureLoadingAvailable(): boolean;
sys.gl.loadTexture2DAsync(path: string, options: TextureLoadOptions, callbacks: TextureLoadCallbacks): number;
sys.gl.loadTexture2DFromBufferAsync(bytes: ArrayBuffer | ArrayBufferView, options: TextureLoadOptions, callbacks: TextureLoadCallbacks): number;
sys.gl.cancelTextureLoad(requestId: number): boolean;

Both load functions return a request ID, not a texture, promise, or lazy pixel object. Supply an options object ({} for defaults) and callbacks:

interface TextureLoadOptions {
    mount?: 'assets' | 'files';
    maxWidth?: number;
    maxHeight?: number;
    maxBytes?: number;
    maxPixels?: number;
}
interface TextureLoadInfo {
    readonly width: number;
    readonly height: number;
    readonly sourceWidth: number;
    readonly sourceHeight: number;
}
interface TextureLoadError { readonly code: string; readonly message: string; }
interface TextureLoadCallbacks {
    onMetadata?(info: TextureLoadInfo): void;
    onComplete(textureId: number, info: TextureLoadInfo): void;
    onError(error: TextureLoadError): void;
}

onComplete and onError are required. Callbacks run during polling on the owning runtime thread, never on a worker and never before the load call returns. Keep the callbacks object unchanged until completion or cancellation. Lua uses the same names and fields, with tables for options and callbacks and a binary string for encoded bytes:

sys.gl.loadTexture2DAsync('photos/coast.jpg', {maxWidth = 256}, {
    onMetadata = function(info) print(info.sourceWidth, info.sourceHeight) end,
    onComplete = function(texture, info) sys.gl.destroyTexture(texture) end,
    onError = function(err) print(err.code, err.message) end
})

Loading A Thumbnail #

let texture = 0;
let request = 0;
let dimensions = null;
let errorMessage = '';

if (sys.gl.isAsyncTextureLoadingAvailable()) {
    request = sys.gl.loadTexture2DAsync('photos/coast.jpg', {
        maxWidth: 256,
        maxHeight: 256
    }, {
        onMetadata(info) { dimensions = info; },
        onComplete(id, info) {
            request = 0;
            texture = id;
            dimensions = info;
        },
        onError(error) {
            request = 0;
            errorMessage = error.message;
        }
    });
}

Default paths are project-relative, just like synchronous loadTexture2D: photos/coast.jpg refers to that physical project file, and assets/coast.jpg refers to a real project subdirectory named assets. Absolute paths, traversal, and symlink escapes are rejected.

For host files mounted with --file-root, use loadTexture2DAsync('album/coast.jpg', { mount: 'files' }, callbacks). Omit the files/ prefix from the path; mount: 'assets' is the default project mount. Each request snapshots the sandbox's opened roots before worker I/O. No filesystem or network metadata permission is needed for ordinary mounts.

The buffer variant copies the provided bytes before returning, including only a typed array's selected range. The caller may immediately modify or release its buffer. That initial copy runs on the calling thread; file loading avoids it. mount is ignored for buffer sources. No image bytes or pixels cross into JS unless the application explicitly reads the source itself.

Metadata, Completion, And Ownership #

Workers probe file chunks until dimensions are known, or until the file has been read. Valid metadata is delivered before completion for the same request; an image whose header is valid may still fail during subsequent reading/decoding. There is no ordering guarantee between different requests.

sourceWidth/sourceHeight describe stored pixels. width/height describe the output texture. Output bounds preserve aspect ratio, round down to positive dimensions, and never enlarge the source. Omitting both bounds preserves source dimensions. EXIF orientation and ICC color transforms are not applied.

Completion transfers a normal texture handle to the app. Use existing texture binding/drawing calls and release it with sys.gl.destroyTexture(texture). Pending requests own their buffers and partially uploaded textures; cancellation or runtime teardown releases these. Runtime teardown releases remaining GL resources as usual. There is no implicit join when drawing an image.

Accepted requests receive exactly one terminal callback, unless cancelled or the owning runtime shuts down. Exceptions thrown by callbacks are logged using normal runtime callback error reporting; they do not stop other requests. The app still owns a completed texture if its onComplete throws.

Cancellation And Scheduling #

if (request) {
    sys.gl.cancelTextureLoad(request);
    request = 0;
}
if (texture) {
    sys.gl.destroyTexture(texture);
    texture = 0;
}

Cancellation returns immediately and suppresses subsequent callbacks, including completion after metadata. It returns false for an already cancelled, completed, or unknown ID. IDs belong to one runtime and are not texture IDs. A callback already executing cannot be undone. Cancel obsolete requests when navigating; bound any application-side queue as well.

Native reduced-resolution JPEG decoding checks cancellation between batches of 16 decoded rows, allowing obsolete thumbnail jobs to release workers before the whole image is decoded. Codec initialization (including progressive JPEG input processing), other decode paths, and an in-progress OS read cannot be preempted safely. Their results are discarded at the next cancellation check; request capacity and memory may remain occupied until then. Runtime destruction joins owned workers, so shutdown can wait for those operations. Cancellation during upload destroys the partial texture. No detached worker retains a VM, Window, or JS callback.

Pending requests participate in managed runtime polling and keep window-less work alive. A GL call activates graphics under the existing lazy-window policy. While image work is pending, native input-wait sleeps are capped at 16 ms so metadata, uploads, and terminal callbacks progress without an active animation. Completion callbacks should request a redraw when needed. This API does not change the existing mixed canvas/GL idle-presentation contract.

Limits And Errors #

Each runtime has two workers, started on first use, and at most eight active requests, including uploads. Encoded storage is capped at 64 MiB total; pixel reservations are capped at 2048 MiB, accounting for the source and resized output while both exist, including completed pixels waiting for upload. Decoder-internal scratch, allocator overhead, caller-owned buffers, and GPU resources are additional.

maxBytes defaults to 64 MiB and cannot exceed 64 MiB. maxPixels defaults to 256,000,000 and may be lowered but not raised. Each source axis is limited to 32768 pixels; explicit maxWidth and maxHeight output bounds remain limited to 16384. Request smaller output for large sources to fit device texture and memory limits. The reservation ceiling is not preallocated and does not guarantee that the device has enough memory. Specified options must be positive integers. Queue/buffer capacity, invalid arguments/paths, and worker creation errors throw at submission, before a request is accepted. File errors, pixel limits, decode failures, and memory pressure discovered after acceptance use onError.

Error codes are image.read_failed, image.byte_limit, image.pixel_limit, image.invalid_data, image.decode_failed, image.capacity, image.out_of_memory, and image.upload_failed. Capacity errors are not an invitation to retry every frame; reduce outstanding work or retry deliberately.

Uploads advance by at most 4 MiB of rows per runtime poll, one image at a time. This is a byte budget, not a hard frame-time guarantee: texture allocation and driver calls may stall. Bounded JPEG requests use Skia's existing JPEG codec to decode at a supported half, quarter, or eighth resolution when that still covers the requested output. A final worker-side bilinear resize produces the exact output dimensions. Full-resolution requests and other formats retain the STB path; JPEG codec failures also fall back to it. Scaled JPEG pixels need not be identical to a full decode followed by resizing.

Source pixel limits are checked before either decoder runs. Native admission reserves the full source plus output conservatively, then releases the excess after a successful scaled decode. Smaller RGBA buffers reduce working memory, but decoder scratch (including progressive JPEG coefficients) can still depend on the full source dimensions. Higher-quality minification and EXIF/ICC handling remain follow-ups. The image_decode_benchmark target compares full and scaled decoding using identical encoded bytes and reports decoded RGBA buffer sizes. Use image_decode_benchmark photo.jpg 256 cancellable to measure the native worker's scanline path, or image_decode_benchmark photo.jpg 256 cancel to measure cancellation after its first row batch. The latter requires a JPEG that can be reduced and produces more than 16 decoded rows. Compare scaled and cancellable checksums to check output parity; cancellation timings are not end-to-end gallery latency measurements.

Desktop Wasmtime ABI #

These env imports use i32 arguments and results. Request IDs are opaque 32-bit values; zero indicates submission failure. The guest must export memory.

gl_image_load_async(ptr, len, mode, maxWidth, maxHeight, maxBytes, maxPixels) -> request
gl_image_status(request, resultPtr) -> 0 or -1
gl_image_take_texture(request) -> texture or 0
gl_image_release(request) -> 1 or 0

Mode 0 reads an assets/project-relative path, mode 1 reads a files-mount path, and mode 2 copies len encoded bytes from guest memory before returning. Paths omit the mount prefix. Zero options select defaults; other limits match the native APIs. Invalid inputs, unavailable capacity, and startup failures return zero before acceptance. No guest-memory pointer is retained.

gl_image_status writes a 312-byte snapshot to a range checked on every call, returning -1 for an invalid request or range without consuming the record. The record is little-endian: six 32-bit words at offsets 0, 4, 8, 12, 16, and 20 hold status (0 pending, 1 complete, 2 error), metadata-available (0 or 1), width, height, source width, and source height. At offset 24 is a 32-byte NUL-terminated error code; at offset 56 is a 256-byte NUL-terminated message. Metadata may be observed while pending, but a guest polling infrequently may first observe it together with a terminal status.

For a completed request, gl_image_take_texture transfers its ordinary texture exactly once; destroy it with gl_destroy_texture. Release every request using gl_image_release, including errors and results whose texture was taken. It returns 0 for an invalid or released ID. Releasing pending work cancels it; releasing an unclaimed completion destroys its texture. A completed record occupies one of eight slots until released, even after its texture is taken. Runtime teardown cancels and joins workers and releases unclaimed textures. The host advances uploads while the app runs; the guest must poll status in its own frame logic to observe results. Status never blocks or invokes guest code.

Browser Backend #

Web JS/TS and Lua use two classic Blob Workers, each running a standalone STB/Skia JPEG WebAssembly decoder embedded in the main runtime. No additional worker URL, SharedArrayBuffer, COOP/COEP headers, or cross-origin isolation is needed. The host needs Worker, WebAssembly, Blob, and Blob URL APIs. CSP must allow worker-src blob: and WebAssembly compilation (for example wasm-unsafe-eval in script-src, subject to browser policy). The availability probe checks APIs; worker construction or startup can still fail. Construction failures throw before acceptance; later worker/startup/message failures use onError. There is no main-thread decoding fallback.

Workers cannot access mounted files. The host snapshots the authorized sandbox roots and reads virtual files in blocks of at most 64 KiB per service poll before transferring encoded bytes. These in-memory VFS reads and initial buffer copies are synchronous; probing, JPEG scaled decoding, STB fallback, and bilinear resizing run only in Workers. Browser file metadata becomes available after the file is read, but before decoding. A metadata callback can cancel before decoding is granted. Formats, stored-pixel orientation, and lack of EXIF/ICC transforms match the native backend rather than browser image-decoder policies.

The same eight-request, 64 MiB encoded, 2048 MiB pixel, and 4 MiB upload budgets apply. Browser encoded reservations include two copies of each source, so a single source has an effective 32 MiB ceiling even if maxBytes is larger. Pixel reservations cover source plus output, including transfer/copy overlap; source pixels are freed before copying a resized result. Decoder scratch, retained Worker WebAssembly heap capacity, mounted file storage, browser/allocator overhead, and GPU memory are outside these budgets. Worker heaps grow up to 512 MiB each and are released when the Worker is terminated.

Cancellation terminates an assigned Worker without waiting; its replacement is created only when needed. Generation and Worker identity checks discard late messages after cancellation or teardown. Page teardown terminates all owned Workers and revokes their Blob URL. Managed WebGL context loss stops the app with status 1 and releases pending work; reload is required, not transparent texture restoration. Hidden tabs may throttle/pause frame polling: results remain bounded until polling resumes. There are no new permission or network grants.

Platforms And Tests #

Desktop JS/TS, Lua, and Wasmtime are implemented. Android JS/TS and Lua use the same native service, but require device validation. Web JS/TS and Lua have been functionally checked in Chromium using the Worker backend; other browsers and mobile devices still require validation. There is no blocking web fallback. Browser-native WebAssembly and the public native SDK do not expose these async functions. Their synchronous image loading remains unchanged.

The image_async CTest covers probing, resize, scaled JPEG dimensions and colors, full-size compatibility, source ownership, failure limits, queue saturation, cancellation, sandbox paths, and teardown. The gallery's opt-in native test also checks public callbacks, ordering, cancellation, errors, and synchronous compatibility:

BUDO_GALLERY_NATIVE=1 node --test examples-private/gallery/gallery.test.mjs

web_image_bridge runs bounded-worker, byte-range, startup/error, cancellation, peer teardown, and late-message regressions under Node. After building the web runtime, serve the repository and open tests/web_image_test.html for the real JS fixture, ?runtime=lua for Lua, and ?unavailable=1 for missing capabilities. ?pending=1&contextloss=1 and ?pending=1&pagehide=1 exercise teardown during pending work. The fixture records actual GPU texture readbacks in imageTestUploads and reports completion through the budoexit event. Hidden-tab automation must keep animation-frame polling active to reach the teardown trigger.