Tech Stack
Overview
This page describes the underlying technology for the curious. You do not need to know any of this to use Fantasynth.
Build
The instrument is a single-page web application built with Vite. Source files are written as ES modules and bundled at build time. The output is a static site that can be deployed to any host that serves files.
Rendering
- WebGL 2.0 for all GPU rendering
- Three.js as the scene graph and rendering abstraction
- postprocessing (pmndrs) for the effect chain: bloom, SSAO, SSR, tone mapping, color grading, and more
- A custom MRT pass shared across post-processing effects to render normals, roughness, and velocity in a single pass
- troika-three-text for SDF-based 3D text rendering inside actors
Shaders
Fantasynth uses a template-based shader system. Visual shaders are written in GLSL with a small JSON frontmatter that declares parameters and metadata. The template handles all the boilerplate: world-space mapping, palette management, fog, instancing, beat-reactive uniforms, lifecycles, and more.
Shaders are processed at load time. A shader processor handles #include directives, feature injection, and template assembly. The result is a single fragment and vertex shader pair that runs on the GPU.
Actors
Actors are written in JavaScript and exported as ES modules. Each actor is a factory function that receives engine state and returns a set of Three.js objects plus an update callback. Actors can declare parameters that the user can tune live.
Compositions
Compositions, content definitions, music tracks, and scenes are stored as plain JSON. Loading a composition triggers a content preloader that fetches and parses every shader, actor, and texture referenced.
Audio
Music playback uses the standard HTML5 audio element. Beat sync is driven by a clock set manually to the current BPM. The clock advances each frame and exposes uniforms like uTime, uBeat, and uPhase to the shader system. Beat-reactive features in the visual template lock to the clock, which the performer aligns to the music either by syncing to an external source (DJ-style, BPM + nudge) or by uploading a file and mapping BPM plus a grid marker.
The Web Audio API is used for waveform analysis when displaying uploaded audio in the timeline.
MIDI
Controller input uses the Web MIDI API. No drivers, no installation. The app ships with a controller layout for the Akai APC mini MK2 and recognises it automatically.
Remote control
A second device can connect to a running Fantasynth instance and act as a remote. The connection uses PeerJS (a wrapper around WebRTC data channels) for direct peer-to-peer communication with low latency. The room code displayed in the Remote panel is rendered as a QR code using qrcode-generator, so the second device can join by scanning. The remote mirrors the clip grid and fader controls, letting you perform from a phone or tablet while the visuals run on a separate display.
Settings and persistence
User settings, saved compositions, presets, and MIDI mappings are persisted in the browser’s LocalStorage. Nothing is sent to a server.
Testing
- Vitest for unit tests covering the engine, composition system, and content handlers
- Playwright for end-to-end smoke tests, actor-loading checks, and screenshot automation used by the docs
Site
This documentation site is built with Astro. It is a separate static site from the instrument itself, deployed independently. The pages you are reading are written in Markdown and rendered to HTML at build time. The live preview embedded on the landing page is a stripped-down build of the engine that lazy-loads via IntersectionObserver as you scroll.
AI collaboration
Fantasynth was built in a long collaboration with Claude (Anthropic). Shader features, UI panels, the composition engine, the embed running on the landing page, the SVG diagrams in these docs, and this site itself. Claude wrote most of the code. The author made the choices.
The tone and direction are human. A lot of the typing is not.