by GANTASMO
This is the one-click Pinokio launcher for theDAW, a free music studio that runs on your own computer. You can generate a track from a text prompt, edit and mix it, turn it into sheet music, sing along to it with timed lyrics, DJ it, play it live, and run visuals behind it. Everything stays on your machine. There is no account and no subscription. The generator is Stable Audio 3, trained on licensed, ethically sourced audio, and UNDERFIT turns your own recordings into your own LoRA on your own GPU, so even the training step never touches a datacenter.
Install and Start below handle the whole stack: Python, CUDA, FFmpeg, the frontend, the LoRA trainer, the visual engine and the default model.
- Install clones the repo into
app/, pulls the Magenta sidecar submodule, installs FFmpeg through conda, resolves all Python dependencies withuv sync --group dev, builds the optional Underfit trainer environment, installs the frontend and VST Foundry packages withnpm install, clones the VJ-9000 app, clones or refreshes the SwayCommand cockpit for the SWAY tab and builds it, builds the live VST3 host on Windows, and pre-fetches the default generation model from the public mirror (Medium on Windows and Linux, Small on macOS) so the first CREATE does not wait on a download. - Start launches the FastAPI backend through its restart supervisor on
http://localhost:8600, then the Vite frontend onhttp://localhost:5173, and opens the app once the URL appears. Settings → Restart Server works under the launcher: the backend comes back inside the same Pinokio terminal. - Update pulls the launcher, moves the app clone onto the published
mainwith a fetch and a hard reset so a lockfile the last run rewrote cannot block it, refreshes the submodule, re-syncs the Python, Underfit and npm dependencies, re-provisions the VJ app if it is missing, refreshes and rebuilds the SwayCommand cockpit, and rebuilds the live VST3 host on Windows. Untracked content stays put:app/data,app/vj, the venvs,node_modulesand the downloaded models. - Reset deletes the dependency trees only:
app/.venv,app/underfit/.venv, and thenode_modulesfolders of the frontend, VST Foundry and VJ app. Your library, settings and generated audio underapp/datastay put. The next Install rebuilds the dependencies from clean. - Build VST3 Host (Windows only) compiles theDAW's native plugin host and shows the compiler output. Install and Update run the same build and carry on when it fails, so this is the button to press when VST3 plugins do not play live.
The other Stable Audio 3 checkpoints are one click away under Download Models (Small ARC, Small RF, Medium ARC, Medium RF), and theDAW also downloads any model the first time a generation needs it. The launcher points HF_HOME at the standard user Hugging Face cache (~/.cache/huggingface) rather than an isolated per-app cache, so checkpoints and the Hugging Face auth token already on the machine are reused. The official Stable Audio 3 and t5gemma repos are gated; the app falls back to a public mirror of the same weights automatically, and a Hugging Face token (the in-app sign-in, or hf auth login) unlocks the official repos.
- Click Install and wait for the dependency sync to finish.
- Click Start. The backend comes up first, then the web UI; the Open App tab appears when the URL is ready.
- Open the MAKE tab, type a prompt, press CREATE. Models do not download by themselves — allow downloads once in Settings → Models and the first CREATE fetches what it needs. The
smallmodel runs on a CPU;mediumneeds an NVIDIA GPU. - Right-click the new track in the library to open it in EDIT, MIX, SCORE or SING, or to load it on a DJ deck. The assistant orb in the bottom left answers questions about the app from the manual, and the in-app TOUR walks through every tab.
| Tab | What it is for |
|---|---|
| MAKE | Generate audio from a text prompt, from your own audio, or by filling in a painted region. Chimera combines several clips into one track. Suno (cloud) and Magenta RealTime 2 are in the same model list. |
| EDIT | A multitrack timeline. Cut, move and fade clips, record automation, add insert effects per track, and render the arrangement to a WAV file. MIDI tracks play sound banks or VST3 instruments, and a Symphony template seats an orchestra in a measured concert hall. |
| MIX | Mastering and effects. A chain of 25 effects, each with its own control panel, Quick Master knobs, VST3 plugins, .gan web-plugins and LUFS metering. |
| SCORE | Audio to MIDI to notation. Sheet music, tablature, arrangements, drum notation, and four play-along views that follow the track. Exports a Beat Saber level. |
| SING | Lyrics that follow the song word by word. Paste lyrics and a forced aligner times every word against the vocal, or tap the timing yourself. Imports and exports LRC. Scores your pitch. Puts the score, or a reading of the lyric's rhyme scheme and literary devices, beside the words. |
| LYRIC | A notebook for lyrics that belong to no song yet. Write with syllable counts and rhyme classes in the gutter, the analysis reading along beside you, and save a draft into a song when it is ready. |
| DJ | Two decks with beat sync, key lock, hotcues, loops, live stems, an FX rack, a sampler, and Automix that plays prepared performance sets and takes instructions from the assistant mid-show. |
| VJ | The VJ-9000 visual engine: audio-reactive terrain, cameras, GLSL shaders, cymatics, a GPU effect chain, and recording. |
| LOOM | A living colony of loops cut from your own library. Cells divide, envelop and wither on the beat clock while it plays. |
| SWAY | The SwayCommand gesture cockpit: scenes, a timeline and gesture axes bound to macros, driven by a camera or the Audima Labs Sway. |
| PERFORM | Launch scenes and clips from a grid. Opens Ableton sets and .tasmo projects. Pad effects and controller routing. Every track slot takes any rack effect or VST3 plugin. |
| FOUNDRY | Design a plugin interface on a canvas and export it as a .gan web-plugin. |
| NODEFI | Connect generation, effects and library nodes into a graph. Run it as a pipeline or play it live. |
| UNDERFIT | Train LoRA adapters on your own audio and use them when generating. |
| LEARN | A graph of your library: every remix, stem split, blend and cover, drawn in 3D or 2D. On a large library, counted lists you can search and page. |
| TOUR | Plan live dates on a map: venues, promoters, festivals, booking contacts and a route. |
Included at no cost. Stem separation up to 12 stems, a mastering suite, VST3 hosting, the HRTF spatializer The Owl, DJ decks with sync and Automix, audio-to-MIDI with engraving, LoRA training, forced-aligned lyrics with a whisper review, a rhyme and literary reading of any lyric, and export to WAV, MP3, FLAC, OGG, AIFF, Opus, M4A, MIDI, MusicXML and LRC. Every model in that list runs on the GPU when there is one, one at a time, and never twice for the same song.
Only in theDAW. theDAW-XR hand-tracked control on Meta Quest 3, Chimera clip fusion, DRAW (draw on a canvas to play generative music), native Audima Labs Sway motion-controller support, The Foundry plugin designer, import of Ableton, Reaper, FL Studio, Audacity, Audition, Bitwig and Resolume projects, the first non-Mac port of Magenta RealTime 2, and 28 themes plus a custom theme built from any image.
Type a prompt in the PROMPT box and press CREATE. The CONTROLS panel sets the model, the length in seconds, the number of sampler steps, the CFG scale, the seed and the batch size. To generate from your own audio, drop a file on the INIT slot and set the noise level: a low value stays close to the source, a high value moves away from it. To regenerate only part of a track, drop it on the INPAINT slot and paint the region to replace. Templates save a full set of controls, and the SAVED list keeps your prompts. Every result is saved to the library.
Chimera combines several clips into one track. Drop two or more clips on the CHIMERA STACK. Chimera analyzes the tempo and key of each clip, cuts them on the beat grid, pitches them into one key, arranges the pieces into a song, and asks the model to regenerate the joins so they do not click.
Suno generates in the cloud. Pick it in the model list for simple, custom, cover and mashup modes. Magenta RealTime 2 runs through magenta-rt2-nvidia, theDAW's own port for Windows with WSL2, native Linux, or a cloud GPU. Steer it with a style clip, MIDI notes on the keyboard, or both.
Drag clips along a track or onto another track with the Move tool. Split a clip with the Cut tool. Drag a clip's corner handles to set fade in and fade out. Each track has mute, solo, volume, pan and its own insert effects. Turn on WRITE and move a control during playback to record automation. COMMIT EDIT renders every audible track into one 44.1 kHz stereo WAV.
A MIDI track plays live on an instrument of its own: the bundled General MIDI bank, a sound bank you add (SF2, SF3 or DLS), or a scanned VST3 instrument picked in the track header, and every bounce, freeze and export prints it. Settings → Sound banks downloads the theDAW Orchestra bank: strings, woodwinds, brass, harp and orchestral percussion built from CC0 samples, with spiccato, staccato, pizzicato, tremolo and mutes as articulations. The Symphony orchestra template adds sixteen seated section tracks on five section buses that share a hall send, and the hall is the Detmold Konzerthaus, measured by seat and stage position. A track can also send its MIDI to a hardware port with clock, song position and Start/Stop.
The arrangement holds a tempo map and a meter map, and Use song tempo sets both from a song's rhythm analysis. Every track row draws its volume as a line: drag it for the fader, right-click it for a keyframe. A bus takes VST3 inserts, an automation lane rides any plugin parameter on a track, a bus or the master, and a plugin's window opens at the plugin's own size. Recording keeps takes for comping and punches on the loop region, with a metronome and a count-in. A song's sections land as markers, and Ctrl+drag copies a clip.
Add effects from the EFFECTS list to the CHAIN. Audio flows through the chain from left to right. The 25 effects cover mastering, compression, filters, vocal processing, lo-fi, stereo widening, reverb, delay, LUFS normalization and pitch shift, and each one opens its own control panel. The four QUICK MASTER knobs (PUNCH, AIR, DRIVE, CEIL) set the most common mastering moves in one place. VST3 plugins found in the standard plugin folders appear in the same list, and .gan web-plugins open in the effect stage. Press PROCESS CHAIN to render.
The Owl is a .gan web-plugin included with theDAW, alongside the Ares multi-effect shown under FOUNDRY below. Any FOUNDRY design exports to the same format.
Right-click a track in the library and choose Convert to MIDI (a drum stem gets a drum-kit transcription). Then open the SCORE tab in the bottom panel with that track selected. MAKE SHEET engraves the MIDI as MusicXML sheet music, MAKE TABS writes guitar, bass or ukulele tablature for a chosen tuning, capo and difficulty, ARRANGE builds a lead sheet, a piano reduction, a simplified part or a band score with drums on a percussion staff, and MAKE CHORDS derives a chord track from the lead sheet or estimates one from the audio. Scores export to PDF, SVG, ABC and MusicXML.
A score does not need a recording. IMPORT SCORE FILE opens a MusicXML, .mxl, kern or ABC file as a composition, and BROWSE CORPUS opens a piece from the music21 corpus. The EXPORT menu adds PERFORM (MIDI), which plays the sheet's dynamics, hairpins and articulations as an expressive MIDI with a tempo map, MIDI (SOUNDING) for the whole sheet or one part, and an audio render through MuseScore 4 and Muse Sounds when they are installed.
Every score is also a play-along. PAGE moves a cursor over the engraved pages, STRIP scrolls one continuous staff under a now-line, CHORDS shows guitar, bass or ukulele chord diagrams, and HIGHWAY shows the notes travelling toward a hit line in a notation, block or drum skin. The INSTRUMENT menu picks the parts and the view for your instrument, CALIBRATE measures the delay of your audio device so the notation lines up with the sound, NOW puts the now-line at the left or the centre, and INK picks the colour of the played notes. TRAIL decides what happens to a note after it sounds: Hold (the default) keeps every played note in the ink colour, so nothing flashes and the score fills in behind the now-line; Flash colours only the note that is sounding. Hold is the setting for anyone sensitive to flashing. The same chart exports as a Beat Saber level pack.
Lyrics come from the song's own lyrics field, from PASTE LYRICS, from an LRC file through IMPORT, or from whisper through TRANSCRIBE. The line being sung is white and slightly bigger, past lines dim, and the word being sung fills from left to right.
- ALIGN keeps your words and takes the timing from the vocal. A forced aligner (Meta's MMS aligner, through torchaudio) places every one of your words on the vocal stem, so no word is ever replaced by a guess. SING runs the stem separator first when the song has no stems. After the timing is saved, whisper listens to the same vocal as a review: a word it heard differently gets an amber underline and the header shows how many words differ. Hover the word to read what whisper heard. You stay the authority on the words.
- TAP times the lyrics by hand. Turn TAP on, play the song, and press Space at the start of each line. Backspace undoes the last tap. The − and + buttons move a line 50 ms.
- OFFSET shifts every line at once.
AUTO (on by default) runs ALIGN by itself when a song opens with lyrics but no timings, and an import with lyrics (a Suno track, a tagged file) is aligned in the background right after its stems, so the song is ready to sing when you open it. PITCH shows the melody of the vocal and draws what you sing into the microphone over it. EXPORT writes LRC, LRC with word tags, or plain text.
SING has four layouts. LYRICS is the karaoke alone; BOTH puts the whole SCORE tab beside it; SCORE is the score alone; STUDY puts the lyric's analysis beside the words.
Latin is in the language picker. A Latin lyric aligns with its macrons and ligatures kept on the page, and STUDY reads it by the Latin rules: syllables, stress and classical scansion.
STUDY reads the words back to you: the rhyme scheme letter by letter and section by section, the near and multisyllabic rhymes drawn on the syllables that rhyme, internal and cross-line rhymes drawn as arcs, and the alliteration, assonance, anaphora, refrains, enjambment and meter marked on the words themselves. Every finding carries a confidence you can see and a floor you can raise, so a loose slant rhyme looks loose. It runs on your machine from the words. The only part that asks a model is the optional pass for metaphor, irony and puns, and it is off until you turn it on.
The LYRIC tab is the same analysis beside a blank page: write lyrics with no song attached, with syllable counts and rhyme classes in the gutter, and save the draft into a song when it is ready.
Load a track on each deck from the browser at the bottom. Press SYNC to match the tempo of the incoming deck to the playing deck. Each deck has pitch, key lock, a 3-band EQ, a filter, hotcues, beat loops, loop rolls, slip and quantize. The FX rack has flanger, reverb and wah per deck and a master limiter. STEMS separates a deck into stems with a fader for each one. CUE sends a deck to a headphone output. AUTOMIX plays a set on its own, and START AUTO DJ in the header starts one in a single press. Each blend begins on a 16-beat phrase line, hands the bass from the outgoing track to the incoming one, corrects phase by bending the platter, and skips a key clash for the nearest compatible track. Hot cues are seeded from the analysis: the first downbeat, then the 16, 32 and 48-bar phrase starts.
Prepared performance sets. Drop a set folder with its audio files and a performance.json timeline into app/data/performance-sets/. It appears under Sets, and automix follows each track's cue-in, mix-out and transition length exactly as prepared. During the show the assistant orb can read what is on, start or stop the set, blend into the next track now, or move a track to play next.
Pick a source in the SOURCES panel: a webcam, a phone, tablet or Quest camera over the LAN, a GLSL shader, cymatics, a depth cloud, a spectrum, or a screen capture. Deck A applies geometry effects and deck B applies corruption effects. AUTOPILOT changes the picture on its own, BPM SYNC ties changes to the beat, and MIDI maps any control to a controller. REC records to WebM, and the backend transcodes it.
Open an Ableton set or a .tasmo project in the OPEN field. Each column is a track and each row is a scene. Click a clip to launch it, or click a scene to launch every clip in that row. Clips loop, warp to the tempo, and run through the track mixer and effects. Sway Perform adds pad effect punches, a template per song, and the SwayCommand deck for assigning a controller. Every track slot takes every rack effect and every scanned VST3 plugin; a hosted plugin opens its own window, and a knob, an XY axis or a gesture can drive its parameters.
Shown above with the included Ares multi-effect open as an editable design. Drag knobs, sliders, meters, buttons, displays and images from the left palette, or describe what you want to the AI panel on the right and let it place and style them. Upload a background image or pick a texture. OPEN .GAN opens an existing plugin to edit, such as the included Ares shown above. DEMO MODE switches between editing the controls and operating them. EXPORT CODE and PACKAGE write the design as a .gan web-plugin, GANTASMO's plugin format, which loads in the MIX chain next to VST3 plugins and the built-in effects.
Every song in the library is torn into bar- and beat-aligned fragments of each stem, indexed with its key, energy, rhythm, chords and words. LOOM plays that index as one dish that grows while it runs. It starts as a single spore you click; from there cells divide on the beat (a child is born on its parent at zero vitality and ripens over three bars), colonies form around existing loops, and idle cells are hollowed out over two bars and removed. Loops, rules, gates and mods wire to each other with rope-physics tendrils, and a colony is itself a cell with its own meter, so 7/8 grouped 3+2+2 can run inside a 4/4 dish at half speed.
SWAY embeds the SwayCommand cockpit whole: scenes down the left, a timeline underneath, and gesture axes (X, Y, PULSE, PRESS, SWAY) bound to macro knobs and named pads. Move in front of a camera, or move the Audima Labs Sway motion controller, and you are playing those controls. theDAW owns the only requestMIDIAccess() in the app and relays hardware into the cockpit, so a controller you plug in reaches it with no extra setup. Click a track's name on the cockpit's timeline to open its panel: EFFECTS lists every effect of theDAW's rack, and the VST3 list runs scanned plugins live in theDAW's plugin host. The kit and the synth have effect chains of their own, and LOAD .gan asks theDAW for the plugin file.
Drag nodes from the left list onto the canvas and connect their ports. In Run mode the graph executes through the AI stack (Stable Audio and Magenta generation, effects, merges, feedback loops) and saves the result to the library. In Live mode the same canvas plays stems, racks and routes in real time without a model.
Press NEW DATASET to add audio, then NEW FINETUNE to set the adapter type (eight types), the layer filter, the interval gate and the SVD base, and start the run. Finished adapters appear in the LORA panel on the MAKE tab, where they stack and each one has a strength control. The launcher's Install step builds the trainer environment, and UNDERFIT repairs it on its own if it breaks.
LEARN draws every track and the links between them as a 3D graph, a 2D graph, or a layered diagram. A remix, an inpaint, a stem split, a Chimera blend and a Suno cover each link to the track they came from.
On a large library LEARN opens on counted headlines instead of one drawing. Every number opens a list you can search, sort and page, built to hold 200,000 songs, and Classic graph draws the family of the song in focus.
The library is on disk under app/data, with its metadata in app/data/library.db. Every generated track is saved with its prompt, model and settings. Imported tracks keep their lyrics and tags. Sub-tabs list a track's STEMS, MIDI, VIDEO and SCORE files. A MIDI row's menu brings every stem MIDI of its song in at once: All stems to piano roll gives each stem a part of its own, and All stems to EDIT as tracks gives each stem an EDIT track, each on its stem's instrument. SUGGEST orders tracks into a playlist by Camelot key and BPM. The Catalogue is the full-width view of the same library with an inspector, spectrograms on demand and a lineage panel.
Each entry carries a provider badge: Stable Audio, theDAW, Suno, Udio, Riffusion, Magenta or Imported. Media roots in Settings → Storage name folders of your own, and an entry whose audio lives there plays in place with nothing copied. DETAILS shows a song's sections as a strip, found on its bar grid and labelled A B A', and the same sections become EDIT markers and roll markers.
theDAW remembers every path it writes. A file it installs, saves, downloads or exports is registered, so pickers open in the folder you last used for that kind of file, a Recent list hands the file straight back, and Show in Folder opens it in the file manager. Saving goes through the native Save As dialog.
Right-click a track and choose Convert to MIDI, or draw notes in by hand. The roll plays what you write and sends it to the EDIT timeline.
The roll carries a meter map, so the time signature can change across one piece, with additive groupings like 7/8 as 2+2+3 and a pickup bar before the first full bar. Polymeter lanes each run their own meter and loop length against the same clock. A pitch bend lane sits under the keys with a semitone range and LINE, HOLD and CURVE point shapes.
The SHAPE row under the roll transforms what is there: harmony, ragtime, runs, polyrhythm and humanize, each with syncopation and accent amounts. MATCH pulls a song's meter map, tempo and lanes from its rhythm analysis, and a part transcribed from the song keeps its notes at their seconds under the new tempo. With KEEP TIME on, beside BPM, a tempo you type keeps every note at its second too. CLEAN reduces a transcription to one line or to a pitch range. GEN writes LOOM generator rules into the active lane. ARP writes a chord-progression arpeggio. CAPTURE, SONG and FORM assemble a multi-section arrangement from intro to outro. A microphone turns a sung line into notes, and an AI compose flyout writes a piano part from a key and mode.
One roll document holds a whole orchestra as parts, each on a sound of its own: a sound bank preset or a VST3 instrument. The roll plays a tempo map with ramps and fermatas, snaps to triplets, quintuplets and septuplets, and carries named section and movement markers. Lanes under the keys draw each part's controllers (modulation, volume, pan, expression, pedal, brightness and reverb send), its articulations, and a note's own pressure, timbre and bend. MPE files import and export with that expression.
COMPOSE on the rail writes music by rule and checks it: a seeded harmony planner in roman numerals voiced in four parts, a voice-leading check, figured bass realization, form plans (sonata, rondo, variations, minuet, scherzo and a four-movement symphony), species counterpoint one to five on a cantus firmus, two-voice canons, fugue expositions with tonal answers, and ten composer style profiles. TRANSFORM inverts, reverses, augments, diminishes, sequences and fragments the selected notes.
It reads MusicXML, ABC, Humdrum and MIDI, taking time signatures and tempo maps from MIDI files, and writes MIDI back out with the roll's own tempo and meters.
- LEVELS meters loudness, true peak, dynamics and stereo image against a delivery target.
- VISUALIZE shows an oscilloscope, a spectrum or a radial view.
- MIDI is the piano roll described above: parts, meter and tempo maps, polymeter lanes, controller and bend lanes, COMPOSE and the SHAPE transforms.
- SEQUENCE is a step sequencer with 16 steps per voice.
- DRAW plays generative music from strokes on a canvas.
- SCORE, SING and DETAILS show the selected song's notation, lyrics and metadata.
- LYRIC is a notebook for writing and analysing lyrics with no song attached.
- DETAILS also holds the media bucket: dropped files and URL imports (YouTube and SoundCloud) waiting to go to a tab or the library.
- SLIDE is a touch control surface. SWAY controls music from camera-tracked movement.
Search a city and TOUR returns the venues in it, 513 for Austin above, each with its type, address, and the website, email and phone to book it. Add the ones you want as stops and it works out the drive between them, with EV charging stops if that is what you drive.
Controller recognition knows about 110 device profiles, detects a connected controller, learns one by capture, and Controller Vision identifies a controller from a photo. The Audima Labs Sway motion controller works natively. theDAW-XR turns a Meta Quest 3 into a hands-only controller with hand-tracked MIDI, passthrough video into VJ and co-located multiplayer. A phone web app pairs with the desktop for remote MAKE, transport, DJ and library control, and a switch in Settings serves theDAW over HTTPS on the network.
The footer is on every tab with transport, a seek bar, volume and download. A track menu key opens every audio action for the loaded track, grouped, with a row per stem that sends that stem to EDIT, the init slot, the inpaint slot or the Chimera stack. The action key beside it changes with the tab (CREATE, EDIT, TRAIN, MIX, DJ), showing a progress fill while the job runs and cancelling it on a second press. Repeat is off, all or one; an output picker chooses the audio device; a Master FX chip is reachable from any tab; Ctrl+S saves the project. A camera key in the header, or F9, records the app window with its sound into a WebM file.
Status notices go to the processing log and to the assistant orb's speech bubble. The log keeps the last 500 entries, filters to errors only, and carries a live GPU, VRAM, CPU and RAM readout.
The assistant orb streams chat from any configured provider (Claude Code over the CLI, Gemini, Anthropic, OpenAI, Grok, Groq, OpenRouter, Ollama, LM Studio, llama.cpp, vLLM), accepts attachments, and answers questions from theDAW's own documentation through a RAG index. Point it at Ollama or LM Studio and the assistant stays local too. It keeps its chat history across sessions, asks before it acts in Ask mode, and works on the project through tools: tracks and roll parts, composer and score calls, sound banks and tuning.
| Key | Type | Params | Autoencoder | Hardware | Max duration |
|---|---|---|---|---|---|
small |
ARC | 433 M | SAME-S | CPU | 120 s |
medium |
ARC | 1.4 B | SAME-L | GPU (CUDA) | 380 s |
small-rf / medium-rf |
RF | 433 M / 1.4 B | SAME-S / SAME-L | CPU / GPU | 120 / 380 s |
same-s / same-l |
Autoencoder | 266 M / 1.7 B | n/a | CPU / GPU | n/a |
ARC checkpoints are post-trained for 8-step inference at cfg_scale=1. RF checkpoints are rectified-flow bases for LoRA training at cfg_scale=7 and about 50 steps. Beyond the default model the Install step pre-fetches, nothing downloads at startup — Local only is on by default. Once downloads are allowed in Settings → Models, a model loads on the first generation that needs it. Checkpoints already on disk can be registered in the same panel or placed in app/models/.
The Python dependency set self-selects per platform through uv:
| Platform | Torch build | Notes |
|---|---|---|
| Windows | CUDA 13 wheels (torch 2.14) + prebuilt flash-attention | Full feature set. Flash-attention is enabled only on Ampere or newer GPUs; Turing cards (RTX 20xx, GTX 16xx) fall back to standard attention automatically. |
| Linux x86_64 | CUDA 13 wheels (torch 2.14) | Full feature set; Magenta sidecar supported. On glibc older than 2.38 the Azure Kinect backend (pyk4a-bundle) is skipped and only the Kinect point-cloud source is lost. |
| macOS | Standard PyPI torch (CPU / MPS) | Small model recommended; flash-attention, Azure Kinect, and the Magenta sidecar are skipped automatically. |
The Small generation model runs on CPU, so machines without an NVIDIA GPU still generate audio. The Medium model, Magenta, Demucs and GPU whisper want an NVIDIA driver of 580 or newer: the torch build is a CUDA 13 wheel, which needs the R580 driver branch.
Live VST3 (instruments, plugin windows and hosted effects in EDIT, PERFORM and the piano roll) runs in theDAW's native plugin host. The host is Windows-only and built from source. Install and Update build it with the Visual Studio Build Tools that Pinokio's AI bundle installs, and with CMake, which the launcher installs through conda. Build VST3 Host in the launcher menu runs the build by itself. Without the host, VST3 effects still process offline.
The hamburger menu opens Change Theme: 28 themes in seven groups (dark, metal, duotone, light, light duotone, pastel and gradient) plus a custom theme built from any background image. A theme recolours every surface through shared design tokens. Obsidian is the default; the screenshots on this page use Brushed Steel.
The library panel collapses, the right panel resizes, the bottom panel switches between its tabs and maximizes, the LEARN graph goes fullscreen, and the DJ tab's Design Mode rearranges the console.
The app fixes its own ports: the frontend proxies /api to localhost:8600 and Vite runs with strictPort on 5173. If Start fails immediately, close anything already using 5173 or 8600 (for example a copy launched through theDAW.bat). The VJ sidecar (port 5187) is spawned by the backend on first use and bootstraps its own npm packages.
Both servers bind 0.0.0.0, so the web UI, the phone companion, Quest streaming and XR control are reachable from other devices on the same network, exactly as under theDAW.bat and theDAW.sh.
theDAW is a React frontend over a FastAPI backend. The backend wraps the Stable Audio 3 pipeline, a plugin module system, and sidecar processes it starts on demand. Large features load on first use, not at startup, which is why Start returns quickly even though the app is this big. Everything below app/ is what Install put there.
flowchart TD
UI["theDAW UI<br/>MAKE EDIT MIX PERFORM DJ VJ SWAY LOOM FOUNDRY UNDERFIT NODEFI LEARN TOUR"]:::in
API["FastAPI backend :8600<br/>job queue, FFmpeg, introspection"]:::proc
SA3["Stable Audio 3<br/>DiT + SAME AE"]:::eng
MODS["Plugin modules<br/>stems, notation, lyrics, midi, vocal ..."]:::proc
MRT2["magenta-rt2-nvidia<br/>WSL2 + JAX"]:::side
WSP["whisper + Demucs<br/>isolated venvs"]:::side
VJ["VJ-9000<br/>WebGL engine"]:::side
XR["theDAW-XR<br/>Quest 3"]:::side
UI -->|/api/*| API
API --> SA3
API --> MODS
MODS -. spawn .-> MRT2
MODS -. spawn .-> WSP
MODS -. iframe .-> VJ
XR <-->|ADB, MIDI, video| MODS
classDef in fill:#0f3d57,stroke:#3aa0db,color:#eaf6ff;
classDef eng fill:#3a2356,stroke:#a877e0,color:#f3ecff;
classDef proc fill:#0e3b3b,stroke:#2bb3a3,color:#e6fffb;
classDef side fill:#4a3115,stroke:#e09a3a,color:#fff4e3;
Generation. The prompt, the init audio, the inpaint region and the Chimera stack all condition one generation. The DiT produces latents, the autoencoder decodes them to audio, the result is saved to the library, and LEARN records where it came from.
One song, many outputs. A single library entry can be separated into stems, converted to MIDI, engraved as a score, played along to, and sung to. Each result is stored with the entry, so no model is ever run twice for the same song.
The backend serves a full HTTP API on http://localhost:8600, with interactive documentation at http://localhost:8600/docs.
curl:
curl -X POST http://localhost:8600/api/generate \
-F "prompt=warm analog synth arpeggio, 120 bpm" \
-F "duration=30" \
-F "steps=8"Python:
import requests
r = requests.post(
"http://localhost:8600/api/generate",
data={"prompt": "warm analog synth arpeggio, 120 bpm", "duration": 30, "steps": 8},
timeout=600,
)
r.raise_for_status()
print(r.json())JavaScript:
const form = new FormData();
form.append("prompt", "warm analog synth arpeggio, 120 bpm");
form.append("duration", "30");
form.append("steps", "8");
const res = await fetch("http://localhost:8600/api/generate", { method: "POST", body: form });
console.log(await res.json());curl:
curl -X POST http://localhost:8600/api/library/import \
-F "file=@track.wav" \
-F 'metadata={"title": "My Track", "source": "import"}'Python:
import json
import requests
with open("track.wav", "rb") as f:
r = requests.post(
"http://localhost:8600/api/library/import",
files={"file": f},
data={"metadata": json.dumps({"title": "My Track", "source": "import"})},
)
r.raise_for_status()
print(r.json()["id"])JavaScript:
const form = new FormData();
form.append("file", fileBlob, "track.wav");
form.append("metadata", JSON.stringify({ title: "My Track", source: "import" }));
const res = await fetch("http://localhost:8600/api/library/import", { method: "POST", body: form });
console.log(await res.json());The rest of the surface (stems, MIDI conversion, notation, DJ, VJ, project import, plugins) is browsable at /docs while the backend is running.
The inference library is installed in the launcher's venv, so it can also be driven directly from app/:
from stable_audio_3 import StableAudioModel
pipe = StableAudioModel.from_pretrained("medium")
audio = pipe.generate(prompt="Lo-fi boom bap meets orchestral strings, 84 BPM", duration=180)
pipe.load_lora("style.safetensors") # adapters stack; strength is settable at runtime
pipe.set_lora_strength(0.8)The full manual ships with the app, is served by the Docs button in the UI, and is on disk under app/docs/.
| Document | Contents |
|---|---|
| USER_GUIDE.md | The complete manual: every feature, control and endpoint. |
| pinokio-launcher.md | This launcher: Install, Start, Update, Reset and what each step does. |
| prompting.md | How to write prompts, conditioning signals, and a style reference. |
| notation-and-score.md | Audio to MIDI, sheet music, tabs, arrangements and play-along. |
| sing-along-and-lyrics.md | Where lyrics come from, ALIGN and the review pass, tapping, LRC. |
| sound-banks-and-midi-out.md | Sound banks, how a track picks a sound, MIDI out, and printing a VST3 instrument. |
| song-sections.md | The section finder: boundaries, repeat letters, roles and confidence. |
| edit-automation.md | Automation lanes in EDIT, including the volume line. |
| nodefi.md | Node graphs: AI pipelines and live performance. |
| sway-perform-live.md | PERFORM, the SwayCommand deck, scenes, punches and templates. |
| dj-and-genealogy.md | The DJ console, the LEARN graph and the watch-link broadcast. |
| lora.md | LoRA adapter types, layer filters and training. |
The GitHub Wiki has the same index across theDAW and its sidecars.
| Project | Role |
|---|---|
| theDAW | The app this launcher installs. |
| VJ-9000 | The WebGL audio-reactive visual engine in the VJ tab. Cloned by Install; also runs standalone. |
| magenta-rt2-nvidia | The first non-Mac port of Magenta RealTime 2, pulled as a submodule. |
| theDAW-XR | The Meta Quest 3 companion: hand-tracked MIDI, passthrough streaming and colocation. |
Start fails immediately. Something else is on 5173 or 8600, most often a copy of theDAW launched outside Pinokio through theDAW.bat. Close it and press Start again.
"API UNREACHABLE" banner. The backend is not listening yet. Check the Pinokio terminal for the backend log, and test with curl http://localhost:8600/api/health.
Out of memory on the Medium model. Use the small model, a shorter duration, or close other CUDA processes.
Static or noise from the Medium model on Windows. Check GET /api/health for flash_attention_active. On Turing GPUs (RTX 20xx, GTX 16xx) it reads false by design and the model runs on an equivalent fallback.
VST3 plugins do not play live. Live VST3 needs the native plugin host, which exists on Windows only. Stop the app, press Build VST3 Host and read the output. The build needs the Visual Studio Build Tools with the C++ workload.
A dependency tree got wedged. Run Reset, then Install. Reset removes only the dependency trees; your library, settings and audio under app/data are untouched.
User Guide §23 has the full list.
GANTASMO is an amorphous entity by Daniel Joaquin Trujillo and Josh Valenzuela that defies conventional classification. We make thought provoking, highly technical, yet listenable music inspired by the underappreciated pioneers of modern music. Beyond musical composition and performance, GANTASMO is a powerhouse of research and development in the fields of Artificial Intelligence, Augmented Reality, Virtual Reality, the democratization of musical tools and education, and the preservation and evolution of musical history and traditions predating modern recording infrastructure.
theDAW was built by GANTASMO as part of the Music Hackspace Music Technology Hackathon at Berklee College of Music.
- Stability AI provides Stable Audio 3 and stable-audio-tools, the diffusion model and pipeline at the core of theDAW.
- Magenta RealTime by Google DeepMind provides real-time music generation, running through theDAW's own NVIDIA/CUDA port.
- Suno provides cloud music generation.
- T5Gemma by Google handles text conditioning.
- Demucs by Meta AI separates stems, basic-pitch by Spotify converts audio to MIDI, the MMS forced aligner by Meta AI times lyrics, and faster-whisper transcribes lyrics and reviews them.
- music21 by MIT builds MusicXML, ABC, tabs and arrangements, alphaTab and OpenSheetMusicDisplay render tablature and scores in the browser, and MuseScore engraves PDF and SVG.
- MLX by Apple is the inference core the Magenta port builds on, extended here with a CUDA backend.
- PyTorch, FFmpeg, three.js, WaveSurfer.js, React, Vite and Tailwind CSS are used throughout, alongside the wider open-source community.
- Pinokio by cocktailpeanut is what this launcher runs on.
Corrections and additions to this list are welcome through a GitHub issue.
To Music Hackspace and Berklee College of Music for hosting the hackathon, and to Zack, CJ, Jordi, Zach, and Matt from Stability AI for their continued help and support.
Made by Daniel Joaquin Trujillo and Josh Valenzuela as GANTASMO.