Skip to content
kevinraymondPublic

About

Free, open-source real-time audio & music visualizer / VJ engine, with audio-reactive GPU shaders, particles, and layer compositing, driven by WGSL and controlled with MIDI, OSC, or your phone. Cross-platform (Vulkan/Metal), with NDI, Spout, Syphon, and virtual-camera output.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

65 stars

Watchers

1 watching

Forks

Latest commit

 

History

731 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Fosfora

Turn whatever your computer is playing into live, beat-synced visuals.

CI Latest release License Wiki

Fosfora reacting to music Dancer footage: Free Stock video by Videezy

Fosfora is a free, open-source music visualizer built for actually performing with. There's nothing to code, no account to make, and no project to set up: open it, play music, and it reacts. When you want more, every knob is there — 58 effects, an eight-layer stack, MIDI, OSC, your phone as a control surface, a webcam, and four ways out to your video mixer: NDI, Spout, Syphon, or a virtual camera.

See it move

Nine of the fifty-eight, eight at default settings. Fluvid reacts to whatever moves on the layers beneath it, so it is shown over a dancer video with tuned settings:

Cymatics
Cymatics
Chladni standing waves, per band
Morph
Morph
Particles spring between images
Murmur
Murmur
Starling murmuration at twilight
Intarsia
Intarsia
Beat-revealed mosaic inlay
Symbiosis
Symbiosis
Particle life, ecosystems emerging
Fluvid
Fluvid
Motion stirs a fluid, here a dancer
Genesis
Genesis
Two species, self-organizing
Accretion
Accretion
N-body gravity, discs and slingshots
Lumen
Lumen
Real-time global illumination

→ See all 58 effects in the gallery

Past the defaults

Those are effects out of the box. This is what the rest of the app does to them — six setups you can build from the panels, each one a preset you can save and recall:

Obstacle collision
Flow around a body
Water parts around a silhouette — a photo, or your webcam
Layer stack
Four layers, blended
Voronoi and curtains screened over a particle flock
Gaussian splat scene
A 3D capture, playing
A photoscan rounded into points, breathing with the mix
Custom bindings
Wired to the music
Drums, harmony and the drop on separate parameters
Image as particles
Your own image, as particles
Bass shoves it apart, springs pull it back
Scene cue dissolve
Cued and dissolved
Presets chained into a set, crossfading on cue

→ How to build these · the presets behind them are in scripts/capture/demos/

→ Take the tour on the wiki: what Fosfora can look like and how to get there, with a clip for every recipe. It starts with trama node chains, which move, spin and echo a single layer or the whole frame.

Your first 60 seconds

Three steps, and you can't break anything — every setting saves to its own file and can be reset.

  1. Download and open it. A visual is already running when the window appears. The interface fades in after a second or two, or press D to show it right away, and a short tour points out each part of it.
  2. Point it at your audio. This is the one step worth getting right — see below if nothing moves. Most of the time it just works.
  3. Play something. Click effects in the catalog along the bottom until one grabs you, then press F for the full output.

The Fosfora interface: the layer stack, Fluvid's controls, the output and the effect catalog
Fluvid over a dancer video and Aurora. Footage: Free Stock video by Videezy

Every output is built into the official downloads. Spout (Windows) and Syphon (macOS) just work — nothing to install. NDI® needs the free NDI® runtime; the Linux virtual camera needs the v4l2loopback kernel module (Setup ▸ Outputs and streams shows the one-line setup).

Build from source instead

Needs Rust 1.97+ (pinned via rust-toolchain.toml) and a Vulkan-capable GPU.

git clone https://github.com/kevinraymond/fosfora.git
cd fosfora
cargo run --release                    # no extra deps
cargo run --release --features release # everything the downloads have (needs the deps below)
cargo run --release --features video   # video layers — needs ffmpeg on PATH
cargo run --release --features webcam  # webcam layers — needs libclang-dev, v4l-utils (Linux)
cargo run --release --features ndi     # NDI out — needs the NDI runtime
cargo run --release --features "v4l2,spout,syphon"  # virtual camera / Spout / Syphon — each inert off its OS
cargo run --release --features depth   # webcam + MiDaS depth — needs libssl-dev, libclang-dev
cargo run --release --features link    # Ableton Link tempo sync — needs cmake; see licensing note below

Ableton Link licensing: the link feature builds Ableton's Link library, which is GPL-2.0-or-later — a binary built with it is distributed under GPL terms, so the prebuilt downloads don't include it. Building it from source for your own rig is fine.

What you can do with it

Stack and blend Layers work like Photoshop or OBS — up to 8 of them, 13 blend modes, drag to reorder. Put a slow shader under a particle storm, dial the opacity, and it's a new look. Three of the modes warp instead of tint: set a layer to Displace and its shapes bend everything underneath, so a ring effect becomes a shockwave through whatever it's sitting on. Layers can also be images, GIFs, video files or a live webcam, not just effects.
Perform it live Map any MIDI knob to any parameter by clicking M and wiggling the knob. Same for OSC. Or open your phone's browser and use it as a touch surface — no app to install.
It genuinely listens Not just "loud = big". Fosfora tracks 85 things about your music 86 times a second — beat and tempo, key, percussive and harmonic content pulled apart, the moment a build turns into a drop — and any of them can drive any parameter.
Bring the room in Feed it a webcam and let particles flow around your silhouette, or a photo, or a depth map. Hand and body tracking stream in over the bridges.
Send it anywhere Spout (Windows), Syphon (macOS) or NDI out to your video mixer, a Linux virtual camera that Zoom and browsers see as a webcam, or record straight to a file — H.264, HEVC or AV1 in MP4 or MKV, up to 8K, hardware-encoded, with the audio muxed in.
Save the moment Presets store your whole layer stack. Scenes chain them into a cue list that advances on a timer, on the beat, or when you hit the spacebar.
Make it yours Every effect is a WGSL shader you can open in the built-in editor and edit while it's running — it recompiles on save and tells you where you broke it.

The effects

58 built-in, all audio-reactive out of the box, every parameter a slider you can map to MIDI or OSC. Full gallery with clips →

Shaders — Aurora · Beam · Drift · Fluvid · Frost · Iris · Plateau · Prism · Pulse · Shards · Storm · Strand · Strata · Tunnel

Particle simulations — Accretion · Array · Cascade · Chaos · Cleave · Cymatics · Flux · Genesis · Morph · Murmur · Mycelium · Polycephalum · Raster · Splat · Symbiosis · Tesla · Tide · Turing · Vessel

Lattice (3D cellular automata, ray-marched) — 445 · Brain · Builder · Chunky · Clouds · Pulse · Pyroclastic · Shells

Any particle effect can also be switched into Volumetric mode, which renders the layer you're on as ray-marched fog instead of discrete points — the same simulation, made of smoke.

Controls

Key Action
D Show / hide the UI
F Fullscreen
B Binding matrix
[ ] Previous / next layer
Space Next scene cue
T Play / pause the scene timeline
Esc Quit

Binding matrix (press B) — a full-screen patch bay. Drag a line from any source (a MIDI knob, an OSC message, an audio feature, your phone, a hand-tracking bridge) to any target (a slider, layer opacity, a particle setting) and it moves with the music or with you.

MIDI — click the M next to any parameter, move a knob, done. Auto-connects and hot-plugs.

OSC — receives on port 9000 (from this computer until you switch on Other devices), transmits on 9001. Click O next to a parameter and send any message to bind it, or address things directly: oscsend localhost 9000 /fosfora/param/warp_intensity f 0.8. Full address list →

Signal — run fosfora --signal and the analysis engine broadcasts beats, bars, sections, key, phrase position and drop prediction over OSC with no window at all — Fosfora as the analysis brain of a TouchDesigner/Resolume/lighting rig. The schema →, and a ready-made TouchDesigner component and examples with a five-minute setup: TouchDesigner →

Your phone — enable the web surface, switch on Other devices under Setup ▸ Control, and scan the QR code it shows (or open the network address beside it) on any phone or tablet on the same network. The link carries the access key other devices need; New key disconnects everyone holding the old one. Multiple people can connect at once.

Documentation

Wiki The picture-first tour: clips, recipes and demo presets to load
Gallery Every effect, in motion
Tutorials The full guide — effects, audio, layers, scenes, MIDI, OSC
Quick reference Shortcuts, blend modes, OSC addresses, config files
Signal The headless analysis broadcast — --signal and the /fosfora/v1/ OSC schema
TouchDesigner Quick start for the TouchDesigner examples and the fosfora_signal component: beats, phases, sections and key as CHOP channels
Audio features All 85 features in plain English, and the research behind them
Technical Architecture, shader authoring, the .pfx format
Credits The libraries and papers this is built on

Settings, presets, scenes and mappings all live in ~/.config/fosfora/ (and the equivalent on macOS and Windows). Delete it to reset; nothing else on your system is touched. (Upgrading from an older release? Your ~/.config/phosphor/ directory is moved over automatically on first launch.)

FAQ

The visuals aren't reacting to my music. Fosfora listens to an input device. Pick the right source in the Input list under Audio, beside the output preview (or in Setup ▸ Audio):

  • Linux — pick the Monitor of your output device. That's your system audio; a plain microphone only hears the room.
  • Windows — pick the WASAPI loopback device for system audio, or a mic for the room.
  • macOS — the default input is your mic. To visualize system audio, route it with a loopback tool such as BlackHole.

I just get a black screen, or it won't start. Fosfora needs a Vulkan-capable GPU (Vulkan on Linux and Windows, Metal on macOS). Update your graphics drivers first.

macOS says it can't verify the app. The official .dmg is signed and notarized, so downloading and dragging should just work. If macOS still blocks it, right-click the app → Open the first time.

Can I use it with my VJ software? Yes — turn on Spout (Windows) or Syphon (macOS) and Fosfora shows up as a live source in Resolume, VDMX, OBS, TouchDesigner and anything else that speaks them, with nothing to install. NDI does the same across the network. Or record to a file and drop that in.

From the dev

Thanks for checking this project out! I've gone through a couple experimental projects recently - EASE and EASEy-GLYPH - and with Fosfora I'm trying to put it all together.

I hope this project is genuinely useful. I've tried to make it as easy to use as possible, but I'm no kind of performer so I'm open to suggestions for improvements (drop an issue)!

I started writing software during the time of C-64 and TRaSh-80, and even with the field of AI "before it was cool" ... even so, I never imagined we would get to the point where I can architect and design and not need to write out everything. Wild times.

Give it a try! Let me know how I can make it better for you.

Contributing

New effects are the easiest way in — a WGSL shader plus a small JSON file, and it shows up in the browser automatically. See CONTRIBUTING.md and the shader authoring guide.

Built on wgpu, egui, cpal, rustfft and a lot of published research — full credits.

License

Dual-licensed under MIT or Apache 2.0, at your option.

About

Free, open-source real-time audio & music visualizer / VJ engine, with audio-reactive GPU shaders, particles, and layer compositing, driven by WGSL and controlled with MIDI, OSC, or your phone. Cross-platform (Vulkan/Metal), with NDI, Spout, Syphon, and virtual-camera output.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

65 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages