hellyee: a closed-loop MCP server for Ableton Live

Why this exists

I have been making music as a hobby for a long time. What kept bugging me was not the ideas — it was that the sounds kept clashing and the track never quite carried the feeling I heard in my head. Most of it was technical: mixing calls I could not hear, mastering I could not get right. And because this is a hobby, I never carved out the time to properly get better at those crafts. So I built hellyee instead.

hellyee is an MCP project sitting on top of AbletonOSC, with a set of skills bundled alongside. A small bit of setup and you can drive Ableton Live through an AI. There are limits — most of them come from Ableton not exposing certain APIs. Third-party plugin parameters, for example, need to be mapped by hand, so a bit of manual config is unavoidable.

Architecture

Nothing exotic:

Claude ⇄ MCP ⇄ hellyee (Python) ⇄ OSC ⇄ AbletonOSC ⇄ Live Object Model

Claude launches hellyee as a subprocess and speaks MCP to it. hellyee turns those calls into OSC messages to AbletonOSC, a remote script by Daniel Jones that runs inside Live’s own Python and drives the Live Object Model. Credit where it is due — AbletonOSC is what makes any of this possible.

Stock AbletonOSC does not expose the browser, the master track, or arrangement clips. hellyee ships handlers that add all three, plus a patcher that installs them into your local AbletonOSC.

Install

uvx hellyee setup

pip install hellyee && hellyee setup

The installer downloads AbletonOSC, patches it with hellyee’s extra handlers, and writes the Claude MCP config. It is idempotent — safe to re-run.

Then there is one step that cannot be automated. Live has no API for enabling its own control surfaces, so you have to do it by hand, once:

Quit and reopen Live (remote scripts are only scanned at startup).
Preferences/Settings → Link/Tempo/MIDI.
Pick AbletonOSC as a Control Surface. Leave Input and Output as None.

That is the whole manual bit. From then on:

hellyee doctor   # verifies connection and handler patch
hellyee smoke    # end-to-end: creates a track, writes and quantizes notes,
                 # loads an instrument, deletes the track

Core install is ~20 MB. The audio features — hum-to-MIDI, voice commands, stem separation — are opt-in and add ~380 MB, so you only pay that cost if you want them.

Three conventions worth knowing

Time is in beats. One 4/4 bar is 4 beats; a 16th note is 0.25. If a tool asks for start or length, that is what it wants.
Pitches use Live’s display: C3 = 60. Standard MIDI calls that C4. hellyee follows Live, so the note Claude writes matches what you see on the piano roll.
Device parameters are set in the unit you mean, never raw. Live’s raw values sit on internal scales that are not what the UI shows — Auto Filter’s Frequency runs 20–135 but reads as “265 Hz” in the interface. So the tools take hz=250, display="1/4" for named settings, or percent= for unitless controls, and report the displayed value back so you can confirm what actually landed.

That last one matters more than it sounds. It is the difference between “set the frequency to 80” (which value did I mean?) and a tool that says “set to 250 Hz, confirmed as 250 Hz.”

Bonus: stem separation

Alongside the closed-loop control, hellyee wraps a stem-separation model you can call over MCP. Hand it a track and it splits vocals and instrumentals out. Not perfect — bleed is normal on busy mixes, vocals separate most cleanly — but it does the job.

Closing

Give it a try. If you like it, PRs are welcome. If you hit something, open an issue and I will take a look.

AI cannot make music. But it knows the math of music better than we do — you can use it like a nerdy music student. Emotion? We barely understand our own emotions; let’s not expect the AI to.

Repo, install, issues: github.com/guvense/hellyee.