How UTAU Works: The Studio, the Sheet Music, and the Singer

Updated

UTAU works like a collaboration with a real singer, just done entirely inside your computer. You set up the UTAU program as your "studio," load a UST as the "sheet music," assign a voicebank as the "singer," and then tell the program to render the performance into vocals you can mix into a music track. This explanation follows the framework laid out on utauguides.carrd.co, a site that organizes UTAU learning into Overview, Basic, and Advanced levels.

The four-step process

The utauguides overview breaks UTAU down into a sequence that mirrors recording a real vocalist:

  1. Prepare your studio — Install and open the UTAU program. This is the environment where everything happens.
  2. Prepare your sheet music — Load a UST (UTAU Sequence Text), which tells the singer what notes to sing and how.
  3. Bring in your singer — Assign a voicebank, which gives the program the audio data and configuration needed to actually produce singing.
  4. Record — Tell the program to render the entire UST. The output is vocals you can drop into a music track.

The site's framing is deliberate: by refining both the UST and the voicebank, you can make the singer perform any song in any way. Neither piece alone is enough.

What a voicebank actually contains

A voicebank is not a single file. According to utauguides, it has two essential parts plus supporting files:

Part Role
Audio files The raw recorded samples the voicebank uses to sing (e.g. n_ba_bi_bu_be_bo_bo.wav, a_tsu.wav)
Configuration files (oto.ini) Split, time, and name the audio files so UTAU knows where each sample begins and ends, and the "perfect" timing to match a recording to a note
Aliases Names for cut samples that let a UST "command" the singer to summon that specific sample
Frequency files (.frq) Pitch information from a recording, used to "flatten" pitch so the singer stays on-tune
Other files Folders, readme.txt, character.yaml — more relevant for managing and releasing voicebanks across programs

The distinction between audio files and configuration files matters because a voicebank with good recordings but bad oto.ini timing will still sound wrong. The configuration is what turns raw samples into usable notes.

What a UST controls

A UST is the sheet music. utauguides lists the properties it holds, with four called out as especially important:

  • BPM (Beats Per Minute) — the tempo of the piece
  • Notes — the pitches and their placement
  • Aliases — the commands that pull specific samples from the voicebank
  • Pitchbends and vibratos — how the pitch moves within and between notes

USTs can hold other properties as well, but these are the ones the guide treats as foundational. The alias field is the direct link between sheet music and voicebank: a UST doesn't reference a sound directly, it references an alias that the voicebank must be able to answer.

Where to start based on your level

utauguides splits its content by skill level, and the split is useful for deciding what to read first:

  • Overview — for understanding the concepts: how UTAU works, what a voicebank is, how USTs work. Start here if you've never used UTAU.
  • Basic — for people who already understand the concepts and want to learn how to actually use the program.
  • Advanced — for people familiar with using UTAU who want to improve further.

If you're asking "how does UTAU work?" you're in the Overview stage. The natural next step is Basic, where the concepts above turn into hands-on operation.