[id="C4276659642"] .page-layout {
align-items: center;
}
[id="C4276659642"] .page-layout {
align-items: center;
}
Passengers of the Ship Named Daughter of Mountains🌋
Immersive Interactive Opera🕗 2 Months
🛠️ Unity, C#, Touch Designer, QLC+ (for lighting), DMX (for audio), Premiere Pro
🧢Unity Engineer
[↗]🔗 Download macOS Demo
A live-piloted immersive theater vehicle where audience choices steer a branching, real-time narrative. The audience sits inside a hand-built ship, watching virtual worlds unfold through three projection windows that tilt and rotate in response to the ship's movement, as live performers and pre-shot footage blend into one continuous reality, staged as an interactive opera exploring what it means to become human in a technological era.
I was the sole Unity developer and built the entire digital backbone of the production:
- Real-time control system: integrated a flight controller joystick as the pilot's live input device, driving playback and transitions across the experience
- Multi-window video system: engineered the three-screen ("window") output pipeline in Unity, synchronizing footage across displays to sell the illusion of a single vehicle moving through virtual space.
- Lighting and sound integration: bridged Unity to DMX and QLC+ so lighting and audio cues could be triggered and sequenced directly from the same real-time system driving the visuals
- Systems ownership: designed and built the show-control architecture end to end, unifying input, video, lighting, and audio into one live-operable Unity build
1. Hovercraft Controller
Kate steered the ship live with a joystick, so it had to feel physical and a little unpredictable, never scripted or smooth.- Physics-based, not scripted. Real forces and torques on a Rigidbody give it weight, momentum, and drift.
- Hover is a spring, not a fixed height. It's always correcting against the ground, which is where the constant bounce comes from.
- Idle bob and sway keep it turbulent even at rest, so it never feels dead between moves.
- Self-righting, not rigid. It nudges back toward upright without locking to a fixed orientation, so it stays recoverable but never feels stiff.
- Eased throttle. Stick input ramps in and out instead of applying instantly, smoothing out jitter from live, human control.
- Banking turns. Steering leans the craft into the turn rather than just rotating it flat.
- Scene-aware override. Input zeroes out during one sequence so the same rig could be handed off to a different control scheme (a drone).
Net effect: always a little restless, but predictablee enough for Kate to pilot live, night after night.
After leaving the studio, the ship descends into a vast rocky cavern. Scale was the goal here: the world needed to dwarf the ship, so I built and lit an oversized cavern environment in Unity, including full set dressing, to sell that contrast.
I designed and implemented the entire scene, including lighting, set decoration, and the drone combat sequence.
Kate piloting the spaceship through the caverns
For the drone combat sequence , production wanted the spaceship to feel fragile, I implemented a multi camera setup where each window can be shot at by the drone and bullets deliver physics based impact and fracture the windows, I also worked with DMX lighting to flash the lights in the ship red on every impact . Later in the production after several runs with real audiecne we got feedback that audience wanted to shoot at the drone so I also coded a quick combat system to fire at the drone.
2. The Caverns
After leaving the studio, the ship descends into a vast rocky cavern. Scale was the goal here: the world needed to dwarf the ship, so I built and lit an oversized cavern environment in Unity, including full set dressing, to sell that contrast.I designed and implemented the entire scene, including lighting, set decoration, and the drone combat sequence.
For the drone combat sequence , production wanted the spaceship to feel fragile, I implemented a multi camera setup where each window can be shot at by the drone and bullets deliver physics based impact and fracture the windows, I also worked with DMX lighting to flash the lights in the ship red on every impact . Later in the production after several runs with real audiecne we got feedback that audience wanted to shoot at the drone so I also coded a quick combat system to fire at the drone.
2. Sound, Light and Fire
Beyond the ship and its systems, a big part of the job was making the world around it feel physically present, not just projected.Lighting (QLC+ over DMX): All of the ship's practical lighting ran through QLC+, controlled directly from Unity. Rather than treating lighting as a separate cue system run by hand, I wired scene events (drone impacts, transitions, story beats) to trigger DMX signals in real time, so the physical room reacted in lockstep with what was happening on screen. This is what let a single hit from the drone register as a red flash across the actual space, not just a visual on a window.
Fire (TouchDesigner): The fire effects seen in the pre-recorded video content were built in TouchDesigner rather than composited or shot practically. This gave more control over look and timing during the edit and let the fire read consistently across the different projection windows, matched to the color and pacing of the rest of the footage.
Spatial audio (FMOD): All pre-recorded content was routed through FMOD for spatial audio, so sound tied to specific windows or events felt like it was coming from a location in the room rather than playing back flatly. This mattered a lot for an experience built around three separate viewports, since audio needed to reinforce which window an audience member should be paying attention to at any given moment.
Together, these three systems (lighting, fire, and audio) were what pushed the production from "video playing behind a set" to something that behaved like a single, reactive environment.