Fun outfits, cool exhibits, and beneath the layers... a secret society experiencing some pesky ghost trouble.

RAVE IN PEACE was an immersive experience created by puzzle designers Jesse Gazic and Bryan Weiser. Their company Lockwood Immersive was contracted by Toronto's Hand Eye Society to build a puzzle woven into the ball itself. A ghost hunt that ended up being the talk of the night.

The boys contracted me to help design and execute a particularly tech-heavy puzzle: a magic mirror that responds to candlelight.

The Project

A magic mirror that gives you the key word for the next puzzle, but ONLY if you provide it candlelight (in the right order)!

The project taught me a lot about image detection models, VFX, and acrylic sheets (of all things). What came out the other side was a pretty badass mirror that I'm proud of sharing here.

Candlelight

Using some cheap LED candles, a webcam, and a bit of Python, I built an image detection model that detected and colour-mapped the candles. Show them to the camera in the right sequence and the puzzle is solved.

I wasn't familiar with CV2 or image detection models at the project conception so this was by far the steepest learning curve. I tried a number of models and techniques but ended up settling with training a simple cascade classifier model but brought along Aruco markers as a fallback (fortunately these were not needed).

 

The detected candle bounds are then colour-averaged and compared against a table of samples configured in the candle configuration JSON file. This determines the "winner" and which colour will be shown in the next sequence.

The candles turned out to be the most arduous process of the whole deal. But I'm thrilled with how it turned out and grateful for the learning experience.

Mirror

A TV donated from our lovely friend Mia. We built a frame around it and overlaid a one-way acrylic sheet so only screen light leaks through.

I handled the VFX in Godot because of its jam-friendly nature and because every second I spend in Godot is bliss.

 

The project composition is small. Some basic port hooks receive messages from the Python backend and transmit signals based on the result. Fireballs are instantiated based on the specific candle that was lit, and a final effect plays whether the sequence is correct or incorrect.

I used FMOD for audio (though this was absolutely not necessary - you could easily get away with Godot's built-in audio system). I followed this guide to get the fireballs just right, though I translated the effect from Unity VFX graph to Godot's GPU Particle System.

Summary

 

 

I'm very proud of the project and am glad I finally took the time to brag about it here. If you live in Toronto and want to keep up with one of the best immersive experience companies in the biz, you should check out Lockwood Immersive.

If you're here for the project files, I have the Git repos for both the Python backend and the Godot frontend below.

Big thanks to Jesse and Bryan at Lockwood Immersive for giving me the opportunity to work on this project. I'm so excited for what we have coming next!