In this experience you use BITalino biosensors and the Oculus Rift to immerse yourself in a strange and wonderful world. The visuals are generated by using ray marching to render the amazing surface fractal. The experience consists of multiple stages each with a unique look. In each stage the biosensors can be used by the player to control some of the visual aspects. While traversing the space a music track is playing that seems to react to changes in the visuals. This is done by syncing the phase transitions to changes in musical tone.

PROJECT INFORMATION

Date: 4th Semester (2017)
Time limit: 4 weeks
Team size: 3 People
Engine: Unity 3D
Constraint: Use the Oculus Rift combined with a BITalino biosensor-kit, to immerse the player in a “levitation” experience. The creation process is confined to a three week time frame.

My Part: I created an implementation of the ray marching algorithm for Unity 3D with the ability to render stereoscopic vision output suitable for the Oculus Rift. Furthermore I learned how the ray marching algorithm can be used to render 3D fractals and wrote a custom tool that allows you to to change the visual appearance of the fractal by manipulating variables exposed by the ray marching shader. This tool can be used to define what parameters are controlled via the biosensors and also allows to predefined animations. I also provided some improvised keyboard play for the soundtrack.

What I learned: Working on this project showed me that you can get very interesting results when you break conventions. You can use the ray marching algorithm to visualize the distance field of a fractal to get stunning results, that are impossible to achieve with a conventional rendering method. I learned a lot about lighting, different rendering pipelines and shader programming in general. I also could deepen my knowledge in how to effectively design and implement useful development tools.