In Shift

Creating a rich user experience in VR with motion capture and real-time graphics generation.


The Why

I'm tired of waving at Kinect projects.
As a tech focused designer/creator I feel that it's important to look at technology as a medium for ideas, rather than rely on the gimmick of its "newness". I wanted to dive a little deeper and give people a reason to engage with an interactive, motion capture work beyond simply having the ability to do so.

The How

  • TouchDesigner
  • Python
  • Oculus Rift S
  • Kinect v2 (for motion capture)
  • Arduino (to trigger fans blowing)
  • Kurt Kaminski fluid shader
  • GLSL
  • Real-time 3D graphics
  • UX design based on phenomenology

The What

In Shift resulted in two iterations: a physical room-scale version, and an VR version. In both cases, participants must move their bodies until they match the posture left by the previous participant. Once achieved, they experience their movements translated into an abstract liquid/smoke in real-time. The session gets recorded, and when it ends, their last position gets captured by the installation for the next person to match. Since each participant picks up where the last left off, each of their recordings links together in a seamless manner, creating a dynamically growing video chain.

Some of the Results




Along with providing this movement version of the exquisite corpse practice, the installation provides some insight into my practice as a dancer embodying physical states. I often use image based physical states to find new ways of moving (as an example: imagining myself as steam, and then moving as steam). In this case, In Shift provides immersive feedback that responds to the movements of the participant, giving them real-time visual representation of their transformation.

Video Documentation of Physical Installation

Participant: Andrew McCormack. Video shot by Francesca Chudnoff, and edited by Luke Garwood

Images of physical set up


Photographs by Francesca Chudnoff



Screengrabs of projected content




Screengrabs from VR project



Install at dance: made in Canada|fait au Canada 2019


Photographs by Irvin Chow and Marlowe Porter