The project

Immerse is Augmented Reality app for children and young people in the first year following diagnosis with epilepsy tailored to each individual to enhance patient and carer understanding of epilepsy, provide safety advice for greater confidence in managing epilepsy at home, and enable greater communication with clinicians to improve patient outcomes.

The technology

The app was built using the Flutter framework and Unity 3D's unity as a library functionality. Within the app we embeded a 3D hero component that introduces a customisable 3D character. This character guides the user through the material on the app and and swith to an AR experience to explore 3D content in the real world.

To support multiple characters we explore using AI to create consistently themed characters in 2D which could then be converted into 3D models to render in 3D.

The character voice was additionally created using AI voice generation, which were in turn was fed the scripts to drive the characters face animations.

We additionally explored how to leverage realtime face tracking to animated the characters face with our own. This same technology stack was used to capture full body motion to animate some of the scenes.

My work

I've worked across multiple disciplines, working on the design, ux and code to deliver the initial Proof of concept. In Figma i've designed the look and feel, addressed a UX conundrum with having an ever presence avatar across screens and seemless switching between AR and on screen experiences.

I additionally retopoligised the 3D generated character mesh to match the required face structure for lip syncing.