01 / Meet
A Star Guides You In
Xiaoxing, the little star, welcomes the player in Chinese. The opening teaches a close-range pinch: pinch the star once to practise, then again to begin.
Mixed Reality / Interaction Design / Unity
Heart Isle
A little star, four emotions, and a choice you can reach out and hold.
Heart Isle is a collaborative mixed reality experience designed to aid researchers in understanding children with autism spectrum disorder (ASD). Built for Quest 3, it brings together short stories, a spoken star guide, and physical face selection. Participants move through happiness, fear, sadness, and anger, then identify the requested expression by grabbing a face in the space in front of them.
01 / Player Journey
Learn The Gesture.
Follow The Feeling.
The experience introduces its physical language before asking the player to use it. The same small gesture carries them from meeting the guide to making an emotion choice.
02 / Watch
A Story Sets The Context
Short films establish the island's emotional situations. The story sequence moves from Happy to Scared, Sad, and Angry.
03 / Choose
Reach For An Expression
After the star finishes its prompt, four face objects appear within reach. A physical grab commits the player's answer, followed by spoken feedback.
04 / Return
Complete The Journey
The current prototype collects three selections per phase, then advances to the next story. After the final phase, the player returns to the island for a closing message and results.
02 / Narrative Media
Four Stories.
Four Emotional Contexts.
Four short films introduce the emotional situations participants encounter on the island. Chinese narration and captions carry each story.
01 / Happy · 开心
An Island Full Of Life
The opening film introduces a bright island and its lively animals.
Open film ↗02 / Scared · 害怕
A Threat Arrives
A looming creature and a darkened sky change the island's mood.
Open film ↗03 / Sad · 伤心
Something Has Been Lost
The star looks over the broken statue and tries to put its pieces back together.
Open film ↗04 / Angry · 生气
The Island Pushes Back
The guardian reacts to the creature's destruction of the island.
Open film ↗03 / Interaction Design
Make The Choice Reachable
Each round needs a clear prompt, a readable arrangement, and a deliberate physical answer. The guide and face layout work together to keep those steps distinct.
Teach Through Contact
The star's pinch tutorial introduces the same close-range interaction used later for face selection. Hand and controller grab components connect the player's reach to the virtual object.
Finish Speaking Before Choices Appear
Faces remain hidden during the prompt. Once the speech ends, the row is positioned relative to the player's current view and the response timer begins. Listening time is kept outside the selection measurement.
Vary The Positions
All four expressions appear in each round. A shuffled Latin-square sequence counterbalances their positions across blocks of four layouts. Later prompts change their target emotion without repeating it immediately.
Keep Guidance Nearby
The star accompanies the face-selection row with Chinese speech and persistent subtitles. Correct choices receive confirmation; incorrect choices receive a short, gentle response. Inactivity triggers a hint about the eyebrows, eyes, or mouth.
04 / Observation & Results
Keep The Choice
And Its Timing Together
The results system records the requested emotion, selected expression, correctness, and response time for each choice. It then brings individual choices together into a session summary.
Choice
Every Answer Counts
Correct and incorrect selections both remain in the record, including which alternative was chosen. A phase completes after its configured number of choices.
Timing
Measure The Decision
Timing starts when the faces appear and ends at the grab event, before the star's response. Feedback duration therefore stays outside the recorded selection time.
Review
Read The Full Session
The closing view includes overall accuracy, average response time, and breakdowns by phase and emotion. The Quest export stores a JSON record, and the history view can reopen saved sessions.
Research Focus
The experience is designed to support researchers as they work to understand children with ASD. Stories and physical choices provide a structured interaction, while the results system keeps each selected expression and its response time available for review.
05 / Unity Development
From Interaction Idea
To A Connected Prototype
Heart Isle is a collaboration across game design, UI/UX, Unity engineering, medical research, and experience design. My contribution focuses on game design and Unity engineering.
Progression Across Scenes
The project connects an island introduction, a story scene, and a face-selection scene. A persistent progression manager carries the current phase across transitions and returns the completed journey to the island.
Physical Interaction & XR Setup
Development iterations addressed close-range grabbing, headset-pose readiness after scene changes, and where the faces and star appear relative to the player. Those details determine whether the next action is visible and reachable.
Selection, Speech & Timing
Separating the prompt, visible choices, grab event, and feedback made the interaction easier to reason about. That structure also gives the data collector clear events for starting and stopping each timer.
Quest Results & History
The session collector, results panel, and saved-session browser share the same data structure. Android MediaStore supports the Quest JSON export and history lookup.
06 / Project Credits
The Team
Heart Isle brings together design, engineering, and medical research expertise.
Bowen Shi
Game Designer / Unity Engineer
Galina Zhu
UI/UX Designer / Unity Engineer
Iris Xu
Unity Engineer / Medical Research Specialist / Experience Designer