Client: *SCAPE Singapore
Project dates: Sep 2022 – Jan 2023
Services: consultancy, mentorship, project management, mixed reality

OVERVIEW

Exploring the relationship between us and the spaces we occupy, with technology

As part of the Singapore Art Week in early January 2023, we were glad to work with two residents of the *SCAPE Dance Residency, which focused on experimenting and performing with technology for the first time and culminated in my_space, a showcase over 6-7 January.

The Tusitala team worked with two choreography residents, Yenny Jenny and Jozef Chua, to explore ideas of space and the relationships between people and the spaces we occupy by combining mixed reality and dance.

Jenny’s project used tech to invite reflection on the experiences of people with hearing disabilities, while Jozef explored questions about how digital spaces and AI systems shape us, through the use of audience input in designing a living room. 

PROCESS

The collaboration began four months ago with an introductory HoloLens workshop at *SCAPE that was jointly led by Tusitala and HelloHolo, for dancers and visual artists keen on interdisciplinary work.  

Two dance residency participants later submitted proposals for dance projects incorporating mixed reality, and we then began working together to assist them in developing their ideas further – especially in terms of integrating the HoloLens into each resident’s concepts and refining the overall user experience.

Jenny and Jozef were challenged to think about different aspects, such as the use of sound, user experience, how mixed reality fits with the physical ‘set’ for the performance, and interactions with the real world.

Integrating the HoloLens into dance

Tusitala’s own experience with creating a HoloLens storytelling experience enabled us to guide both residents through the process of understanding the unique capabilities of HoloLens and considering the role it can play. What could the technology enables that cannot otherwise be achieved?

During this process, our tech partner HelloHolo built custom apps for the participants to test out and prototype their envisioned performances. Together, we worked on the user flow and visuals assets for each project, with several rehearsals held to refine the user experience before the final showcase.

Key takeaways

For Tusitala, this was our first time working with movement, which we found intriguing because HoloLens works with a digital layer within physical space. It was also interesting to learn how a choreographed performance is developed and see how people approach creative work differently. Even for HelloHolo, it was gratifying to see tech used in new innovative and unprecedented ways. If we had more time, our team would have loved to explore other possibilities discussed, too – such as having the HoloLens recognise sign language gestures!

#1 – Importance of aligning to a concept and finding the central motif
Project execution then follows more easily, and decisions are easier to make when guided by a clear focus. We helped Jenny and Jozef explore what role mixed reality plays in their projects, to simplify and clarify each project’s direction.

#2 – Importance of clear written and visual documentation
Having to commit the plans to a cue sheet with different functions (e.g. dancers, audience, HoloLens, audio, set) helped ensure that the showcase would run smoothly.

#3 – Importance of user experience design
From start to finish, it was crucial to plan for both audience member(s) wearing the HoloLens and those who are not, taking into account the wearer’s ability to roam freely in the space. 

Moving Density by Yenny Jenny		Homepage by Jozef Chua
Hello, can you read me? Please give me some time. Don’t leave me behind As society returns to life before covid. Have we taken the time to look back and ponder whom we have left behind? Have we exercised empathy for unique individuals in our society?		Homepage is a work that encourages reflection of our own online activity patterns, in the way we shape our digital spaces through our virtual presences. Relying on audience input, Homepage bears an uncanny resemblance to the algorithms and computer learning systems that refresh our feeds, suggest searches and configures to our preferences. Homepage tailors each experience to the participant’s individual likes and dislikes, yet also streamlines it into the data patterns of the majority. I hope this experimental performance tingles at the back of viewers’ minds as they design their virtual ‘lived spaces’ , and ponder about how much our software know about us. Do these digital systems influence our patterns as much as we influence theirs?
Finally seeing the piece coming together was rewarding for Jenny, whose project combined storytelling, dance, sound, and videos of phrases in sign language (seen through the HoloLens), altogether designed to create a greater awareness of and empathy for the experiences of people with hearing impairments.		One major challenge was the short time frame we had to work within: in terms of technical design and programming, much of the software that would be ideal for this project was beta or would have to be designed from scratch. Jozef had to manage his expectations accordingly and find compromises that still made sense for his concept. Integrating ‘AI’ (in this case, simulated by audience input) into art-making practices also meant relinquishing an extent of control over the final product.

The showcase:

PROJECT TEAM

• Organised by *SCAPE Singapore
  
• Project Mentor – Tusitala
• Mixed Reality Mentor and Developer – HelloHolo
• Dance Residency Artistes – Yenny Jenny, Jozef Chua