Volumetric video (VV) is an emerging 3D format that allows the integration of real people into XR (extended reality) applications, enhancing immersive experiences across various fields. Traditionally, the creation of VVs requires dedicated studios that include many RGB cameras, depth sensors, lights, green screens, etc.
In this project, together with our partners from Aargauer Kunsthaus and Freisicht, we showcase an affordable workflow for creating VVs using a single smart phone and integrating them in an AR application as part of a museum experience.
Content Creation
VV content creation is now possible from a single video by AI-based methods such as the service provided by Volograms. We used a single smart phone (iPhone 12) to record the video. After post-processing, the video was sent to the Volograms cloud service for reconstruction. The final assets, including dynamic meshes and associated video textures, were delivered in a specific VV format, which can be integrated in game engines such as Unity.
The curator and artists were recorded on location in the museum and in one of their own studios. This flexibility to capture anywhere is a big advantage over classical VV studios
Experience Design
We designed our prototype experience with 3 VV-AR stations, a curator, a solo artist, and a team of two artists. The AR application creates a simulation of meeting real people as 3D holograms in the real 3D space and enhances the museum experience by establishing a more personal connection to the artists, artwork, location, and exhibition.
Results
The AR application with VV was received positively by many visitors. It enhanced the museum experience, and the possibility to meet the artists, albeit only virtually, created the sense of personal connection to the artworks and artists.
For further details, check out our technical report and poster at VRST 2024.
Project Data
| Project Type | Research |
| Duration | 01.06.2021 – 31.12.2024 |
| Project Partner | Aargauer Kunsthaus Freisicht |
| Funding | Innosuisse, EU-HORIZON (TRANSMIXR) |
| Project Team | Anh Nguyen, Ariana Huwiler, Philipp Haslbauer, Lam Kit Yung, Gareth W. Young, Jan Lässig, Anna Kälin, Thomas Stettler, Aljosa Smolic |