Next-Generation Astronaut Training: ESA Partners with Photon Using Meta Quest XR
ESA & Photon Use Meta Quest XR to Advance Astronaut Training
In times when Europe’s return to the Moon is only a matter of time, a new wave of technological innovation to support the astronauts emerges. In a unique collaboration, the European Space Agency (ESA) and Photon Engine joined forces to explore how multi-user XR technology can elevate astronaut training. The result is a powerful demonstration of how Photon’s real-time networking stacks can enhance complex, mission-critical workflows, on Earth and in Space.
The Client
The European Space Agency (ESA) is one of the biggest drivers in space exploration and is conducting many missions across the Solar System. With those missions they are driving scientific and technological progress for its 22 member states.
At the core of Europe’s upcoming lunar efforts is Argonaut, a versatile lander designed to give Europe autonomous and reliable access to the Moon. It will deliver cargo, support robotic and future crewed missions and contribute to NASA’s Artemis program.
To prepare the astronauts for these operations, ESA is expanding its training capabilities with advanced XR technologies. Much of this work takes place at LUNA, the agency’s lunar analogue facility in Cologne, where Moon-like conditions allow realistic equipment testing and mission rehearsal. LUNA provides the perfect environment to experiment with XR-powered training tools such as the collaborative project developed with Photon.
The Challenge
Together, ESA and Photon were focused on a scenario that involved the collaboration between an astronaut working around the Argonaut lander and a remote support engineer back on earth. To set up a training programme for this interaction meant to be aware of unique challenges.
First, the engineer needed a precise understanding of the astronaut’s position relative to the lander.
Second, the Argonaut lander is a large structure, too large to be displayed at full scale in most real-world training environments without breaking immersion. Traditional passthrough XR doesn’t play well when virtual content extends beyond the user’s physical boundaries.
Third, locomotion for the remote support engineer needed to feel natural. The astronaut physically walks around the lander, but the engineer’s environment remains stationary. A traditional teleportation system would disrupt presence and spatial understanding.
Finally, situational awareness needed enhancement through live video streaming, enabling both roles to share visual context in real time.
The Solution
To face and conquer these unique challenges, Photon and ESA developed an XR training sample that addresses each of these constraints through innovative, real-time features.
Photon adopted ArUco markers to achieve fast, reliable determination of both position and orientation. Meta later expanded this approach by incorporating their own QR-code detection technology, enabling QR codes and ArUco markers to function together within a unified spatial calibration pipeline.
After that, Photon created Giant Mode to handle the lander’s massive scale. It’s a feature that lets the remote engineer view the entire scene as a small-scale tabletop model without losing real-time interaction with the astronaut.
To ensure intuitive locomotion, a world-grab movement system was introduced. Instead of teleporting, the engineer can reposition themselves by “grabbing” and moving the environment, maintaining immersion even within a physically limited space.
And to enhance communication, this sample integrates the Photon Video SDK, enabling both participants to start real-time video streams from their Meta Quest cameras whenever the situation requires additional clarity.
Platform / Devices
The project was built in Unity for Meta Quest XR headsets, taking advantage of their passthrough capabilities, tracking APIs, and onboard cameras.
The combination of Photon Fusion, Photon Voice and Photon Video SDKs formed the backbone for real-time multi-user interaction across both local and remote roles.
Use of Photon
Photon provided the networking, sync, and media infrastructure powering the entire multi-user experience. With our comprehensive XR samples and SDKs, Photon enabled:
- Stable, low-latency multi-user interaction between astronaut and engineer
- Real-time sync of positional markers and the 3D lander model
- Custom locomotion and environment-scaling interactions
- High-quality video streaming directly from the headset cameras
Photon’s sample ecosystem allowed ESA to rapidly prototype and iterate, ensuring a smooth integration into the training pipeline.
Europe is entering a new era of lunar exploration, and projects like this show how rapidly training, simulation, and collaboration are evolving. XR has moved beyond an experimental add-on and is now actively serving as a practical tool for mission readiness, engineering support, and operational decision-making.
By combining ESA’s vision with Photon’s real-time networking stack, this prototype offers a glimpse of how astronauts and ground teams may one day work together on the Moon, around the Argonaut lander, or across future off-world environments.
Want to explore how Photon’s real-time technology can power your next XR experience? We’d love to hear from you. Contact us and let’s create something extraordinary.
