Production
Production technologies in immersive arts and sciences enable the creation, capture, and rendering of immersive content at scale. The category extends from real-time engines through volumetric capture and spatial audio to the codec layer that delivers immersive content to devices.
Real-Time Production
Real-time production renders digital content interactively rather than offline. Real-time rendering is what makes VR and MR experiences possible; the rendering pipeline must complete a new frame for each eye at frame rates the visual system reads as natural. Real-time engines (Unity, Unreal Engine, Godot) provide the framework inside which real-time production happens.
Virtual Production
Virtual production merges physical sets with computer-generated environments using real-time rendering during the filming process. The technique was popularized by the Mandalorian production team's use of Industrial Light & Magic's StageCraft system in 2019 and has since spread across film, television, news (BBC and other broadcasters use virtual studios with LED-wall integration), sports (live broadcast augmented with real-time graphics), and education and corporate communication.
In-Camera VFX
In-Camera VFX (ICVFX) uses LED walls to display virtual environments behind live action, so that the camera captures the integrated scene in-frame without compositing in post-production. The technique requires camera tracking, real-time rendering, color and lighting calibration across the physical-and-virtual boundary, and high-resolution LED display walls capable of displaying production-grade imagery.
Virtual Set Extensions
Virtual set extensions add digital elements to physical sets to expand the captured environment beyond what can be built physically. The technique is the natural complement to ICVFX and ranges from set-extension shots in film and television to live-event coverage where the broadcast augments the physical venue with real-time graphics.
Immersive Video Formats
360 Video
360 video captures the complete spherical surroundings of the camera using omnidirectional rigs. Participants view 360 video in head-worn devices or through smartphone and tablet apps that respond to device orientation. 360 video is widely used for VR documentary, travel, sports, and event capture where the participant's freedom is to look around rather than to move.
VR180
VR180 captures 180 degrees of the scene in front of the camera in stereoscopic 3D. The format simplifies production (one direction rather than two) and supports higher per-pixel quality at the same bandwidth as 360. VR180 is supported on YouTube VR and a range of other platforms.
Stereoscopic 3D Video
Stereoscopic 3D video uses two cameras (or a stereo capture rig) to mimic human binocular vision and produce a sense of depth on playback through head-worn devices or 3D-capable displays. Stereoscopic capture is the basis of much VR180 production and increasingly of spatial-video capture on consumer devices.
Volumetric Video
Volumetric video renders photo-realistic three-dimensional captures of people and objects that can be viewed from any angle. Volumetric capture stages (Microsoft Mixed Reality Capture Studios, Metastage, 4DViews, Mantis Vision, and a growing field of specialized studios) use dozens to hundreds of synchronized cameras around the subject to produce the source data; the processed output is a 3D model that can be placed into any immersive environment.
Holographic Video
Holographic video, in the strict technical sense, captures light-field information that can be reconstructed as true 3D imagery without head-worn devices. True holographic video remains a research frontier; consumer-grade applications labeled "holographic" typically combine stereoscopic display, light-field approximation, or projection-based illusion techniques. Light-field capture and display companies (including Looking Glass and Light Field Lab) continue to push the category toward broader availability.
Spatial Video Codecs
The spatial-video codec layer matured significantly with the Apple Vision Pro launch. Apple uses MV-HEVC (a multiview extension of HEVC) for spatial video capture on iPhone 15 Pro and later, with stereoscopic delivery at consumer scale for the first time. ProRes RAW supports professional spatial-video production. The MPEG-I standard set (MPEG's immersive successor family) defines codecs and formats for volumetric video, 6DoF media, and the next-generation immersive delivery layer; deployment is progressing through standards work and reference implementations.
glTF Volumetric Extensions
glTF has historically supported static and animated 3D assets at delivery scale. Extensions for volumetric content (point clouds, Gaussian splats, volumetric video, and the metadata that supports playback and synchronization) are in active development within the Khronos Group's working groups. The extensions matter for the medium because glTF's wide runtime support makes it the natural delivery format for volumetric content that needs to play across many devices.
Capture Techniques
Motion Capture (MoCap)
Motion capture records the movement of people and objects using markers, optical systems, inertial sensors, or markerless computer-vision approaches. MoCap supports animation in film, games, and immersive content, sports analysis, biomechanics research, and the avatar embodiment that powers shared social-VR experiences.
Photogrammetry
Photogrammetry uses multiple photographs of an object or scene from different angles to reconstruct a precise 3D model. The technique supports cultural-heritage capture, archaeology, architecture and construction, game-asset capture, and the immersive applications where high-fidelity capture of real places and objects matters. Production tools include Agisoft Metashape, RealityCapture (acquired by Epic Games in 2021 and now bundled with Unreal Engine), and a range of mobile and web-based photogrammetry applications.
Volumetric Capture
Volumetric capture uses arrays of synchronized cameras around a subject to produce a complete 3D model that can be viewed from any angle. Volumetric capture supports immersive storytelling, sports and live-event capture, performing-arts documentation, and the increasingly common use of volumetric subjects placed inside immersive environments. The category has grown to support both studio-based capture and increasingly mobile and on-location capture.
Spatial Audio Capture
Spatial audio capture records sound with directional and positional information so that playback reproduces the auditory geometry of the original space. The category combines several techniques:
Binaural Audio
Binaural audio uses two microphones placed to approximate human ear positioning, capturing audio as it is heard in the environment. Played back through headphones, binaural recordings produce a strong sense of three-dimensional sound. Binaural is widely used in VR, immersive podcasting, and audio storytelling.
Ambisonics
Ambisonics captures the full sphere of sound around the recording position and supports flexible playback orientation. Ambisonics is widely used in VR, AR, and immersive video where the participant's head orientation changes the audio they hear.
Object-Based Audio Formats
Object-based audio formats treat audio elements as discrete objects with position, motion, and properties that the playback system places into the listener's environment. Dolby Atmos and MPEG-H Audio are the most widely deployed object-based formats. Object-based audio supports immersive content where the listener's position and orientation determine the audio they hear.
Wave Field Synthesis
Wave Field Synthesis (WFS) uses arrays of speakers to reproduce sound wavefronts that approximate the original sound sources' positions. WFS is used in specialized installations (immersive venues, high-end audio research, planetariums) where the listener's position should not affect the apparent location of sound sources.
Gaussian Splatting Capture
Gaussian splatting capture, the production application of the rendering technique described in the Generative AI section, uses image and video input to produce 3D Gaussian-splat representations of scenes that can be rendered at high frame rates on consumer hardware. Through 2024 and 2025 the technique matured into a production-ready capture-and-rendering pipeline that has substantially eclipsed NeRFs for most immersive use cases that require real-time playback.