Photogrammetry in Production

Film and VFX

Real environments - digitizing sets and locations is now standard practice in modern VFX productions. CG extensions of physically limited sets let a location be virtually enlarged while maintaining perfect visual coherence with the live-action footage. Inaccessible locations can be virtually reconstructed from photographic archives. VFX destruction of real scanned structures produces a level of authenticity impossible to reach otherwise.

Digital twins - facial scanning of actors for performance capture is central to contemporary blockbuster productions. High-resolution facial scans, captured with synchronized multi-camera scan rigs, create digital doubles of striking fidelity. Body scanning with complete costumes lets teams simulate fabric behavior with the actual production costumes.

AAA Games

Open world - modern AAA productions (The Witcher, Red Dead Redemption, Horizon) make extensive use of photogrammetry to populate their vast environments. LiDAR- and camera-equipped drones photograph real terrain to extract topography and textures; libraries of rock, ground, vegetation and prop assets are built from scans.

Architectural visualization - directly benefits from photogrammetry for digitizing existing buildings destined for renovation or extension. Real scanned materials enable precise lighting studies and real-time client presentations under Unity HDRP or Unreal Engine; VR walkthroughs let clients move through their future spaces well before construction begins.

Historical games - history-themed titles (Assassin's Creed, Ghost of Tsushima) leverage photogrammetry to digitize preserved historical sites and museum artifacts, guaranteeing a cultural authenticity artists alone couldn't reach.

Heritage Conservation

Digital archiving - photogrammetry and LiDAR scanning are now standard heritage conservation tools. Monuments threatened by decay, conflict or climate change are digitized at high resolution to build a permanent digital archive. Museum collections - sculptures, ceramics, archaeological objects - are digitized for public online access without handling the originals.

Virtual restoration - digitizing the same monument at different periods enables quantitative analysis of temporal degradation. Virtually reconstructing original states - before erosion, before vandalism - becomes possible with digitized historical documents.

Virtual Production

Virtual production on LED walls - popularized by The Mandalorian and since adopted by many productions - relies almost entirely on environments digitized via photogrammetry or LiDAR. Actors perform in front of walls of LED screens displaying real-time environments rendered in Unreal Engine, with a camera-tracking system adapting the environment's perspective in real time.

The Standard Production Pipeline

A complete production pipeline for a photogrammetry asset breaks into six steps: Capture (photos or LiDAR scan on location or in studio), Process (generating the initial point cloud and mesh), Houdini (mesh cleanup, retopology, LOD generation), Texturing (baking PBR textures at 4K-8K), Unity (HDRP import, material and LOD Group setup, scene integration), and Final (lighting, compositing, render or export).

Capture Best Practices

Photogrammetry

• Overlap - maintain 60-80% overlap between all images; insufficient overlap produces holes in the reconstruction.
• Lighting - favor diffuse, even lighting - overcast sky outdoors, softboxes in studio. Hard shadows create irreparable geometric artifacts.
• Focus - ensure every area of the subject is sharp; inconsistent focus is one of the most frequent causes of reconstruction failure.
• Stability - use a tripod or a very steady hand-hold to avoid motion blur.
• Coverage - vary shooting angles across multiple planes (low, medium, high) to capture geometry in every dimension.

LiDAR Scanning

• Multiple positions - place the scanner at several complementary positions to ensure full coverage without shadow zones.
• Registration - plan for at least 30% overlap between adjacent scans to allow automatic alignment.
• Targets - use reference spheres or planar markers to improve alignment accuracy between positions.
• Resolution - calibrate scan resolution to the final use; maximum resolution isn't always necessary and generates needlessly large files.
• Weather conditions - for outdoor scans, avoid rain, fog and extreme heat.

Production Optimization

Mesh - an effective mesh optimization workflow relies on clean quad-flow retopology respecting the geometry's natural flow lines. Automatic LOD generation should be systematic for any asset destined for real-time use. Intelligent decimation via PolyReduce should target a polygon budget set by the technical director based on the target platform's constraints.

Textures - texture resolution should match the asset's apparent size in the scene: 4K for a foreground prop, 2K for medium-distance assets, 1K or less for distant assets. Automatic mipmap generation is non-negotiable.

Performance - occlusion culling eliminates rendering of non-visible objects. Draw-call batching - GPU Instancing for repeated assets - is particularly effective in scanned environments where the same rocks, slabs and props get instanced hundreds of times. Regular profiling with the Unity Profiler and Frame Debugger identifies and fixes bottlenecks before they become production-stage problems.

Essential Pipeline Tools

Where This Is Heading

AI and machine learning are already transforming automatic reconstruction: neural networks improve reconstruction quality from incomplete captures, reduce noise, and fill in missing areas through inference - real-time smartphone reconstruction is becoming realistic. The democratization of mobile capture via Apple smartphone LiDAR and dedicated apps (Polycam, Scaniverse, RealityScan) puts decent-quality photogrammetry in the pocket of any artist or field technician.

Neural Radiance Fields (NeRF) may represent the deepest revolution yet: this technology encodes a three-dimensional scene into a neural network trained on a set of photos, generating new views of the scene from any viewpoint with remarkably photoreal quality.

Photogrammetry and 3D scanning have fundamentally transformed the standards of realism achievable in VFX and real-time production. By capturing the exact geometry and appearance of the physical world, these technologies erode the line between real and virtual - provided the full pipeline is mastered, from field capture to the final optimized asset. Beyond the tools themselves, what matters is pipeline thinking: the ability to choose the right technique for each production constraint, anticipate problems upstream, and maintain a procedural organization that allows fast iteration.

Resources

Software

• RealityCapture: capturingreality.com
• Agisoft Metashape: agisoft.com
• CloudCompare: cloudcompare.org
• Meshroom (AliceVision): alicevision.org
• PCX (Keijiro Takahashi): github.com/keijiro/Pcx
• VFXToolbox (Unity): github.com/Unity-Technologies/VFXToolbox

Free Data Sources

For testing PCX or VFX Graph workflows in Unity, a file containing colorized points (.ply or .las) is needed. Three free, royalty-free sources: Sketchfab (filter by Downloadable + Point Cloud - the creator Laslo.S offers massive scans like a 950k-vertex forest, useful for stress-testing VFX Graph performance), Artec 3D (a free catalog of high-precision geometric scans - mechanical parts, the human body, art pieces - from their professional scanners), and LiDAR data banks like Arrival3D (laser scan samples in .las/.e57 specifically for testing post-production workflows) and NSF NEON (educational LiDAR point cloud packages of natural landscapes, requiring Houdini cleanup before baking for Unity).

Learning

• SideFX tutorials: sidefx.com/tutorials
• Unity Learn HDRP: learn.unity.com
• Sketchfab (scanned model library): sketchfab.com
• Polyhaven (royalty-free PBR assets): polyhaven.com
• NeRF / Gaussian Splatting documentation: nerf.studio