Photogrammetry in Unity HDRP

Why Unity for Scanned Assets?

Unity in HDRP mode has established itself as a platform of choice for integrating photogrammetry assets. Real-time feedback on lighting and composition allows much faster creative iteration than traditional offline render pipelines. HDRP's render quality - PBR, raytracing, cinematic post-processing - gets close enough to photorealism to be used in a VFX production context, notably for virtual production on LED walls.

Importing Scanned Assets

Supported formats: FBX (the production reference, supporting hierarchies and LODs), OBJ (simple and universal with its associated MTL textures), Alembic (for animation and simulation caches from Houdini), and USD (Pixar's format, increasingly adopted in production pipelines).

A few import settings deserve particular attention for photogrammetry assets: the Scale Factor must be adjusted to match the object's real-world scale in meters, and Generate Colliders automatically creates colliders from the imported geometry.

PBR map correspondence between photogrammetry output and Unity HDRP's workflow:

LOD Setup in Unity

The LOD Group component manages transitions between detail levels based on the object's apparent screen size: LOD 0 is active from 100% screen height (foreground, very close), LOD 1 from 50-100% (medium distance), LOD 2 from 25-50% (far distance), and below 5% the object is fully culled.

Complete import workflow:

1. Houdini export - generating a multi-LOD FBX file with associated textures.
2. Unity import - drag-and-drop into the Project panel, verifying import settings.
3. Materials - creating HDRP/Lit materials and assigning PBR textures.
4. LOD Group - configuring the component and transition distances.
5. Scene - placement in the scene, lighting setup and composition adjustments.

HDRP/Lit Material Configuration

The HDRP/Lit shader is the main surface shader for opaque assets in Unity HDRP. Base properties: Surface Type set to Opaque in the vast majority of cases for scanned assets; Base Map the photogrammetry Albedo texture encoding diffuse color; Metallic at 0 for organic materials (stone, wood, concrete), 1 for metallic materials; Smoothness inversely proportional to the baked Roughness map (1 − roughness).

Detail maps: a Normal Map encoding micro-geometry detail not represented in mesh topology; a Detail Normal - a second, high-frequency tiling normal map adding surface granularity; Ambient Occlusion from baked cavity and contact-shadow information; and an optional Height Map for parallax-simulated surface depth.

Texture Optimization

Compression format choice determines the balance between visual quality and runtime memory footprint:

Automatic mipmap generation (Generate Mip Maps ON) is essential to avoid moiré on distant surfaces; the Kaiser filter gives the best mipmap quality for photogrammetry textures. For scenes with many scanned objects, atlasing - grouping several objects' textures into a single texture - significantly reduces draw calls and improves performance; Trim Sheets build a reusable texture atlas that multiple assets can share.

Specialized Materials

Terrain and Landscape - topographic scans can be integrated into Unity's Terrain system by importing a Heightmap. Splat Maps blend multiple material layers (rock, earth, vegetation, asphalt) on the terrain based on slope, altitude, or hand-painted masks.

Vegetation - the HDRP/Nature shader is specifically built for scanned vegetation assets, supporting translucency (light passing through leaf blades), subsurface scattering for stems and buds, and built-in wind animation.

Delighting and Lighting

One of photogrammetry's major challenges is that the scene's natural lighting gets baked directly into the texture during capture. Unity provides the De-Lighting Tool, an official utility designed to extract that baked light so assets can react physically to dynamic lighting in your scene.

Lighting a scene with photogrammetry assets requires a carefully calibrated approach to preserve surface photorealism: Baked GI (precomputed lightmaps for static objects give the best quality for architecture and open-world environments), Reflection Probes (local environment captures, essential for reflective surfaces - metal, glass, water), and Light Probes (GI samples for dynamic objects, keeping moving characters and props lighting-consistent).

Rendering Point Clouds Directly

Three main methods exist for importing and rendering a photogrammetric point cloud directly in Unity.

Method 1 - PCX Open-Source Package

The most widely used free solution. Keijiro Takahashi's PCX package allows direct import with optimized shaders (Unlit or Lit). Required format: .ply (Stanford format) - a raw .las, .laz or .e57 cloud must first be cleaned and converted via CloudCompare or Houdini. Workflow: add the PCX GitHub repository via Unity's Package Manager; dragging a .ply file into Assets automatically generates a PointCloudData container ready to place in the scene.

Method 2 - Advanced Rendering via VFX Graph

To fully exploit the GPU, handle tens of millions of points without performance loss, or apply artistic manipulation, the Visual Effect Graph is ideal. VFX Graph doesn't read photogrammetry files directly - the data (XYZ and colors) must first be converted into a Point Cache or RGBA Float textures. The advantage: each point is treated as a particle managed by a Compute Shader, letting you dynamically change display size or apply mathematical forces. Houdini is an excellent candidate for preparing and baking this kind of textured data before export.

Method 3 - Unity Industry Toolkit (formerly Pixyz)

For massive, industrial-scale datasets with a Unity Industry license: supported formats include raw scan formats like .e57, .rcp, .pts and .ptx. The tool natively handles decimation and automatic LOD generation on import - importing more than 100 million points requires a minimum of 32GB system RAM.

Houdini to VFX Graph via Point Cache

The most performant approach relies on Point Caches (.pcache), converting point data into GPU-optimized textures.

1. Preparation in Houdini - install Unity's official Houdini plugin (Unity-Technologies/VFXToolbox GitHub repo) and its HDA. Import the raw scan and clean attributes down to just Position (P) and Color (Cd). Convert axes: Houdini is right-handed (Y up, Z forward), Unity is left-handed - invert the Z axis (@P.z = -@P.z;) via an Attribute Wrangle.
2. Exporting the Point Cache - connect the cleaned cloud to the VFX Point Cache Exporter node (Unity's HDA), mapping Houdini's P attribute to position and Cd to color for Unity, then click Save to Disk to generate the .pcache file.
3. Configuring Unity VFX Graph - drag the .pcache file into your Unity Assets. In the Initialize Particle block, add Set Position from Map and Set Color from Map blocks, connecting your .pcache asset to them. In the Output block, choose Output Particle Quad to visualize the cloud.

Physics and Collision for Imported Scans

For a 3D scan to integrate with Unity's physics engine, manually assign a Mesh Collider and a Rigidbody component. The Mesh Collider generates collision geometry faithful to the scan's shape, while the Rigidbody exposes the object to the scene's physical forces.