Gaussian Splatting in Unreal Engine 5: The Capture-to-Game Pipeline in 2026

3D Gaussian Splatting went from a SIGGRAPH 2023 research paper to something shipped in production games in under three years. That is an unusually short runway for a rendering primitive, and it happened because the technique solves a specific problem that photogrammetry and Nanite both struggle with: capturing real-world spaces with view-dependent appearance, semi-transparent structures, and complex micro-geometry without spending weeks on mesh cleanup.

UE5.7 now has three viable paths for getting a splat into the engine — the XVerse plugin, Luma AI's official UE integration, and Epic's experimental native GS module gated behind a console variable. The workflow is stable enough for environment art teams to rely on it, but the performance characteristics and limitations are still misunderstood. This post walks through the full pipeline we use internally, with the numbers we measured on a Scene II benchmark project (a 60m × 60m exterior with 14 captured splat regions blended into a traditional Nanite/Lumen scene).

We'll cover capture, conversion, authoring in UE5.7, runtime cost, the hybrid-scene pattern we recommend, and the limitations you need to know about before you commit a production asset to splat form.

What Gaussian Splatting Actually Is

Before the pipeline details, a short refresher, because a lot of the authoring decisions downstream only make sense if you understand the primitive.

A 3D Gaussian Splat is an unstructured cloud of anisotropic 3D Gaussians. Each Gaussian has a position, a 3×3 covariance matrix (stored as a scale + rotation quaternion), an opacity, and a set of spherical harmonics coefficients that encode view-dependent color. Rendering works by projecting each Gaussian into screen space as a 2D ellipse, sorting front-to-back within tiles, and alpha-blending.

Key consequences:

• No mesh, no UVs, no shading network. A splat is its own shading model. You do not apply PBR materials to it in the normal sense.
• View-dependent appearance is baked in. Specular highlights, subsurface, wet surfaces, glass — the spherical harmonics capture the observed appearance from any angle within the training view set. This is why splats look so good for real-world captures and so mediocre for things outside the captured view hemisphere.
• Point count, not triangle count, is your budget. A reasonable production splat is 1.5M–6M Gaussians. Rendering cost scales with splat count and screen coverage, because each on-screen splat participates in the tile sort.
• Lighting is static. The training captures lighting conditions. You can not relight a splat with dynamic lights in any physically meaningful way. You can tint, modulate opacity, apply post effects — but the baked radiance is what it is.

That last point is the most important authoring constraint. Everything else in the pipeline follows from it.

Capture: Moving Past Photogrammetry

The capture phase is where splat quality is won or lost. The good news is the capture requirements are strictly more forgiving than photogrammetry. You do not need controlled lighting, polarized cross-filters, or a turntable. You do need discipline about view coverage.

Hardware

Anything from a modern smartphone up to a cinema camera works. We've shipped splats captured on:

• iPhone 15 Pro / 16 Pro. Polycam, Luma, and Scaniverse all produce usable training sets directly on-device. Fast turnaround, lower resolution ceiling.
• Insta360 X4 / X5. Equirectangular video at 8K, decomposed into perspective frames. Excellent for interior captures where a 360 rig moves continuously through the space.
• Sony A7 IV or similar mirrorless. Highest quality ceiling, most post-processing flexibility. Required for print-quality or cinematic splat assets.
• Drones (Mavic 3 Pro / Mini 4 Pro). For exterior rooftops, facades, and terrain splats that you cannot capture from ground level.

Capture Patterns

Three patterns cover most environments:

Orbital — subject in the center, camera orbits at two or three heights. Best for hero props, statues, and isolated objects. Aim for roughly 150 frames across two or three rings. Overlap between adjacent frames should be 70–80%.

Corridor — camera moves forward through a space while sweeping left and right. Best for interiors and linear exteriors (alleyways, corridors, tunnels). Keep the sweep amplitude consistent and plan for horizontal figure-eights at waypoints to capture side geometry.

Dome — camera orbits at increasing height bands, covering a hemisphere above the subject. Best for large exterior monuments and terrain features. Typically requires a drone or a boom.

The single most common capture mistake is under-covering the down angle. If you never photograph looking down onto the top of an object, the spherical harmonics have no data for those directions and the splat looks correct at eye level but falls apart when a player crouches under something or a cinematic camera dips low.

Lighting and Motion

• Capture at a consistent time of day. Splats bake the lighting, so a capture that spans two hours in golden hour will have warm-to-cool gradient artifacts.
• Avoid overcast-to-sun transitions. Consistent overcast is ideal for most game environments because it maximizes texture detail and minimizes specular discontinuities.
• Remove moving elements (cars, people, swaying vegetation) from the capture if possible. Moving objects produce floating "ghost" Gaussians that the training pass cannot resolve.
• Pedestrians and vehicles passing through a single frame are fine; the training pass treats them as outliers and they get assigned near-zero opacity.

Conversion: From Capture to .ply

The conversion pipeline has three stages: structure-from-motion (SfM), Gaussian initialization, and training.

For SfM, the industry has settled on Colmap (open source, CPU-bound, slow) or Glomap (open source, much faster, marginally less accurate on pathological captures). Commercial alternatives like Reality Capture's SfM-only mode exist but are rarely worth the license cost if you are not also doing photogrammetric mesh output.

For training, the two production tools are:

• gsplat (nerfstudio) — open-source, Python, CUDA. Excellent research tooling, slower training.
• 3DGS-Studio / gsplat-fast — optimized CUDA kernels, 2–4× faster training on the same hardware.

Training a 2M-Gaussian scene from a 200-frame capture on an RTX 4090 takes about 25–40 minutes with gsplat-fast, or 90–120 minutes with the reference gsplat implementation. VRAM is the binding constraint — expect to need 24GB for scenes over 3M Gaussians at training resolution.

The output is a .ply file (or the compact .splat variant, which we recommend for engine import). A typical production splat is 80–400MB on disk depending on Gaussian count and spherical harmonics order.

Spherical Harmonics Order

This is a decision you have to make explicitly. SH order controls how much view-dependent detail each Gaussian can encode:

• SH 0 — no view-dependence, just a color. 3 floats per Gaussian. Use for matte/diffuse subjects where appearance does not change with angle. Smallest file size.
• SH 1 — 12 floats per Gaussian. Captures gentle directional shading. A reasonable compromise for diffuse scenes.
• SH 2 — 27 floats per Gaussian. Captures specular lobes and soft reflections. Default for most production captures.
• SH 3 — 48 floats per Gaussian. Captures sharp reflections and subsurface. Required for wet surfaces, polished materials, foliage.

SH 3 quadruples your storage and VRAM cost versus SH 0. We default to SH 2 for environments and only escalate to SH 3 for specific hero assets that require it.

Importing into UE5.7

As of UE5.7, three paths work reliably.

Path 1: XVerse Gaussian Splatting Plugin

Third-party, Marketplace-distributed, the most mature option. Import a .ply or .splat via the Content Browser. The plugin creates a UGaussianSplatAsset with its own actor type and a custom renderer that runs before translucency.

Pros:

• Mature, stable, used in shipped titles.
• Supports splat-to-mesh occlusion culling.
• Editor viewport renders splats natively, no external preview required.

Cons:

• Closed source, licensing fee for commercial projects.
• Custom render pass sits outside Lumen / post-process volumes in non-trivial ways.

Path 2: Luma AI UE Integration

Luma's official UE integration is free, open-ish, and integrates with their capture app pipeline. Best if you are already capturing through the Luma mobile app.

Pros:

• Zero-friction capture-to-engine for Luma users.
• Free for most use cases.

Cons:

• Tied to Luma's capture ecosystem.
• Renderer is less optimized than XVerse on DX12.

Path 3: Epic's Experimental Native GS Module

UE5.7 ships an experimental Gaussian Splatting module. It is disabled by default and gated behind r.GaussianSplatting.Experimental 1. The API is unstable and Epic has marked it "not for production" — but it is the most tightly integrated with Lumen, Nanite, and the temporal super-resolution pipeline.

Pros:

• Native depth integration with Nanite meshes.
• Proper TSR integration (splats anti-alias correctly).
• No third-party licensing.

Cons:

• API will change in 5.8+.
• Fewer authoring tools than XVerse.
• Some features (culling, LOD) are missing entirely.

Our recommendation: XVerse for shipping now, native module for evaluation with an eye on 5.8.

Runtime Cost: The Numbers

The question we get asked most is "how expensive is a splat compared to a Nanite mesh?" The honest answer is "it depends on screen coverage and Gaussian density," but here are measured numbers from our Scene II benchmark on an RTX 4070 Ti at 1440p, TSR Balanced:

A few takeaways:

• At low to moderate screen coverage, splats are competitive with high-detail Nanite meshes. They are not free, but they are in the same ballpark.
• Screen coverage matters more than total Gaussian count, because off-screen Gaussians are culled before the tile sort.
• The cost curve is superlinear past ~60% screen coverage due to tile sort contention.
• On a 4060 / 3060-class GPU, divide those budgets by roughly 2.2× and plan accordingly.

Memory: 1M Gaussians at SH 2 is approximately 220MB VRAM including the SH buffer, position buffer, and sort scratch space. A scene with three active splats totaling 6M Gaussians is already a 1.3GB VRAM commitment — non-trivial on 8GB cards.

The Hybrid Scene Pattern

Splats work best as components of a scene, not as a whole scene. The pattern we've standardized on:

• Hero volumetric regions — captured environments where the player spends time and the view-dependent appearance pays off. Ancient ruins, overgrown forests, cluttered markets.
• Backdrops and skyboxes — distant terrain, cityscapes, matte-painting-style horizons. A single 2M splat can replace 30–40MB of skybox textures and look significantly better at middle distances.
• Nanite for gameplay-critical geometry — anything the player touches, collides with, or manipulates. Splats have no collision primitives, so anything with gameplay collision needs a mesh under or alongside it.
• Standard PBR meshes for dynamic objects — characters, props, interactables. These need dynamic lighting, which splats cannot provide.

The blend points between these zones are where production problems happen. A splat that abruptly terminates at a mesh boundary is visually jarring because the shading models do not match. Three mitigations:

1. Depth-based fade. Modulate splat opacity based on distance from the splat's bounding volume. Fade in over 2–4 meters at the boundary.
2. Color-matching pass. Bake a low-resolution cubemap from inside the splat and use it to tint the adjacent Nanite meshes' ambient term.
3. Overlap, don't abut. Let splat regions extend 1–2 meters past where they visually "end" under meshes. The sort handles the blending gracefully.

We use the Unreal MCP Server to batch-configure these fade parameters across dozens of splat actors in a scene — manually tuning each one in the details panel is tedious and error-prone, especially when you re-import an updated capture and the bounds shift slightly.

Authoring Workflow

For environment artists working with splats daily, a few patterns save hours.

Clean-Up in Blender

Before import to UE5.7, open the .ply in a splat-aware DCC. Blender 4.3+ has a community-maintained Gaussian Splatting add-on that renders splats in the viewport and supports box-select editing. The workflow:

1. Load the .ply.
2. Box-select floating Gaussians (the "floaters" produced by under-covered regions) and delete them.
3. Box-select any pedestrians, vehicles, or temporary objects you want gone.
4. Export the cleaned splat as .splat.

This step alone can reduce a raw 3M-Gaussian capture to a clean 1.8M-Gaussian asset with better visual quality, because removing floaters eliminates most of the "haze" artifacts that splats are infamous for. The Blender MCP Server can automate the floater-removal heuristic across batch captures — we have a prompt template that looks for Gaussians with opacity below 0.02 outside the main bounding cluster and removes them, which catches ~90% of floaters without manual cleanup.

LOD and Streaming

Splats don't have natural LODs the way meshes do. Two strategies:

• Gaussian decimation — generate simplified splats at 50%, 25%, 10% Gaussian count. Most trainers can output decimated variants directly during training.
• Impostor fallback — at very far distances, replace the splat with a flat billboard rendered from the splat's dominant view direction. Cheap, and at sufficient distance the difference is invisible.

For streaming, splats load and unload at the asset level. A 200MB splat blocks a frame hard if it is loaded synchronously. Always async-load splats and fade them in once their buffers are uploaded.

Collision

Splats have no intrinsic collision. Three approaches:

1. Simple collision primitive. Hand-author a low-poly collision mesh that approximates the splat. Works for props and small environments.
2. Automated mesh extraction. Tools like Gaussian2Mesh and Sugar extract a proxy mesh from the Gaussian cloud. Quality varies — works well for solid geometry, poorly for foliage.
3. Keep a reference mesh. For captures where you also did photogrammetric meshing, use the (invisible) mesh for collision and the splat for rendering.

Limitations You Cannot Engineer Around

Splats have hard limits. Know them before committing an asset.

• No dynamic lighting. The baked radiance cannot respond to a flashlight, muzzle flash, or time-of-day system. You can fake a vignette or tint, but the specular highlights in the capture are fixed.
• No animation. A splat is a static cloud of Gaussians. You can translate, rotate, and scale the whole asset, but you cannot rig it, deform it, or morph it.
• Transparency is a mess. Semi-transparent splats (glass, foliage edges) sort correctly within themselves but interact badly with engine translucency (water, particles, UI).
• Cast shadows are approximated. Splats do not generate proper shadow maps. UE5.7 has a shadow-proxy approach where a low-resolution voxelized representation casts shadows, but it loses fine detail.
• Reflection probe capture misbehaves. Splats do not render correctly into reflection captures because the capture camera's view direction is outside the spherical harmonics' interpolation range. Bake reflection probes with the splat hidden, or use screen-space reflection only in splat regions.

When to Pick Splats Over Nanite

A decision framework:

Pick Gaussian Splatting when:

• You are capturing a real location.
• The asset has complex microgeometry (foliage, ruins, organic clutter) that photogrammetry-to-mesh destroys.
• The asset does not need dynamic lighting.
• The asset is static in the scene.
• View-dependent appearance matters (wet surfaces, polished materials, subsurface).

Pick Nanite when:

• The asset is authored (not captured).
• Dynamic lighting matters.
• The asset is interactable or deformable.
• You need precise collision.
• The asset needs to cast high-quality shadows.

The split is clean in practice. A ruined temple environment might have a splat-based ground plane and rubble, Nanite-based walls and columns, and standard PBR props for player-interactable items. Each primitive does what it is best at.

Integration With Cinematic Camerawork

Splats pair unusually well with cinematic sequences, and this is where a lot of teams find their first production use case before committing to gameplay-space splats. The reason is straightforward: cinematic cameras have constrained paths, and constrained camera paths stay inside the capture hemisphere that the splat's spherical harmonics were trained on.

A typical pattern we've shipped: a real-world location captured as a 2.5M-Gaussian splat, a handful of Nanite hero props placed into the splat volume for narrative interaction, and a spline-driven camera that moves through the space along a known path. The camera spline is authored so it never dips outside the angular coverage of the capture set, which keeps the spherical harmonics well-fed and the specular behavior stable frame-to-frame.

The Cinematic Spline Tool includes a camera-coverage overlay that visualizes the capture hemisphere for a selected splat actor and flags spline segments that leave it. This is a small quality-of-life feature but it eliminates the most common cinematic-splat bug, which is a camera drift that produces a sudden specular pop on a hero reflective surface.

For action-cinematic hybrids (playable cinematics, interactive cutscenes), the same coverage overlay tells you which camera freedom budget you can give the player without risking visible artifacts. A typical budget for a well-covered capture is a 50-degree yaw freedom and a 20-degree pitch freedom around the designed camera axis. Beyond that, the specular behavior becomes unstable.

Storage and Distribution

A production concern that catches teams off-guard: splats are large on disk and uncompress poorly. A 2M-Gaussian splat is 150–300MB depending on SH order. Multiple splat regions across a large game install can balloon disk footprint quickly — a game with 20 hero splat locations can spend 3–6GB of install size on splats alone.

Two mitigations:

• Vector-quantized splats. Research from 2024–2025 (Compact-3DGS, Lightweight 3DGS) showed that Gaussian parameters can be vector-quantized with minimal quality loss, producing 5–10× smaller files. UE5.7 supports VQ-compressed splats at import; the engine transparently decompresses on load.
• Streaming from chunks. Splats can be partitioned spatially and streamed as the camera approaches. The XVerse plugin and the native experimental module both support chunked splats. Chunk boundaries do introduce blending artifacts, which require the same fade-at-boundary treatment as mesh-splat transitions.

What Ships Next

UE5.8 will likely graduate the native GS module from experimental, add proper integration with Lumen's reflection probes, and expose a first-party LOD system. Epic has also hinted at volumetric cloud and fluid Gaussian representations in research previews — training a Gaussian cloud for animated phenomena is a harder problem, but research in 2025 (Dynamic 3D Gaussians, Deformable 3DGS) has made it tractable for short sequences.

For now: capture real locations that could not otherwise be authored, keep splats for environment and backdrop roles, and leave gameplay geometry on Nanite. The pipeline is production-ready, the runtime cost is understood, and the quality-per-artist-hour ratio is the best it has ever been for photoreal environments.