Displacements in Hammer: Build Terrain Without the Usual Errors

A displacement is a four-sided brush face that Hammer subdivides into a grid of triangles, which you then deform freely. You create it in the texture tool (Shift+A) on the "Displacement" tab with "Create", shape it with "Paint Geometry" and join neighbouring faces with "Sew". The usual problems nearly always have one of three causes: a displacement does not seal the map, it does not block visibility, and its edges only join without gaps under certain conditions.

What a displacement can and cannot do

VBSP treats a brush with a displacement differently from any other brush: it is removed from the world geometry completely, and only the deformed face ends up in the BSP.

The remaining sides of the brush also disappear during the compile, whatever texture they carry. That is why every piece of terrain needs a simple world brush with tools/toolsnodraw underneath and behind it to seal the map. If the red line of a leak runs through the ground, that brush is missing. See fixing the map leak error for how to find the spot. If a hill should also separate two areas for visibility, it needs a core of world brushes, see optimizing GMod map performance.

Creating a displacement and choosing the power

1. Build a brush whose top face has four corners. For larger areas, place several brushes side by side whose edges match exactly.
2. Open the texture tool with Shift+A (Tools → Texture Application) and select the faces that should become terrain.
3. Switch to the "Displacement" tab, click "Create" and enter the power.

The power determines how finely the face is subdivided. The allowed values are 2, 3 and 4.

Power 4 costs more than just triangles. If a map contains a single displacement with power 4, VBSP stores the collision data of all displacements uncompressed and warns:

WARNING: Map using power 4 displacements, terrain physics cannot be compressed, map will need additional memory and CPU.

In a test with GMod's VBSP, 120 displacements with power 3 took up less than 10 KB of collision data. A single additional one with power 4 made the physics section of the BSP grow to about 1.8 MB, 42 percent of the maximum in the usage table. With 120 displacements at power 4, the table showed SIZE OVERFLOW!!!. We did not test whether such a map still loads. Use power 3 as the normal choice and power 2 for gentle, large areas. If you need more resolution, split the face into four smaller brushes with power 3. That gives the same density.

Shaping and blending textures

The two most important tools on the "Displacement" tab:

• Paint Geometry: The dialog offers the effects "Raise/Lower", "Raise To" and "Smooth". "Axis" sets the direction, "Distance" the strength and "Radius" the size of the painting sphere. With "Raise/Lower", the left mouse button raises and the right one lowers.
• Paint Alpha: Blends between two textures. The left mouse button raises the blend value of the vertices, the right one lowers it. "Invert Alpha" inverts the values of the selected displacements, so the two textures swap places.

Paint Alpha only works with a blend material. It uses the WorldVertexTransition shader with two textures in $basetexture and $basetexture2, for example nature/blendgrassgravel001a. Hammer stores the blend value per vertex. The blend therefore follows the grid: a displacement with power 2 has only 5 by 5 points, so narrow paths cannot be painted on it. The texture coordinates remain those of the flat base face. If you pull points steeply upwards, the texture is stretched there.

Sewing neighbours: when Sew works

Every displacement has its own vertices along its edge. A crack appears when the vertices of two neighbours are not in the same position on the shared edge. "Sew" moves them onto each other. Three points decide whether that works:

• Both displacements are selected. "Sew" only works on the selection.
• The brush edges lie exactly on each other. Either both edges have the same end points, or two edges of half the length meet exactly at the midpoint of the longer one. The engine's neighbour data does not know any other case.
• The power may differ. According to the Valve Developer Community wiki, "Sew" then moves the vertices of the finer displacement onto those of the coarser one. The finer side loses its extra detail along that edge.

So plan the layout on the grid before you create displacements. Every time you paint near an edge, sew again. According to the same wiki, "Sew" also joins a displacement to a normal brush face if both base faces share an edge with the same end points. Where they do not, run the terrain a little way into the wall.

Typical errors and their cause

Three messages abort the compile:

Error: displacement found on a(n) func_detail entity - not supported (entity 3, brush 0)
Trying to create a non-quad displacement! (entity 0, brush 12)
ParseDispInfoChunk: nummapdispinfo > MAX_MAP_DISPINFO

The first one appears when a displacement brush is tied to an entity. Select it and choose Tools → Move to World (Ctrl+Shift+W). The second one concerns a base face without exactly four corners, the third one too many displacements. The Source SDK 2013 sets this limit at 2048. For Garry's Mod the Valve Developer Community wiki lists 16384, and a test with GMod's VBSP fits: 2100 displacements compiled without a message, and the abort only came above 16384. Both values are in the overview of Source map limits. The usage table at the end of the log lists no maximum for dispinfos and therefore never warns with VERY FULL!. You can paste unclear messages as a compile log into the Map Scanner.

When a model or brushwork is the better choice

• Brushwork: Flat floors, roads, walls and everything that has to seal, block visibility or move as an entity.
• Model: Cliffs with overhangs, cave ceilings, arches and individual boulders. A model brings undistorted texturing and can be reused as a prop_static as often as you like. The prop types are explained in prop_static, prop_dynamic and prop_physics.
• Displacement: Large, organic ground areas with soft texture transitions and shadows from lightmaps.
• 3D skybox: Terrain nobody ever walks on belongs, scaled down, in the 3D skybox.

FAQ

Do the other sides of the displacement brush have to be nodraw?

Not for the compile, because VBSP discards the entire brush. In the test, a brush with textured sides produced the same number of faces as one with tools/toolsnodraw. Nodraw is still right in case you later remove the displacement with "Destroy".

Do displacements count towards the brush limit?

No. The brush of a displacement does not show up as a brush in the compiled map. Each displacement does take up one face and counts towards the separate displacement limit.

How large should a single displacement be?

What matters is the spacing of the vertices: edge length divided by subdivisions. A brush 512 units long has a point every 64 units at power 3. That is enough for a meadow, but a narrow path needs smaller brushes.