Normal mapping (GLSL shader)

[Omega]

I've been toying with per-pixel lighting (see my Sun shader thread), but found that the normals generated by the tesselator are quite low resolution. So I implemented a normal map layer that calculates the normals from an elevation model.

It pulls the elevation data directly from the ElevationModel, generates a normal map for each elevation tile, and passes the normal map as a texture in a TiledImageLayer to a shader, which calculates per-pixel lighting and shading. The results are quite encouraging (see screenshots).

The exaggeration in the shader can be changed in realtime, and so could the elevation colors (by editing the fragment shader to accept different color gradients as uniforms).

I'm not sure if something like this would be useful as part of the SDK, as it's pretty specialized, but it may help some people.

I'd really love to see multitexturing make it's way into the SDK. Then we could pass an image texture (eg Blue Marble or Landsat) AND a normal map to the shader as different texture units, and then do beautiful per-pixel lighting using the standard texture layers.

Anyway, feel free to use and extend this.

Kind regards,
-Michael

[bull]

Wow, nice work.

[patmurris]

Hi Omega, glad to hear from you... i was wondering what you where up to these days

Nice work again with the shaders. I'm just not sure why you need to generate a normal map. Don't you get interpolated normals for each fragment without it?

[Omega]

Originally Posted by patmurris

Nice work again with the shaders. I'm just not sure why you need to generate a normal map. Don't you get interpolated normals for each fragment without it?

Yes, but the tesselator only produces density x density (default 20 x 20) vertices for each elevation tile it generates, and there is one normal per vertex. This is a MUCH smaller number of normals as when you retrieve the normals from a texture, therefore the resolution is unacceptable. This is the whole point of normal mapping. You have a low-polygon model (the elevation model rendered as a globe), and you map a high resolution normal map to the surface to make it appear as a high-polygon model.

As an example, I've taken two screenshots from a similar viewpoint; one rendered with normals per vertex (from the RectangularNormalTesselator), and the other with normals per pixel (with a normal map). The difference is clear

[patmurris]

The difference is quite clear at the globe level, but does it make a big difference once close to the terrain?
To complete the comparison it would have been interesting to see 'normals par vertex without fragment interpolation'.

[bull]

Darn now I really want this in .NET you guys are finally getting to a stage where WWJ looks as pretty as WW .NET

[Omega]

Originally Posted by patmurris

To complete the comparison it would have been interesting to see 'normals par vertex without fragment interpolation'.

This would look very similar the 'normals per vertex with fragment interpolation'. You would just set the color at each vertex, and then the color would be interpolated across each triangle anyway, so there wouldn't be much difference.

Originally Posted by patmurris

The difference is quite clear at the globe level, but does it make a big difference once close to the terrain?

Yes it makes just as big a difference when viewing close to terrain. See attached screenshots.

[patmurris]

That's quite spectacular indeed!

I'm guessing the bottleneck is generating the normal maps on the fly. How big are the tiles and how does it perform? I did something similar some time ago with procedural tiled image layers and straight layer overlaying for shading (no fragment shader involved). One issue was that elevation data may not be available right away when you need to compute the normals. How did you deal with this situation? OK i'll look at the source

Thanks again for sharing your experiments.

[Omega]

Originally Posted by patmurris

I'm guessing the bottleneck is generating the normal maps on the fly.

The normal maps are generated in a RequestTask just like BasicTiledImageLayer (and executed by the TaskService), so it doesn't slow down the rendering thread at all.

Originally Posted by patmurris

How big are the tiles and how does it perform?

The layer copies most of its LevelSet data from the elevation model. The default Earth elevation model has 150x150 tiles, and therefore so does the normal map layer when using that elevation model as the data.

Performance mostly depends on the video card, and how quickly it renders GLSL shaders. It also depends on what the fragment shader is doing (a shader that does per-pixel lighting with diffuse and specular components will be much slower that one that simply sets pixel colors using the pixel's elevation).

Originally Posted by patmurris

I did something similar some time ago with procedural tiled image layers and straight layer overlaying for shading (no fragment shader involved). One issue was that elevation data may not be available right away when you need to compute the normals. How did you deal with this situation? OK i'll look at the source

The normal map layer actually depends on the ElevationModel; whenever it receives a tile request, if the ElevationModel doesn't have the elevation tile in memory, it requests it using the BasicElevationModel.requestTile() function (note: I had to extend BasicElevationModel to make this function public; this is what the ExtendedBasicElevationModel is for, among other things). Therefore it never runs into the problem of missing elevation data, because if it is missing, it requests it until it's available.

Originally Posted by patmurris

Thanks again for sharing your experiments.

You're welcome

One more thing I should mention: the normal map texture contains the x,y,z components of the normal in the red,green,blue components of the texture, but it also contains the elevation of each pixel in the alpha component of the texture. This means that the pixel shader can access the exact elevation of each fragment. This allows for coloring of the pixels according to the elevation; perfect for pseudocoloring like your procedural layers example.

However, currently the texture is a 32-bit texture (8-bits per component). This means that the elevation data only has a range of 0-255, which is not enough as the elevation data used in World Wind is 16-bit. To get around this, I've offset & scaled the elevation data for each tile using the min and max elevation found in that tile (the min and max elevation are also stored in the texture file). The min elevation will then equal 0 and the max elevation equal 255 for each tile. When each tile's texture is bound, the layer passes the min and max elevation to the shader, and the shader rescales & offsets the elevations back to their original value.

This problem could be fixed by using a texture format that supports a 16-bit alpha channel (only more recent video cards), or by enabling multi-texturing and passing the normal map as a 24-bit rgb texture and the elevation map as a 16-bit luminance texture (again, only more recent video cards).

[tle]

Hi,

It's really an impressive work. I'm trying to extend your work in my WWJ's prototype. I have a few questions :

- I think that the normal vector could be computed directly in the fragment shader. You have to send only the elevation as an one channel texture to it. Maybe the problem is how to compute the step between two neighbour vertices ? I'm curious to know if there is a solution to do this only from shader.

- Is there a way to get a bigger texture resolution (that will cover 1 line and 1 collum more in each side than just the tile's surface). I notice an artifact on the last line (column) of each side of one tile - due to the lack of elevation datas

Kind regards,