uGridsToLOD: LOD Fallout 4 Improvement Project. Problem statement
In this article, I am starting a series of posts dedicated to my personal project to modify the Fallout 4 game. The goal of this project is to improve the quality of drawing objects through the processing of LOD, which leaves much to be desired in the game.
As you know, in Fallout 4, the rendering of objects is controlled by several parameters. Objects with a high level of detail are loaded around the player at a distance that is determined by the uGridsToLoad parameter. It determines how many cells of the game world should be loaded, considering the center of the cell in which the player is located. The standard value of the parameter is 5, it means loading objects with high detail in a square area, one side of which is formed by five cells.
Everything outside of this area is controlled by the LOD parameters (Levels Of Detail). The LOD is divided into 2 parts and 4 levels. The first part is the LOD of objects, the second is the LOD of the landscape, including land and water. Both parts of the LOD are made up of square blocks. Each block includes a fixed number of cells of the game world, a multiple of two. The LOD level to which the block belongs is denoted by the length of the side of this block, which allows you to determine its dimension: LOD 4 (16 cells), LOD 8 (64 cells), LOD 16 (256 cells) and LOD 32 (1024 cells). By default, the block size is inversely proportional to the level of detail, but if desired, you can generate a LOD with any level of detail.
In game archives, the finished LOD is represented by three types of files: landscape meshes. btr, meshes of objects .bto and textures .dds. It does not use material files, shader properties are sewn into mesh files, although materials in the form of separate files are used in the LOD generation process. The names of files with meshes are formed according to this principle:
where worlds_name is the name of the game worldspace, in the vanilla game it is Commonwealth, DiamondCity and SanctuaryHillsWorld; lod_level is a number defining the LOD level (4, 8, 16 or 32); X and Y are the coordinates of the lower left cell of the block having the length of lod_level along the X and Y axes to the right and up, respectively. The names of the LOD textures of the earth are formed according to a similar principle.
For example, let's look at two files:
- meshes/terrain/Commonwealth /Commonwealth.32.0.0.btr - this is a mesh file extracted from the archive “Fallout4 – Meshes.ba2”, is a 32x32 cell landscape LOD block (see the square on the left in the picture - a screenshot taken in the NifSkope program);
- textures/terrain/Commonwealth/Commonwealth.32.0.0.dds - this texture file, extracted from the archive “Fallout4 – Textures4.ba2", is a diffuse map of the earth for this block of landscape (see the square on the right in the picture).
On the coordinate grid of the Falout 4 world, fragment 32.0.0 fills the upper right quarter of the map: this is a 32x32 square, the left (X) lower (Y) cell of which has the coordinate 0, 0 (see the picture).
In order not to get confused when working with the coordinate grid and LOD files, it must be remembered that LOD is formed on the principle of redundancy, so there may be more blocks of different levels, and especially the 32nd, than they physically fit on the coordinate grid of the world. For example, if you look closely at the map, it becomes clear that only 4 32x32 squares fit in the world of Fallout 4. But in the Commonwealth alone we will find 36 blocks of this level. How so? Very simple: the blocks go beyond the space available for the player to move. Otherwise, the boundaries of the game world would not look very good. (By the way, you can turn off the borders and take a walk from the “outher side”.)
Let's see where the LOD files are located.
Meshes of the LOD landscape: Data/Meshes/Terrain/[worlds_name] (for example, these are 3060 files for the Commonwealth world).
Meshes of LOD objects: Data/Grids/Terrain/[world_name]/Objects (461 files for the Commonwealth).
Textures of the LOD landscape: Data/Textures/Relief/[world_name] (6123 files for the Commonwealth).
Texture atlases for LOD objects: Data/Textures/Terrain/[worlds_name]/Objects (for any of the game worldspaces, these are always three large prefabricated texture maps - diffuse, normal and specular).
In the picture you can see an example of an atlas of diffuse maps, in the upper left corner of which you can see the map I redrawn manually for the LOD version of the overpasses.
It may seem that everything you need seems to be in two places - the Meshes/Terrain and Textures/Terrain folders. But this is not the case. These folders contain ready-made landscape and object LOD files and resources for generating landscape LOD. The resources for generating LOD objects that my project needs are located elsewhere. These are primarily the Materials/LOD, Meshes/LOD and Textures/LOD folders, where the necessary materials, meshes and textures are located. In addition, to work with the LOD of objects, it is useful to know where the information about which objects should be included in it during generation comes from. This information is contained in the form of links from the basic game models to their LOD versions, which can be seen through the FO4Edit program. It can also be used to add any missing object to the LOD. To do this, it is enough to create a new plugin .esp, transfer the record of the desired object to it, add the Has Distant LOD flag to this record and specify a reference to one or more LOD meshes, of course, having previously created it (them) physically. The following picture shows what the entry about the DiamondCity exterior element looks like, which has meshes on three LOD levels.
Note that the same mesh with the _LOD_0 postfix is used at all three levels. That is, it is possible. Moreover, there are many LOD meshes in the game, in the name of which the postfix does not have a numeric index at all, and there we will find just _LOD or LOD.
In in-game resources located outside the ../Terrain folders, as well as in the Fallout 4 settings, the LOD of objects is divided into 4 levels:
LOD_0, LOD #0, Level0 or Level 0,
LOD_1, LOD #1, Level1 or Level 1,
LOD_2, LOD #2, Level2 or Level 2,
LOD_3, LOD #3, Level3 or Level 3.
When loading the LOD during the game, the reference point is the cell with the player. LOD_0 objects belong to the LOD 4 level, they are loaded directly behind the high-detail area, which, as we remember, by default is a square with a dimension of 5x5 cells. LOD_1 objects belong to the LOD 8 level and are located behind the LOD_0 object area. Etc. LOD_2 objects belong to the LOD 16 level, LOD_3 objects belong to the LOD 32 level, i.e. they are at the greatest distance from the player and have the least detail. For the LOD 16 level, the game offers a total of 47 LOD_2 files, of which 44 are models of the trees. The LOD 32 level includes three tree model files and five files .bto: four from the Commonwealth world, in which the largest objects of the game are selectively included: high-rise buildings and an airship; one from the world of SanctuaryHillsWorld, in which only trees got into. For clarity, the lists of all 50 LOD_2 and LOD_3 meshes and all five LOD 32 blocks are shown in the following pictures.
The distance of rendering objects of different LOD levels relative to the player character is regulated by the settings in the .ini files: fBlockLevel0Distance, fBlockLevel1Distance, fBlockLevel2Distance и fBlockMaximumDistance.
Most of the objects of the game world are represented in LOD at LOD 4 and LOD 16 levels: these are 938 mesh files with the _LOD_0 postfix, 198 mesh files with the _LOD_1 postfix, one (!) mesh file with the _ LOD_1 postfix (i.e. with a space between _ and LOD) and 2805 files with the LOD postfix, the attribution of which to a particular LOD level can be found out only in one way: by examining the records of objects using FO4Edit. In addition, there are 5 files with the LOD_Fake prefix (models of high-rise buildings from the lod/architecture/unique folder) and 2 files with the LOD_FLAT postfix in the landscape/trees folder. For the most part, these are not complete models of objects, but their fragments, often just planes of two triangles, which are compiled into complete objects when generating LOD. Almost all of these objects are very rough, and even a zero-level LOD doesn't look very good. The models here are characterized by a very uneven quality, which varies from tolerable to frankly unacceptable, which is especially noticeable on large screens, when using an optical sight and in the VR version of the game. I think that everyone who has played Fallout 4 understands what we are talking about, but still I will give a couple of examples of what I would not like to see on the screen, because it spoils the experience of the game.
The central idea of my project is to transfer the most noticeable highly detailed game objects to a zero-level LOD, having previously optimized meshes and textures. This is reflected in the code designation of the uGridsToLOD project. The wordplay with uGridsToLoad reflects an alternative way to increase the detail of drawing remote objects, without increasing, and if desired, even lowering the value of uGridsToLoad, i.e. to improve the quality of graphics in the game, not only without increasing, but in some cases reducing the load on the computer. In general, increasing the detail through improving the LOD should not lead to a noticeable drop in performance, because the drawing of LOD objects is organized differently in the game than near the player. The main difference is that all objects of each LOD block are combined into several huge meshes, and the textures of all objects of all blocks are combined into one large atlas. This technique reduces the number of objects for processing hundreds of times, as a result of which the load on the computer is reduced several times.
For example, if we take the largest LOD file (Commonwealth.8.0.-8.bto), which contains the center of Boston with most high-rise buildings, and open it in the NifSkope program, we can observe that this does not create any significant load on the video card (in our example, 1050 Ti is used). You can see this block in the following picture (by the way, if you pay attention to the location of the frigate “Constitution”, you can conclude how I solved the problem of Captain Ironsides).
If we rebuild the LOD, forbidding the generator to merge meshes (there are at least 17 xLODGen fine-tuning parameters that are not reflected in the documentation for this program, which can be extracted by analyzing the source code from .cs files available on GitHub), and then open this file again, we will see that it has increased in 2 times and it is almost impossible to work with it, since even a simple rotation of the scene turns into a slideshow with a frame rate not exceeding 10 per second. That is, the computer's performance drops several times, although we are talking about only one block of the LOD 8 level (LOD_1). If we take into account that a lot of LOD_0 objects are not as economically built as one would expect, then the idea of the project gets a good chance of life. Of course, I repeat, a prerequisite here is to reduce the number of meshes faces and the size of textures for the texture atlas. In this regard, it should be noted one feature of LOD, useful from the point of view of the meshes retopology task: it allows access to materials and textures outside the atlas. Due to this, it is possible to use UV-unwraps that go beyond the boundaries of the U1V1 texture coordinates.
The first results have already been obtained and they are encouraging. I will tell you about this in the next article.
Have a nice day, everyone.
Any use of this article must include a link to this publication.