Saints Row IV How are textures are cached and read as "unique" in-game?

[GPZ]

I've been recently trying to improve the various Gang options in the game. One of the troubles I ran into was how the game reads and assigns textures to the NPC models.

This happens when spawning a regular Morning Star in the world along with a Saints variant that uses "npc_mstar_female_soldier" as the base model (because it has cloth simulation and the original Saints version does not; I realized that this model is an unused leftover from SR:TT and is not spawned when fighting Morning Stars). The textures will literally change in-game as seen in this picture:

In theory - this shouldn't happen. Duplicate models and duplicate textures should allow for two independently working variants, however this is not the case here apparently; The game will read two materials as one. I assume it is done to save memory, as it does not happen when the Saints version is spawned alone.

Is there a way for us with the current tools to make the game read each variant as independent so they could co-exist? The other Morning Star Saints also seem dependent on the regular versions - when changing the Saints' face textures the changes apply to the regular Morning Star too - something that isn't problematic but also isn't ideal for what I'm trying to do. On another model, copying the matlib from the original to the Saints variant also made this fun little thing happen when both variants are in the world:

I think it happens because these textures aren't stored as unique ones so the game will simply load whichever texture or material that is there. I hope you guys could shed some light on this, and that it is something that can be fixed.

EDIT: It seems that the hair colors on NPCs are also cached, but where is that cache stored? When there aren't any NPCs spawned with a specific hair color other than the basic 4 (black 01, brunette 01, blonde 01 and white 01) the hair displayed will be white (color-less like in the diffuse texture it seems). It looks nice when it does work, though:

 

[.Swami]

I've been collaborating with GPZ on a specific instance of this situation/problem, and would like to give it as a specific example.

In trying to change one of the gang member choices in Gang Customization...

Successfully (GPZ, not me) replaced one of the gang member choices' head and body (both mesh and texture).
• The head was from another gang member choice, and a is only used for Saints gang members.
• The body was from a pedestrian, so peds would spawn with the same mesh and texture.

Editing the head texture went fine, seeing as it wasn't a shared item.

Editing the body (as expected) meant that the pedestrian whose body was used would also spawn with that edited texture.

I previously (thanks to huge help from [V] Knobby) got around that issue when there are duplicates to be loaded by getting the game to load renamed versions of these things, and I got it to the point where it did load the renamed textures.

Weirdly, despite the different names, the ped NPC will "flicker" (slowly; have to follow them for a few seconds, occasionally moving the camera) between the textures in that ped's packfile and the custom, renamed ones.

Why is that? The ped's packfile's .cvbm/.gvbm don't have the same name anymore.

I even tried changing the name of the ped's textures/meshes to something completely different as well.
It worked in the sense that it loaded, but...it still would change back and forth.

 

[[V] Knobby]

The texture manager doesn't know about peds or anything else. It just knows texture handles and data pointers. These handles are derived from the name of the texture. A single texture filename will map to a single handle forever. The renderer will say "Hey, texture manager! I need texture 0x192a." And the texture manager looks up that handle and hands the renderer the highest resolution texture currently loaded. When multiple textures of the same resolution are loaded, it just hands it one of them. I *think* it is the one that was registered most recently. So what you describe is a situation where you load your texture and it uses it, but then as you move around, the engine loads the other character and their texture is used in place of yours since it is loaded. That character is evicted and the texture goes away so your ped starts using your specific texture again.

I would verify that you hacked the cvbm file to alter the name of the texture that is being loaded so it is assigned a different handle. You will also need to make sure everything else points to that name.

 

[.Swami]

[...]
I would verify that you hacked the cvbm file to alter the name of the texture that is being loaded so it is assigned a different handle.

Just checked the last version of those files that was working as stated before, and the .cpeg and .gpeg files are all renamed in both the saints_female_lindas' str2_pc files.

The character_containers.asm_pc was hex edited to load those renamed .cpeg files as well.

You will also need to make sure everything else points to that name.

Now, here's where I re-checked, searching everything inside misc_tables for the text "lindas" (getting hits in character.xtbl [Line 375] and character_definitions.xtbl [Line 298])

I didn't see anything that referenced textures, just meshes/rigs/animations.

The only thing I saw that could maybe be related was her entry in character_definitions.xtbl (Line 298).

It has a number of files I recognized, and I thought they were all meshes/rigs/animations—things I didn't want (need?) to rename, seeing as they all have extensions in their <Filename> tags (.cmeshx, .rigx, .simx).

But in the <Heads> tags I see this:

    <Heads>
        <Head>
            <Name>lindas_head</Name>
            <Base_Head>npc_basehead</Base_Head>
            <Lod_Head>lindas_head_lod</Lod_Head>
            <Eyelashes>False</Eyelashes>
        </Head>
    </Heads>

I had assumed that since since one of them was "lindas_head_lod" that that text was also mesh-specific information, irrelevant for textures.

But since there's no extensions, I'm not sure.
There is a lindas_head.cpeg_pc, and it's since been renamed in this process.
But, there isn't any *_lod cpeg, so I just assumed both were mesh information.

Would that <Name> tag have anything to do with texture names?

Because aside from that, I don't see why the texture switching back and forth would happen. :-\

 

[[V] Knobby]

The reference you're looking for would be inside the cmesh_pc or a material file or something. When that character loads and is rendered, something internal to the data loaded tells it what resources it needs. I'm not overly familiar with this so maybe Randall can help us find it.

 

[Meatplowz]

The npc's head materials are based on the "basehead.gcmesh" material layout. All npc heads will inherit the base material but then override with their "name" derived matlib/peg files. So in this case lindas_head.matlib_pc, lindas_head.gpeg_pc, etc. The lindas_head_lod mesh, that replaces the basehead when far enough away, will get the same linda_head material/textures applied to it as the lindas_head morph.

I mentioned in another thread about the extended hair pieces and how they are colored. They don't inherit the same coloring as the head as they are not "technically" a part of the head.
http://www.saintsrowmods.com/forum/threads/about-npc-hair-shaders.4507/

The only thing I can think of in regards to the flickering is perhaps the high and low resolution textures haven't both been updated. Each character, as you may have seen has a high and low resolution version of their materials/textures. The materials are essentially the same they just now reference higher resolution textures, so their pegs will be much larger. So if for whatever reason, the pegs are different then you will seeing popping when the high and low res pegs stream in/out.

As for Linda and LindaSaints, I just looked at their raw files and they do in fact share texture references between the two while there are some that are different. I don't know if for whatever reason the game may optimize very similar materials out or not. I've seen stranger things happen.

 

[.Swami]

The reference you're looking for would be inside the cmesh_pc or a material file or something.

Across both the regular and *_high.str2_pc files, I (re-)checked out all .matlib_pc and .ccmesh_pc files in my hex editor, and the filenames I see (in all of them) are all the .tga texture files.

In other words, what's inside the peg files.

And in my initial quest to rename textures, I learned that the .tga files' names (which with current tools extract as .tgs.DDS; so they aren't really in the Targa format exactly) are irrelevant.
Just the .cpeg/.gpeg (or .cvbm/.gvbm) need to be changed, because the .asm_pc indexes only contain the name of the .cpeg (or .cvbm), not what's in them.

In fact, when the .tga files' names are changed, and all the necessary work is done to get the game to load them, you get this (from Modding Mishaps - SRIV version):

Today I learned...
Renaming textures = stick with the .cpeg & .gpeg files.​

If you touch the .tga/.DDS files, you get a Korean horror film:

It's the .*peg files that need to be renamed, and since peg files don't all contain single textures with the exact same filename, there's no need for any correlation.

-----------------------------------------------------------------------------------

As for Linda and LindaSaints, I just looked at their raw files and they do in fact share texture references between the two while there are some that are different. I don't know if for whatever reason the game may optimize very similar materials out or not. I've seen stranger things happen.

I'm aware of the two Lindas files (and that whole system), but it's worth noting that the saints_female_lindas pair of .str2's (regular and *_high) I'm working with are filled mostly with replacements.

The head is from inside saints_female_johanna's (regular and *_high) files.
The body is from generic_female_businesswoman02's (regular and *_high) files.

And the issue is only with the latter, because the textures taken from businesswoman02 were edited to be different, whereas johanna's were left vanilla.

The only thing I can think of in regards to the flickering is perhaps the high and low resolution textures haven't both been updated. Each character, as you may have seen has a high and low resolution version of their materials/textures.

It's not that, I've been familiar with the regular and _high packfiles since the previous game.

Since my initial posting in this thread, I've tried...:
• Changing the names of all the files in both .str2_pc's (and updating the indexes when referencing)
• Additionally changing the names of the textures that businesswoman02 uses ( ″ ″ )
• Changing the .tga file names ( ″ ″ )
• Editing the characters' entries in character_definitions.xtbl to load renamed everythings

None of those tactics, including combinations of them, have worked. She would be invisible when you get to her Gang Customization slot (STRIPPER 3). The game wouldn't freeze, but she just wouldn't show up, as opposed to before those experiments, where I was getting her to show, as well as businesswoman02, and johanna, but the problem at the base of this (switching/flickering of textures) was there.

If you'd like, I can upload the 3 files that produce the switching problem; I just need to clear it with @GPZ . The original files were ones he had first modified (mainly just the .DDS' inside the pegs) — he asked me about it, since I'd gotten a renamed texture to load, so what I have still has his work in it.

 

[GPZ]

I already understood it's related to the texture names (taking Saints variants as examples - the modified purple textures all have an "s" added to their name so they are unique). We can't really alter these yet unfortunately and that does cause some mismatches with replacements that should be relatively easy to do. Along with Swami I could also upload the Zoey character that I've given as an example above so you could see how she switches between the edited and non-edited textures when both model variants are spawned in the world.

 

[.Swami]

Here's what happens with all the changes I made:

My backup is all the modded version of lindas, and she shows up as hoped, with the custom textures (I'm not concerned with the color matching, nor the clipping)

I find a genericbusinesswoman02, and she has her textures:

But after walking a few feet:

You can see that the new lindas (all 3 of them) don't switch back-and-forth at all. But bw02 just...does.

 

[[V] Knobby]

I didn't think about the high res stuff. I'd have to see it locally to see if it is high res character maps fighting with the normal res or if it is the same filename thing. Is this something you've released?