Delete subassets; only asset processing!

[andriyDev]

Motivation

Subassets cause all sorts of crazy issues (e.g., #12756) and a never-ending stream of edge cases (e.g., #21222). Loading a single subasset can result in the entire asset being reloaded even if other subassets from the same root asset are still around. A lot of complexity in the asset server is due to juggling subassets.

Asset processing is only barely used by Bevy and AFAIK users. Part of the reason for this is that asset processing has very confusing functionality. It currently behaves as though it should be totally transparent (turning on/off asset processing should make no difference), but that means users are unlikely to want to make their game "dependent" on asset processing. By making it so transparent, it becomes confusing to have an asset processor change the type of an asset/subasset (since now callers might loading the asset as a different type than they'd otherwise expect). Asset processing is also primarily demonstrated as an optimization for build time.

We expect in the future that formats like GLTF should be preprocessed into BSN and several other files (known as "one to many" asset processing, see #9714). However, this is difficult to implement since it's unclear which files to write each subasset to. How do we load this up at runtime? Will our file paths include #?

The Scheme

Delete subassets, and make asset processing "mandatory". Let's see how this would work. I will be using GLTF as an example.

You take your .gltf/.glb file and place it into your assets folder (for this example "path/to/my.gltf"). If you were to attempt to load this in Bevy without asset processing, the GltfLoader will load it as a Gltf asset. There are no subassets, so all the meshes, textures, materials, scenes, etc must be stored inside this one big Gltf asset. So far this is only a downgrade.

But now, we bring in the GltfProcessor. This will take that one Gltf asset, split it into a bunch of smaller objects (e.g., each mesh, each scene, etc) and call something like ProcessContext::write_subasset for each of them. This would write the assets as "path/to/my.gltf/Mesh0" perhaps. We now have a directory full of all the "subassets" that the Gltf asset contained (note: it doesn't have to be all subassets, discussed later).

Now at runtime, users just load "path/to/my.gltf/Mesh0". They load that singular mesh. That's it. There are no subassets, so either the asset is loaded or it's not loaded. All the "useful" parts of the Gltf have been extracted out through the GltfProcessor. The subasset can be hot reloaded easily since we know there's a one-to-one mapping between files and assets. The asset processor can also be hot reloaded easily since there's also a one-to-one mapping between original files and assets that need to be processed.

This effectively turns the asset processor into an import process! It now has a very clear purpose - we are turning raw source assets into just the "useful" parts of the asset. This also makes the asset processor easier to wrap. For example, you can just take the Bevy default GltfProcessor, wrap it in your own processor that extracts some other data (e.g., GLTF extras), and writes them to some other asset - while keeping all the original processing. This means that we do not need to expose every piece of data in the GLTF in our GltfProcessor. For example, the default GltfProcessor should not do anything with the GLTF extras. If a user wants that data, they will need to wrap the default processor, or write their own entirely (usually wrapping should be sufficient though).

Risks/Regressions

1. We're turning one big file like .glb into many small files. Reading many small files is usually less efficient than one big file, so it could be a performance concern.
2. Loading UGC is harder. You can't just accept a GLTF file path, load it, and try to spawn it as a scene. You must first load it as a Gltf asset, and then decompose that file manually (e.g., through a system). This is effectively running the asset processor at runtime.
   • We can probably add a manual convenience function for applying a processor at runtime. Unclear if it's necessary.
3. "Archive assets" can't exist anymore Supporting archive assets #12279.
   • Archive assets shouldn't exist anyway IMO. If it's truly an archive, it should be an asset source that we "mount" at runtime. This means supporting changing the set of asset sources at runtime though.
4. Users are now forced to use the asset processor.
   • We need to make this easy to run.
   • I don't see this as a big loss. We expect this to be the future anyway with BSN coming.
5. The asset processor would only really make sense at publish time.
   • This is kinda the case today. It would be weird to be running the asset processor after publish time during game time.
6. The asset processor currently has an issue to do with async-fs, resulting in corrupted files. Dropped writes resulting in corrupted files smol-rs/async-fs#45
   • This is presumably a temporary issue though.

Positive Side-effects

1. Since the asset processor only makes sense at publish time, we should lean into that and remove as much of the runtime logic as possible. Yay for simpler code!
2. Our AssetPath format becomes simpler - it's basically all paths!
3. We need to "guess the type" of an asset less often, since it's whatever the user asked for, since users can't request subassets.
4. Hot reloading is simpler! Just find the assets for that one file - no subassets!
5. One to many asset processing is easier to implement. Since asset processing is necessary, the processor can write whatever file paths it wants (with some fixed prefix), and the user just loads those assets. There's no confusion whether the subasset exists in the original asset or just the processed asset - it has to be the processed asset!

[viridia]

I use GLTF extras quite a bit. For example, I used an outline property on the material to indicate which materials should have outlines. Similarly, I have other extras properties for animated materials such as flames, or magical glows. Basically I use extras for any asset property that can't be encoded in the GLTF file format. (I also tend to use extras for unlit materials since I've never gotten the unlit extension to work properly.)

Currently I do all this by processing the scene hierarchy after it's been loaded - replacing the various materials based on the properties in extras. I admit that this is clunky and not optimal, but at least it does work.

I agree with you this much: sub-assets as they exist today are badly broken. And I can see advantages of your scheme; but I also worry that what you're proposing is to replace something that works with something that hasn't been written yet: removing features that we depend on now with a promise of something that might exist in the future.

Consider: to get feature parity for the use case I've outlined, we'd need to have all of the following in place:

• A binary format for encoding meshes (no, RON files aren't going to cut it. Nor will any other format that involves parsing lots of text.)
• ...which also includes a way to encode arbitrary Bevy components
• ...with a fully working asset build pipeline that convert between the formats
• ...that is also extensible enough to process game-specific extras (like outlines and animated materials)

That's a big challenge; until all of those things are in place, the use case I have described is broken. I think it's infeasible to try and make that big of a change in a single release cycle. What's missing from your proposal is an incremental plan where we can take on these tasks one at a time without breaking everything in the interim.

I also don't think that we can say that AssetSource can be the only way to load an archive. Asset sources are fine in cases where all the sources are known at compile time, but this will often not be the case.

The AssetSourceId is like the schema of a URL: ("http://" or "ftp://") - there's currently no mechanism for defining an arbitrary number of asset sources at runtime, and making them dynamic is not really what asset sources are meant for. If I have a directory containing 5 zip files, this shouldn't be 5 asset sources. Instead, there should be a single asset source for the root, which can contain an arbitrary number of archives. Maybe we need a new concept in Bevy assets to handle archives under a root.

(As an aside, I'm not a big fan of RON: it adds extra verbosity to the file that provides very little benefit other than giving the user the opportunity to misspell complex symbol names, and it has no support for schema validation or autocompletion.)

But part of the reason why they are broken is due to limitations that are built into the way Bevy loads assets. One can imagine, however, an alternate design where an asset containing sub-assets gets loaded, and all of the subassets in the archive cached in a LRU, so that subsequent calls to load that subasset would return the entry already loaded; this wouldn't be perfect, as you'd still have occasional cache misses, or subassets loaded and cached unnecessarily, but it would be better than what we have now. I'm not saying this is the route we should go down, I'm just point out that there are other options.

[ChristopherBiscardi]

I feel like its important to mention the caveat that gltf->bsn conversion can't actually happen until we have a bsn asset format, which doesn't seem likely until after 0.18 since it is left out of the current bsn draft PR in favor of being a stretch goal, and that we don't actually have things like the file formats defined for storing mesh vertex data that we would need to actually split a gltf up.

Loading a single subasset can result in the entire asset being reloaded even if other subassets from the same root asset are still around

more context: The current sub-asset system makes it seem like you can arbitrarily access any sub-asset at any time via #Thing but gltf, for example, doesn't enable arbitrary scans of data halfway through a file, so accessing a sub-asset requires loading the full asset anyway. the sub-asset behavior of "load everything and then dump what isn't needed" is confusing and surprising, as things like the AssetEvent for a GltfAsset can't even access the Gltf data if a sub-asset triggered the load.

For example, the default GltfProcessor should not do anything with the GLTF extras. If a user wants that data, they will need to wrap the default processor, or write their own entirely (usually wrapping should be sufficient though).

So if there are two sources of data in the extras (or more likely, extensions). As a concrete example, lets say, skein inserts component data and the end-user also uses blender custom properties. How do both of these sources get processed? This description reads like one will have to win out over the other since the processor will be wrapped. Skein would have to supply "processor pieces" and the end-user would have to write their own wrapper to use the gltfprocessor and those pieces.

Then additionally, metadata doesn't sound like it would be stored with the relevant assets, so you'd always need to fetch the custom Gltf asset and navigate to the metadata anyway, even if using meshes directly.

Now at runtime, users just load "path/to/my.gltf/Mesh0"

As far as I know, indices are technically only meant to be internal references

Whereas indices are used for internal glTF references, optional names are used for application-specific uses such as display.

So referencing "Mesh0" could reference a different mesh any time the processed gltf was updated by a user. There's been a lot of conversation around using names when importing, and if we're encoding those further into the creation of assets I think we should also confirm a switch to names.

A gltf object not having a name means it is not meant to be publicly addressable.

---

I generally agree with @viridia and would also love to see some concrete wins in the asset processing department. I've tried to make use of asset processing and custom loaders quite a few times and I always fall back to it being easier to process outside of bevy. LoadTransformAndSave today, with the same-type requirement, same extension, etc is an awkward restriction. I'd love to have more "build the gpu-ready file" functionality built-in for textures.

We can probably add a manual convenience function for applying a processor at runtime. Unclear if it's necessary.

I do feel a bit like a hard split between build-time and runtime is awkward. I don't really see why a loading process couldn't happen at runtime as well for ugc.

Users are now forced to use the asset processor

Potential increase in build times (or rather, time-to-seeing-my-game-running)

[andriyDev]

@viridia Going down your list of "things we need":

• yes we need a binary format for meshes - but we want this anyway. This was already the plan in regards to BSN: convert all the meshes to our binary format and reference them in BSN.
• encoding arbitrary components isn't necessary? At least, not from the perspective of the default processor. A user's custom processor should take the GLTF extras and apply them to the scene that is produced from the GLTF processor. This should be entirely user code because otherwise, we're forced to write every piece of data of the GLTF into some file, which is overkill.
• this is already asset processing today. The only reason it's weird today is because you need to keep the same file extension, which may not be true in this model (except for the prefix path path/to/my.gltf/.
• just write a custom asset processor to extract that info. Asset processors should be about taking the raw assets like GLTF and taking all the useful stuff out of it on import.

So we either have this, want it anyway, or it's a matter of leaning on user code for portions that are user specific anyway!

As for "asset sources are not meant to be dynamic", I don't understand why this is a "requirement". Currently this is a limitation because of how asset processing is implemented. I would argue we should just limit asset processing to only the default asset source. Then all other sources should just be free to be dynamic. Users can absolutely build asset path strings, seems like asset sources can be just as dynamic? I like the example of a DLC: it seems reasonable to "mount" your DLC assets, and then otherwise treat it like a normal asset source.

(I am also a RON hater, but the particular formats is just bike shedding IMO).

[andriyDev]

@ChristopherBiscardi This is certainly not a proposal of "let's do this in a weekend" haha. Though specifically for GLTF, this could be done with Scenes as they exist today I think?

one will have to win out over the other

Yes, win necessarily wins out over the other? Today whichever system runs last between a user's system and skein's system will "win". This is no different except with more control.

I suspect the GLTF processor would be designed to be hackable so you can take just the pieces you want. Skein would probably want to follow a similar model. A default skein processor though would probably keep 99% of people happy.

indices are technically only meant to be internal references

Yes. This is filed as #16529 and #17984. TL;DR GLTF names are kinda fake, and we're waiting on assets as entities. I suspect deleting subassets to help, since we can do more shenanigans with asset processing by forwarding the GLTF name path to the GLTF index path. But maybe that's a distraction.

I don't really see why a loading process couldn't happen at runtime as well for ugc.

Primarily because I don't think we should be writing a bunch of files to decompose a UGC file. Seems very gross and feels like it should be all in memory. There may be a way to make asset processors write the assets into the world instead of to disk, but that seems hacky, and gets us right back to the weird situation that the root asset could be unloaded while the subassets are. All the hassle of subassets with a worse API lol. UGC is a more niche use case, and we can provide some manual tools to do reasonable things including doing asset processing to decompose - but it's up to the user to manage.

[ChristopherBiscardi]

Today whichever system runs last between a user's system and skein's system will "win". This is no different except with more control.

Today the gltf extras get placed on the entities that have them as Gltf*Extras components, allowing anyone to process them with an observer (they are also stored in the Gltf asset for additional access patterns). One consumer doesn't preclude another from processing unless that consumer chooses to behave poorly and delete the extras components before any other processor can access them. the process specified here eagerly drops the data, presumably resulting in no Gltf*Extras components, and no extras data in the Gltf asset. https://docs.rs/bevy/latest/bevy/gltf/struct.GltfExtras.html

So the behavior described here is a downgrade, since the data is dropped by default and requires writing a custom processor per-consumer if you want to access it.

this isn't really an asset processing issue at the moment, because extras become components, but if the new system decides to arbitrarily drop the data by default it will be.

[Igor-dvr]

This should probably be implemented separately as experimental first and not instead of the current solution. This needs to be properly tested, compared and formats that support referencing should be taken into account as well as various formats and extensions we don't support yet.