Version 2.5.2 / 2.6.0 (Major Refactor - Need Feedback)

[iMrShadow]

Current Situation

Currently, version v2.5.1 is serviceable for the most popular titles. Last year I announced this would be the last version of the tool, however, there are leftover issues which impact the current user experience. The tool feels unpolished - there are GUI inconsistencies, missing basic UX features and bugs. From the developer side, the codebase became harder to maintain. I've decided that I want to deal with these issues, since the end results would also transfer to Telltale-Editor, so I am very open to feedback.

Index

1. Code refactoring
   1.1 Developing a stand-alone texture library / rewriting texture container storage
   1.2 Developing a Telltale Tool library with extensions
   1.3 Utilizing .NET more effectively
2. Issues Status
   2.1 Features
   2.2 GUI changes
   2.3 Bugs

[iMrShadow]

1. Code refactoring

A major code refactoring has already started (see commits after v2.5.1). The end goal should result in more maintainable, performant and reliable codebase, while also contributing to Telltale-Editor.

1.1 Developing a stand-alone texture library / rewriting texture container storage

Currently, the tool is using Hexa Engine's DirectXTex Wrapper for decoding and manipulating the most popular image formats. At the same time, it made the project heavily dependent on the library. This brings some issues:

• It's harder to add support for other file types - KTX, KTX2 and Font files support cannot be added trivially.
• Limited pixel format support - DirectXTex only supports DXGI formats. While these formats are sufficient for Desktop games, for mobile and console platforms it is not.
• Cross platform inconsistencies - DirectXTex's image writer outputs inconsistent data on Linux and MacOS.
• Memory leaks - DirectXTex is a C++ library. Although memory leaks are rare, those that do crash the whole application.
• Performance issues - On load, D3DTX files are first internally converted to DDS, then they are loaded in DirectXTex, and finally they are decoded. This method slows down image loading and increases memory usage. (Edit: The effects of the new loading effect are noticeable. In stress tests (using gobo textures), RAM usage was reduced by 30% and the loading speed was halved, relatively.)

Creating my own texture library should fix these issues, but there will be downsides: the codebase won't be reliable, there will be bugs, and I will still rely on DirectXTex for utility functionality, like generating mips.
Edit: I am also wondering if I should remove the mipmap generation feature. People do seem to use it, but at the moment it's hardcoded to the 'Box' filter type. Additionally, there's also the auto-compression feature, which sometimes it works, other times it doesn't. Sorry MCSM modders, I am aware you hate that feature.

1.2 Developing a Telltale Tool library with extensions

This is a big one - developing a stand alone library has its benefits. The goals for the library will be:

• Unified file handling
• Extensibility
• Safe and reliable data modification (encrypting/decrypting will be easier)
• Loose coupling - The repo will be cleaner and other projects can use it
• Gateway to more features (ttarch/ttarch2 reading and extracting)
• Unit testing

In a nutshell, in the Texture Tool I want to able to open the D3DTX, modify the data, and save. I don't want to maintain dozens of different classes and 2000 lines function. This lead to some annoying present bugs: missing a single line of code breaks the serialization or the debug printing process, forgetting to add additional 4 bytes to some calculation functions or forgetting to edit a string for the debug printer. The library will also help with documenting the metadata.

The development of the library will be reflected on Telltale-Editor.

1.3 Utilizing .NET more effectively

This is a strange bullet point. In a nutshell, the .NET ecosystem has large amount of quality libraries. Some examples including: multi-threading, JSON, LINQ, Reflection and others. This will result in better code in terms of performance, maintainability and readability. One example is the FileSystemWatcher. This class alone fixes most of the Texture Tool's File Viewer bugs. (For e.g. "ghost files", files that are cached in the 'File Viewer' but no longer exist.)

[iMrShadow]

2. Issues Status

I have curated a list of issues, which is still growing, from either Github issues or from the Discord server.

2.1 Features

• Wii U, Wii and PS2 swizzling support
• Decrypting and encrypting D3DTX's (semi-done)
• Directly Open TTArch2/TTArch with options for extraction
• Old Xbox 360, PS4, iOS texture support
• Font files support?
• Reduce dependencies on JSON files? (only store needed data for reading). Extracting additional data should be optional. We already have a debug print output.

2.2 GUI

• Open File option
• Add file filter (search bar)
• Rework the File Explorer
• Add column filters (currently there's a hidden 'Date Modified' column, because I never bothered to remove it...) -
• Rework Image Properties panel - show more consistent and relevant data
• Rework Advanced Options panel
• Rework Menu - return to more traditional menu
• Rework Theme - make widgets more consistent
• Cubemap and volumemap views
  • Option to toggle views
• Option to disable mip viewing (enabled by default)
• Add custom backgrounds (white, black, transparent, checkerboard)
• Migrate to toast notifications rather than message boxes

2.3 Bugs

• When picking a folder in the file explorer, the path has an additional separator
• Incorrect version 7 header size calculation (TWD: Michonne 2016, Borderlands 2021)
• Older telltale textures are not decrypted (CSI: Hard Evidence corrupted textures #16)
• Some swizzling methods do not work for some pixel formats (Xbox 360)