D1 Internal Alpha support (d1_devalpha)

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "quicktag"
-version = "0.4.1"
+version = "0.5.0"
 edition = "2021"
 
 [dependencies]
@@ -35,7 +35,7 @@ rayon = "1.8.0"
 base64 = "0.22.0"
 bincode = "2.0.0-rc.3"
 binrw = "0.13.3"
-destiny-pkg = { version = "0.9.7", features = ["bincode"] }
+destiny-pkg = { version = "0.10.0", features = ["bincode"] }
 serde = { version = "1.0.188", features = ["derive"] }
 serde_json = "1.0.108"
 vgmstream = { git = "https://github.com/cohaereo/vgmstream-rs/", version = "0.1.2" }

README.md

@@ -2,61 +2,88 @@
 
 # QuickTag
 
-QuickTag is a tool that scans and analyzes Tiger engine structure files (also known as 8080 files) to discover structures, patterns, strings and more.
-
+QuickTag is a tool that scans and analyzes Tiger engine structure files (also known as 8080 files) to discover
+structures, patterns, strings and more.
 
 ## How does QuickTag work?
-When first starting QuickTag (or after a game update), QuickTag goes through every package file and scans the data for file references, string hashes and raw strings. <!-- TODO(cohae): Document these -->
-It does this by checking file references and string hashes against a set of valid values, which is generated from the information [destiny-pkg](https://github.com/v4nguard/destiny-pkg) provides. It then stores the scanned information to a cache file so the next time QuickTag is started, it doesn't have to scan every package file again.
 
+When first starting QuickTag (or after a game update), QuickTag goes through every package file and scans the data for
+file references, string hashes and raw strings. <!-- TODO(cohae): Document these -->
+It does this by checking file references and string hashes against a set of valid values, which is generated from the
+information [destiny-pkg](https://github.com/v4nguard/destiny-pkg) provides. It then stores the scanned information to a
+cache file so the next time QuickTag is started, it doesn't have to scan every package file again.
 
 ### Tag Viewer
+
 ![tag view](./.github/readme_tag_view.png)
 
 #### References (left panel)
+
 *(TODO)*
 
 #### Tag Traversal (central panel)
+
 *(TODO)*
 
 Upwards traversal is not supported yet.
 
 #### Type analysis (right panel)
+
 *(TODO)*
 
 ### Named Tags
+
 *(TODO)*
 
 ### Packages
+
 *(TODO)*
 
 ### (Localized) Strings
-The strings tab shows any localized strings found in package files. These strings are referenced by a 32-bit FNV hash (note that this is not the hash of the string itself, but rather the hash of a localization key). Due to this, there will be multiple strings with the same hash, but different text. QuickTag will collapse these into a single entry, showing any strings that use this hash, as well as the tags that reference them. 
+
+The strings tab shows any localized strings found in package files. These strings are referenced by a 32-bit FNV hash (
+note that this is not the hash of the string itself, but rather the hash of a localization key). Due to this, there will
+be multiple strings with the same hash, but different text. QuickTag will collapse these into a single entry, showing
+any strings that use this hash, as well as the tags that reference them.
 
 > [!IMPORTANT]  
-> Note that due to the nature of FNV hashes, there will be a lot of false positives and overlapping strings in some cases.
+> Note that due to the nature of FNV hashes, there will be a lot of false positives and overlapping strings in some
+> cases.
 
-These strings only include the English version of the string. Other languages can be dumped through Quicktag, but they can't be used for the built-in search.
+These strings only include the English version of the string. Other languages can be dumped through Quicktag, but they
+can't be used for the built-in search.
 
 ### Raw Strings
-Like the strings tab, the raw strings tab shows any strings that are found with the raw string table tag (0x80800065). These strings are usually used for debugging purposes, and can't be found in-game. Unlike localized strings, which are referenced by a hash, raw strings are referenced by other data in the file that the string is in.
 
-QuickTag will remove duplicates and collapse them into a single entry. Just like the strings view, selecting a string will show the tags that reference it.
+Like the strings tab, the raw strings tab shows any strings that are found with the raw string table tag (0x80800065).
+These strings are usually used for debugging purposes, and can't be found in-game. Unlike localized strings, which are
+referenced by a hash, raw strings are referenced by other data in the file that the string is in.
+
+QuickTag will remove duplicates and collapse them into a single entry. Just like the strings view, selecting a string
+will show the tags that reference it.
 
 ### Asset preview
+
 QuickTag can also preview certain files, such as:
+
 - Textures (D1 (PS4 only) and D2)
 - WWise audio streams (D1+D2)
 
 ## Running
-Either download the [latest release](https://github.com/v4nguard/quicktag/releases) or [build QuickTag yourself](#building).
+
+Either download the [latest release](https://github.com/v4nguard/quicktag/releases)
+or [build QuickTag yourself](#building).
 
 QuickTag can then be run with the following command:
+
 ```sh
 quicktag.exe -v <version> <path to packages directory>
 ```
 
-Where `<version>` is the version of the game used by the given packages. These correspond to the ones used by `destiny-pkg`:
+Where `<version>` is the version of the game used by the given packages. These correspond to the ones used
+by `destiny-pkg`:
+
+- `d1_devalpha` Destiny 2013 Internal Alpha
 - `d1_ttk` "Legacy" version of Destiny (The Taken King)
 - `d1_roi` The latest version of Destiny (Rise of Iron)
 - `d2_beta` Destiny 2 Beta
@@ -69,14 +96,17 @@ Where `<version>` is the version of the game used by the given packages. These c
 > `d1_ttk` support is WIP
 
 ## Building
+
 Alkahest needs Rust 1.70 or newer to build. You can install Rust from [rustup.rs](https://rustup.rs/).
 
 QuickTag can then be built as follows:
+
 ```sh
 git clone https://github.com/v4nguard/quicktag
 cd quicktag
 cargo build --release
 ```
+
 The resulting binary will be located at `target/release/quicktag.exe`.
 > [!IMPORTANT]  
 > For performance reasons, it is recommended to only build QuickTag in release mode.

src/gui/tag.rs

@@ -87,7 +87,7 @@ impl TagView {
         let mut raw_string_hashes = vec![];
 
         macro_rules! swap_to_ne {
-            ($v:ident, $endian:ident) => {
+            ($v:expr, $endian:ident) => {
                 if $endian != Endian::NATIVE {
                     $v.swap_bytes()
                 } else {
@@ -97,14 +97,29 @@ impl TagView {
         }
 
         let endian = package_manager().version.endian();
-        let data_chunks_u32 = bytemuck::cast_slice::<u8, u32>(&tag_data[0..tag_data.len() & !3])
-            .iter()
-            .map(|&v| swap_to_ne!(v, endian))
-            .collect_vec();
-        let data_chunks_u64 = bytemuck::cast_slice::<u8, u64>(&tag_data[0..tag_data.len() & !7])
-            .iter()
-            .map(|&v| swap_to_ne!(v, endian))
-            .collect_vec();
+        let mut data_chunks_u32 = vec![0u32; tag_data.len() & !3];
+        let mut data_chunks_u64 = vec![0u64; tag_data.len() & !7];
+
+        unsafe {
+            std::ptr::copy_nonoverlapping(
+                tag_data.as_ptr(),
+                data_chunks_u32.as_mut_ptr() as *mut u8,
+                tag_data.len(),
+            );
+            std::ptr::copy_nonoverlapping(
+                tag_data.as_ptr(),
+                data_chunks_u64.as_mut_ptr() as *mut u8,
+                tag_data.len(),
+            );
+        }
+
+        for value in data_chunks_u32.iter_mut() {
+            *value = swap_to_ne!(*value, endian);
+        }
+
+        for value in data_chunks_u64.iter_mut() {
+            *value = swap_to_ne!(*value, endian);
+        }
 
         for (i, &value) in data_chunks_u32.iter().enumerate() {
             let offset = i as u64 * 4;
@@ -113,7 +128,7 @@ impl TagView {
                 array_offsets.push(offset + 4);
             }
 
-            if value == 0x80800065 {
+            if matches!(value, 0x80800065 | 0x808000CB) {
                 raw_string_offsets.push(offset);
             }
 

src/gui/texture.rs

@@ -204,7 +204,8 @@ impl Texture {
         }
 
         match package_manager().version {
-            destiny_pkg::PackageVersion::DestinyTheTakenKing => todo!(),
+            destiny_pkg::PackageVersion::DestinyInternalAlpha
+            | destiny_pkg::PackageVersion::DestinyTheTakenKing => todo!(),
             destiny_pkg::PackageVersion::DestinyRiseOfIron => {
                 let (texture, texture_data, comment) = Self::load_data_roi_ps4(hash, true)?;
                 Self::create_texture(
@@ -358,9 +359,12 @@ impl TextureCache {
             loading_placeholder: (Arc::new(loading_placeholder), loading_placeholder_id),
         }
     }
-    
+
     pub fn is_loading_textures(&self) -> bool {
-        self.cache.read().iter().any(|(_, v)| matches!(v, Either::Right(_)))
+        self.cache
+            .read()
+            .iter()
+            .any(|(_, v)| matches!(v, Either::Right(_)))
     }
 
     pub fn get_or_default(&self, hash: TagHash) -> LoadedTexture {

src/scanner.rs

@@ -120,7 +120,8 @@ pub fn scan_file(context: &ScannerContext, data: &[u8]) -> ScanResult {
             });
         }
 
-        if value == 0x80800065 {
+        // cohae: 0x808000CB is used in the alpha
+        if matches!(value, 0x80800065 | 0x808000CB) {
             r.raw_strings.extend(
                 read_raw_string_blob(data, offset as u64)
                     .into_iter()
@@ -163,16 +164,18 @@ pub fn read_raw_string_blob(data: &[u8], offset: u64) -> Vec<(u64, String)> {
     let mut c = Cursor::new(data);
     (|| {
         c.seek(SeekFrom::Start(offset + 4))?;
-        let (buffer_size, buffer_base_offset) =
-            if package_manager().version == PackageVersion::DestinyTheTakenKing {
-                let buffer_size: u32 = c.read_be()?;
-                let buffer_base_offset = offset + 4 + 4;
-                (buffer_size as u64, buffer_base_offset)
-            } else {
-                let buffer_size: u64 = c.read_le()?;
-                let buffer_base_offset = offset + 4 + 8;
-                (buffer_size, buffer_base_offset)
-            };
+        let (buffer_size, buffer_base_offset) = if matches!(
+            package_manager().version,
+            PackageVersion::DestinyInternalAlpha | PackageVersion::DestinyTheTakenKing
+        ) {
+            let buffer_size: u32 = c.read_be()?;
+            let buffer_base_offset = offset + 4 + 4;
+            (buffer_size as u64, buffer_base_offset)
+        } else {
+            let buffer_size: u64 = c.read_le()?;
+            let buffer_base_offset = offset + 4 + 8;
+            (buffer_size, buffer_base_offset)
+        };
 
         let mut buffer = vec![0u8; buffer_size as usize];
         c.read_exact(&mut buffer)?;
@@ -208,10 +211,7 @@ pub fn create_scanner_context(package_manager: &PackageManager) -> anyhow::Resul
     info!("Creating scanner context");
 
     // TODO(cohae): TTK PS4 is little endian
-    let endian = match package_manager.version {
-        PackageVersion::DestinyTheTakenKing => Endian::Big,
-        _ => Endian::Little,
-    };
+    let endian = package_manager.version.endian();
 
     let stringmap = create_stringmap()?;
 
@@ -417,7 +417,9 @@ pub fn load_tag_cache(version: PackageVersion) -> TagCache {
             };
 
             let mut all_tags = match version {
-                PackageVersion::DestinyTheTakenKing => [pkg.get_all_by_type(0, None)].concat(),
+                PackageVersion::DestinyInternalAlpha | PackageVersion::DestinyTheTakenKing => {
+                    [pkg.get_all_by_type(0, None)].concat()
+                }
                 PackageVersion::DestinyRiseOfIron => [
                     pkg.get_all_by_type(16, None),
                     pkg.get_all_by_type(128, None),

src/tagtypes.rs

@@ -155,6 +155,7 @@ impl Display for TagType {
 impl TagType {
     pub fn from_type_subtype(t: u8, st: u8) -> TagType {
         match package_manager().version {
+            PackageVersion::DestinyInternalAlpha => Self::from_type_subtype_devalpha(t, st),
             PackageVersion::DestinyTheTakenKing => Self::from_type_subtype_ttk(t, st),
             PackageVersion::DestinyRiseOfIron => Self::from_type_subtype_roi(t, st),
             PackageVersion::Destiny2Shadowkeep => Self::from_type_subtype_sk(t, st),
@@ -168,6 +169,17 @@ impl TagType {
         }
     }
 
+    pub fn from_type_subtype_devalpha(t: u8, st: u8) -> TagType {
+        match (t, st) {
+            (0, 0) => TagType::Tag,
+            (0, 2) => TagType::TagGlobal,
+            (0, 4) => TagType::Havok,
+            (15, 8) => TagType::WwiseBank,
+            (16, 8) => TagType::WwiseStream,
+            (ftype, fsubtype) => TagType::Unknown { ftype, fsubtype },
+        }
+    }
+
     pub fn from_type_subtype_ttk(t: u8, st: u8) -> TagType {
         match (t, st) {
             (0, 0) => TagType::Tag,