frostbite2-stuff

frostbite2-stuff copied to clipboard

\ | /

• < > - Frostbite2 Stuff | |

This repository contains libraries that access various formats that Battlefield 3 uses. Currently it reads most of the .sb/.toc/.m/.cas and .cat files as well as the usersettings of the game.

// Superbundle Files

- Overall Format

  I don't know how the engine calls these files internally
  but they are by far the most common file format that
  Frostbite 2 has.  They exist in two formats: unencrypted
  and signed and XOR encrypted.  To keep them apart from
  each other the engine seems to use the 4 byte header
  00D1CE00 at the beginning of files.

  If a file starts with 00D1CE00 a hash will be located at
  0x08 in the file and start with the ascii letter "x",
  then 256 ascii letters which are the hash and then again
  the letter "x".  What this hash is used for I do not know.

  The lookup table for the XOR encryption then is located
  at 0x0128 and is 257 characters long.  It is followed by
  XOR encrypted data.  For each character read the following
  decryption has to be applied:

    for (int i = 0; i < data_size; i++)
      data[i] = data[i] ^ magic[i % 257] ^ 0x7b;

  If the file does not start with that header the payload
  starts at byte zero directly.

- File Format Description

  The file can be described as a form of binary JSON.  It
  generally is a nested structure of dictionarys or lists
  and some scalar types as well as strings and blobs.

  Each object is prefixed with a typecode and some flags:

    uint8 byte = read_single_byte();
    int flags = byte >> 5;
    int typecode = byte & 0x1f;

  I don't know what the flags are doing but for reading the
  files it seems that you can ignore them.  The following
  typecodes exist:

    0           None / nil
    1           List
    2           Dict
    5           Unknown (8 bytes)
    6           Bool
    7           Binary String
    8           int32
    9           int64
    15          UUID
    16          SHA1
    19          Variable Length Quantity (varint)

  Strings are prefixed with the number of bytes as varint.
  Lists and dicts are prefixed with the number of elements
  they contain of but they are also delimited by `None` at
  the end.

  Dicts are interesting in that they move the typecode of
  the value of the dict before the key which means that
  you need to read "typecode key value" where value is no
  longer prefixed with a typecode.

// CAS / CAT files

- CAS File Format

  CAS files are the data source for everything the engine
  reads.  All the assets are stored in CAS files.  The CAS
  files by themselves can be dumped to the filesystem
  without the help of the CAT file but finding things in
  that file would be O(n) as such the CAT file exists which
  is a catalog of all the CAS files.

  struct entry {
      char header[4];
      char sha1[20];
      int32 data_length;
      char padding[4];
  };

  The header for all entries in the CAS file are always
  FACE0FF0.

- CAT File Format

  CAS Catalogs are very similar.  They start with the
  header "NyanNyanNyanNyan" and after that they list all
  the contents of all CAS files with the additional
  information of position, size and which CAS file they
  are from:

  struct entry {
      char sha1[20];
      int32 offset;
      int32 size;
      int32 cas_num;
  }

  The cas num is the name of the cas file.  "2" would
  indicate "cas_02.cas" etc.

// How Does The Engine Find Stuff?

- File Identifiers

  The sha1 hashes in the CAS/CAT are the SHA1 hash of the
  contents of the file.  This is also how the engine looks
  them up which indicates that it does not do a md5 hash
  of the filename.  The .toc/.sb files in combination have
  UUIDs for bundles which then have a bunch of SHA1 hashes
  which point to files in the CAS files.

  Additionally there is a bom.fb2 file which seems to be
  unused by the game.  Once decrypted with the algorithm
  from above it exposes a zip file made up of .m files
  which point to various parts in the archives.  Yet the
  patch does not have such a bom.fb2 file.