2015-10-20-dumping-the-undumped.markdown

layout	title	date	authors	tags
post	Dumping the Undumped	2015-10-20	endrift	hardware debugging

The Game Boy Advance is an old console. It's definitely nowhere near the oldest console out there, but it's had two successors by now, and certainly no one publishes games for it anymore, so it would seem reasonable to assume that each and every Game Boy Advance title has been dumped by one group or another. Maybe there's an odd edge case here or there for a Mary-Kate and Ashley game's European release, but everything good or interesting would be dumped within a few years of the console dropping out of popularity. Right? For the most part, right, but it always seems like there'll be a strange case here or there.

It's not unusual to see obscure or unlicensed hardware, like the Campho Advance to go undumped, due to the scarcity and complexity of the devices, but for everything less advanced, it's generally a simple matter of tossing together existing tools and getting the cartridge. So when it was brought to my attention that there were a handful of Game Boy Advance Video movies that had larger ROMs than any games on the market, and that they hadn't been dumped, my curiosity was piqued.

Initially I was skeptical. The Game Boy Advance has a 25-bit address space, so only 32 megabytes of data can be accessed, but claims were that these had 64MiB, 128MiB, or maybe even 256MiB ROMs. However, too-small address space hasn't stopped other game consoles from having cartridges with larger ROMs than are directly addressable by the console. In fact, older consoles, including the original Game Boy, all used mappers that enabled bank switching of the cartridge address space. Even some Game Boy Advance cartridges used bank switching to enable access to larger save files than the SRAM address space allowed.

I've seen dumps of Game Boy Advance Video cartridges floating around, and they were certainly in the No-Intro dats, but upon closer inspection, it turned out that indeed, the Game Boy Advance Video movies were conspicuously absent; only the TV show episode compilations were present. The cartridges were even called out explicitly as "undumpable with standard equipment" on the No-Intro forums. Well, now my curiosity was off the charts. "Undumpable", huh? That sounds like a challenge to me. And I certainly love a challenge.

I went on eBay and found that some of these Game Boy Advance Video movies were pretty cheap. It turns out that there were only mediocre Dreamworks films available, so I purchased a copy of Shrek 2 and excitedly waited for it to arrive. I even asked online how long people thought it would take me, and I estimated about two hours, since I figured it was just doing a pretty simple bank switch to swap out either part or all of the address space, perhaps with the lower 16MiB of the address space being all static, and the top 16MiB being remappable. Upon receiving the cartridge, I quickly dumped what was immediately visible to the Game Boy Advance without doing any special tricks, and I was quite surprised with what I found.

{: .hero }

The first 4kiB of the dump looked pretty usual. It had the usual cartridge header and a bit of initialization code. Nothing special there. I decided to drop the dump into mGBA to see what it did, expecting it to get to the menu screen and then be very confused by things not getting remapped as it expected. Nope, white screen. Well okay, I guess it maps stuff even sooner. mGBA logs when games try to write to the ROM address space, since it's generally read-only, although the bus does allow for data to be output to the cartridge. In fact, this is how devices like the realtime clock and solar sensor in Boktai work; a bit of data is written to the right address on the bus, and then the cartridge sees it and does a bit of I/O with a secondary chip that can produce dynamic data. It looked like the dump wrote four values to addresses 0x08800180 – 0x0880018C with specific values on startup. This was clearly how it did bank switching, so I took the values it was writing and modified the dumping tool to write these values to the ROM bus before dumping, and dumped the ROM again.

This time the dump got a bit further. It tried bank switching a few more times before it gave up and crashed. But I noticed something very odd. I figured there were only a handful of banks that it could switch to, but the commands it was writing to the ROM bus looked like it was doing much finer addressing than I figured the mapper would be able to manage. I took a deeper look into the dump in a hex editor, and I found something very surprising. There was a block of data that was 2kiB long that repeated every 4kiB for a good chunk of the dump, and everything above 8MiB + 512B was the 16-bit value 0xDEAD repeating over and over again, up until the 16MiB boundary. After that, it was all unmapped.

This was an extremely surprising result. Not only did the cartridge use just a little more than 8MiB of the 32MiB address space, but it was doing a much more complex mapping than I thought it would. This was going to take more than 2 hours to figure out. I started looking more closely at the values it was writing to the bus and looked at these addresses in the dump.

0x08800184 ← 0x10000A00
0x08800188 ← 0x08001800
0x0880018C ← 0x00000004
0x08800180 ← 0x00000011

The entire block of 0x08800000 – 0x08800200 appeared to be unmapped, with repeating 0xFFFFFFFF or 0xFFFFFFFE, implying that it had a much larger set of memory-mapped I/O registers than I observed being used, and that they were write-only. It only wrote to four addresses, and it wrote 32-bit values. The cartridge bus is only 16 bits wide, so this means it would actually be writing twice per 32-bit write. More interestingly, it was writing out of order. This looked like a command sequence, where writing to the first address in the block would actually commit the command. But given the value written, it looked like there might be other commands, too…

Subsequent writes always changed the value written to 0x08800184, implying that this value was the address in ROM to be mapped to the ROM bus. The value was always a multiple of 512, which seemed very small and fine-grained for mappable addresses, and the value written to 0x08800188 seemed to be an address on the ROM bus. Strange. But most interesting of all, the value written to 0x0880018C seemed to be a set of flags, perhaps for controlling how the mapping was done. This was clearly a more advanced mapper than I had been expecting.

Opening up the cartridge, I found something a bit surprising. There were two sizable chips, of different form factors, inside the cartridge. One was clearly the ROM. It was a long TSSOP chip, and looked like most other ROM or DRAM chips. But the second chip, clearly the mapper, was significantly larger than I expected. It was an LQFP chip, similar in size to the ROM chip. The two chips were labeled as Matrix Memory chips, but the ROM chip was labeled as "3D Memory", and the mapper was labeled as "Controller IC". There was quite a bit more going on here than I expected.

{: .hero }

Trying to find more information on these chips led me to a thread on AssemblerGames that was also looking into these cartridges. Unfortunately, they only had only a bit of information on the ROM chip, and no information on the "controller" chip, and there doesn't seem to be any publicly available information anywhere else. I'd need to do this by hand.

Indeed, for the next several hours, I spent a fair bit of time poking and changing the values that were written to the bus and arrived at some results. Firstly, the block size was 512 bytes, meaning that the smallest amount of information that could be mapped and the granularity with which the ROM can be addressed is 512 bytes. It also appeared that the value written to 0x08800188 was the base address onto the ROM bus to where the blocks should be mapped. Most mysterious was the flags write.

I perpetually ran into problems with having some blocks appearing to be mirrored and others not. Especially peculiar was the fact that, apart from a handful of bytes very early on, remapping the base 512 bytes of the cartridge appeared to be impossible. This led to a lot of confusion when I tried dumping multi-kilobyte regions of memory at once. Eventually, however, I realized that the flags write was not in fact a set of flags: it was a number of blocks to map at once. Tweaking the value anywhere from 1 to 8 had an obvious effect, and with this in mind, it was easy to map an entire 4kiB block at once, so long as I didn't try to map to the first virtual block.

Taking a hint from the cartridge's mapping locations, I finally ended up mapping a 4kiB region at 0x08001000, which worked fine. In fact, the next region worked fine too. The problems of repeated data in odd places had vanished, and consecutive blocks appeared to make sense. All the pieces had fallen into place, so now it seemed I could dump the cartridge. Moving things into a loop and dumping an entire 64MiB…success! The data ended at around 50MiB, with the remainder of the dump being entirely zeroes. The entire thing seemed to check out, although oddly, the first 4kiB was slightly shifted compared to what the cartridge looks like when not touching the mapping at all. But it did appear that the cartridge had successfully dumped. Even overdumping left the remainder zero. No more mirroring, no more garbage data.

So naturally, what I did next was toss it into mGBA to see what happened. It crashed immediately, but that was no surprise, considering the mapper wasn't implemented at all in mGBA. In fact, there are no mappers in mGBA at all. It might be interesting to see if the mapper could be implemented, but there are still many unknowns that could use investigation.

The first step of investigation might be to try changing the command write, the 0x11. But instead of doing that, I instead ordered another cartridge. This one claimed to be a combination of two movies, so I wondered if there might be any differences in the mapper or if the ROM was larger. Upon receiving it, I tossed it directly at my ROM dumper and…success! This ROM appeared to be almost exactly 64MiB. Overdumping yielded nothing past that block, implying that all of these use a 64MiB ROM chip. Given that there are only four of these titles, it seems all but guaranteed that they all have the same mapper.

While I have no intent of distributing these dumps (and I'm not sure what anyone would get out of them), the tool I wrote could definitely be of interest to any other dumpers and archivists out there, so it's been uploaded to GitHub under the name Duplo, released under a two-clause BSD license. While there aren't prebuilt binaries available, it should be easy to build for the DS using devKitPro. I'm sure No-Intro might be interested, since there are now tools to dump the "undumpable".