Provide emulated versions of IDE64 and/or SD2IEC and/or CMD-HD as drive #9, #10, etc.

[sy2002]

This topic came up in a discord chat:

The game that Boris talks about is this one: https://csdb.dk/release/?id=231208

And there is also an old entry in our ROADMAP.md that reads:

"Offer an SD2IEC compatible IEC device that browses the SD card and that you can for example use to flash the EF1CR cartridge without the need of splitting large CRTs into multiple D64 disks using EasySplit"

So to generalize this a bit: There are multiple solutions which offer subdirectory support and support for files that are larger than the capacity of an original disk such as IDE64, SD2IEC, CMD-HD .

At this stage I am not fully aware yet, how these devices interact with the C64 so that the C64 can seamlessly for example read 800kB+ PRG files. This issue needs a bit of research before we can get started. Here is what chatGPT spit out:

Yes, there are some commonalities in how SD2IEC, IDE64, and CMD-HD interact with the Commodore 64, but they are distinct devices with different protocols and functionalities.

1. SD2IEC:

   • Protocol: SD2IEC is a firmware implementation of a subset of the IEC protocol, which is the serial bus protocol used by Commodore disk drives.
   • Functionality: It allows the C64 to read from and write to an SD card as if it were a floppy disk drive. It supports common disk image formats like D64, D71, and D81.

2. IDE64:

   • Protocol: IDE64 uses its own proprietary protocol to interface with IDE hard drives and CompactFlash cards. It includes an integrated DOS (IDEDOS) to manage files.
   • Functionality: It allows the C64 to use IDE hard drives, providing a much larger storage capacity and faster access times compared to traditional floppy drives.

3. CMD-HD:

   • Protocol: CMD-HD (Commodore Hard Drive) also uses a proprietary protocol designed by Creative Micro Designs (CMD). It communicates with the C64 over the IEC bus but extends the protocol to handle hard drive operations.
   • Functionality: Similar to IDE64, CMD-HD offers large storage capacities and faster access times, but it uses SCSI or IDE drives with CMD's proprietary interface.

Emulation Approach

To emulate these devices, you would need to replicate their respective protocols and functionalities. Here’s a general outline:

1. SD2IEC Emulation:

   • Emulate the IEC protocol for basic communication.
   • Implement file system handling for FAT32 and disk image formats (D64, etc.).
   • Handle IEC commands like OPEN, CLOSE, READ, and WRITE.

2. IDE64 Emulation:

   • Emulate the IDE64 proprietary protocol.
   • Implement IDEDOS commands and file handling.
   • Manage interactions with virtual IDE devices or disk images.

3. CMD-HD Emulation:

   • Emulate the CMD proprietary protocol for hard drives.
   • Support CMD DOS commands and operations.
   • Handle SCSI or IDE drive emulation, possibly using disk images or virtual drives.

Common Aspects

While each device has its unique protocol and set of commands, they share some common aspects:

• File System: Most of them interact with FAT32 file systems in some manner, either directly (SD2IEC) or through an intermediary DOS layer (IDE64, CMD-HD).
• IEC Bus: SD2IEC and CMD-HD use the IEC bus for communication, though CMD-HD extends it significantly.
• DOS Commands: They have their own DOS implementations for file management (SD2IEC uses a subset of Commodore DOS, IDE64 uses IDEDOS, CMD-HD uses CMD DOS).

[Kugelblitz360]

Thank you for the writeup.

Approach One: Have a specific kernal for the C64 that uses the same jumppoints as the IDE64 routines. They are documented quite well here: https://www.ide64.org/ide_fix.html.
So basically the communication with the file system on the SD card can be any way that is easy to implement, we just need the same kernal calls. Games that were patched with the instructions will then just work. This does create something that is not really an FGPA implementation of real hardware, and that might be philosophically incorrect :-) On the other hand, an IDE64 machine runs a different kernel anyway and there would not be any software that requires a 100% implementation. AFAIK, if we go this route, we get SD2IEC compatibility for free (not for disk images but for file systems) as the calls implemented on the device side of SD2IEC are just a subset of the CMD-HD or IDE64 commands.

Approach Two: Build a complete SD2IEC device as FPGA code so that the standard kernel just works. Looks like a lot of work to rebuild a device that actually is not compatible with every C64 disk that exists. As there are many different SD2IEC devices anyway. And sort of SD2IEC implementations in U1541 et al. - and in theory you also would have to build a CMD-HD and an IDE64 too.

I think a special Kernel for the Core that is useless on real hardware would be a good compromise - especially as you only activate this Kernel when you would like to run an IDE64/SD2IEC optimized game.

[sy2002]

@Kugelblitz360 Thank you for the input - highly appreciated.

[Kugelblitz360]

To validate my thinking, I have just ordered a Pi1541 hat (have a couple of unused 3B+ anyway) to see if the incomplete SD2IEC setup works with my target games. If it does, just "fudging" a SD2IEC device with a specific kernal that talks to something that might not follow IEC protocols but understands the commands (in the CMD HD, IDE and SD2IEC variations, they just have slightly different syntax and you can just follow all variations) would be a solution that allows usage of stuff that comes in files.
Of course, using images of hard drives and/or partitions in the CMD format is still off limits for this solution and there would be SOME use cases outside of gaming for this.
"But Boris, if a Pi1541 does solve the problem, why even bother implementing something in the core?"

• It would be significantly faster because it could bypass the serial bus
• It keeps stuff inside the Mega65 sd cards and no external device
• We can capture device 8. The current implementation in the core does not allow a external device on 8.

[Kugelblitz360]

Okay, IDE64 would need a full emulation including copyrighted ROMs, especially for Games written for IDE64 specifically. These games are easy to identify - set up an IDE64 environment in VICE, start the game and set breakpoints for the DE00 to DFFF range. IDE64 has specific data read/write vectors set up in this space. If a programm calls $DEF4 or $DEF1, an SD2IEC can not capture the call, this requires the FULL emulation.
List of games that do this:

• Alter Ego IDE64 version by Grue
• Portal IDE64 version by Grue
• Wasteland IDE64 version by Grue
  ...you begin to see a pattern here.

In the end it looks like just using an SD2IEC device (they are really cheap) is a better solution than spending enormous amounts of work to add a virtual filesystem device into the Core. The low hanging fruit, a virtual drive #10 that can access the FAT file system and understands the commands to switch in to and out of directories, would still be a lot of work and I would deprioritize my own request.

[Kugelblitz360]

Okay, I see ONE possibility to hack this: Capture the Jumps into $DEF4 and $DEF1 and process them in the virtual file system.

On the other hand, I will now try my luck to hack into Grues code for these to create cartridges out of these games and approach this totally from the other side.