Skip to content

Latest commit

 

History

History
573 lines (462 loc) · 42.4 KB

README.md

File metadata and controls

573 lines (462 loc) · 42.4 KB

C64 for MEGA65 Regression Testing

Before releasing a new version we strive to run all regression tests described here. Since running through all the demos takes some serious effort, it might be that we are not always doing it.

Version 5.1 - June 28, 2024

Status Test Done by Date
Basic regression tests: Main menu AmokPhaze101 6/25/24
Basic regression tests: Additional Smoke Tests AmokPhaze101 6/25/24
HDMI & VGA AmokPhaze101 6/25/24
SID AmokPhaze101 6/25/24
C64 Emulator Test Suite V2.15 AmokPhaze101 6/25/24
Demos AmokPhaze101 6/25/24
Writing to *.d64 images AmokPhaze101 6/25/24
GEOS: REU (sim), GeoRAM (HW), mouse, disk write test AmokPhaze101 6/25/24
PLA Test AmokPhaze101 6/25/24
Dedicated REU tests AmokPhaze101 6/25/24
Dedicated hardware cartridge tests AmokPhaze101 6/25/24
Dedicated simulated cartridge tests AmokPhaze101 6/25/24

Basic regression tests

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Main menu

Work with the main menu and run software that allows to test the following and make sure that you have a JTAG connection and an active serial terminal to observe the debug output of the core:

  • Filebrowser
  • Mount disk
  • Load *.prg
  • Short reset vs. long reset: Test drive led's behavior
  • Stress the OSM ("unexpected" resets, opening closing "all the time" while things that change the OSM are happening in the background, etc.)
  • Play with the Expansion Port settings, start a hardware CRT and an emulated CRT (there are more detailed and dedicated cartridge tests later)
  • Flip joystick ports
  • Save configuration: Switch off/switch, check configuration
  • Save configuration: Switch the SD card while the core is running and observe how settings are not saved.
  • Save configuration: Omit the config file and use a wrong config file
  • CIA: Use 8521 (C64C)
  • Kernal: Test all Kernal variants including Jiffy DOS.
  • Audio Improvements
  • About and Help
  • Close Menu

Additional Smoke Tests

  • Try to mount disk while SD card is empty
  • Work with both SD cards (and switch back and forth in file-browser)
  • Remove external SD card while menu and file browser are not open; reinsert while file browser is open
  • Work with large directory trees / game libraries
  • Eagle's Nest: Reset-tests: Short reset leads to main screen. Long reset resets the whole core (not only the C64).
  • Giana Sisters: Joystick and latency
  • Space Lords: Support for 4 paddles

HDMI & VGA

All done by AmokPhaze101 on June 2024 (on R3A & R6)

HDMI

Test if the resolutions and frequencies are correct:

16:9 720p 50 Hz = 1,280 x 720 pixel
16:9 720p 60 Hz = 1,280 x 720 pixel
4:3  576p 50 Hz =   720 x 576 pixel 
5:4  576p 50 Hz =   720 x 576 pixel 

Test HDMI modes:

VGA

Switch-off "HDMI: Flicker-free" before performing the following VGA tests and check for each VGA mode if the OSM completely fits on the screen:

  • Standard
  • Retro 15 kHz with HS/VS
  • Retro 15 kHz with CSYNC

Make sure that the Retro 15 kHz tests are performed on real analog retro CRTs.

SID

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Identical to Version 5.0 on R3A - so we consider this as a success.

The folder sidtests in this repo contains all the test files including D420.d64 and D500.d64.

Writing to *.d64 images

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Identical to Version 5.0 on R3A - so we consider this as a success.

  • Work with Disk-Write-Test.d64 and create some files and re-load them
  • Try to interrupt the saving by pressing Reset while the yellow light is on. Do this with the OSM open and also with the OSM closed. Watch if the <Saving> is being influenced by the reset attempt.
  • Katakis: High score saving/loading

C64 Emulator Test Suite V2.15

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Different from V5.0 and V4.0 on R3A - However this was investigated GitHub issue #133 and since the involved change has no impact on the core behaviour, we consider this as a success.

Status Detail Done by Comment
Disc 1: Complete AmokPhaze101
Disc 2: From start to and incl. "Trap15" AmokPhaze101
Disc 2: "Trap16" AmokPhaze101 GitHub issue #133
Disc 2: "Trap17" AmokPhaze101 GitHub issue #133
Disc 2: "Branchwrap" to "MMU" AmokPhaze101
Disc 2: "CPUPort" AmokPhaze101
Disc 2: "CPUTiming" to "Cntdef" AmokPhaze101
Disc 2: "CIA1TA" AmokPhaze101
Disc 2: "CIA1TB" AmokPhaze101
Disc 3: "CIA2TA" AmokPhaze101
Disc 3: "CIA2TB" AmokPhaze101

Dedicated REU tests

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Identical to Version 5.0 on R3A - so we consider this as a success.

Demos

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Status Demo Comment
Dark Mights - Movie 32 CSDB
Expand by Bonzai CSDB
fREUd CSDB In the part with boucing balls all the backgrounds are screwed up. Same issue on MiSTer C64_20221117.rbf. Perfectly runs on real Commodore C64 with Ultimate Cartridge.
Frontier CSDB
Globe 2016 CSDB Takes a few minutes to be fully rendered
Life will never be the same Digidemo 286K_1 CSDB
Qi CSDB
REU demo Zelda CSDB
Treu Love CSDB Ensure to use this file: TreuLove_ForReal1750Reu.d64

Games

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Status Game Comment
Sonic The Hedgehog v1.2+5 CSDB
Creatures II +9Hi - Mystic CSDB
Exterminator_1991_Audiogenic_(REU) CSDB
From the West CSDB
Ski_or_Die_1990_Electronic_Arts_REU CSDB Takes ages to load but it's ok
Walkerz +3 CSDB

Dedicated hardware cartridge tests

All done by AmokPhaze101 on June 2024 (on R6 only)

Better than Version 5.0 on R3A - so we consider this as a success.

Status Test Comment
Using a correct core #0: Test if we can directly boot to a hardware cartridge
Using a correct core #0: Test if the hardware cartridge is ignored in case simulated REU or simulated cartridge is selected
Test a bunch of original old game cartridges Tested Grid Runner, Avenger
Test an old Ultimax game Tested Pinball Spectacular, SeaWolves
Test a bunch of new game cartridges Tested ZetaWing, Sam's Journey, Galencia, Soul Force, Robot Jet Action, Super Bread Box
Save game and load game to/from the original Sam's Journey cartridge
Final Cartridge III Not tested this time (last test: Version 5.0)
Action Replay Professional 6.0 Not tested this time (last test: Version 5.0)
Power Cartridge
Flash the EasyFlash 1CR with a small (<64k) and large (>512k) game and playtest these games
Flash the EasyFlash 3 with a small (<64k) and large (>512k) game and playtest these games
EasyFlash 3: Test all freezers that the EF3 supports as described in cartridges.md
Kung Fu Flash Success: No workaround needed on R6 in contrast to R3A
Work with GEOS and GeoRAM Not tested this time (last test: Version 5.0)
Xpander 3 with Datel Midi interface, Cynth Cart 64 and Midi Keyboard

Dedicated simulated cartridge tests

All done by AmokPhaze101 on June 2024 (on R3A & R6)

Identical to Version 5.0 on R3A - so we consider this as a success.

Status Game Name Cartridge Type Comment
Beamrider 0 - generic cartridge
Centipede 0 - generic cartridge
Gridrunner 0 - generic cartridge
Gyruss 0 - generic cartridge
Pac-Man 0 - generic cartridge
Action Replay v6.0 Professional 1 - Action Replay GitHub issue #69
Black Box V8 3 - Final Cartridge III
Final Cartridge III 3 - Final Cartridge III GitHub issue #71
Kung Fu Master 5 - Ocean type 1
Ghostbusters 5 - Ocean type 1
Batman The Movie 5 - Ocean type 1
Robocop 2 5 - Ocean type 1
Soul Force 5 - Ocean type 1
Codemasters - Fast Food, Pro Skateboard, Pro Tennis 7 - Fun Play, Power Play
Microprose - Soccer, Rick Dangerous & Stunt Car Racer 7 - Fun Play, Power Play
Colossus Chess, International Football & Silicon Syborgs 8 - Super Games
Vegetables Deluxe 8 - Super Games
Fiendish Freddy's Big Top o' Fun, Flimbo’s Quest, Klax & International Soccer 15 - C64 Game System, System 3
Last Ninja Remix 15 - C64 Game System, System 3
Myth - History in the Making 15 - C64 Game System, System 3
After the War 17 - Dinamic
Astro Marine Corps 17 - Dinamic
Narco Police 17 - Dinamic
L'Abbaye Des Morts 19 - Magic Desk, Domark, HES Australia
Archon II - Adept 19 - Magic Desk, Domark, HES Australia
Arkanoid - Revenge of Doh 19 - Magic Desk, Domark, HES Australia
Park Patrol 19 - Magic Desk, Domark, HES Australia
Super Bread Box 19 - Magic Desk, Domark, HES Australia
A Pig Quest 32 - EasyFlash
Bruce Lee II 32 - EasyFlash
Monstro Giganto 32 - EasyFlash
Muddy Racer 32 - EasyFlash
ZetaWing 32 - EasyFlash
Freaky Fish DX 60 - GMod2
Metal Warrior Ultra 60 - GMod2
Outrage 60 - GMod2
Polar Bear 60 - GMod2
Sam's Journey 60 - GMod2

Version 5 - June 23, 2023

Status Test Done by Date
Basic regression tests: Main menu AmokPhaze101 6/4/23
‼️ Basic regression tests: Additional Smoke Tests sy2002 6/19/23
HDMI & VGA AmokPhaze101 6/3/23
SID AmokPhaze101 6/3/23
C64 Emulator Test Suite V2.15 AmokPhaze101 6/4/23
Demos AmokPhaze101 6/19/23
Writing to *.d64 images sy2002 6/19/23
GEOS: REU (sim), GeoRAM (HW), mouse, disk write test sy2002 6/19/23
PLA Test sy2002 6/19/23
Dedicated REU tests AmokPhaze101 6/3/23
Dedicated hardware cartridge tests sy2002 6/19/23
Dedicated simulated cartridge tests AmokPhaze101 6/7/23

Basic regression tests

Main menu

Work with the main menu and run software that allows to test the following and make sure that you have a JTAG connection and an active serial terminal to observe the debug output of the core:

  • Filebrowser
  • Mount disk
  • Load *.prg
  • Short reset vs. long reset: Test drive led's behavior
  • Stress the OSM ("unexpected" resets, opening closing "all the time" while things that change the OSM are happening in the background, etc.)
  • Play with the Expansion Port settings, start a hardware CRT and an emulated CRT (there are more detailed and dedicated cartridge tests later)
  • Flip joystick ports
  • Save configuration: Switch off/switch, check configuration
  • Save configuration: Switch the SD card while the core is running and observe how settings are not saved.
  • Save configuration: Omit the config file and use a wrong config file
  • CIA: Use 8521 (C64C)
  • Kernal: Test all Kernal variants including Jiffy DOS.
  • Audio Improvements
  • About and Help
  • Close Menu

Additional Smoke Tests

  • Try to mount disk while SD card is empty
  • Work with both SD cards (and switch back and forth in file-browser)
  • Remove external SD card while menu and file browser are not open; reinsert while file browser is open
  • Work with large directory trees / game libraries
  • Eagle's Nest: Reset-tests: Short reset leads to main screen. Long reset resets the whole core (not only the C64).
  • Giana Sisters: Joystick and latency
  • Space Lords: Support for 4 paddles
Regression

‼️ Long reset does not reset Eagle's Nest any more: GitHub issue #79

HDMI & VGA

HDMI

Test if the resolutions and frequencies are correct:

16:9 720p 50 Hz = 1,280 x 720 pixel
16:9 720p 60 Hz = 1,280 x 720 pixel
4:3  576p 50 Hz =   720 x 576 pixel
5:4  576p 50 Hz =   720 x 576 pixel

Test HDMI modes:

VGA

Switch-off "HDMI: Flicker-free" before performing the following VGA tests and check for each VGA mode if the OSM completely fits on the screen:

  • Standard
  • Retro 15 kHz with HS/VS
  • Retro 15 kHz with CSYNC

Make sure that the Retro 15 kHz tests are performed on real analog retro CRTs.

SID

The folder sidtests in this repo contains all the test files including D420.d64 and D500.d64.

Writing to *.d64 images

  • Work with Disk-Write-Test.d64 and create some files and re-load them
  • Try to interrupt the saving by pressing Reset while the yellow light is on. Do this with the OSM open and also with the OSM closed. Watch if the <Saving> is being influenced by the reset attempt.
  • Katakis: High score saving/loading

C64 Emulator Test Suite V2.15

Identical to Version 4 - so we consider this as a success.

Status Detail Done by Date
Disc 1: Complete AmokPhaze101 06/04/23
Disc 2: From start to and incl. "Trap16" AmokPhaze101 06/04/23
Disc 2: "Trap17" AmokPhaze101 06/04/23
Disc 2: "Branchwrap" to "MMU" AmokPhaze101 06/04/23
Disc 2: "CPUPort" AmokPhaze101 06/04/23
Disc 2: "CPUTiming" to "Cntdef" AmokPhaze101 06/04/23
Disc 2: "CIA1TA" AmokPhaze101 06/04/23
Disc 2: "CIA1TB" AmokPhaze101 06/04/23
Disc 3: "CIA2TA" AmokPhaze101 06/04/23
Disc 3: "CIA2TB" AmokPhaze101 06/04/23

Dedicated REU tests

All done by AmokPhaze101 on 6/3/23

Demos

Status Demo Comment
Dark Mights - Movie 32 CSDB
Expand by Bonzai CSDB
fREUd CSDB In the part with boucing balls all the backgrounds are screwed up. Same issue on MiSTer C64_20221117.rbf. Perfectly runs on real Commodore C64 with Ultimate Cartridge.
Frontier CSDB
Globe 2016 CSDB Takes a few minutes to be fully rendered
Life will never be the same Digidemo 286K_1 CSDB
Qi CSDB
REU demo Zelda CSDB
Treu Love CSDB Ensure to use this file: TreuLove_ForReal1750Reu.d64

Games

Status Game Comment
Sonic The Hedgehog v1.2+5 CSDB
Creatures II +9Hi - Mystic CSDB
Exterminator_1991_Audiogenic_(REU) CSDB
From the West CSDB
Ski_or_Die_1990_Electronic_Arts_REU CSDB Takes ages to load but it's ok
Walkerz +3 CSDB

Dedicated hardware cartridge tests

All done by sy2002 on 6/19/23

Status Test Comment
Using a correct core #0: Test if we can directly boot to a hardware cartridge
Using a correct core #0: Test if the hardware cartridge is ignored in case simulated REU or simulated cartridge is selected
Test a bunch of original old game cartridges Tested Last Ninja Remix, Super Games, Wizard of Wor
Test an old Ultimax game Tested Pinball Spectacular
Test a bunch of new game cartridges Tested Muddy Racers, Sam's Journey, Soul Force, The Curse of Rabenstein, Wormhole
Save game and load game to/from the original Sam's Journey cartridge
Final Cartridge III Works with workarounds
Action Replay Professional 6.0
Power Cartridge
Flash the EasyFlash 1CR with a small (<64k) and large (>512k) game and playtest these games
Flash the EasyFlash 3 with a small (<64k) and large (>512k) game and playtest these games
EasyFlash 3: Test all freezers that the EF3 supports as described in cartridges.md
Kung Fu Flash using the workaround described in cartridges.md
Work with GEOS and GeoRAM

Dedicated simulated cartridge tests

All done by AmokPhaze101 on 6/7/23

Status Game Name Cartridge Type Comment
Beamrider 0 - generic cartridge
Centipede 0 - generic cartridge
Gridrunner 0 - generic cartridge
Gyruss 0 - generic cartridge
Pac-Man 0 - generic cartridge
Action Replay v6.0 Professional 1 - Action Replay GitHub issue #69
Black Box V8 3 - Final Cartridge III
Final Cartridge III 3 - Final Cartridge III GitHub issue #71
Kung Fu Master 5 - Ocean type 1
Ghostbusters 5 - Ocean type 1
Batman The Movie 5 - Ocean type 1
Robocop 2 5 - Ocean type 1
Soul Force 5 - Ocean type 1
Codemasters - Fast Food, Pro Skateboard, Pro Tennis 7 - Fun Play, Power Play
Microprose - Soccer, Rick Dangerous & Stunt Car Racer 7 - Fun Play, Power Play
Colossus Chess, International Football & Silicon Syborgs 8 - Super Games
Vegetables Deluxe 8 - Super Games
Fiendish Freddy's Big Top o' Fun, Flimbo’s Quest, Klax & International Soccer 15 - C64 Game System, System 3
Last Ninja Remix 15 - C64 Game System, System 3
Myth - History in the Making 15 - C64 Game System, System 3
After the War 17 - Dinamic
Astro Marine Corps 17 - Dinamic
Narco Police 17 - Dinamic
L'Abbaye Des Morts 19 - Magic Desk, Domark, HES Australia
Archon II - Adept 19 - Magic Desk, Domark, HES Australia
Arkanoid - Revenge of Doh 19 - Magic Desk, Domark, HES Australia
Park Patrol 19 - Magic Desk, Domark, HES Australia
Super Bread Box 19 - Magic Desk, Domark, HES Australia
A Pig Quest 32 - EasyFlash
Bruce Lee II 32 - EasyFlash
Monstro Giganto 32 - EasyFlash
Muddy Racer 32 - EasyFlash
ZetaWing 32 - EasyFlash
Freaky Fish DX 60 - GMod2
Metal Warrior Ultra 60 - GMod2
Outrage 60 - GMod2
Polar Bear 60 - GMod2
Sam's Journey 60 - GMod2

Additional tests of cartridge releases have been performed by AmokPhaze101 in May 2023.

Version 4 - November 25, 2022

Status Test Done by Date
Basic regression tests sy2002 11/24/22
C64 Emulator Test Suite V2.15 sy2002 11/19/22
Demos AmokPhaze101 October & November 2022
Disk-Write-Test.d64 sy2002 11/24/22
Dedicated REU tests AmokPhaze101 11/19/22
GEOS: REU, mouse, disk write test sy2002 11/24/22

How to interpret the test results

We consider the pattern of success (:white_check_mark:) and failure (:x:) in the Demos, the C64 Emulator Test suite and the dedicated REU tests (scroll down, see below) as the baseline for Version 4 and therefore as "success". Future versions must deliver the same - or better.

Basic regression tests

Main menu

Work with the main menu and run software that allows to test the following:

  • Mount disk
  • Filebrowser
  • Save configuration, switch off/switch, check configuration
  • Flip joystick ports
  • SID: 6581 and 8580
  • REU: 1750 with 512KB
  • HDMI: CRT emulation
  • HDMI: Zoom-in
  • HDMI: 16:9 50 Hz
  • HDMI: 16:9 60 Hz
  • HDMI: 4:3 50 Hz
  • HDMI: 5:4 50 Hz
  • HDMI: Flicker-free
  • HDMI: DVI (no sound)
  • VGA: Retro 15Khz RGB
  • CIA: Use 8521 (C64C)
  • Audio Improvements
  • About and Help
  • Close Menu

Additional Smoke Tests

  • Try to mount disk while SD card is empty
  • Work with both SD cards (and switch back and forth in file-browser)
  • Remove external SD card while menu and file browser are not open; reinsert while file browser is open
  • Work with large directory trees / game libraries
  • Eagle's Nest: Reset-tests: Short reset leads to main screen. Long reset resets the whole core (not only the C64).
  • Giana Sisters: Scrolling while flicker-free is ON/OFF, joystick, latency
  • Katakis: High score saving/loading
  • Smile to the Sky (demo): SID 8580 filters
  • Sonic the Hedgehog: REU
  • Space Lords: Support for 4 paddles

C64 Emulator Test Suite V2.15

Tested with 6526 CIA. We consider the following test pattern, i.e. "Disc 1 Complete" and Disc 2 "everything works but the below-mentioned exceptions" as "success" and our baseline for Release 4.

Status Detail Done by Date
Disc 1: Complete sy2002 11/19/22
Disc 2: From start to and incl. "Trap16" sy2002 11/19/22
Disc 2: "Trap17" sy2002 11/19/22
Disc 2: "Branchwrap" to "MMU" sy2002 11/19/22
Disc 2: "CPUPort" sy2002 11/19/22
Disc 2: "CPUTiming" to "Cntdef" sy2002 11/19/22
Disc 2: "CIA1TA" sy2002 11/19/22
Disc 2: "CIA1TB" sy2002 11/19/22
Disc 3: "CIA2TA" sy2002 11/19/22
Disc 3: "CIA2TB" sy2002 11/19/22

Dedicated REU tests

All done by AmokPhaze101 on 11/19/22

Demos

Status Demo Comment
Dark Mights - Movie 32
Expand by Bonzai
fREUd In the part with boucing balls all the backgrounds are screwed up. Same issue on MiSTer C64_20221117.rbf. Perfectly runs on real Commodore C64 with Ultimate Cartridge.
Globe 2016 Wait 7 minutes before rendering starts
Life will never be the same Digidemo 286K_1 Press SPACE after having swapped disk
Qi
REU demo Zelda Just scroll the map with joystick in port 2
Treu Love OK but not 100%: In the main first scroller sprites have horizontal white pixel lines when on left and right borders, while they should not. Same issue on MiSTer C64_20221117.rbf. Perfectly runs on real Commodore C64 with Ultimate Cartridge.

Games

Status Game Comment
Sonic The Hedgehog v1.2+5 Joystick in port 2, choose options with ARROWS and RETURN, accept to load full game into the REU when asked
Creatures II +9Hi - Mystic Impossible to load the game until the end. Same issues on MiSTer C64_20221117.rbf and real C64+Ultimate Cartridge.
Exterminator_1991_Audiogenic_(REU)
from_the_west[r] All is happening in REU (no disk access) but interraction is quite slow
Ski_or_Die_1990_Electronic_Arts_REU Joystick in port 2. Takes ages to load from disk to the REU on our core as well as on MiSTer and a real C64.
Walkerz +3 Joystick in port 2