CLIC Architecture tests

[dansmathers]

needed for definition of done. creating an issue for tracking

[allenjbaum]

The arch-test SIG is supposed to be designing infrastructure to test interrupts,
primarily a dummy device that can be programmed to issue interrupts at specific times
(as measured by a delta from the value of the instret counter at the time that the mmio write to the dummy device was executed)
The precise interface to this dummy device is TBD,
** we'd appreciate any thought on what this TG thinks is necessary.**

The other part of this is describing the scenarios that this TG thinks need to be tested to indicate architectural compatibility, e.g.

• simultaneous interrupts of different priorities from different modes with different priorities and different target modes
• different configuration values (especially extreme ones) and combinations of them,
• etc.

[dansmathers]

Adding 2021/10/12 discussion (from meeting minutes)
Philosophical Interrupt testing question. Fake interrupt testing device. Need a way at a future
time an event will be generated. interface is a macro probably be a memory mapped widget. as
long as interface is a macro that implementor can define. one possible implementation is a csr
write. e.g. Use for SAIL. spike can do that too. some CSR that can normally cause a trap. trap
handler redirects. most environments can handle a memory map. testbed implementation issue.

Mock-device at a memory map that can see write to memory. macro is the interface. in SAIL
can you call an external function, like floating point. if have tests that don’t need the macro.
some you want to cause interrupts and multiple interrupts. expose retired counter so can have
device schedule when event happen

[dansmathers]

Adding 2021/06/08 discussion (from meeting minutes)
Imperas shared short presentation how they test interrupts in verilog testbenches and ISS:

• Discussed external interrupt stimulator/interrupt agent/async event generator which acts
  as a virtual peripheral and the use of macros in test programs to allow different designs
  different methods to stimulate interrupts.
• Discussed separating DV testing from compatibility testing.
• Desire for at least some subset of compatibility tests to run on actual hardware/FPGA.
  and a class of tests that require something generating interrupts.
• Confirm with architectural test group that the direction they are taking still allows for
  implemented hardware to be tested as well.

[dansmathers]

Adding 2021/05/11 discussion (from meeting minutes)
Discussed compliance simulations. Use macros to setup interrupts to abstract interrupt setup
from test and would be platform responsibility to define the macros. Could also add a nondeterministic
compliance sim that does a repeating sequence of instructions and a sanitized trap
handler e.g., signature checks trap did happen and was right one but doesn’t check epc or other
values that could change based on implementation differences.

[dansmathers]

Adding 2021/04/27 discussion (from meeting minutes)
Current proposed test-plan-clic.adoc does simple, minimal testing. E.g. Only does
synchronous tests, tests if CLIC registers can be set, and checks if it vectors to the correct
address.

• But async behavior and debug are extremely important and where the bugs are. Bugs
  mean incompatibility and fragmentation. Isn’t it important that compliance covers items
  at highest risk of compatibility?
• Since every pipeline will take an interrupt in a different place, it can’t be verified
  trivially. For actual verification to know CLIC is functioning correctly, need to have a
  way of testing async behaviors. Should CLIC have a standalone test? Generic test
  harness? Unit testing of behavioral component called the CLIC?

[dansmathers]

Adding 2021/03/30 discussion (from meeting minutes)
Current framework: load a test (binary), start a hart, hart runs, hart signals finished,
extract results from memory to verify it ran correctly/check signature.

• how to handle async events with sail, implementations with varying behavior.
• Coverage discussion
• What can be leveraged from work teams have already done.

[allenjbaum]

My goal is to define a general event generator interface, for both AIA wired or message signaled interrupts (which would also subsume arbitrary MMIO reads and writes in order to test memory models). I actually don't know if CLIC requires something completely different than what AIA has, or if there is an architectural definition of what an interface to a hart is. My plan requires the implementation to export an additional set of signals, instret[ ], that indicates how many instructions have retired in that cycle. That lets a test schedule the order and timing of events in a deterministic fashion (tests may require a bit more work for superscalar implementations to be consistent....) Writing a single CSR (or MMIO) may not work for this general interface; you need to configure - when the interrupt will occur (in terms of #insts retired from the execution of the MMIO or CSR write that schedules it), - for wired interrupts, a wire# and possibly level/edge+polarity - for Message Signalled Interrupts (and arbitrary MMIO writes) it requires the physical address, and size and type of the transaction (possibly beyond just Rd and Wt) - for MMIO reads, it additionally requires the data, and a RW signal. That might be a sequence of writes to different MMIO addresses (e.g. one for data, one for address, and a 3rd setting the instret delta, event type {wire#, MMIOWt/Rd/other) and arming the event. To be useful, each of those command writes would be buffered up in fifos, and popped off when the event is asserted. (there are details to be worked out. In theory, multiple wires could be asserted simultaneously, but only a single MSI can be. Every cycle would decrement the captured deltas until they become <=0, at which time the event is asserted on the interface, and all associated data is popped off I do have an opinion as to whether the interface should be MMIO or CSR: There is almost certainly an MMIO interface to memory and devices, but there is not necessarily an external CSR interface. A CSR interface, then, likely requires a trap and emulate with MMIO anyway, and causes extra overhead both to configure the trap handler to special case that specific opcode with that specific CSR#. That extra latency from the CSR op may be implementation specific, and makes test development nearly impossible because the test can't rely on any specific instret delta

…

On Sun, Jan 9, 2022 at 11:46 AM Dan Smathers ***@***.***> wrote: Adding 2021/10/12 discussion (from meeting minutes) Philosophical Interrupt testing question. Fake interrupt testing device. Need a way at a future time an event will be generated. interface is a macro probably be a memory mapped widget. as long as interface is a macro that implementor can define. one possible implementation is a csr write. e.g. Use for SAIL. spike can do that too. some CSR that can normally cause a trap. trap handler redirects. most environments can handle a memory map. testbed implementation issue. Mock-device at a memory map that can see write to memory. macro is the interface. in SAIL can you call an external function, like floating point. if have tests that don’t need the macro. some you want to cause interrupts and multiple interrupts. expose retired counter so can have device schedule when event happen — Reply to this email directly, view it on GitHub <#186 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJXXVCLQDZDKQVD5B53UVHQ2HANCNFSM5KQ6YVCQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you commented.Message ID: ***@***.***>

[kasanovic]

TG Meeting 2/1/2022. - attendees felt above was sufficient for arch tests

[dansmathers]

Draft Pull request for Smclic, Ssclic testcases added to riscv-arch-test github. This should help enable spike/SAIL CLIC development.

riscv-non-isa/riscv-arch-test#436

[dansmathers]

architecture tests need to be created for ratification but don't not need to be accepted int riscv-arch-test. That work is part of ecosystem phase. Since they have been created, changed this label to post-v1.0 to indicate this is now an ecosystem task.

[allenjbaum]

Hmm; I wonder which branch this should be against: main, dev, or something new.... The intent now is all PRs are against dev, and maintainers will merge, the pull from dev (with the approprioate changelog), so we don't want to put things in dev that we don't intent to merge at the next merge point.

…

On Sat, Feb 10, 2024 at 6:47 AM Dan Smathers ***@***.***> wrote: architecture tests need to be created for ratification but don't not need to be accepted int riscv-arch-test. That work is part of ecosystem phase. Since they have been created, changed this label to post-v1.0 to indicate this is now an ecosystem task. — Reply to this email directly, view it on GitHub <#186 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJSYNIBTTFYDKDZ6XNDYS3DEBAVCNFSM5KQ6YVC2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJTGY3TSMJVG43A> . You are receiving this because you commented.Message ID: ***@***.***>