Weird NRPN Behavior?

[edonv]

In a project I'm working on, I'm receiving every NRPN MIDIEvent twice, once with dataLSB at 0, then exactly the same message with dataLSB at whatever the actual value is at (which is sometimes 0). At first, I figured it was unavoidable, so I started working on a way to filter out the "leading duplicates" before processing them. I just realized that it could be possible that it's MIDIKit. Is it possible that after MIDIKit parses the first 3 CC messages of the NRPN message, it thinks it's just an NRPN event without the 4th message? Then, when it receives the 4th CC message, it merges it with the 3-message-long NRPN event to make the full one? I know that the 4th CC message is optional for an NRPN message, so that's my best guess? For now, I'll stick with the filtering though.

Examples:

• nrpn(raw(param: [msb: 0x00, lsb: 0x49], dataMSB: 76, dataLSB: 0), change: absolute, chan: 0x0, group: 0x0)

• nrpn(raw(param: [msb: 0x00, lsb: 0x49], dataMSB: 76, dataLSB: 14), change: absolute, chan: 0x0, group: 0x0)

• nrpn(raw(param: [msb: 0x00, lsb: 0x48], dataMSB: 77, dataLSB: 0), change: absolute, chan: 0x0, group: 0x0)

• nrpn(raw(param: [msb: 0x00, lsb: 0x48], dataMSB: 77, dataLSB: 14), change: absolute, chan: 0x0, group: 0x0)

• nrpn(raw(param: [msb: 0x00, lsb: 0x49], dataMSB: 25, dataLSB: 0), change: absolute, chan: 0x0, group: 0x0)

• nrpn(raw(param: [msb: 0x00, lsb: 0x49], dataMSB: 25, dataLSB: 15), change: absolute, chan: 0x0, group: 0x0)

I'll add that I believe the device I'm working with uses MIDI 1.0, but I seem to get better parsing leaving the MIDIKit API settings at the default. Maybe I should switch to newer API with MIDI 1.0?

[orchetect]

What you're seeing is actually intended behavior - it's what Core MIDI does in a mixed MIDI 1.0 and MIDI 2.0 environment. MIDIKit is not parsing received RPN/NRPN messages in any special way internally.

The MIDI 2.0 Spec outlines conventions that must be implemented when translating messages between MIDI 1.0 and 2.0 and back again. Core MIDI implements these conventions under the hood.

• In MIDI 1.0, RPN/NRPN is a sequence of between 2-4 CC messages. They can be packed into a single packet or arrive in separate packets.
• In MIDI 2.0, RPN/NRPN is a single UMP packet that can contain all four param/data bytes.

According to the MIDI 2.0 Spec, when translating 1.0 → 2.0, what Core MIDI is doing is aggregating the first 3 raw MIDI 1.0 CC messages and producing a single RPN/NRPN MIDI 2.0 UMP packet (Param MSB/LSB and Data Entry MSB). When it sees a Data Entry LSB following, it sends the same packet again but including the LSB. This might feel unintuitive but it's by design. The Data LSB may not always be present, and they decided that this algorithm was best. Data Entry is a cumulative value as per the MIDI Spec, so this is not a bug or an error.

If you want to filter them until they contain non-zero data LSBs that's your prerogative -- that comes down to the specific behavior of the device you're talking to. When working in a purely legacy MIDI 1.0 API environment, you would have to parse it in a similar way any how since the CC messages all come in separately. You would have to make the decision to act upon MSBs or wait until LSBs show up.

So MIDIKit is just handling events as Core MIDI is providing them.

• When operating in legacy API (MIDI 1.0) mode, MIDKit simply receives the individual CC messages as-is. (CC 99, CC98, CC6, and CC38). They would be .cc() MIDIEvent cases, not .rpn(). .nrpn(). MIDIKit is not attempting to 'bundle' them up because the MIDI Manager is in legacy API mode where separate CC messages would be expected like this.
• When operating in default API (MIDI 2.0), MIDIKit simply receives UMP packets which always contain all four bytes. That doesn't mean all four bytes will be 'complete' when talking to a MIDI 1.0 device/endpoint, as described above due to Core MIDI's translation.

You can use whichever API mode is most convenient, but keep in mind that the legacy API has been deprecated for several years now and one day it may be completely obsoleted.

[orchetect]

I'm actually working on a refactor that implements this and it's working pretty well. I'll let you know when it's done.

[orchetect]

Do you want to try setting your MIDIKit dependency to branch rpn-nrpn-bundling and try this:

Wherever you are setting up your Input / Input Connection event receiver(s), add the .bundleRPNAndNRPNDataEntryLSB option.

try midiManager.addInput(
    name: "My Input",
    tag: "My Input",
    uniqueID: .userDefaultsManaged(key: "MyInput"),
    receiver: .events(options: [.bundleRPNAndNRPNDataEntryLSB]) { events in
        // ...
    }
)

This should introduce the effect of bundling the two UMP messages together (when a data entry LSB follows the MSB in a 2nd packet), but allowing RPN/NRPN messages with a data entry LSB of 0 if no 2nd packet arrives.

It does this by utilizing a very short 50ms holdback delay (which can be adjusted) only for RPN/NRPN messages with a data entry LSB of 0. If a 2nd message arrives immediately after it (of any kind, not just RPN/NRPN), there is no effective latency. And it does not affect any other MIDI event types.

[edonv]

I so so so appreciate the detail here! I'll give the branch a try next chance I get. I'm working on a package for interfacing with specifically the Allen and Heath Qu line of digital audio mixers via their MIDI spec, and I don't own one of them myself. I have to swing by my wife's job for real testing, so I'll check it out next time I'm there and report back. Meanwhile, your explanation has led me to realize that maybe each of those first NRPN messages that show 0 for dataEntryLSB might actually be nil? So maybe I could test to see if that's actually what's coming in, and I could then filter out NRPNs without a dataEntryLSB value, rather than comparing to previous message. I'll let you know!

[orchetect]

UMP will fill zeroes for Data Entry bytes so I'm afraid you will probably come up dry testing for nil. That was implemented in MIDIKit more for when using legacy API and talking to MIDI 1.0 devices, it was possible to transmit only some of the bytes. I imagine once legacy API is fully obsoleted, I would change the Data Entry bytes to non-Optionals.

[edonv]

Ah ok, that makes sense. That I'll test out my filtering and your filtering algorithms when I get the chance. Thanks again!

[edonv]

I did some testing, and yeah, the dataLSB comes in as zero no matter what. But my algorithm worked. I didn't test your branch, but I can next time, if you'd like. For my purposes, my algorithm works fine because I know that with the device I'm working with, I'll always get NRPNs with dataLSB, even when the dataLSB is intended to be 0.

[orchetect]

If it works for you great. If you want to test the branch let me know. I'll be merging it to main eventually because it seems like a useful feature.

[orchetect]

The feature branch will be merged with main and the new .bundleRPNAndNRPNDataEntryLSB receiver option will be a part of release 0.9.5.