pyocd remote protocol

[koendv]

Hi. I have a question about the pyocd remote protocol. (tcp_probe_server.py and tcp_probe_client.py)
What would be needed to be able to implement pyocd remote protocol in a microcontroller?

At this moment, the length of write_block32 and write_block8 is unlimited. That is a problem if memory is limited, as in microcontrollers.

There are two possibilities to process json:

• the server reads the whole json request in a buffer before processing.
• the server does not wait until the end of the json request to start processing, but processes the json request as it is received.

If the server reads the whole json request in a buffer before processing there ought to be a way for the server to tell the client the maximum json request length.

If the server processes the json request as it comes in, then the receiving buffer can be quite small, but then the order of the attribute-value pairs in the json request is no longer arbitrary. To process json as it comes in, "request" has to come before "arguments"; "address" has to come before "data", etc. The order as specified in remote_probe_protocol is fine; the only thing that needs to be added to the protocol spec is that the client has to send the attribute-value pairs in the order specified.

The most general solution would be to do both - server tells client maximum json length, and order of attribute-value pairs is specified.

Opinion?

koen

[flit]

Hi @koendv, a very interesting question!

First, let me ask you some questions:

• What is your use case?
• Over what transport (USB, Ethernet, etc) will the client communicate?
• What is the downstream side? Are you driving SWD/JTAG, or controlling self-hosted debug, etc?

What I'm really asking is whether CMSIS-DAP might be a better fit for a protocol, depending on the answers to the above questions. Although CMSIS-DAP has currently only ever been implemented on USB, it is (mostly) meant to be transport-independent and would work across other transports like TCP/IP. The main benefits of CMSIS-DAP over the pyocd remote protocol are binary encoding, built-in packet size/count controls, and off the shelf implementations for MCUs. Although it doesn't have high level memory transfer commands, its transfer commands plus the MEM-AP register architecture make that unnecessary (I've looking into adding mem transfer commands, but the benefit would be below the noise level.)

That said, don't let me stop you from implementing the pyocd protocol on an MCU! 😄 I just want to help you make the best choice for your application.

Your concerns and suggestions are spot on. Probably the best option is to do as you suggest and implement both improvements.

A configuration/capabilities handshake should be performed immediately after connection. This could just be an extension of hello, or a second handshake. We should go ahead and specify max sizes for both directions, since notifications/events from server–>client are something planned for the future.

Another possibility, not exclusive of the others, would be to switch to a JSON-RPC compatible protocol. This would basically change from 1-line commands to a HTTP-style (or email) headers + content. A good example is Microsoft's Debug Adapter Protocol. It currently uses a single Content-Length header to specify the size of the JSON content. I'm not really recommending switching, but it's something I've thought about (especially when designing the protocol, of course), and there would be some potential benefits with the standardisation. It would also be useful to have reusable JSON-RPC infrastructure in pyocd, because at some point I want to implement Microsoft's DAP referenced above so pyocd can be a self-contained debugger.

Regardless, I won't have time to work on this for a while since I'm finishing up CMSIS DFP debug sequences for the 0.33 release. There is also a fairly sizable list of additional todos for the remote protocol. 😅

[koendv]

On Tue, 04 Jan 2022 12:05:45 -0800 Chris Reed ***@***.***> wrote: Hi @koendv, a very interesting question! First, let me ask you some questions: - What is your use case? - Over what transport (USB, Ethernet, etc) will the client communicate? - What is the downstream side? Are you driving SWD/JTAG, or controlling self-hosted debug, etc?

OK. esp32-c3 is a small risc-v processor with wifi, bluetooth, usb. 160MHz clock, 400 kbyte ram, a generous 4 mbyte flash. Runs micropython. If you try to turn this board in a debugger probe the old-fashioned way - writing code in c, rust or whatever - you will be busy forever. So what I propose is the following: I add a module "dap" to micropython, with three functions: dap.init() -- start dap device dap.deinit() -- stop dap device dap.process(req, resp) -- process dap request This "dap" module is in c, and bit-bangs gpios. I'd like to use free-dap. Free-dap has no bloat, and is not encumbered with an Arm EULA. What you get is micropython with a local cmsis-dap probe, no usb delay. Port tcp_probe_server.py to micropython, and you have a small, inexpensive probe that runs pyocd server over wifi. Add cmsis-dap over bluetooth hid, and you have a versatile tool. Opinion?

Regardless, I won't have time to work on this for a while since I'm finishing up CMSIS DFP debug sequences for the 0.33 release. There is also a fairly sizable list of additional todos for the remote protocol. 😅

I'm just dreaming out loud. best, koen

[koendv]

Maybe use two micropython platforms, one arm, one risc-v, to catch processor dependencies in the code.

[flit]

(Sorry for taking a few days to reply… back to work this week from holidays.)

That's actually a really cool idea! There are probably some really interesting things that could be done with having micropython in your probe. (I'm using micropython running atop Zephyr for a research project, but only for what are effectively benchmarks.)

Various notes/thoughts… (and I don't mean to sound negative! Just being pragmatic. 😄)

Fyi, the reference CMSIS-DAP code (in the CMSIS_5 repo) and DAPLink are both licensed with Apache 2.0, so there is no special Arm EULA. (This was not always the case years ago).

There was a successful project to port DAPLink to Zephyr, which of course supports the ESP32-C3 (I don't have a link handy though). So it's not clear whether micropython would actually gain you anything in terms of implementation effort. Flexibility is a different case.

What you get is micropython with a local cmsis-dap probe, no usb delay.

What use would that be without USB or wireless connection to a host for debug? It might be very good for manufacturing and provisioning, especially complex cases; except that this would require target support, flash programming algorithms, etc. But beyond that, the performance of micropython would (most likely) negate any benefit of local control of the DAP interface.

If I were to create such a tool, I'd run micropython on top the port of DAPLink to Zephyr. The CMSIS-DAP implementation would be hooked into USB, WiFi, and BLE HID. It would be interesting to see how difficult porting parts of pyOCD to micropython would be, to enable flash programming for manufacturing use cases. The pyocd remote protocol would be cool but not a top priority.

Regardless whether you end up using pyocd's remote protocol or CMSIS-DAP over a wireless connection, your suggestions for the remote protocol are excellent and will be implemented at some point sooner than later.

Best of luck! Let us know how it goes, and if you need any help just ask!

Cheers
-c