[COM] Communication issue with RIGOL oscilloscope (model DS1104z Plus) using pyvisa-py

[Fellypao]

Instrument details

• Model: RIGOL DS1104z Plus
• Communication: USB
• Link to the documentation: https://www.batronix.com/files/Rigol/Oszilloskope/_DS&MSO1000Z/MSO_DS1000Z_ProgrammingGuide_EN.pdf

Output of pyvisa-info

Machine Details:
Platform ID: Linux-5.15.38-x86_64-AMD_Ryzen_5_3500U_with_Radeon_Vega_Mobile_Gfx-with-glibc2.33
Processor: AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx

Python:
Implementation: CPython
Executable: /home/fellype/progs_wo_installer/spyder/bin/python
Version: 3.9.18
Compiler: GCC 11.2.0
Bits: 64bit
Build: Sep 15 2023 13:03:38 (#main)
Unicode: UCS4

PyVISA Version: 1.13.0

Backends:
ivi:
Version: 1.13.0 (bundled with PyVISA)
#1: /usr/lib64/libvisa.so.0.0.0:
found by: auto
bitness: 64
Could not get more info:
VI_ERROR_NSUP_ATTR (-1073807331): The specified attribute is not defined or supported by the referenced object.

py:
Version: 0.7.1
ASRL INSTR: Available via PySerial (3.5)
USB INSTR: Available via PyUSB (1.2.1). Backend: libusb1
USB RAW: Available via PyUSB (1.2.1). Backend: libusb1
TCPIP INSTR: Available
Resource discovery:
- VXI-11: ok
- hislip: disabled (zeroconf not installed)
TCPIP SOCKET: Available
VICP INSTR:
Please install PyVICP to use this resource type.
GPIB INSTR:
Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of functionalities.
No module named 'gpib'
GPIB INTFC:
Please install linux-gpib (Linux) or gpib-ctypes (Windows, Linux) to use this resource type. Note that installing gpib-ctypes will give you access to a broader range of functionalities.
No module named 'gpib'

Problem description:

Almost all the communication commands I've tested works fine with the DS1104z+ oscilloscope. The only one that is not working properly is the ':WAV:DATA?' command - see page 2-219 (235 in sequential counting) of the documentation. It looks to work when executed, but only 9 points are obtained and they do not look to be correct.
Everything works fine when using NI-visa instead of pyvisa-py (tested on windows 10).

Here follows the code I'm using in the tests:

import pyvisa as visa
resm = visa.ResourceManager('@py') # the ivi backend just doesn't work
osc = resm.open_resource(resm.list_resources()[0], timeout = 2000) # ivi backend fails here with "AttributeError: 'IVIVisaLibrary' object has no attribute 'viParseRsrcEx'"
osc.write(':WAV:MODE RAW')
osc.write(':WAV:FORM ASC')
osc.write(':WAV:STAR 1')
osc.write(':WAV:STOP 60000')
osc.write(':STOP')
osc.write(':WAV:SOUR CHAN1')
data = osc.query(':WAV:DATA?')[11:] #waveform data starts at position 11

[MatthieuDartiailh]

First of all, query is not the right way to retrieve binary values. You can have a look at PyVISA documentation on the topic: https://pyvisa.readthedocs.io/en/latest/introduction/rvalues.html

That being said USB support for Rigol device is challenging due to some deviation from the USBTMC norm. Could you enable logging (pyvisa.log_st_screen()) and share the result ?

[Fellypao]

First of all, query is not the right way to retrieve binary values. You can have a look at PyVISA documentation on the topic: https://pyvisa.readthedocs.io/en/latest/introduction/rvalues.html

Thank you for the tip. I started from there, but with 'CURV?' (not supported by the instrument) instead of 'WAV:DATA?'
Anyway, changing from 'query' to 'query_ascii_values' does not work as well: the command
data = osc.query_ascii_values(':WAV:DATA?', container=numpy.array)[11:]
returns an empty array.

That being said USB support for Rigol device is challenging due to some deviation from the USBTMC norm. Could you enable logging (pyvisa.log_st_screen()) and share the result ?

Yes, of course!
Here follows what you've asked:

Input:

import pyvisa as visa
visa.log_to_screen()
resm = visa.ResourceManager('@py') 
osc = resm.open_resource(resm.list_resources()[0], timeout = 2000) 
osc.write(':WAV:MODE RAW')
osc.write(':WAV:FORM ASC')
osc.write(':WAV:STAR 1')
osc.write(':WAV:STOP 60000')
osc.write(':STOP')
osc.write(':WAV:SOUR CHAN1')
data = osc.query(':WAV:DATA?')[11:] #waveform data starts at position 11
osc.close()

Output:

2024-02-15 10:27:19,500 - pyvisa - DEBUG - SerialSession was correctly imported.
2024-02-15 10:27:19,519 - pyvisa - DEBUG - USBSession and USBRawSession were correctly imported.
2024-02-15 10:27:19,537 - pyvisa - DEBUG - TCPIPSession was correctly imported.
2024-02-15 10:27:19,538 - pyvisa - DEBUG - GPIBSession was not imported No module named 'gpib'.
2024-02-15 10:27:19,538 - pyvisa - DEBUG - Created library wrapper for py
2024-02-15 10:27:19,538 - pyvisa - DEBUG - Created ResourceManager with session 9252453
2024-02-15 10:27:19,549 - pyvisa - WARNING - Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::3034::33107::???::0::RAW
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/tcpip.py:121: UserWarning: TCPIP::hislip resource discovery requires the zeroconf package to be installed... try 'pip install zeroconf'
  warnings.warn(
2024-02-15 10:27:20,553 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - opening ...
2024-02-15 10:27:20,568 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - is open with session 6072646
2024-02-15 10:27:20,571 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - reading 20480 bytes (last status <StatusCode.success_max_count_read: 1073676294>)
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/protocols/usbtmc.py:115: UserWarning: Unexpected MsgID format. Consider updating the device's firmware. See https://github.com/pyvisa/pyvisa-py/issues/20
  warnings.warn(
2024-02-15 10:27:20,932 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - closing
2024-02-15 10:27:20,933 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - is closed
2024-02-15 10:27:20,933 - pyvisa - DEBUG - Closing ResourceManager (session: 9252453)
2024-02-15 10:27:20,933 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - closing

And an additional test (just ignore it if not useful):
Input 2 (just changing from 'query' to 'query_ascii_values'):

import pyvisa as visa
import numpy
visa.log_to_screen()
resm = visa.ResourceManager('@py') 
osc = resm.open_resource(resm.list_resources()[0], timeout = 2000) 
osc.write(':WAV:MODE RAW')
osc.write(':WAV:FORM ASC')
osc.write(':WAV:STAR 1')
osc.write(':WAV:STOP 60000')
osc.write(':STOP')
osc.write(':WAV:SOUR CHAN1')
try:
  data = osc.query_ascii_values(':WAV:DATA?', container=numpy.array)
except Exception as error:
  print("\nAn error occurred:",type(error).__name__, "–", error)
osc.close()

Output 2:

2024-02-15 10:29:07,628 - pyvisa - DEBUG - SerialSession was correctly imported.
2024-02-15 10:29:07,645 - pyvisa - DEBUG - USBSession and USBRawSession were correctly imported.
2024-02-15 10:29:07,665 - pyvisa - DEBUG - TCPIPSession was correctly imported.
2024-02-15 10:29:07,666 - pyvisa - DEBUG - GPIBSession was not imported No module named 'gpib'.
2024-02-15 10:29:07,666 - pyvisa - DEBUG - Created library wrapper for py
2024-02-15 10:29:07,666 - pyvisa - DEBUG - Created ResourceManager with session 4019873
2024-02-15 10:29:07,676 - pyvisa - WARNING - Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::3034::33107::???::0::RAW
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/tcpip.py:121: UserWarning: TCPIP::hislip resource discovery requires the zeroconf package to be installed... try 'pip install zeroconf'
  warnings.warn(
2024-02-15 10:29:08,681 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - opening ...
2024-02-15 10:29:08,696 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - is open with session 7485427
2024-02-15 10:29:08,699 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - reading 20480 bytes (last status <StatusCode.success_max_count_read: 1073676294>)
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/protocols/usbtmc.py:115: UserWarning: Unexpected MsgID format. Consider updating the device's firmware. See https://github.com/pyvisa/pyvisa-py/issues/20
  warnings.warn(
2024-02-15 10:29:09,019 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - closing
2024-02-15 10:29:09,019 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - is closed
2024-02-15 10:29:09,019 - pyvisa - DEBUG - Closing ResourceManager (session: 4019873)
2024-02-15 10:29:09,020 - pyvisa - DEBUG - USB0::6833::1230::DS1ZC251902766::0::INSTR - closing

[Fellypao]

Just for info:
The TCPIP protocol with data = osc.query(':WAV:DATA?')[11:] works fine, but data = osc.query_ascii_values(':WAV:DATA?', container=numpy.array) or data = osc.query_ascii_values(':WAV:DATA?') returns an empty array.

[MatthieuDartiailh]

Sorry it appears I misread your code since I though you were retrieving binary data.

As suggested in the log it looks like you are getting some incorrect message which is a somewhat known issue for Rigol. Can you check if a firmware update is available ?

[Fellypao]

I've already checked for a firmware update. The device is already running the most recent (https://beyondmeasure.rigoltech.com/acton/attachment/1579/f-c9c0c4a9-722f-44b1-840e-a395ee6afb95/1/-/-/-/-/MSO_DS1000Z%20Release%20Notes.txt?sid=TV2:Q2jKaGpUj ).

[MatthieuDartiailh]

Ok. Could you place a breakpoint in the usbtmc.py file and look at what is the MsgId we get and that we do not expect (you can even make a PR to make this info part of the message if you feel like it).
Next i will look into why it is not covered by the Rigol quirk support we have.

[Fellypao]

Well, I'm not a python expert... so, I'll need more instructions in order to provide useful info.
Up to now, this is what I could do:

fellype@ideaplasma:oscilloscope_progs_for_rigol_ds1104z$ python 0_gh_issue_test.py 
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/tcpip.py:121: UserWarning: TCPIP::hislip resource discovery requires the zeroconf package to be installed... try 'pip install zeroconf'
  warnings.warn(
/home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/protocols/usbtmc.py:115: UserWarning: Unexpected MsgID format. Consider updating the device's firmware. See https://github.com/pyvisa/pyvisa-py/issues/20
  warnings.warn(
> /home/fellype/progs_wo_installer/spyder/lib64/python3.9/site-packages/pyvisa_py/protocols/usbtmc.py(119)from_bytes()
-> return BulkInMessage.from_quirky(data)
(Pdb) p msgid
57
(Pdb) p btag
54
(Pdb) p btaginverse
50
(Pdb) continue
fellype@ideaplasma:oscilloscope_progs_for_rigol_ds1104z$

[MatthieuDartiailh]

Thanks !
At this point I have no good solution, however it would be very helpful if you could do the following:

• retrieve or attempt to retrieve the same waveform using both TCPIP and USB (having both data to compare will be helpful)
• when the USB raise a warning, print also the raw data block so that I can try to poke around to figure out if we need a different workaround.

[Fellypao]

Here follows the data retrieved using both TCPIP (data_tcpip.txt) and USB (data_usb.txt). In both cases, the first 11 characters of the data block are included.

By comparing the data, it looks that when using USB the data is truncated at position 117 of the data string.

data_tcpip.txt
data_usb.txt

[MatthieuDartiailh]

Sorry for the slow pace here. The last weeks have been very busy. I agree with your diagnostic.

It is not a solution but you could try configuring your device to read binary data since in that case the information from the header will be used in place of using the hardware termination. Another workaround could be to specify read_termination="\n" to force pyvisa to look for it.

Those are only workaround but they could help in your case.

The root of the issue is that when we get a non conform packet we assume it is the last one which in your case is wrong. I am not sure what to do here. Could you print the raw response we get on line 482 in pyvisa_py/protocols/usbtmc.py (you can find this file in the site_packages folder of your Python installation).