NAO v6 dark image issue

[raymonlo-git]

I know that it may not be the right place to ask my question here but as I find no help from Softbank, so just try and see if I can get some help here. I have both NAO v5 and v6. Compared to v5, I find that, with the default settings, the images captured from both the V6’s cameras are pretty dark. I have to adjust the brightness setting in order to make the images bright. Is it normal? Do you have similar issue on your V6v

[rkost]

I am very sorry. I must have missed the notification on this one. Have you resolved your issue in the meantime? We had some serious problems with the v6 cameras, yes. I can ask my team what we changed in comparison to v5 if you still need that information.

[raymonlo-git]

Hi, My camera issue is still unresolved. It is good to know that there is solution to fix it! Could you kindly give me the information or method to fix the issue? Best Regards, Raymond

On 10 Apr 2020, at 5:50 AM, René Kost ***@***.***> wrote:

 I am very sorry. I must have missed the notification on this one. Have you resolved your issue in the meantime? We had some serious problems with the v6 cameras, yes. I can ask my team what we changed in comparison to v5 if you still need that information. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[rkost]

What settings do you currently use to setup the camera? Are you using v4l2 only? Maybe check out our code release which contains functions to modify camera register values that are not accessible via v4l2: https://github.com/HULKs/HULKsCodeRelease/blob/master/src/tuhhsdk/Hardware/Nao/v6/Nao6Camera.cpp
The register documentation can be found here. We disabled the digital effects to reduce image noise but that also disables various settings that are available via v4l2.

Currently we are using these default settings.

We also found out that the camera settings are not always applied. Have a look at our V4L2CtrlSetting class and it's implementation to get an idea what operations may fail and need to be retried. So maybe you already have a sane config, but it is not applied correctly.

[raymonlo-git]

Really thanks for your help!!

According to your default settings, you have auto-exposure turned off and have exposure fixed at 25000. Could you turn on your auto-exposure and see if the images captured are dark or normal?

Actually I have done quite some studies in the past few months and if my finding is correct, the issue is because incorrect default settings are being set to the following registers:

1. 0x3A0E (50Hz Max Bands in One Frame) and 0x3A0D (60Hz Max Bands in One Frame)
   At startup time, these two registers are set properly but once the camera is accessed, these two registers are set tot 1. They should have values set at least bigger than 3. If they are set to 1, the exposure time of the camera is reduced by 2/3 (or even more) and this is the main reason for the dark image problem.

2. 0x3A18 & 0x3A19 (AEC Gain Ceiling)
   This register bounds the gain the camera can have. It should be at least 32 or 64 but the default is 16, which is way too low.

3. 0x5583 & 0x5584 (Saturation) The saturation register may be set incorrectly too.

I had reported my findings to Softbank Robotics but they said they couldn’t repeat the dark image problem I have. So I don’t know if they will look into the issue seriously.

This is the reason I am eager to ask if there is other people who has the same dark image problem on their NAO v6. At least I want to see if the problem is unique to me or common.

[rkost]

We have played with auto.exposure turned on and I remember that we used it especially on locations with bad lighting. So that works just fine.
I currently don't have access to v6 robots (thanks COVID-19 🦄) but I don't think that we fiddled around with these registers or encountered such problems. Let me ask our vision guys...

[schmidma]

Our default settings enable auto exposure. The autoExposure value that can be set via V4L2 is a mode switch and 0 is enabled auto exposure!
These are the register values with our default settings applied and during image capture:

0x3A0E : 1
0x3A18 | 0x3A19 : 248
0x5583 | 0x5584 : 128 | 128 (please keep in mind, that our digital effects are entirely disabled and therefore saturation values do not have any effect)

Indeed i am able to reproduce the dark image issue! But i need to say that we never had any brighter images. The dark image was our default. Only with configuration of 0x3A0E set to 0x06 (data sheet default value) manually, the image pops in way brighter.
I skimmed the data sheet roughly and as i understood this register setting (50Hz Max Bands in One Frame) directly affects the exposure time. Is that correct?

[raymonlo-git]

I took my NAO v6 to another place to give it a test. It is a big room with proper lighting. Bright images could be captured (with luminance 0x56A1 > 40). It makes me believe that the default settings come with NAO v6 work in some very standard environments only. In a small room with furnitures in different colors, like mine, even with all the lights turned on, dark image is resulted (with luminance 0x56A1 < 20).

I had NAO v5 before and it didn’t have any dark camera issue and values of these two registers in NAO v5’s ov5640.ko are set bigger than one. So it should be the improper default settings come with NAO v6 which limit the range of exposure the cameras can have.

I do believe the 50Hz/60Hz Max Bands in One Frame registers shouldn’t be 1 as these two registers can control the maximum exposure time the camera can have and these two registers should change according to the screen size and fps of the camera. With bigger value set, bright images can be got.

Unfortunately the OV5640 cameras in NAO v6 are USB-based. The issue could be fixed in firmware level only. I have this finding, together with the low AEC Gain Ceiling issue, reported to Softbank Robotics. Hope that they will look into it seriously and ask Leopard, the camera manufacturer, to fix the firmware issue.