Caching

[lightheaded]

Is your feature request related to a problem? Please describe.
I have a pretty terrible connection at home and often the Nordpool API requests time out. This results in my dashboard showing N/A for the electricity price, and what's more critical - some of my automations such as car charging and house heating optimization etc. don't run. I would prefer stale data over no data, as it's never incorrect - rather just not up to date. And here I mean the hour-by-hour prices. Obviously, if we have no data for the current hour, we should not show incorrect prices.

Describe the solution you'd like
The nordpool sensor should never be unavailable, but rather use last known state. Ideally, there would be a retry mechanism as well, so if a request times out, it tries x times again. Possibly with an exponential back-off (try again in 10 sec, 1 min, 5 min, 10 min, 30 min etc...)

Describe alternatives you've considered
I've been using the HTTP request sensor which I think handled timeouts better.

[Hellowlol]

Hi,

Thanks for your suggestions. The integration will never show stale data, I'll instead display an error than incorrect numbers. The integration already has retries, with exponential backoff and jitter, but the problem is that there isn't any connection error per ce but that the data is missing or incorrect. Currently, there is no easy way to detect whether the data are bad or not as we only do one pull per currency, meaning the data for zone kr.sand might be ok, but se4 is missing data.

I would love to get a debug log of the errors you get when you have a bad internet connection

Retries are handled in this line: https://github.com/custom-components/nordpool/blob/master/custom_components/nordpool/aio_price.py#L182

[lightheaded]

The integration will never show stale data, I'll instead display an error than incorrect numbers

I agree 100%. But if I have the following data (pseudo):

- time: 2023-06-11 00:00
   price: 0.1
- time: 2023-06-11 01:00
   price: 0.11
- time: 2023-06-11 02:00
   price: 0.12
- time: 2023-06-11 03:00
   price: 0.13
...
- time: 2023-06-11 23:00
   price: 0.15

Now the integration fetches data successfully at, say, 00:05. Then it tries again at 01:05 and gets a timeout. Although it has the data for the full day, it will clear the fetched time-price table and display an error. This is not desired. What I tried to convey is that I'd rather have this table stored and pull from it, rather than show Unavailable.

I'll try to get some logs and get more info on that.

[Hellowlol]

Thats not how the integration should work, it only poll data startup and when a data for a new day is released (ca 1315 cet) if there is a connection error the the integration should retry indefinitely with exponential back off and jitter.

logs would be awesome, thanks.

[degreal]

When you update HA, does nordpool load from cache or it queries it from the internet? I had data loaded but after update I lost price data for 3 hours after I reloaded integration manually. Not robust enough for automation.

2023-06-19 14:18:29.824 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:19:30.048 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:20:30.929 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:21:32.193 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:22:33.101 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:23:33.694 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:24:33.893 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:25:34.875 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: sensor.nordpool
2023-06-19 14:25:58.991 WARNING (MainThread) [homeassistant.components.sensor] Timed out adding entities for domain sensor with platform nordpool after 500s

[Hellowlol]

Its always load from internet. There is no cache. We fetch new price on restart, integration reload, and when new prices are released.

[degreal]

Its always load from internet. There is no cache. We fetch new price on restart, integration reload, and when new prices are released.

I restarted HA at least 100x yesterday during configuration. Doesn't it spam nordpool too much? I wonder what is the daily limit, maybe thats why nordpool timed out?

[myllo]

If I understand line 182 in the file aio_price.py correctly, the function checks if there is data having time stamp for the next date (line 177).

I my case, as I reside in Finland and we have an hour time difference to the Nordpool time, I always have the first hour value of the next day already occupied. In practise this means, that if the first connection attempt fails, first index of next day data is there (hour 00-01), thus data for next date fulfills the condition of line 182. Still all other values are missing (except value of the first index ) and no retries occur.

Please change the code to check if all index values are present and refetch the data if not.

[Hellowlol]

@myllo Please link to the code that you are referring to. I cant find it.