Perf: Handle small requests more efficiently

[AlbertoVPersonal]

Expected Behavior

If it was possible, to reduce the time of the method GetBulkStateAsync lower than 10 ms.

Actual Behavior

The current operation takes about 64 ms to read four keys.

Steps to Reproduce the Problem

Scenario configuration

• MODE
  • self-hosted
• STATE STORE
  • MongoDB
• DAPR INFO
  • CLI version: 1.13.0
  • Runtime version: 1.13.4
• SYS INFO
  • OS: Windows 10 IoT 21H2

Description

In my scenario, I have profiled the code using the MiniProfiler.NET package.

IReadOnlyList<BulkStateItem> multipleStateResult;
using (profiler.Step("Bulk op"))
{
    multipleStateResult = await daprClient.GetBulkStateAsync(_daprStoreName, keys, parallelism: 2);
}

And one of the tests have delivered these results:

== APP ==  === PROFILING RESULTS ===
== APP ==
== APP == MYPC at 10/21/2024 6:58:21 AM
== APP == My Profiler Name 0ms
== APP == >> Main Work 98.84ms
== APP == >>>> Load state 66.67ms
== APP == >>>>>> Bulk op 66.1ms
== APP == >>>> Load dict 7.87ms

In this code we can conclude that:

• the time spent to my library and come back is very fast (0,57 ms aprox),
• the bulk operation is taking the most time.

I have done some tests using PowerShell calling to the State API and the results are much better (Duration: 00:00:00.0040811 ms).

$uri = "http://localhost:10001/v1.0/state/state.mongodb/bulk"
$headers = @{
    "Content-Type" = "application/json"
}
$body = @{
    keys = @("key1", "key2", "key3", "key4")
    parallelism = 20 # it is not relevant
} | ConvertTo-Json

# Capture the start time
$startTime = Get-Date

# Invoke the REST method
$response = Invoke-RestMethod -Uri $uri -Method Post -Headers $headers -Body $body

# Capture the end time
$endTime = Get-Date

# Calculate the duration
$durationMs = ($endTime - $startTime).TotalMilliseconds

# Output the response and duration
$response
"Duration: $duration ms"

Release Note

RELEASE NOTE: IMPROVEMENT Better performance in the GetBulkStateAsync operation.

[WhitWaldo]

As discussed in Discord, I believe the issue here is not specific to this method, but rather one of the overhead of using gRPC in the SDK for all requests where the issue author is instead opting to use the HTTP API for their PowerShell sample.

The 1.15 release will come with some (significant package updates)[https://github.com//pull/1366} which have hopefully themselves added performance boosts.

Further, given a large enough bulk operation (as with the amount of data transferred in other client operations), I would expect that there's a point somewhere that the gRPC operation becomes more performant than the HTTP approach (primarily as binary encoding is smaller), but I don't know where that is for each operation.

Regardless, this is worth keeping in mind when designing the next generation state stores to look at opportunities to handle small requests more efficiently.

[olitomlinson]

@AlbertoVPersonal How many times are you running this test in your .NET code?

I wonder if there is an initial penalty/tax on that first operation, and subsequent operations will be faster?

[AlbertoVPersonal]

Usually I run a minimum of ten times at least because it is my baseline in my benchmark project. For other tests, I usually run during 100 and 1000 iterations.

Yes, .NET has a penalty on the first run so by that reason I run it several times. And my times are an average.
Maybe your data could be better. It is a choice.

@olitomlinson