The "New Libraries" Update Thread

[adrianhall]

Hey folks,

This is a chronological (and - hopefully - frequent) update on where I am with the new client re-design. Everything is a reply - if you want to ask a question on what was done, then please do so as a comment on the reply.

[adrianhall]

Update for 8/2/2024

• Completed the DatasyncServiceClient, which includes a HttpClientFactory for building HttpClients (you can also use your own) and full OData LINQ coverage. While the full API is not there, the underlying functionality is available.

Next up:

• All the extension methods required for a full API coverage.
• Moving onto the OfflineDbContext work - specifically, the PullAsync() method.

[adrianhall]

Update for 8/16/2024

• Completed all the LINQ extension methods necessary to support the full API coverage
• Completed all the push logic (I skipped over the PullAsync method since push was easier to test)

Next up:

• The pull logic.
• Figuring out what is necessary to store and compare UpdatedAt fields in the database properly.

What can you do right now (after the open PR is merged):

• Create an OfflineDbContext with synchronizable and non-synchronizable entities.
• Use both byte[] and string versions in your synchronizable entities.
• When you SaveChanges(Async), the DatasyncOperationsQueue (which is visible) will be updated.
• PushAsync() with multi-threading (up to 8 threads) on the DbSet<>, or for a subset of entity types.

When creating an OfflineDbContext, you need to implement OnDatasyncInitialization to set up the remote service.

[adrianhall]

Update for 8/23/2024

• Completed all the Pull and Push methods (see below)
• Completed 100% of the push tests and about 75% of the pull tests

Next up:

• Complete the pull tests
• Add in the extension methods that handle convenience methods for pullasync

Once I've done that (and end of next week is looking really good right now), I'll be code complete on the library. What's left to do?

• All the integration tests for the client against a "real" server. (The real server is already there - the integration tests aren't)
• Documentation
• Samples and tutorial code
• A bug bash

I really want to get folks to sign up for helping with the bug bash. Can you commit to a few hours learning the new library and checking it out, with an intent to find the corner cases I didn't cover?

On to the new PullAsync() code, which is currently in PR being built. There are two methods of pulling data. First, the tested version:

public class MyDbContext : OfflineDbContext
{
    // Whatever is necessary to set up the DbContext
    // Don't forget to do what is necessary to store byte[] and DateTimeOffset for your situation

    protected override void OnDatasyncInitialization(DatasyncOfflineOptionsBuilder builder)
    {
        HttpClientOptions options = new()
        {
            Endpoint = new Uri("https://myendpoint.azurewebsites.net"),
            HttpPipeline = [ YourHandler ]
        };
        builder.UseHttpClientOptions(options);
    }
}

There is more you can do in the OnDatasyncInitialization - you can choose from a HttpClient, HttpClientFactory, or a basic endpoint, resulting in lots of ways to set up your HTTP pipeline. For each entity, you can add a relative URI (or an absolute URI) and a client name (used if you are using an IHttpClientFactory to generate clients). You can also specify a default query for each entity:

options.Entity<Movie>(cfg => 
{
    cfg.ClientName = "movies";
    cfg.Endpoint = new Uri("/tables/movies", UriKind.Relative);
    cfg.Query.Where(x => x.Rating != MovieRating.R);
});

Now the fun part:

PullResult pullResult = await context.PullAsync([ typeof(Movie) ], new PullOptions());

Yes, there will be convenience methods, so you can just do PullAsync() and so on - I just haven't included them yet. This will use the default query and provide a default queryId for you - incrementally pulling records from the remote service. I've parallelized different types and split the storage in the database from the service request, so there is a good amount of parallelization going on here when you pull everything.

If you want something move, you'll have to go to the configurator. I HAVE NOT TESTED THE CONFIGURATOR VERSION YET. However, it looks like this:

PullResult result = await context.PullAsync(cfg => {
  cfg.SetParallelOperations(8);
  cfg.AddPullRequest<TEntity>();   // Use the defaults
  cfg.AddPullRequest<TEntity>(e => {
    e.Endpoint = new Uri("/tables/foo", UriKind.Relative);
    e.HttpClient = new HttpClient();
    e.Query.Where(x => x.Rating == MovieRating.G);
    e.QueryId = "myQueryId";
  });
}); 

This shows you both the basic version (where you use all the default) and the complex version (where you have absolute control over everything). There are reasonable defaults, so cfg.AddPullRequest<TEntity>(e => { /* Do nothing */ }); is the same as using the defaults.

I believe this API gives you considerable flexibility - first, you can do basic "give me everything" pulls really easily. But you can break out of that and modify every bit of the pull configuration if you want to get complicated.

Another thing I'm doing in the pull/push is getting rid of exceptions. In each case, there is a "FailedRequests" in the result object, together with an "IsSuccessful" flag. You can use these to generate your own exceptions if you desire, but you can also just handle the exceptions.

Hope you all have a wonderful weekend!

[adrianhall]

... and with that last checkin, we are "code complete" on the library!

What's left (as of now):

• Integration tests against a live service with the client.
• Samples
• Documentation

I'm going to work on a MAUI, WinUI3, and WPF sample next so you all have something to reference. However, the issues are now open for business - find a bug? Please send me an issue with as much detail as you can!