Refine workspace APIs and settings persistence

[pmconne]

User problem

Administrators of infrastructure projects want to be able to configure how applications work, including the resources available for their use. We refer to this configuration of settings and resources as a "workspace". Those familiar with MicroStation complain of a lack of control over their workspace configuration. We want to offer them better, easier-to-understand tools when working with iTwins and iModels. For example, iTwin workspaces should meet the following requirements:

• Prohibit ordinary (non-admin) users from modifying the workspace.
• Permit reuse of common settings and resources across different contexts (e.g., all iModels within a specific iTwin; or all iTwins of a specific project type) without duplication.
• A single app in which administrators can configure workspaces across their organization with clear descriptions of available settings and possible values.
• Administrators can quickly push workspace configuration changes to their users without significantly disrupting their work.
• It must be possible to configure settings outside of the context of an iModel.

General solution

We have implemented workspace and settings APIs that store resources in versioned distributed cloud databases. Settings are configured at run-time to point to those resources. Settings can be defined at various levels, in order of ascending priority: defaults, application-supplied, organization, iTwin, branch, and iModel. Settings of levels iTwin and above can be stored in an iModel's property table and automatically loaded when the iModel is opened. Settings of organization level are not stored anywhere. Default and application-supplied settings are delivered with the application.

Really, there are two workspaces:

• IModelHost.appWorkspace comprising application defaults and organization-level settings; and
• IModelDb.workspace comprising the rest of the settings, in the context of a single iModel. It includes and in some cases overrides the settings defined in IModelHost.appWorkspace.

Problems with current APIs and implementation

The current implementation contradicts some of the design goals.

• core-backend provides Settings APIs that any app or application can use to modify the in-memory contents of the workspace at run-time, violating the rule that only admins should have that ability.
• Any user (not just an admin) can modify the workspace by writing settings into an iModel's property table.
• iTwin- and branch-level settings must be written into the iModel's property table, because there is nowhere else to store them. This encourages duplication and risks them becoming out of sync. It would be very hard for an admin to update the iTwin-level settings across all iModels belonging to that iTwin - they would need to obtain the schema lock on each one, forcing all users to first push or abandon their local changes.
• There is no place to store organization-level settings.
• Terminology is confusing, even to the people who wrote or are using the APIs, let alone for end-users.
  • "WorkspaceContainer" sounds like it contains a workspace. It contains WorkspaceDbs, serving as the unit of access control.
  • People constantly conflate "WorkspaceDb" and "workspace".
  • "WorkspaceDb" is a technical term that doesn't clearly convey what it represents - a versioned, immutable library of resources.

Proposed adjustments

All settings are stored in a single place

Settings dictionaries are created by administrators. The administrator defines the settings for a particular object (iTwin, iModel, branch, or organization) by specifying the associated settings dictionaries. Some "workspace settings" API - details TBD - exists to persist and query those associations. Settings are never stored inside of iModels - remove those APIs.

Settings do not change after workspace initialization

Settings are configured by administrators. Users simply consume the workspace as defined by those settings - they cannot change the settings during the application session. Remove the APIs that currently permit that.

IModelHost.appWorkspace is initialized at startup from settings dictionaries supplied by the startup application and obtained for the organization from the workspace settings API. When a non-startup app is loaded, it can provide settings dictionaries for its own settings. The app workspace otherwise remains static for the duration of the session.

IModelDb.workspace is initialized when an iModel is opened. The workspace settings API is queried for the settings dictionaries from which to populate the workspace, which remains static thereafter. Any settings not defined in the iModel workspace fall back to the app workspace.

Applications that do not operate on iModels can create a Workspace from an iTwin. Like an iModel workspace, it is populated from settings dictionaries obtained from the workspace settings API and subsumes settings in the app workspace.

Clean up user-facing terminology

Come up with better names for WorkspaceDb and WorkspaceContainer.

[pmconne]

@Josh-Schifter @williamkbentley @kabentley @wgoehrig @bbastings interested in your feedback.

[kabentley]

core-backend provides Settings APIs that any app or application can use to modify the in-memory contents of the workspace at run-time, violating the rule that only admins should have that ability.

Do you propose that apps cannot add/modify/remove any setting at runtime? That seems drastic.

[kabentley]

Come up with better names for WorkspaceDb and WorkspaceContainer.

What do you propose? A WorkspaceDb is a SQLiteDb and they're stored in cloud containers. It won't be helpful to obscure those concepts.

[kabentley]

In general, I think locking down the concept that settings always come from WorkspaceDbs rather than "on the fly" or from within an iModel is a good idea.

However, there's the chicken/egg problem of associating (versions of) WorkspaceDbs with (a version of) an iModel. Where do you propose to persist that? I still think the best place is in the property table of the iModel itself.

[pmconne]

Thanks for your feedback, @kabentley.

What do you propose? A WorkspaceDb is a SQLiteDb and they're stored in cloud containers. It won't be helpful to obscure those concepts.

I'm primarily interested in the terminology end-users see (project administrators, not software developers), though @Josh-Schifter and @williamkbentley can attest to how frequently developers trip over the current terminology, and ideally the API names resemble the end-user concepts. The name should convey to the end-user what the thing is for, not its technical implementation details.

We're currently using "workspace resource" to refer to the discrete "things" we store inside WorkspaceDbs for use by applications. That's necessarily generic and I have no problem with it.

The closest analog to WorkspaceDb with which our users are familiar is a dgnlib, which tempts me toward terms like "resource library". I could see "resource database" or anything else that captures the concept of a collection of workspace resources.

Workspace containers group together and define access control for any number of WorkspaceDbs. I haven't come up with a satisfying alternative yet.

Do you propose that apps cannot add/modify/remove any setting at runtime? That seems drastic.

On what basis would the app be modifying the settings at run-time? How would the administrator be able to administer the settings effectively if the app can do whatever it wants?

In my opinion, a usable settings editor that an admin can use to manage the settings that define a workspace requires that each app declares in its manifest its available settings and default values - aka their settings schemas. The admin can override/customize/extend those defaults. FWIW that is how VSCode extension settings work.

There may be use cases for a startup app to customize settings for other apps - e.g., Booster customizing the settings for the drawing production app. But if so, I think those customizations belong in the manifest, not hidden away from the administrator in run-time code. Otherwise the admin will lack the sense of control over the configuration that we're trying to provide.

there's the chicken/egg problem of associating (versions of) WorkspaceDbs with (a version of) an iModel. Where do you propose to persist that? I still think the best place is in the property table of the iModel itself.

The property table is one answer to the specific question of where to store the association between an iModel and its settings. It does not solve the question of where to store the settings for an iTwin or an organization.

One could propose that the property table should also contain pointers to the settings for the iTwin and organization. But that still only solves the problem: given an iModel, what are its iTwin- and org-level settings? It doesn't answer the question: given an iTwin or org, what are its settings? A workspace administration tool based on defining settings at iTwin, org, and iModel levels requires that. Organizations necessarily exist before iTwins, and iTwins before iModels. Must I query some arbitrary iModel within the iTwin to look up a pointer in its property table to find the settings for an iTwin, and does anything enforce that every iModel in that iTwin contains the same pointer?

I fully expect us to develop authoring apps that operate on iTwins (reality data, GIS, etc) outside of the context of iModels, and admins to want to configure settings for an iTwin or organization before creating iModels within them. Hence the invocation of "some workspace settings API".

I think locking down the concept that settings always come from WorkspaceDbs rather than "on the fly" or from within an iModel is a good idea.

I didn't actually propose that the settings themselves are store in WorkspaceDbs, though that was my initial automatic inclination too, in part because the system currently permits you to point from the iModel's property table to settings dictionaries stored in WorkspaceDbs. But that seems unnecessarily self-referential and recursive. I know it gave @Josh-Schifter a headache.

Settings are the starting point. They define the workspace, and may point at WorkspaceDbs inside of WorkspaceContainers. I propose that settings dictionaries - and the associations between them an iTwins, iModels, and organizations - are stored in the "workspace settings API (details TBD)".

[Josh-Schifter]

I didn't actually propose that the settings themselves are store in WorkspaceDbs, though that was my initial automatic inclination too, in part because the system currently permits you to point from the iModel's property table to settings dictionaries stored in WorkspaceDbs. But that seems unnecessarily self-referential and recursive. I know it gave @Josh-Schifter a headache.

To be clear, my headache doesn't come from the settings stored in the workspaceDbs. Rather it bugs me that it is possible to store settings in the iModel's property table and that the only seemingly reasonable value for those properties is to point to the ones in the workspaceDbs.