diff --git a/oeps/architectural-decisions/oep-0065-arch-frontend-composability.rst b/oeps/architectural-decisions/oep-0065-arch-frontend-composability.rst index 0e599c725..3f8b8cad6 100644 --- a/oeps/architectural-decisions/oep-0065-arch-frontend-composability.rst +++ b/oeps/architectural-decisions/oep-0065-arch-frontend-composability.rst @@ -29,7 +29,10 @@ OEP-65: Frontend Composability * - References - * `FC-0054 - Composable Micro-frontends Discovery `_ * `FC-0007 - Modular MFE Domains Discovery `_ - * :doc:`ADR 0001 - Create a unified platform repository ` + * :doc:`ADR 0001 - Create a unified platform repository ` + * :doc:`ADR 0002 - Frontend app migrations ` + * :doc:`ADR 0003 - Frontend projects ` + * :doc:`OEP-65 Frontend Glossary ` .. contents:: :local: @@ -38,14 +41,14 @@ OEP-65: Frontend Composability Abstract ******** -This OEP proposes that Open edX micro-frontends (MFEs) adopt shared dependencies and runtime module loading - implemented via webpack module federation - as an approach to improve the consistency, performance, and flexibility of the MFE architecture. +This OEP proposes that Open edX micro-frontends (MFEs) adopt :term:`shared dependencies` and :term:`runtime module loading` - implemented via webpack :term:`module federation` - as an approach to improve the consistency, performance, and flexibility of the frontend architecture. Motivation ********** Micro-frontends were originally designed to avoid some of the limitations of the monolithic ``edx-platform`` frontend, namely that otherwise independent teams in edX's large engineering organization were beholden to the build, test, and release lifecycle of the rest of the codebase. This dramatically slowed down the pace of frontend feature development, experimentation, and innovation on the master branch where the rate of change is particularly high. -As a result, the Open edX "micro-frontend" architecture focused on creating an un-opinionated, loosely coupled set of applications with the goal of enabling teams to iterate quickly and independently. This goal was successful. +As a result, the Open edX :term:`"micro-frontend" architecture ` focused on creating an un-opinionated, loosely coupled set of applications with the goal of enabling teams to iterate quickly and independently. This goal was successful. However, we've discovered that the completely siloed nature of these MFEs has created its own set of problems. Our frontends are more like independent single-page applications than `micro-frontends `_, as we never invested in ways of composing them together. The problems inherent in siloed MFE architecture are described below. @@ -85,7 +88,7 @@ Each MFE bundles all of its own dependencies, even if they're the same version a Full Page Refreshes ------------------- -Each MFE has its own index.html page, and needs to load all its own dependencies from scratch whenever you navigate to it. This means that the browser performs a full page refresh each time a user transfers from MFE to MFE. +Each MFE has its own ``index.html`` page, and needs to load all its own dependencies from scratch whenever you navigate to it. This means that the browser performs a full page refresh each time a user transfers from MFE to MFE. Build time ---------- @@ -106,7 +109,7 @@ The reality of MFEs is that while each application seeks to represent a single c Alternately, site operators may want to show different *versions* of MFEs to different users while keeping the rest of the app (header, navigation, other MFEs) unchanged. -There are alternatives to runtime module loading and shared dependencies which are used in some situations. These are not *rejected* alternatives, and so we include them here to help illuminate how their limitations are motivation for adopting runtime module loading and shared dependencies. +There are alternatives to :term:`runtime module loading` and :term:`shared dependencies` which are used in some situations. These are not *rejected* alternatives, and so we include them here to help illuminate how their limitations are motivation for adopting runtime module loading and shared dependencies. Shared Libraries (Alternative #1) ------------------------------------ @@ -116,7 +119,7 @@ Because each MFE is siloed from each other - both in repositories and at runtime Build-time package overrides (Alternative #2) ------------------------------------------------ -NPM and package.json allow site operators to override dependency resolution by installing an alternate version of a dependency prior to build-time. This has historically been how we've allowed operators to override the header, footer, and brand. +NPM and ``package.json`` allow site operators to override dependency resolution by installing an alternate version of a dependency prior to build-time. This has historically been how we've allowed operators to override the header, footer, and brand. The system is confusing, brittle, and only works at build-time. If a site operator needs different headers/footers/brands for different instances, this multiplies the number of required build processes for an instance. @@ -130,16 +133,16 @@ Direct plugins create some flexibility, but couple our repositories' builds toge Specification ************* -Our approach centers on enabling *runtime module loading* and *shared dependencies*. Together, these two capabilities address the majority of the motivating problems described above (Consistency, User and Developer Experience, and Composability). +Our approach centers on enabling :term:`runtime module loading` and :term:`shared dependencies`. Together, these two capabilities address the majority of the motivating problems described above (Consistency, User and Developer Experience, and Composability). We intend to enable runtime module loading and shared dependencies via `Webpack module federation `_. Further, we need to complement this new architectural approach with ways of *maintaining dependency consistency* between MFEs or we won't be able to realize the benefits of sharing dependencies. Capability: Runtime Module Loading ================================== -The capability to lazily load content from independently built modules into the page - without iframes - solves many of the `Composability`_ and `User and Developer Experience`_ issues above. In particular, it gives us a way of composing UI elements from different MFEs/Domains dynamically at runtime without a "host" application needing to know anything about the "guest" at build-time. The two remain completely decoupled, save some shared runtime configuration. It also cuts down on the number of full page refreshes experienced by users. +The capability to lazily load content from independently built modules into the page - without iframes - solves many of the `Composability`_ and `User and Developer Experience`_ issues above. In particular, it gives us a way of composing UI elements from different MFEs/Domains dynamically at runtime without a :term:`host` application needing to know anything about the :term:`guest` at build-time. The two remain completely decoupled, save some shared runtime configuration. It also cuts down on the number of full page refreshes experienced by users. -Not only do we want to load modules at runtime, but we want to *configure* which modules to load at runtime. This feature is called **runtime remote discovery**, i.e., discovering which "guests" exist, perhaps from some API like the MFE configuration API, at runtime as the application is loading. We believe this is a hard requirement of our runtime module loading implementation because we want to avoid rebuilding 'hosts' just because their module federation configuration changed. +Not only do we want to load modules at runtime, but we want to *configure* which modules to load at runtime. This feature is called **runtime** :term:`remote discovery`, i.e., discovering which "guests" exist, perhaps from some API like the MFE configuration API, at runtime as the application is loading. We believe this is a hard requirement of our runtime module loading implementation because we want to avoid rebuilding "hosts" just because their module federation configuration changed. It's worth noting that adopting runtime module loading has a high degree of overlap with the capabilities of the `frontend-plugin-framework `_ (FPF) and is a natural extension of its feature set. @@ -206,7 +209,7 @@ Advantages: Shared Dependencies Caveat: "build-time" and "dependency maintenance" ------------------------------------------------- -Note that "build-time" and "dependency maintenance" are not mentioned in the advantages above. The reasons for this have to do with how shared dependencies are resolved for modules at runtime, and are described in more detail in the `Approach: Webpack Module Federation`_ section below. +Note that "build-time" and "dependency maintenance" are not mentioned in the advantages above. The reasons for this have to do with how shared dependencies are resolved for :term:`modules ` at runtime, and are described in more detail in the `Approach: Webpack Module Federation`_ section below. At a high level, even if MFEs share dependencies, we want to preserve the ability for them to "fall back" to their own version of a shared dependency if a version already loaded on the page is incompatible with their own. To do this, each MFE builds and deploys its own version of all its dependencies in case dependency resolution determines they're needed. @@ -215,7 +218,7 @@ This means that the build of a given MFE has to take time to bundle their own de Approach: Webpack Module Federation =================================== -Module federation is implemented as a `plugin for Webpack `_ that enables micro-frontends to be composed into the same page at runtime even if they're built separately and independently deployed. The pieces being composed are "modules". It lets us configure which dependencies should be shared between modules on a page and what modules a particular frontend exposes to be loaded by other frontends. +:term:`Module federation` is implemented as a `plugin for Webpack `_ that enables :term:`micro-frontends ` to be composed into the same page at runtime even if they're built separately and independently deployed. The pieces being composed are :term:`modules `. It lets us configure which dependencies should be shared between modules on a page and what modules a particular frontend exposes to be loaded by other frontends. More information on module federation beyond its webpack implementation can be found on its `dedicated site at module-federation.io `_. @@ -245,8 +248,12 @@ Process We need to ensure maintainers and developers know what dependency versions to use, and when they need to upgrade to stay consistent. Open edX release documentation should include information on which frontend dependency versions are compatible with the release, likely pinned to a major version (i.e., React 17.x, Paragon 22.x, etc.) +Further, we recommend that each Open edX release have a single supported major version of all shared dependencies, and that all MFEs be upgraded to it prior to release. + We also need a process to migrate Open edX repositories through breaking changes in third-party dependencies. Ideally following the `Upgrade Project Runbook `_. +The :term:`module architecture` allows for migrations through breaking changes in third-party dependencies via the :term:`Linked Module` loading method, which will allow a frontend to run as separate, linked :term:`sites ` while migrating modules incrementally. + Best Practices -------------- @@ -284,16 +291,16 @@ Out of Scope There are a few important - but tangential - concerns which are considered out of scope for this OEP and its resulting reference implementation. -* Implementation details of how module federation would be added in the frontend-plugin-framework. -* How Tutor and other distributions will need to change to adopt module federation. -* Opinions on which dependencies we should adopt going forward (such as redux or other state management solutions) +* Implementation details of how module federation would be added in the `frontend-plugin-framework `_. +* How `Tutor `_ and other distributions will need to change to adopt module federation. +* Opinions on which dependencies we should adopt going forward (such as ``redux`` or other state management solutions) Rationale ********* The majority of the concerns expressed in the `Motivation`_ section revolve around a lack of shared dependencies and the way in which MFEs are currently siloed from each other, preventing us from creating a more seamless, cohesive experience. -Module federation specifically addresses these use cases exactly. It's right-sized to the problem at hand, can be accomplished with a minimum of impact on our existing MFEs, and can be done in a backwards compatible way (more on that below). +:term:`Module federation` specifically addresses these use cases exactly. It's right-sized to the problem at hand, can be accomplished with a minimum of impact on our existing MFEs, and can be done in a backwards compatible way (more on that below). An approach to maintaining dependency consistency is essential to realize the benefits of sharing dependencies. Without it, we've accomplished very little even though we've added the capability. An approach to providing this consistency is not a prerequisite for implementing module federation, to be clear, but the *success* of module federation is tightly coupled to it. @@ -304,7 +311,7 @@ We intend to maintain backwards compatibility while migrating to and adopting mo Ultimately MFEs will no longer be responsible for initializing `frontend-platform `_ or rendering the header and footer. We will follow the `DEPR process `_ for retiring this code in MFEs once (and if) we make the module federation architecture required. -In the interim, MFEs will have both a Webpack configuration that exposes modules for consumption by other hosts as alternate entry points (to use Webpack parlance) _and_ the primary entry point which initializes `frontend-platform `_ and loads the header/footer. The POC below suggests this won't be a problem. +In the interim, MFEs will have both a Webpack configuration that exposes :term:`modules ` for consumption by other hosts as alternate entry points (to use Webpack parlance) _and_ the primary entry point which initializes `frontend-platform `_ and loads the header/footer. The POC below suggests this won't be a problem. Reference Implementation ************************ @@ -321,23 +328,23 @@ Proposed MFE Architecture .. image:: oep-0065/proposed-mfe-architecture.png -Diagram description: A diagram showing the proposed MFE architecture using Webpack module federation (`LucidChart source`_). Contains the shell application and a "guest" MFE. Shows how the `Shell MFE`_ loads a manifest from MFEs (remoteEntry.js), and then uses that to load modules from the MFE, on demand, at runtime. The decision process around incompatible dependencies is shown, showing how an MFE that needs an incompatible version of a shared dependency loads its own version into the page as necessary - unless that dependency is a "singleton", in which case it will always resolve to the first version loaded. +Diagram description: A diagram showing the proposed MFE architecture using Webpack module federation (`LucidChart source`_). Contains the shell application and a "guest" MFE. Shows how the `Shell`_ loads a manifest from MFEs (remoteEntry.js), and then uses that to load modules from the MFE, on demand, at runtime. The decision process around incompatible dependencies is shown, showing how an MFE that needs an incompatible version of a shared dependency loads its own version into the page as necessary - unless that dependency is a "singleton", in which case it will always resolve to the first version loaded. MFEs and Modules ---------------- -Each of our MFEs will export a set of one or more modules that can be loaded by other MFEs or the `Shell MFE`_. For instance, ``frontend-app-profile`` would likely export the ``ProfilePage`` component. Other MFEs may export their own pages, or perhaps plugins/widgets/components to be loaded by the `frontend-plugin-framework `_ via a "module" plugin type based on this implementation. +Each of our MFEs will export a set of one or more modules that can be loaded by other MFEs or the :term:`Shell`. For instance, ``frontend-app-profile`` would likely export the ``ProfilePage`` component. Other MFEs may export their own pages, or perhaps plugins/widgets/components to be loaded by the `frontend-plugin-framework `_ via a "module" plugin type based on this implementation. Hosts and Guests ---------------- -MFEs become either "hosts" or "guests" or both. A host is an MFE that loads runtime modules from a guest. A guest may itself act as a host to modules from another guest. For example, the `Shell MFE`_ is only a host and all MFEs are guests in the shell; further, some pairs of MFEs might have a host/guest relationship with each other. +MFEs become either :term:`hosts ` or :term:`guests ` or both. A host is an MFE that loads runtime modules from a guest. A guest may itself act as a host to modules from another guest. For example, the :term:`Shell` is only a host and all MFEs are guests in the shell; further, some pairs of MFEs might have a host/guest relationship with each other. -Shell MFE ---------- +Shell +----- -We will create a new "shell" MFE to act as the top-level host for all other MFEs. It is exclusively responsible for: +We will create a new "shell" frontend to act as :term:`site` which hosts all the modules from our ``frontend-app-*`` repositories. It is exclusively responsible for: * Initializing the application via `frontend-platform `_. * Loading the default, expected version of all our shared dependencies. @@ -361,7 +368,7 @@ Converting the POC to a reference implementation To convert this POC into a reference implementation, we need to minimally: -* Create a new "shell" micro-frontend to be the top-level "host" for all our other micro-frontends. +* Create a new "shell" frontend to be the top-level "host" for all our other micro-frontends. * Create module federation-based development and production Webpack configurations in `frontend-build `_. * Modify the Webpack configuration to share the complete list of shared dependencies from the shell. * Pick an existing MFE (or two) to convert to use module federation. Add build targets to these "guest" micro-frontends that can be used to build them in module-federation mode. @@ -485,3 +492,9 @@ Change History ========== * Adding a reference to ADR-0001, which describes creation of a unified platform repository. + +2024-09-13 +========== + +* Updating language to match OEP-65's ADRs and leverage the frontend glossary. +* Adding a recommendation to standardize on a single major version of shared dependencies in a given Open edX release. diff --git a/oeps/architectural-decisions/oep-0065/README b/oeps/architectural-decisions/oep-0065/README index dc250bda2..7b46ad962 100644 --- a/oeps/architectural-decisions/oep-0065/README +++ b/oeps/architectural-decisions/oep-0065/README @@ -1,5 +1,5 @@ OEP-0065 File README -The source for the architecture diagram lives in LucidChart. +The source for the architecture diagrams lives in LucidChart. It can be found `at this LucidChart URL `_ and should be visible by everyone. If you need to edit the chart, either copy the chart into your own Lucid account or get in touch with the original authors if possible (David Joy - listed on the OEP) for access. diff --git a/oeps/architectural-decisions/oep-0065/decisions/0001-unified-platform-repository.rst b/oeps/architectural-decisions/oep-0065/decisions/0001-unified-platform-repository.rst index 1e8c00270..633931756 100644 --- a/oeps/architectural-decisions/oep-0065/decisions/0001-unified-platform-repository.rst +++ b/oeps/architectural-decisions/oep-0065/decisions/0001-unified-platform-repository.rst @@ -1,5 +1,5 @@ -OEP-65 Create a unified platform repository -########################################### +Create a unified platform repository +#################################### Status ****** @@ -9,12 +9,14 @@ Accepted Summary ******* -This ADR proposes combining a number of repositories into a single, unified frontend platform library as a migration strategy for implementing OEP-65. +This ADR proposes combining a number of repositories into a single, unified frontend platform library as a migration strategy for implementing :doc:`OEP-65 <../../oep-0065-arch-frontend-composability>`. This library is named `frontend-base `_. Context ******* -OEP-65 proposes adopting webpack module federation for Open edX micro-frontends (MFEs) as a means to enable *runtime module loading* and *shared dependencies* in the Open edX frontend. The OEP lays out a series of changes necessary to enable these capabilities, which it refers to as building a "reference implementation" of runtime module loading and shared dependencies. This reference implementation is effectively a new underlying architecture for our frontend. This ADR refers to this new architecture as the "module MFE" architecture, as opposed to the historical "application MFE" architecture that has existed prior to OEP-65. +OEP-65 proposes adopting webpack :term:`module federation` for Open edX :term:`micro-frontends ` (MFEs) as a means to enable :term:`runtime module loading` and :term:`shared dependencies` in the :term:`Open edX frontend`. The OEP lays out a series of changes necessary to enable these capabilities, which it refers to as building a "reference implementation" of runtime module loading and shared dependencies. + +This reference implementation is effectively a new underlying architecture for our frontend. This ADR refers to this new architecture as the :term:`module architecture`, as opposed to the historical :term:`micro-frontend architecture` that has existed prior to OEP-65. The bulk of the work to build OEP-65's reference implementation is related to the *library repositories* that MFEs depend on: @@ -24,79 +26,79 @@ The bulk of the work to build OEP-65's reference implementation is related to th * frontend-component-header * frontend-component-footer -It will also involve the creation of a new *shell application* to initialize the frontend at runtime and load existing MFEs as modules. Prior to this decision, we might have considered putting the shell application in its own repository as well: +It will also involve the creation of a new :term:`Shell` to initialize the frontend at runtime and load existing MFEs as :term:`modules `. Prior to this decision, we might have considered putting the shell in its own repository as well: * frontend-app-shell -There are several related concerns which influence our strategy for migrating these library repositories to the new module MFE architecture, and which influence where we decide to put the shell. +There are several related concerns which influence our strategy for migrating these library repositories to the new module architecture, and which influence where we decide to put the shell. Significant API changes ======================= -The creation of the shell application and the move toward using plugins and module federation is expected to significantly change the API surface across our various libraries. +The creation of the shell and the move toward using :term:`plugins ` and module federation is expected to significantly change the API surface across our various libraries. ``frontend-platform`` --------------------- -As the shell takes responsibility for much of what ``frontend-platform`` provides to MFEs in the application MFE architecture, many of the APIs it exposes will become internal to the shell. We will also likely need to add new APIs for interactions between the shell and MFEs. +As the shell takes responsibility for much of what ``frontend-platform`` provides to MFEs in the micro-frontend architecture, many of the APIs it exposes will become internal to the shell. We will also likely need to add new APIs for interactions between the shell and modules. ``frontend-build`` ------------------ -The development environment will also change, as we want to be able to run a single development server that loads the MFE under development into the shell. This means creating alternatives to frontend-build's existing helpers to support the new architecture. +The development environment will also change, as we want to be able to run a single development server that loads the MFE under development into the shell. This means creating alternatives to frontend-build's existing helpers to support the new architecture. ``frontend-component-header`` / ``frontend-component-footer`` ------------------------------------------------------------- -The header and footer will be owned by the shell and need not be libraries any more. A caveat to this is, of course, that we need to maintain good ways to customize them. +The header and footer will be owned by the shell and need not be libraries any more. A caveat to this is, of course, that we need to maintain good ways to customize them. ``frontend-plugin-framework`` ----------------------------- -We expect ``frontend-plugin-framework`` to be folded into ``frontend-platform`` to unify the two, as they will be intimately tied to each other going forward. We'll also be creating new APIs around loading plugins that support module federation. +We expect ``frontend-plugin-framework`` to be folded into ``frontend-platform`` to unify the two, as they will be intimately tied to each other going forward. We'll also be creating new APIs around loading plugins that support module federation. ``frontend-app-*`` Repositories ------------------------------- -While we don't expect to merge these into a unified library, they are the consumers of all the API changes above. We expect it will be simpler for developers to absorb these changes - and the inevitable bug fixes - by updating a single library dependency, rather than trying to navigate an interconnected dependency tree with cascading version updates. +While we don't expect to merge these into ``frontend-base``, they are the consumers of all the API changes above. We expect it will be simpler for developers to absorb these changes - and the inevitable bug fixes - by updating a single library dependency, rather than trying to navigate an interconnected dependency tree with cascading version updates. Reducing Dependency Maintenance =============================== -One of the motivators of OEP-65 was reducing dependency maintenance. Not explicitly called out in the OEP is the desire to have fewer repositories to manage, which will in turn further reduce the number of dependencies in MFEs. We see value in combining the library repositories listed above into a single library, which will in turn reduce the number of dependencies to manage in downstream repositories. +One of the motivators of OEP-65 was reducing dependency maintenance. Not explicitly called out in the OEP is the desire to have fewer repositories to manage, which will in turn further reduce the number of dependencies in our frontend. We see value in combining the library repositories listed above into a single library, which will in turn reduce the number of dependencies to manage in downstream repositories. Modernization ============= -We also feel the need to continue to modernize our library repositories by adopting industry standard technologies like Typescript, or more performant webpack loaders, and there's some sentiment that this may be the right time to make these changes as we're already undergoing a paradigm shift. ``frontend-platform``, for instance, simplifies significantly if we use TypeScript types instead of the bespoke "interface" and "service implementation" system in that repository. +We also feel the need to continue to modernize our library repositories by adopting industry standard technologies like TypeScript, or more performant webpack loaders, and there's some sentiment that this may be the right time to make these changes as we're already undergoing a paradigm shift. ``frontend-platform``, for instance, simplifies significantly if we use TypeScript types instead of the bespoke "interface" and "service implementation" system in that repository. -Deprecation of the application MFE architecture -=============================================== +Deprecation of the micro-frontend architecture +============================================== -As described in OEP-65, we're migrating from an "application" MFE architecture, where each MFE is a standalone, independently deployed app, to a "module" MFE architecture where the MFEs are modules loaded into a common "shell" application which owns the header, footer, initialization, and shared libraries. +As described in OEP-65, we're migrating from the :term:`micro-frontend architecture`, where each MFE is a standalone, independently deployed app, to a :term:`module architecture` where the MFEs are modules loaded into a common :term:`Shell` which owns the header, footer, initialization, and shared dependencies. -One way or another, this is a paradigm shift that will involve breaking changes and migration work on behalf of community members. We would like to provide a clear a path forward for operators, developers, and maintainers to adopt the module MFE architecture. We believe that a clean break and a cohesive and clear platform for module MFEs is more approachable than a more granular deprecation of certain features and code in the existing library repositories. +One way or another, this is a paradigm shift that will involve breaking changes and migration work on behalf of community members. We would like to provide a clear a path forward for operators, developers, and maintainers to adopt the module architecture. We believe that a clean break and a cohesive and clear platform for modules is more approachable than a more granular deprecation of certain features and code in the existing library repositories. Decision ******** -We will migrate our existing foundational library repositories to the new module MFE architecture by creating a new, unified platform library to act as the primary dependency of module MFE frontends. +We will migrate our existing foundational library repositories to the new module architecture by creating a new, unified platform library - `frontend-base `_ - to act as the primary dependency of frontend modules in the module architecture. -This library will include the parts of ``frontend-build``, ``frontend-platform``, ``frontend-plugin-framework``, ``frontend-component-header``, and ``frontend-component-footer``, as well as the new shell application that make sense for the module MFE architecture, along with new features and capabilities necessary to implement our vision. We will also opportunistically fold in ``eslint-config``, which sees very little development but is its own source of dependency management overhead. +This library will include the parts of ``frontend-build``, ``frontend-platform``, ``frontend-plugin-framework``, ``frontend-component-header``, and ``frontend-component-footer``, as well as the new shell application that make sense for the module architecture, along with new features and capabilities necessary to implement our vision. We will also opportunistically fold in ``eslint-config``, which sees very little development but is its own source of dependency management overhead. This new library will be released as one npm package with the following responsibilities. Runtime library =============== -The library will export a subset of the APIs in ``frontend-platform``, along with ``frontend-plugin-framework``'s API and new APIs specific to the module MFE architecture and the shell. This library will be a dependency of MFEs, as ``frontend-platform`` has been historically. +The library will export a subset of the APIs in ``frontend-platform``, along with ``frontend-plugin-framework``'s API and new APIs specific to the module architecture and the shell. This library will be a dependency of MFEs, as ``frontend-platform`` has been historically. Development tool configurations =============================== -The library will provide base ESLint, Jest, TypeScript, and Webpack configurations. ESLint, Jest, and TypeScript will be similar to what ``frontend-build`` provides. +The library will provide base ESLint, Jest, TypeScript, and Webpack configurations. ESLint, Jest, and TypeScript will be similar to what ``frontend-build`` provides. -The webpack configurations will support a variety of build targets for the new module MFE architecture, as well as application MFE configurations to enable us to migrate to the new unified platform library in a backwards compatible way. The new build targets include: +The webpack configurations will support a variety of build targets for the new module architecture, as well as application MFE configurations to enable us to migrate to the new unified platform library in a backwards compatible way. The new build targets include: * A production configuration suitable for deploying the MFE's modules to be consumed via module federation. * A release configuration which will package those same modules to be released as an npm package for use as direct plugins. @@ -108,21 +110,24 @@ These webpack configurations will be exposed via a function like ``frontend-buil CLI tool ======== -Similar to ``fedx-scripts`` provided by ``frontend-build``, the unified library will provide a CLI tool. This tool will expose commands to run the above webpack configurations, as well as other commands related to frontend projects. The rationale and details of frontend projects and these CLI commands are the subject of a future ADR. +Similar to ``fedx-scripts`` provided by ``frontend-build``, the ``frontend-base`` library will provide a CLI tool. This tool will expose commands to run the above webpack configurations, as well as other commands related to frontend projects. The rationale and details of frontend projects and these CLI commands are the subject of a future ADR. Consequences ************ -This approach allows us to treat the "module MFE" architecture as an independent effort, as opposed to a set of features that need to fit in with the existing application MFE architecture. We greatly reduce or eliminate the risk of regressions in libraries that the entire community relies on while we build a replacement which resembles and borrows code from them, but is otherwise significantly different. This division makes it explicitly clear which features are used by the old architecture and which are used by the new one. +This approach allows us to treat the :term:`module architecture` as an independent effort, as opposed to a set of features that need to fit in with the existing :term:`micro-frontend architecture`. We greatly reduce or eliminate the risk of regressions in libraries that the entire community relies on while we build a replacement which resembles and borrows code from them, but is otherwise significantly different. This division makes it explicitly clear which features are used by the old architecture and which are used by the new one. -Further, it means we can migrate to the new architecture in a backwards compatible way by atomically migrating MFEs to use the new unified platform library, again, without running the risk of destabilizing existing libraries. +Further, it means we can migrate to the new architecture in a backwards compatible way by atomically migrating MFEs to use the new `frontend-base `_ library, again, without running the risk of destabilizing existing libraries with multiple breaking changes over time. -We believe this is a reasonable migration strategy given the generally low rate of feature development in these libraries. We incur incrementally more maintenance burden in the interim while both sets of libraries exist, and a level of effort in absorbing any features or bug fixes in the existing libraries. However, we believe that will be offset by an increase in development velocity for the new library, a clearer deprecation process, less risk of regressions in existing code, and an easier mental model of how the architecture is changing. +We believe this is a reasonable migration strategy given the generally low rate of feature development in these libraries. We incur incrementally more maintenance burden in the interim while both sets of libraries exist, and a level of effort in absorbing any features or bug fixes in the existing libraries. However, we believe that will be offset by an increase in development velocity for the new library, a clearer deprecation process, less risk of regressions in existing code, and an easier mental model of how the architecture is changing. References ********** * :doc:`OEP-65: Frontend Composability <../../oep-0065-arch-frontend-composability>` +* :doc:`OEP-65 Frontend Glossary <./frontend-glossary>` +* :doc:`ADR-0002: Frontend App Migrations <./0002-frontend-app-migrations>` +* :doc:`ADR-0003: Frontend Projects <./0003-frontend-projects>` Change History ************** @@ -132,3 +137,8 @@ Change History * Document created * `Pull request #598 `_ + +2024-09-13 +========== + +* Updating the language use to match and reference the frontend glossary. diff --git a/oeps/architectural-decisions/oep-0065/decisions/0002-frontend-app-migrations.rst b/oeps/architectural-decisions/oep-0065/decisions/0002-frontend-app-migrations.rst new file mode 100644 index 000000000..28472750c --- /dev/null +++ b/oeps/architectural-decisions/oep-0065/decisions/0002-frontend-app-migrations.rst @@ -0,0 +1,152 @@ +Frontend App Migrations +####################### + +Status +****** + +Accepted + +Summary +******* + +This ADR describes a migration path and simplified repository organization for the ``frontend-app-*`` repositories that is compatible with :term:`Module Architecture`: the :term:`Shell` provided by the `frontend-base `_ library, :term:`module federation`, and frontend :term:`projects `. + +Please see :doc:`0001-unified-platform-repository` for more information on ``frontend-base``. + +Context +******* + +OEP-65 proposes adopting webpack :term:`module federation` for Open edX :term:`micro-frontends ` (MFEs) as a means to enable :term:`runtime module loading` and :term:`shared dependencies` in the :term:`Open edX frontend`. The OEP lays out a series of changes necessary to enable these capabilities, which it refers to as building a "reference implementation" of runtime module loading and shared dependencies. + +This reference implementation is effectively a new underlying architecture for our frontend. This ADR refers to this new architecture as the :term:`module architecture`, as opposed to the historical :term:`micro-frontend architecture` that has existed prior to OEP-65. + +As part of this paradigm shift, our ``frontend-app-*`` repositories (MFEs) will need to be migrated to work with `frontend-base `_. Of particular note, this will require the repositories to: + +* Adopt a new set of build/development CLI helpers +* Use the :term:`shell` to provide the header, footer, and runtime initialization code, amongst other things. +* Organize their code into loosely-coupled top-level components, which are called :term:`application modules `. + +As we adopt ``frontend-base``, the libraries it replaces will undergo their own deprecation processes, which we will need to coordinate with the migration of micro-frontends included in Open edX releases. After that deprecation, the micro-frontend architecture will cease to be supported. + +Decision +******** + +Each of our ``frontend-app-*`` repositories will migrate from being an independent "micro-frontend application" to being a library of modules that can be loaded into a common :term:`Shell`, deployed as a :term:`Site`. These are called :term:`module libraries `. We will document the migration process in detail. At a high level, this will involve the following changes. + +New Deployment Methods +====================== + +The module libraries will be buildable in several different ways. + +* Built as :term:`imported modules ` into an independent Site using the Shell for initialization, the header and footer, configuration, and other foundational services (logging, analytics, i18n, etc.) +* Built as :term:`federated modules ` to be loaded into the Shell at runtime via webpack module federation. +* Built and released as an NPM package for build-time inclusion in a frontend :term:`Project`, perhaps alongside other modules from other libraries. + +Environment Agnostic +==================== + +The :term:`module libraries ` will no longer contain ``.env`` or ``env.config`` files for any specific environment, including `Devstack `_ and `Tutor `_. Config filename patterns will be added to the ``.gitignore`` file. They will continue to support adding a (git ignored) config file into the repository to build or develop it, but we also expect operators to use Projects and check their config files into those project repositories as their primary way of working with the module libraries. + +Please see :doc:`0003-frontend-projects` for more information on projects. + +Removed Dependencies +==================== + +Application module libraries will cease to use the following libraries in favor of ``frontend-base``: + +* @openedx/frontend-build +* @edx/frontend-plaform +* @openedx/frontend-plugin-framework +* @edx/frontend-component-header +* @edx/frontend-component-footer +* @openedx/frontend-slot-footer +* @edx/brand +* core-js +* regenerator-runtime + +Peer Dependencies +================= + +We expect module libraries to be dependencies of Frontend Projects by default for most operators. Because of this, the following dependencies will become peer dependencies in the module libraries themselves: + +* @openedx/frontend-base +* @openedx/paragon +* react +* react-dom +* react-redux +* react-router +* react-router-dom +* redux + +New CLI Tools +============= + +The ``fedx-scripts`` CLI tools from ``frontend-build`` will be replaced with the ``openedx`` CLI tools from ``frontend-base``. We'll discuss some of them in detail here, as they help illustrate what the library will be able to do: + +* ``dev`` will start a dev server, loading the repository's modules into the shell in a site. +* ``dev:module`` will start a dev server that provides the modules via module federation. +* ``build`` will create a standalone deployable artifact that uses the shell (similar to the micro-frontend architecture) +* ``build:module`` will create a standalone deployable artifact that provides the modules via module federation. +* ``release`` will package the library for distribution on npm. +* ``serve`` will work with ``build`` or ``build:module`` to locally serve the production assets they generated. +* ``pack`` will work with ``release`` to create a ``.tgz`` file suitable for installing in local git checkouts that depend on the library. (this is a development tool) + +The ``dev``, ``dev:module``, ``build``, and ``build:module`` CLI commands will rely on the existence of a :term:`Site Config` file (the replacement for .env/env.config files) which will not be checked into the module library's repository. + +Distributed as NPM Packages +=========================== + +``frontend-app-*`` repositories that are part of Open edX releases will be expected to be published on NPM as a library which exports its modules. These libraries will primarily be consumed by :term:`projects `. + +Consequences +************ + +As the module architecture stabilizes, ``frontend-app-*`` maintainers and developers will be encouraged to migrate their micro-frontends into module libraries, and to adopt the module architecture provided by ``frontend-base``. (There will be a migration guide.) + +For micro-frontends that are migrated to module libraries using the shell, there will be a deployment approach that mimics the micro-frontend architecture, but which will require operators to adopt a new underlying configuration and build process to achieve a similar result. Each ``frontend-app-*`` repository will need a deprecation process for the micro-frontend configuration and build infrastructure. + +Thinking in Modules +=================== + +Our definition of :term:`module` aligns with the `industry standard definition `_. It is also used in the context of `module federation `_. It's a self-contained part of the frontend that represents a specific part of the :term:`Site`, and can be loaded in a variety of ways. We have several sub-types of module: + +* An *application module* represents a well-bounded sub-area of the Open edX frontend at a particular route path. This might be "courseware", "the login page", or "account settings". There are a number of application modules that are *required* for a functioning Open edX frontend Site. +* A "plugin module" represents an optional UI component that is generally added somewhere in an application module, or in the shell. The header and footer, for instance, would be overridden with alternate implementations via plugin modules. New tabs added to the course homepage are also plugin modules. +* *Service modules* which act as implementations of the logging or analytics services. +* *Script modules* which allow attaching arbitrary scripts to the page. + +Our ``frontend-app-*`` repositories go from being "micro-frontend applications" to being a collection of modules centered around a particular domain (learning, authoring, authn, etc.) The question of which modules belong in which repositories, and where the right boundaries are, is beyond the scope of this ADR. + +Unsupported Customizations +========================== + +The :term:`micro-frontend architecture` took an extreme approach to "flexibility", allowing MFEs to diverge from each other in a variety of ways as described in :doc:`OEP-65 <../../oep-0065-arch-frontend-composability>`. As a result, in the process of migrating them to the :term:`module architecture`, there could be unforeseen refactoring that may need to happen in some MFEs that don't map into modules well, or which have customizations that aren't supported by the Shell. While we hope to provide enough extensibility mechanisms to reduce the need for forking or hacky customizations, there will be customizations we haven't anticipated, which the community will need to work around or find ways to support. + +Consistent Dependency Versions +============================== + +Addressing our *lack* of dependency version consistency is one of the primary drivers of OEP-65. + +The shell will support specific versions of shared dependencies (such as React, Paragon, or React Router). All applications loaded into the shell's Site will be expected to use (or at least be compatible) with that version. We intend to create lock-step version consistency of shared dependencies across all applications in the platform. We envision each Open edX release supporting a particular major version of each shared dependency. + +References +********** + +* :doc:`OEP-65: Frontend Composability <../../oep-0065-arch-frontend-composability>` +* :doc:`OEP-65 Frontend Glossary <./frontend-glossary>` +* :doc:`ADR-0001: Unified Platform Repository <./0001-unified-platform-repository>` +* :doc:`ADR-0003: Frontend Projects <./0003-frontend-projects>` + +Change History +************** + +2024-08-28 +========== + +* Document created +* `Pull request #626 `_ + +2024-09-13 +========== + +* Updating the language use to match and reference the frontend glossary. diff --git a/oeps/architectural-decisions/oep-0065/decisions/0003-frontend-projects.rst b/oeps/architectural-decisions/oep-0065/decisions/0003-frontend-projects.rst new file mode 100644 index 000000000..9c6b648d2 --- /dev/null +++ b/oeps/architectural-decisions/oep-0065/decisions/0003-frontend-projects.rst @@ -0,0 +1,147 @@ +Frontend Projects +################# + +Status +****** + +Accepted + +Summary +******* + +This ADR describes the architecture of and rationale for frontend :term:`Projects `, a way of building and customizing the :term:`Open edX Frontend`. + +Context +******* + +OEP-65 proposes adopting webpack :term:`module federation` for Open edX :term:`micro-frontends ` (MFEs) as a means to enable :term:`runtime module loading` and :term:`shared dependencies` in the :term:`Open edX frontend`. The OEP lays out a series of changes necessary to enable these capabilities, which it refers to as building a "reference implementation" of runtime module loading and shared dependencies. + +This reference implementation is effectively a new underlying architecture for our frontend. This ADR refers to this new architecture as the "module" architecture, as opposed to the historical "micro-frontend" architecture that has existed prior to OEP-65. + +As part of this re-architecture, we need a way for operators to work with and deploy the :term:`Shell`, which is a wrapper around our existing micro-frontends that will provide application initialization, the header and footer, and the services (logging, analytics, i18n, etc.) provided by ``frontend-base``'s runtime library. With micro-frontends, operators are expected to check out the MFE, add a configuration file (``.env`` or ``env.config.js``) co-located with the code, and run the build/development commands within the repository. + +If we carried over this paradigm to the Shell, that would imply that operators should check out frontend-base, add a configuration file to their checkout, and run the CLI commands within the frontend-base repository. + +In both cases, there are several important developer experience issues with this approach: + +* A developer or operator should not have to check out the source code of a platform or framework in order to work with it. +* Adding a configuration file to the source code could tacitly encourage forking when it isn't necessary, as operators would like somewhere to check in their configuration. +* There's no clear path or best practice for operators to make customizations beyond simple configuration file changes. +* The approach of adding a config file and customizations to the source code is at odds with industry best practices and developer expectations when working with nearly any popular framework, such as Django, React, Next.js, or Rails. + +Finally, our documentation and best practices around how to build and customize the Open edX frontend have always been a bit of an afterthought, and operators have been left to "figure it out" and rely on tools like Tutor to manage the complexity and uncertainty for them. We believe that the platform itself should take more responsibility for providing best in class tools and patterns for working with the code. + +Decision +******** + +We will create a well-supported, built-in configuration and customization framework by adding support for frontend :term:`Projects `. + +A frontend Project is a repository, created and owned by an operator, responsible for containing all of the frontend configuration and customized modules for an operator's various environments. + +There are two primary types of projects: *site projects* and *module projects*. + +Site Projects +============= + +Site projects serve the shell, contain the bulk of the frontend's configuration, and are a place to check in customizations and extensions to the shell and modules built as part of the project. + +A site project uses a :term:`Site Config` file to define the configuration for a :term:`Site`, which is an independently deployed portion of the Open edX frontend which renders a header, footer, and loads one or more Modules. + +A site project consists of: + +* One or more configuration files which account for all of an operator's config and customizations across all their environments. +* A set of build targets expressed as ``"scripts"`` in ``package.json`` which point at ``openedx`` CLI commands from ``frontend-base``. +* (Optional) A ``src`` sub-folder containing the operator's custom modules and extensions. + +A site project can load modules in several different ways (:term:`Importing `, :term:`Federating `, or :term:`Linking `), as defined in its Site Config. The simplest is to ``import`` the module into the Site Config file so that it is bundled with the Site Project. + +.. image:: ../site-project-architecture.png + +To describe the steps in the above image: + +1. A build is started with the ``npm run build`` command, which references a ``scripts`` entry in the project's package.json. +2. That script delegates to the ``openedx`` CLI ``build`` command provided by ``frontend-base``. +3. The ``build`` CLI command runs webpack with the ``build`` webpack config. +4. Webpack uses the :term:`Shell` - in ``frontend-base`` - as it's entry point. +5. The Shell initialization code imports the :term:`Site Config` (i.e., ``site.config.build.tsx``) file from the project. +6. The :term:`Site Config` file imports any :term:`Application Modules ` from libraries it depends on (defined as ``dependencies`` in package.json), along with any other Modules from the ``src`` sub-folder. + +Module Projects +=============== + +Module projects are a place to put customizations and extensions to federated modules, and contain configuration specific to those modules. + +.. image:: ../module-project-architecture.png + +A Module Project uses a ``config`` data structure in ``package.json`` to define what modules it should bundle for :term:`module federation`. This mirrors the ``config`` data structure in the :term:`module libraries ` (``frontend-app-*`` repositories). It also uses a :term:`Module Config` file for additional configuration of those modules beyond that which will be supplied by the site's :term:`Site Config`, i.e., module-specific configuration. + +To describe the steps in the above image: + +1. A build is started with the ``npm run build:module`` command, which references a ``scripts`` entry in the project's ``package.json``. +2. The script delegates to the ``openedx`` CLI ``build:module`` command provided by ``frontend-base``. +3. The ``build:module`` CLI command runs webpack with the ``build:module`` webpack config. +4. The webpack config reads the module federation config from the project's ``package.json``. This will look something like: + + .. code-block:: json + + { + // ... + "config": { + "name": "myModuleProject", + "exposes": { + "./ModuleOne": "./src/module-one/ModuleOne", + "./ModuleTwo": "./src/module-two/ModuleTwo", + "./ModuleThree": "./src/module-three/ModuleThree" + } + }, + // ... + } + +5. Webpack builds :term:`federated modules ` from the ``src`` based on the above configuration in ``package.json``. +6. The modules will be responsible for importing module-specific configuration from a :term:`Module Config` file. +7. The federated modules in ``src`` may be custom modules, or "pass-through" modules that re-export :term:`modules ` from one of the project's dependencies. Note that webpack module federation cannot export modules from dependencies directly; a thin shim module is required (unfortunately). This will function the same for all module sub-types. + +Implicit Projects +================= + +Fundamentally, site and module projects consist of: + +* A (site or module) config file. +* Appropriate build scripts which use ``openedx`` CLI commands. +* Optionally, the source code of modules to bundle into the project (either in-project or as dependencies). +* For module projects, a ``config`` field in package.json with ``name`` and ``exposes`` sub-fields. + +This means that any repository that satisfies these requirements can act as a project. These are :term:`"implicit" projects `. + +Of particular note, ``frontend-app-*`` repositories will satisfy these requirements if we add a git-ignored Site Config or Module Config file to them, and in fact, we anticipate this will be a desirable way to do local development on module libraries. + +Consequences +************ + +The addition of projects creates a first class way of managing the configuration and customization of an Open edX frontend instance without checking out the source of the Open edX Platform frontend itself. + +As we begin to migrate the frontend to the :term:`module architecture`, operators will need to adjust their development, build and deployment processes to use projects. While this will require some effort, we believe that focusing the customization of the Open edX frontend around projects is a clearer, more approachable paradigm that has significant precident in the industry. + +We expect that there will be edge cases that we didn't anticipate in the module architecture and project paradigm, particularly around customization, which may still require operators to fork the source, but we should endeavor to minimize cases where that's necessary. + +References +********** + +* :doc:`OEP-65: Frontend Composability <../../oep-0065-arch-frontend-composability>` +* :doc:`OEP-65 Frontend Glossary <./frontend-glossary>` +* :doc:`ADR-0001: Unified Platform Repository <./0001-unified-platform-repository>` +* :doc:`ADR-0002: Frontend App Migrations <./0002-frontend-app-migrations>` + +Change History +************** + +2024-09-04 +========== + +* Document created +* `Pull request #626 `_ + +2024-09-13 +========== + +* Updating the language use to match and reference the frontend glossary. diff --git a/oeps/architectural-decisions/oep-0065/decisions/frontend-glossary.rst b/oeps/architectural-decisions/oep-0065/decisions/frontend-glossary.rst new file mode 100644 index 000000000..2b0ef6198 --- /dev/null +++ b/oeps/architectural-decisions/oep-0065/decisions/frontend-glossary.rst @@ -0,0 +1,149 @@ +Frontend Glossary +################# + +High Level Concepts +******************* + +.. glossary:: + + Guest + A Guest is the :term:`Module` being loaded into a :term:`Site` via module federation. + + Host + A Host is the :term:`Site` loading a :term:`Module` via module federation. + + Micro-frontend + Micro-frontend is an industry standard term for small, composable, independently deployed pieces of a frontend. It has a specific and narrower meaning in Open edX's frontend. Open edX's decoupled frontend architecture has been called the "micro-frontend architecture" since 2018 or so, and the ``frontend-app-*`` repositories, specifically, are referred to as "micro-frontends" or "MFEs" for short. They're often called "micro-frontend applications" as well. Some might argue it's a misnomer, as many of our MFEs are quite large. Regardless, MFEs in Open edX refer to our independently deployed, siloed frontends which do not share dependencies, and which may contain one or more distinct parts of the overall frontend. + + To support a cohesive vernacular for our post-MFE architecture (the :term:`Module Architecture`), we propose co-opting "application" to refer to a sub-type of modules - :term:`application modules `. Each of the ``frontend-app-*`` repositories is really a collection of related applications co-located in the repository together because they share code and dependencies unique to their domain. For instance, the modules in the "learning" MFE - course outline, courseware, progress page, dates page, etc. - share a significant amount of code, but may be better thought of as a collection of related apps, not as "the learning app". + + The rest of this glossary reflects the proposed usage of "application". + + Micro-frontend Architecture + This is the name for the Open edX frontend architecture based on ``frontend-platform``, ``frontend-build``, and independent ``frontend-app-*`` repositories that represent :term:`micro-frontends `. It has been super-ceded by the :term:`Module Architecture`. + + Module + A piece of code that provides some specific, tightly coupled functionality, and which is loaded into a :term:`Site` via one of several loading mechanisms (:term:`Imported Module`, :term:`Federated Module`, or :term:`Linked Module`) which can happen at buildtime or runtime. + + There are different sub-types of modules, such as :term:`Applications `, :term:`Plugins `, :term:`Services `, or :term:`Scripts