Code structure changes

[stephenwf]

Proposed structure changes

Source /src folder is the top level for all Typescript.

Top level folders

• frontend
• server
• queue
• utility

Frontend

• admin - All pages and components specific to the admin
• site - All pages and components specific to the site
• shared - All pages and components that are shared
• server-side-rendering - Contains server code specific for rendering the frontend on the server
• capture-models - Specific area for all capture model specific code
• core - I hate this name, but there are some things that do not fit in the above - reduce noise

Admin / Site / Shared common structure

Directories:

• atoms - Single component per file, atomic and reusable components
• blocks - Will contain organised folders of components (e.g. blocks/viewers/XYZ/XYZ.tsx) but must be page-blocks
• components - Same as above, but not (yet) page blocks
• features - Features are all page-blocks and usually themselves compositions
• hooks - Reusable React hooks
• pages - See below

Files (admin/site only):

• routes.tsx - All routes for the frontend
• render-client.tsx - Client-only entrypoint
• render-server.tsx - Server-only entrypoint
• config-client.ts - Client-only config
• config-server.ts - Server-only config
• index.tsx - Common entrypoint (loaded by the render-(client|server).tsx)

New Component structure

At the moment components are a mix of styles. The following will be applied for new components:

1. Moving to prefer function components over const expressions

// before
export const Component1: React.FC<ComponentProps> = (props) => { ... }

// After
export function Component2(props: ComponentProps) { ... }

2. Component folders

Before:

• components/Button.tsx
• ../stories/Button.stories.tsx

After:

• components/Button/Button.tsx
• components/Button/Button.styles.ts
• components/Button/Button.stories.tsx
• components/Button/Button.tests.ts
• components/Button/Button.types.ts
• components/Button/Button.[feature].ts

These component folder can grow with a component, and can be split where it makes sense.

3. Styled components -> Stitches

Currently, we use Styled components for all CSS styling. This has a few problems at the scale of Madoc and is
not been possible to solve these issues with the library itself. It's not practical to migrate everything at once
but for new components they should use the Stitches library instead.

Before:

import styled from 'styled-components';

const Button = styled.button`
   background: blue;
   border: 3px solid orange;
   &:hover {
     border-color: red;
   }
`;

// <Button />

After:

import { styled } from '... TBC ...';

const Button = styled('button', {
  background: 'blue',
  border: '3px solid orange',
  '&:hover': {
    borderColor: 'red',
  }
});

// <Button />

New page structure

At the moment pages mostly work using serverRendererFor() helper or createUniversalComponent to load
server side information. This will slow change for new pages using vite for discovery of pages.

Current page example:

// pages/some-page.tsx

function SomePage() {
  const { data } = useData(SomePage); // Uses self

  return <div>This is my page: {data.someProperty}</div>
}

// This data will be pre-loaded when server-rendering.
serverRendererFor(SomePage, {
  getKey: () => 'some-page',
  getData: (key, vars, api) => api.getSomethingFromApi(),
});

New example:

pages/SomePage/SomePage.tsx

export default function SomePage() {
  const { data } = useClientData(); // No self-ref

  return <div>This is my page: {data.someProperty}</div>
}

pages/SomePage/SomePage.loader.ts

export function loader({ api }: LoaderType) {
  return api.getSomethingFromApi();
}

This splitting allows for bundles to be more optimised, we can know what data is required without loading
the full component and dependencies. This also means we can load data and components in parallel using the
latest version of React Router (v6.4).

Other features

Loader options:

// pages/SomePage/SomePage.loader.ts
export function loader() { ... }
export function action() { ... }
export function shouldRevalidate() { ... }
export const hooks = [];

Server-only loaders / actions:

// pages/SomePage/SomePage.server.ts
export function loader() { ... }
export function action() { ... }

These will available at special endpoints: /api/madoc/loaders?page=SomePage and are a tool to compose APIs

Error boundary:

export default function SomePageError() {
  return <div>Show this if there is an error</div>;
}

Similar to the new component model we can expand this structure with more features we might need in the future.

Backend

Changes to the backend are fairly limited, some are simply technical debt to be addressed.

New route structure

For new routes, a structure similar to the frontend will be used, with the goal to migrate everything across
eventually.

Current system:

// routes/ping.ts
import { RouteMiddleware } from '../types/route-middleware';

export const ping: RouteMiddleware = async context => {
  context.response.body = { ping: 'pong' };
};

and in routes:

// router.ts

const routes = new TypedRouter({
  'get-ping': [TypedRouter.GET, '/api/madoc', ping],
  // ...
});

New system

// All routing information exported.
export default {
  id: 'ping',
  method: 'GET',
  path: '/api/madoc/ping/:slug',
  middleware: [ping]
}

// JSON schema-lite definition
export const params = {
  slug: { type: 'string' }
}

function ping(context: RouteMiddlewareFn<typeof params>) {
  context.response.body = { ping: 'pong', slug: context.params.slug }; // type checking on params.
}

This has a few benefits:

• Single file routing, and vite auto-loading to make DX smoother
• Strongly typed params, with option to add runtime-validation
• Can generate documentation from definition more easily
• Model can be extended (e.g. export const statusMap = {200: {ping: 'boolean'}, 400: {error: 'string'}})
• Vite can be added to hot-reloading of individual endpoints

Moving all SQL to /repository folder

At the moment there are routes that have Postgres queries inline. This should be switched to use the repository
pattern that has cleaned up these routes considerably. This will also make it much easier to test servers with
repository-only mocks.

Consolidate environment variables

These should be configured in a single place, and imported or injected in config. This is mostly done but still
has some gaps.

Possibly reformat of "tasks"

At the moment a task contains ALL the code for a task including types. Splitting these down may reduce bundle sizes
and keep the backend and frontend split.

Auto-loading of extensions

These should use vite to autoload extensions and make development easier.

Splitting monolithic API class

Currently, the api.ts file is massive. This should be split down into distinct API clients, accessible and lazy-loaded
through the existing interface.

Example before:

class ApiClient {
  async getSomething() {
    return this.request( ... );
  }
}

After:

export class SomethingApiClient {
  async getSomething() {
    return this.request( ... );
  }
}

class ApiClient {
  
  getSomethingClient() {
    if (!this._somethingClient) {
      this._somethingClient = import('./apis/SomethingApiClient').then(mod => new mod.SomethingApiClient());
      // Any set up.
    }
    return this._somethingClient;
  }
  
  async getSomething() {
    deprecationWarning('Use SomethingApiClient instead');
    return (await this.getSomethingClient()).getSomething();
  }
}

This should make splitting large APIs much easier, and also reduce the size of APIs floating around. For context
the current API is 66KB on its own. We may also be able to generate some API clients automatically, further
reducing the size and maintenance of the API client.