Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

Commit

Permalink
Update the architecture diagram into excalidraw
Browse files Browse the repository at this point in the history
  • Loading branch information
irfanpena committed Aug 16, 2024
1 parent 467f23a commit 0ae0171
Show file tree
Hide file tree
Showing 18 changed files with 7,212 additions and 43 deletions.
5 changes: 3 additions & 2 deletions docs/architecture.md → docs/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ The following guide details Cortex's core components, providing insights and ins

## Architecture

![Architecture](/img/docs/architecture.png)
import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/Architecture.excalidraw"} />

### Main Components

Expand Down Expand Up @@ -119,7 +121,6 @@ sequenceDiagram
Chat Controller/Service -->> Model Entity: findOne()
Cortex->>Model Entity: Store the model data
Chat Controller/Service -->> Extension Repository: findAll()
%% Responses
Extension Repository ->> Chat Controller/Service: Response stream
Chat Controller/Service ->> Chat Controller/Service: Formatted response/stream
Expand Down
6 changes: 5 additions & 1 deletion docs/benchmarking-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@ slug: "benchmarking-architecture"
:::

## Architecture
![benchmark-architecture](/img/docs/benchmark-architecture.png)

import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/benchmarking.excalidraw"} />

The benchmarking capability comprises several key components:
1. **User Devices**:
- **Workstation, Desktop, Server**: These devices collect and send benchmarking data directly to the Supabase Backend in the Jan Cloud.
Expand Down
7 changes: 5 additions & 2 deletions docs/cortex-llamacpp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,9 @@ You can download precompiled models from the [Cortex Hub](https://huggingface.co

## Architecture

![Cortex llamacpp architecture](/img/docs/llama-cpp1.png)
import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/llamacpp1.excalidraw"} />

### Main Components

Expand All @@ -145,7 +147,8 @@ You can download precompiled models from the [Cortex Hub](https://huggingface.co

### Communication Protocols

![Cortex llamacpp architecture](/img/docs/llama-cpp2.png)
<Diagram diagramPath={"/diagrams/llamacpp2.excalidraw"} />

The diagram above illustrates how `cortex.llamacpp` communication protocol works:

- `Streaming`: Responses are processed and returned one token at a time.
Expand Down
4 changes: 3 additions & 1 deletion docs/cortex-onnx.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,9 @@ All the interfaces above contain the following parameters:


## Architecture
![onnx Architecture](/img/docs/onnx-1.png)
import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/ONNX.excalidraw"} />

### Main Components
These are the main components that interact to provide an API for `inference` tasks using the `onnxruntime-genai` library:
Expand Down
6 changes: 5 additions & 1 deletion docs/cortex-tensorrt-llm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,11 @@ All the interfaces above contain the following parameters:


## Architecture
![tensorrt-llm Architecture](/img/docs/trt1.png)

import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/TensorRT-LLM.excalidraw"} />

These are the main components that interact to provide an API for `inference` tasks using the `tensorrt-llm`:
1. **cortex-cpp**: Acts as an intermediary between `cortex-js` and the inference engine (`cortex.tensorrt-llm`). It processes incoming HTTP requests and forwards them to the appropriate components for handling. Once a response is generated, it sends it back to `cortex-js`.

Expand Down
4 changes: 3 additions & 1 deletion docs/telemetry-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ import TabItem from '@theme/TabItem';
:::

## Architecture
![cortex-architecture.png](/img/docs/telemetry1.png)
import Diagram from "../src/components/Diagram"

<Diagram diagramPath={"/diagrams/Telemetry.excalidraw"} />
The telemetry capability comprises several key components:
1. **Telemetry Collector**: This component gathers telemetry data from the Cortex and exports it to other observability tools for further analysis and monitoring.
2. **Jan Telemetry Server**: This component ingests telemetry data from Cortex. It processes this data and forwards events to Google Logging for further handling.
Expand Down
3 changes: 2 additions & 1 deletion docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,7 @@ const config: Config = {
},
} as ScalarOptions,
],
"docusaurus-plugin-dotenv",
],

scripts: [
Expand Down Expand Up @@ -370,7 +371,7 @@ const config: Config = {
docId: "overview",
label: "Docs",
},
{ to: "/docs/cli", label: "CLI", position: "left" },
// { to: "/docs/cli", label: "CLI", position: "left" },
{ to: "/models", label: "Models", position: "right" },
{
type: "search",
Expand Down
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
"@docusaurus/preset-classic": "^3.4.0",
"@docusaurus/theme-live-codeblock": "^3.4.0",
"@docusaurus/theme-mermaid": "^3.4.0",
"@excalidraw/excalidraw": "^0.17.6",
"@huggingface/hub": "^0.15.1",
"@mdx-js/react": "3.0.1",
"@radix-ui/react-select": "^2.1.1",
Expand All @@ -32,6 +33,7 @@
"class-variance-authority": "^0.7.0",
"clsx": "^2.1.1",
"date-fns": "^3.6.0",
"docusaurus-plugin-dotenv": "^1.0.1",
"docusaurus-plugin-sass": "^0.2.5",
"framer-motion": "^11.2.12",
"highlight.js": "^11.9.0",
Expand Down
52 changes: 18 additions & 34 deletions sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,24 @@ const sidebars: SidebarsConfig = {
{ type: "doc", id: "chat-completions", label: "Chat Completions" },
{ type: "doc", id: "embeddings", label: "Embeddings" },
{ type: "doc", id: "benchmarking", label: "Benchmarking" },
// CLI
{
type: "html",
value: "CLI",
className: "sidebar-divider",
},
{ type: "doc", id: "cli/cortex", label: "cortex" },
{ type: "doc", id: "cli/chat", label: "cortex chat" },
{ type: "doc", id: "cli/embeddings", label: "cortex embeddings" },
{ type: "doc", id: "cli/presets", label: "cortex presets" },
{ type: "doc", id: "cli/pull", label: "cortex pull" },
{ type: "doc", id: "cli/run", label: "cortex run" },
{ type: "doc", id: "cli/models/index", label: "cortex models" },
{ type: "doc", id: "cli/engines/index", label: "cortex engines" },
{ type: "doc", id: "cli/stop", label: "cortex stop" },
{ type: "doc", id: "cli/ps", label: "cortex ps" },
{ type: "doc", id: "cli/telemetry", label: "cortex telemetry" },
{ type: "doc", id: "cli/benchmark", label: "cortex benchmark" },
// ARCHITECTURE
{
type: "html",
Expand Down Expand Up @@ -149,40 +167,6 @@ const sidebars: SidebarsConfig = {
},
{ type: "doc", id: "troubleshooting", label: "Troubleshooting" },
],
cli: [
{
type: "html",
value: "Usage",
className: "sidebar-divider",
},
{ type: "doc", id: "cli/cortex", label: "cortex" },
{ type: "doc", id: "cli/chat", label: "cortex chat" },
{ type: "doc", id: "cli/embeddings", label: "cortex embeddings" },
{ type: "doc", id: "cli/presets", label: "cortex presets" },
{
type: "html",
value: "Model Operations",
className: "sidebar-divider",
},
{ type: "doc", id: "cli/pull", label: "cortex pull" },
{ type: "doc", id: "cli/run", label: "cortex run" },
{ type: "doc", id: "cli/models/index", label: "cortex models" },
{ type: "doc", id: "cli/engines/index", label: "cortex engines" },
{
type: "html",
value: "System",
className: "sidebar-divider",
},
{ type: "doc", id: "cli/stop", label: "cortex stop" },
{ type: "doc", id: "cli/ps", label: "cortex ps" },
{ type: "doc", id: "cli/telemetry", label: "cortex telemetry" },
{
type: "html",
value: "Benchmarking",
className: "sidebar-divider",
},
{ type: "doc", id: "cli/benchmark", label: "cortex benchmark" },
],
};

export default sidebars;
51 changes: 51 additions & 0 deletions src/components/Diagram/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import BrowserOnly from "@docusaurus/BrowserOnly";
import { Excalidraw, loadFromBlob } from "@excalidraw/excalidraw";
import { ExcalidrawImperativeAPI } from "@excalidraw/excalidraw/types/types";
import { useEffect, useState } from "react";

interface DiagramProps {
diagramPath: string;
}

export default function Diagram({ diagramPath }: DiagramProps) {
// return (
// <BrowserOnly>
// {() => {
const [excalidrawAPI, setExcalidrawAPI] =
useState<ExcalidrawImperativeAPI | null>(null);

useEffect(() => {
if (excalidrawAPI !== null) {
fetch(diagramPath)
.then((res) => res.blob())
.then((blob) => loadFromBlob(blob, excalidrawAPI.getAppState(), null))
.then((data) => {
excalidrawAPI.updateScene(data);

// excalidrawAPI.setActiveTool({ type: "hand", locked: true });
});
}
}, [excalidrawAPI]);

return (
<div style={{ height: "600px" }}>
<Excalidraw
excalidrawAPI={(api) => setExcalidrawAPI(api)}
isCollaborating={false}
zenModeEnabled={false}
viewModeEnabled={true}
gridModeEnabled={false}
UIOptions={{
canvasActions: {
export: false,
loadScene: false,
saveToActiveFile: false,
},
}}
/>
</div>
);
// }}
// </BrowserOnly>
// );
}
Loading

0 comments on commit 0ae0171

Please sign in to comment.