feat: support cortex.cpp engine variants

[louis-jan]

Describe Your Changes

This PR aims to integrate the engine management update from cortex.cpp and engine variants.

Where:

The cortex.cpp file now requires the client to set a default variant for engines to run, such as llama-cpp for the mac-arm64 or mac-amd64 platforms.

The folder is structured as follows:

• Jan/data/extensions/@janhq/inference-cortex-extension
  • bin (where engines are downloaded, as before)
    • engines
      • cortex.llamacpp
        • mac-amd64
          • engine_files…
        • mac-arm64
          • engine_files…
      • cortex-server

Cortex extension life-cycle

• Extension Launch -> Boot cortex.cpp -> Health check -> Get System Information -> Set default llama.cpp engine variant

janhq/cortex.cpp#1660

Related Issues

• feat: update inference-cortex-extension to work with the new engine management of cortex.cpp #4025

Code changes

1. Download Script Modifications:

   • Updated download paths in download.sh to use a new directory structure for macOS engine locations, specifically including the version v0.1.35 in the path.

2. Rollup Configurations:

   • Added a new constant CORTEX_ENGINE_VERSION set to 'v0.1.35' in rollup.config.ts.

3. TypeScript Definitions:

   • Declared a new string constant CORTEX_ENGINE_VERSION in global.d.ts.

4. Main Application Logic Enhancements:

   • Added a method setDefaultEngine in index.ts to set the default engine variant on launch.
   • Converted several functions in index.ts to private methods for better encapsulation (healthz, clean, subscribeToEvents).

5. Testing Improvements:

   • Added tests to verify the engineVariant function in execute.test.ts.
   • Generalized enginePath expectations in tests to reflect changes in directory structure.

6. Execution Logic Adjustments:

   • Removed the binPath field from CortexExecutableOptions in execute.ts.
   • Updated the logic for determining the engineVariant based on GPU settings and architecture in execute.ts.

7. Node Module Exports and Utility Functions:

   • Updated the export and utility functions to include engineVariant in both index.ts and engine-executable logic.

These changes seem to focus on organizing engine binaries by version, improving modularity, and enhancing the setup for different system configurations.

[github-actions]

Barecheck - Code coverage report

Total: 69.22%

Your code coverage diff: 0.00% ▴

✅ All code changes are covered