From 9767cc0fdd43ef04a5a43f397db967630e87f253 Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Wed, 4 Dec 2024 10:50:25 +0000 Subject: [PATCH 1/6] Splice orc Enhancements --- jac-splice-orc/ReadMe.md | 95 ++++++++++++++++++- jac-splice-orc/docker/Dockerfile | 1 + .../jac_splice_orc/config/config.json | 6 +- .../jac_splice_orc/managers/pod_manager.py | 53 ++++++++--- .../jac_splice_orc/server/server.py | 10 ++ 5 files changed, 144 insertions(+), 21 deletions(-) diff --git a/jac-splice-orc/ReadMe.md b/jac-splice-orc/ReadMe.md index fea42303e9..920ef7daac 100644 --- a/jac-splice-orc/ReadMe.md +++ b/jac-splice-orc/ReadMe.md @@ -18,7 +18,8 @@ JAC Cloud Orchestrator (`jac-splice-orc`) is a system designed to dynamically im - [1. Clone the Repository](#1-clone-the-repository) - [2. Install Dependencies](#2-install-dependencies) - [3. Configure the System](#3-configure-the-system) - - [4. Initialize the System](#4-initialize-the-system) + - [4. Recreate the Kind Cluster with Port Mappings](#4-recreate-the-kind-cluster-with-port-mappings) + - [5. Initialize the System](#5-initialize-the-system) - [Docker Usage](#docker-usage) - [Usage](#usage) - [Client Application](#client-application) @@ -131,7 +132,7 @@ Before you begin, ensure that you have the following installed and configured: - **Python** (version 3.9 or later): [Install Python](https://www.python.org/downloads/) - **Docker** (version 20.10 or later): [Install Docker](https://docs.docker.com/get-docker/) -- **Kubernetes** (version 1.21 or later): [Install Kubernetes](https://kubernetes.io/docs/setup/) +- **Kind** (Kubernetes IN Docker): [Install Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) - **kubectl** command-line tool: [Install kubectl](https://kubernetes.io/docs/tasks/tools/) - **Jac**: [Install Jaclang](https://github.com/Jaseci-Labs/jasecii) - **Kubernetes Cluster**: Ensure you have access to a Kubernetes cluster (local or remote). @@ -217,11 +218,97 @@ Edit `jac_splice_orc/config/config.json` to match your environment. Here's an ex - Replace `jaseci/jac-splice-orc:latest` with your own image if you have customized it. - Adjust resource requests and limits according to your environment. -### 4. Initialize the System +### 4. Recreate the Kind Cluster with Port Mappings + +To ensure that your Kubernetes cluster can expose services correctly, especially when using **Kind** (Kubernetes IN Docker), you need to recreate the Kind cluster with specific port mappings. This allows services like the Pod Manager to be accessible from your host machine without relying solely on port-forwarding. + +**Why Recreate the Kind Cluster?** + +- **Port Accessibility**: By mapping container ports to host ports, you can access Kubernetes services directly via `localhost:` on your machine. +- **Simplified Access**: Eliminates the need for manual port-forwarding or additional networking configurations. + +**Steps to Recreate the Kind Cluster with Port Mappings:** + +1. **Delete the Existing Kind Cluster** + + If you already have a Kind cluster running, delete it to allow recreation with new configurations. + + ```bash + kind delete cluster --name little-x-kind + ``` + + **Note**: Replace `jac-splice-orc with your cluster name if different. + +2. **Create a Kind Configuration File** + + Create a YAML configuration file named `kind-config.yaml` with the desired port mappings. This file instructs Kind to map specific container ports to host ports. + + ```yaml + kind: Cluster + apiVersion: kind.x-k8s.io/v1alpha4 + nodes: + - role: control-plane + extraPortMappings: + - containerPort: 30080 + hostPort: 30080 + protocol: TCP + ``` + + **Explanation:** + + - **containerPort**: The port inside the Kubernetes cluster (i.e., the port your service listens on). + - **hostPort**: The port on your local machine that maps to the `containerPort`. + - **protocol**: The network protocol (`TCP` or `UDP`). + +3. **Create the New Kind Cluster with Port Mappings** + + Use the `kind-config.yaml` to create a new Kind cluster with the specified port mappings. + + ```bash + kind create cluster --name little-x-kind --config kind-config.yaml + ``` + + **Output Example:** + + ``` + Creating cluster "little-x-kind" ... + ✓ Ensuring node image (kindest/node:v1.21.1) đŸ–ŧ + ✓ Preparing nodes đŸ“Ļ + ✓ Writing configuration 📜 + ✓ Starting control-plane node kind-control-plane 🕹ī¸ + ✓ Installing CNI 🔌 + ✓ Installing StorageClass 💾 + Set kubectl context to "kind-little-x-kind" + You can now use your cluster with: + + kubectl cluster-info --context kind-little-x-kind + + Thanks for using Kind! 🎉 + ``` + + +### Summary of Steps: + +1. **Delete Existing Cluster**: `kind delete cluster --name jac-splice-orc +2. **Create Config File**: Define `kind-config.yaml` with desired port mappings. +3. **Create New Cluster**: `kind create cluster --name little-x-kind --config kind-config.yaml` +4. **Verify Mappings**: Ensure ports are correctly mapped using `kubectl` and `docker` commands. + +**Important Considerations:** + +- **Port Conflicts**: Ensure that the `hostPort` values you choose are not already in use on your host machine. +- **Cluster Name**: Adjust the cluster name (`jac-splice-orc) as per your preference or organizational standards. +- **Security**: Exposing ports directly to `localhost` can have security implications. Ensure that only necessary ports are exposed and consider implementing authentication or network policies if needed. + +--- + +### 5. Initialize the System + +Once the cluster is set up with the appropriate port mappings, proceed to initialize the Pod Manager and Kubernetes resources. Use the provided CLI command to initialize the Pod Manager and Kubernetes resources: -```bash +```jac jac orc_initialize jac-splice-orc ``` diff --git a/jac-splice-orc/docker/Dockerfile b/jac-splice-orc/docker/Dockerfile index 3aeba516c8..6fb2d80d2b 100644 --- a/jac-splice-orc/docker/Dockerfile +++ b/jac-splice-orc/docker/Dockerfile @@ -5,6 +5,7 @@ FROM python:3.12-slim RUN pip install --no-cache-dir \ grpcio \ grpcio-tools \ + grpcio-health-checking\ fastapi \ uvicorn \ kubernetes \ diff --git a/jac-splice-orc/jac_splice_orc/config/config.json b/jac-splice-orc/jac_splice_orc/config/config.json index c0abf86aff..8295bcb551 100644 --- a/jac-splice-orc/jac_splice_orc/config/config.json +++ b/jac-splice-orc/jac_splice_orc/config/config.json @@ -6,14 +6,14 @@ "deployment_name": "pod-manager-deployment", "service_account_name": "jac-orc-sa", "container_name": "pod-manager", - "image_name": "ashishmahendra/jac-splice-orc:0.0.6", + "image_name": "ashishmahendra/jac-splice-orc:0.0.8", "container_port": 8000, "service_name": "pod-manager-service", "service_type": "LoadBalancer", "env_vars": { "SERVICE_TYPE": "pod_manager", "NAMESPACE": "jac-splice-test", - "IMAGE_NAME": "ashishmahendra/jac-splice-orc:0.0.6" + "IMAGE_NAME": "ashishmahendra/jac-splice-orc:0.0.8" }, "resources": { "requests": { @@ -71,6 +71,6 @@ } }, "environment": { - "POD_MANAGER_URL": "http://a88a549ed32f14b14b1333a81ebd7a2a-1627559794.us-west-2.elb.amazonaws.com:8000" + "POD_MANAGER_URL": "http://localhost:8000" } } \ No newline at end of file diff --git a/jac-splice-orc/jac_splice_orc/managers/pod_manager.py b/jac-splice-orc/jac_splice_orc/managers/pod_manager.py index 0b5dbf35ff..384fa6cbbc 100644 --- a/jac-splice-orc/jac_splice_orc/managers/pod_manager.py +++ b/jac-splice-orc/jac_splice_orc/managers/pod_manager.py @@ -110,7 +110,13 @@ def create_pod(self, module_name: str, module_config: dict) -> Any: "mountPath": f"/app/requirements/{module_name}", } ], - } + "readinessProbe": {"grpc": {"port": 50051}}, + "initialDelaySeconds": 10, + "periodSeconds": 5, + "timeoutSeconds": 5, + "failureThreshold": 3, + "successThreshold": 1, + }, ], "volumes": [ { @@ -124,20 +130,23 @@ def create_pod(self, module_name: str, module_config: dict) -> Any: try: existing_configmap = self.v1.read_namespaced_config_map( - name=f"{module_name}-requirements", - namespace=self.namespace + name=f"{module_name}-requirements", namespace=self.namespace ) print(f"ConfigMap '{module_name}-requirements' already exists.") except client.exceptions.ApiException as e: - if e.status == 404: + if e.status == 404: # Create the ConfigMap - print(f"ConfigMap '{module_name}-requirements' not found. Creating it...") + print( + f"ConfigMap '{module_name}-requirements' not found. Creating it..." + ) _ = self.v1.create_namespaced_config_map( self.namespace, body={ "metadata": {"name": f"{module_name}-requirements"}, "data": { - "requirements.txt": open(requirements_file_path, "r").read() + "requirements.txt": open( + requirements_file_path, "r" + ).read() }, }, ) @@ -199,20 +208,36 @@ def delete_pod(self, module_name: str) -> Any: def wait_for_pod_ready(self, pod_name: str) -> None: """Wait until the pod is ready.""" - max_retries = 30 + max_retries = 120 retries = 0 while retries < max_retries: - pod_info = self.v1.read_namespaced_pod( - name=pod_name, namespace=self.namespace - ) - if pod_info.status.phase == "Running": - logging.info( - f"Pod {pod_name} is running with IP {pod_info.status.pod_ip}" + try: + pod_info = self.v1.read_namespaced_pod( + name=pod_name, namespace=self.namespace ) + except client.exceptions.ApiException as e: + logging.error(f"Error fetching pod info for {pod_name}: {e}") + raise Exception(f"Error fetching pod info for {pod_name}: {e}") + + conditions = pod_info.status.conditions or [] + ready = False + logging.info(f"Pod {pod_name} is in phase: {pod_info.status.phase}") + for condition in conditions: + logging.info(f"Condition: {condition.type} - {condition.status}") + if condition.type == "Ready" and condition.status == "True": + ready = True + break + if ready: + logging.info(f"Pod {pod_name} is ready and ready to serve requests.") return + elif pod_info.status.phase in ["Failed", "Unknown"]: + raise Exception(f"Pod {pod_name} is in {pod_info.status.phase} phase.") retries += 1 + logging.info( + f"Waiting for pod {pod_name} to be ready... (Attempt {retries}/{max_retries})" + ) time.sleep(2) - raise Exception(f"Timeout: Pod {pod_name} failed to reach 'Running' state.") + raise Exception(f"Timeout: Pod {pod_name} failed to become ready.") def get_pod_service_ip(self, module_name: str) -> str: """Look up the service IP for the pod corresponding to the module.""" diff --git a/jac-splice-orc/jac_splice_orc/server/server.py b/jac-splice-orc/jac_splice_orc/server/server.py index c2c0355c6e..bd4c76cc3e 100644 --- a/jac-splice-orc/jac_splice_orc/server/server.py +++ b/jac-splice-orc/jac_splice_orc/server/server.py @@ -10,6 +10,8 @@ import logging import traceback +from grpc_health.v1 import health, health_pb2_grpc, health_pb2 + logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s") @@ -182,6 +184,14 @@ def serve(module_name): module_service_pb2_grpc.add_ModuleServiceServicer_to_server( ModuleService(module_name), server ) + + health_servicer = health.HealthServicer() + health_pb2_grpc.add_HealthServicer_to_server(health_servicer, server) + + health_servicer.set( + service="ModuleService", + status=health_pb2.HealthCheckResponse.SERVING, + ) server.add_insecure_port("[::]:50051") server.start() logging.info("gRPC server started and listening on port 50051") From 2ee11fdd65d4466672d5ac66be1a5dcb3ace28ff Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Fri, 6 Dec 2024 12:02:34 +0000 Subject: [PATCH 2/6] test cases updated --- .../jac_splice_orc/test/test_pod_manager.py | 211 +++++++++++++----- 1 file changed, 150 insertions(+), 61 deletions(-) diff --git a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py index 3d8bb179c4..a1a3fad3f9 100644 --- a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py +++ b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py @@ -1,14 +1,26 @@ import pytest from fastapi.testclient import TestClient -from unittest import mock - -# Mock gRPC imports to avoid ImportError in test environment -with mock.patch.dict( +from unittest.mock import patch, MagicMock +from kubernetes.client import ( + V1Pod, + V1Condition, + V1Service, + V1PodStatus, + V1ObjectMeta, + V1PodSpec, + V1Status, +) +from kubernetes.client.rest import ApiException +from datetime import datetime, timezone +import os + +# Mock gRPC imports +with patch.dict( "sys.modules", { - "grpc_local": mock.MagicMock(), - "grpc_local.module_service_pb2": mock.MagicMock(), - "grpc_local.module_service_pb2_grpc": mock.MagicMock(), + "grpc_local": MagicMock(), + "grpc_local.module_service_pb2": MagicMock(), + "grpc_local.module_service_pb2_grpc": MagicMock(), }, ): from ..managers.pod_manager import app, PodManager @@ -16,73 +28,150 @@ client = TestClient(app) -@pytest.fixture -def mock_kubernetes_and_grpc(): - with mock.patch( - "kubernetes.config.load_incluster_config" - ) as mock_load_incluster_config, mock.patch( - "kubernetes.client.CoreV1Api" - ) as mock_v1_api, mock.patch.object( - PodManager, "create_pod" - ) as mock_create_pod, mock.patch.object( - PodManager, "delete_pod" - ) as mock_delete_pod, mock.patch.object( - PodManager, "forward_to_pod" - ) as mock_forward_to_pod, mock.patch.object( - PodManager, "get_pod_service_ip" - ) as mock_get_pod_service_ip: - - # Mock load_incluster_config to avoid loading in-cluster config during tests - mock_load_incluster_config.return_value = None - - # Mock Kubernetes CoreV1Api methods - mock_v1_api.return_value = mock.Mock() - - # Mock responses for Kubernetes actions - mock_create_pod.return_value = { - "message": "Pod numpy-pod and service numpy-service created." - } - mock_delete_pod.return_value = { - "message": "Pod numpy-pod and service numpy-service deleted." - } - - # Mock get_pod_service_ip to avoid real Kubernetes API calls - mock_get_pod_service_ip.return_value = "127.0.0.1" # Return a mock IP address +def create_v1_condition(type, status): + return V1Condition( + type=type, + status=status, + last_transition_time=datetime.now(timezone.utc), + message="Condition stable", + reason="TestingCondition", + observed_generation=1, + ) - # Mock gRPC method call to return expected value - mock_forward_to_pod.return_value = "[1, 2, 3, 4]" - yield +def mock_create_requirements_file(pod_manager, module_name, module_config): + req_path = "/tmp/requirements.txt" + deps = module_config.get("dependency", []) + with open(req_path, "w") as f: + for dep in deps: + f.write(dep + "\n") + return req_path -def test_create_pod(mock_kubernetes_and_grpc): +@pytest.fixture +def mock_kubernetes(): + with patch( + "kubernetes.config.load_incluster_config" + ) as mock_load_incluster_config, patch( + "kubernetes.client.CoreV1Api" + ) as mock_core_v1_api: + mock_load_incluster_config.return_value = None + mock_v1_api_instance = mock_core_v1_api.return_value + + def pod_read_side_effect(name, namespace, *args, **kwargs): + if name == "numpy-pod": + if not hasattr(pod_read_side_effect, "called"): + pod_read_side_effect.called = True + raise ApiException(status=404, reason="Not Found") + else: + return V1Pod( + metadata=V1ObjectMeta(name="numpy-pod"), + spec=V1PodSpec(containers=[]), + status=V1PodStatus( + phase="Running", + conditions=[ + create_v1_condition(type="Ready", status="True") + ], + ), + ) + raise ApiException(status=404, reason="Not Found") + + pod_read_side_effect.calls = 0 + mock_v1_api_instance.read_namespaced_pod.side_effect = pod_read_side_effect + + # ConfigMap read -> always 404 to force creation + mock_v1_api_instance.read_namespaced_config_map.side_effect = ApiException( + status=404, reason="Not Found" + ) + + # Creating configmap returns a mock success + mock_v1_api_instance.create_namespaced_config_map.return_value = MagicMock() + + # Creating pod returns a Running pod + mock_v1_api_instance.create_namespaced_pod.return_value = V1Pod( + metadata=V1ObjectMeta(name="numpy-pod"), + spec=V1PodSpec(containers=[]), + status=V1PodStatus( + phase="Running", + conditions=[create_v1_condition(type="Ready", status="True")], + ), + ) + + # Creating service returns a mock service + mock_v1_api_instance.create_namespaced_service.return_value = V1Service( + metadata=V1ObjectMeta(name="numpy-service"), spec=MagicMock() + ) + + # Deletion returns success statuses + mock_v1_api_instance.delete_namespaced_pod.return_value = V1Status( + message="Pod numpy-pod deleted successfully." + ) + mock_v1_api_instance.delete_namespaced_service.return_value = V1Status( + message="Service numpy-service deleted successfully." + ) + + yield mock_v1_api_instance + + +@patch.object( + PodManager, + "create_requirements_file", + side_effect=mock_create_requirements_file, + autospec=True, +) +@patch("os.makedirs", return_value=None) +def test_create_pod(mock_req_file, mock_makedirs, mock_kubernetes): module_config = { - "numpy": { - "lib_mem_size_req": "100MB", - "dependency": ["math", "mkl"], - "lib_cpu_req": "500m", - "load_type": "remote", - } + "lib_mem_size_req": "100Mi", + "dependency": ["math", "mkl"], + "lib_cpu_req": "500m", + "load_type": "remote", } - response = client.post("/create_pod/numpy", json={"module_config": module_config}) - - assert response.status_code == 200 - assert response.json() == { - "message": "Pod numpy-pod and service numpy-service created." + response = client.post("/create_pod/numpy", json=module_config) + assert response.status_code == 200, response.text + assert response.json() == {"message": "Pod numpy-pod is already running."} + + +@patch.object( + PodManager, + "create_requirements_file", + side_effect=mock_create_requirements_file, + autospec=True, +) +@patch("os.makedirs", return_value=None) +def test_run_module(mock_req_file, mock_makedirs, mock_kubernetes): + module_config = { + "lib_mem_size_req": "100Mi", + "dependency": ["math", "mkl"], + "lib_cpu_req": "500m", + "load_type": "remote", } + # Ensure pod created first + resp = client.post("/create_pod/numpy", json=module_config) + assert resp.status_code == 200 -def test_run_module(mock_kubernetes_and_grpc): response = client.post( "/run_module?module_name=numpy&method_name=array", json={"args": [1, 2, 3, 4]} ) assert response.status_code == 200 - assert response.json() == "[1, 2, 3, 4]" # Expected output from gRPC mock -def test_delete_pod(mock_kubernetes_and_grpc): - response = client.delete("/delete_pod/numpy") - assert response.status_code == 200 - assert response.json() == { - "message": "Pod numpy-pod and service numpy-service deleted." +@patch.object( + PodManager, + "create_requirements_file", + side_effect=mock_create_requirements_file, + autospec=True, +) +@patch("os.makedirs", return_value=None) +def test_create_pod_with_readiness(mock_req_file, mock_makedirs, mock_kubernetes): + module_config = { + "lib_mem_size_req": "100Mi", + "dependency": ["math", "mkl"], + "lib_cpu_req": "500m", + "load_type": "remote", } + + response = client.post("/create_pod/numpy", json=module_config) + assert response.status_code == 200 + assert response.json() == {"message": "Pod numpy-pod is already running."} From faecc5258ff1c25ae912f6d09f3643402520e8f4 Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Fri, 6 Dec 2024 13:51:51 +0000 Subject: [PATCH 3/6] Added Readiness Probes Simulation in Test Suite --- .../jac_splice_orc/test/test_pod_manager.py | 175 ++++++++---------- 1 file changed, 77 insertions(+), 98 deletions(-) diff --git a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py index a1a3fad3f9..101b547fda 100644 --- a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py +++ b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py @@ -1,26 +1,26 @@ import pytest from fastapi.testclient import TestClient -from unittest.mock import patch, MagicMock +from unittest import mock +from unittest.mock import MagicMock from kubernetes.client import ( V1Pod, - V1Condition, - V1Service, - V1PodStatus, V1ObjectMeta, V1PodSpec, + V1PodStatus, + V1Condition, + V1Service, V1Status, ) from kubernetes.client.rest import ApiException from datetime import datetime, timezone -import os -# Mock gRPC imports -with patch.dict( +# Mock gRPC imports to avoid ImportError in test environment +with mock.patch.dict( "sys.modules", { - "grpc_local": MagicMock(), - "grpc_local.module_service_pb2": MagicMock(), - "grpc_local.module_service_pb2_grpc": MagicMock(), + "grpc_local": mock.MagicMock(), + "grpc_local.module_service_pb2": mock.MagicMock(), + "grpc_local.module_service_pb2_grpc": mock.MagicMock(), }, ): from ..managers.pod_manager import app, PodManager @@ -28,81 +28,82 @@ client = TestClient(app) -def create_v1_condition(type, status): +def create_condition_ready(status): return V1Condition( - type=type, + type="Ready", status=status, last_transition_time=datetime.now(timezone.utc), - message="Condition stable", reason="TestingCondition", - observed_generation=1, + message="Condition simulation", ) -def mock_create_requirements_file(pod_manager, module_name, module_config): - req_path = "/tmp/requirements.txt" - deps = module_config.get("dependency", []) - with open(req_path, "w") as f: - for dep in deps: - f.write(dep + "\n") - return req_path - - @pytest.fixture -def mock_kubernetes(): - with patch( +def mock_kubernetes_and_grpc(): + with mock.patch( "kubernetes.config.load_incluster_config" - ) as mock_load_incluster_config, patch( + ) as mock_load_incluster_config, mock.patch( "kubernetes.client.CoreV1Api" - ) as mock_core_v1_api: + ) as mock_core_v1_api, mock.patch.object( + PodManager, "create_pod" + ) as mock_create_pod, mock.patch.object( + PodManager, "delete_pod" + ) as mock_delete_pod, mock.patch.object( + PodManager, "forward_to_pod" + ) as mock_forward_to_pod, mock.patch.object( + PodManager, "get_pod_service_ip" + ) as mock_get_pod_service_ip: + mock_load_incluster_config.return_value = None + mock_v1_api_instance = mock_core_v1_api.return_value - def pod_read_side_effect(name, namespace, *args, **kwargs): + call_count = [0] + + def read_pod_side_effect(name, namespace, *args, **kwargs): if name == "numpy-pod": - if not hasattr(pod_read_side_effect, "called"): - pod_read_side_effect.called = True - raise ApiException(status=404, reason="Not Found") + if call_count[0] == 0: + call_count[0] += 1 + # First check: pod phase = Pending, not ready yet + return V1Pod( + metadata=V1ObjectMeta(name="numpy-pod"), + spec=V1PodSpec(containers=[]), + status=V1PodStatus(phase="Pending", conditions=[]), + ) else: + # Second check: pod is now Running and Ready return V1Pod( metadata=V1ObjectMeta(name="numpy-pod"), spec=V1PodSpec(containers=[]), status=V1PodStatus( - phase="Running", - conditions=[ - create_v1_condition(type="Ready", status="True") - ], + phase="Running", conditions=[create_condition_ready("True")] ), ) raise ApiException(status=404, reason="Not Found") - pod_read_side_effect.calls = 0 - mock_v1_api_instance.read_namespaced_pod.side_effect = pod_read_side_effect + mock_v1_api_instance.read_namespaced_pod.side_effect = read_pod_side_effect - # ConfigMap read -> always 404 to force creation + # ConfigMap read -> assume not found to trigger creation mock_v1_api_instance.read_namespaced_config_map.side_effect = ApiException( status=404, reason="Not Found" ) - # Creating configmap returns a mock success + # Creating configmap returns success mock_v1_api_instance.create_namespaced_config_map.return_value = MagicMock() - # Creating pod returns a Running pod + # Creating pod returns immediately a Pending pod (simulate creation success) mock_v1_api_instance.create_namespaced_pod.return_value = V1Pod( metadata=V1ObjectMeta(name="numpy-pod"), spec=V1PodSpec(containers=[]), - status=V1PodStatus( - phase="Running", - conditions=[create_v1_condition(type="Ready", status="True")], - ), + status=V1PodStatus(phase="Pending", conditions=[]), ) - # Creating service returns a mock service + # Creating service returns success mock_v1_api_instance.create_namespaced_service.return_value = V1Service( - metadata=V1ObjectMeta(name="numpy-service"), spec=MagicMock() + metadata=V1ObjectMeta(name="numpy-service") ) - # Deletion returns success statuses + # Deletion returns success messages mock_v1_api_instance.delete_namespaced_pod.return_value = V1Status( message="Pod numpy-pod deleted successfully." ) @@ -110,68 +111,46 @@ def pod_read_side_effect(name, namespace, *args, **kwargs): message="Service numpy-service deleted successfully." ) - yield mock_v1_api_instance + # Mock PodManager actions + mock_create_pod.return_value = { + "message": "Pod numpy-pod and service numpy-service created." + } + mock_delete_pod.return_value = { + "message": "Pod numpy-pod and service numpy-service deleted." + } + mock_get_pod_service_ip.return_value = "127.0.0.1" # mock IP + mock_forward_to_pod.return_value = "[1, 2, 3, 4]" + yield -@patch.object( - PodManager, - "create_requirements_file", - side_effect=mock_create_requirements_file, - autospec=True, -) -@patch("os.makedirs", return_value=None) -def test_create_pod(mock_req_file, mock_makedirs, mock_kubernetes): - module_config = { - "lib_mem_size_req": "100Mi", - "dependency": ["math", "mkl"], - "lib_cpu_req": "500m", - "load_type": "remote", - } - response = client.post("/create_pod/numpy", json=module_config) - assert response.status_code == 200, response.text - assert response.json() == {"message": "Pod numpy-pod is already running."} - -@patch.object( - PodManager, - "create_requirements_file", - side_effect=mock_create_requirements_file, - autospec=True, -) -@patch("os.makedirs", return_value=None) -def test_run_module(mock_req_file, mock_makedirs, mock_kubernetes): +def test_create_pod(mock_kubernetes_and_grpc): module_config = { - "lib_mem_size_req": "100Mi", - "dependency": ["math", "mkl"], - "lib_cpu_req": "500m", - "load_type": "remote", + "numpy": { + "lib_mem_size_req": "100MB", + "dependency": ["math", "mkl"], + "lib_cpu_req": "500m", + "load_type": "remote", + } + } + response = client.post("/create_pod/numpy", json={"module_config": module_config}) + assert response.status_code == 200 + assert response.json() == { + "message": "Pod numpy-pod and service numpy-service created." } - # Ensure pod created first - resp = client.post("/create_pod/numpy", json=module_config) - assert resp.status_code == 200 +def test_run_module(mock_kubernetes_and_grpc): response = client.post( "/run_module?module_name=numpy&method_name=array", json={"args": [1, 2, 3, 4]} ) assert response.status_code == 200 + assert response.json() == "[1, 2, 3, 4]" -@patch.object( - PodManager, - "create_requirements_file", - side_effect=mock_create_requirements_file, - autospec=True, -) -@patch("os.makedirs", return_value=None) -def test_create_pod_with_readiness(mock_req_file, mock_makedirs, mock_kubernetes): - module_config = { - "lib_mem_size_req": "100Mi", - "dependency": ["math", "mkl"], - "lib_cpu_req": "500m", - "load_type": "remote", - } - - response = client.post("/create_pod/numpy", json=module_config) +def test_delete_pod(mock_kubernetes_and_grpc): + response = client.delete("/delete_pod/numpy") assert response.status_code == 200 - assert response.json() == {"message": "Pod numpy-pod is already running."} + assert response.json() == { + "message": "Pod numpy-pod and service numpy-service deleted." + } From 41a74e79b4329652477960a60226a3c088629820 Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Mon, 9 Dec 2024 10:02:39 +0000 Subject: [PATCH 4/6] cleanup --- jac-splice-orc/jac_splice_orc/test/test_pod_manager.py | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py index 101b547fda..a444465ebc 100644 --- a/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py +++ b/jac-splice-orc/jac_splice_orc/test/test_pod_manager.py @@ -83,27 +83,22 @@ def read_pod_side_effect(name, namespace, *args, **kwargs): mock_v1_api_instance.read_namespaced_pod.side_effect = read_pod_side_effect - # ConfigMap read -> assume not found to trigger creation mock_v1_api_instance.read_namespaced_config_map.side_effect = ApiException( status=404, reason="Not Found" ) - # Creating configmap returns success mock_v1_api_instance.create_namespaced_config_map.return_value = MagicMock() - # Creating pod returns immediately a Pending pod (simulate creation success) mock_v1_api_instance.create_namespaced_pod.return_value = V1Pod( metadata=V1ObjectMeta(name="numpy-pod"), spec=V1PodSpec(containers=[]), status=V1PodStatus(phase="Pending", conditions=[]), ) - # Creating service returns success mock_v1_api_instance.create_namespaced_service.return_value = V1Service( metadata=V1ObjectMeta(name="numpy-service") ) - # Deletion returns success messages mock_v1_api_instance.delete_namespaced_pod.return_value = V1Status( message="Pod numpy-pod deleted successfully." ) @@ -111,14 +106,13 @@ def read_pod_side_effect(name, namespace, *args, **kwargs): message="Service numpy-service deleted successfully." ) - # Mock PodManager actions mock_create_pod.return_value = { "message": "Pod numpy-pod and service numpy-service created." } mock_delete_pod.return_value = { "message": "Pod numpy-pod and service numpy-service deleted." } - mock_get_pod_service_ip.return_value = "127.0.0.1" # mock IP + mock_get_pod_service_ip.return_value = "127.0.0.1" mock_forward_to_pod.return_value = "[1, 2, 3, 4]" yield From 8575d9334b521182a46d72bd149479ae3b14abb5 Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Tue, 17 Dec 2024 14:27:05 +0000 Subject: [PATCH 5/6] Updated ReadMe --- jac-splice-orc/ReadMe.md | 28 ++++++++++++---------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/jac-splice-orc/ReadMe.md b/jac-splice-orc/ReadMe.md index 920ef7daac..1a4283f747 100644 --- a/jac-splice-orc/ReadMe.md +++ b/jac-splice-orc/ReadMe.md @@ -130,25 +130,15 @@ jac-splice-orc/ Before you begin, ensure that you have the following installed and configured: -- **Python** (version 3.9 or later): [Install Python](https://www.python.org/downloads/) -- **Docker** (version 20.10 or later): [Install Docker](https://docs.docker.com/get-docker/) -- **Kind** (Kubernetes IN Docker): [Install Kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) -- **kubectl** command-line tool: [Install kubectl](https://kubernetes.io/docs/tasks/tools/) -- **Jac**: [Install Jaclang](https://github.com/Jaseci-Labs/jasecii) +- **Python** (version 3.11 or later) +- **Docker** (version 20.10 or later) +- **Kubernetes** (version 1.21 or later) +- **kubectl** command-line tool - **Kubernetes Cluster**: Ensure you have access to a Kubernetes cluster (local or remote). Ensure that your Kubernetes cluster is up and running, and that you can connect to it using `kubectl`. -### 1. Clone the Repository - -Clone the `jac-splice-orc` repository to your local machine: - -```bash -git clone https://github.com/Jaseci-Labs/jac-splice-orc.git -cd jac-splice-orc -``` - -### 2. Install Dependencies +### 1. Install Dependencies Create a virtual environment and install the required Python packages: @@ -157,9 +147,15 @@ python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install -r requirements.txt ``` - **Note**: The `requirements.txt` file includes all necessary dependencies, such as `kubernetes`, `grpcio`, `PyYAML`, and others. +### 2. Install via pip + +You can install `jac-splice-orc` directly from PyPI: +```bash +pip install jac-splice-orc +``` + ### 3. Configure the System The application uses a `config.json` file located in the `jac_splice_orc/config/` directory for all configurations. From 891acaf9826a369438131e18e1cda7432ac08120 Mon Sep 17 00:00:00 2001 From: Ashish Mahendra Date: Thu, 19 Dec 2024 12:14:12 +0000 Subject: [PATCH 6/6] updating requirements --- jac-splice-orc/setup.py | 1 + 1 file changed, 1 insertion(+) diff --git a/jac-splice-orc/setup.py b/jac-splice-orc/setup.py index 2d39b0813d..03de3555e7 100644 --- a/jac-splice-orc/setup.py +++ b/jac-splice-orc/setup.py @@ -21,6 +21,7 @@ "requests", "python-dotenv", "numpy", + "jaclang", ], entry_points={ "jac": [