Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests description #509

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
88 changes: 46 additions & 42 deletions preset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,30 +25,33 @@ docker pull intel/deep-learning:latest-py<version>

There are 3 modes to run these containers:

* [Interactive](#run-in-interactive-mode)
* [Jupyter](#run-using-jupyter-notebook)
* [Interactive](#run-in-interactive-mode)
* [Multi-Node Distributed Training](#run-in-multi-node-distributed-mode-advanced) (Deep Learning and Inference Optimization)

> [!NOTE]
> Modify the commands below to fit your use case, especially the image, environment variables, and GPU device path.

### Run in Interactive Mode
### Run using Jupyter Notebook

This mode allows running the container in an interactive shell. This enables the ability to interact with the container's bash shell. Below is the command to start the container in interactive mode:
This mode launches a jupyterlab notebook server. The command below will start the jupyterlab server which can be accessed from a web browser. Each container includes jupyter kernel to enable conda environment in jupyter notebook. The port for this server is `8888` and is exposed by default when you run the container.

#### Run on CPU
> [!NOTE]
> When launching a jupyter notebook server this way, docker assigns a [network](https://docs.docker.com/engine/tutorials/networkingcontainers/) such that the container can communicate with other applications like a web browser. By default docker launches containers with the `bridge` network, but if you are trying to access this server from a machine you are `ssh`'ing into, change the network mode with the `--net=host` flag and ensure you are local port forwarding with `ssh -L 8888:8888`.

#### Run on Jupyter CPU

```bash
docker run -it --rm \
-p 8888:8888 --shm-size=12G \
-v ${PWD}:/home/dev/workdir \
intel/deep-learning:latest-py<version> bash
intel/deep-learning:latest-py<version>
```

> [!NOTE]
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications (Ex. Modin*). Docker recommends increasing shared memory size using `--shm-size`.

#### Run on GPU
#### Run on Jupyter GPU

Find your machine's `RENDER` and `VIDEO` group values to enable [Intel® Flex/Max GPU](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu.html).

Expand All @@ -64,52 +67,53 @@ docker run -it --rm \
${RENDER_GROUP} \
${VIDEO_GROUP} \
--device=/dev/dri \
-p 8888:8888 --shm-size=12G \
-p 8888:8888 --shm-size=12G \
-v ${PWD}:/home/dev/workdir \
-v /dev/dri/by-path:/dev/dri/by-path \
intel/deep-learning:latest-py<version> bash
intel/deep-learning:latest-py<version>
```

> [!NOTE]
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications(Ex. Modin). Docker recommends increasing shared memory size using `--shm-size`.
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications (Ex. Modin*). Docker recommends increasing shared memory size using `--shm-size`.

#### Next Steps

1. For Deep Learning and Inference Optimization containers there will be separate conda environments for each AI framework: `pytorch-cpu`, `pytorch-gpu`, `tensorflow-cpu` and `tensorflow-gpu`. Use the command below to activate one environment:
1. After running this command the terminal should display an output similar to displayed below in the image ![image](https://github.com/intel/ai-containers/assets/18349036/0a8a2d05-f7b0-4a9f-994e-bcc4e4b703a0) The server address together with the port set can be used to connect to the jupyter server in a web browser. For example `http://127.0.0.1:8888`. The token displayed after the `token=` can be used as a password to login into the server. For example in the image displayed above the token is `b66e74a85bc2570bf15782e5124c933c3a4ddabd2cf2d7d3`.

```bash
conda activate <env-name>
```
2. Select a notebook sample from the Overview notebook found in directory you launched the server with. In this example, the `intel/deep-learning` container has a notebook titled [`Deep_Learning_Samples_Overview.ipynb`](https://github.com/intel/ai-containers/blob/main/preset/deep-learning/notebooks/Deep_Learning_Samples_Overview.ipynb) when launched in jupyter mode.

2. Select a test from the `sample-tests` folder and run it using the following command as an example:
3. After selecting a notebook sample, select the preset kernel found in the dropdown menu presented when loading the notebook. For Deep Learning and Inference Optimization containers there will be multiple kernels, one for each framework: `pytorch`, `pytorch-gpu`, and `tensorflow`.

```bash
python sample-tests/intel_extension_for_tensorflow/test_itex.py
```
##### Advanced Jupyter Server Configuration

> [!NOTE]
> The `sample-tests` folder may differ in each container, and some tests use a bash script.
Modify your notebook server command by using the default example below to change the network (port/ip) and security (privilege) settings by appending it to the docker run commands above:

### Run using Jupyter Notebook
```bash
docker run ... intel/deep-learning:latest-py<version> \
bash -c "jupyter notebook --notebook-dir=~/jupyter \
--port 8888 \
--ip 0.0.0.0 \
--no-browser \
--allow-root"
```

This mode launches a jupyterlab notebook server. The command below will start the jupyterlab server which can be accessed from a web browser. Each container includes jupyter kernel to enable conda environment in jupyter notebook. The port for this server is `8888` and is exposed by default when you run the container.
### Run in Interactive Mode

> [!NOTE]
> When launching a jupyter notebook server this way, docker assigns a [network](https://docs.docker.com/engine/tutorials/networkingcontainers/) such that the container can communicate with other applications like a web browser. By default docker launches containers with the `bridge` network, but if you are trying to access this server from a machine you are `ssh`'ing into, change the network mode with the `--net=host` flag and ensure you are local port forwarding with `ssh -L 8888:8888`.
This mode allows running the container in an interactive shell. This enables the ability to interact with the container's bash shell. Below is the command to start the container in interactive mode:

#### Run on Jupyter CPU
#### Run on CPU

```bash
docker run -it --rm \
-p 8888:8888 --shm-size=12G \
-v ${PWD}:/home/dev/workdir \
intel/deep-learning:latest-py<version>
intel/deep-learning:latest-py<version> bash
```

> [!NOTE]
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications (Ex. Modin*). Docker recommends increasing shared memory size using `--shm-size`.

#### Run on Jupyter GPU
#### Run on GPU

Find your machine's `RENDER` and `VIDEO` group values to enable [Intel® Flex/Max GPU](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu.html).

Expand All @@ -125,35 +129,35 @@ docker run -it --rm \
${RENDER_GROUP} \
${VIDEO_GROUP} \
--device=/dev/dri \
-p 8888:8888 --shm-size=12G \
-p 8888:8888 --shm-size=12G \
-v ${PWD}:/home/dev/workdir \
-v /dev/dri/by-path:/dev/dri/by-path \
intel/deep-learning:latest-py<version>
intel/deep-learning:latest-py<version> bash
```

> [!NOTE]
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications (Ex. Modin*). Docker recommends increasing shared memory size using `--shm-size`.
> Certain applications use shared memory to share data between processes. But the default shared memory segment size is 64M for docker containers, and is not enough for multithreaded applications(Ex. Modin). Docker recommends increasing shared memory size using `--shm-size`.

#### Next Steps

1. After running this command the terminal should display an output similar to displayed below in the image ![image](https://github.com/intel/ai-containers/assets/18349036/0a8a2d05-f7b0-4a9f-994e-bcc4e4b703a0) The server address together with the port set can be used to connect to the jupyter server in a web browser. For example `http://127.0.0.1:8888`. The token displayed after the `token=` can be used as a password to login into the server. For example in the image displayed above the token is `b66e74a85bc2570bf15782e5124c933c3a4ddabd2cf2d7d3`.
1. For Deep Learning and Inference Optimization containers there will be separate conda environments for each AI framework: `pytorch-cpu`, `pytorch-gpu`, `tensorflow-cpu` and `tensorflow-gpu`. Use the command below to activate one environment:

2. Select a notebook sample from the Overview notebook found in directory you launched the server with. In this example, the `intel/deep-learning` container has a notebook titled [`Deep_Learning_Samples_Overview.ipynb`](https://github.com/intel/ai-containers/blob/main/preset/deep-learning/notebooks/Deep_Learning_Samples_Overview.ipynb) when launched in jupyter mode.
```bash
conda activate <env-name>
```

3. After selecting a notebook sample, select the preset kernel found in the dropdown menu presented when loading the notebook. For Deep Learning and Inference Optimization containers there will be multiple kernels, one for each framework: `pytorch`, `pytorch-gpu`, and `tensorflow`.
2. Select a test from the `sample-tests` folder and run it using the following command as an example:

##### Advanced Jupyter Server Configuration
```bash
python sample-tests/intel_extension_for_tensorflow/test_itex.py
```

Modify your notebook server command by using the default example below to change the network (port/ip) and security (privilege) settings by appending it to the docker run commands above:
The `test_itex.py` script uses TensorFlow to classify images with a pre-trained ResNet model from TensorFlow Hub.
It creates a random image, preprocesses it, and then runs the model 100 times to measure the average inference time, excluding the first 10 runs which are considered warm-up iterations.
Additionally, the script identifies and lists the available GPUs and CPUs, and prints the version of TensorFlow being used.

```bash
docker run ... intel/deep-learning:latest-py<version> \
bash -c "jupyter notebook --notebook-dir=~/jupyter \
--port 8888 \
--ip 0.0.0.0 \
--no-browser \
--allow-root"
```
> [!NOTE]
> The `sample-tests` folder may differ in each container, and some tests use a bash script.

## Run in Multi-Node Distributed Mode [Advanced]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,20 @@
# See the License for the specific language governing permissions and
# limitations under the License.


# pylint: skip-file
def print_script_description():
description = """
This script utilizes PyTorch to classify images with a pre-trained ResNet-50 model.
Users can specify whether to run the script on a CPU or XPU, and there is an option to apply optimizations using Intel Extension for PyTorch (IPEX).
Transfers both the model and data to the chosen device, and then measures the average inference time over 100 runs, excluding the initial warm-up phase.
Finally, the script prints the average inference time.
"""
print(description)


print_script_description()

import argparse

import intel_extension_for_pytorch as ipex
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,17 @@

#!/usr/bin/env python
# coding: utf-8
def print_script_description():
description = """
This script uses TensorFlow to classify images with a pre-trained ResNet model from TensorFlow Hub.
It creates a random image, preprocesses it, and then runs the model 100 times to measure the average inference time, excluding the first 10 runs which are considered warm-up iterations.
Additionally, the script identifies and lists the available GPUs and CPUs, and prints the version of TensorFlow being used.
"""
print(description)


print_script_description()

import tensorflow as tf

print("TensorFlow version {}".format(tf.__version__))
Expand Down
Loading