Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use dev_me.py for shortfin integration tests #527

Closed
wants to merge 8 commits into from
Closed

Use dev_me.py for shortfin integration tests #527

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7c53f1a
use devme for shortfin integration tests
renxida Nov 14, 2024
0c61e03
add iree cloning
renxida Nov 14, 2024
965c850
install cmake
renxida Nov 14, 2024
d28b29b
remove iree clone if not the same ver
renxida Nov 14, 2024
54bd109
init submodules of iree
renxida Nov 15, 2024
7243e8f
simplify iree clone and submodule versioning
renxida Nov 15, 2024
34fa1c7
move to a separate script so i can compare runtimes
renxida Nov 15, 2024
8633cd9
remove empty lines
renxida Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion .github/workflows/ci-shark-platform.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ jobs:
# from non default locations first. Installing the PyTorch CPU
# wheels saves multiple minutes and a lot of bandwidth on runner setup.
pip install --no-compile -r pytorch-cpu-requirements.txt
pip install --no-compile -r requirements.txt -e sharktank/ shortfin/
pip install --no-compile -r requirements.txt -e sharktank/

# Install latest iree-tubrine.
pip install --no-compile -f https://iree.dev/pip-release-links.html --src deps \
Expand All @@ -71,5 +71,20 @@ jobs:
iree-base-compiler \
iree-base-runtime

- name: Clone IREE with matching version
run: |
# Get IREE version and extract the version number
IREE_VERSION=$(pip show iree-base-compiler | grep Version: | cut -d' ' -f2)
echo "Found IREE version: ${IREE_VERSION}"
# Clone IREE repository at the parent directory level. Git clone --branch supports tags.
cd .. && git clone --depth 1 --branch "iree-${IREE_VERSION}" https://github.com/iree-org/iree.git
cd SHARK-Platform

- name: Install CMake
uses: ssrobins/install-cmake@v1 # this action installs cmake into the action workspace

- name: Install shortfin via dev_me.py
run: cd shortfin && python dev_me.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally integration tests would use built packages, so they can't assume anything about the source environment. Developer scripts could still be used for unit tests though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh good point. Will take the setup section and swap it into a unit test once done.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDK what you're doing with the git clone stuff, but I also want more workflows to just use the FetchContent code in CMake that downloads automatically. Developer flows and GitHub Actions should both be simple. Branchy bash scripts that peek at output from pip commands are not simple :P

dev_me.py appears to handle the automatic download case:

SHARK-Platform/shortfin/dev_me.py

Lines 29 to 31 in 778b567

# By default, if there is an iree source dir adjacent to this parent repository,
# that will be used (so you can just directly edit IREE runtime code and build.
# Otherwise, the shortfin build will download a pinned IREE source tree.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just trying to not repeatedly download code - - i was also trying to make this integration test faster

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(now that i look at it iree_repo_dir already seems to cache the iree repo for the unittests; might get that over so that shortfin build doesn't have to pull iree)


- name: Run LLM Integration Tests
run: pytest -v build_tools/integration_tests/llm --log-cli-level=INFO
Loading