Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

open source ivy's transpiler #28849

Merged
merged 35 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
4d2000e
add transpiler files
Sam-Armstrong Dec 12, 2024
9de183d
remove sourcecode license logic
Sam-Armstrong Dec 12, 2024
80ec868
update import name references
Sam-Armstrong Dec 12, 2024
baac48e
remove compiler/
Sam-Armstrong Dec 12, 2024
47cc20b
update compiler references
Sam-Armstrong Dec 12, 2024
4008f2c
add graph tracer
Sam-Armstrong Dec 14, 2024
51dc977
add ivy.trace_graph to __init__
Sam-Armstrong Dec 14, 2024
1dab0b2
docs: adding a README for the source-to-source transpiler
Sam-Armstrong Dec 16, 2024
f09d694
docs: added README for the core transformations/ submodule
Sam-Armstrong Dec 16, 2024
faaa034
docs: added README for the tracer/ submodule
Sam-Armstrong Dec 16, 2024
bb46a72
add tracer tests
Sam-Armstrong Dec 16, 2024
f3d3886
add transpiler tests
Sam-Armstrong Dec 16, 2024
034a5b0
chore: cleaning up `CONTRIBUTING.md`
YushaArif99 Dec 17, 2024
616baed
docs: adding transpiler tasks to `CONTRIBUTING.md` [skip ci]
YushaArif99 Dec 17, 2024
2e60708
add first transpiler tests to ci
Sam-Armstrong Dec 18, 2024
1f4490e
fix: transpiler import path bugs
Sam-Armstrong Dec 18, 2024
0dd776a
add ivy.module
Sam-Armstrong Dec 18, 2024
6bda842
remove caching 'get_ivy_root' dependency on tracing_caching
Sam-Armstrong Dec 18, 2024
df5ade0
change to using relative imports within the transpiler
Sam-Armstrong Dec 18, 2024
6d9147c
remove continue-on-error from transpiler tests
Sam-Armstrong Jan 2, 2025
8ee0f66
add translations, module, sourcegen tests to ci
Sam-Armstrong Jan 2, 2025
c6b7656
add kornia and hf tests to ci
Sam-Armstrong Jan 2, 2025
a6704ee
lint
Sam-Armstrong Jan 2, 2025
585e1dd
switch to running transpiler ci tests via shell script
Sam-Armstrong Jan 2, 2025
3bb56e1
add missing datasets and transformers installs to script
Sam-Armstrong Jan 2, 2025
cfc2aab
remove 'inplace' arg from module tests
Sam-Armstrong Jan 2, 2025
ffa698e
only run transpiler ci on push/pr to main
Sam-Armstrong Jan 2, 2025
8f1b24b
remove redundant functionality
Sam-Armstrong Jan 2, 2025
e7ff42b
add control_flow_experimental/
Sam-Armstrong Jan 2, 2025
391d4ac
change tracer to use relative imports
Sam-Armstrong Jan 2, 2025
1cfce5d
remove publish binaries step from release workflow
Sam-Armstrong Jan 4, 2025
5b50c11
remove loading binaries from release pipeline
Sam-Armstrong Jan 4, 2025
966615a
fix: don't pull the binaries when running the integration tests
Sam-Armstrong Jan 4, 2025
fe7fe71
fix: don't pass binaries branch to integration tests workflows
Sam-Armstrong Jan 4, 2025
e0f4a61
fix: remove test dispatcher functionality from test_translations
Sam-Armstrong Jan 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions .github/workflows/binaries.yml

This file was deleted.

2 changes: 0 additions & 2 deletions .github/workflows/full-integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,4 @@ permissions:
jobs:
integration-tests:
uses: ivy-llc/ivy-integration-tests/.github/workflows/qa-integration-tests.yml@main
with:
binaries-branch: testing-nightly
secrets: inherit
2 changes: 0 additions & 2 deletions .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,4 @@ jobs:
integration-tests:
name: Run Integration Tests
uses: ivy-llc/ivy-integration-tests/.github/workflows/qa-integration-tests.yml@main
with:
binaries-branch: testing-nightly
secrets: inherit
8 changes: 0 additions & 8 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,6 @@ jobs:
integration-tests:
name: Run Integration Tests
uses: ivy-llc/ivy-integration-tests/.github/workflows/qa-integration-tests.yml@main
with:
binaries-branch: testing-nightly
secrets: inherit

publish-binaries:
name: Publish Binaries
needs: integration-tests
uses: ./.github/workflows/binaries.yml
secrets: inherit

deploy:
Expand Down
107 changes: 107 additions & 0 deletions .github/workflows/test-transpiler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
name: transpiler-tests
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
jobs:

transformations:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Run Transformations Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh transformations

translations:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Run Translations Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh translations

module:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"

- name: Run Module Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh module

sourcegen:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Run SourceGen Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh sourcegen

hf:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Run HuggingFace Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh hf

kornia:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
path: ivy
persist-credentials: false
- uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Run Kornia Tests
id: tests
run: |
cd ivy
docker run --rm -v "$(pwd)":/ivy ivyllc/ivy:latest scripts/shell/run_transpiler_tests.sh kornia
11 changes: 2 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ Please, follow the next process when you work on your subtask:
- is not mentioned in the comments.

Currently, there are three open tasks:

- [Function Reformatting](https://docs.ivy.dev/overview/contributing/open_tasks.html#function-formatting)
- [Transpiler](https://github.com/ivy-llc/ivy/issues?q=is%3Aopen+is%3Aissue+label%3ATranspiler)
- [Frontend APIs](https://docs.ivy.dev/overview/contributing/open_tasks.html#frontend-apis)
- [Ivy Experimental API](https://docs.ivy.dev/overview/contributing/open_tasks.html#ivy-experimental-api)

Expand All @@ -33,15 +32,9 @@ Please, follow the next process when you work on your subtask:
- When you have finished PR or need help open the PR make sure to follow our PR template.

5. **Review Process:**
- Wait for us to review your PR. Please be patient, our engineers will look into your PR based on the queue we have, no need to ping them.
- Wait for us to review your PR.
- Every time you respond to our requested changes you must re-request a review in order for us to re-engage with the PR.
- Once the PR is in good shape, we will merge into main, and then you become an Ivy contributor!

### Important Notes

- If your PR is not created within 7 days of creating the issue, then a warning message will appear on the issue, we do this in order to keep our ToDo lists moving quickly,
- Please don't take it personally if your issue or PR gets closed because of this 7-day inactivity time limit.

- Finally, we limit the maximum number of open and incomplete sub-task issues to three per person.

For questions, please reach out on [discord](https://discord.gg/uYRmyPxMQq)!
23 changes: 9 additions & 14 deletions ivy/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -804,24 +804,19 @@ class Node(str):
except: # noqa: E722
pass
try:
from .compiler.compiler import (
clear_graph_cache,
from .tracer import trace_graph
from .transpiler import (
source_to_source,
transpile,
trace_graph,
)
except: # noqa: E722
pass # Added for the finally statement
try:
from .compiler.replace_with import replace_with, transform_function
except: # noqa: E722
pass
finally:
# Skip framework imports done by Ivy compiler for now
for backend_framework in _not_imported_backends.copy():
if backend_framework in sys.modules:
if backend_framework not in _imported_frameworks_before_compiler:
_not_imported_backends.remove(backend_framework)

# Skip framework imports done by Ivy compiler for now
for backend_framework in _not_imported_backends.copy():
if backend_framework in sys.modules:
if backend_framework not in _imported_frameworks_before_compiler:
_not_imported_backends.remove(backend_framework)


# add instance methods to Ivy Array and Container
Expand Down Expand Up @@ -1189,7 +1184,7 @@ def unset_dynamic_backend():


@handle_exceptions
def set_cython_wrappers_mode(flag: bool = True) -> None:
def set_cython_wrappers_mode(flag=True) -> None:
"""Set the mode of whether to use cython wrappers for functions.

Parameter
Expand Down
86 changes: 0 additions & 86 deletions ivy/compiler/LICENSE

This file was deleted.

Loading
Loading