Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Modify Existing Triton Kernels to Support Newer Triton Versions #56

Closed
achew010 opened this issue Jul 29, 2024 · 1 comment
Closed

Modify Existing Triton Kernels to Support Newer Triton Versions #56

achew010 opened this issue Jul 29, 2024 · 1 comment
Assignees
@achew010
Labels
dependency Issue arises because of a dependency

Comments

@achew010
Copy link
Contributor

achew010 commented Jul 29, 2024 •
edited
Loading

Description:

Newer versions of torch (>= 2.4.0) requires installation of Triton version (>=3.0.0), this will cause an error in the JIT compile
of triton kernels that currently access global variables inside new checks here

Example:

NameError("Cannot access global variable ROPE_GROUP_SIZE from within @jit'ed function. Triton kernels can only access global variables that are annotated as constexpr (x: triton.language.constexpr = 42 or x = triton.language.constexpr(42)).  Alternatively, set the envvar TRITON_ALLOW_NON_CONSTEXPR_GLOBALS=1, but we do not promise to support this forever.")
  • This is currently mitigated using a TRITON_ALLOW_NON_CONSTEXPR_GLOBALS variable that temporarily allows for access to globals.
  • A fix to the triton kernels to access global variables by annotating as constexpr (x: triton.language.constexpr = 42 or x = triton.language.constexpr(42)) would be more permanent.
@fabianlim fabianlim added the dependency Issue arises because of a dependency label Jul 30, 2024
@fabianlim fabianlim mentioned this issue Jul 31, 2024
Merged
2 tasks
fabianlim added a commit that referenced this issue Jul 31, 2024
@fabianlim
annote triton constexpr #56
1ef30a9
fabianlim added a commit that referenced this issue Jul 31, 2024
@fabianlim
annote triton constexpr #56
ba579e3
fabianlim added a commit that referenced this issue Jul 31, 2024
@fabianlim
annote triton constexpr #56
50b9404 
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
fabianlim added a commit that referenced this issue Aug 1, 2024
@fabianlim
Address Package Bound and Triton Issues for Torch 2.4 (#58)
4ea65e0 
* release transformers bound

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* annote triton constexpr #56

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* fix some benchmark issues

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* more benchmark fixes

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

---------

Signed-off-by: Yu Chin Fabian Lim <[email protected]>
@fabianlim
Copy link
Contributor

Closing with the resolution of #58

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@achew010 achew010

Labels
dependency Issue arises because of a dependency
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fabianlim @achew010