Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add IREE numerics test for Llama 3.1 8B FP16 TP8 #394

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add IREE numerics test for Llama 3.1 8B FP16 TP8 #394

wants to merge 1 commit into from
+435 −364

Commits on Oct 31, 2024

  1. Add IREE numerics test for Llama 3.1 8B FP16 TP8 

    Introduce a Llama 3.1 8B FP16 TP8 test that appears to not have good
numerical accuracy. It is compared to an fp64 unsharded torch variant
to ensure that the reference is of high accuracy.

Refactor the sharded Llama tests. Increase code reuse and use the
TorchGenerator in the toy-sized tests. Use the shard_llm_dataset and
export_paged_llm_v1 scripts in the test flow to increase their test
coverage.
    @sogartar
    sogartar committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    15fae20 View commit details
    Browse the repository at this point in the history