Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support CUDA Graph for MoE models #1233

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Support CUDA Graph for MoE models #1233

wants to merge 13 commits into from

Commits on Oct 9, 2024

  1. Align RNG tracker with megatron 

    Signed-off-by: Robin Zhang <[email protected]>
Co-authored-by: Yifei Song <[email protected]>
    @buptzyb @yifeis-nv
    buptzyb and yifeis-nv committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    a95c7e3 View commit details
    Browse the repository at this point in the history

  2. Fix module_params order and warmup bug in cudagraph 

    Signed-off-by: Robin Zhang <[email protected]>
Co-authored-by: Yifei Song <[email protected]>
    @buptzyb @yifeis-nv
    buptzyb and yifeis-nv committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    1c9e35b View commit details
    Browse the repository at this point in the history

  3. Add fp8_group argument and fix fp8 accuracy issue for cudagraph 

    Signed-off-by: Robin Zhang <[email protected]>
Co-authored-by: Yifei Song <[email protected]>
    @buptzyb @yifeis-nv
    buptzyb and yifeis-nv committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    cef94e4 View commit details
    Browse the repository at this point in the history

  4. Add TE modules and weights filters to support MoE models 

    Signed-off-by: Robin Zhang <[email protected]>
Co-authored-by: Yifei Song <[email protected]>
    @buptzyb @yifeis-nv
    buptzyb and yifeis-nv committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    34967b6 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Merge branch 'main' into cudagraph_moe

    @timmoon10
    timmoon10 authored Oct 10, 2024
    Configuration menu
    Copy the full SHA
    076414c View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. Revert self.fp8 

    Signed-off-by: Robin Zhang <[email protected]>
    @buptzyb
    buptzyb committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    88b614c View commit details
    Browse the repository at this point in the history

  2. Use hooks to filter module params 

    Signed-off-by: Robin Zhang <[email protected]>
    @buptzyb
    buptzyb committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    10abcde View commit details
    Browse the repository at this point in the history

  3. Filter all TE modules in hooks 

    Signed-off-by: Robin Zhang <[email protected]>
Co-authored-by: Yifei Song <[email protected]>
    @buptzyb @yifeis-nv
    buptzyb and yifeis-nv committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    f920486 View commit details
    Browse the repository at this point in the history

  4. Format code 

    Signed-off-by: Robin Zhang <[email protected]>
    @buptzyb
    buptzyb committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    0c378e5 View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d639cfe View commit details
    Browse the repository at this point in the history

  6. Update graph.py 

    Signed-off-by: Xin Yao <[email protected]>
    @yaox12
    yaox12 authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    13208f3 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. Revert CudaRNGStatesTracker 

    Signed-off-by: Robin Zhang <[email protected]>
    @buptzyb
    buptzyb committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    f92d901 View commit details
    Browse the repository at this point in the history

  2. Format Update 

    Signed-off-by: Yifei Song <[email protected]>
    @yifeis-nv
    yifeis-nv committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    e6e1eeb View commit details
    Browse the repository at this point in the history