[WIP][Core] feat: enable true cfg in hunyuanvideo.

[sayakpaul]

What does this PR do?

Similar to how it's done in Flux, this PR adds support for doing CFG in HunyuanVideo.

Here are results: https://wandb.ai/sayakpaul/hunyuanvideo_cfg/runs/41dhd69p

Code

import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import export_to_video
import argparse

prompt = "A cat walks on the grass, realistic"
negative_prompt = "worst quality, inconsistent motion, blurry, jittery, distorted"

def load_pipeline():
    model_id = "hunyuanvideo-community/HunyuanVideo"
    transformer = HunyuanVideoTransformer3DModel.from_pretrained(
        model_id, subfolder="transformer", torch_dtype=torch.bfloat16
    )
    pipe = HunyuanVideoPipeline.from_pretrained(
        model_id, transformer=transformer, torch_dtype=torch.float16
    ).to("cuda")
    return pipe

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--true_cfg_scale", type=float, default=1.0)
    args = parser.parse_args()

    pipe = load_pipeline()
    output = pipe(
        prompt="A cat walks on the grass, realistic",
        negative_prompt=negative_prompt,
        true_cfg_scale=args.true_cfg_scale,
        height=320,
        width=512,
        num_frames=61,
        generator=torch.manual_seed(0),
    ).frames[0]
    path = f"output_cfg@{args.true_cfg_scale}.mp4"
    export_to_video(output, path, fps=15)

TODOs

• Add docs
• Add support for negative embeds in __call__()
• Add tests

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.