Research OneDiff inference speed optimization [LPT]

[rickstaa]

Overview

Inference speed is crucial for our applications, and we aim to enhance the performance of our T2I and I2I pipelines. Previous optimizations like DeepCache have improved inference times but at the expense of image quality. We are interested in exploring whether another optimization, OneDiff, can provide better results. As we are currently focused on the core AI subnet software, we are inviting community builders to conduct this research in exchange for a software bounty. If OneDiff proves effective, we will create another bounty to implement this optimization in the T2I and I2I pipelines. The successful researcher will have the first opportunity to take on this new bounty, allowing them to implement this potentially valuable feature for the community and accelerate our pipelines ⚡.

Bounty Tips

Avoid testing DeepCache directly on the livepeer/ai-worker repository, as its complexity may hinder your process. Instead, create a simple inference Python script for your tests.

Requirements

• The submission must include a dedicated report outlining the inference speedup and output quality with and without the OneDiff optimization.
• The report should cover each of the specified models.
• Inference times should be averaged over 10 runs, and comparison images for each run should be provided.

How to Apply

1. Express Your Interest: Comment on the issue to let us know you're interested.
2. Wait for Review: Our team will review expressions of interest within 14 days and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Begin your work! For help or guidance, join the #🛋│developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on the GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: Once your pull request is approved, we'll arrange the bounty payment.

Thank you for your interest in contributing to our project 💛!