Code

merge#1232

Code

Merged

bghira merged 76 commits intoreleasebghira/SimpleTuner:releasefrom mainbghira/SimpleTuner:mainCopy head branch name to clipboardDec 19, 2024

+2,770-6,504

Commits

Commits on Dec 4, 2024

• sana: add to configurator

  

  bghira

  committed
  4106533

  Copy full SHA for 4106533

• sana: text embed caching

  

  bghira

  committed
  3c3b51f

  Copy full SHA for 3c3b51f

• modify vaecache to support Sana return values

  

  bghira

  committed
  e671730

  Copy full SHA for e671730

• add sana to --model_family and enforce "no" mixed_precision

  

  bghira

  committed
  f605790

  Copy full SHA for f605790

• allow exiting on configuration load error

  

  bghira

  committed
  a711d73

  Copy full SHA for a711d73

• add sana configuration safety checks

  

  bghira

  committed
  d01dd32

  Copy full SHA for d01dd32

• sana: model loading

  

  bghira

  committed
  a642d9a

  Copy full SHA for a642d9a

• bugfix for fp16 training and bf16 optim being disabled

  

  bghira

  committed
  12220e4

  Copy full SHA for 12220e4

• savehooks: enable saving and resuming sana models

  

  bghira

  committed
  442d0ea

  Copy full SHA for 442d0ea

• add sana model to list

  

  bghira

  committed
  f513d41

  Copy full SHA for f513d41

• sana: load gemma2 text encoder

  

  bghira

  committed
  0e12e11

  Copy full SHA for 0e12e11

• sana: support for model validations, noise scheduler

  

  bghira

  committed
  f840fe2

  Copy full SHA for f840fe2

• trainer: exit on error when loading config

  

  bghira

  committed
  2fb5764

  Copy full SHA for 2fb5764

• trainer: add sana support, remove --validation_on_startup logic since we have benchmarking now

  

  bghira

  committed
  61e58eb

  Copy full SHA for 61e58eb

• sana: quickstart

  

  bghira

  committed
  baa68db

  Copy full SHA for baa68db

• sana: add to readme

  

  bghira

  committed
  99a9f27

  Copy full SHA for 99a9f27

• add sana to quickstart list

  

  bghira

  committed
  f54e40d

  Copy full SHA for f54e40d

• update diffusers for sana dependencies

  

  bghira

  committed
  bc0b81e

  Copy full SHA for bc0b81e

• Merge pull request #1187 from bghira/feature/sana

  Show description for d5124f9
  bghira

  authored
  d5124f9

  Copy full SHA for d5124f9

• update sana toc link

  

  bghira

  committed
  5222bf2

  Copy full SHA for 5222bf2

• Merge pull request #1188 from bghira/feature/sana

  Show description for 1ea2858
  bghira

  authored
  1ea2858

  Copy full SHA for 1ea2858

• Update SANA.md

  bghira

  authored
  783a3a9

  Copy full SHA for 783a3a9

• sana: remove autocast

  

  bghira

  committed
  462f4cd

  Copy full SHA for 462f4cd

• remove print statement

  

  bghira

  committed
  ee063c4

  Copy full SHA for ee063c4

Commits on Dec 6, 2024

• update sd3 seqlen to 154 max for t5

  

  bghira

  committed
  ae8bd1d

  Copy full SHA for ae8bd1d

• Merge pull request #1190 from bghira/bugfix/sd3-t5-seqlen

  Show description for 80c263d
  bghira

  authored
  80c263d

  Copy full SHA for 80c263d

• multigpu rename error masking

  

  bghira

  committed
  4b73d90

  Copy full SHA for 4b73d90

Commits on Dec 7, 2024

• deepspeed: disable annoying log info prints

  

  bghira

  committed
  e402ae5

  Copy full SHA for e402ae5

• remove test log prints

  

  bghira

  committed
  cf79b38

  Copy full SHA for cf79b38

• apple: update dependencies

  

  bghira

  committed
  e63ce4e

  Copy full SHA for e63ce4e

• Merge pull request #1192 from bghira/chore/log-cleanup

  Show description for ff628b5
  bghira

  authored
  ff628b5

  Copy full SHA for ff628b5

Commits on Dec 10, 2024

• update diffusers branch for Sana

  

  bghira

  committed
  f5eb71a

  Copy full SHA for f5eb71a

Commits on Dec 11, 2024

• trainer should not disable logs after configuring them

  

  bghira

  committed
  3ad04ea

  Copy full SHA for 3ad04ea

• add --grad_clip_method with options for norm or max value

  

  bghira

  committed
  217f6fc

  Copy full SHA for 217f6fc

• Merge pull request #1205 from bghira/feature/clip-value-vs-norm

  Show description for eb80f5f
  bghira

  authored
  eb80f5f

  Copy full SHA for eb80f5f

• local backend: use atomicwrites library to resolve rename errors and parallel overwrites

  

  bghira

  committed
  0b33498

  Copy full SHA for 0b33498

• add atomicwrites dependency

  

  bghira

  committed
  3f6054d

  Copy full SHA for 3f6054d

• max_grad_norm value limit removal for sd3

  

  bghira

  committed
  f2736cd

  Copy full SHA for f2736cd

• Merge pull request #1207 from bghira/chore/remove-sd3-grad-norm-limit

  Show description for b06c4a3
  bghira

  authored
  b06c4a3

  Copy full SHA for b06c4a3

• Merge pull request #1206 from bghira/feature/atomicwrites-library-for-embeddings

  Show description for ea66ac8
  bghira

  authored
  ea66ac8

  Copy full SHA for ea66ac8

• apple: update quanto dependency to upstream repository

  

  bghira

  committed
  9470b8b

  Copy full SHA for 9470b8b

• fix log output of grad_norm

  

  bghira

  committed
  71bd7e6

  Copy full SHA for 71bd7e6

• Merge pull request #1208 from bghira/chore/apple-quanto-dependency

  Show description for d3cb505
  bghira

  authored
  d3cb505

  Copy full SHA for d3cb505

• swith clip method to "value" by default

  

  bghira

  committed
  1492efc

  Copy full SHA for 1492efc

• Merge pull request #1210 from bghira/change/default-norm-method

  Show description for e5c1cfd
  bghira

  authored
  e5c1cfd

  Copy full SHA for e5c1cfd

Commits on Dec 12, 2024

• add vae in example

  MrTuanDao

  authored
  e1b48bc

  Copy full SHA for e1b48bc

• Merge pull request #1212 from MrTuanDao/patch-2

  Show description for 636cd7f
  bghira

  authored
  636cd7f

  Copy full SHA for 636cd7f

Commits on Dec 13, 2024

• sana: work with bf16 weights

  

  bghira

  committed
  e9614f5

  Copy full SHA for e9614f5

• use terminusresearch source for sana 1.6b weights by default so that we use bf16 without variant shenanagins

  

  bghira

  committed
  e135751

  Copy full SHA for e135751

• sana: final pipeline export for full tune

  

  bghira

  committed
  e3791b9

  Copy full SHA for e3791b9

• sana: update quickstart to mention bf16 weights

  

  bghira

  committed
  82aacec

  Copy full SHA for 82aacec

• Merge pull request #1213 from bghira/sana/bf16-weights-fixes

  Show description for 4eb7aee
  bghira

  authored
  4eb7aee

  Copy full SHA for 4eb7aee

• configurator should avoid asking about checkpointing intervals when the model family does not support it

  

  bghira

  committed
  1014665

  Copy full SHA for 1014665

• Merge pull request #1214 from bghira/chore/configurator-grad-checkpointing-interval-model-compat

  Show description for 749ae60
  bghira

  authored
  749ae60

  Copy full SHA for 749ae60

• vaecache: sana should grab .latent object

  

  bghira

  committed
  4042365

  Copy full SHA for 4042365

• Merge pull request #1215 from bghira/bugfix/sana-vae-cache

  Show description for b323d1b
  bghira

  authored
  b323d1b

  Copy full SHA for b323d1b

• sana: add complex human instruction to user prompts by default (untested)

  

  bghira

  committed
  b3cb108

  Copy full SHA for b3cb108

Commits on Dec 14, 2024

• safety_check: Fix gradient checkpointing interval error message

  clayne

  committed
  ad47a58

  Copy full SHA for ad47a58

• Merge pull request #1221 from clayne/1734205326-gradient-checkpointing-interval-error-fix

  Show description for 2d52cda
  bghira

  authored
  2d52cda

  Copy full SHA for 2d52cda

Commits on Dec 17, 2024

• sana: no CHI for negative prompt

  

  bghira

  committed
  d3500aa

  Copy full SHA for d3500aa

• Merge pull request #1216 from bghira/feature/sana-complex-human-instruction

  Show description for fb58cef
  bghira

  authored
  fb58cef

  Copy full SHA for fb58cef

Commits on Dec 18, 2024

• diffusers: bump to main branch instead of Sana branch

  

  bghira

  committed
  c3d573e

  Copy full SHA for c3d573e

• flux: use rank 0 for h100 detection since that is the most realistic setup

  

  bghira

  committed
  90f4ba4

  Copy full SHA for 90f4ba4

• Merge pull request #1225 from bghira/bugfix/flux-multinode-rank-detection

  Show description for e9c9ef0
  bghira

  authored
  e9c9ef0

  Copy full SHA for e9c9ef0

• Merge pull request #1226 from bghira/feature/sana-complex-human-instruction

  Show description for 020bf37
  bghira

  authored
  020bf37

  Copy full SHA for 020bf37

• torchao: update to v0.7.0

  

  bghira

  committed
  3662525

  Copy full SHA for 3662525

• Merge pull request #1224 from bghira/chore/torchao-update

  Show description for aad95ad
  bghira

  authored
  aad95ad

  Copy full SHA for aad95ad

• deepspeed from 0.15 to 0.16.1

  

  bghira

  committed
  9a6cbc7

  Copy full SHA for 9a6cbc7

• Merge pull request #1227 from bghira/chore/deepspeed-update

  Show description for 3bd614d
  bghira

  authored
  3bd614d

  Copy full SHA for 3bd614d

• accelerate: from v0.34 to v1.2

  

  bghira

  committed
  a711e9f

  Copy full SHA for a711e9f

• Merge pull request #1228 from bghira/chore/accelerate-update

  Show description for dcc9ec5
  bghira

  authored
  dcc9ec5

  Copy full SHA for dcc9ec5

• quanto: bump to newer commit

  

  bghira

  committed
  c65b299

  Copy full SHA for c65b299

• update transformers, wandb, boto3, torchsde, numpy, peft, tensorboard

  

  bghira

  committed
  1dc94dc

  Copy full SHA for 1dc94dc

• Merge pull request #1229 from bghira/chore/accelerate-update

  Show description for de9a7c3
  bghira

  authored
  de9a7c3

  Copy full SHA for de9a7c3

• sd3: allow setting grad checkpointing interval

  

  bghira

  committed
  18fe891

  Copy full SHA for 18fe891

• Merge pull request #1230 from bghira/feature/sd3-checkpointing-interval

  Show description for 7815be2
  bghira

  authored
  7815be2

  Copy full SHA for 7815be2