SD3December26UNTESTED.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "name": "SD3December26UNTESTED.ipynb",
      "authorship_tag": "ABX9TyM9SLnXMQE1iLVk0byegu72",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Sewlell/KohyaColab/blob/main/SD3December26UNTESTED.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is honestly a super bare-foot notebook. You may encounter some errors in the middle of the way. If you encountered one, you can report those to my repository."
      ],
      "metadata": {
        "id": "PNC-ZqQW5gIa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Follow huggingface_hub's instruction so you can download SD3.\n",
        "\n",
        "import huggingface_hub\n",
        "\n",
        "!huggingface-cli login\n"
      ],
      "metadata": {
        "id": "Os2jGFDSATDD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n"
      ],
      "metadata": {
        "id": "eAnJNfVhA3YY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from huggingface_hub import hf_hub_download\n",
        "\n",
        "# Define the repository ID and the filename you want to download (i.e. stabilityai/stable-diffusion-3-medium)\n",
        "### If you want to download a model inside a folder, type it like this \"the_folder/pytorch_model_diffusion.safetensors\"\n",
        "repo_id = 'stabilityai/stable-diffusion-3-medium'\n",
        "filename = 'text_encoders/clip_g.safetensors'\n",
        "\n",
        "# Download the file\n",
        "file_path = hf_hub_download(repo_id=repo_id, filename=filename)\n",
        "\n",
        "print(f\"File downloaded to {file_path}\")\n"
      ],
      "metadata": {
        "id": "XDGavWUPFYhc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import zipfile\n",
        "import os\n",
        "\n",
        "# Path to the zip file in Google Drive\n",
        "zip_file_path = '/content/drive/MyDrive/LORAImage/SD++A1.zip'\n",
        "\n",
        "# Directory to extract the contents to\n",
        "extract_dir = '/content/dataset/'\n",
        "\n",
        "os.makedirs(extract_dir, exist_ok=True)\n",
        "\n",
        "# Extract the zip file\n",
        "with zipfile.ZipFile(zip_file_path, 'r') as zip_ref:\n",
        "    zip_ref.extractall(extract_dir)\n",
        "\n",
        "print(f'Extracted {zip_file_path} to {extract_dir}')\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HwR9mQYAndNn",
        "outputId": "23f193df-7973-459e-8f37-da561eafae79"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Extracted /content/drive/MyDrive/LORAImage/SD++A1.zip to /content/dataset/\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content\n",
        "!git clone -b sd3 https://github.com/kohya-ss/sd-scripts\n",
        "%cd /content/sd-scripts\n",
        "\n",
        "!pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124\n",
        "!pip install accelerate==0.33.0\n",
        "!pip install --upgrade -r requirements.txt\n",
        "#!pip install xformers==0.0.26.post1 --index-url https://download.pytorch.org/whl/cu124"
      ],
      "metadata": {
        "id": "4bFa6Z5JhEO3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Training**\n",
        "\n",
        "Hey, you need a dataset_config.toml first before anything. This notebook doesn't contain creation of that unfortunately at the time. (Still you can add your own here) Checkout this doc from kohya-ss (https://github.com/darkstorm2150/sd-scripts/blob/main/docs/config_README-en.md)\n",
        "\n",
        "Your dataset must be captioned first as well. You can tag your image automatically with the tagger (https://github.com/jhc13/taggui/)"
      ],
      "metadata": {
        "id": "2LaheXyx0Q21"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!accelerate launch --num_cpu_threads_per_process=2 \"/content/sd-scripts/sd3_train.py\" \\\n",
        "                  --enable_bucket --min_bucket_reso=256 --max_bucket_reso=1536 \\\n",
        "                  --pretrained_model_name_or_path=\"/root/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3-medium/snapshots/c7c317ea9032a4ef316cbb12e3e81f9700bb81cc/sd3_medium.safetensors\" \\\n",
        "                  --train_data_dir=\"hey put your absolute address here\" \\\n",
        "                  --optimizer_type=\"adafactor\" \\\n",
        "                  --learning_rate=5e-5 \\\n",
        "                  --optimizer_args=[ \"scale_parameter=False\", \"relative_step=False\", \"warmup_init=False\" ] \\\n",
        "                  --vae_batch_size=2 --text_encoder_batch_size=3 \\\n",
        "                  --caption_extension=\".txt\" --output_name=\"SD++A2\" --save_model_as=safetensors \\\n",
        "                  --save_precision=\"fp16\" --train_batch_size=\"2\" \\\n",
        "                  --lr_scheduler=\"constant\" --resolution=\"1024,1024\" --save_every_n_epochs=\"2\" --mixed_precision=\"fp16\" --lr_scheduler_num_cycles=\"20\" \\\n",
        "                  --clip_g=\"/root/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3-medium/snapshots/c7c317ea9032a4ef316cbb12e3e81f9700bb81cc/text_encoders/clip_g.safetensors\" \\\n",
        "                  --clip_l=\"/root/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3-medium/snapshots/c7c317ea9032a4ef316cbb12e3e81f9700bb81cc/text_encoders/clip_l.safetensors\" \\\n",
        "                  --t5xxl=\"/root/.cache/huggingface/hub/models--stabilityai--stable-diffusion-3-medium/snapshots/c7c317ea9032a4ef316cbb12e3e81f9700bb81cc/text_encoders/t5xxl_fp8_e4m3fn.safetensors\" \\\n",
        "                  --max_train_epochs=4 \\\n",
        "                  --sample_prompts=\"1girl, solo, bangs, blonde, yellow eyes, short hair, :o, spear, treant, ornament, pants, leather jacket, leather boots, fennec ears, fennec girl, scarf, accessories, medieval, dynamic pose, pose, stance, dynamic stance, masterpiece, best quality, detailed,\" \\\n",
        "                  --sample_sampler=ddim \\\n",
        "                  --sample_every_n_epochs=2 \\\n",
        "                  --cache_text_encoder_outputs --cache_latents \\\n",
        "                  --output_dir=\"/content/drive/MyDrive/LORAImage\" --xformers --t5xxl_dtype=\"fp8\" --dataset_config=\"/content/dataset_config.toml\""
      ],
      "metadata": {
        "id": "Iogx1LRY-PD0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "`--train_data_dir` change your train_data_dir to your desired folder.\\\n",
        "\n",
        "`--optimizer_type`  original repository reccomended adafactor as `--optimizer_type`, but again you can experimenting your own with dadaption or adamw (adamw8bit with --use_8bit_adam) \\\n",
        "\n",
        "`--learning_rate` knob the learning rate along with the batch size. it could influence the result heavily \\\n",
        "\n",
        "`--caption_extension` keep the `caption_extension` .txt unless your caption is other format. \\\n",
        "\n",
        "`--output_name` is your project name, for `save_model_as` keep it safetensors unless you know what you really wanted. \\\n",
        "\n",
        "`--lr_scheduler` feel free to change lr_scheduler with `constant_with_warmup` or `cosine`, keep the `resolution` to either 512,512 , 756,756 , or 1024,1024 \\\n",
        "\n",
        "change `--sample_prompts` if you dont want my default prompt. you can try to add some command like `--h` (height) `--w` (wide) `--n` (negative prompt) `--s` (steps) `--l` (cfg). Prompt weighting do not worked. \\\n",
        "\n",
        "you got quite a lot of choices here but primarily you can choose default `ddim` and `euler`. DO NOT USE ANCESTRAL VARIANTS (euler_a, dpm_2_ancestral) AND SDE (dpmpp_2m_sde), SD3 ISN'T COMPATIBLE WITH THOSE FUNDAMENTALLY.\n",
        "\n",
        "change your `output_dir` and `dataset_config` to your desired folder."
      ],
      "metadata": {
        "id": "5JToy7JA--Ny"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Run the cell below if you know what you are doing and the errors show above.**"
      ],
      "metadata": {
        "id": "wiX_I41zy6Us"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Changing transformers version.\n",
        "\n",
        "!pip install transformers==4.28.0"
      ],
      "metadata": {
        "id": "bdGj7PweX555"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!accelerate config"
      ],
      "metadata": {
        "id": "UqkZ8_wdJfmE"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}