Unable to tune for ComfyUI via flux1 but work via replicate API

[Anashel-RPG]

I know it's not an issue with the node in ComfyUI, as the same image trained via the Replicate API service produces a model that works with the same workflow in ComfyUI. But the config below do not produce ComfyUI compatible safetensors.

I have gone through the entire process of building a solid Docker image for RunPod with Syncthing, and I've completed all the optimizations and permissions. When I train a test model, I can clearly see the changes in my prompts reflected in the validation folder. However, when I load the final model in ComfyUI, the images never change, regardless of the checkpoints used during training.

If I can figure this out, I’ll have a full pipeline from Docker to RunPod ready to go.

Setup: 1 x H100 SXM
26 vCPU 251 GB RAM
80 GB VRAM

Here are my config.env:

# SimpleTuner Configuration File
# Version 036 for RunPod

# 01 Model Configuration
#######################
export MODEL_TYPE='lora'
export MODEL_NAME="black-forest-labs/FLUX.1-dev"
export FLUX=true

# 02 Training Parameters
#######################
export LEARNING_RATE=5e-4
export MAX_NUM_STEPS=1000
export NUM_EPOCHS=0
export TRAIN_BATCH_SIZE=1
export GRADIENT_ACCUMULATION_STEPS=1
export VAE_BATCH_SIZE=4

# 03 Learning Rate Schedule
#########################
export LR_SCHEDULE="constant"
export LR_WARMUP_STEPS=10

# 04 Checkpointing and Validation
###############################
export CHECKPOINTING_STEPS=250
export CHECKPOINTING_LIMIT=4
export RESUME_CHECKPOINT="latest"
export VALIDATION_STEPS=250
export VALIDATION_NUM_INFERENCE_STEPS=40
export VALIDATION_SEED=777
export GRADIENT_PRECISION="bf16"

# 05 Additional Training Arguments
################################
export TRAINER_EXTRA_ARGS="--lora_rank=16 --adam_bfloat16 --max_grad_norm=1.0 --flux_lora_target=mmdit --lora_init_type=default --base_model_default_dtype=bf16 --user_prompt_library=config/user_prompt_library.json"


################################
# Advanced                                                #
################################

# Performance Optimizations
############################
export MAX_WORKERS=32
export READ_BATCH_SIZE=25
export WRITE_BATCH_SIZE=64
export IMAGE_PROCESSING_BATCH_SIZE=32
export AWS_MAX_POOL_CONNECTIONS=128
export TORCH_NUM_THREADS=8

# Advanced Training Options
############################
export TRAINING_SCHEDULER_TIMESTEP_SPACING="trailing"
export INFERENCE_SCHEDULER_TIMESTEP_SPACING="trailing"
export MIN_SNR_GAMMA=5
export USE_XFORMERS=false
export USE_GRADIENT_CHECKPOINTING=true
export ALLOW_TF32=true
export OPTIMIZER="adamw"
export USE_EMA=false
export EMA_DECAY=0.999

# Model Specific Settings
##########################
export STABLE_DIFFUSION_3=false
export PIXART_SIGMA=false
export STABLE_DIFFUSION_LEGACY=false
export KOLORS=false
export CONTROLNET=false
export USE_DORA=false

# Output and Logging
#####################
export OUTPUT_DIR="output/models"
export DEBUG_EXTRA_ARGS="--report_to=wandb"
export TRACKER_PROJECT_NAME="${MODEL_TYPE}-training"
export TRACKER_RUN_NAME="simpletuner-sdxl"

# Hugging Face Hub Integration
###############################
export PUSH_TO_HUB="false"
export PUSH_CHECKPOINTS="false"
export HUB_MODEL_NAME=$TRACKER_PROJECT_NAME

# Image Processing
###################
export RESOLUTION=1024
export RESOLUTION_TYPE="pixel"
export MINIMUM_RESOLUTION=$RESOLUTION

# Prompts and Captions
#######################
export VALIDATION_GUIDANCE=3
export VALIDATION_GUIDANCE_RESCALE=1.0
export VALIDATION_NEGATIVE_PROMPT="blurry, cropped, ugly"
export CAPTION_DROPOUT_PROBABILITY=0.1

# Reproducibility
##################
export TRAINING_SEED=31337

# Precision Settings
#####################
export BASE_MODEL_PRECISION="bf16"
export MIXED_PRECISION="bf16"
export PURE_BF16=false

# Distributed Training
#######################
export TRAINING_NUM_PROCESSES=1
export TRAINING_NUM_MACHINES=1
export ACCELERATE_EXTRA_ARGS=""

# Compilation Options
######################
export TRAINING_DYNAMO_BACKEND='no'
export TOKENIZERS_PARALLELISM=false

# Data Management
##################
export DELETE_ERRORED_IMAGES=0
export DELETE_SMALL_IMAGES=0
export DATALOADER_CONFIG="config/multidatabackend.json"
export METADATA_UPDATE_INTERVAL=65

And the multidatabackend.json :

[
  {
    "id": "Config-036",
    "type": "local",
	"dataset_type": "image",
    "crop": true,
    "crop_aspect": "square",
    "crop_style": "center",
    "resolution": 1024,
    "minimum_image_size": 1024,
    "maximum_image_size": 1024,
    "target_downsample_size": 1024,
    "resolution_type": "pixel",
    "cache_dir_vae": "cache/vae/flux/my_dataset",
    "instance_data_dir": "/workspace/SimpleTuner/datasets/my_dataset",
    "disabled": false,
	"skip_file_discovery": "",
    "caption_strategy": "filename",
    "metadata_backend": "json"
  },
  {
    "id": "text-embeds",
    "type": "local",
    "dataset_type": "text_embeds",
    "default": true,
    "cache_dir": "cache/text/flux/my_dataset",
	"disabled": false,
    "write_batch_size": 128
  }
]

Again:

• Work well, I see my validation image update
• I get all my checkpoint in my validation folder
• I get my model in ComfyUI and the node are correct
• I always validate first with the same image train on Simpletuner via Replicate API
• All the checkpoint does not change any image on ComfyUI.