[Inadequate documentation] What models are actually required to run Flux.2-dev?

[sniggyfigbat]

I'm referring to flux2.md, and it's extremely unclear which files are used for --diffusion-model, --vae, and --llm. In particular the link for vae takes you to a location with no file named 'flux2_ae.safetensors', and when I used the Comfy.org 'flux2-vae.safetensors' from here (which I suspect is what is actually intended?) that also doesn't seem to work.

More generally, the program fails without generating any kind of error output, making it very difficult to tell where I'm going wrong.

---

Using:

• Diffusion model: flux2-dev-Q5_K_M.gguf
• VAE: flux2-vae.safetensors
• LLM: Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf
• ./sd-master-1d6cb0f-bin-win-cuda12-x64/sd-cli --diffusion-model ./models/flux/flux2-dev-Q5_K_M.gguf --vae ./models/flux/flux2-vae.safetensors --llm ./models/flux/Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf -p "Example" -o ./output/output_%03d.png -b 20 --offload-to-cpu

I get the following output:

[DEBUG] main.cpp:515  - version: stable-diffusion.cpp version unknown, commit 1d6cb0f
[DEBUG] main.cpp:516  - System Info:
    SSE3 = 1 |     AVX = 1 |     AVX2 = 1 |     AVX512 = 1 |     AVX512_VBMI = 0 |     AVX512_VNNI = 0 |     FMA = 1 |     NEON = 0 |     ARM_FMA = 0 |     F16C = 1 |     FP16_VA = 0 |     WASM_SIMD = 0 |     VSX = 0 |
[DEBUG] main.cpp:517  - SDCliParams {
  mode: img_gen,
  output_path: "./output/output_%03d.png",
  verbose: true,
  color: false,
  canny_preprocess: false,
  convert_name: false,
  preview_method: none,
  preview_interval: 1,
  preview_path: "preview.png",
  preview_fps: 16,
  taesd_preview: false,
  preview_noisy: false
}
[DEBUG] main.cpp:518  - SDContextParams {
  n_threads: 16,
  model_path: "",
  clip_l_path: "",
  clip_g_path: "",
  clip_vision_path: "",
  t5xxl_path: "",
  llm_path: "./models/flux/Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf",
  llm_vision_path: "",
  diffusion_model_path: "./models/flux/flux2-dev-Q5_K_M.gguf",
  high_noise_diffusion_model_path: "",
  vae_path: "./models/flux/diffusion_pytorch_model-from-hugging-flux2dev-vae.safetensors",
  taesd_path: "",
  esrgan_path: "",
  control_net_path: "",
  embedding_dir: "",
  embeddings: {
  }
  wtype: NONE,
  tensor_type_rules: "",
  lora_model_dir: ".",
  photo_maker_path: "",
  rng_type: cuda,
  sampler_rng_type: NONE,
  offload_params_to_cpu: true,
  enable_mmap: false,
  control_net_cpu: false,
  clip_on_cpu: false,
  vae_on_cpu: false,
  flash_attn: false,
  diffusion_flash_attn: false,
  diffusion_conv_direct: false,
  vae_conv_direct: false,
  circular: false,
  circular_x: false,
  circular_y: false,
  chroma_use_dit_mask: true,
  qwen_image_zero_cond_t: false,
  chroma_use_t5_mask: false,
  chroma_t5_mask_pad: 1,
  prediction: NONE,
  lora_apply_mode: auto,
  force_sdxl_vae_conv_scale: false
}
[DEBUG] main.cpp:519  - SDGenerationParams {
  loras: "{
  }",
  high_noise_loras: "{
  }",
  prompt: "Example",
  negative_prompt: "",
  clip_skip: -1,
  width: -1,
  height: -1,
  batch_count: 20,
  init_image_path: "",
  end_image_path: "",
  mask_image_path: "",
  control_image_path: "",
  ref_image_paths: [],
  control_video_path: "",
  auto_resize_ref_image: true,
  increase_ref_index: false,
  pm_id_images_dir: "",
  pm_id_embed_path: "",
  pm_style_strength: 20,
  skip_layers: [7, 8, 9],
  sample_params: (txt_cfg: 7.00, img_cfg: 7.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: NONE, sample_method: NONE, sample_steps: 20, eta: 0.00, shifted_timestep: 0, flow_shift: inf),
  high_noise_skip_layers: [7, 8, 9],
  high_noise_sample_params: (txt_cfg: 7.00, img_cfg: 7.00, distilled_guidance: 3.50, slg.layer_count: 3, slg.layer_start: 0.01, slg.layer_end: 0.20, slg.scale: 0.00, scheduler: NONE, sample_method: NONE, sample_steps: 20, eta: 0.00, shifted_timestep: 0, flow_shift: inf),
  custom_sigmas: [],
  cache_mode: "",
  cache_option: "",
  cache: disabled (threshold=inf, start=0.15, end=0.95),
  moe_boundary: 0.875,
  video_frames: 1,
  fps: 16,
  vace_strength: 1,
  strength: 0.75,
  control_strength: 0.9,
  seed: 42,
  upscale_repeats: 1,
  upscale_tile_size: 128,
  vae_tiling_params: { 0, 0, 0, 0.5, 0, 0 },
}
[DEBUG] stable-diffusion.cpp:172  - Using CUDA backend
[INFO ] ggml_extend.hpp:81   - ggml_cuda_init: found 1 CUDA devices (Total VRAM: 8191 MiB):
[INFO ] ggml_extend.hpp:81   -   Device 0: NVIDIA GeForce RTX 2070 SUPER, compute capability 7.5, VMM: yes, VRAM: 8191 MiB
[INFO ] stable-diffusion.cpp:266  - loading diffusion model from './models/flux/flux2-dev-Q5_K_M.gguf'
[INFO ] model.cpp:330  - load ./models/flux/flux2-dev-Q5_K_M.gguf using gguf format
[DEBUG] model.cpp:376  - init from './models/flux/flux2-dev-Q5_K_M.gguf'
[INFO ] stable-diffusion.cpp:313  - loading llm from './models/flux/Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf'
[INFO ] model.cpp:330  - load ./models/flux/Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf using gguf format
[DEBUG] model.cpp:376  - init from './models/flux/Mistral-Small-3.2-24B-Instruct-2506-UD-Q5_K_XL.gguf'
[INFO ] stable-diffusion.cpp:327  - loading vae from './models/flux/diffusion_pytorch_model-from-hugging-flux2dev-vae.safetensors'
[INFO ] model.cpp:333  - load ./models/flux/diffusion_pytorch_model-from-hugging-flux2dev-vae.safetensors using safetensors format
[DEBUG] model.cpp:467  - init from './models/flux/diffusion_pytorch_model-from-hugging-flux2dev-vae.safetensors', prefix = 'vae.'
[INFO ] stable-diffusion.cpp:352  - Version: Flux.2
[INFO ] stable-diffusion.cpp:380  - Weight type stat:                      f32: 457  |    q8_0: 1    |    q4_K: 20   |    q5_K: 303  |    q6_K: 117  |    bf16: 11
[INFO ] stable-diffusion.cpp:381  - Conditioner weight type stat:          f32: 81   |    q8_0: 1    |    q4_K: 20   |    q5_K: 183  |    q6_K: 77
[INFO ] stable-diffusion.cpp:382  - Diffusion model weight type stat:      f32: 128  |    q5_K: 120  |    q6_K: 40   |    bf16: 11
[INFO ] stable-diffusion.cpp:383  - VAE weight type stat:                  f32: 248
[DEBUG] stable-diffusion.cpp:385  - ggml tensor size = 400 bytes
[DEBUG] llm.hpp:392  - vocab size: 131072
[DEBUG] llm.hpp:407  - merges size 269443

...And then it just exits back to prompt with no error output.

[wbruna]

I have never ran that model, but the files seem correct. You are most likely running out of VRAM, though: it would usually print an error, but the OS may be killing the process first.

ggml_cuda_init: found 1 CUDA devices (Total VRAM: 8191 MiB)
Size of remote file: 24.1 GB

That model won't fit inti an 8G video card. --offload-to-cpu doesn't work the same as LLM layer offloading: as a rule of thumb, you need enough free VRAM to fit the whole diffusion model file, plus working memory (and Flux.2 is notoriously resource-hungry in that regard, too).

Klein 9B at Q4 may fit, depending on how much VRAM is in use by other programs.

Also, note that -b 20 means "generate 20 images".