type: user

role: user

content:
I need to figure out the VRAM usage of the FireRed-Edit-1.1 model to estimate concurrency on an A100 GPU.

The model is loaded like this (from diffusers):
```python
from diffusers import QwenImageEditPlusPipeline
self._pipe = QwenImageEditPlusPipeline.from_pretrained(
    "FireRedTeam/FireRed-Image-Edit-1.1",
    torch_dtype=torch.bfloat16,
).to("cuda")
```

Then a Lightning LoRA is loaded on top:
```python
self._pipe.load_lora_weights(
    "FireRedTeam/FireRed-Image-Edit-LoRA-Zoo",
    weight_name="FireRed-Image-Edit-Lightning-8steps-v1.0.safetensors",
    adapter_name="lightning",
)
```

Inference runs with 6 steps, cfg=1.0, single image at a time.

I need to know:
1. What is the base architecture (is it SDXL-based, Flux-based, or something else)?
2. What are the component sizes (UNet/transformer, VAE, text encoder)?
3. Approximate total VRAM for model weights in bfloat16
4. Approximate VRAM for inference (activations, KV cache, intermediate tensors) for a single image
5. Can we estimate max batch size that fits on A100 80GB? And A100 40GB?

Search the web for FireRed-Edit-1.1 model card, architecture details, and VRAM requirements. Also check the HuggingFace model page.

userType: external

entrypoint: cli

gitBranch: HEAD

slug: encapsulated-foraging-sutton


type: message

role: assistant

type: text

text: I'll research the FireRed-Edit-1.1 model architecture and VRAM requirements. Let me start by looking at available information.

service_tier: standard

type: assistant


Bash

command: pip install huggingface_hub 2>/dev/null | tail -1

description: Install huggingface_hub for API access

type: direct


content: Requirement already satisfied: certifi>=2017.4.17 in ./.local/lib/python3.10/site-packages (from requests->huggingface_hub) (2026.2.25)


command:
python3 -c "
from huggingface_hub import HfApi, hf_hub_url, model_info
import json

api = HfApi()
info = api.model_info('FireRedTeam/FireRed-Image-Edit-1.1', files_metadata=True)
print('Model ID:', info.modelId)
print('Tags:', info.tags)
print('Pipeline tag:', info.pipeline_tag)
print('Library:', info.library_name)
print()
print('Files:')
for s in info.siblings:
    size_mb = s.size / (1024*1024) if s.size else 0
    print(f'  {s.rfilename:60s} {size_mb:>10.1f} MB')
" 2>&1

description: Get model info and file sizes from HuggingFace


content:
Model ID: FireRedTeam/FireRed-Image-Edit-1.1
Tags: ['diffusers', 'safetensors', 'image-to-image', 'en', 'zh', 'arxiv:2602.13344', 'license:apache-2.0', 'diffusers:QwenImageEditPlusPipeline', 'region:us']
Pipeline tag: image-to-image
Library: diffusers

Files:
  .gitattributes                                                      0.0 MB
  README.md                                                           0.0 MB
  model_index.json                                                    0.0 MB
  processor/added_tokens.json                                         0.0 MB
  processor/chat_template.jinja                                       0.0 MB
  processor/merges.txt                                                1.6 MB
  processor/preprocessor_config.json                                  0.0 MB
  processor/special_tokens_map.json                                   0.0 MB
  processor/tokenizer.json                                           10.9 MB
  processor/tokenizer_config.json                                     0.0 MB
  processor/video_preprocessor_config.json                            0.0 MB
  processor/vocab.json                                                2.6 MB
  scheduler/scheduler_config.json                                     0.0 MB
  text_encoder/config.json                                            0.0 MB
  text_encoder/generation_config.json                                 0.0 MB
  text_encoder/model-00001-of-00004.safetensors                    4738.1 MB
  text_encoder/model-00002-of-00004.safetensors                    4760.3 MB
  text_encoder/model-00003-of-00004.safetensors                    4704.2 MB
  text_encoder/model-00004-of-00004.safetensors                    1613.5 MB
  text_encoder/model.safetensors.index.json                           0.1 MB
  tokenizer/added_tokens.json                                         0.0 MB
  tokenizer/chat_template.jinja                                       0.0 MB
  tokenizer/merges.txt                                                1.6 MB
  tokenizer/special_tokens_map.json                                   0.0 MB
  tokenizer/tokenizer_config.json                                     0.0 MB
  tokenizer/vocab.json                                                3.2 MB
  transformer/config.json                                             0.0 MB
  transformer/diffusion_pytorch_model-00001-of-00005.safetensors     9511.5 MB
  transformer/diffusion_pytorch_model-00002-of-00005.safetensors     9524.7 MB
  transformer/diffusion_pytorch_model-00003-of-00005.safetensors     9524.6 MB
  transformer/diffusion_pytorch_model-00004-of-00005.safetensors     9470.6 MB
  transformer/diffusion_pytorch_model-00005-of-00005.safetensors      936.6 MB
  transformer/diffusion_pytorch_model.safetensors.index.json          0.2 MB
  vae/config.json                                                     0.0 MB
  vae/diffusion_pytorch_model.safetensors                           242.0 MB


command:
python3 -c "
from huggingface_hub import HfApi
api = HfApi()
info = api.model_info('FireRedTeam/FireRed-Image-Edit-LoRA-Zoo', files_metadata=True)
print('LoRA Zoo files:')
for s in info.siblings:
    size_mb = s.size / (1024*1024) if s.size else 0
    if 'Lightning' in s.rfilename or 'lightning' in s.rfilename or s.rfilename.endswith('.safetensors'):
        print(f'  {s.rfilename:60s} {size_mb:>10.1f} MB')
" 2>&1

description: Get LoRA zoo file sizes

speed: standard


content:
LoRA Zoo files:
  FireRed-Image-Edit-1.0-Lightning-8steps-v1.1.safetensors          810.2 MB
  FireRed-Image-Edit-Covercraft.safetensors                         561.4 MB
  FireRed-Image-Edit-Lightning-8steps-v1.0.safetensors              810.2 MB
  FireRed-Image-Edit-Makeup.safetensors                             564.1 MB


text: Now let me get the config files for the transformer and text encoder to understand the architecture.


command:
python3 -c "
from huggingface_hub import hf_hub_download
import json

# Get transformer config
path = hf_hub_download('FireRedTeam/FireRed-Image-Edit-1.1', 'transformer/config.json')
with open(path) as f:
    print('=== TRANSFORMER CONFIG ===')
    print(json.dumps(json.load(f), indent=2))
" 2>&1

description: Download transformer config


content:
=== TRANSFORMER CONFIG ===
{
  "_class_name": "QwenImageTransformer2DModel",
  "_diffusers_version": "0.36.0.dev0",
  "attention_head_dim": 128,
  "axes_dims_rope": [
    16,
    56,
    56
  ],
  "guidance_embeds": false,
  "in_channels": 64,
  "joint_attention_dim": 3584,
  "num_attention_heads": 24,
  "num_layers": 60,
  "out_channels": 16,
  "patch_size": 2
}


command:
python3 -c "
from huggingface_hub import hf_hub_download
import json

path = hf_hub_download('FireRedTeam/FireRed-Image-Edit-1.1', 'text_encoder/config.json')
with open(path) as f:
    print('=== TEXT ENCODER CONFIG ===')
    print(json.dumps(json.load(f), indent=2))
" 2>&1

description: Download text encoder config


content:
=== TEXT ENCODER CONFIG ===
{
  "architectures": [
    "Qwen2_5_VLForConditionalGeneration"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "dtype": "bfloat16",
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 3584,
  "initializer_range": 0.02,
  "intermediate_size": 18944,
  "max_position_embeddings": 128000,
  "max_window_layers": 28,
  "model_type": "qwen2_5_vl",
  "num_attention_heads": 28,
  "num_hidden_layers": 28,
  "num_key_value_heads": 4,
  "rms_norm_eps": 1e-06,
  "rope_scaling": {
    "mrope_section": [
      16,
      24,
      24
    ],
    "rope_type": "default",
    "type": "default"
  },
  "rope_theta": 1000000.0,
  "sliding_window": 32768,
  "text_config": {
    "architectures": [
      "Qwen2_5_VLForConditionalGeneration"
    ],
    "attention_dropout": 0.0,
    "bos_token_id": 151643,
    "dtype": "bfloat16",
    "eos_token_id": 151645,
    "hidden_act": "silu",
    "hidden_size": 3584,
    "image_token_id": 151655,
    "initializer_range": 0.02,
    "intermediate_size": 18944,
    "layer_types": [
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention",
      "full_attention"
    ],
    "max_position_embeddings": 128000,
    "max_window_layers": 28,
    "model_type": "qwen2_5_vl_text",
    "num_attention_heads": 28,
    "num_hidden_layers": 28,
    "num_key_value_heads": 4,
    "rms_norm_eps": 1e-06,
    "rope_scaling": {
      "mrope_section": [
        16,
        24,
        24
      ],
      "rope_type": "default",
      "type": "default"
    },
    "rope_theta": 1000000.0,
    "sliding_window": null,
    "use_cache": true,
    "use_sliding_window": false,
    "video_token_id": 151656,
    "vision_end_token_id": 151653,
    "vision_start_token_id": 151652,
    "vision_token_id": 151654,
    "vocab_size": 152064
  },
  "tie_word_embeddings": false,
  "transformers_version": "4.57.1",
  "use_cache": true,
  "use_sliding_window": false,
  "vision_config": {
    "depth": 32,
    "dtype": "bfloat16",
    "fullatt_block_indexes": [
      7,
      15,
      23,
      31
    ],
    "hidden_act": "silu",
    "hidden_size": 1280,
    "in_channels": 3,
    "in_chans": 3,
    "initializer_range": 0.02,
    "intermediate_size": 3420,
    "model_type": "qwen2_5_vl",
    "num_heads": 16,
    "out_hidden_size": 3584,
    "patch_size": 14,
    "spatial_merge_size": 2,
    "spatial_patch_size": 14,
    "temporal_patch_size": 2,
    "tokens_per_second": 2,
    "window_size": 112
  },
  "vision_token_id": 151654,
  "vocab_size": 152064
}


command:
python3 -c "
from huggingface_hub import hf_hub_download
import json

for fname in ['vae/config.json', 'model_index.json', 'scheduler/scheduler_config.json']:
    path = hf_hub_download('FireRedTeam/FireRed-Image-Edit-1.1', fname)
    with open(path) as f:
        print(f'=== {fname} ===')
        print(json.dumps(json.load(f), indent=2))
        print()
" 2>&1

description: Download VAE config, model index, and scheduler config


content:
=== vae/config.json ===
{
  "_class_name": "AutoencoderKLQwenImage",
  "_diffusers_version": "0.36.0.dev0",
  "attn_scales": [],
  "base_dim": 96,
  "dim_mult": [
    1,
    2,
    4,
    4
  ],
  "dropout": 0.0,
  "latents_mean": [
    -0.7571,
    -0.7089,
    -0.9113,
    0.1075,
    -0.1745,
    0.9653,
    -0.1517,
    1.5508,
    0.4134,
    -0.0715,
    0.5517,
    -0.3632,
    -0.1922,
    -0.9497,
    0.2503,
    -0.2921
  ],
  "latents_std": [
    2.8184,
    1.4541,
    2.3275,
    2.6558,
    1.2196,
    1.7708,
    2.6052,
    2.0743,
    3.2687,
    2.1526,
    2.8652,
    1.5579,
    1.6382,
    1.1253,
    2.8251,
    1.916
  ],
  "num_res_blocks": 2,
  "temperal_downsample": [
    false,
    true,
    true
  ],
  "z_dim": 16
}

=== model_index.json ===
{
  "_class_name": "QwenImageEditPlusPipeline",
  "_diffusers_version": "0.36.0.dev0",
  "processor": [
    "transformers",
    "Qwen2VLProcessor"
  ],
  "scheduler": [
    "diffusers",
    "FlowMatchEulerDiscreteScheduler"
  ],
  "text_encoder": [
    "transformers",
    "Qwen2_5_VLForConditionalGeneration"
  ],
  "tokenizer": [
    "transformers",
    "Qwen2Tokenizer"
  ],
  "transformer": [
    "diffusers",
    "QwenImageTransformer2DModel"
  ],
  "vae": [
    "diffusers",
    "AutoencoderKLQwenImage"
  ]
}

=== scheduler/scheduler_config.json ===
{
  "_class_name": "FlowMatchEulerDiscreteScheduler",
  "_diffusers_version": "0.36.0.dev0",
  "base_image_seq_len": 256,
  "base_shift": 0.5,
  "invert_sigmas": false,
  "max_image_seq_len": 8192,
  "max_shift": 0.9,
  "num_train_timesteps": 1000,
  "shift": 1.0,
  "shift_terminal": 0.02,
  "stochastic_sampling": false,
  "time_shift_type": "exponential",
  "use_beta_sigmas": false,
  "use_dynamic_shifting": true,
  "use_exponential_sigmas": false,
  "use_karras_sigmas": false
}


text: Now let me compute exact parameter counts and VRAM estimates.


command:
python3 << 'PYEOF'
import json

# ============================================
# 1. TRANSFORMER: QwenImageTransformer2DModel
# ============================================
# Based on Flux-like architecture (flow matching, joint attention)
# Config: 60 layers, 24 heads, head_dim=128, so hidden_dim = 24*128 = 3072
# joint_attention_dim = 3584 (text encoder hidden size)
# in_channels=64, out_channels=16, patch_size=2

# File sizes from HF (these are safetensors = raw bf16 weights)
transformer_bytes = (9511.5 + 9524.7 + 9524.6 + 9470.6 + 936.6) * 1024 * 1024
transformer_gb = transformer_bytes / (1024**3)
# In bf16, each param = 2 bytes
transformer_params_b = transformer_bytes / 2 / 1e9

print("=" * 60)
print("COMPONENT SIZES (from safetensors file sizes, bf16)")
print("=" * 60)
print(f"Transformer: {transformer_gb:.2f} GB  (~{transformer_params_b:.1f}B params)")

# ============================================
# 2. TEXT ENCODER: Qwen2.5-VL (7B class)
# ============================================
text_encoder_bytes = (4738.1 + 4760.3 + 4704.2 + 1613.5) * 1024 * 1024
text_encoder_gb = text_encoder_bytes / (1024**3)
text_encoder_params_b = text_encoder_bytes / 2 / 1e9
print(f"Text Encoder (Qwen2.5-VL): {text_encoder_gb:.2f} GB  (~{text_encoder_params_b:.1f}B params)")

# ============================================
# 3. VAE: AutoencoderKLQwenImage
# ============================================
vae_bytes = 242.0 * 1024 * 1024
vae_gb = vae_bytes / (1024**3)
vae_params_b = vae_bytes / 2 / 1e9
print(f"VAE: {vae_gb:.2f} GB  (~{vae_params_b:.2f}B params)")

# ============================================
# 4. LoRA
# ============================================
lora_bytes = 810.2 * 1024 * 1024
lora_gb = lora_bytes / (1024**3)
print(f"Lightning LoRA: {lora_gb:.2f} GB")

# ============================================
# TOTALS
# ============================================
total_weights_gb = transformer_gb + text_encoder_gb + vae_gb + lora_gb
print(f"\nTotal model weights: {total_weights_gb:.2f} GB")

# ============================================
# INFERENCE VRAM ESTIMATION
# ============================================
print("\n" + "=" * 60)
print("INFERENCE VRAM ESTIMATION")
print("=" * 60)

# PyTorch CUDA overhead
cuda_overhead = 0.5  # ~500MB for CUDA context

# Text encoder runs first, then can be offloaded
# But if kept on GPU:
print(f"\nScenario: All components on GPU simultaneously")
print(f"  Model weights: {total_weights_gb:.1f} GB")

# Activation memory for transformer (dominant cost)
# 60 layers, hidden=3072, sequence length depends on image resolution
# For 1024x1024 image: latent = 1024/8/2 = 64 tokens per side = 64*64 = 4096 patches
# But patch_size=2 and in_channels=64 suggests deeper latent
# With VAE z_dim=16 and temporal_downsample, spatial downsample ~8x
# So 1024/8 = 128, then /patch_size(2) = 64 per side, 64*64 = 4096 latent tokens
# Plus source image tokens (same count) = ~8192 total for joint attention
# Plus text tokens ~77-256

# Activation per layer (rough):
# Self-attention on seq_len=8192, hidden=3072, bf16
# QKV: 3 * 8192 * 3072 * 2 = ~150MB per layer
# Attention scores: 24 heads * 8192 * 8192 * 2 = ~3GB per layer (but only one at a time)
# With gradient checkpointing off (inference), only need current layer

# For a single forward pass with 8192 sequence length:
seq_len = 4096  # single image latent tokens (conservative for 512x512 or 1024x1024)
hidden = 3072
heads = 24
head_dim = 128

# Peak activation per transformer layer (kept in memory briefly):
# attention: seq_len * seq_len * heads * 2 bytes (attention matrix)
attn_peak_per_layer = seq_len * seq_len * heads * 2 / (1024**3)
# But with flash attention this is much lower
# With FlashAttention: ~O(seq_len * hidden) per layer, not O(seq_len^2)

print(f"\n  Assuming ~1024x1024 output, latent seq_len ~{seq_len}")
print(f"  Transformer hidden dim: {hidden}")

# With flash attention (standard in modern diffusers):
# Intermediate activations per step ~ a few GB
# Key intermediates: input/output of each layer, QKV projections
# Per layer: ~3 * seq_len * hidden * 2 (QKV) + seq_len * hidden * 2 (output)
# = ~4 * 4096 * 3072 * 2 = ~100MB per layer, but only peak matters
# Plus FFN intermediates: seq_len * 4*hidden * 2 = ~100MB

# Conservative estimate for peak activation memory with flash attention:
# ~2-4 GB for the full transformer pass
activation_est_low = 2.0
activation_est_high = 5.0

# For diffusion: need to keep the noisy latent + predicted noise
# latent size: 4096 * 64 * 2 = ~0.5MB (tiny)
# The scheduler keeps ~2 copies of latents (prev + current)

# Text encoder forward pass activations: ~1-2 GB
text_enc_activations = 1.5

# VAE decode: ~0.5-1 GB
vae_activations = 0.5

print(f"\n  Estimated activation/intermediate memory:")
print(f"    Transformer (flash attn): {activation_est_low}-{activation_est_high} GB")
print(f"    Text encoder: ~{text_enc_activations} GB")
print(f"    VAE decode: ~{vae_activations} GB")
print(f"    CUDA overhead: ~{cuda_overhead} GB")

# With sequential offloading (text enc -> transformer -> VAE):
# Only one major component active at a time
# Peak = max(text_enc_weights + text_enc_act, transformer_weights + transformer_act, ...)

print(f"\n{'='*60}")
print("SCENARIO 1: All on GPU (no offloading)")
print(f"{'='*60}")
total_min = total_weights_gb + activation_est_low + cuda_overhead
total_max = total_weights_gb + activation_est_high + cuda_overhead
print(f"  Weights: {total_weights_gb:.1f} GB")
print(f"  Activations (peak): {activation_est_low}-{activation_est_high} GB")
print(f"  CUDA overhead: {cuda_overhead} GB")
print(f"  TOTAL: {total_min:.1f} - {total_max:.1f} GB")
print(f"  -> Fits on A100 80GB: YES (barely, with ~{80-total_max:.0f} GB headroom)")
print(f"  -> Fits on A100 40GB: NO")

print(f"\n{'='*60}")
print("SCENARIO 2: CPU offload text encoder after encoding")
print(f"{'='*60}")
# After text encoding, offload text encoder to CPU
# Peak during transformer inference:
peak_transformer = transformer_gb + lora_gb + vae_gb + activation_est_high + cuda_overhead
print(f"  Peak (transformer phase): {peak_transformer:.1f} GB")
print(f"  -> A100 80GB headroom: {80-peak_transformer:.1f} GB")

print(f"\n{'='*60}")
print("SCENARIO 3: enable_model_cpu_offload()")
print(f"{'='*60}")
# Only one component on GPU at a time
peak_offload = max(transformer_gb + lora_gb + activation_est_high,
                    text_encoder_gb + text_enc_activations,
                    vae_gb + vae_activations) + cuda_overhead
print(f"  Peak (largest component): {peak_offload:.1f} GB")
print(f"  -> Fits on A100 40GB: {'YES' if peak_offload < 40 else 'NO'}")

print(f"\n{'='*60}")
print("CONCURRENCY ESTIMATES (batch_size=1 per request)")
print(f"{'='*60}")
# For concurrent inference, each request needs its own activation memory
# but model weights are shared (same model instance)
# With a single pipeline instance, concurrency = 1 (pipeline is not thread-safe)
# Multiple pipeline instances: each needs full weights + activations

print("\nWith SINGLE pipeline instance (sequential requests):")
print(f"  A100 80GB: 1 concurrent request (pipeline not thread-safe)")
print(f"  Need {total_max:.0f} GB for weights + activations")

print("\nWith MULTIPLE pipeline instances:")
# Each instance = full weights copy + activations
per_instance = total_weights_gb + activation_est_high + cuda_overhead
print(f"  Per instance: ~{per_instance:.0f} GB")
print(f"  A100 80GB: {int(80 // per_instance)} instance(s)")

print("\nWith SINGLE instance + batching (if supported):")
# Weights shared, activations scale with batch
# Extra per batch item: ~activation_est_high GB (rough, scales sub-linearly due to overhead)
weights_fixed = total_weights_gb + cuda_overhead
per_batch_item = activation_est_high  # conservative
print(f"  Fixed (weights): {weights_fixed:.1f} GB")
print(f"  Per batch item (activations): ~{per_batch_item:.0f} GB")
for gpu_mem in [40, 80]:
    avail = gpu_mem - weights_fixed
    max_batch = max(1, int(avail // per_batch_item))
    print(f"  A100 {gpu_mem}GB: max batch ~{max_batch}")

# With cfg=1.0 (no classifier-free guidance), no doubling of forward passes
print(f"\nNote: cfg=1.0 means NO classifier-free guidance doubling.")
print(f"With cfg>1, the effective batch doubles (conditional + unconditional).")

# 6 steps with Lightning LoRA
print(f"\n6 denoising steps with Lightning LoRA = 6 transformer forward passes")
print(f"+ 1 text encoder forward + 1 VAE decode")

PYEOF

description: Compute VRAM estimates and concurrency analysis


content:
============================================================
COMPONENT SIZES (from safetensors file sizes, bf16)
============================================================
Transformer: 38.05 GB  (~20.4B params)
Text Encoder (Qwen2.5-VL): 15.45 GB  (~8.3B params)
VAE: 0.24 GB  (~0.13B params)
Lightning LoRA: 0.79 GB

Total model weights: 54.53 GB

============================================================
INFERENCE VRAM ESTIMATION
============================================================

Scenario: All components on GPU simultaneously
  Model weights: 54.5 GB

  Assuming ~1024x1024 output, latent seq_len ~4096
  Transformer hidden dim: 3072

  Estimated activation/intermediate memory:
    Transformer (flash attn): 2.0-5.0 GB
    Text encoder: ~1.5 GB
    VAE decode: ~0.5 GB
    CUDA overhead: ~0.5 GB

============================================================
SCENARIO 1: All on GPU (no offloading)
============================================================
  Weights: 54.5 GB
  Activations (peak): 2.0-5.0 GB
  CUDA overhead: 0.5 GB
  TOTAL: 57.0 - 60.0 GB
  -> Fits on A100 80GB: YES (barely, with ~20 GB headroom)
  -> Fits on A100 40GB: NO

============================================================
SCENARIO 2: CPU offload text encoder after encoding
============================================================
  Peak (transformer phase): 44.6 GB
  -> A100 80GB headroom: 35.4 GB

============================================================
SCENARIO 3: enable_model_cpu_offload()
============================================================
  Peak (largest component): 44.3 GB
  -> Fits on A100 40GB: NO

============================================================
CONCURRENCY ESTIMATES (batch_size=1 per request)
============================================================

With SINGLE pipeline instance (sequential requests):
  A100 80GB: 1 concurrent request (pipeline not thread-safe)
  Need 60 GB for weights + activations

With MULTIPLE pipeline instances:
  Per instance: ~60 GB
  A100 80GB: 1 instance(s)

With SINGLE instance + batching (if supported):
  Fixed (weights): 55.0 GB
  Per batch item (activations): ~5 GB
  A100 40GB: max batch ~1
  A100 80GB: max batch ~4

Note: cfg=1.0 means NO classifier-free guidance doubling.
With cfg>1, the effective batch doubles (conditional + unconditional).

6 denoising steps with Lightning LoRA = 6 transformer forward passes
+ 1 text encoder forward + 1 VAE decode


text:
Here is the full analysis:

---

## 1. Base Architecture

FireRed-Edit-1.1 is **NOT** SDXL-based or Flux-based, though it shares architectural DNA with Flux. It is a custom architecture called **QwenImageEditPlusPipeline** built by the FireRed team:

- **Diffusion backbone**: `QwenImageTransformer2DModel` -- a Flux-style DiT (Diffusion Transformer) with flow matching (`FlowMatchEulerDiscreteScheduler`), 60 joint-attention layers, 24 heads x 128 head_dim = 3072 hidden dim
- **Text/Vision encoder**: `Qwen2_5_VLForConditionalGeneration` -- the full Qwen2.5-VL 7B multimodal model (28 text layers + 32-layer ViT), used as the conditioning encoder. This is what makes it an *image editing* model: it sees both the source image and the text instruction
- **VAE**: `AutoencoderKLQwenImage` -- custom VAE with 16 latent channels, spatial 8x downsampling
- **Scheduler**: Flow matching Euler with dynamic shifting (similar to Flux)

The arxiv paper is `2602.13344`.

---

## 2. Component Sizes (bf16)

| Component | File Size | Params (approx) |
|---|---|---|
| Transformer (DiT) | **38.1 GB** | ~20.4B |
| Text Encoder (Qwen2.5-VL-7B) | **15.4 GB** | ~8.3B |
| VAE | **0.24 GB** | ~0.13B |
| Lightning LoRA | **0.79 GB** | -- |
| **Total weights** | **54.5 GB** | ~28.8B |

---

## 3. VRAM for Model Weights in bf16

**~54.5 GB** for all weights loaded simultaneously on GPU.

---

## 4. VRAM for Inference (single image)

With all components on GPU, cfg=1.0 (no CFG doubling), 6 steps:

- Model weights: ~54.5 GB
- Peak activations (transformer with flash attention): ~2-5 GB
- Text encoder activations: ~1.5 GB
- VAE decode: ~0.5 GB
- CUDA context overhead: ~0.5 GB
- **Total: ~57-60 GB**

The key factors: cfg=1.0 avoids doubling the batch through the transformer. 6 steps means 6 transformer forward passes but activations don't accumulate across steps.

---

## 5. Concurrency Estimates

**A100 80GB:**
- Single pipeline instance, sequential requests: fits comfortably (~60 GB used, ~20 GB headroom)
- Batching with single instance: **max batch ~4** (weights are shared at 55 GB, each additional batch item adds ~5 GB of activation memory)
- Multiple separate pipeline instances: only **1** fits (each needs its own 54.5 GB of weights)

**A100 40GB:**
- **Does not fit** even with `enable_model_cpu_offload()`. The transformer alone is 38 GB + 0.8 GB LoRA + activations exceeds 40 GB
- Would require **int8/int4 quantization** of the transformer or aggressive model sharding across multiple GPUs

**Practical recommendation for A100 80GB:** Run a single pipeline instance, handle concurrency via batching (batch up to ~3-4 safely) or via request queuing. If you offload the text encoder to CPU after the encoding step, you free up 15 GB and can batch more aggressively (up to ~6-7 images). The 6-step Lightning LoRA already makes per-request latency low, so sequential processing with queuing may be sufficient depending on your throughput needs.