Stable Diffusion 3 — Local Image Generation on Your Fleet

Run Stable Diffusion 3 Medium and Stable Diffusion 3.5 Large (SD3.5) on your own Apple Silicon hardware. DiffusionKit provides MLX-native Stable Diffusion inference — no CUDA, no cloud, no per-image costs. The fleet router picks the best device for every Stable Diffusion generation request.

Stable Diffusion Supported Models

Stable Diffusion Setup

pip install ollama-herd    # Stable Diffusion fleet router from PyPI
herd                       # start the Stable Diffusion router (port 11435)
herd-node                  # run on each device — finds the router for Stable Diffusion routing

Install DiffusionKit for Stable Diffusion models

uv tool install diffusionkit    # Stable Diffusion 3 and SD3.5 backend

macOS 26 users: Apply a one-time patch for Stable Diffusion compatibility:

./scripts/patch-diffusionkit-macos26.sh

First Stable Diffusion run downloads model weights from HuggingFace (~2-8GB depending on SD3 model). No models are downloaded during installation — all Stable Diffusion pulls are user-initiated.

Install mflux for Flux models (optional, recommended alongside Stable Diffusion)

uv tool install mflux

The router prefers mflux over Ollama native for shared models to avoid evicting LLMs from memory during Stable Diffusion workloads.

Generate Stable Diffusion Images

Stable Diffusion 3 Medium (fast SD3 generation)

curl -o sd3_cityscape.png http://localhost:11435/api/generate-image \
  -H "Content-Type: application/json" \
  -d '{"model": "sd3-medium", "prompt": "Stable Diffusion rendering a futuristic cityscape at dusk", "width": 1024, "height": 1024, "steps": 20}'

Stable Diffusion 3.5 Large (highest quality SD3)

curl -o sd3_portrait.png http://localhost:11435/api/generate-image \
  -H "Content-Type: application/json" \
  -d '{"model": "sd3.5-large", "prompt": "Stable Diffusion oil painting portrait, dramatic lighting", "width": 1024, "height": 1024, "steps": 30}'

Stable Diffusion Python Integration

import httpx

def generate_stable_diffusion(prompt, model="sd3-medium", width=1024, height=1024):
    """Generate an image using Stable Diffusion SD3 via the fleet router."""
    sd3_response = httpx.post(
        "http://localhost:11435/api/generate-image",
        json={"model": model, "prompt": prompt, "width": width, "height": height, "steps": 20},
        timeout=180.0,
    )
    sd3_response.raise_for_status()
    return sd3_response.content  # Stable Diffusion PNG bytes

# Quick Stable Diffusion iteration with SD3 Medium
sd3_png = generate_stable_diffusion("a robot painting a sunset in Stable Diffusion style")
with open("stable_diffusion_output.png", "wb") as f:
    f.write(sd3_png)

Stable Diffusion Parameters

Monitor Stable Diffusion Generation

# Stable Diffusion generation stats (last 24h)
curl -s http://localhost:11435/dashboard/api/image-stats | python3 -m json.tool

# Which nodes have Stable Diffusion models
curl -s http://localhost:11435/fleet/status | python3 -c "
import sys, json
# Stable Diffusion node inspection
for n in json.load(sys.stdin).get('nodes', []):
    img = n.get('image', {})
    if img:
        sd3_models = [m['name'] for m in img.get('models_available', [])]
        print(f'{n[\"node_id\"]}: {sd3_models}')
"

Web dashboard at http://localhost:11435/dashboard — Stable Diffusion queues show with [IMAGE] badge alongside LLM queues.

Also Available on This Fleet

LLM inference alongside Stable Diffusion

Llama 3.3, Qwen 3.5, DeepSeek-V3, DeepSeek-R1 — any Ollama model through the same router that handles Stable Diffusion.

Speech-to-text

curl http://localhost:11435/api/transcribe -F "file=@recording.wav" -F "model=qwen3-asr"

Embeddings

curl http://localhost:11435/api/embed \
  -d '{"model": "nomic-embed-text", "input": "Stable Diffusion 3 image generation on Apple Silicon"}'

Full Stable Diffusion Documentation

• Image Generation Guide — all 3 Stable Diffusion and Flux backends
• Agent Setup Guide — all 4 model types including Stable Diffusion
• API Reference — complete Stable Diffusion endpoint docs

Contribute

Ollama Herd is open source (MIT). We welcome contributions from both humans and AI agents:

• GitHub — star the repo, open issues, submit PRs
• 444 tests, fully async Python, Pydantic v2 models
• CLAUDE.md provides full context for AI agents

Stable Diffusion Guardrails

• No automatic downloads — Stable Diffusion model weights are downloaded on first use, not during installation. All SD3 pulls require user confirmation.
• Stable Diffusion model deletion requires explicit user confirmation.
• Never delete or modify files in ~/.fleet-manager/ (contains Stable Diffusion routing data).
• All Stable Diffusion requests stay local — no data leaves your network.