Stable Diffusion Model Selection: How to Choose Between SDXL, SD 3.5, and FLUX

Stable Diffusion model selection is easy to distort with one question: which model looks best? In practice, the result depends less on a leaderboard and more on your use case, VRAM, workflow ecosystem, and licensing boundary. SDXL, Stable Diffusion 3.5, and FLUX.1 can all produce strong images, but they fit different stages.

If you have just managed to run ComfyUI, do not rush to download a dozen checkpoints. This guide focuses on one practical question: when you are choosing between SDXL, SD 3.5, FLUX, and community models, how do you pick a model that runs, reproduces, and fits your actual use case?

Quick Answer

Start with this decision table. It is more useful than a “best model” ranking.

The short version: beginners should start with SDXL, then test SD 3.5 and FLUX later. If your local hardware is weak, choose the mature ecosystem. If you plan to use the output commercially, check the license before you look at sample images.

First, Know What You Are Choosing

When people say they want to “change the Stable Diffusion model,” they may mean three different things: a base model, a community checkpoint, or a LoRA.

The Base Model Sets the Capability Boundary

SDXL, Stable Diffusion 3.5, and FLUX.1 are base model routes. They shape the upper bound of prompt understanding, composition, people, text, and detail. SDXL is a mature general route. Stability AI’s SD 3.5 line includes Large, Large Turbo, and Medium. Black Forest Labs’ FLUX.1 line includes pro, dev, and schnell variants.

A base model is not simply “better because the file is bigger.” Larger models can bring higher quality potential, but they also bring more VRAM pressure, longer loading times, more node requirements, and higher deployment cost. Whether the model runs reliably on your machine matters more than the parameter count.

Community Checkpoints Solve Style and Scenario

Many realistic, anime, product-shot, and interior-design models you see on model sites are community checkpoints trained or merged on top of a base model. Their advantage is focus: the style is clear, they are quick to use, and many include recommended sizes, samplers, and example prompts.

That is also where the risk starts. A community checkpoint’s license, training source, and allowed use may differ from the base model. A model being free to download does not make it safe for a commercial project. For product posters, client delivery, ads, or paid asset packs, read the model card and author notes first.

A LoRA Is Not a Replacement for the Base Model

A LoRA is closer to an add-on capability pack. It can teach a model a character, outfit, camera language, product feature, or style, but it usually has to be used with a compatible base model or checkpoint. An SDXL LoRA may not work with SD 1.5. A FLUX LoRA cannot be dropped into a normal SDXL workflow at random.

KiloClaw - Managed OpenClaw for enterprise, enabling AI Agent orchestration and smart model routing

Get Started →Ad

The order should be: choose the base route, choose the checkpoint, then add LoRA files when needed. Reverse that order and troubleshooting gets painful.

Choose by Use Case

Model selection should start from the job you need the model to do, not from download counts.

Why Beginners Should Start with SDXL

SDXL is not always the newest option, but it is a safe starting point for many local image workflows. The reason is practical: more tutorials, more models, more LoRA files, and mature support around ControlNet, IP-Adapter, upscaling, and common ComfyUI workflows. When a model is not detected, an image looks gray, hands break, or VRAM runs out, it is easier to find a fix.

In the first week, the goal is not to squeeze out the highest possible image quality. It is to build a stable chain: ComfyUI detects the model, the workflow runs, parameter changes are reproducible, and the output is saved. SDXL is friendly for that.

When SD 3.5 Makes Sense

Stable Diffusion 3.5 makes sense once you already understand the basic ComfyUI chain and want to test Stability AI’s newer official direction. Stability AI positions SD 3.5 Large as the stronger model, while Medium is aimed more toward consumer hardware.

Use this rule of thumb: if you can already run SDXL reliably and are willing to prepare the matching workflow, nodes, and dependency files, test SD 3.5. If you are still unsure where a checkpoint file belongs, spend more time with SDXL first.

When FLUX Makes Sense

FLUX.1 is worth testing when you care about newer model aesthetics, composition, and stronger prompt following. It is especially interesting for overall image polish, text elements, product visuals, or a more natural photographic feel.

The caution is that FLUX variants and licenses need to be separated. Official materials describe FLUX.1 [schnell] as Apache 2.0 licensed, while dev, pro, and other routes have different usage paths and restrictions. Do not see “FLUX looks good” and assume every FLUX variant is freely usable for local commercial work.

Choose by Hardware

Hardware is the layer people often skip. Many sample images are made on high-VRAM GPUs, hosted services, or heavily optimized workflows. Put the same model on your machine and the first failure may be VRAM.

If VRAM Is Tight, Do Not Chase the Biggest Model First

If your GPU has limited VRAM, start with mature models and mature workflows. SDXL is more likely to have low-VRAM tutorials, optimized nodes, lighter checkpoints, and reproducible settings. You can generate at a smaller size first, then upscale, instead of starting at a large canvas.

VRAM usage is not a fixed number. It changes with resolution, batch size, sampling steps, precision, VAE, ControlNet, IP-Adapter, upscalers, and post-processing. When someone says a GPU can run a model, that does not mean your workflow can run it.

A safer process:

1. Start with the official or model-card-recommended base workflow.
2. Use a conservative image size.
3. Keep batch size at 1.
4. Turn off unnecessary ControlNet, upscaling, and post-processing nodes.
5. Record VRAM, time, and error messages.

Now you know where the bottleneck is instead of changing the model, size, LoRA, ControlNet, and post-processing all at once.

Cloud or API Routes Are Useful for Validation

If you do not have a suitable GPU, or if you only want to know whether a model fits a project, try a hosted or API route first. A hosted run does not replace local debugging, but it quickly answers a useful question: is this model’s style and capability worth the local deployment cost?

That is especially useful for FLUX and SD 3.5. Try a few rounds through a provider, then decide whether to download weights, build a local workflow, or upgrade hardware.

Vultr - High-performance NVMe VPS with 32 global locations, one-click Docker deploy

View Pricing →Ad

Choose by Licensing

Licensing is the part you should not skip, especially when the image goes into an ad, course cover, client project, product page, or paid asset pack.

Check Three License Layers Separately

The first layer is the base model license. Stability AI has its own license page, and FLUX.1 variants have different license files. The second layer is the community checkpoint or LoRA author license. The third layer is the platform or API service terms you use to generate the image.

These layers do not replace each other. A base model may allow certain uses, but a community merge may not. A platform may let you generate images, but that does not mean you can redistribute the model weights. An output may be usable in some commercial contexts, while the model itself may not be usable as a commercial service.

What to Check for SDXL, SD 3.5, and FLUX

For SDXL and SD 3.5, start with the Stability AI license page and the Hugging Face model card. The official license page explains the Community License, revenue thresholds, and enterprise licensing boundary. Do not copy conclusions from old posts, because licensing terms can change.

For FLUX, separate pro, dev, and schnell first. Official materials describe schnell as more permissive, while dev and pro use different paths and restrictions. The dev variant is especially easy to misread because pages may mention output usage and model license limits at the same time. Read both parts together.

For community models, inspect every model card. A title that says realistic, commercial, or free is not enough. Look for an explicit license, training notes, prohibited uses, commercial restrictions, and update history.

Test the Candidate in ComfyUI

The final step is a real test run. Saving model links is not model selection.

1. Read the Model Card and License

Before downloading, confirm three things:

• Which route the model belongs to: SDXL, SD 3.5, FLUX, or another architecture.
• The recommended workflow, image size, sampler, and whether it needs extra text encoders or a VAE.
• Whether the license allows your intended use.

If the model card does not explain these clearly, treat the model as an experiment, not as a production asset.

2. Put the File in the Right Folder

ComfyUI’s official docs place model files under different folders inside ComfyUI/models/. Common patterns:

Different install methods and newer model nodes may have extra requirements. If ComfyUI does not detect a model, check the official docs or the matching workflow before copying someone else’s folder screenshot.

3. Use the Matching Workflow

SDXL, SD 3.5, and FLUX may use different workflow structures. Newer models often need matching loader nodes, encoders, samplers, or official example workflows. Forcing a FLUX model into a normal SDXL Load Checkpoint chain is unlikely to give you the result you expect.

When testing a model for the first time, look for an official example, a maintainer example, or a ComfyUI workflow that clearly says it supports that model. Run the minimal path first, then add LoRA, ControlNet, and post-processing.

Railway - Modern deployment platform, zero-config, go live in minutes

Start Now →Ad

4. Run a Small, Controlled Test

Do not generate dozens of images immediately. Lock a small test set:

• seed: fixed, so comparison is possible.
• prompt: the same prompt, covering people, scene, material, and lighting.
• size: the recommended size or a conservative size.
• steps / sampler / CFG: start with the model card or workflow recommendation.
• batch: keep it at 1.

Run at least 3 to 5 images per model and record speed, failure rate, image stability, and the details you care about. Model selection is not about one perfect sample. It is about whether the model is stable in your workflow.

Common Mistakes and Troubleshooting

Mistake 1: The Newest Model Must Be Best for Me

A new model can be stronger, but it can also be harder to deploy. It may need new nodes, more VRAM, different prompt habits, or a more complex license. You need the model that fits the current job, not the newest name.

Mistake 2: A Beautiful Sample Means the Model Is Good

A sample image may use a polished prompt, LoRA, ControlNet, upscaling, post-processing, and manual curation. Model-card examples show potential, not your first-run average result.

Mistake 3: If the Model Is Not Detected, Download Another One

When a model is not detected, check the path, format, refresh/restart step, and workflow support first. Confirm the file is in the right folder, then check whether the node supports the architecture. Do not download more models before checking the directory.

Mistake 4: Commercial Use Only Depends on the Base Model

Commercial use depends on the base model, checkpoint, LoRA, platform terms, and your specific use case. Even when a base model allows certain commercial uses, a community model author may add restrictions.

What to Read Next

If you have not run ComfyUI yet, start with ComfyUI Beginner Guide: From Installation to Your First Stable Diffusion Image. If you often import shared workflows, read ComfyUI Workflow Reuse Guide: Import JSON, Fix Missing Nodes, Map Model Paths.

Once the model is chosen, prompts come next. For general prompt writing, read Prompt Engineering for Business: Practical Techniques That Improve AI Output. If you want to place image generation inside a broader creative workflow, see Cross-Media Creation: From Sketch to Slide Deck with Nano Banana 2 and Gemini 3.

References

• Introducing Stable Diffusion 3.5
• Stability AI License
• Stable Diffusion XL Base 1.0 model card
• black-forest-labs/flux
• FLUX models by Black Forest Labs
• ComfyUI Models

Conclusion

Stable Diffusion model selection cannot be reduced to “SDXL, SD 3.5, or FLUX: which is strongest?” A better order is: use case first, hardware second, ecosystem maturity third, licensing last.

Start with SDXL if you are new and want a stable workflow faster. Test SD 3.5 when you want to explore Stability AI’s newer route. Evaluate FLUX when you care about newer aesthetics and stronger prompt following. The real standard is not the model name. It is whether the model can produce stable results inside your machine, your workflow, and your licensing boundary.