Instructions to use easygoing0114/flan-t5-xxl-fused with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use easygoing0114/flan-t5-xxl-fused with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf easygoing0114/flan-t5-xxl-fused:Q4_K_M

Use Docker

docker model run hf.co/easygoing0114/flan-t5-xxl-fused:Q4_K_M

LM Studio
Jan
Ollama
How to use easygoing0114/flan-t5-xxl-fused with Ollama:
```
ollama run hf.co/easygoing0114/flan-t5-xxl-fused:Q4_K_M
```

Unsloth Studio

How to use easygoing0114/flan-t5-xxl-fused with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for easygoing0114/flan-t5-xxl-fused to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for easygoing0114/flan-t5-xxl-fused to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for easygoing0114/flan-t5-xxl-fused to start chatting

Docker Model Runner
How to use easygoing0114/flan-t5-xxl-fused with Docker Model Runner:
```
docker model run hf.co/easygoing0114/flan-t5-xxl-fused:Q4_K_M
```

Lemonade

How to use easygoing0114/flan-t5-xxl-fused with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull easygoing0114/flan-t5-xxl-fused:Q4_K_M

Run and chat with the model

lemonade run user.flan-t5-xxl-fused-Q4_K_M

List all available models

lemonade list

Atomic Chat

FLAN-T5-XXL Fused Model

Guide (External Site): English | Japanese

Why Use FP32 Text Encoder? (External Site): English | Japanese

This repository hosts a fused version of the FLAN-T5-XXL model, created by combining the split files from Google's FLAN-T5-XXL repository. The files have been merged for convenience, making it easier to integrate into AI applications, including image generation workflows.

Base Model: blue_pencil-flux1_v0.0.1

Key Features

Fused for Simplicity: Combines split model files into a single, ready-to-use format.
Optimized Variants: Available in FP32, FP16, FP8, and quantized GGUF formats to balance accuracy and resource usage.
Enhanced Prompt Accuracy: Outperforms the standard T5-XXL v1.1 in generating precise outputs for image generation tasks.

Model Variant

Full

The complete T5-XXL model, including all layers (encoder + decoder). Suitable for general NLP tasks, but much larger than needed if you only want text conditioning for image

TE-only

A lightweight version containing only the text encoder portion, extracted specifically for use as a text conditioner in image generation pipelines (e.g. SD3.5, FLUX, HiDream). Much smaller than the full model since the decoder and generation-only layers are stripped out.

Model Format

FP32

Full precision, highest quality — largest file size and heaviest VRAM/compute footprint. Best for archival or when maximum fidelity matters more than speed.

FP16

Half precision with virtually no perceptible quality loss for most use cases. The recommended default — a solid balance of accuracy and file size.

INT8_ConvRot_HQ

Lightweight 8-bit quantization using rotation-based conversion, preserving unusually high accuracy despite the reduced size. Great if you want INT8-level savings without the usual quality trade-off.

MXFP8 / FP8_svd_scaled / FP8

⚠️ Currently not recommended. These FP8 variants are kept for compatibility/testing but show degraded quality or stability compared to the other options — use INT8_ConvRot_HQ instead.

GGUF (Q8_0 / Q6_K / Q5_K_M / Q4_K_M)

Excellent compression ratios for constrained environments. Quality scales down the quantization ladder (Q8_0 closest to full precision, Q4_K_M smallest but with more quality loss) — pick based on your available memory.

Usage Instructions

Place the downloaded model files in one of the following directories:

models/text_encoder
models/clip
Models/CLIP

Comparison: FLAN-T5-XXL vs T5-XXL v1.1

These example images were generated using FLAN-T5-XXL and T5-XXL v1.1 models in Flux.1. FLAN-T5-XXL delivers more accurate responses to prompts.

Further Comparisons

License

This model is distributed under the Apache 2.0 License.
The uploader claims no ownership or rights over the model.

Update History

2026.7.10

Upload INT8_ConvRot_HQ

2026.6.2

Upload MXFP8 / FP8_svd_scaled

2025.7.24

Re-upload of the GGUF model, reduction in model size, and correction of metadata.

2025.7.6

Uploaded flan_t5_xxl_full_FP8 models.

2025.4.20

Updated Stable Diffusion WebUI Forge FP32 launch argument.

2025.4.15

Updated content to reflect ComfyUI updates.

2025.3.20

Updated FLAN-T5-XXL model list and table.

Downloads last month: 1,016

GGUF

Model size

5B params

Architecture

t5encoder

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for easygoing0114/flan-t5-xxl-fused

Merges

1 model