imagen.research.google

DrawBench is a benchmark introduced by the Imagen team (Google Research, Brain Team) specifically to evaluate text-to-image models in greater depth than existing datasets allow. It is used because standard metrics like FID on COCO measure only part of the picture: DrawBench adds a comprehensive, challenging set of prompts that supports side-by-side comparison of models on both sample quality and image-text alignment, as judged by human raters.

What DrawBench is

According to the Imagen project page, DrawBench is described as "a comprehensive and challenging benchmark for text-to-image models." It was released alongside the Imagen paper and the model itself, and the page links to it directly as a companion resource.

The key design goal is depth of evaluation. Rather than relying on a single dataset score, DrawBench is built to probe how well a model handles a wide range of prompts — including ones that stress composition, unusual combinations, and detailed descriptions.

Why it exists: the limits of FID on COCO

The Imagen page reports a state-of-the-art FID score of 7.27 on COCO, achieved without ever training on COCO, and notes that human raters find Imagen samples on par with the COCO data itself in image-text alignment. That is a strong result, but FID alone answers a narrow question: how close are generated distributions to a reference set?

DrawBench was introduced to assess text-to-image models "in greater depth." In practice this means moving beyond a single aggregate number toward comparisons that reflect what users actually care about — whether the image matches the prompt and whether it looks good.

What DrawBench measures

The page frames evaluation along two axes that DrawBench is used to compare:

These are the same two qualities the page uses to describe Imagen's overall contribution ("unprecedented photorealism × deep level of language understanding"), so DrawBench operationalizes the model's stated goals into something measurable and comparable.

How it is used in practice

The page describes a concrete evaluation procedure:

1. Collect prompts from the DrawBench set.
2. Generate images from each model being tested using those prompts.
3. Run side-by-side comparisons judged by human raters.
4. Record preferences on sample quality and image-text alignment.

Using this setup, the Imagen team compared Imagen against recent methods including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2, and found that human raters preferred Imagen over the other models in side-by-side comparisons on both dimensions.

Why this matters if you are choosing or evaluating a model

• Aggregate scores hide failure modes. A model can score well on average while failing on specific prompt types. A prompt-based benchmark surfaces those cases.
• Human preference is the relevant signal for generation quality. FID is a proxy; side-by-side human judgment directly reflects what a user would pick.
• Comparability requires shared prompts. Because DrawBench is a fixed, public set of challenging prompts, different models can be evaluated on the same inputs, which is what makes side-by-side results meaningful.

If you are assessing a text-to-image model, the practical takeaway is to look for evaluations that report both a distribution-level metric and prompt-level human comparisons — DrawBench is the example the Imagen team provides for the latter.

Where to find it

The Imagen page links to both the Research Paper and DrawBench directly, and also points to related work in the Imagen family: Imagen Video and Imagen Editor. Start from the DrawBench link on the page to access the benchmark itself.

Imagen is a text-to-image diffusion model from Google Research's Brain Team that turns a written prompt into a photorealistic image. It works by pairing a large pretrained language model (such as T5) that encodes your text with a diffusion model that generates the image. The key finding behind its quality: scaling up the language model improves both image fidelity and image-text alignment more than scaling up the image diffusion model.

The Two-Stage Pipeline

Imagen separates understanding text from generating pixels.

Stage 1: Text encoding with a large language model

• A generic large language model pretrained only on text (for example T5) reads your prompt and converts it into a numerical representation.
• This model was never trained on images, yet the research found it is "surprisingly effective at encoding text for image synthesis."
• Because it captures deep language structure, it handles prompts with unusual combinations and relationships between objects.

Stage 2: Diffusion-based image generation

• A diffusion model takes that text representation and generates a high-fidelity image.
• Diffusion models work by starting from noise and iteratively refining it into a coherent picture, which is where Imagen's photorealism comes from.

Why the Language Model Matters Most

The central discovery is about where to invest scale:

In other words, a stronger text encoder does more for overall quality than a bigger image generator. This is why Imagen "builds on the power of large transformer language models in understanding text."

Evidence of Quality

• Imagen achieved a state-of-the-art FID score of 7.27 on the COCO dataset without ever training on COCO.
• Human raters found Imagen samples on par with the COCO data itself in image-text alignment.
• On DrawBench, a benchmark introduced alongside Imagen, human raters preferred Imagen over VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2 in side-by-side comparisons, for both sample quality and image-text alignment.

What Kind of Prompts It Handles

The model is designed for detailed, compositional prompts. Examples shown on the site include:

• "A brain riding a rocketship heading towards the moon."
• "A dragon fruit wearing karate belt in the snow."
• "A small cactus wearing a straw hat and neon sunglasses in the Sahara desert."
• "A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat."
• "Teddy bears swimming at the Olympics 400m Butterfly event."
• "A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a landscape."

These illustrate the combination of photorealism and deep language understanding the model targets.

Related Models in the Imagen Family

The same research direction extends to:

• Imagen Video — video generation
• Imagen Editor — image editing

Bottom Line

Imagen generates images by letting a large text-only language model interpret your prompt and a diffusion model render it. Its advantage comes from the discovery that scaling the language model boosts fidelity and alignment more than scaling the image model, which is how it reached leading FID and human-preference results without training on COCO.

The Imagen page at imagen.research.google is a research landing page, not a product sign-up or image generation tool. It links to the Imagen research paper, the DrawBench benchmark, and two related models in the Imagen family (Imagen Video and Imagen Editor), and it displays example images generated from text prompts. If you want to read the paper, inspect the benchmark, or see what the model can produce, this page is the entry point.

What the page actually offers

The page presents Imagen as a text-to-image diffusion model developed by the Google Research, Brain Team, described as combining "unprecedented photorealism" with a "deep level of language understanding." From a navigation standpoint, the useful elements are:

• Research Paper — the link to the full paper describing the model.
• DrawBench — a benchmark introduced alongside Imagen for evaluating text-to-image models.
• Imagen family links — Imagen Video and Imagen Editor.
• Example generations — a set of prompt-and-image pairs illustrating what the model produces.

Core claims stated on the page

These are the technical points the page itself makes, useful context before you read the paper:

• Imagen builds on large transformer language models for text understanding and diffusion models for image generation.
• The key finding: generic large language models (e.g. T5) pretrained on text-only corpora are effective at encoding text for image synthesis. Scaling the language model improves both sample fidelity and image-text alignment more than scaling the image diffusion model.
• Imagen reports a state-of-the-art FID score of 7.27 on COCO, without training on COCO.
• Human raters found Imagen samples on par with the COCO data itself in image-text alignment.
• In side-by-side comparisons via DrawBench, human raters preferred Imagen over VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2, in both sample quality and image-text alignment.

Example prompts shown on the page

The page illustrates the model with prompts such as:

• "A brain riding a rocketship heading towards the moon."
• "A dragon fruit wearing karate belt in the snow."
• "A small cactus wearing a straw hat and neon sunglasses in the Sahara desert."
• "A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat."
• "Teddy bears swimming at the Olympics 400m Butterfly event."
• "Sprouts in the shape of text 'Imagen' coming out of a fairytale book."
• "A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a landscape."

These are useful as reference points for the kind of compositional, attribute-heavy prompts the model was evaluated on.

What the page does not provide

The page evidence contains no pricing signals, no payment platforms, and no login or access instructions. It is a research presentation page, so do not expect a way to generate images directly from it. For hands-on use, you would need to look elsewhere; the page's role is to point you to the paper, the benchmark, and the family models.

How to use this page

1. Open imagen.research.google.
2. Follow the Research Paper link for the full technical description.
3. Follow the DrawBench link to see the evaluation benchmark and comparisons.
4. Use the Imagen Video and Imagen Editor links to explore related models in the family.
5. Review the example prompts and images to gauge the model's capabilities before diving into the paper.

Imagen is a text-to-image diffusion model from the Google Research Brain Team, presented at imagen.research.google as a research project rather than a consumer product. The page is the official hub for the model's paper, sample outputs, and the DrawBench benchmark. It is the right place to look if you want to understand what Imagen claims to do, see example generations, or find the research paper — not if you are looking for a sign-up page or an image generator you can use directly.

What the page presents

The site is organized around a single research claim, stated in the heading: "unprecedented photorealism × deep level of language understanding." From there, the page offers:

• A research paper — the full technical write-up of the model.
• DrawBench — a benchmark the team introduced to evaluate text-to-image models in greater depth.
• Sample images — a gallery of generated outputs paired with their text prompts.
• Links to the wider Imagen family — Imagen Video and Imagen Editor.

How Imagen works, in the authors' own framing

The page describes Imagen as building on two components: large transformer language models for understanding text, and diffusion models for high-fidelity image generation.

The key finding the team highlights is about where scale matters most. Generic large language models such as T5, pretrained only on text, turn out to be "surprisingly effective at encoding text for image synthesis." Increasing the size of the language model improved both sample fidelity and image-text alignment more than increasing the size of the image diffusion model did. In other words, the text encoder — not the image generator — was the bigger lever.

Reported results

The page states two headline numbers and one comparison:

The page also notes Imagen ranked #1 in COCO FID and #1 in DrawBench.

What the sample prompts show

The gallery is useful for judging the model's range, because the prompts are deliberately varied. Examples on the page include:

• "A brain riding a rocketship heading towards the moon."
• "A dragon fruit wearing karate belt in the snow."
• "A small cactus wearing a straw hat and neon sunglasses in the Sahara desert."
• "A photo of a Corgi dog riding a bike in Times Square. It is wearing sunglasses and a beach hat."
• "Teddy bears swimming at the Olympics 400m Butterfly event."
• "Sprouts in the shape of text 'Imagen' coming out of a fairytale book."
• "A transparent sculpture of a duck made out of glass. The sculpture is in front of a painting of a landscape."

These skew toward compositional and unusual scenes — objects wearing clothing, text rendered as shapes, animals in specific settings — which is consistent with the page's emphasis on language understanding rather than just image quality.

The Imagen family

The page links to two related projects:

• Imagen Video — extends the approach to video generation.
• Imagen Editor — an editing variant.

What the page does not offer

The site is a research presentation. It does not describe pricing, an API, a waitlist, or a way to run Imagen yourself, and nothing on the page indicates that access is free or open. If your goal is to generate images, this page will tell you what the model is and what it can do, but not how to use it.

When this page is worth your time

• You want the primary source for Imagen's architecture claims and benchmark results.
• You need the DrawBench description to understand how text-to-image models were being evaluated at that point.
• You are comparing Imagen against DALL-E 2, Latent Diffusion, or VQ-GAN+CLIP and want the authors' own comparison.
• You want to see prompt-and-output pairs before deciding whether the photorealism claim holds up for your use case.

If you need a working tool rather than a research reference, look elsewhere — this page is documentation of a model, not a product.

Imagen reports an advantage over DALL-E 2, VQ-GAN+CLIP, and Latent Diffusion in side-by-side human comparisons, based on the results published on imagen.research.google. The site states that human raters preferred Imagen over these models in both sample quality and image-text alignment, and that Imagen reached a state-of-the-art FID score of 7.27 on COCO without ever training on COCO. These claims come from Google Research's own evaluation, so treat them as the project's reported results rather than an independent verdict.

The comparison dimensions

The Imagen page evaluates models on two axes:

• Sample quality — how photorealistic and artifact-free a generated image looks.
• Image-text alignment — how faithfully the image matches the prompt.

Both are judged by human raters in side-by-side comparisons, and FID is used as an automated quality metric.

What the reported numbers show

Two details are worth separating:

• COCO FID 7.27 is a quality score. The page notes Imagen achieved it without training on COCO, which makes the number more notable than a score obtained by training on the same dataset.
• DrawBench is the benchmark Imagen's authors introduced specifically to test text-to-image models more deeply. Because it was created alongside Imagen, it is a project-defined test rather than a neutral third-party one.

Why Imagen performs this way

The page attributes Imagen's results to a specific design choice: it relies on large pretrained language models (such as T5) to encode text, combined with diffusion models for image generation. The stated key discovery is that scaling the language model improves both sample fidelity and image-text alignment more than scaling the image diffusion model. In other words, the reported strength in prompt understanding comes mainly from the text encoder, not the image generator.

What this means for your choice

• If your priority is prompt adherence on complex descriptions, Imagen's reported DrawBench and alignment results are the relevant evidence.
• If you want an independent, third-party ranking, the page's comparisons are self-reported and DrawBench is author-introduced, so look for external evaluations before treating the ordering as settled.
• If you need access or pricing, the page does not state them — do not assume Imagen is free or openly available based on this page alone.

The page also lists related models in the Imagen family (Imagen Video, Imagen Editor) and shows example prompts, but it does not provide a feature-by-feature comparison table against DALL-E 2 beyond the benchmark results above.