justinpinkney.com

What is Justin Pinkney?

Justin Pinkney is a personal website and blog by an independent machine-learning experimenter. The site describes itself as "my occasionally updated blog" of creative machine-learning projects, with a focus on generative image models and image datasets.

What you'll find there

• Project write-ups: posts on topics such as swapping autoencoders (splitting images into style and content), Image Mixer Diffusion (blending images by fine-tuning an image-variation diffusion model), and fine-tuning Würstchen v2 for text-to-Pokémon generation.
• Datasets: for example, a Trailer Faces HQ dataset of about 187k high-resolution face images with varied expressions taken from movie trailers.
• Personal context: an about page, links to the author elsewhere online, and an RSS feed for following new posts.

Who it suits

The site is most useful if you follow generative AI as a hobby or practitioner and want concrete accounts of experiments—what was tried, what model was used, and what came out. It is a maker's notebook rather than a tutorial series, product site or news outlet, so expect first-person project notes rather than structured courses or benchmarks. Posts are infrequent, so it rewards RSS subscription over regular checking.

If you want to judge whether it's worth following, open the latest post and read the method section: if the level of model detail and experimentation matches your interests, subscribe to the feed. For broader context on generative image work, Hugging Face hosts many of the models and datasets these kinds of projects build on.

How can I create a dataset of high-resolution face images with varied expressions from movie trailers?

The most practical route is to treat it as a small pipeline: find trailer frames containing faces, crop and align them, filter for quality and expression variety, then deduplicate. Justin Pinkney's own project, the Trailer Faces HQ Dataset described on Justin Pinkney, is a concrete reference point: he published a dataset of 187k high-resolution face images with varied expressions taken from movie trailers, which confirms that this approach works at scale and that "varied expressions" is a realistic goal when sourcing from trailers rather than posed photos.

A workable pipeline

1. Collect source video. Trailers are short, high-bitrate and freely available, which makes them efficient raw material. Download at the highest resolution offered, since you want high-resolution crops later.
2. Sample frames. Extract frames at a fixed interval (e.g. one every half-second) rather than every frame; consecutive frames are near-duplicates and add little. A shot-boundary detector can cut redundant frames further.
3. Detect faces. Run a face detector over each frame and keep bounding boxes above a minimum pixel size so the final crops stay sharp after resizing.
4. Align and crop. Use facial landmarks to align eyes and mouth to a canonical position, then crop with a margin. Alignment matters more than raw crop size for downstream training or analysis.
5. Filter aggressively. Remove blurry frames (variance-of-Laplacian or similar), extreme poses, occlusions, heavy motion blur and frames where the face is tiny or partly out of frame.
6. Score expressions. Either classify expressions with a pretrained model or cluster landmark configurations to find distinct mouth/eyebrow states. Keep a balanced spread rather than whatever the trailers happen to contain.
7. Deduplicate. Compare embeddings or perceptual hashes across frames and across films; the same actor in the same scene will otherwise appear many times.
8. Record metadata. Store film, timestamp, crop coordinates and detection confidence so you can audit and re-crop later.

Decisions that shape the result

Practical cautions

Trailers are edited for drama, so you will get more extreme expressions (shouting, shock, tears) and fewer neutral ones than in a posed dataset — useful if you want expression range, less so if you need calm, frontal faces. Casting also skews demographically; if representativeness matters, sample deliberately across genres, eras and regions. Check the terms under which trailers are distributed before redistributing crops, and keep your metadata so attribution and takedown requests are easy to handle.

A sensible first step is to build a 1,000-image pilot from a handful of trailers, run the full pipeline end to end, and inspect a contact sheet of the results. Problems with alignment, blur filtering or duplicate rates show up immediately at that size and are cheap to fix before scaling.

What practical uses does Swapping Autoencoder have for separating style and content in images?

Swapping Autoencoder is useful when you want to keep what an image shows while changing how it looks. It splits an image into a content component (the underlying structure) and a style component (texture, colour and finish), so you can recombine them. Justin Pinkney frames his experiments this way in his post "Experiments with Swapping Autoencoder" on Justin Pinkney, describing a model that "breaks images into style and content components."

Practical uses

• Style transfer without redrawing content: Apply the look of one image (a painting, a photo taken at dusk) to the structure of another, useful for moodboards, concept art and visual exploration.
• Consistent art direction across a set: Give a series of images a shared texture or palette while keeping each subject distinct, helpful for thumbnails, editorial illustration or game asset mockups.
• Texture and material swaps: Change surface appearance—stone to fabric, matte to glossy—while preserving the shape of the object underneath.
• Content-preserving variation: Generate several stylistic versions of the same composition to compare options before committing to a direction.
• Dataset and pipeline testing: Use style/content separation as a diagnostic to see which parts of an image a model treats as structure versus surface.

Who benefits

Designers and illustrators exploring variations, researchers probing how generative models represent images, and anyone building a small internal tool for rapid visual iteration. It suits exploratory work more than precise, production-critical edits, because recombination can introduce artefacts and the separation is never perfect.

Trade-offs to weigh

A practical next step: pick two images—one for structure, one for style—and swap only the style, then inspect where content bleeds into the result. If your priority is blending whole images rather than separating components, Pinkney's related "Image Mixer Diffusion" experiments are a closer fit.

How can I fine-tune an image variation diffusion model to blend or mix images together?

The most concrete starting point on this site is the Image Mixer Diffusion post (April 2024), which describes blending or mixing images together by fine-tuning an Image Variation Diffusion model. That is the closest match to your question: rather than training a model from scratch, you adapt an existing image-variation model so it learns to combine inputs the way you want.

What the approach involves

• Start from an image-variation model, not a text-to-image model. These models already accept an image as input and produce variations, so fine-tuning teaches them to interpolate or merge rather than generate from noise.
• Prepare paired or blended training examples that show the kind of mixing you want (e.g., two faces combined, style from one image with content from another). The model learns the blending behavior from these examples.
• Fine-tune rather than retrain, which keeps compute and data needs modest—one reason this kind of experiment appears on a personal blog rather than a large lab.

The related Swapping Autoencoder post (May 2024) takes a different angle: it uses a model that splits images into style and content components, which is useful if you want more controlled mixing (swap style, keep content). The Face Mixer Diffusion post (January 2024) shows face-conditioned diffusion experiments, relevant if your blends involve faces.

A practical next step

Decide what "blend" means for your use case before choosing a method:

Then collect a small set of example outputs you consider good blends and fine-tune toward them. For broader context on creative ML experiments, see Justin Pinkney.

How do I train face-conditioned diffusion models for creative image generation?

The most practical starting point is to treat face-conditioned diffusion as a small, controlled experiment rather than a full training project. Justin Pinkney's site documents exactly this kind of work: his Face Mixer Diffusion post is described as experiments with training face-conditioned diffusion models, and his Image Mixer Diffusion post covers blending images by fine-tuning an Image Variation Diffusion model. Both suggest a workflow built on fine-tuning an existing model, not training from scratch. You can see these write-ups at Justin Pinkney.

A workable training loop

1. Pick a base model with image variation or image conditioning. Fine-tuning a model that already understands images is far cheaper than starting from text-only weights, which is the approach implied by the Image Mixer Diffusion entry.
2. Assemble a face dataset with variety. The Trailer Faces HQ Dataset on the same site is a concrete example: 187k high-resolution face images with varied expressions taken from movie trailers. Varied expressions and lighting matter more than sheer count for creative results.
3. Condition on identity or embedding, then fine-tune. Train the conditioning pathway on your face data while keeping most of the base model frozen, and save checkpoints obsessively—Pinkney's own note about "the best model checkpoint I ever lost" is a real warning.
4. Evaluate creatively, not just numerically. Generate blends, expression changes and style shifts, and judge whether the face stays recognizable while the image changes.

Choosing between approaches

A concrete scenario

A illustrator wants to generate the same character's face across many moods. Start with a few hundred varied images of that face, fine-tune a face-conditioned model for a short run, and generate a grid of expressions. If identity drifts, increase conditioning strength before adding data.

As a next step, read the Face Mixer Diffusion and Image Mixer Diffusion posts for the specific training choices, then compare with public experiment logs from Hugging Face to see how others structure fine-tuning runs.

How can I fine-tune Würstchen v2 for text-to-Pokémon image generation?

For a practical, reproducible starting point, follow the approach Justin Pinkney describes in his "Würstchen v2 Pokémon" experiment: fine-tune the Würstchen v2 text-to-image model on a curated Pokémon image-caption dataset. His write-up on Justin Pinkney is the most relevant reference for this specific task, and it is framed as a quick experiment rather than a polished tutorial, so treat it as a template you adapt rather than a step-by-step recipe.

What the approach involves

• Dataset first: Assemble Pokémon images with short, consistent captions (e.g., "a water-type Pokémon with blue fins"). Caption style matters more than volume; consistency teaches the model the target style quickly.
• Base model: Start from the released Würstchen v2 weights and fine-tune rather than training from scratch. Würstchen's compressed latent space makes fine-tuning comparatively cheap, but you still need a GPU with enough memory for the stage you train.
• Training setup: Use a standard text-to-image fine-tuning loop (e.g., Diffusers-based) with a low learning rate and a modest number of steps, then sample regularly to catch overfitting.
• Evaluation: Generate a fixed set of prompts each checkpoint and compare side by side; visual inspection is the practical metric here.

Decision criteria

A useful next step: read Pinkney's post for his dataset and training choices, then replicate with a small subset (a few hundred images) before scaling up. If you want broader context on fine-tuning diffusion models, the Hugging Face Diffusers documentation at Hugging Face covers the mechanics, and the original Würstchen paper and repository are worth checking for architecture details.