# How I’m building Pilgrimage with AI

Pilgrimage devlog 2 · September 5, 2026

[Pilgrimage](https://www.youtube.com/watch?v=OWHyFdEDSh8)

The day after [my first devlog](https://www.tomjohn.design/content/pilgrimage-starting-from-scratch), GPT Astra came out, and I spent a rainy morning working with it on [Pilgrimage](https://www.tomjohn.design/content/pilgrimage) [(0:01)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=1s). A lot changed, and the changelog had 64 versions in it by the end [(7:54)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=474s).

## Characters that aren't image sequences

I'd tried making character sprites with image generation before, as PNG sequences, and they looked great at a glance but fell apart once they moved. They were inconsistent, satchels switched sides, and making new ones took too long [(0:49)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=49s). So these characters are 3D shapes with a shader on each asset that gives them the pixel look [(2:11)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=131s). Each one faces eight directions, because the game is isometric, and they can walk, sleep, sit, pray and cut wood [(1:34)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=94s).

![A character walking in each of the eight isometric directions, captioned as I say: “got um monks and um different walking sequences because this is isometric. They all need to be facing these uh seven and eight cardinal directions. Um one, two, three, four,”](https://nlzkl2bo0qmd3uti.public.blob.vercel-storage.com/talks/clips/pilgrimage-building-with-ai/eight-directions-MxWM8Gs6peMQ2PKl9HvoDSHQSRakqX.mp4 "Clip")

Because the shader does the drawing, I don't have to re-render a sequence when I want a taller head [(2:38)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=158s). I can change shoulders and head size on the fly, so every person on the screen can be a little different: tall, short, broad-shouldered or skinny [(2:51)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=171s). Trees work the same way, and there are six species, all ones you'd find in England, grouped by density, with some preferring water [(3:03)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=183s).

![Adjusting a character's proportions in the sprite settings, captioned as I say: “here. So I'm not like I don't have to rerender a sequence. I don't have to go back into a pipeline and say like, okay, now now let me rerender this because I want the the head to be taller. I”](https://nlzkl2bo0qmd3uti.public.blob.vercel-storage.com/talks/clips/pilgrimage-building-with-ai/no-rerendering-1WbRiQaid8TMYXaQSbLAfJvnmoxl2K.mp4 "Clip")

## The map

There's a dark forest now, which only brave or hungry people walk through, so the path has to meander around it, and later you'll be able to clear it with knights [(3:49)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=229s). The map has elevation, waterfalls and water that flows in a direction, and bridges: suspension bridges where the path crosses a cliff and ordinary ones lower down [(4:11)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=251s). The path has two tracks, and it gets more worn the more people walk it [(4:56)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=296s).

![A suspension bridge over a river between cliffs, captioned as I say: “only that, but there's bridges that are suspended bridges if they're on top of a cliff. And if it's lower, then that would actually be a”](https://nlzkl2bo0qmd3uti.public.blob.vercel-storage.com/talks/clips/pilgrimage-building-with-ai/bridges-cKMLJitGt7fkgzJPskVJsv2D1rGQMK.mp4 "Clip")

The relic in this one is the scale of Jonah's whale [(5:38)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=338s). There's a list of about 20 generated relics, all a bit obscure, like the hipbone of St. John. Pilgrims pray there, the monks feed them, and if they need a job the monks give them one at the lumber camp, so they cut trees and stack the wood in piles, a bit like the depots in Pharaoh or Caesar [(6:00)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=360s).

![Pilgrims walking up to the relic, captioned as I say: “relic here is the scale of Jonah's whale. So these are all generated. There's a list of like 20 of them that I have.”](https://nlzkl2bo0qmd3uti.public.blob.vercel-storage.com/talks/clips/pilgrimage-building-with-ai/jonahs-whale-R3xRV96Zla9rzbBxuPYaQGfLPjB1Iz.mp4 "Clip")

I'm not big on the whole one-shot thing. I got a base level and then kept riffing on small things [(7:54)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=474s), like making sand collect on the inside of a river's bend, keeping cliffs to one side, and making sure people don't look like they're moonwalking down the path [(8:39)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=519s). The repo is open if you want to help [(10:13)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=613s). What's next is the actual game: how to get people to come down the path, work and stay [(10:56)](https://www.youtube.com/watch?v=OWHyFdEDSh8&t=656s).