I decided to have fun this weekend and train my own image diffusion model from scratch. Training on a single local GPU. This is 2 days in. It is a purely experimental arch that is working surprisingly well for its size and how it works. There is no text encoder. I am using the qwen3 tokenizer and dumping those token embeddings straight in next to image tokens with a time 0. Only self attention. It is pixel space. I am just doing patch 16 straight into the transformer. I am not doing any projection layers. The hidden size is 768 so the blocks work directly on the pixels not a projected image embedding. There were visible patches, but I added a shuffler to fix it. It does an alternating unpatching, mirror padding 8 on all sides and then re patching so the cross seams land in the center of patches, runs through a block, then depatch, crop and repatch every other block. It cleaned up immediately when I added it, something only possible without projection. There is no VAE, no text encoder, and it is tiny and super fast. Really tiny. It is 441 million parameters total. 116 (1/4) of those is the token embedding table. To put that in perspective, SD 1.5 was 983 million (te and unet) Will it be able to make coherent images? Not sure. I will keep training it for a bit though to find out. It seems to be understanding the basic concept of the prompts and if you squint really hard, it kind of looks like the other AITK samples with the same prompts.