An SM (the basic unit of a GPU) is actually quite simple: memory, register files and a bunch of FMAs (fused multiply-adds) along with warp schedulers. All GPU architecture changes began from SMs. Scaling up is the hard part.