Gpt3 is algorithmically easier to reproduce, in the sense that it’s a simpler architecture with fewer and more robust hyperparameters. It’s engineering is harder to reproduce, because it needs more model parallelism. People have estimated gpt3 to cost 5M, and StableDiffusion to cost 300k, which is similar to the 36k number you quoted
Gpt3 is algorithmically easier to reproduce, in the sense that it’s a simpler architecture with fewer and more robust hyperparameters. It’s engineering is harder to reproduce, because it needs more model parallelism. People have estimated gpt3 to cost 5M, and StableDiffusion to cost 300k, which is similar to the 36k number you quoted