I’m trying to make “FLOPstacles” happen for things that mean we can’t just take max FLOP per GPU and multiply by number of GPUs, e.g. mem or interconnect bandwidth.
I’m trying to make “FLOPstacles” happen for things that mean we can’t just take max FLOP per GPU and multiply by number of GPUs, e.g. mem or interconnect bandwidth.