thought it was just Google researchers who invented the Transformer?
Google engineers published the first version of a transformer. I don’t think it was in a vacuum, but I don’t know how much they drew from outside sources. Their model was designed for translation, and was somewhat different from Bert and GPT 2. I meant that there were a lot of different people and companies whose work resulted in the form of LLM we see today.
To put in enough effort to make it hard for sophisticated attackers (e.g. governments) to steal the models is a far heavier lift and probably not something AI companies will do of their own accord. (Possibly you already agree with this though.
This is outside my expertise. I imagine techniques are even easier to steal than weights. But if theft is inevitable, I am surprised OpenAI is worth as much as it is.
Google engineers published the first version of a transformer. I don’t think it was in a vacuum, but I don’t know how much they drew from outside sources. Their model was designed for translation, and was somewhat different from Bert and GPT 2. I meant that there were a lot of different people and companies whose work resulted in the form of LLM we see today.
This is outside my expertise. I imagine techniques are even easier to steal than weights. But if theft is inevitable, I am surprised OpenAI is worth as much as it is.