(Originally sent as feedback privately regarding an earlier version, then Nuño requested publishing as a comment instead)
Overall I think it’s a fair point and would be curious what the response would be from Ajeya. I think you downplay the importance others have placed in the evolution anchor more than needed and you also could engage a bit more with Ajeya’s existing arguments in the report.
You say that you think the evolution anchor isn’t viewed as as important by others, but I’m not so sure: see e.g. Holden’s blog post here which emphasizes it as an upper bound. I’d also look through this shortform. In particular it notes that Ajeya mentions in the report “I expect that some ML researchers would want to argue that we would need substantially more computation than was performed in the brains of all animals over evolutionary history; while I disagree with this, it seems that the Evolution Anchor hypothesis should place substantial weight on this possibility.”
There’s also this section of the report of which some parts somewhat close to directly address your concern, e.g.: “There are also some specific ways it seems that we could improve upon the “simulate natural selection” baseline, prima facie. For example, population sizes are a consequence of the carrying capacity of an ecological niche rather than being tuned to minimize the amount of computation used to evolve intelligent animals; it seems likely that they were far too large from a computational-efficiency standpoint. Additionally, the genetic fitness signal in the natural environment was often highly noisy, whereas we could plausibly design artificial environments which provide much cleaner tests of precisely the behaviors we are looking to select for.”
(Originally sent as feedback privately regarding an earlier version, then Nuño requested publishing as a comment instead)
Overall I think it’s a fair point and would be curious what the response would be from Ajeya. I think you downplay the importance others have placed in the evolution anchor more than needed and you also could engage a bit more with Ajeya’s existing arguments in the report.
You say that you think the evolution anchor isn’t viewed as as important by others, but I’m not so sure: see e.g. Holden’s blog post here which emphasizes it as an upper bound. I’d also look through this shortform. In particular it notes that Ajeya mentions in the report “I expect that some ML researchers would want to argue that we would need substantially more computation than was performed in the brains of all animals over evolutionary history; while I disagree with this, it seems that the Evolution Anchor hypothesis should place substantial weight on this possibility.”
There’s also this section of the report of which some parts somewhat close to directly address your concern, e.g.: “There are also some specific ways it seems that we could improve upon the “simulate natural selection” baseline, prima facie. For example, population sizes are a consequence of the carrying capacity of an ecological niche rather than being tuned to minimize the amount of computation used to evolve intelligent animals; it seems likely that they were far too large from a computational-efficiency standpoint. Additionally, the genetic fitness signal in the natural environment was often highly noisy, whereas we could plausibly design artificial environments which provide much cleaner tests of precisely the behaviors we are looking to select for.”
Thanks Eli