From an outsider perspective, this looks like the sort of thing that almost anyone could get started on and I like the phrasing you used to signal that. AI progress moves so fast that you are most likely going to the only one looking at something and so you can do very basic things like
“How deterministic are these models? If you take the first K lines of the CoT and regenerate it, do you get the same output?”
It’s pretty easy to imagine taking 1 line of CoT and regenerating and then 2 lines...
I think a lot of people can just do this and getting to do it under Neel Nanda is likely to lead to a high quality paper.
From an outsider perspective, this looks like the sort of thing that almost anyone could get started on and I like the phrasing you used to signal that. AI progress moves so fast that you are most likely going to the only one looking at something and so you can do very basic things like
“How deterministic are these models? If you take the first K lines of the CoT and regenerate it, do you get the same output?”
It’s pretty easy to imagine taking 1 line of CoT and regenerating and then 2 lines...
I think a lot of people can just do this and getting to do it under Neel Nanda is likely to lead to a high quality paper.