Really cool, just last week I was thinking about whether the alignment community should (massively) scale up prizes with relatively low barriers to entry!
Having you considered making this bigger? E.g. with more prices and a more active outreach to other universities?
I initially thought that ideally every contribution that clears a certain bar should be rewarded accordingly, that way there’s less uncertainty about payoffs and more people will contribute
I think you likely could find more texts to recommend, but even duplicated distillations are still valuable for getting students into thinking about alignment research and identifying particularly promising candidates
Evaluation time is a likely bottleneck, but probably you could find a handful of e.g. AGI Safety Fundamentals alumni to volunteer a few hours, or many more if you offer compensation for helping out
Thank you! These are thoughtful comments! I think I will try to add more texts and find more readers, as you suggest.
I’ve been thinking of going into working on creating contests in the future as a potentially serious work project, so I hope to create some contests that can be larger scale then! Right now, I’m rather limited in capacity. Thankfully, I’m connected with some other great university organizers who I’ve let know about advertising at their schools.
I think it would be tricky to have clear baseline cutoffs for distillation that still capture quality since writing varies so much between people. Do you have any ideas of clear cutoffs that would retain quality (for future contests if nothing else)?
You probably already have seen that the contest was featured on AstralCodexTen, so you might get more obviously good submissions than you have prices for and it would kinda feel like a wasted opportunity to not clearly signal (i.e. with money) to those authors that their work is highly appreciated and that we would love for them to do more of this work.
I think I will try to add more texts and find more readers, as you suggest.
I’ve been thinking of going into working on creating contests in the future as a potentially serious work project
Nice and nice! :)
Do you have any ideas of clear cutoffs that would retain quality (for future contests if nothing else)?
Hmm, is your worry that distillations that in hindsight seem to be fairly sub-optimal (e.g. with major mistakes or confusing explanations) end up receiving the lowest tier price because there is some noise introduced by the people who rate the distillations? I think this might happen only rarely, for maybe 2 in 100 distillations? I think your list of scoring criteria already goes a long way giving raters a good idea for what solid work looks like. The money for the lowest tier would also not be a lot, maybe 200$. Giving a price to in-hindsight subpar quality work would maybe reduce the prestige of the price a little bit, but I think it’s a fairly junior price anyway that mostly encourages and rewards initial solid efforts. Also you still would have the higher tiers for especially good work which would lose little prestige.
I do think it’s possible that we might award more prizes retroactively if we recognize that we receive a lot of valuable submissions! Maybe an “honorable mentions” category.
Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality. If I give a couple of examples, people may feel limited to having their work look like those examples. I might say “make your distillation 1,000 words and explain two papers and I’ll give you a prize” but 1,500 words on one paper might have made an optimal submission and I would have limited people’s abilities. I think I find it hard to quantify a bar on writing since everyone has such different approaches. I think the real bar is something more like “the judges who know more about AI Safety than me believe that you have communicated this idea really well” and because of that it feels wrong for me to try to say “and if you do x you will definitely win something.”
If they already get a price, I wouldn’t call it “honorable mentions” because that unnecessarily diminishes it in my eyes. Just have anything that seems that would get at B- in school be in the same category as the 250$ price?
Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality.
Ah, interesting, I have the opposite intuition!:D I completely agree that you shouldn’t give advice about the length of the distillations, but the criteria you mention here just seem really useful and like I’d be surprised if e.g. you find something clearly presented and accessible, and I wouldn’t.
Depth of understanding
Clarity of presentation
Rigor of work
Concision/Length (longer papers will need to present more information than shorter papers)
Originality of insight
Accessibility
And I feel like somebody who has spend like ~40 hours reading and discussing AI Safety material (e.g. as part AGI Safety Fundamentals course) could do a reasonably coherent job at rating the understanding and rigor. Originality seems maybe the trickiest, as you probably have to have some grasp of what ideas/framings are already in the water and which aren’t.
Really cool, just last week I was thinking about whether the alignment community should (massively) scale up prizes with relatively low barriers to entry!
Having you considered making this bigger? E.g. with more prices and a more active outreach to other universities?
I initially thought that ideally every contribution that clears a certain bar should be rewarded accordingly, that way there’s less uncertainty about payoffs and more people will contribute
I think you likely could find more texts to recommend, but even duplicated distillations are still valuable for getting students into thinking about alignment research and identifying particularly promising candidates
Evaluation time is a likely bottleneck, but probably you could find a handful of e.g. AGI Safety Fundamentals alumni to volunteer a few hours, or many more if you offer compensation for helping out
Thank you! These are thoughtful comments! I think I will try to add more texts and find more readers, as you suggest.
I’ve been thinking of going into working on creating contests in the future as a potentially serious work project, so I hope to create some contests that can be larger scale then! Right now, I’m rather limited in capacity. Thankfully, I’m connected with some other great university organizers who I’ve let know about advertising at their schools.
I think it would be tricky to have clear baseline cutoffs for distillation that still capture quality since writing varies so much between people. Do you have any ideas of clear cutoffs that would retain quality (for future contests if nothing else)?
You probably already have seen that the contest was featured on AstralCodexTen, so you might get more obviously good submissions than you have prices for and it would kinda feel like a wasted opportunity to not clearly signal (i.e. with money) to those authors that their work is highly appreciated and that we would love for them to do more of this work.
Nice and nice! :)
Hmm, is your worry that distillations that in hindsight seem to be fairly sub-optimal (e.g. with major mistakes or confusing explanations) end up receiving the lowest tier price because there is some noise introduced by the people who rate the distillations? I think this might happen only rarely, for maybe 2 in 100 distillations? I think your list of scoring criteria already goes a long way giving raters a good idea for what solid work looks like. The money for the lowest tier would also not be a lot, maybe 200$. Giving a price to in-hindsight subpar quality work would maybe reduce the prestige of the price a little bit, but I think it’s a fairly junior price anyway that mostly encourages and rewards initial solid efforts. Also you still would have the higher tiers for especially good work which would lose little prestige.
I do think it’s possible that we might award more prizes retroactively if we recognize that we receive a lot of valuable submissions! Maybe an “honorable mentions” category.
Ah, I think my worry is that it feels difficult for me to find a standard to rate that actually tracks quality. If I give a couple of examples, people may feel limited to having their work look like those examples. I might say “make your distillation 1,000 words and explain two papers and I’ll give you a prize” but 1,500 words on one paper might have made an optimal submission and I would have limited people’s abilities. I think I find it hard to quantify a bar on writing since everyone has such different approaches. I think the real bar is something more like “the judges who know more about AI Safety than me believe that you have communicated this idea really well” and because of that it feels wrong for me to try to say “and if you do x you will definitely win something.”
If they already get a price, I wouldn’t call it “honorable mentions” because that unnecessarily diminishes it in my eyes. Just have anything that seems that would get at B- in school be in the same category as the 250$ price?
Ah, interesting, I have the opposite intuition!:D I completely agree that you shouldn’t give advice about the length of the distillations, but the criteria you mention here just seem really useful and like I’d be surprised if e.g. you find something clearly presented and accessible, and I wouldn’t.
And I feel like somebody who has spend like ~40 hours reading and discussing AI Safety material (e.g. as part AGI Safety Fundamentals course) could do a reasonably coherent job at rating the understanding and rigor. Originality seems maybe the trickiest, as you probably have to have some grasp of what ideas/framings are already in the water and which aren’t.