It’s great to have a plan, and work through various projects.
I think you correctly noted the need for “tight feedback loops”, but at the same time I didn’t see any work in a shape that you could actually get feedback on; perhaps you’re optimising too hard for completing something to 100%, when instead you could be getting feedback sooner?
You mention you’ve already developed your own research tastes—that’s great! Taking on something so ambitious is commendable, but I wonder whether you are again falling prey to not getting feedback often enough. Have you run your research ideas by folks who have been thinking about AI safety for a while? Are there similarities between your research and any agendas being explored (and have you talked to those people)? Have you articulated why popular research agendas are unlikely to work, and what missing pieces your research might add? Taking such actions create surface areas for folks to challenge your thinking, and help you improve your understanding of the core questions and what needs to be done to solve them.
There is lots of great advice about building the underlying tools/mindset/aptitudes/instincts needed for AI safety research, including on getting feedback; e.g. this piece by ARC/METR researcher Lawrence Chan states “touching reality requires receiving actual concrete feedback on your ideas.” I think it would be helpful for you to build relationships with folks who can give you high quality input regularly. (This is a central kind of value that many budding Alignment researchers get out of moving to the SF Bay Area for a few weeks/months.)
I’ll leave it at that for now; feel free to apply for an advising chat if you’re like to talk some more!
Let me know if I should apply for an advising chat instead, but I have a few followup questions...
I’m hearing “(a) prioritize producing feedback-ready work, (b) making it more clear where my feedback-ready work is, and (c) campaigning to get people with relevant knowledge and skill actually give me feedback on my work.”
That’s totally the goal of my SSJ : )
-- Regarding (a), I do have work that I think is feedback ready, so I think I may have more of a problem with (b). For example, I failed to mention the paper I have on arxiv with video explanation and video presentation with followup work that I made for my honour project. I did write “I would love comments on my WIP here: OIS” in the section on a document I am drafting, but it is not prominently displayed. Do you think those are good examples of (a), or should I focus on making my work easier to give feedback on?
-- Regarding (c), I have had email discussions with the authors of papers that inspired my work, but I probably failed to emphasize a request for critical feedback. I admit I find doing this kind of campaigning quite exhausting and so do less than I should. Do you have any advice or links for how to “build relationships with folks who can give you high quality input regularly”, especially as an introvert?
Great that you have work like the arxiv paper! You could even explicitly ask for feedback on that work
Make it easy for people to understand your work: Try and answer questions like “Why did I do this? What did I learn and/or what update did I make? What is my theory of change?”, and so on...
Make it easy for people to engage with your work: Display it prominently, tweet about it, write a blogpost on lesswrong about it. Polish and publish the code base (see an example here), and so on...
Everyone has their own style of building relationships. I think a powerful way to do so is to try and add value to others: can you summarise/discuss their work in public, or give them feedback, or extend it in an interesting way? Are there volunteer or part-time opportunities that you can help out with? Can you identify issues in their codebases and improve them?
Hi Tristan, thanks for writing in.
It’s great to have a plan, and work through various projects.
I think you correctly noted the need for “tight feedback loops”, but at the same time I didn’t see any work in a shape that you could actually get feedback on; perhaps you’re optimising too hard for completing something to 100%, when instead you could be getting feedback sooner?
You mention you’ve already developed your own research tastes—that’s great! Taking on something so ambitious is commendable, but I wonder whether you are again falling prey to not getting feedback often enough. Have you run your research ideas by folks who have been thinking about AI safety for a while? Are there similarities between your research and any agendas being explored (and have you talked to those people)? Have you articulated why popular research agendas are unlikely to work, and what missing pieces your research might add? Taking such actions create surface areas for folks to challenge your thinking, and help you improve your understanding of the core questions and what needs to be done to solve them.
There is lots of great advice about building the underlying tools/mindset/aptitudes/instincts needed for AI safety research, including on getting feedback; e.g. this piece by ARC/METR researcher Lawrence Chan states “touching reality requires receiving actual concrete feedback on your ideas.” I think it would be helpful for you to build relationships with folks who can give you high quality input regularly. (This is a central kind of value that many budding Alignment researchers get out of moving to the SF Bay Area for a few weeks/months.)
I’ll leave it at that for now; feel free to apply for an advising chat if you’re like to talk some more!
Thanks : )
Let me know if I should apply for an advising chat instead, but I have a few followup questions...
I’m hearing “(a) prioritize producing feedback-ready work, (b) making it more clear where my feedback-ready work is, and (c) campaigning to get people with relevant knowledge and skill actually give me feedback on my work.”
That’s totally the goal of my SSJ : )
-- Regarding (a), I do have work that I think is feedback ready, so I think I may have more of a problem with (b). For example, I failed to mention the paper I have on arxiv with video explanation and video presentation with followup work that I made for my honour project. I did write “I would love comments on my WIP here: OIS” in the section on a document I am drafting, but it is not prominently displayed. Do you think those are good examples of (a), or should I focus on making my work easier to give feedback on?
-- Regarding (c), I have had email discussions with the authors of papers that inspired my work, but I probably failed to emphasize a request for critical feedback. I admit I find doing this kind of campaigning quite exhausting and so do less than I should. Do you have any advice or links for how to “build relationships with folks who can give you high quality input regularly”, especially as an introvert?
Thanks again!
Quick thoughts:
Great that you have work like the arxiv paper! You could even explicitly ask for feedback on that work
Make it easy for people to understand your work: Try and answer questions like “Why did I do this? What did I learn and/or what update did I make? What is my theory of change?”, and so on...
Make it easy for people to engage with your work: Display it prominently, tweet about it, write a blogpost on lesswrong about it. Polish and publish the code base (see an example here), and so on...
Everyone has their own style of building relationships. I think a powerful way to do so is to try and add value to others: can you summarise/discuss their work in public, or give them feedback, or extend it in an interesting way? Are there volunteer or part-time opportunities that you can help out with? Can you identify issues in their codebases and improve them?