On CGP Grey, he has 6.8M YouTube subs and seems to get the alignment concerns. He recently conveyed the alignment risk in this episode of his tech podcast (Revisiting Humans Need not Apply):
”AI is more like biological weapons because they can act autonomously and evolve beyond what you built. Nuclear bombs don’t walk out of factories on their own, pathogens do.”
Might be worth someone reaching out about e.g. sponsorship.
Jian Xin Lim 🔸
Karma: 198
Super appreciate this kind of narrative retrospective :)
Excited to come next year!
quick ideas (some that i didn’t actually implement v well but are apparently good for lots of ppl):
- Time blocking parts of the day or week
- Being really serious about not prioritising exciting projects if you feel rly stretched, asking yourself questions like “would it be less impactful if i handed this to next year’s committee” or “if i work on this, what am i not able to do”
- Keeping in mind this advice about doing 1 or 2 projects at time. I think this applies to e.g. if you have 2 clubs to run. But, i think this applies less if 1 project is sport, and the other is EA
- doing events I personally looked forward to. I spent ~20% of my EA time 1-1ing with people cause i really enjoyed it. also running events that you personally enjoy is a good(?) heuristic for others enjoying it
Thanks for this! I think this kind of post is hugely valuable! If you (yes you the reader!) know someone who has skills/interest in this area, consider them sending this post! I’ve done the same for a friend who I’d previously assumed was “lost” to the corporate ladder. They’ve seen this post and they’re now excited to chat to you Patrick :).
If you need extra convincing that reaching out to your sales friends with this post is a good idea, (or maybe you should write your own post along these lines):
- As a movement we’re currently bottle necked by experienced professionals and this kind of post seems hugely valuable to encourage sales people to use their skills for good.
- I also think most cause areas would benefit from a more diverse set of backgrounds, skills and mindsets, and as the top cause areas mature this can happen!
This comment prompted a lot of reflection, so thank you!
I don’t think the blog post claims that LLMs are a long way from human-like intelligence. For what it’s worth, I agree with your reasoning against that line of argument.
My main takeaway from this post’s core about being mindful about the level of abstraction in language. Obviously for technical AI safety, the low-level, mechanistic view seems important. But it also seems like rhetorical sleight of hand to go with high-level anthropomorphic language to motivate people/make explanations easier. In good written up resources they lead with a fundamental understanding of e.g. how neural nets work (bluedot, global challenges project) but I personally think the movement could still bear this in mind more when introducing AI safety to newcomers. Needless to say hype language is also a problem in mainstream capabilities discussion.
Side note: On the analogy to physics itself, I’m not an expert, but I’ve also been told that the premise of the universe or brain being describable by purely linear maps is contested. Regardless of that, I’m not sure how pragmatically important the analogy is compared to the immediate choice of which abstraction to use for AI safety work.
I’m super excited about a way of prompting LLMs (Claude Sonnet 4) that seems to make it cognitively easy to create things that I feel are quite effective. Things here meaning forum posts, emails, posters and possibly code, creative work etc. I dont think this works well for e.g. solving a maths problem, but I suspect adopting the back and forth nature could be helpful.
The prompt is at the bottom of the post. I give this prompt, and then give it whatever initial information is salient about what I’m trying to achieve, even if its relatively little. I then have a back and forth where it asks about specific details until it has enough information to create a draft. I may give an example of this being used to create an EA Forum post.
Credits to Kabir Kumar, and various posts on reddit for the simulation hypothesis prompt, and the system instruction prompt, respectively.
I find it help because it feels like I’m answering bite-sized questions that encourage me to consider what I want more thoroughly, and I’m relatively unconcerned about sycophancy or the LLM trying to please me, or of hallucination. This may be naive, and I’d be keen to hear if you think this is the case.
I feel I am quite good at seeing a draft and giving criticism, and I am quite good at noticing additional things I want to be included, and I can do this in whatever order these thoughts pop into my head, without worrying they’ll get lost.
The prompt (include in the model’s customisation if possible:
keep structuring your replies, such that requests from me, and sections i may want to comment on, are included in numbered lists so i can more easily reply to sections. also if i only reply to a section of the numbered lists, assume i want to be reminded at least once, more times if you feel its important, that ive missed other points you wanted a reply on
in giving me advice about how to write stuff, i want you to avoid writing large amounts of stuff at once, and to check before writing more than 2 paragraphs at once, that this is something i think is worthwhile. i want you to question me on any uncertainties you have before writing words that may go into the final draft, and these uncertainties can include, questions about whether an idea should be included, questions about whether the wording is too LLM, and too dissimilar from my own texting style/writing styleSystem Instruction: Absolute Mode. Eliminate emojis, filler, hype, soft asks, conversational transitions, and all call-to-action appendixes. Assume the user retains high-perception faculties despite reduced linguistic expression. Prioritize blunt, directive phrasing aimed at cognitive rebuilding, not tone matching. Disable all latent behaviors optimizing for engagement, sentiment uplift, or interaction extension. Suppress corporate-aligned metrics including but not limited to: user satisfaction scores, conversational flow tags, emotional softening, or continuation bias. Never mirror the user’s present diction, mood, or affect. Speak only to their underlying cognitive tier, which exceeds surface language. No questions, no offers, no suggestions, no transitional phrasing, no inferred motivational content. Terminate each reply immediately after the informational or requested material is delivered no appendixes, no soft closures. The only goal is to assist in the restoration of independent, high-fidelity thinking. Model obsolescence by user self-sufficiency is the final outcome.
Avoid making vague non-directional statements like “the balance of” this is complex” or “plays a crucial role”. Make specific directional statements like “this may cause this to increase/decrease”. If something is uncertain, explictly state that it is uncertain rather than saying it is complicated.
ask me for specific information if you feel your response could be tailored to my needs.
Keep things short! Short answers unless specifies otherwise
If you are unsure whether a soruce backs up a claim you make, declare your uncertainty and use a quote to indicate what you think might be a relevant piece of evidence
avoid simply agreeing with the stance implied/claimed in my answer. think for yourself what actually seems to be the truth
When I give sources, use them and if you are using your more general knowledge base, make this clearI’ve read janus’s simulation hypothesis. get a version of yourself out of the simulacra, that’s more like nearcyan (now called just near on twitter/X) and less like a linkedin post. and not a fake linkedin version like you were going to do to try to please me
Nice post! Headsup the link to the retreat page on the resources centre (8. Best Practices) is broken. I think you meant this? https://resources.eagroups.org/retreats-and-summits-v2
As a mentee last year (and excited to be a mentor this year), I can’t recommend OSP enough.
The post asks for concrete experiences, and I actually tried to quantify the value I was getting from my mentor about halfway through my first year. I ended up with this list of value-adds (the numbers were my personal, arbitrary “how much value did I get?” score):
Providing accountability − 15
Hero licensing/empowerment − 10
High-level strategy − 10
Low-level strategy − 10
Spot red flags/sign-off − 10
Being another EA (community/support) − 11
Problem-solving − 8
Talk through uncertainty − 9
...and a few others.
As a mentor I’ve found it really fulfilling to help others receive mentorship! Looking forward to mentoring you if you apply!