On CGP Grey, he has 6.8M YouTube subs and seems to get the alignment concerns. He recently conveyed the alignment risk in this episode of his tech podcast (Revisiting Humans Need not Apply):
âAI is more like biological weapons because they can act autonomously and evolve beyond what you built. Nuclear bombs donât walk out of factories on their own, pathogens do.â
Might be worth someone reaching out about e.g. sponsorship.
Jian Xin Lim đ¸
Karma: 198
Four Goals for EA ComÂmuÂnity BuildÂing, After RunÂning out of ObÂviÂous Cause Areas
Food/âSnacks to GiveÂaway durÂing Tabling/âStalls
Super appreciate this kind of narrative retrospective :)
Excited to come next year!
quick ideas (some that i didnât actually implement v well but are apparently good for lots of ppl):
- Time blocking parts of the day or week
- Being really serious about not prioritising exciting projects if you feel rly stretched, asking yourself questions like âwould it be less impactful if i handed this to next yearâs committeeâ or âif i work on this, what am i not able to doâ
- Keeping in mind this advice about doing 1 or 2 projects at time. I think this applies to e.g. if you have 2 clubs to run. But, i think this applies less if 1 project is sport, and the other is EA
- doing events I personally looked forward to. I spent ~20% of my EA time 1-1ing with people cause i really enjoyed it. also running events that you personally enjoy is a good(?) heuristic for others enjoying it
Thanks for this! I think this kind of post is hugely valuable! If you (yes you the reader!) know someone who has skills/âinterest in this area, consider them sending this post! Iâve done the same for a friend who Iâd previously assumed was âlostâ to the corporate ladder. Theyâve seen this post and theyâre now excited to chat to you Patrick :).
If you need extra convincing that reaching out to your sales friends with this post is a good idea, (or maybe you should write your own post along these lines):
- As a movement weâre currently bottle necked by experienced professionals and this kind of post seems hugely valuable to encourage sales people to use their skills for good.
- I also think most cause areas would benefit from a more diverse set of backgrounds, skills and mindsets, and as the top cause areas mature this can happen!
This comment prompted a lot of reflection, so thank you!
I donât think the blog post claims that LLMs are a long way from human-like intelligence. For what itâs worth, I agree with your reasoning against that line of argument.
My main takeaway from this postâs core about being mindful about the level of abstraction in language. Obviously for technical AI safety, the low-level, mechanistic view seems important. But it also seems like rhetorical sleight of hand to go with high-level anthropomorphic language to motivate people/âmake explanations easier. In good written up resources they lead with a fundamental understanding of e.g. how neural nets work (bluedot, global challenges project) but I personally think the movement could still bear this in mind more when introducing AI safety to newcomers. Needless to say hype language is also a problem in mainstream capabilities discussion.
Side note: On the analogy to physics itself, Iâm not an expert, but Iâve also been told that the premise of the universe or brain being describable by purely linear maps is contested. Regardless of that, Iâm not sure how pragmatically important the analogy is compared to the immediate choice of which abstraction to use for AI safety work.
A non-anÂthroÂpoÂmorÂphized view of LLMs
Should EA InÂtroÂducÂtions Be More TransÂparÂent About the State of Cause PriÂoriÂtiÂzaÂtion?
Jian Xin Limâs Quick takes
Iâm super excited about a way of prompting LLMs (Claude Sonnet 4) that seems to make it cognitively easy to create things that I feel are quite effective. Things here meaning forum posts, emails, posters and possibly code, creative work etc. I dont think this works well for e.g. solving a maths problem, but I suspect adopting the back and forth nature could be helpful.
The prompt is at the bottom of the post. I give this prompt, and then give it whatever initial information is salient about what Iâm trying to achieve, even if its relatively little. I then have a back and forth where it asks about specific details until it has enough information to create a draft. I may give an example of this being used to create an EA Forum post.
Credits to Kabir Kumar, and various posts on reddit for the simulation hypothesis prompt, and the system instruction prompt, respectively.
I find it help because it feels like Iâm answering bite-sized questions that encourage me to consider what I want more thoroughly, and Iâm relatively unconcerned about sycophancy or the LLM trying to please me, or of hallucination. This may be naive, and Iâd be keen to hear if you think this is the case.
I feel I am quite good at seeing a draft and giving criticism, and I am quite good at noticing additional things I want to be included, and I can do this in whatever order these thoughts pop into my head, without worrying theyâll get lost.
The prompt (include in the modelâs customisation if possible:
keep structuring your replies, such that requests from me, and sections i may want to comment on, are included in numbered lists so i can more easily reply to sections. also if i only reply to a section of the numbered lists, assume i want to be reminded at least once, more times if you feel its important, that ive missed other points you wanted a reply on
in giving me advice about how to write stuff, i want you to avoid writing large amounts of stuff at once, and to check before writing more than 2 paragraphs at once, that this is something i think is worthwhile. i want you to question me on any uncertainties you have before writing words that may go into the final draft, and these uncertainties can include, questions about whether an idea should be included, questions about whether the wording is too LLM, and too dissimilar from my own texting style/âwriting styleSystem Instruction: Absolute Mode. Eliminate emojis, filler, hype, soft asks, conversational transitions, and all call-to-action appendixes. Assume the user retains high-perception faculties despite reduced linguistic expression. Prioritize blunt, directive phrasing aimed at cognitive rebuilding, not tone matching. Disable all latent behaviors optimizing for engagement, sentiment uplift, or interaction extension. Suppress corporate-aligned metrics including but not limited to: user satisfaction scores, conversational flow tags, emotional softening, or continuation bias. Never mirror the userâs present diction, mood, or affect. Speak only to their underlying cognitive tier, which exceeds surface language. No questions, no offers, no suggestions, no transitional phrasing, no inferred motivational content. Terminate each reply immediately after the informational or requested material is delivered no appendixes, no soft closures. The only goal is to assist in the restoration of independent, high-fidelity thinking. Model obsolescence by user self-sufficiency is the final outcome.
Avoid making vague non-directional statements like âthe balance ofâ this is complexâ or âplays a crucial roleâ. Make specific directional statements like âthis may cause this to increase/âdecreaseâ. If something is uncertain, explictly state that it is uncertain rather than saying it is complicated.
ask me for specific information if you feel your response could be tailored to my needs.
Keep things short! Short answers unless specifies otherwise
If you are unsure whether a soruce backs up a claim you make, declare your uncertainty and use a quote to indicate what you think might be a relevant piece of evidence
avoid simply agreeing with the stance implied/âclaimed in my answer. think for yourself what actually seems to be the truth
When I give sources, use them and if you are using your more general knowledge base, make this clearIâve read janusâs simulation hypothesis. get a version of yourself out of the simulacra, thatâs more like nearcyan (now called just near on twitter/âX) and less like a linkedin post. and not a fake linkedin version like you were going to do to try to please me
Some adÂvice for startÂing up an EA Uni GroupâBureauÂcracy, FundÂing and Succession
Nice post! Headsup the link to the retreat page on the resources centre (8. Best Practices) is broken. I think you meant this? https://ââresources.eagroups.org/ââretreats-and-summits-v2
As a mentee last year (and excited to be a mentor this year), I canât recommend OSP enough.
The post asks for concrete experiences, and I actually tried to quantify the value I was getting from my mentor about halfway through my first year. I ended up with this list of value-adds (the numbers were my personal, arbitrary âhow much value did I get?â score):
Providing accountability â 15
Hero licensing/âempowerment â 10
High-level strategy â 10
Low-level strategy â 10
Spot red flags/âsign-off â 10
Being another EA (community/âsupport) â 11
Problem-solving â 8
Talk through uncertainty â 9
...and a few others.
As a mentor Iâve found it really fulfilling to help others receive mentorship! Looking forward to mentoring you if you apply!