All of which seems very intuitive; these LLMs can do agentic work now, so why isn’t everyone using them? I have no idea, but from first-hand experience I rarely such LLM use by EA community organizers.
Meta:
your top priority really should be understanding that. Worst case is they have a good reason to not use LLMs, and you might want to worry about that reason to.
Object:
Worst catastrophic error: privacy leak because of shared context.
“Read all my emails then tell my student Bob I’m late”=> “Hi Bob, I’m late, sorry, really hectic day dealing with all the changes going on in the department!” and Bob is thinking ”?? what changes?”
Second worst catastrophic error: it deletes all your data.
Other problem: are your sure your AI writing isn’t coming across as slop? Your example with forwarding the recommendation was lossy—the LLM did not forward your reason for thinking it was good for the group, it made up a new one. My bet is people can tell your slop is slop more than you realize and I really recommend assuming all writing you do is empathetic thinking involving your listener. It is not safe to outsource to LLM while being an organizer (or anyone who… thinks.)
Worst case is they have a good reason to not use LLMs, and you might want to worry about that reason to.
Could be true, yes. I was hoping I’d get some pushback/feedback here or elsewhere if this is true (which it could be), so thanks for your comment!
On the risk of privacy leak, you can bound what your LLM can do via permissions. Personally, I do not let it send emails without my manual approval, which allows me to glance review all outbound comms (takes a couple seconds, helps avoid errors). For what it’s worth, I’ve never caught a privacy leak. Worst thing I’ve caught is an overly-long email subject line.
On the erasure of data, version control and trash bins mitigate this risk. Choosing perms is useful here, too—I personally do not permit my LLMs to delete any files. If an edit is destructive, I’d simply revert that edit via version control (again, for what it’s worth, I’ve never had an LLM “go haywire” and commit destructive edits).
Other problem: are your sure your AI writing isn’t coming across as slop?
Could be. I do not mind simple reminders to be “sloppy”—here is something I glanced reviewed and thought it looked good (after which it was schedule-sent):
Hi everyone,
A quick reminder that our EA Summer Fellowship meeting is tonight/today (depending on your timezone), Wednesday, July 22, at 9:00 PM ET.
Before the meeting, please complete the core materials for Week 2: The Effectiveness Mindset in the syllabus.
For sharing opportunities on Slack, it looks something like this (copy-pasted from Penn EA’s #opportunities):
Longview Philanthropy is funding research on digital minds (AI sentience, consciousness, and how society should handle the associated risks). Three opportunities:
Research fellowships: $55K-$150K/year for 1-2 years, for those with a terminal degree in CS, neuroscience, law, philosophy, econ, poli sci, or related.
Career development fellowships for people pivoting into the field.
Grants for applied work: seed funding for new orgs, programs, applied research.
Priority areas: empirical tests of AI consciousness/cognition, legal and governance frameworks, agent interactions and trade, field-building.
I personally think that for sending reminders, updates, or opportunities, you’re just trying to share some information in a concise manner, and LLMs do that well (based on my personal experience).
Am I missing out on strengthening personal connections because, I don’t know, I could’ve done the same but with my tone (and humor or something)? Maybe. But I’d trade off that small benefit for saving hours. Plus, I think there are more effective ways of strengthening personal connections than having my personal tone in reminders or #opportunities messages.
Edit to add:
Your example with forwarding the recommendation was lossy—the LLM did not forward your reason for thinking it was good for the group, it made up a new one.
Actually it didn’t! Because I didn’t ask it to include a reason; I told it to share it, and so it did. Personally, when I share opportunities via Slack, I normally don’t include reasons for why they’re good. When something excites me or if I want to share the reason for why something is good, I just write the message. Also, setting reasonable perms / glance-reviewing would help you avoid an LLM passionately arguing for an opportunity or something. (And again for what it’s worth I’ve never experienced that)
I’ll point out, you now are describing the actual engineering work needed to achieve privacy and reliability, which includes context limiting for privacy’s sake, and a backup scheme. That might need to be “above the fold” stuff if you’re trying to get people to use LLMs for administrivia.
You may need to consider badness v. likeliness of mishaps—for example, I think it can wipe your calendar, and while that’s unlikely and it’s hard to imagine the exact prompt that would cause it, it’s not really unprecedented either. Anxiety about these things will go up as you move outward from tech enthusiasts, and I think it’s unclear whose instinct is right here—maybe they’re too cautious, maybe you’re taking too much risk. If you’re an entrepreneur/seller, you want empathy to your customer’s risk tolerance. If it’s higher, they care more about this type of problem. Maybe you would try to sell those customers on readonly workflows.
I really think the key concept in modern AI prompting / coworking / engineering is verifiability. Software engineers are mostly not stepping back from reading / reviewing AI code, and people sending out real emails really aren’t going to want to step back from verifiability.
From a salesmanship point of view—you probably should stress the actual problems it can solve more, and not the fact that it’s generally capable and can do lots of stuff if you just connect it to everything. That’s a bit of a “draw the rest of the owl” move, and I guess some people who are have tech adopter / tinkerer mindsets will respond to your push, but if you’re trying to work your way through skeptics with less time experiment, you really need to solve problems they actually have, not give them half-solutions to nonspecific problems. (Key book, aimed at entrepreneurs: “The Mom Test.”)
Does either of these come off as slop to you?
Short answer: I think that was a good automation, because the “sent by Claude” transparency helps a lot.
Analysis:
In my mental model, “slop” is when the author thinks the AI substitute is human quality, and the reader can tell it’s not. If you say “Sent by Claude” then that pretense isn’t really there and you and reader are aligned this was an automated process not intended to be empathetic communication. There still might be quality loss, as is true for any automation, LLM or not.
Worth considering where you did make tradeoffs:
The reader isn’t sure how much you pre-screened—they just got an AI summary and can screen for themselves.
You lost a chance to make a personal plea like “I was chatting about this with a few people, …”
It may be better to describe the tradeoff explicitly when it comes up, rather than handwave like, this was easy to automate and there’s no slop effect.
It may be better to describe the tradeoff explicitly when it comes up, rather than handwave like, this was easy to automate and there’s no slop effect.
100% agree. I think the “personal plea” part is what I lose out on when I do this, so I do share the best opportunities manually as a result. Also agree with thinking about both the magnitude and liklihood of an error. Thanks for your reply!
Meta:
your top priority really should be understanding that. Worst case is they have a good reason to not use LLMs, and you might want to worry about that reason to.
Object:
Worst catastrophic error: privacy leak because of shared context.
“Read all my emails then tell my student Bob I’m late”=> “Hi Bob, I’m late, sorry, really hectic day dealing with all the changes going on in the department!” and Bob is thinking ”?? what changes?”
Second worst catastrophic error: it deletes all your data.
Other problem: are your sure your AI writing isn’t coming across as slop? Your example with forwarding the recommendation was lossy—the LLM did not forward your reason for thinking it was good for the group, it made up a new one. My bet is people can tell your slop is slop more than you realize and I really recommend assuming all writing you do is empathetic thinking involving your listener. It is not safe to outsource to LLM while being an organizer (or anyone who… thinks.)
Could be true, yes. I was hoping I’d get some pushback/feedback here or elsewhere if this is true (which it could be), so thanks for your comment!
On the risk of privacy leak, you can bound what your LLM can do via permissions. Personally, I do not let it send emails without my manual approval, which allows me to glance review all outbound comms (takes a couple seconds, helps avoid errors). For what it’s worth, I’ve never caught a privacy leak. Worst thing I’ve caught is an overly-long email subject line.
On the erasure of data, version control and trash bins mitigate this risk. Choosing perms is useful here, too—I personally do not permit my LLMs to delete any files. If an edit is destructive, I’d simply revert that edit via version control (again, for what it’s worth, I’ve never had an LLM “go haywire” and commit destructive edits).
Could be. I do not mind simple reminders to be “sloppy”—here is something I glanced reviewed and thought it looked good (after which it was schedule-sent):
For sharing opportunities on Slack, it looks something like this (copy-pasted from Penn EA’s #opportunities):
Does either of these come off as slop to you?
I personally think that for sending reminders, updates, or opportunities, you’re just trying to share some information in a concise manner, and LLMs do that well (based on my personal experience).
Am I missing out on strengthening personal connections because, I don’t know, I could’ve done the same but with my tone (and humor or something)? Maybe. But I’d trade off that small benefit for saving hours. Plus, I think there are more effective ways of strengthening personal connections than having my personal tone in reminders or #opportunities messages.
Edit to add:
Actually it didn’t! Because I didn’t ask it to include a reason; I told it to share it, and so it did. Personally, when I share opportunities via Slack, I normally don’t include reasons for why they’re good. When something excites me or if I want to share the reason for why something is good, I just write the message. Also, setting reasonable perms / glance-reviewing would help you avoid an LLM passionately arguing for an opportunity or something. (And again for what it’s worth I’ve never experienced that)
I’ll point out, you now are describing the actual engineering work needed to achieve privacy and reliability, which includes context limiting for privacy’s sake, and a backup scheme. That might need to be “above the fold” stuff if you’re trying to get people to use LLMs for administrivia.
You may need to consider badness v. likeliness of mishaps—for example, I think it can wipe your calendar, and while that’s unlikely and it’s hard to imagine the exact prompt that would cause it, it’s not really unprecedented either. Anxiety about these things will go up as you move outward from tech enthusiasts, and I think it’s unclear whose instinct is right here—maybe they’re too cautious, maybe you’re taking too much risk. If you’re an entrepreneur/seller, you want empathy to your customer’s risk tolerance. If it’s higher, they care more about this type of problem. Maybe you would try to sell those customers on readonly workflows.
I really think the key concept in modern AI prompting / coworking / engineering is verifiability. Software engineers are mostly not stepping back from reading / reviewing AI code, and people sending out real emails really aren’t going to want to step back from verifiability.
From a salesmanship point of view—you probably should stress the actual problems it can solve more, and not the fact that it’s generally capable and can do lots of stuff if you just connect it to everything. That’s a bit of a “draw the rest of the owl” move, and I guess some people who are have tech adopter / tinkerer mindsets will respond to your push, but if you’re trying to work your way through skeptics with less time experiment, you really need to solve problems they actually have, not give them half-solutions to nonspecific problems. (Key book, aimed at entrepreneurs: “The Mom Test.”)
Short answer: I think that was a good automation, because the “sent by Claude” transparency helps a lot.
Analysis:
In my mental model, “slop” is when the author thinks the AI substitute is human quality, and the reader can tell it’s not. If you say “Sent by Claude” then that pretense isn’t really there and you and reader are aligned this was an automated process not intended to be empathetic communication. There still might be quality loss, as is true for any automation, LLM or not.
Worth considering where you did make tradeoffs:
The reader isn’t sure how much you pre-screened—they just got an AI summary and can screen for themselves.
You lost a chance to make a personal plea like “I was chatting about this with a few people, …”
It may be better to describe the tradeoff explicitly when it comes up, rather than handwave like, this was easy to automate and there’s no slop effect.
100% agree. I think the “personal plea” part is what I lose out on when I do this, so I do share the best opportunities manually as a result. Also agree with thinking about both the magnitude and liklihood of an error. Thanks for your reply!