edit: I accidentally retracted this without meaning to. The comment has been reposted below.
Hey Kit. I was in charge of marketing for EAG this year so I can explain what we did and why. I’m very sorry you felt like we were being deceptive. On my end, there’s a difficult line to walk between using the copy that best accomplishes our goals and using copy that is maximally clear. Feedback on whether we’ve made the right calls will be helpful for better calibration in the future.
Sending emails ‘from’ other people. Friends I recommended received emails with ‘from’ name ‘Kit Surname via EAG’. Given that I did not create the content of these emails, this seemed somewhat creepy, and harmed outreach.
For some context here, when a person is nominated we put them into an automated Mailchimp flow. If the person applied or clicked a link in an email they were removed from the automation. If they did neither of these we sent them 3 emails asking them to apply. The email you’re referring to was the last of three emails.
Julia and I actually thought quite a bit about the message and concluded that it wasn’t deceptive for a few reasons:
1) The from line included “via EAG”
2) the email address the message came from was “hello@eaglobal.org.”
3) The email itself seems to be clearly from us and not the nominator. For example, the first paragraph is “If you want to use the nomination [fname] [lname] gave you for Effective altruism Global 2016 you need to do it by the end of today.”
4) The email copy contains an unsubscribe link.
5) The email copy contains CEA’s name and address.
I’m interested in whether you think it’s deceptive given this information.
fake deadlines
Over the course of marketing EAG we found that deadlines were effective in getting people to apply for EA Global. Yet, it’s also very important for us that a large percentage of people apply before we have to do things like order shirts, send the headcount to the caterer, etc.
To try to get some of the power of deadlines without encouraging a ton of last-minute applications, we set rolling application review deadlines. For most applications, we promised a two-week turnaround to hear back. The rolling application deadlines were dates where we promised a faster turnaround time if you applied by a particular date.
After reviewing our copy I think we were insufficiently clear about this in some of our emails. I’ll try to be more vigilant in the future. However, I think plan itself is not deceptive if properly communicated. Interested in what people think.
‘I was looking through our attendee database’
Unless you’re referring to something different than I think you are, this was true. I went through our attendee database and found everyone that hadn’t yet claimed a ticket. I then put these folks into some email automation for follow-up on whether they planned to claim their ticket.
Do you mean the automated “you’re a cool person, come to EA” emails or something else? FWIW I thought those were pretty childish.
I agree. These emails were a mistake. They were written by an intern and didn’t go through the proper review channels before sending them out. We instituted some policies after this happened to make the review process more clear, but don’t have enough data to know if that will solve the problem. It’s something we’ll be working to address in the future.
Hi Kerry! Congratulations again for the exceptional conference, and thanks for adding detail.
Updates I’ve made:
while in my tiny sample of 13 the emails with ‘from’ names like ‘Kit Surname via EAG’ worked out badly, it looks like you produced the most reasonable emails of that form possible without the benefit of hindsight. In answer to your question, I call this dishonest primarily because it gives the appearance of endorsement of content which I do not endorse. I would still not do this.
the deadlines at first appeared to be mainly to generate haste, but some or all had operational function. My blanket terming ‘fake deadlines’ was therefore wrong.
aside from ‘we trust Kit’s judgement’, I see that most/all other statements made in the campaign were true in a technical sense. However, I maintain that this is insufficient. ‘I was looking through our attendee database’ is a great example, precisely because the whole message implies specificity to the recipient, while it appears that the looking could have been replaced by a single filter for people who hadn’t bought tickets. Likewise for ‘ideal participant’. At the very least, I’d bin these along with the “you’re a cool person, come to EA” emails Michael mentioned.
Additional arguments against my position:
if CEA has standards substantially above average for its reference class, people might still not trust EAs to the extent I would like
maybe we don’t particularly need highly involved EAs to trust each other more, and this kind of marketing won’t materially affect what less involved people think.
I had also suspected that my concerns put me in a niche group which holds a small proportion of total relevance. I have updated away from this suspicion because the ratio of people who at the present time register a desire for greater honesty (17-27, probably nearer 17) to those who register no concern (3-5) is much higher than I had anticipated, and I suspect that forum participants are a highly relevant class for cooperation considerations.
To the other 16+ of those 17+ people: if my views are not representative of yours, it could be valuable for you to say so.
Really appreciate the response here, Kerry. Adding some extra feedback for calibration. Apologies in advance that I’m realizing I’m struggling to balance the strength of my opinion with my knowledge that this was well-intended. Just to be clear, my views only here, not my employer’s.
I didn’t end up nominating anybody because I’d rather reach out to people myself. The “via EAG” thing makes me really relieved that I made this choice and will prevent me from nominating people in the future. I’m actually a bit surprised at the strength of my reaction but this would’ve felt like a major violation to me. I really dislike the idea of feeling accountable for words that I didn’t endorse. Just for example, I could plausibly have invited work contacts who I’m not super close with and whom I would be very sensitive to being perceived as spamming.
After your explanation the practice still does seem (very) deceptive to me. At the very least, I’d expect a lot of people to click on the email because they think it’s coming from me and then to realize that it came from someone else. If I received this email, I’m sure I’d eventually figure out it wasn’t from the person in the “from” line but I’d be confused for a bit and might assume that they approved it even if they didn’t write it.
Moreover, if I wanted to not only nominate someone but also send them an email advising them to attend, I could easily do so. Some people may even have done that so their nominees would have felt like they received multiple unsolicited pings from the same person. I know it would have a lower yield but I feel like EAG should have emailed [Firstname at Lastname] and asked them to ping their nominee instead of spoofing their identity in the “from” line and taking this decision out of their hands.
I’d acknowledge that most of the other practices on this thread seem like basically standard marketing techniques. They seem off-putting according to my personal taste and I’d guess they’re counter-productive but because they’re so standard it also seems likely that I’m just being biased against them because I find marketing distasteful. I want to make clear that I’d put the “via EAG” thing in another category—substantially worse than I’d expect from a typical sales email.
Lower priority stuff:
Deadlines
I don’t have a problem with rolling deadlines if it’s clear that’s what they are. I didn’t pay a ton of attention to this so I don’t have a strong take. It did seem like discounts went up as it got closer to the actual date and I think that did feel a bit like taking advantage of the people who helped out by signing up early.
Looking through the attendee database
This language feels off-putting and slightly deceptive to me. As Kit says, it’s intended to make it sound like you were thinking of that specific person when it wasn’t the case. Unlike the “via EAG,” I think this practice is basically standard but I still really dislike it. Kit’s comment that “my vanity fooled me for a solid few seconds, by the way!” strikes me as a really good reason to discontinue this practice. I think it’s a bad experience and kind of embarrassing to feel like you’re getting a personal compliment and then realize it’s a form email.
I feel similarly about some other language Kit mentioned.
I was unfortunately unable to go to EAG this year due to work commitments. By all accounts I missed out—additionally, I had less exposure to the EAG marketing than others—I mostly ignored the promotional material as I knew I was unavailable.
I confess I find these practices pretty shady, and I am unpleasantly surprised that EAG made what I view to be a fairly large error of judgement on appropriate marketing tactics. (I am pleasantly unsurprised in the straightforward and open manner with which this criticism has been received). On the issues raised by Kit above.
If I recommended (e.g.) Kit to EAG and he doesn’t reply a couple of times, he gets an email ‘from Greg via EAG’, despite: 1) I’m not sending it, 2) none of the content is written by me, 3) I’m not asked whether I consent to this message being sent ‘on my behalf’, and 4) I’m not told it happens unless the recipient gets back to me.
Although uncharitable, this looks like ventriloquism or arrogation—and such an impression may well occur in cases where do not read the body of the message. Perhaps the most accurate account is: ‘Greg’s social tie with Kit is leveraged (without Greg’s knowledge and consent) in a misleading subject line to get Kit to click on the last ditch sales pitch’, which I think should be avoided.
Especially given in this case Kit is receiving emails he did not solicit on Greg’s say so. I might think Kit might value knowing about the opportunity, but I might also have a sufficiently high view of his powers of judgement that he can decide after the first email whether he’s interested or not. When he gets a third such email, ostensibly with my approbation and/or involvement, he might start feeling irritation towards me. If I really want Kit to attend EAG so much I’d send him multiple emails, I can send these myself; if instead I think the costs of ‘pushing it’ to whatever social tie we have outweigh the increased likelihood of another email prodding him to EAG, I definitely do not want it being done ostensibly ‘on my behalf’ without asking—and especially without telling—me. (The same concerns apply to my implicit endorsement of whatever this email actually says).
[This paragraph is mistaken, and remains just so people can follow the thread of discussion] The worry with ‘rolling deadlines’ is if the deadline isn’t really a deadline for those in the earlier waves. The threat of missing out is scares them to commit early to help you, but after the ‘sham deadline’ passes, the mask drops, and you are happy for them to confirm etc. Although often just akrasia or poor organisation, people may have good reason for waiting if they are weighing up other ways to spend their time, and there’s an (admittedly remote) risk getting them to commit to tickets earlier than they need to deprives them of other opportunities. I confess I’m still not entirely clear what the rolling deadlines entailed, so please disregard if it is inapposite to what EAG’s deadlines represented.
I think the ‘looking through the attendee database’ also sails too close to the wind. I think the impression that evokes is something like, “I was checking our list of attendees, and I suddenly realised you hadn’t got a ticket (!) Given your high status and reputation, I realised your non-attendance would be a great shame, so I thought I’d take the trouble to reach our personally”. What actually happened, I assume, is the list of non-ticket buying attendees are pulled via a query on 1-2 Booleans from the database, a stock email is constructed, and a mail merge is performed.
These less than wholly candid approaches weren’t necessary: one could have sent a final reminder without anyone’s name—if one thought social proof was really key, one could have asked the nominators if they were happy for this email to be sent, or prodded the nominators to reach out to the nominees, module suggestions or even an email template. One could genuinely pre-commit to treating early deadlines as deadlines and making this clear, or simply urge people to sign up earlier to help the logistics. “Our records show” or similar is strictly more accurate than “I was looking through our database”.
I agree there is likely a trade off between candour and efficacy: the alternatives are probably less persuasive, entail more overhead, or both. I think this should be unsurprising on reflection—to whatever degree marketing is subterfuge, or trying to encourage as much ‘buying’ as possible, good marketing strategies can be hindered by frank honesty of the objective merits. Yet I aver one should take candour as all-but-lexically prior to efficacy concerns, as this is much consonant with EA norms (whatever exactly they are).
It is a common refrain to object to overblown empirical claims about (e.g.) how many lives you can save for a dollar, and to insist it is important to see how the world really works to understand how to best improve it. I think the sample principles should apply to our interactions with one another: groups shouldn’t ‘oversell’ their impact, and we should not mislead other EAs into our own designs. We should counter-signal many marketing gimmicks in the same way we (try to) countersignal shoddy empirical work.
There is both a commons problem and an increasingly common problem. The costs of increasing marketing and other behaviours (one of the other commonly remarked upon is how frequently EAG posts were shared to all EA related fb groups) are external to the group itself, who are likely much more sensitive to their own efficacy. They will have a skewed impression of the true exchange between these goods: I got the impression—correct me if I’m wrong—that EAG was at times struggling to secure the anticipated attendance, and in such situations high-handed and often unobserved restraint are unappealing. There have also been deeply regrettable behaviours of a particular EA org which will likely be described on this forum soon. Although I stress these are far, far more egregious, they are not a million miles away from stuff mentioned by Kit above.
Relying on all officers for EA orgs to have the resilience of Penelope refuting endless suitors in fealty to the ideal of extremely honest communication is perhaps utopian. Inculcating a general norm across the community to view this stuff poorly may work better. I’d suggest that marketing techniques mentioned by Kit are not used in future by anyone (plus maybe other behaviours—Kit mentioned these were ‘highlights’). I would also recommend caution and circumspection before adopting anything that even treads into the penumbra of the duplicitous—Caesar’s wife principles would be good to internalize. I hope to encourage the wider EA ecosystem to uphold an ethos along these lines, and robustly challenge mistakes—as, happily, Kit exemplified above.
Agree with most of what you said here. But I had a different interpretation of the facts with respect to the “via EAG” issue than you did.
Your impression is that:
if I recommended (e.g.) Kit to EAG and he doesn’t reply a couple of times, he gets an email with ‘greetings from Greg’ or similar in the subject heading
My impression is that “Greg Lewis (via EAG)” would appear in the “from” line. (In the way that email clients often replace the sender’s email address with their name.
If I understand correctly then the practice strikes me as much more likely to deceive a recipient.
It’d be helpful if you could clear this up. If I was confused and you actually just put “Greetings from FirstName LastName” in the subject line or some such, I’d have a substantially weaker reaction.
Thanks for the note. I’ve responded to some aspects of it below.
1) I’m not sending it, 2) none of the content is written by me, 3) I’m not asked whether I consent to this message being sent ‘on my behalf’, and 4) I’m not told it happens unless the recipient gets back to me.
I’ve updated away from sending messages of this type in the future. But, I do think you’re representing the decision as a clear violation whereas I think it’s less clear.
I think we disagree on two things: 1) does the third email cause people to think that you send it and 2) what did people consent to when filling out the nomination form.
With regards to 1), I don’t think there’s any realistic risk that nominees walked away from the third email thinking that the nominator sent it. They might wonder whether this is the case initially, but I think it’s pretty clear that the email was a template send by EA Global.
With regards to 2)
I think the relevant question seems to be what nominators expected would happen after entering someone’s contact information.
I think it’s reasonable to expect that we wouldn’t attempt to cause the recipient to think the email came directly from you. It would be a clear breach if the email address, name, and copy all appeared to be sent directly from you. However, I also think it would be unreasonable to expect that we wouldn’t mention you at all in the email. For example, if the body of the email said that [fname] [lname] nominated you, that seems uncontroversial to me.
The relevant question is where in between those extremes is OK and where is not OK. My current sense is that email body mentions are OK, and subject line mentions are probably OK, but other tactics are not OK unless we explicitly asked for consent before doing it. Do you have a different take?
The worry with ‘rolling deadlines’ is if the deadline isn’t really a deadline for those in the earlier waves. The threat of missing out is scares them to commit early to help you, but after the ‘sham deadline’ passes, the mask drops, and you are happy for them to confirm etc. Although often just akrasia or poor organisation, people may have good reason for waiting if they are weighing up other ways to spend their time, and there’s an (admittedly remote) risk getting them to commit to tickets earlier than they need to deprives them of other opportunities. I confess I’m still not entirely clear what the rolling deadlines entailed, so please disregard if it is inapposite to what EAG’s deadlines represented.
This worry doesn’t seem to apply to the expedited application review version of the rolling deadline. That is, if the deadline is one to apply and hear back quick then attendees can simply not apply for that deadline if they need additional information.
Yet I aver one should take candour as all-but-lexically prior to efficacy concerns, as this is much consonant with EA norms (whatever exactly they are).
Does this claim entail that one should always choose candor over efficacy? If so, that would seem to me to be a very difficult claim to justify.
Kerry, many thanks for your reply. On the matters you remark upon:
The email ‘from X via EAG’
Re. the risk to ‘mistaken identity of sender’, I think a lot of this depends on whether people read the message body or not. I doubt I’m the only person who deletes emails based on the top-line of subject, sender, etc. Perhaps savvier people would realize the ‘X via Y (and with Y’s email address)’ always means a sort of mass-mailing approach that X didn’t have a huge amount to do with, but I doubt that applies to everyone. It doesn’t seem crazy to interpret this as ‘Y is sending this email from X to me’ before they read the message body. If they just delete the email before doing so, it seems likely they will maintain this impression.
Re. Nominator consent, I agree that if you recommend X to ‘pitch’ at Y, I don’t think one can be too aghast to find X uses your recommendation as part of their pitch—I agree mentioning it in the message body or the subject is fine, and also agree the other approach you say is inappropriate is inappropriate.
I still think what actually happened falls on the wrong side of the line. The problem is not so much ‘where do you use X’s name in the pitch’, but the manner of its use. If I nominate Kit to go to EAG, it just isn’t the case that the pitch for EAG is sent ‘from me via EAG’ - especially in the present case where I have no hand in the content of the pitch email. It is not being sent from me in any sense. I think uses along these lines should demand explicit consent.
Rolling deadlines
Sounds fine to me. Forgive my misunderstanding.
Candour/efficacy trade-off
Yet I aver one should take candour as all-but-lexically prior to efficacy concerns, as this is much consonant with EA norms (whatever exactly they are).
Does this claim entail that one should always choose candor over efficacy? If so, that would seem to me to be a very difficult claim to justify.
Not always (hence the ‘all but’ lexical priority) but something like, ‘barring very exceptional circumstances’. I think having this sort of high bar is justifiable on grounds of community norms and overall brand equity: judging from Kit’s remarks, this behaviour has considerably damaged his trust in CEA; judging by the number of upvotes, he may not be the only one.
Naturally, one may worry about how representative this is of overall opinion. What happened ex post is not always the best steer on what was wise ex ante, but if you know how many extra attendees were marshalled by this sort of email, that could be helpful information.
I offer an analogy to one of my interests—mental health law. In the UK (and many places elsewhere) there is a huge emphasis on ‘the least restrictive option’. Out of the variety of coercive actions that might be appropriate in for a person with mental health issues, one should use whatever is the least restrictive and has the least impact on personal freedom. This is pretty much lexically prior to other issues.
I am sure there are many occasions where the least restrictive option is not the best. One may be justifiably confident that the least restrictive option is unlikely to succeed, and early escalation to a more restrictive option would give better outcomes. One may also think that a less restrictive option may be acceptable, yet give chronically worse outcomes compared to a more restrictive one. Yet I think the law is right to uphold this principle even conditioned on consequentialism which sees no non-instrumental value in personal freedom: the risks (realized many times historically) about the overuse and misuse of psychiatric detention and coercion recommend considerable circumspection and reluctance in their present application, and the costs of a rule obliging the least restrictive option are justified by the reduction in risk of similar incidents happening prospectively.
There are going to be occasions more generally in EA where candour is inappropriate (e.g. information hazards). Yet when it comes to marketing and other aspects of intra-EA communication, I think there should be a similar drive towards picking the ‘most candid option’: that, out of the communication content and approaches available, one should select the one that has the lowest chance of misleading or misunderstanding. Although I am sure the alternatives I suggested above would have proven less effective marketing, I am similarly sure the benefits of better preserving commons of mutual trust and a brand equity of extremely high epistemic standards is outweigh them.
The relevant question is where in between those extremes is OK and where is not OK. My current sense is that email body mentions are OK, and subject line mentions are probably OK, but other tactics are not OK unless we explicitly asked for consent before doing it. Do you have a different take?
My read is that this is correct, with the additional caveat that it depends on the wording of the mentions in the body and the subject line. Saying “X nominated you” is fine; implying something stronger probably wouldn’t be.
I find it difficult to think about this issue in a principled way. Not using standard marketing tactics is not costless. We used the language we used because it was the most effective. Using different language would have caused a decrease in EAG attendees and a decrease in the total value of the conference.
For the sake of argument, let’s assume that some kinds of off-putting language are more effective at getting people to attend. How would you model the tradeoff between generating extra value at EA Global on the one hand and the harm of off-putting language on the other?
Suppose the off-putting but more effective language causes an additional 100 people to attend EA Global. Suppose also that in expectation a marginal EAG attendee is worth $4,200 in donations to effective charities. Would you be willing to use off-putting but more effective language if it generated $420,000 in donations to effective charities? If not, is there a different number where you’d be willing to switch?
This discussion is really valuable by the way and I appreciate the time you’ve both put into it.
My take is that there’s a trade-off here between being most effective for short-term value (getting more attendees at EA global) and most effective building a powerful and supported long-term brand. We have better data on what’s effective for the short-term value, because the feedback loops are tighter. This could mean that it should get more weight (because we actually know what we’re doing), but there’s a danger that it means we swing too far towards it. The off-putting messaging could in some low-grade ways lower a whole lot of people’s opinions towards EA/CEA/EAG. For a movement that trades so much on intellectual leadership this worries me.
Here’s one guesstimate. Take with a vast amount of salt, and I don’t even really believe the framework I’m using, I just want to show how you might get going with these comparisons:
Damage to brand = 0.1% of brand value; brand value = ~$1B; so would be willing to switch if generating >$1M
[Also of course it’s not binary. We can probably look for compromise solutions which get a lot of the marketing value and a lot of the long-term brand value]
There’s even a case that at the margin we should prefer more consolidation over more growth (of EA community generally and EAG specifically), in which case it would be good to have emails which are differentially attractive to people (like Howie and Kit) who are/could become high-value community members, rather than differentially off-putting to them.
I think a stronger argument can be made in favour of the chosen marketing methods. It would probably conclude with something like ‘the huge value of a small number of extra links formed between otherwise-disjoint groups outweighed the minor weakening of cooperation standards across the community’.
Owen’s comment shows that the numbers can be big on the other side too, but valuing brands is a notoriously hard problem. In the hope that people refer back to this discussion when considering future strategies, here is an explicit estimate of one component of the value of avoiding minor harm to trust, for this specific case. It works by assuming that anyone put off from CEA simply shifts collaboration from one organisation to another, causing efficiency loss from wasting comparative advantages, not total loss. It also recognises that I made an unusually large update, and the average will be much smaller. Bracketed items are multiplied together to give an italicised item in the next line.
(present value of a GWWC pledge, $73,292 x number of pledges next year, 856) x size of CEA compared to GWWC proxied by headcount, 2.45 x (my unusually large update to engagement with CEA, 30% x perceived relative strength of other affected people’s reactions, 17.5%) x relative advantage of CEA over competition, 17% x proportion of people with negative reactions, 48%
= (value realised by GWWC next year, $62,737,952 x size of CEA compared to GWWC proxied by headcount, 2.45) x (average affected person’s shift from CEA to elsewhere, 5.25% x relative advantage of CEA over competition, 17%) x proportion of people with negative reactions, 48%
= value realised by CEA next year, $153,707,982 x (inefficiency from one affected person’s shift, 0.88% x proportion of people with negative reactions, 48%)
= (value realised by CEA next year, $153,707,982 x proportion of CEA value lost 0.42%)
= value of one year of CEA minor reputation preservation, $638,849
This model does not incorporate the effects EAG marketing can have on other EA organisations’ reputations (I suspect large), the value of not putting people off the movement entirely (unsure), or the effort required to clean up one’s reputation in the unlikely case that lasting harm is incurred (low in expectation?) To handle overoptimisation, I have tried to keep inputs conservative rather than discounting explicitly.
My guess after public and private discussion is that the approach which captures the most total value would be something like aggressive marketing (including pushing known EAs hard to tell their friends, slightly-more-than-comfortable numbers of chaser emails to applicants, and focussing almost entirely on the positives of attending) while avoiding anyone feeling deliberately misled. Obviously CEA is better placed to make this call, and I hope the broad discussion will help guide future decisions.
I realized I never indicated what I thought after the discussion. I now endorse the position Kit suggests:
My guess after public and private discussion is that the approach which captures the most total value would be something like aggressive marketing (including pushing known EAs hard to tell their friends, slightly-more-than-comfortable numbers of chaser emails to applicants, and focussing almost entirely on the positives of attending) while avoiding anyone feeling deliberately misled. Obviously CEA is better placed to make this call, and I hope the broad discussion will help guide future decisions.
I’m not sure if this discussion has changed your view on using deceptive marketing for EA Global, but if it has, what do you plan to do to avoid it happening in future work by EA Outreach?
Also, it’s easy for EAs with mainly consequentialist ethics to justify deception and non-transparency for the greater good, without considering consequences like the ones discussed here about trust and cooperation. Would it be worth EAO attempting to prevent future deception by promoting the idea that we should be honest and transparent in our communications?
I agree with this. One frame is that the marginal move towards ‘effective marketing tactics’ is also capturing marginal attendees. This seems like it could be tested against data in the long term: attendees at a conference vs active in the community X months later.
edit: I accidentally retracted this without meaning to. The comment has been reposted below.
Hey Kit. I was in charge of marketing for EAG this year so I can explain what we did and why. I’m very sorry you felt like we were being deceptive. On my end, there’s a difficult line to walk between using the copy that best accomplishes our goals and using copy that is maximally clear. Feedback on whether we’ve made the right calls will be helpful for better calibration in the future.
For some context here, when a person is nominated we put them into an automated Mailchimp flow. If the person applied or clicked a link in an email they were removed from the automation. If they did neither of these we sent them 3 emails asking them to apply. The email you’re referring to was the last of three emails.
Julia and I actually thought quite a bit about the message and concluded that it wasn’t deceptive for a few reasons: 1) The from line included “via EAG” 2) the email address the message came from was “hello@eaglobal.org.” 3) The email itself seems to be clearly from us and not the nominator. For example, the first paragraph is “If you want to use the nomination [fname] [lname] gave you for Effective altruism Global 2016 you need to do it by the end of today.” 4) The email copy contains an unsubscribe link. 5) The email copy contains CEA’s name and address.
I’m interested in whether you think it’s deceptive given this information.
Over the course of marketing EAG we found that deadlines were effective in getting people to apply for EA Global. Yet, it’s also very important for us that a large percentage of people apply before we have to do things like order shirts, send the headcount to the caterer, etc.
To try to get some of the power of deadlines without encouraging a ton of last-minute applications, we set rolling application review deadlines. For most applications, we promised a two-week turnaround to hear back. The rolling application deadlines were dates where we promised a faster turnaround time if you applied by a particular date.
After reviewing our copy I think we were insufficiently clear about this in some of our emails. I’ll try to be more vigilant in the future. However, I think plan itself is not deceptive if properly communicated. Interested in what people think.
Unless you’re referring to something different than I think you are, this was true. I went through our attendee database and found everyone that hadn’t yet claimed a ticket. I then put these folks into some email automation for follow-up on whether they planned to claim their ticket.
I agree. These emails were a mistake. They were written by an intern and didn’t go through the proper review channels before sending them out. We instituted some policies after this happened to make the review process more clear, but don’t have enough data to know if that will solve the problem. It’s something we’ll be working to address in the future.
Hi Kerry! Congratulations again for the exceptional conference, and thanks for adding detail.
Updates I’ve made:
while in my tiny sample of 13 the emails with ‘from’ names like ‘Kit Surname via EAG’ worked out badly, it looks like you produced the most reasonable emails of that form possible without the benefit of hindsight. In answer to your question, I call this dishonest primarily because it gives the appearance of endorsement of content which I do not endorse. I would still not do this.
the deadlines at first appeared to be mainly to generate haste, but some or all had operational function. My blanket terming ‘fake deadlines’ was therefore wrong.
aside from ‘we trust Kit’s judgement’, I see that most/all other statements made in the campaign were true in a technical sense. However, I maintain that this is insufficient. ‘I was looking through our attendee database’ is a great example, precisely because the whole message implies specificity to the recipient, while it appears that the looking could have been replaced by a single filter for people who hadn’t bought tickets. Likewise for ‘ideal participant’. At the very least, I’d bin these along with the “you’re a cool person, come to EA” emails Michael mentioned.
Additional arguments against my position:
if CEA has standards substantially above average for its reference class, people might still not trust EAs to the extent I would like
maybe we don’t particularly need highly involved EAs to trust each other more, and this kind of marketing won’t materially affect what less involved people think.
I had also suspected that my concerns put me in a niche group which holds a small proportion of total relevance. I have updated away from this suspicion because the ratio of people who at the present time register a desire for greater honesty (17-27, probably nearer 17) to those who register no concern (3-5) is much higher than I had anticipated, and I suspect that forum participants are a highly relevant class for cooperation considerations.
To the other 16+ of those 17+ people: if my views are not representative of yours, it could be valuable for you to say so.
Really appreciate the response here, Kerry. Adding some extra feedback for calibration. Apologies in advance that I’m realizing I’m struggling to balance the strength of my opinion with my knowledge that this was well-intended. Just to be clear, my views only here, not my employer’s.
I didn’t end up nominating anybody because I’d rather reach out to people myself. The “via EAG” thing makes me really relieved that I made this choice and will prevent me from nominating people in the future. I’m actually a bit surprised at the strength of my reaction but this would’ve felt like a major violation to me. I really dislike the idea of feeling accountable for words that I didn’t endorse. Just for example, I could plausibly have invited work contacts who I’m not super close with and whom I would be very sensitive to being perceived as spamming.
After your explanation the practice still does seem (very) deceptive to me. At the very least, I’d expect a lot of people to click on the email because they think it’s coming from me and then to realize that it came from someone else. If I received this email, I’m sure I’d eventually figure out it wasn’t from the person in the “from” line but I’d be confused for a bit and might assume that they approved it even if they didn’t write it.
Moreover, if I wanted to not only nominate someone but also send them an email advising them to attend, I could easily do so. Some people may even have done that so their nominees would have felt like they received multiple unsolicited pings from the same person. I know it would have a lower yield but I feel like EAG should have emailed [Firstname at Lastname] and asked them to ping their nominee instead of spoofing their identity in the “from” line and taking this decision out of their hands.
I’d acknowledge that most of the other practices on this thread seem like basically standard marketing techniques. They seem off-putting according to my personal taste and I’d guess they’re counter-productive but because they’re so standard it also seems likely that I’m just being biased against them because I find marketing distasteful. I want to make clear that I’d put the “via EAG” thing in another category—substantially worse than I’d expect from a typical sales email.
Lower priority stuff:
Deadlines I don’t have a problem with rolling deadlines if it’s clear that’s what they are. I didn’t pay a ton of attention to this so I don’t have a strong take. It did seem like discounts went up as it got closer to the actual date and I think that did feel a bit like taking advantage of the people who helped out by signing up early.
Looking through the attendee database This language feels off-putting and slightly deceptive to me. As Kit says, it’s intended to make it sound like you were thinking of that specific person when it wasn’t the case. Unlike the “via EAG,” I think this practice is basically standard but I still really dislike it. Kit’s comment that “my vanity fooled me for a solid few seconds, by the way!” strikes me as a really good reason to discontinue this practice. I think it’s a bad experience and kind of embarrassing to feel like you’re getting a personal compliment and then realize it’s a form email.
I feel similarly about some other language Kit mentioned.
[Edited for some corrections and conciseness.]
I was unfortunately unable to go to EAG this year due to work commitments. By all accounts I missed out—additionally, I had less exposure to the EAG marketing than others—I mostly ignored the promotional material as I knew I was unavailable.
I confess I find these practices pretty shady, and I am unpleasantly surprised that EAG made what I view to be a fairly large error of judgement on appropriate marketing tactics. (I am pleasantly unsurprised in the straightforward and open manner with which this criticism has been received). On the issues raised by Kit above.
If I recommended (e.g.) Kit to EAG and he doesn’t reply a couple of times, he gets an email ‘from Greg via EAG’, despite: 1) I’m not sending it, 2) none of the content is written by me, 3) I’m not asked whether I consent to this message being sent ‘on my behalf’, and 4) I’m not told it happens unless the recipient gets back to me.
Although uncharitable, this looks like ventriloquism or arrogation—and such an impression may well occur in cases where do not read the body of the message. Perhaps the most accurate account is: ‘Greg’s social tie with Kit is leveraged (without Greg’s knowledge and consent) in a misleading subject line to get Kit to click on the last ditch sales pitch’, which I think should be avoided.
Especially given in this case Kit is receiving emails he did not solicit on Greg’s say so. I might think Kit might value knowing about the opportunity, but I might also have a sufficiently high view of his powers of judgement that he can decide after the first email whether he’s interested or not. When he gets a third such email, ostensibly with my approbation and/or involvement, he might start feeling irritation towards me. If I really want Kit to attend EAG so much I’d send him multiple emails, I can send these myself; if instead I think the costs of ‘pushing it’ to whatever social tie we have outweigh the increased likelihood of another email prodding him to EAG, I definitely do not want it being done ostensibly ‘on my behalf’ without asking—and especially without telling—me. (The same concerns apply to my implicit endorsement of whatever this email actually says).
[This paragraph is mistaken, and remains just so people can follow the thread of discussion] The worry with ‘rolling deadlines’ is if the deadline isn’t really a deadline for those in the earlier waves. The threat of missing out is scares them to commit early to help you, but after the ‘sham deadline’ passes, the mask drops, and you are happy for them to confirm etc. Although often just akrasia or poor organisation, people may have good reason for waiting if they are weighing up other ways to spend their time, and there’s an (admittedly remote) risk getting them to commit to tickets earlier than they need to deprives them of other opportunities. I confess I’m still not entirely clear what the rolling deadlines entailed, so please disregard if it is inapposite to what EAG’s deadlines represented.
I think the ‘looking through the attendee database’ also sails too close to the wind. I think the impression that evokes is something like, “I was checking our list of attendees, and I suddenly realised you hadn’t got a ticket (!) Given your high status and reputation, I realised your non-attendance would be a great shame, so I thought I’d take the trouble to reach our personally”. What actually happened, I assume, is the list of non-ticket buying attendees are pulled via a query on 1-2 Booleans from the database, a stock email is constructed, and a mail merge is performed.
These less than wholly candid approaches weren’t necessary: one could have sent a final reminder without anyone’s name—if one thought social proof was really key, one could have asked the nominators if they were happy for this email to be sent, or prodded the nominators to reach out to the nominees, module suggestions or even an email template. One could genuinely pre-commit to treating early deadlines as deadlines and making this clear, or simply urge people to sign up earlier to help the logistics. “Our records show” or similar is strictly more accurate than “I was looking through our database”.
I agree there is likely a trade off between candour and efficacy: the alternatives are probably less persuasive, entail more overhead, or both. I think this should be unsurprising on reflection—to whatever degree marketing is subterfuge, or trying to encourage as much ‘buying’ as possible, good marketing strategies can be hindered by frank honesty of the objective merits. Yet I aver one should take candour as all-but-lexically prior to efficacy concerns, as this is much consonant with EA norms (whatever exactly they are).
It is a common refrain to object to overblown empirical claims about (e.g.) how many lives you can save for a dollar, and to insist it is important to see how the world really works to understand how to best improve it. I think the sample principles should apply to our interactions with one another: groups shouldn’t ‘oversell’ their impact, and we should not mislead other EAs into our own designs. We should counter-signal many marketing gimmicks in the same way we (try to) countersignal shoddy empirical work.
There is both a commons problem and an increasingly common problem. The costs of increasing marketing and other behaviours (one of the other commonly remarked upon is how frequently EAG posts were shared to all EA related fb groups) are external to the group itself, who are likely much more sensitive to their own efficacy. They will have a skewed impression of the true exchange between these goods: I got the impression—correct me if I’m wrong—that EAG was at times struggling to secure the anticipated attendance, and in such situations high-handed and often unobserved restraint are unappealing. There have also been deeply regrettable behaviours of a particular EA org which will likely be described on this forum soon. Although I stress these are far, far more egregious, they are not a million miles away from stuff mentioned by Kit above.
Relying on all officers for EA orgs to have the resilience of Penelope refuting endless suitors in fealty to the ideal of extremely honest communication is perhaps utopian. Inculcating a general norm across the community to view this stuff poorly may work better. I’d suggest that marketing techniques mentioned by Kit are not used in future by anyone (plus maybe other behaviours—Kit mentioned these were ‘highlights’). I would also recommend caution and circumspection before adopting anything that even treads into the penumbra of the duplicitous—Caesar’s wife principles would be good to internalize. I hope to encourage the wider EA ecosystem to uphold an ethos along these lines, and robustly challenge mistakes—as, happily, Kit exemplified above.
Agree with most of what you said here. But I had a different interpretation of the facts with respect to the “via EAG” issue than you did.
Your impression is that:
My impression is that “Greg Lewis (via EAG)” would appear in the “from” line. (In the way that email clients often replace the sender’s email address with their name.
If I understand correctly then the practice strikes me as much more likely to deceive a recipient.
@Kerry_Vaughan:
It’d be helpful if you could clear this up. If I was confused and you actually just put “Greetings from FirstName LastName” in the subject line or some such, I’d have a substantially weaker reaction.
The third email has the sender name of “[fname] [lname] via EAG” but with hello@eaglobal.org as the email address.
Thanks. That’s what I thought.
Thanks for the correction, Howie and Kerry—I’ll edit my original comment.
Hey Greg,
Thanks for the note. I’ve responded to some aspects of it below.
I’ve updated away from sending messages of this type in the future. But, I do think you’re representing the decision as a clear violation whereas I think it’s less clear.
I think we disagree on two things: 1) does the third email cause people to think that you send it and 2) what did people consent to when filling out the nomination form.
With regards to 1), I don’t think there’s any realistic risk that nominees walked away from the third email thinking that the nominator sent it. They might wonder whether this is the case initially, but I think it’s pretty clear that the email was a template send by EA Global.
With regards to 2)
I think the relevant question seems to be what nominators expected would happen after entering someone’s contact information.
I think it’s reasonable to expect that we wouldn’t attempt to cause the recipient to think the email came directly from you. It would be a clear breach if the email address, name, and copy all appeared to be sent directly from you. However, I also think it would be unreasonable to expect that we wouldn’t mention you at all in the email. For example, if the body of the email said that [fname] [lname] nominated you, that seems uncontroversial to me.
The relevant question is where in between those extremes is OK and where is not OK. My current sense is that email body mentions are OK, and subject line mentions are probably OK, but other tactics are not OK unless we explicitly asked for consent before doing it. Do you have a different take?
This worry doesn’t seem to apply to the expedited application review version of the rolling deadline. That is, if the deadline is one to apply and hear back quick then attendees can simply not apply for that deadline if they need additional information.
Does this claim entail that one should always choose candor over efficacy? If so, that would seem to me to be a very difficult claim to justify.
Kerry, many thanks for your reply. On the matters you remark upon:
The email ‘from X via EAG’
Re. the risk to ‘mistaken identity of sender’, I think a lot of this depends on whether people read the message body or not. I doubt I’m the only person who deletes emails based on the top-line of subject, sender, etc. Perhaps savvier people would realize the ‘X via Y (and with Y’s email address)’ always means a sort of mass-mailing approach that X didn’t have a huge amount to do with, but I doubt that applies to everyone. It doesn’t seem crazy to interpret this as ‘Y is sending this email from X to me’ before they read the message body. If they just delete the email before doing so, it seems likely they will maintain this impression.
Re. Nominator consent, I agree that if you recommend X to ‘pitch’ at Y, I don’t think one can be too aghast to find X uses your recommendation as part of their pitch—I agree mentioning it in the message body or the subject is fine, and also agree the other approach you say is inappropriate is inappropriate.
I still think what actually happened falls on the wrong side of the line. The problem is not so much ‘where do you use X’s name in the pitch’, but the manner of its use. If I nominate Kit to go to EAG, it just isn’t the case that the pitch for EAG is sent ‘from me via EAG’ - especially in the present case where I have no hand in the content of the pitch email. It is not being sent from me in any sense. I think uses along these lines should demand explicit consent.
Rolling deadlines
Sounds fine to me. Forgive my misunderstanding.
Candour/efficacy trade-off
Not always (hence the ‘all but’ lexical priority) but something like, ‘barring very exceptional circumstances’. I think having this sort of high bar is justifiable on grounds of community norms and overall brand equity: judging from Kit’s remarks, this behaviour has considerably damaged his trust in CEA; judging by the number of upvotes, he may not be the only one.
Naturally, one may worry about how representative this is of overall opinion. What happened ex post is not always the best steer on what was wise ex ante, but if you know how many extra attendees were marshalled by this sort of email, that could be helpful information.
I offer an analogy to one of my interests—mental health law. In the UK (and many places elsewhere) there is a huge emphasis on ‘the least restrictive option’. Out of the variety of coercive actions that might be appropriate in for a person with mental health issues, one should use whatever is the least restrictive and has the least impact on personal freedom. This is pretty much lexically prior to other issues.
I am sure there are many occasions where the least restrictive option is not the best. One may be justifiably confident that the least restrictive option is unlikely to succeed, and early escalation to a more restrictive option would give better outcomes. One may also think that a less restrictive option may be acceptable, yet give chronically worse outcomes compared to a more restrictive one. Yet I think the law is right to uphold this principle even conditioned on consequentialism which sees no non-instrumental value in personal freedom: the risks (realized many times historically) about the overuse and misuse of psychiatric detention and coercion recommend considerable circumspection and reluctance in their present application, and the costs of a rule obliging the least restrictive option are justified by the reduction in risk of similar incidents happening prospectively.
There are going to be occasions more generally in EA where candour is inappropriate (e.g. information hazards). Yet when it comes to marketing and other aspects of intra-EA communication, I think there should be a similar drive towards picking the ‘most candid option’: that, out of the communication content and approaches available, one should select the one that has the lowest chance of misleading or misunderstanding. Although I am sure the alternatives I suggested above would have proven less effective marketing, I am similarly sure the benefits of better preserving commons of mutual trust and a brand equity of extremely high epistemic standards is outweigh them.
My read is that this is correct, with the additional caveat that it depends on the wording of the mentions in the body and the subject line. Saying “X nominated you” is fine; implying something stronger probably wouldn’t be.
Hey HowieL and Kit,
I find it difficult to think about this issue in a principled way. Not using standard marketing tactics is not costless. We used the language we used because it was the most effective. Using different language would have caused a decrease in EAG attendees and a decrease in the total value of the conference.
For the sake of argument, let’s assume that some kinds of off-putting language are more effective at getting people to attend. How would you model the tradeoff between generating extra value at EA Global on the one hand and the harm of off-putting language on the other?
Suppose the off-putting but more effective language causes an additional 100 people to attend EA Global. Suppose also that in expectation a marginal EAG attendee is worth $4,200 in donations to effective charities. Would you be willing to use off-putting but more effective language if it generated $420,000 in donations to effective charities? If not, is there a different number where you’d be willing to switch?
This discussion is really valuable by the way and I appreciate the time you’ve both put into it.
My take is that there’s a trade-off here between being most effective for short-term value (getting more attendees at EA global) and most effective building a powerful and supported long-term brand. We have better data on what’s effective for the short-term value, because the feedback loops are tighter. This could mean that it should get more weight (because we actually know what we’re doing), but there’s a danger that it means we swing too far towards it. The off-putting messaging could in some low-grade ways lower a whole lot of people’s opinions towards EA/CEA/EAG. For a movement that trades so much on intellectual leadership this worries me.
Here’s one guesstimate. Take with a vast amount of salt, and I don’t even really believe the framework I’m using, I just want to show how you might get going with these comparisons: Damage to brand = 0.1% of brand value; brand value = ~$1B; so would be willing to switch if generating >$1M [Also of course it’s not binary. We can probably look for compromise solutions which get a lot of the marketing value and a lot of the long-term brand value]
There’s even a case that at the margin we should prefer more consolidation over more growth (of EA community generally and EAG specifically), in which case it would be good to have emails which are differentially attractive to people (like Howie and Kit) who are/could become high-value community members, rather than differentially off-putting to them.
I think a stronger argument can be made in favour of the chosen marketing methods. It would probably conclude with something like ‘the huge value of a small number of extra links formed between otherwise-disjoint groups outweighed the minor weakening of cooperation standards across the community’.
Owen’s comment shows that the numbers can be big on the other side too, but valuing brands is a notoriously hard problem. In the hope that people refer back to this discussion when considering future strategies, here is an explicit estimate of one component of the value of avoiding minor harm to trust, for this specific case. It works by assuming that anyone put off from CEA simply shifts collaboration from one organisation to another, causing efficiency loss from wasting comparative advantages, not total loss. It also recognises that I made an unusually large update, and the average will be much smaller. Bracketed items are multiplied together to give an italicised item in the next line.
(present value of a GWWC pledge, $73,292 x number of pledges next year, 856) x size of CEA compared to GWWC proxied by headcount, 2.45 x (my unusually large update to engagement with CEA, 30% x perceived relative strength of other affected people’s reactions, 17.5%) x relative advantage of CEA over competition, 17% x proportion of people with negative reactions, 48%
= (value realised by GWWC next year, $62,737,952 x size of CEA compared to GWWC proxied by headcount, 2.45) x (average affected person’s shift from CEA to elsewhere, 5.25% x relative advantage of CEA over competition, 17%) x proportion of people with negative reactions, 48%
= value realised by CEA next year, $153,707,982 x (inefficiency from one affected person’s shift, 0.88% x proportion of people with negative reactions, 48%)
= (value realised by CEA next year, $153,707,982 x proportion of CEA value lost 0.42%)
= value of one year of CEA minor reputation preservation, $638,849
This model does not incorporate the effects EAG marketing can have on other EA organisations’ reputations (I suspect large), the value of not putting people off the movement entirely (unsure), or the effort required to clean up one’s reputation in the unlikely case that lasting harm is incurred (low in expectation?) To handle overoptimisation, I have tried to keep inputs conservative rather than discounting explicitly.
My guess after public and private discussion is that the approach which captures the most total value would be something like aggressive marketing (including pushing known EAs hard to tell their friends, slightly-more-than-comfortable numbers of chaser emails to applicants, and focussing almost entirely on the positives of attending) while avoiding anyone feeling deliberately misled. Obviously CEA is better placed to make this call, and I hope the broad discussion will help guide future decisions.
I realized I never indicated what I thought after the discussion. I now endorse the position Kit suggests:
Thank for the very valuable discussion!
Thanks for this. Very helpful.
I’m not sure if this discussion has changed your view on using deceptive marketing for EA Global, but if it has, what do you plan to do to avoid it happening in future work by EA Outreach?
Also, it’s easy for EAs with mainly consequentialist ethics to justify deception and non-transparency for the greater good, without considering consequences like the ones discussed here about trust and cooperation. Would it be worth EAO attempting to prevent future deception by promoting the idea that we should be honest and transparent in our communications?
I agree with this. One frame is that the marginal move towards ‘effective marketing tactics’ is also capturing marginal attendees. This seems like it could be tested against data in the long term: attendees at a conference vs active in the community X months later.