Introducing Envision: A new EA-Aligned Organization
The difference between human extinction and a wide range of positive outcomes is dependent upon how technology is developed and implemented. The EA community knows this better than anyone.
But a sustainable process to maximize the positive contributions of technology to human well-being while minimizing the probability of existential and global catastrophic risks requires participation by domain leaders in tech development, policy, academia, and business. This is currently not the case.
One of the ways to remedy this is by targeting future domain leaders. That’s why I’m excited to announce a new EA-aligned organization, Envision, with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia, and business by intervening at the college level. The goal is two-fold: convince future domain leaders that technology will play a pivotal role in humanity’s future, and instill in them the careful consideration of technological safety as a core value.
This post will:
Outline Envision’s current status
Justify the claims made above
Outline Envision’s goals and strategy
Define metrics of success and measurement mechanisms
Review failure modes
Explain how EAs can potentially be involved
Current status
As a general outline, Envision will be a global network of student chapters with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia and in relevant business leaders.
Envision is currently a 6-month old student group at Princeton, with 15 officers and 91 members, which will hopefully significantly increase with fall recruiting. Envision’s website, although still under construction, provides more information about past events.
Students at MIT, Stanford, Cambridge, and UPenn have tentatively expressed interest in opening a chapter.
The locus of Envision is our Conference, Dec 2-4, which will serve as a focal point for recruiting and kickstart many of the processes and strategies I am about to describe.
Our current official partners include MIRI, Future of Life Institute, Jaan Tallinn, and Sam Altman among others. Confirmed speakers at our conference include Andrew Critch from MIRI, Robin Hanson, Anders Sandberg from FHI, and Ruth Chadwick from the Centre for Economic and Social Aspects of Genomics. MIRI will most likely host a workshop during the conference.
Justification of claims
This section will expand upon the claims made in the introduction.
The difference between human extinction and a wide range of positive outcomes is dependent upon how technology is developed and implemented.
Existential risk is inherent in artificial intelligence, synthetic biology, and nanotechnology, as argued by Nick Bostrom among others.
Even given that existential risk is avoided, these technologies, in addition to genetic engineering, space technology, fusion, virtual and augmented reality, and neuro-engineering, have a wide range of possible impacts, from the catastrophic to unprecedented advances in human well-being and capabilities.
Given the rapid pace of technological development,
Other technologies are likely to arise which will be crucially important to humanity’s future, and
The impacts of the development of these technologies is likely to be shaped by actions in the short to medium-term and will be felt within a century.
But a sustainable process to minimize the probability of existential and global catastrophic risks while maximizing the positive contributions of technology to human well-being requires participation by domain leaders in tech development, policy, academia, and business.
The argument is often made that AI is more safely developed by a small group of people, without significant domain leadership involvement. However, the key word is sustainable: the future is unpredictable and new technologies and issues will crop up. It is not sustainable to have to convince the relevant players and assemble a team anew every time. As Nick Beckstead argues, broader and more general interventions, including in education, are important to shaping the far future.
Furthermore, ‘participation’ is different from ‘exposure’ and is assumed to be positive. A misinformed elite meddling in AI safety is not a desirable outcome; an elite participating in safe technological development is a positive outcome. Participation is far harder to achieve than exposure, but because of how Envision is set up, is achievable.
Technology does not exist in a vacuum. How it is implemented is as important as how it is developed. The actions taken by governments, academic institutions, and businesses both influence technological development and determine implementation.
This is currently not the case.
Although a small number of domain leaders in tech development and policy are now Effective Altruists and/or are engaged with AI and other technologies in a safety-conscious manner, this remains the exception.
The way to remedy this is by targeting future domain leaders during university.
College strikes a balance between flexibility and accessibility on the one hand, and intellectual maturity and indentifiability on the other.
Flexibility: Still sufficiently early that beliefs are in flux as students are exposed to new ways of viewing the world, and the most important career choices still lie in the future. Many smart, socially aware students default to tech, consulting, and finance for lack of prestigious and effective alternatives, which can be more easily avoided pre- than post-decision.
Intellectual maturity: However, it is late enough such that existential risk, exponential growth, the technical details of technology, and other important concepts can be grasped.
Accessibility: Future domain leaders are still easily accessible at low cost.
Identifiability: However, it is late enough that future domain leaders are considerably more readily identifiable and can be specifically reached out to.
One major disadvantage is the more extended time frame before interventions reap benefits, as it takes time before current students reach the top positions within their domain. This is valid, but there are reasons to think a college intervention now will almost certainly pay off.
Sub-elites exert significant influence on the culture of domain leaders, so effects will be realized considerably before the future domain leaders that are exposed to Envision assume full control. The impact of the intervention begins increasing from the moment students enter the workforce and is thus not as long-term as it appears.
The massive impact a successful university-level intervention would have in causing a significant shift in mindsets is likely to justify the longer time until full realization of impact.
Intervening with future domain leaders also reaps longer rewards: current domain leaders have considerably shorter time left in power.
Identifiability is an issue; current domain leaders can be far more easily identified than future elites. There will certainly be a lower success rate of identification, but what matters is the absolute number of future domain leaders affected, not the percentage of those affected who become domain leaders. It is thus theoretically possible for all future leaders to be influenced despite a low success rate of identification if enough students are reached.
There currently does not exist an organization that does what Envision does.
There is no mechanism by which college students are exposed in a single context to a balanced view of both the potential and the dangers of rapidly developing disruptive technologies.
X-Risk organizations and student groups do not appeal to most future domain leaders, who are more excited by the potential of technology to improve the world.
Tech-specific and issue-specific groups exist, but none are holistically oriented to take into account the bigger picture.
There is no mechanism for pulling in future policy makers and business leaders into the same space and mindset.
Although online resources exist, they are not easy to find, and the message is not brought to most future domain leaders. There is thus no easy way to access the information without an existing initial interest or recommendation.
Envision – The Strategy
As a summary, Envision will be a network of student chapters at top unis worldwide with the goal of imbuing a forward-looking but safety-conscious mindset towards technology in future leaders in tech development, policy, academia and in relevant business leaders.
This section provides an overview of Envision’s strategy for accomplishing this.
Goals can be broken down into four separate categories:
Attract and retain future leaders across the domains relevant to future technology development.
Convince future leaders that technology will play a pivotal role in humanity’s future and integrate technology safety into their mindset.
Provide the tools to contribute towards the positive development of technology.
Providing a pipeline into EA.
Strategies for accomplishing goals
For attracting and retaining future leaders
Leaders must first be identified. For this, we are developing a predictive model identifying key variables current domain leaders had in common during university.
Leaders must then be attracted. There seem to be three factors that have widespread appeal:
Positive messaging emphasizing the potential of technology while integrating concern for safety. Envision’s official mission is inspiring students to pioneer a brighter future through the responsible and innovative use of rapidly developing disruptive technologies, and a lot of our events focus on the positive applications of technology.
Domain leadership in the fields they’re interested in. To appeal to this, our strategies are:
-
Partnering with domain-leading organizations and individuals. Our current list includes MIRI, FLI, Sam Altman, and Jaan Tallinn, and we are in the process of expansion.
Interfacing with firms on the cutting edge of technology, through inviting them to demo their technologies and visiting their offices and labs.
Partnering at each university with established domain-leading student groups. At Princeton, our founding location, we are partnered with Entrepreneurship Club.
Inviting domain-leading speakers.
Developing prestige to attract the most talented students. This includes:
emphasizing selectivity in officer and fellow selection, although membership will be open to all
partnering with prestigious organizations
including a competitive element
ensuring sleek design and streamlined operations
targeted reach-out.
Additionally, we will maintain independence from EA. Although there will be links with EA, partnerships with EA orgs, and many EA members, Envision will remain officially non-affiliated to maintain its appeal to those who would be off-put by EA and maintain the differentiation of its message.
The above strategy is highly subject to change as we learn more about what most attracts future leaders and we will continuously update our strategies in response to new information.
Convince future leaders that technology will play a pivotal role in humanity’s future and integrating technology safety into their mindset.
It’s important to integrate both these aspects and present safety as a standard component of technological development, not a stand-alone add-on as it is often portrayed or interpreted.
This is best accomplished by emphasizing technology’s vast potential while consistently addressing safety concerns. A dichotomy between positive applications and negative risks is to be avoided.
Strategies:
Provide exposure to companies and labs on the cutting edge of rapidly developing disruptive technologies. This makes clear the speed with which technological development is occurring and how close we already are to revolutionary breakthroughs, and the concrete practical applications and implications.
Integrate safety as a principle into all events and issues, even for small concerns. This will establish it as an ingrained attitude. To do this, we will ask all speakers invited, firms visited, and other relevant parties to address how to mitigate the potential negative outcomes of their technology, and we will address safety in discussions.
Host specific safety events in the form of talks by safety-related experts, including in:
Tech development, ie companies such as DeepMind
Technical safety, such as MIRI
Safety considerations, such as FHI and FLI
Curation of online resources, including papers such as Nick Bostrom’s, major news updates, and overviews of relevant technologies.
Discussion. This is a crucial element to complement the outside expert views with the inside, socially acceptable peer views. Envision will hopefully have many Effective Altruists, so discussion will be one of the most potent tools for integrating a safety mindset.
It is important to note Envision focuses on technological developments and issues that are not commonly known, which is generally correlated with the longer-term. For example, the ethics of self-driving cars are already widely discussed and thus do not fall within the scope of Envision.
For providing the tools to contribute to the safe development of technology
Envision Entrepreneurship: A competition for the most innovative application of a rapidly developing disruptive technology for large-scale impact, with prize money. With time, partnering with organizations to provide a more direct route for the winning ideas to be implemented. As with everything, safety considerations will be integrated. Being piloted at the conference.
Interfacing with organizations that are working on innovative tech development. Includes tech firms and start-ups, VC firms, government agencies, research labs, think tanks, and tech safety research organizations. Exact programming highly dependent on organizations. Those deemed to be negatively contributing to human well-being due to an active disregard for safety considerations will be avoided.
The purpose of this interfacing is both to provide the means for working on safe technological development and smoothing the path towards domain leadership.
Will include career-oriented interaction with relevant organizations, including career fairs and in select cases presentations on how to work towards the positive development of technology in different careers.
For providing a talent pipeline into EA
This is quite straightforward: sufficiently involved Envision members will be exposed to EA. We’re collaborating with EA Build, the body coordinating EA Student Chapters, to accomplish this, and will coordinate with individual student chapters at the universities where we open chapters.
However, this exposure will be light, in order to retain those future leaders who are not interested in EA – this is a crucial part of Envision’s value-add.
Planned structure
Envision will be a global network of student chapters initially headquartered at Princeton University. Governance may become more decentralized and distributed across chapters with time, depending upon developments.
There will be an annual main Envision conference, and several smaller technology- and issue-specific conferences hosted by different chapters.
Envision-hosted events will generally be open to the entire student body of the university in question. There will be officers, who run the chapter, and fellows, who are selected by invitation from event attendees and gain several perks such as priority of attendance for events limited in size, and access to more intimate fellow-only discussions with professors and visiting experts.
The exact structure of Envision chapters, while open to change and additions, initially includes:
Understand: guest speaker series and workshops with university and outside speakers and organizations.
Interface: Trips to nearby companies and labs developing or working on implementing rapidly developing disruptive technologies, and attendance of nearby relevant conferences.
Envision Entrepreneurship: Smaller local versions of the conference event. Winners will compete at the main Envision conference or at a separate Envision Entrepreneurship event.
Discuss: Discussions among members, sometimes including faculty. There will be a more selective subset of Fellows who will discuss more frequently and have more intimate discussions with professors and visiting experts.
Success metrics and measurement
These are incomplete. Many measurements and success criteria are not well defined, several are subject to Goodhart’s Law and are not optimized to measure counter-factual impact, and the details of implementation are missing. Feedback and advice for addressing these issues is welcome.
|
Goal |
Metric |
Measurement |
Success criterion |
|
Attracting future leaders |
Success in attraction |
Quality of applicants |
Qualitative—compare to criterion of future leader |
|
Quantity of applicants – Chapters |
10% of freshman class |
||
|
Quantity of applicants—Conference |
700 applications |
||
|
Quality of attractiveness |
Potential applicants not applying |
No talented, potentially interested people not applying. A key issue is how to measure this. |
|
|
Successful screening process |
Fellows alumni turn into future leaders |
50% become leaders |
|
|
Accurate selection |
Every year after acceptance, fellows are re-evaluated whether they still match future leader criterion. 75% yearly success rate. |
||
|
Convince future leaders that technology will play a pivotal role in humanity’s future. |
Successful events |
Pre- and post-event survey—Conference |
Significant update towards understanding power of technology in 85% of cases |
|
Post-event surveys for each event – Chapters |
70% significant updates to deem an event successful |
||
|
Overall successful mindset change |
End-of-year survey, more detailed informal check-in for 50% of most involved members by non-central members (to avoid bias) |
Significant positive updates, tangible excitement in 80% of cases |
|
|
Integrating safety into future leaders’ mindset |
Theoretical updates |
Pre and post conference survey |
70% Significant update towards concern for safety |
|
Post safety event surveys |
30% major update, 60% minor update to deem event successful |
||
|
Observed changes |
Alumni concerned with safety |
60% of alumni leaders demonstrate concern |
|
|
Engagement with safety issues |
End-of-year survey; 50% show increased engagement and concern |
||
|
Providing the tools to accomplish above |
Career changes |
End of year survey question on career change |
70% update, of which 45% major update (positive) |
|
Observed changes |
Qualitative; how to measure? |
||
|
Career advancement |
Opportunities through Envision |
15% of career-related event attendees find opportunity. Based on post event survey, several months after. |
|
|
Partner meetings |
2 partnerships a year formed. |
||
|
Provide a talent pipeline |
Envision alumni working for the organizations in question who otherwise would not have |
At least 5 researchers within 10 years who updated through Envision. Based on self-reporting. |
Key terms to define
Positive update
Safety update
Leader
Future leader criteria
Positive career change
Partnerships
Minor versus major update
Metrics to keep track of
Number of applicants
Alumni paths and tech safety concerns
Pre and post conference views on tech and tech safety
Post event updates chapters
End of year major updates and tech safety views—number asked and answers (non-anonymous)
Pre and post conference career plans
Observed career changes
Opportunities reached through career-related events
Partnerships formed through Envision
Failure Modes
Envision faces many challenges, of which I will outline some of the most salient.
Inability to identify future leaders
Our predictive model is designed to counteract this, but success is far from guaranteed. Some potential causes of failure:
Goodhart’s Law.
Identifying only the visible manifestations, not the underlying traits.
Finding there are no common identifiable traits.
Inability to attract future leaders
Once identified, being unable to attract future leaders. Potential causes:
Factors that attract leaders vary so widely between domains it is not possible to appeal to all.
Path dependence—mistakes early on and the attracting of the wrong initial set of students permanently places Envision on a path that cannot lead to success. This also applies to individual chapters; if initially badly set up, it would be close to impossible to gain traction at the university in question.
It is not possible to provide the factors that would attract future domain leaders within the boundaries of what is possible given Envision’s goals.
Failure to instill a safety mindset in future domain leaders
This is quite a broad failure mode which can result from several different causes:
Resources are not well-provided. This could be due to speakers and firms refusing to address, or inadequately addressing, risks.
Resources are not accessed. For example, students may simply not show up to safety-specific events and not read resources provided.
Resources fail in instilling a safety mindset: Even if well-provided and accessed, students may refuse to update. This can be partially mitigated by presenting safety as an integrated component, that has concrete solutions.
Integration of safety leads to a competitive disadvantage, causing those who do update to have a lower likelihood of achieving domain leadership.
These are but the most likely failure modes that we are most actively addressing. There are more, and we are continuously re-evaluating to ensure we remain aware of failure modes, are not in one, and are not missing any.
How you can be involved
Envision is in its early stages. Strategy is subject to change if another route is deemed more conducive to fulfilling Envision’s goals. Critical feedback, questions, and advice on any part of the value proposition and strategy are encouraged.
If you are a student or are affiliated with a university, further involvement is welcomed. You can apply to the conference and, if you have the time, dedication, and interest, help set up a chapter, in which case you can contact me.
If you are not but think Envision is important and are willing to help, further involvement is welcomed. This is especially true if you have organization-building/management experience, in which case you can advise, or model-building skills (quantitative or qualitative), in which case you can help build our predictive model. If the model is successful, it can be widely applied within EA beyond Envision.
If you are in charge of, employed by, or affiliated with an organization or company that potentially would be interested in partnering with Envision, further involvement is welcomed. If Envision is successful, this is an opportunity to potentially have a significant impact on the future of humanity at low cost. More directly, we also have recruiting and marketing opportunities if these are relevant.
Partnership can be content-based, as in the case of our partnership with MIRI, who are providing a speaker and possibly hosting a workshop; the provision of funding (we currently have some but are far from fully funded to execute on all of our projects); or a combination.
I can be reached through Direct Message or at lrade@princeton.edu.
To summarize, in this post I:
Introduced Envision, a new EA-aligned organization, and its value proposition.
Outlined Envision’s current status.
Justified the claims made with regards to its value and effectiveness.
Outlined Envision’s strategy
Presented our success metrics and measurement mechanisms
Went through some of the most salient failure modes
Explained how EAs can potentially be involved
This is an interesting idea, with a good plan, and carries an implicit demonstration of impressive amounts of work done (particularly re. the conference). The merits of the plan are well-discussed, and I am also pleased you note some of the potential difficulties—in most real cases, reasonable considerations do not all point one way.
I offer some further considerations below. These are all critical, but only because the positive ones I conceive recapitulate ones already stated already, and better, in the OP.
1) Targeting future elites is an extremely crowded area
To a first approximation, every organisation on the planet wants to get the attention of future elites. Companies want them to become employees, ideologies want them ‘on side’, political parties want them in their own talent pipeline, and so on and so forth. Envision may not directly compete with (e.g.) MiscTechCo directly as one can both support the former and intern for the latter, but the indirect competition to get elite university undergrads to go to your event not someone elses is pretty fierce.
2) The difficulty of getting students to run the org
As Ben suggests below, you are also competing for leaders to join your org and found chapters, and this is also super-crowded: students who are able to lead or even willingly and competantly administer aspects of a student group are in extremely short supply, and there are strong incentives (i.e. CV points) prefering these people to strike out on their own project or take a subsidiary role at a prestigious or well-established student group. Playing second (or nth) fiddle to be a junior officer in satellite chapter of a new student group is likely for true-believers only. And (as ben also suggests) the rapid turnover means this challenge repeats, and also applies to keeping the org going: it is easy to overpredict longevity and success if one doesn’t account for the likelihood that ones successors are unlikely to be as invested in the project as you are. Most orgs I was involved in helping set up as a student no longer exist.
3) The predictive model will almost certainly fail to identify future leaders
I would be amazed if you find any novel signal predictive of someone becoming a future elite leader: everyone wants this sort of human capital, and so means of identifying it given even mild instrumental rationality on the part of competitor groups should be very well trodden, if only via informal metrics rather than a formal model. The outside view on a recent student run org finding an edge in human capital identification over large organisations which may include professionals specializing in this area with potentially decades of individual and corporate experience and very large resources addressed to this task does not augur well.
My hunch is that is extremely hard for reasons you do not identify. The first big issue is range restriction: basically everyone at (e.g.) Princeton is highly intelligent, conscientious, and motivated, because getting into an elite Ivy selects strongly on those traits. Within such a rarefied group discrimination is going to be very tough: although (e.g.) IQ is likely good a predictor of success in most fields, the difference between smart-for-princeton and dumb-for-princeton is unlikely to be great—the smart cohort might fare better, but the degree of scatter makes individual predictions next to worthless (maybe its 25% for smart versus 22% for dumb). Worse, most universities want to ‘equip the leaders for tomorrow’ or similar, and so are already selecting in part for leadership traits. The factors fairly orthogonal to this may be unmeasurable, or simply the vicissitudes of chance (do you happen to get on really well with the boss you interned for, do you happen to know socially the person with the complementary skillset for your start-up, etc. etc.) Even without goodhart’s law, the tails come apart for the distributions of actual performance and performance on proxy measures. I’d guess any such model constructed repeats what employers/grad schools/etc. use already: GPA, test scores, particular softs, etc.
(Obviously, validation for any model will take several years to a decade as you at need to follow up the cohort long enough to get to early-career stages where you might be able to predict future trajectory).
[Continued]
4) On the perils of obnoxious elitism
I find the idea of selecting ‘fellows’ amongst the students who get perks as a way of fostering elitism disquieting orthogonal to the difficulties I suggest above for ‘picking winners’, and I believe I have significantly greater tolerance for elitism than most. I expect significant downsides with resentment of the majority to be judged as relatively ‘un-elite’ (good luck getting them to attend your next event!), a lot of unpleasantness attendant to whatever procedure is used for selection (who decides, and how?), and it looking absurd with the power dynamic generally applicable to most universities: most student groups start off small and desperate for both committee members and attendees to their events so as not to offend the speakers they enlist. It sounds like you have cleared this bump at Princeton, but it may not prove so easy somewhere else.
I fear the strategy implied puts the cart before the horse: you don’t get to ‘be elite’ by just offering codified means of differential status, but rather by that status forming a gatekeeper to unique opportunities and experiences which the audience cannot get elsewhere. The only group at my university which could plausibly reach this bar is the Cambridge Union, which is hundreds of years old, has (by student society standards) an absolutely vast budget and infrastructure, and a reputation that extends beyond the university itself. Getting to even a fraction of this, even segmented for tech, requires building considerable reputational resources, which I expect will take a while even if things go well.
5) Extremely optimistic targets
Less important, but although the metrics appear reasonably well chosen, the targets are stratospheric. Getting 10% of princeton 2016s to apply surely won’t happen (do any other groups on campus get this level of interest?), getting 10% of another university to sign up would be flabberghasting—if application means any more than ‘join the mailing list’, I don’t think any group outside the Union gets that in Cambridge, for example. Getting 50% of the alumni to be leaders (however defined) looks impossible: beating the base rate by a OR > 1.5 would be extraordinary. Ditto changing 70% of your audiences minds with a talk, and ditto almost everything else. I know setting aggressive goals fits the gung-ho start-up ethos, but this can become meaningless if they have no prospect of success—which I believe the balance of these are (prove me wrong on this and I will start offering you money).
6) Over and under-segmentation
You are explicit in aiming to be an ‘EA-liminal’ org, attempting to avoid some of the negative connotations of the label. I agree this is probably wise (although note regret en passant that the EA brand is somewhat toxic) but I am unsure there is a sufficient niche to carve out the worldwide org you have in mind. There are usually groups that cater for ‘interested in science and technology’ group, which have generally been around for a while and thus usually have more clout. The ‘implications and existential threat of emerging technologies’ is pretty solidly in the EA ambit.
This poses a problem where there is an existing ecosystem of both groups where you are trying to enter. Cambridge has SciSoc, the triple helix (looking at science at society), biosoc, chemsoc, comsoc, a couple of student think tanks, a GWWC chapter, an 80k chapter, and a nascent x-risk group—and I am sure some further groups that escaped my knowledge. There seems an area around Envision’s remit which isn’t at least covered once, and I doubt other target unis will be much less saturated here. My guess would be adding another group with a narrowly tailored (but overlapping) focus will likely be more of a detriment in terms of dividing efforts than a dividend. EA (or EA-related) groups trying to set up chapters at universities are also pretty saturated (I count GWWC, EA groups simpliciter, SHIC, 80k. Perhaps others?)
7) Summing up
My overall view is, with qualifications, negative. The outside view on initiatives to produce a hyper-successful university org that wins significant outside prestiege and access to extra-university elites (i.e. entrepreneurs, tech companies, et c.) and then repeats the same process multiple times across elite universities is that the vast majority fail in their ambition. The plan seems to be over-optimistic in setting targets, and relies on success in several highly competitive areas (e.g. predicting elite leaders, tempting them invest their time and attention in you, ensuring momentum and resilence for the org to persist for years, getting a nucleus of highly effective leaders to start parallel groups other universities). For each of these, there is neither a clear edge in principle (and often the outside view will favour competitors with more people, more experience, more cash, more insitutional capital, etc.) nor—with no desire to ‘do down’ the considerable achievement of getting to a mid-sized student org in 6 months or arranging a conference—a sufficiently impressive track record to anticipate such the requisite (very substantial) degree of success going forward.
My hunch is there remains considerable overlap between Envision and what various EA groups are already trying to accomplish. My suggestion would be a better means of accomplishing what Envision has in mind by trying to penetrate elite student audiences with EA-esque messages about Tech whilst avoiding the brand baggage of EA is to instead work within the existing ecosystem to deliver progress in that direction, both leveraging existing resources and capital, and developing the right sort of loose collaborations. An example:
The Wiberforce Society is a student run think tank in Cambridge. It ran a day of panel discussions on the impacts of future technology, and had one planned on ‘navigating AI in the 21st century’. I signed up to help produce the paper draft, with a mix of people—some were EA-xrisky types, but not all. The panelists who would discuss our paper included a mix of people from Xrisk orgs, academia, and industry. There have been several attractive dividends: some of the coauthors have gone on to tech start-ups and discussing publically the implications of (e.g.) drone technologies, others have been invited to attend conferences, and so on and so forth. It seems very challenging for an org started in Cambridge to have emerged similarly successfully.
8) Warm wishes
I hope this persuades you to change course. Obviously, if you decide to continue along the lines suggested in the above strategy, I wish Envision every possible success, and sincerely hope my concern proves misplaced. I am fairly good at stats (although you might have access to professorial statistical firepower) and I am happy to advise re. predictive modelling etc. if you don’t already have much better people on board. :)
Thank you, Gregory. You raise excellent points. I will address them individually and then alltogether in conclusion.
1) That’s correct, we will have to compete with other student groups. So far, our message appears powerful enough to give us a significant advantage, which will help partially compensate for our lack of a track record.
We also don’t necessarily have to compete. The strategy of partnering with other successful student groups (ie Entrepreneurship Club at Princeton; similar organizations at a handful of universities have expressed excitement at partnering with us and helping set up a chapter, although of course excitement does not necessarily equal actual work) appears to be sufficient to allow us to compete at a level that we’re able to grow and sustain. Envision is in many ways a welcome addition to entrepreneurship groups so they’ve been very receptive so far to partnering and sharing resources.
2) Excellent point. To counter this, we’re focusing on building up an ecosystem of faculty advisors and partner organizations, which adds both prestige in the competition for student leaders and significantly increases the likelihood of sustainability.
I also think you underestimate the appeal of helping build a new organization, especially one working on something exciting, even if you don’t run it. However, I could be wrong on this.
Lack of investment by later leaders is certainly a problem. However, 1) investment is less important since the organization already exists so far less work is required, 2) continued involvement by a board of alumni will help keep the organization on track, and 3) with faculty advisor buy-in some of the continuity will stem from them. With this combination, a weak leader should not cause the organization to collapse.
3) A good point, and one we had not thought of in detail. A few thoughts: 1) we could just use external validation criteria, eg internships at the most competitive companies, although this is not necessarily indicative of future domain leadership. 2) Breadth is a strong solution to this; the more people we reach in absolute terms, the higher the likelihood we touch future leaders.
In light of your point, do you think it’s worth creating a predictive model at all? It would use up valuable man-power, and you’ve convinced me it would likely have limited impact.
4) This is a good point. Although it seems like having fellows could actually increase attendance at events. In any case, you’ve convinced me to defer a fellow program until at least after the conference before re-considering in light of the new evidence we’ll have gathered.
5) I disagree that the targets are ‘stratospheric’ – although they are optimistic. I also don’t see the problem with the ‘gung-ho start-up ethos’ – it gets quite a lot done. The targets are hard to achieve, and it’s very unlikely we’ll hit all of them, but we’ll try and get pretty far in the process. Failing at achieving optimistic targets but getting quite far towards them in striving is much preferable to achieving unambitious ones and sitting back in satisfaction.
Having made that philosophical objection, I do agree our targets are in some cases probably unrealistic. I’d welcome a more detailed explanation of which ones you think are unrealistic and why, as well as suggestions for more reasonable targets.
6) From my perspective, Envision fills a clear and gaping niche. ‘Interested in science and technology’ is different from ‘interested in the medium-term and long-term future of science and technology and what we can do to pioneer a better future with the tools we have and will have available.’ The differences: more action-oriented; focused on future issues that do not receive much attention on college campuses; more broadly focused on multiple technologies and how they interact; integrating technology and science with ethics, policy, and entrepreneurship.
To use reductio ad absurdum on your argument of there existing separate groups that each touch on an aspect of Envision: there have always been groups for altruistic people and groups for effective people. That does not mean a group for effectively altruistic people can add no value.
I also don’t see much overlap with EA groups – most of our members are not EA, even though many have heard of it. Keep in mind we’re targeting future leaders, in particular those who do not yet have a concern for safety or awareness of the future of technology, to help them learn about it.
A final note – I think you overestimate how many similar student groups exist. We’ve now exhaustively gone through all student organizations at over a dozen universities, and have not yet come across an organization with significant overlap that is run well to the point of making Envision unnecessary. Cambridge is among those with the most potential candidates (although we’ve also had the most interest from students there, in addition to MIT, about starting a chapter).
Finally, as a thought experiment – how many student organizations contain entrepreneurs and policy-makers not in the EA sphere and have Andrew Critch and Robin Hanson as speakers?
Summing up
There is a lot to be said here. First, to break down your second sentence:
• Hyper-successful: I’m not entirely sure what this means, but I don’t see how Envision succeeding at its goals requires it. We certainly need success as a student group, but I don’t think we need substantially more than what would traditionally qualify as success for a student group, albeit repeated several times (which is certainly harder).
• Significant outside prestige: We need prestige among students, but I don’t see why outside prestige is necessary (if I’m interpreting this correctly). It helps with getting external organizations on board, but prestige is sufficient for this, not necessary – being students excited about the future of tech and an organization with the prospect of hiring opportunities, and in the EA sphere being a student organization with the goal of promoting safety, goes a long way.
• Access to extra-university elites: This is true, but we only need very limited access. Ie a few hours on a weekend to come speak or showcase your technology at a conference that pays for your flight, setting up a recruitment booth in return for providing a (for a company) small amount of money, having your name on a student organization and occasionally speaking to excited students, etc. We’ve so far been pretty successful at getting this since it’s low-cost. To be successful, we don’t need more than this – the more the better, but the acceptable threshold of access to ensure success is quite low. In light of this, looking at some other student groups, I think the amount that fail is less than the vast majority, especially when keeping in mind we’re partnering with organizations like Entrepreneurship Club that have already been successful on most relevant metrics and can advise us, lend us their credibility, and help us with recruiting.
On repeating the same process: Here I agree with you. This is certainly one of the most difficult parts of Envision’s strategy. Even if our success is limited to Princeton, though (and I have high confidence we’ll establish at least a few additional chapters at significant universities), I think the net impact is still sufficient to justify building Envision.
With regards to your second sentence:
• Predicting elite leaders – I agree this is difficult, but as I explained in the point about this, not necessary to success. Casting a wide net at universities most likely to produce future leaders ensures high probability of impacting the correct people. Predicting future leaders would be hugely beneficial, but failure here does not invalidate the value proposition.
• Tempting them to invest their time and attention in you – They need only attend a few events and change their minds. We need officers, but this is only a tiny subset, and I think there’s sufficient message attraction to fill this. Getting people to attend events is not trivial, but certainly doable, and has been done before, including by us.
• Ensuring momentum and resilience – I agree that this is a major challenge. However, as I outlined above in the relevant point, I think building a framework will make this significantly easier.
• Getting a nucleus of highly effective leaders to start parallel groups at other universities – I completely agree that this is extremely difficult and highly competitive. But in my experience the draw of starting a new organization, even as a chapter, is quite high. And again, I think the message is powerful and will in itself attract several such highly effective leaders.
For reasons I have already elaborated, I disagree that we have no edge in principle (interpreting this as a synonym for message – correct me if I’m wrong).
Track record – I agree we don’t yet have this, but neither did any organization (especially student organization) upon founding. And the partnership with existing student organizations quite significantly mitigates this.
On your last paragraph about other means of accomplishing what Envision has in mind:
• I agree on the importance of using existing ecosystems, and I think Envision is doing what you describe. Ie Entrepreneurship Club, a conference, being a student group, running a pitch competition, leveraging existing opinions and resources rather than producing our own. I would argue what Envision is doing is leveraging the existing ecosystem more than a student think-tank producing papers would be.
• The work of the Wilberforce Society is admirable, and we will certainly seek collaboration with them given this information about their concern for AI and the future of technology. But it doesn’t seem like the type of organization to attract entrepreneurs, future business leaders, and hardcore tech developers (I could be wrong about this). And signing up to write a paper about AI suggests a pre-established interest. Running such a panel is also high-effort, and it seems like it affected a handful of people. Again, not to say anything negative about Wilberforce Society at all – just to make the point that I don’t think this is necessarily far more effective than Envision. I think they serve different goals and should both exist.
Conclusion
First of all, thank you so much again for taking the time to write out this post. You’ve convinced me of a fair amount of significant changes to our plan, and helped better clarify others.
However, you have not persuaded me to change course. I think I’ve argued quite convincingly why Envision adds value – and I welcome any additional arguments for why it does not, or disagreements with my points. For the sake of keeping this at a readable length I skimmed over some points and left the details to inference, so also feel free to ask for clarification.
Thank you for the model-building offer. Depending on whether you think the model is still worth it, we would be interested in discussing further.
As a final note, Envision is exciting. And excitement is powerful.
This is an impressive plan.
The main thing I want to mention is that it seems like a big undertaking. Student groups that promote novel ideas only succeed if there’s a strong leader in each location, who can drive the group and persuade others. These people are very hard to recruit, and tend to have strong alternative options (e.g. founding their own project). Moreover, there’s constant turnover, so you need to recruit one strong leader every 1-2 years in every location. If you miss one year, the group can easily end up in a negative cycle: a weak leader recruiters a weaker replacement, and so on, until the group falls apart.
We’ve had plenty of experience of this in 7 years of student group organising around EA.
(Note that it’s not the same if you’re promoting an idea that’s already widely adopted, because then you have an existing pool of talented people to draw on.)
Building this network out to 10-20 locations will already be years of work, and it will require constant maintenance. Making the groups awesome enough that you get significant penetration in each location will be much harder again. You’ll need to develop your messaging from scratch, and figure out how to make the whole thing seem like “a big deal” with a generation of students. My guess is it’s a 5-10 year project with several full-time staff.
Just look at how much investment there has been in EA student groups so far over 7 years, and there’s still under ~20 successful ones (my rough guess), and these only engage a few percent of the student body, so it’s a long way from getting the future leaders. Perhaps this idea will be a bit easier to spread than EA, but that’s not obvious. Also consider that EA student groups can piggypack off the main organisations, which have received excellent media coverage and produce all kinds of quality content and provide lots of support and experience.
So one question is whether you want to spend the next multiple years of your life doing this. Of course you don’t have to decide right away, but at some point someone will have to make this kind of commitment.
My other thought is that although there’s interesting differences, there’s still a lot of overlap with the strategy of EA groups (and also 80k). They’re going after basically the same audience, and one of the key messages of EA groups is the importance of shaping future tech. And the EA groups could maybe adopt some of your other differences, such as more positive messaging.
Given that both projects require a major amount of work, and it’s better to have one big success than two mediocre efforts (lognormal returns), and similar projects cause confusion, I wonder if it would be better to put your considerable talents into promoting EA groups instead.
Hi Ben,
You raise good points, thank you for taking the time. To address them:
I don’t think Envision is anywhere near as difficult a message to get across as EA. The basic idea already exists in latent form in many students, and the messaging is naturally attractive to those with ambition (who tend to have world-scale goals already), without the negative associations that often exist around the words “altruism” and “impact.” The Princeton Futurist Society (Envision’s previous name) has only been around for one semester and already has 91 members without a strong marketing effort and with an off-putting name; over 30 student groups at universities across the US have said they are planning to attend the conference (including in tech, engineering, entrepreneurship, and policy). We’re not peddling a controversial message, or one that many perceive as in opposition to their own interests (which is how, in my experience, many see altruism and EA); the way I see it, we’re giving words and tangible action paths to what people already want. I certainly could be wrong about this; it will become clearer over the next year. If I’m wrong strategy will be adjusted accordingly.
I also don’t think we’re developing our message from scratch. We’re combining several different messages into one; ie the massive potential of technology, and the importance of safety in realizing that potential. There’s many existing resources to draw from and existing ideas which make it a lot easier to build off of what exists, especially as compared to EA, which had less precedent. As a concrete example, we don’t need to write any books about our message, we just need to promote books and invite speakers.
As a result of the above two points, I think it will be easier than you suggest to grow Envision, although ensuring the integrity of the organization and its message as it grows will certainly be a major challenge. That said, easier does not mean easy, and we certainly acknowledge that it will be difficult.
The danger of weak leaders is indeed serious, and one of the most likely failure scenarios to pan out, at least on a localized level. That’s why we’ll be cautious in founding chapters and are devoting significant time and effort to figuring out how best to identify good chapter founders. Any advice on this is much appreciated, as we have little prior experience to go off of.
I disagree that there’s much overlap between EA and Envision; although they may appear similar, there’s a deep distinction. The majority of those interested in Envision so far are either not, or barely, EA, including many that have heard of it. For various reasons, most entrepreneurs are not attracted to EA, but are attracted to Envision (our conference is co-hosted with Princeton Entrepreneurship Club). Although I don’t want to speculate too much about the causes of this, I think there’s a strong psychological difference between a movement whose primary goal is helping all sentient beings, with one of the tools being technology (a crude but I think sufficiently accurate description of EA) and a movement whose primary goal is the realization of technological development, done in a way that is beneficial to humanity. I could be wrong here and welcome any counter-points. So to summarize, although EA and Envision are pursuing a similar end state and there’s some similarity in the means, there’s a pretty fundamental distinction in the mindset and implementation, which means Envision appeals to many who are not attracted by EA. And I think that audience will play a pivotal role in shaping humanity’s future.
I also agree with AGB’s points below; will comment separately.
I hope that addresses all your points; let me know if it didn’t or if you have any additional questions and/or counter-points.
I’m aware of this from the main post, but I think it’s pretty weak evidence.
You’re essentially trying to integrate the idea of concern for existential risk into tech development, which seems like a similarly difficult task to EA.
Moreover, EA had many excellent existing resources and powerful ideas to draw on, such as the importance of global poverty, the biases literature, the evidence-based movement, and so on. I don’t see a significant difference in difficulty here.
Entrepreneurs are perhaps EAs best target audience. Almost all of GiveWell’s donors are either from tech or finance, and then they partnered with Dustin Moskovitz. Ried Hoffman and the Gates Foundation endorsed Will’s book. Our blog posts are regularly front page of Hacker News. I could go on.
Overall I agree there’s some nice features of the messaging that are different (more positive frame etc.) but I think these benefits are relatively small, and don’t obviously outweigh the large costs of setting up a new org, in an area that’s already extremely crowded by EA effort, and potentially diverting attention from EA groups.
I think a more cost-effective strategy would be to try to spread these messages through existing groups. Or by trying to integrate the positive features of the messaging into EA, perhaps starting in the Princeton group. I think with some ingenuity you could get the Princeton EA group to seriously engage 5% of students then become self-sustaining, and that would be an extremely valuable project that would only take a couple of years.
I share some of these concerns and don’t have anything like a settled opinion on what to do, but there are also arguments against simply having this idea promoted by EA groups, many of which are mentioned in the post. Notably:
EA is generally much more narrow-base/high-ask than Enivision would be. We’ve done this because we seem to get the most impact out of a relatively small number of people doing relatively dramatic things, but it makes the targeting poorly suited for a broad-based low-ask group.
EA already has a political dimension to it that I suspect ‘make technology developments safe’ might be able to avoid. Again, for EA this isn’t super-problematic because we’re only going after a fairly narrow base to start with and it’s not obvious that what negative optics EA already has are hugely affecting that narrow base. But it would be quite sad if, e.g., the terrible reputation of Peter Singer in Germany meant that we couldn’t make headway with future German leaders on technology safety given how far apart the actual topics are.
A related question is what kinds of percentages you really need to make Envision work, or rather at what point the value starts to flatten off. I find it fairly intuitive that 90% of an organisation working on a dangerous technology (e.g. AI) being safety-conscious to start with isn’t that much better than 70% or probably even 50%; all you need is a critical mass to get these ideas seriously considered in circulation. But how low can you go here? Is a 10% starting point enough because that 10% can then teach the others? What about 5%?
Hey Alex,
I definitely agree there are some arguments against, but I’m concerned they’re not strong enough to offset the downsides of setting up a new org.
Also, my understanding is that Envision is also narrow-base. They’re explicitly aiming at future leaders in tech. EA is aiming at that group plus others, so if anything is a wider base than Envision. Rather, Envision differs in being low-ask.
If envision really is only aiming at tech and adjacent undergrads I’ll be disappointed, but that wasn’t my read; what I see “leaders in tech development, policy, academia, and business”. So for instance I assume an high-flying Oxford PPE graduate with a side interest in tech would qualify*.
I think we might be talking past each other slightly on the base point though, when I said EA was narrow-base/high-ask I meant to imply that our available base is narrowed (a lot) by the high ask; it only appeals to people with a fairly strong to very strong altruistic bent. So I think I could sell Envision or something like it to a much wider cross-section of maths/comp sci types than I could EA in general (within JS, maybe 55% versus 20% to give you some idea of percentages).
*For non-UK readers, Oxford PPE graduates have fairly insane levels of penetration into the highest levels of UK politics.
Ah, I got you.
Also just to clarify I was saying with Envision the audience is future leaders, whereas with EA it’s future leaders plus others; so that’s a sense in which EA has a broader audience.
Alex is correct, Envision is not only targeting future tech leaders, it’s targeting future leaders in tech development, policy, academia, and business.
Hi AGB,
Great points, I completely agree. On your last question, this is an intriguing one. I think 10% is too low; they’ll be sidelined, unless those 10% include most of the leadership and most socially influential individuals. Probably 50% is a good starting level, as long as this increases quite quickly.
Curious to hear everyone else’s thoughts on this!
This is really cool! It’s exactly the kind of X-risk intervention I’m excited about (capacity building among elites). I think this investment in the future is even more important than tackling technical problems today.
I noticed that you didn’t mention any need for funding. Does the mean that your current funding needs are adequately met?
Hi Michael,
Great to hear! They are not; although we have some funding, we are still far from fully funded to execute on all our projects. If you’re interested, shoot me an email at lrade@princeton.edu and we can discuss further.
I want to add my voice to those who are very excited to see an initiative along these lines. At least a few people have had to listen to me rant that pretty much exactly this doesn’t exist and should exist in some form.
Serious thought should be given to Ben Todd’s points if that’s not happening already, which it probably is. I replied there also.
Thanks for sharing about the project! I’m curious how do you plan to engage with existing EA chapters in colleges?
Hi Gleb,
The specifics aren’t worked out yet, but we’re working with EA Build and will coordinate with individual EA chapters at the universities we found chapters at. The general idea is that members of EA chapters who are interested in technology and the future will help with the setting up and growing of Envision chapters, and we will direct Envision members who seem interested in EA towards the EA chapter. There may be some events co-hosted; this is probably context-specific.
Ok, thanks for clarifying. Sounds like there will be a significant focus on collaboration. Also consider collaborating with SHIC if you aren’t yet!
I really think it will become very important initiative in the near future.