SHIC Will Suspend Outreach Operations

A Q1 up­date and 2018 in review

By Bax­ter Bul­lock and Cather­ine Low

Since launch­ing in 2016, Stu­dents for High-Im­pact Char­ity (SHIC), a pro­ject of Re­think Char­ity, has fo­cused on ed­u­ca­tional out­reach for high school stu­dents (pri­mar­ily ages 16-18) through our in­ter­ac­tive con­tent. In Jan­uary 2018, we be­gan im­ple­ment­ing in­struc­tor-led work­shops, mostly in the Greater Van­cou­ver area. Below, we sum­ma­rize our ex­pe­riences of 2018 and ex­plain why we are choos­ing to sus­pend our out­reach op­er­a­tions.

Summary

  • 2018 saw strong up­take, but difficulty se­cur­ing long-term en­gage­ment—Within a year of in­struc­tor-led work­shops, we pre­sented 106 work­shops, reach­ing 2,580 par­ti­ci­pants at 40 (mostly high school) in­sti­tu­tions. We ex­pe­rienced strong stu­dent en­gage­ment and en­courag­ing feed­back from both teach­ers and stu­dents. How­ever, we strug­gled in get­ting stu­dents to opt into ad­vanced pro­gram­ming, which was our be­hav­ioral proxy for fur­ther en­gage­ment.

  • By the end of April, SHIC out­reach will func­tion in min­i­mal form, re­quiring very lit­tle staff time—Over the next two months, our team will grad­u­ally wind down de­liv­ered work­shops at schools. We plan on main­tain­ing a web­site with re­sources and field­ing in­quiries through a con­tact form for those who are look­ing for in­for­ma­tion on how best to im­ple­ment EA ed­u­ca­tion.

  • The most promis­ing el­e­ments of SHIC may be in­cor­po­rated into other high-im­pact pro­jects—The SHIC cur­ricu­lum could likely be re­pur­posed for other high-im­pact pro­jects within the wider Re­think Char­ity um­brella. For ex­am­ple, it could be a tool for en­gag­ing po­ten­tial high-net-worth donors, or as con­tent to provide lo­cal group lead­ers.

  • We be­lieve in the po­ten­tial of ed­u­ca­tional out­reach and hope to re­visit this in the fu­ture—While we ac­knowl­edge the pos­si­bil­ity that poor at­ten­dance at ad­vanced work­shops is in­dica­tive of gen­eral in­ter­est level in our pro­gram and/​or EA in gen­eral, it’s also pos­si­ble that the meth­ods we used to fa­cil­i­tate long term en­gage­ment were in­ad­e­quate. We think that un­der the right cir­cum­stances, ed­u­ca­tional out­reach could be more fruit­ful.

  • SHIC will re­lease an ex­haus­tive eval­u­a­tion of our ex­pe­rience with ed­u­ca­tional out­reach in the com­ing months.

2018 in review

In late 2017 we made a strate­gic shift to­wards a high-fidelity model of stu­dent en­gage­ment through in­struc­tor-led work­shops. We tested this model through­out 2018, with our in­struc­tors vis­it­ing schools in Greater Van­cou­ver, Canada[1].

Most stu­dents (56%) par­ti­ci­pated in a sin­gle-ses­sion work­shop last­ing ap­prox­i­mately 80 min­utes. Th­ese work­shops con­sisted of a giv­ing game[2], fol­lowed by an overview of the core ideas of effec­tive al­tru­ism[3], in­clud­ing cov­er­age of key cause ar­eas. The re­main­ing 44% of par­ti­ci­pants par­ti­ci­pated in multi-ses­sion (typ­i­cally three), in-depth work­shops which usu­ally in­cluded a giv­ing game, in­ter­ac­tive ex­plo­ra­tions of the top­ics men­tioned above, a cause pri­ori­ti­za­tion ac­tivity, and a dis­cus­sion of effec­tive ca­reer paths.

Our goal for the sec­ond half of 2018 was to iden­tify high-po­ten­tial stu­dents from our school vis­its, and en­gage them fur­ther with sup­ple­men­tary ad­vanced work­shops at a cen­tral lo­ca­tion in Van­cou­ver. To gauge in­ter­est ini­tially, we be­gan with an opt-in ap­proach for all in­ter­ested stu­dents who pro­vided an email ad­dress in or­der to ob­tain more in­for­ma­tion. We ran a work­shop in Novem­ber which pri­mar­ily con­sisted of an in-depth ac­tivity on cause pri­ori­ti­za­tion, and a work­shop in De­cem­ber fo­cused on effec­tively cre­at­ing on­line fundraisers for the holi­days.

Our results

The met­rics we iden­ti­fied to gauge our suc­cess were:

  1. Teach­ers and school uptake

  2. Stu­dent sur­vey re­sults in­di­cat­ing shifts of opinion and/​or behavior

  3. The num­ber of stu­dents who con­tinue to en­gage with the ma­te­rial be­yond our ini­tial visit, and the ex­tent to which they re­main in­volved.

We ex­ceeded our ex­pec­ta­tion with met­ric 1 - De­mand from teach­ers was higher than ini­tially ex­pected, with 25% of the schools we con­tacted book­ing at least one work­shop. We also saw some suc­cess through rep­re­sen­ta­tion at lo­cal teacher con­fer­ences. All told, in 2018 we reached a to­tal of 2,580 stu­dents at 40 in­sti­tu­tions, mostly in high school So­cial Stud­ies and Math classes through­out Greater Van­cou­ver[4]. Qual­i­ta­tive feed­back from teach­ers and stu­dents, which in­cludes our reads on the re­ac­tions of work­shop par­ti­ci­pants, sug­gested that stu­dents were en­gaged. Teach­ers gen­er­ally thought the con­tent and de­liv­ery was ex­cel­lent and well-suited to their cur­ricu­lum area. The num­ber of teach­ers who asked us to re­turn for ad­di­tional classes and/​or the for the au­tumn semester af­ter our ini­tial visit serves as fur­ther ev­i­dence of con­tinued in­ter­est in the pro­gram.

We met our ex­pec­ta­tions for met­ric 2 - Qual­i­ta­tive and quan­ti­ta­tive data gath­ered from pre- and post-work­shop sur­veys showed small but sig­nifi­cant im­prove­ments in knowl­edge, at­ti­tudes, and val­ues for the ma­jor­ity of our ques­tions, sug­gest­ing that stu­dents gen­er­ally shift their per­spec­tives as a re­sult of our pro­gram. How­ever, there is likely a so­cial de­sir­a­bil­ity effect to con­sider[5]. Ad­di­tion­ally, it was un­clear to what ex­tent these stu­dents are likely to act upon these newfound per­spec­tives.

We at­tempted to gauge whether these shifts were tem­po­rary, but only 37 out of the 388 stu­dents who were emailed a 3-month sur­vey chose to start the sur­vey, so we were un­able to draw con­clu­sions from such a small sam­ple.

We did not meet our ex­pec­ta­tions for the third, and most im­por­tant met­ricDe­spite the suc­cess of our first two met­rics, we strug­gled to fur­ther en­gage stu­dents with opt-in ad­vanced work­shops. We feel the turnout was poor enough to sug­gest that this was not an effec­tive method for SHIC to en­gage stu­dents be­yond the ini­tial work­shop. 1247 stu­dents were el­i­gible to be in­formed about our ad­vanced work­shops, and we sent out 392 in­vi­ta­tions[6]. 17 stu­dents ex­pressed in­ter­est, and five ended up at­tend­ing our first ad­vanced work­shop in Novem­ber, and two at­tended the De­cem­ber work­shop (both re­turn­ing from the first). There may have been lo­gis­ti­cal rea­sons for why the De­cem­ber work­shop was less at­tended by stu­dents. The de­crease in at­ten­dance could be at­tributed to a change in the work­shop time (moved to be­gin on a Sun­day morn­ing) or its prox­im­ity to the holi­days, or it could have been the par­tic­u­larly dis­cour­ag­ing weather (cold, rain­ing heav­ily).

Based on these re­sults, we’re left with one or more of the fol­low­ing three con­clu­sions:

  1. The stu­dents were en­gaged in the SHIC pro­gram in class, but our meth­ods for en­gag­ing stu­dents be­yond the class­room were in­effec­tive.

  2. Stu­dents had the will, but not the band­width to en­gage fur­ther with the SHIC pro­gram.

  3. Stu­dents were not as en­gaged by the SHIC pro­gram as our post-pro­gram sur­vey data and ex­pe­rience sug­gested, and there­fore un­in­clined to par­ti­ci­pate in fur­ther pro­gram­ming.

Our best guess is that all three of these con­clu­sions are true to some ex­tent. High school-aged stu­dents in North Amer­ica tend to en­gage in a plethora of ex­tracur­ricu­lar ac­tivi­ties, of­ten leav­ing them with lit­tle band­width. We also be­lieve that us­ing email as our pri­mary method of con­tact­ing stu­dents was not the op­ti­mal form of cor­re­spon­dence, as email tends not to be a widely used form of com­mu­ni­ca­tion with this age group[7].

Our rea­son­ing for sus­pend­ing work­shop operations

Our de­ci­sion to sus­pend SHIC’s op­er­a­tions was an un­usu­ally difficult one be­cause, de­spite our failure to de­liver on our key met­ric, we still be­lieve ed­u­ca­tional out­reach in some form could be effec­tive. For ex­am­ple, we could have shifted our au­di­ence to more tar­geted groups be­yond high schools, or used other meth­ods of en­gage­ment within high schools. It’s also pos­si­ble that SHIC work­shops had more of an effect than we were able to mea­sure, such as in­creas­ing the chances that stu­dents will donate or fundraise for effec­tive char­i­ties in the fu­ture, or be more re­cep­tive next time they en­counter EA ideas.

How­ever, as a re­sult of not reach­ing our goals, we came to be­lieve SHIC is now less mar­ketable to new and ex­ist­ing donors who were also in­ter­ested in long-term en­gage­ment of stu­dents. Had this not been the case, we would have more strongly con­sid­ered ex­plor­ing the effec­tive­ness of our pro­gram with new au­di­ences, such as uni­ver­sity stu­dents and adults.

Fi­nally, the sec­tion be­low out­lines our be­lief that or­ga­ni­za­tional band­width and re­sources would be more valuable if re­al­lo­cated to other promis­ing pro­jects, such as RC For­ward and the Lo­cal Effec­tive Altru­ism Net­work (LEAN).

We are nonethe­less proud of what SHIC has ac­com­plished, and be­lieve there is a pos­si­bil­ity that this pro­gram made an im­pact on many stu­dents’ lives, de­spite our difficul­ties in track­ing that im­pact.

Look­ing Forward

A SHIC web­site con­tain­ing our ma­te­ri­als and a con­tact form will re­main ac­tive—While ac­tive out­reach will be sus­pended, we think it’s im­por­tant to main­tain ac­cessibil­ity to the user-friendly con­tent SHIC has cre­ated. We would also like to put vol­un­teer time and min­i­mal staff time to­wards con­sult­ing those with in­quiries about how best to im­ple­ment EA ed­u­ca­tion.

A de­tailed eval­u­a­tion of SHIC and high school out­reach is forth­com­ing—By May 1, we plan on re­leas­ing an in-depth look at not only our ex­pe­riences with high school work­shops, but all pro­grams SHIC has con­ducted, as well as ag­gre­gate data from other similar pro­jects re­lated to EA ed­u­ca­tional out­reach. It is our hope that this post will provide a foun­da­tion upon which ex­per­i­men­ta­tion with ed­u­ca­tional out­reach can con­tinue within EA.

Repur­posed SHIC re­sources could be very valuable—A dis­solu­tion of one of our pro­jects al­lows us to re­al­lo­cate some of the more promis­ing el­e­ments of that pro­ject to­wards other ini­ti­a­tives. In SHIC, we have the re­sources and staff to effec­tively make com­plex philo­soph­i­cal and math­e­mat­i­cal con­cepts ac­cessible to the pub­lic. We be­lieve this could prove very valuable for other pro­jects within Re­think Char­ity, such as ed­u­cat­ing high net worth donors in Canada about effec­tive giv­ing op­por­tu­ni­ties as part of RC For­ward, and pro­vid­ing high qual­ity ma­te­ri­als and guidance for group lead­ers who wish to run work­shops on EA con­cepts as part of the Lo­cal EA Net­work (LEAN).

A po­ten­tial re­turn to ed­u­ca­tional out­reach in the fu­ture—We still be­lieve that ed­u­ca­tional out­reach could be very im­pact­ful, and are pleased that oth­ers within the EA com­mu­nity are con­tin­u­ing with their out­reach pro­grams. Pend­ing clear­ance of sev­eral hur­dles from a strate­gic stand­point, this could be worth re­turn­ing to in the fu­ture. Those hur­dles in­clude re­con­sid­er­ing the op­ti­mal au­di­ence, meth­ods of data col­lec­tion, and meth­ods of long-term en­gage­ment.

_______________________________________________

Endnotes

1. Prior to be­com­ing SHIC Man­ager, Cather­ine Low tested SHIC ma­te­ri­als in the class­room as a high school teacher in 2016 and 2017. Her ex­pe­riences pro­vided ev­i­dence that an in­struc­tor-based model may be more effec­tive than stu­dent-led mod­els.

2. SHIC’s giv­ing game in­volves stu­dents an­a­lyz­ing 3 to 4 char­i­ties com­par­a­tively, then de­cid­ing which char­ity will re­ceive a dona­tion.

3. In most cases, the term “effec­tive al­tru­ism” was not used as part of the SHIC pro­gram.

4. In Van­cou­ver high schools speci­fi­cally, we reached 2,228 stu­dents in 31 in­sti­tu­tions.

5. We in­cluded a so­cial de­sir­a­bil­ity test in our sur­vey to at­tempt to mea­sure this bias, how­ever the re­sults have not been fac­tored into our sur­vey anal­y­sis.

6. For stu­dents who par­ti­ci­pated in the full SHIC pro­gram in the Win­ter and Spring terms of early 2018, pro­vid­ing a con­tact email was manda­tory. In Sum­mer and Fall 2018, 19% of the stu­dents vol­un­tar­ily gave us a con­tact email.

7. We con­sid­ered other (so­cial me­dia fo­cused) meth­ods of com­mu­ni­ca­tion that ul­ti­mately felt in­ap­pro­pri­ate.