Effective Thesis project review

This ar­ti­cle was jointly writ­ten by David Janku and Jan Kul­veit from the Czech EA As­so­ci­a­tion.

Within the Czech EA As­so­ci­a­tion we have been run­ning the Effec­tive Th­e­sis pro­ject for al­most a year now, and would like to share some more in­for­ma­tion about the pro­ject, and our fu­ture plans.

For those in­ter­ested in con­tribut­ing, such as EA-al­igned re­searchers, or even just any­body with a great re­search topic idea, please con­tact us at david.janku@efek­tivni-al­tru­is­mus.cz

Introduction

Effec­tive Th­e­sis is a pro­ject that di­rects stu­dents’ re­search to­wards EA causes by offer­ing them EA-re­lated top­ics for their fi­nal dis­ser­ta­tions and the­ses. The aim is to de­liver three valuable out­comes:

  • Chang­ing stu­dent tra­jec­to­ries at a par­tic­u­larly cru­cial junc­ture in their lives

  • Gen­er­at­ing ad­di­tional re­search in EA cause ar­eas, with lit­tle cost to EA fund­ing sources

  • Mak­ing cur­rent aca­demics (stu­dents’ su­per­vi­sors and com­mit­tees) more fa­mil­iar with EA per­spec­tive and EA top­ics

Our hope is this could be po­ten­tially high-lev­er­age, even more so than usual ca­reer coach­ing, as we are fo­cus­ing on a par­tic­u­larly cru­cial junc­ture.

Background

In most Euro­pean uni­ver­si­ties, a dis­ser­ta­tion is a com­pul­sory part of any bach­e­lor’s, mas­ter’s and doc­toral de­grees. Peo­ple spend hun­dreds to thou­sands of hours work­ing on it (de­pen­dent on study dis­ci­pline and course re­quire­ments) and re­fer to it as the great­est challenge and achieve­ment in their stud­ies. Yet, most of them ex­pe­rience a huge strug­gle when try­ing to find the right topic. The qual­ity of dis­ser­ta­tions differs very much by uni­ver­sity, course, su­per­vi­sor availa­bil­ity and in­di­vi­d­ual stu­dents’ fac­tors. All stu­dents also have their own su­per­vi­sor ad­vis­ing them through­out the re­search pro­cess and the aca­demic qual­ity of the dis­ser­ta­tion is in the end checked by a com­mit­tee who reads it and de­cides on the mark the stu­dent re­ceives.

Main idea

Since dis­ser­ta­tions are com­pul­sory for many stu­dents, they in­vest much time and effort into them, they have aca­demic sup­port in in­creas­ing the qual­ity of their re­search but of­ten strug­gle to find a topic, so we had an idea to uti­lize this pro­cess by offer­ing stu­dents EA-re­lated top­ics.

To make it mul­ti­plica­tive, we have de­cided to ask EA orgs for these top­ics and agreed they will provide stu­dents with con­sul­ta­tions to en­sure that stu­dents’ work will be of best use to EA orgs. Con­sul­ta­tions also serve as a mo­ti­va­tor for stu­dents and should in­crease the qual­ity of their work and, in a broader sense, should help to al­ign academia and prac­tice.

We have de­cided to cre­ate an on­line plat­form to make this pro­ject more scal­able. We have cre­ated a web­site where peo­ple can read about why some prob­lems are more im­por­tant than oth­ers and see the top­ics filtered by their study dis­ci­pline and in­ter­ests. For each topic pro­file, we have put to­gether de­scrip­tions, ex­plain­ing why the topic is im­por­tant and in­clud­ing sources to start with when the stu­dent is in­ter­ested in the topic.

Reflec­tions on the pro­ject development

Outcomes

We have put about 1500 hours into the pro­ject. So far, we have 12 stu­dents work­ing on some of our top­ics and we es­ti­mate that each of them will in­vest about 200-300 hours into their the­ses, re­sult­ing in ap­prox. 2400-3600 ex­tra re­search hours. Up un­til now, the pro­ject bud­get was £5400, so each tra­jec­tory change cost roughly £450. From in­ter­views with 2 stu­dents who have already finished their dis­ser­ta­tions, it seems that our pro­ject helped them re­al­ize that they could do a dis­ser­ta­tion on an EA topic and it also helped them to find sub-dis­ci­plines and spe­cific prob­lems to fo­cus on within the EA prob­lem ar­eas. Both of them worked on AI safety prob­lems and are now are look­ing for a job in this field, al­though the coun­ter­fac­tual in­fluence of dis­ser­ta­tion on their long-term ca­reer plans is difficult to eval­u­ate since both of them also con­sulted with 80k and were in con­tact with other EA or­ga­ni­za­tions and in­for­ma­tion sources.

Marketing

Since we op­er­ate on­line, one of our main mar­ket­ing venues was an AdGrants Google ac­count en­abling us to spend $10.000 per month on AdWords. We got to­gether with the mar­ket­ing com­pany Brainslab who offered help to EA pro­jects and who took care of our ac­count and in­creased its qual­ity. How­ever, af­ter sev­eral months, it seems we still don’t have a good way to reach the right stu­dents, since it hasn’t brought us a sin­gle stu­dent who would like to work on some of our top­ics. We are now chang­ing the strat­egy from us­ing gen­eral key­words (like “the­sis topic”) to us­ing ti­tles of well known and im­por­tant pa­pers and con­cepts spe­cific to each study dis­ci­pline. The most effec­tive strate­gies so far seem to be post­ing in EA face­book groups and pro­mo­tion by word of mouth. In­ter­est­ingly, the strat­egy which may be­come sev­eral times more effec­tive was to google our biggest com­pe­ti­tion in search re­sults and ask them to pro­mote our web­site. Since our main com­peti­tors in search re­sults are blogs ad­vis­ing stu­dents on “how to choose a the­sis topic” but not offer­ing spe­cific top­ics like we do, they can im­prove their con­tent by link­ing us to their ar­ti­cles and we might get a great deal of rele­vant traf­fic from their web­sites. This idea came about from in­ter­views with mar­ket­ing pro­fes­sion­als who we reached out to, and is a good ex­am­ple of non-zero-sum mind­set.

Management

Since we offer top­ics via our web­site, suc­cess de­pends heav­ily on how the web­site is de­signed. We didn’t have very much ex­pe­rience with web de­vel­op­ment and de­sign and there­fore it took us longer to make changes, with re­sults be­ing sub-op­ti­mal. How­ever, we man­aged to cre­ate and run the web­site much more cheaply than if we paid pro­fes­sion­als and im­proved valuable skills (web­site de­vel­op­ment, pro­ject man­age­ment, team­work) along the way.


Steps that proved valuable in­cluded let­ting parts of the pro­ject which don’t re­quire a deep un­der­stand­ing of EA be man­aged by pro­fes­sional or­ga­ni­za­tions, who now take care of ar­eas of it for free as a part of their CSR pro­gramme (as we did with AdWords). Another use­ful step was to con­sult with other in­di­vi­d­ual pro­fes­sion­als who we found via web­sites con­nect­ing non­prof­its and skil­led short-term vol­un­teers (here is the list of such plat­forms we found).

Forth­com­ing plans

As we re­flected on the pro­ject de­vel­op­ment, we found that cur­rent model of work­ing via EA orgs is not op­ti­mal. Some of the top EA re­search or­ga­ni­za­tions are heav­ily time-con­strained, and un­able to spend much effort on propos­ing the­sis top­ics, so the cur­rent top­ics se­lec­tion;

  • Doesn’t rep­re­sent the whole EA re­search land­scape well (with more em­pha­sis on wild an­i­mal suffer­ing, for ex­am­ple, be­cause the re­spec­tive re­search or­ga­ni­za­tion was more co­op­er­a­tive in pro­vid­ing top­ics—a great help!)
  • Many top­ics seem to be highly in­ter­est­ing, but likely not tractable within the scope of mas­ters or even doc­toral the­sis.

  • With many top­ics, it may be difficult to find an ap­pro­pri­ate ad­vi­sor.

In ad­di­tion, the stu­dents, in gen­eral, do not have a com­par­a­tive ad­van­tage in choos­ing top­ics, typ­i­cally hav­ing nei­ther pre­vi­ous ex­pe­rience with dis­ser­ta­tion writ­ing nor a very good overview of the re­search land­scape.

There­fore, we have re­cently launched a differ­ent ap­proach: in­di­vi­d­ual Th­e­sis Topic Coach­ing, a de­liber­a­tive pro­cess in which we try to un­der­stand each stu­dent’s needs and op­por­tu­ni­ties, and sug­gest tai­lored top­ics or re­search di­rec­tions. When stu­dents choose one of our top­ics, we will try to find them con­sul­tants fo­cus­ing speci­fi­cally on their top­ics and will share their work with rele­vant or­ga­ni­za­tions. The change should re­sult in an in­creased up­take of our ser­vices and the qual­ity of fi­nal re­search pieces. This will also en­able us to fo­cus less on web de­vel­op­ment, which took a sig­nifi­cant pro­por­tion of the time we de­voted to the pro­ject.

Another thing we are con­sid­er­ing do­ing is to reach out to cur­rent aca­demics with top­ics rele­vant to their ex­per­tise and ask­ing them to start offer­ing these as dis­ser­ta­tion top­ics for their stu­dents. This might po­ten­tially be a more effec­tive way of ad­dress­ing stu­dents than an on­line web­site since con­sult­ing with teach­ers is a de­fault op­tion for most stu­dents search­ing for dis­ser­ta­tion top­ics. At the same time this would al­low aca­demics to ex­plore and be­come more fa­mil­iar with EA top­ics via their stu­dents.

We would like to en­courage any­one in­ter­ested in this path to con­tact us.

Re­quest for help

As the pro­ject is ul­ti­mately try­ing to cre­ate a two-sided mar­ket­place, it de­pends on the will­ing­ness of EA al­igned re­searchers and re­search or­ga­ni­za­tions to share promis­ing re­search ideas and to offer the stu­dents con­sul­ta­tions.

We want to ask EA al­igned re­searchers for co­op­er­a­tion in this re­spect—if you have the ca­pac­ity to help, please con­tact us. It does not have to be a par­tic­u­larly costly pro­cess and does not cre­ate a com­mit­ment on your side.

We also have a fund­ing gap for 2018 sized circa 12000 USD, which we hope to cover mostly by grants, but in­di­vi­d­ual dona­tions are wel­come and im­por­tant—if you would like to donate, you can do it via Czech Effec­tive Altru­ism As­so­ci­a­tion in a tax-de­ductible way

Our takeaways

  • We are still con­vinced that choos­ing a dis­ser­ta­tion topic is an im­por­tant branch­ing mo­ment in a re­searcher’s tra­jec­tory, which may not only gen­er­ate hun­dreds of hours of ad­di­tional re­search on an im­por­tant topic, but may in­fluence a re­searcher’s long-term ca­reer.
  • Even in case of stu­dents plan­ning non-re­search ca­reers, work­ing on an EA the­sis topic can lead to deeper en­gage­ment with the ideas and com­mu­nity, gen­er­at­ing more knowl­edge­able EAs and mak­ing cur­rent aca­demics more fa­mil­iar with the EA per­spec­tive.

  • We are less sure that let­ting stu­dents choose for them­selves from a list of top­ics pro­posed by EA re­search in­sti­tu­tions is an effec­tive way to uti­lize this branch­ing mo­ment.

Credits

David Janku man­ages the pro­ject and put in most of the work. The web de­vel­op­ment was mainly un­der­taken by Dan Hnyk (pro­gram­ming) and David Horák (copy­writ­ing), helped by many other vol­un­teers. Jan Kul­veit sug­gested the ini­tial idea of steer­ing re­search by pro­mot­ing EA al­igned the­ses top­ics and the di­rec­tion change to the­sis topic coach­ing.

The pro­ject was fi­nan­cially sup­ported by Greg Colbourn with the help of the vol­un­teers Pär Flodin, John Li­di­ard, Ronke Bankole, Nik­ita Hay­ward, Jiří Nád­vorník and oth­ers, and uses an AdWords grant from Google man­aged by Brain­labs.

Come meet us!

We are hold­ing an Effec­tive Th­e­sis meetup on 7.5. at 7 pm in MIRI/​CFAR office in Berkeley, so if you´re in­ter­ested and around, we will be happy to meet you!