Survey of 2018 EA Survey

A few months ago, be­fore the re­sults of the 2018 EA sur­vey came out, I asked if peo­ple would be in­ter­ested in mak­ing pre­dic­tions about the an­swers, and 35 peo­ple com­pleted a sur­vey of the sur­vey. I think that the ma­jor­ity of peo­ple an­swer­ing this were from the group or­ganisers group on Face­book and so prob­a­bly have a bet­ter idea of what the EA com­mu­nity looks like in gen­eral, and hope­fully by test­ing our pre­dic­tions we can im­prove our cal­ibra­tion.

For each ques­tion the re­spon­dent was sup­posed to give lower and up­per bounds for which they thought there is an 80% chance that the an­swer is in­side those bounds. All but the first ques­tion would have an­swers as a per­centage be­tween 0 and 100.

Here is the pro­gram, run by guided track. If you want to see the raw re­sults data for the sur­vey of the sur­vey you can email me us­ing david@ealon­don.com.

A few things that might be interesting

  • There was gen­er­ally over­con­fi­dent an­swers with peo­ple get­ting an av­er­age of 60% cor­rect whilst aiming for 80%, with only 2 out of 35 peo­ple un­der confident

  • The ques­tion with the most cor­rect re­sponses was on the pro­por­tion of peo­ple iden­ti­fy­ing as male, 91% got this one right.

  • Most re­spon­dents pre­dicted there would be many more peo­ple say­ing they were poli­ti­cally cen­trist, right or far right than the re­sults suggested

  • It looks like most peo­ple thought that the in­di­vi­d­ual cause pri­ori­ties would be less pop­u­lar, or that there was more ex­clu­sivity be­tween the choices rather than peo­ple choos­ing mul­ti­ple causes as a top or near top pri­or­ity. For ex­am­ple the me­dian an­swer for global poverty was 45% whereas on the 2018 EA sur­vey 66% said that it was a top or near top pri­or­ity, and the other se­lected causes also had much lower pre­dic­tions than the ac­tual results

  • Un­sur­pris­ingly, peo­ple with wider in­ter­vals gen­er­ally got more an­swers cor­rect. I haven’t made a score com­bin­ing these two but that would prob­a­bly give a bet­ter idea of how well in­di­vi­d­u­als are calibrated

The chart be­low is show­ing the cor­rect an­swer per­centages and av­er­age in­ter­val ranges for the 35 par­ti­ci­pants.

Here is a table look­ing at each ques­tion and see­ing what per­centage of the 35 gave a cor­rect an­swer to that ques­tion.

Here is a table look­ing at the me­dian mid­point an­swer for each ques­tion and how that com­pares to the cor­rect an­swer, with a pos­i­tive differ­ence sug­gest­ing peo­ple thought that the an­swer would be higher and a nega­tive differ­ence sug­gest­ing they thought it would be lower (on av­er­age).