Aaron Gertler(Aaron L. Gertler)

Ex­ter­nal eval­u­a­tion of GiveWell’s research

• Aaron Gertler 22 May 2020 4:03 UTC

  2 points

  on: The case for build­ing more and bet­ter epistemic in­sti­tu­tions in the effec­tive al­tru­ism com­mu­nity

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  This post con­tains a well-struc­tured ar­gu­ment for ad­dress­ing a prob­lem that could be drag­ging down the over­all im­pact of EA work across many differ­ent ar­eas. You could sum­ma­rize the main point in a way that makes it seem ob­vi­ous (“EA should try to figure things out in a bet­ter way than it does now”), but in do­ing so, you’d be ig­nor­ing the de­tails that make the post great:

  • Point­ing out ex­am­ples of things the com­mu­nity has done that pushed EA in the right di­rec­tion (e.g. in­fluen­tial crit­i­cism, ex­pert sur­veys) in or­der to show that we could do even more work along the same lines.

  • Com­par­ing one rea­son­able pro­posal (bet­ter in­sti­tu­tions) to other rea­son­able pro­pos­als (bet­ter norms, other types of in­sti­tu­tion, fo­cus­ing on growth over in­sti­tu­tion-build­ing) with­out ar­gu­ing too vo­cif­er­ously in fa­vor of the first pro­posal. I liked the lan­guage “I sketch a few con­sid­er­a­tions,” where some posts might have used “I show how X is su­pe­rior to Y and Z.”

  If you read this post, I also strongly recom­mend read­ing the com­ments! (This ap­plies to the post above as well.)

• Aaron Gertler 22 May 2020 4:03 UTC

  3 points

  on: Effec­tive An­i­mal Ad­vo­cacy Non­profit Roles Spot-Check

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  Many peo­ple have strong opinions on the state of the EA job mar­ket, but it can be difficult to find enough data to sup­port any par­tic­u­lar view­point. I ap­pre­ci­ate AAC’s efforts to chase down facts, and to pre­sent its method­ol­ogy and re­sults very clearly. I don’t have much to say about the style or struc­ture of this post; it’s just clear and thor­ough, and I’d be happy to hear about other re­searchers us­ing it as a tem­plate for pre­sent­ing their own work.

  (One de­tail: I like that the “limi­ta­tions” sec­tion also in­cludes sug­ges­tions for fur­ther re­search. Posts that show how oth­ers can build on them seem likely to en­courage fur­ther in­tel­lec­tual progress.)

• Aaron Gertler 22 May 2020 4:02 UTC

  3 points

  on: Effec­tive Altru­ism and Free Riding

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  This post de­scribes is­sues that could ap­ply to nearly ev­ery kind of EA work, with clear nega­tive con­se­quences for ev­ery­one in­volved. I es­pe­cially liked the prob­lem state­ment in this pas­sage:

  The key in­tu­ition is that in an un­co­op­er­a­tive set­ting each al­tru­ist will donate to causes based on their own value sys­tem with­out con­sid­er­ing how much other al­tru­ists value those causes. This leads to un­der­in­vest­ment in causes which many differ­ent value sys­tems place pos­i­tive weight on (causes with pos­i­tive ex­ter­nal­ities for other value sys­tems) and over­in­vest­ment in causes which many value sys­tems view nega­tively (causes with nega­tive ex­ter­nal­ities).

  The post sup­ports this point with a well-struc­tured ar­gu­ment. Ele­ments I es­pe­cially liked:

  • The use of ta­bles to demon­strate a sim­ple ex­am­ple of the problem

  • Refer­ences to crit­i­cism of EA from peo­ple out­side the move­ment (show­ing that “free-rid­ing” isn’t just a po­ten­tial is­sue, but may be in­fluenc­ing how peo­ple per­ceive EA right now)

  • Refer­ences to rele­vant work already hap­pen­ing within the move­ment (so that read­ers have a sense for ex­ist­ing work they could sup­port, rather than feel­ing like they’d have to start from scratch in or­der to ad­dress the prob­lem)

  • The au­thor start­ing their “What should we do about this?” sec­tion by not­ing that they weren’t sure whether “defect­ing in pris­oner’s dilem­mas” was ac­tu­ally a bad thing for the EA com­mu­nity to do. It’s re­ally good to dis­t­in­guish be­tween “be­hav­ior that might look bad” and “be­hav­ior that is ac­tu­ally so harm­ful that we should stop it.”

• Aaron Gertler 22 May 2020 4:00 UTC

  3 points

  on: A Qual­i­ta­tive Anal­y­sis of Value Drift in EA

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  Value drift isn’t dis­cussed of­ten on the Fo­rum, but I’d like to see that change.

  I re­mem­ber meet­ing quite a few peo­ple when I started to learn about EA (in 2013), and then re­al­iz­ing later on that I hadn’t heard from some of them in years — even though they were highly al­igned and in­ter­ested in EA work when I met them.

  If we can figure out how to make that sort of thing hap­pen less of­ten, we’ll have a bet­ter chance of keep­ing the move­ment strong over the long haul.

  Marisa’s piece doesn’t try to draw any strong con­clu­sions — which makes sense, given the sam­ple size and the ex­plo­ra­tory na­ture of the re­search — but I ap­pre­ci­ated its beau­tiful for­mat­ting. I also like how she:

  • Refer­ences non-EA re­search on so­cial move­ments. (This is some­thing the com­mu­nity as a whole may not be do­ing enough of.)

  • In­cludes a set of di­rect quotes from in­ter­vie­wees. (Ac­tual hu­man speech offers nu­ance and de­tail that are hard to match with a sum­mary of mul­ti­ple an­swers.).

  • Offers fu­ture re­search di­rec­tions for peo­ple who see this post and want to work on similar is­sues.

• Aaron Gertler 22 May 2020 4:00 UTC

  4 points

  on: Bi­ases in our es­ti­mates of Scale, Ne­glect­ed­ness and Solv­abil­ity?

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  Cause pri­ori­ti­za­tion is still a young field, and it’s great to see some­one come in and ap­ply a sim­ple, rea­son­able cri­tique that may im­prove many differ­ent re­search pro­jects in a con­crete way.

  It’s also great to check the com­ments and re­al­ize that Michael ed­ited the post af­ter pub­lish­ing to im­prove it fur­ther — a prac­tice I’d like to see more of!

  Aside from that, this is just a lot of solid math be­ing con­ducted around an im­por­tant sub­ject, with im­pli­ca­tions for any­one who wants to work on pri­ori­ti­za­tion re­search. If we want to be effec­tive, we need to have strong epistemic norms, and avoid­ing bi­ased es­ti­mates is a key part of that.

• Aaron Gertler 22 May 2020 4:00 UTC

  2 points

  on: My per­sonal cruxes for work­ing on AI safety

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  “I ed­ited [the tran­script] for style and clar­ity, and also to oc­ca­sion­ally have me say smarter things than I ac­tu­ally said.”

  The “en­hanced tran­script” for­mat seems very promis­ing for other Fo­rum con­tent, and I hope to see more peo­ple try it out!

  As for this en­hanced tran­script: here, Buck rea­sons through a difficult prob­lem us­ing tech­niques we en­courage — lay­ing out his “cruxes,” or points that would lead him to change his mind if he came to be­lieve they were false. This prac­tice en­courages dis­cus­sion, since it makes it eas­ier for peo­ple to figure out where their views differ from yours and which points are most im­por­tant to dis­cuss. (You can see this both in the Q&A sec­tion of the tran­script and in com­ments on the post it­self.)

  I also re­ally ap­pre­ci­ated Buck’s in­tro­duc­tion to the talk, where he sug­gested to listen­ers how they might best learn from his work, as well as his con­clud­ing sum­mary at the end of the post.

  Fi­nally, I’ll quote one of the com­menters on the post:

  I think the part I like the most, even more than the awe­some de­con­struc­tion of ar­gu­ments and their un­der­ly­ing hy­pothe­ses, is the sheer num­ber of times you said “I don’t know” or “I’m not sure” or “this might be false”.

• Aaron Gertler 22 May 2020 3:58 UTC

  2 points

  on: Do­ing good is as good as it ever was

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  In this piece, Denise ar­gues that many peo­ple in­volved with EA are un­satis­fied be­cause they’ve de­vel­oped high ex­pec­ta­tions about how much good they’ll be able to do — ex­pec­ta­tions which, when up­set, lead to a lack of mo­ti­va­tion.

  While the scope of this prob­lem isn’t clear to me, I do think the es­say was beau­tifully writ­ten, and it struck a chord with many read­ers. There are sev­eral lines I ex­pect to refer­ence well into the fu­ture:

  “Maybe there are other peo­ple who are able to have an even higher im­pact than you. But that doesn’t change the amount of good you can do.”

  ““Par­ti­ci­pat­ing in the EA com­mu­nity should make you feel more mo­ti­vated about the amount of good you are able to do, not less. If it makes you feel less mo­ti­vated on bal­ance, then the EA com­mu­nity is do­ing some­thing fun­da­men­tally wrong and ev­ery­body might be bet­ter off some­where else un­til this is fixed.”

• Aaron Gertler 22 May 2020 3:58 UTC

  4 points

  on: EAF’s bal­lot ini­ti­a­tive dou­bled Zurich’s de­vel­op­ment aid

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  A re­mark­able pro­ject and a re­mark­able writeup. There were many things I ap­pre­ci­ated about this piece, in­clud­ing:

  • The use of pho­tos and other vi­sual aids

  • Links to origi­nal sources (e.g. meet­ing min­utes, gov­ern­ment web­sites)

  • Suggested pro­jects for peo­ple who want to repli­cate the bal­lot ini­ti­a­tive elsewhere

  • A col­lec­tion of me­dia cov­er­age so that read­ers (at least those who speak Ger­man) could see how non-EA sources viewed the initiative

  While this post pro­vides an in­ter­est­ing his­tory of the Zurich ini­ti­a­tive, I’m more ex­cited by the way it hints at be­ing a “recipe” for this form of EA suc­cess: I can imag­ine many other such ini­ti­a­tives pass­ing in the next decade.

• Aaron Gertler 22 May 2020 3:58 UTC

  2 points

  on: Growth and the case against ran­domista development

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  This piece starts off very well, with the au­thors ex­plain­ing which claims they ac­tu­ally mean to defend and how strongly they sup­port those claims. I also ap­pre­ci­ated the rest of the out­line in the in­tro­duc­tion.

  A lot of writ­ing about effec­tive al­tru­ism ar­gues that more re­search should be done on a given cause. “Ran­domista de­vel­op­ment” goes fur­ther by mak­ing ex­plicit pre­dic­tions about the value of ad­di­tional re­search:

  “A ~4 per­son-year re­search effort will find dona­tion op­por­tu­ni­ties work­ing on eco­nomic growth in LMICs which are sub­stan­tially bet­ter than GiveWell’s top char­i­ties from a cur­rent gen­er­a­tion hu­man welfare-fo­cused point of view.”

  The rest of the piece is well-writ­ten and uses a lot of helpful graph­ics, but my fa­vorite part came at the end, where the au­thors give a fair ac­count­ing of a num­ber of coun­ter­ar­gu­ments, some of which are even ac­cepted as be­ing partly true!

• Aaron Gertler 22 May 2020 3:56 UTC

  2 points

  on: Manag­ing risk in the EA policy space

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  As more EA en­ergy goes into policy change, the com­mu­nity will benefit from hav­ing good heuris­tics about mak­ing change hap­pen. I ap­pre­ci­ate the au­thor’s fo­cus on this im­por­tant goal, as well as:

  • Their use of bold text to call at­ten­tion to im­por­tant points.

  • Their re­al­is­tic ap­proach to risk man­age­ment, and their ac­knowl­edge­ment that risk can’t be en­tirely re­moved from poli­ti­cal ad­vo­cacy. (I some­times see ideas be­ing pushed against be­cause they are “risky”, with­out much con­sid­er­a­tion for how those risks might be re­duced, or how they ac­tu­ally im­pact the idea’s ex­pected value.)

  • Their will­ing­ness to call out spe­cific ideas as be­ing risky, and to ex­plain the risks (rather than in­vent­ing a sam­ple idea that doesn’t nec­es­sar­ily have ac­tual sup­port­ers in the com­mu­nity, which is the ap­proach I might have taken — and which I think wouldn’t have worked as well)

  The win­ning comments

  I won’t write up an anal­y­sis of each com­ment. In­stead, here are my thoughts on se­lect­ing com­ments for the prize.

• Aaron Gertler 22 May 2020 3:56 UTC

  2 points

  on: Will the Treaty on the Pro­hi­bi­tion of Nu­clear Weapons af­fect nu­clear de­pro­lifer­a­tion through le­gal chan­nels?

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  Many Fo­rum posts try to cover very broad top­ics and wind up strug­gling to do them jus­tice — not be­cause the au­thors haven’t done ex­cel­lent work (they of­ten have), but be­cause the world is enor­mously de­tailed and com­plex.

  This post, on the other hand, aims at a ques­tion that… is still de­tailed and com­plex, but also suffi­ciently nar­row that the au­thor can tackle what seem to be most of the key sub-ques­tions.

  Par­tic­u­lar as­pects I liked:

  • The “ques­tions that could be in­ter­est­ing to ex­plore” sec­tion at the end. More posts should have these!

  • The sum­mary that opens the post, which de­scribes both the au­thor’s pro­cess and her con­clu­sions. Again: more posts should have these!

  • The for­mu­la­tion of prob­a­bil­ities for var­i­ous claims: (you get the idea)

  I do wish, how­ever, that the post had been a bit more clear on how this ques­tion fits into Luisa’s over­all per­spec­tive on nu­clear risk, and what her con­clu­sions in this post might im­ply for EA-al­igned work in this space. (Of course, those might come up in some fu­ture post!)

• Aaron Gertler 22 May 2020 3:56 UTC

  2 points

  on: Thoughts on do­ing good through non-stan­dard EA ca­reer pathways

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  Many peo­ple who want to do a lot of good are pur­su­ing or will pur­sue ca­reers that aren’t among the top sug­ges­tions of 80,000 Hours. Thus, it seems highly valuable to con­sider ways in which peo­ple can aim to in­crease their im­pact across a wide range of ca­reer paths.

  Aside from tack­ling a promis­ing topic, Buck’s post also does some spe­cific things I like:

  • He be­gins with an ex­tended quote from an­other ar­ti­cle, and later spends a lot of time ad­dress­ing the par­tic­u­lar ex­am­ple cited in that quote (the po­ten­tial im­pact of an­thro­pol­ogy). When con­tent di­rectly ex­pands upon pre­vi­ous con­tent that’s of­ten a sign that we’re mak­ing progress in an area. I’d love to see more posts that ex­plore the deeper im­pli­ca­tions of briefly-stated ideas in pre­vi­ous EA ma­te­rial.

  • He pre­sents most of his con­crete ad­vice in list form, mak­ing it rel­a­tively easy for some­one to re­visit this post and skim through it to find sug­ges­tions that might be ap­pli­ca­ble to their cur­rent po­si­tion.

  • He care­fully points out the ways in which his ad­vice does and doesn’t break with “stan­dard” ad­vice. For ex­am­ple, while he dis­cusses ways to make an im­pact in ca­reers that aren’t stan­dard 80,000 Hours recom­men­da­tions, he also notes that do­ing so might be a lot harder, and that there are still strong rea­sons to con­sider recom­mended po­si­tions.

  One thing I’d have been in­ter­ested to see: More real-world ex­am­ples of peo­ple in the com­mu­nity who have done a lot of good through un­usual ca­reer paths. This could have pro­vided ev­i­den­tiary sup­port (or the op­po­site) for some of the ideas Buck pre­sented.

• Aaron Gertler 22 May 2020 3:54 UTC

  2 points

  on: Eight high-level un­cer­tain­ties about global catas­trophic and ex­is­ten­tial risk

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  “Open ques­tions” are a key driver of new aca­demic re­search, and can be a good way for aca­demics to ap­proach a new field.

  For this rea­son, I like see­ing lists like Siebe’s — it’s not quite a set of open ques­tions, but it lays out key un­cer­tain­ties that could be used to pro­duce such ques­tions. It also pro­vides a strong set of cita­tions, giv­ing the afore­men­tioned aca­demics a sense for where to start if they want to work on one of these ar­eas.

• Aaron Gertler 22 May 2020 3:54 UTC

  3 points

  on: In­sti­tu­tions for Fu­ture Generations

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  I have a soft spot for posts that fea­ture a long list of ideas, with just enough de­scrip­tion to spark a reader’s imag­i­na­tion.

  While Tyler’s full re­port on these ideas re­mains un­finished, I ap­pre­ci­ate that he shared the ini­tial list to gather feed­back and ad­di­tional ideas. And I es­pe­cially ap­pre­ci­ate that he was very clear about the pur­pose of the post and how read­ers could con­tribute to his work (this may be one of the rea­sons it brought in so many thought­ful com­ments).

• Aaron Gertler 22 May 2020 3:50 UTC

  2 points

  on: Why and how to start a for-profit com­pany serv­ing emerg­ing markets

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  This novel ca­reer pro­file, de­rived from di­rect ex­pe­rience, goes into im­pres­sive de­tail on sev­eral key points:

  • Choos­ing where to found an emerg­ing-mar­ket startup

  • Select­ing a product that might provide value to a large pop­u­la­tion of emerg­ing-mar­ket users

  • Chang­ing the usual “Sili­con valley startup play­book” to ap­ply to your lo­cal context

  It’s hard to imag­ine some­one with an in­ter­est in this path read­ing Ben’s post and not dis­cov­er­ing some­thing use­ful. While I haven’t tracked its true im­pact, there are enough ideas here to launch a dozen plau­si­ble com­pa­nies. But im­por­tantly, there are also enough caveats that read­ers should get a strong im­pres­sion of how difficult the work can be. This post is a won­der­ful ex­am­ple of “scout mind­set”: Ben’s pri­mary aim isn’t to per­suade, but to provide in­for­ma­tion that helps peo­ple make an ac­cu­rate de­ci­sion.

• Aaron Gertler 22 May 2020 3:49 UTC

  2 points

  on: Shap­ley val­ues: Bet­ter than counterfactuals

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  To be hon­est, my fa­vorite part of this post may be the very hon­est epistemic sta­tus (“en­thu­si­asm on the verge of par­ti­san­ship”).

  ...but the rest of the post was also quite good: many, many ex­am­ples, plus a helpful link to a calcu­la­tor that read­ers could use to try ap­ply­ing Shap­ley val­ues them­selves. As with “Real­ity is of­ten un­der­pow­ered”, the ad­vice here could be used in many differ­ent situ­a­tions (the ex­am­ples help to lay out how Shap­ley val­ues might help us un­der­stand the im­pact of giv­ing, hiring, di­rect work, pub­lic com­mu­ni­ca­tion…).

  I was also pleased to see the au­thor’s replies to com­menters (and the fact that they ed­ited their epistemic sta­tus af­ter one ex­change).

• Aaron Gertler 22 May 2020 3:48 UTC

  4 points

  on: Tech­ni­cal AGI safety re­search out­side AI

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  To quote one com­menter, “I think posts of this type (which list op­tions for peo­ple who want to work in a cause area) are valuable”. I have a sense that fields of re­search are more likely to thrive when they can pre­sent schol­ars with in­ter­est­ing open prob­lems, and Ngo takes the ex­tra step of iden­ti­fy­ing prob­lems that might ap­peal to peo­ple who might not oth­er­wise con­sider work­ing on AGI safety. This post is a good idea, ex­e­cuted well, and I don’t have much else to say — but I will note the abun­dant hy­per­links to sources in­side and out­side of EA.

• Aaron Gertler 22 May 2020 3:48 UTC

  2 points

  on: Real­ity is of­ten underpowered

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  In this post, Lewis makes a pow­er­ful ar­gu­ment that we ought to pay more at­ten­tion when we find our­selves work­ing with what­ever data we can scrounge from data-poor en­vi­ron­ments, and con­sider other ways of de­vel­op­ing our judg­ments and pre­dic­tions.

  Some el­e­ments of this post I es­pe­cially ap­pre­ci­ated:

  • The au­thor’s points are ap­pli­ca­ble to work in many differ­ent cause ar­eas, and he ex­plic­itly points out ways in which they are more or less ap­pli­ca­ble de­pend­ing on the prob­lem at hand.

  • He opens with a mem­o­rable story be­fore mak­ing his gen­eral points (I ex­pect that this prac­tice will of­ten make Fo­rum posts more mem­o­rable, and thus more likely to be ap­plied when they mat­ter).

  • Rather than sim­ply iden­ti­fy­ing a prob­lem, he points out ways in which we might be able to over­come it, in­clud­ing a sec­tion with “fi­nal EA take­aways”; I love to see posts that, when rele­vant, end with a set of ac­tion­able sug­ges­tions.

• Aaron Gertler 22 May 2020 3:46 UTC

  2 points

  on: [up­dated] Global de­vel­op­ment in­ter­ven­tions are gen­er­ally more effec­tive than Cli­mate change interventions

  This post was awarded an EA Fo­rum Prize; see the prize an­nounce­ment for more de­tails.

  My notes on what I liked about the post, from the an­nounce­ment:

  My note on Leopold’s pa­per also ap­plies here, and in this case, we have at least some ev­i­dence that Hauke’s post was ac­cessible to crit­ics — one of them was able to point out a calcu­la­tion they be­lieved was mis­taken. After this, Hauke took the im­pres­sive step of cor­rect­ing the post, re­vers­ing his con­clu­sion, and chang­ing the ti­tle to re­flect these changes.

  Good epistemic prac­tice aside, Hauke also or­ga­nized the post very well. He pro­vides a strong refer­ence sec­tion, a set of use­ful ap­pen­dices, and (my per­sonal fa­vorite) de­scrip­tions of sce­nar­ios un­der which some­one might reach differ­ent con­clu­sions about the rel­a­tive effec­tive­ness of cli­mate and de­vel­op­ment in­ter­ven­tions. Im­pact es­ti­ma­tion in­volves a lot of un­cer­tainty; good Fo­rum posts gen­er­ally find a bal­ance be­tween point­ing out this un­cer­tainty and ac­tu­ally draw­ing con­clu­sions that peo­ple can use to guide their ac­tions.