jacobpfau

• jacobpfauJul 26, 2020, 6:26 PM

  1 point

  0 ∶ 0

  in reply to: jacobpfau’s comment on: What ques­tions would you like to see fore­casts on from the Me­tac­u­lus com­mu­nity?

  Somewhat unrelated, but I’ll leave this thought here anyway: Maybe EA metaculus users could perhaps benefit from posting question drafts as short-form posts on the EA forum.

• jacobpfauJul 26, 2020, 6:25 PM

  10 points

  0 ∶ 0

  on: What ques­tions would you like to see fore­casts on from the Me­tac­u­lus com­mu­nity?

  Thanks for doing this, great idea! I think Metaculus could provide some valuable insight into how society’s/​EA’s/​philosophy’s values might drift or converge over the coming decades.

  For instance, I’m curious about where population ethics will be in 10-25 years. Something like, ‘In 2030 will the consensus within effective altruism be that “Total utilitarianism is closer to describing our best moral theories than average utilitarianism and person affecting views”?’

  Having your insight on how to operationalize this would be useful, since I’m not very happy with my ideas: 1. Polling FHI and GW 2. A future PhilPapers Survey if there is one 3. Some sort of citation count/​ number of papers on total/​average/​person utilitarianism. It would probably also be useful to get the opinion of a population ethicist.

  Stepping back from that specific question, I think Metaculus could play a sort of sanity-checking, outside-view role for EA. Questions like ‘Will EA see AI risk (climate change/​bio-risk/​etc.) as less pressing in 2030 than they do now?‘, or ‘Will EA in 2030 believe that EA should’ve invested more and donated less over the 2020s?’

• jacobpfauJul 14, 2020, 6:01 PM

  3 points

  0 ∶ 0

  on: AMA or dis­cuss my 80K pod­cast epi­sode: Ben Garfinkel, FHI researcher

  You discuss at one point in the podcast the claim that as AI systems take on larger and larger real world problems, the challenge of defining the reward function will become more and more important. For example for cleaning, the simple number-of-dust-particles objective is inadequate because we care about many other things e.g. keeping the house tidy and many side constraints e.g. avoiding damaging household objects. This isn’t quite an argument for AI alignment solving itself, but it is an argument that the attention and resources poured into AI alignment may naturally rise to the challenge without EA effort, and thus perhaps EA effort is misplaced.

  First off, I think this is a great steel-man of the Lecun/​Etzioni safety skeptic position, and, importantly, I think it gives a more concrete/​falsifiable position to argue against. On the other hand, this argument seems to go through only if most of the tasks worked on by AI researchers are of the kind described—i.e. the designer of the system has it in their own interest to deal with side constraints and fix reward function specification. In my view, this condition is unlikely to be met. It seems to me likely that most tasks AI corps work on will have a principal-agent complication. Recommender system alignment, automated advertising, stock trading, etc. work in all of these domains maximize profit for the AI researchers’ company when they run roughshod over side constraints. The side constraints here being mostly the preferences of users on the platform for tech, and other investors for finance.

  Does this seem right? If so, what are the upshots? Could the legal/​lobbying work of strengthening the positions of these principals become a high-value task for EA to take on?

• jacobpfauJun 24, 2020, 9:19 PM

  1 point

  0 ∶ 0

  on: EA Fo­rum fea­ture sug­ges­tion thread

  Perhaps include a short form subsection under the Forum Favorites section? It seems to me that most short form posts have very low visibility.

  If the forum admins have traffic statistics, they should be able to get a better sense of the visibility issue than I can. In particular, I suspect the short form section receives a fraction of the traffic of the frontpage, but this should be verified empirically.

• jacobpfauJun 12, 2020, 8:44 PM

  11 points

  0 ∶ 0

  on: Mo­ral Anti-Real­ism Se­quence #3: Against Irre­ducible Normativity

  I enjoyed reading this post! I like Wittgensteinian arguments, and applying them to ethics, so hurrah for this. There was also some lively discussion of it on the EA corner chat.

  Another possible misleading motivation for irreducible normativity may be linguistic. It seems to me plausible that anyone who uses the word agony in the standard sense is committing her/​himself to agony being undesirable. This is not an argument for irreducible normativity, but it may give you a feeling that there is some intrinsic connection underlying the set of self-evident cases.

  From an EA perspective, I thought it could be useful to get a sense of the effectiveness of this post (series)? You could, for instance, identify a few philosophy graduate students who hold the position you’re arguing against and compare their credence in the relevant position before and after reading. In my experience, people’s cruxes for disagreement in ethics are all over the place, and you run a risk of missing the arguments which compel those who believe in e.g. irreducible normativity. I very much like Wittgensteinian arguments against motivations and coherence, but for those who subscribe to irreducible normativity I’m not sure they will find these arguments compelling. If this concern actualizes, you might find it useful to first poll people who disagree with you about the position of interest, and then write a post to address the cruxes you have identified.

  Edit: At the moment EA forum spam filter is, for some reason, preventing me from replying to @antimonyanthony, so I will reply by edit instead: I think this is quite a subtle point, and as I understand it, there is some ongoing disagreement among philosophers about these issues. Let’s make things clearer by replacing ‘agony’ with ‘bad experience’. A bad experience for a paperclip maximizer is likely to involve difficulty producing paperclips. More generally, which experiences are considered bad is determined by the agent’s nature. However for humans there’s sufficient overlap in our neural nature for their to be self-evident cases of badness, e.g. extreme pain. If someone does not call these self-evident cases bad, then she/​he is not using the word bad in its standard sense. There are a lot of complications in this argument cf. Kripke on c-fibers, but I believe the general argument I sketched holds.

• jacobpfauMay 10, 2020, 10:46 PM

  1 point

  0 ∶ 0

  in reply to: Lukas_Gloor’s comment on: ja­cobp­fau’s Shortform

  Yes, I recently asked a metaculus mod about this, and they said they’re hoping to bring back the ai.metaculus sub-domain eventually. For now, I’m submitting everything to the metaculus main domain.

• jacobpfauMay 10, 2020, 10:20 PM

  2 points

  0 ∶ 0

  on: ja­cobp­fau’s Shortform

  Medium term AI forecasting with Metaculus

  I’m working on a collection of metaculus.com questions intended to generate AI domain specific forecasting insights. These questions are intended to resolve in the 1-15 year range, and my hope is that if they’re sufficiently independent, we’ll get a range of positive and negative resolutions which will inform future forecasts.

  I’ve already gotten a couple of them live, and am hoping for feedback on the rest:

  1. When will AI out-perform humans on argument reasoning tasks?

  2. When will multi-modal ML out-perform uni-modal ML?

  3. (Not by me) When will image recognition be made robust against unrestricted adversary?

  4. (WIP) When will reinforcement learning methods achieve sample efficiency within four orders of magnitude of human efficiency?*

  5. (WIP) When will unsupervised learning methods achieve human level performance on image classification?

  The more questions the better, so please make suggestions. Of course we have to avoid burdening the good folks working at metaculus, so 8-10 questions is probably the maximum I’d be willing to personally submit.

  *I am not very familiar with reinforcement learning, so input here would be particularly helpful! What is the best way to operationalize this question? How many orders of magnitude? Is there a relevant benchmark? etc. I’d be happy for someone else to take the credit, and post the question themselves as well!

ja­cobp­fau’s Quick takes

• jacobpfauDec 25, 2019, 5:27 PM

  2 points

  0 ∶ 0

  in reply to: ianps’s comment on: Tak­ing ad­van­tage of cli­mate change con­cerns to chan­nel dona­tions to EA-recom­mended or­ga­ni­za­tions at low marginal cost (pro­posal and call for more re­search)

  True, it seems like solar-aid’s own estimate these days suggests around $5 per tonne. I can’t find a more recent external review unfortunately.

• jacobpfauDec 25, 2019, 11:30 AM

  2 points

  0 ∶ 0

  on: Tak­ing ad­van­tage of cli­mate change con­cerns to chan­nel dona­tions to EA-recom­mended or­ga­ni­za­tions at low marginal cost (pro­posal and call for more re­search)

  This is a great point!

  I’m somewhat hesitant about the CATF recommendation though. After a brief skim of the Founder’s Pledge report, looks like they broke down CATF’s efforts into three projects which have/​are working out well. If we assume that Founder’s Pledge reviewed a number of public advocacy/​lobbying groups there’s likely to have been a multiple testing issue. In that light, the retrospective cost of CO2e/​$ estimate may not be predictive of their future CO2e/​$ ratio. That said so long as the majority of their funds go into these current efforts, the first concern doesn’t apply. A quick ctrl+f didn’t pull up anything on their funding gap either.

  I personally recommend SolarAid to friends. An old estimate put them at $1.34 per tonne. SolarAid’s project may be easier to explain briefly as well?

  Whatever you recommend, in my experience, telling non-EA people it only costs 10-30 dollars to offset your carbon for the year has gotten very enthusiastic responses!