WillPearson

Karma: 108

WillPearson 27 Sep 2017 20:01 UTC
8 points
0 ∶ 0
on: Personal thoughts on careers in AI policy and strategy
I would broadly agree. I think this is an important post and I agree with most of the ways to prepare. I think we are not there yet for large scale AI policy/strategy.

There are few things that I would highlight as additions. 1) We need to cultivate the skills of disentanglement. Different people might be differently suited, but like all skills it is one that works better with practice and people to practice with. Lesswrong is trying to place itself as that kind of place. It is having a little resurgence with the new website www.lesserwrong.com. For example there has been lots of interesting discussion on the problems of Goodheart’s law, which will be necessary to at least somewhat solve if we are to get AISafety groups that actually do AISafety research and don’t just optimise some research output metric to get funding.

I am not sure if lesswrong is the correct place, but we do need places for disentanglers to grow.

2) I would also like to highlight the fact that we don’t understand intelligence and that there have been lots of people studying it for a long time (psychologists etc) that I don’t think we do enough to bring into discussing artificial versions of the thing they have studied. Lots of work on policy side of AI safety models it as utility maximimising agent in the economic style. I am pretty skeptical that is a good model of humans or of the AIs we will create. Figuring out what better models might be, is on the top of my personal priority list.

Edited to add 3) It seems like a sensible policy is to fund a competition in the style of at super forecasting aimed at AI and related technologies. This should give you some idea of the accuracy of peoples view on technology development/forecasting.

I would caution that we are also in the space of wicked problems so it may be there is never a complete certainty of the way we should move.

WillPearson 28 Dec 2019 12:44 UTC
7 points
0 ∶ 0
in reply to: Linch’s comment on: 8 things I believe about climate change
On 1) not being able to read the full text of the impactlab report, but it seem they just model the link between heat and mortality, but not the impact of heat on crop production causing knock on health problems. E.g. http://dels.nas.edu/resources/static-assets/materials-based-on-reports/booklets/warming_world_final.pdf suggests that each degree of warming would reduce the current crop yields by 5-15%. So for 4 degrees warming (baseline according to https://climateactiontracker.org/global/temperatures/ ), this would be 20-60% of world food supply reduction.
If governments stick to their policies (which they have been notoriously bad at so far) then the reduction would only be 10-30%. I’d expect even a 10% decrease to have massive knock on effects to the nutrition and mortality of the world. I expect that is not included in the impact lab report because it is very hard to have papers that encompass the entire scope of the climate crisis.
Of course there could be a lot of changes to how and where we grow crops to avoid these problems, but making sure that we manage this transition well, so that people in the global south can adopt the appropriate crops for whatever their climate becomes seems like something that could use some detailed analysis. It seems neglected as far as I can tell, there may be simple things we can do to help. It is not mainstream climate change mitigation though, so might fit your bill?

WillPearson 17 Jun 2017 11:32 UTC
5 points
0 ∶ 0
on: Cognitive Science/Psychology As a Neglected Approach to AI Safety
There has recently been an effort started to make the pipeline better for getting people up to speed with AGI safety. I’m trying to champion a broad view of AGI safety including psychology.

Would anyone be interested in providing digested content? It would also be good to have an exit for the pipeline for psychology people interested in AGI. Would that be FHI? Who else would be good to talk to about what is required.

WillPearson 29 Dec 2019 13:46 UTC
4 points
0 ∶ 0
in reply to: Linch’s comment on: 8 things I believe about climate change
That said, I personally would be quite surprised if worldwide crop yields actually ended up decreasing by 10-30%. (Not an informed opinion, just vague intuitions about econ).
I hope they won’t too, if we manage to develop the changes we need to make before we need them. Economics isn’t magic
But I wanted to point out that there will probably be costs associated with stopping deaths associated with food shortages with adaptation. Are they bigger or smaller than mitigation by reducing CO2 output or geoengineering?
This case hasn’t been made either way to my knowledge and could help allocate resources effectively.

WillPearson 28 Dec 2019 21:14 UTC
4 points
0 ∶ 0
in reply to: Davidmanheim’s comment on: 8 things I believe about climate change
I’m expecting the richer nations to adapt more easily, So I’m expecting a swing away from food production in the less rich nations as poorer farmers would have a harder time adapting as there farms get less productive (and they have less food to sell). Also farmers with now unproductive land would struggle to buy food on the open market
I’d be happy to be pointed to the people thinking about this and planning on having funding for solving this problem. Who are the people that will be funding the teaching of subsistence rice farmers (of all nationalities) how to farm different crops they are not used to etc? Providing tools and processing equipment for the new crop. Most people interested in climate change I have met are still in the hopeful mitigation phase and if they are thinking about adaptation it is about their own localities.
This might not be a pressing problem now[1], but it could be worth having charities learning in the space about how to do it well (or how to help with migration if land becomes uninhabitable).
[1] https://blogs.ei.columbia.edu/2018/07/25/climate-change-food-agriculture/ suggests that some rice producing regions might have problems soon

WillPearson 27 Oct 2017 19:53 UTC
4 points
0 ∶ 0
on: Why & How to Make Progress on Diversity & Inclusion in EA
Since I’ve not seen it mentioned here, unconferences seem like a inclusive type of event as described above. I’m not sure how EAG compare.

WillPearson 27 Oct 2017 19:44 UTC
4 points
0 ∶ 0
in reply to: kbog’s comment on: Why & How to Make Progress on Diversity & Inclusion in EA

Yes, there are a few EA leftists whose main priority is to systemically reform capitalism, but not significantly more than there were in the first place, and they are a tiny group in comparison to the liberals, the conservatives, the vegans, the x-risk people, and so on. As far as I can tell, the impact of all these articles and comments in bringing leftists into active participation with EA was totally nonexistent.

I’m not sure I count or not. My work on autonomy can be seen as investigating systemic change. I’ve been to a couple of meetups and hung around this forum a bit and I can tell you why the community is not very enticing or inviting from my point of view, if you are interested.

Edit to add:

I can only talk about EA London which I went to a couple of the meetups. To preface things I had generally good interactions with people they were nice and we chatted a bit about non-systemic EA interests (which I am also interested in). There was lots of conversation and not too much holding-forth.

I was mainly trying to find people interested in discussing AI/future things as any systemic change has to take this into consideration and there is lots of uncertainty. I was asked what I was interested in by organisers and asked if anyone knew people primarily interested in AI, and I didn’t get any useful responses. At the time I didn’t know enough about EA to ask about systemic change (and wasn’t as clear on what I exactly wanted).

This slightly rambling point is to illustrate that it is hard to connect with people on niche topics (which AI seems to be in London). There probably needs to be a critical mass of people joining at once for a locality to support a topic.

I’ve joined a London EA facebook group focused on the future so I have my hopes.

That is pretty benign, a problem but not a large one. More could be done, but more could always be done.

The second, which I think might be more exclusionary, is EAG. I applied for tickets and to volunteer but I’ve heard nothing so far. I’m unsure why there is even selection on tickets.

I suspect I don’t look like lots of EAs on an application form: I don’t earn to give, but have taken a pay cut to work part time on my project, which I hope will help everyone in the long run. I may not have quite the same chipper enthusiasm.

I suspect other people interested in systemic change will look similarly different from lots of EAs, and the curation of EAG might be biased against them. If it is, then I probably have not lost out much by not going!

I mainly wrote this comment to try and give some possible reasons for the lack of a significant group interested in systemic change (despite articles/comments to the contrary). I’m not expecting EA to change, you can’t be a group for everyone and you do interesting and good things. But it is good to know some of the potential reasons why things are how they are.

Edit2: I got a polite email from Julia Wise telling me that the reason I didn’t get an invite was because London was a smaller event and that people were selected on the basis of “those who will benefit most from attending EA Global London.” It would be nicer if these things were a little more transparent, e.g. you are applicant #X we can only accept #Y applicants, to give you a better idea of the chances. From my own perspective for the people that are interested in current niche EA topics it is important to be able to potentially meet other people from around the world interested in their topics. EAG might not be the place for that though.

WillPearson 28 Sep 2017 11:03 UTC
4 points
0 ∶ 0
in reply to: John_Maxwell’s comment on: Personal thoughts on careers in AI policy and strategy
I agree that creativity is key.

I’d would point out that you may need discipline to do experiments based upon your creative thoughts (if the information you need is not available). If you can’t check your original reasoning against the world, you are adrift in a sea of possibilities.

WillPearson 28 Sep 2020 10:41 UTC
3 points
0 ∶ 0
on: Open Thread #45
Hi, I’m thinking about a possibly new approach to AI safety. Call it AI monitoring and safe shutdown.
Safe shutdown, riffs on the idea of the big red button, but adapts it for use in simpler systems. If there was a big red button, who gets to press it and how? This involves talking to law enforcement, legal and policy. Big red buttons might be useful for non learning systems, large autonomous drones and self-driving cars are two system that might suffer from software failings and need to be shutdown safely if possible (or precipitously if the risks from hard shutdown are less than it’s continued operation).
The monitoring side of thing asks what kind of registration and monitoring we should have for AIs and autonomous systems. Building on work on aircraft monitoring, what would the needs around autonomous system be?
Is this a neglected/valuable cause area? If so, I’m at an early stage and could use other people to help out.

WillPearson 22 Aug 2017 13:04 UTC
3 points
0 ∶ 0
on: Open Thread #38
One last one.

I’m writing more on my blog about my approach to intelligence augmentation

I’ll be coding and thinking about how to judge it’s impact this week (a lot of it depends on things like hard vs soft takeoff, possibilities of singletons and other crucial considerations). I’m also up for spending a few hours helping people with IA or autonomy based EA work, if anyone needs it?

WillPearson 22 Aug 2017 10:19 UTC
3 points
0 ∶ 0
on: Open Thread #38
I’ve been reading Superforecasting and my take away is that to have good predictions at the world you need to have a multiplicity of view points and quantify and breakdown the estimates fermi-style.

So my question is, has there been any collective attempts at model building for prediction purposes? Try and get all the hedgehogs together with their big ideas and synthesize them to form a collective fox-y model?

I know there are prediction markets, but you don’t know what information that a price has synthesized so it is hard to bet on them, if you only have a small bit of information and do not think you know better than the market as a whole.

It would seem that if we could share a pool of predictive power between us we could make better decisions about how to intervene in the world.

WillPearson 29 Dec 2019 12:38 UTC
2 points
0 ∶ 0
in reply to: Davidmanheim’s comment on: 8 things I believe about climate change
Are there any states that have committed to doing geoengineering, or even experimenting with geoengineering, if mitigation fails?
Having some publicly stated sufficient strategy would convince me that this was not a neglected area.

WillPearson 19 Dec 2018 11:22 UTC
2 points
0 ∶ 0
in reply to: Denkenberger’s comment on: Critique of Superintelligence Part 2
Let’s say they only mail you as much protein as one full human genome.
This doesn’t make sense. Do you mean proteome? There is not a 1-1 mapping between genome and proteome. There are at least 20,000 different proteins in the human proteome, it might be quite noticeable (and tie up the expensive protein producing machines), if there were 20,000 orders in a day. I don’t know the size of the market, so I may be off about that.
I will be impressed if the AI manages to make a biological nanotech that is not immediately eaten up or accidentally sabotaged by the soup of hostile nanotech that we swim in all the time.
There is a lot of uranium in the sea, only because there is a lot of sea. From the pages I have found, there is only 3 micrograms of U per liter, and 0.72 percent is U235. To get the uranium 235 (80% enriched 50Kg bomb) required for a single bomb you would need to process roughly 18 km3 of sea water or 1.8 * 10^13 liters.
This would be pretty noticeable if done in a short time scale (you might also have trouble with diluting the sea locally if you couldn’t wait for diffusion to even out the concentrations globally).
To build 1 million nukes you would need more sea water than the Mediterranean (3.75 million km3)

WillPearson 23 Jul 2018 1:00 UTC
2 points
0 ∶ 0
in reply to: Kit’s comment on: Impact Investing—A Viable Option for EAs?
There might be a further consideration, people might not start or fund impactful startups if there wasn’t a good chance of getting investment. The initial investors (if not impact oriented), might still be counting on impact oriented people to buy the investment. So while each individual impact investor is not doing much in isolation, collectively they are creating a market for things that might not get funded otherwise. How you account for that I’m not sure.

WillPearson 25 Jun 2018 19:12 UTC
2 points
0 ∶ 0
in reply to: Brian Wang’s comment on: Informational hazards and the cost-effectiveness of open discussion of catastrophic risks

Ah right. I suppose the unilateralist’s curse is only a problem insofar as there are a number of other actors also capable of releasing the information; if you are a single actor then the curse doesn’t really apply. Although one wrinkle might be considering the unilateralist’s curse with regards to different actors through time (i.e., erring on the side of caution with the expectation that other actors in the future will gain access to and might release the information), but coordination in this case might be more challenging.

Interesting idea. This may be worth trying to develop more fully?

Probably it’s best to discuss privately with a number of other trusted individuals first, who also understand the unilateralist’s curse,

I’m still coming at this from a lens of “actionable advice for people not in ea”. It might be that the person doesn’t know many other trusted individuals, what should be the advice then? It would probably also be worth giving advice on how to have the conversation as well. The original article gives some advice on what happens if consensus can’t be reached (voting/such like).

As I understand it you shouldn’t wait for consensus else you have the unilateralist’s curse in reverse. Someone pessimistic about an intervention can block the deployment of an intervention needed to avoid disaster (this seems very possible if you consider crucial considerations flipping signs, rather than just random noise in beliefs in desirability).

Would you suggest discussion and vote (assuming no other courses of action can be agreed upon)? Do you see the need to correct for status quo bias in any way?

This seems very important to get right. I’ll think about this some more.

WillPearson 24 Jun 2018 14:13 UTC
2 points
0 ∶ 0
in reply to: Brian Wang’s comment on: Informational hazards and the cost-effectiveness of open discussion of catastrophic risks

My understanding is that it applies regardless of whether or not you expect others to have the same information. All it requires is a number of actors making independent decisions, with randomly distributed error, with a unilaterally made decision having potentially negative consequences for all.

Information determines the decisions that can be made. For example you can’t spread the knowledge of how to create effective nuclear fusion without the information on how to make it.

If there is a single person with the knowledge of how to create safe efficient nuclear fusion they cannot expect other people to release it on their behalf. They may expect it to be net positive but they also expect some downsides and are unsure of whether it will be net good or not. To give a potential downside of nuclear fusion, let us say they are worried about creating excess heat over what the earth can dissipate due to widescale deployment in the world (even if it fixes global warming due to trapping solar energy, it might cause another heat related problem). I forget the technical term for this unfortunately.

The fusion expert(s) cannot expect other people to release this information for them, for as far as they know they are the only people making that exact decision.

I’m also not sure how the strategy of “preemptively release, but mitigate” would work in practice. Does this mean release potentially dangerous information, but with the most dangerous parts redacted? Release with lots of safety caveats inserted? How does this preclude the further release of the unmitigated info?

What the researcher can do is try and build consensus/lobby for a collective decision making body on the internal climate heating (ICH) problem. Planning to release the information when they are satisfied that there is going to be a solution in time for fixing the problem when it occurs.

If they find a greater than expected number of people lobbying for solutions to the ICH problem, then they can expect they are in a unilateralist’s curse scenario. And they may want to hold off on releasing information even when they are satisfied with the way things are going (in case there is some other issue they have not thought of).

They can look to see what the other people are doing that have been helping with ICH and see if there other initiatives they are starting, that may or may not be to do with the advent of nuclear fusion.

I think I am also objecting to the expected payoff being thought of as a fixed quantity. You can either learn more about the world to alter your knowledge of the payoff or try and introduce things/insituttions into the world to alter the expected payoff. Building useful institutions may rely on releasing some knowledge, that is where things become more hairy.

I’ve not had the best luck reaching out to talk to people about my ideas. I expect that the majority of new ideas will come from people not heavily inside the group and thus less influenced by group think. So you might want to think of solutions that take that into consideration.

I’m not sure I’m fully understanding you here. If you’re saying that the majority of potentially dangerous ideas will originate in those who don’t know what the unilateralist’s curse is, then I agree –– but I think this is just all the more reason to try to –– but I think this is just all the more reason to try to spread norms of consensus.

I was suggesting that more norm spreading should be done outwards, keeping it simple and avoiding too much jargon. Is there a presentation of the unilateralist’s curse aimed at micro biologists for example?

Also as the the unilaterlist’s curse suggests discussing with other people such that they can undertake the information release, sometimes increases the expectation of a bad out come. How should consensus be reached in those situations?

Increasing the number of agents capable of undertaking the initiative also exacerbates the problem: as N grows, the likelihood of someone proceeding incorrectly increases monotonically towards 1.7 The magnitude of this effect can be quite large even for relatively small number of agents. For example, with the same error assumptions as above, if the true value of the initiative V* = −1 (the initiative is undesirable), then the probability of erroneously undertaking the initiative grows rapidly with N, passing 50% for just 4 agents.

WillPearson 23 May 2018 21:46 UTC
2 points
0 ∶ 0
on: Ineffective entrepreneurship: post-mortem of Hippo, the happiness app that never quite was
Thanks for writing this up! I’ve forwarded it to a friend who was interested in the happiness app space a while back.

I would add to the advice, from my experience, pick something not too far out of people’s comfort zones for a startup or research idea. There seems to be a horizon beyond which you don’t get feedback or help at all.

WillPearson 12 Feb 2018 21:15 UTC
2 points
0 ∶ 0
on: Open Thread #39
I’m thinking about radically more secure computer architectures as a cause area.
1. Radical architecture changes are neglected because it hard to change computer architecture
2. Bad Computer security costs a fair amount at the moment
3. Having a computer architecture that is insecure is making it hard to adopt more useful technology like Internet of Things.
I’d be interested in doing an analysis of whether it is effective altruistic cause. I’m just doing it as a hobby at the moment. Anyone interested in the same region want to collaborate?

WillPearson 23 Oct 2017 20:28 UTC
2 points
0 ∶ 0
on: Open Thread #39
I’ve posted about an approach to AGI estimation.

I would love to find collaborators on this kind of thing. In London would be great.

WillPearson 22 Aug 2017 10:04 UTC
2 points
0 ∶ 0
on: Open Thread #38
Drawdown a book on possible climate change solutions seems EA relevant. It is interesting that it only allows peer reviewed data/models in it and systematically surveys all the solutions they could find.