Hi, I am a Physicist, Effective Altruist and AI safety student/researcher/organiser
Resume—Linda Linsefors—Google Docs
Linda Linsefors
The Learning-by-doing AI Safety workshop is now full, but due to the enthusiasm I have received I am going to organize a second Learning-by-doing AI Safety workshop some time in October/November this year. If you want to influence when it will be you can fill in our doodle: https://doodle.com/poll/haxdy8iup4hes9xy
I am leaving the application form open. You can fill it in to show interest in the second Learning-by-doing AI Safety workshop and future similar events.
There is still room for more participants at TAISU, but sleeping space is starting to fill up. The EA Hotel dorm rooms are almost fully booked. Fore those who don’t fit in the dorm or want some more privet space, there are lots of near by hotel. However since TAISU happens to be on a UK bank holiday, these might fill up too.
For me the most important consideration is flexibility, i.e. not having to wait for a grant comity to make up their mind before I can start. For this problem, the hybrid model is no better than a grant, unless it can speed up the application process by an order of magnitude.
Also, any major improvement in evaluation time also needs to be combined with running applications. Otherwise the applicant still have to wait a few months (on expectation) for the next grant round to come around. I guess there is a reason no major grant agency has running applications?
Let’s say you have running applications that evaluation time is proportional to the amount of money (I have no idea of that is true). Then funding something 20% up front would take a 1⁄5 of the time to evaluate, which is not bad. But I’m not sure how useful that would be for the applicant. I think most applications will fall into one of two cases:
1) The applicants is fine with taking the risk of not getting paid anything
2) The applicant needs to know that the majority of the budget will get covered
For example, if I run an event online or at CEEALAR (formally EA Hotel), I’m ok with taking the risk of not getting paid, I’ll just adjust my calculations for deciding if I want to run that event again. But if I run an event that has actual cost for me (other than my time), like travel and/or venue, then I need to know that those cost will be covered, 20% up front is probably not good enough.
But if an applicant is willing to put up with the hassle of applying for a grant (because they need the guaranteed money), then having some token amount depend on the outcome might be motivating. However, this also means that the grant maker need to evaluate the project twice, which takes even more time. But if I imagine myself as the recipient, I would very much welcome a post project evaluation from the grant maker, if this is something they want to do.
I think a improvement of this suggestion, is to cover any necessary cost in an initial grant (weather that be 0% or 90%). And offer an additional payment as a bonus if the project is successful. Where projects that request 0% in advance are “auto accepted” for the first half (which is £0). There might still be some point to pre-register projects with the grant makers, I think? Maybe they can say what metrics to track for the post evaluation? E.g. what questions they want in an event evaluation survey, and similar?
Prices and impact purchase are very similar. I would say that impact purchase would be an improvement though.
For an impact purchase the amount of money is decided based on how good impact of the project was. For a price, the price money usually set in advance, and there is often a winner takes it all dynamic.
Prices feels aversive to me becasue I win by being better than others, which means I’m disincentivised to help. This is fine whenever I don’t expect to win anyway, like with the forum prices. Very few blogpost gets awarded. Because the base rate is low I don’t feel like I’m loosing much by helping others. But if I’m trying to make a living out of selling impacts more regularly, I would not want this comparative aspect.
Although, it is not that simple, becasue there is inevitably competition in the market for selling impact. So in practice maybe it is not so different? I’m honestly a bit confused about if I think there is a real difference or not.
Maybe the difference is that a price often have a narrower focus, which pushes the competition to be between very similar work.
I would hate there to be an EA events price, because that would make me view fellow organisers as my competition, and those are exactly the people I should exchange experience and advise with. It would be much less bad to compete on “team organisers” against everyone else, about who can have the biggest overall impact.
Lest assume for now that impact is clustered as the tails.
(I don’t have a strong prior, but this at least don’t seem implausible to me)
Then how would you like to spend funding? Since there will limited amour of money, what is your motivation for giving the low impact projects anything at all?
Is it to support the people involved to do keep working, and eventually learn and/or get lucky enough to do something really important?
I sort of agree with this, but I want to add some things.
I agree that money is not the best motivator. If I was trying to solve [people are not motivated enough] I would probably suggest some community measure rather than a new funding structure.
Money is for buying people the time (i.e. not having do some day-job just to earn a living), or funding other things they need for whatever awesome project they are doing.
However money can defiantly influence motivation. 80k mentions list “Pay you feel is unfair.” as one of four “major negatives” which “tend to be linked to job dissatisfaction.”
https://80000hours.org/2014/09/update-dont-follow-your-passion/
In this situation I would think you evaluated my project as “small impact” which is possibly useful information, depending on how reliable I think you evaluation is. If I trust your judgement, this would obviously be discouraging, since I though it was much more impressive. But in the end I rather be right then proud, so that I can make sure to do better things in the future.
How I react would also depend on if your £10 is all I get, or if I get £10 each from lots of people, becasue that could potentially add up, maybe?
What it mainly comes down to in the end is: Do I get paid enough to sustainably afford to do this work. Or do I need to focus my effort on getting a paid job instead.
If you are a funder, and you think what I’m doing is good, but not good enough to pay me a liveable wages, then I’d much prefer that you don’t try to encourage me, but instead just be upfront about this. Encouraging people to keep up an unsustainable work situation is exploitative and will backfire in the long run.
Regardless of whether you use an impact purchase or provide funding normally, there’s always a chance that the project would have taken place regardless. However, impact purchases greatly increase this problem since the person already did the project without knowing whether or not they would be funded.
I agree. This is not a crux for me.
Another argument is that people who had an impact in the past are likely to spend the money to have an impact in the future. This might be the case, but if this is the primary vehicle through which these purchases have an impact, it might be worthwhile crafting a mechanism that more narrowly focuses on this.
I currently think “people who had an impact in the past are likely to spend the money to have an impact in the future”, is the main argument for Impact Purchase. It is possible that impact purchase is not the optimal format. I am still thinking about this. But I think it is important that “look at all the stuff I did in the past” is *enough* to get funded, no explanation of what I will do next needed, becasue that is too inflexible (see my post).
And if we are going to trust people and hand them money based on what they did in the past, it does make very much sense to me to trust them to the extent of how much good they done in the past. We want to give our trust and money to people who are competent (can successfully complete their plans) and have good judgement (have a good idea of which projects are potentially very important). Impact tracks both those metric.
When making a guess, one should start with the outside view, and then adjust from there. In most cases, the best outside view for what a person will do in the future, is what have they done in the past. Then maybe we want to do some adjustment from there?
If a person seem very reckless, maybe don’t fund them. Or if you think an outcome was mostly bad luck, fund them more than just impact purchase. But in most cases I would suggest straight up impact purchase, because anything else is is really hard and you’ll probably get the adjustments wrong.
Another issue with impact purchases is the potential to harm relations between people, lead to bitterness or to demotivate people.
This is already happening. People do start things in the hope of getting funding later along the way. And it is not just about projects. There is a reason posts complaining about how EA treats people gets very upvoted.
I think everything would be much better if we stopped worrying so much and started treating people as adults.
If a person does something that is good, but not good enough to be worth paying for, than this means that EA rather have money than this work. This means that if this person want to maximise impact, they should find something better to do, or up-skill, or switch to earning to give. Under most circumstances they should not keep doing that thing, so we should not encourage it.
(People are of course allowed to things that are less than optimal, if this is what they want. I am very much in favour of people doing what they want. But we should not pretend that what they do is more important than it is, just for encouragement.)
I went to my first EAG in 2016. Earlier that year I had finished my physics PhD and found out about AI Safety. At EAG almost everyone I talked to encouraged me to retrain to do AI Safety, and I felt super motivated. But I was running out of saving, so I asked for some money to take me through the transition, and most people just thought that I was weird for asking. That was demotivation as hell. These people where encouraging me to use my last savings to retrain to a risky career, but putting in their money was out of the question. This told me that they considerer spending my time and my resources to be costless. I was seen as a tool to be used, not an allay.
(Things have gotten better for me after that, but this is still a painful memory. And to be honest, I am still a bit bitter about it.)
People are not dumb. If encouragement is not backed up by action or money, they will notice.
If there is not enough money to go around to pay reasonable salaries for everyone who does important work, then this means that we a money constrained, which means that we would be better of if some of these people should switch to earning to give. If this is the case we as a community should be upfront about this. People will understand and adjust.
The way to not antagonise people is to be upfront about everything.
But on the other hand, refusing to pay someone who’s good idea didn’t work out and ‘have impact’ for no fault of their own also seems exploitative!
Letting the person running the project take all the risk, might not be optimal, but I would also say it is not exploitative as long as they know this from the start.
I’m not yet sure if I think the amount of money should be 100% based on actual impact, or if we also want to reward people for project that had high expected impact but low actual impact. The main argument for focusing on actual impact is that it is less objective.
I think people who are using this type of work as a living should get paid a salary with benefits and severance. A project to project lifestyle doesn’t seem conducive to focusing on impact.
Um, I was going to argue with this. But actually I think you are right.
Something like: “We like what you have done so far, so we will hire you to keep doing good things based on your own best judgment.”
I’m not sure. The vibe I got from the original post was that it would be good to have small rewards for small impact projects?
I’m unsure what size you have in mind when you say small.
I don’t think small monetary rewards (~£10) are very useful for anything (unless lots of people are giving small amounts, or if I do lot that add up to something that matters).
I also don’t think small impact projects should be encouraged. If we respect peoples time and effort, we should encourage them to drop small impact projects and move on to bigger and better things.
I think the high impact projects are often very risky, and will most likely have low impact.
If you think that the projects with highest expected impact also typically have low success rate, then standard impact purchase is probably not a good idea. Under this hypothesis, what you want to do is to reward people for expected success rather than actual success.
I talk about success rather than impact, because for most project, you’ll never know the actual impact. By “success” I mean your best estimate of the projects impact, from what you can tell after the project is over. (I really meant success not impact from the start, probably should have clarified that some how?)
I’d say that for most events, success is fairly predictable, and more so with more experience as an organiser. If I keep doing events the randomness will even out. Would you say that events are low impact? Would you say events are worth funding?
Can you give an example of the type of high impact project you have in mind? How does your statement about risk change if we are talking about success instead?
I did get help from my parents later, so if that where their assumptions they where not wrong. But I did not know this at the time, and when I asked why I could not get funding I got answers of the type: “that is not how things are done”, which made no sense to me.
It is possible that not funding me back then where the right decision for the right reason. But since I where not told the reason, the experience for me where very discouraging and antagonising. That’s why transparency is important!
(I’m not really blaming anyone. I think that the people I where talking to did not have explicit knowledge, so where therefore not even able to answer me. But I think we can do better.)
I guess this would be a key point where we differ. I haven’t thought deeply about this, but my intuition would be that adjustments would greatly improve impact. For example, a small project extremely competently implemented and a big project poorly implemented might have the exact same impact, but the former would be a stronger signal.
In this case, the competent person can just do more great small projects and get more money.
I’m confused by this response. I answered all of this in the blogpost. Did I fail to communicate? I am not saying that you have to agree, but if you read what I wrote and still don’t understand why *I* think some times paying after a project is a good idea, that is confusing to me, and I would like to understand better what part of the blogpost you found confusing.
Whait what?
100 000 AI Safety Events?
Like 100 000 individual events?
There is a typo here right?
It can’t take more that ~50 events for every AI Safety researcher to get to know each other.
And key ideas are not seeded at a single point in time, it is something that comes together from lots of reading and talking.
There is not *the one event* that made the different and all the others where practically useless. That’s not how research work. Sure there are randomness and some meetings are more important than others.
But if it took on average 50 000 events for one such a key introduction to happen, then we might as well give up on having events. Or find a better way to do it. Otherwise we are just wasting everyone’s time.
I’m ok with hit based impact. I just disagree about events.
I think you are correct about this for some work, but not for others. Things like operations and personal assistant are multipliers, which can consistently increase the productivity of those who are served.
Events that are focused on sharing information and networking fall in this category. People in a small field will get to know each other and each others work eventually, but if there are more events it will happen sooner, which I model as an incremental improvement.
But some other events feels much more hits based not that I think of it. Anything focused on getting people started (e.g. helping them choose the right career) or events focused on ideation.
But there are other types of event that are more hit based, and I notice that I’m less interested in doing them. This is interesting. Because these events also differ in other ways, there are alternative explanations. But seems worth looking at.
Thanks for providing the links, I should read them.
(Of course everything relating to X-risk is all or nothing in therms of impact, but we can’t measure and reward that until it does not matter anyway. Therefore in terms of AI Safety I would measure success in terms of research output, which can be shifted incrementally.)
Update:
I have changed my mind quite a bit since writing this blogpost. The updates are coming from the discussions with you in the comments, so thanks for everyone discussing with me.
Everything in this comment are still work in progress. I’ll write something more formal and well though through later, when I have a more stable opinion. But my views have already change enough so that I wanted to add this update.
------------------------------------------------
What I actually want there to be is some sort of trust based funding. If I proven my self enough (e.g. by doing good work) then I get money, and no questions asked. The reason I want this is becasue of flexibility (see main post).
Giving away money = Giving away power
Impact perches has the neat structure that if I done X amount of good I get X amount worth of trust (i.e. money). This seems to be the exact right amount, because it is the most you can give away and still be protected from exploitation. If someone who are not aligned with the goal of the funder tires to use impact purchase as a money pump, they still have to do an amount of good equal to the payout they want.
But...
A project to project lifestyle doesn’t seem conducive to focusing on impact.
We actually know this form an other field. In most of academia, the law of the land is publish or perish. Someone living of impact purchases will face a similar situation, and it is not good, at least not in the long run.
I think the high impact projects are often very risky, and will most likely have low impact.
To the extent that this is true, impact purchase will not work.
In theory we could have impact investors, who funds a risky project and earn money by selling the impact of the few projects which impact reached the stars (literally and/or figuratively). But this requires an other layer which may or may not happen in reality (probably won’t happen). Also, from the perspective of the applicant, how is this any different from applying for a grant? So what have we gained?
If not impact purchase, then what?
I still would like to solve the problem of inflexibility that grants have. An actually I think the solutions already exist (to some extent).
1) Get a paid job, with high autonomy.
2) Start an organisation and fundraise. I did not think of this until now, but when orgs fundraise, they typically don’t present a plan for what they will do with the money. They mainly point towards what they have done so far, and ask for continued trust.
3) …? I’d be very interested in other suggestions. I would not be surprised if there are other obvious things I have missed.
There are also other solutions that don’t exist yet (or not very much) in EA, but could be implement by any institution or person with spare money:
a) “Trusted person”-job: A generic employment you offer to anyone who you like to keep up the good work, or something like that.
b) Support people on Ko-fi or Patreon, or similar, and generally encourage this behaviour from others too. (I know this is happening already, but not enough for people to make a living.)
Some more additions:
I) I found out what happened to impactpurchase.org
Paul Christiano (from privet email, with the permission to quote):
Basically just a lack of time, and a desire to focus on my core projects. I’d be supportive of other people making impact purchases or similar efforts work, I hope our foray into the space doesn’t discourage anyone.
II) Justin Shovelain told me (and gave me permission to share this information) that he would probably have focused more on Coronavirus stuff early on, if he though there where a way to get paid for this work.
This is another type of situation where grants are too slow.
I very much enjoyed the video. But I don’t think it would have been able to change my mind in some alternative reality where I didn’t already know about EA.
Thanks for pointing this out :)
Should be fixed now