Building bespoke quantitative models to support decisionmakers in AI and bio. Right now that means: forecasting capabilities gains due to post-training enhancements on top of frontier foundation models, and estimating the annual burden of airborne disease in the US.
Joel Becker
Karma: 1,717
I think 333 QARYs/$1m via the CAIS framework is significantly too optimistic, for two reasons:
The CAIS framework would probably make several adjustments downwards that you have not considered here, in particular for scientist-equivalence (where research engineers are valued at 0.1x research scientists).
At the 20% time discount rate that CAIS uses for default estimates, 10 years of time-discounted research is implausible (the infinite geometric sum of time-discounted years is equal to 5 non-discounted years).
Fair enough, I agree.
I think my estimate isn’t going to be very informative—I intentionally spent more time than I might otherwise endorse working on Manifund stuff, because it was fun and seemed like good skills-building. My best guess as to how much time I would have spent on an otherwise similar process in the absence of this factor is (EDIT: there was a mistake in my BOTEC) 59 (42 to 85) hours.
Like @MarcusAbramovitch , I’d feel pretty comfortable allocating ~$1m part-time. I mean just on my existing grants I would’ve been happy to donate another ~$150k without thinking more about it! Concrete >$50k grants I had to pass up but would otherwise have wanted to fund total >$200k (extremely rough). So I’m already at >$400k (EDIT: per 5 months!) without even thinking about how my behavior or prospective grantee behavior might have changed if I had a larger pot.
That said, I think there’s a sense in which I hit strongly diminishing returns at ~$10k, albeit still above-bar. The Robert Long grant was by far my best, and I knew from day 0 that I wanted to make it. After that, a bet on me became a bet on my taste, not a bet on my private information, which seems less exciting. (Again I’m optimistic that the past 5 months was an unusually low-private-information period for me, but you see my point.)
And I’m somewhat skeptical that others had $200k-$500k/year of productive grants to make. To me it’s a bad sign that >30% of Manifund funding went to 3 projects (MATS, Apollo, and LTFF) that I wouldn’t think especially benefit from the regranting model.
Some fun lessons I learned as a junior regrantor
Ah! Thank you. That makes most sense to me both as a literal reading and as a position. Sorry Kirsten if I misread.
I took
If what you describe is actually what she told you, how dare you use it for your own gain here?
to imply something like “if the alleged victim shared private and very personal information, you should not publish it.” This still makes most sense to me as a literal reading.
(I would agree that “don’t publish plausibly false allegation [that you don’t see reason to litigate]” feels like a stronger position.)
Contradicting myself to write comments that it wouldn’t be helpful for me to sit with...
At that hourly rate, he spent perhaps ~$130,000 of Lightcone donors’ money on this. But it’s more than that. When you factor in our time, plus hundreds/thousands of comments across all the posts, it’s plausible Ben’s negligence cost EA millions of dollars of lost productivity. If his accusations were true, that could have potentially been a worthwhile use of time—it’s just that they aren’t, and so that productivity is actually destroyed. And crucially, it was very easy for him to have not wasted everybody’s time—he just had to be willing to look at our evidence.
Posting a price at which you’re willing to do investigative work does not imply that this price is your current average wage.
The lost productivity claim somewhat rubs me the wrong way. It feels like this could be used as motivated reasoning to underinvest in community norm/safety enforcement.
That said, I totally agree that if someone does have cheap ways to spare the (very real) productivity costs, they should do so.
I think the crux might be whether or not Ben did have a cheap option. My memory (maybe misremembered) is that he and Habryka felt that engaging further with Nonlinear could come with a large delay. I’m not sure if you think Ben was right to think this but disagree about whether the large delay was cheap, or if you think that Ben was wrong to think this.
I even think that this reasoning can hold true in cases where you are 100% sure that the bad thing was bad. (If the badness is sufficiently low/productivity costs sufficiently high.)
Isn’t the implication that the (EDIT: alleged) victim gave consent for Kat to share anonymously?
Kat, thank you for this post. I appreciate the very helpful/understanding manner in which it is written. I’m really sorry that you needed to invest so much into this, although I think you made the right decision in doing so.
I’ll read more fully, probably sit with this for some time, and respond properly after that. (Keeping in mind my conditional pre-commitments to signal boost and seriously engage.)
I think you’ve revealed that my thinking was muddled in the earlier response! The sequence of events from my POV is:
Before university, I did extremely little academic work. (Can expand; I really think it’s outlier-low.)
For my first 2.5 out of 3 years at university I did as close to zero work as was feasible. (For example, I attended very few lectures.)
If I sat down to try to work on this (without an impending exam in <2 weeks time), it felt like I was physically unable to work.
During this period, I spent lots of time on side projects/nascent businesses, and internships related to these things. I am describing this situation as ‘not hard working’ above because I think about ‘hard working’ as more or less meaning ‘hard working [on traditional academic or professional pursuits, not part-time, barely paid sports analytics side-projects].’ I would describe hard work on side projects as part of me being ‘intense’ or something—if you want to describe it as ‘hard working’, fair enough.
At the 2-2.5 year mark, I had been very fortunate to have obtained some strong grades. I don’t think this is false modesty—I just just had a lot of variance (on both sides). One of my friends’ parents, who had been to university in the US, encouraged me to think about US options. It seemed like a very aspirational option, and for the first time ‘a thing I could do.’ In my last 0.5 years of university I worked very hard to this end.
Still, this didn’t feel like becoming a ‘hard working person’—there was a year between me leaving university and moving to the US etc., during which I worked quite little.
Then I came to the US in June 2018 and became ~‘the kind of person who works hard on traditional academic or professional pursuits’ more or less permanently.
TL;DR “I became significantly harder-working in ~June 2018” feels true from my perspective, but depends on definitions, and in some ways isn’t as sharp as I might have communicated.
Re: novel work behaviors, some examples:
Very system 2 thinking about time allocation—maybe we should completely kill this project early, or ignore this professional request, or spend disproportionate time on this small ask from an important stakeholder, etc.
Reasoning backwards from goals to what goals imply for today’s work. I had very little feeling for thought patterns like “I want to be in position X in 10 years, which requires Y between now and then, so today I will do Z.”
Just working hard? Much harder than I’d seen people work previously.
Trying to pay close attention to how other parties think, using this model to assess how your actions will be received, and using these predictions to inform your decision about which action to take.
Maybe these seem obvious to you. They seem obvious to me now! But it all felt a bit mind-blowing for me at the time.
Looks great! Good luck on the market Zach!
I too am working on impact evaluation! Feel free to message me.
I would’ve ignored this important opportunity if not for this post (because I was not aware that UK situation had changed this year). Thank you Tyrone!
This is perfectly written and very helpful—thank you JueYan!
Jakob, I sincerely apologize for my unhelpful (or at the very least unelightening) phrases that have come across as soldier mindset/rude.
I was commenting as I would on the unshared google doc of a friend asking for feedback. But perhaps this way of going about things is too curt for a public forum. Again, I’m sorry.
(I will probably reply on the substance later; currently too busy. I think there’s a decent chance that I will agree with you that, in addition to being rude and craply communicated and coming across as soldier mindset, my previous comments reflected sloppy thinking.)
By then, this is no longer a bet on short AI timelines, but rather a bet about whether the typical consumer will realize that AI timelines are short sufficiently long enough before AI that you have time to enjoy your profits.
I get your point, but it just seems a bit 4D-chess.
If I believe that TAI is coming, it seems obvious to me that I should expect people beyond my peers to understand that TAI is coming. I could even encourage this by shouting it from the rooftops after making the bet. (The strategy might not be effective given that these views are already not well-kept secrets, but this seems to strengthen the possibility that others will understand without me shouting.) At which point we’d be in the new equilibrium.
Some possible answers that haven’t seemed to have had large effects on hard-workingness for me:
Switching roles (to the extent that this doesn’t come with one of the above factors).
Medication. (ADHD meds had a very sizeable short-term effect, but not a persistent one.)
Internal work.
Productivity hacks.
Exercise, diet, and sleep.
(Several of these had important impacts on my mood.)
I became significantly harder-working in ~June 2018, age 22-23. This was not the case beforehand and has persisted for years afterwards. Some highly-overlapping-so-hard-to-disentangle things that happened around the time:
Moving to the US.
Moving to an elite university town.
Living or working with hard-working people.
Living or working with people who were ~smarter than me.
Having professional mentors who modeled (what to me at the time were) novel work behaviors.
Being thought of as a smart/hard-working person by people I met for the first time.
Working on interesting problems.
All of these improved quite sharply around the time boundary.
Strong agree—I enjoyed Brad Delong on this point.