I’m currently researching forecasting and epistemics as part of the Quantified Uncertainty Research Institute.
Ozzie Gooen(Ozzie Gooen)
Karma: 3,225
but a piece of advice that I’ve heard is that career capital for these things is only useful for getting your foot in the door (e.g. getting a coding test) and then your actually performance (rather than your resume) is what ends up getting you/not getting you the job.
In my experience, most orgs are much more excited about bringing on “senior engineers” than “junior engineers”, and often the only way to get the performance to be a “senior engineer” is to work in a company to build those skills. It doesn’t have to take long though.
I think I agree with Jack on the AI side.
If you can get a job at Anthropic, Redwood, maybe OpenAI or Deepmind or other AI companies, those might be better than most software startups.
The first 1-2 years of software engineering work can be amazing for the experience. It really can depend on what team you work with though. Some have great standards and help train junior engineers. Others don’t, and won’t teach decent practices. My intuition is that the startup you’re considering is likely good (startups with lots of respect in tech circles, like Stripe, can be great), but I suggest asking some people who work there.
Many of the EA tech groups are much smaller or very early, so won’t be able to provide the same experience. (and, if you join them anyway after, you’ll get the experience they do provide then, anyway). Just chat to people in them and see what mentorship they can provide.
2 years seems like a bit much to me. If you join the software engineering role (and it’s not directly EA), I’d suggest being really intense about it if you can. Try to learn as much as possible, as quickly as possible. Then, after 12 months, gauge how much you’re learning (and maybe, how quickly the stock options are rising in value) and decide things from then.
For me, I found that my first real “junior developer job” was incredibly interesting and useful, but the work got pretty boring after around 6 months. I left around 14 months in, and in retrospect, sort of wish I left ~2 months sooner.One other thing to note: try to learn full-stack skills, not narrow domain skills. Spending 1-2 years on a company-specific database solution is bad, doing it with similar tools to what a tiny startup would use is good (for many EA sorts of things)
Personally, I’m excited about more strong generalist EAs having 1-2 years of strong software engineering experience. Software engineering comes up all over the place. It’s a highly general-purpose skill. It would give you a bunch of options later on.
I think the QURI one is a good pass, though if I were to make it, I’d change a few details of course.
I looked over an earlier version of this, just wanted to post my takes publicly.[1]
I like making diagrams of impact, and these seem like the right things to model. Going through them, many of the pieces seem generally right to me. I agree with many of the details, and I think this process was useful for getting us (QURI, which is just the two of us now) on the same page.
At the same time though, I think it’s surprisingly difficult to make these diagrams to be understandable for many people.
Things get messy quickly. The alternatives are to make them much simpler, and/or to try to style them better.
I think these could have been organized much neater, for example, by:Having the flow always go left-to-right.
Using a different diagram editor that looks neater.
Reducing the number of nodes by maybe 30% or so.
Maybe neater arrow structures (having 90% lines, rather than diagonal lines) or something.
That said, this would have been a lot of work to do (required deciding on and using different software), and there’s a lot of stuff to do, so this is more “stuff to keep in mind for the future, particularly if we want to share these with many more people.” (Nuno and I discussed this earlier)
One challenge is that some of the decisions on the particularities of the causal paths feel fairly ad-hoc, even though they make sense in isolation. I think they’re useful for a few people to get a grasp on the main factors, but they’re difficult to use for getting broad buy-in.
If you take a quick glance and just think, “This looks really messy, I’m not going to bother”, I don’t particularly blame you (I’ve made very similar things that people have glanced over).
But the information is interesting, if you ever consider it worth your time/effort!
So, TLDR:Impact diagrams are really hard. At these levels of details, much more so.
This is a useful exercise, and it’s good to get the information out there.
I imagine some viewers will be intimidated by the diagrams.
I’m a fan of experimenting with things like this and trying out new software, so that was neat.
[1] I think it’s good to share these publicly for transparency + understanding.
Thanks so much, that’s really useful to know (it’s really hard to tell if these metaphors are useful at all), and also makes me feel much better about this. :)
Yea; lightcone is much closer to any other group I knew of before. I was really surprised by their announcement.
I think it’s highly unusual (this seems much higher than the other and previous non-ai eng roles I knew of).
I’d also be very surprised if Lightcone chose someone for $400,000 or more. My guess is that they’ll be aiming for the sorts of people who aren’t quite that expensive.
So, I think Lightcone is making a reasonable move here, but it’s an unusual move. Also, if we thought that future ea/engineering projects would have to pay $200k-500k per engineer, I think that would change the way we think about them a lot.
Yea, I was briefly familiar.
I think it’s still tough, and agree with Ben’s comment here.
https://forum.effectivealtruism.org/posts/kQ2kwpSkTwekyypKu/part-1-ea-tech-work-is-inefficiently-allocated-and-bad-for?commentId=ypo3SzDMPGkhF3GfP
But I think consultancy engineers could be a fit for maybe ~20-40% of EA software talent.
Working at an EA org to discover needs: This seems much slower than asking people who work there, no? (I am not trying to guess the needs myself)
It really depends on how sophisticated the work is and how tied it is to existing systems.
For example, if you wanted to build tooling that would be useful to Google, it would probably be easiest just to start a job at Google, where you can see everything and get used to the codebases, than to try to become a consultant for Google, where you’d ask for very narrow tasks that don’t require you to be part of their confidential workflows and similar.
I could see a space for software consultancies that work with EA orgs, that basically help build and maintain software for them.
I’m not sure what you mean by SaaS in this case. If you only have 2-10 clients, it’s sort of weird to have a standard SaaS business model. I was imagining more of the regular consultancy payment structure.
I’ll note:
When you say “paid”, do you mean full-time? I’ve found that “part-time” people often drop off very quickly. Full-time people would be the domain of 80,000 Hours, so I’d suggest working with them on this.
“no place for orgs to surface such needs beyond posting a job” → This is complicated. I think that software consultancy models could be neat, and of course, full-time software engineering jobs do happen. Both are a lot of work. I’m much less excited about volunteer-type arrangements, outside of being used to effectively help filter candidates for later hiring.
I think that a lot of people just really can’t understand or predict what would be useful without working in an EA org or in an EA group/hub. It took me a while! The obvious advice would be for people who want to really kickstart things, is to first try to work in or right next to an EA org for a year or so; then you’ll have a much better sense.
this post is not an attack on you or on your position
Thanks! I didn’t mean to say it was, just was clarifying my position.
An EA VC which funds projects based mostly on expected impact might be a good idea to consider
Now that I think about it, the situation might be further along than you might expect. I think I’ve heard about small “EA-adjacent” VCs starting in the last few years.[1] There are definitely socially-good-focused VCs out there, like 50 Year VC.
Anthropic was recently funded for $124 Million as the first round. Dustin Moskovitz, Jaan Tallinn, and the Center for Emerging Risk Research all were funders (all longtermists). I assume this was done fairly altruistically.
I think Jaan has funded several altruistic EA projects; including ones that wouldn’t have made sense just on a financial level.
[1]: Sorry for forgetting the 1-2 right names here.
I agree with (basically) all of this. I’ve been looking more into enterprise tools for QURI and have occasionally used some. As EA grows, enterprise tools make more sense for us.
I guess this seemed to me like a different topic, but I probably should have flagged this somewhere in this post.
On Guesstimate in particular, I’m very happy for other groups to use different tools (like Analytica, Causal and probabilistic programming languages. Normally when I talk to people about this, I wind up recommending other options. All that said, I think there are some areas where some smart programming efforts on our behalf could go a long way. I think the space of “monte carlo” tools is still quite small, and I’d like to see other efforts in it (like more startups).
One issue with what you mention is that I expect that managing a bunch of corporate licenses will be a pain. It wouldn’t be great if some smart people can’t see or work with the relevant information because their small team doesn’t have a license. So in some situations, it’s worth it, but if we can (cheaply) get/use open-source tools and standards, that can also be preferable.
I’m really happy to see this posted and to see more discussion on the topic.
However, I strongly disagree with him on what kinds of projects we should focus on. Most of his examples are of tools or infrastructure for other efforts. I think that as a community we should be even more ambitious—I think we should try to execute multiple tech-oriented R&D projects (not necessarily software-oriented) that can potentially have an unusually large direct impact.
This is a good point. My post had a specific frame in mind, of “Tech either for EAs or funded mostly by EAs”.
“Tech startups created by EAs” is a very different category, and I didn’t mean to argue that it should be less important. We’ve already seen several tech startups by EAs (FTX, Wave, as you mention); which is one reason why I was trying to draw attention on the other stuff. I’ve also been part of 3 EA startups earlier on that were more earning-to-give focused. (Guesstimate was one)
I didn’t mean to argue that “Tech either for EAs or funded mostly by EAs” were more important than “Tech startups by EAs”. The latter has been a big deal and probably will continue to be so, in large part because of opportunity costs (which I wrote about here).
funding situation in EA is currently very unfavorable to such efforts.
Most “Tech startups” already have an existing ecosystem of funding in the VC world. It’s unclear to me where and how EA Funders could best encourage these sorts of projects. I could picture there being a EA VC soon; there are starting to be more potential things to fund.
Sigh… sorry;
This is a question post, but it’s more specific than my post. It’s asking groups what their needs are, which will result in different answers than the sorts of ideas I provided.
The ideas I gave weren’t ones that were explicitly asked for. They were instead ones I’ve noticed, and, having spent a while investigate, think they would be good bets. Many are more technical/abstract than I’d expect people would understand, especially when thinking “what are my software needs”
In my experience, this is one nice way of coming up with ideas, but it’s definitely not the only way.
I think this might be getting into the weeds though. The TLDR is that I expect that this question will be useful for surfacing a subset of ideas from the community, but it doesn’t seem like the be-all-end-all of feedback for software projects.
Maybe it could be it’s own post? Like, we write a Question post, and write all of the options as answers. We could do that after this one is live for a few days, and include the top ideas in it.
I’m happy with others doing it, but it’s a whole lot of ideas, so it feels to me like it would get messy. Maybe there’s some way to use a more formal survey or identify some other software solution.
I also would very much want others to suggest ideas. (Like in this post!) I wasn’t trying to make any sort of definitive list, just a generative one.
I think it would be cool for someone to create an “engineering agenda” that entrepreneurial software developers could take ideas from and start working on, analogous
I think my hunch is that this is almost like asking for an “entrepreneur agenda”. There are really a ton of options.
I’m happy to see people list all the ideas they can come up with.
I imagine “agendas” would be easier to rigorously organize if you limit yourself to a more specific area. (So, I’d want to see many “research agendas” :) )
Possibly you are planning this for later posts in this sequence already
My main area is forecasting and evaluation. (A la QURY). The plan is to spend time writing up my thoughts on it and then use that to describe an agenda of some kind. This is more cause-specific than skill-specific, but maybe 2/3rds of the work would be for software engineering.
Noted!
these projects generally seem to fail not because of software engineering but because of some non-technical thing
Agreed, though this seems mainly for getting them off the ground (making sure you find an important problem). Software startups also have this problem; and there’s a lot of discussion & best practices about the right kinds of people & teams for software startups.
I think this is a major factor. From what I can tell, some people have very easy times getting EA jobs, and some have very hard times getting EA jobs. This in itself really isn’t much information; we’d really need many stats to get a better sense of things.
For what it’s worth, I wouldn’t read this as, “the people who have a hard time… are just bad candidates”. It’s more that EA needs some pretty specific things, and there are some sorts of people for who it’s been very difficult to find a position; even though some of these people are quite brilliant in many ways.