Luise

Karma: 549

Currently researching how involved the US government may get in the development of AGI and by what method. I try to learn from history and generalize from past cases of US government involvement in developing general-purpose technologies. (As a participant of the Pivotal Research Fellowship.)

Previously, I researched whether cost-benefit analysis used by US regulators might stop/discourage frontier AI regulations. (Supervised by John Halstead, GovAI.)

I also sometimes worry about the big-picture epistemics of EA à la “Is EA just an ideology like any other?”.

In the past, I’ve done operations and recruiting at GovAI, CEA, and the SERI ML Alignment Theory Scholars program. My degree is in Computer Science.

Luise 4 Sep 2024 10:01 UTC
3 points
0 ∶ 0
in reply to: Tamay’s comment on: Will the US Government Control the First AGI?—Finding Base Rates
I don’t think it is clear what the “crucial step” in AGI development will look like—will it be a breakthrough in foundational science, or massive scaling, or combining existing technologies in a new way? It’s also unclear how the different stages of the reference technologies would map onto stages for AGI. I think it is reasonable to use reference cases that have a mix of different stages/‘cutoff points’ that seem to make sense for the respective innovation.
Ideally, one would find a more principled way to control for the different stages/”crucial steps” the different technologies had. Maybe one could quantify the government control for each of these stages for each technology. And assign weights to the different stages depending on how important the stages might be for AGI. But I had limited time and I think my approach is a decent approximation.

Luise 3 Sep 2024 9:45 UTC
3 points
0 ∶ 0
in reply to: Tamay’s comment on: Will the US Government Control the First AGI?—Finding Base Rates
Thank you, these are good points!
On the notion of “USG control”:
I agree that the labeling of USG control is imperfect and only an approximation. I think it’s a reasonable approximation though.
Almost all of the USG control labels I used were taken from Anderson-Samways’s research. He gives explanations for each of his labels, e.g. for the airplane he considers the relevant inventors the Wright brothers who weren’t government-affiliated at the time. It’s probably best to refer to his research if you want to verify how much to trust the labels.
You may have detailed contentions with each of these labels but you might still expect that, on average, they give a reasonable approximation of USG control. This is how I see the data.

On the list of innovations feeling arbitrary:
I share this concern but, again, I feel the list of innovations is still reasonably meaningful. As I said in the piece:
Choices regarding which stage of development and deployment to identify as “the invention” of the technology aren’t consistent. The most important scientific breakthroughs are often made some time before the first full deployment of a technology which in turn is often done before crucial hurdles to deployment at scale are overcome. This matters for the data insofar as the labeling of the invention year and the extent of USG control aren’t applied consistently to the same stage of development and deployment. This should not be detrimental, considering it’s not clear what the crucial stage of development and deployment for AGI will be either.^[6] Nevertheless, it makes the data less precise and more of an approximation.
(I was trying to get at something similar as your concern about “specific versus broad” innovations. “Early stage development versus mass-scale deployment” is often pretty congruent with “specific scientific breakthrough” versus “broad set of related breakthroughs and their deployment”.)
The reason why many other important innovations are not on the list is mostly time constraints.

Luise 1 May 2024 22:59 UTC
1 point
0 ∶ 0
on: Killing the moths
I’ve got moths in my flat right now and this post made me take solving this more seriously. Thank you!

Luise 20 Jun 2023 9:41 UTC
2 points
0 ∶ 0
on: Ingroup Deference
I found the framing of “Is this community better-informed relative to what disagreers expect?” new and useful, thank you!
To point out the obvious: Your proposed policy of updating away from EA beliefs if they come in large part from priors is less applicable for many EAs who want to condition on “EA tenets”. For example, longtermism depends on being quite impartial regarding when a person lives, but many EAs would think it’s fine that we were “unusual from the get-go” regarding this prior. (This is of course not very epistemically modest of them.)
Here are a more not-well-fleshed-out, maybe-obvious, maybe-wrong concerns with your policy:
- It’s kind of hard to determine whether EA beliefs are weird because we were weird from the get-go or because we did some novel piece of research/thinking. For example, was Toby Ord concerned about x-risks in 2009 because he had unusual priors or because he had thought about novel considerations that are obscure to outsiders? People would probably introduce their own biases while making this judgment. I think you could even try to make an argument like this about polyamory.
- People probably generally think a community is better-informed than expected when spending more time engaging with it. At least this is what I see empirically. So for people who’ve engaged a lot with EA, your policy of updating towards EA beliefs if EA seems better-informed than expected probably leads to deferring asymmetrically more to EA than other communities. Since they will have engaged less with other communities. (Ofc you could try to consciously correct for that.)
- I overall often have the concern with EA beliefs that “maybe most big ideas are wrong”, just like most big ideas have been wrong throughout history. In this frame, our little inside pet theories and EA research provide almost no Bayesian information (because they are likely to be wrong) and it makes sense to closely stick to whatever seems most “common sense” or “established”. But I’m not well-calibrated on how true “most big ideas are wrong” is. (This point is entirely compatible with what you said in the post but it changes the magnitude of updates you’d make.)
Side-note: I found this post super hard to parse and would’ve appreciated it a lot if it was more clearly written!

Luise 8 Jun 2023 9:47 UTC
18 points
7 ∶ 5
in reply to: Guy Raveh’s comment on: Large epistemological concerns I should maybe have about EA a priori
My impression is that others have thought so much less about AI x-risk than EAs and rationalists, and for generally bad reasons, that EAs/rats are the “largest and smartest” expert group basically ‘by default’. Unfortunately with all the biases that come with that. I could be misunderstanding the situation tho.

Luise 29 May 2023 9:19 UTC
1 point
0 ∶ 0
in reply to: tlevin’s comment on: How I solved my problems with low energy (or: burnout)
Thanks a lot, I think it’s really valuable to have your experience written up!

Luise 26 May 2023 9:56 UTC
4 points
0 ∶ 0
in reply to: MaxRa’s comment on: How I solved my problems with low energy (or: burnout)
Thanks Max!
Sounds like a plausible theory that you lost motivation because you pushed yourself too hard. I’d also pay attention to “dumber” reasons like maybe you had more motivation from supervisors/social environment/more achievable goals in the past.
Similar to my call to take a vacation, maybe it’s worth it for you to only do motivating work (like a side project) for 1.5 weeks and see if the tiredness disappears.
All of this with the caveat that you understand your situation a lot better than I do ofc!

Luise 25 May 2023 10:44 UTC
3 points
0 ∶ 0
in reply to: jeffreyyun’s comment on: How I solved my problems with low energy (or: burnout)
yes! From reading about burnout it can seem like it only happens to people who hate their job, work in bad environments, etc. But it can totally happen to people who love their job!

Luise 25 May 2023 10:42 UTC
1 point
0 ∶ 0
in reply to: rime’s comment on: How I solved my problems with low energy (or: burnout)
thanks and big agree; I want to see many more different experiences of energy problems written up!

Luise 25 May 2023 10:41 UTC
3 points
0 ∶ 0
in reply to: Jeroen Willems🔸’s comment on: How I solved my problems with low energy (or: burnout)
the causes of people’s energy problems are so many and varied! It would be great to have many different experiences written up, including stress and anxiety-induced problems.

Thanks for feedback re:appendix, will see if others say the same :)

Luise 14 Apr 2023 12:50 UTC
5 points
3 ∶ 1
on: My updates after FTX
Optimistic note with low confidence:
In my impression, SBF thought he was doing an ‘unpalatable’ but right thing given the calculations (and his epistemic immodesty). Promoting a central meme in EA like “naïve calculations like this are too dangerous and too fallible” might solve a lot of the issue. I think dangerously-optimize-y people in EA are already updating in this direction as a result of FTX. Before FTX, being “hardcore” and doing naïve calculations was seen as cool sometimes. If we correct hard for this right now, it may be less of an issue in the future.
2 main caveats:
1. The whole “don’t do naïve calculations” idea is quite complex and not easy to communicate. This may make it hard to correct for it.
2. The movements of the memes of a space as large and complex as EA are probably hard to predict. All sorts of crazy things might happen and I have no clue. Like for example there could be a new counterculture part of EA that becomes super dangerously-optimize-y. (But at least they would face more of an uphill battle in this world.)

Luise 27 Oct 2022 10:29 UTC
3 points
0 ∶ 0
in reply to: Habryka’s comment on: On funding, trust relationships, and scaling our community [PalmCone memo]
ah, the thing about fragile cooperative equilibria makes sense to me.
I’m not as sure as you that this shift would happen to core EA though. I could also imagine that current EAs will have a very allergic reaction to new, unaligned people coming in and trying to take advantage of EA resources. I imagine something like a counterculture forming where aligned EAs start purposefully setting themselves apart from people who’re only in it for a piece of the pie, by putting even more emphasis on high EA alignment. I believe I’ve already seen small versions of this happening in response to non-altruistic incentives appearing in EA.
The faster the flood of new people and change of incentives happens, the more confident I am in this view. Overall, I’m not extremely confident at all though.
On your last point, if I understand this right this is not the thing you’re most worried about though? Like, these people hijacking EA are not the mechanism by which EA may collapse in your view?

Luise 26 Oct 2022 16:10 UTC
3 points
0 ∶ 0
on: On funding, trust relationships, and scaling our community [PalmCone memo]
It’s unclear to me whether you are saying that the potentially huge number of new people in EA will try to take advantage of EA resources for personal gain or that WE, who are currently in EA for altruistic reasons, will do so. The former sounds likely to me, the latter doesn’t.
I might be missing crucial context here since I’m not familiar with the Thielosphere and all that, but overall I also don’t think a huge number of new, unaligned people will be the downfall of EA. As long as leadership, thought-leaders, and grantmakers in EA stay aligned, it may be harder for them to determine whom to give that grant (or that stamp of approval), but wouldn’t that just simply lead to less grants? Which seems bad but not like the end?
Or are you imagining highly intelligent people with impressive resumes who strategically aim to hijack EA resources for their aims and get into important positions in EA?

Luise 21 Sep 2022 17:28 UTC
1 point
0 ∶ 0
in reply to: elisemb’s comment on: A bunch of reasons why you might have low energy (or other vague health problems) and what to do about it
Thanks for commenting this. Any tips for how to get disordered breathing diagnosed reliably?

Luise 9 Aug 2022 1:15 UTC
1 point
0 ∶ 0
in reply to: Yhw’s comment on: What’s the theory of change of “Come to the bay over the summer!”?
pls make one!

Luise 21 Jul 2022 0:51 UTC
9 points
0 ∶ 0
on: EAs should use Signal instead of Facebook Messenger
If effective altruists’ messages are hacked, taken out of context, and publicly revealed, it could substantially and even permanently harm the movement. Consider the example of John Podesta, chair of Hillary Clinton’s 2016 presidential campaign. Many of his emails, including those that made Clinton and her campaign look bad, were obtained by hackers in a data breach and published in Wikileaks.
How likely is it that someone would target the EA movement by hacking messages and taking them out of context?

Luise 8 Jun 2022 21:20 UTC
19 points
0 ∶ 0
in reply to: Jan_Kulveit’s comment on: What’s the theory of change of “Come to the bay over the summer!”?
I agree with you, being “a highly cool and well networked EA” and “do things which need to be done” are different goals. This post is heavily influenced by my experience as a new community builder and my perception that, in this situation, being “a highly cool and well networked EA” and “do things which need to be done” are pretty similar. If I wasn’t so sociable and network-y, I’d probably still be running my EA reading group with ~6 participants, which is nice but not “doing things which need to be done”. For technical alignment researchers, this is probably less the case, though still much more than I would’ve expected.

Luise 24 Mar 2022 12:13 UTC
2 points
0 ∶ 0
on: Update from Open Philanthropy’s Longtermist EA Movement-Building team
Hi Claire,
what are your thoughts on “going one meta-level up” and trying to build the meta space? Specifically creating opportunities like UGAP, the GCP internships, or running organisers’ summits to get more and better community builders? I’m unsure but I thought this might be at odds with some of the points you raised, e.g., that we might neglect object-level work and its community-building effect. I’d love to hear your thoughts!

Luise 9 Feb 2022 22:45 UTC
1 point
0 ∶ 0
on: Sustained effort, potential, and obligation
So when I entered university I was probably capable of doing 0.5 hours per day on average.
hahah I feel this

Luise 7 Feb 2022 16:45 UTC
6 points
0 ∶ 0
on: Call for action for German university groups!
(I’m an organiser at EA Edinburgh and from Germany.)
Yes. Your point about the social culture at German universities seems crucial. The lack of an extensive extracurricular life in and around the university should lead to smaller EA groups (because of people not looking for student groups, less enthusiasm from organisers, lacking knowledge about how to build such groups, …)
In terms of action plans, I think an important component is getting EA group organisers excited and ambitious. Communication between large, vibrant EA groups and German groups would be good for this. Show them what is possible. And then we probably need upskilling.
Apart from that, German groups probably need the same things as other groups. I think that this is mainly, again, excited organisers, who are willing to put in time. Just in case this is interesting to you, I’m thinking about running a “bootcamp” in the UK for new organisers, fellowship facilitators, etc. to get them excited about organising. (Approaches similar to this seem to have an amazing track record.) Would be happy to chat about this!