trevor1

Karma: 230

trevor1 Mar 19, 2025, 5:16 AM
1 point
0 ∶ 0
on: Discussion Thread: Existential Choices Debate Week
I haven’t read enough about this yet, and I need to shrink the gap between me and others who’ve read a lot about this by, like, 3 OOMs or something.

trevor1 Mar 8, 2025, 7:33 PM
1 point
0 ∶ 0
in reply to: qwertyops900’s comment on: Emergency pod: Judge plants a legal time bomb under OpenAI (with Rose Chan Loui)
For an AG, should you handwrite the letter, like with congressmember offices, or type and print it like with normal legal work?
Congressional offices often ignore typed letters because they’ve learned years ago that some people set up mills that mass produce cookie-cutter letters mimicking civic engagement, instead of the legitimate paradigm of reaching out to interested people and convincing them to write their own letters, and see handwritten letters to be a costly signal that a large number of highly engaged people are involved; but if attorney generals aren’t in an equilibria like that then they’d probably prefer typed and printed letters.

trevor1 Mar 6, 2025, 10:24 PM
1 point
0 ∶ 0
on: EA Core Skillset
Strong upvoted, I’m glad I read this and really wish I’d read this or something like this back in 2017. Finding out about some of these skills and their importance, at all, was a very long slog for me and many others; the process of trial and error often yields valuable insight, but for most of these skills, it didn’t have to happen and the waste from the delay was not worth it.

trevor1 Dec 19, 2024, 6:13 AM
3 points
2 ∶ 0
in reply to: Karthik Tadepalli’s comment on: Karthik Tadepalli’s Quick takes
It seems like there’s a scale between rubber-ducking and mentorship that a person can operate at depending on the skill differences. Furthermore, some advice-givers are better at rubberducking and others are better at mentorship, adding another dimension.
Yudkowsky’s old sequence post on Cached Thoughts is pretty brief and goes rather deep into the watering-down phenomenon you described.

Stone Age Herbalist’s notes on ant warfare and slavery

trevor1Nov 9, 2024, 2:42 AM

6 points

0 comments1 min readEA link

(x.com)

trevor1 Sep 22, 2024, 9:43 PM
1 point
0 ∶ 0
on: I’m creating a deep dive podcast episode about the original Leverage Research—would you like to take part?
How can I get an email notification for this if it comes out? It probably won’t be in your regular email newsletter so I’ll probably miss it, but I’d like to see it.

trevor1 Jul 23, 2024, 2:39 AM
4 points
0 ∶ 0
on: Peter Singer AMA (July 30th)
Do you know about your cameo in Scott Alexander’s novel Unsong? What probability would you have placed on you shifting career paths from academia to more like your Unsong character if, in the 1970s, your younger self and everyone you knew witnessed the sky shattering?
John S Wentworth wrote a 1 minute post considering whether individual innovators cause major discoveries to happen many decades or even centuries earlier than they would have without that one person, or whether they only accelerate the discovery by a few months or years before someone else would have made the advancement. Based on your impact on the philosophy scene in the 1970s and EA’s emergence decades later (the counterculture movement is considered by many to have died down during the mid-1970s which notably is around the time when some of your most famous works came out), what does your life indicate about Wentworth’s models of innovation, particularly conceptual and philosophical innovation?
What do you think about the current state of introductory philosophy education, with the ancient texts (Greek, kant, etc) being Schelling points that work great in low-trust environments, but still follow the literary traditions of the times? Do you think undergrads and intellectuals outside contemporary philosophy culture (e.g. engineers, historians, anthropologists, etc) would prefer introductory philosophy classes be restructured to produce a more logical foundations to produce innovators and reductionists like your 1970s self and less literary-analysis-minded thinking?

trevor1 Jul 10, 2024, 4:32 AM
14 points
0 ∶ 0
on: AMA: Beast Philanthropy’s Darren Margolias
One of the things I found extraordinary about MrBeast videos was how it seems like viewers come for the extreme content on the thumbnail, and then stay to see the details of how exactly people succeed at doing extraordinary things.
On an economic basis, it looks like it scales really well to find ways to do really big things (unambiguously net positive) that you can also bundle into an entertainment product that also inspires other people. I don’t know what the median viewer would think of this, but I found it really vivid to see the videos “ramp up” with more and more people getting helped per minute of the video.
Have you thought of other ways you could set up something where the complexity of the situation or the number people helped gets “ramped up” over the course of the video, or where the subjects of the story find increasingly extraordinary ways to overcome increasingly extraordinary challenges? Showing people shine brightly, being their best self and then winning for it, seems to be a common theme for the channel.

trevor1 Jun 10, 2024, 5:06 AM
6 points
1 ∶ 0
on: Summary of Situational Awareness—The Decade Ahead
One thing I really liked about it was the title: “situational awareness”. I think that phrase is very well-put given the situation, and I got pretty good results from it in conversations which were about AI but not Leopold’s paper.
I also found “does this stem from the pursuit of situational awareness” or “how can I further improve situational awareness” to be helpful questions to ask myself every now and then, but I haven’t been trying this for very long and might get tired of it eventually (or maybe they will become reflexive automatic instincts which stick and activate when I’d want them to activate; we’ll see).

trevor1 Jun 7, 2024, 12:59 AM
2 points
0 ∶ 0
in reply to: Cullen 🔸’s comment on: Cullen_OKeefe’s Shortform
I might be mistaken about this, but I thought there was a possibility that Khrushchev and others anticipated that leaders and influential people in the US/USSR and elsewhere in the world would interpret space race victories as a costly signal of strategic space superiority (while simultaneously being less aggressive and less disruptive to diplomacy than developing and testing more directly military-related technology such as Starfish Prime), and separately there was a possibility that this anticipation was a correct prediction about what stakeholders around the world would conclude about the relative power of the US and USSR (including the third world and “allied” countries which often contained hawk and dove factions and regime change etc).
Momentum behind the space race itself had died out by 1975, possibly as part of the trend described in the 2003 paper “The Nuclear Taboo” which argued that a strong norm against nuclear weapon use developed over time; during the Korean War in 1950, American generals were friendly towards the idea of using nuclear weapons to break the stalemate and ultimately decided not to, but were substantially less friendly towards nuclear weapon use by the time the Vietnam War started and since then have only considered it progressively more unthinkable (the early phases of the Ukraine War in 2022, particularly the period leading up to the invasion, might have been an example of backsliding).

trevor1 Jun 2, 2024, 11:03 PM
4 points
0 ∶ 1
in reply to: NunoSempere’s comment on: NunoSempere’s Shortform
At some point in the 90s or the 00s, the “whole of person” concept became popular in the US Natsec community for security clearance matters.
It distinguishes between a surface level vibe from a person, and trying to understand the whole person. The surface level vibe is literally taking the worst of a person and taking it out of context, whereas the whole person concept is making any effort at all to evaluate the person and the odds that they’re good to work with and on what areas. Each subject has their own cost-benefit analysis in the context of different work they might do, and more flexible people (e.g. younger) and weirder people will probably have cost-benefit analysis that change somewhat over time.
In environments where evaluators are incompetent, lack the resources needed to evaluate each person, or believe that humans can’t be evaluated, then there’s a reasonable justification to rule people out without making an effort to optimize.
Otherwise, evaluators should strive to make predictions and minimize the gap between their predictions of whether a subject will cause harm again, and the reality that comes to pass; for example, putting in any effort at all to succeed at distinguishing between individuals causing harm due to mental health, individuals causing harm due to mistakes due to unpreventable ignorance (e.g. the pauseAI movement), mistakes caused by ignorance that should have been preventable, harm caused by malice correctly attributed to the subject, harm caused by someone spoofing the point of origin, or harm caused by a hostile individual, team, or force covertly using SOTA divide-and-conquer tactics to disrupt or sow discord in an entire org, movement, or vulnerable clique; see Conflict vs mistake theory.

trevor1 May 10, 2024, 4:00 PM
12 points
0 ∶ 0
on: Potential Pitfalls in University EA Community Building
Thanks for making a post for this! Coincidentally (probably both causally downstream of something) I had just watched part of the EAG talk and was like “wow, this is surprisingly helpful, I really wish I had access to something like this back when I was in uni, so I could have at least tried to think seriously about plotting a course around the invisible helicopter blades, instead of what I actually did, which was avoiding it all with a ten-foot pole”.
I’m pretty glad that it’s an 8-minute post now instead of just a ~1-hour video.

trevor1 May 8, 2024, 1:54 PM
1 point
1 ∶ 0
in reply to: Benjamin_Todd’s comment on: The market plausibly expects AI software to create trillions of dollars of value by 2027
My bad- I should have looked into Nvidia more before commenting.
Your model looked like something that people were supposed to try to poke holes in, and I realized midway through my comment that it was actually a minor nitpick + some interesting dynamics rather than a significant flaw (e.g. even if true it only puts a small dent in the OOM focus).

trevor1 May 6, 2024, 2:19 PM
7 points
2 ∶ 0
on: The market plausibly expects AI software to create trillions of dollars of value by 2027
Stock prices represent risk and information asymmetry, not just the P/E ratio.
The big 5 tech companies (google, amazon, microsoft, facebook, apple) primarily do data analysis and software (with apple as a partial exception). That puts each of the five (except apple to some extent, as their thread to hang on is iphone marketing) at the cutting edge of all the things that high-level data analysis is needed for, which is a very diverse game where each of the diverse elements add in a ton of risk (e.g. major hacks, data poisoning, military/geopolitical applications, lighting-quick historically unprecedented corporate espionage strategies, etc).
The big 5 are more like the wild west, everything that’s happening is historically unprecedented and they could easily become the big 4, since a major event e.g. a big data leak could cause a staff exodus or a software exodus that allows the others to subsume most of their market share (imagine how LLMs affected Google’s moat for search, except LLMs are just one example of historical unprecedence (that EA happens to focus way closer on, relative to other advancements, than wall street and DC), and most of the big 5 companies are vulnerable in ways as brutal and historically unprecedented as the emergence of LLMs).
Nvidia, on the other hand, is exclusively hardware and has a very strong moat (obviously semiconductor supply chains are a big deal here). This reduces risk premiums substantially, and I think it’s reasonable likely that they would even be substantially lower risk per dollar than holding stock diversified between all 5 of the big 5 tech companies combined; I think the big 5 set a precedent that the companies making up the big leagues are each very high risk including in aggregate and Nvidia’s unusual degree of stability, while also emerging on the bigleagues stage without diversifying or getting great access to secure data, might potentially shatter the high-risk bigtech company investment paradigm. I think this could cause people’s p/e ratio for Nvidia to maybe be twice or even three times higher than it should, if they depend heavily on comparing Nvidia specifically to google, amazon, facebook, microsoft, and apple. This is also a qualitative risk that can also spiral into other effects e.g. a qualitatively different kind of bubble risk than what we’ve seen from the big 5 over the last ~15 years of the post-2008 paradigm where data analysis is important and respected.
tl;dr Nvidia’s stable hardware base might make comparisons to the 5 similarly-sized tech companies unhelpful, as those companies probably have risk premiums that are much higher and more difficult to calculate for investors.

trevor1 Apr 15, 2024, 7:32 PM
2 points
0 ∶ 0
in reply to: Clare_Diane’s comment on: Reasons for optimism about measuring malevolence to tackle x- and s-risks
Ah, I see; for years I’ve been pretty pessimistic about the ability of people to fool systems (namely voice-only lie detectors facilitated by large numbers of retroactively-labelled audio recordings of honest and dishonest statements in the natural environments of different kinds of people) but now that I’ve read more about humans genetic diversity, that might have been typical mind fallacy on my part; people in the top 1% of charisma and body-language self-control tend to be the ones who originally ended up in high-performance and high-stakes environments as they formed (or forming around then, just as innovative institutions form around high-intelligence and high-output folk).
I can definitely see the best data coming from a small fraction of the human body’s outputs such as pupil dilation; most of the body’s outputs should yield bayesian updates but that doesn’t change the fact that some sources will be wildly more consistent and reliable than others.

trevor1 Apr 13, 2024, 11:32 PM
2 points
0 ∶ 0
in reply to: Clare_Diane’s comment on: Reasons for optimism about measuring malevolence to tackle x- and s-risks
Why are you pessimistic about eyetracking and body language? Although those might not be as helpful in experimental contexts, they’re much less invasive per unit time, and people in high-risk environments can agree to have specific delineated periods of eyetracking and body language data collected while in the high-performance environments themselves such as working with actual models and code (ie not OOD environments like a testing room).
AFAIK analysts might find uses for this data later on, e.g. observing differences in patterns of change over time based on based on the ultimate emergence of high risk traits, or comparing people to others who later developed high risk traits (comparing people to large amounts of data from other people could also be used to detect positive traits from a distance), spotting the exact period where high risk traits developed and cross-referencing that data with the testimony of a high risk person who voluntarily wants other high risk people to be easier to detect, or depending on advances in data analysis, using that data to help refine controlled environment approaches like pupillometry data or even potentially extrapolating it to high-performance environments. Conditional on this working and being helpful, high-impact people in high-stakes situations should have all the resources desired to create high-trust environments.

trevor1 Apr 13, 2024, 3:00 PM
18 points
6 ∶ 2
on: To what extent & how did EA indirectly contribute to financial crime—and what can be done now? One attempt at a review
The crypto section here didn’t seem to adequately cover a likely root cause of the problem.
The “dark side” of crypto is a dynamic called information asymmetry; in the case of Crypto, it’s that wealthier traders are vastly superior at buying low and selling high, and the vast majority of traders are left unaware of how profoundly disadvantaged they are in what is increasingly a zero-sum game. Michael Lewis covered this concept extensively in Going Infinite, the Sam Bankman-Fried book.
This dynamic is highly visible to those in the crypto space (and quant/econ/logic people in general who catch so much as a glimpse), and many elites in the industry like Vitalik and Altman saw it coming from a mile away and tried to find/fund technical solutions e.g. to fix the zero-sum problem e.g. Vitalik’s d/acc concept.
It was very clear that SBF also appeared to be trying to find technical solutions, rather than just short-term profiteering, but his decision to commit theft points towards the hypothesis that this was superficial.
I can’t tell if there’s any hope for crypto (I only have verified information on the bad parts, not the good parts if there are any left), but if there is, it would have to come from elite reformers, who are these types of people (races to the bottom to get reputation and outcompete rivals) and who each come with the risk of being only superficially committed.
Hence why the popular idea of “cultural reform” seems like a roundaboutly weak plan. EA needs to get better at doing the impossible on a hostile planet, including successfully sorting/sifting through accusationspace/powerplays/deception, and evaluating the motives of powerful people in order to determine safe levels of involvement and reciprocity. Not massive untested one-shot social revolutions with unpredictable and irreversible results.

trevor1 Apr 12, 2024, 3:45 PM
2 points
3 ∶ 0
in reply to: Amber Dawn’s comment on: Mediocre EAs: career paths and how do they engage with EA?
There’s people who are good at EA-related thinking and people who are less good at that.
There’s people who are good at accumulating resume padding, and people who are less good at that.
Although these are correlated, there will still be plenty of people who are good at EA thinking, but bad at accumulating resume padding. You can think of these people as having fallen through the cracks of the system.
Advances in LLMs give me the impression that we’re around ~2-5 years out from most EA orgs becoming much better at correctly identifying/drawing talent from this pool e.g. higher-quality summaries of posts and notes, or tracing upstream origins of original ideas.
I’m less optimistic about solutions to conflict theory/value alignment issues, but advances in talent sourcing/measurement might give orgs more room to focus hiring/evaluation energy on character traits. If talent is easy to measure then there’s less incentive to shrug and focus on candidates based on metrics that historically correlated with talent e.g. credentials.

trevor1 Apr 2, 2024, 2:34 PM
3 points
0 ∶ 0
on: Reasons for optimism about measuring malevolence to tackle x- and s-risks
Understanding malevolent high-performance individuals is a highly data-constrained area, even for just high-performance individuals; for example, any so-called “survey of CEOs” should be taken dubiously due to a high risk of intense nonresponse bias (e.g. most of the people on the survey have the CEO title but are only answering the survey because they aren’t actually doing most of the tasks undertaken by real CEOs).
Harder still to get data on individuals who are also malevolent on top of being high-performance. I’m pretty optimistic about technical solutions like fMRI and lie detectors (lie detection systems today are probably much more powerful than they have been for their ~100 years of history), especially when combined with genomics.
But data on high-performance individuals must be labelled based on the history/cases of revealed malevolent traits, and data on non-high-performance individuals might be plentiful but it’s hard to tell if it’s useful because high-performance individuals are in such an OOD environment.

trevor1 Mar 29, 2024, 5:04 AM
1 point
0 ∶ 0
in reply to: Emma Richter🔸’s comment on: Recent and upcoming media related to EA
Ah, my bad, I did a ctrl + f for “sam”! Glad that it was nothing.

trevor1

Stone Age Her­bal­ist’s notes on ant war­fare and slavery

Stone Age Herbalist’s notes on ant warfare and slavery