anonymous6

Karma: 204

anonymous6 6 Aug 2023 14:09 UTC
1 point
0 ∶ 0
on: An appeal to people who are smarter than me: please help me clarify my thinking about AI
I think this is an extremely good post laying out why the public discussion on this topic might seem confusing:
https://www.lesswrong.com/posts/BTcEzXYoDrWzkLLrQ/the-public-debate-about-ai-is-confusing-for-the-general
It might be somewhat hard to follow, but this little prediction market is interesting (wouldn’t take the numbers too seriously):

In December of last year it seemed plausible to many people online that by now, August 2023, the world would be a very strange, near-apocalyptic place full of inscrutable alien intelligences. Obviously, this is totally wrong. So it could be worth comparing others’ “vibes” here to your own thought process to see if you’re overestimating the rate of progress.
Paying for GPT-4 if you have the budget may also be helpful to calibrate. It’s magical, but you run into embarrassing failures pretty quickly, which most commentators tend to talk about rarely.

anonymous6 6 Jun 2023 9:19 UTC
7 points
2 ∶ 0
on: [Linkpost] Given Extinction Worries, Why Don’t AI Researchers Quit? Well, Several Reasons
I thought this was a great point.
There is absolutely nothing hypocritical about an AI researcher who is pursuing either research that’s not on the path to AGI or alignment research to be sounding the alarm about the risks of AGI. Consider if we had one word for “energy researcher” which included all of: a) studying the energy released in chemical reactions, b) developing solar panels, and c) developing methods for fossil fuel extraction. In such a situation, it would not be hypocritical for someone from a) or b) to voice concerns about how c) was leading to climate change — even though they would be an “energy researcher” expressing concerns about “energy research.”
Probably the majority of “AI researchers” are in this position. It’s an extremely broad field. Someone can come up with a new probabilistic programming language for Bayesian statistics, or prove some abstruse separation of two classes of MDPs, and wind up publishing at the same conference as the people trying to hook up a giant LLM to real-world actuators.

anonymous6 30 Mar 2023 17:36 UTC
8 points
5 ∶ 1
on: Recruit the World’s best for AGI Alignment
Right now it seems like AI safety is more of a scene (centered around the Bay Area) than a research community. If you want to attract great scientists and mathematicians (or even mediocre scientists and mathematicians), something even more basic than coming up with good “nerd-sniping” problems is changing this. There are many people who could do good work in technical AI safety, but would not fit in socially with EAs and rationalists.
I’m sympathetic to arguments that formal prepublication peer review is a waste of time. However, I think formalizing and writing up ideas to academic standards, such that they could be submitted for peer review, is definitely a very good use of time, it’s not even that hard, and there should be more of it. This would be one step towards making a more bland, boring, professionalized research community where a wider variety of people might want to be involved.

anonymous6 15 Mar 2023 13:25 UTC
5 points
2 ∶ 0
in reply to: Geoffrey Miller’s comment on: How bad a future do ML researchers expect?
I think a better analogy than “ICBM engineering” might be “all of aeronautical engineering and also some physicists studying fluid dynamics”. If you were an anti-nuclear protester and you went and yelled at an engineer who runs wind tunnel simulations to design cars, they would see this as strange and unfair. This is true even though there might be some dual use where aerodynamics simulations are also important for designing nuclear missiles.

anonymous6 13 Mar 2023 20:19 UTC
4 points
3 ∶ 1
in reply to: Geoffrey Miller’s comment on: How bad a future do ML researchers expect?
I think this would be counterproductive. Most ML researchers don’t think the research they personally work on is dangerous, or could be dangerous, or contributes to a research direction that could be dangerous, and most of them actually are right about this. There is all kinds of stuff people work on that’s not on the critical path to dangerous AI.

anonymous6 9 Feb 2023 13:31 UTC
1 point
0 ∶ 0
on: Which ML skills are useful for finding a new AIS research agenda?
Some basic knowledge of (relatively) old-school probabilistic graphical models, along with basic understanding of variational inference. Not that graphical models are going to be used directly for any SOTA models any more, but the mathematical formalism is still very useful.
For example, understanding how inference on a graphical model works motivates the control-as-inference perspective on reinforcement learning. This is useful for understanding things like decision transformers, or this post on how to interpret RLHF on language models.
It would also be essential background to understand the causal incentives research agenda.
So the same tools come up in two very different places, which I think makes a case for their usefulness.
This is in some sense math-heavy, and some of the concepts are pretty dense, but without many mathematical prerequisites. You have to understand basic probability (how expected values and log likelihoods work, mental comfort going between $E$ and $\int$ notation), basic calculus (like “set the derivative = 0 to maximize”), and be comfortable algebraically manipulating sums and products.

anonymous6 29 Dec 2022 3:32 UTC
3 points
0 ∶ 0
on: Book recommendations for the history of ML?
Each chapter of Russell & Norvig’s textbook “Artificial Intelligence: A Modern Approach” ends with historical notes. These are probably sparser than you want, but they are good and cover a very broad array of topics. The 4th edition of the book is decently up to date (for the time being!).

anonymous6 1 Dec 2022 9:03 UTC
3 points
0 ∶ 1
on: SBF’s comments on ethics are no surprise to virtue ethicists
Trying to “do as the virtuous agent would do” (or maybe “do things for the sake of being a good person”) seems to be a really common problem for people.
Ruthless consequentialist reasoning totally short-circuits this, which I think is a large part of its appeal. You can be sitting around in this paralyzed fog, agonizing over whether you’re “really” good or merely trying to fake being good for subconscious selfish reasons, feeling guilty for not being eudaimonic enough—and then somebody comes along and says “stop worrying and get up and buy some bednets”, and you’re free.
I’m not philosophically sophisticated enough to have views on metaethics, but it does seem sometimes that the main value of ethical theories is therapeutic, so different contradictory ethical theories could be best for different people and at different times of life.

anonymous6 25 Nov 2022 9:24 UTC
4 points
1 ∶ 0
in reply to: Habryka’s comment on: Rethink Priorities’ Leadership Statement on the FTX situation
I would be inclined to replace “not thinking carefully” with “not thinking formally”. In real life everything tends to have exceptions and this is most people’s presumption, so they don’t feel a need to reserve language for the truly universal claims which are never meaningful.

Some people have practice in thinking about formal systems, where truly universal statements are meaningful, and where using different language to draw fine distinctions is important (“always” vs “with probability 1” vs “with high probability” vs “likely”).

Trying to push the second group’s norms on the first group might be tough even if perhaps it would be good.

anonymous6 25 Nov 2022 7:39 UTC
5 points
2 ∶ 1
in reply to: Habryka’s comment on: Rethink Priorities’ Leadership Statement on the FTX situation
I think when most people say “unequivocally” and “all”, they almost always mean “still maybe some exceptions” and “almost all”. If you don’t need to make mathematical/logical statements, which most people don’t, then reserving these words to act as universal quantifiers is not very useful. I used to be annoyed by this but I’ve learned to accept it.

anonymous6 20 Oct 2022 13:02 UTC
1 point
0 ∶ 0
in reply to: jh’s comment on: We can do better than argmax
Here’s one set of lecture notes (don’t endorse that they’re necessarily the best, just first I found quickly) https://lucatrevisan.github.io/40391/lecture12.pdf
Keywords to search for other sources would be “multiplicative weight updates”, “follow the leader”, “follow the regularized leader”.
Note that this is for what’s sometimes called the “experts” setting, where you get full feedback on the counterfactual actions you didn’t take. But the same approach basically works with some slight modification for the “bandit” setting, where you only get to see the result of what you actually did.

anonymous6 12 Oct 2022 17:50 UTC
12 points
2 ∶ 0
on: Cultural EA considerations for Nordic folks
I have the feeling people sometimes just disappear even if we already agreed to have a call or to meet up (but for example did not agree on the time yet).
This is stereotypically seen as something people in California do, and is complained about by East Coasters. People will both agree to get coffee or lunch at some point and then never follow up. Maintaining the ambiguity is considered polite. Overrepresentation of Bay Area residents might be the explanation here.

anonymous6 12 Oct 2022 17:26 UTC
18 points
2 ∶ 0
in reply to: jh’s comment on: We can do better than argmax
One justification might be that in an online setting where you have to learn which options are best from past observations, the naive “follow the leader” approach—exactly maximizing your action based on whatever seems best so far—is easily exploited by an adversary.
This problem resolves itself if you make actions more likely if they’ve performed well, but regularize a little to smooth things out. The most common regularizer is entropy, and then as described on the “Softmax demystified” page, you basically end up recovering softmax (this is the well-known “multiplicative weight updates” algorithm).

anonymous6 11 Oct 2022 17:43 UTC
1 point
0 ∶ 0
on: Prediction market does not imply causation
Ignoring the exponential blowup, one could have a prediction market over all the causal models to elicit the best one (including an option “all of these are wrong/important variables are missing”).
[on reflection, this seems hard unless you commit to doing a bunch of experiments or otherwise have a way to get the right outcome]
Then, with a presumptively trustworthy causal model, the “make adjustments to observational data” approach would be more reliable to estimate from other markets.
However, it feels like it could be the case that trying to do both of these things at once might screw up the incentives—in other cases there are sometimes impossibility results like this.
“How can we design mechanisms to elicit causal information, not just distributional properties” seems like a really interesting question that seemingly hasn’t received much attention.

anonymous6 6 Oct 2022 6:29 UTC
1 point
0 ∶ 0
in reply to: Joseph Bloom’s comment on: Does EA need better explanations of why Climate Change isn’t so popular a cause area?
unfortunately no, just idle musings. i would be interested in reading it, though.

anonymous6 4 Oct 2022 15:11 UTC
16 points
5 ∶ 1
on: Does EA need better explanations of why Climate Change isn’t so popular a cause area?
How would EAs talk about climate change, if it were a weird niche issue that few people were working on and didn’t have any political connotations? One can imagine that “catastrophic climate impacts due to carbon dioxide” would be another EA cause area that made normal people scratch their heads.
Giving a short description from that hypothetical world might be a good way to communicate why EAs worry less about climate in our actual world.

anonymous6 29 Sep 2022 15:34 UTC
5 points
6 ∶ 0
in reply to: Nathan Young’s comment on: William MacAskill—The Daily Show
Apart from a few things (rs) I actually think his accent might be phonetically closer to an American accent than most UK accents. It didn’t seem hard to understand.

anonymous6 12 Sep 2022 9:22 UTC
4 points
0 ∶ 0
on: The (Allegedly) Best Business Books
“Business Adventures” by John Brooks is a collection of midcentury New Yorker articles about business, obviously very old-fashioned but they are really quite good. There’s something to be said for learning about:
- a short squeeze in shares of the Piggly-Wiggly supermarket chain (founder went broke, later tried to make Amazon Just Walk Out but with 1950s punch card technology)
- the first insider trading lawsuit
- the first “Big Tech” information technology company that spun off from university research and promoted liberal political causes (Xerox)
- the earliest cases of employees being sued over noncompete/intellectual property agreements
Among many other interesting but less obviously relevant topics.

anonymous6 7 Sep 2022 15:27 UTC
16 points
0 ∶ 0
in reply to: Jackson Wagner’s comment on: Who are some less-known people like Petrov?
José Figueres Ferrer was victorious in the Costa Rican civil war, after which he appointed himself head of the provisional junta.

Sounds like trouble — but he only ruled for 18 months, during which time he abolished the army and extended the franchise to women and nonwhite people. Then he stepped down and there have been fair elections since.

anonymous6 8 Aug 2022 2:55 UTC
2 points
0 ∶ 0
on: Why I Hope (Certain) Hedonic Utilitarians Don’t Control the Long-term Future
In meditation there are the jhanas, which include states of intense physical pleasure (like a runner’s high). I learned how to do this, but the pleasure gets boring—though not less intense—after about 10 minutes or so and I feel tempted to go do something which is less pleasurable (and not less painful or offering greater future benefits). (And you’d think it would be habit forming, but in fact I have a hard time keeping up my meditation habit...)
What this taught me is that I don’t always want to maximize pleasure, even if I can do it with zero cost. I thus have a hard time making sense of what hedonists mean by “pleasure”.
If it’s just positive emotions and physical pleasure, then that means sometimes hedonists would want to force me to do things I don’t want to do, with no future benefit to me, which seems odd. (I guess a real bullet-biting hedonist could say that I have a kind of akrasia, but it’s not very persuasive.)
It also seems that, sometimes, hedonists who say “pleasure” mean some kind of more subtle multidimensional notion of subjective experiences of human flourishing. Giving a clear definition of this seems beyond anybody now living, but in principle it seems like a safer basis for hedonistic utilitarianism, and bothers me a lot less.
But now I’m not so sure, because I think most of your arguments here also go through even for a complicated, subtle notion of “pleasure” that reflects all our cherished human values.