Ward A

Karma: 36

The world would have the wisest minds cower in fear, and the fair of heart despair. Human nature, once reflected upon and made to cohere, is revealed as near unbearable to share. Even so, it seems plain from here—the latent harmony makes it clear! -- we tend to converge when we start to care.

I’m trying to align AI and learn how to think. I care a lot about animals and machine sentience. I prefer a state wherein things are happy, and dislike it when things suffer.

I’m clueless about most of what I say, so I’d be happy if you informed me of what you found objectionable. I tend to optimise for sharing weird ideas without worrying about being wrong or justifying them. If I have something valuable to say, saying it without nuance is the cheapest-for-reader way to learn it. Sometimes the most epistemically virtuous and altruistic thing is to not waste anyone’s time signalling that virtue.

Ward A 3 May 2023 13:33 UTC
2 points
0 ∶ 0
on: My choice of AI misalignment introduction for a general audience
This is good, thanks! I’m not sure if it’s my preferred choice for any use case, but it’s close.

Ward A 24 Apr 2023 7:09 UTC
3 points
1 ∶ 0
on: Silly idea to enhance List representation accuracy
I like the intent and I love the silliness, but the reasons we bring up alternatives include...
1. So we seem less biased and less like we’ve only considered a single option.
2. To cooperate with people who prioritise differently than us.
  1. It seems better if e.g. when someone asks what the best religion is, Christians, Muslims, Buddhists, all agree on providing a list of options while also making clear that they themselves consider their own religion best for XYZ reasons.
And accurately representing our beliefs about the relative merits of focusing on each risk will to some extent negate 1 and 2, especially 1.

Ward A 19 Apr 2023 8:19 UTC
1 point
2 ∶ 2
on: What concrete question would you like to see debated? I might organise some debates.
Make it an ‘undebate’. 10 points for every time you learn something, and 150 points for changing your mind on the central proposition.
Also, I’d like to see RLHF^[1] debated. Whether any form of RL on realistic text data will be able to take us to a point where it’s “smart enough”, either to help us align higher intelligences or just smart enough for what we need.
1. ^
  Reinforcement Learning from Human Feedback.^[2] A strategy for AI alignment.
2. ^
  I wish the forum had the feature where if you write [[RLHF]], it automatically makes an internal link to the topic page or where RLHF is defined in the wiki. It’s standard in personal knowledge management systems like Obsidian, Roam, RemNote, and I think Arbital does it.

Ward A 18 Apr 2023 15:09 UTC
5 points
4 ∶ 0
on: But why would the AI kill us?
“One of the big reasons that humans haven’t disassembled cows for spare parts...”
What. This is all that we do to them.

Ward A 18 Apr 2023 2:41 UTC
4 points
1 ∶ 0
in reply to: Eevee🔹’s comment on: Nobody’s on the ball on AGI alignment
Thanks for the explanation! Though I think I’ve been misunderstood.
I think I strongly prefer if e.g. Sam Altman, Demis Hassabis, or Elon Musk ends up with majority influence over how AGI gets applied (assuming we’re alive), over leading candidates in China (especially Xi Jinping). But to state that preference as “I hope China doesn’t develop AI before the US!” seems … unusually imprecise and harmful. Especially when nationalistic framings like that are already very likely to fuel otherisation and lack of cross-cultural understanding.
It’s like saying “Russia is an evil country for attacking Ukraine,” when you could just say “Putin” or “the Russian government” or any other way of phrasing what you mean with less likelihood of spilling over to hatred of Russians in general.

Ward A 16 Apr 2023 17:13 UTC
1 point
0 ∶ 0
in reply to: Riccardo’s comment on: All AGI Safety questions welcome (especially basic ones) [April 2023]
Hmm, I have a different take. I think if I tried to predict as many tokens as possible in response to a particular question, I would say all the words that I could guess someone who knew the answer would say, and then just blank out the actual answer because I couldn’t predict it.
Ah, you want to know about the Riemann hypothesis? Yes, I can explain to you what this hypothesis is, because I know it well. Wise of you ask me in particular, because you certainly wouldn’t ask anyone you knew didn’t have a clue. I will state its precise definition as follows:
~Kittens on the rooftop they sang nya nya nya.~
And that, you see, is what the hypothesis that Riemann hypothesised.
I’m not very good at even pretending to pretend to know what it is, so even if you blanked out the middle, you could still guess I was making it up. But if you blank out the substantive parts of GPT’s answer when it’s confabulating, you’ll have a hard time telling whether it knows the answer or not. It’s just good at what it does.

Ward A 15 Apr 2023 2:26 UTC
7 points
0 ∶ 0
on: Announcing Innovate Animal Ag (like GFI but for Animal Welfare Tech)
So happy to hear somebody’s carrying the torch wrt in-ovo sexing and many more technologies I can’t keep up with. I wish I could prioritise it, but I chose AI and it doesn’t make sense for me to whiffle-whaffle between the two. Best of luck!

Ward A 15 Apr 2023 2:19 UTC
1 point
1 ∶ 0
in reply to: Riccardo’s comment on: All AGI Safety questions welcome (especially basic ones) [April 2023]
Thanks for reply! I don’t think the fact that they hallucinate is necessarily indicative of limited capabilities. I’m not worried about how dumb they are at their dumbest, but how smart they are at their smartest. Same with humans lol.
Though, for now, I still struggle with getting GPT-4 to be creative. But this could either be because it’s habit to stick to training data, and not really about it being too dumb to come up with creative plans. …I remember when I was in school, I didn’t much care for classes, but I studied math on my own. If my reward function hasn’t been attuned to whatever tests other people have designed for me, I’m just not going to try very hard.

Ward A 13 Apr 2023 16:03 UTC
4 points
1 ∶ 0
in reply to: Brian_Tomasik’s comment on: All AGI Safety questions welcome (especially basic ones) [April 2023]
Personally, my worry stems primarily from how difficult it seems to prevent utter fools from mixing up something like ChaosGPT with GPT-5 or 6. That was a doozy for me. You don’t need fancy causal explanations of misalignment if the doom-mechanism is just… somebody telling the GPT to kill us all. And somebody will definitely try.
Secondarily, I also think a gradually increasing share of GPT’s activation network gets funneled through heuristics that are generally useful for all the tasks involved in minimising its loss function at INT<20, and those heuristics may not stay inner- or outer-aligned at INT>20. Such heuristics include:
1. You get better results if you search a higher-dimensional action-space.
2. You get better results on novel tasks if you model the cognitive processes producing those results, followed by using that model to produce results. There’s a monotonic path all the way up to consequentialism that goes something like the following.
  1. ...index and reuse algorithms that have been reliable for similar tasks, since searching a space of general algorithms is much faster than the alternative.
  2. ...extend its ability to recognise which tasks count as ‘similar’.^[1]
  3. ...develop meta-algorithms for more reliably putting algorithms together in increasingly complex sequences.
  4. This progression could result in something that has an explicit model of its own proxy-values, and explicitly searches a high-dimensional space of action-sequences for plans according to meta-heuristics that have historically maximised those proxy-values. Aka a consequentialist. At which point you should hope those proxy-values capture something you care about.
This is just one hypothetical zoomed-out story that makes sense in my own head, but you definitely shouldn’t defer to my understanding of this. I can explain jargon upon request.
1. ^
  Aka proxy-values. Note that just by extending the domain of inputs for which a particular algorithm is used, you can end up with a proxy-value without directly modelling anything about your loss-function explicitly. Values evolve as the domains of highly general algorithms.

Ward A 13 Apr 2023 13:28 UTC
4 points
0 ∶ 0
on: All AGI Safety questions welcome (especially basic ones) [April 2023]
Are there any concrete reasons to suspect language models to start to act more like consequentialists the better they get at modelling them? I think I’m asking something subtle, so let me rephrase. This is probably a very basic question, I’m just confused about it.
If an LLM is smart enough give us a step-by-step robust plan that covers everything with regards to solving alignment and steering the future to where we want it, are there concrete reasons to expect it to also apply a similar level of unconstrained causal reasoning wrt its own loss function or evolved proxies?
At the moment, I can’t settle this either way, so it’s cause for both worry and optimism.

Ward A 13 Apr 2023 13:05 UTC
3 points
0 ∶ 0
in reply to: Geoffrey Miller’s comment on: [Linkpost] 538 Politics Podcast on AI risk & politics
My advice was badly targeted. I regret it. Empirical research is relevant for measuring average appeal across a variety of opinion-markets, but people also tend to overapply it in cases where they know more about their audience. Nate Silver appeals to a broad group, and my suggestion is more appropriate appealing to a niche.
Just like the most profitable strategy for a spaghetti-sauce-selling company is to offer a variety of flavours (preferably informed by factor analysis of preferences), I imagine the goal should be to have a nice mix between Eliezer-style and everyday-folk public messaging, even though the two offset each other a little.

Ward A 12 Apr 2023 5:21 UTC
1 point
0 ∶ 4
on: [Linkpost] 538 Politics Podcast on AI risk & politics
I think saying “even a small risk of extinction is worth prioritising!” sends the wrong message. It’s (usually) aimed at assuring people that they can go on as normal. What should be communicated instead is something like:
No, I’m serious. It’s not a small risk, and we lack the emotional freedom to deal with it properly as a society. But this also means that we can make an actual difference by taking it seriously early. We now have a unique opportunity to influence the trajectory of public opinion because people haven’t got used to it yet.

Ward A 11 Apr 2023 11:36 UTC
1 point
0 ∶ 0
on: Measuring artificial intelligence on human benchmarks is naive
Hm, I believe there was a button for cross-posting to both EA and LW forums a year ago, but I couldn’t find the option.

Ward A 11 Apr 2023 10:03 UTC
2 points
0 ∶ 0
in reply to: JustinShovelain’s comment on: AI: 4 alignment criteria to keep in mind
Thanks!
Agree that values have to be learned indirectly (as we do), but I’m either skeptical or confused about what could uncharitably be framed as “inserting a math equation into an AI”. Making an idea technically precise, or specifying it with mathy symbols, does not guarantee that it can be used for training a neural network (which realistically is the only paradigm of AI we can work with, I think).
We will probably have to find a balance between a precisely specified loss function we believe will indirectly lead to good values, and a loss function that we have a large amount of corresponding training data for.
Feel free to ignore the following section, it’s a hurried rambling before breakfast.
||One way of potentially getting extra power out a limited set of data, is to first 1) constrain the amount of information the network receives about its true loss function per unit of computation (i.e. an information bottleneck), maybe via something like training on a larger “batch size”. And then 2) use something like temporal difference learning so the network gets trained on a proxy (an estimator) for the true loss function that it refines over time.
This would amount to having a fluid proximal (high-information) reward function and a fixed distal (low-information) reward function. The brain does something similar, and it predictably leads to mesa-optimisation if the influence from proximal rewards dominates the distal ones. The trick would then be to balance their respective learning rates so the distal rewards always constrains the evolution of the network more than the estimator. While the evolution of DNA resulted in a mesa-optimisation “catastrophe”, we have the advantage that we get to monitor the network and intervene intelligently on the learning process in real-time.||

Ward A 11 Apr 2023 9:12 UTC
−2 points
1 ∶ 0
in reply to: D0TheMath’s comment on: Which stocks or ETFs should you invest in to take advantage of a possible AGI explosion, and why?
Right? I might be misunderstanding something about investing, but my presumption was that if you invest in a company, you help it do more of what it does. Please do correct me if I’m wrong.

Ward A 11 Apr 2023 9:10 UTC
3 points
0 ∶ 0
on: EA & “The correct response to uncertainty is *not* half-speed”
That post is one of my all-time favourites from LW. I’ve struggled a lot with cause uncertainty, but sometimes the best way to resolve it is just to go full speed on one thing until you discover that it was not for you. The compromise is usually less effective than any of the options. :)

Ward A 10 Apr 2023 18:52 UTC
2 points
3 ∶ 1
in reply to: Ward A’s comment on: Nobody’s on the ball on AGI alignment
I’m honestly curious what motivates people to support this framing. I don’t live in an English-speaking country, and for all I know, I might have severely misunderstood broader EA culture. Are people here sincerely feeling nationalistic (a negative term in Norway) about this? I’d be appreciative if somebody volunteered to help me understand.

Ward A 10 Apr 2023 15:37 UTC
−3 points
5 ∶ 1
on: Nobody’s on the ball on AGI alignment
″...or block deployment when we actually really should deploy, e.g. to beat China.”
What the heck? Keep me out of your “we”. I’m vehemently against framing this as a nationalist us-them issue. If instead you meant something entirely reasonable like “e.g. so Xi doesn’t become god-emperor,” then say so. Leave “China” out of it, that’s where my friends live.
Otherwise, nice post btw.

Ward A 10 Apr 2023 8:01 UTC
2 points
0 ∶ 0
on: GPTs are Predictors, not Imitators
I think an easier way to see it is realising that its loss is uniform over all pieces of text, whereas humans only care about predicting an extreme minority of that text. If you see a sentence like
“It was a rainy day in Nairobi, the capital of_”
...it’s obvious to you that the salient piece of knowledge here is what country Nairobi is the capital of, so that’s how you design your benchmarks for AI performance. But the AI cares equally about predicting ‘capital’ after ‘the’, and ‘rainy’ after ‘It was a’. GPT-2 was already past human level at almost all text except the very selective subset humans put all their optimisation into (e.g. answers to math tests, long-term coherence in stories, etc.).
And yet GPT-4 rivals us at what we care about.
It’s comparable to a science fiction author who only cares about writing better stories yet ends up rivalling top scientists in every field as an instrumental side quest. Human-centric benchmarks^[1] vastly underestimate the objective intelligence and generality of GPTs.
1. ^
  Lessons from Are We Smart Enough to Know How Smart Animals Are seem relevant here.