Gerald Monroe

Karma: −69

Gerald Monroe Sep 19, 2023, 1:24 AM
2 points
1 ∶ 2
in reply to: Joe Collman’s comment on: AI Pause Will Likely Backfire
So just to summarize:

No deceptive or dangerous AI has ever been built or empirically tested. (1)

Historically AI capabilities have consistently been “underwhelming”, far below the hype. (2)

If we discuss “ok we build a large AGI, give it persistent memory and online learning, and isolate it in an air gapped data center and hand carry data to the machine via hardware locked media, what is the danger” you are going to respond either with:

“I don’t know how the model escapes but it’s so smart it will find a way” or (3)

“I am confident humanity will exist very far into the future so a small risk now is unacceptable (say 1-10 percent pDoom)”.

and if I point out that this large ASI model needs thousands of H100 accelerator cards and megawatts of power and specialized network topology to exist and there is nowhere to escape to, you will argue “it will optimize itself to fit on consumer PCs and escape to a botnet”. (4)

Have I summarized the arguments?

Like we’re supposed to coordinate an international pause and I see 4 unproven assertions above that have zero direct evidence. The one about humanity existing far into the future I don’t know I don’t want to argue that because it’s not falsifiable.

Shouldn’t we wait for evidence?

Gerald Monroe Sep 19, 2023, 12:08 AM
1 point
0 ∶ 0
in reply to: titotal’s comment on: Yonatan Cale’s Shortform
Why doesn’t this translate to AI risk.

“We should avoid building more powerful AI because it might kill us all” breaks to
1. No prior AI system has tried to kill us all
2. We are not sure how powerful a system we can really make scaling known techniques and adjacent to known techniques in the next 10-20 years. A system 20 years from now might not actually be “AGI” we don’t know.
This sounds like someone should have the burden of proof of showing near future AI systems are (1) lethal (2) powerful in a utility way, not just a trick but actually effective at real world tasks

And like the absence of unicorns caught on film someone could argue that ¹⁄₂ are unlikely by prior due to AI hype that did not pan out.

The counter argument seems to be “we should pause now, I don’t have to prove anything because an AI system might be so smart it can defeat any obstacles even though I don’t know how it could do that, it will be so smart it finds a way”. Or “by the time there is proof we will be about to die”.

Gerald Monroe Sep 19, 2023, 12:00 AM
3 points
0 ∶ 0
in reply to: Chris Leong’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
If you think it’s like a nuclear weapon but better (less indiscriminate, potentially offers a defense against the nukes of rival countries) what choice do you have? Notably Israel has a nuclear arsenal for precisely this reason, now they see a need to get the next weapons upgrade.

Gerald Monroe Sep 18, 2023, 11:50 PM
3 points
0 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: AI Pause Will Likely Backfire
Just something that jumped out at me. Suppose a pause is on 1e28+ training runs.

The human brain is made of modules organized in a way we don’t understand. But we do know the frontal lobes associated with executive functions are a small part of the total tissue.

This means an AI system could be a collection of a few dozen specialized 1e28 models separated by api calls, hosted in a common data center for low latency interconnects.

If a “few dozen” is 100+ modules the total compute used would be 1e30 and it might be possible to make this system an AGI with difficult training tasks to cause this level of cognitive development through feedback.

Especially with “meta” system architectures where new modules could be automatically added to improve score where deficiencies are present in a way that training existing weights is leading to regressions.

Gerald Monroe Sep 18, 2023, 11:37 PM
1 point
0 ∶ 0
in reply to: Steven Byrnes’s comment on: AI Pause Will Likely Backfire
Steven the issue is without empirical data you end up with a branching tree of possible futures. And if you make some faulty assumptions early—such as assuming the amount of compute needed to host optimal AI models is small and easily stolen via hacking—you end up lost in a tree of possibilities where every one you consider is “doom”. And thus you arrive at the conclusion of “pDoom is 99 percent”, because you are only cognitively able to consider adjacent futures in the possibility tree. No living human can keep track of thousands of possibilities in parallel. This is where I think Eliezer and Zvi are lost, where they simply ignore branches that would lead to different outcomes.

(And vice versa, you could arrive at the opposite conclusion).

It becomes angels at the head of a pin. There is no way to make a policy decision based on this. You need to prove you beliefs with data. It’s how we even got here as a species.

Gerald Monroe Sep 18, 2023, 11:28 PM
−1 points
1 ∶ 1
in reply to: titotal’s comment on: AI Pause Will Likely Backfire
Or another rephrase. How is the “secretly is planning to murder all humans” improving the models scores on a benchmark? If you think about it first, what gradient from the training set even led to this capability of an inner cognitive process looking for a chance to betray. What force is causing this cognitive process to come out of the random initial weights?

Humans seem to have such a force but it’s because modeling “if I kill this rival then the reward for me is...” was evolutionarily useful. Also it’s probably a behavior that is learned.

And second, yeah, SGD should push “neutral” weights inside the network that are not contributing to correct answers towards weights that do increase the odds of a correct output distribution. So it should actively destroy “unnecessary ” cognitive processes inside the model.

You could prove this. Make a psychopathic model designed to “betray” in a game like world and then see how many rounds of training on a new dataset clear the ability for the model to kill when it improves score.

Gerald Monroe Sep 18, 2023, 11:22 PM
1 point
0 ∶ 2
in reply to: DanielFilan’s comment on: AI Pause Will Likely Backfire
As a side note the actual things that break this loop are

(1) we don’t use superintelligent singletons and probably won’t, I hope. We instead create context limited model instances of a larger model and tell it only about our task and the model doesn’t retain information. This “break an ASI into a billion instances each which lives only in the moment” is a powerful alignment method

(2) it seems to take an absolutely immense amount of compute hardware to host even today’s models which are significantly below human intelligence in some expensive to fix ways. (For example how many H100s would you need for useful realtime video perception?)

This means a “rogue” Singleton would have nowhere to exist, as it would be too heavy in weights and required bandwidth to run on a botnet.

This breaks everything else.

It’s telling that Bostroms PhD is in philosophy and I don’t see any industry experience on his wiki page. He is correct if you ignore real world limitations on AI.

Gerald Monroe Sep 18, 2023, 11:16 PM
0 points
0 ∶ 0
in reply to: NickLaing’s comment on: AI Pause Will Likely Backfire
The assumptions are that more powerful models won’t be like weaker models but more accurate. They will show emergent abilities. Many things that gpt-4 can solve gpt-3 cannot, and those models share a similar lineage.

Safety issues show up when you have a model powerful enough to even exhibit them, and they may not be anything you predicted will happen from theory. Waluigi effect, hallucinations—both were not predicted by any theory by AI safety research groups. They seem to be the majority of the issues with models at the current level of capabilities.

Gerald Monroe Sep 18, 2023, 11:01 PM
2 points
0 ∶ 0
on: Comments on Manheim’s “What’s in a Pause?”
So I think the error here is disagreements over:
1. Is AI as useful as a nuclear weapon. If you think AI is so dangerous it will immediately turn on you and be useless, then it’s not like a nuclear weapon that only detonates when it was designed to.
If you think you can make AI systems using variations on current tech that are as useful as nuclear weapons then:
1. Has a technology even 1 percent as useful as a nuclear weapon ever been successfully “paused”. You might bring up gunpowder and China so then the followup question would be “in the last 150 years”.
If your answers are:
1. Yes it’s as useful as a nuclear weapon because an AI can be like adding billion of unpaid new workers to an economy
2. You are not guaranteed doom by using myopia to contain each instance in (1). It is a more dangerous world because the tools will be available to build non myopic systems.
3. No. The few technologies ever successfully paused barely work or have almost as good alternatives. For instance CFCs have alternatives, nuclear power has alternatives that are cheaper, genetic engineering of humans takes too long for results to be useful, nerve gas is not as good as nukes, bioweapons are uncontrollable and infect everyone or are not as good as nukes.
I personally think from what I know now, the above statement are true. What am I ignorant about? How am I wrong? Please tell me.

Because..if the above are true, then as morally correct as a pause might be, it wouldn’t happen unless the world is also one able to abolish nuclear weapons. Which is also morally correct but generally thought to be impossible.

Gerald Monroe Sep 15, 2023, 12:28 AM
−1 points
0 ∶ 0
in reply to: Zach Stein-Perlman’s comment on: Is EA a well-intentioned version of ‘survival of the fittest’ (or largest)?
What if you could either add an average 5 years to 1000 people, or spend the same resources to make 1 person have a life expectancy of 10,000 years?

What bothers me is that obviously 10k>5k, but the vibes of the decision. Especially as the “1” chosen individual is likely already powerful and well connected etc.

Gerald Monroe Sep 15, 2023, 12:10 AM
9 points
3 ∶ 1
on: Is slowing AI progress misguided because we need AI capabilities to achieve alignment?
It’s hard to think of examples of real human technologies that were developed or improved without essentially building full scale examples. Including ultimately the fully assembled machine. I can’t really think of an counter example.

It’s true you can build just a part and test that. Develop the turbo pump on a rocket engine by building some and separately perf testing them. Develop the full engine with pumps without the actual rocket. Various models to measure the forces the full assembly will see before actually building a full engine assembly and testing that.

But with all that you still need several launches of the full rocket system before you can use it for astronauts or payload.

You’re going to need full scale AGI and you’re going to need to create simulated scenarios where the AI will have an incentive to betray humans. There’s probably no other way.

Like hallucinations, the REAL problems with AGI systems may in fact turn out to be nothing anyone could have predicted. Instrumental convergence may turn out to be just a mirage.

I think it’s important to plan to control the fuel supply if early AGI systems do escape. We humans don’t actually control fire we just know the 3 conditions it needs and it is impossible to form without. Similarly we know AGI likely needs large quantities of compute to exist at all, so controlling access to that is important. (I can imagine in extreme scenarios technicians in bunny suits have to carry data on flash drives with hardware switches to block writes to air gapped clusters hosting the largest and most dangerous ASI models)

Another argument about the same lines is that building and testing is how you make progress. SpaceX being a marque example, outracing moribund competitors who have made minimal progress in 50 years. One of their key innovations seems to be their willingness to fail often. Each exploded complete rocket brings back more information than probably any amount of whiteboard or simulator modeling.

If someone were to demand safety “beyond a reasonable doubt” for any future AI research, obstructing any permits to train models behind months to years of review, it would be a way to make an entire country, perhaps an entire power block, moribund. Unable to make progress until their law enforcement authority is stripped away at gunpoint by competitors or worse.

Gerald Monroe Sep 14, 2023, 11:50 PM
2 points
1 ∶ 1
on: The state of AI in different countries — an overview
I wonder how large the USSRs fission research budget—the total across every institution—was prior to the Trinity test. My point is that this model is a lagging indicator. If the Trinity equivalent already happened then you will see a spike of research, a clean out of plagiarized results, and real effort from China.

There are routes that would allow Chinese labs to match current top Western labs and even proceed forward if they had full nation support. The obvious routes are espionage- historically this was how the USSR caught up on fission—and RSI. (Which requires an immense quantity of silicon. It’s as crucial as finding a source of uranium ore that the USSR faced for China to catch up on silicon fabrication if it is possible to do so)

It’s hard to say if Trinity has happened. Gpt-4 is strong but it’s not undeniably capable and it makes frequent errors. Maybe the next major model will be that moment.

I don’t know how much AI slowdown the west can afford, but maybe it should focus on measures that won’t lead to a significant deceleration. For example simply disallowing large interconnected GPU clusters in data centers unregistered with the government would be a start. Logging the users of the hardware, the source of the funds, their human contact information, and how much compute they are using would be another.

Requiring better cyber security especially on prototype AI systems is another measure that wouldn’t cost much or slow things down but would improve safety.

Gerald Monroe Sep 13, 2023, 6:36 PM
1 point
0 ∶ 0
in reply to: Arturo Macias’s comment on: Summary: High risk, low reward: A challenge to the astronomical value of existential risk mitigation
Agree. Also the very idea of a “hostile AGI” being able to exist assumes a bunch of things.

Notably :

(1) humans build large powerful models with the cognitive capacity to even be hostile (2) it is possible for a model to be optimized to actually run on stolen computers. This may be flat impossible due to fundamental limits on computation (3) once humans learn of the escaped hostile model they are ineffective in countermeasures or actually choose to host the model and trade with it instead of licensing more limited safer models (4) the hostile model is many times more intelligent than safer cognitively restricted models (5) intelligence has meaningful benefits at very high levels, it doesn’t saturate, where “5000 IQ” is meaningfully stronger in real world conflicts than “500 IQ” where the weaker model has a large resource advantage

I don’t believe any of these 5 things are true based on my current knowledge, and all 5 must be true or AGI doom is not possible.

Gerald Monroe Sep 13, 2023, 5:45 PM
1 point
0 ∶ 0
in reply to: Arturo Macias’s comment on: Summary: High risk, low reward: A challenge to the astronomical value of existential risk mitigation
The counter argument is that nuclear war was maybe never existential. That even at cold war peaks, there were not enough warheads and missiles and they were not targeted on enough industrial centers on earth to extinct humanity. Entire continents and global regions would have been uninvolved in the war (Africa, south America) and had enough copies of human knowledge and tools to respond to the new reality. Even global cooling scenarios ignore humans building makeshift greenhouses or other countermeasures.

Theoretically a hostile AGI with unlimited access to self replicating machinery, and freedom from interference from human militaries with comparable technology, could locate and kill every human on earth.

Gerald Monroe Sep 13, 2023, 5:38 PM
1 point
1 ∶ 1
in reply to: Jeremy Gillen’s comment on: Summary: High risk, low reward: A challenge to the astronomical value of existential risk mitigation
It would depend on if the risk from AGI is a one time risk that goes away when humans figure out alignment or an ongoing effort.

Alignment may be impossible. As a SWE working in AI I don’t know of any plausible method for the kind of alignment discussed here.

Risk mitigation is possible, in that we can stack together serial steps that must fail for the AGI to escape and do meaningful damage, as well as countermeasures (pre constructed weapons and detection institutions) ready to respond when this happens.

But the risk remains nonzero and recurring . Each century there is always this risk that the AGIs escape human control and the risk remains as long as “humans” are significantly stupider and less rational than AGI. I don’t know if “humans augment themselves so much to compete they are not remotely humanlike” counts as the extinction of humanity or not.

Gerald Monroe Sep 12, 2023, 5:09 PM
1 point
0 ∶ 1
in reply to: Greg_Colbourn ⏸️ ’s comment on: Debate series: should we push for a pause on the development of AI?
Perhaps. I can’t really engage on that because “moral disgust” doesn’t explain multiple distinct nations with slightly different views on morality all refusing to practice it. My main comment is I think it’s helpful to look at the potential gain vs potential risks.

Potential gain : yes you could identify alleles with promoters associated with the nervous system and statistically correlated with higher IQ. 20-30 years later, if this were done on large scales, people might be marginally smarter.

But how much gain is this? How long has it been since humans had the tools to even attempt genetic engineering. Can you project any gain whatsoever 20-30 years from now?

I would argue the answers are minimal, less than 10 years since reliable tools existed, and almost no gain, because in 20-30 years any task that “average or below” IQ individuals struggle with, AI tools will be able to complete in seconds.

Potential risks : each editing error undetected in early fetal development saddles the individual human with lifelong birth defects which may require a permanent caretaker or hospitalization. This can cost many millions, and is essentially so much liability that only a government could afford to practice genetic engineering.

Governments are slow, remember it’s only been about 10 years since it has been even feasible.

Conclusion: the risk to benefits ratio for human genetic engineering offers minimal gain and even in the best case is slow, which means little annual roi. The decision to say “industrialize China” has generated more wealth than all of humanity had prior to that point, and took only slightly longer than one iteration of human genetic engineering.

The potential benefits for AI could double the wealth of humanity in a few years, aka 10-100 percent annual ROI.

Gerald Monroe Sep 11, 2023, 9:51 PM
1 point
2 ∶ 9
on: Shrimp: The animals most commonly used and killed for food production
Shouldn’t you normalize for brain tissue? So instead of counting individuals, sum the entire mass of the brains of the animals humans breed for consumption.

A “sapience estimate” would also be useful. Smaller brains are likely less capable of sapience and it should be possible estimate the scaling empirically.

How do you handle ecological displacement? Every shrimp farm occupies a chunk of biomass and ocean territory that other animals would have lived in. So you’re substituting billions of shrimp lives lived for billions of displaced other animals and insects.

So if you got an animal rights bill passed that made illegal all forms of animal protein, the fallow areas where the farms were would now have other species living there. These species lives are “nasty brutish and short”, as whatever species occupy where the farm was will likely have short lifespan and face ocean predators. Maybe the lack of nets to keep them in the farm will make their lives better.

Though given the limited cognitive capacity of shrimp I wonder if they can even be upset by a large enough cage. They find one edge of it, but there are barriers in the natural ocean, and then do they “know” they are trapped when they wander to the other side of the cage they are in and find another impossible barrier? I don’t know the answer but this could be investigated empirically.

Gerald Monroe Sep 11, 2023, 9:31 PM
−1 points
0 ∶ 1
in reply to: Nathan Young’s comment on: Debate series: should we push for a pause on the development of AI?
So do any of these not exist in some form proving the tech is real?

Do any of these not have real world data of the drawbacks?

Note for example human genetic engineering, we never tried that but when we edit mammals errors are common and we know the incredible cost of birth defects. Also since we edit other mammals we know 100 percent this isn’t just a possibility down the road, it’s real, same tooling that works on rats will work on humans.

Do any of these, if you researched the tech and developed it to it’s full potential, allow you to invade every country on earth and/or threaten to kill every living person within that countrys borders?

Note those last paragraph. This is not me being edgy. If exactly one nation has a large nuclear arsenal they could do exactly that. Once other powers started to get nukes in the 1950s, every nation had to get their own or have trusted friends with nukes and a treaty protecting them.

AGI technology if developed to it’s full potential, and kept exclusive to one superpower, would allow the same.

It means any multilateral agreement to stop agi research because of dangers that haven’t been shown yet to exist means each party is throwing away all the benefits, like becoming incredibly wealthy, immortal, and having almost limitless automated military hardware.

So much incentive to cheat it seems like a non starter.

For an agreement to be even possible I think there would need to be evidence that

(1) it’s too dangerous to even experiment with AGI systems because it could “break out and kill everyone”. Break out to where? What computers can host it? Where are they? How does the AGI stop humans cutting the power or bombing the racks of interconnected H100s?

(2) There’s no point in building models, benchmarking them, finding the ones that are safe to use and also AGI, and using them, they will all betray you.

It’s possible that (1,2) are true facts of reality but there is no actual evidence.

Gerald Monroe Sep 10, 2023, 8:11 AM
−1 points
0 ∶ 1
in reply to: Marcel D’s comment on: Debate series: should we push for a pause on the development of AI?
The ‘final paragraph’ was simply noting that when you try to make concrete AI risks—instead of an abstract machine that is overwhelmingly smarter than human intelligence and randomly aligned, but a real machine that humans have to train and run on their computers—numerous technical mitigation methods are obvious. The one I was alluding to was (1)
(1) https://www.lesswrong.com/posts/C8XTFtiA5xtje6957/deception-i-ain-t-got-time-for-that
Sparsity and myopia are general alignment strategies and as it happens are general software engineering practices. Many of the alignment enthusiasts on lesswrong have rediscovered software architectures that already exist. Not just exist, but are core to software systems ranging from avionics software to web hyperscalers.
Sparsity happens to be TDD.
Myopia happens to be stateless microservices.

Gerald Monroe Sep 8, 2023, 10:43 PM
−1 points
0 ∶ 8
on: Debate series: should we push for a pause on the development of AI?
Regardless of whether a pause has benefits, can someone succinctly explain how a pause could even be possible? Nuclear arms pauses never meaningfully happened - SALT treaties were decades later and didn’t meaningfully reduce the “nuclear apocalypse” doom risk, at least for citizens living in targeted countries. (Maybe it reduced worldwide risks)

A nuclear bombardment is a quantifiable, object level thing. You can use a special tool and draw circles on a paper map of your own cities and wargame out where the enemy will allocate their missiles if their goal is maximum destruction. Multiply a series probability estimate of how likely the warhead will reach the target and give ful yield and add up the estimated damage.

The “we must pause” threat of AI doom I have seen so far doesn’t have any data of any threat. And a frequent threat model is the “treacherous turn”—you wouldn’t be able to quantify how risky the current AI systems you have at any point or their capacity for damage in numbers because the AI systems are hiding their motives and capabilities until they see a route to victory.

With no data of a threat how could any policymakers agree to a pause?

The other examples of human technologies we did agree to not pursue all exist and the threats are concrete. The damage from nerve gas, CFCs, bio warfare, and human genetic engineering we have data on.

As a side note the teacherous turn threat model has an obvious technical mitigation measure : use the least compute on any task by selecting from a pool of models the most efficient one. This discourages models with the cognitive capacity for sedition or hidden capabilities as they will need more memory and compute to run.