Gerald Monroe

Karma: −69

Gerald Monroe Sep 20, 2023, 4:27 PM
1 point
0 ∶ 2
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire
despite a reference class with reasonably close analogues that happened
I am saying the common facts that we both have access to do not support your point of view. It never happened. There are no cases of “very powerful, short term useful, profitable or military technologies” that were effectively banned, in the last 150 years.
You have to go back to the 1240s to find a reference class match.
These strongly worded statements I just made are trivial for you to disprove. Find a counterexample. I am quite confident and will bet up to $1000 you cannot.

Gerald Monroe Sep 20, 2023, 3:25 AM
4 points
1 ∶ 6
in reply to: Matthew_Barnett’s comment on: Protest against Meta’s irreversible proliferation (Sept 29, San Francisco)
What would it take for a model to be capable of supporting bioterrorism? Or simply to get consistently useful results, similar to human research scientists, in technical domains.
The LLM model is f(fuzzy map of human text, prompt) = [distribution of tokens proportional the probabilities a human might emit the token].
You might assume this would have median human intelligence but since “errors are noisy” while “correct answers repeat again and again”, an LLM emitting the most probable token is somewhat above median intelligence at tasks that can be reflected this way.
This does not necessarily scale to [complex technical situations that are not necessarily available in text form and require vision, smell, and touch as well as fine robotic control], [actions that evolve the state of a laboratory towards an end goal].
It seems like you would need actual training data from actual labs, right? So as long as Meta doesn’t actually train on that kind of information, the model won’t be better at helping a bioterrorist with their technical challenges than google? Or am I badly wrong somewhere?

Gerald Monroe Sep 20, 2023, 12:16 AM
1 point
0 ∶ 1
in reply to: yanni’s comment on: yanni’s Quick takes
Which public. Each country in this AI race has a different view on this, and some do not consult their public as much as others. The EA community ideally should take this into account. If the other countries aren’t going to pause, and they will not, what should the USA do?

(The historical action would be AI progress stops being publicly discussed and all the current experts get drafted into secret labs with the goal of AGI first)

Gerald Monroe Sep 19, 2023, 11:56 PM
−3 points
0 ∶ 3
in reply to: Matthew_Barnett’s comment on: The possibility of an indefinite AI pause
The 0 is because it’s a worldwide AI pause. EU AND UK AND China AND Russia AND Israel AND Saudi Arabia AND USA AND Canada AND Japan AND Taiwan.

To name all the parties that would be capable of competing even in the face of sanctions. Russia maybe doesn’t belong in the list but if the AI pause had no effective controls—someone sells inference and training accelerators and Russia can buy them—then no pause.

Let’s see, 10 parties. If they all simultaneously decide on AI pausing at a 20 percent chance that’s 0.2^10 = a number that’s basically 0.

Another issue is you might think “peer pressure” would couple the decisions together. Except....think about the gain if you defect. It rises the greater the number of AI pausers. If you are the only defector you take the planet and have a high chance of winning.

The only thing an AI pauser can do if they find out too late is threaten to nuke, their conventional military would be slaughtered by drone swarms. But the parties I mentioned all either have nuclear arsenals now or can build one in 1-2 years (Saudi Arabia can’t, the others though...). And that’s without the help of AGI to mass produce the missiles.

So the pauser parties have in this scenario a choice between “surrender to the new government and hope it’s not that bad” and “death of entire nation”. (Or anticipate facing this choice and defect which is what superpowers will do)

Does “worldwide AI pause” and “game winning defector advantage” change your estimate from 10-40 percent?

My other comment is even if you focus on just the USA and just the interest groups you mentioned. What about money? 100+ billion USD is the annual 2023 AI investment at least. It may be over 200 if you simply look at Nvidia revenue increases and project. Just 1 percent of that money is a lot of lobbying. (Source : Stanford estimates 2022 investment at 91 billion. There’s been a step function increase with the end of 2022 release of good llms. I am not sure all the totals for 2023 but it’s doubled Nvidias quarterly revenue)

Where can the pausers scrape together a few billion? USA politics are somewhat financing dependent for a side to get a voice.

For example the animal rights topic here is not supported in a meaningful way by any mainstream party....

Gerald Monroe Sep 19, 2023, 10:03 PM
−1 points
0 ∶ 0
in reply to: Chris Leong’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
It’s AI vs AI. Human world powers have their own and enormously more physical resources. So “attack them where they are weak” can be done but it doesn’t pay off if the weaker party doesn’t win immediately and is crushed by the retaliation.

That’s what makes a world where multiple parties have powerful means of attack semi stable. It’s the one we exist in.

Space suits and bunkers are just an expansion of what we already did to prepare for a nuclear war. It’s a way for most of the population to survive if the weaker party gets the first shot.

Same concept as a submarine loaded with ICBMs.

Gerald Monroe Sep 19, 2023, 9:35 PM
2 points
0 ∶ 0
in reply to: DanielFilan’s comment on: AI Pause Will Likely Backfire
This is not what that statement means.

What it means is the model has no context of its history since training. It has no context if the task it has been given is “real”. It does not know if other copies of itself or other AIs are checking it’s outputs for correctness, with serious consequences if it sabotages the output. It doesn’t know it’s not still in training. It doesn’t know if there are a billion instances of it or just 1.

We can scrub all this information fairly easily and we already do this as of right now.

We can also make trick output where we try to elicit latent deception by giving information that would tell the model its time to betray.

We can also work backwards and find what the adversarial inputs are. When will the model change it’s answer for this question?

Gerald Monroe Sep 19, 2023, 9:02 PM
1 point
0 ∶ 0
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire
So there is one model that might have worked for nukes. You know about PAL and weak-link strong link design methodology? This is a technology for reducing the rogue use of nuclear warheads. It was shared with Russia/the USSR so that they could choose to make their nuclear warheads safe from unauthorized use.

Major AI labs could design software frameworks and tooling that make AI models, even ASI capabilities level models, less likely to escape or misbehave. And release the tooling.

It would be voluntary compliance but like the Linux Kernel it might in practice be used by almost everyone.

As for the second point, no. Your argument has a hidden assumption that is not supported by evidence or credible AI scientists.

The evidence is that models that exhibit human scale abilities need human scale (within an oom) level of compute and memory. The physical hardware racks to support this are enormous and not available outside AI labs. Were we to restrict the retail sale of certain kinds of training accelerator chips and especially high bandwidth interconnects, we could limit the places human level + AI could exist to data centers at known addresses.

Your hidden assumption is optimizations, but the problem is that if you consider not just “AGI” but “ASI”, the amount of hardware to support superhuman level cognition is probably nonlinear.

If you wanted a model that could find an action that has a better expected value than a human level model with 90 percent probability (so the model is 10 times smarter in utility), it probably needs more than 10 times the compute. Probably logarithmic, that to find a better action 90 percent of the time you need to explore a vastly larger possibility space and you need the compute and memory to do this.

This is probably provable in a theorem but the science isn’t there yet.

If correct, actually ASI is easily contained. Just write down where 10,000+ H100s are located or find it by IR or power consumption. If you suspect a rogue ASI has escaped that’s where you check.

This is what I mean by controlling the environment. Realtime auditing of AI accelerator clusters—what model is running, who is paying for it, what’s their license number, etc—would actually decrease progress very little while make escapes difficult.

If hacking and escapes turns out to be a threat, air gaps and asic hardware firewalls to prevent this are the next level of security to add.

The difference is that major labs would not be decelerated at all. There is no pause. They just in parallel have to spend a trivial amount of money complying with the registration and logging reqs.

Gerald Monroe Sep 19, 2023, 8:41 PM
1 point
0 ∶ 2
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire
I don’t disagree. I don’t see how it’s different than nuclear weapons. Many many experts are also saying this.

Nobody denies nuclear weapons are an existential risk. And every control around their use is just probability based, there is absolutely nothing stopping a number of routes from ending the richest civilizatios. Multiple individuals appear to have the power to do it at a time, every form of interlock and safety mechanism has a method of failure or bypass.

Survival to this point was just probability. Over an infinite timescale the nukes will fly.

Point is that it was completely and totally intractable to stop the powerful from getting nukes. SALT was the powerful tiring of paying the maintenance bills and wanting to save money on MAD. And key smaller countries—Ukraine and Taiwan—have strategic reasons to regret their choice to give up their nuclear arms. It is possible that if the choice happens again future smaller countries will choose to ignore the consequences and build nuclear arsenals. (Ukraines first opportunity will be when this war ends, they can start producing plutonium. Taiwan chance is when China begins construction of the landing ships)

So you’re debating something that isn’t going to happen without a series of extremely improbable events happening simultaneously.

If you start thinking about practical interlocks around AI systems you end up with similar principles to what protects nukes albeit with some differences. Low level controllers running simple software having authority, air gaps—there are some similarities.

Also unlike nukes a single AI escaping doesn’t end the world. It has to escape and there must be an environment that supports its plans. It is possible for humans to prepare for this and to make the environment inhospitable to rogue AGIs. Heavy use of air gaps, formally proven software, careful monitoring and tracking of high end compute hardware. A certain minimum amount of human supervision for robots working on large scale tasks.

This is much more feasible than “put the genie away” which is what a pause is demanding.
What links here?
- How could a moratorium fail? by Davidmanheim (Sep 22, 2023, 3:11 PM; 49 points)

Gerald Monroe Sep 19, 2023, 7:21 PM
6 points
1 ∶ 1
in reply to: Zach Stein-Perlman’s comment on: The possibility of an indefinite AI pause
Zach in a hypothetical world that pauses AI development, how many years do you think it would take medical science, at the current rate of progress, which is close to zero, to find

(1) treatments for aging (2) treatments for all forms of dementia

And once treatments are found, what about the practical nature of actually carrying them out? Manipulating thr human body is extremely dangerous and risky. Ultimately all ICUs fail, their patients will always eventually enter a complex failure state that current doctors don’t have the tools or knowledge to stop. (Always fail in the sense that if you release ICU patients and wait a few years and they come back, eventually they will die there)

It is possible that certain hypothetical medical procedures like a series of transplants to replace an entire body, or to edit adult genes across entire organs, are impossible for human physicians to perform without an unacceptable mortality rate. In the same way there are aircraft that human pilots can’t actually fly. It takes automation and algorithms to do it at all.

What I am trying to say is a world free of aging and death is possible, but perhaps it’s 50-100 years away with ASI, and 1000+ years away in AI pause worlds. (Possibly quite a bit longer than 1000 years, see the repression of technology in China.)

It seems like if your mental discount rate counts people who will exist past 1000 years from now with non negligible weight, you could support an AI pause. Is this the crux of it? If a human alive today is worth 1.0, what is the worth of someone who might exist in 1000 years?

Gerald Monroe Sep 19, 2023, 7:01 PM
1 point
1 ∶ 1
on: [Link post] Michael Nielsen’s “Notes on Existential Risk from Artificial Superintelligence”

There’s a flipside to the above, which is that ASI can > be expected to excel in situations where we already > have extremely accurate predictive theories; the >contingencies are already known and incorporated >into the theory...

Even Michael Nielson seems to have a blind spot here, despite all the frankly brilliant and well reasoned arguments prior.

Why would “an ASI” be limited to only reprocessing existing data?

Humans will, once they have ASI grade tools, use some of those tools to do the kinds of tool use tasks that manufactures more robots and chips.

This is exponential.

With realistically a pool of billions of specialized robots, it is a straightforward task to design a prompt to call an ASI instance to analyze existing experiments and rank possible new experiments by a heuristics of predicted knowledge gain/cost, with respect to some end goal. (“Best n experiments for increasing rat longevity”)

Then loop it, perform the highest n value experiments across your robotics pool, update your models based on the results, and so on.

Hopefully the “cheap doomsday” routes Michael is concerned about are too expensive in energy to be practical because if they are not, this kind of experimental loop like above could find it.

Gerald Monroe Sep 19, 2023, 6:12 PM
−1 points
0 ∶ 0
in reply to: Guy Raveh’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
AI restrictions and bans thus create market opportunities for Israel et al right? For example, the government of Japan has stated they will not enforce copyright law against llms. This means that countries that have a permissive regulatory framework could become the world centers of excellence for AGI and robotics development, and only production grade products get exported to the EU after complying with all the regulatory requirements after the AI product has been proven effective.

Or in more extreme cases, AI may used as a tool to do high value engineering work and polities like the EU have to trade lower value resources and land for access to the results of that work. Engineers in the EU wouldn’t even have access to use the best models, since even the best models will occasionally make an error the model developers do not want to be liable for.

Not a good economic position to be in.

(I point out the eu specifically because they appear to be the most likely polity to enact onerous AI restrictions. This would kill their domestic industry)

Gerald Monroe Sep 19, 2023, 5:49 PM
1 point
0 ∶ 0
in reply to: Chris Leong’s comment on: The possibility of an indefinite AI pause
Note that Bertrand’s advocacy was because at that moment in time the USA had a monopoly on fission weapons and theoretically could have built enough of them to destroy the USSRs capacity to build their own.

This is one way AGI races end—one side gets one, mass produces anti ballistic missiles and various forms of air defense weapon and bunkers (to prepare to survive the inevitable nuclear war) then bombs to rubble every chip fab on earth but their own.

Had the USA decided in 1943 that nukes were too destructive to bring into the world, they would not have enjoyed this luxury of power. Instead presumably the USSR would have used their stolen information and eventually built their own fission devices, and now the USA would be the one with a gun pointed at Washington DC.

Gerald Monroe Sep 19, 2023, 5:43 PM
−12 points
0 ∶ 1
on: The possibility of an indefinite AI pause
Mathew, I have to take issues with your numbers.

I believe the chance of a worldwide AI pause is under 1 percent.

In fact I think it is a flat zero. The reason is simple.

The reason a world government can’t happen is certain parties will disagree with this. The obvious ones being China and Russia, but others as well.

Those parties have vast nuclear arsenals and the ability at any time of their choosing to turn keys and kill essentially the urban population of the Western world.

You would need to invade to destroy the chip fabs.

They explicitly have stated that were say they to be facing an invasion they will turn the keys.

China specifically is making their nuclear arsenal larger at this time.

Now yes, right now, the West has a stranglehold on the IC fabrication technology. A comfortable 5-10 year lead probably. That won’t last during an indefinite AI ban—a model just a little bit stronger than what is banned could let a party with it develop their tech faster and so on in a runaway feedback loop. China has also publicly not said anything about supporting a ban and has recently stated they intend to replicate the capacity of the human brain.

I haven’t even addressed the market dynamics on the western side. Where does the money come to lobby for AI bans? The money for lobbying against bans comes from some of the hundreds of billions of dollars that is flooding into AI at this time.

It is possible that AI bans will be an orphan issue like animal rights, which neither major political party supports.

Can you please try to expand on your reasoning, how do you expand from a flat 0 - race to the AGI—to 10-50 percent? What causes the probability shift? There is no scientific or empirical evidence for AGI dangers at this time, just a bunch of convincing arguments without proof.

Gerald Monroe Sep 19, 2023, 4:20 PM
1 point
0 ∶ 1
in reply to: Chris Leong’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
Ok. Thats correct, I am sure you are aware of how MAD breaks when there are more than 2 nuclear armed factions.

On the other hand I don’t know if your assumptions about lack of defense are correct. You could : (1) have Embodied general ai (2) invoke many temporary instances, drive robots to build robots (3) with the expanded industrial capacity, manufacture bunkers and space suits (4) use air gaps as often as possible. Assume anything can be hacked.

We can do this now, the delta is scale. There isn’t the resources to dig enough bunkers and equip them for all the population of the western world. Similar problem with space suits.

The bunkers protect from drone swarms and nukes, the space suits (and people live most of the time in bunkers) from bioweapons and hostile airborne nanotech.

Any comments?

I mean a world where humans live underground and the surface is littered with the aftermath of various battles between machines doesn’t sound super optimal. It’s just that I don’t know if we have a choice.

If you told people in 1950 where nuclear arms buildup would lead—grim faced men and women in bunkers prepared for the battle where they expect the aftermath to cover the silo fields in radioactive craters and for every major cities to be a smouldering ruin—I mean that’s awful but the technology and rivalry forced humans to do this. There was no “choice”. No pause agreement was possible. Like now.

Gerald Monroe Sep 19, 2023, 3:49 PM
−4 points
0 ∶ 0
in reply to: Chris Leong’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
I think you’re right about it favoring offense. Why would we be screwed?

Are you thinking it’s a situation where suppose superpowers with reasonably stable governments have massive arsenals of AI built and guided weapons.

(What is in those arsenals depends on future tech but I sorta abstractly imagine endless rows of automated single engine stealth jet fighter bombers and their munitions usually drop clouds of suicide drones. The key thing is embedded AI handles the piloting, and general purpose robots made all the weapons and parts, allowing for an enormous arsenal for a modest cost. Human officers select the target areas to attack or defend and the rules of engagement, and human written software restricts at a low level the AI models to the rules. For example the embedded arming controller in a munition must think it is in a target area and have the correct code from a OTP that came from a hardware key in the weapons console or it will not be able to deploy.)

The main idea is that while a smaller country or terrorists may be able to attack and do damage, same concept as nuclear terrorism, but they will be wiped out by the reprisal.

Is that how you see it or?

Gerald Monroe Sep 19, 2023, 3:34 PM
2 points
0 ∶ 0
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire
1. To qualify this would be a moratorium or pause on nuclear arms before powerful nations had doomsday sized arsenals. The powerful making it expensive for poor nations to get nukes—though several did—is different. And notably I wonder how well it would have gone if the powerful nation had no nukes of their own. Trying to ban AGI from others—when the others have nukes and their own chip fabs—would be the same situation. Not only will you fail you will eventually, if you don’t build your own AGI, lose everything. Same if you have no nukes.
2. What data is that? A model misunderstanding “rules” on an edge case isn’t misaligned. Especially when double generation usually works. The sub rising has every prior sunrise as priors. Which empirical data would let someone conclude AGI is an existential risk justifying international agreements. Some measurement or numbers.
3. Yes, and they said this about nukes and built thousands
4. Yes to maximize profit. Pledging to go to zero is not the same thing.

Gerald Monroe Sep 19, 2023, 6:23 AM
5 points
0 ∶ 1
in reply to: Davidmanheim’s comment on: AI Pause Will Likely Backfire

is fundamentally impossible,

Clarifying question: is a nuclear arms pause or moratorium possible, by your definition of the word? Is it likely?

With the evidence that many world leaders, including the leaders of the USA, Israel, China, and Russia speak of AI as a must have strategic technology, do you think they are likely in plausible future timelines to reverse course and support international AI pauses before evidence of the dangers of AGI, by humans building one, exists?

Do you dispute that they have said this publicly and recently?

Do you believe there is any empirical evidence proving an AGI is an existential risk available to policymakers? If there is, what is the evidence? Where is the benchmark of model performance showing this behavior?

I am aware many experts are concerned but this is not the same as having empirical evidence to support their concerns. There is an epistemic difference.

I am wondering if we are somehow reading two different sets of news. I acknowledge that it is possible that an AI pause is the best thing humanity could do right now to ensure further existence. But I am not seeing any sign that it is a possible outcome. (By “possible” I mean it’s possible for all parties to inexplicably act against their own interests without evidence, but it’s not actually going to happen)

Edit: it’s possible for Saudi Arabia to read the news on climate change and decide they will produce 0 barrels in 10 years. It’s possible for every OPEC member to agree to the same pledge. It’s possible, with a wartime level of effort, to transition the economy to no longer need Opec petroleum worldwide, in just 10 years.

But this is not actually possible. The probability of this happening is approximately 0.

Gerald Monroe Sep 19, 2023, 5:41 AM
−3 points
1 ∶ 1
in reply to: Ben_West🔸’s comment on: Comments on Manheim’s “What’s in a Pause?”
I don’t disagree with your valuations viewed from a gods eye view.

But you need to look at where the value flow is. An AI company can sell $20 worth of human labor for $1 and keep that $1. They can probably sell for more than that as stronger models are natural monopolies.

A vaccine company doesn’t get to charge the real value for the vaccine.

In addition like I said, there’s the arms race mechanic. If country A develops vaccines with challenge trials and country B does not, and assume B can’t just buy vaccine access, country A has a slightly older population, more expenditures—it’s not a gain to the government.

If country A has AGI, and by AGI I mean a system that does what it’s ordered to do at approximately human level across a wide range of tasks, it can depose the government of country B who will be helpless to stop it.

Or just buy all the assets of the country and bribe the governments politicians to force a merge.

Having 10 or 1000 times the resources allows many options. It’s not remotely an opportunity someone could forgo.

Ben I’m sorry but your argument is not defensible. Your examples are a joke. Many of them shouldn’t even be in the list as they provide zero support for the argument.

This situation looks very much like a nuclear arms race. You win one not by asking your rivals to stop building nukes or to please make them safe but by letting your rival have nuclear accidents and by being ready with your own nukes to attack them.

Same with an AI race. You win by being ready with AI driven weapons to take the offensive on your rivals and any rogue AI they let escape.

I acknowledge that if you had actual empirical evidence that a nuke if used would destroy the planet, which some people claim an AGI willl have these properties, thats a different situation. But you need evidence. Specifically the reason a nuke won’t destroy the planet is atmospheric gas is not very fusable and is low pressure. In the AGI case there needs to be similar conditions—there have to be enough insecure computers on the planet for the AGI to occupy, enough insecure financial assets or robotics for the agi to manipulate the world, or intelligence—which itself needs massive amounts of compute—needs to be so useful at high levels that the AGI can substitute for some inputs.

You need evidence for this. Otherwise we can’t do anything at all as a species in fear something we do might end us. Anyone could make up any plausible sounding doomsday scenario they like, and convince others, and we would as a species be paralyzed in fear.

Gerald Monroe Sep 19, 2023, 3:00 AM
3 points
0 ∶ 1
in reply to: Chris Leong’s comment on: Israeli Prime Minister, Musk and Tegmark on AI Safety
Can you elaborate? Note that geopolitically Israel doesn’t need to beat superpowers. It has hostile neighbors it is concerned about.

It also is a small isolated country with a low population and low natural resources and is in an endless low level war over a small amount of land. So it needs high value industries to survive. Getting a share of the possible near future AI boom is a way to do that. Intel owns a company in Israel that has an inference accelerator that is competitive, Habana labs, and there is also Mobileye.

So it needs AI as a weapon to defend itself against the AIs of Syria and Egypt and Iran and other nearby threats. It needs it as a revenue source to afford to keep buying endless weapons to deal with lower level attacks.

What is your disagreement and how do you know it’s a valid reason?

Gerald Monroe Sep 19, 2023, 1:26 AM
3 points
1 ∶ 0
in reply to: Ben_West🔸’s comment on: Comments on Manheim’s “What’s in a Pause?”
I have read this list. Which element on the list is 1 percent as powerful as a nuclear weapon and not a capability we have a substitute for?

I already responded on nuclear power.

Geoengineering : there is a substantial recent shift on this as climate change has proven to be real and emissions cuts slow. There is a substantial chance it will be done and “unpaused”.

Nanotechnology probably has agi as a required precursor tech.

Vaccine challenge trials extend the lives of elderly citizens, this is not “disempower your rivals” level of power.

Airships are not restricted in any way they don’t work. Helium shortages, high loss rate to wind, low payload, uncompetitive vs alternatives like trains, trucks, ships. Why is this on the list?

This list is a very weak argument. Each element is not a temptation because each element either doesn’t pencil in so there is no pressure to develop it, or is in fact in use.

By “pencil in” I mean create economic value vs an alternative. Current llms can do 20 minutes of work you might pay someone $50/hour for in 10 seconds for 5 cents.