I’m taking decision making under deep uncertainty as a base. So being comfortable with making decisions under many view points. So trying to avoid any one dominant view point or analysis paralysis.
WillPearson
Karma: 125
Should AI Safety Funders Require Governance Documentation?
I’m trying to create a website/organisation/community around exploring difficult problems and improving the decisions people make.
I’ve currently got an alpha website where people can interact with AI in different scenarios and record the decisions and reasoning they make, to inform others.
I’m curious how others would approach this endeavour (I don’t have a broad network)
So I’ve been trying to think of ways to improve the software landscape. If we do this it might make traditional software more aligned with human values and it’s models for building more advanced systems too.
One piece I’ve been looking at is software licensing.
Instead of traditional open source, have an easy to get license for a version of software, based on a cryptographic identity. This could make it less frictional to be a bad actor.
This license is checked on startup that it matches the version of the software running (git sha stored somewhere). If it doesn’t the software fails to start. It can also be used by clients and servers to identify each other but does not have to marry up one to one with a person’s identity.
The license is acquired as easily as a let’s encrypt style certificate, but the identity has to part of the reputation system (which might require a fee).
The software might require a license from one of many reputation monitoring systems. So that no monitoring system becomes a single point of failure.
Edit: effective altruism might decide to fund awards for work of software ecosystem engineering with non software engineers as the judges to bring this digital infrastructure to the publics consciousness and incentivise making it understandable as well
I had an idea for a new concept in alignment that might allow nuanced and human like goals (if it can be fully developed).
Has anyone explored using neural clusters found by mechanistic interpretability as part of a goal system?
So that you would look for clusters for certain things e.g. happiness or autonomy and have that neural clusters in the goal system. If the system learned over time it could refine that concept.
This was inspired by how human goals seem to have concepts that change over time in them.
I’ve got an idea for a business that could help biosecurity by helping stop accidental leaks of data to people that shouldn’t have it. I’m thinking about proving the idea with personal identifiable information. Looking for feedback and collaborators.
My expectation is that software without humans in the loop evaluating it, will Goodhart’s law itself and over fit to the metrics/measures given.
My blog might be of interest to people
Here is a blog post also written with Claudes help that I hope to engage with home scale experimenters with
Fractal Governance: A Tractable, Neglected Approach to Existential Risk Reduction
I appreciate your views on space and AI working with ML systems in that way might be useful.
But I think that I am drawn to the base reality a lot because of threats to that from things like gamma ray bursts or aliens. These things can only be represented probabilistically in simulations because they are out of context. The branching tree explodes with possibilities.
I agree that we aren’t ready for agents , but I would like to try to build time non-static intelligence augmentation as slowly as possible. Starting with building systems to control and shape them tested out with static ML systems. Then testing them with people. Then testing them inside simulations etc
I find your view of things interesting. A few questions, how do you deal with democracy when people might be inhabiting worlds unlike the real one and have forgotten the real one exists?
I think static AI models lack corrigibility, humans can’t give them instruction on how to change how to act, so they might be a dead end in terms of day to day usefulness. They might be good as scientists though as they can be detached from human needs. So worth exploring.
There is a concept of utility, but I’m expecting these systems to mainly be user focussed so not agents in their own rights, so the utility is based on user feedback about the system. So ideally the system would be an extension of the feedback systems within humans.
There is also karma which is separate from utility which is given by one ml system to another, if it is helped it out or hindered it in a non-economic fashion.
I’ve been thinking that AGI will require an freely evolving multi-agent approach. So I want to try out the multi-agent control patterns on ML models without the evolution. Which should prove them out in a less dangerous setting. The multi-agent control patterns I am thinking are things like karma and market based alignment patterns. More information on my blog
[Question] Intellectual property of AI and existential risk in general?
Does anyone have recommendations for people I should be following for structural AI risk discussion and possible implications of post-current deep learning AGI systems.
I suppose I’m interested in questions around what is an existential threat. How bad a nuclear winter would it have to be to cause the collapse of society (and how easily could society be rebuilt afterwards). Both require robust models of agriculture in extreme situations and models of energy flows in economies where strategic elements might have been destroyed (to know how easy rebuilding would be). Since pandemic/climate change also have societal collapse as a threat the models needed would apply to them too (they might trigger nuclear exchange or at least loss of control over nuclear reactors, depending upon what societal collapse looks like).
The national risk register is the closest I found, in the public domain. It doesn’t include things like large meteorites, that I found.
[Question] Existential risk management in central government? Where is it?
It’s true that all data and algorithms are biased in some way. But I suppose the question is, is the bias from this less than what you get from human experts, who often have a pay cheque that might lead them to think in a certain way.
I’d imagine that any system would not be trusted implicitly, to start with, but would have to build up a reputation of providing useful predictions.
In terms of implementation, I’m imagining people building complex models of the world, like decision making under deep uncertainty with the AI mainly providing a user friendly interface to ask questions about the model.
An important early question I’ve been thinking about is “Even with aligned AI there might be a narrowing of human society, we need to make sure this is not permanent or ameliorated. How can we do this?” By narrowing of society I mean people interacting with a widely deployed AI being trained to act in a way that the dominant AI does not see as a threat or otherwise select against, e.g. with culture specific morals. Otherwise we might lose some important culture and not be able to get it back due to convergence.