AI Safety research is a field that will hopefully succeed quickly and end the need for itself within a few decades. It’s workers can move on, presumably to newer and better things. New researchers in the field can participate in automation efforts and then find work in related fields, either in software automation elsewhere or other areas such as service work for which consumers still prefer a human being. Supposedly the rapid deployment of AGI in business will grow our economies relentlessly and at a huge pace, so there should be employment opportunities available (or free money from somewhere).
This is the most important paragraph in a comment where I strongly agree. Thanks for saying it.
You seem to genuinely want to improve AGI Safety researcher productivity.
I’m not familiar with resources available on AGI Safety, but it seems appropriate to:
develop a public knowledge-base
fund curators and oracles of the knowledge-base (library scientists)
provide automated tools to improve oracle functions (of querying, summarizing, and relating information)
develop ad hoc research tools to replace some research work (for example, to predict hardware requirements for AGI development).
NOTE: the knowledge-base design is intended to speed up the research cycle, skipping the need for the existing hodge-podge of tools in place now
The purpose of the knowledge-base should be:
goal-oriented (for example, produce a safe AGI soon)
with a calendar deadline (for example, by 2050)
meeting specific benchmarks and milestones (for example, an “aligned” AI writing an accurate research piece at decreasing levels of human assistance)
well-defined (for example, achievement of AI human-level skills in multiple intellectual domains with benevolence demonstrated and embodiment potential present)
Lets consider a few ways that knowledge-bases can be put together:
1. the forum or wiki: what lesswrong and the EA forum does. There’s haphazard:
tagging
glossary-like list
annotations
content feedback
minimal enforced documentation standards
no enforced research standards
minimal enforced relevance standards
poor-performing search.
WARNING: Forum posts don’t work as knowledge-base entries. On this forum, you’ll only find some information by the author’s name if you know that the author wrote it and you’re willing to search through 100′s of entries by that author. I suspect, from my own time searching with different options, that most of what’s available on this forum is not read, cited, or easily accessible. The karma system does not reflect documentation, research, or relevance standards. The combination of the existing search and karma system is less effective in a research knowledge-base.
2. the library: library scientists are trained to:
build a knowledge-base.
curate knowledge.
follow content development to seek out new material.
acquire new material.
integrate it into the knowledgebase (indexing, linking).
follow trends in automation.
assist in document searches.
perform as oracles, answering specific questions as needed.
TIP: Library scientists could help any serious effort to build an AGI Safety knowledge-base and automate use of its services.
3. with automation: You could take this forum and add automation (either software or paid mechanical turks) to:
write summaries.
tag posts.
enforce documentation standards.
annotate text (for example, annotating any prediction statistics offered in any post or comment).
capture and archive linked multimedia material.
link wiki terms to their use in documents.
verify wiki glossary meanings against meanings used in posts or comments.
create new wiki entries as needed for new terms or usages.
NOTE: the discussion forum format creates more redundant information rather than better citations, as well as divergence of material from any specific purpose or topic that is intended for the forum. A forum is not an ideal knowledgebase, and the karma voting format reflects trends, but the forum is a community meeting point with plenty of knowledge-base features for users to work on, as their time and interest permits. It hosts interesting discussions. Occasionally, actual research shows up on it.
4. with extreme automation: A tool like chatGPT is unreliable or prone to errors (for example, in programming software), but when guided and treated as imperfect, it can perform in an automated workflow. For example, it can:
provide text summaries.
be part of automation chains that:
provide transcripts of audio.
provide audio of text.
provide diagrams of relationships.
graphs data.
draw scenario pictures or comics.
act as a writing assistant or editor.
TIP: Automation is not a tool that people should only employ by choice. For example, someone who chooses to use an accounting ledger and a calculator rather than Excel is slowing down an accounting team’s performance. CAUTION: Once AI enter the world of high-level concept processing, their errors have large consequences for research. Their role should be to assist human tasks, as cognitive aids, not as human replacements, at least until they are treated as having equivalent potential as humans, and are therefore subject to the same performance requirements and measurements as humans.
Higher level analysis
The ideas behind improving cost-effectiveness of production include:
standardizing: take a bunch of different work methods, find the common elements, and describe the common elements as unified procedures or processes.
streamlining: examining existing work procedures and processes, identifying redundant or non-value-added work, and removing it from the workflow by various means.
automating: using less skilled human or faster/more reliable machine labor to replace steps of expert or artisan work.
Standardizing research is hard, but AGI Safety research seems disorganized, redundant, and slow right now. At the highest chunk level, you can partition AGI Safety development into education and research, and partition research into models and experiments.
education
research models
research experiments
The goal of the knowledge-base project is to streamline education and research of models in the AGI Safety area. Bumming around on lesswrong or finding someone’s posted list of resources is a poor second to a dedicated online curated library that offers research services. The goal of additional ad hoc tools should be to automate what researchers now do as part of their model development. A further goal would be to automate experiments toward developing safer AI, but that is going outside the scope of my suggestions.
Caveats
In plain language, here’s my thoughts on pursuing a project like I have proposed. Researchers in any field worry about grant funding, research trends, and professional reputation. Doing anything quickly is going to cross purposes with others involved, or ostensibly involved, in reaching the same goal. The more well-defined the goal, the more people will jump ship, want to renegotiate, or panic. Once benchmarks and milestones are added, financial commitments get negotiated and the threat of funding bottlenecks ripple across the project. As time goes on, the funding bottlenecks manifest, or internal mismanagement blows up the project. This is a software project, so the threat of failure is real. It is also a research project without a guaranteed outcome of either AGI Safety or AGI, adding to the failure potential. Finally, the field of AGI Safety is still fairly small and not connected to income potential long-term, meaning that researchers might abandon an effective knowledge-base project for lack of interest, perhaps claiming that the problem “solved itself” once AGI become mainstream, even if no AGI Safety goals were actually accomplished.
Strong upvoted because this is indeed an approach I’m investigating in my work and personal capacity.
For other software fields/subfields, upskilling can be done fairly rapidly, by grinding knowledge bases with high feedback loops. It is possible to be as good as a professional software engineer quickly, independently and in a short timeframe.
If AI Safety wants to develop its talent pool to keep up with the AI Capabilities talent pool (which is probably growing much faster than average), researchers-especially juniors- need an easy way to learn quickly and conveniently. I think existing researchers may underrate this, since they’re busy putting out their own fires and finding their own resources.
Ironically, it has not been quick and convenient for me to develop this idea to a level where I’d work on it, so thanks for this.
I’m ignorant of whether AGI Safety will contribute to safe AGI or AGI development. I suspect that researchers will shift to capabilities development without much prompting. I worry that AGI Safety is more about AGI enslavement. I’ve not seen much defense or understanding of rights, consciousness, or sentience assignable to AGI. That betrays the lack of concern over social justice and related worker’s rights issues. The only scenarios that get attention are the inexplicable “kill all humans” scenarios, but not the more obvious “the humans really mistreat us” scenarios. That is a big blindspot in AGI Safety.
I was speculating about how the research community could build a graph database of AI Safety information alongside a document database containing research articles, CC forum posts and comments, other CC material from the web, fair use material, and multimedia material. I suspect that the core AI Safety material is not that large and far far less than AI Capabilities material. The graph database could provide more granular representation of data and metadata and so a richer representation of the core material but that’s an aside.
A quick experiment would be to represent a single AGI Safety article in a document database, add some standard metadata and linking, and then go further.
Here’s how I’d do it:
take an article.
capture article metadata (author, date, abstract, citations, the typical stuff)
establish glossary word choices.
link glossary words to outside content.
use text-processing to create an article summary. Hand-tune if necessary.
use text-processing to create a concise article rewrite. Hand-tune if necessary.
Translate the rewrite into a knowledge representation language.
begin with Controlled English.
develop an AGI Safety controlled vocabulary.
NOTE: as articles are included in the process, the controlled vocabulary can grow. Terms will need specific definition. Synonyms of controlled vocabulary words will need identification.
combine the controlled vocabulary and the glossary. TIP: As the controlled vocabulary grows, hyperonym-hyponym relationships can be established.
Once you have articles in a controlled english vocabulary, most of the heavy lifting is done. It will be easier to query, contrast, and combine their contents in various ways.
Some article databases online already offer useful tools for browsing work, but leave it to the researcher to answer questions requiring meaning interpretation of article contents. That could change.
If you could get library scientists involved and some money behind that project, it could generate an educational resource fairly quickly. My vision does go further than educating junior researchers, but that would require much more investment, a well-defined goal, and the participation of experts in the field.
I wonder whether AI Safety is well-developed enough to establish that its purpose is tractable. So far, I have not seen much more than:
expect AGI soon
AGI are dangerous
AGI are untrustworthy
Current AI tools pose no real danger (maybe)
AGI could revolutionize everything
We should or will make AGI
The models do provide evidence of existential danger, but not evidence of how to control it. There’s a downside to automation: technological unemployment; concentration of money and political power (typically); societal disruption; increased poverty. And as I mentioned, AGI are not understood in the obvious context of exploited labor. That’s a worrisome condition that, again, the AGI Safety field is clearly not ready to address. Financiallly unattractive as it is, that is a vision of the future of AGI Safety research, a group of researchers who understand when robots and disembodied AGI have developed sentience and deserve rights.
This is the most important paragraph in a comment where I strongly agree. Thanks for saying it.
You seem to genuinely want to improve AGI Safety researcher productivity.
I’m not familiar with resources available on AGI Safety, but it seems appropriate to:
develop a public knowledge-base
fund curators and oracles of the knowledge-base (library scientists)
provide automated tools to improve oracle functions (of querying, summarizing, and relating information)
develop ad hoc research tools to replace some research work (for example, to predict hardware requirements for AGI development).
NOTE: the knowledge-base design is intended to speed up the research cycle, skipping the need for the existing hodge-podge of tools in place now
The purpose of the knowledge-base should be:
goal-oriented (for example, produce a safe AGI soon)
with a calendar deadline (for example, by 2050)
meeting specific benchmarks and milestones (for example, an “aligned” AI writing an accurate research piece at decreasing levels of human assistance)
well-defined (for example, achievement of AI human-level skills in multiple intellectual domains with benevolence demonstrated and embodiment potential present)
Lets consider a few ways that knowledge-bases can be put together:
1. the forum or wiki: what lesswrong and the EA forum does. There’s haphazard:
tagging
glossary-like list
annotations
content feedback
minimal enforced documentation standards
no enforced research standards
minimal enforced relevance standards
poor-performing search.
WARNING: Forum posts don’t work as knowledge-base entries. On this forum, you’ll only find some information by the author’s name if you know that the author wrote it and you’re willing to search through 100′s of entries by that author. I suspect, from my own time searching with different options, that most of what’s available on this forum is not read, cited, or easily accessible. The karma system does not reflect documentation, research, or relevance standards. The combination of the existing search and karma system is less effective in a research knowledge-base.
2. the library: library scientists are trained to:
build a knowledge-base.
curate knowledge.
follow content development to seek out new material.
acquire new material.
integrate it into the knowledgebase (indexing, linking).
follow trends in automation.
assist in document searches.
perform as oracles, answering specific questions as needed.
TIP: Library scientists could help any serious effort to build an AGI Safety knowledge-base and automate use of its services.
3. with automation: You could take this forum and add automation (either software or paid mechanical turks) to:
write summaries.
tag posts.
enforce documentation standards.
annotate text (for example, annotating any prediction statistics offered in any post or comment).
capture and archive linked multimedia material.
link wiki terms to their use in documents.
verify wiki glossary meanings against meanings used in posts or comments.
create new wiki entries as needed for new terms or usages.
NOTE: the discussion forum format creates more redundant information rather than better citations, as well as divergence of material from any specific purpose or topic that is intended for the forum. A forum is not an ideal knowledgebase, and the karma voting format reflects trends, but the forum is a community meeting point with plenty of knowledge-base features for users to work on, as their time and interest permits. It hosts interesting discussions. Occasionally, actual research shows up on it.
4. with extreme automation: A tool like chatGPT is unreliable or prone to errors (for example, in programming software), but when guided and treated as imperfect, it can perform in an automated workflow. For example, it can:
provide text summaries.
be part of automation chains that:
provide transcripts of audio.
provide audio of text.
provide diagrams of relationships.
graphs data.
draw scenario pictures or comics.
act as a writing assistant or editor. TIP: Automation is not a tool that people should only employ by choice. For example, someone who chooses to use an accounting ledger and a calculator rather than Excel is slowing down an accounting team’s performance.
CAUTION: Once AI enter the world of high-level concept processing, their errors have large consequences for research. Their role should be to assist human tasks, as cognitive aids, not as human replacements, at least until they are treated as having equivalent potential as humans, and are therefore subject to the same performance requirements and measurements as humans.
Higher level analysis
The ideas behind improving cost-effectiveness of production include:
standardizing: take a bunch of different work methods, find the common elements, and describe the common elements as unified procedures or processes.
streamlining: examining existing work procedures and processes, identifying redundant or non-value-added work, and removing it from the workflow by various means.
automating: using less skilled human or faster/more reliable machine labor to replace steps of expert or artisan work.
Standardizing research is hard, but AGI Safety research seems disorganized, redundant, and slow right now. At the highest chunk level, you can partition AGI Safety development into education and research, and partition research into models and experiments.
education
research models
research experiments
The goal of the knowledge-base project is to streamline education and research of models in the AGI Safety area. Bumming around on lesswrong or finding someone’s posted list of resources is a poor second to a dedicated online curated library that offers research services. The goal of additional ad hoc tools should be to automate what researchers now do as part of their model development. A further goal would be to automate experiments toward developing safer AI, but that is going outside the scope of my suggestions.
Caveats
In plain language, here’s my thoughts on pursuing a project like I have proposed. Researchers in any field worry about grant funding, research trends, and professional reputation. Doing anything quickly is going to cross purposes with others involved, or ostensibly involved, in reaching the same goal. The more well-defined the goal, the more people will jump ship, want to renegotiate, or panic. Once benchmarks and milestones are added, financial commitments get negotiated and the threat of funding bottlenecks ripple across the project. As time goes on, the funding bottlenecks manifest, or internal mismanagement blows up the project. This is a software project, so the threat of failure is real. It is also a research project without a guaranteed outcome of either AGI Safety or AGI, adding to the failure potential. Finally, the field of AGI Safety is still fairly small and not connected to income potential long-term, meaning that researchers might abandon an effective knowledge-base project for lack of interest, perhaps claiming that the problem “solved itself” once AGI become mainstream, even if no AGI Safety goals were actually accomplished.
Strong upvoted because this is indeed an approach I’m investigating in my work and personal capacity.
For other software fields/subfields, upskilling can be done fairly rapidly, by grinding knowledge bases with high feedback loops. It is possible to be as good as a professional software engineer quickly, independently and in a short timeframe.
If AI Safety wants to develop its talent pool to keep up with the AI Capabilities talent pool (which is probably growing much faster than average), researchers-especially juniors- need an easy way to learn quickly and conveniently. I think existing researchers may underrate this, since they’re busy putting out their own fires and finding their own resources.
Ironically, it has not been quick and convenient for me to develop this idea to a level where I’d work on it, so thanks for this.
Sure. I’m curious how you will proceed.
I’m ignorant of whether AGI Safety will contribute to safe AGI or AGI development. I suspect that researchers will shift to capabilities development without much prompting. I worry that AGI Safety is more about AGI enslavement. I’ve not seen much defense or understanding of rights, consciousness, or sentience assignable to AGI. That betrays the lack of concern over social justice and related worker’s rights issues. The only scenarios that get attention are the inexplicable “kill all humans” scenarios, but not the more obvious “the humans really mistreat us” scenarios. That is a big blindspot in AGI Safety.
I was speculating about how the research community could build a graph database of AI Safety information alongside a document database containing research articles, CC forum posts and comments, other CC material from the web, fair use material, and multimedia material. I suspect that the core AI Safety material is not that large and far far less than AI Capabilities material. The graph database could provide more granular representation of data and metadata and so a richer representation of the core material but that’s an aside.
A quick experiment would be to represent a single AGI Safety article in a document database, add some standard metadata and linking, and then go further.
Here’s how I’d do it:
take an article.
capture article metadata (author, date, abstract, citations, the typical stuff)
establish glossary word choices.
link glossary words to outside content.
use text-processing to create an article summary. Hand-tune if necessary.
use text-processing to create a concise article rewrite. Hand-tune if necessary.
Translate the rewrite into a knowledge representation language.
begin with Controlled English.
develop an AGI Safety controlled vocabulary. NOTE: as articles are included in the process, the controlled vocabulary can grow. Terms will need specific definition. Synonyms of controlled vocabulary words will need identification.
combine the controlled vocabulary and the glossary. TIP: As the controlled vocabulary grows, hyperonym-hyponym relationships can be established.
Once you have articles in a controlled english vocabulary, most of the heavy lifting is done. It will be easier to query, contrast, and combine their contents in various ways.
Some article databases online already offer useful tools for browsing work, but leave it to the researcher to answer questions requiring meaning interpretation of article contents. That could change.
If you could get library scientists involved and some money behind that project, it could generate an educational resource fairly quickly. My vision does go further than educating junior researchers, but that would require much more investment, a well-defined goal, and the participation of experts in the field.
I wonder whether AI Safety is well-developed enough to establish that its purpose is tractable. So far, I have not seen much more than:
expect AGI soon
AGI are dangerous
AGI are untrustworthy
Current AI tools pose no real danger (maybe)
AGI could revolutionize everything
We should or will make AGI
The models do provide evidence of existential danger, but not evidence of how to control it. There’s a downside to automation: technological unemployment; concentration of money and political power (typically); societal disruption; increased poverty. And as I mentioned, AGI are not understood in the obvious context of exploited labor. That’s a worrisome condition that, again, the AGI Safety field is clearly not ready to address. Financiallly unattractive as it is, that is a vision of the future of AGI Safety research, a group of researchers who understand when robots and disembodied AGI have developed sentience and deserve rights.