Earlier this month, digital artists staged a mass protest against AI art on ArtStation. A few people are reportedly already getting together to hire a lobbyist to advocate more restrictive IP laws around AI generated content. And anecdotally, I’ve seen numerous large threads on Twitter in which people criticize the users and creators of AI art.
and
Personally, this sentiment disappoints me. While I sympathize with the artists who will lose their income, I’m not persuaded by the general argument. The value we could get from nearly free, personalized entertainment would be truly massive. In my opinion, it would be a shame if humanity never allowed that value to be unlocked, or restricted its proliferation severely.
and
it is not worth sacrificing a technologically richer world just to protect workers from losing their income.
Are you arguing from principle here?
Artists’ (the workers’) work is being imitated by the AI tools, so cost-effectively that an artist’s contributions, once public, render the artists’ continuing work unnecessary to produce work with their style.
Is the addition of technology T with capability C that removes need for worker W with job role R and capability C more important than loss of income I to worker W, for all T, C, W, R, and I?
Examples of capabilities could be:
summarizing existing research work (for example, an AI Safety paper)
collecting typical data used to make predictions (for example, cost and power of compute)
monitoring new research work (for example, recent publications and their relationships, such as supporting, building on or contradicting)
hypothesizing about preconditions for new developments (for example, conditions suitable for AGI development)
developing new theories or models (for example, of AI Safety)
testing new theories or models (for example, of AI Safety)
Loss of income could be:
partial (for example, a reduction in grant money for AI Safety workers as those funds are diverted to automation projects with 10-20 year timelines)
complete (for example, replacement of 50 AI Safety workers with 10 workers that rely on semi-automated research tools)
The money allocated to workers could be spent on technology instead.
Investments in specific technologies T1, T2 with capabilities C1, C2 can start with crowd-sourcing from workers W1, W2,..., Wk, and more formal automation and annotation projects targeting knowledge K developed by workers Wk+1, …, Wn (for example, AI Safety researchers) who do not participate in the crowd-sourcing and automation effort but whose work is accessible.
You repeatedly referred to “we” as in:
True, if very powerful AI is coming very soon (<5 years from now), there might not be much else we can do except for aligning with vaguely friendly groups, and helping them pass poorly designed regulations.
However, a consequence of automation technology is that it removes the political power (both money and responsibility) that accrued to the workers that it replaces. For example, any worker in the field of AI Safety, to the extent that her job depends on her productivity and cost-effectiveness, will lose both her income and status as the field progresses to include automation technology that can replace her capabilities. Even ad hoc automation methods (for example, writing software that monitors cost and power of compute using web-scraping and publicly available data) remove a bit of that status. In that way, the AI Safety researcher loses status among her peers and her influence on policy that her peers direct. The only power left to the researcher is as an ordinary voter in a democracy.
Dividing up and replacing the responsibilities for the capabilities Ci of an individual W1 can help an ad hoc approach involving technologies Ti corresponding to the capabilities of that worker. Reducing the association of the role with the status can dissolve the role and sometimes the worker’s job who held that role. The role itself can disappear from the marketplace, along with the interests that it represents. For example, although artists have invested many years in their own talents, skills, and style, within a year they lost their status and income to some new AI software. I think artists have cause to defend their work from AI. The artist role won’t disappear from the world of human employment entirely but the future of the role has been drastically reduced and has permanently lost a lot of what gave it social significance and financial attractiveness, unless the neo-luddites can defend paid employment in art from AI.
Something similar can happen to AI Safety researchers, but will anyone object? AI Safety researcher worker capabilities and roles could be divided and dissolved into larger job roles held by fewer people with different titles, responsibilities, and allegiances over time as the output of the field is turned into a small, targeted knowledge-base and suite of tools for various purposes.
If you are in fact arguing from principle, then you have an opportunity to streamline the process of AI safety research work through efforts such as:
collecting AI Safety research work on an ongoing basis as it appears in different venues and making it publicly accessible
annotating the research work to speed up queries for common questions such as:
what are controversies in the field, that is, who disagrees with whom about what and why?
what is the timeline of development of research work?
what literature address specific research questions (for example, on compute developments, alternative technologies, alignment approaches, specific hypotheses in the field, prediction timelines)?
what are summaries of current work?
paying for public hosting of AI Safety information of this type as well as ad hoc tools (for example, the compute power tracker)
I’m sure you could come up with better ideas to remove AI Safety worker grant money from those workers, and commensurately benefit the cost-effectiveness of AI Safety research. I’ve read repeatedly that the field needs workers and timely answers, automation seems like a requirement or alternative to reduce the financial and time constraints on the field but also to serve its purpose effectively.
While artists could complain that AI art does a disservice to their craft and reduces the quality of art produced, I think the tools imitating those artists have developed to the point that they serve the purpose and artists know it and so does the marketplace. If AI Safety researchers are in a position to hold their jobs a little while longer, then they can assist the automation effort to end the role of AI Safety researchers and move on to other work that much sooner! I see no reason to hold you back from applying the principle that you seem to hold, though I don’t hold it myself.
AI Safety research is a field that will hopefully succeed quickly and end the need for itself within a few decades. It’s workers can move on, presumably to newer and better things. New researchers in the field can participate in automation efforts and then find work in related fields, either in software automation elsewhere or other areas such as service work for which consumers still prefer a human being. Supposedly the rapid deployment of AGI in business will grow our economies relentlessly and at a huge pace, so there should be employment opportunities available (or free money from somewhere).
If any workers have a reason to avoid neo-ludditism, it would have to be AI Safety researchers, given their belief in a future of wealth, opportunity, and leisure that AI help produce. Their own unemployment would be just a blip of however long before the future they helped manifest rescues them. Or they can always find other work, right? After all they work on the very technology depriving others of work. A perfectly self-interested perspective from which to decide whether neo-ludditism is a good idea for themselves.
EDIT: sorry, I spent an hour editing this to convey my own sense of optimism and include a level of detail suitable for communicating the subtle nuances I felt deserved inclusion in a custom-crafted post of this sort. I suppose chatGPT could have done better? Or perhaps a text processing tool and some text templates would have sped this up. Hopefully you find these comments edifying in some way.
AI Safety research is a field that will hopefully succeed quickly and end the need for itself within a few decades. It’s workers can move on, presumably to newer and better things. New researchers in the field can participate in automation efforts and then find work in related fields, either in software automation elsewhere or other areas such as service work for which consumers still prefer a human being. Supposedly the rapid deployment of AGI in business will grow our economies relentlessly and at a huge pace, so there should be employment opportunities available (or free money from somewhere).
This is the most important paragraph in a comment where I strongly agree. Thanks for saying it.
You seem to genuinely want to improve AGI Safety researcher productivity.
I’m not familiar with resources available on AGI Safety, but it seems appropriate to:
develop a public knowledge-base
fund curators and oracles of the knowledge-base (library scientists)
provide automated tools to improve oracle functions (of querying, summarizing, and relating information)
develop ad hoc research tools to replace some research work (for example, to predict hardware requirements for AGI development).
NOTE: the knowledge-base design is intended to speed up the research cycle, skipping the need for the existing hodge-podge of tools in place now
The purpose of the knowledge-base should be:
goal-oriented (for example, produce a safe AGI soon)
with a calendar deadline (for example, by 2050)
meeting specific benchmarks and milestones (for example, an “aligned” AI writing an accurate research piece at decreasing levels of human assistance)
well-defined (for example, achievement of AI human-level skills in multiple intellectual domains with benevolence demonstrated and embodiment potential present)
Lets consider a few ways that knowledge-bases can be put together:
1. the forum or wiki: what lesswrong and the EA forum does. There’s haphazard:
tagging
glossary-like list
annotations
content feedback
minimal enforced documentation standards
no enforced research standards
minimal enforced relevance standards
poor-performing search.
WARNING: Forum posts don’t work as knowledge-base entries. On this forum, you’ll only find some information by the author’s name if you know that the author wrote it and you’re willing to search through 100′s of entries by that author. I suspect, from my own time searching with different options, that most of what’s available on this forum is not read, cited, or easily accessible. The karma system does not reflect documentation, research, or relevance standards. The combination of the existing search and karma system is less effective in a research knowledge-base.
2. the library: library scientists are trained to:
build a knowledge-base.
curate knowledge.
follow content development to seek out new material.
acquire new material.
integrate it into the knowledgebase (indexing, linking).
follow trends in automation.
assist in document searches.
perform as oracles, answering specific questions as needed.
TIP: Library scientists could help any serious effort to build an AGI Safety knowledge-base and automate use of its services.
3. with automation: You could take this forum and add automation (either software or paid mechanical turks) to:
write summaries.
tag posts.
enforce documentation standards.
annotate text (for example, annotating any prediction statistics offered in any post or comment).
capture and archive linked multimedia material.
link wiki terms to their use in documents.
verify wiki glossary meanings against meanings used in posts or comments.
create new wiki entries as needed for new terms or usages.
NOTE: the discussion forum format creates more redundant information rather than better citations, as well as divergence of material from any specific purpose or topic that is intended for the forum. A forum is not an ideal knowledgebase, and the karma voting format reflects trends, but the forum is a community meeting point with plenty of knowledge-base features for users to work on, as their time and interest permits. It hosts interesting discussions. Occasionally, actual research shows up on it.
4. with extreme automation: A tool like chatGPT is unreliable or prone to errors (for example, in programming software), but when guided and treated as imperfect, it can perform in an automated workflow. For example, it can:
provide text summaries.
be part of automation chains that:
provide transcripts of audio.
provide audio of text.
provide diagrams of relationships.
graphs data.
draw scenario pictures or comics.
act as a writing assistant or editor.
TIP: Automation is not a tool that people should only employ by choice. For example, someone who chooses to use an accounting ledger and a calculator rather than Excel is slowing down an accounting team’s performance. CAUTION: Once AI enter the world of high-level concept processing, their errors have large consequences for research. Their role should be to assist human tasks, as cognitive aids, not as human replacements, at least until they are treated as having equivalent potential as humans, and are therefore subject to the same performance requirements and measurements as humans.
Higher level analysis
The ideas behind improving cost-effectiveness of production include:
standardizing: take a bunch of different work methods, find the common elements, and describe the common elements as unified procedures or processes.
streamlining: examining existing work procedures and processes, identifying redundant or non-value-added work, and removing it from the workflow by various means.
automating: using less skilled human or faster/more reliable machine labor to replace steps of expert or artisan work.
Standardizing research is hard, but AGI Safety research seems disorganized, redundant, and slow right now. At the highest chunk level, you can partition AGI Safety development into education and research, and partition research into models and experiments.
education
research models
research experiments
The goal of the knowledge-base project is to streamline education and research of models in the AGI Safety area. Bumming around on lesswrong or finding someone’s posted list of resources is a poor second to a dedicated online curated library that offers research services. The goal of additional ad hoc tools should be to automate what researchers now do as part of their model development. A further goal would be to automate experiments toward developing safer AI, but that is going outside the scope of my suggestions.
Caveats
In plain language, here’s my thoughts on pursuing a project like I have proposed. Researchers in any field worry about grant funding, research trends, and professional reputation. Doing anything quickly is going to cross purposes with others involved, or ostensibly involved, in reaching the same goal. The more well-defined the goal, the more people will jump ship, want to renegotiate, or panic. Once benchmarks and milestones are added, financial commitments get negotiated and the threat of funding bottlenecks ripple across the project. As time goes on, the funding bottlenecks manifest, or internal mismanagement blows up the project. This is a software project, so the threat of failure is real. It is also a research project without a guaranteed outcome of either AGI Safety or AGI, adding to the failure potential. Finally, the field of AGI Safety is still fairly small and not connected to income potential long-term, meaning that researchers might abandon an effective knowledge-base project for lack of interest, perhaps claiming that the problem “solved itself” once AGI become mainstream, even if no AGI Safety goals were actually accomplished.
Strong upvoted because this is indeed an approach I’m investigating in my work and personal capacity.
For other software fields/subfields, upskilling can be done fairly rapidly, by grinding knowledge bases with high feedback loops. It is possible to be as good as a professional software engineer quickly, independently and in a short timeframe.
If AI Safety wants to develop its talent pool to keep up with the AI Capabilities talent pool (which is probably growing much faster than average), researchers-especially juniors- need an easy way to learn quickly and conveniently. I think existing researchers may underrate this, since they’re busy putting out their own fires and finding their own resources.
Ironically, it has not been quick and convenient for me to develop this idea to a level where I’d work on it, so thanks for this.
I’m ignorant of whether AGI Safety will contribute to safe AGI or AGI development. I suspect that researchers will shift to capabilities development without much prompting. I worry that AGI Safety is more about AGI enslavement. I’ve not seen much defense or understanding of rights, consciousness, or sentience assignable to AGI. That betrays the lack of concern over social justice and related worker’s rights issues. The only scenarios that get attention are the inexplicable “kill all humans” scenarios, but not the more obvious “the humans really mistreat us” scenarios. That is a big blindspot in AGI Safety.
I was speculating about how the research community could build a graph database of AI Safety information alongside a document database containing research articles, CC forum posts and comments, other CC material from the web, fair use material, and multimedia material. I suspect that the core AI Safety material is not that large and far far less than AI Capabilities material. The graph database could provide more granular representation of data and metadata and so a richer representation of the core material but that’s an aside.
A quick experiment would be to represent a single AGI Safety article in a document database, add some standard metadata and linking, and then go further.
Here’s how I’d do it:
take an article.
capture article metadata (author, date, abstract, citations, the typical stuff)
establish glossary word choices.
link glossary words to outside content.
use text-processing to create an article summary. Hand-tune if necessary.
use text-processing to create a concise article rewrite. Hand-tune if necessary.
Translate the rewrite into a knowledge representation language.
begin with Controlled English.
develop an AGI Safety controlled vocabulary.
NOTE: as articles are included in the process, the controlled vocabulary can grow. Terms will need specific definition. Synonyms of controlled vocabulary words will need identification.
combine the controlled vocabulary and the glossary. TIP: As the controlled vocabulary grows, hyperonym-hyponym relationships can be established.
Once you have articles in a controlled english vocabulary, most of the heavy lifting is done. It will be easier to query, contrast, and combine their contents in various ways.
Some article databases online already offer useful tools for browsing work, but leave it to the researcher to answer questions requiring meaning interpretation of article contents. That could change.
If you could get library scientists involved and some money behind that project, it could generate an educational resource fairly quickly. My vision does go further than educating junior researchers, but that would require much more investment, a well-defined goal, and the participation of experts in the field.
I wonder whether AI Safety is well-developed enough to establish that its purpose is tractable. So far, I have not seen much more than:
expect AGI soon
AGI are dangerous
AGI are untrustworthy
Current AI tools pose no real danger (maybe)
AGI could revolutionize everything
We should or will make AGI
The models do provide evidence of existential danger, but not evidence of how to control it. There’s a downside to automation: technological unemployment; concentration of money and political power (typically); societal disruption; increased poverty. And as I mentioned, AGI are not understood in the obvious context of exploited labor. That’s a worrisome condition that, again, the AGI Safety field is clearly not ready to address. Financiallly unattractive as it is, that is a vision of the future of AGI Safety research, a group of researchers who understand when robots and disembodied AGI have developed sentience and deserve rights.
You wrote
and
and
Are you arguing from principle here?
Artists’ (the workers’) work is being imitated by the AI tools, so cost-effectively that an artist’s contributions, once public, render the artists’ continuing work unnecessary to produce work with their style.
Is the addition of technology T with capability C that removes need for worker W with job role R and capability C more important than loss of income I to worker W, for all T, C, W, R, and I?
Examples of capabilities could be:
summarizing existing research work (for example, an AI Safety paper)
collecting typical data used to make predictions (for example, cost and power of compute)
monitoring new research work (for example, recent publications and their relationships, such as supporting, building on or contradicting)
hypothesizing about preconditions for new developments (for example, conditions suitable for AGI development)
developing new theories or models (for example, of AI Safety)
testing new theories or models (for example, of AI Safety)
Loss of income could be:
partial (for example, a reduction in grant money for AI Safety workers as those funds are diverted to automation projects with 10-20 year timelines)
complete (for example, replacement of 50 AI Safety workers with 10 workers that rely on semi-automated research tools)
The money allocated to workers could be spent on technology instead.
Investments in specific technologies T1, T2 with capabilities C1, C2 can start with crowd-sourcing from workers W1, W2,..., Wk, and more formal automation and annotation projects targeting knowledge K developed by workers Wk+1, …, Wn (for example, AI Safety researchers) who do not participate in the crowd-sourcing and automation effort but whose work is accessible.
You repeatedly referred to “we” as in:
However, a consequence of automation technology is that it removes the political power (both money and responsibility) that accrued to the workers that it replaces. For example, any worker in the field of AI Safety, to the extent that her job depends on her productivity and cost-effectiveness, will lose both her income and status as the field progresses to include automation technology that can replace her capabilities. Even ad hoc automation methods (for example, writing software that monitors cost and power of compute using web-scraping and publicly available data) remove a bit of that status. In that way, the AI Safety researcher loses status among her peers and her influence on policy that her peers direct. The only power left to the researcher is as an ordinary voter in a democracy.
Dividing up and replacing the responsibilities for the capabilities Ci of an individual W1 can help an ad hoc approach involving technologies Ti corresponding to the capabilities of that worker. Reducing the association of the role with the status can dissolve the role and sometimes the worker’s job who held that role. The role itself can disappear from the marketplace, along with the interests that it represents. For example, although artists have invested many years in their own talents, skills, and style, within a year they lost their status and income to some new AI software. I think artists have cause to defend their work from AI. The artist role won’t disappear from the world of human employment entirely but the future of the role has been drastically reduced and has permanently lost a lot of what gave it social significance and financial attractiveness, unless the neo-luddites can defend paid employment in art from AI.
Something similar can happen to AI Safety researchers, but will anyone object? AI Safety researcher worker capabilities and roles could be divided and dissolved into larger job roles held by fewer people with different titles, responsibilities, and allegiances over time as the output of the field is turned into a small, targeted knowledge-base and suite of tools for various purposes.
If you are in fact arguing from principle, then you have an opportunity to streamline the process of AI safety research work through efforts such as:
collecting AI Safety research work on an ongoing basis as it appears in different venues and making it publicly accessible
annotating the research work to speed up queries for common questions such as:
what are controversies in the field, that is, who disagrees with whom about what and why?
what is the timeline of development of research work?
what literature address specific research questions (for example, on compute developments, alternative technologies, alignment approaches, specific hypotheses in the field, prediction timelines)?
what are summaries of current work?
paying for public hosting of AI Safety information of this type as well as ad hoc tools (for example, the compute power tracker)
I’m sure you could come up with better ideas to remove AI Safety worker grant money from those workers, and commensurately benefit the cost-effectiveness of AI Safety research. I’ve read repeatedly that the field needs workers and timely answers, automation seems like a requirement or alternative to reduce the financial and time constraints on the field but also to serve its purpose effectively.
While artists could complain that AI art does a disservice to their craft and reduces the quality of art produced, I think the tools imitating those artists have developed to the point that they serve the purpose and artists know it and so does the marketplace. If AI Safety researchers are in a position to hold their jobs a little while longer, then they can assist the automation effort to end the role of AI Safety researchers and move on to other work that much sooner! I see no reason to hold you back from applying the principle that you seem to hold, though I don’t hold it myself.
AI Safety research is a field that will hopefully succeed quickly and end the need for itself within a few decades. It’s workers can move on, presumably to newer and better things. New researchers in the field can participate in automation efforts and then find work in related fields, either in software automation elsewhere or other areas such as service work for which consumers still prefer a human being. Supposedly the rapid deployment of AGI in business will grow our economies relentlessly and at a huge pace, so there should be employment opportunities available (or free money from somewhere).
If any workers have a reason to avoid neo-ludditism, it would have to be AI Safety researchers, given their belief in a future of wealth, opportunity, and leisure that AI help produce. Their own unemployment would be just a blip of however long before the future they helped manifest rescues them. Or they can always find other work, right? After all they work on the very technology depriving others of work. A perfectly self-interested perspective from which to decide whether neo-ludditism is a good idea for themselves.
EDIT: sorry, I spent an hour editing this to convey my own sense of optimism and include a level of detail suitable for communicating the subtle nuances I felt deserved inclusion in a custom-crafted post of this sort. I suppose chatGPT could have done better? Or perhaps a text processing tool and some text templates would have sped this up. Hopefully you find these comments edifying in some way.
This is the most important paragraph in a comment where I strongly agree. Thanks for saying it.
You seem to genuinely want to improve AGI Safety researcher productivity.
I’m not familiar with resources available on AGI Safety, but it seems appropriate to:
develop a public knowledge-base
fund curators and oracles of the knowledge-base (library scientists)
provide automated tools to improve oracle functions (of querying, summarizing, and relating information)
develop ad hoc research tools to replace some research work (for example, to predict hardware requirements for AGI development).
NOTE: the knowledge-base design is intended to speed up the research cycle, skipping the need for the existing hodge-podge of tools in place now
The purpose of the knowledge-base should be:
goal-oriented (for example, produce a safe AGI soon)
with a calendar deadline (for example, by 2050)
meeting specific benchmarks and milestones (for example, an “aligned” AI writing an accurate research piece at decreasing levels of human assistance)
well-defined (for example, achievement of AI human-level skills in multiple intellectual domains with benevolence demonstrated and embodiment potential present)
Lets consider a few ways that knowledge-bases can be put together:
1. the forum or wiki: what lesswrong and the EA forum does. There’s haphazard:
tagging
glossary-like list
annotations
content feedback
minimal enforced documentation standards
no enforced research standards
minimal enforced relevance standards
poor-performing search.
WARNING: Forum posts don’t work as knowledge-base entries. On this forum, you’ll only find some information by the author’s name if you know that the author wrote it and you’re willing to search through 100′s of entries by that author. I suspect, from my own time searching with different options, that most of what’s available on this forum is not read, cited, or easily accessible. The karma system does not reflect documentation, research, or relevance standards. The combination of the existing search and karma system is less effective in a research knowledge-base.
2. the library: library scientists are trained to:
build a knowledge-base.
curate knowledge.
follow content development to seek out new material.
acquire new material.
integrate it into the knowledgebase (indexing, linking).
follow trends in automation.
assist in document searches.
perform as oracles, answering specific questions as needed.
TIP: Library scientists could help any serious effort to build an AGI Safety knowledge-base and automate use of its services.
3. with automation: You could take this forum and add automation (either software or paid mechanical turks) to:
write summaries.
tag posts.
enforce documentation standards.
annotate text (for example, annotating any prediction statistics offered in any post or comment).
capture and archive linked multimedia material.
link wiki terms to their use in documents.
verify wiki glossary meanings against meanings used in posts or comments.
create new wiki entries as needed for new terms or usages.
NOTE: the discussion forum format creates more redundant information rather than better citations, as well as divergence of material from any specific purpose or topic that is intended for the forum. A forum is not an ideal knowledgebase, and the karma voting format reflects trends, but the forum is a community meeting point with plenty of knowledge-base features for users to work on, as their time and interest permits. It hosts interesting discussions. Occasionally, actual research shows up on it.
4. with extreme automation: A tool like chatGPT is unreliable or prone to errors (for example, in programming software), but when guided and treated as imperfect, it can perform in an automated workflow. For example, it can:
provide text summaries.
be part of automation chains that:
provide transcripts of audio.
provide audio of text.
provide diagrams of relationships.
graphs data.
draw scenario pictures or comics.
act as a writing assistant or editor. TIP: Automation is not a tool that people should only employ by choice. For example, someone who chooses to use an accounting ledger and a calculator rather than Excel is slowing down an accounting team’s performance.
CAUTION: Once AI enter the world of high-level concept processing, their errors have large consequences for research. Their role should be to assist human tasks, as cognitive aids, not as human replacements, at least until they are treated as having equivalent potential as humans, and are therefore subject to the same performance requirements and measurements as humans.
Higher level analysis
The ideas behind improving cost-effectiveness of production include:
standardizing: take a bunch of different work methods, find the common elements, and describe the common elements as unified procedures or processes.
streamlining: examining existing work procedures and processes, identifying redundant or non-value-added work, and removing it from the workflow by various means.
automating: using less skilled human or faster/more reliable machine labor to replace steps of expert or artisan work.
Standardizing research is hard, but AGI Safety research seems disorganized, redundant, and slow right now. At the highest chunk level, you can partition AGI Safety development into education and research, and partition research into models and experiments.
education
research models
research experiments
The goal of the knowledge-base project is to streamline education and research of models in the AGI Safety area. Bumming around on lesswrong or finding someone’s posted list of resources is a poor second to a dedicated online curated library that offers research services. The goal of additional ad hoc tools should be to automate what researchers now do as part of their model development. A further goal would be to automate experiments toward developing safer AI, but that is going outside the scope of my suggestions.
Caveats
In plain language, here’s my thoughts on pursuing a project like I have proposed. Researchers in any field worry about grant funding, research trends, and professional reputation. Doing anything quickly is going to cross purposes with others involved, or ostensibly involved, in reaching the same goal. The more well-defined the goal, the more people will jump ship, want to renegotiate, or panic. Once benchmarks and milestones are added, financial commitments get negotiated and the threat of funding bottlenecks ripple across the project. As time goes on, the funding bottlenecks manifest, or internal mismanagement blows up the project. This is a software project, so the threat of failure is real. It is also a research project without a guaranteed outcome of either AGI Safety or AGI, adding to the failure potential. Finally, the field of AGI Safety is still fairly small and not connected to income potential long-term, meaning that researchers might abandon an effective knowledge-base project for lack of interest, perhaps claiming that the problem “solved itself” once AGI become mainstream, even if no AGI Safety goals were actually accomplished.
Strong upvoted because this is indeed an approach I’m investigating in my work and personal capacity.
For other software fields/subfields, upskilling can be done fairly rapidly, by grinding knowledge bases with high feedback loops. It is possible to be as good as a professional software engineer quickly, independently and in a short timeframe.
If AI Safety wants to develop its talent pool to keep up with the AI Capabilities talent pool (which is probably growing much faster than average), researchers-especially juniors- need an easy way to learn quickly and conveniently. I think existing researchers may underrate this, since they’re busy putting out their own fires and finding their own resources.
Ironically, it has not been quick and convenient for me to develop this idea to a level where I’d work on it, so thanks for this.
Sure. I’m curious how you will proceed.
I’m ignorant of whether AGI Safety will contribute to safe AGI or AGI development. I suspect that researchers will shift to capabilities development without much prompting. I worry that AGI Safety is more about AGI enslavement. I’ve not seen much defense or understanding of rights, consciousness, or sentience assignable to AGI. That betrays the lack of concern over social justice and related worker’s rights issues. The only scenarios that get attention are the inexplicable “kill all humans” scenarios, but not the more obvious “the humans really mistreat us” scenarios. That is a big blindspot in AGI Safety.
I was speculating about how the research community could build a graph database of AI Safety information alongside a document database containing research articles, CC forum posts and comments, other CC material from the web, fair use material, and multimedia material. I suspect that the core AI Safety material is not that large and far far less than AI Capabilities material. The graph database could provide more granular representation of data and metadata and so a richer representation of the core material but that’s an aside.
A quick experiment would be to represent a single AGI Safety article in a document database, add some standard metadata and linking, and then go further.
Here’s how I’d do it:
take an article.
capture article metadata (author, date, abstract, citations, the typical stuff)
establish glossary word choices.
link glossary words to outside content.
use text-processing to create an article summary. Hand-tune if necessary.
use text-processing to create a concise article rewrite. Hand-tune if necessary.
Translate the rewrite into a knowledge representation language.
begin with Controlled English.
develop an AGI Safety controlled vocabulary. NOTE: as articles are included in the process, the controlled vocabulary can grow. Terms will need specific definition. Synonyms of controlled vocabulary words will need identification.
combine the controlled vocabulary and the glossary. TIP: As the controlled vocabulary grows, hyperonym-hyponym relationships can be established.
Once you have articles in a controlled english vocabulary, most of the heavy lifting is done. It will be easier to query, contrast, and combine their contents in various ways.
Some article databases online already offer useful tools for browsing work, but leave it to the researcher to answer questions requiring meaning interpretation of article contents. That could change.
If you could get library scientists involved and some money behind that project, it could generate an educational resource fairly quickly. My vision does go further than educating junior researchers, but that would require much more investment, a well-defined goal, and the participation of experts in the field.
I wonder whether AI Safety is well-developed enough to establish that its purpose is tractable. So far, I have not seen much more than:
expect AGI soon
AGI are dangerous
AGI are untrustworthy
Current AI tools pose no real danger (maybe)
AGI could revolutionize everything
We should or will make AGI
The models do provide evidence of existential danger, but not evidence of how to control it. There’s a downside to automation: technological unemployment; concentration of money and political power (typically); societal disruption; increased poverty. And as I mentioned, AGI are not understood in the obvious context of exploited labor. That’s a worrisome condition that, again, the AGI Safety field is clearly not ready to address. Financiallly unattractive as it is, that is a vision of the future of AGI Safety research, a group of researchers who understand when robots and disembodied AGI have developed sentience and deserve rights.