As someone who is exploring a transition to full-time TAIS work, I appreciate this series of posts and other efforts like it. Having a detailed public critique and “adversarial summary” of multiple organizations will make any future due-diligence process (by me or others) much quicker and easier.
That said, I am pretty unconvinced of the key points of this post.
I suspect some of the criticism is more convincing and more relevant to those who share the authors’ views on specific technical questions related to TAIS.
My own personal experience (very limited, detailed below) engaging with Conjecture and their public work conflicts with the claim that it is low quality and that they react defensively to criticism.
Some points are vague and I personally do not find them damning even if true.
Below, I’ll elaborate on each bullet.
Criticism which seems dependent on technical views on TAIS
I agree with Eliezer’s views on the difficulty and core challenges posed by the development of AGI.
I realize these views are not necessarily consensus among EAs or the TAIS community, and I don’t want to litigate them on the object level here. I merely want to remark that, having personally accepted them, I find some of the criticism and suggestions offered in this post unconvincing, overly generic, or even misguided (hire more experienced ML researchers, engage with the broader ML community, “average ML conference paper” as a quality benchmark, etc.) for reasons that aren’t specific to Conjecture.
My own experience engaging directly with Conjecture
I myself am skeptical of Conjecture’s current alignment plan and much of their current and past research, as I understand it. However, in engaging with some of their published work, I have not found it to be low quality or disconnected from relevant work of others, and some of the views I disagree most strongly with are actually shared by other prominent TAIS researchers or funders.
I commented (obliquely) on their CoEm strategy in the thread starting here, and on a post by one of their researchers starting here.
These posts by me cite the work of researchers at Conjecture, and some of them are partially criticism or counters to views that I perceive they hold:
Conjecture has not engaged directly with most of my posts or comments, but this is explained by the fact that the posts and comments have received very little engagement in general. My point here is mainly that I would not have written many of the posts and comments above at all, if I did not personally find the work which they cite and / or criticize to be above a pretty high quality threshold.
To the very limited degree that Conjecture’s researchers have engaged directly with my own work, I did not find it to be defensive.
I think Conjecture is hopelessly confused and doomed to fail, but mostly for inside-view technical reasons mentioned in the previous section, and my own criticism is not really specific to Conjecture. When the criteria I most care about are graded on a curve, I think Conjecture and their research stacks up well against most other organizations. My views on this are low-confidence and based on scant firsthand evidence, but the information in this post was not a meaningful update.
These are important points to consider when diligencing any organization, and even more critical when the organization in question is working on TAIS. I appreciate the authors compiling and summarizing their views on these topics.
However, there is limited detail to the accusations and criticism in these sections. Even if all of the points were true and exactly as bad as the authors claim or imply, none of them are severe enough that I would consider them damning or even far outside the norm for a non-TAIS organization.
I think that EA and particularly TAIS organizations should strive to meet a higher standard, and agree Conjecture has room for improvement in these areas, but there is nothing in these sections which I consider a dealbreaker if I were choosing to work for or collaborate with or fund them.
Given the sparsity and viewpoint diversity of TAIS organizations, unless the issues on these topics are extremely serious, I personally would weigh the following factors much more heavily when evaluating TAIS organizations:
Clarity of thought, epistemic hygiene, and general sanity of the researchers at the organization.
As someone who is exploring a transition to full-time TAIS work, I appreciate this series of posts and other efforts like it. Having a detailed public critique and “adversarial summary” of multiple organizations will make any future due-diligence process (by me or others) much quicker and easier.
That said, I am pretty unconvinced of the key points of this post.
I suspect some of the criticism is more convincing and more relevant to those who share the authors’ views on specific technical questions related to TAIS.
My own personal experience (very limited, detailed below) engaging with Conjecture and their public work conflicts with the claim that it is low quality and that they react defensively to criticism.
Some points are vague and I personally do not find them damning even if true.
Below, I’ll elaborate on each bullet.
Criticism which seems dependent on technical views on TAIS
I agree with Eliezer’s views on the difficulty and core challenges posed by the development of AGI.
In particular, I agree with his 2021 assessment of the AI safety field here, and List of Lethalities 38-41.
I realize these views are not necessarily consensus among EAs or the TAIS community, and I don’t want to litigate them on the object level here. I merely want to remark that, having personally accepted them, I find some of the criticism and suggestions offered in this post unconvincing, overly generic, or even misguided (hire more experienced ML researchers, engage with the broader ML community, “average ML conference paper” as a quality benchmark, etc.) for reasons that aren’t specific to Conjecture.
My own experience engaging directly with Conjecture
I myself am skeptical of Conjecture’s current alignment plan and much of their current and past research, as I understand it. However, in engaging with some of their published work, I have not found it to be low quality or disconnected from relevant work of others, and some of the views I disagree most strongly with are actually shared by other prominent TAIS researchers or funders.
I commented (obliquely) on their CoEm strategy in the thread starting here, and on a post by one of their researchers starting here.
These posts by me cite the work of researchers at Conjecture, and some of them are partially criticism or counters to views that I perceive they hold:
“Aligned” foundation models don’t imply aligned systems
Gradient hacking via actual hacking
LLM cognition is probably not human-like
Conjecture has not engaged directly with most of my posts or comments, but this is explained by the fact that the posts and comments have received very little engagement in general. My point here is mainly that I would not have written many of the posts and comments above at all, if I did not personally find the work which they cite and / or criticize to be above a pretty high quality threshold.
To the very limited degree that Conjecture’s researchers have engaged directly with my own work, I did not find it to be defensive.
I think Conjecture is hopelessly confused and doomed to fail, but mostly for inside-view technical reasons mentioned in the previous section, and my own criticism is not really specific to Conjecture. When the criteria I most care about are graded on a curve, I think Conjecture and their research stacks up well against most other organizations. My views on this are low-confidence and based on scant firsthand evidence, but the information in this post was not a meaningful update.
Other points
Regarding:
CEO trustworthiness and consistency
funding sources / governance structure / profit motive
scaling too quickly
These are important points to consider when diligencing any organization, and even more critical when the organization in question is working on TAIS. I appreciate the authors compiling and summarizing their views on these topics.
However, there is limited detail to the accusations and criticism in these sections. Even if all of the points were true and exactly as bad as the authors claim or imply, none of them are severe enough that I would consider them damning or even far outside the norm for a non-TAIS organization.
I think that EA and particularly TAIS organizations should strive to meet a higher standard, and agree Conjecture has room for improvement in these areas, but there is nothing in these sections which I consider a dealbreaker if I were choosing to work for or collaborate with or fund them.
Given the sparsity and viewpoint diversity of TAIS organizations, unless the issues on these topics are extremely serious, I personally would weigh the following factors much more heavily when evaluating TAIS organizations:
Clarity of thought, epistemic hygiene, and general sanity of the researchers at the organization.
The organization’s operational adequacy (relative to other orgs)
Understanding of, and a plan to actually work on (or at least engage with) the most difficult and important problems.