I’d say that’s a fair assessment — one wrinkle that isn’t a critique of what you wrote, but seems worth mentioning, is that it’s an open question if these are the metrics we should be optimizing for. If we were part of academia, citations would be the de facto target, but we have different incentives (we’re not trying to impress tenure committees). That said, the more citations the better of course.
As you say, if STV is true, it would essentially introduce an entirely new subfield. It would also have implications for items like AI safety and those may outweigh its academic impact. The question we’re looking at is how to navigate questions of support, utility, and impact here: do we put our (unfortunately rather small) resources toward academic writing and will that get us to the next step of support, or do we put more visceral real-world impact first (can we substantially improve peoples’ lives? How much and how many?), or do we go all out towards AI safety?
It’s of course possible to be wrong; I’m also understanding it’s possible to be right, but take the wrong strategic path and run out of gas. Basically I’m a little worried that racking up academic metrics like citations is less a panacea than it might appear, and we’re looking to hedge our bets here.
For what it’s worth, we’ve been interfacing with various groups working on emotional wellness neurotech and one internal metric I’m tracking is how useful a framework STV is to these groups; here’s Jay Sanguinetti explaining STV to Shinzen Young (first part of the interview):
I think of the metrics I mentioned above as proxies rather than as the underlying targets, which is some combination of:
a) Is STV true? b) Conditional upon STV being true, is it useful?
What my forecasting questions aimed to do is shedding light on a). I agree that academia and citations isn’t the best proxy. They may in some cases have conservatism bias (I think trusting the apparent academic consensus on AI risk in 2014 would’ve been a mistake for early EAs), but are also not immune to falseties/crankery (cf replication crisis). In addition, standards for truth and usefulness are different within EA circles than academia, partially because we are trying to answer different questions.
This is especially an issue as the areas that QRI is likely to interact with (consciousness, psychedelics) seem from the outside to be more prone than average to falseness and motivated cognition, including within academia.
This is what I was trying to get at with “will Luke Muelhauser say statements to the effect that the Symmetry Theory of Valence is substantively true?” because Luke is a non-QRI affiliated person within EA who’s a) respected and b) have thought about concepts adjacent to QRI’s work. Bearing in mind that Luke is very far from a perfect oracle, I would still trust Luke’s judgement on this more than an arbitrarily selected academic in an adjacent field.
I think the actual question I’m interested in is something like “In X year, will a panel of well-respected EAs a) not affiliated with QRI and b) have very different thoughts from each other and c)who have thought about things adjacent to QRI’s work have updated to believing STV to be substantively true” but I was unable to come up with a clean question operationalization in the relatively brief amount of time I gave myself to come up with this.
People are free to counterpropose and make their own questions.
Hi Linch, that’s very well put. I would also add a third possibility (c), which is “is STV false but generative.” — I explore this a little here, with the core thesis summarized in this graphic:
I.e., STV could be false in a metaphysical sense, but insofar as the brain is a harmonic computer (a strong reframe of CSHW), it could be performing harmonic gradient descent. Fully expanded, there would be four cases:
STV true, STHR true
STV true, STHR false
STV false, STHR true
STV false, STHR false
Of course, ‘true and false’ are easier to navigate if we can speak of absolutes; STHR is a model, and ‘all models are wrong; some are useful.’
I’d say that’s a fair assessment — one wrinkle that isn’t a critique of what you wrote, but seems worth mentioning, is that it’s an open question if these are the metrics we should be optimizing for. If we were part of academia, citations would be the de facto target, but we have different incentives (we’re not trying to impress tenure committees). That said, the more citations the better of course.
As you say, if STV is true, it would essentially introduce an entirely new subfield. It would also have implications for items like AI safety and those may outweigh its academic impact. The question we’re looking at is how to navigate questions of support, utility, and impact here: do we put our (unfortunately rather small) resources toward academic writing and will that get us to the next step of support, or do we put more visceral real-world impact first (can we substantially improve peoples’ lives? How much and how many?), or do we go all out towards AI safety?
It’s of course possible to be wrong; I’m also understanding it’s possible to be right, but take the wrong strategic path and run out of gas. Basically I’m a little worried that racking up academic metrics like citations is less a panacea than it might appear, and we’re looking to hedge our bets here.
For what it’s worth, we’ve been interfacing with various groups working on emotional wellness neurotech and one internal metric I’m tracking is how useful a framework STV is to these groups; here’s Jay Sanguinetti explaining STV to Shinzen Young (first part of the interview):
https://open.spotify.com/episode/6cI9pZHzT9sV1tVwoxncWP?si=S1RgPs_CTYuYQ4D-adzNnA&dl_branch=1
I think of the metrics I mentioned above as proxies rather than as the underlying targets, which is some combination of:
a) Is STV true?
b) Conditional upon STV being true, is it useful?
What my forecasting questions aimed to do is shedding light on a). I agree that academia and citations isn’t the best proxy. They may in some cases have conservatism bias (I think trusting the apparent academic consensus on AI risk in 2014 would’ve been a mistake for early EAs), but are also not immune to falseties/crankery (cf replication crisis). In addition, standards for truth and usefulness are different within EA circles than academia, partially because we are trying to answer different questions.
This is especially an issue as the areas that QRI is likely to interact with (consciousness, psychedelics) seem from the outside to be more prone than average to falseness and motivated cognition, including within academia.
This is what I was trying to get at with “will Luke Muelhauser say statements to the effect that the Symmetry Theory of Valence is substantively true?” because Luke is a non-QRI affiliated person within EA who’s a) respected and b) have thought about concepts adjacent to QRI’s work. Bearing in mind that Luke is very far from a perfect oracle, I would still trust Luke’s judgement on this more than an arbitrarily selected academic in an adjacent field.
I think the actual question I’m interested in is something like “In X year, will a panel of well-respected EAs a) not affiliated with QRI and b) have very different thoughts from each other and c)who have thought about things adjacent to QRI’s work have updated to believing STV to be substantively true” but I was unable to come up with a clean question operationalization in the relatively brief amount of time I gave myself to come up with this.
People are free to counterpropose and make their own questions.
Hi Linch, that’s very well put. I would also add a third possibility (c), which is “is STV false but generative.” — I explore this a little here, with the core thesis summarized in this graphic:
I.e., STV could be false in a metaphysical sense, but insofar as the brain is a harmonic computer (a strong reframe of CSHW), it could be performing harmonic gradient descent. Fully expanded, there would be four cases:
STV true, STHR true
STV true, STHR false
STV false, STHR true
STV false, STHR false
Of course, ‘true and false’ are easier to navigate if we can speak of absolutes; STHR is a model, and ‘all models are wrong; some are useful.’