Completed this, but was difficult!
ElliotJDavies
Karma: 1,956
It takes a significant amount of time to mark a test task. But this can be fixed by just adjusting the height of the screening bar, as opposed to using credentialist and biased methods (like looking at someone’s LinkedIn profile or CV).
My guess is that, for many orgs, the time cost of assessing the test task is larger than the financial cost of paying candidates to complete the test task
This is an empirical question, and I suspect is not true. For example, it took me 10 minutes to mark each candidates 1 hour test task. So my salary would need to be 6* higher (per unit time) than the test task payment for this to be true.
I also think the justice-implications of compensating applicants are unclear (offering pay for longer tasks may make them more accessible to poorer applicants)
This is a good point.
I’d be curious to know the marginal cost of an additional attendee—I’d put it between 5-30 USD, assuming they attend all sessions.
Assuming you update your availability on swapcard, and that you would get value out of attending a conference, I suspect attending is positive EV.
Paying candidates to complete a test task likely increases inequality, credentialism and decreases candidate quality. If you pay candidates for their time, you’re likely to accept less candidates and lower variance candidates into the test task stage. Orgs can continue to pay top candidates to complete the test task, if they believe it measurably decreases the attrition rate, but give all candidates that pass an anonymised screening bar the chance to complete a test task.
A man of integrity
I read Saul’s comment to be discussing two different events. 1 event he was uninvited to, the other he would have been able to attend if he would have so wished.
potential employers, neighbors, and others might come across it
I think saying “I am against scientific racism” is within the overton window, and it would be extraordinarily unlikely to be”cancelled” as a result of that. This level of risk aversion is straightforwardly deleterious for our community and wider society.
While I’m cognizant of the downsides of a centralized authority deciding what events can and cannot be promoted here, I think the need to maintain sufficient distance between EA and this sort of event outweighs those downsides.
Can I also nudge people to be more vocal when they perceive there to a problem? I find it’s extremely common that when a problem is unfolding nobody says anything.
Even the post above is posted anonymously. To me, I see this as being part of a wider trend where people don’t feel comfortable expressing their viewpoint openly, which I think is not super healthy.
Sentient AI ≠ AI Suffering.
Biological life forms experience unequal (asymmetrical) amounts of pleasure and pain. This asymmetry is important. It’s why you cannot make up for starving someone for a week by giving them food for a week.
This is true for biological life, because a selection pressure was applied (evolution by natural selection). This selection pressure is necessitated by entropy, because it’s easier to die than it is to live. Many circumstances result in death, only a narrow band of circumstances results in life. Incidentally, this is why you spend most of your life in a temperature controlled environment.
The crux: there’s no reason to think that a similar selection effect is being applied to AI models. LLMs, if they were sentient, would be equally as likely to enjoy predicting the next token as to dislike predicting the next token.
you claim that it’s relevant when comparing lifesaving interventions with life-improving interventions, but it’s not quite obvious to me how to think about this: say a condition C has a disability weight of D, and we cure it in some people who also have condition X with disability weight Y. How many DALYs did we avert? Do they compound additively, and the answer is D? Or multiplicatively, giving D*(1-Y)? I’d imagine they will in general compound idiosyncratically, but assuming we can’t gather empirical information for every single combination of conditions, what should our default assumption be? I think there’s arguments each way, and this will have an impact on whether failing to discount for a typical background level of disability is relevant to between-cause comparisons or not.
(Low Confidence, this is a new area for me).
DALYs averted = Without Intervention (Years of life lost + Years Lived With Disability) - After intervention (Years of life lost + Years Lived With Disability)
Years Lived With Disability (YLD) = Disability Weight * Duration .If the duration of the disability is the entire lifespan of somebody, then it becomes quite a significant factor.
Is it obvious that disability weights don’t already include this?
Do you mean age wighting, or discounting or disability weighting? The crux of this post (to the extent I understand it) is that disability weights are not being calculated or factored in for interventions. I.e. post intervention Years Lived With Disability is assumed to be zero, and Years of Life Lost is also assumed to be zero.
Regarding age weighting or discounting: I do think the burden of proof is on the organisation doing the Cost Effectiveness analysis to elucidate on what there discounting entails. I’d argue that OP has done their due diligence here by delving into the WHOs methodology for age weighting
Disclosure: I discussed this with OP (Mikołaj) previous to it being posted.
Low confidence in what I am saying being correct, I am brand new to this area and trying to get my head around it.
Yes, we can fix this fairly easily. We should decrease the number of DALYs gained from interventions (or components of interventions) that saves lives by roughly 10%.
I agree this is not a bad way to fix post-hoc. One concern I would have using this model going forward, is that you may overweight interventions that leave the beneficiary with some sort of long lasting disability.
Take the example of administering snakebite anti-venom, if we assumed that 1⁄2 of beneficiaries that counterfactually survive are likely to have lost a limb, if you don’t account for that in your DALY’s averted, then snakebite anti-venom’s DALYs averted will be artificially inflated compared to an interventions who’s counterfactual beneficiaries don’t have high levels of Years Lived with Disability.
Sounds like a very interesting intervention. I’d be keen to give it a try but I am only in the UK for 1-2 weeks a year.
To a large extent I don’t buy this. Academics and Journalists could interview an arbitrary EA forum user on a particular area if they wanted to get up to speed quickly. The fact they seem not to do this, in addition to not giving a right to reply, makes me think they’re not truth-seeking.
This is great!
Just to note: I have a COI in commenting on this subject.
I strong downvoted your comment, as it reads to me as making bold claims whilst providing little supporting evidence. References to “lots of people in this area” could be considered to be a use case of the bandwagon fallacy.
As you write:
The result will be a singularity, understood as a fundamental discontinuity in human history beyond which our fate depends largely on how we interact with artificial agents
The discontinuity is a result of humans no longer being the smartest agents in the world, and no longer being in control of our own fate. After this point, we’ve entered an event horizon where the output is almost entirely unforeseeable.
If you have accelerating growth that isn’t sustained for very long, you get something like population growth from 1800-2000
If, after surpassing humans, intelligence “grows” exponentially for another 200 years, do you not think we’ve passed an event horizon? I certainly do!
If not, using the metric of single agent intelligence (i.e. not the sum of intelligence in a group of agents), at what point during an exponential growth curve that intersects human level intelligence, would you defining as crossing the event horizon?
I feel this claim is disconnected with the definition of the singularity given in the paper:
The singularity hypothesis begins with the supposition that artificial agents will gain the ability to improve their own intelligence. From there, it is claimed that the intelligence of artificial agents will grow at a rapidly accelerating rate, producing an intelligence explosion in which artificial agents quickly become orders of magnitude more intelligent than their human creators. The result will be a singularity, understood as a fundamental discontinuity in human history beyond which our fate depends largely on how we interact with artificial agents
Further in the paper you write:
The singularity hypothesis posits a sustained period of accelerating growth in the general intelligence of artificial agents.
[Emphasis mine]. I can’t see any reference for either the original definition and later addition of “sustained”.
Intelligence Explosion: For a sustained period
[...]
Extraordinary claims require extraordinary evidence: Proposing that exponential or hyperbolic growth will occur for a prolonged period [Emphasis mine]I’m not sure why “prolonged period” or “sustained” was used here?
I am also not sure what is meant by prolonged period? 5 years? 100 years?
For the answer to the above, why do you believe would this be required?
Just to help nail down the crux here, I don’t see why more than a few days of an intelligence explosion is required for a singularity event.
Circuits’ energy requirements have massively increased—increasing costs and overheating.[6]
I’m not sure I understand this claim, and I can’t see that it’s supported by the cited paper.
Is the claim that energy costs have increased faster than computation? This would be cruxy, but it would also be incorrect.
I see a dynamic playing out here, where a user has made a falsifiable claim, I have attempted to falsify it, and you’ve attempted to deny that the claim is falsifiable at all.
I recognise it’s easy to stumble into these dynamics, but we must acknowledge that this is epistemically destructive.
I don’t think we should dismiss empirical data so quickly when it’s brought to the table—that sets a bad precedent.
I can also provide empirical data on this if that is the crux here?
Notice that we are discussing a concrete empirical data point, that represents a 600% difference, while you’ve given a theoretical upper bound of 100%. That leaves a 500% delta.
Would you be able to provide any concrete figures here?
I view pointing to opportunity cost in the abstract as essentially an appeal to ignorance.
Not to say that opportunity costs do not exist, but you’ve failed to concretise them in a way, and that makes it hard to find the truth here.
I could make similar appeals to ignorance in support of my argument, like the idea the benefit of getting a better candidate is very high, as candidate performance is fat-tailed ect. - but I believe this is similarly epistemically destructive. If I were to make a similar claim, I would likely attempt to concretise it.