I figure most people don’t know a statistician (I don’t) but there are plenty of people in LessWrong discussion who know how to do a power calculation so it might be good to start there (or just to dig a bit deeper here).
It really won’t help address the problem I’m talking about at all, which is unknown design flaws/statistical techniques/study design tools. Once you’ve figured out that you have a problem like “how should I power my study?”, smart people plus Internet is fine; I’m worried about the other 10 issues we haven’t noticed yet. That’s the kind of thing that statisticians are useful for.
Fortunately, it turns out you can still talk to statisticians even if you don’t know them personally. If you’re spending money on your study, you could even go so far as to hire a consultant. I also know statisticians and would be happy to refer Jonathon.
As someone who did a lot of study design in undergraduate, is currently a “data scientist”, and considers myself smart, I can confirm that I still make approximately 10 huge mistakes every time I run a study.
Yeah, if you give me the contact info of a statistician that you recommend that would be great. I don’t know if we have the budget for it, but I would definitely reach out.
I’m checking for people who would be interested in doing it pro bono. If that doesn’t work, I’m 99% sure you can find some people to fund a couple consultant-hours.
I don’t know if we have the budget for it
Not to put too fine a point on it, but if the alternative is TLYCS designing the experiment themselves, this is pretty much like running a charity that spends nothing on overhead. It looks good on paper, but in reality, that last bit of money is a huge effectiveness multiplier.
I figure most people don’t know a statistician (I don’t) but there are plenty of people in LessWrong discussion who know how to do a power calculation so it might be good to start there (or just to dig a bit deeper here).
It really won’t help address the problem I’m talking about at all, which is unknown design flaws/statistical techniques/study design tools. Once you’ve figured out that you have a problem like “how should I power my study?”, smart people plus Internet is fine; I’m worried about the other 10 issues we haven’t noticed yet. That’s the kind of thing that statisticians are useful for.
Fortunately, it turns out you can still talk to statisticians even if you don’t know them personally. If you’re spending money on your study, you could even go so far as to hire a consultant. I also know statisticians and would be happy to refer Jonathon.
Makes sense. Also, if the EA survey is redone, that might be an even more important place to have a statistician.
As someone who did a lot of study design in undergraduate, is currently a “data scientist”, and considers myself smart, I can confirm that I still make approximately 10 huge mistakes every time I run a study.
Yeah, if you give me the contact info of a statistician that you recommend that would be great. I don’t know if we have the budget for it, but I would definitely reach out.
I’m checking for people who would be interested in doing it pro bono. If that doesn’t work, I’m 99% sure you can find some people to fund a couple consultant-hours.
Not to put too fine a point on it, but if the alternative is TLYCS designing the experiment themselves, this is pretty much like running a charity that spends nothing on overhead. It looks good on paper, but in reality, that last bit of money is a huge effectiveness multiplier.
I’d consider funding this if it’s “worth it” and not too much money. I’m sure others would as well.
I’m fairly surprised the EA movement doesn’t have official statisticians. The EAA movement has a lot of people claiming to be official statisticians.