A statistically sound study design is important for two major reasons I can see. Firstly it will maximise your chance of answering the question you are trying to answer (ie be adequately powered, have robust confidence intervals etc). But in addition it will help make sure you are studying what you think you are studying. Giving adequate consideration to sampling, randomisation, controls etc are all key, as is using the correct tests to measure your results, and these are all things a good stats person will help with. Having a ‘precise’ result is no good if you didn’t study what you thought you were studying, and a small p value is meaningless if you didn’t make the right comparison.
Regarding why I think bad data is worse than no data, I think it comes to a question of human psychology. We love numbers and measurement. It’s very hard for us to unhear a result even when we find out later it was exaggerated or incorrect. (For example the MMR vaccine and Wakefield’s discredited paper). Nick Bostrum refers to ‘data fumes’ - unreliable bits of information that permeate out ideas and to which we give excessive attention.
(Sorry for taking so long to reply)
A statistically sound study design is important for two major reasons I can see. Firstly it will maximise your chance of answering the question you are trying to answer (ie be adequately powered, have robust confidence intervals etc). But in addition it will help make sure you are studying what you think you are studying. Giving adequate consideration to sampling, randomisation, controls etc are all key, as is using the correct tests to measure your results, and these are all things a good stats person will help with. Having a ‘precise’ result is no good if you didn’t study what you thought you were studying, and a small p value is meaningless if you didn’t make the right comparison.
Regarding why I think bad data is worse than no data, I think it comes to a question of human psychology. We love numbers and measurement. It’s very hard for us to unhear a result even when we find out later it was exaggerated or incorrect. (For example the MMR vaccine and Wakefield’s discredited paper). Nick Bostrum refers to ‘data fumes’ - unreliable bits of information that permeate out ideas and to which we give excessive attention.