This is very reasonable; ‘no predictive power’ is a simplification.
Purely academically, I am sure a well-reasoned Bayesian approach would get us closer to the truth. But I think the conclusions drawn still make sense for three reasons.
I did not specify in the table, but the p-values for the insignificant coefficients were very high; often around p=0.85. I think this constitutes so little evidence that it would be too minor a Bayesian update to have to formally conduct.
Given that we do have evidence of some other variables being predictive, updating in favour of weighting those higher still makes sense (although maybe to a lesser degree than I implied in the post).
The time applicants and facilitators spend on the many different criteria we used is a cost (and a meaningful one for smaller groups). I would guess that cutting down the number of variables used would increase productivity more than what can be outweighed by the small updates we could make with little (but non-zero) predictive power.
Thanks for writing this!
This is very reasonable; ‘no predictive power’ is a simplification.
Purely academically, I am sure a well-reasoned Bayesian approach would get us closer to the truth. But I think the conclusions drawn still make sense for three reasons.
I did not specify in the table, but the p-values for the insignificant coefficients were very high; often around p=0.85. I think this constitutes so little evidence that it would be too minor a Bayesian update to have to formally conduct.
Given that we do have evidence of some other variables being predictive, updating in favour of weighting those higher still makes sense (although maybe to a lesser degree than I implied in the post).
The time applicants and facilitators spend on the many different criteria we used is a cost (and a meaningful one for smaller groups). I would guess that cutting down the number of variables used would increase productivity more than what can be outweighed by the small updates we could make with little (but non-zero) predictive power.