Great argument. My guess for why this isn’t common based on a little experience is that the decision is usually sequential. First you calculate a sample size based on power requirements, and then you fundraise for that budget (and usually the grantmaker asks for your power calculations, so it does have to be sequential). This doesn’t inherently prevent you from factoring intervention cost into the power calculations, but it does mean the budget constraint is not salient.
I wouldn’t be too surprised ex ante if there are inefficiencies in how we do randomization. This is an area with quite active research, such as this 2022 paper which proposes a really basic shift in randomization procedures and yet shows its power benefits.
I’m confused why the process being sequential is a reason that this isn’t occurring. Suppose someone was writing a RCT grant proposal and knew in advance how expensive the treatment was compared to the control. They find the optimal ratio of treatment to control, based on the post above. Then, they ask for however much money they need to get a certain amount of power (which would be less money than they would have needed to ask for not doing this).
Or alternatively, run the sample size calculation as you suggest. Convert that into a $ figure, then use the information in the post above to get more power for that same amount of money and show the grant-maker the second version of one’s power calculations.
I’m surprised you retracted the comment because I agree with it and I’m not 100% sure what I meant. It is still a salience issue but I don’t think the sequential process really matters for that
To explain why I retracted: I re-read your original post and noticed that you were talking about salience, and I think you’re probably right that this isn’t a very salient aspect of the process. At first, I thought you were saying something like ‘the steps occur sequentially, so the suggestion of the post can’t be implemented’ which seems wrong. But ‘the steps occur sequentially, so it might not occur to someone to back-track in their thinking and revise the result they got in the first step afterwards’ seems probably right, although I have no idea how big of an explanation that is compared to other reasons the OP’s suggestion isn’t very common.
Great argument. My guess for why this isn’t common based on a little experience is that the decision is usually sequential. First you calculate a sample size based on power requirements, and then you fundraise for that budget (and usually the grantmaker asks for your power calculations, so it does have to be sequential). This doesn’t inherently prevent you from factoring intervention cost into the power calculations, but it does mean the budget constraint is not salient.
I wouldn’t be too surprised ex ante if there are inefficiencies in how we do randomization. This is an area with quite active research, such as this 2022 paper which proposes a really basic shift in randomization procedures and yet shows its power benefits.
I’m confused why the process being sequential is a reason that this isn’t occurring. Suppose someone was writing a RCT grant proposal and knew in advance how expensive the treatment was compared to the control. They find the optimal ratio of treatment to control, based on the post above. Then, they ask for however much money they need to get a certain amount of power (which would be less money than they would have needed to ask for not doing this).
Or alternatively, run the sample size calculation as you suggest. Convert that into a $ figure, then use the information in the post above to get more power for that same amount of money and show the grant-maker the second version of one’s power calculations.
I’m surprised you retracted the comment because I agree with it and I’m not 100% sure what I meant. It is still a salience issue but I don’t think the sequential process really matters for that
To explain why I retracted: I re-read your original post and noticed that you were talking about salience, and I think you’re probably right that this isn’t a very salient aspect of the process. At first, I thought you were saying something like ‘the steps occur sequentially, so the suggestion of the post can’t be implemented’ which seems wrong. But ‘the steps occur sequentially, so it might not occur to someone to back-track in their thinking and revise the result they got in the first step afterwards’ seems probably right, although I have no idea how big of an explanation that is compared to other reasons the OP’s suggestion isn’t very common.