do you have a vague impression of when randomisation might be a big win purely by reducing costs of evaluation?
Not really I’m afraid. I’d expect that due to the risk of inadvertent negative impacts and large improvements from weeding out obviously suboptimal options a pure lottery will rarely be a good idea. How much effort to expend beyond weeding out clearly suboptimal options to me likely seems to depend on contextual information specific to the use case. I’m not sure how much there is to be said in general except for platitudes along the lines of “invest time into explicit evaluation until the marginal value of information has diminished sufficiently”.