Thanks for engaging with my criticism in a positive way.
Regarding how timely the data ought to be, I don’t think live data is necessary at all—it would be sufficient in my view to post updated information every year or two.
I don’t think “applied in the last 30 days” is quite the right reference class, however, because by-definition, the averages will ignore all applications that have been waiting for over one month. I think the most useful kind of statistics would:
Restrict to applications from n to n+m months ago, where n>=3
Make a note of what percentage of these applicants haven’t received a response
Give a few different percentiles for decision-timelines, e.g. 20th, 50th, 80th, 95th percentiles.
Include a clear explanation of which applications are being included, or excluded, for example, are you including applications that were not at all realistic, and so were rejected as soon as they landed on your desk?
With such statistics on the website, applications would have a much better sense of what they can expect from the process.
Thanks for engaging with my criticism in a positive way.
Regarding how timely the data ought to be, I don’t think live data is necessary at all—it would be sufficient in my view to post updated information every year or two.
I don’t think “applied in the last 30 days” is quite the right reference class, however, because by-definition, the averages will ignore all applications that have been waiting for over one month. I think the most useful kind of statistics would:
Restrict to applications from n to n+m months ago, where n>=3
Make a note of what percentage of these applicants haven’t received a response
Give a few different percentiles for decision-timelines, e.g. 20th, 50th, 80th, 95th percentiles.
Include a clear explanation of which applications are being included, or excluded, for example, are you including applications that were not at all realistic, and so were rejected as soon as they landed on your desk?
With such statistics on the website, applications would have a much better sense of what they can expect from the process.
Oh, I thought you might have suggested the live thing before, my mistake. Maybe I should have just given the 90-day figure above.
(That approach seems reasonable to me)