When speaking about “merit” of judges, the useful for what that means in practice.
For low-level judges, that often means exam scores. In merit-based systems, you need good exam scores to become a judge.
For experienced judges you can measure merit by how much of their judgements get overturned by higher courts. A judge who constantly makes judgements that get overturned is bad at seeking legal consensus.
In common law jurisdiction you could also measure how often the opinion of the judge on cases get cited by opinions from other courts.
Yes, there are some measures, but beware of Goodhart Law: if you over-incentive consensus, you get herd behaviour. Many “consensus building” mechanisms end producing the same kind of problems as “peer review”: conformity, statu quo bias, and above all, guild mentality. In Law, external measures of goodness (that counterbalance statu quo bias) are even more difficult to create than in academy...
When speaking about “merit” of judges, the useful for what that means in practice.
For low-level judges, that often means exam scores. In merit-based systems, you need good exam scores to become a judge.
For experienced judges you can measure merit by how much of their judgements get overturned by higher courts. A judge who constantly makes judgements that get overturned is bad at seeking legal consensus.
In common law jurisdiction you could also measure how often the opinion of the judge on cases get cited by opinions from other courts.
Yes, there are some measures, but beware of Goodhart Law: if you over-incentive consensus, you get herd behaviour. Many “consensus building” mechanisms end producing the same kind of problems as “peer review”: conformity, statu quo bias, and above all, guild mentality. In Law, external measures of goodness (that counterbalance statu quo bias) are even more difficult to create than in academy...
https://www.palladiummag.com/2024/08/02/the-academic-culture-of-fraud/