There is a problem that basically every award judge runs into, and almost nobody talks about it openly. It is called score drift. You start your first nomination with a fairly clear sense of what a seven out of ten looks like. Two hours and thirty nominations later, your seven has quietly become a five. Or maybe it became a nine. The point is, it moved, and you probably did not notice.
Score drift is not a character flaw. It is just a natural consequence of how human judgment works under sustained cognitive load. But in an award evaluation context, it produces genuinely unfair outcomes. A company that would have scored an eight at nine in the morning scores a six at three in the afternoon, not because their work is any less strong but because your internal benchmark has shifted. After spending years building and running judging programs at Business Intelligence Group, I have seen this pattern enough times to know it is nearly universal, and nearly avoidable.
The Five Minutes That Change Everything
The fix is actually pretty simple, and the best judges I know do some version of it instinctively. Before you score a single nomination, spend five minutes writing down in plain language what each score level means to you for the specific criteria you are evaluating.
Not the rubric language the program provides. Your own words. For innovation, what does a ten actually look like? What does a four look like? What is the specific difference between a six and a seven in your mental model? Write it down. Even bullet points. Even messy notes that nobody else will ever read.
This process forces you to externalize your standard before your judgment gets contaminated by the submissions themselves. According to research published in the Journal of Applied Psychology, anchoring your evaluative criteria before exposure to stimuli significantly reduces evaluator inconsistency over time. That is a formal way of saying what experienced judges already know: write it down first, and you will be a lot more consistent at the end of the day than at the beginning.
Separating Personal Preference From Objective Criteria
The other piece of this that tends to trip people up is the difference between what you personally find impressive and what the criteria actually call for. These two things are related but not identical, and confusing them is one of the most common sources of inconsistent scoring.
Say you are evaluating a category for customer service innovation. You might have strong opinions about what good customer service looks like based on your own company's approach or your own experiences as a customer. Those opinions are real and they have value. But they are not the same thing as an objective assessment of whether the nominated company achieved a meaningful, measurable improvement in their service delivery against their own context and goals.
Good judges learn to hold both things at once. You can personally prefer a different approach while still recognizing that the approach in front of you was executed well and produced results. Keeping that distinction clear, especially hours into a long evaluation session, is actually a skill that gets better with practice.
The Calibration Read
Here is a practical technique that a lot of experienced judges use, and it is genuinely helpful. Before scoring the full batch, read three or four nominations all the way through without scoring anything. Just read them to get a feel for the range of what you are working with. What is the best thing you have seen so far? What is the weakest? Where does the middle of the distribution seem to be?
Then go back and start scoring. This calibration pass gives your internal benchmark a foundation in the actual submissions rather than an abstract expectation. It does not take long, and it meaningfully reduces the chance that your first few scores are outliers relative to the rest of your evaluation.
Think of it like the warm-up laps before a race. The actual scoring is the race. The calibration read is just making sure your engine is running at the right temperature before you push.
Taking Breaks on Purpose
Score drift accelerates when you are tired. This is not complicated, but it is easy to ignore when you are trying to get through a stack of nominations efficiently. The research on evaluator fatigue is pretty clear that decision quality degrades meaningfully after sustained cognitive effort, and that short breaks restore performance in ways that willpower alone cannot.
The practical implication for judges is simple: build breaks into your evaluation schedule and treat them as part of the work, not as interruptions to the work. A ten-minute break every hour or so, stepping away from the nominations entirely, will produce more consistent scores than grinding straight through. And it is worth reviewing your notes on what your score levels mean before you start back up after each break, just to reset your anchor.
The Quality That Separates Great Judges From Good Ones
The judges who are most valuable to any award program are the ones who can separate their genuine, informed expertise from their stylistic preferences, maintain consistency over a long evaluation session, give fair consideration to approaches that differ from their own, and articulate the reasoning behind their scores in ways that are useful to others. None of that requires extraordinary intelligence or decades of experience. It requires a little self-awareness and a willingness to slow down enough to do the calibration work before you dive in.
Five minutes of preparation. Dramatically better outcomes. That is a trade most judges are very happy to make once they try it.
If you are interested in putting these skills to work, Business Intelligence Group runs twelve annual award programs and is always looking for sharp, engaged judges who care about getting it right. The nominations are genuinely interesting, the company is good, and you will probably learn something.