April 17, 2017 - Comments Off on Let’s talk about judging bias
Let’s talk about judging bias
We're a few weeks away from the 4th annual Art and Olfaction Awards, and now feels like a great time to post about one challenge we face as an organization: The fairness of our judging process.
In a perfect world, we would identify and persuade 28 people who both have excellent noses AND zero existing relationships in the industry to spend months of their time assessing the submissions for nothing more than an enthusiastic thank you.
However, our world is not quite perfect, and people who understand perfume tend to... well... work in the perfume industry. In other words, pre-existing relationships are a given: There is no reasonable way to avoid them, short of coralling a pack of robots to evaluate the submissions (and even that might create some issues along the way).
We are, however, very aware of a relationship's potential to bias the awards. Although every mechanism has potential flaws (and every year brings us a new exception to what we thought was a hard rule), we do our very, very best to make the awards as fair as possible. Here's how!
