For the curious, what follows is a brief post about this year's new administrative challenges, and resulting changes to our judging process.
In our effort to better internationalize the awards, for the 2017 awards we invited many new shortlisting + Sadakichi Award judges who are not local to us in Los Angeles. In addition to that, submissions spiked this year.
This has presented us with a challenge: Our centralized judging system from previous years no longer makes sense. Rather than bringing the judges in to the IAO to make their assessments, we now must organize and coordinate the fair and consistent evaluation of the submissions outside of our immediate reach - in eight countries, to be exact.
Since there was also an overall increase in submissions this year, we have had to ask the judges to spend more time on the job - while accommodating their busy schedules and other commitments.
It all points to one thing: Ch-ch-ch-changes. We need to adapt our process while retaining the uncompromising fairness that is crucial to how we run the awards.
So: Put on your thinking caps, and follow our thoughts for a few short minutes. We hope that you see the logic behind how we are running things, this year.
Changes to judging process:
- Longer judging window: This year we have a created longer preliminary judging window in which the shortlisting judges can make their evaluations. We've allotted just about two months for the shortlisting judges to spend with the vials. We've then allotted one month for the finalist judges to spend with the shortlisted perfumes.
- At home judging: With one or two exceptions, our judges are now making their assessments in the privacy of their homes. This allows them to spend more time with the submissions - on their own schedule - while still of course adhering to our judging rules and the confidentiality of the submission process.
- More effective judging categories: We've streamlined the judging categories to avoid redundancy and scoring fatigue. There's a lot to talk about here, so we'll spend more time on this in a separate post.
These are small changes, granted, but the net result is this: The preliminary judges will have a lot more time in which to judge, and will therefore be able to accommodate the higher amount of submissions while spending more time with each submission. We hope that this makes the process easier for the judges while producing the same thoughtful and thorough evaluations that we have come to expect from this pool of dedicated volunteers.
Since the judging is now decentralized, each judge needs their own judging packet containing instructions for judging, a judging form, the statement of intent associated with each submission and - of course - the perfume samples themselves.
Over the past few weeks Submissions Manager Minetta Rogers and A+O Awards director Saskia Wilson-Brown have spent a lot of time together at Saskia's L.A. home, decanting and labeling submissions and preparing a total of approximately 2,150 little vials of scent for the judges. This behind the scenes work is not glamorous. At the same time, it's one of the most important steps in the process to get right. Thanks to a steady flow of strong coffee (and the occasional feline visitor), it's also a lot of fun.
This last week we ramped it up and spent several days preparing the judging packets, printing out the instructions and statements of intent from the perfumers and coordinating the trajectory of each submission in our master database. Since some submissions have been lost in the mail, we also have the unique challenge of tracking which submissions each judge is currently missing so that we can send them later, when they arrive.
We'll start this process over again for the artisan and independent categories when we start the second round of judging in late February.
And that's about it! The administration work associated with the judging process is mostly taken care of, and now it's in the judges' very capable hands.
Expect an update on judging itself next week (as well as an explanation of how we simplified the judging form.)
Thanks, as always, for your participation.
Any questions? email us at [email protected].
Published by: artandolfactionawards in process