In every sport, the competitors are the ones who are celebrated and idolized. They are the superstars who are the focus of the public and media’s attention. After all, nobody watches the NBA game or World Cup match to appreciate the BRILLIANT WORLD CLASS officiating of the referees. While most people would pay lip-service to the importance of the role which these official’s play, the truth is their job is a largely thankless one. Good officiating is taken for granted and easily forgotten.
Adjudication in debating is pretty much the same, (to my knowledge) besides the top 10 awards given out at the Asian Championships and the cash prizes at tournaments in India for the top-rated judges, debaters get most of the glory. For adjudicators, the greatest accomplishment one can attain is to become the adjudication core of a major tournament, though it must be said that being a fantastic adjudicator isn’t necessarily going to land you a spot on the Chief Adjudication Panel (CAP).
CAPs tend to be decided by
1) VOTING! At the Asian Championships (and maybe others)
Skeptics may see this as nothing more than a popularity contest. I shall neither confirm nor deny whether I am a skeptic, but regardless of your stance on the merits of voting, it certainly seems more meritocratic than the alternatives.
2) The host institution, or host, can select the Chief Adjudicator (CA), or in some cases, multiple Co-CAs
Some might argue that this is a worthy trade-off to encourage potential hosts to bid for tournaments, given the largely thankless work necessary to host a major. The alternative would be to rely on profit-making entities to host tournaments for money, often in a hotel at a third-world country where costs are low and margins are high. Ill leave it to Reuben to rant more about this in his UADC 2018 report.
3) Having a good relationship with the powers at be who decides the CAP, whether it is the organizers, host institution, or existing CAP already in place
This process, as you would expect, isn’t necessarily the most transparent and may lead to external influences on the tournament proceedings. #thisisanorder
4) Very often, even if this isn’t technically an iron-clad rule, being a decorated debater is almost a prerequisite to becoming CAP of a major.
It is quite common for great debaters without much judging experience to be in the CAP of a major but very rare for great judges without much debate experience to do the same. There are some who may argue that being a good adjudicator isn’t the only skill-set required to excelling as a CAP member, and I would be inclined to agree. Certainly, a good CAP member should also be able to select and create excellent motions and have a good grasp on tournament organization and management, but there is no reason why someone with a long-list of debate accomplishments would be good at these things either.
Don’t get me wrong, I’m sure an overwhelming majority of people who have served on CAPs are really deserving and have done a really excellent job, really. (or at least that’s the politically correct thing to say, as some might have learnt) In my honest opinion, nobody really remembers much about the CAP after the tournament, unless something truly scandalous happens, such as when someone erased negative community feedback on a Google Sheet while logged in to their Gmail account and then blamed an alleged mental illness, or gropes someone using homosexuality as an excuse.
To further prove my point (while also engaging in some humble bragging, sort of), barely a year after the United Asian Debate Championship 2016 for which yours truly was the Chief Adjudicator, I was barely recognized at the subsequent edition, beyond those who had participated in UADC the previous year. 1st time participants even mistook me for a freshman, owing in part to my youthful good looks.
Now that I have spent 600 words leading up to that very important conclusion, I would like to provide a guide? commentary? critique? on adjudication, based on my limited personal experiences. As I would expect this to be controversial, and since I’m a debater, here are some caveats:
1) Consider this an Op-ed article, this means that I am representing my own opinions and not facts, lest someone whose name got struck through accuses me of false reporting to stir controversy. (Though that did double our website traffic, so…)
2) I will be ranting about certain behaviours of adjudicators which I myself have resorted to in the past.
3) I acknowledge that this article may encourage the exact type of behaviour I am advocating against, because it is effective in helping judges get better feedback scores. However, the purpose of this article is to hopefully spark a discussion about adjudication and hopefully solve some of the problems highlighted in the long run.
So, in the Asian circuit at least, we have a problem with adjudication that to my knowledge there has been not much discussion over. Namely, that there is a trend of judges bending over backwards to suck up to teams in the hopes of getting a good score from them. The very idea of this would have been ludicrous around 5 years ago back when I was an active debater. (yes, I am old, even with my youthful good looks) As far as I can remember, the best, most established adjudicators I got tended to be brutally harsh and honest, they would let you know in no uncertain terms if they thought you screwed up, and found constructive criticism even if you did relatively well. Judges used to be appreciated for their ability to painfully highlight the mistakes that teams made and point out how they could improve, because that is what teams cared about, useful feedback they could apply in future rounds to make them better.
I have no doubt that the best teams today would still appreciate clear and constructive criticism, and I’m happy to have met many debaters who ask me what they could do to improve in the next round, which is something I have always advocated for (link). However, it is an open secret that the easiest way to climb up the ranks if you are being assigned to average to below average rooms is to give debaters the impression that they performed better than they actually did. I’m sure many of you have encountered judges who have said these things, or some variation of them.
1) “I loved your case, I particularly enjoyed this brilliant argument, and I was pleasantly surprised to heard such rigorous analysis.” When you check the tab later on, you realize you got an average to below average score.
This is probably the one which Asian teams have heard alot at international tournaments over the past decade, which we complain and rant about because it is often racist and patronizing. Yet, for some strange reason, it seems like Asian judges are themselves resorting to this.
2) “This was an above average to excellent round which was pretty close, so if you are worrying about your speaker scores which will affect the speaker rankings or the break, don’t worry, you are in a good position.” The adjudicator later explains to the winning team during constructive: “You guys actually thrashed the other team, I just wanted them to feel better.”
This is a classic example of judges trying to please the debaters while being completely two-faced about it, where very often they may not be at all truthful.
3) This is probably the worst anecdote I’ve come across, and as a result, also the most amusing. Judges writing inflated scores on their track sheet, basically scribbling many 80s all over it so that teams will see it when they come over to get constructive feedback. Sometimes, the judge may even approach the losing team to offer feedback before they leave, “accidentally” leaving their track sheet on the bench to flash the scores so that the team may feel better about themselves.
This would be outright disgusting behaviour even if the indicated scores were accurate, and I have no doubt that, assuming the legitimacy of this anecdote, the scores were definitely inflated. The fact that a judge, or many judges, may resort to this in order to rank higher makes me sick, and I would probably write negative feedback for all that judge’s future community feedback if I knew for sure this happened to me.
Adjudicators trying to suck up to debaters to get better feedback scores is symptomatic of a larger problem. There is a reason why they resort to such behaviours; they are pretty effective. I have to shake my head every time I see a team coming up to me to dig for more information, the exact margin, the exact scores I gave, who the best speaker was etc. I usually try to give some vague but truthful response just so that I can extricate myself from their approaches while not breaking the rules or offending them.
I never understood the purpose of debaters or teams doing this. Well, I get that debaters are nervous and would always like to know where they stand, though in most instances I feel like this obsession to get a better grasp of how likely your team is to make the breaks is almost always counterproductive. Having more information isn’t going to change what your team should be trying to do, which is to perform as well as possible in the rest of the rounds. Being told that your team is doing well can only make you complacent, whereas knowing that your team did badly in the previous round can only make you depressed and unmotivated. Your judge doesn’t even have any real incentive to tell you the truth, they might just decide to claim that you did well just so you will give them a higher score (which you are less likely to do if you are depressed), or they may just claim you did badly to troll you (which brings great utility), or because for some strange reason they want you to feel depressed and bomb the next round so that the team from their own institution might have a better shot at breaking.
It was amusing and shocking in equal measure to hear the adjudication core at UADC 2018 have to remind judges to not lie to participants about how well they did. The fact that they saw a need to make a public announcement about this suggests that someone probably made a report about this and/or that they were aware of the tendency for judges to resort to such behavior. Given that teams and judges tend not to speak publicly about this, there is a reasonable chance that this was more than an isolated incident.
This is a serious problem, which I strongly believe that we as a community need to engage with. At the point in time where adjudicators are more interested in bootlicking the debaters to boost their own scores rather than to provide the best justification possible for the decision coupled with honest constructive feedback, the integrity of the tournament is likely to be undermined in a great number of ways.
1) The oral adjudication will not clearly and accurately reflect the dynamics of the round
If adjudicators are too busy trying to sugarcoat their words to make debaters feel better about themselves, it will compromise their ability to provide a clear and accurate justification for their decision. This is especially since they also operate under time constraints, and would likely spend a significant proportion of their preparation time considering how to get debaters to give them a higher score. In theory, this may not be a bad thing, because judges should take great care with their words to justify their decision to the teams. However, when they have to pretend that the debate was close and excellent when it really wasn’t, the mental gymnastics necessary to delude oneself can take up a lot of energy. Trust me, I have tried.
More importantly, this defeats the entire purpose of adjudication and by extension, the debate event. The precious registration money collected from participants to subsidize adjudicators, the time spent conducting an adjudication test, marking the test, releasing the results, explaining the results, allowing judges to appeal, electing a shadow adjudication core to vet those who need to appeal, significant resources are expended simply to ensure that the participants enjoy quality adjudication. If the oral adjudication isn’t clear or accurate, the entire process becomes pointless because teams do not get the best possible justification.
2) Teams are not given the opportunity to improve
Negative feedback, delivered in a direct and honest fashion, is the best way to help teams identify their most glaring weaknesses so that they can attempt to begin addressing them. When the adjudicator tells the losing team that their argument was brilliant when it made no sense whatsoever, when they claim the result is much closer than it actually is, when they neglect to mention all the terrible mistakes being committed, they encourage delusion from bad teams which forms a vicious cycle via confirmation bias. (I KNEW IT! I AM IN FACT BRILLIANT!) When the next adjudicator criticizes them even though they performed equally well (this means badly) as the previous round, they are going to think that this new adjudicator is biased or has an agenda against them, or is too stupid to recognize their brilliance.
I remember all the times my seniors, coaches and judges have torn my speech apart and sought to brutally embarrass me for my mistakes. They were not necessarily rude while doing so, but they never minced their words. Honest and direct feedback really helped me as a debater to quickly identify what were the key weaknesses that I had to work on and I always appreciated that. In fact, I would usually give higher feedback scores to judges which were harsh with their feedback if they were able to provide legitimate justifications for their criticism, those were also the adjudicators I would pester for further constructive because I knew that I could trust them to provide their honest opinion.
3) Judges may favor teams that are more likely to give them a low score if they lose
When the adjudicator is desperate enough to get high feedback scores such that they can break into the elimination rounds and hopefully make top 10, they may end up making their decision based on which team is more likely to give them a higher score even if they lose. This is terrible because it rewards bad behaviour, where teams which are well-known for being vindictive, and behave aggressively during the debate may basically bully the judge into submission. To a certain extent, this will be inevitable as long as we rely on a feedback system for adjudicators, which does seem to be the most fair and meritocratic system that is available.
In this case, I really do not know what the solution is, my personal opinion is that there should be some sort of system which would require teams with 3 or more losses to retroactively amend their scores such that their feedback scores to the judges who gave them losses should average no less than a 5. This would hopefully curb excesses of the worst teams which give a 1 to every single adjudicator who ever gave them a loss.
Eliminating the top 10 awards for adjudicators may also curb their efforts to climb the feedback score ladder, but at the same time the need to recognize good judges and provide transparency is probably more valuable.
As usual, I have vastly exceeded my word count, so I will probably write a part 2 soon to write more about what I believe debaters and judges should do to make the judging process serve everyone better. In the meantime, hopefully this article can spark a long-due discussion over how we handle adjudication at tournaments. Feel free to leave a comment below!