In any sensory analysis endeavor, common errors are likely to be made if they are not controlled for. “Common sensory errors” are human errors. They describe a sort of troubleshooting guide for human sensory testers. These are not “good cupper” vs “bad cupper” metrics. They could more appropriately be considered “good program” (those that control for the errors) vs “bad program” (those that do not) metrics.
While it may be helpful for assessors to be self-aware and cognizant of what these errors are and how they are made, it is crucial to account for them in our experimental design (that is, when we are figuring out how we are going to set up and run our cuppings). In fact, leaving control of these errors up to our assessors’ awareness, discretion, and responsibility can contribute to their falling into various errors (e.g., time error/positional bias).
Time error or positional bias states that one’s attitude undergoes subtle changes over a series of tests. These changes can include anticipation, hunger, indifference, fatigue, and mental tiring as the day progresses. Anyone who has cupped many coffees in a day, let alone in a week or harvest season, knows the task is mentally taxing. Just maintaining focus in the face, for example, of adaptation (a decrease or change in sensitivity to a given stimulus due to continued exposure to that stimulus or a similar one) can be extremely challenging. Adding the responsibility to continually self-check for sensory errors will add to the challenge and accelerate mental fatigue, even if one manages to avoid initiating a neurotic feedback loop of self-panopticonic error policing!
Cupping and coffee assessment is famously complex. As administrators of cupping programs, we are responsible for setting our cuppers up for success in a field littered with hidden pitfalls. Given the amount of ink, paper, class time, and expense devoted to both legacy and future industry cupping protocols, it is astonishing that these most basic, actionable, and immediately beneficial principles have been so rarely addressed. As Director of Sensory Analysis at Cafe Imports, I’m often asked how people can become better cuppers. While there is work that can be done along the lines that most people imagine (tasting and smelling prepared solutions, doing various exercises), the biggest bang for your buck thing that you can do to be a better cupper is cup within the context of a program designed to control for the common sensory errors.
As much as possible, we need to set up our cupping experiments to avoid or mitigate the following physiological and psychological factors or errors:
Failure to design these errors out of our cupping procedure will result in… these errors. Even if the errors do not occur, we are left with reasonable doubt as to the presence of these errors and, therefore, the robustness of the cupping procedure. Note that sensory design must generate reliable results and demonstrate the basis for that reliability.
This common sensory error list provides baseline guidance for navigating the sensory universe. This is a very good place to start if we want to assess, update, or improve our assessment protocols. At Cafe Imports, every time we make a change to our cupping protocol, we tie it back here. Even if we make a change to something seemingly unrelated, for example, some workflow alteration aimed at improving efficiency, we still check it against this list to ensure that we are not compromising our baseline process by inadvertently increasing the likelihood of some error. It’s important to note that much of sensory design deals with preparation and presentation. It’s also important to note that we in Specialty Coffee have done an excellent job with some of these aspects. We pay attention to things like water composition and temperature, grind size, and roast degree, all examples of limiting error via preparation.
Here is the big point: We want to ensure as far as possible that the attributes, descriptions, scores, similarities, and differences identified on the cupping table are attributable to the coffees and not the processes or protocols by which the coffees are assessed.
The common errors above are where our cupping output can be doubted. These errors describe human tendencies that are best addressed via test design before they arise on the cupping table.
Being an expert cupper is not enough. Expert cuppers still get tired, get excited, have bad days, and can be influenced by non-important inputs, etc. Expert cuppers are still human cuppers, and the common sensory errors are specifically human errors. Experts can certainly generate good data. However, as long as the primary metric for the goodness of that data is merely the expertise of an expert, and as long as the doubts raised around sensory errors have not been addressed, we cannot have great confidence in the output.
This is a big part of why so much of sensory science revolves around experimental design. Experimental design is about best setting your assessors up for success (repeatability, accuracy, sustainability, sensitivity, calibration). At the same time, experimental design allows you to “show your work” and alleviate doubts regarding results. How do we know that a given score does not result from an error? We cannot rule everything out completely, but if our cupping protocol explicitly addresses the inherent challenges of sensory analysis, we can point to it and say, “We have done X, Y, and Z to control for the possibility of errors A, B, and C which we observe to occur in situations L, M, and N.”
We’ve paired our new cupping form, the Coffee Rose, with a full spectrum or holistic approach to cupping protocol design. Controlling for common sensory errors is something that nearly anyone can do in some way, and doing so will benefit nearly every cupping program. The first step is learning what they are and observing them in action. In the coming weeks, I’ll detail studies we performed at Cafe Imports specifically to test, demonstrate, and observe various errors as the groundwork for developing the Coffee Rose and the protocols supporting it.
This blog is the first in a Common Sensory Errors series by Ian Frethiem, Director of Sensory Analysis. Want to be notified of the next blog release? Sign up for our newsletter here.