When I started at Cafe Imports in 2011, we were cupping around 2000 samples per year. We had also already taken our first steps beyond the Conventionally Held Beliefs (CHB) of cupping form design, and were using a streamlined take on the Specialty Coffee Association (SCA) form. That cupping form dropped the more poorly defined categories from the SCA version, assumed specialty as a starting point, uncluttered the layout, and moved sweetness into the same scaling as the other attributes.
In 2013 we released processing based scoring standards. At that time the standard, as it was, for specialty coffee was washed centric. Our goal was to identify and make more transparent our own expectations for coffee attributes and profiles on the basis of how they were processed. Jason pitched the idea with a simple analogy of beer styles. No matter how good a stout might be, it will always be a terrible lager. It doesn’t make sense to judge stouts on lager metrics. Neither does it make sense to judge wet hulled or natural coffees on washed metrics.
By 2016 we were cupping around 5,500 unique samples each year. The steadily increasing sample load combined with my participation in a sensory science workshop transformed what had been theoretical conversations about scaling and improving our cupping protocols into the practical pursuit of a new solution that would allow us to both scale as needed and avoid the pitfalls that we had identified in the common industry cupping protocol.
In 2017 we introduced a cupping form based on Quantitative Descriptive Analysis (QDA). While this form marked a significant departure from the industry’s CHBs and took a definitive step into the world of sensory science, its output was still indexed to and designed to mirror the scoring compression endemic to the SCA form and cupping protocol.
The QDA form stood out because it was explicitly quantitative and analytic whereas the majority of industry CHB forms are qualitative and affective, meaning they are based on the preference or liking of the cupper. This is a crucial difference. Many stakeholders, cuppers, producers, and buyers alike, believe that these affective cupping forms yield information about coffees. The reality is that preference based forms ask questions and therefore yield information about assessors and not the coffees that they assess.
In distinction to this, QDA designates attributes of interest and then asks assessors to indicate the intensity of those attributes. Attribute value is determined at the administrative level rather than on the cupping table. This means that valuation is transparently knowable to stakeholders prior to coffee submission, as opposed to being “discovered” by assessors on the fly and by stakeholders after the fact. As an example, one of the flavor attributes on our QDA form was Floral. In our model, the more floral a coffee was, the higher the score was. Floral intensity is intrinsic to a coffee. The value, or quality assignment of floral intensity is not.
Beyond being more transparent and descriptive than the preference based forms that still dominate specialty coffee today, our QDA form was also more efficient to use. Reducing assessment to just the observed intensity of attributes and associated descriptor words meant that cuppers no longer had to weigh goodness or badness, nor search coffees for any of the non-coffee attributes featured on affective cupping forms. This was so deceptively simple that it became a regular experience in training for people to say that they didn’t get it while simultaneously generating both calibrated and concept aligned output. This efficiency also meant that as our sample volume increased, we were able to not only keep pace, but also to increase sample replication, making our protocol and blinding even more robust. As effective as the QDA form was, it left plenty of room for improvement.
QDA is technically descriptive. This does not mean that it is colloquially descriptive,
We were still asking our cuppers to perform multiple cognitive functions in the form of generating descriptive language while also measuring attribute intensity.
Scores on the QDA were still decoupled from descriptors.
New entries in variety (Sidra) and processing modulation (anaerobic) challenged and in some ways broke the form.
Cuppers were still required to know and mentally switch from one processing based scoring standard to another.
On the preparation and presentation side, we still had to designate the appropriate processing standard, reducing the degree of effective blinding on flights.
In 2019 I began developing a new methodology to separate valuation of intensity from its measurement. This work quickly turned into an early version of what I call the scoring engine. Even at this stage, the scoring engine was designed to factor our processing based scoring standards into value outputs. At each stage the design process included a goal for reducing as much testing error and variability as possible. The underlying principle was simple: cuppers should describe coffees, while standards should score (value) those descriptions.
The cupping form that developed alongside that early conception of the scoring engine was a bimodal, more purely QDA style form. Wrapped in the cocoon of months of highly caffeinated, solo covid cupping, this very linear cupping form emerged in a flurry of inspiration one morning as what my colleague Devon would two years later dub the Coffee Rose.
This isn’t the final word in score cards. For me, it’s just the second or third word. Remember: We’re perfecting an enemy of the good. Should we find success such that our scorecard is one day itself good, another enemy will be needed.
While I critique industry cupping as received and many of the commonly held beliefs that protect it, the hard work, talent, and professionalism of so many of the cuppers that I’ve worked with and learned from cannot be overstated in all of this. It is unfortunate that so much of that passion and effort has been bled by cupping forms that neither do what they claim to do nor support transparency in protocol and process, even as they make up a primary pillar of our industry transactions. Our QDA form previously, and the Coffee Rose today, are far from perfect. But they are honest.
The Rose in particular frees cuppers to describe the coffees that they taste, frees them from the Keyensian Coffee Contest and Organoleptic Stockholm Syndrome of guessing “correct” scores glossed as “calibration”. The Rose receives each bit of descriptive information that a cupper generates. It does not drop typos or smudged marks. It contextualizes niche and obscure language, making it not only usable, but rich. It generates a score value for the coffees that we describe based directly on how we describe them. That value doesn’t change because of a bad day, or a good one. Of course not everyone will agree with those values. That’s ok, we don’t all agree already anyway and no amount of SmuckersTM will make that happen. Even so, the Coffee Rose brings its receipts. It makes disagreement computable, transparent, and more importantly, opens it up as a path forward for communication.