Cook’s Illustrated Coffee Brewer Review: Are Their Results Credible?

Like a lot of “foodie” types, I read Cooks Illustrated. I’m not a subscriber but I scan each issue for recipes. Naturally, when they covered coffee brewers I had to read what they said. Technivorm’s Gerard-Clement Smit told me he was excited as they’d just chosen his unit as their best choice. That’s a good start. Consumer Reports has never, ever really chosen a decent coffee brewer as a Best Buy. Right away, I thought that someone credible might be reviewing if they recognize the Technivorm. How about the rest of the review?

Well, I have not yet reviewed every brewer they did, so I’ll reserve judgment unit per unit. But, I have a few quibbles with the big picture. I believe Cooks Illustrated did three tests: an objective temperature test, a brewing time measurement and a subjective taste panel evaluation. They did not identify who did their tests, but it’s a reasonable start and Consumer Reports has never identified any but a subjective panel test, so Cooks is to be commended for at least including two real tests.

In appliance theory, the time and temperature are perceived useful because it is easy to taste that the amount of time it takes and how hot the water is are the two major factors to extraction. But, in our desire to make things measurable we sometimes make things too easy. Consider fact that, according to Cooks Illustrated standards, they would never have chosen several of my favorites and one particularly well-regarded coffee brewers as best buys because they don’t meet those standards very well is interesting.

I looked for the French press measurements but they were nowhere to be seen. Sure enough, the brewing time is set by the user, so that’s understandable. But, how about brew temperatures? Does anyone but me measure the French press’s brewing temperature? And, I’m assuming we’re all following the longstanding practice of boiling water, letting it stand for a minute off-the-boil and pouring. It might be instructive because it explains my theory; that is that temperature is number three of the three brewing parameters that should be measured.

There, I’ve said it. And, now that I’ve said that, I’ll list what I think are the factors, in order of importance, for a coffee brewer:

Brewing contact time – we want to know just how long the hot water is in contact with the grounds. With some brewers this is easy. You can simply pour hot water into the French press and start a stopwatch, press the top down and stop your timer. In a drip brewer, you must factor in a delay, sometimes up a minute after you turn it on and the last water drips through. Also, some automatic drip makers actually stop from time to time during brewing, either to allow their heaters to recharge their energies or as designed to offer the grounds a chance to swell up and rest between pulses; some connoisseurs think this gives a better extraction, particularly with fresh coffee, but it makes timing the contact time a more complex procedure. But, contact time is the single most important factor in the final taste in your cup. Too short, and it will be underdeveloped; too long and it will be overextracted and bitter.

Grounds saturation – how well saturated are the grounds in your coffee brewer? Consider what it’s like when you take a shower. There is a fixed head above you (most commonly) and you move about to make sure every inch of you is properly cleaned and rinsed. Coffee grounds cannot likewise adjust their position during brewing. It is the role of the designer to make a showerhead that is designed to saturate the grounds thoroughly from beginning to end of the brew cycle. It becomes particularly challenging as consumers begin using larger grounds portions as they discover the joys of full-flavored coffee. Often the showerhead is simply a drip spout or it sprays at a midpoint in the grounds bed, hoping that the water will eventually fill and cover all the grounds. Fresh roasting and fresh ground exacerbate the situation because fresh coffee de-gasses carbon dioxide as it’s brewing, interfering with extraction chemically, but also causing the grounds to physically expand, even making them a larger and more difficult-to-cover target for the showerhead. This is THE notable flaw in many brewers, even those who reach the industry’s recommended water brewing temperature. Methods such as French press and vacuum have maintained cultish favor with connoisseurs precisely due to their inborn skill provide great grounds saturation.

Brewing temperature – the single most overrated attribute of a coffee brewer is its brew temperature. Am I saying it’s unimportant? Absolutely not, but it is in third place. Why is it number 3? I suppose because it’s the easiest to measure. If you want my honest opinion based upon years of observing coffee brewers and measuring the brewing temperature, it is that any brewer that brews between 180 and 212 CAN conceivably brew very good tasting coffee, PROVIDED the contact time is kept under six minutes AND the grounds are well and equally saturated.

Take a few examples. Have you measured your French press’ temperature lately? I think you might be surprised at how low it is, in consideration of the Specialty Coffee Association’s recommended brewing temperature (200 degrees Fahrenheit +/- 5 degrees). If you don’t like the press (and Cooks Illustrated didn’t) it is likely due to other factors, such as the amount of grounds left in the cup (I prefer paper filtered “cleaner” coffee – press coffee is too muddy for me). Most press brewing is done well under 195. Generally, if used according to directions, the press will drop off below 195 within a minute of pouring the water into it. It regularly ends contact time at under 185. I once heated a press during operation to maintain 200 degree contact heat and I thought the coffee tasted worse for it. If you are fond of manual drip brewing, such as Melitta’s or Chemex, the results are similar. You continuously pour off-the-boil water into the grounds and wait for the water to run through the grounds bed. If you chart the temperatures over the course of manual drip brewing, it will be series of low hanging telephone lines, each dipping well below 195. If it were not for having proper vacuum preparation coffee, I’d be inclined to regard below 195 temperatures as superior.

Speaking of the vacuum, it is my considered belief that the original industry brewing standards adopted by the SCAA and other trade organizations was based upon observing the once-standard vacuum machine as a peerless example of good brewing. So, the standards were simply reverse-engineered to become THE standards. While at first glance, this seems sensible, it is too confining and it sends well-meaning appliance designers in the wrong direction. Again, my observation is that there are various good and interesting coffee flavors extracted throughout a wide range of temperatures. Some years ago I was a member of the Specialty Coffee Technical Standards Committee. I challenged our group that we’d recently denied certification to Kitchen Aid for their 4-cup maker because it only reached about 190. Meanwhile, we were enjoyably sharing our Chair Kevin Knox’s French press coffee (Kevin at the time ran Allegro Coffee’s roasting operations) which I pointed out was brewed at a temperature in line or below the Kitchen Aid unit. Is there anything wrong with a solid 200 degree contact time? No, and it might arguably extract some of the very finest flavors from coffee. But, there are credible machines that produce excellent tasting coffee (above and below) this range. Meanwhile, there are machines that get the water to the right temperature but fail to keep contact times between four and six minutes and/or fail to get all the grounds consistently wet. Give a choice between a machine that is outside the temperature range or one that either does not get all the grounds wet or takes longer than six minutes, I’d choose the one that’s temperature is non-standard.

So, Cooks is to be commended for trying to follow industry standards and indeed for even emphasizing the importance of a good brewer to coffee taste. But the industry needs to reexamine the standards as well as their order of importance, given the new consumer interest in fine coffees and how to enjoy them at their best.

Does the SCAA Brew Chart Need To Be Changed?

The Specialty Coffee Association of America endorses an objective measurement based upon an analysis of some brewed coffee. It measures the total number of dissolved solids as an analog of coffee flavor. While no one, including the SCAA, claims this is a perfect measurement of true coffee taste, nor to they claim it’s the taste everyone agrees to be perfect, the standards was established on the principle that we have to start somewhere and to help create some sort of objective standard for the industry to use to compare taste intensity.

What good’s an industry if it doesn’t attempt to establish and promote good standards? I think that’s a fair question and it’s to the SCAA’s credit that they are trying to do something to standardize measurements. But, here are some questions I have for the SCAA.

  • Are they doing a service if the standards really don’t represent the taste people really want in their coffee?
  • How do they know the standards are applicable to most public coffee drinkers?
  • Is the current dissolved solids test really accurate in measuring what consumers call coffee taste?
  • Has the espresso drink trend (really the café latte trend) caused consumer coffee taste expectations to dramatically change?

Where the Standards Came From

The SCAA’s standards are pretty clear. They suggest 18 to 22% weight of the ground coffee’s oils to extracted to a brew, which will, by their calculations become a beverage containing 1.15 to 1.35 percent dissolved solids. In effect, it’s like making a lemonade and saying we want this percentage of a lemon to be used and the resulting lemonade should contain thus amount of juice in its makeup. The standards were established by a committee of (probably) pretty learned and sincere coffee people. I’ve heard negative comments that the standards were simply a way to sell more coffee, but (and I’ve never sold a bean in my life) I think that’s an unfair allegation. These were people who recognized that a full flavored, developed cup of brew was actually not bitter and that many food service operators and home users were habitually destroying coffee’s flavor by over-extracting undersized portions of ground coffee. Their intentions were, in my opinion, good and honorable.

Now to a problem: How do you take this noble measurement? The most obvious method was chosen, and that’s using a hydrometer which measures water and then measures the same water’s density once it has been changed, in this case water that’s been changed into brewed coffee. Roast and bean variety will affect density It’s a pretty ingenious idea, and the industry seems to agree that it’s a reasonable way to “see” how much coffee taste is in there. It does not answer questions such as if the coffee flavor’s strength altered by such things as roast or varietal or any number of other factors. The test is not able to tell if the coffee tastes good. That is not its role. So, within the parameters of what it’s designed to do, it seems the test can achieve a goal.

But, what happens if tastes change? Do you think that they do? Let’s take wine. What if we found out that the French wine board set standards for pH and alcohol in wine? Would those big California clarets meet those standards? I think not, and that is part of what has happened. Like it or not, the fun and profit of espresso-based beverages has moved the American coffee drinking public away from the light-roasted vacuum-brewed cup of coffee that this august panel decided was a great cup of coffee back many years ago. To go back to the wine analogy for a moment, the French oenophile who developed the years ago wine standards would have spit out the latest Kendall-Jackson Chardonnay, although some of those fine old French firms have taken to selling their Chardonnay as motor fuel due to changing tastes.

What happens when roasts change? Frank Chambers did some research on measuring coffee strength and discovered that different varieties and different roasts brewed using the same formula, brewed in the same coffee brewer, will give different results on the SCAA chart. The short cut way to say it is that dark roasts will provide stronger coffee and coffees from different growing regions provide more or less strength. In fact, the chart’s developers specified that only one variety roasted to one roast color be used for testing.

My concern is that the American Specialty Coffee Association might be inadvertently imposing a no-longer valid standard. There are only two current coffee brewer manufacturers known to me who ever mention the SCAA’s standards: Kitchen Aid and Bunn-O-Matic. Kitchen Aid designed their wonderful 4-cup brewer several years ago following the SCAA’s standards only to be denied certification when their brewer’s heating element failed to jack up the brewing temperature above 195 Fahrenheit during its first minute of brewing. While I publicly disagreed with this because I think the temperature standards are too narrow, I have to applaud the SCAA for at least sticking to their guns and having some standards. Meanwhile, Bunn keeps putting out coffee brewers that do meet the standards, but they continue to lose competitions won by brewers that don’t.

The Coffee Brewing Chart

Part of the SCAA’s brewing standard was the creation of a chart that showed where a coffee’s flavor strength, as measured using a hydrometer, appeared in relation to its strength versus bitterness, versus body. Although interesting to only the most advanced home users, the idea was to get the commercial coffee servers to “get religion” and see that their adherence to the SCAA’s standards would produced a full-flavored but not bitter cup. The chart, which is difficult if not impossible for most people to understand (even in the industry), contains a box on it that represents the ideal balance of flavor strength, so-called “development” (supposedly “which” flavors), lack of bitterness (over development?) and thrift or economy. The message is clear: brew coffee that falls into the box and you’ll be making people’s taste buds happy. We know, we’re the Specialty Coffee Association. The last tests I saw of various consumer coffee brewers, only one brand made it into the box and it was a Bunn-O-Matic. None of the others did, and the list was extensive. So, either the other manufacturers are ignoring the SCAA or they are incapable of meeting the standards or don’t know how to measure their machines. I’m guessing it’s number one and that they just don’t care or thing anyone will notice.

Now, I just gave Bunn a free ride. But, how about them? Why did they fail to impress Cooks Illustrated? Why do high-end roasters who want their customers to own the best brewer often recommend other models, some of the same ones that don’t brew coffee that fall near the SCAA’s box? You might speculate that the other brewer manufacturers don’t care or know how to test, but the high end bean sellers do care and do know how to test. This tells us in a dramatic way that those bean sellers don’t think their customers want coffee that’s “in the box”—the SCAA’s box at any rate.

I’ve tested the Bunn machine. My test results confirm what the SCAA’s box says. The Bunn delivers a nice no-bitterness cup of coffee, one that is very full flavored as far as acidity. My taste observations are that many consumers, including some who buy very expensive beans and grind them carefully before brewing, are now weaned on espresso. The level of bitterness that bothered an honest coffee merchant or taster in 1970 no does not bother this end-user and in fact he desires it. I challenged Cook’s Illustrated on some of their methodology but I spoke extensively with Cook’s Illustrated’s Lisa McManus and they did consult with some honest and notable coffee experts before drawing her conclusions. I think she represents fairly what many consumers would taste. Cook’s Illustrated did not report performing any total dissolved solids tests.

In my opinion, we have here an honest mistake in the making. The sad thing is several industry leaders are trying to do the right thing. Two of the industry’s most noble and honest manufacturers are losing sales because they are adhering to seriously deficient and possibly outdated standards. And, the SCAA is failing to reexamine and update their standards given emerging consumer taste pattern changes.

The risk is what happened to me when I followed the outdated road maps on my GPS this summer. I didn’t reach my goal and someone without guidance did.

Pin It on Pinterest