Why Don't All These Numbers Add Up?

Chris Jennings

By Scott Yaich, Ph.D.

Well, Bobby Joe, maybe your hunting wouldn't be going downhill if you didn't shoot so many ducks every year. According to the federal harvest and midwinter waterfowl surveys, Arkansans killed more ducks than were even in the state!"

"Look, Lars, that midwinter survey showed we had almost twice as many ducks as the year before, and our harvest was barely up. Besides, if y'all up North raised anywhere near the ducks that were supposedly produced according to the feds' surveys, we'd have been covered up in ducks."

Dr. Will, having grown up in a small town on the Illinois River, was accustomed to duck hunting being the central topic of every discussion, in every cafe, on every morning of duck season. However, as he waited this March morning for the hostess to seat him at a diner near his hotel in Kansas City, his caffeine-deprived brain gradually became aware of two gentlemen in the corner having a heated discussion about duck hunting.

Lars and Bobby Joe served together in the same Air Force squadron more than three decades ago. Every March, their old unit got together, and being the only dedicated duck hunters in the group, they spent much time comparing notes on the past season, arguing about what was wrong and how things could be improved, and making plans for the next season. Except for his years in the service, Lars, a mechanic and owner of an engine-rebuilding business, spent his entire life just outside Bismarck, North Dakota. He grew up on a ranch full of potholes and native prairie, so he knew ducks. Bobby Joe, a surgeon in Little Rock, spent his youth chasing ducks in the flooded timber on the wintering grounds near Stuttgart, Arkansas, so he knew ducks too.

Will, a college professor specializing in waterfowl management, was dismayed to realize that the only remaining table was adjacent to the two gentlemen arguing about ducks. As he sat down, he resolved himself to keep out of the discussion. He talked ducks for a living and would be in meetings with other biologists for the next three days. All he wanted this morning was a newspaper, an omelet, some coffee, and quiet.

"You're right, Bobby Joe," Lars admitted at the next table. "I've looked at the numbers from all the surveys the state and federal biologists do each year, and I have a hard time making sense of them. I'm no dummy, but they just don't seem to add up. I've read some things on the Internet by people who have apparently spent a lot of time crunching numbers and trying to get them to jibe, and they say the managers are messing up, if not outright hiding something."
"Well," Bobby Joe asked, "you know what Mark Twain said about statistics, don't you?"

"Everybody knows about lies, damned lies, and statistics!" Lars exclaimed.

From behind the newspaper at the next table they heard a voice say, "Yeah, it's fun to criticize something you don't understand, but how many people are aware of Florence Nightingale's positive take on statistics: ‘To understand God's thoughts we must study statistics, for these are the measure of His purpose.'"

Hackles raised, Bobby Joe challenged, "Who the heck are you, and what the heck does that mean?"

Will, having abandoned his resolution to hold his peace when his neighbors moved to questioning the ethics and motives of waterfowl managers, asked if he could join them. After introductions and an exchange of pleasantries about their personal and professional backgrounds, Will offered, "Look, I apologize if I offended you. You're obviously both intelligent and accomplished in your respective professions. But every profession has its own statistics, usually difficult for others to understand and get to ‘add up,' as you put it. But that by no means implies that waterfowl managers are either incompetent or trying to mislead you with statistics.

"What Nightingale meant was that to understand nature, you have to study the statistics, the numbers, which quantify all the interconnected living organisms that make up natural systems," he continued. "By definition, statistics are used to ‘present significant information,' but not just any set of numbers qualifies as statistics. I appreciate your earnest desire to understand the underlying statistics of waterfowl management, and I appreciate the difficulty you're having in making things add up. Sometimes, though, they just don't."

"But I've seen some pretty compelling analyses of waterfowl survey statistics on the Internet," Bobby Joe countered. "They look at the breeding populations, harvest figures, midwinter counts, and other numbers that seem to represent straightforward additions and subtractions from the population. The problem is the dots don't connect in a way that makes sense of what managers are saying and doing. I just want to see more ducks when I go hunting, and I honestly want to know what has to be done."

"The survey data we use to manage waterfowl are all valuable, but different surveys are often like apples and oranges," Will explained. "They have to be used in their proper context to be meaningful. For example, people making comparisons of harvest and midwinter population data usually assume that winter surveys count every duck in the state. In reality, a significant proportion is missed, even in states that count all the ducks they can find. And even if biologists could count every bird in the state, that is still just a snapshot in time. On migration and wintering areas, ducks are constantly moving in and out of the state as they seek food, water, and safety. In the course of a season, many more ducks than are counted actually move through the state. On the other hand, the harvest data represents the entire season.

"In addition, the midwinter counts are notoriously variable among states, each using slightly different procedures," Will continued. "Some try to count every duck, and others only count birds on public areas and in major concentrations. Nevertheless, as long as states do their annual surveys consistently—and somehow account for weather and other factors that strongly influence waterfowl survey data and their meaningful interpretation—the midwinter counts can be useful for measuring trends in waterfowl numbers from year to year statewide or in specific areas.

"This brings up another important point: You have to keep in mind the scale at which surveys are intended to be used," Will added. "Some surveys and data are only valid at the national or continental level, others are useful at the state level, and some are useful only on a local scale. For example, the 2004 harvest survey information for each of the 14 Mississippi Flyway states had an average sampling error of ±20 percent, meaning that the actual duck harvest in each of these states could have been anywhere from 20 percent more or less than the published estimate."

Bobby Joe jumped in, "Twenty percent! That's a pretty wide range. Are those numbers good for anything?"

"Although they only give a rough indication of state-level harvests, you have to remember that the federal government isn't trying to manage waterfowl at the state level," Will replied. "At the national level, the sampling error was only ±5 percent, plenty good for duck management on a continental scale.

"So, in light of all this, it should be clear that while comparing statistics from midwinter surveys and annual harvest estimates might make for a good argument, they can't possibly add up in any meaningful way. Each of these statistics is useful when used in its appropriate context, but meaningless, or worse, beyond that. The article "Setting the Season" in the July/August 2005 issue of Ducks Unlimited provides a good overview of the major waterfowl surveys and how they are used."

"All well and good," Lars said, scratching his chin, "but my hunting has been a lot worse the last few years. Some of the Internet analysts have crunched some numbers and said that we are harvesting more than the surveys show, driving populations down. What about that?"

"These claims are usually made by people who are convinced the seasons have been too liberal," Will explained. "They are correlating high harvests in recent liberal seasons with declining breeding populations. But that doesn't mean that harvest has driven down populations. You can make almost any correlation you want if you selectively use data. Yes, total duck breeding populations were down 25 percent in 2005 compared to 1997 (the year current liberal harvest regulations were implemented). But May ponds—the main barometer for the quantity of wetland breeding habitat on the prairies—were down 28 percent during the same period. Furthermore, the collective breeding population of the five duck species most closely associated with prairie pothole country—mallards, gadwalls, blue-winged teal, shovelers, and pintails—was down 29 percent, almost exactly the same amount as the pond counts. If harvests were having as much impact on duck populations as these people say, we should have seen a much greater decline in these duck populations. 

"The disagreements that often arise over how waterfowl data is interpreted is one of the reasons the Adaptive Harvest Management (AHM) process is so important," Will added. "This process is allowing managers to more rigorously quantify the relationship between hunting, habitat, and populations. Prior to AHM, managers typically restricted seasons when populations declined. But because declining populations usually coincided with drier breeding habitats, it was impossible to determine the role of hunting in regulating populations."

Bobby Joe pressed the argument. "But I also read on the Internet, and it seems reasonable, that if we shoot more ducks one year, we should fully expect to have fewer ducks and worse hunting the next year."

"You are making an assumption here," Will responded. "While many assumptions are valid, you shouldn't take them as fact unless they're backed up by data. In this case, harvest statistics since 1961 demonstrate absolutely no relationship between the size of the duck harvest one year with that of the next. Annual survival of adult female mallards is estimated to be 60 to 69 percent, while harvest rates for the birds have averaged only about 9 percent a year. Clearly, there's a lot more going on with waterfowl populations than harvest. In fact, it would almost be good news to discover that hunting was important in regulating populations. We have a lot more control over hunting than over weather and habitat conditions."

After a long pause Lars asked, "So why don't  biologists just use statistics that we can all understand and don't require so much context and interpretation?"

Laughing, Will said, "Lars, why don't mechanics make it easier to understand and fix car engines? And Bobby Joe, why don't doctors make those two pages of numbers I get after my annual checkup more understandable to non-physicians? Fact is, waterfowl management is a profession, just like medicine and engine repair. Each takes specialized knowledge, and each requires getting the right information, using it in the proper context, making a diagnosis, and taking appropriate action. There's some art, as well as science, in what each of us does.

"At the end of the day, it's useful to step back and look at the ‘big picture' statistics," Will continued. "Over the last 50 years, the average total breeding population estimate for the 10 species of surveyed ducks is 30.9 million. The average for the last five years is 29.9 million, a difference of only 3 percent. Looking at that statistic, let me ask: How do both of you think professional managers and hunters are doing in taking care of waterfowl?"

Bobby Joe reflected a moment before acknowledging with a smile, "You know, when you look at that statistic in the face of all the changes I've seen in habitat across the continent over the last 50 years, I'd say we're all doing pretty darn good." Lars nodded in agreement.
Getting up from the table and exchanging handshakes, Will added, "Keep questioning and thinking critically, but most of all, enjoy your hunting. That's why we managers do what we do!"