A Brief History of Polling
Public opinion polls, as we know them today, had to earn their spurs. Polling organizations needed to prove that they could accurately determine public sentiment using a relatively small sample of the population. The first convincing demonstration came in the 1936 presidential election.
1936 - Literary Digest
When it comes to a survey sample size, more is not always better. That lesson was learned the hard way by the Literary Digest in 1936. Over the years Literary Digest had developed a sizable mailing list. In 1928 the magazine decided to use its list to conduct a poll for that year's presidential election. People were sent mock ballots which they were asked to mark with their preference and return to the magazine. The poll's final outcome was within four percentage points of Hoover's actual victory margin. Four years later the magazine's prediction was within two points of Roosevelt's winning percentage.
Buoyed by its previous successes the magazine launched its largest survey ever in 1936. Going beyond its own mailing list, Literary Digest also added names from auto registration lists and telephone directories to send out a total of more than 10 million ballots. Considerable time and expense had to be devoted to tabulating the flood of 2.4 million ballots returned. When they were all in, the magazine predicted that Alf Landon would carry 32 states and defeat Roosevelt by 57% to 43%.
Needless to say, the Digest was sorely embarrassed by the final outcome, a 61% to 37% Roosevelt landslide that left Landon with only two states and eight electoral votes compared to Roosevelt's 523. Not long after, the magazine went out of business.
End of an era
What makes the Literary Digest experience noteworthy is not that the magazine had two good years and one bad call, or that the prediction was so egregiously wrong. The failure of the Digest poll marked the end of one era of polling and the beginning of another.
George Gallup criticized the Digest's methodology. Even a sample as large as two and a half million could not get an accurate picture of national sentiment, Gallup contended, if the sample was not properly selected. Using a sample of only 5,000, Gallup predicted that Roosevelt would carry at least 40 states and win the popular vote by a 56% to 44% margin. Subsequent Gallup Polls would use a smaller sample size and come much closer to the final vote total, but the essential point had been established. A scientifically selected sample of the population was not only much cheaper and easier to handle, it also produced more accurate results.
Random sampling comes of age
Why would such a small sample be so much better than the enormous Literary Digest sample? The key is in mathematics worked out in the 18th and 19th centuries, and applied to polling only in the second quarter of the 20th century. Gallup and other polling pioneers like Elmo Roper and Archibald Crossley based their work on sampling and probability theory.
Reduced to their essence, the mathematical formulations demonstrate that a randomly drawn sample will have certain predictable properties. It does not matter whether the sample is selected from all the people in the United States, or from a bunch of colored balls in a barrel; the same mathematical rules apply. But those rules will apply only if every person in the country or every ball in the barrel has an equal chance of being selected every time a choice is made for the sample. In the case of the barrel, that would probably mean turning it regularly to ensure that balls at the bottom come to the top and increase the chance they will be selected. In the case of a national sample, it means that a person in California is just as likely to be chosen as a person in Delaware.