Polling is “scientific,” in the sense that it attempts to follow well-established mathematical concepts of random sampling, but political polls remain as much art as science, and each polling cycle presents different challenges to pollsters’ ability to accurately capture public sentiment. Quick summary: dating back roughly to George Gallup’s introduction of modern political polling in the 1936 election, a pollster seeks to extrapolate the voting behavior of many millions of people (130 million people voted in the 2008 presidential election) from a poll of several hundred or a few thousand people. In a poll that seeks only the opinion of the public at large, the pollster will seek to use a variety of sampling techniques to ensure that the population called actually matches the population as a whole in terms of age, gender, race, geography and other demographic factors. In some cases, where the raw data doesn’t provide a random sample, the pollster may re-weight the sample to reflect a fair cross-section.
Political polling is a somewhat different animal, however: not all adults are registered voters, and not all registered voters show up to vote every time there’s an election. So, a pollster has to use a variety of different methods – in particular, a “likely voter” screen designed to tease out the poll respondent’s likelihood of voting – to try to figure out whether the pollster’s results have sampled a group of people who correspond to the actual electorate for a given election. This is complicated by the fact that voter turnout isn’t uniform: in some years and some states Republican enthusiasm is higher than others, in some Democratic enthusiasm is higher than others. You can conduct the best poll in the world in terms of accurately ascertaining the views of a population that mirrors your sample – but if your sample doesn’t mirror that season’s electorate, your poll will mislead its readers in the same way that the Literary Digest’s unscientific poll did in 1936, or the RCP averages in the Senate elections in Colorado and Nevada in 2010, or the polls that failed to capture the GOP surge in 2002.
Technology, economics and other factors affect polling. The rise of caller ID in particular has dramatically reduced response rates – that is, pollsters have to call 8 or 9 people for every one who will answer their poll. That raises the level of difficulty in ensuring that the people who actually do answer the questions are a representative sample. Liberals argue that pollsters undersample people who have only cell phones (a disproportionately younger and/or poorer group) and non-English speakers; conservatives counter that Tea Partiers may be less likely to talk to pollsters and that polls in some cases can suffer a “Shy Tory Factor” where voters are less likely to admit to voting Republican. Partisans dispute the relative merits of in-person versus automated polling and the structure of polls that ask a lot of leading questions before asking for voter preferences. And the economics of the polling business itself is under stress, as news organizations have less money to spend on polls and pollsters do public political polling for a variety of business reasons, only some of which have anything to do with a desire to be accurate – some pollsters like PPP make most of their money off serving partisan clients, news organizations do it to drive news, universities do it for name recognition.