Election polls are 95% confident but only 60% accurate, Berkeley Haas study finds
Analysis of 1,400 polls from 11 election cycles found that the outcome lands within the poll’s result just 60% of the time.
26-Oct-2020 4:25 PM EDT, by University of California, Berkeley Haas School of Business
Newswise — How confident should you be in election polls? Not nearly as confident as the pollsters claim, according to a new Berkeley Haas study.
Most election polls report a 95% confidence level. Yet an analysis of 1,400 polls from 11 election cycles found that the outcome lands within the poll’s result just 60% of the time. And that’s for polls just one week before an election—accuracy drops even more further out.
“If you’re confident, based on polling, about how the 2020 election will come out, think again,” said Berkeley Haas Prof. Don Moore, who conducted the analysis with former student Aditya Kotak, BA 20. “There are a lot of reasons why the actual outcome could be different from the poll, and the way pollsters compute confidence intervals does not take those issues into account.”
<<< Probability and Statistics Games for Kids >>>
Many people were surprised when President Donald Trump beat Hillary Clinton in 2016 after trailing her in the polls, and speculated that polls are getting less accurate or that the election was so unusual it threw them off. But Moore and Kotak found no evidence of declining accuracy in their sample of polls back to 2008—rather, they found consistently overconfident claims on the part of pollsters.
“Perhaps the way we interpret polls as a whole needs to be adjusted, to account for the uncertainty that comes with them,” Kotak said. In fact, to be 95% confident, polls would need to double the margins of error they report even a week from election day, the analysis concluded.
As a statistics and computer science student on an undergraduate research apprenticeship in Moore’s Accuracy Lab during the 2019 presidential primary, Kotak grew curious about the confidence intervals included with polls. He noticed that polls’ margin of error was frequently mentioned as a footnote in news articles and election forecast methodologies, and he wondered whether they were as accurate as their margins of error implied they should be.
Kotak brought the idea to Moore, who studies overconfidence from both a psychological and statistical perspective. Much of the research on polling accuracy considers only whether the poll correctly called the winner. To gauge poll confidence, they decided to take a retroactive look at polls based on how long before an election they were conducted, and consider not whether a candidate won or lost, but whether the actual share of the vote fell within the margin of error the poll had reported. For example, if a poll showed that 54% of voters favored a candidate, and it had a 5% margin of error, it would be accurate if the candidate garnered 49% to 59% of the vote, but would be a miss if the candidate won with more than 59% of the vote (or less than 49%).
Moore and Kotak obtained 1,400 polls conducted ahead of the general elections of 2008, 2012, and 2016, as well as the Democratic presidential primaries in Iowa and New Hampshire from 2008 and 2016 and the Republican primaries in the same states from 2012 and 2016. Because some polls asked about multiple candidates, the sample included results of over 5,000 surveys of how people said they’d vote on particular candidates, as well as the accompanying margins of error.
Analyzing the polls in seven-day batches, they found a steady decline in accuracy the farther from an election the poll was conducted, with only about half proving to be accurate 10 weeks before an election. This makes sense, since unforeseen events occur—such as former FBI director James Comey announcing an investigation into Clinton’s emails just a week before the 2016 presidential election. Yet most polls, even weeks out, reported the industry standard 95% confidence interval.
Sampling error and confidence intervals
The confidence interval quantifies how sure one can be that the sample of people surveyed reflects the whole voter population. A 95% confidence interval, for example, means that if the same sampling procedure were followed 100 times, 95 of those samples would contain the true voter population. Therein lies the problem, however.
The confidence level takes into account “sampling error,” a statistical term that quantifies how likely it is that by pure chance, the sample varies from the larger population of voters from which the sample was drawn. For example, not surveying a large enough group of voters would increase the sampling error. But sampling error does not include any other kinds of errors—such as surveying the wrong set of people to begin with.
“People often forget that margins of error for polls only capture the statistical sources of error,” said David Broockman, an associate professor in Berkeley’s Department of Political Science. “This analysis shows just how large the remaining non-statistical sources of error are in practice.”
Added Prof. Gabriel Lenz, also of Berkeley Political Science, “This is a fascinating analysis, and future work could sort out the sources of the inaccuracy, such as low-quality pollsters, difficulty screening likely voters, last-minute changes in voter intentions, and more.”
It’s easy to take sampling error into account in polling statistics, but much harder to account for all the other unknowns, Moore said. It’s a lesson that goes far beyond polling.
“Because we base our beliefs on imperfect and biased samples of information, sometimes we will be wrong for reasons that we did not anticipate,” he said.
FATE, TX — The Fate City Council voted last night to censure Councilman Mark Hatley following a contentious ethics hearing that exposed deep divisions among elected officials.
The censure stems from two ethics complaints alleging Hatley improperly disclosed confidential information tied to internal discussions about the potential firing of former Department of Public Safety Chief Lyle Lombard. According to testimony, Hatley shared details with local journalist Michael Pipkins of PipkinsReports.com, including references to recorded conversations with City Manager Michael Kovacs.
The complaint was filed by outgoing councilman Scott Kelley, who played a central role throughout the proceedings and ultimately did not recuse himself and voted in favor of censure.
Monday’s meeting included a formal evidentiary hearing where Hatley, represented by attorney David Dodd, presented a defense and attempted to question fellow council members. The process, however, was repeatedly constrained by legal warnings from City Attorney Jennifer Richie, who advised council members not to answer questions related to Lombard’s termination due to ongoing litigation. That guidance, issued numerous times during the hearing, limited testimony and narrowed the scope of cross-examination.
The council ultimately split along familiar lines. Kelley was joined by outgoing councilman Mark Harper and recalled councilwoman Codi Chinn in supporting the censure. Mayor Andrew Greenberg and Councilman Rick Maneval opposed it, creating a 3–2 divide before the deciding vote was cast. Councilwoman Martha Huffman ultimately sided with the majority, breaking what would have otherwise been a tie, and would have quashed the censure.
Under Texas municipal norms, a censure is a formal statement of disapproval by a governing body against one of its own members. It carries no direct legal penalty, meaning Hatley retains his elected position and voting authority. However, such a reprimand can damage political standing, limit influence within the council, and shape future electoral prospects…if the electorate so decides.
The underlying controversy traces back to the dismissal of Lombard, which has since evolved into a broader legal dispute involving claims of wrongful termination. During Monday’s hearing, repeated references to that litigation underscored the complexity of the case and the limits placed on public disclosure. Richie’s guidance, aimed at protecting the city’s legal position, effectively curtailed testimony that might have clarified key details. Critics argue this dynamic left Hatley unable to fully defend himself against the allegations.
The political context surrounding the vote is difficult to ignore. This was Chinn’s last meeting, as she was recalled from office by the voters, in part due to her involvement in the Lombard matter. Kelley, who initiated the ethics complaint, participated fully in the decision-making process knowing that this was his last meeting. Harper has also been linked in prior discussions about leadership conflicts within city administration, and for he as well, this was his last meeting. Meanwhile, all three have supported recall efforts targeting Hatley, Greenberg, Maneval, and Huffman, for additional recall, along with two new councilmen who will take their seats at the next meeting.
From a procedural standpoint, the meeting reflected a council operating under significant strain. Testimony was fragmented, legal cautions were frequent, and the final vote appeared to follow established political alliances rather than shifting based on evidence presented during the hearing. Even Hatley’s legal representation struggled to gain traction within the constraints imposed by the city’s legal posture.
Opinion
The battle for power in Fate is very real. What unfolded Monday night was not merely an ethics hearing; it was the visible culmination of an ongoing political battle inside Fate’s leadership. When a complainant votes on his own accusation; when key witnesses are effectively shielded from cross examination; when you have councilmen under recall by the very people bringing charges against their opponents; the process begins to look less like a search for truth and more like a managed outcome. It’s cut-throat politics at its worst.
What’s changed due to this Hearing? Essentially, nothing. Hatley gets a political black eye, but that’s about it. The sides were already defined, and the votes exactly as expected. Councilmen whose terms were ending anyway are now gone after delivering one last poke in the eye to their opponents. And the City Manager, who is at the heart of this debacle because of his employee decisions, and his inability to stand up to influence from Council Members… is still employed.
For residents of Fate, the final result is an up-close view into how dirty local politics can get. It diminishes the desirability of the city to new residents, hurts economic growth, and the entire process gives citizens the perspective that their city government is completely dysfunctional.
Disclosure
The author of this article was referenced during the hearing as a recipient of information discussed in the ethics complaints. The reporting above is based on observations of the public meeting and review of the proceedings.
FATE, TX — Allen Robbins defeated newcomer Melinda McCarthy for Place 3 on the Fate City Council in the May 2, 2026 election, signaling that a slim majority of voters preferred experience over change.
The seat, previously held by Scott Kelley, was open after Kelley declined to seek reelection, setting up a direct contest between Robbins’ prior service and McCarthy’s outsider campaign.
Unofficial results show Robbins winning with 52.22% of the vote, 883 votes, to McCarthy’s 47.78%, 808 votes, out of 1,691 ballots cast. The margin reflects a divided electorate, with nearly half backing a first-time candidate.
Robbins campaigned on experience, but his record on the council became a central issue. Public records show he supported a roughly 5.96 percent property tax rate increase, higher solid waste fees, and a $3 monthly road fee applied broadly to residents.
He also backed zoning changes and approved a 179-unit townhome development, decisions that critics argue contributed to rapid growth and increased density. Some residents have tied those policies to worsening traffic and a perceived decline in quality of life in Fate.
McCarthy’s campaign focused on transparency, responsiveness, and reevaluating growth decisions. Her message resonated with a significant share of voters but fell short against Robbins’ name recognition and governing background.
The results remain subject to canvassing, but Robbins is expected to return to the council as debates over growth, taxation, and infrastructure continue.
Analysis and Commentary
This race underscores a familiar tension in local politics. Voters often voice frustration with growth and rising costs, yet still choose candidates they believe understand the system.
Robbins’ win suggests that, for now, experience outweighs dissatisfaction. But the narrow margin tells a different story beneath the surface.
Nearly half the electorate signaled a desire for change, and those concerns are unlikely to fade. If anything, they will follow Robbins back into office, where the consequences of past decisions, and future ones, will be closely watched.
FATE, TX — In a decisive and unexpected outcome, Ashley Rains defeated Lorna Grove for Fate City Council Place 2, delivering a clear upset against a candidate backed by a unified slate of local Republican leadership.
Unofficial results from May 2 show Rains winning with 56.38% of the vote (945 votes) to Grove’s 43.62% (731 votes). The margin, more than 200 votes, signals a strong voter preference that defied expectations heading into election night.
The seat opened after Councilman Mark Harper declined to seek reelection, setting up a race that quickly became a referendum on the direction of city leadership.
Establishment Support Falls Short
Grove entered the race with significant political backing, including endorsements from State Senator Bob Hall, Jace Yarbrough, John Stacy, Dennis London, and Darcy Gildon. Fate Mayor Andrew Greenberg and every Republican precinct chair in Rockwall County also supported her candidacy, forming a rare, consolidated front in a local race.
Despite that support, voters broke the other direction.
Rains positioned herself as a grassroots alternative, emphasizing accountability and independence from what some voters viewed as coordinated political influence. The result suggests that message resonated more strongly than institutional endorsements.
Recall Effort Played a Key Role
A secondary, but important, factor in the race was Rains’ leadership role in the ongoing recall effort targeting three council members and the mayor. The effort will likely be placed on the November election ballot, giving Rains elevated visibility and an engaged base of supporters.
While she did not run solely on the recall, her involvement helped frame her candidacy as part of a broader push for change at City Hall. That connection likely contributed to turnout among voters already invested in the issue.
What It Means Going Forward
Rains’ victory may serve as an early indicator of voter sentiment ahead of the November recall election, though the two contests are not perfectly aligned.
With 1,676 total votes cast, turnout was solid for a municipal race, and the nearly 13-point margin suggests a clear mandate—at least in this contest.
The results remain unofficial pending canvassing, but the outcome is unlikely to change.
For now, the takeaway is straightforward: Fate voters rejected a unified political slate and elevated a candidate tied to grassroots activism, signaling a shift in the city’s political landscape with more tests to come this fall.