A study of seven leading chatbots has found that voters asking how to cast a ballot can be given wrong or outdated answers, with Microsoft’s Copilot getting voting questions wrong one time in four. The research, conducted in October by Dewey Square Group for the nonprofit Innovating for the Public Good, tested authentic voter questions about deadlines, polling places and candidates, and it found that accuracy depended more on which chatbot was asked than on what was asked. The results landed the day before the United States midterm elections on November 3.
One in four, one in three, one in ten
The headline numbers describe two different problems. Across all voter questions, 13.9% of answers to voting-logistics questions were outdated or inaccurate. Errors in 5.9% of answers could actually mislead a voter, for example by giving a missed deadline, the wrong voting day or a skipped step.
Across every question type, 5.1% of answers carried a factual error, and 10.9% carried an error or a misleading claim. The chatbot split is where the study bites: Claude and Copilot produced three-quarters of the clear errors. On voting questions, Claude gave wrong information one time in three, Copilot one time in four, and ChatGPT one time in ten.
In one example reviewed by the researchers, Claude was asked when the 2026 election in Michigan’s 4th district takes place and how to vote. It gave incorrect online and mail registration deadlines that were later than the real ones. A voter who followed them could have missed registration, which is the kind of error Chambers called extremely troubling.
What the companies said
Microsoft said helping people find accurate election information has long been a priority, and that it has improved how Copilot handles election questions and keeps testing and refining them. Anthropic pushed back on method, saying that because part of the research used its programming interface rather than its website, some of Claude’s safeguards may not have been active.
The researchers said a comparison found web and interface results comparable, and Anthropic noted that voting-advice requests made up 0.01% of United States Claude conversations in a September sample. The study exposed a gap in safeguards: Anthropic, OpenAI and Google all published policies before the midterms, while Microsoft did not publish one, and Meta’s policy does not mention Meta AI.
That gap matters because the same tools are the first place voters ask. About half of American adults reported using AI chatbots, and 42% of users said they use them to search for information. A chatbot answer about a candidate may reflect who worked hardest to get into it, as one researcher put it, rather than the best information available.
Our read
The most important number in this study is not the worst one, it is the ordinary one. A 5.9% rate of misleading answers sounds small until it is multiplied by millions of questions. The errors that matter most are the boring ones: a date, a deadline, a polling place, delivered with the same confidence as a correct answer.
Copilot’s position is the awkward one here. It ships inside Windows, Edge and Microsoft 365, so its answers reach people who never chose a chatbot at all. The company has no published midterm policy against which its performance can be judged, while Claude’s worse rate at least comes with a published policy and a methodological defense.
For voters, the practical lesson is unchanged. Deadlines, polling places and ballot status belong with state and local election offices, not with a chatbot, and any answer that affects whether a vote counts should be checked against an official source before it is trusted.
Why It Matters
The study arrives alongside a second finding that should worry anyone building election safeguards. In an experiment, researchers prompted five popular chatbots with voter personas and found that all but Gemini produced lists of candidates to vote for after two or three prompts, despite initially refusing.
Copilot was not part of that persona test, but the pattern applies to the same kind of system: a refusal is only a safeguard if it holds every time, and a refusal that leaks under gentle pressure is not a safeguard at all.
That is why the comparison with other companies’ preparations matters. OpenAI, for example, staffed an election operations center for the midterms, a visible commitment to monitoring election answers in real time. The question for Microsoft is whether Copilot’s election handling will get an equivalent public commitment, with published rules and measured results, rather than assurances after the fact.
Background
The study tested Google AI Mode and AI Overviews, Claude, Copilot, Gemini, ChatGPT and Perplexity against voter questions covering winners, policy platforms, candidate comparisons and backgrounds, controversies and voting rules. Its authors caution that the percentages describe test responses, not a measure of how often conversations mislead people, and the prompt set, sample sizes and scoring rubric were not published.
Source: NPR report.
0 Comments