« previous | TPM CAFÉ READER POSTS HOME | next »
Natl Expert: Instant Runoff marred with flaws
Just today a new report has been released on Instant Runoff Voting, by Mathematician Kathy Dopp, President of The National Election Data Archive. Here is her press release, and the full report is linked if you wish to review it. North Carolina has been targeted by outside groups to be a "beta test" for this election experiment. Many groups are being persuaded to endorse Instant Runoff Voting without hearing these important facts. North Carolina's voting software and machines are not up to mission requirements for this. Senior citizens and less educated voters will be hindered by this system.
Monday, June 9. 2008
New Report Says that Worrisome Realities Mar Alternate Voting Method - 15 Flaws and 3 Benefits of Instant Runoff or Ranked Choice Voting
Park City, UT June 9, 2008
The National Election Data ArchiveThe National Election Data Archive has released a research report “15 Flaws and 3 Benefits of Instant Runoff Voting or Ranked Choice Voting” which concludes that “Ranked choice voting (RCV)/instant runoff voting (IRV) is not worthy of consideration and its use should be avoided” because “there are simpler, fairer, less costly, more auditable alternative voting methods.”
According to Kathy Dopp, the report’s author, “The RCV/IRV method creates serious obstacles to implementing measures to detect and correct machine vote miscount, such as post-election audits. If counting complexity, increased potential for undetected vote fraud and error, increased costs, and difficulty of auditing were not important factors for elections, then RCV/IRV could be considered an improvement over today’s voting method.”
"Instant runoff voting" or “ranked choice voting” is a system where each "vote" is a rank ordering of all the candidates. The counting would proceed in "rounds" where in each round, the candidate with the fewest votes in that round is eliminated (both from the election, and from all orderings inside votes).
RCV/IRV is billed by its proponents as a solution to the “spoiler problem”. The “spoiler problem occurs when two candidates have overlapping support and both candidates are penalized. When a third party candidate receives an amount of votes that is more than the vote margin between the two major political party candidates, it may tip the balance of votes to the major political party candidate who is favored by fewer voters overall.
Support for RCV/IRV has grown since the 2000 election, and it is being tested or considered for adoption in some jurisdictions within Minnesota, North Carolina, and California.
According to Dopp the flaws of IRV/RCV include that it does not solve the “spoiler” problem except in special cases; it requires centralized vote counting procedures at the state-level; it encourages the use of complex high-tech voting systems; it confuses voters and increases over-vote rates; its complex and time-consuming to implement and to count; it makes post election analysis difficult to perform; it is difficult and time-consuming to manually count; it is difficult and inefficient to manually audit; it could necessitate counting all presidential votes in Washington D.C. if a national popular vote compact were passed; it entrenches the two-party system; it could deliver unreasonable outcomes; not all voters’ ballots are treated equally; it is costly; it increases the potential for undetectable vote fraud and erroneous vote counts; and it violates some election fairness principles.
The benefits of RCV/IRV over today’s U.S. voting method, called “plurality voting”, include that RCV/IRV eliminates the spoiler scenario in situations where the minority party candidate is behind both frontrunners; it will not elect a candidate who loses pair-wise to all other candidates; and it gives voters an opportunity to express their preferences among all candidates.
The National Election Data Archive recommends that fundamental integrity of elections should be restored first before considering any alternative voting methods, and notes that today, not one State utilizes all the measures that are required to ensure fundamental election integrity such as:
1. public access to all election records and data necessary to evaluate the integrity of the electoral process,
2. observable post-election independent manual audits of machine vote counts,
3. post-election ballot reconciliation of all printed, counted, unused, and spoiled ballots with voter process records, and
4. public oversight of ballot security.
The eight page report “15 Flaws and 3 Benefits of Instant Runoff or Ranked Choice Voting” explains the flaws and benefits of instant runoff voting in detail plus provides appendices with examples of how RCV/IRV violates fairness principles, plus provides three pages of endnotes of references and additional facts.
The full report is found on-line at http://electionarchive.org/ucvAnalysis/US/RCV-IRV/InstantRunoffVotingFla...
This release is also posted online at
http://electionarchive.org/ucvAnalysis/US/RCV-IRV/FlawsIRV-PressRelease....Press Contact: Kathy Dopp 435-658-4657 kathy@electionarchive.org
About The National Election Archive:
The National Election Data Archive has been organized for educational and scientific purposes of promoting fair and accurate elections by researching, developing and promoting methods and procedures to detect voter disenfranchisement and vote count inaccuracy. Such methods include independent manual vote count audits, exit poll discrepancy analysis, and the public release and scientific analysis of election data along with public release of election records necessary to verify the integrity of elections. NEDA is a completely non-profit organization that relies on the donation of time by its volunteers who donate their time and expertise because of their dedication to vote integrity and public service. The project depends on donations from from citizens who are concerned about fair and free elections in the U.S. in order to continue its work. All donations are tax deductible. To make a donation or become involved in the project, please visit http://electionarchive.org
Fair Vote, a national organization, and Fair Vote NC are asking groups in North Carolina to endorse Instant Runoff Voting, also called IRV. The NC Coalition for Verified Voting vehemently opposes this voting method for many reasons. NC Democrats will be discussing resolutions this week at "town halls" and voting on them at the NCDP state convention on June 21.
IRV directly threatens NC's hard fought for Public Confidence in Elections Act, our law that requires paper ballots, audits and set strict standards for voting vendors. Our law would have to be gutted in order to weaken standards enough to allow for uncertified, untested voting software needed for IRV. Standards requiring vendor responsibility would also have to be removed.
Other North Carolina voters oppose IRV because of voter confusion and also because it is so difficult to count:
Eugene Weeks, Chair of the Wake Voter Education Coalition says: “We feel that IRV will disenfranchise certain segments of voters-especially the challenged and impaired voters. The ballot that is being used now is already confusing to some voters, yet you want to antagonize and confuse the voters more by asking them to not only vote for one candidate, but indicate a second and third choice before leaving the voting booth. Where is the voter's rights in this process?”
Janice Sears of Wake County said: “If the best board of elections in North Carolina had this much trouble counting 3,000 votes, this is too dangerous to try statewide....The claim that 'voters like it' does not impress me because whether they like it or not has nothing to do with whether it is an accurate and effective way to conduct an election and count votes”
A Hendersonville voter said “It doesn't make any sense I call it instant confusion."
Rueben Blackwell, Rocky Mount City Council Member and co-chair for the NC Justice Center advised that: "To cast out an instant runoff speculative experiment in communities that have had historic voting rights violations issues is absolutely wrong…"
Is IRV worth the damage to our verified voting law, is it worth the expense, the voter confusion?In San Francisco, the largest IRV jurisdiction in the US, in 20 contests the results were the same as if a plurality contest were held. In other words, the final winners were still the candidates with the most votes in the first round of voting.
1. If you are a registered democrat, please contact your county and district party chairman ASAP, ask them encourage the delegates attending the State Convention (June 21) to vote YES for the Resolution OPPOSING Instant Runoff Voting.
County Party Chairman:
District Party Chairman:
2. Speak up at one of the town hall meetings this week.
The North Carolina Democratic Party Resolutions and Platforms Committee has 2 Town Hall meetings this week, one in Raleigh this Tuesday, June 10, 2008 - 7:00pm, the other in Greensboro on Thursday, June 12, 2008 both at their county party headquarters. The meetings are open and the Committee invites participation from all Democrats as they review and consider the Committee's recommendations to the State Convention. You can encourage discussion about the Instant Runoff Resolutions being considered for a vote at the state convention.
Vote YES for Resolution Opposing Instant Runoff Voting: The 2nd and 13th districts have passed resolutions in opposition to Instant Runoff Voting(IRV) because of it endangers the Public Confidence in Elections Act that we worked so hard for and that passed in August 2005.
Vote NO for Resolution Supporting Instant Runoff Voting: The 11th district Dems have passed a resolution in favor of Instant Runoff Voting.
Delegates will be attending the NCDP State Convention this Saturday, June 21st at the New Bern Riverfront Convention Center, 203 S. Front St., New Bern, 28563 at 10:30 am.
Support the resolution OPPOSING Instant Runoff Voting (IRV)
Resolution to restore Election Integrity by Opposing Instant Runoff Voting (IRV)
WHEREAS, supporters of Instant Runoff Voting (hereinafter “IRV”), a form of Ranked Choice Voting (hereinafter “RCV”), succeeded in getting an IRV pilot program passed in 2006, allowing IRV to be used in up to 10 municipal elections in 2007 and up to 10 county elections in 2008, and that said law further requiring that the State Board of Elections set up closely monitor the program and report on the results to the General Assembly;
NOW, THEREFORE BE IT RESOLVED, that the North Carolina Democratic Party and elected and appointed Democratic officials urge the North Carolina General Assembly not to extend the IRV pilot project beyond 2008, and
BE IT FURTHER RESOLVED, that the North Carolina Democratic Party and elected and appointed Democratic officials urge the North Carolina General Assembly hereby withhold any further endorsement of IRV and oppose IRV because it has endangered Public Confidence in Elections in North Carolina.
3. Ask your non political group or organizations to adopt their own resolution opposing Instant Runoff Voting and share that with the NC Coalition for Verified Voting, via joyce@ncvoter.net
For more information please read Jan 14, 2008 Point of View: Worrisome realities mar instant runoff voting
Learn more about IRV in North Carolina at http://www.ncvoter.net/irv.html and also at
http://www.instantrunoffvoting.us














Comments (4)
i can imagine recounting by hand is tedious. and the vote integrity issues shouldn't be ignored.
but it is an improvement over the current system which is deeply flawed.
The current system discourages competition because a 3rd party helps the party that is least compatible with it.
The country needs party competition right now. The elephants are a party broken by neocons. That makes for a donkey monopoly. I know there is a lot of donkey love around here, but give them too much power and they will turn into asses.
June 9, 2008 11:45 PM | Reply | Permalink
IRV in real life: it has done nothing whatsoever for third parties in San Francisco, the largest IRV jurisdiction in the US.
San Francisco's "IRV" is a hot mess. And expensive. SF spends about $2.00 per reg voter on education, and will spend about $12 million on new voting machines soon.
Where is the net savings?
The IRV ballot is a portal into hell. An article by Electionline quoted San Francisco voters as describing IRV this way:
"Voters also questioned the value of ranked-choice voting."
"There are a lot of people who only mark one [candidate] or the same person three times,"
"I don't want to vote for a second one, I want this one."
Jay Bordeleau, an election inspector at Notre Dame Des Victoires in Union Square concurred.
"There are a lot of people who only mark one [candidate] or the same person three times," he said.
And what a sick waste of money on printing ballots for the Nov 2007 election:
there were three places on the ballot to rank candidate choices for the sheriff's race, in which only two candidates were running and the district attorney race in which Kamala Harris ran unopposed.
And in 20 consecutive IRV elections in San Francisco, since its adoption, the end result was the same as if the election were a "winner take all" or plurality. In other words, the person who won the first round ended up being the winner of the final round.
Oh, and I keep seeing the phrase "incumbent protection" in articles explaining why the mayor did not have a serious challenger and as a reason for the unchallenged DA contest.
I hope other places in the country will look before they leap.
June 10, 2008 1:07 AM | Reply | Permalink
Kathy Dopp is NOT an expert on voting methods, and in fact has gathered her miss-information from fringe and unreliable sources. The flaws are not with IRV, but with her report.
De-Bunking Kathy Dopp's "15 Flaws" of Instant Runoff Voting
1. Dopp: "Does not solve the "spoiler" problem except in special cases…."
Dopp has her “special cases” reversed. In fact, IRV solves the spoiler problem in virtually all likely U.S. partisan elections. Whenever a third party or independent candidate is unlikely to be one of the top vote-getters (true in over 99% of U.S. elections), IRV eliminates the spoiler problem completely. If a third party grows to the point that its candidates out-poll major party candidates, another issue that is related to the spoiler problem, can occasionally arise. This is where supporters of a third party candidate may worry that by supporting their favorite candidate, they risk causing their less-preferred compromise choice to be eliminated from the final runoff, leading to the election of their least-preferred choice. In other words, the issue of whether to vote for your favorite choice, or to rank your compromise choice first can resurface in this unique circumstance. But this is extremely rare and no different than a candidate in a party’s political primary arguing “Vote for me because I am more electable in the general election.”
2. Dopp: “Requires centralized vote counting procedures at the state-level…"
IRV creates no need to centralize the counting or the ballots themselves, although that is one possible counting procedure -- and indeed a central count is often sensible for smaller jurisdictions. But all that is required to implement IRV is central coordination of the tally. If ballot images are recorded on optical scan equipment, the data from those images can be collected centrally for an IRV ballot. If a hand-count is conducted, vote totals need to be reported to a central tallying office in order to determine what step to take next in the count. In Ireland, for example, there are 43 counting centers in the presidential race. Election administrators count ballots and report their totals to a national office that in turn instructs the administrators at each counting center on what to do next. The entire process takes less than a day even though more than a million ballots are cast.
3. Dopp: “Encourages the use of complex voting systems and …[FairVote promotes] electronic-balloting…”
Most government IRV elections are in fact conducted with hand-count paper ballots, including national elections in Australia, Ireland and Papua New Guinea. FairVote is a leading advocacy organization for IRV, but it is joined in supporting IRV by numerous other organizations and individuals, including the founders of TrueVote Maryland and election integrity leader David Cobb and Anthony Lorenzo.
As to FairVote, it advocates the replacement of all paperless voting machines with paper-ballot systems, such as optical scanners. All three of the major voting machine vendors have created optical scan options for ranked-choice ballots. Not all of these are ideal (some, for example, cannot handle more than three rankings), but FairVote expects IRV elections to be overwhelming run on paper ballot systems in the future. FairVote advocates that all such machines store a redundant electronic record of each ballot, as well as a paper ballot to allow for better fraud detection, and simplify ranked ballot tabulations. Rather than making such elections more complicated, this would simplify the process, while improving transparency and integrity.
4. Dopp: “Confuses voters…”
All the evidence shows that voters are not confused by IRV. The rate of spoiled ballots did not increase in any of the U.S. cities when they switched to IRV. For example, Burlington (VT) used IRV for the first time in a hotly contested race for mayor in 2006, and among those casting votes in the IRV race fully 99.9% of ballots were valid, with the very highest valid ballot rate in the ward in town with the highest number of low-income voters. San Francisco’s rate of valid ballots in the most closely contested race in its first citywide election with IRV was 99.6%. Furthermore, exit polls have been conducted in every city having an IRV election for the first time in the modern era. Each survey shows that voters overwhelmingly prefer IRV to their old method of elections.
5. Dopp: “Confusing, complex and time-consuming to implement and to count…”
IRV certainly is simpler for election officials and voters than conducting a whole separate runoff election to find a majority winner. It is more complicated to administer than a single vote-for-one election, but election officials have adjusted well to their new responsibilities. Note that the winning threshold for an IRV election, as with any election, must be specified in the law.
6. Dopp: “Makes post election data and exit poll analysis much more difficult to perform…”
To date, IRV election can make it easier to do post-election and exit poll analysis. Because optical scan counts with IRV require capturing of ballot images, San Francisco (CA) and Burlington (VT) were able to release the data files showing every single ballot's set of rankings – thereby allowing any voter to do a recount and full analysis on their own.
Exit polls can be done just as well under IRV rules as vote-for-one rules. California requires a manual audit in its elections, which has been done without difficulty in San Francisco’s IRV elections. Manual audits should be required for all elections, regardless of whether IRV is used or not.
7. Dopp: “Difficult and time-consuming to manually count…”
Manual counts can take slightly longer than vote-for-one elections, but aren't difficult, unless many different races on a ballot need to go to a runoff count. As cited earlier, Irish election administrators can count more than a million ballots by hand in hotly contested presidential elections in one standard workday.
8. Dopp: “Difficult and inefficient to manually audit. …”
IRV can be manually audited just as well as vote-for-one elections, although it does take more effort (since voters must be allowed to express more information on their ballot). A manual audit can either be done using a random sample of ballots from all jurisdictions, or a random sample of ballots from a random sample of voting machines, or by a complete re-tally from a random sample of voting machines. A complete re-tally of all ballots (a recount) is, of course, possible but unnecessary unless a court recount is ordered.
9. Dopp: “Could necessitate counting all presidential votes in Washington, D.C.. …”
If the Electoral College were abolished and IRV were then adopted for future national popular vote elections for president, there would need to be national coordination of the tally in order to know which candidates got the fewest votes nationwide and needed to be eliminated – just as in Ireland. But the actual counting of ballots does not need to be federalized any more than if IRV was not used, and could be conducted by counties, states or whatever level is easiest and most secure for that jurisdiction. Note that voters certainly would be pleased to have a majority winner in elections for our highest office.
10. Dopp: “IRV entrenches the two-major-political party system …”
IRV neither "entrenches" nor "overthrows" the two-party system. It simply ensures no candidate wins over majority opposition. If a minor party has the support to earn a majority of vote, it can win in an IRV election. If not, it will not win.
IRV is a winner-take-all method, like plurality voting and two-round runoffs. However, IRV allows independents and candidates with minor parties to run without being labeled as spoilers. This may reveal a higher level of support for these parties, and if these parties are attractive to voters, their support may grow.
Relating to multi-party representation, any winner-take-all, single seat election method tends towards two dominant parties, at least in any given geographic area. To allow for multiple parties to regularly win office, jurisdictions should adopt a form of proportional representation in which candidates will be able to win office with less than 50% of the vote.
Note that Australia’s IRV elections are often cited as an example of two-party domination. But while the two major parties (one of which is divided into two parties, with one party running in one particular region of the country) dominant representation, the minor parties contest elections very vigorously, with an average of seven candidates contesting house elections in 2007. That year the Green Party did not win any seats in house elections, but it ran candidates in every district and earned 8% of the national vote. It naturally would prefer a proportional representation system, but supports IRV over alternate winner-take-all systems and uses it to elect its internal leaders.
11. Dopp: "Could deliver unreasonable outcomes…."
Unreasonable outcomes are less likely with IRV than with any other single-seat voting method in use today. Every single voting method ever proposed can deliver "unreasonable outcomes" in some scenarios, but real-world experience has shown IRV to be one of the best methods. The overwhelming number of election method experts agree that IRV is fairer and more democratic than plurality voting even if some might prefer other theoretical voting methods. The American Political Science Association (the national association of political science professors) has incorporated IRV into their own constitution for electing their own national president. Robert’s Rules of Order recommends IRV over plurality voting.
12. Dopp: “Not all ballots are treated equally…”
This charge reveals a lack of understanding of how IRV works. All ballots are treated equally. Every one has one and only one vote in each round of counting. Just as in a traditional runoff, your ballot counts first for your favorite candidate and continues to count for that candidate as long as he or she has a chance to win.
Your rankings should be considered as backup choices. Your ballot will only count for one of your lesser preferences if your favorite candidate has been eliminated. Every ballot counts as one vote for your highest ranked candidate who is still in the running in every round of counting.
Note that courts have upheld IRV for this very reason and Robert’s Rules of Order recommends it over plurality voting. For quotations from a court decision upholding IRV's equal treatment of ballots, please see below.
13. Dopp: “Costly. …”
The two main expenses associated with the transition to IRV are voting equipment upgrades and voter education. Both of these are one-time costs that will be quickly balanced out by the savings coming from eliminating a runoff election in each election cycle. In San Francisco, for example, the city and county saved approximately $3 million by not holding a separate runoff election in 2005, easily covering the mostly one-time costs spent in 2003-2004 to implement the system.
In North Carolina, counties spent $3.5 million for the Superintendent of Public Instruction runoff in 2004, at election with statewide turnout of only 3%. In 2007, IRV elections in Cary (NC) avoided the need for a runoff in one of the city council districts that would have cost taxpayers $28,000.
An effective voter education program can also be done for relatively little money by learning from what types of education worked well in other jurisdictions and what types did not – with the biggest factors being a good ballot design, clear voter instructions and effective pollworker training in that order. In a report to the Vermont General Assembly, the Vermont Secretary of State estimated that, based on how well IRV was implemented in Vermont’s largest city of Burlington in 2006, voter education for statewide IRV in Vermont would cost less than $0.25 per registered voter. In a city of more than 100,000 people, Cary spent less than $10,000 on voter education – with highly favorable reactions from voters.
14. Dopp: “Increases the potential for undetectable vote fraud and erroneous vote counts…"
Actually, just the opposite is true, so long as paper ballots (such as optical scan) are used. The reason that any attempts at fraud are easier to detect with IRV is that there is a redundant electronic record (called a ballot image) of each ballot that can be matched one-to-one with the corresponding paper ballot. Cities such as San Francisco (CA) and Burlington (VT) release these ballot files so that any voter can do their own count. Without such redundant ballot records (which are not typical with vote-for-one elections) there is no way to know for certain if the paper ballots have been altered prior to a recount.
15. Dopp: “Violates some election fairness principles…."
This charge reveals either a general lack of understanding, or intentional miss-representation. Every single voting method ever devised must violate some "fairness principles" as some of these criteria are mutually exclusive. Dopp's example in appendix B of "Arrow's fairness condition" (the Pareto Improvement Criterion) completely misunderstands the criterion, and gives an example that has no relevance to it (and contrary to her implication, IRV complies with this criterion). IRV works essentially the same as a traditional runoff election to find a majority winner. When the field narrows to the two finalists in the final instant runoff count, the candidate with more support (ranked more favorably on more ballots) will always win. Some theoretical voting methods may satisfy some "fairness' criteria, such as monotonicity, but then violate other more important criteria such as the majority criterion, or the later-no-harm criterion.
Endnotes
The rank order ballot used in instant runoff voting (and other voting systems) is known by political scientists as the "single transferable vote" or STV. This balloting procedure has been consistently upheld in United States courts as constitutional and upholding the "one person, one vote" principle. As an example, here is what the Michigan Court ruled in upholding the use of instant runoff voting in an Ann Arbor, Michigan Mayoral race in a 1975 challenge:
"Under the "M.P.V. System" [IRV], however, no one person or voter has more than one effective vote for one office. No voter's vote can be counted more than once for the same candidate. In the final analysis, no voter is given greater weight in his or her vote over the vote of another voter, although to understand this does require a conceptual understanding of how the effect of a "M.P.V. System" is like that of a run-off election. The form of majority preferential voting employed in the City of Ann Arbor's election of its Mayor does not violate the one-man, one-vote mandate nor does it deprive anyone of equal protection rights under the Michigan or United States Constitutions."
page 11, Stephenson v Ann Arbor Board of City Canvassers File No. 75-10166 AW
Michigan Circuit Court for the County of Jackson
The Judge also observed on page 7,
"Each voter has the same right at the time he casts his or her ballot. Each voter has his or her ballot counted once in any count that determines whether one candidate has a majority of the votes. . . . Far better to have the People's will expressed more adequately in this fashion, than to wonder what would have been the results of a run-off election not provided for."
June 10, 2008 11:05 AM | Reply | Permalink
Kathy Dopp and Experts Deconstruct Instant Runoff Voting, rebutting the rebuttal Fair Vote's "debunk" of Kathy Dopps' report on IRV is now itself - "debunked"
Fair Vote's talking points have been repeated over and over as if they were fact, instead of unsubstantiated opinion! Many people accept these talking points without question because the methods and mechanics of IRV are dauntingly complex, and because the idea of IRV sounds so appealing. People in North Carolina and Minnesota should be espcially alarmed since Fair Vote is pushing IRV so desperately in those states.
Kathy Dopp has laid out the facts and addressed the faults and myths about IRV in a second report on the flaws of instant runoff voting. Kathy enlisted the assistance of top national computer voting experts and also election method experts. Her report is the first one I've seen that addresses whether IRV works, and how it (negatively) impacts election integrity.
The hardest hitting section of the report is where Kathy Dopp deconstructs Fair Vote's rebuttal, "point-by-point". In effect she dismantled the pro IRV talking points. Additionally, in her rebuttal of a rebuttal, Kathy shows us how Fair Vote "crafts" their words and responses.
My favorite part of the report was - Appendix F: Rebuttals to Fair Vote's "De-Bunking Kathy Dopp's 15 Flaws of Instant Runoff Voting"
This appendix relies heavily on the expertise, writing, and research of Adb ul-Rahman Lomax and his rebuttals to Fair Vote on the election-methods@lists.electorama.com with some help by other email list members, including Warren Smith. This appendix rebuts the Fair Vote organization's attempted rebuttal of the first version of this paper.
(See http://www.fairvote.org/?page=2285 or http://www.fairvote.org/dopp for the full text of Fair Vote's rebuttals.) Note: The numbering of IRV flaws is slightly different in this revised version above than in the original version due to the addition of two new flaws in this addition.
1. "Does not solve the "spoiler" problem except in special cases…."
....read the full report here:
http://electionarchive.org/ucvAnalysis/US/RCV-IRV/InstantRunoffVotingFlaws.pdf
June 18, 2008 3:17 PM | Reply | Permalink
Post a Comment