Author: Rajeeva Karandikar

Rajeeva Karandikar is Director at Chennai Mathematical Institute.

On this blog:

• The EVM – VVPAT saga, 24 May 2019.
• Early trends in election results and financial trading , 10 November 2016.
• Going from vote share estimates to seat estimates, 10 May 2014.
• Small samples from big populations shouldn't bother us, 16 December 2013.
• Pre-election polls: Do they work, do they influence voters, and should they be banned?, 13 December 2013.

The EVM – VVPAT saga

by Abhay Bhatt and Rajeeva Karandikar.

There has been lot of grumbling in the Indian media about Electronic voting machines (EVM), esspecially over last 3 months, with opposition parties accusing the government of manipulating EVMs. This reached a crescendo recently when all the opposition parties joined hands and filed a petition in the Supreme Court seeking a directive to the Election Commission to cross verify the vote count reported by the EVM, with that of paper slips produced by the VVPAT (Voter verifiable paper audit trail), in 50% of the booths.

The Election Commission had consulted us about the sampling plan to be put in place to instill confidence about the sanctity of the election process. Here is our take on how to think about these questions.

The background

1. EVMs were first used in the Paravur Assembly constituency in Ernakulam district in the 1982 Assembly poll in about 50 booths. The CPI candidate, Mr. Pillai, defeated the Congress candidate, Mr. Jose, by a thin margin of 123 votes. Mr. Jose went to the High Court with the contention that the Representation of the People Act, 1951, and the Conduct of Elections Rules, 1961, did not empower the Election Commission to use EVMs. While the High Court dismissed the petition, subsequently the Supreme Court upheld the contention and ordered a repoll in these 50 booths. Mr. Jose won the election after this repoll.
2. Subsequently, the Representation of the People Act was amended, and S.61A was inserted, in December 1988, empowering the use of EVMs.
3. These EVMs were designed by public sector undertakings, BEL (a Defence Ministry PSU) and ECIL (an Atomic Energy Ministry PSU), who also manufacture the same. They are subjected to a thorough testing process.
4. The design and production are overseen by a technical expert committee consisting of senior professors in electronics and computer science from leading Indian Institutions.
5. The EVMs do not have any networking hardware. There is no ethernet port, no wifi or bluetooth capability, and thus it is not possible to alter or tamper the memory remotely.
6. The names of candidates on the EVM appear in an order that is determined by the same convention that had been followed since the sixties for the order on the ballot paper. First, the candidates of the national parties appear, in an alphabetical order of their names, then candidates of state parties (again in an alphabetical order of their names) and lastly the rest, again in an alphabetical order of their names. Thus, the order gets determined only after the last date of withdrawal of nomination.
7. The EVMs are distributed across the constituencies via randomisation. The EVMs used in India consist of two units: BU, the balloting unit and CU, the control unit. The BUs and CUs are distributed independently and on the day of polling, the two are connected. If one of them has been tampered or replaced, the handshake between the two units will fail and the pair will have to be replaced.
8. After the voting, the machine is locked by the presiding officer and then the EVM are physically sealed using the same techniques that were used to seal the ballot boxes of old.
9. In the parliamentary elections in 1999, EVMs were used in some constituencies and from 2004, all the elections to the parliament and to the state legislatures are conducted using the EVMs. It should be noted that in several instances, including in 2 out of three parliamentary elections – 2004, 2009 and 2014, the ruling party has lost the election.
10. The fact that there was no way to do a recount, even if a court ordered the same, was a reason for the Supreme Court to ask the EC to find a way of generating a paper trail. The Election Commission appointed a committee of experts, who came up with a design of the VVPAT machine and the Supreme Court ordered that the same be introduced as soon as possible for all elections.
11. The Supreme Court had not ordered the Election Commission to routinely carry out any cross verification of EVM and VVPAT count. The purpose of VVPAT was to have the possibility of a recount if a court so ordered, as a result of an election petition.
12. Since so many doubts had been raised about EVMs, the Election Commission decided that in every assembly constituency, one booth will be picked at random by a draw of lots and for the chosen booth, the VVPAT slips will be counted and cross checked with the count on the EVM. This done in order to increase public confidence in elections.

Our analysis of the recent debate

Over the last year, demands started coming up for verification of much larger number of EVMs. Various petitions were filed. The election commission engaged the two of us along with Mr. Onkar Prasad Ghosh to advise on an appropriate sample size, so if that many EVMs are randomly chosen and the machine count is verified with the VVPAT count and if no mismatches are found, then we can be confident that defective EVMs, if any, are in insignificant proportion.

We felt that given that the EVM's are assigned randomly to constituencies and that the order of names on the EVMs is determined at a late stage, only after the last date of withdrawal, it is not possible to manipulate or tamper the EVMs centrally.

Given that there is no networking component in EVM, tampering is possible only by getting physical access. If someone can get physical access to an EVM and tamper with the EVM, then the VVPAT slips can also be tampered and so validating EVM count by VVPAT count does not give any guarantee that EVM has not been tampered with. For this we must rely on the elaborate process that the Election Commission has about sealing the EVM in a bag, closing the same and storing in such a way that tampering can be detected. This was also the case with the paper ballot and ballot box method of earlier years. We are no worse with EVMs as compared with the old ways, in this regard.

Also, if some smart mind does figure out a way of changing the memory of EVMs remotely, he/she will not stop with doing so in one or two booths. Certainly, the effort will be to tamper with a larger number of constituencies so as to make an impact on the national level.

That is the reason that we took our objective to be to conclude with high degree of confidence that defective EVMs, if any, are less than a certain percentage of all EVMs in use nationwide. In our report we had taken this as 2%, but it can be 1% or 0.5%.

It is true that our suggested method does not give a guarantee at constituency level. But in our view, such a guarantee is not needed. Our sampling scheme can guarantee that the national picture has not been distorted by tampering or by a manufacturing defect. To those who have been insisting upon constituency level guarantees, we would ask: Why stop at the constituency? Should not every voter be assured that his or her vote has been counted correctly? The only way to do so without compromising privacy is to use cryptography. This is the subject of present research, with Prof. Bimal Roy, former director of ISI involved with one such initiative along with a team in UK. But when such a solution would become available, it can and will be attacked as opaque!

The Election Commission, in its affidavit to the Supreme Court, stated that in the last 2 years, over 1500 EVM counts have been matched with VVPAT counts and in all cases the matching has been perfect. Statistically, this alone is sufficient to conclude that EVM-VVAPT in use currently along with all the safeguards and practices in place are good.

We believe there is a lot of misinformation in the media. Various cases of EVM malfunction are being reported in the media and social media. These relate to local body elections or even college student union elections etc., which are outside the purview of the Election Commission. Other reports of EVM malfunction on the day of election are mostly about EVMs which fail the test before voting starts and are replaced.

While our recommendation was to draw a random sample from the population of all the EVMs in use, the EC decided to continue with their policy (for operational simplicity) to draw one booth per assembly segment, which translates to verifying 4125 EVMs by cross checking VVPAT counts. Based on the data about the number of booths across 4125 assembly segments, we were able to assert that if no defective is found in these 4125 chosen booths, we can say with 99.99999% confidence confidence that the proportion defective is less than 1%. This bound is true irrespective of the configuration of the defective EVMs. For example, a group of miscreants could have tried to tamper with EVMs selectively across a few constituencies. However, as long as the number of defective EVMs is 1% or more, the sampling procedure will catch this with a high probability.

The Supreme Court has ordered or suggested that instead of 1 per segment, 5 EVMs per assembly segment be drawn and VVPAT count and EVM count be cross verified. Based on our calculations, we conclude that if there are no defectives found in the 20625 randomly chosen booths (5 per assembly segment), then with 99.99999% confidence, the proportion of defectives, if any, are less than 0.25%, irrespective of the configuration of the defectives.


The authors are Professor, Indian Statistical Institute, Delhi and Director, Chennai Mathematical Institute, respectively.

Early trends in election results and financial trading

by Rajeeva Karandikar and Ajay Shah.

Financial markets and the Trump victory

Donald Trump's election victory was an unpleasant surprise for the financial markets:

On equity index futures markets, the NASDAQ and S&P e-mini futures hit their price limits and for some time, effectively stopped trading. These are dangerous situations, for as is well known in the field of risk management, these `circuit breakers' convert price risk into liquidity risk, which is in many ways a bad deal. When a market stops trading, all risk management stops.

Financial markets in the Indian general elections

On 17 May 2004, when the UPA won the election, this was quite unexpected; most people had expected five more years for the NDA. Nifty crashed dramatically and the risk management systems of financial market infrastructure institutions were tested like never before (or after). High frequency data for that episode is presently not available, hence we're not able to look inside that day.

In 2009, the election result was a positive surprise, when the UPA made it across the finish line with a weakened CPI(M). We have decent data for this day, so here are some pictures. The market was ecstatic:

The turnover on the equity index derivatives market surged:

This reminds us how important trading on such days is to financial market participants, and to the managers of financial market infrastructure institutions. These were also turbulent times, with wide fluctuations of the spot-futures basis and violations of put-call parity on options markets.

Experiences in state level elections

In November 2015, the Bihar elections threw up a surprise. Till about 10am (two hours after counting started), most TV channels were reporting that BJP and allies are ahead of JDU-RJD. Only after about 11 am, a stable picture emerged.

The early hours of counting have been misleading many times. In the last UP elections (in 2012), till about noon (4 hours after counting started) it appeared that the SP would fall well short of the majority mark and BJP would do rather well. By 10 AM, NDTV and Times Now were projecting 180 seats for SP and 100 seats for BJP. There were celebrations at the BJP HQ. In the end SP got 225 seats and BJP 50.

Can elections overwhelm financial markets infrastructure?

Imagine if, on 15th May 2014, when the counting of votes for the Lok Sabha elections had begun, the early picture as reported on TV stations was very different from final outcome. Imagine if the early indicators were showing that we were headed towards a hung Parliament. The market would have crashed. Later in the day, when the full picture emerged, the market would have swung dramatically.

Even with state elections, sometimes, the stakes are very high. Consider the coming UP elections. The BJP won 73 of 80 seats from UP in 2014. The possibilities for the BJP in 2019 critically hinge on their being popular in UP. The market will thus be watching UP closely, interpreting it as a leading indicator about the 2019 general elections.

Why might early results diverge from the final answer?

Before 1999, when elections worked with pieces of paper, early trends were a statistically good predictor. The counting methodology was to first mix all the ballot papers, and divide them in 10-12 parts which would be counted, one at a time, over roughly two days. Each of these lots (which was called "a round") was a large random sample of votes from each constituency. It is not surprising that early trends often prevailed. In 1998 and 1999, we (Rajeeva Karandikar and Yogendra Yadav) had done well by making predictions on Doordarshan about the national tally based on early counting trends.

Things have changed with the induction of Electronic Voting Machines (EVMs). They now take up one booth at a time, which is not random sampling.

Most TV channels get the counting data from one syndicated source. Viewers see the same message in numerous channels, and get lulled into the feeling that the answer is correct as it has come from multiple sources. However, this one source has had methodological problems. This is also introducing errors in the early trends.

This appears to have been at work in the surprises of the UP election in 2012. A similar situation prevailed on the counting day in Bihar 2016 elections, the difference being that on TV channel which was using its own reporters were able to report the correct picture early on.

The way forward

This article is a plea to participants in financial markets to use early trends more carefully. Wait till noon before believing what you are seeing.

Alternatively, the risk management system of clearinghouses may find it useful to have larger collateral for these few days. After all, the only day when the modern Indian financial market system was really stressed was 17 May 2004.

With EVM-based elections, the counting process is pretty rapid. Perhaps we are better off with a brief pause in trading from the start of counting to its end. The Election Commission, and the exchange institutions, should think about these possibilities.



Rajeeva L. Karandikar is Director at Chennai Mathematical Institute. Ajay Shah is a resarcher at the National Institute for Public Finance and Policy.

Going from vote share estimates to seat estimates

by Rajeeva Karandikar, Director, Chennai Mathematical Institute.

My previous blog posts (link, link), showed that opinion polls help us predict the vote shares of major parties (or alliances). This leads us to the next question: converting estimates of vote share into estimates of seats. This is much harder than meets the eye. Sampling is not done in all constituencies. In constituencies where we have respondents in our sample, the sample size is not large enough to predict the winner in these constituencies in isolation.

Hence, we have to build a mathematical model of voting behavior. It is widely believed that an individual's identity (caste, religion, economic status, gender, age) plays a role in his/her vote. Moreover, this behavior varies from state to state. So if one were to build a model incorporating these parameters, we will end up having a large number of free parameters, particularly as these correlations are likely to change from state to state. Data on these parameters at the constituency level is not available, as census data is compiled and reported at the district level. Thus, this approach is unlikely to yield a good result.

To get a tractable solution, we do not need to build a model for voting intention of an individual voter- it suffices to build a model for voting behavior of a constituency. One can assume that the socio-economic composition of a constituency does not undergo a major change in 5 years (this is true for most constituencies). We assume that the change in vote share for a given party in a constituency, from the previous election to the current one, is constant across a state, or a smaller geographic sub-region of a larger state. We call this the uniform swing assumption.

If the sample size at the state level, or a sub-region in a large state is adequate, we can estimate these vote shares via a methodologically proper poll.

Then using the uniform swing assumption described above, using actual data from the previous election and estimating vote shares of parties across a state or a sub-region of a state enables us to estimate vote shares of major parties in every constituency.

This is a crude model! In the historical experience, the reality has diverged from the uniform swing assumption. However, it turns out that with some further work, this model yields fairly good estimates of seats at the national level.

Consider a scenario where in one constituency, out of a sample of size 101, Candidate A gets 52 votes while candidate B gets 49 and in an adjacent constituency also on sample size 101, Candidate C gets 59 votes while candidate D gets 42. While we can be fairly confident that C will win, the same cannot be said of A. What is the best case scenario for B? The scenario is that A and B are almost neck-to-neck with B having a slight edge and yet a sample of size 101 shows B to be behind A by 3 votes. The probability of this happening is the same as the probability of seeing 49 or less heads in 101 tosses of a fair coin, which is 0.42. We assign B a winning probability of 0.42 and A a winning probability of 0.58. On the other hand, the probability of 42 or less heads in 101 tosses of a fair coin is 0.06. Thus, we assign a winning probability of 0.06 to candidate D while 0.94 to C.

This analogy can be extended to 3 or more significant candidates. We have been using this for the top three candidates: first the best case scenario for the third candidate, then the best case scenario for the second candidate and the remaining for the first candidate. This needs an assessment of the standard deviation of the vote estimates.

To summarise, based on an opinion poll (or our day-after poll), we obtain statewide vote shares and vote shares in sub-regions of a state and build vote estimates for all major parties in each constituency. Then we convert the vote share estimates in each constituency to predicted win probabilities for the top three candidates. Finally, we add up the probability of wins for a given party across all the 543 constituencies and this yields an estimate of the seats for the party.

This methodology, developed over 15 years ago, has yielded useful seat estimates. Of course, this element also has an errting to seat estimates go in opposite directions, which is lucky. Sometimes, the two errors conspire to go together and give bad results.

From October 2005 onwards, CNN-IBN, CSDS-Lokniti and I have done numerous poll projections. Most of these are based on post poll surveys, but occasionally these are also based on pre-election polls. Here is the listing of all such occasions: a comparison of what we said and what happened. I am giving vote share estimates and my seat estimate corresponding to that and the actual vote share and seats.

BIHAR(October, 2005)

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
JDU-BJP	36	37	127-137	147
RJD+	31	31	72-80	65
Others	33	32	29-39	31

ASSAM 2006

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	31	31	52-60	53
BJP	11	12	10-15	10
AGP	22	20	25-31	24
Others	36	37	26-35	39

TAMIL NADU 2006

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
AIADMK+	35	40	64-74	69
DMK+	45	45	157-167	163
DMDK	10	8	2-6	1
Others	10	7	-	1

KERALA 2006

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
LDF	51	49	107-117	98
UDF	41	43	25-31	42
Others	8	8	0-1	0

WEST BENGAL 2006

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
LF	53	50	230-240	235
INC	16	15	17-23	24
TMC+	27	29	32-40	31

PUNJAB 2007

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
SAD-BJP	41	45	50-60	68
Congress	41	41	50-60	44
Others	18	14	3-9	5

UTTARAKHAND 2007

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	31	30	21-27	21
BJP	34	32	33-39	35
Others	35	38	8-12	14

UTTAR PRADESH 2007

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
SP	25	25	99-111	97
BSP	29	30	152-168	206
BJP+	22	18	80-90	52
Congress	11	9	25-33	26
Others	13	18	21-27	26

GUJARAT 2007

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	47	49	92-100	117
Congress	42	39	77-85	62
Others	11	12	3-7	3

KARNATAKA 2008

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	30	34	79	110
Congress	35	35	86	80
JDS	21	19	45	28
Others	14	12	14	6

LOK SABHA 2009

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
UPA	36	36	210-225	262
NDA	28	24	180-195	159
Left	-	8	30-40	24
BSP	-	6	24-32	21
Others	-	26	-	77

BIHAR 2010

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
JDU-BJP	46	39	185-201	206
Congress	9	8	6-12	4
RJD-LJP	27	26	22-32	25
Others	18	27	9-19	8

ASSAM 2011

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	36	39	64-72	78
BJP	9	11	7-11	5
AGP	18	16	16-22	10
AIUDF	13	13	11-17	18
Others	24	21	12-20	15

KERALA 2011

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
LDF	36	45	69-77	68
UDF	45	46	63-71	72
Others	9	9	0	0

TAMIL NADU 2011

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
DMK+	44	39	102-114	31
AIDMK+	46	52	120-132	203
BJP Front	3	2	-	-
Others	7	7	-	-

WEST BENGAL 2011

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Left	40	41	60-72	62
TMC+	50	48	222-234	227
Others	10	11	-	5

UTTARAKHAND 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	39	34	31-41	32
BJP	32	33	22-32	31

PUNJAB 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
SAD+BJP	41	42	51-63	68
Congress	40	40	48-60	46

MANIPUR 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	30	42	24-32	42
TMC	14	17	7-13	7

UTTAR PRADESH 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
SP	34	29	232-250	224
BSP	24	26	65-79	80
BJP+	14	15	36-44	47
Congress	12	12	28-38	28

GUJARAT 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	48	48	129-141	116
Congress+	36	39	37-45	60
Others	16	13	4-10	6

HIMANCHAL PRADESH 2012

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
Congress	41	43	29-35	36
BJP	40	38	29-35	26

KARNATAKA 2013

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	23	20	39-49	40
Congress	37	37	117-129	122
JDS	20	20	34-44	40
Others	20	23	14-22	21

MADHYA PRADESH 2013

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	41	45	136-146	165
Congress	35	36	67-77	58
Others	24	19	13-21	7

RAJASTHAN 2013

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	43	45	126-136	162
Congress	33	33	49-57	21
Others	24	22	12-20	16

CHHATISGARH 2013

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	42	41	45-55	49
Congress+	38	40	32-40	39
Others	20	19	7-13	2

DELHI 2013

	Vote Estimate	Vote Actual	Seat Estimate	Seat Actual
BJP	33	34	32-42	31
Congress	23	24	9-17	8
AAP	27	30	13-21	28
Others	17	12	1-5	3

Small samples from big populations shouldn't bother us

by Rajeeva Karandikar, Director, Chennai Mathematical Institute.

In the last few weeks, there has been a lot of discussion about opinion polls. Some people have questioned if these have a scientific basis. Indeed, each time we disclose our findings based on an opinion poll, someone raises this question.

In this article, I offer a simple explanation of the scientific basis of an opinion poll. The key result is this: If the methodology is sound, an opinion poll based on a sample size of 25 thousand respondents in our country, where there are over 500 million voters, can yield surprisingly good projections of the vote shares of major parties.

Consider a lottery. Suppose you are told that a box contains lottery tickets and that each ticket has a number written on it: 1 or 1000. You can pay Rs 100, and then put your hand inside the box and draw one ticket from the box. The prize would be the amount written on the ticket (in Rupees). Most people would not agree to play unless they are told how many tickets in the box have the number 1 or 1000 written on them. However, if they are told that 99 percent of the tickets have the number 1000 on them, many may be willing to play. Indeed, even if the cost of playing the game was Rs 900, many would opt to play if 99 per cent of the tickets have 1000 written on them.

Suppose, instead, you are told that host of the casino will put his hand in the box and draw a ticket. There are still 99 percent tickets with the number 1000 and only 1 percent with the number 1. And you have ascertained that all tickets are identical in all aspects other than the number written on it. Even then, you would be a bit apprehensive, as the host might have put the tickets with the number 1 at the bottom of the box, and given a chance the host can dig deep in and draw a ticket from the bottom. If you are allowed to shake the box and mix the tickets well, you would probably still play.

Now let us consider another scenario. A political party has two claimants for a Lok Sabha constituency, say Raghu and Prasad. Suppose the constituency has 5 lakh voters. Let us imagine that we have lottery tickets with the following characteristics: Each ticket has the names of 2501 voters from the constituency and also that the ticket is coloured Red if 1251 or more voters on that list prefer Raghu over Prasad and the ticket is coloured Blue if 1251 or more voters on that list prefer Prasad over Raghu. Suppose all such lists are written out on otherwise identical lottery tickets.

Let us assume that there is at least a 5 percent gap in the support level of the two candidates. It can then be shown that over 99 percent of the tickets will have the name of the candidate with more support. This is just a question of counting and is purely arithmetical- no element of probability or statistics enters here. Thus it is a matter of fact and not of belief! Indeed, 99.3939507 percent of the tickets will have the colour of the candidate with more support.

Return to our example of two candidates with a gap of 5 percent or more. If the party draws a ticket out of the box after mixing it well, it will end up knowing which candidate is more popular. Here the logic is that since 99 percent of the tickets have the colour of the more popular candidate, we can assume that the colour of the ticket drawn has the winner's colour. Once again the decision maker should ensure that the tickets have been mixed well.

Here are the percentage of tickets that will have the colour of the winner for different combinations of population sizes and sample sizes. In each case, we assume that there is a 5 percent gap in the vote shares of the two candidates.

Sample size	Population size (Total number of voters)
	500000	1000000	2500000	5000000	10000000	25000000
1001	94.35	94.34	94.34	94.33	94.33	94.33
1201	95.87	95.87	95.86	95.86	95.86	95.86
1401	96.96	96.96	96.95	96.95	96.95	96.95
1601	97.75	97.75	97.74	97.74	97.74	97.74
2001	98.75	98.75	98.74	98.74	98.74	98.74
2501	99.39	99.39	99.39	99.38	99.38	99.38
3201	99.77	99.77	99.77	99.77	99.77	99.77

The remarkable thing about this is that while accuracy increases as sample size increases, the population size (total number of voters) has only a negligible influence on the accuracy. This is somewhat counter intuitive but true. A sample of size 2501 will give the same accuracy when the population size is 1 million or 25 million!

The following table gives the percentage of lists which have the colour of the winner when the gap between the winner and loser is only 2 percent. Here again we see that sample size determines the accuracy and population size has very little effect on it.

Sample size	Population size (Total number of voters)
	500000	1000000	2500000	5000000	10000000	25000000
1601	78.86	78.85	78.84	78.83	78.83	78.83
2501	84.2	84.17	84.16	84.15	84.15	84.15
3601	88.58	88.54	88.52	88.51	88.5	88.5
5001	92.24	92.19	92.16	92.15	92.15	92.14
8001	96.44	96.38	96.34	96.33	96.33	96.32
10001	97.83	97.78	97.75	97.74	97.73	97.73
15001	99.36	99.32	99.3	99.29	99.29	99.29

At the bottom of this article is a computer program written in Python which does these computations. You have to believe me or have a mathematical expert confirm the accuracy of the program and then run the same on a computer with Python installed (which is available freely at http://www.python.org/). You can change the population size, the sample size and the gap between the support levels to get the accuracy level of the corresponding sampling scheme.

The same situation applies when we conduct an opinion poll. We select a group of 2501 voters, and ascertain the opinion of this group, called a sample. It is the percentage of votes for a party in this chosen sample that we report as the estimated vote share of the party. The crucial thing is that our choice should be as if we have written all possible lists on lottery tickets and put them in a box, mixed them well and then drew one and the names on the ticket constitute the group. This is what is called random sampling. One can use random number generators to generate such a random sample from any list of voters.

Colloquially, most people think that random means arbitrary. This is far from true in the scientific setting. Random sampling refers to the methodology of choosing a sample. In this context, it means choosing one list out of all possible lists as if we are drawing a lottery ticket (in the scenario described above). What I have described is the simplest sampling scheme. There are variations which may be more appropriate in a given situation.

Suppose we have access to a list of all telephone numbers in use in a constituency. We can use a computer program to generate a list of 2501 randomly generated phone numbers from this list. We can then call these numbers and ascertain the view of the owner. In this case we could estimate the opinion of the group of people who have phones. In this case, richer, urban, educated class will be over represented and our estimate could be biased. This methodology is used in the US and seems to work well (at least over the last 50 years, while it did not work in the '30s and '40s even in the USA when the telephones were not ubiquitous all across the country).

Thus the most important ingredient in the opinion poll is the methodology of sample selection. One must be sure of getting opinions from a representative sample. Unless the sampling is done properly, there is no statistical guarantee that the estimate would fall within 2.5 percent of the true vote share (with 99 percent probability for the sample size of 2501).

Readers can experiment with the program and obtain accuracy of random sample based prediction for a given sample size, population size and gap in the support for the winning candidate and the losing candidate. The program prints the total number of lists, number of lists where winner has majority and then the last line is the accuracy (in percent).

Python code:

g=5#Gap in percent support for winning candidate and the losing candidate
psize=500000
#population size
ssize=2501
#sample size
def binomlist(N, R):
    '''Return [binom(N,0), ... , binom(N, R-1)]'''
    a=[1]
    for k in range(1, R):
        a.append((a[k-1]*(N-k+1))//k)
        assert((a[k-1]*(N-k+1))%k==0)
    return a

n=psize
#Population size
print('Population Size :')
print(n)

m=(100+g)*n//200
print('Gap in the level of support between the two candidates 
(in percent):')
print(g)

#Total number of supporters of the winning candidate
print('Total number of supporters of the winning candidate :')
print(m)

k=n-m
#Number of supporters of the losing candidate
print('Total number of supporters of the losing candidate :')
print(k)

r=ssize
#Sample size
print('Sample Size :')
print(r)

s=1+(ssize)//2
#Majority mark in the sample
print('Majority mark in the sample :')
print(s)

t=r-s
b=binomlist(n,r+1)
c=binomlist(m,r+1)
d=binomlist(k,t+1)
print('Total number of lists :')
print(b[r])

z=sum([ c[r-k]*d[k] for k in range(0,t+1)])
print('Total number of lists in which winning candidate has majority support:')
print(z)

y=(z/b[r])*100
print('Percentage of lists in which winning candidate has majority support:')
print(y)

Output: Population Size: 500000
Gap in the level of support between the two candidates (in percent): 5
Total number of supporters of the winning candidate: 262500
Total number of supporters of the losing candidate: 237500
Sample Size: 2501
Majority mark in the sample: 1251
Total number of lists: 62231690581446480003124486564603608079722664287780679850769754811742042826440472887015830702924480575139486249657512804993096017025966527240485971677012460101302514218686266609441052100836909464169270524814906289825323267820948737888768638306721657325213500920099906234174550459916676877801122648015241862393226740611391693419690393435279384448846498164611917690485938916309022444186853678716540339720996823920632761895486203438380430254590374925296252761868287613362669749365125454631374879693160142819869304875906654921349095055838442562414668977024766179959130011021610575662910956134247564521738477313446196261604802302543410146068132670342155475007095024743323867045795400143176727384029281976933600168079297510291849445093067071083684685003730058946519710247034945376030279821029701472923740192102205025797475452531004667596413727636670465215729867754283833374385303145387948051359404453403594361525378558410033629759275932498192096982291800849470571518287063229431447959133385792138084490304666939123657615189822099874121079295131987178206767084477208423116361539422938568526859676309130466065888802081248462657939570182699815625453901386358318350022709995625288828603793916108904428008609734299699221437566336835240257534085393479491186665079655190103428237800738888006964700812940498236110822184478021780415260136866672164326231310650895521248121755859107866938779565130334913321094933601029730436184082485079029558170569819165053571542795991727217330966100414527221364686964529920726163238492293892228326948001117293468138858023516939457994664567261850006311933756947304285561086248788200803564375093003772848775681842197209982100478555863846338584281906599009475583222084487980818308040033671587984993515974558684475022277901970099053541223542134155842504516032224804445183451317380149589970032212804575628082098277081463957839077920898869597586620515995970008514248167231810555336158368760540858584640697240880859068759980546301544945173321069553721350972811702465776038261514751366432380505653269990734628912787921196828014924849957148325947484479478464943528525829530723712207177801854498505379313242978072796608415660424672105172137755545024900415945428256536045336980540671661266557344947764836566529722714879021182976140139129856559145427658178495007317534394739394235188377026548923486253173751616379130541552155758837114472809783850427754469844587936072840642351778220558057232669828423498123063458914684776588125631122103174618980604765576707899260689467306579408356058711623351399260178055917659963408455273580719144814560484832919904878765961591225700327651593973860864438116094850680456864518725262740928051582341539254987525019787256071659676928373298021282046954594050755383326687971809253561980088484298073161856459452325694037739274837365216230582263924667803583485781780218253284218391730866178085262881994066033816393829013721311131748672183078728900933581558734405974104874756534969248232592313582502037142672654121741282578372979805465908127950187274075397490336844923270342615399463969649554126623283032616619860556580636328136753628805680852851407963270089714073626621120839871909711594994253666342272359471320655869954863460407565986906474615595552330627592635455243206575046823339900668550971352633374485688587126074795047908585598712155945668417607256395345722088439538463447076588501001219788542858153049781833657942552401464715635534332260515218505056897905685877043364499338188335401451485749380823361891367165284575198795795706325965459245970820182968416588569262205864560967474390631523041120889653260589129456152566220058515917669342363795423128034942492225269840456119934077650384674330202082758512409968193166923108572334911544560155774100118425500347543423269124631844558114435125175017555500011451376956685419999212743508413967187310121355328643814596714505035428570563667905329220023568761920372318672748066491538386362503932919067623642006192288629490332239924529275660392604135364518178709645251568947829790968485350757937039473403892835221019104946380501456170261792541779653905610112609257982069288434883821539348456638978706277490266096785801912450185230638539829701924352942262894855270142930259031972046282586563892827754397261394135669853192369909536949252496489905384890519693644219374285311950869838840999862576449738089616498010247917038541443671710835200209258303429618341172828857210172546352033108320714086475275833013658816172562851640153013201594935921805444858511862222986921611649620787025702764928938030416878469784542178572899283792381712549547989957375712162715834971052306908424777553250398897933660688114626175343338296459105052106907815907800493632106153130371390265741118879621773827599268210510657654647567961805679873227774988494734117015705540753609174031970835793477945867907801468391170032234998449398312805494784365015754905261103496492914475522013784154069747889118167745622648817823453096293834552709425038099410751211010184684088478062229567458545385381873514215152003667097109597790568765198065077427946525045384254967053032767662079442599022863825992255697642670195087368061303544182026383279381793756537585195072034501032282072218014776155536686226067590207105059978962856264152015857329823054618874575107943555769313879322107342773045700458905454789107511870461578164581868782223102778218653491871855192270951818997164294958334974045773678167469423485625177317370915848177336736332836777996361609707902987046694214863048890113554854413050567534204094725686390362155721963672141096706581649992815877681713676063878647949708426533918893860041211338666717678506988337856766016039452351562182705851879530723219770661660390683134598036166887734683508116125393617837521663026650845446856218974787651129036755981552785539813024142173328570948759613346757263838152602944234205086327543051588254089090386593113031841672362875582419951045117676214677983892982215768091890388410459618648394701970636811984634408588565396887808598116836276413546396278428362439496050403104026679130181272599410000361010047778136678130879383747883747846895444550308537252212467158365879188315673992959209980186727731070450877815644643251764435971587530519135727567687253468334262068241333011975327403420996864840461485541038137569067936744784955690032673877577363436808601455485916733141230626610962300477240992891148835574452260329156066010179688692107195572679448377022660187493117482552354887255967959473828708949465982835085658015438685159219153348125861982966902706795929866032389611325518509983815234810190419913364567450264118122918625091636534133701922567413915709199174234722642774022748876761798838932368596471473383761819955993306073194192511980655731511111418734812039174839069481997922657860771600177158258301963426575629546453731005562602020307021534742971566271133060854173312518512084425581890020765636041299469386187750782069745075999996018174251440607754688000000
Total number of lists in which winning candidate has majority support: 61854535859474557855990802105237752997003079226915266398295566709786571570865126944390968210637518423542079527102570022999166688984274296235885139162890371514814845152010622805158620168912468690693188245682966954787357816511116779763733163402700476933630959164843305433201850578845370091288748854061390074747439388865845974202161930262599395731727353191068011590081152770196877407971636426951731994085907267804768852937825049803838100473521131063052926267944191522907342658317551764904960334072386631507754440746171433527467144733182012255305326199455319159497759562823152663968262865777457103271700128546028508185840232611523695405934728964035118237960868014973326447922706116544354336034347614678384022162974527037904321191181398534575569848882630055799910749797172304564491772423199336789851612216070203204275524094233112452914612289125662370386046257715156289616617574543451300214501841247096787674912979683070061713605543496243219427259075034171350439369320584086710980928379366489759203262100385124489235015029050551301074751781445992781257981654435667272527606697678067098831709443929240127929128547394461174871163509626564598556318324969348111671167120110424508155676842784775760059444981204834187739812753861969928971222420761131855788632604940532247374491023157683617994324064023325600495426562474895522481817088290306507602098761586556424183165192703406960550038339797967819973579603029824346542759436150333392596915015063750963940450511380855736879240949203056727957820806654033532382652697944335187351538193502302752677438979716069371383839023319581409715533431261519318056963567358615092916567918646344809017192689254441873194607032799176580005518847744513314944645132471407540662504364341425558288361717082939160655446029064670317732228963561205409737330114951013218590804768437809071092058235440817461936484573823922609954200258697063724572845222547780365519185476138723059149923014644555837546472430304327776379190867288941326831870645002682813073558114144397020856651396098641428245880407763399111302626215417904725128805326719787159608089308759310539659291511216704403271866844555932756610742675279329626214802593821101127278170409131325586212124151710165142757100487590767687382044871782315436075121171542572626492638808471680968040729208050469140926077912584273576340641257014821813020214765524809778864954507078512918888714254859177939535150020680594439865146499021168025610541634139341501484522062829556587659324452928788309544618343094453013332661828069966660143355367154612846409938357974206448260728642168060123578255631198332463974385754771418670196985850132396338939942461308023817769221799318621949923468227274010642413121733050935121767411991316887764568127280843446615938837129446630717759517950376477713807259688797866576252414083146600686650880207899667891677604126827061960251420067738418355966327207257392917436336731865485642950327130932493289464011591491047704107981756427219291055264143312206202230577972263205514048071843127797269511291476129899775547198248411450106880937713920234740112362636474232781683958078134701589591040282984364866200009733100910271820163097453414872456237992771088179902024383111236689464059435281063899471133588050088649308537531041274287153585336369394790451199357846698579951685792888991089679835339060335104126017261132825267018640531309505427477537620870911733521553658152454976725660963872168787051805718895517444599140932756344873557326427855400701615942270826310406658514734551562588662900750800729828396653844953945482326221733257969297201087018127132328768798046176023879900010060373993216522794747523289640979321071773794218866605017475730371283611610761350911346861500008729919019301067072919496561285987332861930046572860663808696120651875913544097783371879746140834717719015688160225361094599128341000683910024247665018933294496488786624226050404864707017014357988489253639376601921455654391172067437596345359240709330773941058600329053762687941644065922633145644130469148317865760411103692624307884683174819618671442069152710707134162672027346237374834097350336170928453416600244355207200811054739419529397124154126071051792133199895128726950214690751373964902723667255890924491013232603657299792685025004023713057058893428464868779315295174749486468550601318702477239163614341044641062996932687024872027811548077618001234615864548746268130250171192778617488138004967564306698248535187717421076365106348806106821537346949643846700449330358599365475382470246918611292871944809121969111017864598545829051438437850288221899022100184712042404754086803996345120912030499710004313022194682886744221708406796242511777371707223033713474614368940615606518938162404539717599572710179404281043850851420088512734477536989718707445595772395424620311444817067973609967777834700409297277415549465074115328057064442133675519804938678457212846870080100109455964899729071965181905612555945488493560180366058606748476317280119865991331975082418726690147297736709796082622562532706341896303468093693062647926409712747591212083935703258289194014940709114745569037161608333832469544837654632228442721822702724003627170996788824619104568972975934911932378037473120556765765762195941632676171512945909129242448444638553636536609594131129282152539412714009585868892428320440895663351867733881652239580949142684683481353248562798873757287916757580468627389991155852813550448917969327892911097933990419196015960374273051473175438993206323137256403186109363448282668586166311354485384932931804144339988774403004402405896945245725422702431733690124017022924657227323047668961814477218714072568186313359079987411940130396681799715540521231268441953447718529859895803980262710401387658027606656500057205126849851478255459358279161783808824363571309794095088890379247630444890577923100756584545549108006677956146637604172798742819492521507826898758791892006636607596635419708170718743967976837221333720083327132366807991394197404515600528862762389248905808587268155994320338244998724029406492065210164089814201328490050287743177666822378851971606713160006189128166640525745262825626356517895066557744597519977449314900187385212625277914375251622206262429685098796590400768356218087685302088179153086859311291523273637686891639986221244405181640484253991269897217169690609115907769063954029261659568602193333309944736964180426430695238571165245318661728551017255810873045429123026855411671804358355994634963743745030745365862888759330245538534977291471181991934485986474528502644452587454925992820425856177106615897073589580463926178212465336178760260176190531289730768305813149258314589213372104868616854925650428977268668149331635952396377069894057586620585416152265929820699770161781555434581354014880088969190425658318935898315061156729251402541740877913593377241246796287235439071594744201325364245711979195094423501588318728952447886937824706628041515791435885246207092104530611200000

Percentage of lists in which winning candidate has majority support: 99.39395070510206.

Pre-election polls: Do they work, do they influence voters, and shouldthey be banned?

by Rajeeva Karandikar, Director, Chennai Mathematical Institute.

I have been a psephologist for about 16 years and have had a fair amount of success in predicting seats in an upcoming election. Here is a post-mortem of what I said and what happened in the current round of elections.

From 2005, I have been working with CNN-IBN and Centre for Studies in Developing Societies (CSDS). CNN-IBN engages CSDS for the survey. CSDS does a great job of running surveys `by the book'. I use vote share data from the survey to come out with seat projections, which CNN-IBN carries on air.

How did we fare in the recent state elections?

Seat count predictions on air (CNN-IBN)

	BJP	Congress	AAP	Others
Madhya Pradesh	136-146	67-77	-	13-21
Rajasthan	126-136	49-57	-	12-20
Chhatisgarh	45-55	32-40	-	7-13
Delhi	32-42	9-17	13-21	1-5

The outcome

	BJP	Congress	AAP	Others
Madhya Pradesh	165	58	-	7
Rajasthan	162	21	-	16
Chhatisgarh	49	39	-	2
Delhi	31	8	28	3

These results show the power and the limitations of opinion poll based projections. If one simply counts the number of cases out of 13 that the actual results are within the interval projected, the score is just 4. However, one should see these as having correctly predicted clear and decisive victories for the BJP in Rajasthan and Madhya Pradesh. For Chhatisgarh, we predicted correctly that the BJP will win, with a much smaller gap than Madhya Pradesh and Rajasthan. So I would count all the three as being good predictions, with Chhatisgarh being very good.

As for Delhi, we underestimated AAP support and marginally overestimated BJP but we had the ordering right: Congress in third position with the possibility of touching a single digit, and BJP as the single largest party.

Vote share estimates on air (CNN-IBN)

	BJP	Congress	AAP	Others
Madhya Pradesh	41%	35%	-	24%
Rajasthan	43%	33%	-	24%
Chhatisgarh	42%	38%	-	20%
Delhi	33%	23%	27%	17%

The outcome

	BJP	Congress	AAP	Others
Madhya Pradesh	44.9%	36.4%	-	18.7%
Rajasthan	45.1%	33%	-	21.9%
Chhatisgarh	41%	40.3%	-	18.7%
Delhi	34%	24.5%	29.5%	12%

Here, the survey has worked well, and the errors are generally within the statistically acceptable range. The conversion from votes to seats is a non trivial transformation - since one needs to estimate the distribution of votes across the state in addition to the overall percentage of votes in the state. This requires building an appropriate statistical model. I will explain my methodology for this stage in a future article.

One can see that in MP and Rajasthan, there was underestimation of BJP votes by a few percentage points. The error in vote to seat conversion went in the same direction, and as a result our prediction was much lower than the outcome for the seats obtained by BJP. In Chhattisgarh on the other hand, the survey estimated the gap between BJP and Congress as 4% while the actual gap turned out to be less than one percent. In this case, the error in vote to seat conversion and the error in the vote share estimate cancelled out, and we got a result that was bang on. Of the four, I was the least confident about Chhattisgarh (and I had said so on air) since the estimated gap in vote shares was small.

In my experience, the predictive power of any opinion poll that is done a while before voting is rather poor. For one, any such poll can only measure the mood of state or nation at the time of poll and cannot estimate the potential change that can happen close to the voting day. Some psephologists claim to estimate this change by conducting polls at regular intervals and then extrapolate to get an estimate of this change. However, this assumes there are linear time trends in vote share, which is unlikely.

The other problem is the selection process that determines who in the general population (that is sampled in a survey) shows up to vote. The propensity to vote is not uniform across socio-economic stratums of the society. One can try to factor these in but that can inflate the error.

Exit polls are designed to take care of both these issues. However, choosing respondents in a randomised manner as they exit the booth is rather difficult and our experience has shown that it does not produce a representative sample (as measured by the gap between the socio-economic profile of the sample and population). Hence, we prefer to do a post-election poll, where in the days following actual voting, we do a household survey with sound methodology. In the current round, various exit polls had got numbers close to the actual results for MP, Rajasthan and Chhattisgarh. But for Delhi there was wide variation. And in the past, there have been occasions when exit polls had given an incorrect picture while we got it right with our post-poll.

Going beyond forecasting, these polls are valuable in understanding what was happening on the ground. As an example, the CSDS poll in MP, Rajasthan and Chhattisgarh showed that the gap between BJP and Congress vote shares was higher in rural areas. This diverges from the common view that the BJP is an urban party. The CSDS website gives the breakup of voting intention by various socio-economic groups, and this is valuable knowledge.

Regulation of opinion polls

Do opinion polls influence voter behaviour? In each of the surveys done by CSDS, one standard question is: `Who did the respondent vote for in the last election'. This refers to the last Vidhan Sabha election, if this is a poll for the Vidhan Sabha and the last Lok Sabha election, if the current one is for the Lok Sabha. Almost invariably the recall for whoever won the last time is much higher than the actual votes, even when the winner from previous poll is set to lose the current election.

Thus in 2011, a much larger percentage of voters seem to recall having voted for the left front in 2006 though in 2011 they were voting for Trinamool Congress. I may add that our estimate of the Trinamool vote share and seats in 2011 was very accurate. The same was true in Tamilnadu where, while voting out the DMK, a much larger number of respondents seem to recall having voted for them.

We have observed this time and again across various states and consistently over the last 15 years. The only explanation that I could come up with is that there is a general tendency to go with the winner.

This raises the concern that a political party may run a media campaign claiming that it is ahead in the polls. This justifies regulation (though not a ban). I feel this regulation could be self regulation by the media, e.g. through the Press Council. The regulation should require that each published poll reveals, in public domain, the detailed methodology of sample selection, the sample size, the socio-economic profile of the sample, the dates when sampling was done, the names of the core team members who supervised the survey and the methodology used to convert vote estimates to seat conversion. All agencies that release such information should be open to an audit by an expert group formed by an autonomous body such as Press Council.

CSDS and I have been very open about our methods. The sampling methodology is on the CSDS website, the sample size is always given on air and the socio economic profile of the sample is also given on the CSDS website. I have written about the vote to seat conversion in an academic article: Predicting the 1998 Indian parliamentary election, Karandikar, R. L.; Payne, C.; Yadav, Y., Electoral Studies; 21, 1; 69-89, and have been talking about it in seminars.

Fortunately, over the years, the visibility of any one opinion poll has declined, with so many agencies doing polls and making contradictory predictions. In the recent Delhi elections, the range of seats projected for the Aam Admi Party was from 6 to 31 out of 70 seats! Hence, the salience of this debate has probably declined greatly.

I do believe that there is a feedback loop: if all surveys point in a certain direction, at least some voters tend to get influenced. But this is no reason for a ban. After all, newspapers and TV channels also talk about their assessment of the political situation and if they all seem to point in a certain direction, this too has an effect on the electorate. In addition, under the Indian legal system, while the government can easily do censorship on television, this is harder with newspapers and more generally on the Internet. Hence, even if a ban were desirable, it is not feasible.