Search interesting materials

Showing posts with label author: Viral Shah. Show all posts
Showing posts with label author: Viral Shah. Show all posts

Saturday, October 18, 2014

Elections in Maharashtra: Have the fires of nativism subsided?

by Naman Pugalia, Renuka Sane, Viral Shah.

The results of the Assembly polls in Maharashtra are anxiously awaited. The four main contenders, the Congress, the NCP, the Shiv Sena, and the BJP have all been part of one of the two principal coalitions, the Democratic Front (Congress and the NCP) which ruled the state for the last 15 years, and the Mahayuti (Shiv Sena and the BJP) that has been the principal opposition alliance.

The battle against the `other'


After these two principal alliances in Maharashtra broke up, ahead of the assembly elections, political parties have been quick to rouse nativist sentiments to secure the Marathi vote. Each political party contesting in Maharashtra, and especially in Bombay, has been vying for the "marathi manoos": the BJP by bringing together Narendra Modi and Chattrapati Shivaji, and the Shiv Sena and the upcoming Maharashtra Navanirman Sena reacting strongly against such a comparison, comparing the BJP leaders as foot soldiers of Afzal Khan, the commander of the Adil Shahi, who was killed by Shivaji. At heart seems to be the idea that the son-of-the-soil will never prefer an outsider as the ruler of the state.

The roots of this angst date back to the Samyukta Maharashtra Movement launched in 1955 in Pune. As Kumar Ketar in the Asian Age says:

the business lobbies, mostly consisting of the Gujarati's and Marwaris wanted Mumbai to be an independent city state or a bi-lingual or autonomous city state. But the mass movement led by Samyukta Maharashtra Samiti foiled that plan. The Marathi angst of the time was one of the reasons for the Shiv Sena's rise, and continues to the reason for the undeclared hostility between the Gujarati-Marwari business community and the Marathi working class.

The Gujarati-Marathi antagonism was mostly restricted to Bombay. In other parts of Maharashtra, it has always been a "Maratha" vote, something that the Congress and the NCP had capitalised on over the last few decades. In the 54 years since Maharashtra was formed, the Congress has ruled the State for 49 years. Of its 17 Chief Ministers, 10 have been Marathas. The outgoing cabinet did not have a single non-Maratha!

By this logic, you would have expected that a national party, with a low support base in Maharashtra in the past, with a Gujarati leader and a Gujarati campaign manager, would not fare that well in the coming elections.

Several commentators, have, however argued that the new Marathi middle class has moved on in its economic and cultural ambitions. It no longer shares the sense of injustice that was the cornerstone of the Samyukta movement, and is in fact, brimming with enthusiasm to participate in the new India. In addition, over the years, migration on a large scale has taken place into Bombay and it's environs, and into Poona, which has created a new set of immigrant voters.

How relevant is the issue of the "marathi-manoos"?


FourthLion Technologies has been conducting message testing polls in the run up to the elections in Maharashtra to tease out voter preferences using its Instavaani. The methodology involves using a control and multiple treatments, and comparing the treatments to the control to get a relative understanding of the persuasion power of different messages.

In a message testing poll, the control is a simple horse-race poll, that asks voters to pick the party or candidate of their choice. The poll on October 1, 2014, showed that 41% of voters preferred the BJP, 11% Congress, 14% Shiv Sena and 11% the NCP. BJP was comfortably in the lead. This is the control.

In each treatment, a particular message is read out to the listener, and then the horse-race question is asked again. Differences from the control give us a sense of the immediate short-term impact of this message on the minds of the populace. These polls are conducted by randomly sampling phone numbers across the entire state. The poll typically strives for 200-400 observations. With assumptions of perfect random sampling of a small sample from a representative population, the margin of error is 0.98/sqrt(n). At 200 samples, the margin of error is 7%, and at 400 samples, it is 5%. These polls are typically carried out as soon as news breaks out, and situations develop in real-time, allowing the observation of the mood of the people within hours after an event.

Here are some results which illuminate attitudes to nativism:

  1. `Prithviraj Chavan is a true son of Maharashtra. He went to school in Karad, which is in south Maharashtra. His mother and father, Premalakaki and Dajisaheb Chavan, went to jail because they fought for an independent state of Maharashtra. No other candidate for Chief Minister has the same legacy of fighting for Maharashtra as Prithviraj Chavan'.
    Would you vote for% of respondents
    BJP31%
    Congress26%
    Shiv Sena15%
    NCP11%
    Others17%
    This shows that the CM's background matters quite a bit, and led 10 percentage points of voters to switch from the BJP to the Congress. This also explains why Prithviraj Chavan led the Congress' campaign in the state - his popularity is higher than the party's.
  2. `In 1960, Gujarati minister Morarji Desai ordered police to fire on activists of the Samyukta Maharashtra Samiti, killing 105 Marathis. The Samyukta Maharashtra Samiti activists won their fight to create an independent state of Maharashtra. Today, the BJP is bringing Gujaratis such as Amit Shah to again place Maharashtra under Gujarati dominance'.
    Would you vote for% of respondents
    BJP33%
    Congress14%
    Shiv Sena25%
    NCP11%
    Others17%
    If there was indeed strong antagonism about Gujaratis, this question should have caused a lot of people to switch votes out of the BJP. However, only 8% of the voters seems to have moved away from the BJP, mostly to the Shiv Sena.
  3. `The BJP has no leaders in Maharashtra who are clean, honest and capable of running the state government. That is why the BJP has to parachute in outsiders like the Prime Minister and Amit Shah to campaign for them. The BJP is afraid to announce who their CM candidate will be because their local leaders, including Devendra Fadnavis and Eknath Khadse, are inexperienced and unqualified to run the second-largest state in India, and also have dozens of criminal charges against them'.
    Would you vote for% of respondents
    BJP39%
    Congress15%
    Shiv Sena19%
    NCP10%
    Others17%
    This yielded the least movement away from the BJP: only two percentage points, which is not statistically significant. 39% of voters continue to root for the BJP. It shows there is far greater confidence in the BJP leadership than in that of any other parties.

This post is about nativism, so we don't talk about other measurement of how voters feel. But one point must be made. None of these treatments work as well as other treatment messages that talk about construction of roads, public works, anti-corruption, etc. These results suggest that the passions of caste and creed are now less important; that the history of the Gujarati-Marathi antagonism has faded from memory. By this logic, the BJP was perhaps on the right track in breaking away from the Shiv Sena, and focusing on its core messages of development and good governance. This is what voters in Maharashtra seem to care about.

Implications


We may conjecture that three things are going on:

  1. Part of the reason for this move away from nativist sentiment is the personal appeal of the Prime Minister. His approval ratings, measured in a survey FourthLion did for Mint on August 16, 2014, were highest in Maharashtra and West Bengal. In the bye-polls, there was very little involvement of the Prime Minister, and the BJP did not do well. It is no surprise then that the BJP is seeking votes under the Modi banner, with messages like "Chalo chale Modi ke saath" ("let's walk with Modi") and "Ab ki bar Modi sarkar" ("this time let's make it the Modi administration").
  2. Anti-incumbency against the state government, and the 2 parties (INC + NCP) that jointly governed the state for 15 years, has voters looking for an alternative. Given the BJP's own brand, their assessment of being able to achieve a majority on their own, and the country beginning to taste the benefits of a clear mandate, the BJP has an edge in asking voters in Maharashtra and Haryana to give it a clear mandate in the states too, so that they can work well with the Centre.
  3. But most important is the fact that the Indian electorate has moved on. The desire of the voter to look beyond tribal considerations is the reason why Maharashtra might be the first state to throw up a verdict that challenges preconceived notions about the eternal power of old hatreds.

Does this have implications for regional parties elsewhere in India? Many regional parties may have to go in for radical reconstruction if nativist fires are subsiding. Some, like the BSP, have begun doing this. The entire eastern and southern belt, which sees strong regional parties - West Bengal, Orissa, Telangana, Andhra Pradesh, and Tamil Nadu - could see change. While Jammu and Kashmir and Jharkhand will give us some more intuition in the coming few months, Bihar is going to be the next big test in 2015.

One possible argument is that Maharashtra is a better state, with greater exposure to new ideas, low levels of violence, and a successful economy. In contrast, the backward parts of East India may still be trapped in the old nativist ways. But what about the South? The developments in Maharashtra could be particularly portentious for the better states of the South.

The politics of Bombay has long been benighted by the problem of nativism. What was once a great metropolis has been bogged down by decades of nativist politics. These results show a possibility for becoming a normal city, where the political questions that matter are about efficiently producing local public goods.

Friday, October 10, 2014

Scientific management for election campaigns in India

by Viral Shah.

From 2010, I worked in the Aadhaar project for three years. This helped me learn how Government works and how it does not. The one big takeaway from my experience of working with the Executive arm of the Government was that to bring about true and lasting change, to restructure our defunct institutions and build new ones, one needs to engage with the Legislative arm, with politicians.

Politics in India is a cottage industry. Everybody loves to talk about it; most are cynical; very little is known about how things actually work. The professional ways of working -- which are found in business, science and slowly in government -- are least visible in politics. In particular, the crucible of politics -- the election campaign -- is just black art. In the last 20 years, we have seen family businesses get shaken up, professionalise, and embrace technology and process engineering. We have seen some parts of government do the same. We have seen a transformation of some parts of academiaa. The one place which has seen the least change is politics in general and election campaigns in particular.

This suggests opportunities for achieving important change. Shankar Maruwada, Naman Pugalia and I started a company -- FourthLion Technologies -- to provide professional services to political campaigns. Over a couple of campaigns, we have slowly learned how elections in India work. We have looked at an array of data from various public and private sources, and developed tools and technologies to aid election campaigns in multiple phases.

Elsewhere in the world, election campaigns are run through scientific management. In the US, both the Democratic and Republican parties have voter databases, where one can search for any voter by name. Through a variety of analytical methods, campaigns know fairly well which voters are likely to vote for them, and which ones are marginal, and on which groups of voters, no resources should be expended. Starting with such databases, every voter contact is recorded (a volunteer knock, a telephone call, a letter sent, an email, or a tweet) in the same way companies manage customer services through a CRM system. It took a decade of work for the machinery of election campaigns in the US to get to this stage, to transplant ideas which were well developed in the world of business.

When thinking about election campaigns in India in a professional way, there are many challenges. There are multiple parties, many races are multi-cornered, and with first-past-the-post elections, a candidate can win with as few as 20-30% of the votes. The voter lists are very poor in quality, with every possible error of inclusion and exclusion. They do not capture the large scale of urban migration and are often tampered with. Although the Election Commission of India has made great strides in conducting free and fair elections over the last several decades, much more remains to be done, and the quality of the voter list is perhaps the weakest link in Indian democracy today.

Every election has three natural phases: Registration, Persuasion, and Turnout. A campaign should start 6-12 months before voting date, by registering voters. Three months before the election, voters need to know the candidate and be persuaded, and finally the last week is focussed on "Get The Vote Out", or Turnout. At each stage of the campaign, one has to focus on the message and mobilisation. The message is all about what the candidate says and does, and mobilisation is about execution on the ground, in the digital sphere and in the media. Each stage as a distinct methodology for scientific management, and the problems faced can be quite surprising. As an example, it is not uncommon for 2 to 3% of the population of a constituency to be working for all the candidates, put together, in the last 2-3 weeks. This calls for the processes of large-scale management.

We provide tools and technologies for different parts of the campaign, starting with coalition dynamics, seat selection, analysis of past elections, formulation and testing of messages, calculating the reach of every channel (hoardings, TV, radio, print, etc.), managing call centres, and a control room for the turnout operation and voting day. We use data, analytics, and technology at every stage of a campaign to aid decision making and efficient deployment of resources. Traditional politics often deploys resources in a "Spray and Pray" manner, while we try to combine all available information and intuition so as to use resources more effectively.

In our experience, an incumbent who has a good chance of getting the ticket has a head start as he is able to do preparatory work for the campaign well ahead of time. As emphasised above, the campaign should really start 6-12 months before the voting. All too often, in India, candidate selection is left to the last minute. This makes it impossible to mount a serious campaign, and generally plays in favour of the incumbent. Once we start thinking of an election campaign as a systematic project, this induces the discipline of a minimum time period that is required to execute all the steps, just as is the case with all well planned projects.

On one hand, our thinking about process improvement for election campaigns consciously draws from successful techniques of scientific business management which have been perfected in the worlds of business, science and government in India. Along the way, we have seen that the speed, agility, and scale required in political campaigns in India is something unique when compared with the worlds of business, science and government in India. To some extent, we are seeing innovations in the field of election campaigns that can usefully inform, and sometimes get directly transplanted, into the other three worlds.

We are learning how our democracy works. The accountability is jarring, as any politician will tell you: voters make every possible demand, and speak their mind to the candidate, in as direct a way as can be. Millions of micro-deals are struck with candidates by individuals and groups of people. These are genuine deals about actions of the State and not bribery or corruption. These micro-deals bubble up into the processes of government and ultimately shape policy. It is a rough and tumble world which clashes against our dreams of representative democracy, but it is also astonishing how much this is about representative democracy.

Monday, September 15, 2014

Technology and institutional machinery that will save Rs.75,000 crore a year

by Viral Shah.

In a recent article, Ajay Shah discussed the potential for Aadhaar to help save Rs.75,000 crore per year, by overhauling the administration of susbidies. The first level gains come from using Aadhaar numbers in the databases of every scheme to elimnate ghosts, fakes and duplicates. The second level gains come from enforcing a one price policy. There is an incentive for arbitrage when and only when a commodity has two different prices in the market. Aadhaar technology helps ensure that all subsidised goods are sold at market prices, and the subsidy is transferred directly to the targeted beneficiary. Finally, there is a need to rationalise the subsidy and social safety net policies - who gets subsidies, how do they get them, how are subsidised goods produced, and how they get distributed.

Let's go closer into these questions, envisioning the concrete actions required on some of these fronts.

LPG subsidy

An implementation plan for the LPG was in the Report of the Task Force for Direct Transfer of Subsidy for LPG, Fertilisers and Kerosene where Nandan Nilekani was the Chairman.

The first step of implementing this plan was the cap on the number of cylinders of LPG that can be purchased by a household. Starting with 6 initially, political considerations led to this limit to be raised to 9 and then 12 subsidised cylinders annually. However, as the report notes, the average consumption of cylinders is less than 9 cylinders annually, and this decision should be reverted to 9 cylinders. While this cap has immediately led to a gain, over time, fake LPG consumer connections will be created as there is ample incentive to do so.

To address this requires implementing the second recommendation: to link every LPG connection with an Aadhaar number and deliver cylinders at market prices. The subsidy amount for the first 12 cylinders is transferred directly to the consumer into their Aadhaar-linked bank account as soon as the cylinder is delivered. While this sounds like science fiction, the Dhande Committee Report finds that this scheme had been rolled out in 291 districts with high levels of Aadhaar registration, and Rs.5400 crore of subsidy was transferred into 28 million Aadhaar-linked bank accounts. This is about 10% of the total subsidy for the year of 2013-14, which was Rs.46,000 crore, with 120 million subscribers in all. The tangible outcomes obtained through this work were: 0.6 million duplicate connections detected, and an 18% reduction in the sale of domestic LPG cylinders.

It took years to align all the stakeholders - the Oil Marketing Companies, Banks, National Payments Corporation of India, UIDAI, Ministry of Petroleum and Natural Gas, the Ministry of Finance, the distributors - to develop the IT systems, processes, exceptions, training materials, etc. Today, this is a scheme that can be rolled out at scale, after one signature on a green sheet by the Petroleum minister. Conservatively, this signature can deliver a saving of Rs.10,000 crore within 2 years.

Today, the private sector cannot enter the domestic LPG business, since the subsidies are funded as under-recovery by the Ministry of Petroleum and Natural Gas, and this is available to the Government owned Oil and Marketing Companies. Once LPG is sold to the customers at market price, it also allows for the entry of the private sector in this business, since the subsidy is directly transferred to consumers. There has been very little innovation in this business due to lack of competition. We will most likely see more investment in LPG import terminals, innovations such as LPG cylinders made from lighter materials, and operational improvements such as in logistics as the sector sees higher competition. Consumers will benefit with lower prices and better service if there with more competition. No signatures are required for this to happen!

Fertiliser subsidy

The same report also outlines a solution for fertiliser subsidy based on the same principles. The size of the fertiliser ecosystem is the same as that of LPG. While 120 million consumers benefit from LPG subsidy, there are about 100 million farmers who benefit from subsidised fertilisers. There are major differences though. LPG is largely sold in dense urban areas, whereas fertilisers have to delivered in sparsely populated rural areas. The distribution of LPG is carried out by Oil and Marketing Companies that have fully computerised operations across the country, and directly appoint their own distributors. In the case of fertilisers, the production and imports is mostly in the private sector, which largely receives the subsidy at source. The Central Government administers the movement of fertilisers up to the district level. Beyond that, the State Government's Department of Agriculture takes over, and different States have different distribution models.

Every aspect of fertiliser manufacturing is controlled and subsidised. Given the importance of food security, even inefficiencies are tolerated to a great extent. Take the case of Urea, which is manufactured from both, natural gas and Naphtha, with natural gas being significantly cheaper. However, natural gas pipelines are not available to factories in the south, which continue to use Naphtha, and the Government covers the additional cost. Not only is the price of Urea controlled, but so is the supply of natural gas (yes, fertiliser manufacturing gets priority when there are disruptions in natural gas imports or production), movement, allocation of railway carriages, freight subsidy, etc. There has been no investment in this sector in over a decade, and no technological improvements due to the cost-plus model of manufacturing.

As a result, we live in a society today, where people camping for iphones overnight are cheered, but farmers camping overnight for fertilisers get hell. Here is an interview with a farmer in Uttar Pradesh that sums up all the problems with fertiliser subsidy.

The subsidy on fertilisers has cost us close to Rs.100,000 crore in 2013-14. India has the largest subsidy in place worldwide, which leads to overuse of urea that decreases soil quality, and far too much control in the manufacturing and distribution that leads to perennial shortages and rampant black marketing. (See page 42 of Chapter 2 of the 2014 Economic Survey from the Ministry of Finance).

The savings in the problem of fertiliser subsidy can be conservatively estimated at Rs.20,000 crore per year. However, the implementation is a lot more challenging than the LPG case, due to involvement of the Central and State Governments, a less technologically skilled ecosystem, and the geographical spread of the solution, the grip of co-operatives in many states, etc. At the same time, just because it is difficult, this does not mean it cannot be done.

If Urea is fully decontrolled, we can expect entry of the private sector. Selling at market prices will get rid of the black marketing, and higher customer service levels will emerge from multiple producers competing for customers.

One possible way to deal with fertiliser subsidy is to eliminate it altogether, and instead adjust the higher input costs as part of the Government procurement of foodgrains. In the same way, many other input subsidies such as electricity and water subsidies offered by the state governments can be eliminated altogether and adjusted in procurement.

Reforming the PDS and Kerosene Subsidy

Many of the ideas above can also be implemented in the PDS, where the food subsidy is on the order of Rs.60,000 crore. The Report of the Task Force on an IT strategy for PDS also chaired by Nandan Nilekani provides a comprehensive set of steps to reform the PDS.

The kerosene subsidy is also administered by the State Governments, who receive subsidised kerosene from the Centre. Both, food and kerosene subsidy require better administration.

Again, by Aadhaar-linking and administering these subsidies directly into the bank accounts of the beneficiaries, there are significant gains possible. The beneficiaries can then procure foodgrains from any retail shop or grocer in the country, which has its back-end tied into the subsidy administration platform as outlined in the above report.

The expected savings from these initiatives can be on the order of Rs.20,000 crore.

Payments

An electronic payment architecture for administering Aadhaar-linked subsidies and payments is the backbone of the entire idea. This has already been put in place for purposes of LPG subsidy, and the same architecture can be leveraged for all other subsidies and entitlements.

The ideas behind linking Aadhaar and payments are detailed in the Report on Aadhaar-enabled payment infrastructure. Translating these into implementation gave the following institutional infrastructure:

  • Aadhaar Payments Bridge, operated by National Payments Corporation of India. This is a back-end payments platform through which payments can be sent to bank accounts simply on the basis of the Aadhaar number. This system worked, on scale, for the LPG subsidy transfer.
  • The MicroATM device design and interoperable network through which any shop owner with a mobile phone and biometric reader can become a Business Correspondent. The specifications were designed by an RBI appointed committee and have full stakeholder acceptance, that includes the Indian Banks Association, all banks, IDRBT, and UIDAI. Without issuing any token such as debit cards or mobile phone linkage, anyone in the country can access their accounts simply with their Aadhaar number and biometric authentication. This is working, on scale, in Andhra Pradesh for MGNREGS payments by India Post. A national rollout can be carried out in a matter of months with a decision from the Department of Financial Services as part of the Jan Dhan scheme.
  • The Aadhaar e-KYC service provides electronic KYC for opening of bank accounts and satisfying KYC requirements in any domain.

Every government scheme -- from the MGNREGS, every form of social security pension, Janani Suraksha Yojana, scholarships, and payments to para-workers such as Aanganwadi workers and Asha workers -- can be routed through these payment. These form of payments add up to roughly Rs.100,000 crore. We may guess that Rs.25,000 crore will be saved through computerisation and elimination of fakes and ghosts.

Conclusion

These calculations suggest that it is feasible to save Rs.75,000 crore per year, by fully utilising these ideas and institutional infrastructure. Achieving important change in India is about this two part process: (a) going after big ideas that can move the needle and then (b) building State capacity for the institutional machinery which will make these ideas happen. We fail when we stick to small ideas and we fail when we fail to back big ideas with execution.

The work described above took place over a period of three years at UIDAI, under the leadership of Nandan Nilekani. I was part of the secretariat. The behind-the-scenes stories, and more, are going to be in a book Rebooting Government, that is presently being written by Nandan Nilekani and me.

Wednesday, July 04, 2012

New moves in regulation of debit card payments

by Viral Shah.

The Reserve Bank of India has stepped in to regulate the pricing for debit card transactions. The rationale behind this regulatory change seems to be that lower transaction processing fees paid by merchants will lead to an increase in the adoption of retail electronic payments overall. Issuing banks will have to give up interchange revenue in the short run, but increased transactions will make up for lower fees in the long run. An unintended consequence could be that transaction processing gets adversely affected, and the current growth rate of retail electronic payments slows down. The RBI circular was released on June 28, 2012, and the industry is expected to comply from July 1, 2012.

Card payments background


Electronic payments are an outcome of the delicate combination of technology and incentives. A card scheme (Mastercard, Visa, etc.) brings four stakeholders together. Issuing banks issue payment cards to their customers, who become cardholders, whereas acquiring banks sign up merchants to accept card payments. The card scheme provides the interconnect between issuing and acquiring banks, so that a merchant can accept a payment from any cardholder. The diagram below shows the relationships between all participants in a payments transaction.




The service fee that merchants pays to the acquiring bank for processing transactions is called the merchant discount rate (MDR). The acquiring bank collects the MDR from the merchant and pays an interchange (I) fee to the issuing bank and a network fee (N) to the card scheme. The interchange is usually enabled by the card scheme, which guarantees revenue for the issuing bank. This incentivises the issuing bank to keep issuing more cards, and to spend on marketing and loyalty programs so that cardholders activate the cards and use card payments frequently. The MDR necessarily has to be higher than the interchange and includes the acquirer's processing fee (A), which is used to operate the card processing infrastructure. It is traditionally market determined, and is a contract between the acquiring bank and the merchant, based on the merchant's volumes, risk, chargebacks, infrastructure needs, etc.
We thus have the equation:

MDR (Merchant Discount Rate) = I (Interchange) + N (Network fee) + A (Acquirer's processing fee)

Debit card transaction volumes are growing much faster than credit card transactions, and if one extrapolates the trend from the RBI electronic payments data, it is expected that debit card transactions will have overtaken credit card transactions by volume.

Different methods of payments


Sr. Payment Instrument Cost Who bears Who earns Volume (Million) Value (Rs. Trillion) Source
1. Cheque Rs.25-40 Bank 1,155 67 RBI 2010-11
2. Cash (ATM) Rs.18 Bank 3,500 10 Extrapolated from NPCI 2012 data
3. ECS Rs.2.50 Merchant / Biller Issuing bank 273 2.5 RBI 2010-11
4. NEFT Rs.5 Customer Issuing bank 132 9 RBI 2010-11
5. Net Banking Rs.7-10 Merchant / Biller Issuing bank
6. Credit cards Rs.50 (average txn of Rs.3000) Merchant / Biller Issuing bank 265 0.75 RBI 2010-11
7. Debit cards Rs.25 (average txn of Rs.1500) Merchant / Biller Issuing bank 237 0.35 RBI 2010-11


We have a topsy-turvy world, where banks are willing to bear the cost of transactions for cheques and cash, but expect fees when transactions are processed electronically. Given that electronic payments often lead to customers keeping higher balances in their accounts, and savings on cheque processing and cash withdrawal, it would be rational for banks to incentivise electronic payments for customers and merchants alike.

Should credit card and debit card transactions have the same pricing?


Credit cards are really instruments for lending, whereas debit cards are instruments for making payments. The card transaction model evolved first in the case of credit cards, and was subsequently adopted for debit cards. In the case of credit cards, the interchange fee is used by the issuing bank to fund the cost of credit offered to the customer, and the risk of default, between the time of purchase and the time the customer pays the credit card bill. As a result, in case of a debit card transaction, one would expect (I) to be lower due to absence of credit, (A) to be similar since it is already market determined, and hence, (MDR) to be lower.

Large merchants are often able to negotiate a lower (MDR) with acquirers, even lower than (I), implying that (A) is negative. The acquiring bank offers this service to the merchant if the merchant maintains their current account with the acquirer. Clearly, this model is not scalable and does not work for the long tail of small merchants.

Should point-of-sale (POS) and e-commerce pricing be different?


Today, both credit and debit card transactions have the same MDR - roughly 1.6% for POS transactions, and 2% for e-commerce transactions. E-commerce transactions were once considered riskier with higher rates of fraud, and hence justified higher pricing. Now that two factor authentication is mandatory for internet and mobile transactions, there should be no difference in (I) and (A), and hence in (MDR) for POS vs. e-commerce transactions.

Why do regulators step in?


The card business is a two-sided platform, where the card issuing and merchant acquiring incentives are managed by card schemes. Consider the case of a new entrant in the card scheme business. The new entrant may want to lower prices to establish market share. However, if the entrant offers a lower (MDR) to merchants by lowering (I), issuers find the proposition unattractive. If the new entrant offers issuers a higher (I), merchants face a higher (MDR), and will be unwilling to accept the product. (A) is already market-decided and offers little opportunity for differentiated pricing. A new entrant can at best, charge a lower (N). These are the kinds of challenges faced by the Government backed National Payments Corporation of India (NPCI) in launching the domestic card scheme, RuPay. The issue of debit card transaction pricing was first highlighted in the public domain in the Report of the Task Force on Aadhaar-enabled unified payment infrastructure. Due to such high barriers to entry, card schemes are routinely examined by Governments, and regulators have stepped in to regulate debit interchange pricing. Regulators have typically capped (I), but in India, RBI has decided to cap (MDR). This is likely to have interesting consequences that are not easy to predict.

What will happen on July 1, 2012, when the new pricing kicks in?


On July 1, 2012, the MDR cannot exceed 0.75% for transactions up to Rs.2,000, and 1% for other tansactions. If existing contracts remain in place, then issuers are guaranteed to receive (I) (usually 1.1% or higher) and the card scheme is guaranteed to receve (N) (roughly 0.15%). In such a case, (A) becomes negative, and acquirers will lose money on every transaction they process. However, this announcement by RBI is likely to be considered a material adverse change, one expects contracts to be renegotiated. As per the data above, if debit card volumes are Rs.40,000 crore, MDR paid by merchants at 2% is roughly Rs.800 crore. The new regulation effectively means that the merchants are as a group better off by Rs.400 crore on a notional basis on July 1. The acquirers are likely to be inelastic on pricing, which means that it is issuers and card schemes that will have to largely absorb the notional loss - (I) and (N) will have to be reduced in the new regime. Much of this will be absorbed by five large issuers. Over time, as more transactions are processed electronically, banks will save on processing cheques and cash, and instead earn fees from processing electronic transactions.

Will the lower pricing due to regulation lead to higher acceptance of debit cards overall?


It is clear that merchants who were on the margin, are going to be more likely to accept card payments. It is even likely that merchants will now start demanding debit cards from customers instead of credit cards. At the same time, it is also worth noting that cards are largely accepted by merchants in metros and by e-commerce merchants. India has a network of only 600,000 POS devices and 100,000 ATMs, which is grossly inadequate for a country of our size. (A) is now likely to get fixed due to MDR being capped, and the acquiring business could start seeing stable revenues. This is also likely to lead to an interesting opportunity for the low cost merchant acquiring technologies similar to Square, a number of which are getting ready to launch in India. It could also create an opportunity for NPCI to differentiate itself from established competitors. Overall, the best case scenario is an increased demand for electronic payments with debit cards by merchants and consumers, savings for issuers due to reduced cash and cheque usage, greater acquiring revenues, and more banks entering the acquiring business (PSU Banks are notably absent in the acquiring business). At the very least, one hopes that the oil marketing companies will no longer charge a petrol surcharge fee of 2.5% when paying with a debit card.

What are the possible negative consequences of this regulation?


If (A) is set too low by the card schemes, the acquiring business will be affected. With no further bargaining power, acquirers may have to focus on cost cutting and holding back new investments. Issuing is unlikely to be affected much given the existing base of 300 Million debit cards, and that banks will continue to issue debit cards for ATM usage. One does expect cash-back schemes, loyalty programs, and various other cardholder incentives for debit products to effecively stop, and cardholder fees to increase. Even with ATM interoperability and pricing, RBI has continues to refine its policy (making interoperability mandatory at first, then free interoperable transactions, then restricting the number of free interoperable transactions to five, white labelled ATM policy, etc.). Similarly, this is likely to be the beginning and not the last word on the matter from the regulator. The policy should be stabilized quickly, since it is consumers who suffer during the experimentation phase.

Monday, July 02, 2012

Transparency in the LPG subsidy

by Viral Shah.

Recently, the Petroleum Minister launched the LPG transparency portals for all three Oil Marketing Companies (OMCs):

The Oil Marketing Companies have been constantly leveraging technology to launch various initiatives for offering convenience to their consumers For example, some of them are offering the facility for booking refill cylinders 24x7online through their websites as well as through SMS and IVRS. In continuation of their endeavor to leverage technology to achieve more efficiency and improve business processes, Oil Companies have now put in place systems to capture the complete details of customers and track their LPG consumption pattern with an aim to increase transparency in LPG supply chain. With this information, each OMC has created a transparency portal which is hosted on their individual websites. These portals can also be accessed from MOPNG's website. These portals provide complete details of each customer with their consumer numbers, name, address, no. of cylinders supplied, dates of supply as well as the indicative subsidy amount for the cylinders supplied. The portals feature quick search options to find one's distributor,sort information based on consumer numberand consumer name, see thehighest off take consumer orput in a request to surrender one's connection. Logging a complaint is just as easy. Consumers can now even rate the performance of theirdistributors; and this is expected to help the services to improve further. These portals would truly empower the consumers and civil society to verify or seek information under one roof and bring about transparency in a government program where thousands of crores of subsidy is involved.

Links for the transparency portals are:

LPG distribution

The production, supply, and distribution of Liquified Petroleum Gas (LPG) is governed by the Essential Commodities Act, 1955 and the LPG Control Order, 2000. The LPG Control Order specifies various aspects of LPG distribution in great detail: storage, transport, bottling, packaging, consumer connection, etc. Subsidized LPG is provided largely for domestic use, but institutional use is permitted for Government schools, hospitals, canteens, police stations, etc. Subsidized LPG cylinders are red in colour and contain 14.2kg of LPG, whereas commercial LPG cylinders are purple and contain 19kg of LPG. LPG is supplied to consumers through distributors, who are paid a commission for every cylinder they deliver. Distributors have very thin margins for subsidized LPG, and are given distribution rights by area by the OMC. Margins for commercial LPG are higher, with no restrictiction on distribution. Thus, by design, there is no competition between distributors for subsidized LPG, but commercial LPG is supplied competitively. Almost 90% of the usage in India is domestic, and hence subsidized.

LPG subsidy

A detailed price computation for an LPG cylinder shows that as of June 1, 2012, the consumer pays Rs.399 per cylinder in Delhi, and receives a subsidy of Rs.418 per cylinder. The true subsidy is only Rs.22 per cylinder, and the rest is termed under recovery to OMCs. The Government funds the full subsidy amount of Rs.22, but the under-recovery is funded out of profits of ONGC, OMCs, and partially by the Government. It is also worthwhile to note that LPG is exempt from Excise Duty from Central Government, and often also exempt from VAT.

Sr. Elements (Delhi) Unit Effective 1st June'12
1 Free On Board Price at Arab Gulf of LPG $/MT 852.78
2 Add: Ocean Freight from AG to Indian Ports $/MT 43.39
3 Cost & Freight Price Rs. / Cylinder 689.03
4 Import Charges (Insurance/Ocean Loss/ LC Charge/Port Dues) Rs. / Cylinder 5.82
5 Basic Customs Duty Rs. / Cylinder 0.00
6 Import Parity Price (Sum of 3 to 5) Rs. / Cylinder 694.84
7 Refinery Transfer Price (RTP) for Domestic LPG Rs. / Cylinder 694.84
8 Add : Inland Freight, Delivery Charges etc. Rs. / Cylinder 39.46
9 Add : Marketing Cost of OMCs Rs. / Cylinder 12.38
10 Add : Marketing Margin of OMCs Rs. / Cylinder 6.68
11 Add : Bottling Charges (Filling and Cylinder Cost) Rs. / Cylinder 38.68
12 Total Desired Price (Sum of 7 to 11)
Before Excise Duty, VAT and Distributor Commission
Rs. / Cylinder 792.04
13 Less : Subsidy by Central Government Rs. / Cylinder 22.58
14 Less: Under-recovery to Oil Marketing Companies Rs. / Cylinder 396.03
15 Price Charged to Distributor (12-13-14)
Excluding Excise Duty & VAT
Rs. / Cylinder 373.43
16 Add : Excise Duty (Including Education Cess) Rs. / Cylinder 0.00
17 Add : Distributor Commission Rs. / Cylinder 25.83
18 Add : VAT Rs. / Cylinder 0.00
19 Retail Selling Price (Sum of 15 to 18) Rs. / Cylinder 399.26
20 Retail Selling Price at Delhi (Rounded Off) Rs. / Cylinder 399.00
21 Under Recovery due to Rounding Down Rs. / Cylinder 0.26

The Report of the Task Force on Direct Transfer of Subsidies on Kerosene, LPG and Fertiliser provides some other interesting figures. There are 12.5 crore LPG connections, consuming 6 cylinders on average. The per-capita consumption of LPG is expected to be roughly 1.5 cylinders annually, leading to a family of four requiring 6 cylinders every year. The total subsidy to consumers in FY09-10 was Rs.16,071 crore, when the per cylinder subsidy was Rs.185. With the current international oil prices being much higher, the total subsidy to the consumers could add up to Rs.50,000 crore this year. The total subsidy to the consumer was Rs.76 per cylinder in FY02-03, and has steadily risen ever since to its current value of Rs.418 per cylinder.

Benefits from transparency

The Report of the Task Force on Direct Transfer of Subsidies on Kerosene, LPG and Fertiliser provides a number of suggestions to address the leakage of subsidies:

  • Setting up a transparency portal
  • Per-capita or per-connection cap on number of subsidized cylinders per year
  • Sale of LPG at market price, with subsidy refunded to the consumer's bank account

The Ministry of Petroleum and Natural Gas, along with the OMCs, have launched transparency portals. Even though there is no restriction on the number of subsidized LPG cylinders that a consumer can order, checks have been put in so that there has to be a gap of at least 21 days between two bookings. The media has been quick to report on the heavy consumption of LPG by politicians and industrialists. The Government expects that the transparency portal will also curb the usage of domestic LPG for commercial purposes. Many are even asking whether LPG should be subsidized at all. Over the next few months, we will learn whether LPG diversion is checked due to transparency portals. However, the launch of other features such as rating of dealers, online complaints, and online booking are certainly going to be beneficial for consumers.

The Right to Information Act, 2005 has provisions that require Government to provide data electronically to citizens. Transparency portals are incredibly powerful accountability tools, and should be the first step towards e-Governance for any Ministry or Department. They take existing databases and make them available online for scrutiny, often without requiring major business process re-engineering. Rather than simply make these portals available for browsing online, care should be put in to produce high quality, anonymized, machine-readable datasets that researchers can use for various purposes. Such datasets can provide interesting insight into the microeconomics of households and also macroeconomic trends, when studied over time.

Monday, April 23, 2012

Developments on implementation of the GST

by Viral Shah and Ajay Shah.

As with many other problems in Indian economic reform, getting to the right destination on the GST requires winning on policy, politics and administration. On the policy side, the basic design of the GST needs to be done right. Pulling this off will require great political skill - a coalition of beneficiaries from the GST will need to champion it in the Indian federal setting. Finally, assuming that the policy and the politics has been done well, it will require the right plumbing. In this blog post, we review progress on this third element.

Done right, the GST ought to replace all existing indirect taxes. This would remove barriers to inter-state commerce, and create an Indian common market. It should treat all goods and all services identically. It should be a single administrative system covering tax payments to both Centre and States thus eliminating the compliance cost that is associated with dealing with multiple tax authorities. It should be globalisation-compatible: goods and services sold to non-residents would be fully refunded the entire burden of indirect taxation that has been incurred at all stages of production. India would then follow the principle of not taxing non-residents. This would be fair for domestic producers who face foreign competition, and ensure competitiveness of domestic producers selling abroad.

These are powerful and important economic concepts. However, their translation into reality is critically about execution. In the case of the GST, as with the New Pension System, the problem of execution is substantially (though not entirely) a question about building large IT systems.

While much of the legal and policy framework around GST is still being worked on by the Central Government in consultation with States, some progress has been made on setting up the infrastructure for processing registration, returns, and payments in a standardised manner. A detailed note on the IT infrastructure for GST has been put up by the Ministry of Finance.

In terms of organisation structure, existing success stories include the Tax Information Network (TIN) and the New Pension System, both of which are being managed by NSDL. A more general concept of `National Information Utilities' (NIU) was proposed by the TAGUP Report. This report drew on the success of establishing market infrastructure institutions such as NSE and NSDL, and recommended that NIUs be such non-Government companies, with Central and State Governments as joint shareholders, dispersed shareholding among other institutions, avoiding shareholders that may have a conflict of interest, and avoiding listing on exchanges. In spirit, NIUs must have the efficiency of a private corporations, but be animated by a public purpose.

In the Budget Speech of 2012-13, the Finance Minister announced that a NIU for implementing the GST would be constructed. It would be called GSTN and would be fully operational by August 2012. The first steps towards constructing GSTN have now been taken, with a Cabinet approval for GSTN. The official press release on this says:

The Cabinet has approved a proposal to set up a Special Purpose Vehicle (SPV) namely Goods and Services Tax Network SPV (GSTN SPV) to create enabling environment for smooth introduction of Goods and Services Tax (GST). GSTN SPV will provide IT infrastructure and services to various stakeholders including the Centre and the States. 
The GSTN SPV would be incorporated as a Section 25 (not-for-profit), non-Government, private limited company in which the Government will retain strategic control. It would have an equity capital of Rs. 10 crore, with the Centre and States having equal stakes of 24.5% each. Non Government institutions would hold 51% equity. No single institution would hold more that 10% equity, with the possibility of one private institution holding a maximum of 21% equity. 
GTSN SPV would have a self-sustaining revenue model, based on levy of user charges on tax payers and tax authorities availing its services. While the SPV's services would be critical to actual rollout of GST at a future date, it is also expected to render valuable services to the Centre / State tax administrations prior to the GST implementation.

Sunday, February 12, 2012

An election rally in Amethi, Uttar Pradesh

by Naman Pugalia, Viral Shah and Ajay Shah.

We decided to experience the spectacle of Indian democracy, by going to an election rally in the Uttar Pradesh (UP) assembly elections. Elections in Uttar Pradesh are unbelievably big - in 2007 there were 110,654 polling stations, which gathered up 471 votes each (on average) into electronic voting machines (EVMs), adding up to 52 million votes. (For a comparison, the US presidential election in 2008 had 132 million votes). In UP, the 2012 elections are going to be much bigger than what was seen in 2007.

A microcosm of Uttar Pradesh. 
Deep inside the production function of the political campaign is the rally. From Gandhiji's public meetings to Nehru's memorable speeches, mass gatherings have been an integral element of political communication. Audiences pick up an array of subtle aspects of the voice, body language, and mannerisms of the speaker, through which a great deal is communicated other than the text of what is spoken.

On February 9th, we attended a rally organised by the Congress, where Priyanka Gandhi spoke. The meeting took place in Pari village in the Amethi district. This is a place so obscure that google maps does not know it.

A small rally, with a special platform for the media to mount cameras.
Perhaps 500 to 1000 people from Pari and neighbouring villages had assembled to hear Priyanka, who urged them to vote for their (incumbent) MLA from the Salon constituency, Balak Pasi. A strong media contingent was there, with both local and national media being represented. In 2007, this constituency was reserved for a scheduled caste (SC) candidate. Balak Pasi had won handily, with 45,078 votes compared with his rivals Asha Kishor of SP who got 31,969 votes and Dalbahadur of the BSP who got 26,588 votes.

As with a music concert, you have to warm up the crowd before the main act. Two local politicians kept the audience entertained before Priyanka arrived:

Local politicians, warming up the crowd before the main show.
Priyanka was fluent in Hindi, and was quite at ease. The well-rehearsed speech pitched Congress. There were two main planks of the speech: that UP had done badly with 22 years of non-Congress alternatives, and that Congress was doing good things for the people at the Centre but the failures of the BSP government had prevented these benefits from reaching them. It was a short, well-written speech: not a lengthy tiresome rant or ramble. The video you see ahead is from a different location, but the speech is essentially the same as what we heard.




Needs at least Flash Player Ver.9 to Play Download Here



The audience was not very engaged. We were told that audiences have not been too engaged with any political speaker after Gandhiji. After the speech, she stepped off the podium and walked towards the gathering to shake hands with a few awestruck attendees. In a few minutes, her convoy of SUVs, complete with SPG staff, sped off.

Everyone was comfortable.


UP has a bad reputation for law and order, but we were struck by the civility of the entire process. Everyone was comfortable. It was a day out for the family, and aided by the beautiful winter morning, people were smiling. There was no hint of violence, intimidation or fear.

If mass politics evolves in this direction, a lot more people would be comfortable getting involved in it. It was one more reminder of the importance of law and order as the foundation of democracy and civilisation: when politics involves violence, only the goons will participate in it. As Fareed Zakaria says, the courts are even more important than the elections.

In the olden days, politicians did a small number of speeches for big audiences. Meetings were announced many days in advance, and a large crowd took the trouble of going to the meeting venue. The world has changed. Small rallies are a much more intimate experience. They are also a security nightmare, owing to the very intimacy of the experience. At the same time, we felt that the security arrangements were non-intrusive.

With these small rallies, the politician comes to the audience. The speaker probably does 10 rallies a day, and thus touches perhaps 5000 people a day. Another feature of the new production function of political campaigns is television: A politician may do a speech for a small rally, but the omnipresent television cameras beam it all across Uttar Pradesh, to a vastly larger audience.

The rally was pretty nice, but at the same time, political parties in India are not professional organisations. There were numerous small issues where a relentless process engineering perspective could have added value. Politics in India is a bit like the way big companies used to be twenty years ago (and small companies are today). Big Indian firms have figured out how to graduate from mom and pop operations to professional organisations, how to write process manuals, setup instrumentation of the process in motion, and how to bring in high end brainpower to improve processes. That transformation of Indian politics -- from cottage industry to professional organisations -- has mostly not begun.

After the speech, the crowd peacefully dispersed.

Monday, January 23, 2012

A fueling fable: Consumer protection issues with payments

by Naman Pugalia and Viral Shah.

On 22nd December 2011, we purchased petrol worth Rs.100 from an Indian Oil fueling station in Bombay using an ICICI Bank debit card. The receipt suggested that we could have saved a fuel surcharge of 2.5% had we used an Indian Oil Citibank credit card. Upon seeing this message, we asked the cashier at the petrol pump if we would be charged 2.5% over and above the Rs.100 that we paid for the fuel. The cashier assured us that only Rs.100 would be debited from the account linked to the card. The chargeslip and the receipt were:

The chargeslip

The receipt
A couple of days later, we viewed the account statement online and found that the relevant transaction had been recorded. A full week later, we observed that an additional charge of Rs.11.03 had been
debited from the account for the same vendor. Not only was the entry unusual, the charge did not match the 2.5% figure which was mentioned on the transaction receipt:

The statement

We wrote to the bank asking them to explain the transaction. The bank explained that for fuel purchased at non-HPCL petrol pumps, a surcharge of 2.5% of the fuel cost or Rs.10 (whichever is higher) would be levied. A service tax would be levied additionally.

There is a consumer protection issue here. After the account had been debited, and up until we sought a clarification from the bank, we were not made aware of the surcharge. The chargeslip gave a false impression of the amount being paid.

Upon delving further, we find various websites where people have complained about this surcharge being confusing. Further investigation revealed an interesting combination of participants:
  1. The surcharge on fuel is mentioned in the fine print in Terms and Conditions of a debit card.
  2. The bank that deploys the POS machine (acquiring bank being Citibank in our example), at the end of day, surcharges the higher of 2.5% or Rs.10 and sends it to the customer's bank (issuer bank being ICICI Bank in this case).
  3. The issuing bank then creates a separate debit in the customer's account for the surcharge
  4. The acquiring bank shares much of this surcharge back to Oil Marketing Company (Indian Oil in this example).
  5. Contrast this with typical debit card processing fees in India around 1.5%. In most cases, merchants will inform a customer before surcharging, and the value on the chargeslip is what the
    customer pays.
  6. Many banks apply these surcharges weeks or months after the transaction actually occurs, which helps ensure that most customers do not understand what is going on.
When paying for fuel in India with a debit card, the customer pays the surcharge by being misled, the Oil Marketing Company makes higher profits, the charge is administered in a non-transparent way, and is posted late when the customer may not even recall the transaction. Thus, Government owned companies and banks have created a perverse incentive, whereby customers prefer to use cash rather than pay electronically.

Author: Viral Shah


On this blog:




Tuesday, January 18, 2011

Excitement in electronic payments

by Viral Shah.

Frictionless payments are an important contributor to India's growth story. Cash is costly for everyone in the system: for the central banker, banks, businesses, and individuals. Yet, we are far away from a mass adoption of electronic payments.

For electronic payments to take off, it is essential to have a bridge between the world of physical and electronic money, allowing seamless conversion between the two. Until recently, this could only be done at banks or ATMs. Various developments over the last year have greatly expanded the number of entities that can now offer such services:
  1. July 2009, Cash withdrawal at point-of-sale guidelines: As a step towards enhancing customer convenience, RBI allowed cash withdrawals at point-of-sale terminals with debit cards.
  2. April 2010, Report of the Inter Ministerial Group on the Delivery of Basic Financial Services using mobile phones: This group chaired by the Secretary, DIT and included, among others, representatives from Department of Financial Services, Department of Posts, Ministry of Rural Development, Planning Commission, UID Authority of India, TRAI, RBI, Department of Telecom and the Home Ministry. It suggested a nationwide payments architecture that consisted of a simple centralized account hosting platform as a national infrastructure, full interoperability among payments providers, standards based biometric point of sale terminals, and standards based mobile payments.
  3. April 2010, From Exclusion to Inclusion with micropayments: The UIDAI published a strategy document on micropayments, which provides a detailed framework for biometrically authenticated transactions, as recommended by the IMG. The National Payments Corporation of India has developed an interoperability switch for Aadhaar and mobile based transactions as recommended in the IMG report. Both, Visa and Mastercard have also adopted this framework.
  4. June 2010, Harnessing the India Post Network for financial inclusion: This report was jointly commissioned and produced by Department of Posts, Department of Financial Services, Department of Economic Affairs and Invest India Economic Foundation. It recommended a framework similar to that recommended by the IMG; that of a low-cost account hosting platform and cultivating a payments ecosystem by allowing partners to access its physical and electronic payments network for a fee.
  5. September 2010, Financial Inclusion by Extension of Banking Services - Use of Business Correspondents (BCs): BC guidelines have existed for a while, but RBI recently relaxed the rules for the entities that can act as BCs. These new guidelines allow for-profit companies (except NBFCs) also to become BCs. This appears like a small change but it has had far-reaching consequences.
Over these two years, we have a vivid sense about electronic payments making the grade, from a vaguely posed idea for the deep future to something that is tangibly around the corner. Each of these elements appears to be small in isolation, but the link from public policy developments to outcomes is like a butterfly effect.

These developments have important ramifications for mass adoption of retail electronic payments and financial inclusion. Banks, recognizing the importance of new regulation, have partnered with telcos with retail networks. This allows for the telco to leverage its network of talk-time retailers as BC sub-agents, and to also offer mobile payments between bank accounts. A flurry of announcements has happened recently: SBI-Airtel, ICICI-Vodafone, Axis-Idea, and more will certainly follow.

As much as these announcements are exciting, they raise some worrisome questions. The biggest question is interoperability. If the country is going to have a network of a million BC outlets, shouldn't a customer of any bank be able to use any outlet, much like they can use any ATM? Shouldn't a customer of one bank-telco partnership be able to send money to a customer of another bank-telco partnership effortlessly? Network effects are essential for mass adoption of electronic payments. After all, regulations today allow a person with a debit card to withdraw cash over an interoperable network of point-of-sale terminals.

Banks and telcos are unlikely to want interoperability; to justify investments, gain customers, and want them to stick. Perhaps the regulator ought to take a firm stand on the issue. A good balance may be to ensure that the products are designed to be interoperable with some uniformity of customer experience, but offer an interoperability holiday for the first two years.

Monday, November 19, 2007

Interpreting the 4th fastest supercomputer in the world

In response to my blog entry (and comments about it posted on this blog) about `Eka' making rank 4 in the Top500 list of the world's supercomputers, I got this email from my friend Viral Shah who knows a bit about the field:

Ajay, it seems that the ranking of Eka at number 4 on the Top500 list has resulted in quite a lot of excitement. Hats off to the folks at CRD Labs for achieving the feat of assembling such a large computer in a short amount of time. As some of your readers noted, Eka is a cluster. It is roughly 2,000 nodes, consisting of roughly 15,000 processors and connected by Infiniband. Some readers noted that the benchmark is not representative of real scientific applications. 
Firstly, making a small cluster is quite easy. However, constructing such a large cluster, and operating it is no easy task. It requires some serious skills to administer it, tune the hardware and software for performance, and run scientific applications on it. Second, the Top500 is an interesting benchmark. Sure, it is not representative of a realistic workload, but over the years, the bar has been set quite high. If a general purpose computer does not achieve a good LINPACK score (the top500 benchmark), it is safe to conclude that something is terribly wrong. I am of course excluding special purpose computers that are built to solve specific problems, rather than get a high LINPACK score. 
That said, one needs to think this through clearly. Why was Eka built? To simply show that we can do it, and place a computer in the Top 10 supercomputers? To run specific scientific applications? I am guessing that the answer is "a bit of both". Almost always, it is safe to conclude that the full supercomputer is never used to solve one problem. What are the largest problems that will be run on Eka? What percentage of peak will they achieve? Would it have been a better idea to buy an "off the shelf system" such as the Cray XT4 or the SGI Altix and focus on programmer productivity, instead of getting a high LINPACK score? 
Computers such as Eka achieve extremely high and unrealistic flop rates on the LINPACK benchmark.Typically, they can achieve over 70% of the peak flop rate (Number of floating point operations per second). However, real applications often run at below 5% of the peak flop rate. Let's examine some other possibilities. 
Note that the software industry has been one of India's strong points. It is becoming increasingly clear, that, software is the key. For example, Apple's success with the iphone and ipod have as much to do with well designed software, as with the hardware. If you ask me, the big event at Supercomputing'07 was not that Eka placed at No. 4 on the Top500 list. For me, the most exciting event was one that you will not hear about in media - it has to do with the other part of the HPC Challenge, often called the beauty contest. Instead of asking "which computer can run LINPACK the fastest", it asks, "which programming language implements the benchmarks elegantly". 
The winners of the class II challenge this year were IBM's X10, and Interactive Supercomputing's Python Star-P. For me, the most surprising, and the coolest event was the revelation that some of the compiler work for X10 was done at IBM's research labs in India. This is cutting edge compiler technology, and the fact that part of the team was based in India is a strong statement about HPC innovation in India. 
Not to belittle the effort that went into Eka, but we should be asking the hard questions. I think it is fantastic that we can afford to build a $30 million supercomputer. But how are we going to program it? What applications will run on these large computers? Will we be able to address some important problems such as better meteorological forecasts for our farmers, or better groundwater modeling to solve our long term water problems, or allow our companies to gain that extra edge in the international arena? Are we better off buying computers from those who know how best to make them, and focusing our skill sets on what we are good at - developing software? 
Our universities are indeed not be up to this challenge yet, but, perhaps, we don't need to wait till they catch up. With all the resources available online, a hungry young person can learn this game - the material is all online, after all, like these classes at MIT, at UC Berkeley and at UC Santa Barbara.

Thursday, August 23, 2007