Search interesting materials

Wednesday, May 08, 2024

The usefulness of the CMIE household survey data for electricity research in India

by Susan Das, Renuka Sane and Ajay Shah.

Measurement for electricity research

The problems of the electricity system are of particular importance in India given that the existing institutional arrangements work relatively poorly. While this has been a concern for decades given the importance of electricity in economic growth, it has achieved a fresh prioritisation due to the need for a clean energy transition. The problems of electricity sector have become the critical bottleneck for the decarbonisation of the economy (Jaitly and Shah, 2021).

De jure government subsidies, and de facto theft, take place in this field on a significant scale. While researchers are able to readily observe de juresubsidies, theft is hard to observe. Domestic consumers and agriculturists shape the political economy of sub-national electricity. Policy makers make decisions about the de jure and de facto policy frameworks with an eye on how these pressure groups will react. The mechanisms through which transfers take place are often subtle. As an example, Mahadevan (2023) obtained micro data for one state in India, and found that electricity bills were manipulated to charge less to households in regions that had voted for the ruling party.

Understanding the field, and analysing possible policy pathways, will benefit from quantitative political economy research grounded in household survey data. Once household level electricity expenditures are observed alongside an array of household characteristics, it becomes possible to analyse the incidence of present or alternative subsidy mechanisms. Researchers could try to understand the mechanisms through which tariffs and subsidies shape expenditures, the role of appliance ownership, the problem of theft of electricity, the elasticity of residential electricity demand to price changes and other policy actions, and the impact of electricity consumption upon the household.

These possibilities have been opened up in India through the CMIE CPHS database, a panel dataset where 240,000 households are observed thrice a year. As with many measurement strategies such as crime victimisation surveys, there is value in looking beyond aggregative measurement, and directly asking households about their experiences. The CMIE CPHS dataset opens up diverse array of research possibilities in the field of electricity research. Some research papers have emerged which harness this. Martínez Arranz et al. (2021) have used the CPHS to study the expansion of access to 24 hours of electricity supply between the 2014-2019 period across 24 states and UTs. Kulkarni, Sahasrabudhe and Chunekar (2022) have used the CPHS to analyse the appliance ownership trends in India which potentially can be used to forecast residential electricity demand.

At the same time, there are concerns about the veracity of this data. The world over, it is difficult to supervise field investigators and obtain cooperation from respondents, particularly from high income households. While the CMIE CPHS database represents a remarkable new phase in economic measurement in India, there has been a debate about the difficulties of this data.

In this article, we perform sanity checks of this data, across natural experiments of tariff changes in Tamil Nadu. This helps assess the extent to which the data can be a sound foundation for researchers in this field. It also contributes to the larger literature on the problems of household survey data, on economic measurement in India, and on pathways to measurement in emerging markets that do not rely on the official statistical system.

The CMIE CPHS database

The Consumer Pyramids Household Survey (CPHS) is a longitudinal household survey conducted by the Centre for Monitoring Indian Economy (CMIE) from 2014. A panel of over 240,000 Indian households across 27 states and 514 districts is measured thrice a year. The sample is nationally representative and selected through a multi-stage stratified design. The survey captures many features of the household including labour supply, investment, borrowing, consumption, income, and demographics. It represents the first large scale longitudinal dataset of this nature in India.

A rich literature has emerged based on the CMIE CPHS data, ranging from questions of household portfolio choice, labour markets to health and mortality impacts of diseases. In May 2024, a search for the string CMIE CPHS on Google Scholar showed 310 papers. Pandey, Patnaik and Sane (2019) study the evolution of financial savings with the changes in the design of tax breaks, while Gopalakrishnan, Ritadhi and Tomar (2019) study the effects of adjustment costs in real estate affecting portfolio choice. Patnaik, Sane and Shah (2019) study the effects of a flood in Chennai on the income and consumption pattern of affected households and the heterogeneity in impact by financial constraints and income levels. It has been used to understand gender gaps in the labour market outcomes as a result of the impact of COVID-19 (Deshpande, 2022; Abraham, Basole and Kesar, 2022, excess mortality and prevalence of COVID-19 (Malani and Ramachandran, 2022; Mohanan et al., 2021), and the impact of location and income on the health of individuals (Patnaik, Sane, Shah and Subramanian, 2023).

Household survey measurement is facing challenges the world over. Respondents who are immersed in modern distractions, or are busy, are prone to refuse to answer questions, which gives non-random non-response. Supervision and control of field investigators is difficult in the Indian locale, which makes field research particularly difficult. Publication orientation creates little incentive for researchers to expend resources on correctness of data (or software).

In the specific context of the CMIE CPHS database, there have been concerns about a potential under-counting of poor households. For example, Somanchi (2021) argues that CPHS under-represents women, young children and poor households and, over-represents well-educated households. Similarly, Pais and Rawal (2021) raise questions regarding the lack of a precise sampling frame and the likely exclusion of poor and mobile households. CMIE has responded to these concerns (Vyas, 2021). These debates raise concerns about potential applications of this database.

The purpose of this article is to examine the usefulness of one measure from this database รข€“ expenditure on electricity -- where veracity of measurement is essential for applications in the field of electricity. Towards this objective, we harness a group of natural experiments, and examine the gross regularities seen in the data.

Natural experiments with the price of electricity in Tamil Nadu

We examine four events where the price of electricity changed in the state of Tamil Nadu. Two of these were tariff changes by the regulator (the Tamil Nadu Electricity Regulatory Commission (TNERC)), and the other two were changes in the on-budget subsidy by the state government.

Prior to the first event, the tariff regime prevalent in the state was based on the tariff order passed in 2012 by the TNERC. The effective prices for electricity varied between INR 3.00 - INR 3.25 per kWh for households whose consumption was below 100 kWh of electricity. Further, the prices gradually increased to the range of INR 3.50 - INR 4.60 per per kWh for households whose consumption was up to 250 kWh in a month. For consumption beyond 250 kWh, the price rose to INR 6.60 per kWh. In this backdrop, a sequence of four natural experiments took place:

Subsidy increase in 2016
During the state legislative assembly election campaign in Tamil Nadu in 2016, the then Chief Minister Jayalalithaa announced 50 kWh of free electricity a month to over 19 million residential consumers as one of her poll promises (Srikanth, 2016). She won the election, and the promise was kept in June 2016.
Tariff decrease in 2017
On 11 August 2017, the TNERC issued a tariff order which decreased tariffs for domestic consumers (TNERC, 2017).
Subsidy increase in 2018
In November 2018, the Government of Tamil Nadu (GoTN) introduced additional subsidies. These varied from INR 0.5 - INR 1.00 per kWh for consumption below 250 kWh in a month (TNERC, 2018). This is a small event where very little changed.
Tariff increase along with subsidy in 2022
The TNERC approved a tariff increase for domestic consumers across all consumption slabs in September 2022 (TNERC, 2022). Alongside this, the GoTN announced an additional 50% subsidy on electricity consumption between 50-100 kWh (for consumers having consumption below 250 kWh) in addition to the 50 kWh of free electricity (Guruvanmikanathan, 2022).

These changes are summarised in Table 1, which depicts the full price schedule of electricity vis-a-vis electricity consumption (kWh) for residential consumers in Tamil Nadu under different price regimes. We show the effective price factoring in the subsidies by the GoTN for a household as per its electricity consumption after each event of tariff or subsidy change announced by the state.

Table 1: Price schedule (monthly) for electricity for residential consumers
Slab Electricity
(kWh) (INR)

0-50 kWh 0-50 3.00 0.00 0.00 0.00 0.00
0-100 kWh 0-50 3.25 0.00 0.00 0.00 0.00
51-100 3.25 3.25 2.50 1.50 2.25
0-250 kWh 0-50 3.50 0.00 0.00 0.00 0.00
51-100 3.50 3.50 2.50 2.00 2.25
101-200 4.60 4.60 3.00 3.00 4.50
201-250 4.60 4.60 3.00 3.00 6.00
Above 250 kWh 0-50 3.50 0.00 0.00 0.00 0.00
51-100 3.50 3.50 3.50 3.50 4.50
101-200 4.60 4.60 4.60 4.60 4.50
201-250 4.60 4.60 4.60 4.60 6.00
251-300 6.60 6.60 6.60 6.60 8.00
301-400 6.60 6.60 6.60 6.60 9.00
401-500 6.60 6.60 6.60 6.60 10.00
Above 500 6.60 6.60 6.60 6.60 11.00

What might we expect to see, in expenditure data at a per-household level, across these four events? For the purpose of this article, we bring a simplistic prior: that the short-term price elasticity of energy demand is low, so the household expenditure will just go from $qp1$ to about $qp2$, with adjustments in the consumption basket based on a commensurate income effect only.

We recognise, of course, that the reality is more complex than this. Many other elements of adjustment are in fray, both in the short term and in the long run. With a lag, higher electricity prices change the incentives for energy efficient equipment and rooftop solar generation. Going beyond such conventional economic responses, the incentives for theft change: when electricity is more expensive there is a greater incentive to steal. The changes in the price schedule seen in Table 1 could induce complex effects combining these factors. Sophisticated research projects, in the field of household electricity consumption, are required that seek to tease out the short-term and long-term effects.

This article is not in that field: it is an examination of the concerns around measurement of household level electricity expenditure as seen in the CMIE CPHS database. We examine the gross regularities about how household electricity expenditure changed across these natural experiments. Our ability to do this across four natural experiments, adds up to an opportunity to examine the usefulness of the CMIE household survey data for the purpose of electricity research. If the expenditure change generally goes with the price change, for relevant households, we will conclude the dataset has value for researchers in this field.

Data description

We focus on the 10,000 households observed in Tamil Nadu. Within this, we obtain three facts:

  • Total expenditure (INR) - This is sum of all the expenses incurred by a household on the consumption of goods and services during a month. It includes expenses on rent, food, clothes, utility bills, entertainment, fuel etc. This feature is not directly captured during CPHS survey process, rather it is derived as the sum of all the monthly individual expense heads captured during the survey of a household. We derive this feature from the monthly expenses data.
  • Electricity expenditure (INR) - This is the expenditure incurred by a household on electricity during a month. We obtain this feature from the monthly expenses data.
  • Weights - We use household weights for the state level estimates provided by CPHS for making estimates at the monthly frequency level. We further use the adjustment factor for household non-response to adjust sampling weights to balance for non-responses during the survey. It is the ratio of the total number of sample households in the stratum and the non-surveyed regions to the accepted sample from these.

An important strand of the field of economic measurement is assessing how a variety of methods for measurement fared through the pandemic. In this article, however, the third event was in November 2018 (which was well before the pandemic) and the fourth event was in September 2022 (which represents post-pandemic conditions). This article, thus, does not offer insights on data quality in the pandemic.

What might we expect?

  1. The event: In June 2016, the state government announced 50 kWh of free electricity to domestic consumers, an implementation of a promise that was made in the election campaign.

    Prediction: We expect to see lower expenditure in the aggregate.

  2. The event: The TNERC approved a tariff cut in its 2017 tariff order for domestic consumers with consumption below 250 kWh of electricity.

    Prediction: We expect to see lower expenditure in the aggregate due to tariff cut in addition to existing free 50 kWh of electricity.

  3. The event: In November 2018, the state government announced a further subsidy on electricity sold to domestic consumers. Varying subsidies were introduced ranging from INR 0.50 - INR 1.50 per kWh for different slabs of consumption of up to 250 kWh of electricity consumption. This was in addition to the already existing subsidy of 50 kWh of free electricity across all slabs of consumption. This was the smallest of the four changes in the price schedule that are examined in this article.

    Prediction: We expect to see lower expenditure in the aggregate, with a small effect.

  4. The event: The TNERC approved a tariff increase in its 2022 tariff order for domestic consumers. Alongside this increase in tariffs, the subsidy given by the state government also increased. GoTN announced a further 50% subsidy on the next 50 kWh of electricity consumption (for households below 250 kWh of consumption) in addition to the prior 50 kWh of free electricity.

    Prediction: We expect to see higher expenditure in the aggregate in smaller magnitude due to combination of tariff hike by TNERC and subsidy announcement by GoTN.


We now examine the household survey data and assess the extent to which the predictions, based on a simple prior, are borne out by the statistical evidence. For each price change, we examine household expenditure in the six months before and after the event. We view this in the aggregate. The results are shown in Table 2.

Table 2: Aggregate results across all events for monthly electricity expenditure
Mean Median Mean Median

12/2015 — 5/2016
7/2016 — 12/2016
Electricity expenditure (INR) 313 196 192 150
Share in total expenditure (%) 2.88 2.25 2.05 1.76

2/2017 — 7/2017
9/2017 — 2/2018
Electricity expenditure (INR) 191 163 173 148
Share in total expenditure (%) 1.94 1.86 1.46 1.36

5/2018 — 10/2018
12/2018 — 5/2019
Electricity expenditure (INR) 179 151 174 142
Share in total expenditure (%) 1.59 1.47 1.51 1.37

3/2022 — 8/2022
10/2022 — 3/2022
Electricity expenditure (INR) 142 116 154 137
Share in total expenditure (%) 1.15 1.02 1.16 1.06

  • Event 1, subsidy in 2016:

    The mean electricity expenditure went down from INR 313 to INR 192. The median went down from INR 196 to INR 150. The gap between these two location estimators may potentially reflect extreme values that are influencing the mean.

  • Event 2, tariff cut in 2017:

    The mean and median expenditures went down, as predicted.

  • Event 3, subsidy in 2018:

    The mean and median expenditures went down, as predicted. The change was small, as predicted.

  • Event 4, tariff increase along with subsidy in 2022:

    The mean and median expenditures rose by INR 12 and INR 21. These appear to be unusually small effects given the size of the price increase.


The gross regularities of the CPHS data in Tamil Nadu appear to change in sane ways, across the four tariff or subsidy change events. These findings contribute to the field of economic measurement in India. They encourage applications of this data into the field of electricity. Much more future research is required, of course, in examining household behaviour when faced with policy changes, in the short term and in the long term.


Abraham, Basole and Kesar. 2022. Down and out? The gendered impact of the Covid-19 pandemic on India's labour market. Economia Politica 39.1.

Deshpande. 2022. The Covid-19 pandemic and gendered division of paid work, domestic chores and leisure: evidence from India's first wave. Economia Politica 39.1.

Gopalakrishnan, Ritadhi and Tomar. 2019. Household Finance in Developing Countries: Evidence from India. Rochester, NY.

Guruvanmikanathan. 2022. Tamil Nadu: Scheme to surrender power subsidy unlikely anytime soon. The New Indian Express.

Jaitly and Shah. 2021. The lowest hanging fruit on the coconut tree: India's climate transition through the price system in the power sector. XKDR Forum.

Kulkarni, Sahasrabudhe and Chunekar. 2022. Appliance ownership trends in India: As per Consumer Pyramids Household Survey Data.

Mahadevan. 2023. The Price of Power: Costs of Political Corruption in Indian Electricity. University of California, Irvine.

Malani and Ramachandran. 2022. Using household rosters from survey data to estimate all-cause excess death rates during the COVID pandemic in India. Journal of Development Economics.

Mart́inez Arranz et al. 2021. The uneven expansion of electricity supply in India: The logics of clientelism, incrementalism and maximin. Energy Research and Social Science.

Mohanan et al. 2021. Prevalence of SARS-CoV-2 in Karnataka, India.

Pais and Rawal. 2021. CMIE's consumer pyramids household surveys: An assessment.

Pandey, Patnaik and Sane. 2019. Impact of Tax Breaks on Household Financial Saving in India. National Council of Applied Economic Research.

Patnaik, Sane and Shah. 2019. Chennai 2015: A novel approach to measuring the impact of a natural disaster. National Institute of Public Finance and Policy.

Patnaik, Sane Shah and Subramanian. 2023. Distribution of self-reported health in India: The role of income and geography. PLOS ONE.

Somanchi. 2021. "Missing the poor, big time: A critical assessment of the consumer pyramids household survey".

Srikanth. 23rd May 2016. Free power to benefit 1.90 cr consumers. The Hindu.

TNERC. 2017. TNERC Tariff Order No.01 of 2017 in T.P. No.1 of 2017.

TNERC. 2018. TNERC Tariff Order No.07 of 2018 - Provision of Tariff subsidy for FY 2018-19 by the Government of Tamil Nadu.

TNERC. 2022. TNERC Tariff Order No.07 of 2022 in T.P. No.1 of 2022.

Vyas. 2021. View: There are practical limitations in CMIE's CPHS sampling, but no bias. The Economic Times, 23 June 2021.

Susan Das and Renuka Sane are researchers at TrustBridge. Ajay Shah is a co-founders of XKDR Forum.

Monday, May 06, 2024

Concerns about recent developments on SEBI's regulation-making on market rumours and insider trading

by Bhavin Patel and Renuka Sane.

Insider trading is one of the areas of financial regulation where the order of complexity required of state capability is relatively high, and the gains to society from successful implementation are relatively low. The Indian environment on SEBI enforcement against insider trading has accumulated many difficulties. In recent months, we have seen the next step forward in SEBI's journey, with a novel legal idea around rumours swirling in the market. In this article, we summarise the recent developments, and show two substantive problems with the direction taken by SEBI. These substantive problems are ultimately grounded in failures of process. There are signs that the process adopted in recent months for these developments on rumours, has more deficiencies than usual.

Recent developments on how SEBI thinks about rumours

Over the past year, SEBI has been concerned about the impact of market rumours on security prices. It proposed that certain listed entities be required to verify "market rumours" related to their firm. Amendments to this effect (the June 2023 Amendments) were made in the SEBI (Listing Obligations and Disclosure Requirements) Regulations, 2015 (the LODR Regulations), though the amendments are yet to be enforced. On December 28, 2023, it published a Consultation Paper on Amendments to SEBI Regulations with Respect to Verification of Market Rumour (the Current Consultation Paper) suggesting that listed entities should verify market rumours only if they are material. 'Materiality' meant that such rumours should lead to price movement in the security. It also proposed that if a listed entity classifies certain information as Unpublished Price Sensitive Information (UPSI) but does not verify a market rumour related to such information, it should continue to be treated as UPSI under the SEBI (Prohibition of Insider Trading) Regulations, 2015, (the PIT Regulations). In its Board Meeting held on March 15, 2024, SEBI approved this proposal, confirming that if a rumour related to UPSI is not verified, it will continue to be treated as UPSI. In other words, if a speculative story appears in the public domain, pertaining to UPSI about a firm, it will still be treated as UPSI until the listed entity verifies the story.

These steps contain difficulties that are generic to the working of SEBI. Firms are already mandated to release certain information (through listing obligations), and not release some (through prohibition on insider trading). SEBI's proposals on verification of market rumours have not yet explained the market failure proposed to be solved and how the selected intervention is the least costly way of doing so. In this particular case, the legal effects of the new law can be particularly damaging. The decision in the March 15 Board meeting, to continue to treat Generally Available Information (verified or not) as Unpublished Price Sensitive Information (till the company verifies it), goes against the grain of how price discovery works in the public market. A greater process discipline will help improve the thinking, and the democratic legitimacy, of insider trading regulation.

Substantive problem #1: The impact on the OTC market

Consider the following illustration:

  • Assume a listed entity classifies "defaulting on a large supplier contract" as UPSI.

  • Since the information is classified as UPSI, insiders would be prohibited from trading based on such information under the PIT Regulations. Note that the definition of insider under R. 2(1)(g) of the PIT Regulations includes persons with access to UPSI. This implies that if a retail investor, with no connection to the firm, has access to this information, the retail investor is also classified as an insider. This speaks to the over-inclusiveness of the term 'insider' under prevailing securities laws.

  • Assume that a newspaper carries a story about a "rumour about the default". Since this is now reported in the media, it will be accessible to the general public. The listed entity chooses not to verify the information since it does not trigger the materiality threshold under R. 30 of the LODR Regulations (for example, it might not cause a price movement or may only cause a price movement lower than the limits specified in the threshold).

  • Under the current proposal, an unverified event or information reported in the media would not be considered 'generally available information' under the PIT Regulations. It would continue to be UPSI. The definition of insider regarding UPSI would extend to the general public. As a result, the PIT Regulations would generalise the prohibition on trading based on such information, even though it has been reported in the media, because the listed entity chose not to verify it. This may effectively hinder the price discovery process, a core function of securities markets.

Further, the explanation to R. 4(1) of the PIT Regulations states that if a person possesses UPSI, their trades would be presumed to have been motivated "by the knowledge and awareness of such information". The proviso to this explanation states that an insider can prove their 'innocence' if: (i) the transaction is an off-market inter-se transfer between insiders who were in possession of the same unpublished price sensitive information without being in breach of R. 3 and both parties had made a conscious and informed trade decision, (ii)the transaction was carried out through the block deal window mechanism between persons who were in possession of the unpublished price sensitive information without being in breach of R. 3 and both parties had made a conscious and informed trade decision.

As a result, under proviso (i) two people who read the same unverified story in the media are consequently insiders and who have made a 'conscious and informed trade decision' can trade in the listed entity's securities if such trade is off-market; and under proviso (ii) a block deal trade is also allowed under similar circumstances.

The effect would be that regular trades on the floor of the exchange based on unverified media reports would be prohibited. However, off-market transactions or block deals conducted based on such information would not count as 'insider trading', creating a problem of arbitrary discrimination. This also drastically reduces the pool of investors able to trade, which may possibly lead to a liquidity collapse.

Substantive problem #2: Contradictions with existing regulations

The proposals contradict a literal interpretation of UPSI as set out in the PIT Regulations since they suggest that information which is neither unpublished (since it has been published in the media) nor price-sensitive (since it does not affect the price of securities and therefore falls outside the scope of the materiality threshold verification requirements under R. 30 of the LODR Regulations) would still be considered UPSI. This would suggest a need to amend the definition of UPSI under the PIT Regulations and, potentially, the provisos to the explanation to R. 4(1) and the definition of insider under R. 2(1)(g). The proposals in the Current Consultation Paper also necessitate changes to determining materiality under R. 30(4) of the LODR Regulations. This is because the proposed definition of materiality is not a part of the June 2023 Amendments. The materiality thresholds proposed impact which market rumours listed entities need to verify.

Difficulties with the process

Assuming that SEBI has considered the potential impact on liquidity and does not regard it as problematic, we are still faced with the problem of how these proposals may be implemented. The current proposal may require changes in other substantive regulations. If they are indeed to be implemented, they should be done through amendments to the PIT and LODR Regulations. Board minutes, Circulars, or Guidelines should not suffice to effectuate substantive changes. Under Section 30 of the Securities and Exchange Board of India Act, 1992 (the SEBI Act), changes to regulations must be laid "as soon as may be after it is made, before each House of Parliament, while it is in session, for a total period of thirty days." This allows for Parliamentary oversight of the regulator's exercise of powers of subordinate legislation and prevents the use of such powers beyond permitted limits. The June 2023 Amendments are scheduled to take effect from June 1, 2024, there is little time left for listed entities to prepare, and such amendments should be published soon. The process under S. 30 of the SEBI Act was followed in the case of the June 2023 Amendments, and the Amendments were laid before Parliament on August 11, 2023. There is no reason this process should be side-stepped now. SEBI should come out with a precise legal instrument, and the amendments to the PIT and LODR Regulations, to implement its proposal.

Ideally, the regulator should, in addition to inviting and analysing public comments, identify the problem or market failure these seek to address, the principles governing the proposals, the outcome the regulator aims to achieve through such changes, and an analysis of their costs and benefits as recommended by the Financial Sector Legislative Reforms Commission (FSLRC) in its Handbook on adoption of governance enhancing and non-legislative elements of the draft Indian Financial Code of December 26, 2013. The Financial Stability and Development Council had also approved the implementation of the FSLRC's recommendations in its meeting in October 2013. If this process had been followed, the failures in substantive thinking described earlier would have been avoided.

The authors are researchers at TrustBridge. We thank Amol Kulkarni and Madhav Goel for useful comments.