The usefulness of the CMIE household survey data for electricity research in India

by Susan Das, Renuka Sane and Ajay Shah.

Measurement for electricity research

The problems of the electricity system are of particular importance in India given that the existing institutional arrangements work relatively poorly. While this has been a concern for decades given the importance of electricity in economic growth, it has achieved a fresh prioritisation due to the need for a clean energy transition. The problems of electricity sector have become the critical bottleneck for the decarbonisation of the economy (Jaitly and Shah, 2021).

De jure government subsidies, and de facto theft, take place in this field on a significant scale. While researchers are able to readily observe de juresubsidies, theft is hard to observe. Domestic consumers and agriculturists shape the political economy of sub-national electricity. Policy makers make decisions about the de jure and de facto policy frameworks with an eye on how these pressure groups will react. The mechanisms through which transfers take place are often subtle. As an example, Mahadevan (2023) obtained micro data for one state in India, and found that electricity bills were manipulated to charge less to households in regions that had voted for the ruling party.

Understanding the field, and analysing possible policy pathways, will benefit from quantitative political economy research grounded in household survey data. Once household level electricity expenditures are observed alongside an array of household characteristics, it becomes possible to analyse the incidence of present or alternative subsidy mechanisms. Researchers could try to understand the mechanisms through which tariffs and subsidies shape expenditures, the role of appliance ownership, the problem of theft of electricity, the elasticity of residential electricity demand to price changes and other policy actions, and the impact of electricity consumption upon the household.

These possibilities have been opened up in India through the CMIE CPHS database, a panel dataset where 240,000 households are observed thrice a year. As with many measurement strategies such as crime victimisation surveys, there is value in looking beyond aggregative measurement, and directly asking households about their experiences. The CMIE CPHS dataset opens up diverse array of research possibilities in the field of electricity research. Some research papers have emerged which harness this. Martínez Arranz et al. (2021) have used the CPHS to study the expansion of access to 24 hours of electricity supply between the 2014-2019 period across 24 states and UTs. Kulkarni, Sahasrabudhe and Chunekar (2022) have used the CPHS to analyse the appliance ownership trends in India which potentially can be used to forecast residential electricity demand.

At the same time, there are concerns about the veracity of this data. The world over, it is difficult to supervise field investigators and obtain cooperation from respondents, particularly from high income households. While the CMIE CPHS database represents a remarkable new phase in economic measurement in India, there has been a debate about the difficulties of this data.

In this article, we perform sanity checks of this data, across natural experiments of tariff changes in Tamil Nadu. This helps assess the extent to which the data can be a sound foundation for researchers in this field. It also contributes to the larger literature on the problems of household survey data, on economic measurement in India, and on pathways to measurement in emerging markets that do not rely on the official statistical system.

The CMIE CPHS database

The Consumer Pyramids Household Survey (CPHS) is a longitudinal household survey conducted by the Centre for Monitoring Indian Economy (CMIE) from 2014. A panel of over 240,000 Indian households across 27 states and 514 districts is measured thrice a year. The sample is nationally representative and selected through a multi-stage stratified design. The survey captures many features of the household including labour supply, investment, borrowing, consumption, income, and demographics. It represents the first large scale longitudinal dataset of this nature in India.

A rich literature has emerged based on the CMIE CPHS data, ranging from questions of household portfolio choice, labour markets to health and mortality impacts of diseases. In May 2024, a search for the string CMIE CPHS on Google Scholar showed 310 papers. Pandey, Patnaik and Sane (2019) study the evolution of financial savings with the changes in the design of tax breaks, while Gopalakrishnan, Ritadhi and Tomar (2019) study the effects of adjustment costs in real estate affecting portfolio choice. Patnaik, Sane and Shah (2019) study the effects of a flood in Chennai on the income and consumption pattern of affected households and the heterogeneity in impact by financial constraints and income levels. It has been used to understand gender gaps in the labour market outcomes as a result of the impact of COVID-19 (Deshpande, 2022; Abraham, Basole and Kesar, 2022, excess mortality and prevalence of COVID-19 (Malani and Ramachandran, 2022; Mohanan et al., 2021), and the impact of location and income on the health of individuals (Patnaik, Sane, Shah and Subramanian, 2023).

Household survey measurement is facing challenges the world over. Respondents who are immersed in modern distractions, or are busy, are prone to refuse to answer questions, which gives non-random non-response. Supervision and control of field investigators is difficult in the Indian locale, which makes field research particularly difficult. Publication orientation creates little incentive for researchers to expend resources on correctness of data (or software).

In the specific context of the CMIE CPHS database, there have been concerns about a potential under-counting of poor households. For example, Somanchi (2021) argues that CPHS under-represents women, young children and poor households and, over-represents well-educated households. Similarly, Pais and Rawal (2021) raise questions regarding the lack of a precise sampling frame and the likely exclusion of poor and mobile households. CMIE has responded to these concerns (Vyas, 2021). These debates raise concerns about potential applications of this database.

The purpose of this article is to examine the usefulness of one measure from this database – expenditure on electricity -- where veracity of measurement is essential for applications in the field of electricity. Towards this objective, we harness a group of natural experiments, and examine the gross regularities seen in the data.

Natural experiments with the price of electricity in Tamil Nadu

We examine four events where the price of electricity changed in the state of Tamil Nadu. Two of these were tariff changes by the regulator (the Tamil Nadu Electricity Regulatory Commission (TNERC)), and the other two were changes in the on-budget subsidy by the state government.

Prior to the first event, the tariff regime prevalent in the state was based on the tariff order passed in 2012 by the TNERC. The effective prices for electricity varied between INR 3.00 - INR 3.25 per kWh for households whose consumption was below 100 kWh of electricity. Further, the prices gradually increased to the range of INR 3.50 - INR 4.60 per per kWh for households whose consumption was up to 250 kWh in a month. For consumption beyond 250 kWh, the price rose to INR 6.60 per kWh. In this backdrop, a sequence of four natural experiments took place:

Subsidy increase in 2016

During the state legislative assembly election campaign in Tamil Nadu in 2016, the then Chief Minister Jayalalithaa announced 50 kWh of free electricity a month to over 19 million residential consumers as one of her poll promises (Srikanth, 2016). She won the election, and the promise was kept in June 2016.

Tariff decrease in 2017

On 11 August 2017, the TNERC issued a tariff order which decreased tariffs for domestic consumers (TNERC, 2017).

Subsidy increase in 2018

In November 2018, the Government of Tamil Nadu (GoTN) introduced additional subsidies. These varied from INR 0.5 - INR 1.00 per kWh for consumption below 250 kWh in a month (TNERC, 2018). This is a small event where very little changed.

Tariff increase along with subsidy in 2022

The TNERC approved a tariff increase for domestic consumers across all consumption slabs in September 2022 (TNERC, 2022). Alongside this, the GoTN announced an additional 50% subsidy on electricity consumption between 50-100 kWh (for consumers having consumption below 250 kWh) in addition to the 50 kWh of free electricity (Guruvanmikanathan, 2022).

These changes are summarised in Table 1, which depicts the full price schedule of electricity vis-a-vis electricity consumption (kWh) for residential consumers in Tamil Nadu under different price regimes. We show the effective price factoring in the subsidies by the GoTN for a household as per its electricity consumption after each event of tariff or subsidy change announced by the state.

Table 1: Price schedule (monthly) for electricity for residential consumers

Slab	Electricity Consumption	Tariff 2012	Subsidy 2016	Tariff 2017	Subsidy 2018	Tariff 2022
	(kWh)	(INR)
0-50 kWh	0-50	3.00	0.00	0.00	0.00	0.00
0-100 kWh	0-50	3.25	0.00	0.00	0.00	0.00
	51-100	3.25	3.25	2.50	1.50	2.25
0-250 kWh	0-50	3.50	0.00	0.00	0.00	0.00
	51-100	3.50	3.50	2.50	2.00	2.25
	101-200	4.60	4.60	3.00	3.00	4.50
	201-250	4.60	4.60	3.00	3.00	6.00
Above 250 kWh	0-50	3.50	0.00	0.00	0.00	0.00
	51-100	3.50	3.50	3.50	3.50	4.50
	101-200	4.60	4.60	4.60	4.60	4.50
	201-250	4.60	4.60	4.60	4.60	6.00
	251-300	6.60	6.60	6.60	6.60	8.00
	301-400	6.60	6.60	6.60	6.60	9.00
	401-500	6.60	6.60	6.60	6.60	10.00
	Above 500	6.60	6.60	6.60	6.60	11.00

What might we expect to see, in expenditure data at a per-household level, across these four events? For the purpose of this article, we bring a simplistic prior: that the short-term price elasticity of energy demand is low, so the household expenditure will just go from qp₁ to about qp₂, with adjustments in the consumption basket based on a commensurate income effect only.

We recognise, of course, that the reality is more complex than this. Many other elements of adjustment are in fray, both in the short term and in the long run. With a lag, higher electricity prices change the incentives for energy efficient equipment and rooftop solar generation. Going beyond such conventional economic responses, the incentives for theft change: when electricity is more expensive there is a greater incentive to steal. The changes in the price schedule seen in Table 1 could induce complex effects combining these factors. Sophisticated research projects, in the field of household electricity consumption, are required that seek to tease out the short-term and long-term effects.

This article is not in that field: it is an examination of the concerns around measurement of household level electricity expenditure as seen in the CMIE CPHS database. We examine the gross regularities about how household electricity expenditure changed across these natural experiments. Our ability to do this across four natural experiments, adds up to an opportunity to examine the usefulness of the CMIE household survey data for the purpose of electricity research. If the expenditure change generally goes with the price change, for relevant households, we will conclude the dataset has value for researchers in this field.

Data description

We focus on the 10,000 households observed in Tamil Nadu. Within this, we obtain three facts:

• Total expenditure (INR) - This is sum of all the expenses incurred by a household on the consumption of goods and services during a month. It includes expenses on rent, food, clothes, utility bills, entertainment, fuel etc. This feature is not directly captured during CPHS survey process, rather it is derived as the sum of all the monthly individual expense heads captured during the survey of a household. We derive this feature from the monthly expenses data.
• Electricity expenditure (INR) - This is the expenditure incurred by a household on electricity during a month. We obtain this feature from the monthly expenses data.
• Weights - We use household weights for the state level estimates provided by CPHS for making estimates at the monthly frequency level. We further use the adjustment factor for household non-response to adjust sampling weights to balance for non-responses during the survey. It is the ratio of the total number of sample households in the stratum and the non-surveyed regions to the accepted sample from these.

An important strand of the field of economic measurement is assessing how a variety of methods for measurement fared through the pandemic. In this article, however, the third event was in November 2018 (which was well before the pandemic) and the fourth event was in September 2022 (which represents post-pandemic conditions). This article, thus, does not offer insights on data quality in the pandemic.

What might we expect?

1. The event: In June 2016, the state government announced 50 kWh of free electricity to domestic consumers, an implementation of a promise that was made in the election campaign.

   Prediction: We expect to see lower expenditure in the aggregate.

2. The event: The TNERC approved a tariff cut in its 2017 tariff order for domestic consumers with consumption below 250 kWh of electricity.

   Prediction: We expect to see lower expenditure in the aggregate due to tariff cut in addition to existing free 50 kWh of electricity.

3. The event: In November 2018, the state government announced a further subsidy on electricity sold to domestic consumers. Varying subsidies were introduced ranging from INR 0.50 - INR 1.50 per kWh for different slabs of consumption of up to 250 kWh of electricity consumption. This was in addition to the already existing subsidy of 50 kWh of free electricity across all slabs of consumption. This was the smallest of the four changes in the price schedule that are examined in this article.

   Prediction: We expect to see lower expenditure in the aggregate, with a small effect.

4. The event: The TNERC approved a tariff increase in its 2022 tariff order for domestic consumers. Alongside this increase in tariffs, the subsidy given by the state government also increased. GoTN announced a further 50% subsidy on the next 50 kWh of electricity consumption (for households below 250 kWh of consumption) in addition to the prior 50 kWh of free electricity.

   Prediction: We expect to see higher expenditure in the aggregate in smaller magnitude due to combination of tariff hike by TNERC and subsidy announcement by GoTN.

Results

We now examine the household survey data and assess the extent to which the predictions, based on a simple prior, are borne out by the statistical evidence. For each price change, we examine household expenditure in the six months before and after the event. We view this in the aggregate. The results are shown in Table 2.

Table 2: Aggregate results across all events for monthly electricity expenditure

	Before		After
	Mean	Median	Mean	Median
	12/2015 — 5/2016		7/2016 — 12/2016
Electricity expenditure (INR)	313	196	192	150
Share in total expenditure (%)	2.88	2.25	2.05	1.76
	2/2017 — 7/2017		9/2017 — 2/2018
Electricity expenditure (INR)	191	163	173	148
Share in total expenditure (%)	1.94	1.86	1.46	1.36
	5/2018 — 10/2018		12/2018 — 5/2019
Electricity expenditure (INR)	179	151	174	142
Share in total expenditure (%)	1.59	1.47	1.51	1.37
	3/2022 — 8/2022		10/2022 — 3/2022
Electricity expenditure (INR)	142	116	154	137
Share in total expenditure (%)	1.15	1.02	1.16	1.06

• Event 1, subsidy in 2016:

  The mean electricity expenditure went down from INR 313 to INR 192. The median went down from INR 196 to INR 150. The gap between these two location estimators may potentially reflect extreme values that are influencing the mean.

• Event 2, tariff cut in 2017:

  The mean and median expenditures went down, as predicted.

• Event 3, subsidy in 2018:

  The mean and median expenditures went down, as predicted. The change was small, as predicted.

• Event 4, tariff increase along with subsidy in 2022:

  The mean and median expenditures rose by INR 12 and INR 21. These appear to be unusually small effects given the size of the price increase.

Conclusion

The gross regularities of the CPHS data in Tamil Nadu appear to change in sane ways, across the four tariff or subsidy change events. These findings contribute to the field of economic measurement in India. They encourage applications of this data into the field of electricity. Much more future research is required, of course, in examining household behaviour when faced with policy changes, in the short term and in the long term.

References

Abraham, Basole and Kesar. 2022. Down and out? The gendered impact of the Covid-19 pandemic on India's labour market. Economia Politica 39.1.

Deshpande. 2022. The Covid-19 pandemic and gendered division of paid work, domestic chores and leisure: evidence from India's first wave. Economia Politica 39.1.

Gopalakrishnan, Ritadhi and Tomar. 2019. Household Finance in Developing Countries: Evidence from India. Rochester, NY.

Guruvanmikanathan. 2022. Tamil Nadu: Scheme to surrender power subsidy unlikely anytime soon. The New Indian Express.

Jaitly and Shah. 2021. The lowest hanging fruit on the coconut tree: India's climate transition through the price system in the power sector. XKDR Forum.

Kulkarni, Sahasrabudhe and Chunekar. 2022. Appliance ownership trends in India: As per Consumer Pyramids Household Survey Data.

Mahadevan. 2023. The Price of Power: Costs of Political Corruption in Indian Electricity. University of California, Irvine.

Malani and Ramachandran. 2022. Using household rosters from survey data to estimate all-cause excess death rates during the COVID pandemic in India. Journal of Development Economics.

Mart́inez Arranz et al. 2021. The uneven expansion of electricity supply in India: The logics of clientelism, incrementalism and maximin. Energy Research and Social Science.

Mohanan et al. 2021. Prevalence of SARS-CoV-2 in Karnataka, India.

Pais and Rawal. 2021. CMIE's consumer pyramids household surveys: An assessment.

Pandey, Patnaik and Sane. 2019. Impact of Tax Breaks on Household Financial Saving in India. National Council of Applied Economic Research.

Patnaik, Sane and Shah. 2019. Chennai 2015: A novel approach to measuring the impact of a natural disaster. National Institute of Public Finance and Policy.

Patnaik, Sane Shah and Subramanian. 2023. Distribution of self-reported health in India: The role of income and geography. PLOS ONE.

Somanchi. 2021. "Missing the poor, big time: A critical assessment of the consumer pyramids household survey".

Srikanth. 23rd May 2016. Free power to benefit 1.90 cr consumers. The Hindu.

TNERC. 2017. TNERC Tariff Order No.01 of 2017 in T.P. No.1 of 2017.

TNERC. 2018. TNERC Tariff Order No.07 of 2018 - Provision of Tariff subsidy for FY 2018-19 by the Government of Tamil Nadu.

TNERC. 2022. TNERC Tariff Order No.07 of 2022 in T.P. No.1 of 2022.

Vyas. 2021. View: There are practical limitations in CMIE's CPHS sampling, but no bias. The Economic Times, 23 June 2021.

Susan Das and Renuka Sane are researchers at TrustBridge. Ajay Shah is a co-founders of XKDR Forum.

Concerns about recent developments on SEBI's regulation-making on market rumours and insider trading

by Bhavin Patel and Renuka Sane.

Insider trading is one of the areas of financial regulation where the order of complexity required of state capability is relatively high, and the gains to society from successful implementation are relatively low. The Indian environment on SEBI enforcement against insider trading has accumulated many difficulties. In recent months, we have seen the next step forward in SEBI's journey, with a novel legal idea around rumours swirling in the market. In this article, we summarise the recent developments, and show two substantive problems with the direction taken by SEBI. These substantive problems are ultimately grounded in failures of process. There are signs that the process adopted in recent months for these developments on rumours, has more deficiencies than usual.

Recent developments on how SEBI thinks about rumours

Over the past year, SEBI has been concerned about the impact of market rumours on security prices. It proposed that certain listed entities be required to verify "market rumours" related to their firm. Amendments to this effect (the June 2023 Amendments) were made in the SEBI (Listing Obligations and Disclosure Requirements) Regulations, 2015 (the LODR Regulations), though the amendments are yet to be enforced. On December 28, 2023, it published a Consultation Paper on Amendments to SEBI Regulations with Respect to Verification of Market Rumour (the Current Consultation Paper) suggesting that listed entities should verify market rumours only if they are material. 'Materiality' meant that such rumours should lead to price movement in the security. It also proposed that if a listed entity classifies certain information as Unpublished Price Sensitive Information (UPSI) but does not verify a market rumour related to such information, it should continue to be treated as UPSI under the SEBI (Prohibition of Insider Trading) Regulations, 2015, (the PIT Regulations). In its Board Meeting held on March 15, 2024, SEBI approved this proposal, confirming that if a rumour related to UPSI is not verified, it will continue to be treated as UPSI. In other words, if a speculative story appears in the public domain, pertaining to UPSI about a firm, it will still be treated as UPSI until the listed entity verifies the story.

These steps contain difficulties that are generic to the working of SEBI. Firms are already mandated to release certain information (through listing obligations), and not release some (through prohibition on insider trading). SEBI's proposals on verification of market rumours have not yet explained the market failure proposed to be solved and how the selected intervention is the least costly way of doing so. In this particular case, the legal effects of the new law can be particularly damaging. The decision in the March 15 Board meeting, to continue to treat Generally Available Information (verified or not) as Unpublished Price Sensitive Information (till the company verifies it), goes against the grain of how price discovery works in the public market. A greater process discipline will help improve the thinking, and the democratic legitimacy, of insider trading regulation.

Substantive problem #1: The impact on the OTC market

Consider the following illustration:

• Assume a listed entity classifies "defaulting on a large supplier contract" as UPSI.

• Since the information is classified as UPSI, insiders would be prohibited from trading based on such information under the PIT Regulations. Note that the definition of insider under R. 2(1)(g) of the PIT Regulations includes persons with access to UPSI. This implies that if a retail investor, with no connection to the firm, has access to this information, the retail investor is also classified as an insider. This speaks to the over-inclusiveness of the term 'insider' under prevailing securities laws.

• Assume that a newspaper carries a story about a "rumour about the default". Since this is now reported in the media, it will be accessible to the general public. The listed entity chooses not to verify the information since it does not trigger the materiality threshold under R. 30 of the LODR Regulations (for example, it might not cause a price movement or may only cause a price movement lower than the limits specified in the threshold).

• Under the current proposal, an unverified event or information reported in the media would not be considered 'generally available information' under the PIT Regulations. It would continue to be UPSI. The definition of insider regarding UPSI would extend to the general public. As a result, the PIT Regulations would generalise the prohibition on trading based on such information, even though it has been reported in the media, because the listed entity chose not to verify it. This may effectively hinder the price discovery process, a core function of securities markets.

Further, the explanation to R. 4(1) of the PIT Regulations states that if a person possesses UPSI, their trades would be presumed to have been motivated "by the knowledge and awareness of such information". The proviso to this explanation states that an insider can prove their 'innocence' if: (i) the transaction is an off-market inter-se transfer between insiders who were in possession of the same unpublished price sensitive information without being in breach of R. 3 and both parties had made a conscious and informed trade decision, (ii)the transaction was carried out through the block deal window mechanism between persons who were in possession of the unpublished price sensitive information without being in breach of R. 3 and both parties had made a conscious and informed trade decision.

As a result, under proviso (i) two people who read the same unverified story in the media are consequently insiders and who have made a 'conscious and informed trade decision' can trade in the listed entity's securities if such trade is off-market; and under proviso (ii) a block deal trade is also allowed under similar circumstances.

The effect would be that regular trades on the floor of the exchange based on unverified media reports would be prohibited. However, off-market transactions or block deals conducted based on such information would not count as 'insider trading', creating a problem of arbitrary discrimination. This also drastically reduces the pool of investors able to trade, which may possibly lead to a liquidity collapse.

Substantive problem #2: Contradictions with existing regulations

The proposals contradict a literal interpretation of UPSI as set out in the PIT Regulations since they suggest that information which is neither unpublished (since it has been published in the media) nor price-sensitive (since it does not affect the price of securities and therefore falls outside the scope of the materiality threshold verification requirements under R. 30 of the LODR Regulations) would still be considered UPSI. This would suggest a need to amend the definition of UPSI under the PIT Regulations and, potentially, the provisos to the explanation to R. 4(1) and the definition of insider under R. 2(1)(g). The proposals in the Current Consultation Paper also necessitate changes to determining materiality under R. 30(4) of the LODR Regulations. This is because the proposed definition of materiality is not a part of the June 2023 Amendments. The materiality thresholds proposed impact which market rumours listed entities need to verify.

Difficulties with the process

Assuming that SEBI has considered the potential impact on liquidity and does not regard it as problematic, we are still faced with the problem of how these proposals may be implemented. The current proposal may require changes in other substantive regulations. If they are indeed to be implemented, they should be done through amendments to the PIT and LODR Regulations. Board minutes, Circulars, or Guidelines should not suffice to effectuate substantive changes. Under Section 30 of the Securities and Exchange Board of India Act, 1992 (the SEBI Act), changes to regulations must be laid "as soon as may be after it is made, before each House of Parliament, while it is in session, for a total period of thirty days." This allows for Parliamentary oversight of the regulator's exercise of powers of subordinate legislation and prevents the use of such powers beyond permitted limits. The June 2023 Amendments are scheduled to take effect from June 1, 2024, there is little time left for listed entities to prepare, and such amendments should be published soon. The process under S. 30 of the SEBI Act was followed in the case of the June 2023 Amendments, and the Amendments were laid before Parliament on August 11, 2023. There is no reason this process should be side-stepped now. SEBI should come out with a precise legal instrument, and the amendments to the PIT and LODR Regulations, to implement its proposal.

Ideally, the regulator should, in addition to inviting and analysing public comments, identify the problem or market failure these seek to address, the principles governing the proposals, the outcome the regulator aims to achieve through such changes, and an analysis of their costs and benefits as recommended by the Financial Sector Legislative Reforms Commission (FSLRC) in its Handbook on adoption of governance enhancing and non-legislative elements of the draft Indian Financial Code of December 26, 2013. The Financial Stability and Development Council had also approved the implementation of the FSLRC's recommendations in its meeting in October 2013. If this process had been followed, the failures in substantive thinking described earlier would have been avoided.

The authors are researchers at TrustBridge. We thank Amol Kulkarni and Madhav Goel for useful comments.

Assessing regulatory capability in Tamil Nadu electricity regulation: Evidence from appeals

by Bhavin Patel and Renuka Sane.

The Indian journey to decarbonisation faces the roadblocks of electricity policy. One of the critical impediments faced is that of state capability in electricity regulation. There is a new body of knowledge in India, in regulatory theory, that can usefully be brought to bear on the problem of improving electricity regulation. One element of this field is the question of assessing the state of regulatory capability. At any point in time, how would we judge the extent to which electricity regulation in a certain state is working well?

Electricity is regulated by the Central and State Electricity Regulatory Commissions (ERCs), which perform legislative, executive, and quasi-judicial functions. The concentration of power and the lack of democratic accountability in the new administrative state raise concerns about their functioning in modern economies. In India, the ERCs have quasi-judicial powers that are even broader than those of other regulators such as the SEBI. With one electricity regulator -- the TNERC -- in focus, we took up the research strategy of assessing regulatory capability through the analysis of regulatory orders that go to appeal. A key metric of the quality of orders the regulator passes is how well its orders fare at an appellate forum. If the regulator succeeds in defending its decisions, it signifies that the orders are well-reasoned and have followed the procedures required by the applicable law (whatever they be).

In a recent paper, Performance at the Appellate Tribunal as an indicator of regulatory capacity: The case of TNERC at APTEL, we study the overturn rate of the orders of the Tamil Nadu Electricity Regulatory Commission (TNERC) at the Appellate Tribunal for Electricity (APTEL) between 2013 and 2023.

We find that TNERC failed to hold its ground in 52% of appeals. Of these, about half were remanded back to the TNERC, suggesting that it did not do an adequate job of bringing the evidence necessary to decide a matter, despite being conferred the powers of a Civil Court for such purposes under Section 94 of the Electricity Act, 2003. Further, the TNERC lost 86% of matters that involved issues related to how it uses its regulatory powers.

These results raise concerns about the exercise of quasi-judicial authority by the TNERC. There is considerable knowledge in the field of regulation in India, on how such deficiencies can be addressed. If policy makers were to take up such strategies, it would help improve private sector confidence for electricity investment in Tamil Nadu.

Our work is limited to orders that were appealed. There is a selection bias, in that regulated persons make rational choices when deciding to appeal. In the extreme, it is possible to argue that all sound orders are not appealed, and only defective orders are appealed, so the defect rate is just the appeal rate (which is about 8% with appeals to APTEL that we measure, and some appeals that go to the High Court under writ jurisdiction which we do not measure). We suspect the defect rate is higher. Considerations in appealing include issues such as errors in the regulatory order which create a high chance of winning, the expense in appealing, the rupee value at stake multiplied by the probability of winning, and the non-rule-of-law pressures from TNEB in favour of not appealing. The appealed orders, that are studied by us, offer valuable insights into the limitations of state capability in regulation in TN electricity. Such measurement should motivate regulatory reform, and it should be used in measuring the extent of progress in regulatory reform.

Our approach can be usefully applied in analysing and improving electricity regulation in other states. To assist this process, we have also released a detailed Manual For Reviewing Regulatory Orders: Orders of the Tamil Nadu Electricity Regulatory Commission at the Appellate Tribunal for Electricity. This will enable replication of our results for Tamil Nadu, and porting these methods to other states.

The authors are researchers at TrustBridge.

Announcements

Janaagraha is hiring: Head - Municipal Finance

We at Janaagraha seek an exceptional individual to lead our municipal finance mission to the next level of impact. We aim to increase municipal revenues, ensure transparency in spending, and improve accountability for citizen outputs and outcomes. Transforming public finance in cities is quintessential to human development in India. Over the last four years, we have achieved remarkable milestones in the pursuit of our municipal finance mission, including:

• Public disclosure of audited annual accounts of over 4,200 municipalities in a comparable format on www.cityfinance.in
• Catalyzing property tax reforms in 20 states resulting in a 50% growth in pan-India property tax collections since 2017-18
• Digital grant management of USD 15 billion of XV FC grants on www.cityfinance.in

More importantly, we have put together an exemplary, majority-women team. We are looking for a senior leader with a proven track record in systems practice to join our team and take our municipal finance program to new heights of systems change.

Key Responsibilities

• Deliver on government and donor engagements.
• Engage closely with senior leaders in government and the market ecosystem to both advocate for systemic reforms in municipal finance and to inform Janaagraha's strategy for the same.
• Continuously strive to transform municipal finance in India at scale and with speed and agility, through a whole of systems approach.
• Lead planning and development of Janaagraha's Municipal Finance programme.
• Engage with the urban governance and municipal finance ecosystem.

Qualifications and Experience

• Master's degree or equivalent in business administration, finance, public finance, public administration, public policy or other directly relevant fields OR a Membership of a Professional Institute such as the Institute of Chartered Accountants of India.
• 15+ years of experience in government advisory, public finance, business management/consulting or senior leadership roles in finance.
• Candidates with direct government experience in any civil services are particularly welcome.

Location: Bengaluru/Delhi

Interested candidates can send in their CV to work@janaagraha.org

You can learn more about the role here.

Janaagraha's culture codes can be seen here.

About Janaagraha

Janaagraha is a Bengaluru-based not-for-profit institution working to transform the quality of life in India's cities and towns. It defines quality of life as comprising quality of infrastructure and services, and quality of citizenship. To achieve its mission, Janaagraha works with councillors and citizens to catalyse active citizenship in city neighbourhoods, and with governments to institute reforms to city-systems. Janaagraha has worked extensively on urban policy and governance reforms for over two decades including on JnNURM, and with the XIII, XIV and XV Finance Commissions, Second Administrative Reforms Commission, Comptroller and Auditor General of India, NITI Aayog/Planning Commission, Ministry of Housing and Urban Affairs (MoHUA), as well as the state governments of Odisha, Uttar Pradesh, Tamil Nadu, Rajasthan, and Assam.

Announcements

IIHS Urban Fellows Programme

The Indian Institute for Human Settlements (IIHS) invites applications for the ninth batch of the Urban Fellows Programme (UFP). The UFP is a nine-month, full-time, residential, interdisciplinary programme based at the IIHS Bengaluru City Campus.

The UFP is a unique space that combines classroom teaching, site-based applied learning, live projects and external internships to introduce learners to diverse forms of urban practice. Its interdisciplinary framework encourages learners from different disciplines, and practices diversity across multiple facets. Alumni of the UFP work in a wide range of sectors and organisations, driving urban transformation across India.

The UFP will run from August 2024 to May 2025 and recent graduates and young professionals from varied educational backgrounds or practice domains are eligible to apply. The UFP is committed to providing scholarships to all deserving candidates through a needs-blind process.

This video gives an overview of the UFP and this video explains how the UFP is unique.

Admissions close on 15 April 2024. For queries, you can write to ufp.info@iihs.ac.in or contact +91 99012 55788, 96064 84336 (10:00 am to 6:00 pm India Time, seven days a week).

For more information, please visit the UFP website.

Rethinking innovation policy in India: amplifying spillovers through contracting-out

by R. A. Mashelkar, Ajay Shah, Susan Thomas.

Independent India valued science and rationalism at an early stage of social and economic development. The objectives of building the scientific temper and harnessing the power of science and technology were clearly articulated, for example, in the 1958 Science Policy Resolution. These objectives pertain to improvements in the people, in the society. 

The practical aspects of government expenditures on innovation, and the construction of organisations doing frontiers work, emphasised government organisations. The government used taxpayer resources, built science organisations, hired scientists as civil servants, and developed capabilities within these organisations

In a new paper, Rethinking innovation policy in India: amplifying spillovers through contracting-out, we reopen the objectives of innovation policy in India, applying modern knowledge of public economics and public administration to obtain fresh insights.

We start at the foundations of innovation policy, with new clarity on the questions of why (what motivates state intervention in innovation?), what (in what areas should government intervention into innovation take place in India?), how (what mechanisms should be used when spending public money?) and how much (at what point do the incremental gains to society equal the incremental costs).

The market failure that motivates this field is the problem of spillovers, where the full gains from innovative activity by one person are not captured by her, leading to systematic under-investment into innovation by her. There is a case for public expenditure, where taxpayer resources are spent on innovation, but the expenditure needs to be done in a way that induces spillovers into the society.

We undertake four detailed case studies: the US National Aeronautics and Space Administration (NASA), the US National Institutes of Health (NIH), innovation policy in French defence procurement and CSIR's New Millennium Indian Technology Leadership Initiative (NMITLI). In each case, we understand how tradeoffs are made between `make' and `buy'. Make involves building state organisations, scientists as civil servants. Buy involves contracting-out innovative activities into the society, to private firms and particularly to high-spillover sites of universities and research organisations (whether public or private).

While doing more contracting-out is appealing from the first principles of innovation policy -- the purpose is to obtain greater capabilities in the society, not in the state -- there are many difficulties in implementation. We suggest the strategy for implementation which involves (a) Changes to the GFR; (b) Changes to the founding documents of government innovation organisations; (c) Changes to procurement rules and internal process manuals; (d) Resource planning for a gentle reform trajectory; and (e) a sketch of the required project planning.

Many elements of innovation policy in India have been moving in a similar direction. Three recent initiatives should be pointed out. The Union Interim Budget of 2024-25 envisages a Rs.1 trillion fund that would be channeled to research and innovation in the private sector. The `National Research Foundation' has been setup with a law that came to force on 5 February 2024. The K. Vijay Raghavan Committee report, submitted in early January 2024, has important ideas on improving the working of DRDO. There is a harmony between the philosophy of these three moves, and the ideas of this paper. Conversely, the detailed work of this paper can be useful in translating these initiatives from concept to implementation.

R. A. Mashelkar, FRS, was Director General of CSIR. Ajay Shah and Susan Thomas are co-founders of XKDR Forum.

Author: Charmi Mehta

Charmi Mehta is a Researcher at XKDR Forum, Mumbai.

On this blog:

• The electricity chokepoint in Tamil Nadu public finance, 26 February 2024.
• Offshore wind in Tamil Nadu: from potential to reality, 13 January 2024.
• The difficulties of asset monetisation in the transmission sector, 9 January 2024.
• Identifying roadblocks in highway contracting: lessons from NHAI litigation, 13 July 2022.
• How competitive is bidding in infrastructure public procurement? A study of road and water projects in five Indian states, 29 March 2022.
• Lessons from the COVID-19 vaccine procurement of 2021, 15 November 2021.

The electricity chokepoint in Tamil Nadu public finance

Charmi Mehta, Radhika Pandey, Renuka Sane and Ajay Shah

Each state in India can be visualised as an entity in itself. Vast magnitudes of finance will be needed to put the states on an energy transition pathway. While this pathway will be different for every state, the electricity sector is likely to be the recipient of much of these funds. The investibility of the electricity sector is thus an important field of study.

Many states in India face fiscal distress and many states in India have difficulties in the electricity system. In a new paper, 'The electricity chokepoint in Tamil Nadu public finance', we bring the two streams of knowledge together for the state of Tamil Nadu, and offer fresh insights for fiscal policy and for electricity policy.

The formal toolkit of a `debt sustainability analysis' (DSA) is brought to the standard Tamil Nadu fiscal data. This involves a first stage of comparing a group of fiscal indicators against normative benchmarks, and a second stage of forecasting the debt/GSDP ratio and the interest payments to revenue receipts ratio (IP/RR ratio) for five years; till FY 2028. These results, which we term the `baseline DSA' translate the mainstream intuition towards Tamil Nadu's fiscal difficulties into tangible numbers and forecasts.

A consolidated financial picture is drawn by integrating the two electricity sector utilities -- TANGEDCO and TANTRANSCO -- fused into the Government of Tamil Nadu debt stock. This yields a modified DSA that we term a `Corrected DSA'. This is done to acknowledge the implicit guarantee that state governments hold towards the debt of state-owned entities. This modified picture is thus a truer depiction of the fiscal problems of the state.

The fact that large debt servicing expenditures were successfully achieved for the last decade has helped create a confidence that the fiscal strategy of Tamil Nadu is deplorable but feasible. Of essence in the fiscal outlook of every highly indebted entity is the problem of sustainability. There are three concerns:

1. Sustained large scale borrowing, from the financial system, may potentially face difficulties through the risk appetite of lenders, changes in regulations, systemic crises in the financial system, etc.
2. The most important assumptions that shape the results of this paper are the nominal interest rate ($r$), estimated at 7%, and nominal GSDP growth rate ($g$), estimated at 9%. This has a $r-g$ of -2: it is a very positive environment from the viewpoint of fundamental fiscal dynamics. In the future, if $r-g$ becomes less benign, the debt dynamics could change significantly.
3. The conventional notion of fiscal stress is phrased in terms of bond default. In India, fiscal distress is known to manifest itself as unplanned budget cuts (that disrupt the working of the government), defaults on payments to private firms, and even the deferrals salaries or pensions. We look back upon three instances when state governments faced high fiscal stress in 2001, and find that the present projections for Tamil Nadu for FY 2028 are partially similar to these values.

The fiscal knowledge of this paper has implications for electricity policy. The electricity system requires two large blocks of investment. A big block of capital is required to rebuild the grid for the post-carbon world. And, a big block of capital is required for the investment in renewables and storage that are required to sustain economic growth in the post-carbon world. Of particular importance is the economic upside from exploiting that remarkable natural resource which is found off the coast of Tamil Nadu in the form of offshore wind generation. These investments will not arise in the environment of chronic fiscal stress in the electricity system.

The electricity knowledge of this paper has implications for fiscal policy. Through simulations where electricity subsidies remain constant or they are completely eliminated, we find that the electricity system is material in solving the fiscal problem. Thus, we extract the electricity subsidy problem from the sector, and place upfront its impact on the public finance parameters and the development trajectory of the state. A complete electricity sector reform versus business-as-usual translates into an FY 2028 outcome for the debt/GSDP ratio of 32.47% vs. 43.53%, and an IP/RR ratio outcome of 19.71% vs. 26.12%. These are large differences. They encourage us to prioritise electricity sector reform as a part of the medium-term fiscal strategy.

Charmi Mehta and Ajay Shah are researchers at XKDR Forum. Radhika Pandey is a researcher at NIPFP. Renuka Sane is a researcher at Trustbridge.

The consequences of criminalising cheque bouncing

by Shubho Roy and Ajay Shah

Many countries (e.g. New Zealand, Poland, Germany, Norway) have discontinued paper cheques. In other countries (e.g. the U.K., the U.S.), the use of cheques is declining. But in India, the use of paper cheques in India has stabilised in terms of value and number over the last five years. This is despite the extent to which digital and instantaneous payments are now feasible. Why might this be the case?

Prior to 1988, cheques were primarily used as a tool of payment. In that age, there were delays in clearing. Paper cheques had to be transported to the bank branch where the cheque issuer had an account. In that branch, the issuer's signature would be verified against a sample. If the signatures matched, the balance would be cleared (assuming the issuer had adequate funds). This process took seven working days, even when the issuer and recipient had banks in the same city. If the parties were in different cities, the process would take 15 working days, on average.

Today, a cheque issuer can send a secure document showing that the issuer's bank account has adequate funds to honour the cheque. However, in 1988, there was no such system, and the cheque recipient faced the risk that the issuer was writing a cheque that her account could not honour.

Some technical mistakes can always happen, where a person fails to anticipate the date on which funds are required and there are unpredictable delays in the money moving in and out of the account. Alongside this, many unscrupulous people knowingly wrote bad cheques. This made sellers mistrust cheques and prefer cash. While cash as a payment mechanism has the virtues of instantaneity and privacy, it comes with difficulties on physical security.

The 1988 change of the law

In this setting, the Parliament criminalised the bouncing of cheques in 1988. Now, the cheque writer could be sent to jail if the cheque did not clear. The law also stated that every cheque is presumed to be written to clear a debt. This change helped the recipient of cheques because the recipients did not have to prove any underlying transaction. The recipient only had to demonstrate that the cheque was not honoured.

In 2016, the Supreme Court ratified the practice of using cheques as collateral in the case of Sampelly Satyanarayana Rao v Indian Renewable Energy Development Agency Limited. Sampelly Satyanarana Rao (Mr. Rao) had written post-dated cheques as security for a loan from the Indian Renewable Energy Development Agency (IREDA). Mr. Rao did not take the loan personally but wrote the cheques as a director in the company that borrowed the money. The borrower company failed to pay the instalments when they became due. In response, IREDA (the creditor) initiated criminal proceedings against Mr. Rao under the 1988 law. Mr. Rao defended the claim by stating that the cheques were written before the creditor (IRDEA) disbursed the loan amount. IREDA pointed out that the cheques were deposited after the borrower (the company) had failed to pay the instalments, and therefore, the penal provisions of the 1988 Act applied. The Supreme Court agreed with the creditor and allowed for the criminal prosecution of Mr. Rao. This judgement provides legal certainty to the use of post-dated cheques as security. This date, i.e. 2016, is an important milestone in the journey, over and beyond the amendment of the N. I. Act in 1988.

The role of cheques in India today

The threat of imprisonment restored some faith in cheques. Anecdotally, it seems to have worked well in the initial years after the amendment. Cheques became more acceptable in commercial transactions and helped reduce frictions in economic activity. Many a cheque recipient was willing to take the risk of delivering goods without waiting one or two weeks for a cheque to clear.

These considerations do not exist in the present landscape. Instantaneous payment systems are ubiquitous in India, ranging from small value payments to the largest amounts possible. Any problem of trust between buyers and sellers can be readily solved by resorting to NEFT or RTGS. By this reasoning, the number of cheques written in India should have declined sharply. It has not.

People responded to incentives

Alongide this, the rest of the Indian legal system which enforces contracts works poorly. Ordinarily, a loan dispute would be resolved as a contract dispute through civil law, and, in some cases, bankruptcy law may be involved in situations where the debtor is insolvent. In 2020, in enforcing contracts, India ranked 163 out of 190 countries, while its overall rank was 63 (a difference of 100 ranks). India's rank in resolving insolvency was 52. Hence, creditors are unconfident about ordinary credit enforcement mechanisms.

One strand of credit enforcement systems is seizing assets that are pledged as collateral. This tends to work poorly in India. While the SARFAESI Act of 2002 is reasonably effective in getting collateral into the hands of the lender, many assets are hard to sell. Many land titles have encumbrances, and the land market works poorly, which hinders the recovery rate.

These weaknesses of the ordinary (civil) credit enforcement systems made criminal proceedings under S.138 attractive to creditors. Under S.138, a debtor faces up to two years of imprisonment if the debtor is convicted. In reality, the creditor does not even have to wait for the end of the litigation to get the debtor imprisoned. The debtor can be arrested at the beginning of the litigation so that the debtor can be produced before the court. In some cases, the debtor can also be imprisoned for the duration of trial under S.138. In contrast, a civil case proceeds without the debtor, if the debtor chooses not to appear. Most people will pay up to avoid being imprisoned, which gives heart to creditors.

When faced with legal difficulties around land title, it is better for the creditor to threaten imprisonment, and have the borrower solve the problem of selling the land, instead of seizing collateral and then facing legal difficulties in liquidating them.

As a consequence, after the new law was established in 1988, creditors started using cheques as a security. Creditors frequently demand that the debtor provide post-dated cheques for loan amounts. These cheques are payable deep into the future -- sometimes extending to multiple years. In such cases, both parties are aware that the cheque drawer does not have the money in the bank account at the point in time when the cheque was signed. Creditors sometimes demand a separate cheque for each instalment of loan repayment. Consequently, a debtor for a five-year loan may write 60 post-dated cheques. On a similar note, landlords sometimes asked for post-dated cheques for the payment of rent at multiple time points in the future.

Modern economies do not have a debtors prison: the choice of filing for bankruptcy is always there, in which case the creditor gets a low recovery rate. On one hand, in India, there is no legal framework for personal bankruptcy. When threatened with jail time, the borrower may reach into her web of relationships, and borrow from the community. This increases the resources available to the lender.

Weighing the pros and cons

The introduction of S.138 has thus exerted many complex impacts upon the working of the economy.

An increased level of violence in society

More people go jail, and more threats of incarceration are bandied about. This is a less civilised society.

Increased interest in lending

When lenders are given greater certainty about recoveries, they are likely to be more willing to lend to persons that might otherwise be excluded from the credit market.

Diminished interest in borrowing

When borrowers are shown the possibility of jail time, they will be more cautious and avoid borrowing. That has its own welfare consequences.

Conditions for state failure

The prospect of jail time is a `high stakes' situation where the policing system gets to make decisions which have a high impact upon the life of a citizen. This increases the incentives for corruption.

Hindering the emergence of a modern economy

All advanced economies have moved away from debtor's prison, and evolved civil mechanisms around borrowing, collateral and bankruptcy. These pathways reduce the extent of violence in society and increases user confidence in borrowing.

The threat versus the cash

Jail time is indeed a potent threat and creates strong incentives for the borrower to obtain cash, either by borrowing from someone else or by liquidating opaque assets. But once a person goes to jail, all future payments to the lender are stopped. With more civil processes of collatoral and bankruptcy, there is the strategy of keeping the delinquent active in economic life, and obtaining a stream of cashflows to the lender.

Incentives for policy makers

Lenders that got comfortable with the use of S.138 were less inclined to persuade policy makers of the need for the institutional apparatus of the credit market.

Conclusion

The introduction of S.138 into the N.I. Act in 1988 was a response to a problem of the time. Some other countries, like Taiwan, had also criminalised cheque bouncing. However, most countries have walked back since then because credit systems have improved and the use of cheques have declined. In those countries, cheques are not used as collateral for loans.

There is a strong argument for repealing this section. The consequences of such a repeal will, however, also be far reaching, particularly in the context where the institutional apparatus for contract enforcement remain weak. It is interesting to look at the list, presented above, of the consequences of criminalising cheque bouncing. We can then ask: Which of these would flip around and arise, in reverse, when cheque bouncing is de-criminalised.

Shubho Roy and Ajay Shah are researchers at XKDR Forum.

Announcements

Upcoming Conference: Electricity Reforms in the Economic Strategy of Tamil Nadu

XKDR Forum and TrustBridge Rule of Law Foundation, are hosting a conference on 'Electricity Reforms in the Economic Strategy of Tamil Nadu', on 29 February 2024, in Chennai.

Tamil Nadu is one of the most important states in India from the perspective of energy transition. It was one of the first states in India to have achieved almost universal electrification and was also at the forefront of the transition to renewables - both wind and solar - in the country. The session hosts a discussion on a roadmap for reforms in the electricity sector, bringing together the team's year-long research towards identifying problems and strategies for the state's energy transition and economic growth.

Program Design:

11:00 - 11:45: Electricity Reforms in the Economic Strategy of Tamil Nadu

Presenter: Ajay Shah, XKDR Forum

11:45 - 13:00: Open Discussion

Moderator: Akshay Jaitly, Trilegal and TrustBridge Rule of Law Foundation

13:00 onwards: Lunch

Date: 29 February 2024 (Thursday)
Time: 11:00 am - 1:00 pm
Location: Taj Coromandel, Chennai.

The event is in-person and you can register here!

Solarisation in agriculture in Tamil Nadu

by Susan Das, Renuka Sane, Ajay Shah.

Highly subsidised electricity for farmers is an important problem faced in the working of the electricity sector. With falling prices of solar panels, there is the possibility of government subsidising solar panels established by farmers.

This could potentially induce welfare gains in several directions.

1. The fiscal burden imposed by the subsidy could decline.
2. The pricing distortion in electricity (where commercial / industrial / domestic buyers are overcharged) could decline.
3. Farmers could earn a revenue selling their surplus electricity to the grid.
4. Farmers could become more thoughtful in their extraction of ground water when they face an opportunity cost, in the sense that the electricity that is not used to pump water is a revenue generator for them. This would yield welfare gains by diminishing the `tragedy of the commons' in ground water.

These possibilities turn on careful calculations. Whether some or all of these gains are obtained depends on insolation, the magnitude of water required, the energy cost of extracting the water, the cost of solar panels and the price paid by the grid. There will not be one answer within a state, and therefore there will not be one optimal policy within a state. (Similarly, the economic possibilities from such pathways will vary greatly across the breadth of the country).

In a recent paper, Solarisation in agriculture in Tamil Nadu: A first principles evaluation, we try to engage in this careful calculation for one district (Erode) in Tamil Nadu. We analyse a corner solution: one where the government pays for the full cost of the solar panel. 

The results turn on the price at which the surplus energy, that comes from the farmer to the discom, is sold. If the discom is able to sell this energy at the (high) prices that are charged to the commercial and industrial ("C&I") customers, then the corner solution is financially efficient for the grid. At lower prices, the proposition is less attractive.

 One important parameter that influences the results -- the price of solar panels -- is likely to decline in the future. Hence, we simulate the scenario with lower costs. We find this expands the class of situations where solarisation in agriculture is useful.

This paper is about one district (Erode) in Tamil Nadu. The methods adopted are general and could be applied to other locations in India. The answers are likely to vary considerably depending on the precise setting. There is value in discovering how this varies across India.

This is a field with many intricacies in implementation. These include the mechanism of selling surplus electricity, the choice of the tariff paid to the farmer, the problems of (the lack of) metering of electricity connections to farmers, the trustworthiness of the government on timely payments to farmers, the financing mechanism for the capital cost the puzzles of operations and maintenance at the level of one farmer, and the path to a sound monitoring and evaluation of such programs. These are much studied areas where considerable research has taken place. These discussions will improve through using the carefully constructed numerical estimates, on a per-district basis, all across the country.

Survey-based measurement of Indian courts

by Pavithra Manivannan, Susan Thomas, and Bhargavi Zaveri-Shah.

Public institutions do not face a market test. Achieving state capacity is about establishing checks and balances. The traditional idea is to instrument the operations, and construct an operational MIS, which is released into the public domain. Through this, deficiencies of the working of the organisation are visible to researchers and the public. The other pathway is to ask the persons who interact with the state institution about what they feel, to elicit their perceptions. This is an important pathway to obtain evidence and thus create feedback loops. For instance, citizen surveys are commonly used to assess the quality and impact of public services such as health and education (UNDP 2021, Clifton et al, 2020, OECD-ADB 2019).

In the legal system, perception surveys of court users can generate useful knowledge about how well courts function in their delivery of justice (National Center for State Courts, 2005). Ongoing surveys of user experience of courts can help measure the performance of a component of the entire legal system, and in assessing the impact of interventions made for reforming the legal system.

Surveys of court users and the public on their perception of the judiciary have been prevalent in developed countries from the 1990s, and are gaining currency in India (eg., Dougherty et al, 2006; Rottman and Tyler, 2014; Staats et al, 2005; Daksh 2016). Such surveys seek to capture the perceptions of court users on qualitative metrics (Manivannan et al, 2022). Such metrics can be used to evaluate the functioning of a single court, or compare alternative courts.

On one hand, perceptions are not reality. On the other hand, the views of end-users of the justice system are particularly important because, ultimately, the justice system exists to serve end-users whose interests and preferences may differ from those of judges and lawyers. We can readily discern certain difficulties in survey-based measurement of perceptions:

1. There are many different users of a court, who differ in their extent of knowledge. Litigants who see a court case as a disruption of their daily lives, may see things differently when compared with lawyers, for whom courts are part of their professional lives.
2. A person who loses a case is likely to be unhappy with his experience of the court and vice versa.
3. Different individuals might be working on non-comparable cases, and their subjective experience of the court is then not comparable.
4. It is not clear what is an objective benchmark of sound performance. A perfect court may be prohibitively expensive. Users of courts may have normalised a variety of difficulties; their `satisfaction' may only flow from learned helplessness.
5. It is important to narrowly measure a court or a group of courts, and make claims about the narrow unit of observation, as opposed to bigger claims about the Indian legal system.

In 2023, we conducted two pilot surveys to evaluate their utility as feedback loops for courts.

One survey was administered to understand the functioning of five alternative forums that can be approached to adjudicate matters of debt disputes: the Bombay benches of the National Company Law Tribunal (NCLT), the Debt Recovery Tribunal (DRT), the Bombay High Court (Bom HC), the Metropolitan Magistrate (MM) courts (which adjudicates criminal proceedings for cheque bouncing cases), and the Alternative Dispute Resolution (ADR) process.

To help improve data quality, the survey was conducted on practitioners who had multiple instances of interacting with the five courts. By selecting practitioners that have had repeated instances of approaching these forums to resolve disputes, the survey results are less vulnerable to the 'loser' effect. To obtain comparability, we presented a hypothetical, canonical problem of debt dispute resolution to each survey respondent. We then asked them to rank the five forums on five dimensions of court performance, namely, efficiency, effectiveness, predictability, independence, cost and convenience, and calculated the average rank for each forum on each of these dimensions.

The second survey was conducted with litigants at the DRT, with the objective of understanding the functioning of this court. For this, we deployed a team of four, who visited the premises of the Bombay bench of the DRT. The team administered a survey questionnaire on individuals, in order to evaluate the performance of the DRT on the above mentioned five dimensions. The participants were asked to rate their experience at the DRT on a five-point scale.

Method

Survey design

We used a combination of qualitative (in-depth expert interviews and open-ended comments) and quantitative surveys (multiple choice and scaled questions). Qualitative surveys with experts provide more contextual insights, enable comprehensive analysis. They helped validate our founding conjecture, the idea that there was a class of disputes which could go to multiple different forums. However, these surveys were time-intensive and it was difficult to obtain the interest and involvement of experts.

Survey mode

We administered the survey in both online and offline formats. Surveying litigants on court premises was challenging in two ways. First, litigants do not always accompany their lawyers to courts, especially in disputes of larger sizes involving firms. Second, one forum may deal with multiple type of disputes (civil v. criminal; mergers v. insolvency). This poses difficulty in identifying a litigant with a desired case-type.

The questionnaire used for the surveys and the responses collected can be found here.

Results: The perceptions of practitioners

The practitioner survey involved eliciting their choice of forum for the following hypothetical, canonical problem:

Q is a large public listed company. It has availed of a working capital loan of Rs. 7 crores from N, a small sized NBFC, repayable within three years with simple interest @16% p.a. Q and N are 100% domestically owned. As collateral for the loan, Q has granted N a floating charge over some of its movable assets, for example, its machinery or its inventory. One year into the loan, Q defaults on its loan to N. The outstanding amount exceeds Rs.1 crore. Post-dated cheques issued by Q towards interest payment bounce due to insufficient funds. The collateral is not sufficient to cover the outstanding amount. You are advising N.

The survey respondents were asked to make two assumptions, namely, that the limitation period is the same across all the courts; and that all courts have jurisdiction.

We collected responses from 18 respondents, of which 16 were lawyers and two were key managerial personnel at an asset reconstruction company and a debt restructuring advisory firm. Six of our respondents had between 20 to 30 years of experience in this area, eight of them had experience of less than 20 years, and two of them had more than 30 years experience in this field. They had significant experience with many of the venues of interest: 14 had experience with the NCLT and the Bom HC, 11 with the DRT and ADR process, and 5 with the MM Courts.

We aggregated the ranks assigned by the respondents to each of these forums on the parameters of independence, efficiency, effectiveness, predictability and access, and averaged them to arrive at an overall rank for each forum. The specific statements on which the respondents ranked the forums and their ranks are presented in Table 1. The forums are arranged in increasing order of the average rankings on each parameter. The NCLT was ranked the highest on the parameter of Efficiency, followed by ADR, the Bom HC, the DRT and the Metropolitan Magistrate. On the other hand, the Bom HC was ranked as the most preferred forum of choice on the parameter of independence.

Table 1: Preference ordering of five debt enforcement forums

Metric	Survey Statement	Ranking
		1	2	3	4	5
Efficiency	Most likely to dispose of my matter in a timely manner	NCLT	ADR	Bom HC	DRT	MM Courts
Effectiveness	Easiest to recover the amount awarded in the judgement decree.	NCLT	Bom HC	DRT, ADR	MM Courts
Predictability	(i) Expected sequence of stages in my matter was clear.	NCLT	ADR	Bom HC	DRT	MM Courts
	(ii) Hearings are most likely to be held as scheduled.	ADR	NCLT	Bom HC	MM Courts	DRT
Independence	Decisions are most likely made based on the merits of the case.	Bom HC	ADR	NCLT	MM Courts	DRT
Access	(i) Can afford to take my case to this forum.	MM Courts	DRT	NCLT	Bom HC	ADR
	(ii) Ease of navigation; staff helpfulness; website; ease of filing process	ADR	Bom HC	NCLT	DRT	MM Courts

Table 1 contains new insights on a specific court on each attribute. For example, while the Bom HC and the ADR process are perceived to be most unbiased, they are perceived as more expensive to access. ADR is perceived to be most predictable, but less effective on actually getting the relief. The NCLT, on the other hand, is perceived to be more efficient and effective, when compared to the other forums, but less likely to also be unbiased. The DRT and the Metropolitan Magistrate courts are perceived unfavourably on all aspects, except affordability.

Results: The perceptions of litigants

The in-person survey conducted at the DRT observed 55 persons, who were presently a party to a dispute at the DRT. Among these, 24 were debtors, 19 were creditors, and 12 belonged to the residual category, such as court/privately appointed receivers and auction awardees. Of these, 30.6% were at early stages (admission), 28.6% were at advanced stages (such as post-admission or pending last hearing), and 22.4% were awaiting a final hearing or pronouncement of judgement.

Litigants at the DRT had more positive perceptions than practitioners. Litigants ranked the DRT the highest on predictability of the hearing: most litigants agreed that when a hearing for their case is scheduled at the DRT, it will be held on the scheduled date. About 67-69% of litigants perceived the DRT to be an affordable and unbiased forum to resolve their dispute. More creditors ranked it higher (85-89%) on these two metrics than debtors (58-62%). However, 52% of litigants did not think that the DRT resolves cases in a timely manner.

Discussion

Good performance by the judicial branch in a country is essential. As with all aspects of public policy, this requires the loop of evidence, identification of difficulties, creative policy proposals, policy reforms, and measurement of the gains. In the legal system, generally, evidence and measurement involves quantitative measures. In this article, we have shown a case study where survey-based evidence was useful. This constitutes a useful additional pathway to measurement of the legal system.

Litigants are the ultimate end-users of courts, so their views matter greatly, but their information set may be limited. Legal practitioners have better information through repeated interactions and potentially observation of multiple venues, but their views may not capture the views of the litigants themselves. In the future, it would be useful to go further, by way of surveying the general public, measuring the view of persons who have not experienced litigation at a given location.

References

Shaun Bowler, Joseph L. Staats, and Jonathan T. Hiskey (2005). Measuring Judicial Performance in Latin America, Latin American Politics and Society.

Judith Cliftona, Marcos Fernandez-Gutierrez and Michael Howlett (2020). Assessing public services from the citizen perspective: What can we learn from surveys?, Journal of Economic Policy Reform.

Daksh (2016). Access to Justice Survey, A DAKSH report.

David B. Rottman and Tom R. Tyler (2014). Thinking about judges and judicial performance: Perspective of the Public and Court users, Onati Socio-legal Series.

Devendra Damle and Tushar Anand (2020). Problems with the e-Courts data, NIPFP Working Paper Series 314.

George W. Dougherty, Stephanie A. Lindquist and Mark D. Bradbury (2006). Evaluating Performance in State Judicial Institutions: Trust and Confidence in the Georgia Judiciary, State and Local Government Review.

Institute of Social Studies and Analysis (2021). Satisfaction with Public Services in Georgia, United Nations Development Programme.

National Center for State Courts (2005). CourTools: Trial Court Performance Measures.

Pavithra Manivannan, Susan Thomas and Bhargavi Zaveri-Shah (2022). Evaluating contract enforcement by courts in India: a litigant's lens, XKDR Working Paper No. 16.

Pavithra Manivannan, Susan Thomas and Bhargavi Zaveri-Shah (2023). Helping litigants make informed choices in resolving debt disputes, The Leap Blog.

OECD-ADB (2019). Government at a Glance Southeast Asia, Serving Citizens: Citizen satisfaction with public services and institutions, OECD Publishing, Paris.

Pavithra Manivannan and Susan Thomas are researchers at XKDR Forum, Mumbai. Bhargavi Zaveri-Shah is a doctoral candidate at the National University of Singapore. We thank Surya Prakash B.S., Renuka Sane, and Anjali Sharma for their suggestions on the design of the surveys. We acknowledge the very diligent assistance by Nell Crasto and Balveer Godara, students at Kirit P. Mehta School of Law, NMIMS Mumbai, on conducting the litigant survey. We are grateful to all the survey respondents for their generous participation, and thank Mahesh Krishnamurthy, K.P. Krishnan, Sachin Malhan, Harish Narsappa, Rashika Narain, Geetika Palta, Siddarth Raman, Ajay Shah, and Arun Thiruvengadam for their comments and suggestions on this work.