Problems of the Health Management Information System (HMIS): the experience of Haryana

by Smriti Sharma.

Last year during a "Beti bachao, Beti badhao" video conference, errors in the data became visible. The `Maternal Infant Death Review System' (MIDRS) of Haryana showed that the health staff in some  districts of Haryana had been grossly under-reporting deaths of mothers and infants. As an example, for the trimester of April-June 2015, the number of infant deaths measured in the MIDRS was 3,307, but only 728 were reported into in the Health Management Information System (HMIS). For maternal deaths, HMIS showed 21 deaths while MIDRS showed 145.

MIDRS is a surveillance-based system which was launched by the Haryana government in 2013 to keep tabs on such under-reporting. The system includes a mixture of routine passive data collection and active surveillance by specially recruited and trained field volunteers. Ironically, HMIS too was conceived as a mechanism to monitor the functioning of the National Health Mission (NHM).

Inaccurate data in HMIS raises concerns about the working of NHM. In this article, we take a close look at the HMIS in Haryana and understand the sources of difficulties.

HMIS: A management tool for National Health Mission

The Indian government launched the `National Rural Health Mission' in 2005. This was renamed as the `National Health Mission' (NHM). HMIS was intended as a management information system to oversee the working of NHM. NHM is a national mission that runs through the length and breadth of the country. There are approximately 1.8 lakh health facilities that make use of HMIS to capture data.

HMIS captures data about antenatal coverage, immunisation coverage, delivery services, family planning coverage indicators etc. Some states like Haryana used another system called DHIS for data collection at the State level. These systems remained in operation, but their data was uploaded into HMIS to achieve comprehensive information in HMIS.

Substantial public expenditures are taking place through NHM. For NHM to work effectively, HMIS must be sound. Hence, the reports about errors in HMIS are particularly alarming. If HMIS contains faulty information, there may be substantial failures in the working of NHM. This motivates HMIS as the object of study.

How does HMIS collect data?

Figure 1 shows how data flows into HMIS. The Indian public sector health system has multiple tiers, where the first point of contact between the community and the health system is the sub-centre where the most peripheral health services are provided. Here at the sub-centre, when a pregnant woman walks in, an auxiliary nurse & midwife (ANM) jots down her details into her register.

The ANMs at the sub-centre level do not have access to computers and have to record information in handmade registers. ANMs maintain multiple registers and carry all of them every month to the relevant Primary Health Centre where the information from their registers are transferred onto the DHIS (in Haryana) by a Data Entry Operator.

Figure 1: Data flows in HMIS

Figure 1 also shows the different levels at which the data is aggregated. Data from the sub-centre, primary health centres and community health centres is aggregated at the block level. The data sent from the block level and sub-district hospital and district hospital is aggregated at the District Headquarters. The District Headquarters then sends the aggregated data to the State Headquarters which forwards it to the national level.

Issues of data quality

Numerous concerns have been raised about the quality of the data. Singh et. al., 2014, found that many districts in Haryana were routinely over-reporting data. For example in Palwal district:

• ANC registrations were 47% higher than the total number of expected deliveries. Expected deliveries refer to the probable number of pregnancies which is calculated by multiplying the total population of the area by the birth rate.
  
• Reported deliveries were 11% higher than the expected deliveries.
  
• Measles vaccines administered were 16% higher than the number of reported live births.
  
• Overall, only one district (out of the 21 districts in Haryana) did not have reported occurrence (eg. immunisation rates, deliveries, children weighed) higher than the total population.
  
  

In a recent paper, Sharma et. al., 2016, found that the ANMs over-recorded the following two indicators the most:

• 3 or more Antenatal Care (ANC) visits by pregnant women
  
• Provision of 100 or more Iron/Folic Acid (IFA) tablets
  

When the ANMs reported the data for monthly submission, the data they inflated the most pertained to:

• IFA supplementation
  
• Contraceptive device insertions
  
• Administration of 2nd dose of Tetanus Toxoid (TT) to pregnant women
  

The authors find that data were over reported because it was known to the health staff that these particular indicators were crucial to the success of the program. Numbers were inflated when the actual coverage of service delivery of a sub centre was low; inflating the data helped to hide low performance.

Going by IPHS Guidelines, it is the responsibility of ANMs to register pregnant women and provide at least four antenatal check ups to pregnant women. They are also responsible for administering IFA tablets. We may conjecture that by inflating these indicators, ANMs were making their performance look better than it was.

Reasons for bad data quality

Why is the quality of data so bad?

Lack of capacity

HMIS was launched in 2008, but as yet, computers and the internet have not reached down the entire chain. There are two chronic problems:

• Lack of infrastructure: Data entry at the sub-centre level is by ANMs writing into physical registers. There are bound to be errors at this level because ANMs record data in handmade registers which are very badly designed. These registers sometimes do not have enough space available to write. Also, handmade registers do not necessarily capture all information that is necessary for the DHIS.
  
  
• Over-burdened manpower: At the PHC level, the Data Entry Operator is responsible for entering data for DHIS. Alongside, she is responsible for fulfilling several other reporting requirements too. For example, there is another health information system called Mother and Child Tracking System (MCTS). This too has its parallel reporting requirements and the Data Entry Operator has to report data for MCTS too. Similarly, the Data Entry Operator has to undertake data entry of immunisation report, vaccine and logistics, release and logbook data.
  
  

Lack of accountability

We spoke with the personnel at all levels of HMIS in Haryana. We were told that data errors happen, and are verbally pointed out over telephone calls. Those numbers are then corrected and re-submitted. There is a casual, informal camaraderie between Data Entry Operators and the Monitoring and Evaluation Officer. They all seem to sympathise with each other and have a shared belief that they are over-burdened, which justifies human errors. This situation is not unique to Haryana. According to National Health System Resource Centre's assessment of HMIS in 23 states, 72% of the states give feedback to the districts on the HMIS data. Out of these, 61% give feedback to blocks. However, the feedback is given verbally. Only 38% of the states give feedback via emails or letters. The assessment also showed that the feedback is used to manipulate data and not to improve quality.

The way forward

What you measure is what you can manage. The fact that HMIS has poor measurement raises important concerns about NHM. It is hard to see how NHM can effectively generate bang for the buck when it is grounded in an inaccurate management information system.

While duplication of data reporting is inefficient and a source of discrepencies in data, there are multiple databases which have health related data. These include District-level Household Survey, National Family Health Survey, Annual Health Survey and the Mother and Child Tracking System (which follows each individual mother and child). These should be used on a regular basis to cross-check the information in HMIS, so as to uncover data problems sooner.

The movement of aggregated data betrays IT systems design that is many decades out of date. In a modern IT system that would be constructed today, only transactions would be stored (e.g. one death). All aggregation would be done on the fly when queries are required.

Some early steps towards true business process engineering are easy to envisage. An online application called ANMOL or ANM Online, allows ANMs to use tablets to enter and update the service records of the beneficiaries on real time basis. Since the entire process is digital, the ANMs don't have to carry or maintain the registers and the entire process becomes paperless for them.

The problems of HMIS and NHM are primarily a question of incentives and public administration, and not about computer technology. For a counterpoint, in the 18th century, the recording of deaths in the US and in Europe was being done correctly. It does not require great technology to do these basic things.

Too often in India, there is a temptation to solve problems of public policy with computer technology. As argued in Shah, 2006, these projects must be located around two elements: of doing a full blown business process re-engineering, BPR, (i.e. not a superficial layer of computers on top of the old process), and of removing discretion with front line staff.

References

Singh, Gajinder Pal, Jordan Tuchman and Michael P. Rodriguez, Improving Data for Decision-making: Leveraging Data Quality Audits in Haryana, India, Abt Associates Inc., 2014.

Shah, Ajay, Improving governance using IT systems, page 122-148 in `Documenting reforms: Case studies from India', edited by S. Narayan, Macmillan India, 2006.

Sharma A., Rana S. K., Prinja S., Kumar R., Quality of Health Management Information System for Maternal & Child Health Care in Haryana State, India, PLoS ONE, 2016.




The author is a researcher at the National Institute for Public Finance and Policy, New Delhi. I thank Jeff Hammer and Ajay Shah for useful comments.