Time: 1700 - 1800 hrs.
Venue: Conference Room No.1
Topic: Strengthening the National Evaluation System in India
Guide: Prof. Bakul Rao
RPC members: Prof. Satish B Agnihotri, Prof. Trupti Mishra
Abstract:Every government works towards its citizens' social and economic development in different ways; in today’s world, the objective is to achieve the Sustainable Development Goals through market, bureaucratic, and social efforts. The Government of India has long adopted the programmatic development model. Regular evaluations are also mandated to measure its success or failure, along with other regulatory, controlling, and feedback measures such as monitoring and auditing. However, it is seen that despite a long history of government evaluation offices in the country and regular intermittent efforts to strengthen evaluation systems, the institutionalisation and use of evaluation in India remains weak. It is thus proposed that evaluation is given a stepchild treatment within the government system compared to other controlling measures.
To explore this hypothesis, the history of India's government evaluation system is documented from the establishment of the Programme Evaluation Office in 1952; further insights are taken from the changes in State Evaluation Offices over the last six decades. The evolution of evaluation systems in India can be divided into four phases – the planned growth phase (1952-1972), the contraction phase (1973-1995), the resurgence phase (1995-2013), and the new institutions and frameworks phase (2013 onwards). The repetitive nature of activities, events, and efforts to strengthen the evaluation system form patterns of cyclic efforts that are not sustained. When the history and status of the evaluation system are compared to the national auditing system, it establishes the weaker status of evaluation in comparison. The surface-level causes behind the lacunae are documented, and it is proposed that the weakness in evaluation systems is due to unresolved structural issues. Applying the iceberg model from systems thinking to understand the layers of complexity at play exposes the apathy in the government system towards evaluation. This is further explored to propose the reasons behind this condition based on the behavioural and mental models related to evaluation and evaluation systems amongst the crucial stakeholders.
The assessment of India’s national evaluation system is complemented through an analysis of its important outputs. To assess the evaluation reports, the Monitoring Information Systems, and the results-based outcome indicator framework (currently, the Output Outcome Monitoring Framework and previously the Results Framework Documents) specific tools were developed using literature, feedback was taken from experts, and then applied to available documents of five flagship programs of Ministry of Rural Development, viz., Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGA), Deen Dayal Upadhyaya Antyodaya Yojana - National Rural Livelihoods Mission (NRLM), Pradhan Mantri Gram Sadak Yojana (PMGSY), Pradhan Mantri Awaas Yojana Gramin (PMAY-G), and Mission Antyodaya. The scoring using the MIS assessment tool (MAT) of the five MISs shows that the older MIS of programs such as MGNREGA is better than the newer MIS such as that of Mission Antyodaya, applying the Evaluation report content analysis tool (ERCAT) to the five recent evaluation reports of the five programs show wide variation in their content and structure depending on the author institute, and tabulation of outputs and outcomes defined in budget documents show that neither the RFD nor the OOMF effectively captures the outcomes of the programs, despite annual revisions of indicators for the last two decades. It is seen that the quality of these outputs leaves much to be desired.
Suggestions are given regarding how to proceed further after learning how national evaluation systems have developed in other countries and applying them to the Indian context. Adopting a National Evaluation Policy is instrumental to strengthening the national evaluation system in India, and this needs support from the legislature, bureaucracy, professionals, academicians, and society. How this can be brought about is discussed using approaches in evaluation capacity development literature.