Request for Proposals (RFP) on Methodological Innovations to Improve Data Quality

RFP launch date: July 22, 2019

The last date for submission of proposals has lapsed

Background:

The National Council of Applied Economic Research (NCAER) is one of India’s oldest and largest independent, non-profit, economic and social research institutes. It undertakes grant-funded research and commissioned studies for governments and industry, and is one of the few think tanks globally that also collects primary data. NCAER has set up a National Data Innovation Centre (NDIC) to serve as a laboratory for experiments in data collection, interfacing with partners in think tanks, Indian and international universities, and government. NDIC forms an important core of NCAER’s long-standing data collection activities. NCAER, has partnered with the Universities of Maryland and Michigan for the NDIC. Initial funding for NDIC is being provided by the Bill & Melinda Gates Foundation.

Current Request for Proposals: 

The focus on data-oriented research and the dynamic policy environment impose a great demand for rapid, high-quality, and policy-relevant data. Changing socio-economic conditions and technological innovations necessitate rethinking of both the kind of data being collected and how they are collected. In that context, quality assurance mechanisms play an important role in influencing the credibility of the data being collected.

Therefore, the focus of the current RFP is to seek proposals on methods to improve data quality across the following themes: gender equity, women’s time use, health system, health insurance and healthcare expenditure, employment and unemployment, family planning, and financial inclusion.

The proposals on these themes should ideally focus on generating evidence pertaining to the following specific areas relevant to data quality:

Sampling frame for selection of ultimate sampling units: The list of sampling units or the sampling frame plays an important role in probability sampling and the accuracy and completeness of the frame dictate the magnitude of the coverage bias in survey estimates. Often the sampling frame for the ultimate sampling units is not readily available. Although house listing has traditionally been used for the selection of households, it can be quite resource-intensive.

This RFP seeks to identify alternative ways of constructing sampling frames for different modes of data collection and of validating the quality of the frames for its potential use in surveys. The proposals on these themes should ideally focus on the following specific areas relevant to the following themes:

Use of paradata as a quality assurance mechanism: In the case of surveys based on computer-assisted personal interviewing (CAPI), a lot of process data (paradata) are being generated, almost real-time, throughout the survey. This may include interviewer productivity indicators, call records, number of attempts made per interview to interview the targeted respondent, interview length, question-level time stamp data based on key strokes, use of question-specific remarks, GPS coordinates, and, audio recording of interviews. The RFP is soliciting proposals for innovative ways of using such data for monitoring data collection activities and intervening as per the findings of such a monitoring mechanism. Can remote monitoring based on paradata provide an alternative to field-based monitoring and supervision as traditionally used in surveys in India? Is this targeted monitoring and supervision method more effective as compared to the random back checks method?

Controlling interviewer bias and variability in outcomes: Computer-assisted modes of data collection ensure the availability of survey data on a real-time basis. This RFP also seeks to promote innovative assessments of such data in order to reduce interviewer bias and variability. Examples include measuring interviewer effort based on different indicators, such as household roster size, number of cases having zero value in consumption expenditure items, number of illnesses or hospitalisation episodes recorded, and number of formal and informal loans taken by the household, and comparing them across interviewers, identifying interviewers opting for more negative screening in order to reduce their burden of the interview relative to other interviewers or having a conceptual misunderstanding about specific questions. How can one use such early signs to minimise interviewer bias and variability in the outcome of interest?

Real time survey data and consistency checks: Real-time survey data can be used to identify intentional or unintentional mistakes in data collection and to take necessary actions for preventing such errors going forward. The RFP looks for demonstration of consistency checks based on a single variable or multiple variables in order to identify errors at the nascent stage of data collection, and to determine how regular tracking can reduce errors in the long run. In this context, it also seeks an answer to the following question: What is the most effective mode and hierarchy of communication that makes it a feasible quality assurance mechanism?

Use of machine learning techniques to identify patterns automatically: Depending on the extent and coverage of the survey, and the CAPI software used for data collection, the volume of paradata generated can be so huge that traditional methods of analysing data may not work. Examples of such paradata include key stroke data, and audio files generated in surveys recording the interviews. In such a situation, the following questions need to be addressed: How can one use machine learning techniques to identify interviewers not performing up-to-the mark? Can this process be automated? How can one evaluate machine learning techniques for achieving this objective?

Secondary data analysis to quantify variability between interviewers: This RFP also looks for application of statistical/econometric modelling in existing survey data for quantifying variability among interviewers across different outcomes, such as, sensitive versus non-sensitive data; straightforward versus complicated questions; and questions leading to a substantial skip versus questions without any skip patterns. It is also important to identify ways of interpreting the results, and the potential explanations behind the various findings.

Eligibility 

Applicants affiliated to any academic or research institutes, non-profit organisations, and private companies that have experience in primary data collection and work out of offices located within India are eligible to apply. We hope that the successful applicants will be able to collaborate with NCAER researchers in future activities, allowing NCAER and the Centre to expand both its network as well as the skill sets of professionals associated with it.

Funding

The Centre will support a budget of up to Rs. 20,00,000/- (inclusive of all applicable taxes) for a period of 12 months. The budget should clearly indicate the actual needs and modes of utilisation of the funding for the proposed project. There is a provision for two such grants of up to Rs. 20,00,000/- (inclusive of all applicable taxes). However, only one grant for each applicant will be considered for funding.

Application Procedure 

All applications must be emailed to Ms. Arpita Kayal, Program Manager, NDIC (akayal@ncaer.org) in a single PDF document (font ‘Georgia’, size 12) with the following components:

A) The proposal (no longer than 6 pages in single space) on research work falling under the focus areas outlined above. The proposal should include the following sections:

1. Project Summary

2. Specific Aim(s)

3. Research Strategy, which would further specify:

a. Significance

b. Innovation

c. Approach and Implementation Plan

4. Expected Outcomes

5. Potential Challenges and Alternative Strategies

6. Timeline

7. Budget and Budget Justification

8. Institutional Background

(The page limit for the proposal is exclusive of details on the budget and applicant’s institutional background.)

B) Curriculum Vitae of the key research staff who will undertake the proposed work.

Expression of interest

Applicants interested in participating in this RFP may inform Ms. Arpita Kayal (akayal@ncaer.org) of their interest. We expect to set up an information sharing phone call with potential applicants during early August 2019.

Selection Criteria

The selection of proposals will be based on the merit of the proposal and the CVs of members of the respective research teams. And, the merit of the proposal will be judged on the basis of the following criteria:

  • Alignment of the proposal with the RFP;
  • Innovativeness in methods outlined in the proposal;
  • Rigour and feasibility of approach;
  • Clarity of thought; and
  • Clarity in writing.

Expected Output

The grantees will be expected to submit a report on the methodology and results to the Centre, which will be posted on NCAER’s website. Successful applicants will be encouraged to submit their results for publications in journals. The study instruments and anonymised data sets will also be placed in the public domain, allowing for free online downloads.

 

Key dates

Proposal submission due date: 31st August, 2019.

Announcement of the successful candidates: 1st October, 2019.

Contract signing: 1st November, 2019.

Project period: 1st November, 2019 to 31st October, 2020.

NCAER, Brookings India, and Business Standard pay tribute to their former colleague, Subir Gokarn (1959-2019)

It is with deep sadness that many of Subir Gokarn’s friends and former colleagues heard about his tragic passing away in Washington DC on July 30th after a brief illness: Dr Gokarn was just 59. Since November 2015, Subir Gokarn had been the IMF Executive Director for India, Bangladesh, Sri Lanka and Bhutan. More than that, he was a professional and personal friend to many in India and overseas, known for his wry, keen sense of humour.

To celebrate the life and work of Dr Subir Gokarn, three New Delhi institutions—NCAER, Brookings India, and the Business Standard—came together on Wednesday, August 7, 2019 for a joint memorial meeting. Over 120 guests joined the meeting, held at the new T2 Conference Centre at NCAER. This 74-minute video of the memorial meeting contains tributes from the three institutions—delivered by Shekhar Shah, T N Ninan, and Shamika Ravi—and from a number of Subir’s colleagues and friends from his time at the Delhi School of Economics, IGIDR, NCAER, CRISIL, RBI, Brookings India, and the IMF.

Dr Gokarn started his professional career at the Indira Gandhi Institute of Development Research (IGIDR) in Mumbai, becoming an Associate Professor over 1991-2000. During this time, he also spent an academic year at the Economic Growth Center at Yale on a Fulbright Fellowship. From IGIDR Subir came to NCAER as Chief Economist and IFCI Chair during 1999-2002. In that role, he ran the first three NCAER-NBER Neemrana Conferences, working closely with NBER Research Associate Raghuram Rajan. His association with NCAER continued over the years as an active member of the India Policy Forum Research Panel, co-editor on the India Policy Forum journal for two years, and a regular at the NCAER Neemrana Conferences.

From NCAER, Gokarn went on to become the Chief Economist of CRISIL from 2002 to 2007, and then Chief Economist of Standard & Poor Asia-Pacific during 2007-09. At 49, Gokarn became the RBI’s youngest Deputy Governor responsible for Monetary Policy, Research, Financial Markets, Communications and Deposit Insurance, staying at RBI from 2009-12. From 2012 to 2015, Gokarn was the first Director of Research at Brookings India, building a vibrant institution while overcoming many teething problems. In recent years, Subir Gokarn was a regular columnist with the Business Standard and a frequent participant at its editorial meetings.

Request for Student Research Proposals supervised by Faculty Members

The last date for submission of proposals has lapsed

Background 

The National Council of Applied Economic Research (NCAER) is one of India’s oldest and largest independent, non-profit, economic and social research institute. It does grant-funded research, and commissioned studies for governments and industry, and is one of the few think tanks globally that also collect primary data. NCAER with its consortium partners, the University of Maryland and University of Michigan, has set up a new National Data Innovation Centre (NDIC). Initial funding for NDIC is provided by the Bill & Melinda Gates Foundation.

 

The main objective of NDIC is to serve as a laboratory for experiments in data collection, interfacing with partners in think tanks, Indian and international universities, and government. NDIC forms an important core of NCAER’s long-standing data collection activities.

 

Request for Proposal (RFP)

The focus of the current RFP is to seek for proposals on methodologies for data collection and analysis across the following domains: gender equity, health system, health insurance and healthcare expenditure, employment and unemployment, family planning, and financial inclusion.

 

The proposals should focus on innovative ideas to improve data quality pertaining to the following aspects:

  • Fulfilling data needs and bridging gaps: In the context of the above-mentioned domains, this would entail identifying the key sub-domains that have hitherto received little attention, and developing and evaluating scalable data collection modules.
  • Mode of data collection: This would involve experimenting with alternative modes of data collection such as telephone interviews, various computer-assisted methods, self-administered mode of data collection for sensitive topics, web-surveys and comparing these with the traditional methods of data collection in India.
  • Questionnaire designing: The main question that needs to be addressed during designing of a questionnaire is: What are the elements of a good questionnaire, and how can these be integrated to elicit accurate responses from the target respondents? In this context, the components that require special attention are: framing and wording of the questions, use of open versus closed questions, number of points on rating scales, labelling of the rating scale points, order of response alternatives, use of the ‘don’t know’ response, sequence of the questions, the recall period, and skip patterns, among other things.
  • Social desirability bias: Since it has been observed that a significant number of people provide false information to stay within the socially desirable framework than be seen to be part of the socially undesirable one, it is important to identify methods of overcoming this problem in data collection. This phenomenon may also be related to the interviewer characteristics and interviewing techniques, as often people report what they perceive is considered desirable by the interviewer instead of giving an honest response. In this context, the self-administered mode of data collection could be a viable alternative to reduce social desirability bias.
  • Interview setting: This implies assessment of the interview setting and how it can play a role in improving reporting by the respondents, particularly in the case of questions pertaining to sensitive behaviour (e.g., semi-private setting versus complete privacy).
  • Survey implementation: The method of data collection and survey implementation may also affect the quality of data. This necessitates identification of the key elements of survey implementation for enhancing data quality, such as interviewing techniques, developing a feasible field plan, monitoring and supervision of the fieldwork, and remote monitoring of process data and survey data.

 

Eligibility 

Students pursuing their PhD degree or equivalent in any Indian academic or research institute are eligible to apply. We seek proposals from student–faculty teams in order to promote innovations, build skills, and foster collaborations between the researchers and senior faculty both during and after the grant implementation phase.

 

Funding

The Centre will support a budget of up to Rs 6 lakhs for a period of 12 months. The budget should clearly indicate the actual needs and modes of utilisation of the funding for the proposed project. There is provision for five such grants. Only one grant from each applicant will be considered for funding.

 

Application Procedure 

All applications must be emailed to Arpita Kayal, Programme Manager, NDIC (akayal@ncaer.org), in a single PDF document (with the text in ‘Georgia’ font, point size 12), comprising the following components:

A) The proposal (not longer than four pages of text in single space) on research work falling under the Centre’s focus areas outlined above. The proposal should include the following sections:

1. Project Summary;

2. Specific Aim(s);

3. Research Strategy, which would further specify:

a. Significance

b. Innovation

c. Approach and Implementation Plan, indicating how the proposed questions/innovations will be tested;

4. Expected Outcomes;

5. Potential Challenges and Alternative Strategies;

6. Timeline; and

7. Budget.

(The page limit is inclusive of the first six components delineated above, while additional pages may be used, if needed, for detailing the budget.)

B) Curriculum Vitae of both the applicant and the respective research guide/faculty member.

C) Support letter from the research guide/faculty member.

 

Selection Criteria

The selection of students will be based on the merit of the proposal and the CV of the applicant. The merit of the proposal will be judged based on the following criteria:

  • Alignment of the proposal with the RFP focus areas
  • Innovativeness in method
  • Rigour and feasibility of approach
  • Clarity of thought
  • Clarity in writing
  • Selection will also be guided by the distribution of proposals across various domains and components that determine data quality.

 

Other Requirements

The student and faculty should first check the institutional policies of their respective universities/institutes regarding such grants before applying.

 

It is mandatory that the selected students spend some time at NCAER to make themselves familiar with the activities undertaken by the Centre and also to discuss and present their research findings. Optionally, the selected students may wish to spend the entire project period (one year) at NCAER.

 

At the end of the grant, selected students will have to submit a research/working paper authored by the student. The research/working paper will be uploaded on NCAER-NDIC website after it has been peer-reviewed. All research outputs are expected to follow the open access policy of the Gates Foundation.

 

The selected students and their supervisors will also be encouraged to contribute to the NCAER-NDIC blogs based on their experiences of working on the grant.

 

Key Dates:

Proposal submission due date: 1st August, 2019

Announcement of the successful candidates: 1st September, 2019

Contract signing: 1st October, 2019

Project period: 1st October, 2019 to 30th September, 2020

State of the Economy Seminar July 2019

NCAER presented its Quarterly Review of the Economy, covering the performance of the Indian Economy in the first quarter of 2019-20 and forecast for the year ahead at this seminar held at NCAER. Organized as an integral part of the Quarterly Review, this seminar brought together policymakers, industry leaders and researchers for a discussion.  Bornali Bhandari and Sudipto Mundle, NCAER, presented the analysis and key findings of the report. Devendra Pant from India Ratings and Research and Subhomoy BhattacharyaBusiness Standard were invited as discussants. The seminar also hosted a presentation on Fintech: Prospects, Challenges and Opportunities by Mitul Thapliyal, Partner, MicroSave Consulting.

Key Highlights 

NCAER forecasts growth based on both quarterly and annual models.  The forecast as per the Annual Model is that GDP at market prices will grow at 6.2% and Gross Value Added (GVA) at basic prices at 6.0% for 2019-20 on a year-on-year (y-o-y) basis. The forecast as per the Quarterly Model is that GVA will grow at 6.2% on a y-o-y basis.

In 2019–20, the real agriculture GVA is envisaged to grow at zero per cent, real industry GVA at 6.1%, and real services GVA at 6.9%. The Wholesale Price Index (WPI) inflation is projected at 2.6%. The growth rates in exports and imports, in dollar terms, are estimated at 4.4% and 3.3%, respectively. The current account balance and central fiscal deficit, as percentages of GDP, are projected at –0.6% and 3.5%, respectively.

Agricultural growth collapsed in the last two quarters of 2018-19.  The prospects for 2019-20 for the agricultural sector, in particular, depend largely on the south-west monsoon. As of 5 August, 2019, out of a total number of 36 agro-meteorological sub-divisions in the country, three had received excess rainfall, 21 had received normal rainfall, while the remaining 12 sub-divisions had received deficient rainfall. And the country as a whole has received 7% below normal rainfall by 5th August 2019. Further, the country has experienced temporal variations in rainfall.  The mixed performance of the agricultural sector has led to a high rate of inflation for food articles, which was observed to be 6.8% during the period April to June 2019-20 as compared to 1.5% during the same period last year.

Industrial growth has been slipping since 2018-19:Q1.  The Index of Industrial Production (IIP), a measure of industrial performance, shows a y-o-y growth of 3.7% during the period April–May 2019–20, versus 4.1% during the corresponding period in 2018–19. The major component of IIP by economic activities, that is, manufacturing, shows a decreasing trend in production. The same is true for mining while the electricity sector shows an improvement. As regards the y-o-y growth of capital goods, after being in depression during 2018-19:Q4, it shows positive, albeit very slow, growth in April and May 2019. This signals that investments continue to be weak. Core IIP showed a y-o-y growth of 3.5% in 2019-20:Q1. However, in June 2019, it slowed down to a y-o-y growth of 0.2%.  The Nikkei PMI Manufacturing came down from 53.8 in 2018-19:Q4 to 52.4 in 2019-20:Q1.

While the annual data for the services sector shows decline since 2015-16, the quarterly data shows mixed patterns.  These mixed patterns of growth continued in 2019-20:Q1. Aggregate deposits and tourist arrivals showed a marginally higher positive growth in April-May 2019-20. In contrast, the goods traffic by Indian Railways, cargo handled at major ports, and cumulative addition in the total telephone and bank credit to the commercial sector showed weaker y-o-y growth in 2019-20:Q1. Production in the commercial vehicle and aviation sectors (including in both the domestic and international segments) showed significantly slower growth in the first two months of the current fiscal.  The Nikkei India PMI services Index fell from 52.6 in 2018-19:Q4 to 51.1 in 2019-20:Q1.

The inflation rate has largely remained benign in the last two years.  The Consumer Price Index (CPI) exhibited an upward trend in 2019-20:Q1, primarily owing to increase in food prices.  Core CPI continued its downward trend. Although overall Wholesale Price Index (WPI) inflation went downwards, WPI food inflation exhibited an upward trend in the last quarter. Overall inflation remained benign in the first quarter of 2019-20.

In the external sector, exports and imports have also shown a sliding growth pattern. The total Indian exports (merchandise and services) for the first quarter of FY 2019-20 amount to US$ 137.3 billion, attaining a growth of 3.14% on a y-o-y basis. The total Indian imports for the same period are valued at US$ 164.5 billion, exhibiting a growth of 3.6% on a y-o-y basis. Consequently, the overall trade deficit for April–June 2019–20 stood at US$ 27.2 billion.  While the trade deficit as a ratio to GDP has declined, the rupee has shown appreciation.

In the monetary space, the Reserve Bank of India (RBI) cut the benchmark policy repo rate to 5.75% in June 2019, its third consecutive cut since February 2019. With inflation well below the mandated target, the Monetary Policy Committee (MPC) switched to an accommodative policy stance, largely seen as being an indicator that the RBI is keen to support economic growth.  The data shows that despite a reduction in interest rates, there has been a weak transmission to either deposit or lending rates. Non-food credit showed a slower y-o-y growth of 12 per cent in 2019-20:Q1 versus 12 per cent in 2018-19:Q4. However, M3 y-o-y growth has remained virtually stable at 10.1% in 2019-20:Q1 versus 10.2% in 2018-19:Q4.

In the realm of public finances, the Net Tax Revenue rose by 6.1% in 2018-19 versus 12.8% in 2017-18. It is projected to grow by 25.1% in 2019-20. Meanwhile the 2019-20:Q1 shows a y-o-y growth of 5.9%. Both direct and indirect taxes showed slower growth in 2018-19 versus 2017-18, especially in the case of the latter. Direct taxes (income and corporate taxes) grew by 9.7% in 2019-20:Q1 on a y-o-y basis, and indirect taxes (including custom duties, excise taxes, and GST) grew by (-)3.0%. It is clear that there has been strong compression of Government Expenditure in 2018-19. The total Central Expenditure grew by 7.9% and the total Revenue Expenditure by 6.9% in 2018-19. In 2019-20:Q1, both grew by 2% and 6.1%, respectively.

The IMF projections for all major countries/groups show reduced growth in 2019 compared to 2018.  Sliding export growth is likely to persist.  Unrealistic tax revenue and expenditure proposals imply strong expenditure compression in 2018-19 likely to be followed by further expenditure compression in 2019-20.  With all autonomous demand drivers sliding i.e. exports, investment, government expenditure, growth in 2019-20 is likely to slide further.  Shift to expansionary monetary policy and further lowering of Repo rate may not be effective because transmission to interest rates is not working.  This is mainly because interest rates on alternative financial savings instruments like NSS etc. remain high.  Government may reduce NSS etc. interest rates or allow large slippage in fiscal deficit, but both are unlikely.  Therefore, growth may slip further in 2019-20 to 6% (±0.5%).

About the Quarterly Review of the Economy

The Quarterly Review of the Economy has been designed to meet the needs of policymakers, corporates, and others interested in tracking the latest developments in the Indian economy. It provides an analysis of current policies and tracks developments in both the domestic as well as the global economies. The growth forecasts of NCAER are objective and are widely quoted and referred to in both the Indian as well as international media. An integral part of the Quarterly Review are the State of the Economy seminars, organised quarterly at NCAER, which bring together policymakers, industry leaders, and researchers at one forum.

The Union Budget 2019-20: Reforms and Development Perspectives

This joint seminar brought together the Executive Directors of five institutions, namely, the Centre for Policy Research (CPR), Indian Council for Research on International Economic Relations (ICRIER), India Development Foundation (IDF), National Council of Applied Economic Research (NCAER), and National Institute of Public Finance and Policy (NIPFP) to present their more reflective assessment of the Union Budget 2019-20. NCAER was represented by its Director-General, Dr Shekhar Shah. The discussion and the Q&A with the audience was moderated by Puja Mehra, an Independent Economist.

The Union Budget in India remains the bellwether of where the government is headed with its economic policies. The 2019-20 Union Budget is particularly important since it suggests how the new government is looking at the next five years. But much of the immediate commentary and eager dialogue on TV and the newspapers takes a narrow, short-term view. As a counter, the executive directors of five of India’s leading economic policy research institutes came together in March 2007 for the first time to present their assessment of the longer term reform and development implications of the Budget. This joint seminar runs into its 13th year in 2019. The seminar was attended by a large gathering of researchers, policymakers, academicians, private and public sector experts and media.

As might be expected from a government starting its term and a new Finance Minister, the 2019-20 Budget presented on July 5 has generated much interest.  The slowing of the economy and the need to kick-start the investment cycle sits uneasy with the promise of fiscal consolidation and poor monetary transmission.  Other trade-offs reflected in the Union Budget include domestic compulsions such as ‘Make in India’ versus international engagement, bank recapitalisation vs the need for public investment, structural reforms vs handling short term exigencies and the political imperatives of the three upcoming state elections.  In the budget she has presented, the Finance Minister has tried to balance these and other needs, but her task remains an envious one.  Against this backdrop, the heads of the five institutes shared a thoughtful view of the Union Budget and its longer-term implications for the Indian economy under the leadership of the Modi-led NDA Government, now decisively in its second term.

    Get updates from NCAER