The Poverty Impact of Microfinance

A perfect impact evaluation really needs to answer a counterfacutal question: how does the status of participants in the program compare with how those same individuals would have fared in the absence of the program? Or, alternatively, how would non-participants have fared in the presence of a program? The problem with cross-sections of data (observations on many individuals at a given point in time) is, of course, that at any given point in time individuals are observed to be either participants or not. Even panels of data (observations on many individuals through time) are problematic since over time many other things have happened to the individuals in addition to program participation and it is nearly impossible to separate out the impact of the program from all the other influences. In reality, researchers must settle for estimates of the average impact of the program on a group of participants – the treatment group - to a credible comparison group – a control group. The ideal control group is individuals who would have had outcomes similar to those in the treatment group had they not participated in the program.

But constructing a control group comparable to the treatment group is not straightforward. Participants in the program are usually different from non-participants in many ways: programs are usually carefully placed in specific areas, participants within those areas may be screened for participation, and the final decision on whether or not to participate is usually voluntary. To the extent that these factors are known and can be measured, they can be controlled for in the empirical analysis, but in most cases the placement of the program and self-selection of participants in those areas into the program are based on unobservable factors. These unobservable factors lead to at least two kinds of bias in any empirical impact evaluation: program placement bias and self-selection bias.

Controlling for this bias – determining the effects of just microfinance and separating out the impact of microcredit from what would have happened to the same household without credit - is often the most difficult part of careful empirical impact studies. Wellrun microfinance institutions do not randomize either the location of their operations or their selection of clients. If microfinance institutions tend to operate in areas that have relatively better or worse infrastructure such as access by roads, or more or less active markets, then estimates of the impacts of the program on participants do not measure the effects just of microfinance, but of these other factors as well. Even within a given village, if, as studies by Coleman (2002), Alexander (2001) and Hashemi (1997) suggest, microfinance clients already have initial advantages over non-clients, then the impact of microfinance will be overestimated if these initial biases are not controlled for. Similarly, the impact of microfinance programs that deliberately target relatively disadvantaged households in the areas they operate may find impacts underestimated if these biases are not controlled for.

Despite the importance of thinking carefully about these issues, few studies have addressed them rigorously and for good reason, as rigorous quantitative studies, among other limitations, are costly and time consuming.⁸ Few microfinance institutions have the resources in terms of funds or staff-time to conduct them. There is a movement in the industry to create practitioner-friendly assessment tools (for example, the Imp-Act project based at the Institute of Development Studies at Sussex, USAIDs AIMS project and assessment tools by CGAP), but these assessments, while very useful to the institutions themselves in refining their targeting, products and marketing, are not rigorous quantitative measures of impact and do not adequately address the issues of selection bias.⁹

Armendariz de Aghion and Morduch (2005:pp 238-239) provide a compelling argument to make the substantial investment required to conduct careful impact studies that control for these potential biases:

“Unfortunately, this is not an esoteric concern that practitioners and policymakers can safely ignore. It is not just a difference between obtaining “very good” estimates of impacts versus “perfect” estimates--the biases can be large. In evaluating the Grameen Bank, for example, Signe-Mary McKernan (2002) finds that not controlling for selection bias can lead to overestimation of the effect of participation on profits by as much as 100 percent. In other cases …controlling for these biases reverses conclusions about impacts entirely.”

There are a handful of studies that rigorously address the issues of selection bias and endogeneity. The approaches of Pitt and Khandker (1998), Hulme and Mosely (1996), Coleman (1999), and work in progress by Banerjee and Duflo are discussed below.

Exogenous Eligibility Requirement

In an innovative approach to controlling for selection bias, Pitt and Khandker (1998) combine the use of a quasi-natural experiment and eligibility requirements to study the impacts of the Grameen Bank, Bangladesh Rural Advancement Committee (BRAC) and Bangladesh Rural Development Board (BRDB). The authors sample 1538 participants and 260 non-participants in a number of ‘treatment’ villages where group-lending programs are operating as well as randomly selected households from ‘control’ villages without a program. They use village fixed effects to correct for endogeneity of program placement and take advantage of the fact that the microcredit programs impose eligibility requirements on participants (households with land holdings of more than half an acre are ineligible) to construct eligible and ineligible households in the control villages. Impact is assessed using a difference-in-difference approach between eligible and ineligible households and between program and non-program villages. After controlling for other factors, such as various household characteristics, any remaining difference was attributed to the microfinance programs.

The study draws a number of conclusions, but the main one is that the program had a positive effect on household consumption, which was significantly greater for female borrowers. On average, a loan of 100 taka to a female borrower, after it is repaid, allows a net consumption increases of 18 taka. In terms of poverty impact it is estimated that 5% of participant households are pulled above the poverty line annually.

The accuracy of the original results as presented in Pitt and Khandker (1998) has been disputed on the grounds that the eligibility criteria of low land holdings was not enforced strictly in practice. In a reworking of the results focusing on more directly comparable households, no impact on consumption from participation is found (Morduch 1999:1605). This debate, which in part centers around details of econometric estimation, has not been resolved. An unpublished paper by Pitt reworks the original analysis to address the concerns of Morduch and is said to confirm the original results (Khandker 2003, footnote 1).

Prospective Clients as Control Group

Another approach to controlling for self-selection and placement bias, used by Hulme and Mosley (1996) and Coleman (1999) is to include a sample of microcredit clients who have formed solidarity groups but have not yet received loans as the control group. In this approach, participating and non-participating households are again surveyed in treatment villages where the microcredit program is already operating and has already given loans. The control villages are villages where the micro credit program will operate and households from the village have already self-selected to participate in the program but have not yet actually received loans.

Hulme and Mosley (1996) employ this approach in their study of programs in a number of countries including the Grameen Bank in Bangladesh and the Bank Rakyat Indonesia (BRI). In general a positive impact is found on borrower incomes of the poor with on average an increase over the control groups ranging from 10-12% in Indonesia, to around 30% in Bangladesh and India. Gains are found to be larger for non-poor borrowers, however, and within the poorest group gains are negatively correlated with income.

However, despite the breadth of the study and its use of control group techniques, Hulme and Mosley’s study fails to control for program placement bias, so part of the advantage of program participants relative to the control group may be due to unmeasured village attributes that affect both the supply and demand for credit.¹⁰

Coleman (1999) advances the literature by expanding on this concept to control for selfselection bias and introduces both observable village characteristics and village fixed effects to control for program placement bias in his study of a village-banking program in Thailand. Utilizing data on 455 households, including participating and non-participating households in treatment villages where a village bank is already offering microcredit and selected future participants and non-participants in control villages that have been identified to receive a village bank program but have not yet actually received funds, Coleman uses a difference-in-difference approach that compares the difference between income for participants and non-participants in program villages with the same difference in the control villages, where the programs were introduced later.

Coleman’s study measures the effects of access to rather than participation in a microcredit program and finds no evidence that months of access to a village bank program has an impact on any asset or income variables and no evidence that access to village bank loans increased productive activity. The author cautions, however, against extrapolating these results to other contexts since Thailand is a rather wealthy developing country. One of the reasons there is a weak poverty impact is that there was a tendency for wealthier households to self-select into village banks, and the relatively small sizes of loans may mean that they were largely used for consumption.

This approach as well is not perfect. Karlan (2001) points out that this approach still fails to correct for possible attrition bias – the fact that the control group includes potential future dropouts (or graduates) of the program, while the treatment group of older borrowers (who have in fact remained active borrowers) does not. Depending on the reasons for attrition, attrition bias can be positive or negative. If attrition is due to successful clients graduating out of microfinance into the formal financial sector, then impact will be underestimated. If attrition is due to dropouts who find the program unhelpful or whose microenterprises fail, for example, then impact will be overestimated. Armendáriz de Aghion and Morduch (2005) review a number of studies that find dropout rates between 3.5%-60% per year in various microfinance programs worldwide. Even the lower-end estimates can add up to a substantial effect over time.¹¹

Randomized Program Design

There are a few very recent impact studies underway that use randomized study design to control for selection bias. Duflo and Kremer (2003) describe the use of this type of evaluation for an educational program in Mexico. Banerjee and Duflo (in progress) will apply this approach to a microfinance impact assessment for the Center for Micro Finance Research (CMFR)). This approach eliminates selection bias by randomly selecting treatment groups (those who receive microfinance) and control groups (those who do not) from a potential population of participants. With this type of study design, the researcher can be assured that on average those who are exposed to the program are no different than those who are not, and thus that a statistically significant difference between the groups’ outcomes can be confidently attributed to the program, not to selection bias.

Well-designed studies of this sort have the potential to rigorously address all kinds of potential biases, although they are limited by the fact that they can only estimate partial equilibrium treatment effects, which may differ from general equilibrium treatment effects. In the case of microfinance, this means that if, for example, microfinance is introduced on a large scale, the program could eventually affect the functioning of financial markets and thus have a different impact than the necessarily smaller scale program introduced for the impact study.

A more practical concern in attempting to apply randomized study design is that such studies require tremendous cooperation from the institutions being evaluated; they must be willing to allow researchers to randomize implementation of their services. Such studies must also be longitudinal, making them costly, and it can be difficult to conduct research over a time period long enough for some impacts to show up. In the case of Banerjee and Duflo’s study for CMFR, the time frame between base line and final study is one year, which may not be long enough for some of the impacts of microfinance to show up quantitatively. For these reasons randomized studies are likely to continue to constitute only a tiny fraction of all microfinance evaluations.

Comment(s)

There are [1] comment(s) for this entry. Post a comment.

1. Khan Hidayat Ullah
   (posted 23 November 2006 / 10:14:28 PM)
   
   Econometric Model Selection:
   
   I went through your paper and really liked the approach adopted by you while found your results interesting as well.
   
   I have one comment to make. When it comes the components of microfinance pertaining to the term of loan, e.g., amount, duration and repayment mode, in practice these factors are determined by the both the borrower and lender through negotiation between line staff of MFI and Managers of CBOs. This results into adoption of resolution by members in joint meeting held by field staff MIF and CBO.
   
   Keeping these factors in view, the claim that an explanatory variable relating to terms of micro-credit is exogenous becomes doubtful. This creates endogenity problems in the model and the results which are estimated by this model can not be claimed as unbiased.
   
   In you model you have used variables like amounts of loan and times of loans borrowed in past, which brings in endogenity problem in the model and leads to biased estimates, if I am correct? I would like to know how you addressed this problem in your econometric model. If not what kind of instrumental variable you intend to use in order to deal with situation will.