Interpreting evidence for policy

In the last decade or so, research in economics has moved towards randomized controlled trials (RCTs). In simple English, these are experiments in which policies are carefully structured and implemented in a way that the impact can be properly measured. These careful RCTs have thrown up a lot of interesting findings. Unfortunately, the way in which these findings are understood and interpreted by policy makers has sometimes been lacking.

Take access to finance for instance. In 2015, economists at the World Bank did an evaluation of the YouWin program. If you recall, as part of the program the federal government gave about $50,000 on average to some entrepreneurs who participated in a business plan competition. The economists found that entrepreneurs who got the grant had businesses that were much more likely to survive, ended up being bigger, were likely to employ more people, and were more profitable. All very good, but what does this mean for policy? You might be forgiven for thinking this means government should ramp up grants to businesses but that is not necessarily correct.

To understand why, the first thing to think about is costs. There is a tendency to assume that government money is free, but we know it is not. For any program one barometer for success is that the overall benefits must be larger than the overall costs. The YouWin RCT demonstrated that there are benefits but it says nothing about costs. As a policy maker you have to think about the costs of collecting extra taxes from the public or borrowing extra funding from financial markets. If more jobs are lost due to the extra taxes collected from other businesses for the program, or from the increase in interest rates due to borrowing more money from the financial sector, then the program is bad overall.

The second thing to think about is the substitution effect. Say the government decides that it wants to boost access to finance for rice or wheat farmers and implements a program to directly lend money to them. Sounds good, but what if because of the program banks who previously used to lend to the sector decide to pull back and not lend anymore? For instance, if your program gives out N150bn in loans to rice farmers but as a result the banks opt out of their usual N200bn in loans to the sector, then the program is bad overall.

The third thing to think about is sustainability. Can the program run continuously in perpetuity? What happens when the program stops? Governments change frequently and if history is anything to go by, new governments tend to not want to continue the programs of old governments. What happens to the sector when the program stops. Using the rice lending program again, if the program stops, will the banks go back to lending to the sector or will they do something else? If the effects of the program are not sustainable then the program may be bad overall.

Finally, the last thing to think about is scale. This is particularly important for a country as large as Nigeria. If I asked you to imagine a Nigeria in which the government gives out grants to all businesses who need finance, or a country in which the central bank gives out loans to all farmers, you probably won’t be able to. For context there are about 37 million farmers in Nigeria and maybe half as many businesses. The central bank lending to all farmers would probably make it the largest lender in all of Africa. A scenario which is practically impossible. If the program cannot scale then in terms of policy for access to finance, it may not be particularly useful.

The results from the YouWin RCT are very interesting and demonstrate that dealing with the problems around access to finance has potential to make a real difference. But the RCT does not mean that governments should launch direct lending programs all over the place. Expectedly, there have been a flurry of government programs trying to boost access to finance since that RCT. From GEEP to the ABP and more recently Trader Moni, all programs where the government, or the Central Bank in the case of the ABP, gives loans to some segment of the population. Unfortunately, it is not clear if any of these programs pass the “should we do it” test. These programs cannot replace financial sector policy, because ultimately the financial sector needs to work if the access to finance challenges are to be dealt with. Most importantly, the policy implications of RCTs in general need be thought of as carefully as the RCTs themselves.
Dr. Nonso Obikili is an economist currently roaming somewhere between Nigeria and South Africa. The opinions expressed in this article are the author’s and do not reflect the views of his employers.