Structural Model Estimation: Results and Interpretation

4.1 Model Specification

The econometric specification of the model requires scrutinizing the events and risks that can occur during the entire project cycle, and determining how these could affect project outcomes. The previous sections described these effects and risks; this section will attempt to empirically model them.

A schematic diagram of the risks that can lead to project distress and cancellation is shown in Figure 2 [ PDF 29.6KB | 1 page ]:

4.1.1 Typical Factors Leading to Project Failure: Macroeconomic Channels, Adverse Selection and Moral Hazard

Figure 2 suggests that the typical project failure (distress or cancellation) reported in the PPI dataset occurs through the following channels:

1. A macroeconomic crisis occurs, triggering a devaluation;
2. Depending on what the contract stipulates, currency risk is either borne by the project firm or by the consumers;
3. If it is borne by the firm, it bears the direct impact of currency risk; and
4. If it is borne by consumers, the country's highest government officials freeze tariffs.

Macroeconomic channels of risk are those primarily related to exchange rates, growth, openness, fiscal imbalances, etc. Note that these risks can occur sequentially in projects. If project outcomes can be affected by risks that are realized sequentially, this implies that in addition to non-structural models, structural econometric models can yield important insights into project failures. Thus, the analysis tries to examine the failed events themselves, to gain more insight into the substance of project failures. This is done using narrative accounts and a review of the existing literature on crises and their effects on projects.

Many of the macroeconomic channels of risk may magnify and propagate the effects of adverse selection and moral hazard throughout the project cycle, creating potentially serious incentive problems. For example, project planners are more likely to overestimate demand when economic growth is high during the planning and design phases. High growth during planning and design can lead governments to be less thorough when screening projects and proponents. High growth periods are therefore likely to exacerbate adverse selection, by attract riskier projects and riskier proponents to environments with less stringent controls and screening. Excessive demand forecasts during project design and planning can subsequently lead to larger project costs and subsequent losses, especially during the first few years of operations.

Similarly, errors in exchange rate forecasts are more likely to be committed when exchange rates are rigid or fixed during the design phase. These errors negatively influence project outcomes, as stakeholders fail to anticipate currency collapses that can occur during the operations phase—an indication of moral hazard.

4.1.2 Other Channels: The Role of Project Planning, Design and Contracting

Figure 2 identifies other channels through which risks can affect project outcomes. As described earlier, weaknesses in project planning, design and contracting (such as excessive demand forecasts, severe risk misallocation in contracts, and underestimation of project risks) can also contribute to project failure.

The typical PPP experience in Figure 2 has implications on specifying the econometric model for estimating the determinants of PPP risk. First, political events such as the freezing of tariffs can be the direct outcomes of observable macroeconomic shocks. Second, political events such as contract cancellations and nationalizations, can also be endogenous with respect to institutional and contractual weaknesses; these include weak and non-transparent procurement systems, and contracts that pass on too much risk to government.

All of these different factors may impair the firm's value.¹⁰ Given the often complicated evolution of and relationships between risks and project outcomes, a structural econometric model is therefore appropriate for this study:

Project outcomes (fail or not fail) = f(various endogenous and exogenous factors), with endogenous variables a function of the instruments.

The model lends itself to probit, logit, multinomial, and ordered discrete dependent variable regression techniques.¹¹

4.2 Data Description

The variables used in the regressions are described in Appendix A. Descriptive statistics for the major variables are listed in Appendix B. Figure 3 [ PDF 26.9KB | 1 page ] depicts the general structure of the crosssection data for this study.

Two sets of data are required to estimate the model. The first set is project-specific data, available from the PPI database of the World Bank. The second set includes data that would allow the estimation of various risks that affect a broader set of projects over time.

Since the PPI dataset is cross-sectional, the project information in it is limited to project-specific data at the time the contract was signed, such as the value of the investment commitment, the sector, and the identity of multilateral creditors. However, for each cancelled or concluded project, the year of financial closure and year of cancellation or conclusion are also listed. For projects that are currently operational, macroeconomic conditions during the last few years of operations can be captured. Thus, for each and every project in the PPI dataset, it is possible to capture economic conditions that were prevailing during the project design and operations phases. This allows one to get a sense of how macro conditions affect stakeholder psychology, in the sense that forecasts are affected.

It is difficult to empirically model the wide variety of project risks described in Section 3 and Figure 2, for a number of reasons. First, it is impossible to find global, project-specific data for a broad category of risks; information related to demand risk or currency risk is not available from the PPI dataset or any other source. Second, the extent of these risks is directly proportional to assumptions and forecasts made by stakeholders during the time of contracting; however, data on these assumptions and forecasts are likewise unavailable on a global scale.

In the absence of global, project-specific data, one must rely on creativity and use available data as proxies to capture the impact of these risks. Globally observable macroeconomic data can give one a sense of economic conditions prevailing in two key periods of the project cycle—the project's design phase, and the project's operations phase (see Figure 3 [ PDF 26.9KB | 1 page ] ). It is highly possible that the macroeconomic environment prevailing during the former period influences stakeholder forecasts of growth and exchange rates during the latter period, and this is part of what this study aims to capture.

4.3 Estimation Results

Given the structural nature of political risk and other risk factors, a two stage instrumental variable probit procedure was used for estimation.

In the first stage, endogenous variables were regressed on instruments. Table 6 [ PDF 26.1KB | 1 page ] summarizes the endogenous variables and exogenous or predetermined variables) used in the benchmark model.

The residuals of the first stage were used in the second stage regression, with FAIL as the dependent variable (FAIL is a binary variable that takes the value 1 when a project is listed as distressed or canceled in the World Bank's PPI database and 0 otherwise).

Regressions with highly significant variables are displayed in Table 7 [ PDF 28.3KB | 1 page ] (variables that do not appear in regression results are in large part insignificant).

Projects at the construction phase were excluded from the sample, as no conclusions may be drawn by analyzing them. Wald tests of exogeneity were performed to determine whether there is enough information in the equation to reject the null that there is endogeneity. Significant p-values indicated that a structural estimation was appropriate. In addition, it did not appear that multicollinearity posed a problem for the estimates. First of all, the size of the sample mitigated the effects of any multicollinearity. Second, cross-correlations between the variables were low (see Table 8 [ PDF 26.9KB | 1 page ]).

4.4 Interpretation of Results

A summary of the empirical results is listed in Table 9 [ PDF 30.5KB | 2 page ].

4.4.1 Variables That Tend to Raise the Failure Rate of PPP

Equation 1 in Table 7 is the baseline regression. In the succeeding equations, explanatory variables were added individually.

As expected, stress and failure are positively associated with tariff freezes (their mere imposition, as well as a longer duration raises the failure rate), confirming the strong adverse impact of political risk on project outcomes.

The estimates in equation 1 also reveal that higher average real per capita GDP growth and lower volatility in of the real exchange rate six years prior to financial closure (i.e., prior to the operations phase of a project) raise the PPP failure rate. This tends to confirm the hypothesis that macroeconomic conditions during these phases lead to moral hazard and adverse selection, influencing the subsequent incidence (and depth) of project stress during the operations phase—high growth and more rigid exchange rates during the design, planning, contracting, and screening phases all lead to incentive problems and more adverse project outcomes.

That macroeconomic conditions prior to project financial closure significantly affect subsequent project outcomes is a strong sign that adverse selection exists throughout the project cycle. The data would show that countries with high per capita growth rates and perceptions of good governance attract riskier proponents and projects with excessive and overly-optimistic demand forecasts. These are the projects which subsequently fail during the operations phase.

High growth rates may also: (i) attract more rent-seeking investors; (ii) create a rapid buildup in demand for infrastructure, causing countries to fast-track their procurement procedures for PPP (which is generally good for investment unless done at the cost of due diligence); and (iii) lead to situations where opportunistic governments subsequently extract rents from infrastructure investments that were initially sunk.

Most importantly, however, strong growth and rigid exchange rate regimes may contribute to adverse selection and moral hazard by creating a psychological environment in which financiers, project designers, planners, government executives, and consumers underestimate currency risk and discount the importance of the exchange rate as a major determinant of prices¹², both during the project design phase and during the operations phase. When a large and sudden exchange rate adjustment makes it necessary to have a proportional adjustment in price, government executives are suddenly faced with the task of choosing between the firm and the consumer. This predicament raises political risk dramatically.

Rigid exchange rate regimes may also be incompatible with long-term infrastructure investments. It is unlikely that the rigidity in the exchange rate can be sustained without some large adjustment within the investment horizon for PPP. Indeed, the probability that any large macroeconomic shock can hit the economy is much higher under long-term projects. Political risk is inherently greater, the longer the investment horizon.

Moral hazard is exacerbated by the lack of accumulated fiscal risk monitoring, as well poor coordination between government agencies implementing PPP. Moral hazard can also be exacerbated by investors and multilateral institutions that are overly enthusiastic about countries with rapid per capita income growth.

Measures of explicit government support and guarantees also tend to raise risk of failure. This provides additional evidence of moral hazard and adverse selection in projects, and is consistent with Woodhouse's (2006) analysis of anecdotal evidence from projects.

All PPP projects, even those undertaken without government guarantees, come with a certain level of implicit and explicit fiscal risk. This is apparent from a number of nationalizations and unforeseen bailouts of PPP projects, and is confirmed by the regression results.¹³ That all PPP projects carry large fiscal risks with immediate impact, clearly demonstrates that countries face binding liquidity and fiscal capacity constraints with respect to PPP. The risk of project stress and failure also rises with an increase in the average shortterm debt-to-exports ratio, as well as in the average fiscal deficit. The extent to which a sector's capacity is supplied by PPP also contributes to fiscal risk, as experiences with IPPs tend to demonstrate. These results suggest that some form of soft (or even hard) “PPP restraints” may be necessary, especially in fast-growing countries with problems in interagency coordination and information flow. These countries may have limited ability to coordinate agencies to support PPP or to monitor fiscal risk.

An increase in the number of PPP projects also tends to raise project stress. Since PPP requires some fiscal space and monitoring capability ex ante, the number of projects has profound implications on the fiscal sustainability of PPP in a country in general.

4.4.2 Variables That Tend to Lower the Failure Rate of PPP

Macroeconomic conditions (real growth and exchange rates) do not only influence project design and planning, they also have a profound impact on the operations phase of a project. As expected, average real per capita GDP growth in the six years prior to the terminal or current year is associated with lower project failure. This is intuitive, as high growth during the project operations phase raises demand (and revenues) for any PPP project. An appreciation in the average real exchange rate in the six years prior to the terminal or current year during the operations phase of a project also leads to favorable outcomes. This is likewise expected, given the high import content (and foreign debt leveraging) of PPP projects. Real appreciations make imported inputs cheaper, leading to reductions in debt servicing burdens.

Fiscal surpluses during the operations phase also lead to favorable project outcomes, as governments have the fiscal space to lend limited support to projects. A higher degree of trade openness likewise reduces the risk of project stress. This may be due to the fact that countries tend to strategically situate trade-supporting PPP (roads, seaports, airports, etc.) in a manner that mitigates demand risk (i.e., close to export zones). It could also be that efficiency-enhancing infrastructure is simply valued more in open economies.

Broadly-defined political risk includes executive-instigated tariff manipulation and tariff freezing. These appear to be within the capacity of existing political risk guarantee instruments (PRGs).¹⁴ Except for PRGs, no other instrument (loans, risk management services, or equity) from multilateral and bilateral financial institutions appears to be useful in mitigating stress. It is possible that providers of PRGs provide good advice, which is additional support apart from the PRG itself. Because of their benefits, PRGs should be more accessible to stakeholders. PRGs should be strengthened to address many of the issues identified here, such as other manifestations of political risk, like tariff-freezing. Given the pervasiveness of tariff freezing, the fact that PRGs are utilized by only 3% of the projects in the World Bank's PPI dataset implies that stakeholders grossly underestimate the value of PRGs (or some form of mis-targeting in PRGs occurs). It may also mean that PRGs are simply too expensive to purchase. An initial survey of PRGs being offered by private and public agencies, as well as multilaterals, suggests that many of the risks mentioned in this study may be currently eligible for PRG cover, but affordability is a major hindrance to enhancing demand.

The coefficient of the dummy for contracts transacted with federal governments (FEDCON) was significantly negative, implying that transactions with local governments were riskier. This confirms the earlier hypothesis concerning the advantages of contracting with higher levels of government.

Rate of return regulation lowers risk, suggesting that price cap regulation may be too inflexible during periods of crisis, leading to the demise of many projects. While achieving efficiency in project design and operations is desirable in the long-run, frequent price adjustments may be needed to respond to a crisis as it evolves. Here, the clash between trying to balance affordability and public and political acceptance and project viability is most intense. The best response under these circumstances appears to be orderly coordination and thorough workout of issues within and across projects, instead of outright confrontation between government and private developers.¹⁵ Subsequent mitigation should be held off until the situation normalizes.

Foreign direct investment (FDI) likewise reduces project risk, suggesting that the ability of foreign investors to raise capital and bring in technology helps projects. However, having foreign proponents can also increase project financing costs and risks, because (i) the required returns (to cover the cost of capital) are benchmarked in foreign currencies; and (ii) the capacity to manage domestic political and social risks is lower. This may be why Guasch (2004) found that renegotiations occur more often when FDI is involved in concessions. In addition, devaluations in the host country will have generally negative effects on foreign investors' balance sheets, even if there are bearable effects on demand and tariffs.¹⁶

4.5 Sectoral Analysis

PPP in the upstream power sector (distribution and transmission segments) as well as water treatment and sewerage and water utilities, seem to be associated with higher project failures. These segments tend to be more politicized than other sectors, and are therefore more vulnerable to political risk, tariff freezing, and subsequent failure.

Rehabilitation projects are associated with lower risks of failure, as are power generation (perhaps because of its sheer necessity), and seaports (because of the natural demand for seaports created by commerce and trade).

Relative to concessions, divestitures, and merchant projects, greenfield projects and management contracts are associated with higher failure rates. While coefficients for divestitures and merchant projects were not conclusive, the coefficients for concessions suggest favorable project outcomes in general.

The sectoral results suggest that, with some exceptions, upstream PPP and PPP situated in new markets are associated with higher failure rates.

4.6 Results of Other Qualitative Analysis Conducted for this Study

A qualitative analysis¹⁷ of canceled and distressed projects suggests that firm-embodied traits explain a substantial amount of the observed outcomes in PPP. For example, the success of water concessions hinges greatly on the firm's ability to rapidly reduce system losses at the beginning of the concession period. This in turn depends on in-house management capacity, employee efficiency, innovative ability, and the overall quality of corporate governance—factors not captured by the empirical data. A firm's ability to innovate and increase efficiency and productivity (even if not necessarily subjected to price cap regulation) may also help temper pressures to raise prices. Thus, when designing policy for PPP, governments could concentrate on mechanisms that encourage the attainment of efficiency at the beginning of the operations phase, the most failure-prone part of the project cycle.

Important information can also be gleaned from those variables that were not significantly related to PPP project outcomes (or had perverse signs, such as the World Bank's governance indicators). The type of PPP, such as whether the project was structured as build-operate-transfer, or build-operate-own, did not affect project outcomes. Likewise, empirical results suggested that patterns of ownership and control preferences, such as the dummy variables for build-operate-transfer (BOT), build-operate-own (BOO), etc., were not significant.

4.7 Does the Quality of Governance Determine PPP Investment Outcome?

While the quality of country governance (as measured by the World Bank) can influence the pattern of PPP investment flows,¹⁸ this did not directly explain favorable PPP investment outcomes in the main empirical model. Interestingly enough, many of the stressed projects were located in countries with relatively high scores in governance criteria. When the World Bank's governance indicators were individually entered into second stage probit regressions, they yielded insignificant coefficients or coefficients with perverse signs (i.e., they raised failure rates). The positive correlation between good governance criteria—government effectiveness, control of corruption, political stability, and rule of law—and stress in PPP implies that a vastly different governance paradigm for PPP should be contemplated. Good governance in PPP includes having good macroeconomic policies to prevent shocks that may lead to adverse political decisions.

Nevertheless, governance indicators play an important role as instruments for the other endogenous variables in the empirical model. Of particular interest is the role governance plays in tariff freezes. Table 10 [ PDF 27.2KB | 1 page ] shows the results of a typical first stage regression for the tariff freeze variable (TARIFFFRZ2). The results suggest that the probability of a tariff freeze is greater: (i) the lower the extent to which a nation's citizens can select their government, and enjoy freedom of association, the press, etc. (VOICE); (ii) the more vigorous the enforcement of rule of law (RULE); and (c) the greater the extent of corruption (CORRUPT). Other governance indicators such as regulatory quality (REGQUAL) and government effectiveness (GOVEFF) tended to perform perversely in the first and second stage regressions.

In addition, tariff freezes are more likely, the greater the number of projects implemented by the country.

4.8 Accounting For Other Aspects of Political Risk

When the other binary political risk variables, YRSFRZ (duration of tariff freeze during the investment horizon) and POLSTRESS2 (whether or not projects experienced a tariff freeze or had witnessed a renegotiation or tariff freeze within the first two years of a change in political leadership), replaced TARIFFFRZ2 in the second stage regression, they yielded similar results. The significance of POLSTRESS2 implies that regime change can be a catalyst for project failure. This is significant, given that over half the projects in the PPI dataset were affected by renegotiations and tariff freezes occurring within the first few years of a regime change (see Table 11 [ PDF 27.2KB | 1 page ]). A similar proportion of projects experienced regime change during their most vulnerable years (see Table 12 [ PDF 27.2KB | 1 page ]).

The analysis further revealed that endogenous political risk is also associated with poor procurement systems, poorly managed state-owned utilities, and even poorly designed PPP contracts.

While countries can take steps to address endogenous political risks, political risks exogenous to the country cannot be controlled. In this regard, MFIs should monitor the regional and global economy and the regional and global geo-political environment to forestall the effects of such exogenous risks. The impact of exogenous risks can be mitigated by strengthening consultation during the design phase. Improving project approval and procurement systems would also be beneficial, as this would reduce the risk of subsequent project rejection or cancellation.

4.9 Comparison of Results with Findings in the Existing Literature

It is useful to contrast the results of this study with those of Guasch (2004) (Table 13 [ PDF 26.4KB | 1 page ]). Note that results of the renegotiation study by Guasch holds for cancelled and distressed projects in general, with two important exceptions—governance quality and the existence of regulatory bodies matter for reducing renegotiations, but not as much for reducing failures.

Download this Paper [ PDF 269.4KB| 58 pages ].

Comment(s)

There are [0] comment(s) for this entry. Post a comment.