Comparison of Fractional Splines with Polynomial Splines; An Application on under-five year’s Child Mortality Data in Pakistan (1960-2012)

Cubic splines are commonly used for capturing the changes in economic analysis. This is because of the fact that traditional regression including polynomial regression fail to capture the underlying changes in the corresponding response variables. Moreover, these variables do not change monotonically, i.e. there are discontinuities in the trend of these variables over a period of time. The objective of this research is to explain the movement of under-five child mortality in Pakistan over the past few decades through a combination of statistical techniques. While cubic splines explain the movement of under-five child mortality to a large extent, we cannot deny the possibility that splines with fractional powers might better explain the underlying movement. Hence, we estimated the value of fractional power by nonlinear regression method and used it to develop the fractional splines. Although, the fractional spline model may have the potential to improve upon the cubic spline model, it does not demonstrate a real improvement in results of this case, but, perhaps, with a different data set.


Introduction
Regression is the study of dependency. Regression analysis is based upon the study and analysis of relationships among various variables. Obviously, it is often applied indiscriminately to data with no specific objective as a main priority. The classical linear model is represented by the following equation. It is useful, however, to recognize two of the fundamental purposes for which linear regression is valuable. The main principle motivation behind regression is to give a summary and reduction of the observed data keeping in mind the end goal to investigate and present the relationship between the configuration variable x and the reaction variable y. The other main purpose of regression is to utilize the model for forecast. While, prediction is no doubt an essential part of regression, it is most likely a much more precise reflection of the statistical practice to consider regression fundamentally a model based system for data outline.
Among various statistical techniques, linear regression is preferred for analysis due to a number of reasons. Some of these include its simplicity, flexibility for the choice of deduction method and easy-to-understand techniques. It has widely been used in literature and has given reliable results. However, the method cannot be used if two or more variable are non-linearly related. The method cannot be applied to analyze varying time series data. To analyze such a data, a plot of data over time must be obtained and then be examined for non-linear trends. There are many approaches for estimating nonlinear trends. One of them most popular technique is piecewise polynomial regression splines.
The term "spline" originates from the tool utilized by the shipbuilders and drafters to manufacture smooth shapes having desired properties. Mathematically, a spline is a piecewise function represented by polynomials. The function has a high degree of smoothness at the nodes of polynomial functions. In computer science, the term spline more frequently points out a piecewise polynomial curve. Spline regression models which are also called piecewise or segmented line regression models or broken stick regression models framed of continual linear stages. Despite the fact that spline regression models may sound like something tricky, they are much same as dummy variable models with a couple of constraints set on them. For example, if a person is gaining weight over time, but suddenly decides to lose weight, then with liposuction, there is an instantaneous drop in his or her weight at that moment the decision is made to lose weight. The person's weight could serve as dependent variable in a regression while time is an explanatory variable. There will be a gap between the regression line before and after liposuction. Using unrestricted dummy variables, the model after liposuction may have a different slope and intercept rather than the model before it.
Economic analysis regularly involves circumstances where one is required to investigate the effect of unexpected changes in the data. The 2010 earthquake in Chile and 2011 earthquake in Japan are two of the most important examples. In both the cases, an intermediate, persistent effect was observed for the product availability. Spline regression models searches for points in the data that would identify where these changes happen. These points are named as "knots". Spline regression models give a method for capturing these changes smoothly and joining the segments without the usual break between the segments. Thus, in a spline model, a turning point in the product availability could be represented by a spline knot, which may join the upward regression line to the downward regression line. This type of spline model is often called as a piecewise regression model (Pindyck et al. 1998).
An undeniable inquiry is the reason not to utilize a polynomial regression model rather than splines? Interestingly, spline regression models or piecewise polynomials have considerably additional flexibility than polynomial regression models in low measurements and are mostly less liable to generate perfect multicollinearity in low extent. Other methods such as kernel regression can be used but Carroll (2000) reported that spline methods are generally more efficient than kernel methods.
The credentials of variations in the recent development are vital concerns in the analysis of a data related to mortality and incidence of a disease. Recently, Kim et al (2000) applied segmented line regression to illustrate the continuous changes in cancer mortality and incidence rates. We apply a join point regression model to describe such continuous changes of infant mortality rates data in Pakistan. Tracking the infant mortality rate over time is a good variable for using splines. The set of real numbers is much larger than the set of positive integers. One can allow coefficients and exponential power that is the fractional value (e.g 0.5 or whatever) for each spline knot.
In present research we find the number and location of spline knots where the regression line pivots by considering a new slope. In this regard, we initially fit the cubic spline and estimate the knots and then used these estimated knots in fractional splines (i.e. only degree of the polynomial is changed by some fractional number) to determine the better fit of the model. In last, we compare the fractional spline results with the polynomial splines.
Searching for the location of spline knots by creating a potential adjustment variable for every possible time period (54 in this case) allows one to statistically search for significant adjustment points by using an appropriate technique to locate one or more such points. The first step is to generate the spline adjustment variables. One can use the "+" functions or dummy variables to set up spline adjustment variables. The dummy variable, D, is needed to create a first derivative break at the point where X = K. This creates a kink in the line at X = K. We are not trying to change the overall slope of the line throughout the entire length of the line, but instead create a kink in the line at X = K. The dummy variable plays a critical role in accomplishing this. Similarly the impact of each Year will be captured in the subsequent years. Since we are not sure of the degree of impact of each of the years on the outcome for the subsequent years, we have also taken Quadratic variables (Q1, Q2, Q3, … Q54) and also Cubic variables (C1, C2, C3 … C54).
. . C54=(L54) 3 We create linear, quadratic and cubic splines adjustment and also use year square and year cube for analysis. As we have so few observations it makes sense to restrict the model to fit more smoothly between points. Therefore, it makes sense to run a cubic spline model, which only allows third derivative adjustments. The problem here becomes even more complicated in that we do not even know the number of spline knots. Our general strategy for dealing with this problem is to create a large number of potential spline knots and then use stepwise regression to pick out those that are most statistically significant. Of course, we have to specify the significance level that determines when to stop adding additional spline knots to the stepwise regression.
To convert cubic spline results from the stepwise regression into fractional splines requires treating the exponent which is number "3" in cubic splines into an unknown parameter to be estimated by the nonlinear regression. For estimating fractional spline regressions, one can use any nonlinear regression estimation technique such as Gauss Newton or Newton-Raphson.
This study is conducted on the data of under-five mortality in Pakistan from the period 1960 to 2012. Data for this study has been obtained from secondary source, World Bank.
Here is the link for data download: http://data.worldbank.org/country/pakistan.

Results and Discussion
The first objective of the study is to search for the number and location of spline knots (join points). For our study, we are using the pattern recognition approach. Pattern recognition creates explanatory variables that just fit the pattern of data as a function of time. The research problem here becomes more complicated in that we do not even know the number and also the location of spline knots. We deal with this problem by creating a large number of spline knots and then we use stepwise regression to pick out those which are most statistically significant.
We create a set of dummy variables by using R statistical package. The following mathematical explanation to understand the dummy variables is given as under.

 
It makes sense to run a cubic spline model, which only allows third derivative adjustments. Therefore, we have considered year, year squared, year cubed and cubic spline adjustment variables. Given the large number of variables (57 variables), we have done stepwise regression to select only significant variables that have impact on the under-five deaths. The stepwise regression process starts with an empty model (Null Hypothesis that under-five mortality is not dependent on any of the variables). From this step, it will include any independent variable that has a significance level of below 0.50 (entry criteria). Once a variable is inside the model, the variable will stay inside as long as the significance level is below 0.15 (stay criteria). So at the end of the Stepwise Regression, we have a set of variables (including the knots) that have dependence on under-five child mortality. The stepwise procedure comes to an end at step 23. After this stepwise regression analysis with cubic spline adjustments, now we need to plot a couple of them to get a sense of how realistic they are. We Plot the fitted equation against the observed sample points using the results from different steps of stepwise output.
By comparing the models of different steps of stepwise regression procedure, we have come to the conclusion that model of step 10 looks adequate. The output for step 10 is shown in table 1 and 2. The F-value is greater than as in step 4 and R squared is now over 0.99 and all the regression coefficients are highly significant. There are now 7 spline adjustment variables in the model. Notice that the coefficients alternate in sign. This means that both upward and downward movements have a tendency to go too far. This tendency can complicate the problem of forecasting with spline models beyond the scope of the original data. Our predicted regression curve is represented among the actual values of under-five child deaths in the graph in Figure 3. The predicted values are approximately near the actual values. This graph shows considerably more sensitivity to the data and therefore greater flexibility. A higher step might be needed if we are fitting a 3 dimensional spline to search for caves in a mountain terrain but for simple policy purposes, there is no need to go beyond step 10. Determining the proper fit for each problem depends very much on the purpose of the analysis.
Step 10 shows a much flexible model and in comparison with step 23, it only uses 7 spline adjustment variables.   The second main objective of our research is to fit a fractional spline regression model. To convert cubic splines into fractional splines, we consider that the number of knots and their location is already known. We have estimated the knots by considering it a nonlinear regression problem from stepwise regression procedure in the first objective of our research.
The problem here is to deal with the estimation of degree of the polynomial which is "3" in cubic spline regression model. We estimate this by using Newton Raphson method which have implemented using SAS Programming language. We use year 1972, 1984, 1990, 1997, 1999, 2005 and 2010 as the knot years already known which we have estimated from cubic spline stepwise regression procedure. Thus, we created a SAS code for this.
In the code, the first nonlinear least squares regression starts out with the initial value a=3 representing cubic splines, but allows the nonlinear estimation to search for a fractional power that replaces the "3" with some fractional number that provides a better nonlinear fit to the data. Thus, fractional splines are estimated by turning the exponents into parameters to be estimated as part of the regression equation.
In nonlinear regression, Taylor Series Expansion has an important role to play. A typical nonlinear regression estimation method is to expand the nonlinear regression equation in a Taylor series around initial guesses for the value of the parameters. Then, right after the linear term chop off the rest of the Taylor series, and then run ordinary least squares estimation on the linear part to estimate new estimates of the parameter. Next expand the Taylor series around the new parameter estimates and chop off the higher order terms and run ordinary least squares again on the linear part. Repeat this until convergence. That's how Newton Raphson method works by Taylor series expansion.   Figure 2 which depicts the trend of actual values and values fitted by fractional spline model. This graph is very much similar to as one made by cubic spline model (step10) in Figure 2.

Conclusion
Selection a good proper model, this is one of the most difficult and contentious issues in statistics. By comparing the fractional and cubic spline regression models, we conclude that there is not much difference in the results of both models. Consequently, the fractional spline approach offers a much greater range of flexibility than the traditional polynomial splines. Fractional splines work in the same context that polynomial splines do, but with much greater flexibility and better fit. Consequently, fractional splines can produce a smaller error sum of squares and a higher R-squared than polynomial splines.
In general, spline models work well for interpolation within the range of the observed data. Neither polynomial nor fractional splines are useful for forecasting outside of that range without first transforming the underlying data series. The example provided in this research hopefully has given the reader an understanding of how spline models are more appropriate than traditional linear and polynomial regression models and how these models are set up and estimated and how to choose a proper model among various alternative spline models. We have tried to fit a fractional spline model also. In this case, fractional splines are not depicting a very good result as compared to cubic splines. But, they may work better with a larger data set such as stock market data. As we have worked with only 54 observations and there is an oscillation in our mortality data, therefore it is possible that fractional spline may work more accurately with big data and with data having cycles. Further investigation to reveal the circumstances where fractional splines significantly outperform than traditional splines will require a different model, and, perhaps, a different data set.