A New Bias Corrected Version of Heteroscedasticity Consistent Covariance Estimator

In the presence of heteroscedasticity, different available flavours of the heteroscedasticity consistent covariance estimator (HCCME) are used. However, the available literature shows that these estimators can be considerably biased in small samples. Cribari–Neto et al. (2000) introduce a bias adjustment mechanism and give the modified White estimator that becomes almost bias-free even in small samples. Extending these results, Cribari-Neto and Galvão (2003) present a similar bias adjustment mechanism that can be applied to a wide class of HCCMEs’. In the present article, we follow the same mechanism as proposed by Cribari-Neto and Galvão to give bias-correction version of HCCME but we use adaptive HCCME rather than the conventional HCCME. The Monte Carlo study is used to evaluate the performance of our proposed estimators.


Introduction
Heteroscedasticity is a common problem when estimating linear regression.It not only gives inefficient least squares estimates but also the inconsistent usual covariance matrix estimate.So it results in the inadequacy and inaccuracy of inference as the degree of heteroscedasticy increases.The popular White's (1980) heteroscedastic covariance matrix estimator (HCCME) holds this issue and is the most commonly used covariance estimator.It is known as HC0.Three more improved versions of the HCCME are presented by Mackinnon and White (1985) and Davidson and MacKinnon (1993), which are known as HC1, HC2 and HC3 in the available literature.Long and Ervin (2000) confirmed the superb performance of HC3 by their Monte Carlo findings.Further valuable work is accessible in Cribari-Neto and Zarkos (1999,2001) Ahmad et al. (2007) show that if the HCCMEs are constructed using the residuals obtained from some adaptive estimation method rather from, simply, the OLS method, we can lessen the null rejection rates.
As discussed earlier that in the presence of heteroscedasticity, different available versions of HCCME are used, However, they can be considerably biased in small samples; see, e.g., MacKinnon and White (1985), Cribari-Neto and Zarkos (1999,2001), Long and Ervin (2000), Cribari-Neto et al. (2005) etc. Chesher and Jewitt (1987) also report that the HCCME studied by White (1980) can be severely biased in small samples.Substantial downward bias may exist whatever the form of heteroscedasticity when regression designs contain points of high leverage.They derive the proportionate bias of the HCCME.Cribari-Neto et al. (2000) introduce a bias adjustment mechanism and give the modified White estimator that becomes almost bias-free even in small samples.Their numerical results report that such estimator gives less size distortion.Extending these results, Cribari-Neto and Galvão (2003) present a similar bias adjustment mechanism that can be applied to a wide class of HCCMEs including, among others, the HC0, HC1, HC2 and HC3 estimators.According to them, this bias adjustment mechanism, that can be used to deliver a sequence of bias-corrected estimators where the order of the bias decreases as one advances in the sequence.On the basis of the numerical evidence, they report that the proposed estimators deliver more accurate estimates in small samples.
In this paper, we follow the same mechanism as proposed by Cribari-Neto and Galvão (2003) to give bias-corrected version of HCCME but we use the adaptive construction of HCCME as given in Ahmed et al. (2013).It is also important to show that whether or not the corrected estimators deliver improved inference in small samples.So we evaluate their performance on the basis of null rejection rate (NRR) and power of test.

The Model, Conventional and Kernel HCCME
Consider a heteroscedastic linear regression model of the form, where y is an n x 1 vector of observations, X is an n x p (p < n) nonrandom matrix of covariates,  is a p x 1 vector of unknown regression parameters, and u is an n x 1 vector of random errors with zero mean and variance  having typical ith diagonal elements ) ..., , 2 , 1 ( where I is an identity matrix of order n.

The usual OLS estimator of
It can be shown that OLS  ˆhas mean  and covariance matrix, In case of homoscedasticity, Eq. ( 2), obviously, reduces to 1
In the presence of heteroscedasticity of unknown form, the OLS estimate becomes inefficient and its covariance matrix estimate is also inconsistent as discussed earlier.In this situation, White (1980) derived a covariance matrix estimator which is consistent under both homoscedasticity and heteroscedasticity of unknown form and thus, the idea of HCCME comes in.The said estimator is given as In the literature, this estimator is known as HC0 and is the most commonly used by the practitioners.However, it can be considerably biased in small samples (Chesher and Jewitt, 1987).
MacKinnon and White (1985) raise concerns about the performance of HC0 in small samples.They noted that the estimator HC0 (3) takes no account of the well-known fact that OLS residuals tend to be 'too small'.One simple way to modify HC0 is to use a degree of freedom correction similar to the one conventionally used to obtain unbiased estimates of  2 .Thus, following Hinkely (1977), MacKinnon and White give This version of estimator is known as HC1.The degrees of freedom adjustment in HC1 is not the only way to compensate for the fact that the OLS residuals tend to underestimate the true errors.If there is no heteroscedasticity, it is easily seen that where h i is ith diagonal element of the above mentioned hat matrix H. Finding this fact, Horn et al. (1975) propose to use Based on this approach, MacKinnon and White (1985) propose: where . 1 A third deviation approximates a more complicated Jackknife (Efron, 1979) estimator which is used by Davidson and MacKinnon (1993) In their work, Mackinnon and White (1985) show that HC1, HC2 and HC3 are asymptotically equivalent to HC0 and are expected to have superior properties in small samples.Long and Ervin (2000) also recommend that use of HC standard error estimates and by their Monte Carlo findings they report the superb performance of HC3 for small samples (see MacKinnon, 2011 for more discussion).
So, generally, we can define HCCME as While the diagonal matrices, D i 's are as follows: Carroll (1982) presents an adaptive estimator for heteroscedastic linear regression model by following the key idea as noted by Fuller and Rao (1978) that in most of the heteroscedastic regression problems, the variances appeared to be smooth function of the design or mean response as where g is unknown and  i can be estimated by Carroll (1982) presents a kernel estimator of g as where v is smoothing parameter and the estimated variances are   ˆ() i g t .Using these estimated variances, an estimated weighted least square estimator that is adaptive also in the sense of Carroll (1982).We further refer this estimator as kernel weighted least squares (KWLS) estimator which is defined as follows: (10) As it has been shown that  ˆKWLS is more efficient as compared to OLS  ˆ and the asymptotic properties of  ˆKWLS can be seen by Carroll (1982).Therefore, Ahmed et al.
(2007) find it quite plausible to use KWLS residuals to construct new HCCMEs as follows: and u ~2 denotes the KWLS squared residuals.While the diagonal matrices, D i 's remain the same as in ( 8) but the new HC estimators are replaced by HCK, for distinction:

Bias Correction Mechanism
We have given Carroll's (1982) adaptive estimator as KWLS estimator in (10).In order to follow the bias correction mechanism as given by Cribari-Neto and Galvão (2003), we rewrite (10) as is given in (9).
On the basis of our proposal, we modify the recursive function, defined by Cribari-Neto and Galvão (2003), as follows, (17) where K is the same as defined earlier, , A is an n X n diagonal matrix and r = 0, 1, 2, . . . .Using the same properties as given by Cribari-Neto and Galvão (2003) about the said recursive function, (15) can be written as Similarly, (16), can be written as, By replacing  with  ~in (18), we can have an estimator of  which is nearly unbiased as,

M
and  ~ is given in (11).
So by simplifying, Similarly, we can find the bias of  ~) 1 , where I D  0 then by using (18), we can have that Now by using ( 22) and the properties of the recursive function, we can write (21) as, Again applying the same procedure as given in ( 20) and ( 23), we have the second bias corrected estimator of  as; Repeating the same procedure, recursively, after r iteration, we have where u ~denotes the vector of KWLS residuals.

Numerical Evaluation
In our study, we follow the Monte Carlo scheme as used by Cribari-Neto and Galvão (2003).Under this scheme, the model is; where 1


that ranges from 1 (a case of homoscedasticty) to over 100.The degree of heteroscedasticity will remain fruitful to evaluate the performance of the estimators in mild to severe heteroscedasticity.The sample size (n) is set to 25, 50, 100, 150, and 200.It is to be noted that Cribari-Neto and Galvão (2003) take smallest sample of size 50, in their study while we take 25 so as to assess the performance of the candidate estimators in considerably small samples.In order to keep the degree of heteroscedasticity same for all the sample sizes, firstly, 25 independent x i 's are generated and then replicated twice, thrice and so on for n = 50, 100, 150 and 200, respectively (see also Cribari-Neto, 2004, for such replications).The number of Monte Carlo replications is set to be 5000.
For the estimation of model ( 27), we use the OLS and an adaptive estimator as used by Carroll (1982).Such adaptive estimators are more efficient to those of the OLS in the presence of heteroscedasticity of unknown form.The details of such estimation can also be found by Ahmad et al. ( 2007 2000) note that gains from high-order iterations of the bias-correction process are negligible; two iterations in the bias-correction scheme are enough.Since we are dealing with numerous estimators (5 variants of already available HCCME and 5, according to our new proposal), therefore we limitize our numerical work for bias correction at just first iteration.For bias corrected HCCMEs, we use a prefix BC (Bias Corrected) with all the estimators.Thus, we evaluate the estimators, HC0, BCHC0, HC1, BCHC1, HC2, BCHC2, HC3, BCHC3 and our proposed, HCK0, BCHCK0, HCK1, BCHCK1, HCK2, BCHCK2, HCK3, BCHCK3.

Bias and MSE
Following Cribari-Neto and Galvão (2003) and Cribari-Neto (2004), we compute the total relative bias (TRB) that is defined as Second measure is the square root of total mean square error (also known as total root mean square error, total RMSE) ( X 5000 ).This measure is computed as the square roots of the sum of two individual MSE's and then multiplied by 5000 .These quantities take into account not only bias, but also the variances of the different estimators.
Table 1 presents total relative bias (TRB) of the conventional HC estimators, HCK estimators, and bias corrected version of HC and HCK estimators.We see that bias corrected version of HC and HCK perform better than the conventional HC and adaptive HC estimators.We note that BCHC2, BCHC3, BCHCK2 and BCHCK3 perform better than HC2, HC3, HCK2 and HCK3, respectively.For example, in small sample and for higher degree of heteroscedasticity (see n = 25, a 1 = a 2 = 2.5), the TRBs for the BCHC2, BCHC3, BCHCK2 and BCHCK3 are 1%, 8%, 6% and 4%, respectively, while for the OLS, HC2, HC3, HCK2 and HCK3, the TRB are noted to be 113%, 4%, 35%, 18% and 58% respectively.Similar is the situation for the large samples.However, with the increase in sample size, the TRB decreases, as evident.For (n = 100, a 1 = a 2 = 1.5), the TRB is 97%, 1%, 7%, 3% and 10%, respectively for OLS, HC2, HC3, HCK2 and HCK3 and the TRB for BCHC2, BCHC3, BCHCK2 and BCHCK3 is 0.07%, 0.2%, 1% and 1%, respectively.We observe that among the conventional HC estimators, HC2 performs well in terms of TRB while, in the adaptive estimators, BCHCK2 takes this stand.We further, note that among all the estimators, under consideration, generally, BCHCK2 outperforms in terms of TRB.Its unmatched performance can also be compared if a researcher do not consider the severe heteroscedasticity then he might be using the OLS estimators with 119% TRB as compared to BCHCK2 with just 0.1% TRB (see n = 200, a 1 = a 2 = 2.5).Even for moderate heteroscedasticity ( = 6.44, 16.34), BCHCK2 performs superbly in terms of less bias.Table 2 presents total RMSE (X 5000 ), of the conventional HC estimators, HCK estimators, and the bias corrected version of HC and HCK estimators.We note that, in case of homoscedasticty, all the estimators behave in the same manner in terms of total RMSE.It means that RMSE for BCHC and BCHCK is almost equal to that of HC and HCK for all the sample sizes when there is no heteroscedasticity.We report that the bias corrected version of HC and HCK perform better than its counterparts.For example for the severe heteroscedasticity and small sample (see n = 25, a 1 = a 2 = 2.0), the RMSE for OLS, HC2, HC3, HCK2 and HCK3 is 442.49,538.53, 490.50 and 612.46, respectively, while for BCHC2, BCHC3, BCHCK2 and BCHCK3, the RMSE is 457.76,476.57, 429.59 and 434.17, respectively.We note that the bias corrected version of adaptive HCCME performs better than the other estimators in terms of RMSE.As sample size increases, the total RMSE tends to decline and for large sample (n = 200, a 1 = a 2 = 2.5, λ=105.22),all of the estimators behave in the same fashion.

Null Rejection Rate
In our numerical work, we test H 0 : and 10 % of significance, respectively.Table 3, 4 and 5 present the estimated null rejection rates (NRR, in percentage) based on t-ratios using the estimated variances of  ˆ1 (from all the variance estimators under consideration).Under the stated null hypothesis, the limiting distribution of t-statistic is N(0, 1) and this test is performed by using the critical values of this limiting distribution which is then compared to the absolute values of the said test statistic (see Cribari-Neto, 2004and Ahmad et al., 2007, 2011, for more details).Although, we present the NRRs at nominal levels of significance (LOS) at 1%, 5% and 10% but in our discussion, we just discuss them at 5% LOS.It is just because the performance can be observed in the same fashion for the other two LOSs'.So Table 3 will be discussed only.
Table 3 represents the NRR for the conventional HC estimators, HCK estimators and bias corrected versions of HC and HCK estimators.This table shows that under homoscedasticity, the tests based on OLS variance estimators performs quite well as expected but they become too liberal as the degree of heteroscedasticity increases so as invite to other variance estimators to handle the situation.When we compare KWLS and all the other weighted HCCMEs' in the case of homoscedasticity, they are too liberal as compared to the OLS estimators.Obviously, it shows that the OLSE is the best choice when dealing homoscedasticity.If we take a moderate case of heteroscedasticity in small sample ( n = 25, a 1 = a 2 = 1.5,  = 16.34), the NRR is 20.18%, 10.72%, 8.42%, 9.46% and 7.42% respectively, for, the OLS HC2, HC3, HCK2 and HCK3, while the figures are 10.60%, 10.06%, 9.92% and 9.58% respectively, for BCHC2, BCHC3, BCHCK2 and BCHCK3.We note that only BCHC2 performs better than its competitors.But bias corrected version of adaptive HCCME performs better than the conventional HCCME and its bias corrected versions.It shows the same reflection as verified by Ahmad et al. (2007) that the adaptive HCCM estimators show better size performance than the conventional HCCM estimators.For example for large sample size and severe degree of heteroscedasticity, ( see n = 150, a 1 = a 2 = 2.5,  = 105.22), the NRR for OLS HC2, HC3, BCHC2 and BCHC3 are 23.90%,6.54%, 6.16%, 6.44% and 6.44% respectively, whereas the NRR for BCHCK2 and BCHCK3 are 5.72% and 5.72%, respectively.Among all the bias corrected estimators, the size distortion in BCHCK3 is smallest for all sample sizes and all degree of heteroscedasticity.However, the difference becomes insignificant as sample size increases.

Conclusion
It is common to use the HCCMEs to draw correct inference about the regression parameters of a linear regression model with heteroscedastic errors.However, these estimators can be considerably biased in small samples.Available literature suggests some bias corrected mechanism for these estimators.We use the same mechanism but instead of using the OLS residuals, we propose to use the residuals obtained from the adaptive estimation.Our proposed estimators display less bias and also perform considerably well in the testing of hypothesis.
can define a sequence of estimators of  ,   we define the same sequence of bias corrected HCCME as Cribari-Neto and Galvão (2003) but with our new proposal of , ), Ahmed et al. (2011) and Aslam et al. (2013) etc. Cribari-Neto et al. (