With bootstrap's new jab option, you can obtain jackknife-after-bootstrap (JAB) standard errors to evaluate how precisely the bootstrap standard error is measured. This feature is a part of StataNow™.

A bootstrap standard error is itself an estimate subject to sampling variability. The JAB method evaluates that uncertainty by estimating the standard error of the bootstrap standard error itself.

For each observation, JAB uses the bootstrap replications that omit it to recompute the bootstrap standard error. It applies the jackknife formula to these leave-one-out estimates, without requiring further resampling.

In Stata, the jab option adds these calculations to the bootstrap prefix, reporting both observed JAB standard errors and estimates corrected for finite-replication noise.

Let's see it work

We illustrate this using Stata's automobile dataset. We regress miles per gallon on vehicle weight, gear ratio, and an indicator for foreign cars, scaling weight to thousands of pounds to simplify interpretation of the coefficients.

. sysuse auto, clear
(1978 automobile data)

. generate double weight1000 = weight/1000

. bootstrap, reps(100) nodots rseed(195) jab: regress mpg weight1000 
     gear_ratio foreign

Linear regression                                       Number of obs =     74
                                                        Replications  =    100
                                                        Wald chi2(3)  = 176.98
                                                        Prob > chi2   = 0.0000
                                                        R-squared     = 0.6670
                                                        Adj R-squared = 0.6527
                                                        Root MSE      = 3.4096


             
 
   Observed   Bootstrap                         Normal-based    
         mpg 
 
 coefficient  std. err.      z    P>|z|     [95% conf. interval]
 
 
 
  weight1000 
 
  -6.139031     .61997    -9.90   0.000     -7.35415   -4.923912
  gear_ratio 
 
   1.457113   1.537166     0.95   0.343    -1.555676    4.469902
     foreign 
 
  -2.221682    1.18429    -1.88   0.061    -4.542846    .0994835
       _cons 
 
   36.10135   5.716314     6.32   0.000     24.89758    47.30512


Sampling variability of bootstrap standard errors


             
 
  Observed JAB Corrected JAB
             
 
     std. err.     std. err.
 
 
 
  weight1000 
 
     .4339763       .2008243
  gear_ratio 
 
     1.243602       .4194864
     foreign 
 
     1.037857       .2060507
       _cons 
 
     3.682556              .

Warning: One or more corrected JAB std. err. is negative and thus set to
         missing. This may happen when the number of bootstrap replicates is
         too small relative to the sample size.

We begin with 100 replications and specify a random-number seed for reproducibility. Stata resamples the cars with replacement and refits the regression, using the variation in the coefficients to estimate their standard errors.

While the bootstrap standard error measures the sampling variability of the coefficient estimator, the corresponding JAB standard error measures the sampling variability of that estimated standard error.

Below the coefficient table, Stata displays both “Observed JAB std. err.” and “Corrected JAB std. err”. The corrected standard error adjusts for the upward bias in the JAB variance estimate introduced by Monte Carlo simulation error from a finite number of replications.

For example, the observed JAB standard error for weight1000 is 0.434 but drops to 0.201 after correction. This substantial reduction demonstrates that Monte Carlo simulation error accounts for a large portion of the unadjusted estimate at only 100 replications.

Notice that the corrected JAB estimate for the intercept is missing. When the number of bootstrap replications is too small, instability in the adjustment can lead to a negative corrected variance estimate, as flagged by the warning below the table.

For reliable JAB estimates, Efron (1992) advises using a much larger number of bootstrap replications, generally on the order of 1,000.

Increasing the number of replications

We repeat the analysis with 10,000 replications, holding the data, model, and random-number seed constant.

. bootstrap, reps(10000) nodots rseed(195) jab: regress mpg weight1000 
     gear_ratio foreign

Linear regression                                       Number of obs =     74
                                                        Replications  = 10,000
                                                        Wald chi2(3)  = 141.53
                                                        Prob > chi2   = 0.0000
                                                        R-squared     = 0.6670
                                                        Adj R-squared = 0.6527
                                                        Root MSE      = 3.4096


             
 
   Observed   Bootstrap                         Normal-based    
         mpg 
 
 coefficient  std. err.      z    P>|z|     [95% conf. interval]
 
 
 
  weight1000 
 
  -6.139031   .6275911    -9.78   0.000    -7.369087   -4.908975
  gear_ratio 
 
   1.457113   1.416656     1.03   0.304     -1.31948    4.233707
     foreign 
 
  -2.221682   1.208305    -1.84   0.066    -4.589915    .1465523
       _cons 
 
   36.10135   5.317175     6.79   0.000     25.67988    46.52282


Sampling variability of bootstrap standard errors


             
 
  Observed JAB Corrected JAB
             
 
     std. err.     std. err.
 
 
 
  weight1000 
 
     .1270199       .1159748
  gear_ratio 
 
      .311387       .2861413
     foreign 
 
     .2327646        .209109
       _cons 
 
     1.132275       1.036134

With 10,000 replications, the bootstrap standard error for weight1000 shifts only marginally, moving from 0.620 to 0.628. In contrast, the observed and corrected JAB standard errors are now in close agreement (0.127 versus 0.116, respectively), confirming that Monte Carlo simulation error has decreased.

Corrected JAB estimates are now available for all coefficients, including the intercept. For the weight1000 coefficient, the corrected value of 0.116 quantifies the sampling variability surrounding the estimated bootstrap standard error of 0.628.

While increasing the replication count suppresses simulation noise, it cannot alter the sampling variability dictated by the original sample size. The corrected JAB standard error allows us to isolate and evaluate the true sampling uncertainty underlying the bootstrap standard error.