Dive deeper into H2O
What do you do when traditional statistical models fall short? Unlock the power of high-performance predictive models powered by H2O without leaving your familiar Stata environment.
The introduction of the h2oml suite earlier this year, featuring gradient boosting machines and random forest with tools for hyperparameter tuning, model performance, prediction, and prediction explainability, was just the start. We've been working to give you even more power and insight. Since then, we've released three blog posts detailing different ways you can take your machine learning analysis in Stata to the next level. We've also added three new postestimation commands in the most recent Stata 19 update, making it easier to interpret your models and diagnose potential issues.
Approximate statistical tests for comparing binary classifier error rates using H2OML
Uncertain whether your new model truly outperforms your baseline? The first blog post provides two methods for conducting approximate statistical tests to compare binary classifier error rates: the McNemar test (McNemar 1947) and the combined 5 × 2 cross-validated (5 × 2 cv) F test (Alpaydin 1998). These tests give you the statistical rigor needed to confidently claim your model performs better.
Prediction intervals with gradient boosting machine
The second blog post tackles a critical question: how confident should you be in individual predictions? We construct prediction intervals for a gradient boosting machine, moving beyond single-point predictions to capture the uncertainty around each forecast.
Heterogeneous treatment-effect estimation with S-, T-, and X-learners using H2OML
The third blog explores one of the most powerful applications of machine learning in causal inference: heterogeneous treatment effects. Does a treatment work equally well for everyone, or does it help some groups while harming others? We demonstrate how S-, T-, and X-learners can reveal subgroup-specific effects that traditional approaches might miss.
New postestimation commands
- h2omlgraph permimp graphs permutation variable importance, clearly showing which predictors drive your results. Permutation importance graphs show how much prediction accuracy drops when each variable is shuffled, giving you interpretable insights.
- h2omlgraph rvfplot graphs the residuals against the fitted values for comprehensive model diagnostics. In machine learning, this diagnostic plot is useful for evaluating how well the model fits the data.
- h2omlgraph rvpplot graphs the residuals against a predictor, providing an additional perspective on model fit across values for a specified predictor.
Keep an eye out for more updates and resources as we continue to expand Stata’s h2oml toolkit.
References
McNemar, Q. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12: 153–157. https://doi.org/10.1007/BF02295996.
Alpaydin, E. 1999. Combined 5 x 2 cv F test for comparing supervised classification learning algorithms. Neural Computation 11: 1885–1892. http://doi.org/10.1162/089976699300016007.
