Approximately Normal Tests for Equal Predictive Accuracy
in Nested Models

By Todd E. Clark and Kenneth D. West*
November 2005 
RWP 05-05
Research Division 
Federal Reserve Bank of Kansas City 

Abstract

      Forecast evaluation often compares a parsimonious null model to a larger model that nests the null model. Under the null that the parsimonious model generates the data, the larger model introduces noise into its forecasts by estimating parameters whose population values are zero. We observe that the mean squared prediction error (MSPE) from the parsimonious model is therefore expected to be smaller than that of the larger model. We describe how to adjust MSPEs to account for this noise. We propose applying standard methods (West (1996)) to test whether the adjusted mean squared error difference is zero. We refer to nonstandard limiting distributions derived in Clark and McCracken (2001, 2005a) to argue that use of standard normal critical values will yield actual sizes close to, but a little less than, nominal size. Simulation evidence supports our recommended procedure.
 

Keywords: Forecast evaluation, causality, nested models.

JEL classification: C53, C52

*Todd Clark is a vice president and economist at the Federal Reserve Bank of Kansas City.  Kenneth West is a Ragnar Frisch Professor of Economics at the University of Wisconsin.  West thanks the National Science Foundation for financial support.  We thank Pablo M. Pincheira-Brown and Taisuke Nakata for helpful comments.  The views expressed herein are solely those of the authors and do not necessarily reflect the views of the Federal Reserve Bank of Kansas City or the Federal Reserve System.

Back to top       RWP home