Sylvain Barde

The paper proposes a new algorithm for finding the confidence set of a collection of forecasts or prediction models. Existing numerical implementations use an elimination approach, where one starts with the full collection of models and successively eliminates the worst performing until the null of equal predictive ability is no longer rejected at a given confidence level. The intuition behind the proposed implementation lies in reversing the process, i.e. starting with a collection of two models and updating both the model rankings and p-values as models are successively added to the collection. The first benefit of this approach is a reduction of one polynomial order in both the time complexity and memory cost of finding the confidence set of a collection of M models, falling respectively from O(M^3) to O(M^2) and from O(M^2) to O(M). The second key benefit is that it allows for further models to be added at a later point in time, thus enabling collaborative efforts using the model confidence set procedure. We prove that this implementation is equivalent to the elimination approach and demonstrate the improved performance on a multivariate GARCH collection consisting of 4800 models.

We propose a novel estimation procedure for models with endogenous variables in the presence of spatial correlation based on Eigenvector Spatial Filtering. The procedure, called Moran’s 2-Stage Lasso (Mi-2SL), uses a two-stage Lasso estimator where the Standardised Moran’s I is used to set the Lasso tuning parameter. Unlike existing spatial econometric methods, this has the key benefit of not requiring the researcher to explicitly model the spatial correlation process, which is of interest in cases where they are only interested in removing the resulting bias when estimating the direct effect of covariates. We show the conditions necessary for consistent and asymptotically normal parameter estimation assuming the support (relevant) set of eigenvectors is known. Our Monte Carlo simulation results also show that Mi-2SL performs well against common alternatives in the presence of spatial correlation. Our empirical application replicates Cadena and Kovak (2016) instrumental variables estimates using Mi-2SL and shows that in that case, Mi-2SL can boost the performance of the first stage.

This paper proposes a Lasso-based estimator which uses information embedded in the Moran statistic to develop a selection procedure called Moran’s I Lasso (Mi-Lasso) to solve the Eigenvector Spatial Filtering (ESF) eigenvector selection problem. ESF uses a subset of eigenvectors from a spatial weights matrix to efficiently account for any omitted spatially correlated terms in a classical linear regression framework, thus eliminating the need for the researcher to explicitly specify the spatially correlated parts of the model. We proposed the first ESF procedure accounting for post-selection inference. We derive performance bounds and show the necessary conditions for consistent eigenvector selection. The key advantages of the proposed estimator are that it is intuitive, theoretically grounded, able to provide robust inference and substantially faster than Lasso based on cross-validation or any proposed forward stepwise procedure. Our simulation results and an application on house prices demonstrate Mi-Lasso performs well compared to existing procedures in finite samples.

Large scale, computationally expensive simulation models pose a particular challenge when it comes to estimating their parameters from empirical data. Most simulation models do not possess closed-form expressions for their likelihood function, requiring the use of simulation-based inference, such as simulated method of moments, indirect inference, likelihood-free inference or approximate Bayesian computation. However, given the high computational requirements of large-scale models, it is often difficult to run these estimation methods, as they require more simulated runs that can feasibly be carried out. The aim is to address the problem by providing a full Bayesian estimation framework where the true but intractable likelihood function of the simulation model is replaced by one generated by a surrogate model trained on the limited simulated data. This is provided by a Linear Model of Coregionalization, where each latent variable is a sparse variational Gaussian process, chosen for its desirable convergence and consistency properties. The effectiveness of the approach is tested using both a simulated Bayesian computing analysis on a known data generating process, and an empirical application in which the free parameters of a computationally demanding agent-based model are estimated on US macroeconomic data.

The paper aims to address the issue of comparing agent-based models (ABMs) with more traditional VAR and DSGE models by developing a multivariate extension of the Markov Information Criterion (MIC) of Barde (2017). The univariate MIC measures the informational distance between a simulation model and some empirical data by mapping the simulated data to a Markov transition matrix, and is proven to provide an unbiased measurement for all models reducible to a Markov process. As a result, the MIC can accurately measure distance using only simulated data, for a wide class of data generating processes. The paper first presents the multivariate extension of the MIC and its validation on VAR and DGSE models before carrying the first direct comparison between a macroeconomic ABM and a DGSE model, namely the benchmark ABM of Caiani et al. (2016) and Smets and Wouters (2007).

The recent increase in the breath of computational methodologies has been matched with a corresponding increase in the difficulty of comparing the relative explanatory power of models from different methodological lineages. In order to help address this problem a Markovian information criterion (MIC) is developed that is analogous to the Akaike information criterion (AIC) in its theoretical derivation and yet can be applied to any model able to generate simulated or predicted data, regardless of its methodology. Both the AIC and proposed MIC rely on the Kullback–Leibler (KL) distance between model predictions and real data as a measure of prediction accuracy. Instead of using the maximum likelihood approach like the AIC, the proposed MIC relies instead on the literal interpretation of the KL distance as the inefficiency of compressing real data using modelled probabilities, and therefore uses the output of a universal compression algorithm to obtain an estimate of the KL distance. Several Monte Carlo tests are carried out in order to (a) confirm the performance of the algorithm and (b) evaluate the ability of the MIC to identify the true data-generating process from a set of alternative models.

The present paper tests a new model comparison methodology by comparing multiple calibrations of three agent-based models of financial markets on the daily returns of 24 stock market indices and exchange rate series. The models chosen for this empirical application are the herding model of Gilli and Winker (2003), its asymmetric version by Alfarano et al. (2005) and the more recent model by Franke and Westerhoff (2011), which all share a common lineage to the herding model introduced by Kirman (1993). In addition, standard ARCH processes are included for each financial series to provide a benchmark for the explanatory power of the models. The methodology provides a consistent and statistically significant ranking of the three models. More importantly, it also reveals that the best performing model, Franke and Westerhoff, is generally not distinguishable from an ARCH-type process, suggesting their explanatory power on the data is similar.