Based on their 2015 JPM paper "Backtesting", Campbell Harvey and Yan Liu (HL) propose
and demonstrate three methods of adjusting for potential multiple testing bias.
Using their model they propose haircuts for Sharpe ratios returned by trading strategies
(see haircutSharpe).
HL pose another way of viewing the problem is to ascertain a minimum average monthly
return for a given significance level. This is replicated in quantstrat with the
profitHurdle function.
profitHurdle(portfolios, ..., strategy = NULL, trials = NULL,
alpha_sig = 0.05, vol_annual = 0.1, RHO = 0.2, audit = NULL,
env = .GlobalEnv)string name of portfolio, or optionally a vector of portfolios, see DETAILS
any other passthrough parameters
optional strategy specification that would contain more information on the process, default NULL
optional number of trials,default NULL
Significance level e.g. 0.05 ie. 5%, default 0.05
Annual return volatility e.g. 0.10 ie. 10%, default 0.10
Assumed average correlation, default 0.2
optional audit environment containing the results of parameter optimization or walk forward, default NULL
optional environment to find market data in, if required
an object of type profitHurdle containing:
Significance Level: the significance level used to determine the minimum required average monthly return
Number of Monthly Observations: the number of monthly observations in the strategy, converted from one of either daily, weekly, monthly, quarterly or yearly
Annualized Return Volatility: the annualized volatility of the strategy returns
Assumed Number of Tests: the number of tests assumed and needed to account for in multiple testing adjustment
Assumed Average Correlation: assumed average level of correlation among strategy returns
Independent: unadjusted return hurdle, implying single test significance
Bonferroni: the Bonferroni adjusted return hurdle for the given level of significance
Holm: the Holm adjusted return hurdle for the given level of significance
BHY: the BHY adjusted return hurdle for the given level of significance
Average for Multiple Tests: the average multiple testing return hurdle for the given level of significance
Harvey, Campbell R. and Yan Liu. 2015. Backtesting The Journal of Portfolio Management. 41:1 pp. 13-28.
Harvey, Campbell R., Yan Liu, and Heqing Zhu. 2016. "... and the cross-section of expected returns." The Review of Financial Studies 29, no. 1 (2016): 5-68.
Mackie, Jasen. 2016. R-view: Backtesting - Harvey & Liu (2015). https://opensourcequant.wordpress.com/2016/11/17/r-view-backtesting-harvey-liu-2015/