This post will demonstrate a downside to rankings-based strategies, particularly when using data of a questionable quality (which, unless one pays multiple thousands of dollars per month for data, most likely is of questionable quality). Essentially, by making one small change to the way the strategy filters, it introduces a massive performance drop in terms of drawdown. This exercise effectively demonstrates a different possible way of throwing a curve-ball at ranking strategies to test for robustness.

Recently, a discussion came up between myself, Terry Doherty, Cliff Smith, and some others on Seeking Alpha regarding what happened when I substituted the 63-day SMA for the three month SMA in Cliff Smith’s QTS strategy (quarterly tactical strategy…strategy).

Essentially, by simply substituting a 63-day SMA (that is, using daily data instead of monthly) for a 3-month SMA, the results were drastically affected.

Here’s the new QTS code, now in a function.

qts <- function(prices, nShort = 20, nLong = 105, nMonthSMA = 3, nDaySMA = 63, wRankShort=1, wRankLong=1.01, movAvgType = c("monthly", "daily"), cashAsset="VUSTX", returnNames = FALSE) { cashCol <- grep(cashAsset, colnames(prices)) #start our data off on the security with the least data (VGSIX in this case) prices <- prices[!is.na(prices[,7]),] #cash is not a formal asset in our ranking cashPrices <- prices[, cashCol] prices <- prices[, -cashCol] #compute momentums rocShort <- prices/lag(prices, nShort) - 1 rocLong <- prices/lag(prices, nLong) - 1 #take the endpoints of quarter start/end quarterlyEps <- endpoints(prices, on="quarters") monthlyEps <- endpoints(prices, on = "months") #take the prices at quarterly endpoints quarterlyPrices <- prices[quarterlyEps,] #short momentum at quarterly endpoints (20 day) rocShortQtrs <- rocShort[quarterlyEps,] #long momentum at quarterly endpoints (105 day) rocLongQtrs <- rocLong[quarterlyEps,] #rank short momentum, best highest rank rocSrank <- t(apply(rocShortQtrs, 1, rank)) #rank long momentum, best highest rank rocLrank <- t(apply(rocLongQtrs, 1, rank)) #total rank, long slightly higher than short, sum them totalRank <- wRankLong * rocLrank + wRankShort * rocSrank #function that takes 100% position in highest ranked security maxRank <- function(rankRow) { return(rankRow==max(rankRow)) } #apply above function to our quarterly ranks every quarter rankPos <- t(apply(totalRank, 1, maxRank)) #SMA of securities, only use monthly endpoints #subset to quarters #then filter movAvgType = movAvgType[1] if(movAvgType=="monthly") { monthlyPrices <- prices[monthlyEps,] monthlySMAs <- xts(apply(monthlyPrices, 2, SMA, n=nMonthSMA), order.by=index(monthlyPrices)) quarterlySMAs <- monthlySMAs[index(quarterlyPrices),] smaFilter <- quarterlyPrices > quarterlySMAs } else if (movAvgType=="daily") { smas <- xts(apply(prices, 2, SMA, n=nDaySMA), order.by=index(prices)) quarterlySMAs <- smas[index(quarterlyPrices),] smaFilter <- quarterlyPrices > quarterlySMAs } else { stop("invalid moving average type") } finalPos <- rankPos*smaFilter finalPos <- finalPos[!is.na(rocLongQtrs[,1]),] cash <- xts(1-rowSums(finalPos), order.by=index(finalPos)) finalPos <- merge(finalPos, cash, join='inner') prices <- merge(prices, cashPrices, join='inner') returns <- Return.calculate(prices) stratRets <- Return.portfolio(returns, finalPos) if(returnNames) { findNames <- function(pos) { return(names(pos[pos==1])) } tmp <- apply(finalPos, 1, findNames) assetNames <- xts(tmp, order.by=as.Date(names(tmp))) return(list(assetNames, stratRets)) } return(stratRets) }

The one change I made is this:

movAvgType = movAvgType[1] if(movAvgType=="monthly") { monthlyPrices <- prices[monthlyEps,] monthlySMAs <- xts(apply(monthlyPrices, 2, SMA, n=nMonthSMA), order.by=index(monthlyPrices)) quarterlySMAs <- monthlySMAs[index(quarterlyPrices),] smaFilter <- quarterlyPrices > quarterlySMAs } else if (movAvgType=="daily") { smas <- xts(apply(prices, 2, SMA, n=nDaySMA), order.by=index(prices)) quarterlySMAs <- smas[index(quarterlyPrices),] smaFilter <- quarterlyPrices > quarterlySMAs } else { stop("invalid moving average type") }

In essence, it allows the function to use either a monthly-calculated moving average, or a daily, which is then subset to the quarterly frequency of the rest of the data.

(I also allow the function to return the names of the selected securities.)

So now we can do two tests:

1) The initial parameter settings (20-day short-term momentum, 105-day long-term momentum, equal weigh their ranks (tiebreaker to the long-term), and use a 3-month SMA to filter)

2) The same exact parameter settings, except a 63-day SMA for the filter.

Here’s the code to do that.

#get our data from yahoo, use adjusted prices symbols <- c("NAESX", #small cap "PREMX", #emerging bond "VEIEX", #emerging markets "VFICX", #intermediate investment grade "VFIIX", #GNMA mortgage "VFINX", #S&P 500 index "VGSIX", #MSCI REIT "VGTSX", #total intl stock idx "VUSTX") #long term treasury (cash) getSymbols(symbols, from="1990-01-01") prices <- list() for(i in 1:length(symbols)) { prices[[i]] <- Ad(get(symbols[i])) } prices <- do.call(cbind, prices) colnames(prices) <- gsub("\\.[A-z]*", "", colnames(prices)) monthlySMAqts <- qts(prices, returnNames=TRUE) dailySMAqts <- qts(prices, wRankShort=.95, wRankLong=1.05, movAvgType = "daily", returnNames=TRUE) retsComparison <- cbind(monthlySMAqts[[2]], dailySMAqts[[2]]) colnames(retsComparison) <- c("monthly SMA qts", "daily SMA qts") retsComparison <- retsComparison["2003::"] charts.PerformanceSummary(retsComparison["2003::"]) rbind(table.AnnualizedReturns(retsComparison["2003::"]), maxDrawdown(retsComparison["2003::"]))

And here are the results:

Statistics:

monthly SMA qts daily SMA qts Annualized Return 0.2745000 0.2114000 Annualized Std Dev 0.1725000 0.1914000 Annualized Sharpe (Rf=0%) 1.5915000 1.1043000 Worst Drawdown 0.1911616 0.3328411

With the corresponding equity curves:

Here are the several instances in which the selections do not match thanks to the filters:

selectedNames <- cbind(monthlySMAqts[[1]], dailySMAqts[[1]]) colnames(selectedNames) <- c("Monthly SMA Filter", "Daily SMA Filter") differentSelections <- selectedNames[selectedNames[,1]!=selectedNames[,2],]

With the results:

Monthly SMA Filter Daily SMA Filter 1997-03-31 "VGSIX" "cash" 2007-12-31 "cash" "PREMX" 2008-06-30 "cash" "VFIIX" 2008-12-31 "cash" "NAESX" 2011-06-30 "cash" "NAESX"

Now, of course, many can make the arguments that Yahoo’s data is junk, my backtest doesn’t reflect reality, etc., which would essentially miss the point: this data here, while not a perfect realization of the reality of Planet Earth, may as well have been valid (you know, like all the academics, who use various simulation techniques to synthesize more data or explore other scenarios?). All I did here was change the filter to something logically comparable (that is, computing the moving average filter on a different time-scale, which does not in any way change the investment logic). From 2003 onward, this change only affected the strategy in four places. However, those instances were enough to create some noticeable changes (for the worse) in the strategy’s performance. Essentially, the downside of rankings-based strategies are when the overall number of selected instruments (in this case, ONE!) is small, a few small changes in parameters, data, etc. can lead to drastically different results.

As I write this, Cliff Smith already has ideas as to how to counteract this phenomenon. However, unto my experience, once a strategy starts getting into “how do we smooth out that one bump on the equity curve” territory, I think it’s time to go back and re-examine the strategy altogether. In my opinion, while the idea of momentum is of course, sound, with a great deal of literature devoted to it, the idea of selecting just one instrument at a time as the be-all-end-all strategy does not sit well with me. However, to me, QTS nevertheless presents an interesting framework for analyzing small subgroups of securities, and using it as one layer of an overarching strategy framework, such that the return streams are sub-strategies, instead of raw instruments.

Thanks for reading.

NOTE: I am a freelance consultant in quantitative analysis on topics related to this blog. If you have contract or full time roles available for proprietary research that could benefit from my skills, please contact me through my LinkedIn here.

Pingback: The Whole Street’s Daily Wrap for 2/26/2015 | The Whole Street

The quality of data can indeed explain the discrepancies you highlighted between a daily vs monthly computed SMA. Statistically, you have 21 times more chances to come across an error when using daily data compared to monthly data. But let’s not forget that mathematically, monthly average are most likely to be different than daily one, This is always true, so perfect the quality of the data is. For example, let’s consider a fictitious time series looking like this (and assuming the prices stick to reality – no error due to the data provider):

Day1 : 100

Day 2 to Day 21: 90

Day 22 (which corresponds to Day 1 of next month) : 100

monthly SMA = [ 100 + 100 ] / 2 = 100

daily SMA = [ 100 + 20*90 + 100 ] / 22 = 90.9

Therefore, the results being mathematically different in most of the cases, no wonder that subsequent calculation that use these results as inputs are different too.

Cheers,

Tonio.

Agree with Tonio. There is no reason to expect daily SMA to be the same as monthly SMA.

First rule of developing trading strategies: optimize all you wnat but distrust the results until there is ample reason not to distrust them.

Another issue I see in most ranking strategy articles is the use of dividend adjusted data for both ranking/signals and returns. To calculate more realistic returns, it seems signals/ranking should be calculated on the non-adjusted price time series, with returns using the adjusted price time series data.

This process takes two steps: 1) run the ranking strategy on the non-adjusted time series, and extract the trade entry and exit dates and corresponding symbols. 2) Use the data from step 1 to select the trade entry and exit dates and corresponding symbols on the adjusted price time series. Use the outputs from step 2 to calculate the returns from the rotation system.

Thoughts?

Thanks,

Dave

That’s a good point. At this point, I was trying to just replicate the article. But yeah, the more proper thing to do would be to pass in both returns and prices.

If you’re interested, I wrote an article about this issue on my blog at:

http://dtr-trading.blogspot.com/2014/09/historical-data-and-momentum-rotation.html

I was motivated to look into this because I have been running a rotation system on several of my accounts live for several years now and the actual results do not match the back tests over this same time period…and I was backtesting only with adjusted price data.

Hi Dave,

Thanks for bringing this into the discussion. I have brought this up in the past as this is a legitimate concern. There are several “gotcha’s” in ETF rotation strategies and using adjusted data is one of them. No one can trade on adjusted data and obviously, they can’t do signal generation in real-time adjusted data either. All back testing by its nature always over estimates return and under estimates risk – using adjusted data is one reason.

-Gerald

Why would anyone expect a daily 63day-MA to be the same as a 3-month-MA in the first place ? Imagine a stock trading at 100 at the beginning of a month and gaining 1$ until the last day where the stock loses all his intramonthly gains and ends unchanged at 100. Repeat this monthly pattern for several months. The monthly MA will stay at 100 and the daily MA will stay at 109 !

(`series <- rep(c(100:119,100),6)

plot(rollmean(series,63),type='l')` )

So in one case you compare the monthly closing of 100 with 100 (monthly MA) in the case of using the daily MA the close is compared to 109 !

This certainly is an extreme example to make the point, but if we think of some of the seasonal patterns we see in the stock market this difference has real practical implications.

What version of R are you using? Thanks in advance. Does not work for 3.1.3?

Make sure to have all your packages updated. I’m still on R 3.1.2