The Downside of Rankings-Based Strategies

This post will demonstrate a downside to rankings-based strategies, particularly when using data of a questionable quality (which, unless one pays multiple thousands of dollars per month for data, most likely is of questionable quality). Essentially, by making one small change to the way the strategy filters, it introduces a massive performance drop in terms of drawdown. This exercise effectively demonstrates a different possible way of throwing a curve-ball at ranking strategies to test for robustness.

Recently, a discussion came up between myself, Terry Doherty, Cliff Smith, and some others on Seeking Alpha regarding what happened when I substituted the 63-day SMA for the three month SMA in Cliff Smith’s QTS strategy (quarterly tactical strategy…strategy).

Essentially, by simply substituting a 63-day SMA (that is, using daily data instead of monthly) for a 3-month SMA, the results were drastically affected.

Here’s the new QTS code, now in a function.

qts <- function(prices, nShort = 20, nLong = 105, nMonthSMA = 3, nDaySMA = 63, wRankShort=1, wRankLong=1.01, 
                movAvgType = c("monthly", "daily"), cashAsset="VUSTX", returnNames = FALSE) {
  cashCol <- grep(cashAsset, colnames(prices))
  #start our data off on the security with the least data (VGSIX in this case)
  prices <- prices[![,7]),] 
  #cash is not a formal asset in our ranking
  cashPrices <- prices[, cashCol]
  prices <- prices[, -cashCol]
  #compute momentums
  rocShort <- prices/lag(prices, nShort) - 1
  rocLong <- prices/lag(prices, nLong) - 1
  #take the endpoints of quarter start/end
  quarterlyEps <- endpoints(prices, on="quarters")
  monthlyEps <- endpoints(prices, on = "months")
  #take the prices at quarterly endpoints
  quarterlyPrices <- prices[quarterlyEps,]
  #short momentum at quarterly endpoints (20 day)
  rocShortQtrs <- rocShort[quarterlyEps,]
  #long momentum at quarterly endpoints (105 day)
  rocLongQtrs <- rocLong[quarterlyEps,]
  #rank short momentum, best highest rank
  rocSrank <- t(apply(rocShortQtrs, 1, rank))
  #rank long momentum, best highest rank
  rocLrank <- t(apply(rocLongQtrs, 1, rank))
  #total rank, long slightly higher than short, sum them
  totalRank <- wRankLong * rocLrank + wRankShort * rocSrank 
  #function that takes 100% position in highest ranked security
  maxRank <- function(rankRow) {
  #apply above function to our quarterly ranks every quarter
  rankPos <- t(apply(totalRank, 1, maxRank))
  #SMA of securities, only use monthly endpoints
  #subset to quarters
  #then filter
  movAvgType = movAvgType[1]
  if(movAvgType=="monthly") {
    monthlyPrices <- prices[monthlyEps,]
    monthlySMAs <- xts(apply(monthlyPrices, 2, SMA, n=nMonthSMA),
    quarterlySMAs <- monthlySMAs[index(quarterlyPrices),]
    smaFilter <- quarterlyPrices > quarterlySMAs
  } else if (movAvgType=="daily") {
    smas <- xts(apply(prices, 2, SMA, n=nDaySMA),
    quarterlySMAs <- smas[index(quarterlyPrices),]
    smaFilter <- quarterlyPrices > quarterlySMAs
  } else {
    stop("invalid moving average type")
  finalPos <- rankPos*smaFilter
  finalPos <- finalPos[![,1]),]
  cash <- xts(1-rowSums(finalPos),
  finalPos <- merge(finalPos, cash, join='inner')
  prices <- merge(prices, cashPrices, join='inner')
  returns <- Return.calculate(prices)
  stratRets <- Return.portfolio(returns, finalPos)
  if(returnNames) {
    findNames <- function(pos) {
    tmp <- apply(finalPos, 1, findNames)
    assetNames <- xts(tmp,
    return(list(assetNames, stratRets))

The one change I made is this:

  movAvgType = movAvgType[1]
  if(movAvgType=="monthly") {
    monthlyPrices <- prices[monthlyEps,]
    monthlySMAs <- xts(apply(monthlyPrices, 2, SMA, n=nMonthSMA),
    quarterlySMAs <- monthlySMAs[index(quarterlyPrices),]
    smaFilter <- quarterlyPrices > quarterlySMAs
  } else if (movAvgType=="daily") {
    smas <- xts(apply(prices, 2, SMA, n=nDaySMA),
    quarterlySMAs <- smas[index(quarterlyPrices),]
    smaFilter <- quarterlyPrices > quarterlySMAs
  } else {
    stop("invalid moving average type")

In essence, it allows the function to use either a monthly-calculated moving average, or a daily, which is then subset to the quarterly frequency of the rest of the data.

(I also allow the function to return the names of the selected securities.)

So now we can do two tests:

1) The initial parameter settings (20-day short-term momentum, 105-day long-term momentum, equal weigh their ranks (tiebreaker to the long-term), and use a 3-month SMA to filter)
2) The same exact parameter settings, except a 63-day SMA for the filter.

Here’s the code to do that.

#get our data from yahoo, use adjusted prices
symbols <- c("NAESX", #small cap
             "PREMX", #emerging bond
             "VEIEX", #emerging markets
             "VFICX", #intermediate investment grade
             "VFIIX", #GNMA mortgage
             "VFINX", #S&P 500 index
             "VGSIX", #MSCI REIT
             "VGTSX", #total intl stock idx
             "VUSTX") #long term treasury (cash)

getSymbols(symbols, from="1990-01-01")
prices <- list()
for(i in 1:length(symbols)) {
  prices[[i]] <- Ad(get(symbols[i]))  
prices <-, prices)
colnames(prices) <- gsub("\\.[A-z]*", "", colnames(prices))

monthlySMAqts <- qts(prices, returnNames=TRUE)
dailySMAqts <- qts(prices, wRankShort=.95, wRankLong=1.05, movAvgType = "daily", returnNames=TRUE)

retsComparison <- cbind(monthlySMAqts[[2]], dailySMAqts[[2]])
colnames(retsComparison) <- c("monthly SMA qts", "daily SMA qts")
retsComparison <- retsComparison["2003::"]
rbind(table.AnnualizedReturns(retsComparison["2003::"]), maxDrawdown(retsComparison["2003::"]))

And here are the results:


                          monthly SMA qts daily SMA qts
Annualized Return               0.2745000     0.2114000
Annualized Std Dev              0.1725000     0.1914000
Annualized Sharpe (Rf=0%)       1.5915000     1.1043000
Worst Drawdown                  0.1911616     0.3328411

With the corresponding equity curves:

Here are the several instances in which the selections do not match thanks to the filters:

selectedNames <- cbind(monthlySMAqts[[1]], dailySMAqts[[1]])
colnames(selectedNames) <- c("Monthly SMA Filter", "Daily SMA Filter")
differentSelections <- selectedNames[selectedNames[,1]!=selectedNames[,2],]

With the results:

           Monthly SMA Filter Daily SMA Filter
1997-03-31 "VGSIX"            "cash"          
2007-12-31 "cash"             "PREMX"         
2008-06-30 "cash"             "VFIIX"         
2008-12-31 "cash"             "NAESX"         
2011-06-30 "cash"             "NAESX"  

Now, of course, many can make the arguments that Yahoo’s data is junk, my backtest doesn’t reflect reality, etc., which would essentially miss the point: this data here, while not a perfect realization of the reality of Planet Earth, may as well have been valid (you know, like all the academics, who use various simulation techniques to synthesize more data or explore other scenarios?). All I did here was change the filter to something logically comparable (that is, computing the moving average filter on a different time-scale, which does not in any way change the investment logic). From 2003 onward, this change only affected the strategy in four places. However, those instances were enough to create some noticeable changes (for the worse) in the strategy’s performance. Essentially, the downside of rankings-based strategies are when the overall number of selected instruments (in this case, ONE!) is small, a few small changes in parameters, data, etc. can lead to drastically different results.

As I write this, Cliff Smith already has ideas as to how to counteract this phenomenon. However, unto my experience, once a strategy starts getting into “how do we smooth out that one bump on the equity curve” territory, I think it’s time to go back and re-examine the strategy altogether. In my opinion, while the idea of momentum is of course, sound, with a great deal of literature devoted to it, the idea of selecting just one instrument at a time as the be-all-end-all strategy does not sit well with me. However, to me, QTS nevertheless presents an interesting framework for analyzing small subgroups of securities, and using it as one layer of an overarching strategy framework, such that the return streams are sub-strategies, instead of raw instruments.

Thanks for reading.

NOTE: I am a freelance consultant in quantitative analysis on topics related to this blog. If you have contract or full time roles available for proprietary research that could benefit from my skills, please contact me through my LinkedIn here.

10 thoughts on “The Downside of Rankings-Based Strategies

  1. Pingback: The Whole Street’s Daily Wrap for 2/26/2015 | The Whole Street

  2. The quality of data can indeed explain the discrepancies you highlighted between a daily vs monthly computed SMA. Statistically, you have 21 times more chances to come across an error when using daily data compared to monthly data. But let’s not forget that mathematically, monthly average are most likely to be different than daily one, This is always true, so perfect the quality of the data is. For example, let’s consider a fictitious time series looking like this (and assuming the prices stick to reality – no error due to the data provider):
    Day1 : 100
    Day 2 to Day 21: 90
    Day 22 (which corresponds to Day 1 of next month) : 100

    monthly SMA = [ 100 + 100 ] / 2 = 100
    daily SMA = [ 100 + 20*90 + 100 ] / 22 = 90.9

    Therefore, the results being mathematically different in most of the cases, no wonder that subsequent calculation that use these results as inputs are different too.


  3. Agree with Tonio. There is no reason to expect daily SMA to be the same as monthly SMA.

    First rule of developing trading strategies: optimize all you wnat but distrust the results until there is ample reason not to distrust them.

  4. Another issue I see in most ranking strategy articles is the use of dividend adjusted data for both ranking/signals and returns. To calculate more realistic returns, it seems signals/ranking should be calculated on the non-adjusted price time series, with returns using the adjusted price time series data.

    This process takes two steps: 1) run the ranking strategy on the non-adjusted time series, and extract the trade entry and exit dates and corresponding symbols. 2) Use the data from step 1 to select the trade entry and exit dates and corresponding symbols on the adjusted price time series. Use the outputs from step 2 to calculate the returns from the rotation system.



    • Hi Dave,

      Thanks for bringing this into the discussion. I have brought this up in the past as this is a legitimate concern. There are several “gotcha’s” in ETF rotation strategies and using adjusted data is one of them. No one can trade on adjusted data and obviously, they can’t do signal generation in real-time adjusted data either. All back testing by its nature always over estimates return and under estimates risk – using adjusted data is one reason.


  5. Why would anyone expect a daily 63day-MA to be the same as a 3-month-MA in the first place ? Imagine a stock trading at 100 at the beginning of a month and gaining 1$ until the last day where the stock loses all his intramonthly gains and ends unchanged at 100. Repeat this monthly pattern for several months. The monthly MA will stay at 100 and the daily MA will stay at 109 !
    (`series <- rep(c(100:119,100),6)
    plot(rollmean(series,63),type='l')` )

    So in one case you compare the monthly closing of 100 with 100 (monthly MA) in the case of using the daily MA the close is compared to 109 !

    This certainly is an extreme example to make the point, but if we think of some of the seasonal patterns we see in the stock market this difference has real practical implications.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s