Why Backtesting On Individual Legs In A Spread Is A BAD Idea

So after reading the last post, the author of quantstrat had mostly critical feedback, mostly of the philosophy that prompted its writing in the first place. Basically, the reason I wrote it, as I stated before, is that I’ve seen many retail users of quantstrat constantly ask “how do I model individual spread instruments”, and otherwise try to look like they’re sophisticated by trading spreads.

The truth is that real professionals use industrial-strength tools to determine their intraday hedge ratios (such a tool is called a spreader). The purpose of quantstrat is not to be an execution modeling system, but to be a *strategy* modeling system. Basically, the purpose of your backtest isn’t to look at individual instruments, since in the last post, the aggregate trade statistics told us absolutely nothing about how our actual spread trading strategy performed. The backtest was a mess as far as the analytics were concerned, and thus rendering it more or less useless. So this post, by request of the author of quantstrat, is about how to do the analysis better, and looking at what matters more–the actual performance of the strategy on the actual relationship being traded–namely, the *spread*, rather than the two components.

So, without further ado, let’s look at the revised code:


getSymbols("UNG", from="1990-01-01")
getSymbols("DGAZ", from="1990-01-01")
getSymbols("UGAZ", from="1990-01-01")
UNG <- UNG["2012-02-22::"]
UGAZ <- UGAZ["2012-02-22::"]

spread <- 3*OHLC(UNG) - OHLC(UGAZ)

symbols <- c("spread")
stock(symbols, currency="USD", multiplier=1)

strategy.st <- portfolio.st <- account.st <-"spread_strategy_done_better"
initPortf(portfolio.st, symbols=symbols, initDate=initDate, currency='USD')
initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD')
initOrders(portfolio.st, initDate=initDate)
strategy(strategy.st, store=TRUE)

#### paramters

nEMA = 20

### indicator

add.indicator(strategy.st, name="EMA",
              arguments=list(x=quote(Cl(mktdata)), n=nEMA),

### signals

add.signal(strategy.st, name="sigCrossover",
           arguments=list(columns=c("Close", "EMA.ema"), relationship="gt"),

add.signal(strategy.st, name="sigCrossover",
           arguments=list(columns=c("Close", "EMA.ema"), relationship="lt"),

### rules

add.rule(strategy.st, name="ruleSignal", 
         arguments=list(sigcol="longEntry", sigval=TRUE, ordertype="market", 
                        orderside="long", replace=FALSE, prefer="Open", orderqty=1), 
         type="enter", path.dep=TRUE)

add.rule(strategy.st, name="ruleSignal", 
         arguments=list(sigcol="longExit", sigval=TRUE, orderqty="all", ordertype="market", 
                        orderside="long", replace=FALSE, prefer="Open"), 
         type="exit", path.dep=TRUE)

#apply strategy
t1 <- Sys.time()
out <- applyStrategy(strategy=strategy.st,portfolios=portfolio.st)
t2 <- Sys.time()

In this case, things are a LOT simpler. Rather than jumping through the hoops of pre-computing an indicator, along with the shenanigans of separate rules for both the long and the short end, we simply have a spread as it’s theoretically supposed to work–three of an unleveraged ETF against the 3x leveraged ETF, and we can go long the spread, or short the spread. In this case, the dynamic seems to be on the up, and we want to capture that.

So how did we do?

#set up analytics
dateRange <- time(getPortfolio(portfolio.st)$summary)[-1]

#trade statistics
tStats <- tradeStats(Portfolios = portfolio.st, use="trades", inclZeroDays=FALSE)
tStats[,4:ncol(tStats)] <- round(tStats[,4:ncol(tStats)], 2)
(aggPF <- sum(tStats$Gross.Profits)/-sum(tStats$Gross.Losses))
(aggCorrect <- mean(tStats$Percent.Positive))
(numTrades <- sum(tStats$Num.Trades))
(meanAvgWLR <- mean(tStats$Avg.WinLoss.Ratio[tStats$Avg.WinLoss.Ratio < Inf], na.rm=TRUE))

And here’s the output:

> print(data.frame(t(tStats[,-c(1,2)])))
Num.Txns            76.00
Num.Trades          38.00
Net.Trading.PL       9.87
Avg.Trade.PL         0.26
Med.Trade.PL        -0.10
Largest.Winner       7.76
Largest.Loser       -1.06
Gross.Profits       21.16
Gross.Losses       -11.29
Std.Dev.Trade.PL     1.68
Percent.Positive    39.47
Percent.Negative    60.53
Profit.Factor        1.87
Avg.Win.Trade        1.41
Med.Win.Trade        0.36
Avg.Losing.Trade    -0.49
Med.Losing.Trade    -0.46
Avg.Daily.PL         0.26
Med.Daily.PL        -0.10
Std.Dev.Daily.PL     1.68
Ann.Sharpe           2.45
Max.Drawdown        -4.02
Profit.To.Max.Draw   2.46
Avg.WinLoss.Ratio    2.87
Med.WinLoss.Ratio    0.78
Max.Equity          13.47
Min.Equity          -1.96
End.Equity           9.87
> (aggPF <- sum(tStats$Gross.Profits)/-sum(tStats$Gross.Losses))
[1] 1.874225
> (aggCorrect <- mean(tStats$Percent.Positive))
[1] 39.47
> (numTrades <- sum(tStats$Num.Trades))
[1] 38
> (meanAvgWLR <- mean(tStats$Avg.WinLoss.Ratio[tStats$Avg.WinLoss.Ratio < Inf], na.rm=TRUE))
[1] 2.87

In other words, the typical profile for a trend follower, rather than the uninformative analytics from the last post. Furthermore, the position sizing and equity curve chart actually make sense now. Here they are.

To conclude, while it’s possible to model spreads using individual legs, it makes far more sense in terms of analytics to actually examine the performance of the strategy on the actual relationship being traded, which is the spread itself. Furthermore, after constructing the spread as a synthetic instrument, it can be treated like any other regular instrument in the context of analysis in quantstrat.

Thanks for reading.

NOTE: I am a freelance consultant in quantitative analysis on topics related to this blog. If you have contract or full time roles available for proprietary research that could benefit from my skills, please contact me through my LinkedIn here.


4 thoughts on “Why Backtesting On Individual Legs In A Spread Is A BAD Idea

  1. Pingback: The Whole Street’s Daily Wrap for 12/31/2014 | The Whole Street

  2. great post. But I am not quite sure I understand the output of the trade stats units. Since we are trading a spread how can we convert the values back to a notional? How would this work if the instruments were in 2 different currencies?

    • The point of quantstrat is to backtest the strategy. It’s the responsibility of the user to get his ducks in a row by converting all instruments to one currency — that is, by dividing one instrument by the exchange rate of the other instrument. EG if you were spreading a US stock against an AUD stock, then you’d want to convert the AUD prices to USD prices before creating the spread to begin with.

      Converting back to a notional also seems to be self-defeating. Again, it’s about the statistics of the trading strategy. Other than that, see the previous post on how to do it the less recommended way.

  3. Ilya,

    While this post makes total sense from the Trade Analytics stand point, it is easy to lose sight of what is being traded especially if the constructed spread involves hedge ratios etc..

    See this post and why the constructed spread trading makes astronomical returns, while the actual account trades generate a loss. My explanation is at the bottom of the post in the comments section.


    So for a retail trader, it would be easier to see how the individual legs trade and see the final equity curve (like you did in the part I of this) to know if the strategy works or not rather than getting lost in complex calculations.

    To be clear, I agree that from a trade analytics stand point trading the synthetic spread helps. But need to be careful of what is being traded.

    As always, Thanks for your great work.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s