So, now that I have the ability to generate a term structure and constant expiry contracts, I decided to revisit some of the strategies on Volatility Made Simple and see if any of them are any good (long story short: all of the publicly detailed ones aren’t so hot besides mine–they either have a massive drawdown in-sample around the time of the crisis, or a massive drawdown out-of-sample).

Why this strategy? Because it seemed different from most of the usual term structure ratio trades (of which mine is an example), so I thought I’d check out how it did since its first publishing date, and because it’s rather easy to understand.

Here’s the strategy:

Take XIV, VXX, ZIV, VXZ, and SHY (this last one as the “risk free” asset), and at the close, invest in whichever has had the highest 83 day momentum (this was the result of optimization done on volatilityMadeSimple).

Here’s the code to do this in R, using the Quandl EOD database. There are two variants tested–observe the close, buy the close (AKA magical thinking), and observe the close, buy tomorrow’s close.

require(quantmod) require(PerformanceAnalytics) require(TTR) require(Quandl) Quandl.api_key("yourKeyHere") symbols <- c("XIV", "VXX", "ZIV", "VXZ", "SHY") prices <- list() for(i in 1:length(symbols)) { price <- Quandl(paste0("EOD/", symbols[i]), start_date="1990-12-31", type = "xts")$Adj_Close colnames(price) <- symbols[i] prices[[i]] <- price } prices <- na.omit(do.call(cbind, prices)) returns <- na.omit(Return.calculate(prices)) # find highest asset, assign column names topAsset <- function(row, assetNames) { out <- row==max(row, na.rm = TRUE) names(out) <- assetNames out <- data.frame(out) return(out) } # compute momentum momentums <- na.omit(xts(apply(prices, 2, ROC, n = 83), order.by=index(prices))) # find highest asset each day, turn it into an xts highestMom <- apply(momentums, 1, topAsset, assetNames = colnames(momentums)) highestMom <- xts(t(do.call(cbind, highestMom)), order.by=index(momentums)) # observe today's close, buy tomorrow's close buyTomorrow <- na.omit(xts(rowSums(returns * lag(highestMom, 2)), order.by=index(highestMom))) # observe today's close, buy today's close (aka magic thinking) magicThinking <- na.omit(xts(rowSums(returns * lag(highestMom)), order.by=index(highestMom))) out <- na.omit(cbind(buyTomorrow, magicThinking)) colnames(out) <- c("buyTomorrow", "magicalThinking") # results charts.PerformanceSummary(out['2014-04-11::'], legend.loc = 'top') rbind(table.AnnualizedReturns(out['2014-04-11::']), maxDrawdown(out['2014-04-11::']))

Pretty simple.

Here are the results.

> rbind(table.AnnualizedReturns(out['2014-04-11::']), maxDrawdown(out['2014-04-11::'])) buyTomorrow magicalThinking Annualized Return -0.0320000 0.0378000 Annualized Std Dev 0.5853000 0.5854000 Annualized Sharpe (Rf=0%) -0.0547000 0.0646000 Worst Drawdown 0.8166912 0.7761655

Looks like this strategy didn’t pan out too well. Just a daily reminder that if you’re using fine grid-search to select a particularly good parameter (EG n = 83 days? Maybe 4 21-day trading months, but even that would have been n = 82), you’re asking for a visit from, in the words of Mr. Tony Cooper, a visit from the grim reaper.

****

Moving onto another topic, whenever Dr. Jonathan Kinlay posts something that I think I can replicate that I’d be very wise to do so, as he is a very skilled and experienced practitioner (and also includes me on his blogroll).

A topic that Dr. Kinlay covered is the idea of beta convexity–namely, that an asset’s beta to a benchmark may be different when the benchmark is up as compared to when it’s down. Essentially, it’s the idea that we want to weed out firms that are what I’d deem as “losers in disguise”–I.E. those that act fine when times are good (which is when we really don’t care about diversification, since everything is going up anyway), but do nothing during bad times.

The beta convexity is calculated quite simply: it’s the beta of an asset to a benchmark when the benchmark has a positive return, minus the beta of an asset to a benchmark when the benchmark has a negative return, then squaring the difference. That is, (beta_bench_positive – beta_bench_negative) ^ 2.

Here’s some R code to demonstrate this, using IBM vs. the S&P 500 since 1995.

ibm <- Quandl("EOD/IBM", start_date="1995-01-01", type = "xts") ibmRets <- Return.calculate(ibm$Adj_Close) spy <- Quandl("EOD/SPY", start_date="1995-01-01", type = "xts") spyRets <- Return.calculate(spy$Adj_Close) rets <- na.omit(cbind(ibmRets, spyRets)) colnames(rets) <- c("IBM", "SPY") betaConvexity <- function(Ra, Rb) { positiveBench <- Rb[Rb > 0] assetPositiveBench <- Ra[index(positiveBench)] positiveBeta <- CAPM.beta(Ra = assetPositiveBench, Rb = positiveBench) negativeBench <- Rb[Rb < 0] assetNegativeBench <- Ra[index(negativeBench)] negativeBeta <- CAPM.beta(Ra = assetNegativeBench, Rb = negativeBench) out <- (positiveBeta - negativeBeta) ^ 2 return(out) } betaConvexity(rets$IBM, rets$SPY)

For the result:

> betaConvexity(rets$IBM, rets$SPY) [1] 0.004136034

Thanks for reading.

NOTE: I am always looking to network, and am currently actively looking for full-time opportunities which may benefit from my skill set. If you have a position which may benefit from my skills, do not hesitate to reach out to me. My LinkedIn profile can be found here.

]]>

First off, I would like to thank one Matthew Barry, for helping me modify my HRP algorithm so as to not use the global environment for recursion. You can find his github here.

Here is the modified HRP code.

covMat <- read.csv('cov.csv', header = FALSE) corMat <- read.csv('corMat.csv', header = FALSE) clustOrder <- hclust(dist(corMat), method = 'single')$order getIVP <- function(covMat) { invDiag <- 1/diag(as.matrix(covMat)) weights <- invDiag/sum(invDiag) return(weights) } getClusterVar <- function(covMat, cItems) { covMatSlice <- covMat[cItems, cItems] weights <- getIVP(covMatSlice) cVar <- t(weights) %*% as.matrix(covMatSlice) %*% weights return(cVar) } getRecBipart <- function(covMat, sortIx) { w <- rep(1,ncol(covMat)) w <- recurFun(w, covMat, sortIx) return(w) } recurFun <- function(w, covMat, sortIx) { subIdx <- 1:trunc(length(sortIx)/2) cItems0 <- sortIx[subIdx] cItems1 <- sortIx[-subIdx] cVar0 <- getClusterVar(covMat, cItems0) cVar1 <- getClusterVar(covMat, cItems1) alpha <- 1 - cVar0/(cVar0 + cVar1) # scoping mechanics using w as a free parameter w[cItems0] <- w[cItems0] * alpha w[cItems1] <- w[cItems1] * (1-alpha) if(length(cItems0) > 1) { w <- recurFun(w, covMat, cItems0) } if(length(cItems1) > 1) { w <- recurFun(w, covMat, cItems1) } return(w) } out <- getRecBipart(covMat, clustOrder) out

With covMat and corMat being from the last post. In fact, this function can be further modified by encapsulating the clustering order within the getRecBipart function, but in the interest of keeping the code as similar to Marcos Lopez de Prado’s code as I could, I’ll leave this here.

Anyhow, the backtest will follow. One thing I will mention is that I’m using Quandl’s EOD database, as Yahoo has really screwed up their financial database (I.E. some sector SPDRs have broken data, dividends not adjusted, etc.). While this database is a $50/month subscription, I believe free users can access it up to 150 times in 60 days, so that should be enough to run backtests from this blog, so long as you save your downloaded time series for later use by using write.zoo.

This code needs the tseries library for the portfolio.optim function for the minimum variance portfolio (Dr. Kris Boudt has a course on this at datacamp), and the other standard packages.

A helper function for this backtest (and really, any other momentum rotation backtest) is the appendMissingAssets function, which simply adds on assets not selected to the final weighting and re-orders the weights by the original ordering.

require(tseries) require(PerformanceAnalytics) require(quantmod) require(Quandl) Quandl.api_key("YOUR_AUTHENTICATION_HERE") # not displaying my own api key, sorry # function to append missing (I.E. assets not selected) asset names and sort into original order appendMissingAssets <- function(wts, allAssetNames, wtsDate) { absentAssets <- allAssetNames[!allAssetNames %in% names(wts)] absentWts <- rep(0, length(absentAssets)) names(absentWts) <- absentAssets wts <- c(wts, absentWts) wts <- xts(t(wts), order.by=wtsDate) wts <- wts[,allAssetNames] return(wts) }

Next, we make the call to Quandl to get our data.

symbols <- c("SPY", "VGK", "EWJ", "EEM", "VNQ", "RWX", "IEF", "TLT", "DBC", "GLD") rets <- list() for(i in 1:length(symbols)) { # quandl command to download from EOD database. Free users should use write.zoo in this loop. returns <- Return.calculate(Quandl(paste0("EOD/", symbols[i]), start_date="1990-12-31", type = "xts")$Adj_Close) colnames(returns) <- symbols[i] rets[[i]] <- returns } rets <- na.omit(do.call(cbind, rets))

While Josh Ulrich fixed quantmod to actually get Yahoo data after Yahoo broke the API, the problem is that the Yahoo data is now garbage as well, and I’m not sure how much Josh Ulrich can do about that. I really hope some other provider can step up and provide free, usable EOD data so that I don’t have to worry about readers not being able to replicate the backtest, as my policy for this blog is that readers should be able to replicate the backtests so they don’t just nod and take my word for it. If you are or know of such a provider, please leave a comment so that I can let the blog readers know all about you.

Next, we initialize the settings for the backtest.

invVolWts <- list() minVolWts <- list() hrpWts <- list() ep <- endpoints(rets, on = "months") nMonths = 6 # month lookback (6 as per parameters from allocateSmartly) nVol = 20 # day lookback for volatility (20 ibid)

While the AAA backtest actually uses a 126 day lookback instead of a 6 month lookback, as it trades at the end of every month, that’s effectively a 6 month lookback, give or take a few days out of 126, but the code is less complex this way.

Next, we have our actual backtest.

for(i in 1:(length(ep)-nMonths)) { # get returns subset and compute absolute momentum retSubset <- rets[c(ep[i]:ep[(i+nMonths)]),] retSubset <- retSubset[-1,] moms <- Return.cumulative(retSubset) # select top performing assets and subset returns for them highRankAssets <- rank(moms) >= 6 # top 5 assets posReturnAssets <- moms > 0 # positive momentum assets selectedAssets <- highRankAssets & posReturnAssets # intersection of the above selectedSubset <- retSubset[,selectedAssets] # subset returns slice if(sum(selectedAssets)==0) { # if no qualifying assets, zero weight for period wts <- xts(t(rep(0, ncol(retSubset))), order.by=last(index(retSubset))) colnames(wts) <- colnames(retSubset) invVolWts[[i]] <- minVolWts[[i]] <- hrpWts[[i]] <- wts } else if (sum(selectedAssets)==1) { # if one qualifying asset, invest fully into it wts <- xts(t(rep(0, ncol(retSubset))), order.by=last(index(retSubset))) colnames(wts) <- colnames(retSubset) wts[, which(selectedAssets==1)] <- 1 invVolWts[[i]] <- minVolWts[[i]] <- hrpWts[[i]] <- wts } else { # otherwise, use weighting algorithms cors <- cor(selectedSubset) # correlation volSubset <- tail(selectedSubset, nVol) # 20 day volatility vols <- StdDev(volSubset) covs <- t(vols) %*% vols * cors # minimum volatility using portfolio.optim from tseries minVolRets <- t(matrix(rep(1, sum(selectedAssets)))) minVolWt <- portfolio.optim(x=minVolRets, covmat = covs)$pw names(minVolWt) <- colnames(covs) minVolWt <- appendMissingAssets(minVolWt, colnames(retSubset), last(index(retSubset))) minVolWts[[i]] <- minVolWt # inverse volatility weights invVols <- 1/vols invVolWt <- invVols/sum(invVols) invNames <- colnames(invVolWt) invVolWt <- as.numeric(invVolWt) names(invVolWt) <- invNames invVolWt <- appendMissingAssets(invVolWt, colnames(retSubset), last(index(retSubset))) invVolWts[[i]] <- invVolWt # hrp weights clustOrder <- hclust(dist(cors), method = 'single')$order hrpWt <- getRecBipart(covs, clustOrder) names(hrpWt) <- colnames(covs) hrpWt <- appendMissingAssets(hrpWt, colnames(retSubset), last(index(retSubset))) hrpWts[[i]] <- hrpWt } }

In a few sentences, this is what happens:

The algorithm takes a subset of the returns (the past six months at every month), and computes absolute momentum. It then ranks the ten absolute momentum calculations, and selects the intersection of the top 5, and those with a return greater than zero (so, a dual momentum calculation).

If no assets qualify, the algorithm invests in nothing. If there’s only one asset that qualifies, the algorithm invests in that one asset. If there are two or more qualifying assets, the algorithm computes a covariance matrix using 20 day volatility multiplied with a 126 day correlation matrix (that is, sd_20′ %*% sd_20 * (elementwise) cor_126. It then computes normalized inverse volatility weights using the volatility from the past 20 days, a minimum variance portfolio with the portfolio.optim function, and lastly, the hierarchical risk parity weights using the HRP code above from Marcos Lopez de Prado’s paper.

Lastly, the program puts together all of the weights, and adds a cash investment for any period without any investments.

invVolWts <- round(do.call(rbind, invVolWts), 3) # round for readability minVolWts <- round(do.call(rbind, minVolWts), 3) hrpWts <- round(do.call(rbind, hrpWts), 3) # allocate to cash if no allocation made due to all negative momentum assets invVolWts$cash <- 0; invVolWts$cash <- 1-rowSums(invVolWts) hrpWts$cash <- 0; hrpWts$cash <- 1-rowSums(hrpWts) minVolWts$cash <- 0; minVolWts$cash <- 1-rowSums(minVolWts) # cash value will be zero rets$cash <- 0 # compute backtest returns invVolRets <- Return.portfolio(R = rets, weights = invVolWts) minVolRets <- Return.portfolio(R = rets, weights = minVolWts) hrpRets <- Return.portfolio(R = rets, weights = hrpWts)

Here are the results:

compare <- cbind(invVolRets, minVolRets, hrpRets) colnames(compare) <- c("invVol", "minVol", "HRP") charts.PerformanceSummary(compare) rbind(table.AnnualizedReturns(compare), maxDrawdown(compare), CalmarRatio(compare))

invVol minVol HRP Annualized Return 0.0872000 0.0724000 0.0792000 Annualized Std Dev 0.1208000 0.1025000 0.1136000 Annualized Sharpe (Rf=0%) 0.7221000 0.7067000 0.6968000 Worst Drawdown 0.1548801 0.1411368 0.1593287 Calmar Ratio 0.5629882 0.5131956 0.4968234

In short, in the context of a small, carefully-selected and allegedly diversified (I’ll let Adam Butler speak for that one) universe dominated by the process of which assets to invest in as opposed to how much, the theoretical upsides of an algorithm which simultaneously exploits a covariance structure without needing to invert a covariance matrix can be lost.

However, this test (albeit from 2007 onwards, thanks to ETF inception dates combined with lookback burn-in) confirms what Adam Butler himself told me, which is that HRP hasn’t impressed him, and from this backtest, I can see why. However, in the context of dual momentum rank selection, I’m not convinced that any weighting scheme will realize much better performance than any other.

Thanks for reading.

NOTE: I am always interested in networking and hearing about full-time opportunities related to my skill set. My linkedIn profile can be found here.

]]>

This is a paper that I struggled with until I ran the code in Python (I have anaconda installed but have trouble installing some packages such as keras because I’m on windows…would love to have someone walk me through setting up a Linux dual-boot), as I assumed that the clustering algorithm actually was able to concretely group every asset into a particular cluster (I.E. ETF 1 would be in cluster 1, ETF 2 in cluster 3, etc.). Turns out, that isn’t at all the case.

Here’s how the algorithm actually works.

First off, it computes a covariance and correlation matrix (created from simulated data in Marcos’s paper). Next, it uses a hierarchical clustering algorithm on a distance-transformed correlation matrix, with the “single” method (I.E. friend of friends–do ?hclust in R to read up more on this). The key output here is the order of the assets from the clustering algorithm. Note well: this is the only relevant artifact of the entire clustering algorithm.

Using this order, it then uses an algorithm that does the following:

Initialize a vector of weighs equal to 1 for each asset.

Then, run the following recursive algorithm:

1) Break the order vector up into two equal-length (or as close to equal length) lists as possible.

2) For each half of the list, compute the inverse variance weights (that is, just the diagonal) of the covariance matrix slice containing the assets of interest, and then compute the variance of the cluster when multiplied by the weights (I.E. w’ * S^2 * w).

3) Then, do a basic inverse-variance weight for the two clusters. Call the weight of cluster 0 alpha = 1-cluster_variance_0/(cluster_variance_0 + cluster_variance_1), and the weight of cluster 1 its complement. (1 – alpha).

4) Multiply all assets in the original vector of weights containing assets in cluster 0 with the weight of cluster 0, and all weights containing assets in cluster 1 with the weight of cluster 1. That is, weights[index_assets_cluster_0] *= alpha, weights[index_assets_cluster_1] *= 1-alpha.

5) Lastly, if the list isn’t of length 1 (that is, not a single asset), repeat this entire process until every asset is its own cluster.

Here is the implementation in R code.

First off, the correlation matrix and the covariance matrix for use in this code, obtained from Marcos Lopez De Prado’s code in the appendix in his paper.

> covMat V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 1 1.000647799 -0.003050479 0.010033224 -0.010759689 -0.005036503 0.008762563 0.998201625 -0.001393196 -0.001254522 -0.009365991 2 -0.003050479 1.009021349 0.008613817 0.007334478 -0.009492688 0.013031817 -0.009420720 -0.015346223 1.010520047 1.013334849 3 0.010033224 0.008613817 1.000739363 -0.000637885 0.001783293 1.001574768 0.006385368 0.001922316 0.012902050 0.007997935 4 -0.010759689 0.007334478 -0.000637885 1.011854725 0.005759976 0.000905812 -0.011912269 0.000461894 0.012572661 0.009621670 5 -0.005036503 -0.009492688 0.001783293 0.005759976 1.005835878 0.005606343 -0.009643250 1.008567427 -0.006183035 -0.007942770 6 0.008762563 0.013031817 1.001574768 0.000905812 0.005606343 1.064309825 0.004413960 0.005780148 0.017185396 0.011601336 7 0.998201625 -0.009420720 0.006385368 -0.011912269 -0.009643250 0.004413960 1.058172027 -0.006755374 -0.008099181 -0.016240271 8 -0.001393196 -0.015346223 0.001922316 0.000461894 1.008567427 0.005780148 -0.006755374 1.074833155 -0.011903469 -0.013738378 9 -0.001254522 1.010520047 0.012902050 0.012572661 -0.006183035 0.017185396 -0.008099181 -0.011903469 1.075346677 1.015220126 10 -0.009365991 1.013334849 0.007997935 0.009621670 -0.007942770 0.011601336 -0.016240271 -0.013738378 1.015220126 1.078586686 > corMat V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 1 1.000000000 -0.003035829 0.010026270 -0.010693011 -0.005020245 0.008490954 0.970062043 -0.001343386 -0.001209382 -0.009015412 2 -0.003035829 1.000000000 0.008572055 0.007258718 -0.009422702 0.012575370 -0.009117080 -0.014736040 0.970108941 0.971348946 3 0.010026270 0.008572055 1.000000000 -0.000633903 0.001777455 0.970485047 0.006205079 0.001853505 0.012437239 0.007698212 4 -0.010693011 0.007258718 -0.000633903 1.000000000 0.005709500 0.000872861 -0.011512172 0.000442908 0.012052964 0.009210090 5 -0.005020245 -0.009422702 0.001777455 0.005709500 1.000000000 0.005418538 -0.009347204 0.969998023 -0.005945165 -0.007625721 6 0.008490954 0.012575370 0.970485047 0.000872861 0.005418538 1.000000000 0.004159261 0.005404237 0.016063910 0.010827955 7 0.970062043 -0.009117080 0.006205079 -0.011512172 -0.009347204 0.004159261 1.000000000 -0.006334331 -0.007592568 -0.015201540 8 -0.001343386 -0.014736040 0.001853505 0.000442908 0.969998023 0.005404237 -0.006334331 1.000000000 -0.011072068 -0.012759610 9 -0.001209382 0.970108941 0.012437239 0.012052964 -0.005945165 0.016063910 -0.007592568 -0.011072068 1.000000000 0.942667300 10 -0.009015412 0.971348946 0.007698212 0.009210090 -0.007625721 0.010827955 -0.015201540 -0.012759610 0.942667300 1.000000000

Now, for the implementation.

This reads in the two matrices above and gets the clustering order.

covMat <- read.csv('cov.csv', header = FALSE) corMat <- read.csv('corMat.csv', header = FALSE) clustOrder <- hclust(dist(corMat), method = 'single')$order

This is the clustering order:

> clustOrder [1] 9 2 10 1 7 3 6 4 5 8

Next, the getIVP (get Inverse Variance Portfolio) and getClusterVar functions (note: I’m trying to keep the naming conventions identical to Dr. Lopez’s paper)

getIVP <- function(covMat) { # get inverse variance portfolio from diagonal of covariance matrix invDiag <- 1/diag(as.matrix(covMat)) weights <- invDiag/sum(invDiag) return(weights) } getClusterVar <- function(covMat, cItems) { # compute cluster variance from the inverse variance portfolio above covMatSlice <- covMat[cItems, cItems] weights <- getIVP(covMatSlice) cVar <- t(weights) %*% as.matrix(covMatSlice) %*% weights return(cVar) }

Next, my code diverges from the code in the paper, because I do not use the list comprehension structure, but instead opt for a recursive algorithm, as I find that style to be more readable.

One wrinkle to note is the use of the double arrow dash operator, to assign to a variable outside the scope of the recurFun function. I assign the initial weights vector w in the global environment, and update it from within the recurFun function. I am aware that it is a faux pas to create variables in the global environment, but my attempts at creating a temporary environment in which to update the weight vector did not produce the updating mechanism I had hoped to, so a little bit of assistance with refactoring this code would be appreciated.

getRecBipart <- function(covMat, sortIx) { # keeping track of weights vector in the global environment assign("w", value = rep(1, ncol(covMat)), envir = .GlobalEnv) # run recursion function recurFun(covMat, sortIx) return(w) } recurFun <- function(covMat, sortIx) { # get first half of sortIx which is a cluster order subIdx <- 1:trunc(length(sortIx)/2) # subdivide ordering into first half and second half cItems0 <- sortIx[subIdx] cItems1 <- sortIx[-subIdx] # compute cluster variances of covariance matrices indexed # on first half and second half of ordering cVar0 <- getClusterVar(covMat, cItems0) cVar1 <- getClusterVar(covMat, cItems1) alpha <- 1 - cVar0/(cVar0 + cVar1) # updating weights outside the function using scoping mechanics w[cItems0] <<- w[cItems0] * alpha w[cItems1] <<- w[cItems1] * (1-alpha) # rerun the function on a half if the length of that half is greater than 1 if(length(cItems0) > 1) { recurFun(covMat, cItems0) } if(length(cItems1) > 1) { recurFun(covMat, cItems1) } }

Lastly, let’s run the function.

out <- getRecBipart(covMat, clustOrder)

With the result (which matches the paper):

> out [1] 0.06999366 0.07592151 0.10838948 0.19029104 0.09719887 0.10191545 0.06618868 0.09095933 0.07123881 0.12790318

So, hopefully this democratizes the use of this technology in R. While I have seen a raw Rcpp implementation and one from the Systematic Investor Toolbox, neither of those implementations satisfied me from a “plug and play” perspective. This implementation solves that issue. Anyone here can copy and paste these functions into their environment and immediately make use of one of the algorithms devised by one of the top minds in quantitative finance.

A demonstration in a backtest using this methodology will be forthcoming.

Thanks for reading.

NOTE: I am always interested in networking and full-time opportunities which may benefit from my skills. Furthermore, I am also interested in project work in the volatility ETF trading space. My linkedin profile can be found here.

]]>

The last time I visited this topic, I created a term structure using publicly available data from the CBOE, along with an external expiry calendar.

The logical next step, of course, is to create constant-expiry contracts, which may or may not be tradable (if your contract expiry is less than 30 days, know that the front month has days in which the time to expiry is more than 30 days).

So here’s where we left off: a way to create a continuous term structure using CBOE settlement VIX data.

So from here, before anything, we need to get VIX data. And while the getSymbols command used to be easier to use, because Yahoo broke its API (what else do you expect from an otherwise-irrelevant, washed-up web 1.0 dinosaur?), it’s not possible to get free Yahoo data at this point in time (in the event that Josh Ulrich doesn’t fix this issue in the near future, I’m open to suggestions for other free sources of data which provide data of reputable quality), so we need to get VIX data from elsewhere (particularly, the CBOE itself, which is a one-stop shop for all VIX-related data…and most likely some other interesting futures as well.)

So here’s how to get VIX data from the CBOE (thanks, all you awesome CBOE people! And a shoutout to all my readers from the CBOE, I’m sure some of you are from there).

VIX <- fread("http://www.cboe.com/publish/scheduledtask/mktdata/datahouse/vixcurrent.csv", skip = 1) VIXdates <- VIX$Date VIX$Date <- NULL; VIX <- xts(VIX, order.by=as.Date(VIXdates, format = '%m/%d/%Y')) spotVix <- Cl(VIX)

Next, there’s a need for some utility functions to help out with identifying which futures contracts to use for constructing synthetics.

# find column with greatest days to expiry less than or equal to desired days to expiry shortDurMinMax <- function(row, daysToExpiry) { return(max(which(row <= daysToExpiry))) } # find column with least days to expiry greater desired days to expiry longDurMinMax <- function(row, daysToExpiry) { return(min(which(row > daysToExpiry))) } # gets the difference between the two latest contracts (either expiry days or price) getLastDiff <- function(row) { indices <- rev(which(!is.na(row))) out <- row[indices[1]] - row[indices[2]] return(out) } # gets the rightmost non-NA value of a row getLastValue <- function(row) { indices <- rev(which(!is.na(row))) out <- row[indices[1]] return(out) }

The first two functions are to determine short-duration and long-duration contracts. Simply, provided a row of data and the desired constant time to expiry, the first function finds the contract with a time closest to expiry less than or equal to the desired amount, while the second function does the inverse.

The next two functions are utilized in the scenario of a function whose time to expiry is greater than the expiry of the longest trading contract. Such a synthetic would obviously not be able to be traded, but can be created for the purposes of using as an indicator. The third function gets the last two non-NA values in a row (I.E. the two last prices, the two last times to expiry), and the fourth one simply gets the rightmost non-NA value in a row.

The algorithm to create a synthetic constant-expiry contract/indicator is divided into three scenarios:

One, in which the desired time to expiry of the contract is shorter than the front month, such as a constant 30-day expiry contract, when the front month has more than 30 days to maturity (such as on Nov 17, 2016), at which point, the weight will be the desired time to expiry over the remaining time to expiry in the front month, and the remainder in spot VIX (another asset that cannot be traded, at least conventionally).

The second scenario is one in which the desired time to expiry is longer than the last traded contract. For instance, if the desire was to create a contract

with a year to expiry when the furthest out is eight months, there obviously won’t be data for such a contract. In such a case, the algorithm is to compute the linear slope between the last two available contracts, and add the extrapolated product of the slope multiplied by the time remaining between the desired and the last contract to the price of the last contract.

Lastly, the third scenario (and the most common one under most use cases) is that of the synthetic for which there is both a trading contract that has less time to expiry than the desired constant rate, and one with more time to expiry. In this instance, a matter of linear algebra (included in the comments) denotes the weight of the short expiry contract, which is (desired – expiry_long)/(expiry_short – expiry_long).

The algorithm iterates through all three scenarios, and due to the mechanics of xts automatically sorting by timestamp, one obtains an xts object in order of dates of a synthetic, constant expiry futures contract.

Here is the code for the function.

constantExpiry <- function(spotVix, termStructure, expiryStructure, daysToExpiry) { # Compute synthetics that are too long (more time to expiry than furthest contract) # can be Inf if no column contains values greater than daysToExpiry (I.E. expiry is 3000 days) suppressWarnings(longCol <- xts(apply(expiryStructure, 1, longDurMinMax, daysToExpiry), order.by=index(termStructure))) longCol[longCol == Inf] <- 10 # xts for too long to expiry -- need a NULL for rbinding if empty tooLong <- NULL # Extend the last term structure slope an arbitrarily long amount of time for those with too long expiry tooLongIdx <- index(longCol[longCol==10]) if(length(tooLongIdx) > 0) { tooLongTermStructure <- termStructure[tooLongIdx] tooLongExpiryStructure <- expiryStructure[tooLongIdx] # difference in price/expiry for longest two contracts, use it to compute a slope priceDiff <- xts(apply(tooLongTermStructure, 1, getLastDiff), order.by = tooLongIdx) expiryDiff <- xts(apply(tooLongExpiryStructure, 1, getLastDiff), order.by = tooLongIdx) slope <- priceDiff/expiryDiff # get longest contract price and compute additional days to expiry from its time to expiry # I.E. if daysToExpiry is 180 and longest is 120, additionalDaysToExpiry is 60 maxDaysToExpiry <- xts(apply(tooLongExpiryStructure, 1, max, na.rm = TRUE), order.by = tooLongIdx) longestContractPrice <- xts(apply(tooLongTermStructure, 1, getLastValue), order.by = tooLongIdx) additionalDaysToExpiry <- daysToExpiry - maxDaysToExpiry # add slope multiplied by additional days to expiry to longest contract price tooLong <- longestContractPrice + additionalDaysToExpiry * slope } # compute synthetics that are too short (less time to expiry than shortest contract) # can be -Inf if no column contains values less than daysToExpiry (I.E. expiry is 5 days) suppressWarnings(shortCol <- xts(apply(expiryStructure, 1, shortDurMinMax, daysToExpiry), order.by=index(termStructure))) shortCol[shortCol == -Inf] <- 0 # xts for too short to expiry -- need a NULL for rbinding if empty tooShort <- NULL tooShortIdx <- index(shortCol[shortCol==0]) if(length(tooShortIdx) > 0) { tooShort <- termStructure[,1] * daysToExpiry/expiryStructure[,1] + spotVix * (1 - daysToExpiry/expiryStructure[,1]) tooShort <- tooShort[tooShortIdx] } # compute everything in between (when time to expiry is between longest and shortest) # get unique permutations for contracts that term structure can create colPermutes <- cbind(shortCol, longCol) colnames(colPermutes) <- c("short", "long") colPermutes <- colPermutes[colPermutes$short > 0,] colPermutes <- colPermutes[colPermutes$long < 10,] regularSynthetics <- NULL # if we can construct synthetics from regular futures -- someone might enter an extremely long expiry # so this may not always be the case if(nrow(colPermutes) > 0) { # pasting long and short expiries into a single string for easier subsetting shortLongPaste <- paste(colPermutes$short, colPermutes$long, sep="_") uniqueShortLongPaste <- unique(shortLongPaste) regularSynthetics <- list() for(i in 1:length(uniqueShortLongPaste)) { # get unique permutation of short-expiry and long-expiry contracts permuteSlice <- colPermutes[which(shortLongPaste==uniqueShortLongPaste[i]),] expirySlice <- expiryStructure[index(permuteSlice)] termStructureSlice <- termStructure[index(permuteSlice)] # what are the parameters? shortCol <- unique(permuteSlice$short); longCol <- unique(permuteSlice$long) # computations -- some linear algebra # S/L are weights, ex_S/ex_L are time to expiry # D is desired constant time to expiry # S + L = 1 # L = 1 - S # S + (1-S) = 1 # # ex_S * S + ex_L * (1-S) = D # ex_S * S + ex_L - ex_L * S = D # ex_S * S - ex_L * S = D - ex_L # S(ex_S - ex_L) = D - ex_L # S = (D - ex_L)/(ex_S - ex_L) weightShort <- (daysToExpiry - expirySlice[, longCol])/(expirySlice[, shortCol] - expirySlice[, longCol]) weightLong <- 1 - weightShort syntheticValue <- termStructureSlice[, shortCol] * weightShort + termStructureSlice[, longCol] * weightLong regularSynthetics[[i]] <- syntheticValue } regularSynthetics <- do.call(rbind, regularSynthetics) } out <- rbind(tooShort, regularSynthetics, tooLong) colnames(out) <- paste0("Constant_", daysToExpiry) return(out) }

And here’s how to use it:

constant30 <- constantExpiry(spotVix = vixSpot, termStructure = termStructure, expiryStructure = expiryStructure, daysToExpiry = 30) constant180 <- constantExpiry(spotVix = vixSpot, termStructure = termStructure, expiryStructure = expiryStructure, daysToExpiry = 180) constantTermStructure <- cbind(constant30, constant180) chart.TimeSeries(constantTermStructure, legend.loc = 'topright', main = "Constant Term Structure")

With the result:

Which means that between the CBOE data itself, and this function that creates constant expiry futures from CBOE spot and futures prices, one can obtain any futures contract, whether real or synthetic, to use as an indicator for volatility trading strategies. This allows for exploration of a wide variety of volatility trading strategies.

Thanks for reading.

NOTE: I am always interested in networking and hearing about full-time opportunities related to my skill set. My linkedin can be found here.

Furthermore, if you are a volatility ETF/futures trading professional, I am interested in possible project-based collaboration. If you are interested, please contact me.

]]>

The TL;DR: 4.5 out of 5 stars.

So, I honestly have very little criticism of the book beyond the fact that the book sort of insinuates as though equity momentum is the be-all-end-all of investing, which is why I deduct a fraction of a point.

Now, for the book itself: first off, unlike other quantitative trading books I’ve read (aside from Andreas Clenow’s), the book outlines a very simple to follow strategy, to the point that it has already been replicated over at AllocateSmartly. (Side note: I think Walter’s resource at Allocate Smartly is probably the single best one-stop shop for reading up on any tactical asset allocation strategy, as it’s a compendium of many strategies in the risk/return profile of the 7-15% CAGR type strategies, and even has a correlation matrix between them all.)

Regarding the rest of the content, Antonacci does a very thorough job of stepping readers through the history/rationale of momentum, and not just that, but also addressing the alternatives to his strategy.

While the “why momentum works” aspect you can find in this book and others on the subject (I.E. Alpha Architect’s Quantitative Momentum book), I do like the section on other alternative assets. Namely, the book touches upon the fact that commodities no longer trend, so a lot of CTAs are taking it on the chin, and that historically, fixed income has performed much worse from an absolute return than equities. Furthermore, smart beta isn’t (smart), and many of these factors have very low aggregate returns (if they’re significant at all, I believe Wesley Gray at Alpha Architect has a blog post stating that they aren’t). There are a fair amount of footnotes for those interested in corroborating the assertions. Suffice to say, when it comes to strategies that don’t need daily micromanagement, when it comes to how far you can get without leverage (essentially, anything outside the space of volatility trading strategies), equity momentum is about as good as you get.

Antonacci then introduces his readers to his GEM (Global Equities Momentum) strategy, which can be explained in a few sentences: at the end of each month, calculate the total 12-month return of SPY, EAFE, and BIL. If BIL has the highest return, buy AGG for that month, otherwise buy the asset with the highest return. Repeat. That’s literally it, and the performance characteristics, on a risk-adjusted basis, are superior to just about any equity fund tied down to a tiny tracking error. Essentially, the reason for that is that equity markets have bear markets, and a dual momentum strategy lets you preserve your gains instead of giving it back in market corrections (I.E. 2000-2003, 2008, etc.) while keeping pace during the good times.

Lastly, Antonacci provides some ideas for possibly improving on GEM. I may examine on these in the future. However, the low-hanging fruit for improving on this strategy, in my opinion, is to find some other strategies that diversify its drawdowns, and raise its risk-adjusted return profile. Even if the total return goes down, I believe that an interactive brokers account can offer some amount of leverage (either 50% or 100%) to boost the total returns back up, or combine a more diversified portfolio with a volatility strategy.

Lastly, the appendix includes the original dual momentum paper, and a monthly return table for GEM going back to 1974.

All in all, this book is about as accessible and comprehensive as you can get on a solid strategy that readers actually *can* implement themselves in their brokerage account of choice (please use IB or Robinhood because there’s no point paying $8-$10 per trade if you’re retail). That said, I still think that there are venues in which to travel if you’re looking to improve your overall portfolio with GEM as a foundation.

Thanks for reading.

NOTEL I am always interested in networking and hearing about full-time roles which can benefit from my skill set. My linkedin profile can be found here.

]]>

So this post, as has been the usual for quite some time, will not be about a strategy, but rather, a tool that can be used for exploring future strategies. Particularly, volatility strategies–which seems to have been a hot topic on this blog some time ago (and might very well be still, especially since the Volatility Made Simple blog has just stopped tracking open-sourced strategies for the past year).

This post’s topic is the VIX term structure–that is, creating a set of continuous contracts–properly rolled according to VIX contract specifications, rather than a hodgepodge of generic algorithms as found on some other websites. The idea is, as of the settlement of a previous day (or whenever the CBOE actually releases their data), you can construct a curve of contracts, and see if it’s in contango (front month cheaper than next month and so on) or backwardation (front month more expensive than next month, etc.).

The first (and most code-intensive) part of the procedure is fairly simple–map the contracts to an expiration date, then put their settlement dates and times to expiry into two separate xts objects, with one column for each contract.

The expiries text file is simply a collection of copied and pasted expiry dates from this site. It includes the January 2018 expiration date. Here is what it looks like:

> head(expiries) V1 V2 V3 1 18 January 2006 2 15 February 2006 3 22 March 2006 4 19 April 2006 5 17 May 2006 6 21 June 2006

require(xts) require(data.table) # 06 through 17 years <- c(paste0("0", c(6:9)), as.character(c(10:17))) # futures months futMonths <- c("F", "G", "H", "J", "K", "M", "N", "Q", "U", "V", "X", "Z") # expiries come from http://www.macroption.com/vix-expiration-calendar/ expiries <- read.table("expiries.txt", header = FALSE, sep = " ") # convert expiries into dates in R dateString <- paste(expiries$V3, expiries$V2, expiries$V1, sep = "-") dates <- as.Date(dateString, format = "%Y-%B-%d") # map futures months to numbers for dates monthMaps <- cbind(futMonths, c("01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12")) monthMaps <- data.frame(monthMaps) colnames(monthMaps) <- c("futureStem", "monthNum") dates <- data.frame(dates) dates$dateMon <- substr(dates$dates, 1, 7) contracts <- expand.grid(futMonths, years) contracts <- paste0(contracts[,1], contracts[,2]) contracts <- c(contracts, "F18") stem <- "https://cfe.cboe.com/Publish/ScheduledTask/MktData/datahouse/CFE_" #contracts <- paste0(stem, contracts, "_VX.csv") masterlist <- list() timesToExpiry <- list() for(i in 1:length(contracts)) { # obtain data contract <- contracts[i] dataFile <- paste0(stem, contract, "_VX.csv") expiryYear <- paste0("20",substr(contract, 2, 3)) expiryMonth <- monthMaps$monthNum[monthMaps$futureStem == substr(contract,1,1)] expiryDate <- dates$dates[dates$dateMon == paste(expiryYear, expiryMonth, sep="-")] data <- suppressWarnings(fread(dataFile)) # create dates dataDates <- as.Date(data$`Trade Date`, format = '%m/%d/%Y') # create time to expiration xts toExpiry <- xts(expiryDate - dataDates, order.by=dataDates) colnames(toExpiry) <- contract timesToExpiry[[i]] <- toExpiry # get settlements settlement <- xts(data$Settle, order.by=dataDates) colnames(settlement) <- contract masterlist[[i]] <- settlement } # cbind outputs masterlist <- do.call(cbind, masterlist) timesToExpiry <- do.call(cbind, timesToExpiry) # NA out zeroes in settlements masterlist[masterlist==0] <- NA

From there, we need to visualize how many contracts are being traded at once on any given day (I.E. what’s a good steady state number for the term structure)?

sumNonNA <- function(row) { return(sum(!is.na(row))) } simultaneousContracts <- xts(apply(masterlist, 1, sumNonNA), order.by=index(masterlist)) chart.TimeSeries(simultaneousContracts)

The result looks like this:

So, 8 contracts (give or take) at any given point in time. This is confirmed by the end of the master list of settlements.

dim(masterlist) tail(masterlist[,135:145])

> dim(masterlist) [1] 3002 145 > tail(masterlist[,135:145]) H17 J17 K17 M17 N17 Q17 U17 V17 X17 Z17 F18 2017-04-18 NA 14.725 14.325 14.525 15.175 15.475 16.225 16.575 16.875 16.925 NA 2017-04-19 NA 14.370 14.575 14.525 15.125 15.425 16.175 16.575 16.875 16.925 NA 2017-04-20 NA NA 14.325 14.325 14.975 15.375 16.175 16.575 16.875 16.900 NA 2017-04-21 NA NA 14.325 14.225 14.825 15.175 15.925 16.350 16.725 16.750 NA 2017-04-24 NA NA 12.675 13.325 14.175 14.725 15.575 16.025 16.375 16.475 17.00 2017-04-25 NA NA 12.475 13.125 13.975 14.425 15.225 15.675 16.025 16.150 16.75

Using this information, an algorithm can create eight continuous contracts, ranging from front month to eight months out. The algorithm starts at the first day of the master list to the first expiry, then moves between expiry windows, and just appends the front month contract, and the next seven contracts to a list, before rbinding them together, and does the same with the expiry structure.

termStructure <- list() expiryStructure <- list() masterDates <- unique(c(first(index(masterlist)), dates$dates[dates$dates %in% index(masterlist)], Sys.Date()-1)) for(i in 1:(length(masterDates)-1)) { subsetDates <- masterDates[c(i, i+1)] dateRange <- paste(subsetDates[1], subsetDates[2], sep="::") subset <- masterlist[dateRange,c(i:(i+7))] subset <- subset[-1,] expirySubset <- timesToExpiry[index(subset), c(i:(i+7))] colnames(subset) <- colnames(expirySubset) <- paste0("C", c(1:8)) termStructure[[i]] <- subset expiryStructure[[i]] <- expirySubset } termStructure <- do.call(rbind, termStructure) expiryStructure <- do.call(rbind, expiryStructure)

Again, one more visualization of when we have a suitable number of contracts:

simultaneousContracts <- xts(apply(termStructure, 1, sumNonNA), order.by=index(termStructure)) chart.TimeSeries(simultaneousContracts)

And in order to preserve the most data, we’ll cut the burn-in period off when we first have 7 contracts trading at once.

first(index(simultaneousContracts)[simultaneousContracts >= 7]) termStructure <- termStructure["2006-10-23::"] expiryStructure <- expiryStructure[index(termStructure)]

So there you have it–your continuous VIX futures contract term structure, as given by the official CBOE settlements. While some may try and simulate a trading strategy based on these contracts, I myself prefer to use them as indicators or features to a model that would rather buy XIV or VXX.

One last trick, for those that want to visualize things, a way to actually visualize the term structure on any given day, in particular, the most recent one in the term structure.

plot(t(coredata(last(termStructure))), type = 'b')

A clear display of contango.

A post on how to compute synthetic constant-expiry contracts (EG constant 30 day expiry contracts) will be forthcoming in the near future.

Thanks for reading.

NOTE: I am currently interested in networking and full-time positions which may benefit from my skills. I may be contacted at my LinkedIn profile found here.

]]>

The first four parts of my nuts and bolts of quantstrat were well received. They are even available as a datacamp course. For those that want to catch up to today’s post, I highly recommend the datacamp course.

To motivate this post, the idea is that say you’re using alternative data that isn’t simply derived from a transformation of the market data itself. I.E. you have a proprietary alternative data stream that may predict an asset’s price, you want to employ aÂ cross-sectional ranking system, or any number of things. How do you do this within the context of quantstrat?

The answer is that it’s as simple as binding a new xts to your asset data, as this demonstration will show.

First, let’s get the setup out of the way.

require(quantstrat) require(PerformanceAnalytics) initDate="1990-01-01" from="2003-01-01" to="2012-12-31" options(width=70) options("getSymbols.warning4.0"=FALSE) currency('USD') Sys.setenv(TZ="UTC") symbols <- 'SPY' suppressMessages(getSymbols(symbols, from=from, to=to, src="yahoo", adjust=TRUE)) stock(symbols, currency="USD", multiplier=1)

Now, we have our non-derived indicator. In this case, it’s a toy example–the value is 1 if the year is odd (I.E. 2003, 2005, 2007, 2009), and 0 if it’s even. We compute that and simply column-bind (cbind) it to the asset data.

nonDerivedIndicator <- as.numeric(as.character(substr(index(SPY), 1, 4)))%%2 == 1 nonDerivedIndicator <- xts(nonDerivedIndicator, order.by=index(SPY)) SPY <- cbind(SPY, nonDerivedIndicator) colnames(SPY)[7] = "nonDerivedIndicator"

Next, we just have a very simple strategy–buy a share of SPY on odd years, sell on even years. That is, buy when the nonDerivedIndicator column crosses above 0.5 (from 0 to 1), and sell when the opposite occurs.

strategy.st <- portfolio.st <- account.st <- "nonDerivedData" rm.strat(strategy.st) initPortf(portfolio.st, symbols=symbols, initDate=initDate, currency='USD') initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD') initOrders(portfolio.st, initDate=initDate) strategy(strategy.st, store=TRUE) add.signal(strategy.st, name = sigThreshold, arguments = list(column = "nonDerivedIndicator", threshold = 0.5, relationship = "gte", cross = TRUE), label = "longEntry") add.signal(strategy.st, name = sigThreshold, arguments = list(column = "nonDerivedIndicator", threshold = 0.5, relationship = "lte", cross = TRUE), label = "longExit") tmp <- applySignals(strategy = strategy.st, mktdata=SPY) add.rule(strategy.st, name="ruleSignal", arguments=list(sigcol="longEntry", sigval=TRUE, ordertype="market", orderside="long", replace=FALSE, prefer="Open", orderqty = 1), type="enter", path.dep=TRUE) add.rule(strategy.st, name="ruleSignal", arguments=list(sigcol="longExit", sigval=TRUE, orderqty="all", ordertype="market", orderside="long", replace=FALSE, prefer="Open"), type="exit", path.dep=TRUE) #apply strategy t1 <- Sys.time() out <- applyStrategy(strategy=strategy.st,portfolios=portfolio.st) t2 <- Sys.time() print(t2-t1) #set up analytics updatePortf(portfolio.st) dateRange <- time(getPortfolio(portfolio.st)$summary)[-1] updateAcct(portfolio.st,dateRange) updateEndEq(account.st)

And the result:

chart.Posn(portfolio.st, 'SPY')

In conclusion, you can create signals based off of any data in quantstrat. Whether that means volatility ratios, fundamental data, cross-sectional ranking, or whatever proprietary alternative data source you may have access to, this very simple process is how you can use quantstrat to add all of those things to your systematic trading backtest research.

Thanks for reading.

Note: I am always interested in full-time opportunities which may benefit from my skills. I have experience in data analytics, asset management, and systematic trading research. If you know of any such opportunities, do not hesitate to contact me on my LinkedIn, found here.

]]>

Before beginning this post, I must give credit where it’s due, to one Mr. Fabrizio Maccallini, the head of structured derivatives at Nordea Markets in London. You can find the rest of the repository he did for Dr. John Ehlers’s Cycle Analytics for Traders on his github. I am grateful and honored that such intelligent and experienced individuals are helping to bring some of Dr. Ehlers’s methods into R.

The point of the Ehlers Autocorrelation Periodogram is to dynamically set a period between a minimum and a maximum period length. While I leave the exact explanation of the mechanic to Dr. Ehlers’s book, for all practical intents and purposes, in my opinion, the punchline of this method is to attempt to remove a massive source of overfitting from trading system creation–namely specifying a lookback period.

SMA of 50 days? 100 days? 200 days? Well, this algorithm takes that possibility of overfitting out of your hands. Simply, specify an upper and lower bound for your lookback, and it does the rest. How well it does it is a topic of discussion for those well-versed in the methodologies of electrical engineering (I’m not), so feel free to leave comments that discuss how well the algorithm does its job, and feel free to blog about it as well.

In any case, here’s the original algorithm code, courtesy of Mr. Maccallini:

AGC <- function(loCutoff = 10, hiCutoff = 48, slope = 1.5) { accSlope = -slope # acceptableSlope = 1.5 dB ratio = 10 ^ (accSlope / 20) if ((hiCutoff - loCutoff) > 0) factor <- ratio ^ (2 / (hiCutoff - loCutoff)); return (factor) } autocorrPeriodogram <- function(x, period1 = 10, period2 = 48, avgLength = 3) { # high pass filter alpha1 <- (cos(sqrt(2) * pi / period2) + sin(sqrt(2) * pi / period2) - 1) / cos(sqrt(2) * pi / period2) hp <- (1 - alpha1 / 2) ^ 2 * (x - 2 * lag(x) + lag(x, 2)) hp <- hp[-c(1, 2)] hp <- filter(hp, (1 - alpha1), method = "recursive") hp <- c(NA, NA, hp) hp <- xts(hp, order.by = index(x)) # super smoother a1 <- exp(-sqrt(2) * pi / period1) b1 <- 2 * a1 * cos(sqrt(2) * pi / period1) c2 <- b1 c3 <- -a1 * a1 c1 <- 1 - c2 - c3 filt <- c1 * (hp + lag(hp)) / 2 leadNAs <- sum(is.na(filt)) filt <- filt[-c(1: leadNAs)] filt <- filter(filt, c(c2, c3), method = "recursive") filt <- c(rep(NA, leadNAs), filt) filt <- xts(filt, order.by = index(x)) # Pearson correlation for each value of lag autocorr <- matrix(0, period2, length(filt)) for (lag in 2: period2) { # Set the average length as M if (avgLength == 0) M <- lag else M <- avgLength autocorr[lag, ] <- runCor(filt, lag(filt, lag), M) } autocorr[is.na(autocorr)] <- 0 # Discrete Fourier transform # Correlate autocorrelation values with the cosine and sine of each period of interest # The sum of the squares of each value represents relative power at each period cosinePart <- sinePart <- sqSum <- R <- Pwr <- matrix(0, period2, length(filt)) for (period in period1: period2) { for (N in 2: period2) { cosinePart[period, ] = cosinePart[period, ] + autocorr[N, ] * cos(2 * N * pi / period) sinePart[period, ] = sinePart[period, ] + autocorr[N, ] * sin(2 * N * pi / period) } sqSum[period, ] = cosinePart[period, ] ^ 2 + sinePart[period, ] ^ 2 R[period, ] <- EMA(sqSum[period, ] ^ 2, ratio = 0.2) } R[is.na(R)] <- 0 # Normalising Power K <- AGC(period1, period2, 1.5) maxPwr <- rep(0, length(filt)) for(period in period1: period2) { for (i in 1: length(filt)) { if (R[period, i] >= maxPwr[i]) maxPwr[i] <- R[period, i] else maxPwr[i] <- K * maxPwr[i] } } for(period in 2: period2) { Pwr[period, ] <- R[period, ] / maxPwr } # Compute the dominant cycle using the Center of Gravity of the spectrum Spx <- Sp <- rep(0, length(filter)) for(period in period1: period2) { Spx <- Spx + period * Pwr[period, ] * (Pwr[period, ] >= 0.5) Sp <- Sp + Pwr[period, ] * (Pwr[period, ] >= 0.5) } dominantCycle <- Spx / Sp dominantCycle[is.nan(dominantCycle)] <- 0 dominantCycle <- xts(dominantCycle, order.by=index(x)) dominantCycle <- dominantCycle[dominantCycle > 0] return(dominantCycle) #heatmap(Pwr, Rowv = NA, Colv = NA, na.rm = TRUE, labCol = "", add.expr = lines(dominantCycle, col = 'blue')) }

One thing I do notice is that this code uses a loop that says for(i in 1:length(filt)), which is an O(data points) loop, which I view as the plague in R. While I’ve used Rcpp before, it’s been for only the most basic of loops, so this is definitely a place where the algorithm can stand to be improved with Rcpp due to R’s inherent poor looping.

Those interested in the exact logic of the algorithm will, once again, find it in John Ehlers’s Cycle Analytics For Traders book (see link earlier in the post).

Of course, the first thing to do is to test how well the algorithm does what it purports to do, which is to dictate the lookback period of an algorithm.

Let’s run it on some data.

getSymbols('SPY', from = '1990-01-01') t1 <- Sys.time() out <- autocorrPeriodogram(Ad(SPY), period1 = 120, period2 = 252, avgLength = 3) t2 <- Sys.time() print(t2-t1)

And the result:

> t1 <- Sys.time() > out <- autocorrPeriodogram(Ad(SPY), period1 = 120, period2 = 252, avgLength = 3) > t2 <- Sys.time() > print(t2-t1) Time difference of 33.25429 secs

Now, what does the algorithm-set lookback period look like?

plot(out)

Let’s zoom in on 2001 through 2003, when the markets went through some upheaval.

plot(out['2001::2003']

In this zoomed-in image, we can see that the algorithm’s estimates seem fairly jumpy.

Here’s some code to feed the algorithm’s estimates of n into an indicator to compute an indicator with a dynamic lookback period as set by Ehlers’s autocorrelation periodogram.

acpIndicator <- function(x, minPeriod, maxPeriod, indicatorFun = EMA, ...) { acpOut <- autocorrPeriodogram(x = x, period1 = minPeriod, period2 = maxPeriod) roundedAcpNs <- round(acpOut, 0) # round to the nearest integer uniqueVals <- unique(roundedAcpNs) # unique integer values out <- xts(rep(NA, length(roundedAcpNs)), order.by=index(roundedAcpNs)) for(i in 1:length(uniqueVals)) { # loop through unique values, compute indicator tmp <- indicatorFun(x, n = uniqueVals[i], ...) out[roundedAcpNs==uniqueVals[i]] <- tmp[roundedAcpNs==uniqueVals[i]] } return(out) }

And here is the function applied with an SMA, to tune between 120 and 252 days.

ehlersSMA <- acpIndicator(Ad(SPY), 120, 252, indicatorFun = SMA) plot(Ad(SPY)['2008::2010']) lines(ehlersSMA['2008::2010'], col = 'red')

And the result:

As seen, this algorithm is less consistent than I would like, at least when it comes to using a simple moving average.

For now, I’m going to leave this code here, and let people experiment with it. I hope that someone will find that this indicator is helpful to them.

Thanks for reading.

NOTES: I am always interested in networking/meet-ups in the northeast (Philadelphia/NYC). Furthermore, if you believe your firm will benefit from my skills, please do not hesitate to reach out to me. My linkedin profile can be found here.

Lastly, I am volunteering to curate the R section for books on quantocracy. If you have a book about R that can apply to finance, be sure to let me know about it, so that I can review it and possibly recommend it. Thakn you.

]]>

Before I get into the brunt of this post, I’d like to let my readers know that I formalized my nuts and bolts of quantstrat series of posts as a formal datacamp course. Datacamp is a very cheap way to learn a bunch of R, and financial applications are among those topics. My course covers the basics of quantstrat, and if those who complete the course like it, I may very well create more advanced quantstrat modules on datacamp. I’m hoping that the finance courses are well-received, since there are financial topics in R I’d like to learn myself that a 45 minute lecture doesn’t really suffice for (such as Dr. David Matteson’s change points magic, PortfolioAnalytics, and so on). In any case, here’s the link.

So, let’s start with a summary of the book:

Part 1 is several chapters that are the giant expose- of why momentum works (or at least, has worked for at least 20 years since 1993)…namely that human biases and irrational behaviors act in certain ways to make the anomaly work. Then there’s also the career risk (AKA it’s a risk factor, and so, if your benchmark is SPY and you run across a 3+ year period of underperformance, you have severe career risk), and essentially, a whole litany of why a professional asset manager would get fired but if you just stick with the anomaly over many many years and ride out multi-year stretches of relative underperformance, you’ll come out ahead in the very long run.

Generally, I feel like there’s work to be done if this is the best that can be done, but okay, I’ll accept it.

Essentially, part 1 is for the uninitiated. For those that have been around the momentum block a couple of times, they can skip right past this. Unfortunately, it’s half the book, so that leaves a little bit of a sour taste in the mouth.

Next, part two is where, in my opinion, the real meat and potatoes of the book–the “how”.

Essentially, the algorithm can be boiled down into the following:

Taking the universe of large and mid-cap stocks, do the following:

1) Sort the stocks into deciles by 2-12 momentum–that is, at the end of every month, calculate momentum by last month’s closing price minus the closing price 12 months ago. Essentially, research states that there’s a reversion effect on the 1-month momentum. However, this effect doesn’t carry over into the ETF universe in my experience.

2) Here’s the interesting part which makes the book worth picking up on its own (in my opinion): after sorting into deciles, rank the top decile by the following metric: multiply the sign of the 2-12 momentum by the following equation: (% negative returns – % positive). Essentially, the idea here is to determine smoothness of momentum. That is, in the most extreme situation, imagine a stock that did absolutely nothing for 230 days, and then had one massive day that gave it its entire price appreciation (think Google when it had a 10% jump off of better-than-expected numbers reports), and in the other extreme, a stock that simply had each and every single day be a small positive price appreciation. Obviously, you’d want the second type of stock. That’s this idea. Again, sort into deciles, and take the top decile. Therefore, taking the top decile of the top decile leaves you with 1% of the universe. Essentially, this makes the idea very difficult to replicate–since you’d need to track down a massive universe of stocks. That stated, I think the expression is actually a pretty good idea as a stand-in for volatility. That is, regardless of how volatile an asset is–whether it’s as volatile as a commodity like DBC, or as non-volatile as a fixed-income product like SHY, this expression is an interesting way of stating “this path is choppy” vs. “this path is smooth”. I might investigate this expression on my blog further in the future.

3) Lastly, if the portfolio is turning over quarterly instead of monthly, the best months to turn it over are the months preceding end-of-quarter month (that is, February, May, August, November) because a bunch of amateur asset managers like to “window dress” their portfolios. That is, they had a crummy quarter, so at the last month before they have to send out quarterly statements, they load up on some recent winners so that their clients don’t think they’re as amateur as they really let on, and there’s a bump for this. Similarly, January has some selling anomalies due to tax-loss harvesting. As far as practical implementations go, I think this is a very nice touch. Conceding the fact that turning over every month may be a bit too expensive, I like that Wes and Jack say “sure, you want to turn it over once every three months, but on *which* months?”. It’s a very good question to ask if it means you get an additional percentage point or 150 bps a year from that, as it just might cover the transaction costs and then some.

All in all, it’s a fairly simple to understand strategy. However, the part that sort of gates off the book to a perfect replication is the difficulty in obtaining the CRSP data.

However, I do commend Alpha Architect for disclosing the entire algorithm from start to finish.

Furthermore, if the basic 2-12 momentum is not enough, there’s an appendix detailing other types of momentum ideas (earnings momentum, ranking by distance to 52-week highs, absolute historical momentum, and so on). None of these strategies are really that much better than the basic price momentum strategy, so they’re there for those interested, but it seems there’s nothing really ground-breaking there. That is, if you’re trading once a month, there’s only so many ways of saying “hey, I think this thing is going up!”

I also like that Wes and Jack touched on the fact that trend-following, while it doesn’t improve overall CAGR or Sharpe, does a massive amount to improve on max drawdown. That is, if faced with the prospect of losing 70-80% of everything, and losing only 30%, that’s an easy choice to make. Trend-following is good, even a simplistic version.

All in all, I think the book accomplishes what it sets out to do, which is to present a well-researched algorithm. Ultimately, the punchline is on Alpha Architect’s site (I believe they have some sort of monthly stock filter). Furthermore, the book states that there are better risk-adjusted returns when combined with the algorithm outlined in the “quantitative value” book. In my experience, I’ve never had value algorithms impress me in the backtests I’ve done, but I can chalk that up to me being inexperienced with all the various valuation metrics.

My criticism of the book, however, is this:

The momentum algorithm in the book misses what I feel is one key component: volatility targeting control. Simply, the paper “momentum has its moments” (which I covered in my hypothesis-driven development series of posts) essentially states that the usual Fama-French momentum strategy does far better from a risk-reward strategy by deleveraging during times of excessive volatility, and avoiding momentum crashes. I’m not sure why Wes and Jack didn’t touch upon this paper, since the implementation is very simple (target/realized volatility = leverage factor). Ideally, I’d love if Wes or Jack could send me the stream of returns for this strategy (preferably daily, but monthly also works).

Essentially, I think this book is very comprehensive. However, I think it also has a somewhat “don’t try this at home” feel to it due to the data requirement to replicate it. Certainly, if your broker charges you $8 a transaction, it’s not a feasible strategy to drop several thousand bucks a year on transaction costs that’ll just give your returns to your broker. However, I do wonder if the QMOM ETF (from Alpha Architect, of course) is, in fact, a better version of this strategy, outside of the management fee.

In any case, my final opinion is this: while this book leaves a little bit of knowledge on the table, on a whole, it accomplishes what it sets out to do, is clear with its procedures, and provides several worthwhile ideas. For the price of a non-technical textbook (aka those $60+ books on amazon), this book is a steal.

4.5/5 stars.

Thanks for reading.

NOTE: While I am currently employed in a successful analytics capacity, I am interested in hearing about full-time positions more closely related to the topics on this blog. If you have a full-time position which can benefit from my current skills, please let me know. My Linkedin can be found here.

]]>

So, to start off, this post was motivated by Michael Halls-Moore, who recently posted some R code about using the depmixS4 library to use hidden markov models. Generally, I am loath to create posts on topics I don’t feel I have an absolutely front-to-back understanding of, but I’m doing this in the hope of learning from others on how to appropriately do online state-space prediction, or “regime switching” detection, as it may be called in more financial parlance.

Here’s Dr. Halls-Moore’s post.

While I’ve seen the usual theory of hidden markov models (that is, it can rain or it can be sunny, but you can only infer the weather judging by the clothes you see people wearing outside your window when you wake up), and have worked with toy examples in MOOCs (Udacity’s self-driving car course deals with them, if I recall correctly–or maybe it was the AI course), at the end of the day, theory is only as good as how well an implementation can work on real data.

For this experiment, I decided to take SPY data since inception, and do a full in-sample “backtest” on the data. That is, given that the HMM algorithm from depmix sees the whole history of returns, with this “god’s eye” view of the data, does the algorithm correctly classify the regimes, if the backtest results are any indication?

Here’s the code to do so, inspired by Dr. Halls-Moore’s.

require(depmixS4) require(quantmod) getSymbols('SPY', from = '1990-01-01', src='yahoo', adjust = TRUE) spyRets <- na.omit(Return.calculate(Ad(SPY))) set.seed(123) hmm <- depmix(SPY.Adjusted ~ 1, family = gaussian(), nstates = 3, data=spyRets) hmmfit <- fit(hmm, verbose = FALSE) post_probs <- posterior(hmmfit) post_probs <- xts(post_probs, order.by=index(spyRets)) plot(post_probs$state) summaryMat <- data.frame(summary(hmmfit)) colnames(summaryMat) <- c("Intercept", "SD") bullState <- which(summaryMat$Intercept > 0) bearState <- which(summaryMat$Intercept < 0) hmmRets <- spyRets * lag(post_probs$state == bullState) - spyRets * lag(post_probs$state == bearState) charts.PerformanceSummary(hmmRets) table.AnnualizedReturns(hmmRets)

Essentially, while I did select three states, I noted that anything with an intercept above zero is a bull state, and below zero is a bear state, so essentially, it reduces to two states.

With the result:

table.AnnualizedReturns(hmmRets) SPY.Adjusted Annualized Return 0.1355 Annualized Std Dev 0.1434 Annualized Sharpe (Rf=0%) 0.9448

So, not particularly terrible. The algorithm works, kind of, sort of, right?

Well, let’s try online prediction now.

require(DoMC) dailyHMM <- function(data, nPoints) { subRets <- data[1:nPoints,] hmm <- depmix(SPY.Adjusted ~ 1, family = gaussian(), nstates = 3, data = subRets) hmmfit <- fit(hmm, verbose = FALSE) post_probs <- posterior(hmmfit) summaryMat <- data.frame(summary(hmmfit)) colnames(summaryMat) <- c("Intercept", "SD") bullState <- which(summaryMat$Intercept > 0) bearState <- which(summaryMat$Intercept < 0) if(last(post_probs$state) %in% bullState) { state <- xts(1, order.by=last(index(subRets))) } else if (last(post_probs$state) %in% bearState) { state <- xts(-1, order.by=last(index(subRets))) } else { state <- xts(0, order.by=last(index(subRets))) } colnames(state) <- "State" return(state) } # took 3 hours in parallel t1 <- Sys.time() set.seed(123) registerDoMC((detectCores() - 1)) states <- foreach(i = 500:nrow(spyRets), .combine=rbind) %dopar% { dailyHMM(data = spyRets, nPoints = i) } t2 <- Sys.time() print(t2-t1)

So what I did here was I took an expanding window, starting from 500 days since SPY’s inception, and kept increasing it, by one day at a time. My prediction, was, trivially enough, the most recent day, using a 1 for a bull state, and a -1 for a bear state. I ran this process in parallel (on a linux cluster, because windows’s doParallel library seems to not even know that certain packages are loaded, and it’s more messy), and the first big issue is that this process took about three hours on seven cores for about 23 years of data. Not exactly encouraging, but computing time isn’t expensive these days.

So let’s see if this process actually works.

First, let’s test if the algorithm does what it’s actually supposed to do and use one day of look-ahead bias (that is, the algorithm tells us the state at the end of the day–how correct is it even for that day?).

onlineRets <- spyRets * states charts.PerformanceSummary(onlineRets) table.AnnualizedReturns(onlineRets)

With the result:

> table.AnnualizedReturns(onlineRets) SPY.Adjusted Annualized Return 0.2216 Annualized Std Dev 0.1934 Annualized Sharpe (Rf=0%) 1.1456

So, allegedly, the algorithm seems to do what it was designed to do, which is to classify a state for a given data set. Now, the most pertinent question: how well do these predictions do even one day ahead? You’d think that state space predictions would be parsimonious from day to day, given the long history, correct?

onlineRets <- spyRets * lag(states) charts.PerformanceSummary(onlineRets) table.AnnualizedReturns(onlineRets)

With the result:

> table.AnnualizedReturns(onlineRets) SPY.Adjusted Annualized Return 0.0172 Annualized Std Dev 0.1939 Annualized Sharpe (Rf=0%) 0.0888

That is, without the lookahead bias, the state space prediction algorithm is atrocious. Why is that?

Well, here’s the plot of the states:

In short, the online hmm algorithm in the depmix package seems to change its mind very easily, with obvious (negative) implications for actual trading strategies.

So, that wraps it up for this post. Essentially, the main message here is this: there’s a vast difference between loading doing descriptive analysis (AKA “where have you been, why did things happen”) vs. predictive analysis (that is, “if I correctly predict the future, I get a positive payoff”). In my opinion, while descriptive statistics have their purpose in terms of explaining why a strategy may have performed how it did, ultimately, we’re always looking for better prediction tools. In this case, depmix, at least in this “out-of-the-box” demonstration does not seem to be the tool for that.

If anyone has had success with using depmix (or other regime-switching algorithm in R) for prediction, I would love to see work that details the procedure taken, as it’s an area I’m looking to expand my toolbox into, but don’t have any particular good leads. Essentially, I’d like to think of this post as me describing my own experiences with the package.

Thanks for reading.

NOTE: On Oct. 5th, I will be in New York City. On Oct. 6th, I will be presenting at The Trading Show on the Programming Wars panel.

NOTE: My current analytics contract is up for review at the end of the year, so I am officially looking for other offers as well. If you have a full-time role which may benefit from the skills you see on my blog, please get in touch with me. My linkedin profile can be found here.

]]>