Nuts and Bolts of Quantstrat, Part I

Recently, I gave a webinar on some introductory quantstrat. Here’s the link.

So to follow up on it, I’m going to do a multi-week series of posts delving into trying to explain the details of parts of my demos, so as to be sure that everyone has a chance to learn and follow along with my methodologies, what I do, and so on. To keep things simple, I’ll be using the usual RSI 20/80 filtered on SMA 200 demo. This post will deal with the initial setup of any demo–code which will be largely similar from demo to demo.

Let’s examine this code:

require(IKTrading)
require(quantstrat)
require(PerformanceAnalytics)

initDate="1990-01-01"
from="2003-01-01"
to="2012-12-31"
options(width=70)

source("demoData.R")

The first three lines load the libraries I use in my demos. In R, libraries are loaded with a single line. However, installation procedures may vary from operating system to operating system. Windows systems are the least straightforward, while macs can use unix functionality to function in identical ways to linux machines. It’s often good practice to place functions used repeatedly into a package, which is R’s own version of encapsulation and information hiding. Packages don’t always have to be open-sourced to the internet, and in many cases, some are used just as local repositories. My IKTrading package started off as such a case; it’s simply a toolbox that contains functionality that isn’t thematically attributable in other places.

The next three lines, dealing with dates, all have separate purposes.

The initDate variable needs a date that must occur before the start of data in a backtest. If this isn’t the case, the portfolio will demonstrate a massive drawdown on the initialization date, and many of the backtest statistics will be misleading or nonsensical.

The from and to variables are endpoints on the data that the demoData.R script will use to fetch from yahoo (or elsewhere). The format is yyyy-mm-dd, which means four digit year, two digit month (E.G. January is “01”), and two digit day, in that order.

In some cases, I may write the code:

to=as.character(Sys.Date())

This just sets the current to date to the time that I run the demonstration. Although it may affect the replication of the results, thanks to some of the yearly metrics I’ve come to utilize, those wishing to see the exact day of the end of the data would be able to. However, in cases that I use data to the present, it’s often simply an exploration of the indicator as opposed to trying to construct a fully-fledged trading system.

The options(width=70) line simply controls the width of output to my R console.

The source line is a way to execute other files in the specified directory. Sourcing files works in similar ways to specifying a file path. So if, from your current directory, there’s a file you want to source called someFile, you may write a command such as source(“someFile.R”), but if said file is in a different directory, you would want to use the standard unix file navigation notation to execute it. For instance, if my directory was in “IKTrading”, rather than “IKTrading/demo”, I would write source(“demo/demoData.R”). Note that this notation is relative to my current directory. To see your current working directory, type:

getwd()

To navigate among your working directories, use the setwd command, such as:

setwd("/home/myAccount/someDirectory/NextDirectory/")

In order to obtain the data, let’s look at the demoData.R file once again.

options("getSymbols.warning4.0"=FALSE)

currency('USD')
Sys.setenv(TZ="UTC")

symbols <- c("XLB", #SPDR Materials sector
             "XLE", #SPDR Energy sector
             "XLF", #SPDR Financial sector
             "XLP", #SPDR Consumer staples sector
             "XLI", #SPDR Industrial sector
             "XLU", #SPDR Utilities sector
             "XLV", #SPDR Healthcare sector
             "XLK", #SPDR Tech sector
             "XLY", #SPDR Consumer discretionary sector
             "RWR", #SPDR Dow Jones REIT ETF
             
             "EWJ", #iShares Japan
             "EWG", #iShares Germany
             "EWU", #iShares UK
             "EWC", #iShares Canada
             "EWY", #iShares South Korea
             "EWA", #iShares Australia
             "EWH", #iShares Hong Kong
             "EWS", #iShares Singapore
             "IYZ", #iShares U.S. Telecom
             "EZU", #iShares MSCI EMU ETF
             "IYR", #iShares U.S. Real Estate
             "EWT", #iShares Taiwan
             "EWZ", #iShares Brazil
             "EFA", #iShares EAFE
             "IGE", #iShares North American Natural Resources
             "EPP", #iShares Pacific Ex Japan
             "LQD", #iShares Investment Grade Corporate Bonds
             "SHY", #iShares 1-3 year TBonds
             "IEF", #iShares 3-7 year TBonds
             "TLT" #iShares 20+ year Bonds
)

#SPDR ETFs first, iShares ETFs afterwards
if(!"XLB" %in% ls()) { 
  suppressMessages(getSymbols(symbols, from=from, to=to, src="yahoo", adjust=TRUE))  
}

stock(symbols, currency="USD", multiplier=1)

The getSymbols.warning4.0=FALSE line is simply to remove the initial warning that comes from getting symbols from yahoo. It makes no difference to how the demo runs.

The next two lines are critical.

currency('USD')
Sys.setenv(TZ="UTC")

Currency must be initialized for every demo. Thus far, I’ve yet to see it set to anything besides USD (U.S. Dollars), however, the accounting analytics back-end systems need to know what currency the prices are listed in. So the currency line cannot be skipped, or the demo will not work.

Next, the Sys.setenv(TZ=”UTC”) line is necessary because if you look at, say, the data of XLB, and look at the class of its index, here’s what you see:

> head(XLB)
           XLB.Open XLB.High  XLB.Low XLB.Close XLB.Volume XLB.Adjusted
2003-01-02 15.83335 16.09407 15.68323  16.08617  401095.00        15.58
2003-01-03 16.03877 16.05457 15.91235  15.99926   79105.20        15.50
2003-01-06 16.10988 16.41011 16.10988  16.30740  377806.43        15.80
2003-01-07 16.38641 16.38641 16.18098  16.25209  390463.27        15.75
2003-01-08 16.19679 16.19679 15.80964  15.83335  201496.76        15.34
2003-01-09 15.95186 16.12568 15.92026  16.07827   82522.54        15.58

class(index(XLB))

> class(index(XLB))
[1] "Date"

Since the index of the data is a Date type object, in order for certain orders to work, such as chain rules (which contain stop losses and take profits), the timezone has to be set as UTC, since that’s the time zone for a “Date” class object. If the demo uses the system’s default timezone instead, the timestamps will not match, and so, there will be order failures.

The symbols assignment is simply one long string vector. Here it is, once again:


symbols <- c("XLB", #SPDR Materials sector
             "XLE", #SPDR Energy sector
             "XLF", #SPDR Financial sector
             "XLP", #SPDR Consumer staples sector
             "XLI", #SPDR Industrial sector
             "XLU", #SPDR Utilities sector
             "XLV", #SPDR Healthcare sector
             "XLK", #SPDR Tech sector
             "XLY", #SPDR Consumer discretionary sector
             "RWR", #SPDR Dow Jones REIT ETF
             
             "EWJ", #iShares Japan
             "EWG", #iShares Germany
             "EWU", #iShares UK
             "EWC", #iShares Canada
             "EWY", #iShares South Korea
             "EWA", #iShares Australia
             "EWH", #iShares Hong Kong
             "EWS", #iShares Singapore
             "IYZ", #iShares U.S. Telecom
             "EZU", #iShares MSCI EMU ETF
             "IYR", #iShares U.S. Real Estate
             "EWT", #iShares Taiwan
             "EWZ", #iShares Brazil
             "EFA", #iShares EAFE
             "IGE", #iShares North American Natural Resources
             "EPP", #iShares Pacific Ex Japan
             "LQD", #iShares Investment Grade Corporate Bonds
             "SHY", #iShares 1-3 year TBonds
             "IEF", #iShares 3-7 year TBonds
             "TLT" #iShares 20+ year Bonds
)

There is nothing particularly unique about it. However, I structured the vector so as to be able to comment with the description of each ETF next to its ticker string for the purposes of clarity.

From there, the file gets the symbols from yahoo. The extra verbosity around the command is simply to suppress any output to the screen. Here’s the line of code that does this:

#SPDR ETFs first, iShares ETFs afterwards
if(!"XLB" %in% ls()) { 
  suppressMessages(getSymbols(symbols, from=from, to=to, src="yahoo", adjust=TRUE))  
}

I can control whether or not to rerun the data-gathering process by removing XLB from my current working environment. This isn’t the most general way of controlling the data cache (a more general boolean would be better general style), but it works for the examples I use. If I keep XLB in my working environment, then this line is skipped altogether, to speed up the backtest.

Lastly, the backtest needs the instrument specifications. This is the line of code to do so:

stock(symbols, currency="USD", multiplier=1)

Although it looks fairly trivial at the moment, once a backtest would start dealing with futures, contract multiplier specifications, and other instrument-specific properties, this line becomes far less trivial than it looks.

Moving back to the main scripts, here is the rest of the initialization boilerplate:

#trade sizing and initial equity settings
tradeSize <- 100000
initEq <- tradeSize*length(symbols)

strategy.st <- portfolio.st <- account.st <- "DollarVsATRos"
rm.strat(strategy.st)
initPortf(portfolio.st, symbols=symbols, initDate=initDate, currency='USD')
initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD',initEq=initEq)
initOrders(portfolio.st, initDate=initDate)
strategy(strategy.st, store=TRUE)

The tradeSize and initEq variables are necessary in order to compute returns at the end of a backtest. Furthermore, tradeSize is necessary for the osDollarATR order-sizing function.

Next, I name the strategy, portfolio, and account–all with the same name. The x <- y <- z <- "xyz" format is a multi-assignment syntax that should be used only for assigning several objects that are all initially (or permanently) identical to one another, such as initializing multiple xts objects of the same length.

Next, the removal of the strategy is necessary for rerunning the strategy. If the strategy object exists, and a user attempts to rerun the demo, the demo will crash. Always make sure to remove the strategy.

Next, we have the three initialization steps. Due to the dependencies between the portfolio, account, and orders, the portfolio must be initialized before the account, and it also must be initialized before the orders.

To initialize the portfolio, one needs to name the portfolio something, have a vector of character strings that represent the symbols passed in as the symbols argument, an initial date (initDate), and a currency. This currency was defined earlier in the demoData.R file.

The account initialization replaces the symbols argument with a portfolios argument, and adds an initEq argument, from which to compute returns.

Lastly, the orders initialization needs only a portfolio to reference, and a date from which to begin transactions (initDate, which is earlier than the beginning of the data).

Lastly, we initialize the strategy in the form of the line:

strategy(strategy.st, store=TRUE)

This is where we put all our indicators, signals, and rules. Without supplying this line to the demo, none of the indicators, signals, and rules will know what strategy object to look for, and the demo will crash.

This concludes the initial boilerplate walkthrough. Next: parameters and indicators.

Thanks for reading.

56 thoughts on “Nuts and Bolts of Quantstrat, Part I

  1. Pingback: The Whole Street’s Daily Wrap for 9/9/2014 | The Whole Street

  2. Pingback: Nuts and Bolts of Quantstrat, Part I | QuantStrat TradeR | DRBTK

  3. Like, how do I set up an limit buy order only if there has been N lower lows and if today’s price falls another M times the P-day ATR ?

    • If you mean custom indicators and signals by R functions that originated with me, such as the Ehlers functions, or sigAND, then I’ve already done that. Everything in R, however, is a function, so I suppose the only difference in “custom” vs. “not custom” is if it’s a function not written by the original package authors.

  4. Pingback: Nuts and Bolts of Quantstrat, Part I | mintegration blog

  5. Dear Ilya,
    Thanks a lot for your sharing of how to use quantstrat on BigMike forum and this great blog. I found your video and the four articles very helpful. but I am new to programming and R, and easy to get stuck by small problems. I was following your video on installing different packages. I installed R 3.0.3, Rstudio and Rtools, quantstrat, zoo, and TTR, but stuck at Defaults package.
    after I ran > install.packages(“Defaults”), I got the following message:
    Warning in install.packages :
    package ‘Defaults’ is not available (for R version 3.0.3)

    I have searched on google to find solutions, didn’t find very useful answers though I did get the file “Defaults_1.1-1.tar.gz” updated on 26-Aug-2007 15:20 112K.

    Could you give me some tips on installing this package?

    also, I have installed Rtools but how do I know Rtools is at work?

    Thank you very much for your time.

    Kenny

      • Dear Ilya,
        Thanks for your quick reply and help.
        I replaced R 3.0.3 with R 3.1.2 and it solved the problem with Defaults package.

        however, there comes a new problem.
        when I ran require(quantstrat), in the console it returned the following error:
        failed with error: “did not find …. ‘foreach’ package”
        I am again not able to find any help about this on google.
        do you know what is wrong here?

        Thanks a lot.

        Kenny

      • I’m getting this message for myself after trying the other options

        when i try and install the defaults…

        Warning in install.packages :
        package ‘Defaults’ is not available (for R version 3.1.2)

        I dunno what to do

      • I was able to install the defaults package by doing this
        install.packages(“http://cran.r-project.org/src/contrib/Archive/Defaults/Defaults_1.1-1.tar.gz”, repo=NULL, type=”source”)

        I then installed foreach by doing this
        install.packages(“http://cran.r-project.org/bin/windows/contrib/3.1/foreach_1.4.2.zip”, repo=NULL, type=”source”)

        and I get this error
        > require(quantstrat)
        Loading required package: quantstrat
        Loading required package: foreach
        Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) :
        there is no package called ‘iterators’
        Failed with error: ‘package ‘foreach’ could not be loaded’

        I’m starting R with this video and I appreciate your quick responses
        how should I proceed?

      • Try to find the iterators package, of course. Email the R-SIG-Finance mailing list if you can’t find it. I believe there’s a foreach package on CRAN, but don’t take my word for it.

      • this is what fixed the require(quantstrat)
        install.packages(“http://cran.r-project.org/bin/windows/contrib/3.1/iterators_1.0.7.zip”, repo=NULL, type=”source”)

        I think is is fine but I thought I would share this part with you

        next, I tried
        install_github(repo=”IKTrading”, username=”Ilyakipnis”)
        I got
        Warning message:
        Username parameter is deprecated. Please use Ilyakipnis/IKTrading

        but then i typed require(IKTrading)
        and I got

        Loading required package: IKTrading
        Loading required package: Rcpp
        Loading required package: digest
        Loading required package: roxygen2

        then i typed
        install_github(repo=”DSTrading”, username=”Ilyakipnis”)

        I got
        Warning message:
        Username parameter is deprecated. Please use Ilyakipnis/DSTrading

        then i typed require(DSTrading)
        I got
        Loading required package: DSTrading
        Loading required package: inline

        Attaching package: ‘inline’

        The following object is masked from ‘package:Rcpp’:

        registerPlugin

        I hope this helps you and others thank you for the video , hopefully I can continue now

        I have R 3.1.2, RTools 31.exe and RStudio Version 0.98.1102

      • I then typed sessionInfo() capital I in info

        > sessionInfo()
        R version 3.1.2 (2014-10-31)
        Platform: x86_64-w64-mingw32/x64 (64-bit)

        locale:
        [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
        [4] LC_NUMERIC=C LC_TIME=English_United States.1252

        attached base packages:
        [1] stats graphics grDevices utils datasets methods base

        other attached packages:
        [1] DSTrading_1.0 inline_0.3.13 IKTrading_1.0 roxygen2_4.1.0
        [5] digest_0.6.8 Rcpp_0.11.4 quantstrat_0.9.1669 foreach_1.4.2
        [9] devtools_1.7.0 blotter_0.9.1666 PerformanceAnalytics_1.4.3579 FinancialInstrument_1.2.0
        [13] quantmod_0.4-3 TTR_0.22-0.1 xts_0.9-7 zoo_1.7-11

        loaded via a namespace (and not attached):
        [1] bitops_1.0-6 codetools_0.2-9 grid_3.1.2 httr_0.6.1 iterators_1.0.7 lattice_0.20-29 RCurl_1.95-4.5
        [8] stringr_0.6.2 tools_3.1.2

  6. Thank you Ilya, I eventually installed foreach package directly using Rstudio in its packages window – install – search and install foreach.

    Thanks again for your quick reply and help

    Best,

    Kenny

  7. Dear Ilya,
    one more question:
    In your video you mentioned that after getting all packages installed, we need to make sure to have the version 0.8.19 of Blotter for windows. however, I got the version blotter 0.9.1637. I did check the archive for blotter, but I cannot tell whether this newer version of blotter is somehow less capable than the 0.8.19 version.
    so, do I have to replace it with the blotter 0.8.19?
    or the 0.9.1637 version is fine?

    Thanks a lot for your patience.

    Kenny

  8. I have a quastion about your webinar. When I try to install IKTrading or DSTrading packages i get error message saying:
    “Error: Command failed (1)
    In addition: Warning message:
    Username parameter is deprecated. Please use IlyaKipnis/IKTrading”

    Do you have any suggestions on how to solve this problem?
    I have devtools and other required packages installed

  9. hi Ilya, very useful blog, thanks for putting it together. Any chance you’d be able to share the code for your “intro to quantstrat” webinar. Getting the below error when i try to apply the strategy and not sure what’s causing it. Thanks again.

    “Error in inherits(x, “xts”) : argument “x” is missing, with no default”

  10. It would be very helpful if you can provide step by step instructions to obtain package IKTrading (for first line of code “require(IKTrading)”,

  11. Hi Ilya. Which versions are you running today? (R, R-Studio, + all installed packages, from CRAN and Github). Just thought that the recording is from year 2014 and maybe you are running newer versions. I am working with downgrading the versions using R in VMware to secure getting complete backtest strategy running with the solution presented in your video. Currently I am running windows and MAC, but will probably phase both OS out for benefit of Linux.

  12. Hi Ilya. I have the environment setup and the configuration file in almost working shape. I currently have problems with the package “Defaults”. When running install.packages(“Defaults”) it gives me -“package ‘Defaults’ is not available (for R version 3.3.1). I have backed all the way to R version 3.0.1, and tested also R-versions (3.0.2), (3.0.3), (3.1.2). All R-versions give me the same error message.

    Further on, when running the “apply.strategy” without no previous errors in the R-script (except the “Defaults” problem, I get the following error:
    Error in if (length(j) == 0 || (length(j) == 1 && j == 0)) { :
    missing value where TRUE/FALSE needed

    Your advice would be highly appreciated, since I am stuck for the moment and it would be great to start testing and experiencing the quanstrat setup you provided on your youtube session.

  13. Ok. I will check the braverock github.

    Passing the “Defaults” error, i am now getting following error when running “apply.strategy”:
    Error in if (length(j) == 0 || (length(j) == 1 && j == 0)) { :
    missing value where TRUE/FALSE needed
    In addition: Warning messages:
    1: In match.names(columns, colnames(data)) :
    all columns not located in Close filterMA for SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted atr.atrX EMA.rsi SMA.quickSMA
    2: In min(j, na.rm = TRUE) :
    no non-missing arguments to min; returning Inf
    3: In max(j, na.rm = TRUE) :
    no non-missing arguments to max; returning -Inf

  14. Can you please clarify if you mean to use this line in R script for download latest package from braverock github: install.packages(“TTR”, repos=”http://R-Forge.R-project.org”).

    …or do I need to got to https://github.com/braverock, to fetch the packages, for manual download and installation. If so, do I download the complete masterbranch?.

    Thanks.

  15. Ok, so I took the packages from github now. I managed to get running the quantstrat demo strategy “luxor.strategy.basic.R”.

    I guess there is some faults in my script for the script I built up with reference to your Youtube clip.
    The only line in the script that shows error is:
    out <- applyStrategy(strategy=strategy.st,portfolios=portfolio.st)

    And the error is:
    Error in match.names(column, colnames(data)) :
    argument "column" is missing, with no default
    In addition: Warning messages:
    1: In match.names(columns, colnames(data)) :
    all columns not located in Close filterMA for SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted atr.atrX EMA.rsi SMA.quickSMA
    2: In min(j, na.rm = TRUE) :
    no non-missing arguments to min; returning Inf
    3: In max(j, na.rm = TRUE) :
    no non-missing arguments to max; returning -Inf

    • Hi Ilya. Thanks for the suggestions.
      I am up and running now with RSI_10_06. Found 2 errors.

      1) the name column in the script is both used as “column” and “columns”.
      During duplications of the “add.signal” I ended up with “columns” where
      there was only one column value.

      2) mismatch between “label” & “sigcol”.
      I was using “longEntry1” in one place, and “LongEntry1” in the other.
      The “L” was not matching “l”.

  16. Hello, when I am appying this part: “stock(symbols, currency=”USD”, multiplier=1)”
    I receive the following Error: C stack usage 19924192 is too close to the limit.
    Can anyone please help me to solve it?

  17. Pingback: Nuts and Bolts of Quantstrat, Part V | QuantStrat TradeR

  18. Pingback: Nuts and Bolts of Quantstrat, Part V – 极智投研

  19. Hi, could you kindly explain the meaning of the code :
    #SPDR ETFs first, iShares ETFs afterwards
    if(!”XLB” %in% ls()) {
    suppressMessages(getSymbols(symbols, from=from, to=to, src=”yahoo”, adjust=TRUE))
    }

    While I do understand the purpose of suppressMessages, I am not sure of the purpose of the if function. Thank you!

  20. Hi Ilya, Thank you for these tutorials. May I use my own custom data.frame containing timeseries data to go on with the rest of backtesting functions? Can you give an example of it?

  21. Thanks for the great posts.
    It looks like packages ‘IKTrading’,’quantstrat’ both do not install on R version 3.5 and above. Is there an alternative package or other ways to get the same functions as these packages?

Leave a comment