Recently, I gave a webinar on some introductory quantstrat. Here’s the link.
So to follow up on it, I’m going to do a multi-week series of posts delving into trying to explain the details of parts of my demos, so as to be sure that everyone has a chance to learn and follow along with my methodologies, what I do, and so on. To keep things simple, I’ll be using the usual RSI 20/80 filtered on SMA 200 demo. This post will deal with the initial setup of any demo–code which will be largely similar from demo to demo.
Let’s examine this code:
require(IKTrading) require(quantstrat) require(PerformanceAnalytics) initDate="1990-01-01" from="2003-01-01" to="2012-12-31" options(width=70) source("demoData.R")
The first three lines load the libraries I use in my demos. In R, libraries are loaded with a single line. However, installation procedures may vary from operating system to operating system. Windows systems are the least straightforward, while macs can use unix functionality to function in identical ways to linux machines. It’s often good practice to place functions used repeatedly into a package, which is R’s own version of encapsulation and information hiding. Packages don’t always have to be open-sourced to the internet, and in many cases, some are used just as local repositories. My IKTrading package started off as such a case; it’s simply a toolbox that contains functionality that isn’t thematically attributable in other places.
The next three lines, dealing with dates, all have separate purposes.
The initDate variable needs a date that must occur before the start of data in a backtest. If this isn’t the case, the portfolio will demonstrate a massive drawdown on the initialization date, and many of the backtest statistics will be misleading or nonsensical.
The from and to variables are endpoints on the data that the demoData.R script will use to fetch from yahoo (or elsewhere). The format is yyyy-mm-dd, which means four digit year, two digit month (E.G. January is “01”), and two digit day, in that order.
In some cases, I may write the code:
to=as.character(Sys.Date())
This just sets the current to date to the time that I run the demonstration. Although it may affect the replication of the results, thanks to some of the yearly metrics I’ve come to utilize, those wishing to see the exact day of the end of the data would be able to. However, in cases that I use data to the present, it’s often simply an exploration of the indicator as opposed to trying to construct a fully-fledged trading system.
The options(width=70) line simply controls the width of output to my R console.
The source line is a way to execute other files in the specified directory. Sourcing files works in similar ways to specifying a file path. So if, from your current directory, there’s a file you want to source called someFile, you may write a command such as source(“someFile.R”), but if said file is in a different directory, you would want to use the standard unix file navigation notation to execute it. For instance, if my directory was in “IKTrading”, rather than “IKTrading/demo”, I would write source(“demo/demoData.R”). Note that this notation is relative to my current directory. To see your current working directory, type:
getwd()
To navigate among your working directories, use the setwd command, such as:
setwd("/home/myAccount/someDirectory/NextDirectory/")
In order to obtain the data, let’s look at the demoData.R file once again.
options("getSymbols.warning4.0"=FALSE) currency('USD') Sys.setenv(TZ="UTC") symbols <- c("XLB", #SPDR Materials sector "XLE", #SPDR Energy sector "XLF", #SPDR Financial sector "XLP", #SPDR Consumer staples sector "XLI", #SPDR Industrial sector "XLU", #SPDR Utilities sector "XLV", #SPDR Healthcare sector "XLK", #SPDR Tech sector "XLY", #SPDR Consumer discretionary sector "RWR", #SPDR Dow Jones REIT ETF "EWJ", #iShares Japan "EWG", #iShares Germany "EWU", #iShares UK "EWC", #iShares Canada "EWY", #iShares South Korea "EWA", #iShares Australia "EWH", #iShares Hong Kong "EWS", #iShares Singapore "IYZ", #iShares U.S. Telecom "EZU", #iShares MSCI EMU ETF "IYR", #iShares U.S. Real Estate "EWT", #iShares Taiwan "EWZ", #iShares Brazil "EFA", #iShares EAFE "IGE", #iShares North American Natural Resources "EPP", #iShares Pacific Ex Japan "LQD", #iShares Investment Grade Corporate Bonds "SHY", #iShares 1-3 year TBonds "IEF", #iShares 3-7 year TBonds "TLT" #iShares 20+ year Bonds ) #SPDR ETFs first, iShares ETFs afterwards if(!"XLB" %in% ls()) { suppressMessages(getSymbols(symbols, from=from, to=to, src="yahoo", adjust=TRUE)) } stock(symbols, currency="USD", multiplier=1)
The getSymbols.warning4.0=FALSE line is simply to remove the initial warning that comes from getting symbols from yahoo. It makes no difference to how the demo runs.
The next two lines are critical.
currency('USD') Sys.setenv(TZ="UTC")
Currency must be initialized for every demo. Thus far, I’ve yet to see it set to anything besides USD (U.S. Dollars), however, the accounting analytics back-end systems need to know what currency the prices are listed in. So the currency line cannot be skipped, or the demo will not work.
Next, the Sys.setenv(TZ=”UTC”) line is necessary because if you look at, say, the data of XLB, and look at the class of its index, here’s what you see:
> head(XLB) XLB.Open XLB.High XLB.Low XLB.Close XLB.Volume XLB.Adjusted 2003-01-02 15.83335 16.09407 15.68323 16.08617 401095.00 15.58 2003-01-03 16.03877 16.05457 15.91235 15.99926 79105.20 15.50 2003-01-06 16.10988 16.41011 16.10988 16.30740 377806.43 15.80 2003-01-07 16.38641 16.38641 16.18098 16.25209 390463.27 15.75 2003-01-08 16.19679 16.19679 15.80964 15.83335 201496.76 15.34 2003-01-09 15.95186 16.12568 15.92026 16.07827 82522.54 15.58 class(index(XLB)) > class(index(XLB)) [1] "Date"
Since the index of the data is a Date type object, in order for certain orders to work, such as chain rules (which contain stop losses and take profits), the timezone has to be set as UTC, since that’s the time zone for a “Date” class object. If the demo uses the system’s default timezone instead, the timestamps will not match, and so, there will be order failures.
The symbols assignment is simply one long string vector. Here it is, once again:
symbols <- c("XLB", #SPDR Materials sector "XLE", #SPDR Energy sector "XLF", #SPDR Financial sector "XLP", #SPDR Consumer staples sector "XLI", #SPDR Industrial sector "XLU", #SPDR Utilities sector "XLV", #SPDR Healthcare sector "XLK", #SPDR Tech sector "XLY", #SPDR Consumer discretionary sector "RWR", #SPDR Dow Jones REIT ETF "EWJ", #iShares Japan "EWG", #iShares Germany "EWU", #iShares UK "EWC", #iShares Canada "EWY", #iShares South Korea "EWA", #iShares Australia "EWH", #iShares Hong Kong "EWS", #iShares Singapore "IYZ", #iShares U.S. Telecom "EZU", #iShares MSCI EMU ETF "IYR", #iShares U.S. Real Estate "EWT", #iShares Taiwan "EWZ", #iShares Brazil "EFA", #iShares EAFE "IGE", #iShares North American Natural Resources "EPP", #iShares Pacific Ex Japan "LQD", #iShares Investment Grade Corporate Bonds "SHY", #iShares 1-3 year TBonds "IEF", #iShares 3-7 year TBonds "TLT" #iShares 20+ year Bonds )
There is nothing particularly unique about it. However, I structured the vector so as to be able to comment with the description of each ETF next to its ticker string for the purposes of clarity.
From there, the file gets the symbols from yahoo. The extra verbosity around the command is simply to suppress any output to the screen. Here’s the line of code that does this:
#SPDR ETFs first, iShares ETFs afterwards if(!"XLB" %in% ls()) { suppressMessages(getSymbols(symbols, from=from, to=to, src="yahoo", adjust=TRUE)) }
I can control whether or not to rerun the data-gathering process by removing XLB from my current working environment. This isn’t the most general way of controlling the data cache (a more general boolean would be better general style), but it works for the examples I use. If I keep XLB in my working environment, then this line is skipped altogether, to speed up the backtest.
Lastly, the backtest needs the instrument specifications. This is the line of code to do so:
stock(symbols, currency="USD", multiplier=1)
Although it looks fairly trivial at the moment, once a backtest would start dealing with futures, contract multiplier specifications, and other instrument-specific properties, this line becomes far less trivial than it looks.
Moving back to the main scripts, here is the rest of the initialization boilerplate:
#trade sizing and initial equity settings tradeSize <- 100000 initEq <- tradeSize*length(symbols) strategy.st <- portfolio.st <- account.st <- "DollarVsATRos" rm.strat(strategy.st) initPortf(portfolio.st, symbols=symbols, initDate=initDate, currency='USD') initAcct(account.st, portfolios=portfolio.st, initDate=initDate, currency='USD',initEq=initEq) initOrders(portfolio.st, initDate=initDate) strategy(strategy.st, store=TRUE)
The tradeSize and initEq variables are necessary in order to compute returns at the end of a backtest. Furthermore, tradeSize is necessary for the osDollarATR order-sizing function.
Next, I name the strategy, portfolio, and account–all with the same name. The x <- y <- z <- "xyz" format is a multi-assignment syntax that should be used only for assigning several objects that are all initially (or permanently) identical to one another, such as initializing multiple xts objects of the same length.
Next, the removal of the strategy is necessary for rerunning the strategy. If the strategy object exists, and a user attempts to rerun the demo, the demo will crash. Always make sure to remove the strategy.
Next, we have the three initialization steps. Due to the dependencies between the portfolio, account, and orders, the portfolio must be initialized before the account, and it also must be initialized before the orders.
To initialize the portfolio, one needs to name the portfolio something, have a vector of character strings that represent the symbols passed in as the symbols argument, an initial date (initDate), and a currency. This currency was defined earlier in the demoData.R file.
The account initialization replaces the symbols argument with a portfolios argument, and adds an initEq argument, from which to compute returns.
Lastly, the orders initialization needs only a portfolio to reference, and a date from which to begin transactions (initDate, which is earlier than the beginning of the data).
Lastly, we initialize the strategy in the form of the line:
strategy(strategy.st, store=TRUE)
This is where we put all our indicators, signals, and rules. Without supplying this line to the demo, none of the indicators, signals, and rules will know what strategy object to look for, and the demo will crash.
This concludes the initial boilerplate walkthrough. Next: parameters and indicators.
Thanks for reading.
Pingback: The Whole Street’s Daily Wrap for 9/9/2014 | The Whole Street
Pingback: Nuts and Bolts of Quantstrat, Part I | QuantStrat TradeR | DRBTK
What could really be helpful would be small examples of basic strategies.
How basic do you mean? I feel my RSI 20/80 filtered on SMA200 is about as basic as things get.
Like, how do I set up an limit buy order only if there has been N lower lows and if today’s price falls another M times the P-day ATR ?
You’d need some way of computing the streak of what qualifies. If you’re thinking about Cesar Alvarez’s trading strategy, I may do that in the near future.
Yes I was thinking about this strategy. I’m gonna try to code it on my end.
Great post Ilya !
In your subsequent posts within this series, are you going to be creating custom signals and indicators?
(I’ve yet to see your webcast).
If you mean custom indicators and signals by R functions that originated with me, such as the Ehlers functions, or sigAND, then I’ve already done that. Everything in R, however, is a function, so I suppose the only difference in “custom” vs. “not custom” is if it’s a function not written by the original package authors.
I’ve used the indicators from TTR and I’m now looking to create some of my own ideas too.
Thanks Ilya.
Well, at the end of the day, they’re all just functions. TTR’s just happened to be written by a certain person, that’s all.
Oh, and if you’d be so kind, I’d be appreciative of you sharing them.
Pingback: Nuts and Bolts of Quantstrat, Part I | mintegration blog
Dear Ilya,
Thanks a lot for your sharing of how to use quantstrat on BigMike forum and this great blog. I found your video and the four articles very helpful. but I am new to programming and R, and easy to get stuck by small problems. I was following your video on installing different packages. I installed R 3.0.3, Rstudio and Rtools, quantstrat, zoo, and TTR, but stuck at Defaults package.
after I ran > install.packages(“Defaults”), I got the following message:
Warning in install.packages :
package ‘Defaults’ is not available (for R version 3.0.3)
I have searched on google to find solutions, didn’t find very useful answers though I did get the file “Defaults_1.1-1.tar.gz” updated on 26-Aug-2007 15:20 112K.
Could you give me some tips on installing this package?
also, I have installed Rtools but how do I know Rtools is at work?
Thank you very much for your time.
Kenny
Kenny,
Upgrade to the latest version of R. The R-forge version of TTR is up to date for Windows now.
-Ilya
Dear Ilya,
Thanks for your quick reply and help.
I replaced R 3.0.3 with R 3.1.2 and it solved the problem with Defaults package.
however, there comes a new problem.
when I ran require(quantstrat), in the console it returned the following error:
failed with error: “did not find …. ‘foreach’ package”
I am again not able to find any help about this on google.
do you know what is wrong here?
Thanks a lot.
Kenny
http://cran.r-project.org/web/packages/foreach/index.html
I’m getting this message for myself after trying the other options
when i try and install the defaults…
Warning in install.packages :
package ‘Defaults’ is not available (for R version 3.1.2)
I dunno what to do
I believe the latest iterations of things like quantstrat and TTR don’t have this dependency anymore.
I was able to install the defaults package by doing this
install.packages(“http://cran.r-project.org/src/contrib/Archive/Defaults/Defaults_1.1-1.tar.gz”, repo=NULL, type=”source”)
I then installed foreach by doing this
install.packages(“http://cran.r-project.org/bin/windows/contrib/3.1/foreach_1.4.2.zip”, repo=NULL, type=”source”)
and I get this error
> require(quantstrat)
Loading required package: quantstrat
Loading required package: foreach
Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) :
there is no package called ‘iterators’
Failed with error: ‘package ‘foreach’ could not be loaded’
I’m starting R with this video and I appreciate your quick responses
how should I proceed?
Try to find the iterators package, of course. Email the R-SIG-Finance mailing list if you can’t find it. I believe there’s a foreach package on CRAN, but don’t take my word for it.
this is what fixed the require(quantstrat)
install.packages(“http://cran.r-project.org/bin/windows/contrib/3.1/iterators_1.0.7.zip”, repo=NULL, type=”source”)
I think is is fine but I thought I would share this part with you
next, I tried
install_github(repo=”IKTrading”, username=”Ilyakipnis”)
I got
Warning message:
Username parameter is deprecated. Please use Ilyakipnis/IKTrading
but then i typed require(IKTrading)
and I got
Loading required package: IKTrading
Loading required package: Rcpp
Loading required package: digest
Loading required package: roxygen2
then i typed
install_github(repo=”DSTrading”, username=”Ilyakipnis”)
I got
Warning message:
Username parameter is deprecated. Please use Ilyakipnis/DSTrading
then i typed require(DSTrading)
I got
Loading required package: DSTrading
Loading required package: inline
Attaching package: ‘inline’
The following object is masked from ‘package:Rcpp’:
registerPlugin
I hope this helps you and others thank you for the video , hopefully I can continue now
I have R 3.1.2, RTools 31.exe and RStudio Version 0.98.1102
I then typed sessionInfo() capital I in info
> sessionInfo()
R version 3.1.2 (2014-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] DSTrading_1.0 inline_0.3.13 IKTrading_1.0 roxygen2_4.1.0
[5] digest_0.6.8 Rcpp_0.11.4 quantstrat_0.9.1669 foreach_1.4.2
[9] devtools_1.7.0 blotter_0.9.1666 PerformanceAnalytics_1.4.3579 FinancialInstrument_1.2.0
[13] quantmod_0.4-3 TTR_0.22-0.1 xts_0.9-7 zoo_1.7-11
loaded via a namespace (and not attached):
[1] bitops_1.0-6 codetools_0.2-9 grid_3.1.2 httr_0.6.1 iterators_1.0.7 lattice_0.20-29 RCurl_1.95-4.5
[8] stringr_0.6.2 tools_3.1.2
Thank you Ilya, I eventually installed foreach package directly using Rstudio in its packages window – install – search and install foreach.
Thanks again for your quick reply and help
Best,
Kenny
Very welcome.
Big thank you for asking this question and for Ilya’s response. Sorted out the same problem I had and have spent literally hours searching for a solution until I came across your post. Now looking forward to trying some Ilya’s work.
best
Philip
Dear Ilya,
one more question:
In your video you mentioned that after getting all packages installed, we need to make sure to have the version 0.8.19 of Blotter for windows. however, I got the version blotter 0.9.1637. I did check the archive for blotter, but I cannot tell whether this newer version of blotter is somehow less capable than the 0.8.19 version.
so, do I have to replace it with the blotter 0.8.19?
or the 0.9.1637 version is fine?
Thanks a lot for your patience.
Kenny
I followed your suggestion go to https://r-forge.r-project.org/R/?group_id, and found that the blotter for downloading now is version 0.9.1644 and replaced 0.9.1637. So, now I should be ok with all the packages version, I assume.
Thanks
I have a quastion about your webinar. When I try to install IKTrading or DSTrading packages i get error message saying:
“Error: Command failed (1)
In addition: Warning message:
Username parameter is deprecated. Please use IlyaKipnis/IKTrading”
Do you have any suggestions on how to solve this problem?
I have devtools and other required packages installed
Try repo = “IKTrading”, username=”IlyaKipnis” and the same for DSTrading
Thanks, the problem may have been that I typed it somehow wrong or tried to install it multiple times.
hi Ilya, very useful blog, thanks for putting it together. Any chance you’d be able to share the code for your “intro to quantstrat” webinar. Getting the below error when i try to apply the strategy and not sure what’s causing it. Thanks again.
“Error in inherits(x, “xts”) : argument “x” is missing, with no default”
Haven’t seen that one before.
It would be very helpful if you can provide step by step instructions to obtain package IKTrading (for first line of code “require(IKTrading)”,
You need devtools and to install it from github.
Hi Ilya. Which versions are you running today? (R, R-Studio, + all installed packages, from CRAN and Github). Just thought that the recording is from year 2014 and maybe you are running newer versions. I am working with downgrading the versions using R in VMware to secure getting complete backtest strategy running with the solution presented in your video. Currently I am running windows and MAC, but will probably phase both OS out for benefit of Linux.
I am running the latest from braverock’s github.
Hi Ilya. I have the environment setup and the configuration file in almost working shape. I currently have problems with the package “Defaults”. When running install.packages(“Defaults”) it gives me -“package ‘Defaults’ is not available (for R version 3.3.1). I have backed all the way to R version 3.0.1, and tested also R-versions (3.0.2), (3.0.3), (3.1.2). All R-versions give me the same error message.
Further on, when running the “apply.strategy” without no previous errors in the R-script (except the “Defaults” problem, I get the following error:
Error in if (length(j) == 0 || (length(j) == 1 && j == 0)) { :
missing value where TRUE/FALSE needed
Your advice would be highly appreciated, since I am stuck for the moment and it would be great to start testing and experiencing the quanstrat setup you provided on your youtube session.
Download the latest packages from the braverock github.
I forgot to say I am running on Windows. I now manage to install the “Default” package manually downloading from ” https://cran.r-project.org/src/contrib/Archive/Defaults/“.
Ok. I will check the braverock github.
Passing the “Defaults” error, i am now getting following error when running “apply.strategy”:
Error in if (length(j) == 0 || (length(j) == 1 && j == 0)) { :
missing value where TRUE/FALSE needed
In addition: Warning messages:
1: In match.names(columns, colnames(data)) :
all columns not located in Close filterMA for SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted atr.atrX EMA.rsi SMA.quickSMA
2: In min(j, na.rm = TRUE) :
no non-missing arguments to min; returning Inf
3: In max(j, na.rm = TRUE) :
no non-missing arguments to max; returning -Inf
Can you please clarify if you mean to use this line in R script for download latest package from braverock github: install.packages(“TTR”, repos=”http://R-Forge.R-project.org”).
…or do I need to got to https://github.com/braverock, to fetch the packages, for manual download and installation. If so, do I download the complete masterbranch?.
Thanks.
Download the latest TTR from joshuaulrich ‘s github as well.
Sounds like a naming error on your end.
Ok, so I took the packages from github now. I managed to get running the quantstrat demo strategy “luxor.strategy.basic.R”.
I guess there is some faults in my script for the script I built up with reference to your Youtube clip.
The only line in the script that shows error is:
out <- applyStrategy(strategy=strategy.st,portfolios=portfolio.st)
And the error is:
Error in match.names(column, colnames(data)) :
argument "column" is missing, with no default
In addition: Warning messages:
1: In match.names(columns, colnames(data)) :
all columns not located in Close filterMA for SPY.Open SPY.High SPY.Low SPY.Close SPY.Volume SPY.Adjusted atr.atrX EMA.rsi SMA.quickSMA
2: In min(j, na.rm = TRUE) :
no non-missing arguments to min; returning Inf
3: In max(j, na.rm = TRUE) :
no non-missing arguments to max; returning -Inf
Hi Ilya. Thanks for the suggestions.
I am up and running now with RSI_10_06. Found 2 errors.
1) the name column in the script is both used as “column” and “columns”.
During duplications of the “add.signal” I ended up with “columns” where
there was only one column value.
2) mismatch between “label” & “sigcol”.
I was using “longEntry1” in one place, and “LongEntry1” in the other.
The “L” was not matching “l”.
Hello, when I am appying this part: “stock(symbols, currency=”USD”, multiplier=1)”
I receive the following Error: C stack usage 19924192 is too close to the limit.
Can anyone please help me to solve it?
Pingback: Nuts and Bolts of Quantstrat, Part V | QuantStrat TradeR
Pingback: Nuts and Bolts of Quantstrat, Part V – 极智投研
Hi, could you kindly explain the meaning of the code :
#SPDR ETFs first, iShares ETFs afterwards
if(!”XLB” %in% ls()) {
suppressMessages(getSymbols(symbols, from=from, to=to, src=”yahoo”, adjust=TRUE))
}
While I do understand the purpose of suppressMessages, I am not sure of the purpose of the if function. Thank you!
It’s to make sure I don’t re-download the data.
Hi Ilya, Thank you for these tutorials. May I use my own custom data.frame containing timeseries data to go on with the rest of backtesting functions? Can you give an example of it?
I believe I cover that in Part V.
Thanks for the great posts.
It looks like packages ‘IKTrading’,’quantstrat’ both do not install on R version 3.5 and above. Is there an alternative package or other ways to get the same functions as these packages?
Strange. What’s the error message? Are you trying to install them from CRAN? You need to install them from my github.
Okay that helps. It looks like I had to first reinstall the right version of Rtools for 3.5, too.