lazytrade v0.4.0

0

Monthly downloads

0th

Percentile

Learn Computer and Data Science using Algorithmic Trading

Provide sets of functions and methods to learn and practice data science using idea of algorithmic trading. Main goal is to process information within "Decision Support System" to come up with analysis or predictions. There are several utilities such as dynamic and adaptive risk management using reinforcement learning and even functions to generate predictions of price changes using pattern recognition deep regression learning. Summary of Methods used: Awesome H2O tutorials: <https://github.com/h2oai/awesome-h2o>, Market Type research of Van Tharp Institute: <https://www.vantharp.com/>, Reinforcement Learning R package: <https://CRAN.R-project.org/package=ReinforcementLearning>.

Readme

lazytrade

Travis build
status codecov Lifecycle:
maturing CRAN
status

The goal of lazytrade is to keep all functions and scripts of the lazytrade educational project on UDEMY. Functions are providing an opportunity to learn Computer and Data Science using example of Algorithmic Trading. Please kindly not that this project was created for Educational Purposes only!

Installation

You can install the released version of lazytrade from CRAN with:

install.packages("lazytrade")

And the development version from GitHub with:

# install.packages("devtools")
devtools::install_github("vzhomeexperiments/lazytrade")

Several ideas explored in this package

  • Data manipulation and analysis of performed trades results
  • Reinforcement Learning for Automated Trading Risk Management
  • Data manipulation and preparation for Machine Learning (transposing, aggregation, lagging, etc)
  • Using Deep Learning for prediction of Market Types (Classification)
  • Using Deep Learning for prediction of future price change (Regression)
  • Strategy Tests simulations
  • Utility functions to generate passwords, initialization files, encryption of passwords, etc
  • Explored idea of building a model using random structures combined with an automated functional (strategy) test to improve model performance
  • Overall, all functions have working examples with relevant documented sample data included in the package

Example - prepare data for machine learning

This is a basic example which shows you how to solve a common problem:

library(lazytrade)
library(magrittr, warn.conflicts = FALSE)
## basic example code
# Convert a time series vector to matrix with 64 columns
macd_m <- seq(1:1000) %>% as.data.frame() %>% to_m(20)

head(macd_m, 2)
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
#> [1,]    1    2    3    4    5    6    7    8    9    10    11    12    13    14
#> [2,]   21   22   23   24   25   26   27   28   29    30    31    32    33    34
#>      [,15] [,16] [,17] [,18] [,19] [,20]
#> [1,]    15    16    17    18    19    20
#> [2,]    35    36    37    38    39    40

Example - aggregate multiple log files and visualize results

Multiple log files could be joined into one data object

library(lazytrade)
library(readr)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(magrittr)
library(lubridate)
#> 
#> Attaching package: 'lubridate'
#> The following objects are masked from 'package:base':
#> 
#>     date, intersect, setdiff, union

# files are located in the sample folders
DFOLDER <- system.file("extdata/RES", package = "lazytrade")

DFR <- opt_aggregate_results(fold_path = DFOLDER)

This data object can be visualized

library(ggplot2)
opt_create_graphs(x = DFR, outp_path = dir,graph_type = 'bars')

Or just visualize results with time-series plot

opt_create_graphs(x = DFR, outp_path = dir,graph_type = 'ts')

Example - leverage Reinforcement Learning for Risk Management

Example below would generate RL policy based on the trade results achieved so far

library(dplyr)
library(ReinforcementLearning)
library(magrittr)

data(data_trades)
states <- c("tradewin", "tradeloss")
actions <- c("ON", "OFF")
control <- list(alpha = 0.7, gamma = 0.3, epsilon = 0.1)
generate_RL_policy(data_trades, states, actions, control)
#>   TradeState Policy
#> 1  tradeloss     ON
#> 2   tradewin    OFF

Example - generating passwords for trading platforms login

Multiple trading accounts require passwords, package contains function that may easily generate random passwords:

library(lazytrade)
library(stringr)
library(magrittr)
library(openssl)
library(readr)

#generate 8digit password for trading platform
util_generate_password(salt = 'random text')
#>          .
#> 1 ce37D988

Example - generate initialization files for MT4 platform

Facilitate generation of initialisation files:

library(lazytrade)

dir <- normalizePath(tempdir(),winslash = "/")

# test file to launch MT4 terminal with parameters
write_ini_file(mt4_Profile = "Default",
               mt4_Login = "12345678",
               mt4_Password = "password",
               mt4_Server = "BrokerServerName",
               dss_inifilepath = dir,
               dss_inifilename = "prod_T1.ini",
               dss_mode = "prod")

Notes to remind myself how to create R package

This readme file

What is special about using README.Rmd instead of just README.md? You can include R chunks like so:

summary(cars)
#>      speed           dist       
#>  Min.   : 4.0   Min.   :  2.00  
#>  1st Qu.:12.0   1st Qu.: 26.00  
#>  Median :15.0   Median : 36.00  
#>  Mean   :15.4   Mean   : 42.98  
#>  3rd Qu.:19.0   3rd Qu.: 56.00  
#>  Max.   :25.0   Max.   :120.00

You’ll still need to render README.Rmd regularly, to keep README.md up-to-date.

taken from https://r-pkgs.org/intro.html

Generating Documentation

Title of the package

Create right title case for the title of the package By running this command… tools::toTitleCase("Learn computer and data science using algorithmic trading") the Title will become: “Learn Computer and Data Science using Algorithmic Trading”

Re-generating documentation

Run this code to re-generate documentation devtools::document()

Fixing License

Run this code to fix license: usethis::use_mit_license(name = "Vladimir Zhbanko")

Adding data to the package for internal tests

Run this code to add data to the folder data/ x <- sample(1000) usethis::use_data(x)

To update this data: x <- sample(2000) usethis::use_data(x, overwrite = T)

Note: use option ’LazyLoad` to make data available only when user wants it always include LazyData: true in your DESCRIPTION. Note: to document dataset see https://stackoverflow.com/questions/2310409/how-can-i-document-data-sets-with-roxygen

Document dataset using the R script R/datasets.R

Use data in the function with data(x)

Adding examples to test package function

Tests setup first time

Run this command to setup tests ‘usethis::use_testthat()’

This will create a folder with the name tests

Inside this folder there will be another folder testthat.

Examples in Roxygen code

@examples …

code to execute during package checks

@examples

/donttest{

code to NOT execute during package checks

}

Testing a package

Create a test script

Run this command to create a new script with the test skeleton:

usethis::use_test("profit_factor.R")

Enrich the test script

Details:

  1. add libraries used for test
  2. add function context("profit_factor")
  3. add function test_that(“test description”, {test process})
  4. load data using function data(named_data_object)

Example:

library(testthat)
#> 
#> Attaching package: 'testthat'
#> The following object is masked from 'package:dplyr':
#> 
#>     matches
#> The following objects are masked from 'package:magrittr':
#> 
#>     equals, is_less_than, not
library(dplyr)
library(magrittr)
context("profit_factor")

test_that("test value of the calculation", {

  data(profit_factor_data)

  DF_Stats <- profit_factor_data %>%
    group_by(X1) %>%
    summarise(PnL = sum(X5),
              NumTrades = n(),
              PrFact = profit_factor(X5)) %>%
    select(PrFact) %>%
    head(1) %>%
    as.vector() %>%
    round(3)

  expect_equal(DF_Stats$PrFact, 0.68)

})

Test of the coverage for the script

Test coverage shows you what you’ve tested devtools::test_coverage_file()

devtools::test_coverage_file()

Automated checks

This will add automatic test coverage badge to the readme file on github usethis::use_coverage()

Checking package

Step 1. devtools::document() Step 2. devtools::run_examples() Step

  1. Menu ‘Build’ Clean and Rebuild Step 4. ‘Check’ devtools::check()

Locally checking package with –run-donttest enabled

This is now a default option

Whenever examples construct is used author of the package must insure that those examples are running. Such examples are those that would require longer test execution. To perform this test package needs to be checked with the following command:

devtools::check(run_dont_test = TRUE)

whenever a quick check is required:

devtools::check(run_dont_test = FALSE) ???

Handling functions that write files

In case functions are writing files there are few considerations to take into account:

  • examples section must contain working example of code that writes files
  • example code must write to the temporary directory defined by tempdir() function
  • after package check performed with devtools::check() there should nothing remain in the ‘tmp/’ directory

Considerations

File names defined by function tempdir() would look like this:

# > tempdir()
# [1] "/tmp/RtmpkaFStZ"

File names defined by function tempfile() would look like this:

# > tempfile()
# [1] "/tmp/RtmpkaFStZ/file7a33be992b4"

This is example of how function write_csv example works:

tmp <- tempfile()
write_csv(mtcars, tmp)

results of this code are correctly stored to the temporary file

however this example from readr package function write_csv is showing that file will be written to the ‘/tmp/’ directory

dir <- tempdir()
write_tsv(mtcars, file.path(dir, "mtcars.tsv.gz"))

Deleting files after running examples:

We use function unlink() to do this:

unlink("/tmp/*.csv", recursive = TRUE, force = TRUE)

and we check that there is nothing more remained:

dir("/tmp/*.csv")

CRAN Submission Tips and Tricks

Many notes while using global variables:

see https://stackoverflow.com/questions/9439256/how-can-i-handle-r-cmd-check-no-visible-binding-for-global-variable-notes-when see https://github.com/HughParsonage/grattan/blob/master/R/zzz.R

Unfortunate note on specific flavors

After first submission there are some notes on specific R flavors

This question was addressed here but yet it’s not answered: https://stackoverflow.com/questions/48487541/r-cmd-check-note-namespace-in-imports-field-not-imported

Define min R version

https://stackoverflow.com/questions/38686427/determine-minimum-r-version-for-all-package-dependencies

When functions are writing to the file

It’s important to avoid that function write to the directory other then tempdir() Construct file name must be done using file.name() function as follow:

dir_name <- normalizePath(tempdir(),winslash = "/")
file_name <- paste0('my_file', 1, '.csv')
# this needs to be used in the function
full_path <- file.path(dir_name, file_name)

Versioning of the package

https://r-pkgs.org/description.html#version

..

Test Environments

Clone package from GitHub and test check it in Docker Container

  • started docker container vladdsm/docker-r-studio
  • new project package
  • clone from vzhomeexperiments/lazytrade.git
  • use check button to pass the test

Build package

devtools::build()

Adding Readme Rmd

usethis::use_readme_rmd()

Automatic check with Travis

usethis::use_travis()

Upload package to CRAN

Setup the new version of the package:

usethis::use_release_issue()

Follow checklist before upload to CRAN:

devtools::release_checks()

then:

devtools::release()

before release checks

spelling devtools::spell_check()

checking on R hub rhub::validate_email() rhub::check( platform="windows-x86_64-devel", env_vars=c(R_COMPILE_AND_INSTALL_PACKAGES = "always") ) devtools::check_rhub(interactive = F)

checking with release devtools::check_win_release()

checking win devel devtools::check_win_devel()

checking win old devel devtools::check_win_oldrelease()

check with rocker R in container - use docker image with R Studio, - clone repo, build, check package…

Update news.md file

uploading the package archive to CRAN

https://cran.r-project.org/submit.html

Functions in lazytrade

Name Description
aml_collect_data Function to read new data, transform, aggregate and save data for further retraining of regression model for a single currency pair
create_transposed_data Create Transposed Data
EURUSDM15X75 Table with indicator and price change dataset
aml_score_data Function to score new data and predict change for each single currency pair
aml_test_model Function to test the model and conditionally decide to update existing model for a single currency pair
create_labelled_data Create labelled data
aml_make_model Function to train Deep Learning regression model for a single asset
TradeStatePolicy Table with Trade States and sample of actual policy for those states
DFR Table with predicted price change
check_if_optimize Function check_if_optimize.
evaluate_market_type Function to score data and predict current market type using pre-trained classification model
decrypt_mykeys Function that decrypt encrypted content
data_trades Table with Trade results samples
import_data Import Data file with Trade Logs to R.
mt_make_model Function to train Deep Learning Classification model for Market Type recognition
encrypt_api_key Encrypt api keys
evaluate_macroeconomic_event Function used to evaluate market type situation by reading the file with Macroeconomic Events and writing a trigger to the trading robot
generate_RL_policy_mt Function performs RL and generates model policy for each Market Type
import_data_mt Import Market Type related Data to R from the Sandbox
indicator_dataset Table with indicator dataset
opt_aggregate_results Function to aggregate trading results from multiple folders and files
profit_factor Calculate Profit Factor
get_profit_factorDF Function that returns the profit factors of the systems in a form of a DataFrame
indicator_dataset_big Table with indicator dataset, 30000 rows
generate_RL_policy Function performs RL and generates model policy
log_RL_progress_mt Function to log RL progress, dedicated to Market Types
profit_factorDF Table with Trade results samples
macd_ML2 Table with indicator and market type category used to train model
load_asset_data Load and Prepare Asset Data
record_policy_mt Record Reinforcement Learning Policy for Market Types
price_dataset Table with price dataset
macd_df Table with one column indicator dataset
log_RL_progress Function to log RL progress.
self_learn_ai_R Function to train Deep Learning regression model
result_R Table with predicte price change
policy_tr_systDF Table with Market Types and sample of actual policy for those states
opt_create_graphs Function to create summary graphs of the trading results
result_R1 Table with aggregated trade results
result_prev Table with one column as result from the model prediction
price_dataset_big Table with price dataset, 30000 rows
write_control_parameters Function to find and write the best control parameters.
write_control_parameters_mt Function to find and write the best control parameters.
test_data_pattern Table with several columns containing indicator values and Label values
to_m Convert time series data to matrix with defined number of columns
test_model Test model using independent price data.
util_generate_password R function to generate random passwords for MT4 platform or other needs
write_ini_file Create initialization files to launch MT4 platform with specific configuration
trading_systemDF Table with trade data and joined market type info
macd_100 Table with indicator only used to train model, 128 col 1646 rows
profit_factor_data Table with Trade results samples
x_test_model Table with a dataset to test the Model
writeCommandViaCSV Write csv files with indicated commands to the external system
record_policy Record Reinforcement Learning Policy.
write_command_via_csv Write csv files with indicated commands to the external system
y Table with indicators and price change which is used to train model
No Results!

Last month downloads

Details

Type Package
License MIT + file LICENSE
URL https://vladdsm.github.io/myblog_attempt/topics/lazy%20trading/, https://github.com/vzhomeexperiments/lazytrade
BugReports https://github.com/vzhomeexperiments/lazytrade/issues
Encoding UTF-8
LazyData true
RoxygenNote 7.1.1
NeedsCompilation no
Packaged 2020-09-16 11:56:31 UTC; fxtrams
Repository CRAN
Date/Publication 2020-09-16 12:20:03 UTC

Include our badge in your README

[![Rdoc](http://www.rdocumentation.org/badges/version/lazytrade)](http://www.rdocumentation.org/packages/lazytrade)