From cb7c36a512d0c75b431eb83831386cb655575c79 Mon Sep 17 00:00:00 2001 From: Gerald Lonlas Date: Tue, 2 Jan 2018 22:43:00 -0800 Subject: [PATCH] Add Backtesting and Hyperopt documentation --- docs/backtesting.md | 104 +++++++++++++++ docs/bot-optimization.md | 48 +------ docs/hyperopt.md | 269 +++++++++++++++++++++++++++++++++++++++ docs/index.md | 5 +- 4 files changed, 379 insertions(+), 47 deletions(-) create mode 100644 docs/backtesting.md create mode 100644 docs/hyperopt.md diff --git a/docs/backtesting.md b/docs/backtesting.md new file mode 100644 index 000000000..d7073365f --- /dev/null +++ b/docs/backtesting.md @@ -0,0 +1,104 @@ +# Backtesting +This page explains how to validate your strategy performance by using +Backtesting. + +## Table of Contents +- [Test your strategy with Backtesting](#test-your-strategy-with-backtesting) +- [Understand the backtesting result](#understand-the-backtesting-result) + +## Test your strategy with Backtesting +Now you have good Buy and Sell strategies, you want to test it against +real data. This is what we call +[backtesting](https://en.wikipedia.org/wiki/Backtesting). + + +Backtesting will use the crypto-currencies (pair) from your config file +and load static tickers located in +[/freqtrade/tests/testdata](https://github.com/gcarq/freqtrade/tree/develop/freqtrade/tests/testdata). +If the 5 min and 1 min ticker for the crypto-currencies to test is not +already in the `testdata` folder, backtesting will download them +automatically. Testdata files will not be updated until you specify it. + +The result of backtesting will confirm you if your bot as more chance to +make a profit than a loss. + + +The backtesting is very easy with freqtrade. + +### Run a backtesting against the currencies listed in your config file +**With 5 min tickers (Per default)** +```bash +python3 ./freqtrade/main.py backtesting --realistic-simulation +``` + +**With 1 min tickers** +```bash +python3 ./freqtrade/main.py backtesting --realistic-simulation --ticker-interval 1 +``` + +**Reload your testdata files** +```bash +python3 ./freqtrade/main.py backtesting --realistic-simulation --refresh-pairs-cached +``` + +**With live data (do not alter your testdata files)** +```bash +python3 ./freqtrade/main.py backtesting --realistic-simulation --live +``` + +For help about backtesting usage, please refer to +[Backtesting commands](#backtesting-commands). + +## Understand the backtesting result +The most important in the backtesting is to understand the result. + +A backtesting result will look like that: +``` +====================== BACKTESTING REPORT ================================ +pair buy count avg profit % total profit BTC avg duration +-------- ----------- -------------- ------------------ -------------- +BTC_ETH 56 -0.67 -0.00075455 62.3 +BTC_LTC 38 -0.48 -0.00036315 57.9 +BTC_ETC 42 -1.15 -0.00096469 67.0 +BTC_DASH 72 -0.62 -0.00089368 39.9 +BTC_ZEC 45 -0.46 -0.00041387 63.2 +BTC_XLM 24 -0.88 -0.00041846 47.7 +BTC_NXT 24 0.68 0.00031833 40.2 +BTC_POWR 35 0.98 0.00064887 45.3 +BTC_ADA 43 -0.39 -0.00032292 55.0 +BTC_XMR 40 -0.40 -0.00032181 47.4 +TOTAL 419 -0.41 -0.00348593 52.9 +``` + +The last line will give you the overall performance of your strategy, +here: +``` +TOTAL 419 -0.41 -0.00348593 52.9 +``` + +We understand the bot has made `419` trades for an average duration of +`52.9` min, with a performance of `-0.41%` (loss), that means it has +lost a total of `-0.00348593 BTC`. + +As you will see your strategy performance will be influenced by your buy +strategy, your sell strategy, and also by the `minimal_roi` and +`stop_loss` you have set. + +As for an example if your minimal_roi is only `"0": 0.01`. You cannot +expect the bot to make more profit than 1% (because it will sell every +time a trade will reach 1%). +```json +"minimal_roi": { + "0": 0.01 +}, +``` + +On the other hand, if you set a too high `minimal_roi` like `"0": 0.55` +(55%), there is a lot of chance that the bot will never reach this +profit. Hence, keep in mind that your performance is a mix of your +strategies, your configuration, and the crypto-currency you have set up. + +## Next step +Great, your strategy is profitable. What if the bot can give your the +optimal parameters to use for your strategy? +Your next step is to learn [how to find optimal parameters with Hyperopt](https://github.com/gcarq/freqtrade/blob/develop/docs/hyperopt.md) \ No newline at end of file diff --git a/docs/bot-optimization.md b/docs/bot-optimization.md index ea4ea9ae4..a49ffdc26 100644 --- a/docs/bot-optimization.md +++ b/docs/bot-optimization.md @@ -1,14 +1,10 @@ # Bot Optimization -This page explains where to customize your strategies, validate their -performance by using Backtesting, and tuning them by finding the optimal -parameters with Hyperopt. +This page explains where to customize your strategies, and add new +indicators. ## Table of Contents - [Change your strategy](#change-your-strategy) - [Add more Indicator](#add-more-indicator) -- [Test your strategy with Backtesting](#test-your-strategy-with-backtesting) -- [Find optimal parameters with Hyperopt](#find-optimal-parameters-with-hyperopt) -- [Show your buy strategy on a graph](#show-your-buy-strategy-on-a-graph) ## Change your strategy The bot is using buy and sell strategies to buy and sell your trades. @@ -115,43 +111,7 @@ def populate_indicators(dataframe: DataFrame) -> DataFrame: return dataframe ``` -## Test your strategy with Backtesting -Now you have good Buy and Sell strategies, you want to test it against -real data. This is what we call [backtesting](https://en.wikipedia.org/wiki/Backtesting). - -Backtesting will use the crypto-currencies (pair) tickers located in -[/freqtrade/tests/testdata](https://github.com/gcarq/freqtrade/tree/develop/freqtrade/tests/testdata). -If the 5 min and 1 min ticker for the crypto-currencies to test is not -already in the `testdata` folder, backtesting will download them -automatically. Testdata files will not be updated until your specify it. - -### Run a backtesting against the currencies listed in your config file -**With 5 min tickers (Per default)** -```bash -python3 ./freqtrade/main.py backtesting --realistic-simulation -``` - -**With 1 min tickers** -```bash -python3 ./freqtrade/main.py backtesting --realistic-simulation --ticker-interval 1 -``` - -**Reload your testdata files** -```bash -python3 ./freqtrade/main.py backtesting --realistic-simulation --refresh-pairs-cached -``` - -**With live data (do not alter your testdata files)** -```bash -python3 ./freqtrade/main.py backtesting --realistic-simulation --live -``` - -## Find optimal parameters with Hyperopt -*To be completed, please feel free to complete this section.* - -## Show your buy strategy on a graph -*To be completed, please feel free to complete this section.* ## Next step -Now you have a perfect bot and want to control it from Telegram. Your -next step is to learn [Telegram usage](https://github.com/gcarq/freqtrade/blob/develop/docs/telegram-usage.md). \ No newline at end of file +Now you have a perfect strategy you probably want to backtesting it. +Your next step is to learn [How to use ](https://github.com/gcarq/freqtrade/blob/develop/docs/backtesting.md). diff --git a/docs/hyperopt.md b/docs/hyperopt.md new file mode 100644 index 000000000..212a3f56c --- /dev/null +++ b/docs/hyperopt.md @@ -0,0 +1,269 @@ +# Hyperopt +This page explains how to tune your strategy by finding the optimal +parameters with Hyperopt. + +## Table of Contents +- [Prepare your Hyperopt](#prepare-hyperopt) + - [1. Configure your Guards and Triggers](#1-configure-your-guards-and-triggers) + - [2. Update the hyperopt config file](#2-update-the-hyperopt-config-file) +- [Advanced Hyperopt notions](#advanced-notions) + - [Understand the Guards and Triggers](#understand-the-guards-and-triggers) +- [Execute Hyperopt](#execute-hyperopt) + - [Hyperopt with MongoDB](#hyperopt-with-mongoDB) +- [Understand the hyperopts result](#understand-the-backtesting-result) + +## Prepare Hyperopt +Before we start digging in Hyperopt, we recommend you to take a look at +out Hyperopt file +[freqtrade/optimize/hyperopt.py](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py) + +### 1. Configure your Guards and Triggers +There are two places you need to change to add a new buy strategy for +testing: +- Inside the [populate_buy_trend()](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L167-L207). +- Inside the [SPACE dict](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L47-L94). + +There you have two different type of indicators: 1. `guards` and 2. +`triggers`. +1. Guards are conditions like "never buy if ADX < 10", or never buy if +current price is over EMA10. +2. Triggers are ones that actually trigger buy in specific moment, like +"buy when EMA5 crosses over EMA10" or buy when close price touches lower +bollinger band. + +HyperOpt will, for each eval round, pick just ONE trigger, and possibly +multiple guards. So that the constructed strategy will be something like +"*buy exactly when close price touches lower bollinger band, BUT only if +ADX > 10*". + + +If you have updated the buy strategy, means change the content of +`populate_buy_trend()` function you have to update the `guards` and +`triggers` hyperopts must used. + +As for an example if your `populate_buy_trend()` function is: +```python +def populate_buy_trend(dataframe: DataFrame) -> DataFrame: + dataframe.loc[ + (dataframe['rsi'] < 35) & + (dataframe['adx'] > 65), + 'buy'] = 1 + + return dataframe +``` + +Your hyperopt file must contains `guards` to find the right value for +`(dataframe['adx'] > 65)` & and `(dataframe['plus_di'] > 0.5)`. That +means you will need to enable/disable triggers. + +In our case the `SPACE` and `populate_buy_trend` in hyperopt.py file +will be look like: +```python +SPACE = { + 'rsi': hp.choice('rsi', [ + {'enabled': False}, + {'enabled': True, 'value': hp.quniform('rsi-value', 20, 40, 1)} + ]), + 'adx': hp.choice('adx', [ + {'enabled': False}, + {'enabled': True, 'value': hp.quniform('adx-value', 15, 50, 1)} + ]), + 'trigger': hp.choice('trigger', [ + {'type': 'lower_bb'}, + {'type': 'faststoch10'}, + {'type': 'ao_cross_zero'}, + {'type': 'ema5_cross_ema10'}, + {'type': 'macd_cross_signal'}, + {'type': 'sar_reversal'}, + {'type': 'stochf_cross'}, + {'type': 'ht_sine'}, + ]), +} + +... + +def populate_buy_trend(dataframe: DataFrame) -> DataFrame: + conditions = [] + # GUARDS AND TRENDS + if params['adx']['enabled']: + conditions.append(dataframe['adx'] > params['adx']['value']) + if params['rsi']['enabled']: + conditions.append(dataframe['rsi'] < params['rsi']['value']) + + # TRIGGERS + triggers = { + 'lower_bb': dataframe['tema'] <= dataframe['blower'], + 'faststoch10': (crossed_above(dataframe['fastd'], 10.0)), + 'ao_cross_zero': (crossed_above(dataframe['ao'], 0.0)), + 'ema5_cross_ema10': (crossed_above(dataframe['ema5'], dataframe['ema10'])), + 'macd_cross_signal': (crossed_above(dataframe['macd'], dataframe['macdsignal'])), + 'sar_reversal': (crossed_above(dataframe['close'], dataframe['sar'])), + 'stochf_cross': (crossed_above(dataframe['fastk'], dataframe['fastd'])), + 'ht_sine': (crossed_above(dataframe['htleadsine'], dataframe['htsine'])), + } + ... +``` + + +### 2. Update the hyperopt config file +Hyperopt is using a dedicated config file. At this moment hyperopt +cannot use your config file. It is also made on purpose to allow you +testing your strategy with different configurations. + +The Hyperopt configuration is located in +[freqtrade/optimize/hyperopt_conf.py](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt_conf.py). + + +## Advanced notions +### Understand the Guards and Triggers +When you need to add the new guards and triggers to be hyperopt +parameters, you do this by adding them into the [SPACE dict](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L47-L94). + +If it's a trigger, you add one line to the 'trigger' choice group and that's it. + +If it's a guard, you will add a line like this: +``` +'rsi': hp.choice('rsi', [ + {'enabled': False}, + {'enabled': True, 'value': hp.quniform('rsi-value', 20, 40, 1)} +]), +``` +This says, "*one of guards is RSI, it can have two values, enabled or +disabled. If it is enabled, try different values for it between 20 and 40*". + +So, the part of the strategy builder using the above setting looks like +this: +``` +if params['rsi']['enabled']: + conditions.append(dataframe['rsi'] < params['rsi']['value']) +``` +It checks if Hyperopt wants the RSI guard to be enabled for this +round `params['rsi']['enabled']` and if it is, then it will add a +condition that says RSI must be < than the value hyperopt picked +for this evaluation, that is given in the `params['rsi']['value']`. + +That's it. Now you can add new parts of strategies to Hyperopt and it +will try all the combinations with all different values in the search +for best working algo. + + +### Add a new Indicators +If you want to test an indicator that isn't used by the bot currently, +you need to add it to +[freqtrade/analyze.py](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/analyze.py#L40-L70) +inside the `populate_indicators` function. + +## Execute Hyperopt +Once you have updated your hyperopt configuration you can run it. +Because hyperopt tries a lot of combination to find the best parameters +it will take time you will have the result (more than 30 mins). + +We strongly recommend to use `screen` to prevent any connection loss. +```bash +python3 ./freqtrade/main.py -c config.json hyperopt +``` + +### Hyperopt with MongoDB +Hyperopt with MongoDB, is like Hyperopt under steroids. As you saw by +executing the previous command is the execution takes a long time. +To accelerate it you can use hyperopt with MongoDB. + +To run hyperopt with MongoDb you will need 3 terminals. + +**Terminal 1: Start MongoDB** +```bash +cd +source .env/bin/activate +python3 scripts/start-mongodb.py +``` + +**Terminal 2: Start Hyperopt worker** +```bash +cd +source .env/bin/activate +python3 scripts/start-hyperopt-worker.py +``` + +**Terminal 3: Start Hyperopt with MongoDB** +```bash +cd +source .env/bin/activate +python3 ./freqtrade/main.py -c config.json hyperopt --use-mongodb +``` + +**Re-run an Hyperopt** +To re-run Hyperopt you have to delete the existing MongoDB table. +```bash +cd +rm -rf .hyperopt/mongodb/ +``` + +## Understand the hyperopts result +Once Hyperopt is completed you can use the result to adding new buy +signal. Given following result from hyperopt: +``` +Best parameters: +{ + "adx": 1, + "adx-value": 15.0, + "fastd": 1, + "fastd-value": 40.0, + "green_candle": 1, + "mfi": 0, + "over_sar": 0, + "rsi": 1, + "rsi-value": 37.0, + "trigger": 0, + "uptrend_long_ema": 1, + "uptrend_short_ema": 0, + "uptrend_sma": 0 +} + +Best Result: + 2197 trades. Avg profit 1.84%. Total profit 0.79367541 BTC. Avg duration 241.0 mins. +``` + +You should understand this result like: +- You should **consider** the guard "adx" (`"adx": 1,` = `adx` is true) +and the best value is `15.0` (`"adx-value": 15.0,`) +- You should **consider** the guard "fastd" (`"fastd": 1,` = `fastd` +is true) and the best value is `40.0` (`"fastd-value": 40.0,`) +- You should **consider** to enable the guard "green_candle" +(`"green_candle": 1,` = `candle` is true) but this guards as no +customizable value. +- You should **ignore** the guard "mfi" (`"mfi": 0,` = `mfi` is false) +- and so on... + + +You have to look from +[freqtrade/optimize/hyperopt.py](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L170-L200) +what those values match to. + +So for example you had `adx-value: 15.0` (and `adx: 1` was true) so we +would look at `adx`-block from +[freqtrade/optimize/hyperopt.py](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L178-L179). +That translates to the following code block to +[analyze.populate_buy_trend()](https://github.com/gcarq/freqtrade/blob/develop/freqtrade/analyze.py#L73) +``` +(dataframe['adx'] > 15.0) +``` + +So translating your whole hyperopt result to as the new buy-signal +would be the following: +``` +def populate_buy_trend(dataframe: DataFrame) -> DataFrame: + dataframe.loc[ + ( + (dataframe['adx'] > 15.0) & # adx-value + (dataframe['fastd'] < 40.0) & # fastd-value + (dataframe['close'] > dataframe['open']) & # green_candle + (dataframe['rsi'] < 37.0) & # rsi-value + (dataframe['ema50'] > dataframe['ema100']) # uptrend_long_ema + ), + 'buy'] = 1 + return dataframe +``` + +## Next step +Now you have a perfect bot and want to control it from Telegram. Your +next step is to learn the [Telegram usage](https://github.com/gcarq/freqtrade/blob/develop/docs/telegram-usage.md). diff --git a/docs/index.md b/docs/index.md index 5df745ae8..ed940d132 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,9 +22,8 @@ Pull-request. Do not hesitate to reach us on - [Bot Optimization](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md) - [Change your strategy](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md#change-your-strategy) - [Add more Indicator](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md#add-more-indicator) - - [Test your strategy with Backtesting](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md#test-your-strategy-with-backtesting) - - [Find optimal parameters with Hyperopt](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md#find-optimal-parameters-with-hyperopt) - - [Show your buy strategy on a graph](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-optimization.md#show-your-buy-strategy-on-a-graph) + - [Test your strategy with Backtesting](https://github.com/gcarq/freqtrade/blob/develop/docs/backtesting.md) + - [Find optimal parameters with Hyperopt](https://github.com/gcarq/freqtrade/blob/develop/docs/hyperopt.md) - [Control the bot with telegram](https://github.com/gcarq/freqtrade/blob/develop/docs/telegram-usage.md) - [Contribute to the project](https://github.com/gcarq/freqtrade/blob/develop/CONTRIBUTING.md) - [How to contribute](https://github.com/gcarq/freqtrade/blob/develop/CONTRIBUTING.md)