adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update hyperopt documentation

Browse Source
This commit is contained in:
Janne Sinivirta 2018-07-02 09:56:58 +03:00
parent 0ce08932ed
commit a58d51ded0
1 changed files with 96 additions and 183 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

281
docs/hyperopt.md
View File

@ -1,153 +1,109 @@
# Hyperopt # Hyperopt
This page explains how to tune your strategy by finding the optimal This page explains how to tune your strategy by finding the optimal
parameters with Hyperopt. parameters, process called hyperparameter optimization. The bot uses several
algorithms included in `scikit-optimize` package to accomplish this. The
search will burn all your CPU cores, make your laptop sound like a fighter jet
and still take a long time.
## Table of Contents ## Table of Contents
- [Prepare your Hyperopt](#prepare-hyperopt) - [Prepare your Hyperopt](#prepare-hyperopt)
- [1. Configure your Guards and Triggers](#1-configure-your-guards-and-triggers) - [Configure your Guards and Triggers](#configure-your-guards-and-triggers)
- [2. Update the hyperopt config file](#2-update-the-hyperopt-config-file) - [Solving a Mystery](#solving-a-mystery)
- [Advanced Hyperopt notions](#advanced-notions) - [Adding New Indicators](#adding-new-indicators)
- [Understand the Guards and Triggers](#understand-the-guards-and-triggers)
- [Execute Hyperopt](#execute-hyperopt) - [Execute Hyperopt](#execute-hyperopt)
- [Understand the hyperopts result](#understand-the-backtesting-result) - [Understand the hyperopts result](#understand-the-backtesting-result)
## Prepare Hyperopt ## Prepare Hyperopting
Before we start digging in Hyperopt, we recommend you to take a look at We recommend you start by taking a look at `hyperopt.py` file located in [freqtrade/optimize](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py)
your strategy file located into [user_data/strategies/](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py)
### 1. Configure your Guards and Triggers ### Configure your Guards and Triggers
There are two places you need to change in your strategy file to add a There are two places you need to change in your strategy file to add a
new buy strategy for testing: new buy strategy for testing:
- Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py#L278-L294). - Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L278-L294).
- Inside [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py#L244-L297) known as `SPACE`. - Inside [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L218-L229)
and the associated methods `indicator_space`, `roi_space`, `stoploss_space`.
There you have two different type of indicators: 1. `guards` and 2. There you have two different type of indicators: 1. `guards` and 2. `triggers`.
`triggers`.
1. Guards are conditions like "never buy if ADX < 10", or never buy if 1. Guards are conditions like "never buy if ADX < 10", or never buy if
current price is over EMA10. current price is over EMA10.
2. Triggers are ones that actually trigger buy in specific moment, like 2. Triggers are ones that actually trigger buy in specific moment, like
"buy when EMA5 crosses over EMA10" or buy when close price touches lower "buy when EMA5 crosses over EMA10" or buy when close price touches lower
bollinger band. bollinger band.
HyperOpt will, for each eval round, pick just ONE trigger, and possibly Hyperoptimization will, for each eval round, pick one trigger and possibly
multiple guards. So that the constructed strategy will be something like multiple guards. The constructed strategy will be something like
"*buy exactly when close price touches lower bollinger band, BUT only if "*buy exactly when close price touches lower bollinger band, BUT only if
ADX > 10*". ADX > 10*".
If you have updated the buy strategy, ie. changed the contents of
If you have updated the buy strategy, means change the content of
`populate_buy_trend()` method you have to update the `guards` and `populate_buy_trend()` method you have to update the `guards` and
`triggers` hyperopts must used. `triggers` hyperopts must use.
As for an example if your `populate_buy_trend()` method is: ## Solving a Mystery
```python
Let's say you are curious: should you use MACD crossings or lower Bollinger
Bands to trigger your buys. And you also wonder should you use RSI or ADX to
help with those buy decisions. If you decide to use RSI or ADX, which values
should I use for them? So let's use hyperparameter optimization to solve this
mystery.
We will start by defining a search space:
```
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(20, 40, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal'], name='trigger')
]
```
Above definition says: I have five parameters I want you to randomly combine
to find the best combination. Two of them are integer values (`adx-value`
and `rsi-value`) and I want you test in the range of values 20 to 40.
Then we have three category variables. First two are either `True` or `False`.
We use these to either enable or disable the ADX and RSI guards. The last
one we call `trigger` and use it to decide which buy trigger we want to use.
So let's write the buy strategy using these values:
```
def populate_buy_trend(dataframe: DataFrame) -> DataFrame: def populate_buy_trend(dataframe: DataFrame) -> DataFrame:
conditions = []
# GUARDS AND TRENDS
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
dataframe.loc[ dataframe.loc[
(dataframe['rsi'] < 35) & reduce(lambda x, y: x & y, conditions),
(dataframe['adx'] > 65),
'buy'] = 1 'buy'] = 1
return dataframe return dataframe
return populate_buy_trend
``` ```
Your hyperopt file must contain `guards` to find the right value for Hyperopting will now call this `populate_buy_trend` as many times you ask it (`epochs`)
`(dataframe['adx'] > 65)` & and `(dataframe['plus_di'] > 0.5)`. That with different value combinations. It will then use the given historical data and make
means you will need to enable/disable triggers. buys based on the buy signals generated with the above function and based on the results
it will end with telling you which paramter combination produced the best profits.
In our case the `SPACE` and `populate_buy_trend` in your strategy file ### Adding New Indicators
will look like:
```python
space = {
'rsi': hp.choice('rsi', [
{'enabled': False},
{'enabled': True, 'value': hp.quniform('rsi-value', 20, 40, 1)}
]),
'adx': hp.choice('adx', [
{'enabled': False},
{'enabled': True, 'value': hp.quniform('adx-value', 15, 50, 1)}
]),
'trigger': hp.choice('trigger', [
{'type': 'lower_bb'},
{'type': 'faststoch10'},
{'type': 'ao_cross_zero'},
{'type': 'ema5_cross_ema10'},
{'type': 'macd_cross_signal'},
{'type': 'sar_reversal'},
{'type': 'stochf_cross'},
{'type': 'ht_sine'},
]),
}
...
def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
conditions = []
# GUARDS AND TRENDS
if params['adx']['enabled']:
conditions.append(dataframe['adx'] > params['adx']['value'])
if params['rsi']['enabled']:
conditions.append(dataframe['rsi'] < params['rsi']['value'])
# TRIGGERS
triggers = {
'lower_bb': dataframe['tema'] <= dataframe['blower'],
'faststoch10': (crossed_above(dataframe['fastd'], 10.0)),
'ao_cross_zero': (crossed_above(dataframe['ao'], 0.0)),
'ema5_cross_ema10': (crossed_above(dataframe['ema5'], dataframe['ema10'])),
'macd_cross_signal': (crossed_above(dataframe['macd'], dataframe['macdsignal'])),
'sar_reversal': (crossed_above(dataframe['close'], dataframe['sar'])),
'stochf_cross': (crossed_above(dataframe['fastk'], dataframe['fastd'])),
'ht_sine': (crossed_above(dataframe['htleadsine'], dataframe['htsine'])),
}
...
```
### 2. Update the hyperopt config file
Hyperopt is using a dedicated config file. Currently hyperopt
cannot use your config file. It is also made on purpose to allow you
testing your strategy with different configurations.
The Hyperopt configuration is located in
[user_data/hyperopt_conf.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopt_conf.py).
## Advanced notions
### Understand the Guards and Triggers
When you need to add the new guards and triggers to be hyperopt
parameters, you do this by adding them into the [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py#L244-L297).
If it's a trigger, you add one line to the 'trigger' choice group and that's it.
If it's a guard, you will add a line like this:
```
'rsi': hp.choice('rsi', [
{'enabled': False},
{'enabled': True, 'value': hp.quniform('rsi-value', 20, 40, 1)}
]),
```
This says, "*one of the guards is RSI, it can have two values, enabled or
disabled. If it is enabled, try different values for it between 20 and 40*".
So, the part of the strategy builder using the above setting looks like
this:
```
if params['rsi']['enabled']:
conditions.append(dataframe['rsi'] < params['rsi']['value'])
```
It checks if Hyperopt wants the RSI guard to be enabled for this
round `params['rsi']['enabled']` and if it is, then it will add a
condition that says RSI must be smaller than the value hyperopt picked
for this evaluation, which is given in the `params['rsi']['value']`.
That's it. Now you can add new parts of strategies to Hyperopt and it
will try all the combinations with all different values in the search
for best working algo.
### Add a new Indicators
If you want to test an indicator that isn't used by the bot currently, If you want to test an indicator that isn't used by the bot currently,
you need to add it to the `populate_indicators()` method in `hyperopt.py`. you need to add it to the `populate_indicators()` method in `hyperopt.py`.
@ -164,12 +120,12 @@ python3 ./freqtrade/main.py -c config.json hyperopt -e 5000
The `-e` flag will set how many evaluations hyperopt will do. We recommend The `-e` flag will set how many evaluations hyperopt will do. We recommend
running at least several thousand evaluations. running at least several thousand evaluations.
### Execute hyperopt with different ticker-data source ### Execute Hyperopt with Different Ticker-Data Source
If you would like to hyperopt parameters using an alternate ticker data that If you would like to hyperopt parameters using an alternate ticker data that
you have on-disk, use the `--datadir PATH` option. Default hyperopt will you have on-disk, use the `--datadir PATH` option. Default hyperopt will
use data from directory `user_data/data`. use data from directory `user_data/data`.
### Running hyperopt with smaller testset ### Running Hyperopt with Smaller Testset
Use the `--timeperiod` argument to change how much of the testset Use the `--timeperiod` argument to change how much of the testset
you want to use. The last N ticks/timeframes will be used. you want to use. The last N ticks/timeframes will be used.
Example: Example:
@ -178,7 +134,7 @@ Example:
python3 ./freqtrade/main.py hyperopt --timeperiod -200 python3 ./freqtrade/main.py hyperopt --timeperiod -200
``` ```
### Running hyperopt with smaller search space ### Running Hyperopt with Smaller Search Space
Use the `--spaces` argument to limit the search space used by hyperopt. Use the `--spaces` argument to limit the search space used by hyperopt.
Letting Hyperopt optimize everything is a huuuuge search space. Often it Letting Hyperopt optimize everything is a huuuuge search space. Often it
might make more sense to start by just searching for initial buy algorithm. might make more sense to start by just searching for initial buy algorithm.
@ -193,87 +149,44 @@ Legal values are:
- `stoploss`: search for the best stoploss value - `stoploss`: search for the best stoploss value
- space-separated list of any of the above values for example `--spaces roi stoploss` - space-separated list of any of the above values for example `--spaces roi stoploss`
## Understand the hyperopts result ## Understand the Hyperopts Result
Once Hyperopt is completed you can use the result to adding new buy Once Hyperopt is completed you can use the result to creating a new strategy.
signal. Given following result from hyperopt: Given following result from hyperopt:
```
Best parameters:
{
"adx": {
"enabled": true,
"value": 15.0
},
"fastd": {
"enabled": true,
"value": 40.0
},
"green_candle": {
"enabled": true
},
"mfi": {
"enabled": false
},
"over_sar": {
"enabled": false
},
"rsi": {
"enabled": true,
"value": 37.0
},
"trigger": {
"type": "lower_bb"
},
"uptrend_long_ema": {
"enabled": true
},
"uptrend_short_ema": {
"enabled": false
},
"uptrend_sma": {
"enabled": false
}
}
Best Result: ```
2197 trades. Avg profit 1.84%. Total profit 0.79367541 BTC. Avg duration 241.0 mins. Best result:
135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins.
with values:
{'adx-value': 44, 'rsi-value': 29, 'adx-enabled': False, 'rsi-enabled': True, 'trigger': 'bb_lower'}
``` ```
You should understand this result like: You should understand this result like:
- You should **consider** the guard "adx" (`"adx"` is `"enabled": true`) - The buy trigger that worked best was `bb_lower`.
and the best value is `15.0` (`"value": 15.0,`) - You should not use ADX because `adx-enabled: False`)
- You should **consider** the guard "fastd" (`"fastd"` is `"enabled": - You should **consider** using the RSI indicator (`rsi-enabled: True` and the best value is `29.0` (`rsi-value: 29.0`)
true`) and the best value is `40.0` (`"value": 40.0,`)
- You should **consider** to enable the guard "green_candle"
(`"green_candle"` is `"enabled": true`) but this guards as no
customizable value.
- You should **ignore** the guard "mfi" (`"mfi"` is `"enabled": false`)
- and so on...
You have to look inside your strategy file into `buy_strategy_generator()` You have to look inside your strategy file into `buy_strategy_generator()`
method, what those values match to. method, what those values match to.
So for example you had `adx:` with the `value: 15.0` so we would look So for example you had `rsi-value: 29.0` so we would look
at `adx`-block, that translates to the following code block: at `rsi`-block, that translates to the following code block:
``` ```
(dataframe['adx'] > 15.0) (dataframe['rsi'] < 29.0)
``` ```
Translating your whole hyperopt result to as the new buy-signal Translating your whole hyperopt result as the new buy-signal
would be the following: would then look like:
``` ```
def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame: def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
dataframe.loc[ dataframe.loc[
( (
(dataframe['adx'] > 15.0) & # adx-value (dataframe['rsi'] < 29.0) & # rsi-value
(dataframe['fastd'] < 40.0) & # fastd-value dataframe['close'] < dataframe['bb_lowerband'] # trigger
(dataframe['close'] > dataframe['open']) & # green_candle
(dataframe['rsi'] < 37.0) & # rsi-value
(dataframe['ema50'] > dataframe['ema100']) # uptrend_long_ema
), ),
'buy'] = 1 'buy'] = 1
return dataframe return dataframe
``` ```
## Next step ## Next Step
Now you have a perfect bot and want to control it from Telegram. Your Now you have a perfect bot and want to control it from Telegram. Your
next step is to learn the [Telegram usage](https://github.com/freqtrade/freqtrade/blob/develop/docs/telegram-usage.md). next step is to learn the [Telegram usage](https://github.com/freqtrade/freqtrade/blob/develop/docs/telegram-usage.md).