adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #4596 from rokups/rk/hyper-strategy

Browse Source 
Support for creating auto-hyperoptable strategies.
This commit is contained in:
Matthias 2021-04-05 13:55:32 +02:00 committed by GitHub
commit b470672dfd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 1140 additions and 312 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

363
docs/advanced-hyperopt.md
View File

@ -4,79 +4,6 @@ This page explains some advanced Hyperopt topics that may require higher
coding skills and Python knowledge than creation of an ordinal hyperoptimization coding skills and Python knowledge than creation of an ordinal hyperoptimization
class. class.
## Derived hyperopt classes
Custom hyperopt classes can be derived in the same way [it can be done for strategies](strategy-customization.md#derived-strategies).
Applying to hyperoptimization, as an example, you may override how dimensions are defined in your optimization hyperspace:
```python
class MyAwesomeHyperOpt(IHyperOpt):
...
# Uses default stoploss dimension
class MyAwesomeHyperOpt2(MyAwesomeHyperOpt):
@staticmethod
def stoploss_space() -> List[Dimension]:
# Override boundaries for stoploss
return [
Real(-0.33, -0.01, name='stoploss'),
]
```
and then quickly switch between hyperopt classes, running optimization process with hyperopt class you need in each particular case:
```
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt --hyperopt-loss SharpeHyperOptLossDaily --strategy MyAwesomeStrategy ...
or
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt2 --hyperopt-loss SharpeHyperOptLossDaily --strategy MyAwesomeStrategy ...
```
## Sharing methods with your strategy
Hyperopt classes provide access to the Strategy via the `strategy` class attribute.
This can be a great way to reduce code duplication if used correctly, but will also complicate usage for inexperienced users.
``` python
from pandas import DataFrame
from freqtrade.strategy.interface import IStrategy
import freqtrade.vendor.qtpylib.indicators as qtpylib
class MyAwesomeStrategy(IStrategy):
buy_params = {
'rsi-value': 30,
'adx-value': 35,
}
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
return self.buy_strategy_generator(self.buy_params, dataframe, metadata)
@staticmethod
def buy_strategy_generator(params, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
qtpylib.crossed_above(dataframe['rsi'], params['rsi-value']) &
dataframe['adx'] > params['adx-value']) &
dataframe['volume'] > 0
)
, 'buy'] = 1
return dataframe
class MyAwesomeHyperOpt(IHyperOpt):
...
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by Hyperopt.
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
# Call strategy's buy strategy generator
return self.StrategyClass.buy_strategy_generator(params, dataframe, metadata)
return populate_buy_trend
```
## Creating and using a custom loss function ## Creating and using a custom loss function
To use a custom loss function class, make sure that the function `hyperopt_loss_function` is defined in your custom hyperopt loss class. To use a custom loss function class, make sure that the function `hyperopt_loss_function` is defined in your custom hyperopt loss class.
@ -142,3 +69,293 @@ This function needs to return a floating point number (`float`). Smaller numbers
!!! Note !!! Note
Please keep the arguments `*args` and `**kwargs` in the interface to allow us to extend this interface later. Please keep the arguments `*args` and `**kwargs` in the interface to allow us to extend this interface later.
## Overriding pre-defined spaces
To override a pre-defined space (`roi_space`, `generate_roi_table`, `stoploss_space`, `trailing_space`), define a nested class called Hyperopt and define the required spaces as follows:
```python
class MyAwesomeStrategy(IStrategy):
class HyperOpt:
# Define a custom stoploss space.
def stoploss_space(self):
return [Real(-0.05, -0.01, name='stoploss')]
```
---
## Legacy Hyperopt
This Section explains the configuration of an explicit Hyperopt file (separate to the strategy).
!!! Warning "Deprecated / legacy mode"
Since the 2021.4 release you no longer have to write a separate hyperopt class, but all strategies can be hyperopted.
Please read the [main hyperopt page](hyperopt.md) for more details.
### Prepare hyperopt file
Configuring an explicit hyperopt file is similar to writing your own strategy, and many tasks will be similar.
!!! Tip "About this page"
For this page, we will be using a fictional strategy called `AwesomeStrategy` - which will be optimized using the `AwesomeHyperopt` class.
#### Create a Custom Hyperopt File
The simplest way to get started is to use the following command, which will create a new hyperopt file from a template, which will be located under `user_data/hyperopts/AwesomeHyperopt.py`.
Let assume you want a hyperopt file `AwesomeHyperopt.py`:
``` bash
freqtrade new-hyperopt --hyperopt AwesomeHyperopt
```
#### Legacy Hyperopt checklist
Checklist on all tasks / possibilities in hyperopt
Depending on the space you want to optimize, only some of the below are required:
* fill `buy_strategy_generator` - for buy signal optimization
* fill `indicator_space` - for buy signal optimization
* fill `sell_strategy_generator` - for sell signal optimization
* fill `sell_indicator_space` - for sell signal optimization
!!! Note
`populate_indicators` needs to create all indicators any of thee spaces may use, otherwise hyperopt will not work.
Optional in hyperopt - can also be loaded from a strategy (recommended):
* `populate_indicators` - fallback to create indicators
* `populate_buy_trend` - fallback if not optimizing for buy space. should come from strategy
* `populate_sell_trend` - fallback if not optimizing for sell space. should come from strategy
!!! Note
You always have to provide a strategy to Hyperopt, even if your custom Hyperopt class contains all methods.
Assuming the optional methods are not in your hyperopt file, please use `--strategy AweSomeStrategy` which contains these methods so hyperopt can use these methods instead.
Rarely you may also need to override:
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
* `generate_roi_table` - for custom ROI optimization (if you need the ranges for the values in the ROI table that differ from default or the number of entries (steps) in the ROI table which differs from the default 4 steps)
* `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
* `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)
#### Defining a buy signal optimization
Let's say you are curious: should you use MACD crossings or lower Bollinger
Bands to trigger your buys. And you also wonder should you use RSI or ADX to
help with those buy decisions. If you decide to use RSI or ADX, which values
should I use for them? So let's use hyperparameter optimization to solve this
mystery.
We will start by defining a search space:
```python
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(20, 40, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal'], name='trigger')
]
```
Above definition says: I have five parameters I want you to randomly combine
to find the best combination. Two of them are integer values (`adx-value` and `rsi-value`) and I want you test in the range of values 20 to 40.
Then we have three category variables. First two are either `True` or `False`.
We use these to either enable or disable the ADX and RSI guards.
The last one we call `trigger` and use it to decide which buy trigger we want to use.
So let's write the buy strategy generator using these values:
```python
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by Hyperopt.
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = []
# GUARDS AND TRENDS
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if 'trigger' in params:
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
# Check that volume is not 0
conditions.append(dataframe['volume'] > 0)
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
return populate_buy_trend
```
Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations.
It will use the given historical data and make buys based on the buy signals generated with the above function.
Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured [loss function](#loss-functions)).
!!! Note
The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
When you want to test an indicator that isn't used by the bot currently, remember to
add it to the `populate_indicators()` method in your strategy or hyperopt file.
#### Sell optimization
Similar to the buy-signal above, sell-signals can also be optimized.
Place the corresponding settings into the following methods
* Inside `sell_indicator_space()` - the parameters hyperopt shall be optimizing.
* Within `sell_strategy_generator()` - populate the nested method `populate_sell_trend()` to apply the parameters.
The configuration and rules are the same than for buy signals.
To avoid naming collisions in the search-space, please prefix all sell-spaces with `sell-`.
### Execute Hyperopt
Once you have updated your hyperopt configuration you can run it.
Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result. More time usually results in better results.
We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
```bash
freqtrade hyperopt --config config.json --hyperopt <hyperoptname> --hyperopt-loss <hyperoptlossname> --strategy <strategyname> -e 500 --spaces all
```
Use `<hyperoptname>` as the name of the custom hyperopt used.
The `-e` option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs.
Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results.
The `--spaces all` option determines that all possible parameters should be optimized. Possibilities are listed below.
!!! Note
Hyperopt will store hyperopt results with the timestamp of the hyperopt start time.
Reading commands (`hyperopt-list`, `hyperopt-show`) can use `--hyperopt-filename <filename>` to read and display older hyperopt results.
You can find a list of filenames with `ls -l user_data/hyperopt_results/`.
#### Running Hyperopt using methods from a strategy
Hyperopt can reuse `populate_indicators`, `populate_buy_trend`, `populate_sell_trend` from your strategy, assuming these methods are **not** in your custom hyperopt file, and a strategy is provided.
```bash
freqtrade hyperopt --hyperopt AwesomeHyperopt --hyperopt-loss SharpeHyperOptLossDaily --strategy AwesomeStrategy
```
### Understand the Hyperopt Result
Once Hyperopt is completed you can use the result to create a new strategy.
Given the following result from hyperopt:
```
Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params:
{ 'adx-value': 44,
'rsi-value': 29,
'adx-enabled': False,
'rsi-enabled': True,
'trigger': 'bb_lower'}
```
You should understand this result like:
* The buy trigger that worked best was `bb_lower`.
* You should not use ADX because `adx-enabled: False`)
* You should **consider** using the RSI indicator (`rsi-enabled: True` and the best value is `29.0` (`rsi-value: 29.0`)
You have to look inside your strategy file into `buy_strategy_generator()`
method, what those values match to.
So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
```python
(dataframe['rsi'] < 29.0)
```
Translating your whole hyperopt result as the new buy-signal would then look like:
```python
def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
dataframe.loc[
(
(dataframe['rsi'] < 29.0) & # rsi-value
dataframe['close'] < dataframe['bb_lowerband'] # trigger
),
'buy'] = 1
return dataframe
```
### Validate backtesting results
Once the optimized parameters and conditions have been implemented into your strategy, you should backtest the strategy to make sure everything is working as expected.
To achieve same results (number of trades, their durations, profit, etc.) than during Hyperopt, please use same configuration and parameters (timerange, timeframe, ...) used for hyperopt `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
Should results don't match, please double-check to make sure you transferred all conditions correctly.
Pay special care to the stoploss (and trailing stoploss) parameters, as these are often set in configuration files, which override changes to the strategy.
You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss` or `trailing_stop`).
### Sharing methods with your strategy
Hyperopt classes provide access to the Strategy via the `strategy` class attribute.
This can be a great way to reduce code duplication if used correctly, but will also complicate usage for inexperienced users.
``` python
from pandas import DataFrame
from freqtrade.strategy.interface import IStrategy
import freqtrade.vendor.qtpylib.indicators as qtpylib
class MyAwesomeStrategy(IStrategy):
buy_params = {
'rsi-value': 30,
'adx-value': 35,
}
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
return self.buy_strategy_generator(self.buy_params, dataframe, metadata)
@staticmethod
def buy_strategy_generator(params, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[
(
qtpylib.crossed_above(dataframe['rsi'], params['rsi-value']) &
dataframe['adx'] > params['adx-value']) &
dataframe['volume'] > 0
)
, 'buy'] = 1
return dataframe
class MyAwesomeHyperOpt(IHyperOpt):
...
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by Hyperopt.
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
# Call strategy's buy strategy generator
return self.StrategyClass.buy_strategy_generator(params, dataframe, metadata)
return populate_buy_trend
```

358
docs/hyperopt.md
View File

@ -1,19 +1,22 @@
# Hyperopt # Hyperopt
This page explains how to tune your strategy by finding the optimal This page explains how to tune your strategy by finding the optimal
parameters, a process called hyperparameter optimization. The bot uses several parameters, a process called hyperparameter optimization. The bot uses algorithms included in the `scikit-optimize` package to accomplish this.
algorithms included in the `scikit-optimize` package to accomplish this. The The search will burn all your CPU cores, make your laptop sound like a fighter jet and still take a long time.
search will burn all your CPU cores, make your laptop sound like a fighter jet
and still take a long time.
In general, the search for best parameters starts with a few random combinations (see [below](#reproducible-results) for more details) and then uses Bayesian search with a ML regressor algorithm (currently ExtraTreesRegressor) to quickly find a combination of parameters in the search hyperspace that minimizes the value of the [loss function](#loss-functions). In general, the search for best parameters starts with a few random combinations (see [below](#reproducible-results) for more details) and then uses Bayesian search with a ML regressor algorithm (currently ExtraTreesRegressor) to quickly find a combination of parameters in the search hyperspace that minimizes the value of the [loss function](#loss-functions).
Hyperopt requires historic data to be available, just as backtesting does. Hyperopt requires historic data to be available, just as backtesting does (hyperopt runs backtesting many times with different parameters).
To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation. To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation.
!!! Bug !!! Bug
Hyperopt can crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133) Hyperopt can crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133)
!!! Note
Since 2021.4 release you no longer have to write a separate hyperopt class, but can configure the parameters directly in the strategy.
The legacy method is still supported, but it is no longer the recommended way of setting up hyperopt.
The legacy documentation is available at [Legacy Hyperopt](advanced-hyperopt.md#legacy-hyperopt).
## Install hyperopt dependencies ## Install hyperopt dependencies
Since Hyperopt dependencies are not needed to run the bot itself, are heavy, can not be easily built on some platforms (like Raspberry PI), they are not installed by default. Before you run Hyperopt, you need to install the corresponding dependencies, as described in this section below. Since Hyperopt dependencies are not needed to run the bot itself, are heavy, can not be easily built on some platforms (like Raspberry PI), they are not installed by default. Before you run Hyperopt, you need to install the corresponding dependencies, as described in this section below.
@ -34,7 +37,6 @@ pip install -r requirements-hyperopt.txt
## Hyperopt command reference ## Hyperopt command reference
``` ```
usage: freqtrade hyperopt [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH] usage: freqtrade hyperopt [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH]
[--userdir PATH] [-s NAME] [--strategy-path PATH] [--userdir PATH] [-s NAME] [--strategy-path PATH]
@ -136,47 +138,19 @@ Strategy arguments:
``` ```
## Prepare Hyperopting
Before we start digging into Hyperopt, we recommend you to take a look at
the sample hyperopt file located in [user_data/hyperopts/](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt.py).
Configuring hyperopt is similar to writing your own strategy, and many tasks will be similar.
!!! Tip "About this page"
For this page, we will be using a fictional strategy called `AwesomeStrategy` - which will be optimized using the `AwesomeHyperopt` class.
The simplest way to get started is to use the following, command, which will create a new hyperopt file from a template, which will be located under `user_data/hyperopts/AwesomeHyperopt.py`.
``` bash
freqtrade new-hyperopt --hyperopt AwesomeHyperopt
```
### Hyperopt checklist ### Hyperopt checklist
Checklist on all tasks / possibilities in hyperopt Checklist on all tasks / possibilities in hyperopt
Depending on the space you want to optimize, only some of the below are required: Depending on the space you want to optimize, only some of the below are required:
* fill `buy_strategy_generator` - for buy signal optimization * define parameters with `space='buy'` - for buy signal optimization
* fill `indicator_space` - for buy signal optimization * define parameters with `space='sell'` - for sell signal optimization
* fill `sell_strategy_generator` - for sell signal optimization
* fill `sell_indicator_space` - for sell signal optimization
!!! Note !!! Note
`populate_indicators` needs to create all indicators any of the spaces may use, otherwise hyperopt will not work. `populate_indicators` needs to create all indicators any of the spaces may use, otherwise hyperopt will not work.
Optional in hyperopt - can also be loaded from a strategy (recommended): Rarely you may also need to create a [nested class](advanced-hyperopt.md#overriding-pre-defined-spaces) named `HyperOpt` and implement
* `populate_indicators` - fallback to create indicators
* `populate_buy_trend` - fallback if not optimizing for buy space. should come from strategy
* `populate_sell_trend` - fallback if not optimizing for sell space. should come from strategy
!!! Note
You always have to provide a strategy to Hyperopt, even if your custom Hyperopt class contains all methods.
Assuming the optional methods are not in your hyperopt file, please use `--strategy AweSomeStrategy` which contains these methods so hyperopt can use these methods instead.
Rarely you may also need to override:
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default) * `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
* `generate_roi_table` - for custom ROI optimization (if you need the ranges for the values in the ROI table that differ from default or the number of entries (steps) in the ROI table which differs from the default 4 steps) * `generate_roi_table` - for custom ROI optimization (if you need the ranges for the values in the ROI table that differ from default or the number of entries (steps) in the ROI table which differs from the default 4 steps)
@ -184,31 +158,19 @@ Rarely you may also need to override:
* `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default) * `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)
!!! Tip "Quickly optimize ROI, stoploss and trailing stoploss" !!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything (i.e. without creation of a "complete" Hyperopt class with dimensions, parameters, triggers and guards, as described in this document) from the default hyperopt template by relying on your strategy to do most of the calculations. You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything in your strategy.
```python ```python
# Have a working strategy at hand. # Have a working strategy at hand.
freqtrade new-hyperopt --hyperopt EmptyHyperopt freqtrade hyperopt --hyperopt-loss SharpeHyperOptLossDaily --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100
freqtrade hyperopt --hyperopt EmptyHyperopt --hyperopt-loss SharpeHyperOptLossDaily --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100
``` ```
### Create a Custom Hyperopt File
Let assume you want a hyperopt file `AwesomeHyperopt.py`:
``` bash
freqtrade new-hyperopt --hyperopt AwesomeHyperopt
```
This command will create a new hyperopt file from a template, allowing you to get started quickly.
### Configure your Guards and Triggers ### Configure your Guards and Triggers
There are two places you need to change in your hyperopt file to add a new buy hyperopt for testing: There are two places you need to change in your strategy file to add a new buy hyperopt for testing:
* Inside `indicator_space()` - the parameters hyperopt shall be optimizing. * Define the parameters at the class level hyperopt shall be optimizing.
* Within `buy_strategy_generator()` - populate the nested `populate_buy_trend()` to apply the parameters. * Within `populate_buy_trend()` - use defined parameter values instead of raw constants.
There you have two different types of indicators: 1. `guards` and 2. `triggers`. There you have two different types of indicators: 1. `guards` and 2. `triggers`.
@ -224,24 +186,46 @@ Hyper-optimization will, for each epoch round, pick one trigger and possibly
multiple guards. The constructed strategy will be something like "*buy exactly when close price touches lower Bollinger band, BUT only if multiple guards. The constructed strategy will be something like "*buy exactly when close price touches lower Bollinger band, BUT only if
ADX > 10*". ADX > 10*".
If you have updated the buy strategy, i.e. changed the contents of `populate_buy_trend()` method, you have to update the `guards` and `triggers` your hyperopt must use correspondingly. ```python
from freqtrade.strategy import IntParameter, IStrategy
class MyAwesomeStrategy(IStrategy):
# If parameter is prefixed with `buy_` or `sell_` then specifying `space` parameter is optional
# and space is inferred from parameter name.
buy_adx_min = IntParameter(0, 100, default=10)
def populate_buy_trend(self, dataframe: 'DataFrame', metadata: dict) -> 'DataFrame':
dataframe.loc[
(
(dataframe['adx'] > self.buy_adx_min.value)
), 'buy'] = 1
return dataframe
```
#### Sell optimization #### Sell optimization
Similar to the buy-signal above, sell-signals can also be optimized. Similar to the buy-signal above, sell-signals can also be optimized.
Place the corresponding settings into the following methods Place the corresponding settings into the following methods
* Inside `sell_indicator_space()` - the parameters hyperopt shall be optimizing. * Define the parameters at the class level hyperopt shall be optimizing.
* Within `sell_strategy_generator()` - populate the nested method `populate_sell_trend()` to apply the parameters. * Within `populate_sell_trend()` - use defined parameter values instead of raw constants.
The configuration and rules are the same than for buy signals. The configuration and rules are the same than for buy signals.
To avoid naming collisions in the search-space, please prefix all sell-spaces with `sell-`.
#### Using timeframe as a part of the Strategy ```python
class MyAwesomeStrategy(IStrategy):
# There is no strict parameter naming scheme. If you do not use `buy_` or `sell_` prefixes -
# please specify to which space parameter belongs using `space` parameter. Possible values:
# 'buy' or 'sell'.
adx_max = IntParameter(0, 100, default=50, space='sell')
The Strategy class exposes the timeframe value as the `self.timeframe` attribute. def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
The same value is available as class-attribute `HyperoptName.timeframe`. dataframe.loc[
In the case of the linked sample-value this would be `AwesomeHyperopt.timeframe`. (
(dataframe['adx'] < self.adx_max.value)
), 'buy'] = 1
return dataframe
```
## Solving a Mystery ## Solving a Mystery
@ -251,50 +235,38 @@ help with those buy decisions. If you decide to use RSI or ADX, which values
should I use for them? So let's use hyperparameter optimization to solve this should I use for them? So let's use hyperparameter optimization to solve this
mystery. mystery.
We will start by defining a search space: We will start by defining hyperoptable parameters:
```python ```python
def indicator_space() -> List[Dimension]: class MyAwesomeStrategy(IStrategy):
""" buy_adx = IntParameter(20, 40, default=30)
Define your Hyperopt space for searching strategy parameters buy_rsi = IntParameter(20, 40, default=30)
""" buy_adx_enabled = CategoricalParameter([True, False]),
return [ buy_rsi_enabled = CategoricalParameter([True, False]),
Integer(20, 40, name='adx-value'), buy_trigger = CategoricalParameter(['bb_lower', 'macd_cross_signal']),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal'], name='trigger')
]
``` ```
Above definition says: I have five parameters I want you to randomly combine Above definition says: I have five parameters I want to randomly combine to find the best combination.
to find the best combination. Two of them are integer values (`adx-value` Two of them are integer values (`buy_adx` and `buy_rsi`) and I want you test in the range of values 20 to 40.
and `rsi-value`) and I want you test in the range of values 20 to 40.
Then we have three category variables. First two are either `True` or `False`. Then we have three category variables. First two are either `True` or `False`.
We use these to either enable or disable the ADX and RSI guards. The last We use these to either enable or disable the ADX and RSI guards.
one we call `trigger` and use it to decide which buy trigger we want to use. The last one we call `trigger` and use it to decide which buy trigger we want to use.
So let's write the buy strategy using these values: So let's write the buy strategy using these values:
```python ```python
@staticmethod def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by Hyperopt.
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = [] conditions = []
# GUARDS AND TRENDS # GUARDS AND TRENDS
if 'adx-enabled' in params and params['adx-enabled']: if self.buy_adx_enabled.value:
conditions.append(dataframe['adx'] > params['adx-value']) conditions.append(dataframe['adx'] > self.buy_adx.value)
if 'rsi-enabled' in params and params['rsi-enabled']: if self.buy_rsi_enabled.value:
conditions.append(dataframe['rsi'] < params['rsi-value']) conditions.append(dataframe['rsi'] < self.buy_rsi.value)
# TRIGGERS # TRIGGERS
if 'trigger' in params: if self.buy_trigger.value == 'bb_lower':
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband']) conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal': if self.buy_trigger.value == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above( conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal'] dataframe['macd'], dataframe['macdsignal']
)) ))
@ -308,8 +280,6 @@ So let's write the buy strategy using these values:
'buy'] = 1 'buy'] = 1
return dataframe return dataframe
return populate_buy_trend
``` ```
Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations. Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations.
@ -321,6 +291,20 @@ Based on the results, hyperopt will tell you which parameter combination produce
When you want to test an indicator that isn't used by the bot currently, remember to When you want to test an indicator that isn't used by the bot currently, remember to
add it to the `populate_indicators()` method in your strategy or hyperopt file. add it to the `populate_indicators()` method in your strategy or hyperopt file.
## Parameter types
There are four parameter types each suited for different purposes.
* `IntParameter` - defines an integral parameter with upper and lower boundaries of search space.
* `DecimalParameter` - defines a floating point parameter with a limited number of decimals (default 3). Should be preferred instead of `RealParameter` in most cases.
* `RealParameter` - defines a floating point parameter with upper and lower boundaries and no precision limit. Rarely used as it creates a space with a near infinite number of possibilities.
* `CategoricalParameter` - defines a parameter with a predetermined number of choices.
!!! Tip "Disabling parameter optimization"
Each parameter takes two boolean parameters:
* `load` - when set to `False` it will not load values configured in `buy_params` and `sell_params`.
* `optimize` - when set to `False` parameter will not be included in optimization process.
Use these parameters to quickly prototype various ideas.
## Loss-functions ## Loss-functions
Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results. Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results.
@ -342,16 +326,14 @@ Creation of a custom loss function is covered in the [Advanced Hyperopt](advance
## Execute Hyperopt ## Execute Hyperopt
Once you have updated your hyperopt configuration you can run it. Once you have updated your hyperopt configuration you can run it.
Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result. More time usually results in better results. Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result.
We strongly recommend to use `screen` or `tmux` to prevent any connection loss. We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
```bash ```bash
freqtrade hyperopt --config config.json --hyperopt <hyperoptname> --hyperopt-loss <hyperoptlossname> --strategy <strategyname> -e 500 --spaces all freqtrade hyperopt --config config.json --hyperopt-loss <hyperoptlossname> --strategy <strategyname> -e 500 --spaces all
``` ```
Use `<hyperoptname>` as the name of the custom hyperopt used.
The `-e` option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs. The `-e` option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs.
Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results. Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results.
@ -365,24 +347,17 @@ The `--spaces all` option determines that all possible parameters should be opti
### Execute Hyperopt with different historical data source ### Execute Hyperopt with different historical data source
If you would like to hyperopt parameters using an alternate historical data set that If you would like to hyperopt parameters using an alternate historical data set that
you have on-disk, use the `--datadir PATH` option. By default, hyperopt you have on-disk, use the `--datadir PATH` option. By default, hyperopt uses data from directory `user_data/data`.
uses data from directory `user_data/data`.
### Running Hyperopt with a smaller test-set ### Running Hyperopt with a smaller test-set
Use the `--timerange` argument to change how much of the test-set you want to use. Use the `--timerange` argument to change how much of the test-set you want to use.
For example, to use one month of data, pass the following parameter to the hyperopt call: For example, to use one month of data, pass `--timerange 20210101-20210201` (from january 2021 - february 2021) to the hyperopt call.
Full command:
```bash ```bash
freqtrade hyperopt --hyperopt <hyperoptname> --strategy <strategyname> --timerange 20180401-20180501 freqtrade hyperopt --hyperopt <hyperoptname> --strategy <strategyname> --timerange 20210101-20210201
```
### Running Hyperopt using methods from a strategy
Hyperopt can reuse `populate_indicators`, `populate_buy_trend`, `populate_sell_trend` from your strategy, assuming these methods are **not** in your custom hyperopt file, and a strategy is provided.
```bash
freqtrade hyperopt --hyperopt AwesomeHyperopt --hyperopt-loss SharpeHyperOptLossDaily --strategy AwesomeStrategy
``` ```
### Running Hyperopt with Smaller Search Space ### Running Hyperopt with Smaller Search Space
@ -405,40 +380,9 @@ Legal values are:
The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy. The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy.
### Position stacking and disabling max market positions
In some situations, you may need to run Hyperopt (and Backtesting) with the
`--eps`/`--enable-position-staking` and `--dmmp`/`--disable-max-market-positions` arguments.
By default, hyperopt emulates the behavior of the Freqtrade Live Run/Dry Run, where only one
open trade is allowed for every traded pair. The total number of trades open for all pairs
is also limited by the `max_open_trades` setting. During Hyperopt/Backtesting this may lead to
some potential trades to be hidden (or masked) by previously open trades.
The `--eps`/`--enable-position-stacking` argument allows emulation of buying the same pair multiple times,
while `--dmmp`/`--disable-max-market-positions` disables applying `max_open_trades`
during Hyperopt/Backtesting (which is equal to setting `max_open_trades` to a very high
number).
!!! Note
Dry/live runs will **NOT** use position stacking - therefore it does make sense to also validate the strategy without this as it's closer to reality.
You can also enable position stacking in the configuration file by explicitly setting
`"position_stacking"=true`.
### Reproducible results
The search for optimal parameters starts with a few (currently 30) random combinations in the hyperspace of parameters, random Hyperopt epochs. These random epochs are marked with an asterisk character (`*`) in the first column in the Hyperopt output.
The initial state for generation of these random values (random state) is controlled by the value of the `--random-state` command line option. You can set it to some arbitrary value of your choice to obtain reproducible results.
If you have not set this value explicitly in the command line options, Hyperopt seeds the random state with some random value for you. The random state value for each Hyperopt run is shown in the log, so you can copy and paste it into the `--random-state` command line option to repeat the set of the initial random epochs used.
If you have not changed anything in the command line options, configuration, timerange, Strategy and Hyperopt classes, historical data and the Loss Function -- you should obtain same hyper-optimization results with same random state value used.
## Understand the Hyperopt Result ## Understand the Hyperopt Result
Once Hyperopt is completed you can use the result to create a new strategy. Once Hyperopt is completed you can use the result to update your strategy.
Given the following result from hyperopt: Given the following result from hyperopt:
``` ```
@ -446,49 +390,38 @@ Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367 44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params: # Buy hyperspace params:
{ 'adx-value': 44, buy_params = {
'rsi-value': 29, 'buy_adx': 44,
'adx-enabled': False, 'buy_rsi': 29,
'rsi-enabled': True, 'buy_adx_enabled': False,
'trigger': 'bb_lower'} 'buy_rsi_enabled': True,
'buy_trigger': 'bb_lower'
}
``` ```
You should understand this result like: You should understand this result like:
- The buy trigger that worked best was `bb_lower`. * The buy trigger that worked best was `bb_lower`.
- You should not use ADX because `adx-enabled: False`) * You should not use ADX because `'buy_adx_enabled': False`.
- You should **consider** using the RSI indicator (`rsi-enabled: True` and the best value is `29.0` (`rsi-value: 29.0`) * You should **consider** using the RSI indicator (`'buy_rsi_enabled': True`) and the best value is `29.0` (`'buy_rsi': 29.0`)
You have to look inside your strategy file into `buy_strategy_generator()` Your strategy class can immediately take advantage of these results. Simply copy hyperopt results block and paste them at class level, replacing old parameters (if any). New parameters will automatically be loaded next time strategy is executed.
method, what those values match to.
So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block: Transferring your whole hyperopt result to your strategy would then look like:
```python ```python
(dataframe['rsi'] < 29.0) class MyAwesomeStrategy(IStrategy):
# Buy hyperspace params:
buy_params = {
'buy_adx': 44,
'buy_rsi': 29,
'buy_adx_enabled': False,
'buy_rsi_enabled': True,
'buy_trigger': 'bb_lower'
}
``` ```
Translating your whole hyperopt result as the new buy-signal would then look like:
```python
def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
dataframe.loc[
(
(dataframe['rsi'] < 29.0) & # rsi-value
dataframe['close'] < dataframe['bb_lowerband'] # trigger
),
'buy'] = 1
return dataframe
```
By default, hyperopt prints colorized results -- epochs with positive profit are printed in the green color. This highlighting helps you find epochs that can be interesting for later analysis. Epochs with zero total profit or with negative profits (losses) are printed in the normal color. If you do not need colorization of results (for instance, when you are redirecting hyperopt output to a file) you can switch colorization off by specifying the `--no-color` option in the command line.
You can use the `--print-all` command line option if you would like to see all results in the hyperopt output, not only the best ones. When `--print-all` is used, current best results are also colorized by default -- they are printed in bold (bright) style. This can also be switched off with the `--no-color` command line option.
!!! Note "Windows and color output"
Windows does not support color-output natively, therefore it is automatically disabled. To have color-output for hyperopt running under windows, please consider using WSL.
### Understand Hyperopt ROI results ### Understand Hyperopt ROI results
If you are optimizing ROI (i.e. if optimization search-space contains 'all', 'default' or 'roi'), your result will look as follows and include a ROI table: If you are optimizing ROI (i.e. if optimization search-space contains 'all', 'default' or 'roi'), your result will look as follows and include a ROI table:
@ -498,11 +431,13 @@ Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367 44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
ROI table: # ROI table:
{ 0: 0.10674, minimal_roi = {
0: 0.10674,
21: 0.09158, 21: 0.09158,
78: 0.03634, 78: 0.03634,
118: 0} 118: 0
}
``` ```
In order to use this best ROI table found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `minimal_roi` attribute of your custom strategy: In order to use this best ROI table found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `minimal_roi` attribute of your custom strategy:
@ -548,13 +483,16 @@ Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367 44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params: # Buy hyperspace params:
{ 'adx-value': 44, buy_params = {
'rsi-value': 29, 'buy_adx': 44,
'adx-enabled': False, 'buy_rsi': 29,
'rsi-enabled': True, 'buy_adx_enabled': False,
'trigger': 'bb_lower'} 'buy_rsi_enabled': True,
Stoploss: -0.27996 'buy_trigger': 'bb_lower'
}
stoploss: -0.27996
``` ```
In order to use this best stoploss value found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `stoploss` attribute of your custom strategy: In order to use this best stoploss value found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `stoploss` attribute of your custom strategy:
@ -584,11 +522,11 @@ Best result:
45/100: 606 trades. Avg profit 1.04%. Total profit 0.31555614 BTC ( 630.48Σ%). Avg duration 150.3 mins. Objective: -1.10161 45/100: 606 trades. Avg profit 1.04%. Total profit 0.31555614 BTC ( 630.48Σ%). Avg duration 150.3 mins. Objective: -1.10161
Trailing stop: # Trailing stop:
{ 'trailing_only_offset_is_reached': True, trailing_stop = True
'trailing_stop': True, trailing_stop_positive = 0.02001
'trailing_stop_positive': 0.02001, trailing_stop_positive_offset = 0.06038
'trailing_stop_positive_offset': 0.06038} trailing_only_offset_is_reached = True
``` ```
In order to use these best trailing stop parameters found by Hyperopt in backtesting and for live trades/dry-run, copy-paste them as the values of the corresponding attributes of your custom strategy: In order to use these best trailing stop parameters found by Hyperopt in backtesting and for live trades/dry-run, copy-paste them as the values of the corresponding attributes of your custom strategy:
@ -610,6 +548,46 @@ If you are optimizing trailing stop values, Freqtrade creates the 'trailing' opt
Override the `trailing_space()` method and define the desired range in it if you need values of the trailing stop parameters to vary in other ranges during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt_advanced.py). Override the `trailing_space()` method and define the desired range in it if you need values of the trailing stop parameters to vary in other ranges during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt_advanced.py).
### Reproducible results
The search for optimal parameters starts with a few (currently 30) random combinations in the hyperspace of parameters, random Hyperopt epochs. These random epochs are marked with an asterisk character (`*`) in the first column in the Hyperopt output.
The initial state for generation of these random values (random state) is controlled by the value of the `--random-state` command line option. You can set it to some arbitrary value of your choice to obtain reproducible results.
If you have not set this value explicitly in the command line options, Hyperopt seeds the random state with some random value for you. The random state value for each Hyperopt run is shown in the log, so you can copy and paste it into the `--random-state` command line option to repeat the set of the initial random epochs used.
If you have not changed anything in the command line options, configuration, timerange, Strategy and Hyperopt classes, historical data and the Loss Function -- you should obtain same hyper-optimization results with same random state value used.
## Output formatting
By default, hyperopt prints colorized results -- epochs with positive profit are printed in the green color. This highlighting helps you find epochs that can be interesting for later analysis. Epochs with zero total profit or with negative profits (losses) are printed in the normal color. If you do not need colorization of results (for instance, when you are redirecting hyperopt output to a file) you can switch colorization off by specifying the `--no-color` option in the command line.
You can use the `--print-all` command line option if you would like to see all results in the hyperopt output, not only the best ones. When `--print-all` is used, current best results are also colorized by default -- they are printed in bold (bright) style. This can also be switched off with the `--no-color` command line option.
!!! Note "Windows and color output"
Windows does not support color-output natively, therefore it is automatically disabled. To have color-output for hyperopt running under windows, please consider using WSL.
## Position stacking and disabling max market positions
In some situations, you may need to run Hyperopt (and Backtesting) with the
`--eps`/`--enable-position-staking` and `--dmmp`/`--disable-max-market-positions` arguments.
By default, hyperopt emulates the behavior of the Freqtrade Live Run/Dry Run, where only one
open trade is allowed for every traded pair. The total number of trades open for all pairs
is also limited by the `max_open_trades` setting. During Hyperopt/Backtesting this may lead to
some potential trades to be hidden (or masked) by previously open trades.
The `--eps`/`--enable-position-stacking` argument allows emulation of buying the same pair multiple times,
while `--dmmp`/`--disable-max-market-positions` disables applying `max_open_trades`
during Hyperopt/Backtesting (which is equal to setting `max_open_trades` to a very high
number).
!!! Note
Dry/live runs will **NOT** use position stacking - therefore it does make sense to also validate the strategy without this as it's closer to reality.
You can also enable position stacking in the configuration file by explicitly setting
`"position_stacking"=true`.
## Show details of Hyperopt results ## Show details of Hyperopt results
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter. After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter.

1
freqtrade/commands/cli_options.py
View File

@ -195,6 +195,7 @@ AVAILABLE_CLI_OPTIONS = {
'--hyperopt', '--hyperopt',
help='Specify hyperopt class name which will be used by the bot.', help='Specify hyperopt class name which will be used by the bot.',
metavar='NAME', metavar='NAME',
required=False,
), ),
"hyperopt_path": Arg( "hyperopt_path": Arg(
'--hyperopt-path', '--hyperopt-path',

7
freqtrade/optimize/hyperopt.py
View File

@ -26,6 +26,7 @@ from freqtrade.data.history import get_timerange
from freqtrade.misc import file_dump_json, plural from freqtrade.misc import file_dump_json, plural
from freqtrade.optimize.backtesting import Backtesting from freqtrade.optimize.backtesting import Backtesting
# Import IHyperOpt and IHyperOptLoss to allow unpickling classes from these modules # Import IHyperOpt and IHyperOptLoss to allow unpickling classes from these modules
from freqtrade.optimize.hyperopt_auto import HyperOptAuto
from freqtrade.optimize.hyperopt_interface import IHyperOpt # noqa: F401 from freqtrade.optimize.hyperopt_interface import IHyperOpt # noqa: F401
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F401 from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F401
from freqtrade.optimize.hyperopt_tools import HyperoptTools from freqtrade.optimize.hyperopt_tools import HyperoptTools
@ -61,14 +62,18 @@ class Hyperopt:
hyperopt = Hyperopt(config) hyperopt = Hyperopt(config)
hyperopt.start() hyperopt.start()
""" """
custom_hyperopt: IHyperOpt
def __init__(self, config: Dict[str, Any]) -> None: def __init__(self, config: Dict[str, Any]) -> None:
self.config = config self.config = config
self.backtesting = Backtesting(self.config) self.backtesting = Backtesting(self.config)
if not self.config.get('hyperopt'):
self.custom_hyperopt = HyperOptAuto(self.config)
else:
self.custom_hyperopt = HyperOptResolver.load_hyperopt(self.config) self.custom_hyperopt = HyperOptResolver.load_hyperopt(self.config)
self.custom_hyperopt.__class__.strategy = self.backtesting.strategy self.custom_hyperopt.strategy = self.backtesting.strategy
self.custom_hyperoptloss = HyperOptLossResolver.load_hyperoptloss(self.config) self.custom_hyperoptloss = HyperOptLossResolver.load_hyperoptloss(self.config)
self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function

89
freqtrade/optimize/hyperopt_auto.py Normal file
View File

@ -0,0 +1,89 @@
"""
HyperOptAuto class.
This module implements a convenience auto-hyperopt class, which can be used together with strategies
that implement IHyperStrategy interface.
"""
from contextlib import suppress
from typing import Any, Callable, Dict, List
from pandas import DataFrame
with suppress(ImportError):
from skopt.space import Dimension
from freqtrade.optimize.hyperopt_interface import IHyperOpt
class HyperOptAuto(IHyperOpt):
"""
This class delegates functionality to Strategy(IHyperStrategy) and Strategy.HyperOpt classes.
Most of the time Strategy.HyperOpt class would only implement indicator_space and
sell_indicator_space methods, but other hyperopt methods can be overridden as well.
"""
def buy_strategy_generator(self, params: Dict[str, Any]) -> Callable:
def populate_buy_trend(dataframe: DataFrame, metadata: dict):
for attr_name, attr in self.strategy.enumerate_parameters('buy'):
if attr.optimize:
# noinspection PyProtectedMember
attr._set_value(params[attr_name])
return self.strategy.populate_buy_trend(dataframe, metadata)
return populate_buy_trend
def sell_strategy_generator(self, params: Dict[str, Any]) -> Callable:
def populate_buy_trend(dataframe: DataFrame, metadata: dict):
for attr_name, attr in self.strategy.enumerate_parameters('sell'):
if attr.optimize:
# noinspection PyProtectedMember
attr._set_value(params[attr_name])
return self.strategy.populate_sell_trend(dataframe, metadata)
return populate_buy_trend
def _get_func(self, name) -> Callable:
"""
Return a function defined in Strategy.HyperOpt class, or one defined in super() class.
:param name: function name.
:return: a requested function.
"""
hyperopt_cls = getattr(self.strategy, 'HyperOpt', None)
default_func = getattr(super(), name)
if hyperopt_cls:
return getattr(hyperopt_cls, name, default_func)
else:
return default_func
def _generate_indicator_space(self, category):
for attr_name, attr in self.strategy.enumerate_parameters(category):
if attr.optimize:
yield attr.get_space(attr_name)
def _get_indicator_space(self, category, fallback_method_name):
indicator_space = list(self._generate_indicator_space(category))
if len(indicator_space) > 0:
return indicator_space
else:
return self._get_func(fallback_method_name)()
def indicator_space(self) -> List['Dimension']:
return self._get_indicator_space('buy', 'indicator_space')
def sell_indicator_space(self) -> List['Dimension']:
return self._get_indicator_space('sell', 'sell_indicator_space')
def generate_roi_table(self, params: Dict) -> Dict[int, float]:
return self._get_func('generate_roi_table')(params)
def roi_space(self) -> List['Dimension']:
return self._get_func('roi_space')()
def stoploss_space(self) -> List['Dimension']:
return self._get_func('stoploss_space')()
def generate_trailing_params(self, params: Dict) -> Dict:
return self._get_func('generate_trailing_params')(params)
def trailing_space(self) -> List['Dimension']:
return self._get_func('trailing_space')()

33
freqtrade/optimize/hyperopt_interface.py
View File

@ -44,36 +44,31 @@ class IHyperOpt(ABC):
IHyperOpt.ticker_interval = str(config['timeframe']) # DEPRECATED IHyperOpt.ticker_interval = str(config['timeframe']) # DEPRECATED
IHyperOpt.timeframe = str(config['timeframe']) IHyperOpt.timeframe = str(config['timeframe'])
@staticmethod def buy_strategy_generator(self, params: Dict[str, Any]) -> Callable:
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a buy strategy generator. Create a buy strategy generator.
""" """
raise OperationalException(_format_exception_message('buy_strategy_generator', 'buy')) raise OperationalException(_format_exception_message('buy_strategy_generator', 'buy'))
@staticmethod def sell_strategy_generator(self, params: Dict[str, Any]) -> Callable:
def sell_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a sell strategy generator. Create a sell strategy generator.
""" """
raise OperationalException(_format_exception_message('sell_strategy_generator', 'sell')) raise OperationalException(_format_exception_message('sell_strategy_generator', 'sell'))
@staticmethod def indicator_space(self) -> List[Dimension]:
def indicator_space() -> List[Dimension]:
""" """
Create an indicator space. Create an indicator space.
""" """
raise OperationalException(_format_exception_message('indicator_space', 'buy')) raise OperationalException(_format_exception_message('indicator_space', 'buy'))
@staticmethod def sell_indicator_space(self) -> List[Dimension]:
def sell_indicator_space() -> List[Dimension]:
""" """
Create a sell indicator space. Create a sell indicator space.
""" """
raise OperationalException(_format_exception_message('sell_indicator_space', 'sell')) raise OperationalException(_format_exception_message('sell_indicator_space', 'sell'))
@staticmethod def generate_roi_table(self, params: Dict) -> Dict[int, float]:
def generate_roi_table(params: Dict) -> Dict[int, float]:
""" """
Create a ROI table. Create a ROI table.
@ -88,8 +83,7 @@ class IHyperOpt(ABC):
return roi_table return roi_table
@staticmethod def roi_space(self) -> List[Dimension]:
def roi_space() -> List[Dimension]:
""" """
Create a ROI space. Create a ROI space.
@ -109,7 +103,7 @@ class IHyperOpt(ABC):
roi_t_alpha = 1.0 roi_t_alpha = 1.0
roi_p_alpha = 1.0 roi_p_alpha = 1.0
timeframe_min = timeframe_to_minutes(IHyperOpt.ticker_interval) timeframe_min = timeframe_to_minutes(self.timeframe)
# We define here limits for the ROI space parameters automagically adapted to the # We define here limits for the ROI space parameters automagically adapted to the
# timeframe used by the bot: # timeframe used by the bot:
@ -145,7 +139,7 @@ class IHyperOpt(ABC):
'roi_p2': roi_limits['roi_p2_min'], 'roi_p2': roi_limits['roi_p2_min'],
'roi_p3': roi_limits['roi_p3_min'], 'roi_p3': roi_limits['roi_p3_min'],
} }
logger.info(f"Min roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}") logger.info(f"Min roi table: {round_dict(self.generate_roi_table(p), 5)}")
p = { p = {
'roi_t1': roi_limits['roi_t1_max'], 'roi_t1': roi_limits['roi_t1_max'],
'roi_t2': roi_limits['roi_t2_max'], 'roi_t2': roi_limits['roi_t2_max'],
@ -154,7 +148,7 @@ class IHyperOpt(ABC):
'roi_p2': roi_limits['roi_p2_max'], 'roi_p2': roi_limits['roi_p2_max'],
'roi_p3': roi_limits['roi_p3_max'], 'roi_p3': roi_limits['roi_p3_max'],
} }
logger.info(f"Max roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}") logger.info(f"Max roi table: {round_dict(self.generate_roi_table(p), 5)}")
return [ return [
Integer(roi_limits['roi_t1_min'], roi_limits['roi_t1_max'], name='roi_t1'), Integer(roi_limits['roi_t1_min'], roi_limits['roi_t1_max'], name='roi_t1'),
@ -165,8 +159,7 @@ class IHyperOpt(ABC):
Real(roi_limits['roi_p3_min'], roi_limits['roi_p3_max'], name='roi_p3'), Real(roi_limits['roi_p3_min'], roi_limits['roi_p3_max'], name='roi_p3'),
] ]
@staticmethod def stoploss_space(self) -> List[Dimension]:
def stoploss_space() -> List[Dimension]:
""" """
Create a stoploss space. Create a stoploss space.
@ -177,8 +170,7 @@ class IHyperOpt(ABC):
Real(-0.35, -0.02, name='stoploss'), Real(-0.35, -0.02, name='stoploss'),
] ]
@staticmethod def generate_trailing_params(self, params: Dict) -> Dict:
def generate_trailing_params(params: Dict) -> Dict:
""" """
Create dict with trailing stop parameters. Create dict with trailing stop parameters.
""" """
@ -190,8 +182,7 @@ class IHyperOpt(ABC):
'trailing_only_offset_is_reached': params['trailing_only_offset_is_reached'], 'trailing_only_offset_is_reached': params['trailing_only_offset_is_reached'],
} }
@staticmethod def trailing_space(self) -> List[Dimension]:
def trailing_space() -> List[Dimension]:
""" """
Create a trailing stoploss space. Create a trailing stoploss space.

2
freqtrade/strategy/__init__.py
View File

@ -1,5 +1,7 @@
# flake8: noqa: F401 # flake8: noqa: F401
from freqtrade.exchange import (timeframe_to_minutes, timeframe_to_msecs, timeframe_to_next_date, from freqtrade.exchange import (timeframe_to_minutes, timeframe_to_msecs, timeframe_to_next_date,
timeframe_to_prev_date, timeframe_to_seconds) timeframe_to_prev_date, timeframe_to_seconds)
from freqtrade.strategy.hyper import (CategoricalParameter, DecimalParameter, IntParameter,
RealParameter)
from freqtrade.strategy.interface import IStrategy from freqtrade.strategy.interface import IStrategy
from freqtrade.strategy.strategy_helper import merge_informative_pair, stoploss_from_open from freqtrade.strategy.strategy_helper import merge_informative_pair, stoploss_from_open

269
freqtrade/strategy/hyper.py Normal file
View File

@ -0,0 +1,269 @@
"""
IHyperStrategy interface, hyperoptable Parameter class.
This module defines a base class for auto-hyperoptable strategies.
"""
import logging
from abc import ABC, abstractmethod
from contextlib import suppress
from typing import Any, Iterator, Optional, Sequence, Tuple, Union
with suppress(ImportError):
from skopt.space import Integer, Real, Categorical
from freqtrade.exceptions import OperationalException
logger = logging.getLogger(__name__)
class BaseParameter(ABC):
"""
Defines a parameter that can be optimized by hyperopt.
"""
category: Optional[str]
default: Any
value: Any
opt_range: Sequence[Any]
def __init__(self, *, opt_range: Sequence[Any], default: Any, space: Optional[str] = None,
optimize: bool = True, load: bool = True, **kwargs):
"""
Initialize hyperopt-optimizable parameter.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter field
name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to skopt.space.(Integer|Real|Categorical).
"""
if 'name' in kwargs:
raise OperationalException(
'Name is determined by parameter field name and can not be specified manually.')
self.category = space
self._space_params = kwargs
self.value = default
self.opt_range = opt_range
self.optimize = optimize
self.load = load
def __repr__(self):
return f'{self.__class__.__name__}({self.value})'
@abstractmethod
def get_space(self, name: str) -> Union['Integer', 'Real', 'Categorical']:
"""
Get-space - will be used by Hyperopt to get the hyperopt Space
"""
def _set_value(self, value: Any):
"""
Update current value. Used by hyperopt functions for the purpose where optimization and
value spaces differ.
:param value: A numerical value.
"""
self.value = value
class IntParameter(BaseParameter):
default: int
value: int
opt_range: Sequence[int]
def __init__(self, low: Union[int, Sequence[int]], high: Optional[int] = None, *, default: int,
space: Optional[str] = None, optimize: bool = True, load: bool = True, **kwargs):
"""
Initialize hyperopt-optimizable integer parameter.
:param low: Lower end (inclusive) of optimization space or [low, high].
:param high: Upper end (inclusive) of optimization space.
Must be none of entire range is passed first parameter.
:param default: A default value.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to skopt.space.Integer.
"""
if high is not None and isinstance(low, Sequence):
raise OperationalException('IntParameter space invalid.')
if high is None or isinstance(low, Sequence):
if not isinstance(low, Sequence) or len(low) != 2:
raise OperationalException('IntParameter space must be [low, high]')
opt_range = low
else:
opt_range = [low, high]
super().__init__(opt_range=opt_range, default=default, space=space, optimize=optimize,
load=load, **kwargs)
def get_space(self, name: str) -> 'Integer':
"""
Create skopt optimization space.
:param name: A name of parameter field.
"""
return Integer(*self.opt_range, name=name, **self._space_params)
class RealParameter(BaseParameter):
default: float
value: float
opt_range: Sequence[float]
def __init__(self, low: Union[float, Sequence[float]], high: Optional[float] = None, *,
default: float, space: Optional[str] = None, optimize: bool = True,
load: bool = True, **kwargs):
"""
Initialize hyperopt-optimizable floating point parameter with unlimited precision.
:param low: Lower end (inclusive) of optimization space or [low, high].
:param high: Upper end (inclusive) of optimization space.
Must be none if entire range is passed first parameter.
:param default: A default value.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to skopt.space.Real.
"""
if high is not None and isinstance(low, Sequence):
raise OperationalException(f'{self.__class__.__name__} space invalid.')
if high is None or isinstance(low, Sequence):
if not isinstance(low, Sequence) or len(low) != 2:
raise OperationalException(f'{self.__class__.__name__} space must be [low, high]')
opt_range = low
else:
opt_range = [low, high]
super().__init__(opt_range=opt_range, default=default, space=space, optimize=optimize,
load=load, **kwargs)
def get_space(self, name: str) -> 'Real':
"""
Create skopt optimization space.
:param name: A name of parameter field.
"""
return Real(*self.opt_range, name=name, **self._space_params)
class DecimalParameter(RealParameter):
default: float
value: float
opt_range: Sequence[float]
def __init__(self, low: Union[float, Sequence[float]], high: Optional[float] = None, *,
default: float, decimals: int = 3, space: Optional[str] = None,
optimize: bool = True, load: bool = True, **kwargs):
"""
Initialize hyperopt-optimizable decimal parameter with a limited precision.
:param low: Lower end (inclusive) of optimization space or [low, high].
:param high: Upper end (inclusive) of optimization space.
Must be none if entire range is passed first parameter.
:param default: A default value.
:param decimals: A number of decimals after floating point to be included in testing.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter fieldname is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to skopt.space.Real.
"""
self._decimals = decimals
default = round(default, self._decimals)
super().__init__(low=low, high=high, default=default, space=space, optimize=optimize,
load=load, **kwargs)
def get_space(self, name: str) -> 'Integer':
"""
Create skopt optimization space.
:param name: A name of parameter field.
"""
low = int(self.opt_range[0] * pow(10, self._decimals))
high = int(self.opt_range[1] * pow(10, self._decimals))
return Integer(low, high, name=name, **self._space_params)
def _set_value(self, value: int):
"""
Update current value. Used by hyperopt functions for the purpose where optimization and
value spaces differ.
:param value: An integer value.
"""
self.value = round(value * pow(0.1, self._decimals), self._decimals)
class CategoricalParameter(BaseParameter):
default: Any
value: Any
opt_range: Sequence[Any]
def __init__(self, categories: Sequence[Any], *, default: Optional[Any] = None,
space: Optional[str] = None, optimize: bool = True, load: bool = True, **kwargs):
"""
Initialize hyperopt-optimizable parameter.
:param categories: Optimization space, [a, b, ...].
:param default: A default value. If not specified, first item from specified space will be
used.
:param space: A parameter category. Can be 'buy' or 'sell'. This parameter is optional if
parameter field
name is prefixed with 'buy_' or 'sell_'.
:param optimize: Include parameter in hyperopt optimizations.
:param load: Load parameter value from {space}_params.
:param kwargs: Extra parameters to skopt.space.Categorical.
"""
if len(categories) < 2:
raise OperationalException(
'CategoricalParameter space must be [a, b, ...] (at least two parameters)')
super().__init__(opt_range=categories, default=default, space=space, optimize=optimize,
load=load, **kwargs)
def get_space(self, name: str) -> 'Categorical':
"""
Create skopt optimization space.
:param name: A name of parameter field.
"""
return Categorical(self.opt_range, name=name, **self._space_params)
class HyperStrategyMixin(object):
"""
A helper base class which allows HyperOptAuto class to reuse implementations of of buy/sell
strategy logic.
"""
def __init__(self, *args, **kwargs):
"""
Initialize hyperoptable strategy mixin.
"""
self._load_params(getattr(self, 'buy_params', None))
self._load_params(getattr(self, 'sell_params', None))
def enumerate_parameters(self, category: str = None) -> Iterator[Tuple[str, BaseParameter]]:
"""
Find all optimizeable parameters and return (name, attr) iterator.
:param category:
:return:
"""
if category not in ('buy', 'sell', None):
raise OperationalException('Category must be one of: "buy", "sell", None.')
for attr_name in dir(self):
if not attr_name.startswith('__'): # Ignore internals, not strictly necessary.
attr = getattr(self, attr_name)
if issubclass(attr.__class__, BaseParameter):
if (category and attr_name.startswith(category + '_')
and attr.category is not None and attr.category != category):
raise OperationalException(
f'Inconclusive parameter name {attr_name}, category: {attr.category}.')
if (category is None or category == attr.category or
(attr_name.startswith(category + '_') and attr.category is None)):
yield attr_name, attr
def _load_params(self, params: dict) -> None:
"""
Set optimizeable parameter values.
:param params: Dictionary with new parameter values.
"""
if not params:
return
for attr_name, attr in self.enumerate_parameters():
if attr_name in params:
if attr.load:
attr.value = params[attr_name]
logger.info(f'Strategy Parameter: {attr_name} = {attr.value}')
else:
logger.warning(f'Parameter "{attr_name}" exists, but is disabled. '
f'Default value "{attr.value}" used.')

4
freqtrade/strategy/interface.py
View File

@ -18,6 +18,7 @@ from freqtrade.exceptions import OperationalException, StrategyError
from freqtrade.exchange import timeframe_to_minutes, timeframe_to_seconds from freqtrade.exchange import timeframe_to_minutes, timeframe_to_seconds
from freqtrade.exchange.exchange import timeframe_to_next_date from freqtrade.exchange.exchange import timeframe_to_next_date
from freqtrade.persistence import PairLocks, Trade from freqtrade.persistence import PairLocks, Trade
from freqtrade.strategy.hyper import HyperStrategyMixin
from freqtrade.strategy.strategy_wrapper import strategy_safe_wrapper from freqtrade.strategy.strategy_wrapper import strategy_safe_wrapper
from freqtrade.wallets import Wallets from freqtrade.wallets import Wallets
@ -59,7 +60,7 @@ class SellCheckTuple(NamedTuple):
sell_type: SellType sell_type: SellType
class IStrategy(ABC): class IStrategy(ABC, HyperStrategyMixin):
""" """
Interface for freqtrade strategies Interface for freqtrade strategies
Defines the mandatory structure must follow any custom strategies Defines the mandatory structure must follow any custom strategies
@ -140,6 +141,7 @@ class IStrategy(ABC):
self.config = config self.config = config
# Dict to determine if analysis is necessary # Dict to determine if analysis is necessary
self._last_candle_seen_per_pair: Dict[str, datetime] = {} self._last_candle_seen_per_pair: Dict[str, datetime] = {}
super().__init__(config)
@abstractmethod @abstractmethod
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

4
freqtrade/templates/base_strategy.py.j2
View File

@ -1,4 +1,5 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement # pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
# flake8: noqa: F401
# --- Do not remove these libs --- # --- Do not remove these libs ---
import numpy as np # noqa import numpy as np # noqa
@ -6,6 +7,7 @@ import pandas as pd # noqa
from pandas import DataFrame from pandas import DataFrame
from freqtrade.strategy import IStrategy from freqtrade.strategy import IStrategy
from freqtrade.strategy import CategoricalParameter, DecimalParameter, IntParameter
# -------------------------------- # --------------------------------
# Add your lib to import here # Add your lib to import here
@ -16,7 +18,7 @@ import freqtrade.vendor.qtpylib.indicators as qtpylib
class {{ strategy }}(IStrategy): class {{ strategy }}(IStrategy):
""" """
This is a strategy template to get you started. This is a strategy template to get you started.
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md More information in https://www.freqtrade.io/en/latest/strategy-customization/
You can: You can:
:return: a Dataframe with all mandatory indicators for the strategies :return: a Dataframe with all mandatory indicators for the strategies

12
freqtrade/templates/sample_strategy.py
View File

@ -1,4 +1,5 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement # pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
# flake8: noqa: F401
# isort: skip_file # isort: skip_file
# --- Do not remove these libs --- # --- Do not remove these libs ---
import numpy as np # noqa import numpy as np # noqa
@ -6,6 +7,7 @@ import pandas as pd # noqa
from pandas import DataFrame from pandas import DataFrame
from freqtrade.strategy import IStrategy from freqtrade.strategy import IStrategy
from freqtrade.strategy import CategoricalParameter, DecimalParameter, IntParameter
# -------------------------------- # --------------------------------
# Add your lib to import here # Add your lib to import here
@ -53,6 +55,10 @@ class SampleStrategy(IStrategy):
# trailing_stop_positive = 0.01 # trailing_stop_positive = 0.01
# trailing_stop_positive_offset = 0.0 # Disabled / not configured # trailing_stop_positive_offset = 0.0 # Disabled / not configured
# Hyperoptable parameters
buy_rsi = IntParameter(low=1, high=50, default=30, space='buy', optimize=True, load=True)
sell_rsi = IntParameter(low=50, high=100, default=70, space='sell', optimize=True, load=True)
# Optimal timeframe for the strategy. # Optimal timeframe for the strategy.
timeframe = '5m' timeframe = '5m'
@ -340,7 +346,8 @@ class SampleStrategy(IStrategy):
""" """
dataframe.loc[ dataframe.loc[
( (
(qtpylib.crossed_above(dataframe['rsi'], 30)) & # Signal: RSI crosses above 30 # Signal: RSI crosses above 30
(qtpylib.crossed_above(dataframe['rsi'], self.buy_rsi.value)) &
(dataframe['tema'] <= dataframe['bb_middleband']) & # Guard: tema below BB middle (dataframe['tema'] <= dataframe['bb_middleband']) & # Guard: tema below BB middle
(dataframe['tema'] > dataframe['tema'].shift(1)) & # Guard: tema is raising (dataframe['tema'] > dataframe['tema'].shift(1)) & # Guard: tema is raising
(dataframe['volume'] > 0) # Make sure Volume is not 0 (dataframe['volume'] > 0) # Make sure Volume is not 0
@ -358,7 +365,8 @@ class SampleStrategy(IStrategy):
""" """
dataframe.loc[ dataframe.loc[
( (
(qtpylib.crossed_above(dataframe['rsi'], 70)) & # Signal: RSI crosses above 70 # Signal: RSI crosses above 70
(qtpylib.crossed_above(dataframe['rsi'], self.sell_rsi.value)) &
(dataframe['tema'] > dataframe['bb_middleband']) & # Guard: tema above BB middle (dataframe['tema'] > dataframe['bb_middleband']) & # Guard: tema above BB middle
(dataframe['tema'] < dataframe['tema'].shift(1)) & # Guard: tema is falling (dataframe['tema'] < dataframe['tema'].shift(1)) & # Guard: tema is falling
(dataframe['volume'] > 0) # Make sure Volume is not 0 (dataframe['volume'] > 0) # Make sure Volume is not 0

2
tests/optimize/conftest.py
View File

@ -14,6 +14,7 @@ from tests.conftest import patch_exchange
def hyperopt_conf(default_conf): def hyperopt_conf(default_conf):
hyperconf = deepcopy(default_conf) hyperconf = deepcopy(default_conf)
hyperconf.update({ hyperconf.update({
'datadir': Path(default_conf['datadir']),
'hyperopt': 'DefaultHyperOpt', 'hyperopt': 'DefaultHyperOpt',
'hyperopt_loss': 'ShortTradeDurHyperOptLoss', 'hyperopt_loss': 'ShortTradeDurHyperOptLoss',
'hyperopt_path': str(Path(__file__).parent / 'hyperopts'), 'hyperopt_path': str(Path(__file__).parent / 'hyperopts'),
@ -21,6 +22,7 @@ def hyperopt_conf(default_conf):
'timerange': None, 'timerange': None,
'spaces': ['default'], 'spaces': ['default'],
'hyperopt_jobs': 1, 'hyperopt_jobs': 1,
'hyperopt_min_trades': 1,
}) })
return hyperconf return hyperconf

15
tests/optimize/test_hyperopt.py
View File

@ -16,6 +16,7 @@ from freqtrade.commands.optimize_commands import setup_optimize_configuration, s
from freqtrade.data.history import load_data from freqtrade.data.history import load_data
from freqtrade.exceptions import OperationalException from freqtrade.exceptions import OperationalException
from freqtrade.optimize.hyperopt import Hyperopt from freqtrade.optimize.hyperopt import Hyperopt
from freqtrade.optimize.hyperopt_auto import HyperOptAuto
from freqtrade.optimize.hyperopt_tools import HyperoptTools from freqtrade.optimize.hyperopt_tools import HyperoptTools
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
from freqtrade.state import RunMode from freqtrade.state import RunMode
@ -1089,3 +1090,17 @@ def test_print_epoch_details(capsys):
assert '# ROI table:' in captured.out assert '# ROI table:' in captured.out
assert re.search(r'^\s+minimal_roi = \{$', captured.out, re.MULTILINE) assert re.search(r'^\s+minimal_roi = \{$', captured.out, re.MULTILINE)
assert re.search(r'^\s+\"90\"\:\s0.14,\s*$', captured.out, re.MULTILINE) assert re.search(r'^\s+\"90\"\:\s0.14,\s*$', captured.out, re.MULTILINE)
def test_in_strategy_auto_hyperopt(mocker, hyperopt_conf, tmpdir) -> None:
(Path(tmpdir) / 'hyperopt_results').mkdir(parents=True)
# No hyperopt needed
del hyperopt_conf['hyperopt']
hyperopt_conf.update({
'strategy': 'HyperoptableStrategy',
'user_data_dir': Path(tmpdir),
})
hyperopt = Hyperopt(hyperopt_conf)
assert isinstance(hyperopt.custom_hyperopt, HyperOptAuto)
hyperopt.start()

6
tests/rpc/test_rpc_apiserver.py
View File

@ -1151,7 +1151,11 @@ def test_api_strategies(botclient):
rc = client_get(client, f"{BASE_URI}/strategies") rc = client_get(client, f"{BASE_URI}/strategies")
assert_response(rc) assert_response(rc)
assert rc.json() == {'strategies': ['DefaultStrategy', 'TestStrategyLegacy']} assert rc.json() == {'strategies': [
'DefaultStrategy',
'HyperoptableStrategy',
'TestStrategyLegacy'
]}
def test_api_strategy(botclient): def test_api_strategy(botclient):

173
tests/strategy/strats/hyperoptable_strategy.py Normal file
View File

@ -0,0 +1,173 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
import talib.abstract as ta
from pandas import DataFrame
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.strategy import DecimalParameter, IntParameter, IStrategy, RealParameter
class HyperoptableStrategy(IStrategy):
"""
Default Strategy provided by freqtrade bot.
Please do not modify this strategy, it's intended for internal use only.
Please look at the SampleStrategy in the user_data/strategy directory
or strategy repository https://github.com/freqtrade/freqtrade-strategies
for samples and inspiration.
"""
INTERFACE_VERSION = 2
# Minimal ROI designed for the strategy
minimal_roi = {
"40": 0.0,
"30": 0.01,
"20": 0.02,
"0": 0.04
}
# Optimal stoploss designed for the strategy
stoploss = -0.10
# Optimal ticker interval for the strategy
timeframe = '5m'
# Optional order type mapping
order_types = {
'buy': 'limit',
'sell': 'limit',
'stoploss': 'limit',
'stoploss_on_exchange': False
}
# Number of candles the strategy requires before producing valid signals
startup_candle_count: int = 20
# Optional time in force for orders
order_time_in_force = {
'buy': 'gtc',
'sell': 'gtc',
}
buy_params = {
'buy_rsi': 35,
# Intentionally not specified, so "default" is tested
# 'buy_plusdi': 0.4
}
sell_params = {
'sell_rsi': 74,
'sell_minusdi': 0.4
}
buy_rsi = IntParameter([0, 50], default=30, space='buy')
buy_plusdi = RealParameter(low=0, high=1, default=0.5, space='buy')
sell_rsi = IntParameter(low=50, high=100, default=70, space='sell')
sell_minusdi = DecimalParameter(low=0, high=1, default=0.5001, decimals=3, space='sell',
load=False)
def informative_pairs(self):
"""
Define additional, informative pair/interval combinations to be cached from the exchange.
These pair/interval combinations are non-tradeable, unless they are part
of the whitelist as well.
For more information, please consult the documentation
:return: List of tuples in the format (pair, interval)
Sample: return [("ETH/USDT", "5m"),
("BTC/USDT", "15m"),
]
"""
return []
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame
Performance Note: For the best performance be frugal on the number of indicators
you are using. Let uncomment only the indicator you are using in your strategies
or your hyperopt configuration, otherwise you will waste your memory and CPU usage.
:param dataframe: Dataframe with data from the exchange
:param metadata: Additional information, like the currently traded pair
:return: a Dataframe with all mandatory indicators for the strategies
"""
# Momentum Indicator
# ------------------------------------
# ADX
dataframe['adx'] = ta.ADX(dataframe)
# MACD
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['macdhist'] = macd['macdhist']
# Minus Directional Indicator / Movement
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Plus Directional Indicator / Movement
dataframe['plus_di'] = ta.PLUS_DI(dataframe)
# RSI
dataframe['rsi'] = ta.RSI(dataframe)
# Stoch fast
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['fastk'] = stoch_fast['fastk']
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['bb_middleband'] = bollinger['mid']
dataframe['bb_upperband'] = bollinger['upper']
# EMA - Exponential Moving Average
dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10)
return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators, populates the buy signal for the given dataframe
:param dataframe: DataFrame
:param metadata: Additional information, like the currently traded pair
:return: DataFrame with buy column
"""
dataframe.loc[
(
(dataframe['rsi'] < self.buy_rsi.value) &
(dataframe['fastd'] < 35) &
(dataframe['adx'] > 30) &
(dataframe['plus_di'] > self.buy_plusdi.value)
) |
(
(dataframe['adx'] > 65) &
(dataframe['plus_di'] > self.buy_plusdi.value)
),
'buy'] = 1
return dataframe
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators, populates the sell signal for the given dataframe
:param dataframe: DataFrame
:param metadata: Additional information, like the currently traded pair
:return: DataFrame with buy column
"""
dataframe.loc[
(
(
(qtpylib.crossed_above(dataframe['rsi'], self.sell_rsi.value)) |
(qtpylib.crossed_above(dataframe['fastd'], 70))
) &
(dataframe['adx'] > 10) &
(dataframe['minus_di'] > 0)
) |
(
(dataframe['adx'] > 70) &
(dataframe['minus_di'] > self.sell_minusdi.value)
),
'sell'] = 1
return dataframe

72
tests/strategy/test_interface.py
View File

@ -10,9 +10,11 @@ from pandas import DataFrame
from freqtrade.configuration import TimeRange from freqtrade.configuration import TimeRange
from freqtrade.data.dataprovider import DataProvider from freqtrade.data.dataprovider import DataProvider
from freqtrade.data.history import load_data from freqtrade.data.history import load_data
from freqtrade.exceptions import StrategyError from freqtrade.exceptions import OperationalException, StrategyError
from freqtrade.persistence import PairLocks, Trade from freqtrade.persistence import PairLocks, Trade
from freqtrade.resolvers import StrategyResolver from freqtrade.resolvers import StrategyResolver
from freqtrade.strategy.hyper import (BaseParameter, CategoricalParameter, DecimalParameter,
IntParameter, RealParameter)
from freqtrade.strategy.interface import SellCheckTuple, SellType from freqtrade.strategy.interface import SellCheckTuple, SellType
from freqtrade.strategy.strategy_wrapper import strategy_safe_wrapper from freqtrade.strategy.strategy_wrapper import strategy_safe_wrapper
from tests.conftest import log_has, log_has_re from tests.conftest import log_has, log_has_re
@ -552,3 +554,71 @@ def test_strategy_safe_wrapper(value):
assert type(ret) == type(value) assert type(ret) == type(value)
assert ret == value assert ret == value
def test_hyperopt_parameters():
from skopt.space import Categorical, Integer, Real
with pytest.raises(OperationalException, match=r"Name is determined.*"):
IntParameter(low=0, high=5, default=1, name='hello')
with pytest.raises(OperationalException, match=r"IntParameter space must be.*"):
IntParameter(low=0, default=5, space='buy')
with pytest.raises(OperationalException, match=r"RealParameter space must be.*"):
RealParameter(low=0, default=5, space='buy')
with pytest.raises(OperationalException, match=r"DecimalParameter space must be.*"):
DecimalParameter(low=0, default=5, space='buy')
with pytest.raises(OperationalException, match=r"IntParameter space invalid\."):
IntParameter([0, 10], high=7, default=5, space='buy')
with pytest.raises(OperationalException, match=r"RealParameter space invalid\."):
RealParameter([0, 10], high=7, default=5, space='buy')
with pytest.raises(OperationalException, match=r"DecimalParameter space invalid\."):
DecimalParameter([0, 10], high=7, default=5, space='buy')
with pytest.raises(OperationalException, match=r"CategoricalParameter space must.*"):
CategoricalParameter(['aa'], default='aa', space='buy')
with pytest.raises(TypeError):
BaseParameter(opt_range=[0, 1], default=1, space='buy')
intpar = IntParameter(low=0, high=5, default=1, space='buy')
assert intpar.value == 1
assert isinstance(intpar.get_space(''), Integer)
fltpar = RealParameter(low=0.0, high=5.5, default=1.0, space='buy')
assert isinstance(fltpar.get_space(''), Real)
assert fltpar.value == 1
fltpar = DecimalParameter(low=0.0, high=5.5, default=1.0004, decimals=3, space='buy')
assert isinstance(fltpar.get_space(''), Integer)
assert fltpar.value == 1
fltpar._set_value(2222)
assert fltpar.value == 2.222
catpar = CategoricalParameter(['buy_rsi', 'buy_macd', 'buy_none'],
default='buy_macd', space='buy')
assert isinstance(catpar.get_space(''), Categorical)
assert catpar.value == 'buy_macd'
def test_auto_hyperopt_interface(default_conf):
default_conf.update({'strategy': 'HyperoptableStrategy'})
PairLocks.timeframe = default_conf['timeframe']
strategy = StrategyResolver.load_strategy(default_conf)
assert strategy.buy_rsi.value == strategy.buy_params['buy_rsi']
# PlusDI is NOT in the buy-params, so default should be used
assert strategy.buy_plusdi.value == 0.5
assert strategy.sell_rsi.value == strategy.sell_params['sell_rsi']
# Parameter is disabled - so value from sell_param dict will NOT be used.
assert strategy.sell_minusdi.value == 0.5
strategy.sell_rsi = IntParameter([0, 10], default=5, space='buy')
with pytest.raises(OperationalException, match=r"Inconclusive parameter.*"):
[x for x in strategy.enumerate_parameters('sell')]

6
tests/strategy/test_strategy_loading.py
View File

@ -35,7 +35,7 @@ def test_search_all_strategies_no_failed():
directory = Path(__file__).parent / "strats" directory = Path(__file__).parent / "strats"
strategies = StrategyResolver.search_all_objects(directory, enum_failed=False) strategies = StrategyResolver.search_all_objects(directory, enum_failed=False)
assert isinstance(strategies, list) assert isinstance(strategies, list)
assert len(strategies) == 2 assert len(strategies) == 3
assert isinstance(strategies[0], dict) assert isinstance(strategies[0], dict)
@ -43,10 +43,10 @@ def test_search_all_strategies_with_failed():
directory = Path(__file__).parent / "strats" directory = Path(__file__).parent / "strats"
strategies = StrategyResolver.search_all_objects(directory, enum_failed=True) strategies = StrategyResolver.search_all_objects(directory, enum_failed=True)
assert isinstance(strategies, list) assert isinstance(strategies, list)
assert len(strategies) == 3 assert len(strategies) == 4
# with enum_failed=True search_all_objects() shall find 2 good strategies # with enum_failed=True search_all_objects() shall find 2 good strategies
# and 1 which fails to load # and 1 which fails to load
assert len([x for x in strategies if x['class'] is not None]) == 2 assert len([x for x in strategies if x['class'] is not None]) == 3
assert len([x for x in strategies if x['class'] is None]) == 1 assert len([x for x in strategies if x['class'] is None]) == 1