Merge pull request #2024 from freqtrade/custom_hyperopt_loss
Custom hyperopt loss function (and sharpe-ratio)
This commit is contained in:
commit
790838d897
@ -199,19 +199,22 @@ to find optimal parameter values for your stategy.
|
||||
|
||||
```
|
||||
usage: freqtrade hyperopt [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
|
||||
[--max_open_trades MAX_OPEN_TRADES]
|
||||
[--stake_amount STAKE_AMOUNT] [-r]
|
||||
[--customhyperopt NAME] [--eps] [--dmmp] [-e INT]
|
||||
[-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
|
||||
[--print-all] [-j JOBS]
|
||||
[--max_open_trades INT]
|
||||
[--stake_amount STAKE_AMOUNT] [-r]
|
||||
[--customhyperopt NAME] [--eps] [-e INT]
|
||||
[-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
|
||||
[--dmmp] [--print-all] [-j JOBS]
|
||||
[--random-state INT] [--min-trades INT] [--continue]
|
||||
[--hyperopt-loss NAME]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
-i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
|
||||
Specify ticker interval (1m, 5m, 30m, 1h, 1d).
|
||||
Specify ticker interval (`1m`, `5m`, `30m`, `1h`,
|
||||
`1d`).
|
||||
--timerange TIMERANGE
|
||||
Specify what timerange of data to use.
|
||||
--max_open_trades MAX_OPEN_TRADES
|
||||
--max_open_trades INT
|
||||
Specify max_open_trades to use.
|
||||
--stake_amount STAKE_AMOUNT
|
||||
Specify stake_amount.
|
||||
@ -221,18 +224,18 @@ optional arguments:
|
||||
run your optimization commands with up-to-date data.
|
||||
--customhyperopt NAME
|
||||
Specify hyperopt class name (default:
|
||||
DefaultHyperOpts).
|
||||
`DefaultHyperOpts`).
|
||||
--eps, --enable-position-stacking
|
||||
Allow buying the same pair multiple times (position
|
||||
stacking).
|
||||
-e INT, --epochs INT Specify number of epochs (default: 100).
|
||||
-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...], --spaces {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]
|
||||
Specify which parameters to hyperopt. Space-separated
|
||||
list. Default: `all`.
|
||||
--dmmp, --disable-max-market-positions
|
||||
Disable applying `max_open_trades` during backtest
|
||||
(same as setting `max_open_trades` to a very high
|
||||
number).
|
||||
-e INT, --epochs INT Specify number of epochs (default: 100).
|
||||
-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...], --spaces {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]
|
||||
Specify which parameters to hyperopt. Space separate
|
||||
list. Default: all.
|
||||
--print-all Print all results, not only the best ones.
|
||||
-j JOBS, --job-workers JOBS
|
||||
The number of concurrently running jobs for
|
||||
@ -240,6 +243,19 @@ optional arguments:
|
||||
(default), all CPUs are used, for -2, all CPUs but one
|
||||
are used, etc. If 1 is given, no parallel computing
|
||||
code is used at all.
|
||||
--random-state INT Set random state to some positive integer for
|
||||
reproducible hyperopt results.
|
||||
--min-trades INT Set minimal desired number of trades for evaluations
|
||||
in the hyperopt optimization path (default: 1).
|
||||
--continue Continue hyperopt from previous runs. By default,
|
||||
temporary files will be removed and hyperopt will
|
||||
start from scratch.
|
||||
--hyperopt-loss NAME
|
||||
Specify the class name of the hyperopt loss function
|
||||
class (IHyperOptLoss). Different functions can
|
||||
generate completely different results, since the
|
||||
target for optimization is different. (default:
|
||||
`DefaultHyperOptLoss`).
|
||||
```
|
||||
|
||||
## Edge commands
|
||||
|
101
docs/hyperopt.md
101
docs/hyperopt.md
@ -144,16 +144,85 @@ it will end with telling you which paramter combination produced the best profit
|
||||
|
||||
The search for best parameters starts with a few random combinations and then uses a
|
||||
regressor algorithm (currently ExtraTreesRegressor) to quickly find a parameter combination
|
||||
that minimizes the value of the objective function `calculate_loss` in `hyperopt.py`.
|
||||
that minimizes the value of the [loss function](#loss-functions).
|
||||
|
||||
The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
|
||||
When you want to test an indicator that isn't used by the bot currently, remember to
|
||||
add it to the `populate_indicators()` method in `hyperopt.py`.
|
||||
|
||||
## Loss-functions
|
||||
|
||||
Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results.
|
||||
|
||||
By default, FreqTrade uses a loss function, which has been with freqtrade since the beginning and optimizes mostly for short trade duration and avoiding losses.
|
||||
|
||||
A different version this can be used by using the `--hyperopt-loss <Class-name>` argument.
|
||||
This class should be in it's own file within the `user_data/hyperopts/` directory.
|
||||
|
||||
Currently, the following loss functions are builtin: `SharpeHyperOptLoss` and `DefaultHyperOptLoss`.
|
||||
|
||||
### Creating and using a custom loss function
|
||||
|
||||
To use a custom loss function class, make sure that the function `hyperopt_loss_function` is defined in your custom hyperopt loss class.
|
||||
For the sample below, you then need to add the command line parameter `--hyperopt-loss SuperDuperHyperOptLoss` to your hyperopt call so this fuction is being used.
|
||||
|
||||
A sample of this can be found below, which is identical to the Default Hyperopt loss implementation. A full sample can be found [user_data/hyperopts/](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_loss.py)
|
||||
|
||||
``` python
|
||||
from freqtrade.optimize.hyperopt import IHyperOptLoss
|
||||
|
||||
TARGET_TRADES = 600
|
||||
EXPECTED_MAX_PROFIT = 3.0
|
||||
MAX_ACCEPTED_TRADE_DURATION = 300
|
||||
|
||||
class SuperDuperHyperOptLoss(IHyperOptLoss):
|
||||
"""
|
||||
Defines the default loss function for hyperopt
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def hyperopt_loss_function(results: DataFrame, trade_count: int,
|
||||
min_date: datetime, max_date: datetime,
|
||||
*args, **kwargs) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for better results
|
||||
This is the legacy algorithm (used until now in freqtrade).
|
||||
Weights are distributed as follows:
|
||||
* 0.4 to trade duration
|
||||
* 0.25: Avoiding trade loss
|
||||
* 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
|
||||
"""
|
||||
total_profit = results.profit_percent.sum()
|
||||
trade_duration = results.trade_duration.mean()
|
||||
|
||||
trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
|
||||
profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
|
||||
duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
|
||||
result = trade_loss + profit_loss + duration_loss
|
||||
return result
|
||||
```
|
||||
|
||||
Currently, the arguments are:
|
||||
|
||||
* `results`: DataFrame containing the result
|
||||
The following columns are available in results (corresponds to the output-file of backtesting when used with `--export trades`):
|
||||
`pair, profit_percent, profit_abs, open_time, close_time, open_index, close_index, trade_duration, open_at_end, open_rate, close_rate, sell_reason`
|
||||
* `trade_count`: Amount of trades (identical to `len(results)`)
|
||||
* `min_date`: Start date of the hyperopting TimeFrame
|
||||
* `min_date`: End date of the hyperopting TimeFrame
|
||||
|
||||
This function needs to return a floating point number (`float`). Smaller numbers will be interpreted as better results. The parameters and balancing for this is up to you.
|
||||
|
||||
!!! Note
|
||||
This function is called once per iteration - so please make sure to have this as optimized as possible to not slow hyperopt down unnecessarily.
|
||||
|
||||
!!! Note
|
||||
Please keep the arguments `*args` and `**kwargs` in the interface to allow us to extend this interface later.
|
||||
|
||||
## Execute Hyperopt
|
||||
|
||||
Once you have updated your hyperopt configuration you can run it.
|
||||
Because hyperopt tries a lot of combinations to find the best parameters it will take time you will have the result (more than 30 mins).
|
||||
Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result. More time usually results in better results.
|
||||
|
||||
We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
|
||||
|
||||
@ -168,8 +237,11 @@ running at least several thousand evaluations.
|
||||
|
||||
The `--spaces all` flag determines that all possible parameters should be optimized. Possibilities are listed below.
|
||||
|
||||
!!! Note
|
||||
By default, hyperopt will erase previous results and start from scratch. Continuation can be archived by using `--continue`.
|
||||
|
||||
!!! Warning
|
||||
When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file.
|
||||
When switching parameters or changing configuration options, make sure to not use the argument `--continue` so temporary results can be removed.
|
||||
|
||||
### Execute Hyperopt with Different Ticker-Data Source
|
||||
|
||||
@ -179,12 +251,11 @@ use data from directory `user_data/data`.
|
||||
|
||||
### Running Hyperopt with Smaller Testset
|
||||
|
||||
Use the `--timerange` argument to change how much of the testset
|
||||
you want to use. The last N ticks/timeframes will be used.
|
||||
Example:
|
||||
Use the `--timerange` argument to change how much of the testset you want to use.
|
||||
For example, to use one month of data, pass the following parameter to the hyperopt call:
|
||||
|
||||
```bash
|
||||
freqtrade hyperopt --timerange -200
|
||||
freqtrade hyperopt --timerange 20180401-20180501
|
||||
```
|
||||
|
||||
### Running Hyperopt with Smaller Search Space
|
||||
@ -197,14 +268,14 @@ new buy strategy you have.
|
||||
|
||||
Legal values are:
|
||||
|
||||
- `all`: optimize everything
|
||||
- `buy`: just search for a new buy strategy
|
||||
- `sell`: just search for a new sell strategy
|
||||
- `roi`: just optimize the minimal profit table for your strategy
|
||||
- `stoploss`: search for the best stoploss value
|
||||
- space-separated list of any of the above values for example `--spaces roi stoploss`
|
||||
* `all`: optimize everything
|
||||
* `buy`: just search for a new buy strategy
|
||||
* `sell`: just search for a new sell strategy
|
||||
* `roi`: just optimize the minimal profit table for your strategy
|
||||
* `stoploss`: search for the best stoploss value
|
||||
* space-separated list of any of the above values for example `--spaces roi stoploss`
|
||||
|
||||
### Position stacking and disabling max market positions.
|
||||
### Position stacking and disabling max market positions
|
||||
|
||||
In some situations, you may need to run Hyperopt (and Backtesting) with the
|
||||
`--eps`/`--enable-position-staking` and `--dmmp`/`--disable-max-market-positions` arguments.
|
||||
@ -252,7 +323,7 @@ method, what those values match to.
|
||||
|
||||
So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
|
||||
|
||||
```
|
||||
``` python
|
||||
(dataframe['rsi'] < 29.0)
|
||||
```
|
||||
|
||||
|
@ -213,6 +213,21 @@ AVAILABLE_CLI_OPTIONS = {
|
||||
metavar='INT',
|
||||
default=1,
|
||||
),
|
||||
"hyperopt_continue": Arg(
|
||||
"--continue",
|
||||
help="Continue hyperopt from previous runs. "
|
||||
"By default, temporary files will be removed and hyperopt will start from scratch.",
|
||||
default=False,
|
||||
action='store_true',
|
||||
),
|
||||
"hyperopt_loss": Arg(
|
||||
'--hyperopt-loss',
|
||||
help='Specify the class name of the hyperopt loss function class (IHyperOptLoss). '
|
||||
'Different functions can generate completely different results, '
|
||||
'since the target for optimization is different. (default: `%(default)s`).',
|
||||
metavar='NAME',
|
||||
default=constants.DEFAULT_HYPEROPT_LOSS,
|
||||
),
|
||||
# List exchanges
|
||||
"print_one_column": Arg(
|
||||
'-1', '--one-column',
|
||||
@ -299,7 +314,8 @@ ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_pos
|
||||
|
||||
ARGS_HYPEROPT = ARGS_COMMON_OPTIMIZE + ["hyperopt", "position_stacking", "epochs", "spaces",
|
||||
"use_max_market_positions", "print_all", "hyperopt_jobs",
|
||||
"hyperopt_random_state", "hyperopt_min_trades"]
|
||||
"hyperopt_random_state", "hyperopt_min_trades",
|
||||
"hyperopt_continue", "hyperopt_loss"]
|
||||
|
||||
ARGS_EDGE = ARGS_COMMON_OPTIMIZE + ["stoploss_range"]
|
||||
|
||||
|
@ -259,6 +259,12 @@ class Configuration(object):
|
||||
self._args_to_config(config, argname='hyperopt_min_trades',
|
||||
logstring='Parameter --min-trades detected: {}')
|
||||
|
||||
self._args_to_config(config, argname='hyperopt_continue',
|
||||
logstring='Hyperopt continue: {}')
|
||||
|
||||
self._args_to_config(config, argname='hyperopt_loss',
|
||||
logstring='Using loss function: {}')
|
||||
|
||||
def _process_plot_options(self, config: Dict[str, Any]) -> None:
|
||||
|
||||
self._args_to_config(config, argname='pairs',
|
||||
|
@ -12,6 +12,7 @@ HYPEROPT_EPOCH = 100 # epochs
|
||||
RETRY_TIMEOUT = 30 # sec
|
||||
DEFAULT_STRATEGY = 'DefaultStrategy'
|
||||
DEFAULT_HYPEROPT = 'DefaultHyperOpts'
|
||||
DEFAULT_HYPEROPT_LOSS = 'DefaultHyperOptLoss'
|
||||
DEFAULT_DB_PROD_URL = 'sqlite:///tradesv3.sqlite'
|
||||
DEFAULT_DB_DRYRUN_URL = 'sqlite://'
|
||||
UNLIMITED_STAKE_AMOUNT = 'unlimited'
|
||||
|
@ -1,17 +1,15 @@
|
||||
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
|
||||
|
||||
from functools import reduce
|
||||
from typing import Any, Callable, Dict, List
|
||||
|
||||
import talib.abstract as ta
|
||||
from pandas import DataFrame
|
||||
from typing import Dict, Any, Callable, List
|
||||
from functools import reduce
|
||||
|
||||
from skopt.space import Categorical, Dimension, Integer, Real
|
||||
|
||||
import freqtrade.vendor.qtpylib.indicators as qtpylib
|
||||
from freqtrade.optimize.hyperopt_interface import IHyperOpt
|
||||
|
||||
class_name = 'DefaultHyperOpts'
|
||||
|
||||
|
||||
class DefaultHyperOpts(IHyperOpt):
|
||||
"""
|
||||
|
53
freqtrade/optimize/default_hyperopt_loss.py
Normal file
53
freqtrade/optimize/default_hyperopt_loss.py
Normal file
@ -0,0 +1,53 @@
|
||||
"""
|
||||
DefaultHyperOptLoss
|
||||
This module defines the default HyperoptLoss class which is being used for
|
||||
Hyperoptimization.
|
||||
"""
|
||||
|
||||
from math import exp
|
||||
|
||||
from pandas import DataFrame
|
||||
|
||||
from freqtrade.optimize.hyperopt import IHyperOptLoss
|
||||
|
||||
# Define some constants:
|
||||
|
||||
# set TARGET_TRADES to suit your number concurrent trades so its realistic
|
||||
# to the number of days
|
||||
TARGET_TRADES = 600
|
||||
# This is assumed to be expected avg profit * expected trade count.
|
||||
# For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
|
||||
# self.expected_max_profit = 3.85
|
||||
# Check that the reported Σ% values do not exceed this!
|
||||
# Note, this is ratio. 3.85 stated above means 385Σ%.
|
||||
EXPECTED_MAX_PROFIT = 3.0
|
||||
|
||||
# max average trade duration in minutes
|
||||
# if eval ends with higher value, we consider it a failed eval
|
||||
MAX_ACCEPTED_TRADE_DURATION = 300
|
||||
|
||||
|
||||
class DefaultHyperOptLoss(IHyperOptLoss):
|
||||
"""
|
||||
Defines the default loss function for hyperopt
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def hyperopt_loss_function(results: DataFrame, trade_count: int,
|
||||
*args, **kwargs) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for better results
|
||||
This is the Default algorithm
|
||||
Weights are distributed as follows:
|
||||
* 0.4 to trade duration
|
||||
* 0.25: Avoiding trade loss
|
||||
* 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
|
||||
"""
|
||||
total_profit = results.profit_percent.sum()
|
||||
trade_duration = results.trade_duration.mean()
|
||||
|
||||
trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
|
||||
profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
|
||||
duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
|
||||
result = trade_loss + profit_loss + duration_loss
|
||||
return result
|
@ -7,7 +7,7 @@ This module contains the hyperopt logic
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from math import exp
|
||||
|
||||
from operator import itemgetter
|
||||
from pathlib import Path
|
||||
from pprint import pprint
|
||||
@ -21,7 +21,9 @@ from skopt.space import Dimension
|
||||
from freqtrade.configuration import Arguments
|
||||
from freqtrade.data.history import load_data, get_timeframe
|
||||
from freqtrade.optimize.backtesting import Backtesting
|
||||
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
|
||||
# Import IHyperOptLoss to allow users import from this file
|
||||
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F4
|
||||
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -46,27 +48,46 @@ class Hyperopt(Backtesting):
|
||||
super().__init__(config)
|
||||
self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
|
||||
|
||||
# set TARGET_TRADES to suit your number concurrent trades so its realistic
|
||||
# to the number of days
|
||||
self.target_trades = 600
|
||||
self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
|
||||
self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
|
||||
|
||||
self.total_tries = config.get('epochs', 0)
|
||||
self.current_best_loss = 100
|
||||
|
||||
# max average trade duration in minutes
|
||||
# if eval ends with higher value, we consider it a failed eval
|
||||
self.max_accepted_trade_duration = 300
|
||||
|
||||
# This is assumed to be expected avg profit * expected trade count.
|
||||
# For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
|
||||
# self.expected_max_profit = 3.85
|
||||
# Check that the reported Σ% values do not exceed this!
|
||||
# Note, this is ratio. 3.85 stated above means 385Σ%.
|
||||
self.expected_max_profit = 3.0
|
||||
if not self.config.get('hyperopt_continue'):
|
||||
self.clean_hyperopt()
|
||||
else:
|
||||
logger.info("Continuing on previous hyperopt results.")
|
||||
|
||||
# Previous evaluations
|
||||
self.trials_file = TRIALSDATA_PICKLE
|
||||
self.trials: List = []
|
||||
|
||||
# Populate functions here (hasattr is slow so should not be run during "regular" operations)
|
||||
if hasattr(self.custom_hyperopt, 'populate_buy_trend'):
|
||||
self.advise_buy = self.custom_hyperopt.populate_buy_trend # type: ignore
|
||||
|
||||
if hasattr(self.custom_hyperopt, 'populate_sell_trend'):
|
||||
self.advise_sell = self.custom_hyperopt.populate_sell_trend # type: ignore
|
||||
|
||||
# Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
|
||||
if self.config.get('use_max_market_positions', True):
|
||||
self.max_open_trades = self.config['max_open_trades']
|
||||
else:
|
||||
logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
|
||||
self.max_open_trades = 0
|
||||
self.position_stacking = self.config.get('position_stacking', False),
|
||||
|
||||
def clean_hyperopt(self):
|
||||
"""
|
||||
Remove hyperopt pickle files to restart hyperopt.
|
||||
"""
|
||||
for f in [TICKERDATA_PICKLE, TRIALSDATA_PICKLE]:
|
||||
p = Path(f)
|
||||
if p.is_file():
|
||||
logger.info(f"Removing `{p}`.")
|
||||
p.unlink()
|
||||
|
||||
def get_args(self, params):
|
||||
dimensions = self.hyperopt_space()
|
||||
# Ensure the number of dimensions match
|
||||
@ -134,16 +155,6 @@ class Hyperopt(Backtesting):
|
||||
print('.', end='')
|
||||
sys.stdout.flush()
|
||||
|
||||
def calculate_loss(self, total_profit: float, trade_count: int, trade_duration: float) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for more optimal results
|
||||
"""
|
||||
trade_loss = 1 - 0.25 * exp(-(trade_count - self.target_trades) ** 2 / 10 ** 5.8)
|
||||
profit_loss = max(0, 1 - total_profit / self.expected_max_profit)
|
||||
duration_loss = 0.4 * min(trade_duration / self.max_accepted_trade_duration, 1)
|
||||
result = trade_loss + profit_loss + duration_loss
|
||||
return result
|
||||
|
||||
def has_space(self, space: str) -> bool:
|
||||
"""
|
||||
Tell if a space value is contained in the configuration
|
||||
@ -172,49 +183,40 @@ class Hyperopt(Backtesting):
|
||||
return spaces
|
||||
|
||||
def generate_optimizer(self, _params: Dict) -> Dict:
|
||||
"""
|
||||
Used Optimize function. Called once per epoch to optimize whatever is configured.
|
||||
Keep this function as optimized as possible!
|
||||
"""
|
||||
params = self.get_args(_params)
|
||||
if self.has_space('roi'):
|
||||
self.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params)
|
||||
|
||||
if self.has_space('buy'):
|
||||
self.advise_buy = self.custom_hyperopt.buy_strategy_generator(params)
|
||||
elif hasattr(self.custom_hyperopt, 'populate_buy_trend'):
|
||||
self.advise_buy = self.custom_hyperopt.populate_buy_trend # type: ignore
|
||||
|
||||
if self.has_space('sell'):
|
||||
self.advise_sell = self.custom_hyperopt.sell_strategy_generator(params)
|
||||
elif hasattr(self.custom_hyperopt, 'populate_sell_trend'):
|
||||
self.advise_sell = self.custom_hyperopt.populate_sell_trend # type: ignore
|
||||
|
||||
if self.has_space('stoploss'):
|
||||
self.strategy.stoploss = params['stoploss']
|
||||
|
||||
processed = load(TICKERDATA_PICKLE)
|
||||
|
||||
# Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
|
||||
if self.config.get('use_max_market_positions', True):
|
||||
max_open_trades = self.config['max_open_trades']
|
||||
else:
|
||||
logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
|
||||
max_open_trades = 0
|
||||
|
||||
min_date, max_date = get_timeframe(processed)
|
||||
|
||||
results = self.backtest(
|
||||
{
|
||||
'stake_amount': self.config['stake_amount'],
|
||||
'processed': processed,
|
||||
'max_open_trades': max_open_trades,
|
||||
'position_stacking': self.config.get('position_stacking', False),
|
||||
'max_open_trades': self.max_open_trades,
|
||||
'position_stacking': self.position_stacking,
|
||||
'start_date': min_date,
|
||||
'end_date': max_date,
|
||||
}
|
||||
)
|
||||
result_explanation = self.format_results(results)
|
||||
|
||||
total_profit = results.profit_percent.sum()
|
||||
trade_count = len(results.index)
|
||||
trade_duration = results.trade_duration.mean()
|
||||
|
||||
# If this evaluation contains too short amount of trades to be
|
||||
# interesting -- consider it as 'bad' (assigned max. loss value)
|
||||
@ -227,7 +229,8 @@ class Hyperopt(Backtesting):
|
||||
'result': result_explanation,
|
||||
}
|
||||
|
||||
loss = self.calculate_loss(total_profit, trade_count, trade_duration)
|
||||
loss = self.calculate_loss(results=results, trade_count=trade_count,
|
||||
min_date=min_date.datetime, max_date=max_date.datetime)
|
||||
|
||||
return {
|
||||
'loss': loss,
|
||||
|
25
freqtrade/optimize/hyperopt_loss_interface.py
Normal file
25
freqtrade/optimize/hyperopt_loss_interface.py
Normal file
@ -0,0 +1,25 @@
|
||||
"""
|
||||
IHyperOptLoss interface
|
||||
This module defines the interface for the loss-function for hyperopts
|
||||
"""
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from datetime import datetime
|
||||
|
||||
from pandas import DataFrame
|
||||
|
||||
|
||||
class IHyperOptLoss(ABC):
|
||||
"""
|
||||
Interface for freqtrade hyperopts Loss functions.
|
||||
Defines the custom loss function (`hyperopt_loss_function()` which is evaluated every epoch.)
|
||||
"""
|
||||
ticker_interval: str
|
||||
|
||||
@staticmethod
|
||||
@abstractmethod
|
||||
def hyperopt_loss_function(results: DataFrame, trade_count: int,
|
||||
min_date: datetime, max_date: datetime, *args, **kwargs) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for better results
|
||||
"""
|
42
freqtrade/optimize/hyperopt_loss_sharpe.py
Normal file
42
freqtrade/optimize/hyperopt_loss_sharpe.py
Normal file
@ -0,0 +1,42 @@
|
||||
"""
|
||||
IHyperOptLoss interface
|
||||
This module defines the interface for the loss-function for hyperopts
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from pandas import DataFrame
|
||||
import numpy as np
|
||||
|
||||
from freqtrade.optimize.hyperopt import IHyperOptLoss
|
||||
|
||||
|
||||
class SharpeHyperOptLoss(IHyperOptLoss):
|
||||
"""
|
||||
Defines the a loss function for hyperopt.
|
||||
This implementation uses the sharpe ratio calculation.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def hyperopt_loss_function(results: DataFrame, trade_count: int,
|
||||
min_date: datetime, max_date: datetime,
|
||||
*args, **kwargs) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for more optimal results
|
||||
Using sharpe ratio calculation
|
||||
"""
|
||||
total_profit = results.profit_percent
|
||||
days_period = (max_date - min_date).days
|
||||
|
||||
# adding slippage of 0.1% per trade
|
||||
total_profit = total_profit - 0.0005
|
||||
expected_yearly_return = total_profit.sum() / days_period
|
||||
|
||||
if (np.std(total_profit) != 0.):
|
||||
sharp_ratio = expected_yearly_return / np.std(total_profit) * np.sqrt(365)
|
||||
else:
|
||||
# Define high (negative) sharpe ratio to be clear that this is NOT optimal.
|
||||
sharp_ratio = 20.
|
||||
|
||||
# print(expected_yearly_return, np.std(total_profit), sharp_ratio)
|
||||
return -sharp_ratio
|
@ -8,8 +8,9 @@ from pathlib import Path
|
||||
from typing import Optional, Dict
|
||||
|
||||
from freqtrade import OperationalException
|
||||
from freqtrade.constants import DEFAULT_HYPEROPT
|
||||
from freqtrade.constants import DEFAULT_HYPEROPT, DEFAULT_HYPEROPT_LOSS
|
||||
from freqtrade.optimize.hyperopt_interface import IHyperOpt
|
||||
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss
|
||||
from freqtrade.resolvers import IResolver
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@ -54,7 +55,7 @@ class HyperOptResolver(IResolver):
|
||||
current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
|
||||
|
||||
abs_paths = [
|
||||
current_path.parent.parent.joinpath('user_data/hyperopts'),
|
||||
Path.cwd().joinpath('user_data/hyperopts'),
|
||||
current_path,
|
||||
]
|
||||
|
||||
@ -77,3 +78,66 @@ class HyperOptResolver(IResolver):
|
||||
f"Impossible to load Hyperopt '{hyperopt_name}'. This class does not exist "
|
||||
"or contains Python code errors."
|
||||
)
|
||||
|
||||
|
||||
class HyperOptLossResolver(IResolver):
|
||||
"""
|
||||
This class contains all the logic to load custom hyperopt loss class
|
||||
"""
|
||||
|
||||
__slots__ = ['hyperoptloss']
|
||||
|
||||
def __init__(self, config: Optional[Dict] = None) -> None:
|
||||
"""
|
||||
Load the custom class from config parameter
|
||||
:param config: configuration dictionary or None
|
||||
"""
|
||||
config = config or {}
|
||||
|
||||
# Verify the hyperopt is in the configuration, otherwise fallback to the default hyperopt
|
||||
hyperopt_name = config.get('hyperopt_loss') or DEFAULT_HYPEROPT_LOSS
|
||||
self.hyperoptloss = self._load_hyperoptloss(
|
||||
hyperopt_name, extra_dir=config.get('hyperopt_path'))
|
||||
|
||||
# Assign ticker_interval to be used in hyperopt
|
||||
self.hyperoptloss.__class__.ticker_interval = str(config['ticker_interval'])
|
||||
|
||||
if not hasattr(self.hyperoptloss, 'hyperopt_loss_function'):
|
||||
raise OperationalException(
|
||||
f"Found hyperopt {hyperopt_name} does not implement `hyperopt_loss_function`.")
|
||||
|
||||
def _load_hyperoptloss(
|
||||
self, hyper_loss_name: str, extra_dir: Optional[str] = None) -> IHyperOptLoss:
|
||||
"""
|
||||
Search and loads the specified hyperopt loss class.
|
||||
:param hyper_loss_name: name of the module to import
|
||||
:param extra_dir: additional directory to search for the given hyperopt
|
||||
:return: HyperOptLoss instance or None
|
||||
"""
|
||||
current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
|
||||
|
||||
abs_paths = [
|
||||
Path.cwd().joinpath('user_data/hyperopts'),
|
||||
current_path,
|
||||
]
|
||||
|
||||
if extra_dir:
|
||||
# Add extra hyperopt directory on top of search paths
|
||||
abs_paths.insert(0, Path(extra_dir))
|
||||
|
||||
for _path in abs_paths:
|
||||
try:
|
||||
(hyperoptloss, module_path) = self._search_object(directory=_path,
|
||||
object_type=IHyperOptLoss,
|
||||
object_name=hyper_loss_name)
|
||||
if hyperoptloss:
|
||||
logger.info(
|
||||
f"Using resolved hyperopt {hyper_loss_name} from '{module_path}'...")
|
||||
return hyperoptloss
|
||||
except FileNotFoundError:
|
||||
logger.warning('Path "%s" does not exist.', _path.relative_to(Path.cwd()))
|
||||
|
||||
raise OperationalException(
|
||||
f"Impossible to load HyperoptLoss '{hyper_loss_name}'. This class does not exist "
|
||||
"or contains Python code errors."
|
||||
)
|
||||
|
@ -2,20 +2,25 @@
|
||||
import os
|
||||
from datetime import datetime
|
||||
from unittest.mock import MagicMock
|
||||
from filelock import Timeout
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from arrow import Arrow
|
||||
from filelock import Timeout
|
||||
|
||||
from freqtrade import DependencyException
|
||||
from freqtrade.data.converter import parse_ticker_dataframe
|
||||
from freqtrade.data.history import load_tickerdata_file
|
||||
from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
|
||||
from freqtrade.optimize.hyperopt import Hyperopt, HYPEROPT_LOCKFILE
|
||||
from freqtrade.optimize import setup_configuration, start_hyperopt
|
||||
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
|
||||
from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
|
||||
from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss
|
||||
from freqtrade.optimize.hyperopt import (HYPEROPT_LOCKFILE, TICKERDATA_PICKLE,
|
||||
Hyperopt)
|
||||
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
|
||||
from freqtrade.state import RunMode
|
||||
from freqtrade.tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
|
||||
from freqtrade.strategy.interface import SellType
|
||||
from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
|
||||
patch_exchange,
|
||||
patched_configuration_load_config_file)
|
||||
|
||||
|
||||
@ -25,6 +30,21 @@ def hyperopt(default_conf, mocker):
|
||||
return Hyperopt(default_conf)
|
||||
|
||||
|
||||
@pytest.fixture(scope='function')
|
||||
def hyperopt_results():
|
||||
return pd.DataFrame(
|
||||
{
|
||||
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
|
||||
'profit_percent': [0.1, 0.2, 0.3],
|
||||
'profit_abs': [0.2, 0.4, 0.5],
|
||||
'trade_duration': [10, 30, 10],
|
||||
'profit': [2, 0, 0],
|
||||
'loss': [0, 0, 1],
|
||||
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
# Functions for recurrent object patching
|
||||
def create_trials(mocker, hyperopt) -> None:
|
||||
"""
|
||||
@ -166,6 +186,18 @@ def test_hyperoptresolver(mocker, default_conf, caplog) -> None:
|
||||
assert hasattr(x, "ticker_interval")
|
||||
|
||||
|
||||
def test_hyperoptlossresolver(mocker, default_conf, caplog) -> None:
|
||||
|
||||
hl = DefaultHyperOptLoss
|
||||
mocker.patch(
|
||||
'freqtrade.resolvers.hyperopt_resolver.HyperOptLossResolver._load_hyperoptloss',
|
||||
MagicMock(return_value=hl)
|
||||
)
|
||||
x = HyperOptResolver(default_conf, ).hyperopt
|
||||
assert hasattr(x, "populate_indicators")
|
||||
assert hasattr(x, "ticker_interval")
|
||||
|
||||
|
||||
def test_start(mocker, default_conf, caplog) -> None:
|
||||
start_mock = MagicMock()
|
||||
patched_configuration_load_config_file(mocker, default_conf)
|
||||
@ -254,26 +286,54 @@ def test_start_filelock(mocker, default_conf, caplog) -> None:
|
||||
)
|
||||
|
||||
|
||||
def test_loss_calculation_prefer_correct_trade_count(hyperopt) -> None:
|
||||
|
||||
correct = hyperopt.calculate_loss(1, hyperopt.target_trades, 20)
|
||||
over = hyperopt.calculate_loss(1, hyperopt.target_trades + 100, 20)
|
||||
under = hyperopt.calculate_loss(1, hyperopt.target_trades - 100, 20)
|
||||
def test_loss_calculation_prefer_correct_trade_count(default_conf, hyperopt_results) -> None:
|
||||
hl = HyperOptLossResolver(default_conf).hyperoptloss
|
||||
correct = hl.hyperopt_loss_function(hyperopt_results, 600)
|
||||
over = hl.hyperopt_loss_function(hyperopt_results, 600 + 100)
|
||||
under = hl.hyperopt_loss_function(hyperopt_results, 600 - 100)
|
||||
assert over > correct
|
||||
assert under > correct
|
||||
|
||||
|
||||
def test_loss_calculation_prefer_shorter_trades(hyperopt) -> None:
|
||||
shorter = hyperopt.calculate_loss(1, 100, 20)
|
||||
longer = hyperopt.calculate_loss(1, 100, 30)
|
||||
def test_loss_calculation_prefer_shorter_trades(default_conf, hyperopt_results) -> None:
|
||||
resultsb = hyperopt_results.copy()
|
||||
resultsb['trade_duration'][1] = 20
|
||||
|
||||
hl = HyperOptLossResolver(default_conf).hyperoptloss
|
||||
longer = hl.hyperopt_loss_function(hyperopt_results, 100)
|
||||
shorter = hl.hyperopt_loss_function(resultsb, 100)
|
||||
assert shorter < longer
|
||||
|
||||
|
||||
def test_loss_calculation_has_limited_profit(hyperopt) -> None:
|
||||
correct = hyperopt.calculate_loss(hyperopt.expected_max_profit, hyperopt.target_trades, 20)
|
||||
over = hyperopt.calculate_loss(hyperopt.expected_max_profit * 2, hyperopt.target_trades, 20)
|
||||
under = hyperopt.calculate_loss(hyperopt.expected_max_profit / 2, hyperopt.target_trades, 20)
|
||||
assert over == correct
|
||||
def test_loss_calculation_has_limited_profit(default_conf, hyperopt_results) -> None:
|
||||
results_over = hyperopt_results.copy()
|
||||
results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2
|
||||
results_under = hyperopt_results.copy()
|
||||
results_under['profit_percent'] = hyperopt_results['profit_percent'] / 2
|
||||
|
||||
hl = HyperOptLossResolver(default_conf).hyperoptloss
|
||||
correct = hl.hyperopt_loss_function(hyperopt_results, 600)
|
||||
over = hl.hyperopt_loss_function(results_over, 600)
|
||||
under = hl.hyperopt_loss_function(results_under, 600)
|
||||
assert over < correct
|
||||
assert under > correct
|
||||
|
||||
|
||||
def test_sharpe_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
|
||||
results_over = hyperopt_results.copy()
|
||||
results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2
|
||||
results_under = hyperopt_results.copy()
|
||||
results_under['profit_percent'] = hyperopt_results['profit_percent'] / 2
|
||||
|
||||
default_conf.update({'hyperopt_loss': 'SharpeHyperOptLoss'})
|
||||
hl = HyperOptLossResolver(default_conf).hyperoptloss
|
||||
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
|
||||
datetime(2019, 1, 1), datetime(2019, 5, 1))
|
||||
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
|
||||
datetime(2019, 1, 1), datetime(2019, 5, 1))
|
||||
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
|
||||
datetime(2019, 1, 1), datetime(2019, 5, 1))
|
||||
assert over < correct
|
||||
assert under > correct
|
||||
|
||||
|
||||
@ -371,6 +431,11 @@ def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
|
||||
assert dumper.called
|
||||
# Should be called twice, once for tickerdata, once to save evaluations
|
||||
assert dumper.call_count == 2
|
||||
assert hasattr(hyperopt, "advise_sell")
|
||||
assert hasattr(hyperopt, "advise_buy")
|
||||
assert hasattr(hyperopt, "max_open_trades")
|
||||
assert hyperopt.max_open_trades == default_conf['max_open_trades']
|
||||
assert hasattr(hyperopt, "position_stacking")
|
||||
|
||||
|
||||
def test_format_results(hyperopt):
|
||||
@ -468,7 +533,7 @@ def test_generate_optimizer(mocker, default_conf) -> None:
|
||||
)
|
||||
mocker.patch(
|
||||
'freqtrade.optimize.hyperopt.get_timeframe',
|
||||
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
|
||||
MagicMock(return_value=(Arrow(2017, 12, 10), Arrow(2017, 12, 13)))
|
||||
)
|
||||
patch_exchange(mocker)
|
||||
mocker.patch('freqtrade.optimize.hyperopt.load', MagicMock())
|
||||
@ -510,3 +575,36 @@ def test_generate_optimizer(mocker, default_conf) -> None:
|
||||
hyperopt = Hyperopt(default_conf)
|
||||
generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values()))
|
||||
assert generate_optimizer_value == response_expected
|
||||
|
||||
|
||||
def test_clean_hyperopt(mocker, default_conf, caplog):
|
||||
patch_exchange(mocker)
|
||||
default_conf.update({'config': 'config.json.example',
|
||||
'epochs': 1,
|
||||
'timerange': None,
|
||||
'spaces': 'all',
|
||||
'hyperopt_jobs': 1,
|
||||
})
|
||||
mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
|
||||
unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
|
||||
Hyperopt(default_conf)
|
||||
|
||||
assert unlinkmock.call_count == 2
|
||||
assert log_has(f"Removing `{TICKERDATA_PICKLE}`.", caplog.record_tuples)
|
||||
|
||||
|
||||
def test_continue_hyperopt(mocker, default_conf, caplog):
|
||||
patch_exchange(mocker)
|
||||
default_conf.update({'config': 'config.json.example',
|
||||
'epochs': 1,
|
||||
'timerange': None,
|
||||
'spaces': 'all',
|
||||
'hyperopt_jobs': 1,
|
||||
'hyperopt_continue': True
|
||||
})
|
||||
mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
|
||||
unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
|
||||
Hyperopt(default_conf)
|
||||
|
||||
assert unlinkmock.call_count == 0
|
||||
assert log_has(f"Continuing on previous hyperopt results.", caplog.record_tuples)
|
||||
|
@ -1,18 +1,18 @@
|
||||
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
|
||||
|
||||
from functools import reduce
|
||||
from math import exp
|
||||
from typing import Any, Callable, Dict, List
|
||||
from datetime import datetime
|
||||
|
||||
import numpy as np# noqa F401
|
||||
import talib.abstract as ta
|
||||
from pandas import DataFrame
|
||||
from typing import Dict, Any, Callable, List
|
||||
from functools import reduce
|
||||
|
||||
import numpy
|
||||
from skopt.space import Categorical, Dimension, Integer, Real
|
||||
|
||||
import freqtrade.vendor.qtpylib.indicators as qtpylib
|
||||
from freqtrade.optimize.hyperopt_interface import IHyperOpt
|
||||
|
||||
class_name = 'SampleHyperOpts'
|
||||
|
||||
|
||||
# This class is a sample. Feel free to customize it.
|
||||
class SampleHyperOpts(IHyperOpt):
|
||||
|
47
user_data/hyperopts/sample_hyperopt_loss.py
Normal file
47
user_data/hyperopts/sample_hyperopt_loss.py
Normal file
@ -0,0 +1,47 @@
|
||||
from math import exp
|
||||
from datetime import datetime
|
||||
|
||||
from pandas import DataFrame
|
||||
|
||||
from freqtrade.optimize.hyperopt import IHyperOptLoss
|
||||
|
||||
# Define some constants:
|
||||
|
||||
# set TARGET_TRADES to suit your number concurrent trades so its realistic
|
||||
# to the number of days
|
||||
TARGET_TRADES = 600
|
||||
# This is assumed to be expected avg profit * expected trade count.
|
||||
# For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
|
||||
# self.expected_max_profit = 3.85
|
||||
# Check that the reported Σ% values do not exceed this!
|
||||
# Note, this is ratio. 3.85 stated above means 385Σ%.
|
||||
EXPECTED_MAX_PROFIT = 3.0
|
||||
|
||||
# max average trade duration in minutes
|
||||
# if eval ends with higher value, we consider it a failed eval
|
||||
MAX_ACCEPTED_TRADE_DURATION = 300
|
||||
|
||||
|
||||
class SampleHyperOptLoss(IHyperOptLoss):
|
||||
"""
|
||||
Defines the default loss function for hyperopt
|
||||
This is intended to give you some inspiration for your own loss function.
|
||||
|
||||
The Function needs to return a number (float) - which becomes for better backtest results.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def hyperopt_loss_function(results: DataFrame, trade_count: int,
|
||||
min_date: datetime, max_date: datetime,
|
||||
*args, **kwargs) -> float:
|
||||
"""
|
||||
Objective function, returns smaller number for better results
|
||||
"""
|
||||
total_profit = results.profit_percent.sum()
|
||||
trade_duration = results.trade_duration.mean()
|
||||
|
||||
trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
|
||||
profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
|
||||
duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
|
||||
result = trade_loss + profit_loss + duration_loss
|
||||
return result
|
Loading…
Reference in New Issue
Block a user