Merge pull request #2024 from freqtrade/custom_hyperopt_loss

Custom hyperopt loss function (and sharpe-ratio)

docs/bot-usage.md

@@ -199,19 +199,22 @@ to find optimal parameter values for your stategy.
 
 ```
 usage: freqtrade hyperopt [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
-                        [--max_open_trades MAX_OPEN_TRADES]
+                          [--max_open_trades INT]
-                        [--stake_amount STAKE_AMOUNT] [-r]
+                          [--stake_amount STAKE_AMOUNT] [-r]
-                        [--customhyperopt NAME] [--eps] [--dmmp] [-e INT]
+                          [--customhyperopt NAME] [--eps] [-e INT]
-                        [-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
+                          [-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
-                        [--print-all] [-j JOBS]
+                          [--dmmp] [--print-all] [-j JOBS]
+                          [--random-state INT] [--min-trades INT] [--continue]
+                          [--hyperopt-loss NAME]
 
 optional arguments:
   -h, --help            show this help message and exit
   -i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
-                        Specify ticker interval (1m, 5m, 30m, 1h, 1d).
+                        Specify ticker interval (`1m`, `5m`, `30m`, `1h`,
+                        `1d`).
   --timerange TIMERANGE
                         Specify what timerange of data to use.
-  --max_open_trades MAX_OPEN_TRADES
+  --max_open_trades INT
                         Specify max_open_trades to use.
   --stake_amount STAKE_AMOUNT
                         Specify stake_amount.
@@ -221,18 +224,18 @@ optional arguments:
                         run your optimization commands with up-to-date data.
   --customhyperopt NAME
                         Specify hyperopt class name (default:
-                        DefaultHyperOpts).
+                        `DefaultHyperOpts`).
   --eps, --enable-position-stacking
                         Allow buying the same pair multiple times (position
                         stacking).
+  -e INT, --epochs INT  Specify number of epochs (default: 100).
+  -s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...], --spaces {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]
+                        Specify which parameters to hyperopt. Space-separated
+                        list. Default: `all`.
   --dmmp, --disable-max-market-positions
                         Disable applying `max_open_trades` during backtest
                         (same as setting `max_open_trades` to a very high
                         number).
-  -e INT, --epochs INT  Specify number of epochs (default: 100).
-  -s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...], --spaces {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]
-                        Specify which parameters to hyperopt. Space separate
-                        list. Default: all.
   --print-all           Print all results, not only the best ones.
   -j JOBS, --job-workers JOBS
                         The number of concurrently running jobs for
@@ -240,6 +243,19 @@ optional arguments:
                         (default), all CPUs are used, for -2, all CPUs but one
                         are used, etc. If 1 is given, no parallel computing
                         code is used at all.
+  --random-state INT    Set random state to some positive integer for
+                        reproducible hyperopt results.
+  --min-trades INT      Set minimal desired number of trades for evaluations
+                        in the hyperopt optimization path (default: 1).
+  --continue            Continue hyperopt from previous runs. By default,
+                        temporary files will be removed and hyperopt will
+                        start from scratch.
+  --hyperopt-loss       NAME
+                        Specify the class name of the hyperopt loss function
+                        class (IHyperOptLoss). Different functions can
+                        generate completely different results, since the
+                        target for optimization is different. (default:
+                        `DefaultHyperOptLoss`).
 ```
 
 ## Edge commands

docs/hyperopt.md

@@ -144,16 +144,85 @@ it will end with telling you which paramter combination produced the best profit
 
 The search for best parameters starts with a few random combinations and then uses a
 regressor algorithm (currently ExtraTreesRegressor) to quickly find a parameter combination
-that minimizes the value of the objective function `calculate_loss` in `hyperopt.py`.
+that minimizes the value of the [loss function](#loss-functions).
 
 The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
 When you want to test an indicator that isn't used by the bot currently, remember to
 add it to the `populate_indicators()` method in `hyperopt.py`.
 
+## Loss-functions
+
+Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results.
+
+By default, FreqTrade uses a loss function, which has been with freqtrade since the beginning and optimizes mostly for short trade duration and avoiding losses.
+
+A different version this can be used by using the `--hyperopt-loss <Class-name>` argument.
+This class should be in it's own file within the `user_data/hyperopts/` directory.
+
+Currently, the following loss functions are builtin: `SharpeHyperOptLoss` and `DefaultHyperOptLoss`.
+
+### Creating and using a custom loss function
+
+To use a custom loss function class, make sure that the function `hyperopt_loss_function` is defined in your custom hyperopt loss class.
+For the sample below, you then need to add the command line parameter `--hyperopt-loss SuperDuperHyperOptLoss` to your hyperopt call so this fuction is being used.
+
+A sample of this can be found below, which is identical to the Default Hyperopt loss implementation. A full sample can be found [user_data/hyperopts/](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_loss.py)
+
+``` python
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+TARGET_TRADES = 600
+EXPECTED_MAX_PROFIT = 3.0
+MAX_ACCEPTED_TRADE_DURATION = 300
+
+class SuperDuperHyperOptLoss(IHyperOptLoss):
+    """
+    Defines the default loss function for hyperopt
+    """
+
+    @staticmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               min_date: datetime, max_date: datetime,
+                               *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for better results
+        This is the legacy algorithm (used until now in freqtrade).
+        Weights are distributed as follows:
+        * 0.4 to trade duration
+        * 0.25: Avoiding trade loss
+        * 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
+        """
+        total_profit = results.profit_percent.sum()
+        trade_duration = results.trade_duration.mean()
+
+        trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
+        profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
+        duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
+        result = trade_loss + profit_loss + duration_loss
+        return result
+```
+
+Currently, the arguments are:
+
+* `results`: DataFrame containing the result  
+    The following columns are available in results (corresponds to the output-file of backtesting when used with `--export trades`):  
+    `pair, profit_percent, profit_abs, open_time, close_time, open_index, close_index, trade_duration, open_at_end, open_rate, close_rate, sell_reason`
+* `trade_count`: Amount of trades (identical to `len(results)`)
+* `min_date`: Start date of the hyperopting TimeFrame
+* `min_date`: End date of the hyperopting TimeFrame
+
+This function needs to return a floating point number (`float`). Smaller numbers will be interpreted as better results. The parameters and balancing for this is up to you.
+
+!!! Note
+    This function is called once per iteration - so please make sure to have this as optimized as possible to not slow hyperopt down unnecessarily.
+
+!!! Note
+    Please keep the arguments `*args` and `**kwargs` in the interface to allow us to extend this interface later.
+
 ## Execute Hyperopt
 
 Once you have updated your hyperopt configuration you can run it.
-Because hyperopt tries a lot of combinations to find the best parameters it will take time you will have the result (more than 30 mins).
+Because hyperopt tries a lot of combinations to find the best parameters it will take time to get a good result. More time usually results in better results.
 
 We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
 
@@ -168,8 +237,11 @@ running at least several thousand evaluations.
 
 The `--spaces all` flag determines that all possible parameters should be optimized. Possibilities are listed below.
 
+!!! Note
+    By default, hyperopt will erase previous results and start from scratch. Continuation can be archived by using `--continue`.
+
 !!! Warning
-    When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file.
+    When switching parameters or changing configuration options, make sure to not use the argument `--continue` so temporary results can be removed.
 
 ### Execute Hyperopt with Different Ticker-Data Source
 
@@ -179,12 +251,11 @@ use data from directory `user_data/data`.
 
 ### Running Hyperopt with Smaller Testset
 
-Use the `--timerange` argument to change how much of the testset
+Use the `--timerange` argument to change how much of the testset you want to use.
-you want to use. The last N ticks/timeframes will be used.
+For example, to use one month of data, pass the following parameter to the hyperopt call:
-Example:
 
 ```bash
-freqtrade hyperopt --timerange -200
+freqtrade hyperopt --timerange 20180401-20180501
 ```
 
 ### Running Hyperopt with Smaller Search Space
@@ -197,14 +268,14 @@ new buy strategy you have.
 
 Legal values are:
 
-- `all`: optimize everything
+* `all`: optimize everything
-- `buy`: just search for a new buy strategy
+* `buy`: just search for a new buy strategy
-- `sell`: just search for a new sell strategy
+* `sell`: just search for a new sell strategy
-- `roi`: just optimize the minimal profit table for your strategy
+* `roi`: just optimize the minimal profit table for your strategy
-- `stoploss`: search for the best stoploss value
+* `stoploss`: search for the best stoploss value
-- space-separated list of any of the above values for example `--spaces roi stoploss`
+* space-separated list of any of the above values for example `--spaces roi stoploss`
 
-### Position stacking and disabling max market positions.
+### Position stacking and disabling max market positions
 
 In some situations, you may need to run Hyperopt (and Backtesting) with the 
 `--eps`/`--enable-position-staking` and `--dmmp`/`--disable-max-market-positions` arguments.
@@ -252,7 +323,7 @@ method, what those values match to.
 
 So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
 
-```
+``` python
 (dataframe['rsi'] < 29.0)
 ```
 

freqtrade/configuration/arguments.py

@@ -213,6 +213,21 @@ AVAILABLE_CLI_OPTIONS = {
         metavar='INT',
         default=1,
     ),
+    "hyperopt_continue": Arg(
+        "--continue",
+        help="Continue hyperopt from previous runs. "
+        "By default, temporary files will be removed and hyperopt will start from scratch.",
+        default=False,
+        action='store_true',
+    ),
+    "hyperopt_loss": Arg(
+        '--hyperopt-loss',
+        help='Specify the class name of the hyperopt loss function class (IHyperOptLoss). '
+        'Different functions can generate completely different results, '
+        'since the target for optimization is different. (default: `%(default)s`).',
+        metavar='NAME',
+        default=constants.DEFAULT_HYPEROPT_LOSS,
+    ),
     # List exchanges
     "print_one_column": Arg(
         '-1', '--one-column',
@@ -299,7 +314,8 @@ ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_pos
 
 ARGS_HYPEROPT = ARGS_COMMON_OPTIMIZE + ["hyperopt", "position_stacking", "epochs", "spaces",
                                         "use_max_market_positions", "print_all", "hyperopt_jobs",
-                                        "hyperopt_random_state", "hyperopt_min_trades"]
+                                        "hyperopt_random_state", "hyperopt_min_trades",
+                                        "hyperopt_continue", "hyperopt_loss"]
 
 ARGS_EDGE = ARGS_COMMON_OPTIMIZE + ["stoploss_range"]
 

freqtrade/configuration/configuration.py

@@ -259,6 +259,12 @@ class Configuration(object):
         self._args_to_config(config, argname='hyperopt_min_trades',
                              logstring='Parameter --min-trades detected: {}')
 
+        self._args_to_config(config, argname='hyperopt_continue',
+                             logstring='Hyperopt continue: {}')
+
+        self._args_to_config(config, argname='hyperopt_loss',
+                             logstring='Using loss function: {}')
+
     def _process_plot_options(self, config: Dict[str, Any]) -> None:
 
         self._args_to_config(config, argname='pairs',

freqtrade/constants.py

@@ -12,6 +12,7 @@ HYPEROPT_EPOCH = 100  # epochs
 RETRY_TIMEOUT = 30  # sec
 DEFAULT_STRATEGY = 'DefaultStrategy'
 DEFAULT_HYPEROPT = 'DefaultHyperOpts'
+DEFAULT_HYPEROPT_LOSS = 'DefaultHyperOptLoss'
 DEFAULT_DB_PROD_URL = 'sqlite:///tradesv3.sqlite'
 DEFAULT_DB_DRYRUN_URL = 'sqlite://'
 UNLIMITED_STAKE_AMOUNT = 'unlimited'

freqtrade/optimize/default_hyperopt.py

@@ -1,17 +1,15 @@
 # pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
 
+from functools import reduce
+from typing import Any, Callable, Dict, List
+
 import talib.abstract as ta
 from pandas import DataFrame
-from typing import Dict, Any, Callable, List
-from functools import reduce
-
 from skopt.space import Categorical, Dimension, Integer, Real
 
 import freqtrade.vendor.qtpylib.indicators as qtpylib
 from freqtrade.optimize.hyperopt_interface import IHyperOpt
 
-class_name = 'DefaultHyperOpts'
-
 
 class DefaultHyperOpts(IHyperOpt):
     """

freqtrade/optimize/default_hyperopt_loss.py

@@ -0,0 +1,53 @@
+"""
+DefaultHyperOptLoss
+This module defines the default HyperoptLoss class which is being used for
+Hyperoptimization.
+"""
+
+from math import exp
+
+from pandas import DataFrame
+
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+# Define some constants:
+
+# set TARGET_TRADES to suit your number concurrent trades so its realistic
+# to the number of days
+TARGET_TRADES = 600
+# This is assumed to be expected avg profit * expected trade count.
+# For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
+# self.expected_max_profit = 3.85
+# Check that the reported Σ% values do not exceed this!
+# Note, this is ratio. 3.85 stated above means 385Σ%.
+EXPECTED_MAX_PROFIT = 3.0
+
+# max average trade duration in minutes
+# if eval ends with higher value, we consider it a failed eval
+MAX_ACCEPTED_TRADE_DURATION = 300
+
+
+class DefaultHyperOptLoss(IHyperOptLoss):
+    """
+    Defines the default loss function for hyperopt
+    """
+
+    @staticmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for better results
+        This is the Default algorithm
+        Weights are distributed as follows:
+        * 0.4 to trade duration
+        * 0.25: Avoiding trade loss
+        * 1.0 to total profit, compared to the expected value (`EXPECTED_MAX_PROFIT`) defined above
+        """
+        total_profit = results.profit_percent.sum()
+        trade_duration = results.trade_duration.mean()
+
+        trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
+        profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
+        duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
+        result = trade_loss + profit_loss + duration_loss
+        return result

freqtrade/optimize/hyperopt.py

@@ -7,7 +7,7 @@ This module contains the hyperopt logic
 import logging
 import os
 import sys
-from math import exp
+
 from operator import itemgetter
 from pathlib import Path
 from pprint import pprint
@@ -21,7 +21,9 @@ from skopt.space import Dimension
 from freqtrade.configuration import Arguments
 from freqtrade.data.history import load_data, get_timeframe
 from freqtrade.optimize.backtesting import Backtesting
-from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
+# Import IHyperOptLoss to allow users import from this file
+from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss  # noqa: F4
+from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
 
 
 logger = logging.getLogger(__name__)
@@ -46,27 +48,46 @@ class Hyperopt(Backtesting):
         super().__init__(config)
         self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
 
-        # set TARGET_TRADES to suit your number concurrent trades so its realistic
+        self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
-        # to the number of days
+        self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
-        self.target_trades = 600
+
         self.total_tries = config.get('epochs', 0)
         self.current_best_loss = 100
 
-        # max average trade duration in minutes
+        if not self.config.get('hyperopt_continue'):
-        # if eval ends with higher value, we consider it a failed eval
+            self.clean_hyperopt()
-        self.max_accepted_trade_duration = 300
+        else:
-
+            logger.info("Continuing on previous hyperopt results.")
-        # This is assumed to be expected avg profit * expected trade count.
-        # For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
-        # self.expected_max_profit = 3.85
-        # Check that the reported Σ% values do not exceed this!
-        # Note, this is ratio. 3.85 stated above means 385Σ%.
-        self.expected_max_profit = 3.0
 
         # Previous evaluations
         self.trials_file = TRIALSDATA_PICKLE
         self.trials: List = []
 
+        # Populate functions here (hasattr is slow so should not be run during "regular" operations)
+        if hasattr(self.custom_hyperopt, 'populate_buy_trend'):
+            self.advise_buy = self.custom_hyperopt.populate_buy_trend  # type: ignore
+
+        if hasattr(self.custom_hyperopt, 'populate_sell_trend'):
+            self.advise_sell = self.custom_hyperopt.populate_sell_trend  # type: ignore
+
+            # Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
+        if self.config.get('use_max_market_positions', True):
+            self.max_open_trades = self.config['max_open_trades']
+        else:
+            logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
+            self.max_open_trades = 0
+        self.position_stacking = self.config.get('position_stacking', False),
+
+    def clean_hyperopt(self):
+        """
+        Remove hyperopt pickle files to restart hyperopt.
+        """
+        for f in [TICKERDATA_PICKLE, TRIALSDATA_PICKLE]:
+            p = Path(f)
+            if p.is_file():
+                logger.info(f"Removing `{p}`.")
+                p.unlink()
+
     def get_args(self, params):
         dimensions = self.hyperopt_space()
         # Ensure the number of dimensions match
@@ -134,16 +155,6 @@ class Hyperopt(Backtesting):
             print('.', end='')
             sys.stdout.flush()
 
-    def calculate_loss(self, total_profit: float, trade_count: int, trade_duration: float) -> float:
-        """
-        Objective function, returns smaller number for more optimal results
-        """
-        trade_loss = 1 - 0.25 * exp(-(trade_count - self.target_trades) ** 2 / 10 ** 5.8)
-        profit_loss = max(0, 1 - total_profit / self.expected_max_profit)
-        duration_loss = 0.4 * min(trade_duration / self.max_accepted_trade_duration, 1)
-        result = trade_loss + profit_loss + duration_loss
-        return result
-
     def has_space(self, space: str) -> bool:
         """
         Tell if a space value is contained in the configuration
@@ -172,49 +183,40 @@ class Hyperopt(Backtesting):
         return spaces
 
     def generate_optimizer(self, _params: Dict) -> Dict:
+        """
+        Used Optimize function. Called once per epoch to optimize whatever is configured.
+        Keep this function as optimized as possible!
+        """
         params = self.get_args(_params)
         if self.has_space('roi'):
             self.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params)
 
         if self.has_space('buy'):
             self.advise_buy = self.custom_hyperopt.buy_strategy_generator(params)
-        elif hasattr(self.custom_hyperopt, 'populate_buy_trend'):
-            self.advise_buy = self.custom_hyperopt.populate_buy_trend  # type: ignore
 
         if self.has_space('sell'):
             self.advise_sell = self.custom_hyperopt.sell_strategy_generator(params)
-        elif hasattr(self.custom_hyperopt, 'populate_sell_trend'):
-            self.advise_sell = self.custom_hyperopt.populate_sell_trend  # type: ignore
 
         if self.has_space('stoploss'):
             self.strategy.stoploss = params['stoploss']
 
         processed = load(TICKERDATA_PICKLE)
 
-        # Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
-        if self.config.get('use_max_market_positions', True):
-            max_open_trades = self.config['max_open_trades']
-        else:
-            logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
-            max_open_trades = 0
-
         min_date, max_date = get_timeframe(processed)
 
         results = self.backtest(
             {
                 'stake_amount': self.config['stake_amount'],
                 'processed': processed,
-                'max_open_trades': max_open_trades,
+                'max_open_trades': self.max_open_trades,
-                'position_stacking': self.config.get('position_stacking', False),
+                'position_stacking': self.position_stacking,
                 'start_date': min_date,
                 'end_date': max_date,
             }
         )
         result_explanation = self.format_results(results)
 
-        total_profit = results.profit_percent.sum()
         trade_count = len(results.index)
-        trade_duration = results.trade_duration.mean()
 
         # If this evaluation contains too short amount of trades to be
         # interesting -- consider it as 'bad' (assigned max. loss value)
@@ -227,7 +229,8 @@ class Hyperopt(Backtesting):
                 'result': result_explanation,
             }
 
-        loss = self.calculate_loss(total_profit, trade_count, trade_duration)
+        loss = self.calculate_loss(results=results, trade_count=trade_count,
+                                   min_date=min_date.datetime, max_date=max_date.datetime)
 
         return {
             'loss': loss,

freqtrade/optimize/hyperopt_loss_interface.py

@@ -0,0 +1,25 @@
+"""
+IHyperOptLoss interface
+This module defines the interface for the loss-function for hyperopts
+"""
+
+from abc import ABC, abstractmethod
+from datetime import datetime
+
+from pandas import DataFrame
+
+
+class IHyperOptLoss(ABC):
+    """
+    Interface for freqtrade hyperopts Loss functions.
+    Defines the custom loss function (`hyperopt_loss_function()` which is evaluated every epoch.)
+    """
+    ticker_interval: str
+
+    @staticmethod
+    @abstractmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               min_date: datetime, max_date: datetime, *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for better results
+        """

freqtrade/optimize/hyperopt_loss_sharpe.py

@@ -0,0 +1,42 @@
+"""
+IHyperOptLoss interface
+This module defines the interface for the loss-function for hyperopts
+"""
+
+from datetime import datetime
+
+from pandas import DataFrame
+import numpy as np
+
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+
+class SharpeHyperOptLoss(IHyperOptLoss):
+    """
+    Defines the a loss function for hyperopt.
+    This implementation uses the sharpe ratio calculation.
+    """
+
+    @staticmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               min_date: datetime, max_date: datetime,
+                               *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for more optimal results
+        Using sharpe ratio calculation
+        """
+        total_profit = results.profit_percent
+        days_period = (max_date - min_date).days
+
+        # adding slippage of 0.1% per trade
+        total_profit = total_profit - 0.0005
+        expected_yearly_return = total_profit.sum() / days_period
+
+        if (np.std(total_profit) != 0.):
+            sharp_ratio = expected_yearly_return / np.std(total_profit) * np.sqrt(365)
+        else:
+            # Define high (negative) sharpe ratio to be clear that this is NOT optimal.
+            sharp_ratio = 20.
+
+        # print(expected_yearly_return, np.std(total_profit), sharp_ratio)
+        return -sharp_ratio

freqtrade/resolvers/hyperopt_resolver.py

@@ -8,8 +8,9 @@ from pathlib import Path
 from typing import Optional, Dict
 
 from freqtrade import OperationalException
-from freqtrade.constants import DEFAULT_HYPEROPT
+from freqtrade.constants import DEFAULT_HYPEROPT, DEFAULT_HYPEROPT_LOSS
 from freqtrade.optimize.hyperopt_interface import IHyperOpt
+from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss
 from freqtrade.resolvers import IResolver
 
 logger = logging.getLogger(__name__)
@@ -54,7 +55,7 @@ class HyperOptResolver(IResolver):
         current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
 
         abs_paths = [
-            current_path.parent.parent.joinpath('user_data/hyperopts'),
+            Path.cwd().joinpath('user_data/hyperopts'),
             current_path,
         ]
 
@@ -77,3 +78,66 @@ class HyperOptResolver(IResolver):
             f"Impossible to load Hyperopt '{hyperopt_name}'. This class does not exist "
             "or contains Python code errors."
         )
+
+
+class HyperOptLossResolver(IResolver):
+    """
+    This class contains all the logic to load custom hyperopt loss class
+    """
+
+    __slots__ = ['hyperoptloss']
+
+    def __init__(self, config: Optional[Dict] = None) -> None:
+        """
+        Load the custom class from config parameter
+        :param config: configuration dictionary or None
+        """
+        config = config or {}
+
+        # Verify the hyperopt is in the configuration, otherwise fallback to the default hyperopt
+        hyperopt_name = config.get('hyperopt_loss') or DEFAULT_HYPEROPT_LOSS
+        self.hyperoptloss = self._load_hyperoptloss(
+            hyperopt_name, extra_dir=config.get('hyperopt_path'))
+
+        # Assign ticker_interval to be used in hyperopt
+        self.hyperoptloss.__class__.ticker_interval = str(config['ticker_interval'])
+
+        if not hasattr(self.hyperoptloss, 'hyperopt_loss_function'):
+            raise OperationalException(
+                f"Found hyperopt {hyperopt_name} does not implement `hyperopt_loss_function`.")
+
+    def _load_hyperoptloss(
+            self, hyper_loss_name: str, extra_dir: Optional[str] = None) -> IHyperOptLoss:
+        """
+        Search and loads the specified hyperopt loss class.
+        :param hyper_loss_name: name of the module to import
+        :param extra_dir: additional directory to search for the given hyperopt
+        :return: HyperOptLoss instance or None
+        """
+        current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
+
+        abs_paths = [
+            Path.cwd().joinpath('user_data/hyperopts'),
+            current_path,
+        ]
+
+        if extra_dir:
+            # Add extra hyperopt directory on top of search paths
+            abs_paths.insert(0, Path(extra_dir))
+
+        for _path in abs_paths:
+            try:
+                (hyperoptloss, module_path) = self._search_object(directory=_path,
+                                                                  object_type=IHyperOptLoss,
+                                                                  object_name=hyper_loss_name)
+                if hyperoptloss:
+                    logger.info(
+                        f"Using resolved hyperopt {hyper_loss_name} from '{module_path}'...")
+                    return hyperoptloss
+            except FileNotFoundError:
+                logger.warning('Path "%s" does not exist.', _path.relative_to(Path.cwd()))
+
+        raise OperationalException(
+            f"Impossible to load HyperoptLoss '{hyper_loss_name}'. This class does not exist "
+            "or contains Python code errors."
+        )

freqtrade/tests/optimize/test_hyperopt.py

@@ -2,20 +2,25 @@
 import os
 from datetime import datetime
 from unittest.mock import MagicMock
-from filelock import Timeout
 
 import pandas as pd
 import pytest
+from arrow import Arrow
+from filelock import Timeout
 
 from freqtrade import DependencyException
 from freqtrade.data.converter import parse_ticker_dataframe
 from freqtrade.data.history import load_tickerdata_file
-from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
-from freqtrade.optimize.hyperopt import Hyperopt, HYPEROPT_LOCKFILE
 from freqtrade.optimize import setup_configuration, start_hyperopt
-from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
+from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
+from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss
+from freqtrade.optimize.hyperopt import (HYPEROPT_LOCKFILE, TICKERDATA_PICKLE,
+                                         Hyperopt)
+from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
 from freqtrade.state import RunMode
-from freqtrade.tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
+from freqtrade.strategy.interface import SellType
+from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
+                                      patch_exchange,
                                       patched_configuration_load_config_file)
 
 
@@ -25,6 +30,21 @@ def hyperopt(default_conf, mocker):
     return Hyperopt(default_conf)
 
 
+@pytest.fixture(scope='function')
+def hyperopt_results():
+    return pd.DataFrame(
+        {
+            'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
+            'profit_percent': [0.1, 0.2, 0.3],
+            'profit_abs': [0.2, 0.4, 0.5],
+            'trade_duration': [10, 30, 10],
+            'profit': [2, 0, 0],
+            'loss': [0, 0, 1],
+            'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
+        }
+    )
+
+
 # Functions for recurrent object patching
 def create_trials(mocker, hyperopt) -> None:
     """
@@ -166,6 +186,18 @@ def test_hyperoptresolver(mocker, default_conf, caplog) -> None:
     assert hasattr(x, "ticker_interval")
 
 
+def test_hyperoptlossresolver(mocker, default_conf, caplog) -> None:
+
+    hl = DefaultHyperOptLoss
+    mocker.patch(
+        'freqtrade.resolvers.hyperopt_resolver.HyperOptLossResolver._load_hyperoptloss',
+        MagicMock(return_value=hl)
+    )
+    x = HyperOptResolver(default_conf, ).hyperopt
+    assert hasattr(x, "populate_indicators")
+    assert hasattr(x, "ticker_interval")
+
+
 def test_start(mocker, default_conf, caplog) -> None:
     start_mock = MagicMock()
     patched_configuration_load_config_file(mocker, default_conf)
@@ -254,26 +286,54 @@ def test_start_filelock(mocker, default_conf, caplog) -> None:
     )
 
 
-def test_loss_calculation_prefer_correct_trade_count(hyperopt) -> None:
+def test_loss_calculation_prefer_correct_trade_count(default_conf, hyperopt_results) -> None:
-
+    hl = HyperOptLossResolver(default_conf).hyperoptloss
-    correct = hyperopt.calculate_loss(1, hyperopt.target_trades, 20)
+    correct = hl.hyperopt_loss_function(hyperopt_results, 600)
-    over = hyperopt.calculate_loss(1, hyperopt.target_trades + 100, 20)
+    over = hl.hyperopt_loss_function(hyperopt_results, 600 + 100)
-    under = hyperopt.calculate_loss(1, hyperopt.target_trades - 100, 20)
+    under = hl.hyperopt_loss_function(hyperopt_results, 600 - 100)
     assert over > correct
     assert under > correct
 
 
-def test_loss_calculation_prefer_shorter_trades(hyperopt) -> None:
+def test_loss_calculation_prefer_shorter_trades(default_conf, hyperopt_results) -> None:
-    shorter = hyperopt.calculate_loss(1, 100, 20)
+    resultsb = hyperopt_results.copy()
-    longer = hyperopt.calculate_loss(1, 100, 30)
+    resultsb['trade_duration'][1] = 20
+
+    hl = HyperOptLossResolver(default_conf).hyperoptloss
+    longer = hl.hyperopt_loss_function(hyperopt_results, 100)
+    shorter = hl.hyperopt_loss_function(resultsb, 100)
     assert shorter < longer
 
 
-def test_loss_calculation_has_limited_profit(hyperopt) -> None:
+def test_loss_calculation_has_limited_profit(default_conf, hyperopt_results) -> None:
-    correct = hyperopt.calculate_loss(hyperopt.expected_max_profit, hyperopt.target_trades, 20)
+    results_over = hyperopt_results.copy()
-    over = hyperopt.calculate_loss(hyperopt.expected_max_profit * 2, hyperopt.target_trades, 20)
+    results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2
-    under = hyperopt.calculate_loss(hyperopt.expected_max_profit / 2, hyperopt.target_trades, 20)
+    results_under = hyperopt_results.copy()
-    assert over == correct
+    results_under['profit_percent'] = hyperopt_results['profit_percent'] / 2
+
+    hl = HyperOptLossResolver(default_conf).hyperoptloss
+    correct = hl.hyperopt_loss_function(hyperopt_results, 600)
+    over = hl.hyperopt_loss_function(results_over, 600)
+    under = hl.hyperopt_loss_function(results_under, 600)
+    assert over < correct
+    assert under > correct
+
+
+def test_sharpe_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
+    results_over = hyperopt_results.copy()
+    results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2
+    results_under = hyperopt_results.copy()
+    results_under['profit_percent'] = hyperopt_results['profit_percent'] / 2
+
+    default_conf.update({'hyperopt_loss': 'SharpeHyperOptLoss'})
+    hl = HyperOptLossResolver(default_conf).hyperoptloss
+    correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
+                                        datetime(2019, 1, 1), datetime(2019, 5, 1))
+    over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
+                                     datetime(2019, 1, 1), datetime(2019, 5, 1))
+    under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
+                                      datetime(2019, 1, 1), datetime(2019, 5, 1))
+    assert over < correct
     assert under > correct
 
 
@@ -371,6 +431,11 @@ def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
     assert dumper.called
     # Should be called twice, once for tickerdata, once to save evaluations
     assert dumper.call_count == 2
+    assert hasattr(hyperopt, "advise_sell")
+    assert hasattr(hyperopt, "advise_buy")
+    assert hasattr(hyperopt, "max_open_trades")
+    assert hyperopt.max_open_trades == default_conf['max_open_trades']
+    assert hasattr(hyperopt, "position_stacking")
 
 
 def test_format_results(hyperopt):
@@ -468,7 +533,7 @@ def test_generate_optimizer(mocker, default_conf) -> None:
     )
     mocker.patch(
         'freqtrade.optimize.hyperopt.get_timeframe',
-        MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
+        MagicMock(return_value=(Arrow(2017, 12, 10), Arrow(2017, 12, 13)))
     )
     patch_exchange(mocker)
     mocker.patch('freqtrade.optimize.hyperopt.load', MagicMock())
@@ -510,3 +575,36 @@ def test_generate_optimizer(mocker, default_conf) -> None:
     hyperopt = Hyperopt(default_conf)
     generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values()))
     assert generate_optimizer_value == response_expected
+
+
+def test_clean_hyperopt(mocker, default_conf, caplog):
+    patch_exchange(mocker)
+    default_conf.update({'config': 'config.json.example',
+                         'epochs': 1,
+                         'timerange': None,
+                         'spaces': 'all',
+                         'hyperopt_jobs': 1,
+                         })
+    mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
+    unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
+    Hyperopt(default_conf)
+
+    assert unlinkmock.call_count == 2
+    assert log_has(f"Removing `{TICKERDATA_PICKLE}`.", caplog.record_tuples)
+
+
+def test_continue_hyperopt(mocker, default_conf, caplog):
+    patch_exchange(mocker)
+    default_conf.update({'config': 'config.json.example',
+                         'epochs': 1,
+                         'timerange': None,
+                         'spaces': 'all',
+                         'hyperopt_jobs': 1,
+                         'hyperopt_continue': True
+                         })
+    mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
+    unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
+    Hyperopt(default_conf)
+
+    assert unlinkmock.call_count == 0
+    assert log_has(f"Continuing on previous hyperopt results.", caplog.record_tuples)

user_data/hyperopts/sample_hyperopt.py

@@ -1,18 +1,18 @@
 # pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
 
+from functools import reduce
+from math import exp
+from typing import Any, Callable, Dict, List
+from datetime import datetime
+
+import numpy as np# noqa F401
 import talib.abstract as ta
 from pandas import DataFrame
-from typing import Dict, Any, Callable, List
-from functools import reduce
-
-import numpy
 from skopt.space import Categorical, Dimension, Integer, Real
 
 import freqtrade.vendor.qtpylib.indicators as qtpylib
 from freqtrade.optimize.hyperopt_interface import IHyperOpt
 
-class_name = 'SampleHyperOpts'
-
 
 # This class is a sample. Feel free to customize it.
 class SampleHyperOpts(IHyperOpt):

user_data/hyperopts/sample_hyperopt_loss.py

@@ -0,0 +1,47 @@
+from math import exp
+from datetime import datetime
+
+from pandas import DataFrame
+
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+# Define some constants:
+
+# set TARGET_TRADES to suit your number concurrent trades so its realistic
+# to the number of days
+TARGET_TRADES = 600
+# This is assumed to be expected avg profit * expected trade count.
+# For example, for 0.35% avg per trade (or 0.0035 as ratio) and 1100 trades,
+# self.expected_max_profit = 3.85
+# Check that the reported Σ% values do not exceed this!
+# Note, this is ratio. 3.85 stated above means 385Σ%.
+EXPECTED_MAX_PROFIT = 3.0
+
+# max average trade duration in minutes
+# if eval ends with higher value, we consider it a failed eval
+MAX_ACCEPTED_TRADE_DURATION = 300
+
+
+class SampleHyperOptLoss(IHyperOptLoss):
+    """
+    Defines the default loss function for hyperopt
+    This is intended to give you some inspiration for your own loss function.
+
+    The Function needs to return a number (float) - which becomes for better backtest results.
+    """
+
+    @staticmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               min_date: datetime, max_date: datetime,
+                               *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for better results
+        """
+        total_profit = results.profit_percent.sum()
+        trade_duration = results.trade_duration.mean()
+
+        trade_loss = 1 - 0.25 * exp(-(trade_count - TARGET_TRADES) ** 2 / 10 ** 5.8)
+        profit_loss = max(0, 1 - total_profit / EXPECTED_MAX_PROFIT)
+        duration_loss = 0.4 * min(trade_duration / MAX_ACCEPTED_TRADE_DURATION, 1)
+        result = trade_loss + profit_loss + duration_loss
+        return result