added daily sharpe ratio hyperopt loss method, ty @djacky (#2826)

* more consistent backtesting tables and labels

* added rounding to Tot Profit % on Sell Reasosn table to be consistent with other percentiles on table.

* added daily sharpe ratio hyperopt loss method, ty @djacky

* removed commented code

* removed unused profit_abs

* added proper slippage to each trade

* replaced use of old value total_profit

* Align quotes in same area

* added daily sharpe ratio test and modified hyperopt_loss_sharpe_daily

* fixed some more line alignments

* updated docs to include SharpeHyperOptLossDaily

* Update dockerfile to 3.8.1

* Run tests against 3.8

* added daily sharpe ratio hyperopt loss method, ty @djacky

* removed commented code

* removed unused profit_abs

* added proper slippage to each trade

* replaced use of old value total_profit

* added daily sharpe ratio test and modified hyperopt_loss_sharpe_daily

* updated docs to include SharpeHyperOptLossDaily

* docs fixes

* missed one fix

* fixed standard deviation line

* fixed to bracket notation

* fixed to bracket notation

* fixed syntax error

* better readability, kept np.sqrt(365) which results in  annualized sharpe ratio

* fixed method arguments indentation

* updated commented out debug print line

* renamed after slippage profit_percent so it wont affect _calculate_results_metrics()

* Reworked to fill leading and trailing days

* No need for np; make flake happy

* Fix risk free rate

Co-authored-by: Matthias <xmatthias@outlook.com>
Co-authored-by: hroff-1902 <47309513+hroff-1902@users.noreply.github.com>

docs/bot-usage.md

@@ -337,8 +337,8 @@ optional arguments:
                         generate completely different results, since the
                         target for optimization is different. Built-in
                         Hyperopt-loss-functions are: DefaultHyperOptLoss,
-                        OnlyProfitHyperOptLoss, SharpeHyperOptLoss (default:
+                        OnlyProfitHyperOptLoss, SharpeHyperOptLoss,
-                        `DefaultHyperOptLoss`).
+                        SharpeHyperOptLossDaily (default: `DefaultHyperOptLoss`).
 
 Common arguments:
   -v, --verbose         Verbose mode (-vv for more, -vvv to get all messages).

docs/hyperopt.md

@@ -57,7 +57,7 @@ Rarely you may also need to override:
 !!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
     You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything (i.e. without creation of a "complete" Hyperopt class with dimensions, parameters, triggers and guards, as described in this document) from the default hyperopt template by relying on your strategy to do most of the calculations.
 
-    ``` python
+    ```python
     # Have a working strategy at hand.
     freqtrade new-hyperopt --hyperopt EmptyHyperopt
 
@@ -75,8 +75,8 @@ Copy the file `user_data/hyperopts/sample_hyperopt.py` into `user_data/hyperopts
 
 There are two places you need to change in your hyperopt file to add a new buy hyperopt for testing:
 
-- Inside `indicator_space()` - the parameters hyperopt shall be optimizing.
+* Inside `indicator_space()` - the parameters hyperopt shall be optimizing.
-- Inside `populate_buy_trend()` - applying the parameters.
+* Inside `populate_buy_trend()` - applying the parameters.
 
 There you have two different types of indicators: 1. `guards` and 2. `triggers`.
 
@@ -141,7 +141,7 @@ one we call `trigger` and use it to decide which buy trigger we want to use.
 
 So let's write the buy strategy using these values:
 
-``` python
+```python
         def populate_buy_trend(dataframe: DataFrame) -> DataFrame:
             conditions = []
             # GUARDS AND TRENDS
@@ -192,6 +192,7 @@ Currently, the following loss functions are builtin:
 * `DefaultHyperOptLoss` (default legacy Freqtrade hyperoptimization loss function)
 * `OnlyProfitHyperOptLoss` (which takes only amount of profit into consideration)
 * `SharpeHyperOptLoss` (optimizes Sharpe Ratio calculated on the trade returns)
+* `SharpeHyperOptLossDaily` (optimizes Sharpe Ratio calculated on daily trade returns)
 
 Creation of a custom loss function is covered in the [Advanced Hyperopt](advanced-hyperopt.md) part of the documentation.
 
@@ -323,7 +324,7 @@ method, what those values match to.
 
 So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
 
-``` python
+```python
 (dataframe['rsi'] < 29.0)
 ```
 
@@ -372,6 +373,7 @@ In order to use this best ROI table found by Hyperopt in backtesting and for liv
         118: 0
     }
 ```
+
 As stated in the comment, you can also use it as the value of the `minimal_roi` setting in the configuration file.
 
 #### Default ROI Search Space
@@ -379,7 +381,7 @@ As stated in the comment, you can also use it as the value of the `minimal_roi`
 If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps). Hyperopt implements adaptive ranges for ROI tables with ranges for values in the ROI steps that depend on the ticker_interval used. By default the values vary in the following ranges (for some of the most used ticker intervals, values are rounded to 5 digits after the decimal point):
 
 | # step | 1m     |                   | 5m       |             | 1h         |                   | 1d           |                   |
-|---|---|---|---|---|---|---|---|---|
+| ------ | ------ | ----------------- | -------- | ----------- | ---------- | ----------------- | ------------ | ----------------- |
 | 1      | 0      | 0.01161...0.11992 | 0        | 0.03...0.31 | 0          | 0.06883...0.71124 | 0            | 0.12178...1.25835 |
 | 2      | 2...8  | 0.00774...0.04255 | 10...40  | 0.02...0.11 | 120...480  | 0.04589...0.25238 | 2880...11520 | 0.08118...0.44651 |
 | 3      | 4...20 | 0.00387...0.01547 | 20...100 | 0.01...0.04 | 240...1200 | 0.02294...0.09177 | 5760...28800 | 0.04059...0.16237 |
@@ -416,6 +418,7 @@ In order to use this best stoploss value found by Hyperopt in backtesting and fo
     # This attribute will be overridden if the config file contains "stoploss"
     stoploss = -0.27996
 ```
+
 As stated in the comment, you can also use it as the value of the `stoploss` setting in the configuration file.
 
 #### Default Stoploss Search Space
@@ -452,6 +455,7 @@ In order to use these best trailing stop parameters found by Hyperopt in backtes
     trailing_stop_positive_offset = 0.06038
     trailing_only_offset_is_reached = True
 ```
+
 As stated in the comment, you can also use it as the values of the corresponding settings in the configuration file.
 
 #### Default Trailing Stop Search Space

freqtrade/commands/cli_options.py

@@ -256,7 +256,7 @@ AVAILABLE_CLI_OPTIONS = {
         help='Specify the class name of the hyperopt loss function class (IHyperOptLoss). '
         'Different functions can generate completely different results, '
         'since the target for optimization is different. Built-in Hyperopt-loss-functions are: '
-        'DefaultHyperOptLoss, OnlyProfitHyperOptLoss, SharpeHyperOptLoss.'
+        'DefaultHyperOptLoss, OnlyProfitHyperOptLoss, SharpeHyperOptLoss, SharpeHyperOptLossDaily.'
         '(default: `%(default)s`).',
         metavar='NAME',
         default=constants.DEFAULT_HYPEROPT_LOSS,

freqtrade/optimize/hyperopt_loss_sharpe_daily.py

@@ -0,0 +1,61 @@
+"""
+SharpeHyperOptLossDaily
+
+This module defines the alternative HyperOptLoss class which can be used for
+Hyperoptimization.
+"""
+import math
+from datetime import datetime
+
+from pandas import DataFrame, date_range
+
+from freqtrade.optimize.hyperopt import IHyperOptLoss
+
+
+class SharpeHyperOptLossDaily(IHyperOptLoss):
+    """
+    Defines the loss function for hyperopt.
+
+    This implementation uses the Sharpe Ratio calculation.
+    """
+
+    @staticmethod
+    def hyperopt_loss_function(results: DataFrame, trade_count: int,
+                               min_date: datetime, max_date: datetime,
+                               *args, **kwargs) -> float:
+        """
+        Objective function, returns smaller number for more optimal results.
+
+        Uses Sharpe Ratio calculation.
+        """
+        resample_freq = '1D'
+        slippage_per_trade_ratio = 0.0005
+        days_in_year = 365
+        annual_risk_free_rate = 0.0
+        risk_free_rate = annual_risk_free_rate / days_in_year
+
+        # apply slippage per trade to profit_percent
+        results.loc[:, 'profit_percent_after_slippage'] = \
+            results['profit_percent'] - slippage_per_trade_ratio
+
+        # create the index within the min_date and end max_date
+        t_index = date_range(start=min_date, end=max_date, freq=resample_freq)
+
+        sum_daily = (
+            results.resample(resample_freq, on='close_time').agg(
+                {"profit_percent_after_slippage": sum}).reindex(t_index).fillna(0)
+        )
+
+        total_profit = sum_daily["profit_percent_after_slippage"] - risk_free_rate
+        expected_returns_mean = total_profit.mean()
+        up_stdev = total_profit.std()
+
+        if (up_stdev != 0.):
+            sharp_ratio = expected_returns_mean / up_stdev * math.sqrt(days_in_year)
+        else:
+            # Define high (negative) sharpe ratio to be clear that this is NOT optimal.
+            sharp_ratio = -20.
+
+        # print(t_index, sum_daily, total_profit)
+        # print(risk_free_rate, expected_returns_mean, up_stdev, sharp_ratio)
+        return -sharp_ratio

tests/optimize/test_hyperopt.py

@@ -42,7 +42,13 @@ def hyperopt_results():
             'profit_percent': [-0.1, 0.2, 0.3],
             'profit_abs': [-0.2, 0.4, 0.6],
             'trade_duration': [10, 30, 10],
-            'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI]
+            'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI],
+            'close_time':
+            [
+                datetime(2019, 1, 1, 9, 26, 3, 478039),
+                datetime(2019, 2, 1, 9, 26, 3, 478039),
+                datetime(2019, 3, 1, 9, 26, 3, 478039)
+            ]
         }
     )
 
@@ -336,6 +342,24 @@ def test_sharpe_loss_prefers_higher_profits(default_conf, hyperopt_results) -> N
     assert under > correct
 
 
+def test_sharpe_loss_daily_prefers_higher_profits(default_conf, hyperopt_results) -> None:
+    results_over = hyperopt_results.copy()
+    results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2
+    results_under = hyperopt_results.copy()
+    results_under['profit_percent'] = hyperopt_results['profit_percent'] / 2
+
+    default_conf.update({'hyperopt_loss': 'SharpeHyperOptLossDaily'})
+    hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
+    correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
+                                        datetime(2019, 1, 1), datetime(2019, 5, 1))
+    over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
+                                     datetime(2019, 1, 1), datetime(2019, 5, 1))
+    under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
+                                      datetime(2019, 1, 1), datetime(2019, 5, 1))
+    assert over < correct
+    assert under > correct
+
+
 def test_onlyprofit_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
     results_over = hyperopt_results.copy()
     results_over['profit_percent'] = hyperopt_results['profit_percent'] * 2