Update hyperopt documentation

docs/hyperopt.md

@@ -37,12 +37,20 @@ pip install -r requirements-hyperopt.txt
 Before we start digging into Hyperopt, we recommend you to take a look at
 the sample hyperopt file located in [user_data/hyperopts/](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt.py).
 
-Configuring hyperopt is similar to writing your own strategy, and many tasks will be similar and a lot of code can be copied across from the strategy.
+Configuring hyperopt is similar to writing your own strategy, and many tasks will be similar.
 
-The simplest way to get started is to use `freqtrade new-hyperopt --hyperopt AwesomeHyperopt`.
+!!! Tip "About this page"
-This will create a new hyperopt file from a template, which will be located under `user_data/hyperopts/AwesomeHyperopt.py`.
+    For this page, we will be using a fictional strategy called `AwesomeStrategy` - which will be optimized using the `AwesomeHyperopt` class.
 
-### Checklist on all tasks / possibilities in hyperopt
+The simplest way to get started is to use the following, command, which will create a new hyperopt file from a template, which will be located under `user_data/hyperopts/AwesomeHyperopt.py`.
+
+``` bash
+freqtrade new-hyperopt --hyperopt AwesomeHyperopt
+```
+
+### Hyperopt checklist
+
+Checklist on all tasks / possibilities in hyperopt
 
 Depending on the space you want to optimize, only some of the below are required:
 
@@ -54,17 +62,15 @@ Depending on the space you want to optimize, only some of the below are required
 !!! Note
     `populate_indicators` needs to create all indicators any of thee spaces may use, otherwise hyperopt will not work.
 
-Optional - can also be loaded from a strategy:
+Optional in hyperopt - can also be loaded from a strategy (recommended):
 
 * copy `populate_indicators` from your strategy - otherwise default-strategy will be used
 * copy `populate_buy_trend` from your strategy - otherwise default-strategy will be used
 * copy `populate_sell_trend` from your strategy - otherwise default-strategy will be used
 
-!!! Note
-    Assuming the optional methods are not in your hyperopt file, please use `--strategy AweSomeStrategy` which contains these methods so hyperopt can use these methods instead.
-
 !!! Note
     You always have to provide a strategy to Hyperopt, even if your custom Hyperopt class contains all methods.
+    Assuming the optional methods are not in your hyperopt file, please use `--strategy AweSomeStrategy` which contains these methods so hyperopt can use these methods instead.
 
 Rarely you may also need to override:
 
@@ -83,14 +89,17 @@ Rarely you may also need to override:
     freqtrade hyperopt --hyperopt EmptyHyperopt --hyperopt-loss SharpeHyperOptLossDaily --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100
     ```
 
-### 1. Install a Custom Hyperopt File
+### Create a Custom Hyperopt File
 
-Put your hyperopt file into the directory `user_data/hyperopts`.
+Let assume you want a hyperopt file `AwesomeHyperopt.py`:  
 
-Let assume you want a hyperopt file `awesome_hyperopt.py`:  
+``` bash
-Copy the file `user_data/hyperopts/sample_hyperopt.py` into `user_data/hyperopts/awesome_hyperopt.py`
+freqtrade new-hyperopt --hyperopt AwesomeHyperopt
+```
 
-### 2. Configure your Guards and Triggers
+This command will create a new hyperopt file from a template, allowing you to get started quickly.
+
+### Configure your Guards and Triggers
 
 There are two places you need to change in your hyperopt file to add a new buy hyperopt for testing:
 
@@ -102,14 +111,16 @@ There you have two different types of indicators: 1. `guards` and 2. `triggers`.
 1. Guards are conditions like "never buy if ADX < 10", or never buy if current price is over EMA10.
 2. Triggers are ones that actually trigger buy in specific moment, like "buy when EMA5 crosses over EMA10" or "buy when close price touches lower Bollinger band".
 
+!!! Hint "Guards and Triggers"
+    Technically, there is no difference between Guards and Triggers.  
+    However, this guide will make this distinction to make it clear that signals should not be "sticking".
+    Sticking signals are signals that are active for multiple candles. This can lead into buying a signal late (right before the signal disappears - which means that the chance of success is a lot lower than right at the beginning).
+
 Hyper-optimization will, for each epoch round, pick one trigger and possibly
-multiple guards. The constructed strategy will be something like
+multiple guards. The constructed strategy will be something like "*buy exactly when close price touches lower Bollinger band, BUT only if
-"*buy exactly when close price touches lower Bollinger band, BUT only if
 ADX > 10*".
 
-If you have updated the buy strategy, i.e. changed the contents of
+If you have updated the buy strategy, i.e. changed the contents of `populate_buy_trend()` method, you have to update the `guards` and `triggers` your hyperopt must use correspondingly.
-`populate_buy_trend()` method, you have to update the `guards` and
-`triggers` your hyperopt must use correspondingly.
 
 #### Sell optimization
 
@@ -192,14 +203,14 @@ So let's write the buy strategy using these values:
         return populate_buy_trend
 ```
 
-Hyperopting will now call this `populate_buy_trend` as many times you ask it (`epochs`)
+Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations.  
-with different value combinations. It will then use the given historical data and make
+It will use the given historical data and make buys based on the buy signals generated with the above function.  
-buys based on the buy signals generated with the above function and based on the results
+Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured [loss function](#loss-functions)).
-it will end with telling you which parameter combination produced the best profits.
 
-The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
+!!! Note
-When you want to test an indicator that isn't used by the bot currently, remember to
+    The above setup expects to find ADX, RSI and Bollinger Bands in the populated indicators.
-add it to the `populate_indicators()` method in your custom hyperopt file.
+    When you want to test an indicator that isn't used by the bot currently, remember to
+    add it to the `populate_indicators()` method in your strategy or hyperopt file.
 
 ## Loss-functions
 
@@ -232,14 +243,15 @@ freqtrade hyperopt --config config.json --hyperopt <hyperoptname> --hyperopt-los
 
 Use `<hyperoptname>` as the name of the custom hyperopt used.
 
-The `-e` option will set how many evaluations hyperopt will do. We recommend
+The `-e` option will set how many evaluations hyperopt will do. Since hyperopt uses Bayesian search, running too many epochs at once may not produce greater results. Experience has shown that best results are usually not improving much after 500-1000 epochs.  
-running at least several thousand evaluations.
+Doing multiple runs (executions) with a few 1000 epochs and different random state will most likely produce different results.
 
 The `--spaces all` option determines that all possible parameters should be optimized. Possibilities are listed below.
 
 !!! Note
     Hyperopt will store hyperopt results with the timestamp of the hyperopt start time.
     Reading commands (`hyperopt-list`, `hyperopt-show`) can use `--hyperopt-filename <filename>` to read and display older hyperopt results.
+    You can find a list of filenames with `ls -l user_data/hyperopt_results/`.
 
 ### Execute Hyperopt with different historical data source
 
@@ -247,7 +259,7 @@ If you would like to hyperopt parameters using an alternate historical data set
 you have on-disk, use the `--datadir PATH` option. By default, hyperopt
 uses data from directory `user_data/data`.
 
-### Running Hyperopt with Smaller Testset
+### Running Hyperopt with a smaller test-set
 
 Use the `--timerange` argument to change how much of the test-set you want to use.
 For example, to use one month of data, pass the following parameter to the hyperopt call:
@@ -437,7 +449,7 @@ Stoploss: -0.27996
 
 In order to use this best stoploss value found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `stoploss` attribute of your custom strategy:
 
-```
+``` python
     # Optimal stoploss designed for the strategy
     # This attribute will be overridden if the config file contains "stoploss"
     stoploss = -0.27996
@@ -471,7 +483,7 @@ Trailing stop:
 
 In order to use these best trailing stop parameters found by Hyperopt in backtesting and for live trades/dry-run, copy-paste them as the values of the corresponding attributes of your custom strategy:
 
-```
+``` python
     # Trailing stop
     # These attributes will be overridden if the config file contains corresponding values.
     trailing_stop = True
@@ -496,4 +508,8 @@ After you run Hyperopt for the desired amount of epochs, you can later list all
 
 Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.
 
-To achieve same results (number of trades, their durations, profit, etc.) than during Hyperopt, please use same set of arguments `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
+To achieve same results (number of trades, their durations, profit, etc.) than during Hyperopt, please use same configuration and parameters (timerange, timeframe, ...) used for hyperopt `--dmmp`/`--disable-max-market-positions` and `--eps`/`--enable-position-stacking` for Backtesting.
+
+Should results don't match, please double-check to make sure you transferred all conditions correctly.
+Pay special care to the stoploss (and trailing stoploss) parameters, as these are often set in configuration files, which override changes to the strategy.
+You should also carefully review the log of your backtest to ensure that there were no parameters inadvertently set by the configuration (like `stoploss` or `trailing_stop`).