Clearly highlight potential problems with looking into the future

This commit is contained in:
Matthias 2019-10-20 16:22:11 +02:00
parent 6e938b59c8
commit 20dd3f2d67
2 changed files with 19 additions and 5 deletions

View File

@ -201,6 +201,8 @@ Since backtesting lacks some detailed information about what happens within a ca
Taking these assumptions, backtesting tries to mirror real trading as closely as possible. However, backtesting will **never** replace running a strategy in dry-run mode. Taking these assumptions, backtesting tries to mirror real trading as closely as possible. However, backtesting will **never** replace running a strategy in dry-run mode.
Also, keep in mind that past results don't guarantee future success. Also, keep in mind that past results don't guarantee future success.
In addition to the above assumptions, strategy authors should carefully read the [Common Mistakes](strategy-customization.md#common-mistakes-when-developing-strategies) section, to avoid using data in backtesting which is not available in real market conditions.
### Further backtest-result analysis ### Further backtest-result analysis
To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file). To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).

View File

@ -60,8 +60,7 @@ file as reference.**
!!! Warning Using future data !!! Warning Using future data
Since backtesting passes the full time interval to the `populate_*()` methods, the strategy author Since backtesting passes the full time interval to the `populate_*()` methods, the strategy author
needs to take care to avoid having the strategy utilize data from the future. needs to take care to avoid having the strategy utilize data from the future.
Samples for usage of future data are `dataframe.shift(-1)`, `dataframe.resample("1h")` (this uses the left border of the interval, so moves data from an hour to the start of the hour). Some common patterns for this are listed in the [Common Mistakes](#common-mistakes-when-developing-strategies) section of this document.
They all use data which is not available during regular operations, so these strategies will perform well during backtesting, but will fail / perform badly in dry-runs.
### Customize Indicators ### Customize Indicators
@ -399,10 +398,10 @@ def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
Printing more than a few rows is also possible (simply use `print(dataframe)` instead of `print(dataframe.tail())`), however not recommended, as that will be very verbose (~500 lines per pair every 5 seconds). Printing more than a few rows is also possible (simply use `print(dataframe)` instead of `print(dataframe.tail())`), however not recommended, as that will be very verbose (~500 lines per pair every 5 seconds).
### Where is the default strategy? ### Where can i find a strategy template?
The default buy strategy is located in the file The strategy template is located in the file
[freqtrade/default_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/strategy/default_strategy.py). [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py).
### Specify custom strategy location ### Specify custom strategy location
@ -412,6 +411,19 @@ If you want to use a strategy from a different directory you can pass `--strateg
freqtrade --strategy AwesomeStrategy --strategy-path /some/directory freqtrade --strategy AwesomeStrategy --strategy-path /some/directory
``` ```
### Common mistakes when developing strategies
Backtesting analyzes the whole time-range at once for performance reasons. Because of this, strategy authors need to make sure that strategies do not look into the future.
This is a common pain-point, which can cause huge differences between backtesting and dry/live run methods, since they all use data which is not available during dry/live runs, so these strategies will perform well during backtesting, but will fail / perform badly in real conditions.
The following lists some common patterns which should be avoided to avoid frustration:
- don't use `shift(-1)`. This uses data from the future, which is not available.
- don't use `.iloc[-1]` or any other absolute position in the dataframe, this will be different between dry-run and backtesting.
- don't use `dataframe['volume'].mean()`. This uses the full DataFrame for backtesting, including data from the future. Use `dataframe['volume'].rolling(<window>).mean()` instead
- don't use `.resample('1h')`. This uses the left border of the interval, so moves data from an hour to the start of the hour. Use `.resample('1h', label='right')` instead.
### Further strategy ideas ### Further strategy ideas
To get additional Ideas for strategies, head over to our [strategy repository](https://github.com/freqtrade/freqtrade-strategies). Feel free to use them as they are - but results will depend on the current market situation, pairs used etc. - therefore please backtest the strategy for your exchange/desired pairs first, evaluate carefully, use at your own risk. To get additional Ideas for strategies, head over to our [strategy repository](https://github.com/freqtrade/freqtrade-strategies). Feel free to use them as they are - but results will depend on the current market situation, pairs used etc. - therefore please backtest the strategy for your exchange/desired pairs first, evaluate carefully, use at your own risk.