Merge branch 'develop' into pr/nicolaspapp/6715

This commit is contained in:
Matthias
2022-04-30 14:21:12 +02:00
101 changed files with 5536 additions and 5053 deletions

View File

@@ -0,0 +1,73 @@
# Advanced Backtesting Analysis
## Analyze the buy/entry and sell/exit tags
It can be helpful to understand how a strategy behaves according to the buy/entry tags used to
mark up different buy conditions. You might want to see more complex statistics about each buy and
sell condition above those provided by the default backtesting output. You may also want to
determine indicator values on the signal candle that resulted in a trade opening.
!!! Note
The following buy reason analysis is only available for backtesting, *not hyperopt*.
We need to run backtesting with the `--export` option set to `signals` to enable the exporting of
signals **and** trades:
``` bash
freqtrade backtesting -c <config.json> --timeframe <tf> --strategy <strategy_name> --timerange=<timerange> --export=signals
```
This will tell freqtrade to output a pickled dictionary of strategy, pairs and corresponding
DataFrame of the candles that resulted in buy signals. Depending on how many buys your strategy
makes, this file may get quite large, so periodically check your `user_data/backtest_results`
folder to delete old exports.
To analyze the buy tags, we need to use the `buy_reasons.py` script from
[froggleston's repo](https://github.com/froggleston/freqtrade-buyreasons). Follow the instructions
in their README to copy the script into your `freqtrade/scripts/` folder.
Before running your next backtest, make sure you either delete your old backtest results or run
backtesting with the `--cache none` option to make sure no cached results are used.
If all goes well, you should now see a `backtest-result-{timestamp}_signals.pkl` file in the
`user_data/backtest_results` folder.
Now run the `buy_reasons.py` script, supplying a few options:
``` bash
python3 scripts/buy_reasons.py -c <config.json> -s <strategy_name> -t <timerange> -g0,1,2,3,4
```
The `-g` option is used to specify the various tabular outputs, ranging from the simplest (0)
to the most detailed per pair, per buy and per sell tag (4). More options are available by
running with the `-h` option.
### Tuning the buy tags and sell tags to display
To show only certain buy and sell tags in the displayed output, use the following two options:
```
--enter_reason_list : Comma separated list of enter signals to analyse. Default: "all"
--exit_reason_list : Comma separated list of exit signals to analyse. Default: "stop_loss,trailing_stop_loss"
```
For example:
```bash
python3 scripts/buy_reasons.py -c <config.json> -s <strategy_name> -t <timerange> -g0,1,2,3,4 --enter_reason_list "enter_tag_a,enter_tag_b" --exit_reason_list "roi,custom_exit_tag_a,stop_loss"
```
### Outputting signal candle indicators
The real power of the buy_reasons.py script comes from the ability to print out the indicator
values present on signal candles to allow fine-grained investigation and tuning of buy signal
indicators. To print out a column for a given set of indicators, use the `--indicator-list`
option:
```bash
python3 scripts/buy_reasons.py -c <config.json> -s <strategy_name> -t <timerange> -g0,1,2,3,4 --enter_reason_list "enter_tag_a,enter_tag_b" --exit_reason_list "roi,custom_exit_tag_a,stop_loss" --indicator_list "rsi,rsi_1h,bb_lowerband,ema_9,macd,macdsignal"
```
The indicators have to be present in your strategy's main DataFrame (either for your main
timeframe or for informative timeframes) otherwise they will simply be ignored in the script
output.

View File

@@ -20,7 +20,8 @@ usage: freqtrade backtesting [-h] [-v] [--logfile FILE] [-V] [-c PATH]
[--dry-run-wallet DRY_RUN_WALLET]
[--timeframe-detail TIMEFRAME_DETAIL]
[--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
[--export {none,trades}] [--export-filename PATH]
[--export {none,trades,signals}]
[--export-filename PATH]
[--breakdown {day,week,month} [{day,week,month} ...]]
[--cache {none,day,week,month}]
@@ -63,18 +64,17 @@ optional arguments:
`30m`, `1h`, `1d`).
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
Provide a space-separated list of strategies to
backtest. Please note that timeframe needs to be
set either in config or via command line. When using
this together with `--export trades`, the strategy-
name is injected into the filename (so `backtest-
data.json` becomes `backtest-data-SampleStrategy.json`
--export {none,trades}
backtest. Please note that timeframe needs to be set
either in config or via command line. When using this
together with `--export trades`, the strategy-name is
injected into the filename (so `backtest-data.json`
becomes `backtest-data-SampleStrategy.json`
--export {none,trades,signals}
Export backtest results (default: trades).
--export-filename PATH
Save backtest results to the file with this filename.
Requires `--export` to be set as well. Example:
`--export-filename=user_data/backtest_results/backtest
_today.json`
--export-filename PATH, --backtest-filename PATH
Use this filename for backtest results.Requires
`--export` to be set as well. Example: `--export-filen
ame=user_data/backtest_results/backtest_today.json`
--breakdown {day,week,month} [{day,week,month} ...]
Show backtesting breakdown per [day, week, month].
--cache {none,day,week,month}
@@ -299,6 +299,7 @@ A backtesting result will look like that:
| Final balance | 0.01762792 BTC |
| Absolute profit | 0.00762792 BTC |
| Total profit % | 76.2% |
| CAGR % | 460.87% |
| Trades per day | 3.575 |
| Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC |
@@ -388,6 +389,7 @@ It contains some useful key metrics about performance of your strategy on backte
| Final balance | 0.01762792 BTC |
| Absolute profit | 0.00762792 BTC |
| Total profit % | 76.2% |
| CAGR % | 460.87% |
| Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC |
| | |

View File

@@ -11,7 +11,7 @@ Per default, the bot loads the configuration from the `config.json` file, locate
You can specify a different configuration file used by the bot with the `-c/--config` command-line option.
If you used the [Quick start](installation.md/#quick-start) method for installing
If you used the [Quick start](installation.md/#quick-start) method for installing
the bot, the installation script should have already created the default configuration file (`config.json`) for you.
If the default configuration file is not created we recommend to use `freqtrade new-config --config config.json` to generate a basic configuration file.
@@ -64,7 +64,7 @@ This is similar to using multiple `--config` parameters, but simpler in usage as
"config-private.json"
]
```
``` bash
freqtrade trade --config user_data/config.json <...>
```
@@ -100,7 +100,7 @@ This is similar to using multiple `--config` parameters, but simpler in usage as
"stake_amount": "unlimited",
}
```
Resulting combined configuration:
``` json title="Result"
@@ -173,6 +173,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `order_types` | Configure order-types depending on the action (`"entry"`, `"exit"`, `"stoploss"`, `"stoploss_on_exchange"`). [More information below](#understand-order_types). [Strategy Override](#parameters-in-the-strategy).<br> **Datatype:** Dict
| `order_time_in_force` | Configure time in force for entry and exit orders. [More information below](#understand-order_time_in_force). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Dict
| `custom_price_max_distance_ratio` | Configure maximum distance ratio between current and custom entry or exit price. <br>*Defaults to `0.02` 2%).*<br> **Datatype:** Positive float
| `recursive_strategy_search` | Set to `true` to recursively search sub-directories inside `user_data/strategies` for a strategy. <br> **Datatype:** Boolean
| `exchange.name` | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). <br> **Datatype:** String
| `exchange.sandbox` | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.<br> **Datatype:** Boolean
| `exchange.key` | API key to use for the exchange. Only required when you are in production mode.<br>**Keep it in secret, do not disclose publicly.** <br> **Datatype:** String

View File

@@ -122,5 +122,6 @@ Best avoid relative paths, since this starts at the storage location of the jupy
* [Strategy debugging](strategy_analysis_example.md) - also available as Jupyter notebook (`user_data/notebooks/strategy_analysis_example.ipynb`)
* [Plotting](plotting.md)
* [Tag Analysis](advanced-backtesting.md)
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

View File

@@ -64,7 +64,10 @@ Binance supports [time_in_force](configuration.md#understand-order_time_in_force
For Binance, please add `"BNB/<STAKE>"` to your blacklist to avoid issues.
Accounts having BNB accounts use this to pay for fees - if your first trade happens to be on `BNB`, further trades will consume this position and make the initial BNB trade unsellable as the expected amount is not there anymore.
### Binance Futures' order pricing
### Binance Futures
Binance has specific (unfortunately complex) [Futures Trading Quantitative Rules](https://www.binance.com/en/support/faq/4f462ebe6ff445d4a170be7d9e897272) which need to be followed, and which prohibit a too low stake-amount (among others) for too many orders.
Violating these rules will result in a trading restriction.
When trading on Binance Futures market, orderbook must be used because there is no price ticker data for futures.

View File

@@ -51,6 +51,14 @@ Please read the [exchange specific notes](exchanges.md) to learn about eventual,
- [X] [OKX](https://okx.com/) (Former OKEX)
- [ ] [potentially many others through <img alt="ccxt" width="30px" src="assets/ccxt-logo.svg" />](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_
### Experimentally, freqtrade also supports futures on the following exchanges:
- [X] [Binance](https://www.binance.com/)
- [X] [Gate.io](https://www.gate.io/ref/6266643)
- [X] [OKX](https://okx.com/).
Please make sure to read the [exchange specific notes](exchanges.md), as well as the [trading with leverage](leverage.md) documentation before diving in.
### Community tested
Exchanges confirmed working by the community:

View File

@@ -9,4 +9,4 @@ window.MathJax = {
ignoreHtmlClass: ".*|",
processHtmlClass: "arithmatex"
}
};
};

View File

@@ -1,5 +1,5 @@
mkdocs==1.3.0
mkdocs-material==8.2.9
mkdocs-material==8.2.10
mdx_truly_sane_lists==1.2
pymdown-extensions==9.3
pymdown-extensions==9.4
jinja2==3.1.1

View File

@@ -7,6 +7,7 @@ Depending on the callback used, they may be called when entering / exiting a tra
Currently available callbacks:
* [`bot_start()`](#bot-start)
* [`bot_loop_start()`](#bot-loop-start)
* [`custom_stake_amount()`](#stake-size-management)
* [`custom_exit()`](#custom-exit-signal)
@@ -21,6 +22,29 @@ Currently available callbacks:
!!! Tip "Callback calling sequence"
You can find the callback calling sequence in [bot-basics](bot-basics.md#bot-execution-logic)
## Bot start
A simple callback which is called once when the strategy is loaded.
This can be used to perform actions that must only be performed once and runs after dataprovider and wallet are set
``` python
import requests
class AwesomeStrategy(IStrategy):
# ... populate_* methods
def bot_start(self, **kwargs) -> None:
"""
Called only once after bot instantiation.
:param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
"""
if self.config['runmode'].value in ('live', 'dry_run'):
# Assign this to the class by using self.*
# can then be used by populate_* methods
self.cust_remote_data = requests.get('https://some_remote_source.example.com')
```
## Bot loop start
A simple callback which is called once at the start of every bot throttling iteration (roughly every 5 seconds, unless configured differently).
@@ -122,11 +146,11 @@ See [Dataframe access](strategy-advanced.md#dataframe-access) for more informati
## Custom stoploss
Called for open trade every throttling iteration (roughly every 5 seconds) until a trade is closed.
Called for open trade every iteration (roughly every 5 seconds) until a trade is closed.
The usage of the custom stoploss method must be enabled by setting `use_custom_stoploss=True` on the strategy object.
The stoploss price can only ever move upwards - if the stoploss value returned from `custom_stoploss` would result in a lower stoploss price than was previously set, it will be ignored. The traditional `stoploss` value serves as an absolute lower level and will be instated as the initial stoploss (before this method is called for the first time for a trade).
The stoploss price can only ever move upwards - if the stoploss value returned from `custom_stoploss` would result in a lower stoploss price than was previously set, it will be ignored. The traditional `stoploss` value serves as an absolute lower level and will be instated as the initial stoploss (before this method is called for the first time for a trade), and is still mandatory.
The method must return a stoploss value (float / number) as a percentage of the current price.
E.g. If the `current_rate` is 200 USD, then returning `0.02` will set the stoploss price 2% lower, at 196 USD.
@@ -365,30 +389,30 @@ class AwesomeStrategy(IStrategy):
# ... populate_* methods
def custom_entry_price(self, pair: str, current_time: datetime, proposed_rate: float,
def custom_entry_price(self, pair: str, current_time: datetime, proposed_rate: float,
entry_tag: Optional[str], side: str, **kwargs) -> float:
dataframe, last_updated = self.dp.get_analyzed_dataframe(pair=pair,
timeframe=self.timeframe)
new_entryprice = dataframe['bollinger_10_lowerband'].iat[-1]
return new_entryprice
def custom_exit_price(self, pair: str, trade: Trade,
current_time: datetime, proposed_rate: float,
current_profit: float, **kwargs) -> float:
current_profit: float, exit_tag: Optional[str], **kwargs) -> float:
dataframe, last_updated = self.dp.get_analyzed_dataframe(pair=pair,
timeframe=self.timeframe)
new_exitprice = dataframe['bollinger_10_upperband'].iat[-1]
return new_exitprice
```
!!! Warning
Modifying entry and exit prices will only work for limit orders. Depending on the price chosen, this can result in a lot of unfilled orders. By default the maximum allowed distance between the current price and the custom price is 2%, this value can be changed in config with the `custom_price_max_distance_ratio` parameter.
**Example**:
Modifying entry and exit prices will only work for limit orders. Depending on the price chosen, this can result in a lot of unfilled orders. By default the maximum allowed distance between the current price and the custom price is 2%, this value can be changed in config with the `custom_price_max_distance_ratio` parameter.
**Example**:
If the new_entryprice is 97, the proposed_rate is 100 and the `custom_price_max_distance_ratio` is set to 2%, The retained valid custom entry price will be 98, which is 2% below the current (proposed) rate.
!!! Warning "Backtesting"
@@ -418,7 +442,7 @@ The function must return either `True` (cancel order) or `False` (keep order ali
``` python
from datetime import datetime, timedelta
from freqtrade.persistence import Trade
from freqtrade.persistence import Trade, Order
class AwesomeStrategy(IStrategy):
@@ -430,7 +454,7 @@ class AwesomeStrategy(IStrategy):
'exit': 60 * 25
}
def check_entry_timeout(self, pair: str, trade: 'Trade', order: dict,
def check_entry_timeout(self, pair: str, trade: 'Trade', order: 'Order',
current_time: datetime, **kwargs) -> bool:
if trade.open_rate > 100 and trade.open_date_utc < current_time - timedelta(minutes=5):
return True
@@ -441,7 +465,7 @@ class AwesomeStrategy(IStrategy):
return False
def check_exit_timeout(self, pair: str, trade: Trade, order: dict,
def check_exit_timeout(self, pair: str, trade: Trade, order: 'Order',
current_time: datetime, **kwargs) -> bool:
if trade.open_rate > 100 and trade.open_date_utc < current_time - timedelta(minutes=5):
return True
@@ -459,7 +483,7 @@ class AwesomeStrategy(IStrategy):
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.persistence import Trade, Order
class AwesomeStrategy(IStrategy):
@@ -471,22 +495,22 @@ class AwesomeStrategy(IStrategy):
'exit': 60 * 25
}
def check_entry_timeout(self, pair: str, trade: Trade, order: dict,
def check_entry_timeout(self, pair: str, trade: 'Trade', order: 'Order',
current_time: datetime, **kwargs) -> bool:
ob = self.dp.orderbook(pair, 1)
current_price = ob['bids'][0][0]
# Cancel buy order if price is more than 2% above the order.
if current_price > order['price'] * 1.02:
if current_price > order.price * 1.02:
return True
return False
def check_exit_timeout(self, pair: str, trade: Trade, order: dict,
def check_exit_timeout(self, pair: str, trade: 'Trade', order: 'Order',
current_time: datetime, **kwargs) -> bool:
ob = self.dp.orderbook(pair, 1)
current_price = ob['asks'][0][0]
# Cancel sell order if price is more than 2% below the order.
if current_price < order['price'] * 0.98:
if current_price < order.price * 0.98:
return True
return False
```
@@ -508,7 +532,7 @@ class AwesomeStrategy(IStrategy):
# ... populate_* methods
def confirm_trade_entry(self, pair: str, order_type: str, amount: float, rate: float,
time_in_force: str, current_time: datetime, entry_tag: Optional[str],
time_in_force: str, current_time: datetime, entry_tag: Optional[str],
side: str, **kwargs) -> bool:
"""
Called right before placing a entry order.
@@ -616,35 +640,35 @@ from freqtrade.persistence import Trade
class DigDeeperStrategy(IStrategy):
position_adjustment_enable = True
# Attempts to handle large drops with DCA. High stoploss is required.
stoploss = -0.30
# ... populate_* methods
# Example specific variables
max_entry_position_adjustment = 3
# This number is explained a bit further down
max_dca_multiplier = 5.5
# This is called when placing the initial order (opening trade)
def custom_stake_amount(self, pair: str, current_time: datetime, current_rate: float,
proposed_stake: float, min_stake: float, max_stake: float,
entry_tag: Optional[str], side: str, **kwargs) -> float:
# We need to leave most of the funds for possible further DCA orders
# This also applies to fixed stakes
return proposed_stake / self.max_dca_multiplier
def adjust_trade_position(self, trade: Trade, current_time: datetime,
current_rate: float, current_profit: float, min_stake: float,
max_stake: float, **kwargs):
"""
Custom trade adjustment logic, returning the stake amount that a trade should be increased.
This means extra buy orders with additional fees.
:param trade: trade object.
:param current_time: datetime object, containing the current datetime
:param current_rate: Current buy rate.
@@ -654,7 +678,7 @@ class DigDeeperStrategy(IStrategy):
:param **kwargs: Ensure to keep this here so updates to this won't break your strategy.
:return float: Stake amount to adjust your trade
"""
if current_profit > -0.05:
return None

View File

@@ -93,7 +93,7 @@ from freqtrade.data.btanalysis import load_backtest_data, load_backtest_stats
# if backtest_dir points to a directory, it'll automatically load the last backtest file.
backtest_dir = config["user_data_dir"] / "backtest_results"
# backtest_dir can also point to a specific file
# backtest_dir can also point to a specific file
# backtest_dir = config["user_data_dir"] / "backtest_results/backtest-result-2020-07-01_20-04-22.json"
```

View File

@@ -183,11 +183,11 @@ class AwesomeStrategy(IStrategy):
``` python hl_lines="2 6"
class AwesomeStrategy(IStrategy):
def check_entry_timeout(self, pair: str, trade: 'Trade', order: dict,
def check_entry_timeout(self, pair: str, trade: 'Trade', order: 'Order',
current_time: datetime, **kwargs) -> bool:
return False
def check_exit_timeout(self, pair: str, trade: 'Trade', order: dict,
def check_exit_timeout(self, pair: str, trade: 'Trade', order: 'Order',
current_time: datetime, **kwargs) -> bool:
return False
```