Merge pull request #1089 from freqtrade/feat/backtest_multi_strat
Allow multi strategy backtest without data reload
This commit is contained in:
commit
3a5b435dfa
@ -151,7 +151,7 @@ cp freqtrade/tests/testdata/pairs.json user_data/data/binance
|
|||||||
Then run:
|
Then run:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python scripts/download_backtest_data --exchange binance
|
python scripts/download_backtest_data.py --exchange binance
|
||||||
```
|
```
|
||||||
|
|
||||||
This will download ticker data for all the currency pairs you defined in `pairs.json`.
|
This will download ticker data for all the currency pairs you defined in `pairs.json`.
|
||||||
@ -238,6 +238,31 @@ On the other hand, if you set a too high `minimal_roi` like `"0": 0.55`
|
|||||||
profit. Hence, keep in mind that your performance is a mix of your
|
profit. Hence, keep in mind that your performance is a mix of your
|
||||||
strategies, your configuration, and the crypto-currency you have set up.
|
strategies, your configuration, and the crypto-currency you have set up.
|
||||||
|
|
||||||
|
## Backtesting multiple strategies
|
||||||
|
|
||||||
|
To backtest multiple strategies, a list of Strategies can be provided.
|
||||||
|
|
||||||
|
This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple
|
||||||
|
strategies you'd like to compare, this should give a nice runtime boost.
|
||||||
|
|
||||||
|
All listed Strategies need to be in the same folder.
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strategy-list Strategy001 Strategy002 --export trades
|
||||||
|
```
|
||||||
|
|
||||||
|
This will save the results to `user_data/backtest_data/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
|
||||||
|
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
|
||||||
|
Detailed output for all strategies one after the other will be available, so make sure to scroll up.
|
||||||
|
|
||||||
|
```
|
||||||
|
=================================================== Strategy Summary ====================================================
|
||||||
|
| Strategy | buy count | avg profit % | cum profit % | total profit ETH | avg duration | profit | loss |
|
||||||
|
|:-----------|------------:|---------------:|---------------:|-------------------:|:----------------|---------:|-------:|
|
||||||
|
| Strategy1 | 19 | -0.76 | -14.39 | -0.01440287 | 15:48:00 | 15 | 4 |
|
||||||
|
| Strategy2 | 6 | -2.73 | -16.40 | -0.01641299 | 1 day, 14:12:00 | 3 | 3 |
|
||||||
|
```
|
||||||
|
|
||||||
## Next step
|
## Next step
|
||||||
|
|
||||||
Great, your strategy is profitable. What if the bot can give your the
|
Great, your strategy is profitable. What if the bot can give your the
|
||||||
|
@ -1,13 +1,15 @@
|
|||||||
# Bot usage
|
# Bot usage
|
||||||
This page explains the difference parameters of the bot and how to run
|
|
||||||
it.
|
This page explains the difference parameters of the bot and how to run it.
|
||||||
|
|
||||||
## Table of Contents
|
## Table of Contents
|
||||||
|
|
||||||
- [Bot commands](#bot-commands)
|
- [Bot commands](#bot-commands)
|
||||||
- [Backtesting commands](#backtesting-commands)
|
- [Backtesting commands](#backtesting-commands)
|
||||||
- [Hyperopt commands](#hyperopt-commands)
|
- [Hyperopt commands](#hyperopt-commands)
|
||||||
|
|
||||||
## Bot commands
|
## Bot commands
|
||||||
|
|
||||||
```
|
```
|
||||||
usage: freqtrade [-h] [-v] [--version] [-c PATH] [-d PATH] [-s NAME]
|
usage: freqtrade [-h] [-v] [--version] [-c PATH] [-d PATH] [-s NAME]
|
||||||
[--strategy-path PATH] [--dynamic-whitelist [INT]]
|
[--strategy-path PATH] [--dynamic-whitelist [INT]]
|
||||||
@ -41,6 +43,7 @@ optional arguments:
|
|||||||
```
|
```
|
||||||
|
|
||||||
### How to use a different config file?
|
### How to use a different config file?
|
||||||
|
|
||||||
The bot allows you to select which config file you want to use. Per
|
The bot allows you to select which config file you want to use. Per
|
||||||
default, the bot will load the file `./config.json`
|
default, the bot will load the file `./config.json`
|
||||||
|
|
||||||
@ -49,6 +52,7 @@ python3 ./freqtrade/main.py -c path/far/far/away/config.json
|
|||||||
```
|
```
|
||||||
|
|
||||||
### How to use --strategy?
|
### How to use --strategy?
|
||||||
|
|
||||||
This parameter will allow you to load your custom strategy class.
|
This parameter will allow you to load your custom strategy class.
|
||||||
Per default without `--strategy` or `-s` the bot will load the
|
Per default without `--strategy` or `-s` the bot will load the
|
||||||
`DefaultStrategy` included with the bot (`freqtrade/strategy/default_strategy.py`).
|
`DefaultStrategy` included with the bot (`freqtrade/strategy/default_strategy.py`).
|
||||||
@ -60,6 +64,7 @@ To load a strategy, simply pass the class name (e.g.: `CustomStrategy`) in this
|
|||||||
**Example:**
|
**Example:**
|
||||||
In `user_data/strategies` you have a file `my_awesome_strategy.py` which has
|
In `user_data/strategies` you have a file `my_awesome_strategy.py` which has
|
||||||
a strategy class called `AwesomeStrategy` to load it:
|
a strategy class called `AwesomeStrategy` to load it:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 ./freqtrade/main.py --strategy AwesomeStrategy
|
python3 ./freqtrade/main.py --strategy AwesomeStrategy
|
||||||
```
|
```
|
||||||
@ -70,6 +75,7 @@ message the reason (File not found, or errors in your code).
|
|||||||
Learn more about strategy file in [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).
|
Learn more about strategy file in [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).
|
||||||
|
|
||||||
### How to use --strategy-path?
|
### How to use --strategy-path?
|
||||||
|
|
||||||
This parameter allows you to add an additional strategy lookup path, which gets
|
This parameter allows you to add an additional strategy lookup path, which gets
|
||||||
checked before the default locations (The passed path must be a folder!):
|
checked before the default locations (The passed path must be a folder!):
|
||||||
```bash
|
```bash
|
||||||
@ -77,21 +83,25 @@ python3 ./freqtrade/main.py --strategy AwesomeStrategy --strategy-path /some/fol
|
|||||||
```
|
```
|
||||||
|
|
||||||
#### How to install a strategy?
|
#### How to install a strategy?
|
||||||
|
|
||||||
This is very simple. Copy paste your strategy file into the folder
|
This is very simple. Copy paste your strategy file into the folder
|
||||||
`user_data/strategies` or use `--strategy-path`. And voila, the bot is ready to use it.
|
`user_data/strategies` or use `--strategy-path`. And voila, the bot is ready to use it.
|
||||||
|
|
||||||
### How to use --dynamic-whitelist?
|
### How to use --dynamic-whitelist?
|
||||||
|
|
||||||
Per default `--dynamic-whitelist` will retrieve the 20 currencies based
|
Per default `--dynamic-whitelist` will retrieve the 20 currencies based
|
||||||
on BaseVolume. This value can be changed when you run the script.
|
on BaseVolume. This value can be changed when you run the script.
|
||||||
|
|
||||||
**By Default**
|
**By Default**
|
||||||
Get the 20 currencies based on BaseVolume.
|
Get the 20 currencies based on BaseVolume.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 ./freqtrade/main.py --dynamic-whitelist
|
python3 ./freqtrade/main.py --dynamic-whitelist
|
||||||
```
|
```
|
||||||
|
|
||||||
**Customize the number of currencies to retrieve**
|
**Customize the number of currencies to retrieve**
|
||||||
Get the 30 currencies based on BaseVolume.
|
Get the 30 currencies based on BaseVolume.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 ./freqtrade/main.py --dynamic-whitelist 30
|
python3 ./freqtrade/main.py --dynamic-whitelist 30
|
||||||
```
|
```
|
||||||
@ -102,6 +112,7 @@ negative value (e.g -2), `--dynamic-whitelist` will use the default
|
|||||||
value (20).
|
value (20).
|
||||||
|
|
||||||
### How to use --db-url?
|
### How to use --db-url?
|
||||||
|
|
||||||
When you run the bot in Dry-run mode, per default no transactions are
|
When you run the bot in Dry-run mode, per default no transactions are
|
||||||
stored in a database. If you want to store your bot actions in a DB
|
stored in a database. If you want to store your bot actions in a DB
|
||||||
using `--db-url`. This can also be used to specify a custom database
|
using `--db-url`. This can also be used to specify a custom database
|
||||||
@ -111,14 +122,14 @@ in production mode. Example command:
|
|||||||
python3 ./freqtrade/main.py -c config.json --db-url sqlite:///tradesv3.dry_run.sqlite
|
python3 ./freqtrade/main.py -c config.json --db-url sqlite:///tradesv3.dry_run.sqlite
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
## Backtesting commands
|
## Backtesting commands
|
||||||
|
|
||||||
Backtesting also uses the config specified via `-c/--config`.
|
Backtesting also uses the config specified via `-c/--config`.
|
||||||
|
|
||||||
```
|
```
|
||||||
usage: main.py backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp]
|
usage: freqtrade backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp]
|
||||||
[--timerange TIMERANGE] [-l] [-r]
|
[--timerange TIMERANGE] [-l] [-r]
|
||||||
|
[--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
|
||||||
[--export EXPORT] [--export-filename PATH]
|
[--export EXPORT] [--export-filename PATH]
|
||||||
|
|
||||||
optional arguments:
|
optional arguments:
|
||||||
@ -139,6 +150,13 @@ optional arguments:
|
|||||||
refresh the pairs files in tests/testdata with the
|
refresh the pairs files in tests/testdata with the
|
||||||
latest data from the exchange. Use it if you want to
|
latest data from the exchange. Use it if you want to
|
||||||
run your backtesting with up-to-date data.
|
run your backtesting with up-to-date data.
|
||||||
|
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
|
||||||
|
Provide a commaseparated list of strategies to
|
||||||
|
backtest Please note that ticker-interval needs to be
|
||||||
|
set either in config or via command line. When using
|
||||||
|
this together with --export trades, the strategy-name
|
||||||
|
is injected into the filename (so backtest-data.json
|
||||||
|
becomes backtest-data-DefaultStrategy.json
|
||||||
--export EXPORT export backtest results, argument are: trades Example
|
--export EXPORT export backtest results, argument are: trades Example
|
||||||
--export=trades
|
--export=trades
|
||||||
--export-filename PATH
|
--export-filename PATH
|
||||||
@ -151,6 +169,7 @@ optional arguments:
|
|||||||
```
|
```
|
||||||
|
|
||||||
### How to use --refresh-pairs-cached parameter?
|
### How to use --refresh-pairs-cached parameter?
|
||||||
|
|
||||||
The first time your run Backtesting, it will take the pairs you have
|
The first time your run Backtesting, it will take the pairs you have
|
||||||
set in your config file and download data from Bittrex.
|
set in your config file and download data from Bittrex.
|
||||||
|
|
||||||
@ -162,7 +181,6 @@ to come back to the previous version.**
|
|||||||
To test your strategy with latest data, we recommend continuing using
|
To test your strategy with latest data, we recommend continuing using
|
||||||
the parameter `-l` or `--live`.
|
the parameter `-l` or `--live`.
|
||||||
|
|
||||||
|
|
||||||
## Hyperopt commands
|
## Hyperopt commands
|
||||||
|
|
||||||
To optimize your strategy, you can use hyperopt parameter hyperoptimization
|
To optimize your strategy, you can use hyperopt parameter hyperoptimization
|
||||||
@ -194,10 +212,11 @@ optional arguments:
|
|||||||
```
|
```
|
||||||
|
|
||||||
## A parameter missing in the configuration?
|
## A parameter missing in the configuration?
|
||||||
|
|
||||||
All parameters for `main.py`, `backtesting`, `hyperopt` are referenced
|
All parameters for `main.py`, `backtesting`, `hyperopt` are referenced
|
||||||
in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L84)
|
in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L84)
|
||||||
|
|
||||||
## Next step
|
## Next step
|
||||||
The optimal strategy of the bot will change with time depending of the
|
|
||||||
market trends. The next step is to
|
The optimal strategy of the bot will change with time depending of the market trends. The next step is to
|
||||||
[optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).
|
[optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).
|
||||||
|
@ -142,6 +142,16 @@ class Arguments(object):
|
|||||||
action='store_true',
|
action='store_true',
|
||||||
dest='refresh_pairs',
|
dest='refresh_pairs',
|
||||||
)
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
'--strategy-list',
|
||||||
|
help='Provide a commaseparated list of strategies to backtest '
|
||||||
|
'Please note that ticker-interval needs to be set either in config '
|
||||||
|
'or via command line. When using this together with --export trades, '
|
||||||
|
'the strategy-name is injected into the filename '
|
||||||
|
'(so backtest-data.json becomes backtest-data-DefaultStrategy.json',
|
||||||
|
nargs='+',
|
||||||
|
dest='strategy_list',
|
||||||
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
'--export',
|
'--export',
|
||||||
help='export backtest results, argument are: trades\
|
help='export backtest results, argument are: trades\
|
||||||
|
@ -187,6 +187,14 @@ class Configuration(object):
|
|||||||
config.update({'refresh_pairs': True})
|
config.update({'refresh_pairs': True})
|
||||||
logger.info('Parameter -r/--refresh-pairs-cached detected ...')
|
logger.info('Parameter -r/--refresh-pairs-cached detected ...')
|
||||||
|
|
||||||
|
if 'strategy_list' in self.args and self.args.strategy_list:
|
||||||
|
config.update({'strategy_list': self.args.strategy_list})
|
||||||
|
logger.info('Using strategy list of %s Strategies', len(self.args.strategy_list))
|
||||||
|
|
||||||
|
if 'ticker_interval' in self.args and self.args.ticker_interval:
|
||||||
|
config.update({'ticker_interval': self.args.ticker_interval})
|
||||||
|
logger.info('Overriding ticker interval with Command line argument')
|
||||||
|
|
||||||
# If --export is used we add it to the configuration
|
# If --export is used we add it to the configuration
|
||||||
if 'export' in self.args and self.args.export:
|
if 'export' in self.args and self.args.export:
|
||||||
config.update({'export': self.args.export})
|
config.update({'export': self.args.export})
|
||||||
|
@ -6,7 +6,9 @@ This module contains the backtesting logic
|
|||||||
import logging
|
import logging
|
||||||
import operator
|
import operator
|
||||||
from argparse import Namespace
|
from argparse import Namespace
|
||||||
|
from copy import deepcopy
|
||||||
from datetime import datetime, timedelta
|
from datetime import datetime, timedelta
|
||||||
|
from pathlib import Path
|
||||||
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
||||||
|
|
||||||
import arrow
|
import arrow
|
||||||
@ -52,13 +54,9 @@ class Backtesting(object):
|
|||||||
backtesting = Backtesting(config)
|
backtesting = Backtesting(config)
|
||||||
backtesting.start()
|
backtesting.start()
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, config: Dict[str, Any]) -> None:
|
def __init__(self, config: Dict[str, Any]) -> None:
|
||||||
self.config = config
|
self.config = config
|
||||||
self.strategy: IStrategy = StrategyResolver(self.config).strategy
|
|
||||||
self.ticker_interval = self.strategy.ticker_interval
|
|
||||||
self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
|
|
||||||
self.advise_buy = self.strategy.advise_buy
|
|
||||||
self.advise_sell = self.strategy.advise_sell
|
|
||||||
|
|
||||||
# Reset keys for backtesting
|
# Reset keys for backtesting
|
||||||
self.config['exchange']['key'] = ''
|
self.config['exchange']['key'] = ''
|
||||||
@ -66,9 +64,36 @@ class Backtesting(object):
|
|||||||
self.config['exchange']['password'] = ''
|
self.config['exchange']['password'] = ''
|
||||||
self.config['exchange']['uid'] = ''
|
self.config['exchange']['uid'] = ''
|
||||||
self.config['dry_run'] = True
|
self.config['dry_run'] = True
|
||||||
|
self.strategylist: List[IStrategy] = []
|
||||||
|
if self.config.get('strategy_list', None):
|
||||||
|
# Force one interval
|
||||||
|
self.ticker_interval = str(self.config.get('ticker_interval'))
|
||||||
|
for strat in list(self.config['strategy_list']):
|
||||||
|
stratconf = deepcopy(self.config)
|
||||||
|
stratconf['strategy'] = strat
|
||||||
|
self.strategylist.append(StrategyResolver(stratconf).strategy)
|
||||||
|
|
||||||
|
else:
|
||||||
|
# only one strategy
|
||||||
|
strat = StrategyResolver(self.config).strategy
|
||||||
|
|
||||||
|
self.strategylist.append(StrategyResolver(self.config).strategy)
|
||||||
|
# Load one strategy
|
||||||
|
self._set_strategy(self.strategylist[0])
|
||||||
|
|
||||||
self.exchange = Exchange(self.config)
|
self.exchange = Exchange(self.config)
|
||||||
self.fee = self.exchange.get_fee()
|
self.fee = self.exchange.get_fee()
|
||||||
|
|
||||||
|
def _set_strategy(self, strategy):
|
||||||
|
"""
|
||||||
|
Load strategy into backtesting
|
||||||
|
"""
|
||||||
|
self.strategy = strategy
|
||||||
|
self.ticker_interval = self.config.get('ticker_interval')
|
||||||
|
self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe
|
||||||
|
self.advise_buy = strategy.advise_buy
|
||||||
|
self.advise_sell = strategy.advise_sell
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
|
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
|
||||||
"""
|
"""
|
||||||
@ -132,7 +157,32 @@ class Backtesting(object):
|
|||||||
tabular_data.append([reason.value, count])
|
tabular_data.append([reason.value, count])
|
||||||
return tabulate(tabular_data, headers=headers, tablefmt="pipe")
|
return tabulate(tabular_data, headers=headers, tablefmt="pipe")
|
||||||
|
|
||||||
def _store_backtest_result(self, recordfilename: Optional[str], results: DataFrame) -> None:
|
def _generate_text_table_strategy(self, all_results: dict) -> str:
|
||||||
|
"""
|
||||||
|
Generate summary table per strategy
|
||||||
|
"""
|
||||||
|
stake_currency = str(self.config.get('stake_currency'))
|
||||||
|
|
||||||
|
floatfmt = ('s', 'd', '.2f', '.2f', '.8f', 'd', '.1f', '.1f')
|
||||||
|
tabular_data = []
|
||||||
|
headers = ['Strategy', 'buy count', 'avg profit %', 'cum profit %',
|
||||||
|
'total profit ' + stake_currency, 'avg duration', 'profit', 'loss']
|
||||||
|
for strategy, results in all_results.items():
|
||||||
|
tabular_data.append([
|
||||||
|
strategy,
|
||||||
|
len(results.index),
|
||||||
|
results.profit_percent.mean() * 100.0,
|
||||||
|
results.profit_percent.sum() * 100.0,
|
||||||
|
results.profit_abs.sum(),
|
||||||
|
str(timedelta(
|
||||||
|
minutes=round(results.trade_duration.mean()))) if not results.empty else '0:00',
|
||||||
|
len(results[results.profit_abs > 0]),
|
||||||
|
len(results[results.profit_abs < 0])
|
||||||
|
])
|
||||||
|
return tabulate(tabular_data, headers=headers, floatfmt=floatfmt, tablefmt="pipe")
|
||||||
|
|
||||||
|
def _store_backtest_result(self, recordfilename: str, results: DataFrame,
|
||||||
|
strategyname: Optional[str] = None) -> None:
|
||||||
|
|
||||||
records = [(t.pair, t.profit_percent, t.open_time.timestamp(),
|
records = [(t.pair, t.profit_percent, t.open_time.timestamp(),
|
||||||
t.close_time.timestamp(), t.open_index - 1, t.trade_duration,
|
t.close_time.timestamp(), t.open_index - 1, t.trade_duration,
|
||||||
@ -140,6 +190,11 @@ class Backtesting(object):
|
|||||||
for index, t in results.iterrows()]
|
for index, t in results.iterrows()]
|
||||||
|
|
||||||
if records:
|
if records:
|
||||||
|
if strategyname:
|
||||||
|
# Inject strategyname to filename
|
||||||
|
recname = Path(recordfilename)
|
||||||
|
recordfilename = str(Path.joinpath(
|
||||||
|
recname.parent, f'{recname.stem}-{strategyname}').with_suffix(recname.suffix))
|
||||||
logger.info('Dumping backtest results to %s', recordfilename)
|
logger.info('Dumping backtest results to %s', recordfilename)
|
||||||
file_dump_json(recordfilename, records)
|
file_dump_json(recordfilename, records)
|
||||||
|
|
||||||
@ -307,62 +362,55 @@ class Backtesting(object):
|
|||||||
else:
|
else:
|
||||||
logger.info('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
|
logger.info('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
|
||||||
max_open_trades = 0
|
max_open_trades = 0
|
||||||
|
all_results = {}
|
||||||
|
|
||||||
preprocessed = self.tickerdata_to_dataframe(data)
|
for strat in self.strategylist:
|
||||||
|
logger.info("Running backtesting for Strategy %s", strat.get_strategy_name())
|
||||||
|
self._set_strategy(strat)
|
||||||
|
|
||||||
# Print timeframe
|
# need to reprocess data every time to populate signals
|
||||||
min_date, max_date = self.get_timeframe(preprocessed)
|
preprocessed = self.tickerdata_to_dataframe(data)
|
||||||
logger.info(
|
|
||||||
'Measuring data from %s up to %s (%s days)..',
|
|
||||||
min_date.isoformat(),
|
|
||||||
max_date.isoformat(),
|
|
||||||
(max_date - min_date).days
|
|
||||||
)
|
|
||||||
|
|
||||||
# Execute backtest and print results
|
# Print timeframe
|
||||||
results = self.backtest(
|
min_date, max_date = self.get_timeframe(preprocessed)
|
||||||
{
|
logger.info(
|
||||||
'stake_amount': self.config.get('stake_amount'),
|
'Measuring data from %s up to %s (%s days)..',
|
||||||
'processed': preprocessed,
|
min_date.isoformat(),
|
||||||
'max_open_trades': max_open_trades,
|
max_date.isoformat(),
|
||||||
'position_stacking': self.config.get('position_stacking', False),
|
(max_date - min_date).days
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
if self.config.get('export', False):
|
|
||||||
self._store_backtest_result(self.config.get('exportfilename'), results)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
'\n' + '=' * 49 +
|
|
||||||
' BACKTESTING REPORT ' +
|
|
||||||
'=' * 50 + '\n'
|
|
||||||
'%s',
|
|
||||||
self._generate_text_table(
|
|
||||||
data,
|
|
||||||
results
|
|
||||||
)
|
)
|
||||||
)
|
|
||||||
# logger.info(
|
|
||||||
# results[['sell_reason']].groupby('sell_reason').count()
|
|
||||||
# )
|
|
||||||
|
|
||||||
logger.info(
|
# Execute backtest and print results
|
||||||
'\n' +
|
all_results[self.strategy.get_strategy_name()] = self.backtest(
|
||||||
' SELL READON STATS '.center(119, '=') +
|
{
|
||||||
'\n%s \n',
|
'stake_amount': self.config.get('stake_amount'),
|
||||||
self._generate_text_table_sell_reason(data, results)
|
'processed': preprocessed,
|
||||||
|
'max_open_trades': max_open_trades,
|
||||||
)
|
'position_stacking': self.config.get('position_stacking', False),
|
||||||
|
}
|
||||||
logger.info(
|
|
||||||
'\n' +
|
|
||||||
' LEFT OPEN TRADES REPORT '.center(119, '=') +
|
|
||||||
'\n%s',
|
|
||||||
self._generate_text_table(
|
|
||||||
data,
|
|
||||||
results.loc[results.open_at_end]
|
|
||||||
)
|
)
|
||||||
)
|
|
||||||
|
for strategy, results in all_results.items():
|
||||||
|
|
||||||
|
if self.config.get('export', False):
|
||||||
|
self._store_backtest_result(self.config['exportfilename'], results,
|
||||||
|
strategy if len(self.strategylist) > 1 else None)
|
||||||
|
|
||||||
|
print(f"Result for strategy {strategy}")
|
||||||
|
print(' BACKTESTING REPORT '.center(119, '='))
|
||||||
|
print(self._generate_text_table(data, results))
|
||||||
|
|
||||||
|
print(' SELL REASON STATS '.center(119, '='))
|
||||||
|
print(self._generate_text_table_sell_reason(data, results))
|
||||||
|
|
||||||
|
print(' LEFT OPEN TRADES REPORT '.center(119, '='))
|
||||||
|
print(self._generate_text_table(data, results.loc[results.open_at_end]))
|
||||||
|
print()
|
||||||
|
if len(all_results) > 1:
|
||||||
|
# Print Strategy summary table
|
||||||
|
print(' Strategy Summary '.center(119, '='))
|
||||||
|
print(self._generate_text_table_strategy(all_results))
|
||||||
|
print('\nFor more details, please look at the detail tables above')
|
||||||
|
|
||||||
|
|
||||||
def setup_configuration(args: Namespace) -> Dict[str, Any]:
|
def setup_configuration(args: Namespace) -> Dict[str, Any]:
|
||||||
|
@ -406,6 +406,50 @@ def test_generate_text_table_sell_reason(default_conf, mocker):
|
|||||||
data={'ETH/BTC': {}}, results=results) == result_str
|
data={'ETH/BTC': {}}, results=results) == result_str
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_text_table_strategyn(default_conf, mocker):
|
||||||
|
"""
|
||||||
|
Test Backtesting.generate_text_table_sell_reason() method
|
||||||
|
"""
|
||||||
|
patch_exchange(mocker)
|
||||||
|
backtesting = Backtesting(default_conf)
|
||||||
|
results = {}
|
||||||
|
results['ETH/BTC'] = pd.DataFrame(
|
||||||
|
{
|
||||||
|
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
|
||||||
|
'profit_percent': [0.1, 0.2, 0.3],
|
||||||
|
'profit_abs': [0.2, 0.4, 0.5],
|
||||||
|
'trade_duration': [10, 30, 10],
|
||||||
|
'profit': [2, 0, 0],
|
||||||
|
'loss': [0, 0, 1],
|
||||||
|
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
results['LTC/BTC'] = pd.DataFrame(
|
||||||
|
{
|
||||||
|
'pair': ['LTC/BTC', 'LTC/BTC', 'LTC/BTC'],
|
||||||
|
'profit_percent': [0.4, 0.2, 0.3],
|
||||||
|
'profit_abs': [0.4, 0.4, 0.5],
|
||||||
|
'trade_duration': [15, 30, 15],
|
||||||
|
'profit': [4, 1, 0],
|
||||||
|
'loss': [0, 0, 1],
|
||||||
|
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
result_str = (
|
||||||
|
'| Strategy | buy count | avg profit % | cum profit % '
|
||||||
|
'| total profit BTC | avg duration | profit | loss |\n'
|
||||||
|
'|:-----------|------------:|---------------:|---------------:'
|
||||||
|
'|-------------------:|:---------------|---------:|-------:|\n'
|
||||||
|
'| ETH/BTC | 3 | 20.00 | 60.00 '
|
||||||
|
'| 1.10000000 | 0:17:00 | 3 | 0 |\n'
|
||||||
|
'| LTC/BTC | 3 | 30.00 | 90.00 '
|
||||||
|
'| 1.30000000 | 0:20:00 | 3 | 0 |'
|
||||||
|
)
|
||||||
|
print(backtesting._generate_text_table_strategy(all_results=results))
|
||||||
|
assert backtesting._generate_text_table_strategy(all_results=results) == result_str
|
||||||
|
|
||||||
|
|
||||||
def test_backtesting_start(default_conf, mocker, caplog) -> None:
|
def test_backtesting_start(default_conf, mocker, caplog) -> None:
|
||||||
def get_timeframe(input1, input2):
|
def get_timeframe(input1, input2):
|
||||||
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
|
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
|
||||||
@ -654,6 +698,18 @@ def test_backtest_record(default_conf, fee, mocker):
|
|||||||
records = records[0]
|
records = records[0]
|
||||||
# Ensure records are of correct type
|
# Ensure records are of correct type
|
||||||
assert len(records) == 4
|
assert len(records) == 4
|
||||||
|
|
||||||
|
# reset test to test with strategy name
|
||||||
|
names = []
|
||||||
|
records = []
|
||||||
|
backtesting._store_backtest_result("backtest-result.json", results, "DefStrat")
|
||||||
|
assert len(results) == 4
|
||||||
|
# Assert file_dump_json was only called once
|
||||||
|
assert names == ['backtest-result-DefStrat.json']
|
||||||
|
records = records[0]
|
||||||
|
# Ensure records are of correct type
|
||||||
|
assert len(records) == 4
|
||||||
|
|
||||||
# ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117)
|
# ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117)
|
||||||
# Below follows just a typecheck of the schema/type of trade-records
|
# Below follows just a typecheck of the schema/type of trade-records
|
||||||
oix = None
|
oix = None
|
||||||
@ -686,15 +742,6 @@ def test_backtest_start_live(default_conf, mocker, caplog):
|
|||||||
read_data=json.dumps(default_conf)
|
read_data=json.dumps(default_conf)
|
||||||
))
|
))
|
||||||
|
|
||||||
args = MagicMock()
|
|
||||||
args.ticker_interval = 1
|
|
||||||
args.level = 10
|
|
||||||
args.live = True
|
|
||||||
args.datadir = None
|
|
||||||
args.export = None
|
|
||||||
args.strategy = 'DefaultStrategy'
|
|
||||||
args.timerange = '-100' # needed due to MagicMock malleability
|
|
||||||
|
|
||||||
args = [
|
args = [
|
||||||
'--config', 'config.json',
|
'--config', 'config.json',
|
||||||
'--strategy', 'DefaultStrategy',
|
'--strategy', 'DefaultStrategy',
|
||||||
@ -725,3 +772,60 @@ def test_backtest_start_live(default_conf, mocker, caplog):
|
|||||||
|
|
||||||
for line in exists:
|
for line in exists:
|
||||||
assert log_has(line, caplog.record_tuples)
|
assert log_has(line, caplog.record_tuples)
|
||||||
|
|
||||||
|
|
||||||
|
def test_backtest_start_multi_strat(default_conf, mocker, caplog):
|
||||||
|
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
|
||||||
|
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history',
|
||||||
|
new=lambda s, n, i: _load_pair_as_ticks(n, i))
|
||||||
|
patch_exchange(mocker)
|
||||||
|
backtestmock = MagicMock()
|
||||||
|
mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest', backtestmock)
|
||||||
|
gen_table_mock = MagicMock()
|
||||||
|
mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table', gen_table_mock)
|
||||||
|
gen_strattable_mock = MagicMock()
|
||||||
|
mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table_strategy',
|
||||||
|
gen_strattable_mock)
|
||||||
|
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
|
||||||
|
read_data=json.dumps(default_conf)
|
||||||
|
))
|
||||||
|
|
||||||
|
args = [
|
||||||
|
'--config', 'config.json',
|
||||||
|
'--datadir', 'freqtrade/tests/testdata',
|
||||||
|
'backtesting',
|
||||||
|
'--ticker-interval', '1m',
|
||||||
|
'--live',
|
||||||
|
'--timerange', '-100',
|
||||||
|
'--enable-position-stacking',
|
||||||
|
'--disable-max-market-positions',
|
||||||
|
'--strategy-list',
|
||||||
|
'DefaultStrategy',
|
||||||
|
'TestStrategy',
|
||||||
|
]
|
||||||
|
args = get_args(args)
|
||||||
|
start(args)
|
||||||
|
# 2 backtests, 4 tables
|
||||||
|
assert backtestmock.call_count == 2
|
||||||
|
assert gen_table_mock.call_count == 4
|
||||||
|
assert gen_strattable_mock.call_count == 1
|
||||||
|
|
||||||
|
# check the logs, that will contain the backtest result
|
||||||
|
exists = [
|
||||||
|
'Parameter -i/--ticker-interval detected ...',
|
||||||
|
'Using ticker_interval: 1m ...',
|
||||||
|
'Parameter -l/--live detected ...',
|
||||||
|
'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
|
||||||
|
'Parameter --timerange detected: -100 ...',
|
||||||
|
'Using data folder: freqtrade/tests/testdata ...',
|
||||||
|
'Using stake_currency: BTC ...',
|
||||||
|
'Using stake_amount: 0.001 ...',
|
||||||
|
'Downloading data for all pairs in whitelist ...',
|
||||||
|
'Measuring data from 2017-11-14T19:31:00+00:00 up to 2017-11-14T22:58:00+00:00 (0 days)..',
|
||||||
|
'Parameter --enable-position-stacking detected ...',
|
||||||
|
'Running backtesting for Strategy DefaultStrategy',
|
||||||
|
'Running backtesting for Strategy TestStrategy',
|
||||||
|
]
|
||||||
|
|
||||||
|
for line in exists:
|
||||||
|
assert log_has(line, caplog.record_tuples)
|
||||||
|
@ -132,7 +132,11 @@ def test_parse_args_backtesting_custom() -> None:
|
|||||||
'backtesting',
|
'backtesting',
|
||||||
'--live',
|
'--live',
|
||||||
'--ticker-interval', '1m',
|
'--ticker-interval', '1m',
|
||||||
'--refresh-pairs-cached']
|
'--refresh-pairs-cached',
|
||||||
|
'--strategy-list',
|
||||||
|
'DefaultStrategy',
|
||||||
|
'TestStrategy'
|
||||||
|
]
|
||||||
call_args = Arguments(args, '').get_parsed_arg()
|
call_args = Arguments(args, '').get_parsed_arg()
|
||||||
assert call_args.config == 'test_conf.json'
|
assert call_args.config == 'test_conf.json'
|
||||||
assert call_args.live is True
|
assert call_args.live is True
|
||||||
@ -141,6 +145,8 @@ def test_parse_args_backtesting_custom() -> None:
|
|||||||
assert call_args.func is not None
|
assert call_args.func is not None
|
||||||
assert call_args.ticker_interval == '1m'
|
assert call_args.ticker_interval == '1m'
|
||||||
assert call_args.refresh_pairs is True
|
assert call_args.refresh_pairs is True
|
||||||
|
assert type(call_args.strategy_list) is list
|
||||||
|
assert len(call_args.strategy_list) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_parse_args_hyperopt_custom() -> None:
|
def test_parse_args_hyperopt_custom() -> None:
|
||||||
|
@ -292,6 +292,61 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_setup_configuration_with_stratlist(mocker, default_conf, caplog) -> None:
|
||||||
|
"""
|
||||||
|
Test setup_configuration() function
|
||||||
|
"""
|
||||||
|
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
|
||||||
|
read_data=json.dumps(default_conf)
|
||||||
|
))
|
||||||
|
|
||||||
|
arglist = [
|
||||||
|
'--config', 'config.json',
|
||||||
|
'backtesting',
|
||||||
|
'--ticker-interval', '1m',
|
||||||
|
'--export', '/bar/foo',
|
||||||
|
'--strategy-list',
|
||||||
|
'DefaultStrategy',
|
||||||
|
'TestStrategy'
|
||||||
|
]
|
||||||
|
|
||||||
|
args = Arguments(arglist, '').get_parsed_arg()
|
||||||
|
|
||||||
|
configuration = Configuration(args)
|
||||||
|
config = configuration.get_config()
|
||||||
|
assert 'max_open_trades' in config
|
||||||
|
assert 'stake_currency' in config
|
||||||
|
assert 'stake_amount' in config
|
||||||
|
assert 'exchange' in config
|
||||||
|
assert 'pair_whitelist' in config['exchange']
|
||||||
|
assert 'datadir' in config
|
||||||
|
assert log_has(
|
||||||
|
'Using data folder: {} ...'.format(config['datadir']),
|
||||||
|
caplog.record_tuples
|
||||||
|
)
|
||||||
|
assert 'ticker_interval' in config
|
||||||
|
assert log_has('Parameter -i/--ticker-interval detected ...', caplog.record_tuples)
|
||||||
|
assert log_has(
|
||||||
|
'Using ticker_interval: 1m ...',
|
||||||
|
caplog.record_tuples
|
||||||
|
)
|
||||||
|
|
||||||
|
assert 'strategy_list' in config
|
||||||
|
assert log_has('Using strategy list of 2 Strategies', caplog.record_tuples)
|
||||||
|
|
||||||
|
assert 'position_stacking' not in config
|
||||||
|
|
||||||
|
assert 'use_max_market_positions' not in config
|
||||||
|
|
||||||
|
assert 'timerange' not in config
|
||||||
|
|
||||||
|
assert 'export' in config
|
||||||
|
assert log_has(
|
||||||
|
'Parameter --export detected: {} ...'.format(config['export']),
|
||||||
|
caplog.record_tuples
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None:
|
def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None:
|
||||||
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
|
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
|
||||||
read_data=json.dumps(default_conf)
|
read_data=json.dumps(default_conf)
|
||||||
|
Loading…
Reference in New Issue
Block a user