Merge pull request #1089 from freqtrade/feat/backtest_multi_strat
Allow multi strategy backtest without data reload
This commit is contained in:
		| @@ -151,7 +151,7 @@ cp freqtrade/tests/testdata/pairs.json user_data/data/binance | |||||||
| Then run: | Then run: | ||||||
|  |  | ||||||
| ```bash | ```bash | ||||||
| python scripts/download_backtest_data --exchange binance | python scripts/download_backtest_data.py --exchange binance | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
| This will download ticker data for all the currency pairs you defined in `pairs.json`. | This will download ticker data for all the currency pairs you defined in `pairs.json`. | ||||||
| @@ -238,6 +238,31 @@ On the other hand, if you set a too high `minimal_roi` like `"0":  0.55` | |||||||
| profit. Hence, keep in mind that your performance is a mix of your  | profit. Hence, keep in mind that your performance is a mix of your  | ||||||
| strategies, your configuration, and the crypto-currency you have set up. | strategies, your configuration, and the crypto-currency you have set up. | ||||||
|  |  | ||||||
|  | ## Backtesting multiple strategies | ||||||
|  |  | ||||||
|  | To backtest multiple strategies, a list of Strategies can be provided. | ||||||
|  |  | ||||||
|  | This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple  | ||||||
|  | strategies you'd like to compare, this should give a nice runtime boost. | ||||||
|  |  | ||||||
|  | All listed Strategies need to be in the same folder. | ||||||
|  |  | ||||||
|  | ``` bash | ||||||
|  | freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strategy-list Strategy001 Strategy002 --export trades  | ||||||
|  | ``` | ||||||
|  |  | ||||||
|  | This will save the results to `user_data/backtest_data/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename. | ||||||
|  | There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table). | ||||||
|  | Detailed output for all strategies one after the other will be available, so make sure to scroll up. | ||||||
|  |  | ||||||
|  | ``` | ||||||
|  | =================================================== Strategy Summary ==================================================== | ||||||
|  | | Strategy   |   buy count |   avg profit % |   cum profit % |   total profit ETH | avg duration    |   profit |   loss | | ||||||
|  | |:-----------|------------:|---------------:|---------------:|-------------------:|:----------------|---------:|-------:| | ||||||
|  | | Strategy1  |          19 |          -0.76 |         -14.39 |        -0.01440287 | 15:48:00        |       15 |      4 | | ||||||
|  | | Strategy2  |           6 |          -2.73 |         -16.40 |        -0.01641299 | 1 day, 14:12:00 |        3 |      3 | | ||||||
|  | ``` | ||||||
|  |  | ||||||
| ## Next step | ## Next step | ||||||
|  |  | ||||||
| Great, your strategy is profitable. What if the bot can give your the | Great, your strategy is profitable. What if the bot can give your the | ||||||
|   | |||||||
| @@ -1,13 +1,15 @@ | |||||||
| # Bot usage | # Bot usage | ||||||
| This page explains the difference parameters of the bot and how to run  |  | ||||||
| it. | This page explains the difference parameters of the bot and how to run it. | ||||||
|  |  | ||||||
| ## Table of Contents | ## Table of Contents | ||||||
|  |  | ||||||
| - [Bot commands](#bot-commands) | - [Bot commands](#bot-commands) | ||||||
| - [Backtesting commands](#backtesting-commands) | - [Backtesting commands](#backtesting-commands) | ||||||
| - [Hyperopt commands](#hyperopt-commands) | - [Hyperopt commands](#hyperopt-commands) | ||||||
|  |  | ||||||
| ## Bot commands | ## Bot commands | ||||||
|  |  | ||||||
| ``` | ``` | ||||||
| usage: freqtrade [-h] [-v] [--version] [-c PATH] [-d PATH] [-s NAME] | usage: freqtrade [-h] [-v] [--version] [-c PATH] [-d PATH] [-s NAME] | ||||||
|                  [--strategy-path PATH] [--dynamic-whitelist [INT]] |                  [--strategy-path PATH] [--dynamic-whitelist [INT]] | ||||||
| @@ -41,6 +43,7 @@ optional arguments: | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| ### How to use a different config file? | ### How to use a different config file? | ||||||
|  |  | ||||||
| The bot allows you to select which config file you want to use. Per  | The bot allows you to select which config file you want to use. Per  | ||||||
| default, the bot will load the file `./config.json` | default, the bot will load the file `./config.json` | ||||||
|  |  | ||||||
| @@ -49,6 +52,7 @@ python3 ./freqtrade/main.py -c path/far/far/away/config.json | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| ### How to use --strategy? | ### How to use --strategy? | ||||||
|  |  | ||||||
| This parameter will allow you to load your custom strategy class. | This parameter will allow you to load your custom strategy class. | ||||||
| Per default without `--strategy` or `-s` the bot will load the | Per default without `--strategy` or `-s` the bot will load the | ||||||
| `DefaultStrategy` included with the bot (`freqtrade/strategy/default_strategy.py`). | `DefaultStrategy` included with the bot (`freqtrade/strategy/default_strategy.py`). | ||||||
| @@ -60,6 +64,7 @@ To load a strategy, simply pass the class name (e.g.: `CustomStrategy`) in this | |||||||
| **Example:**   | **Example:**   | ||||||
| In `user_data/strategies` you have a file `my_awesome_strategy.py` which has | In `user_data/strategies` you have a file `my_awesome_strategy.py` which has | ||||||
| a strategy class called `AwesomeStrategy` to load it: | a strategy class called `AwesomeStrategy` to load it: | ||||||
|  |  | ||||||
| ```bash | ```bash | ||||||
| python3 ./freqtrade/main.py --strategy AwesomeStrategy | python3 ./freqtrade/main.py --strategy AwesomeStrategy | ||||||
| ``` | ``` | ||||||
| @@ -70,6 +75,7 @@ message the reason (File not found, or errors in your code). | |||||||
| Learn more about strategy file in [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md). | Learn more about strategy file in [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md). | ||||||
|  |  | ||||||
| ### How to use --strategy-path? | ### How to use --strategy-path? | ||||||
|  |  | ||||||
| This parameter allows you to add an additional strategy lookup path, which gets | This parameter allows you to add an additional strategy lookup path, which gets | ||||||
| checked before the default locations (The passed path must be a folder!): | checked before the default locations (The passed path must be a folder!): | ||||||
| ```bash | ```bash | ||||||
| @@ -77,21 +83,25 @@ python3 ./freqtrade/main.py --strategy AwesomeStrategy --strategy-path /some/fol | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| #### How to install a strategy? | #### How to install a strategy? | ||||||
|  |  | ||||||
| This is very simple. Copy paste your strategy file into the folder  | This is very simple. Copy paste your strategy file into the folder  | ||||||
| `user_data/strategies` or use `--strategy-path`. And voila, the bot is ready to use it. | `user_data/strategies` or use `--strategy-path`. And voila, the bot is ready to use it. | ||||||
|  |  | ||||||
| ### How to use --dynamic-whitelist? | ### How to use --dynamic-whitelist? | ||||||
|  |  | ||||||
| Per default `--dynamic-whitelist` will retrieve the 20 currencies based  | Per default `--dynamic-whitelist` will retrieve the 20 currencies based  | ||||||
| on BaseVolume. This value can be changed when you run the script. | on BaseVolume. This value can be changed when you run the script. | ||||||
|  |  | ||||||
| **By Default**   | **By Default**   | ||||||
| Get the 20 currencies based on BaseVolume.   | Get the 20 currencies based on BaseVolume.   | ||||||
|  |  | ||||||
| ```bash | ```bash | ||||||
| python3 ./freqtrade/main.py --dynamic-whitelist | python3 ./freqtrade/main.py --dynamic-whitelist | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
| **Customize the number of currencies to retrieve**   | **Customize the number of currencies to retrieve**   | ||||||
| Get the 30 currencies based on BaseVolume.   | Get the 30 currencies based on BaseVolume.   | ||||||
|  |  | ||||||
| ```bash | ```bash | ||||||
| python3 ./freqtrade/main.py --dynamic-whitelist 30 | python3 ./freqtrade/main.py --dynamic-whitelist 30 | ||||||
| ``` | ``` | ||||||
| @@ -102,6 +112,7 @@ negative value (e.g -2), `--dynamic-whitelist` will use the default | |||||||
| value (20). | value (20). | ||||||
|  |  | ||||||
| ### How to use --db-url? | ### How to use --db-url? | ||||||
|  |  | ||||||
| When you run the bot in Dry-run mode, per default no transactions are  | When you run the bot in Dry-run mode, per default no transactions are  | ||||||
| stored in a database. If you want to store your bot actions in a DB  | stored in a database. If you want to store your bot actions in a DB  | ||||||
| using `--db-url`. This can also be used to specify a custom database | using `--db-url`. This can also be used to specify a custom database | ||||||
| @@ -111,14 +122,14 @@ in production mode. Example command: | |||||||
| python3 ./freqtrade/main.py -c config.json --db-url sqlite:///tradesv3.dry_run.sqlite | python3 ./freqtrade/main.py -c config.json --db-url sqlite:///tradesv3.dry_run.sqlite | ||||||
| ``` | ``` | ||||||
|  |  | ||||||
|  |  | ||||||
| ## Backtesting commands | ## Backtesting commands | ||||||
|  |  | ||||||
| Backtesting also uses the config specified via `-c/--config`. | Backtesting also uses the config specified via `-c/--config`. | ||||||
|  |  | ||||||
| ``` | ``` | ||||||
| usage: main.py backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp] | usage: freqtrade backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp] | ||||||
|                              [--timerange TIMERANGE] [-l] [-r] |                              [--timerange TIMERANGE] [-l] [-r] | ||||||
|  |                              [--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]] | ||||||
|                              [--export EXPORT] [--export-filename PATH] |                              [--export EXPORT] [--export-filename PATH] | ||||||
|  |  | ||||||
| optional arguments: | optional arguments: | ||||||
| @@ -139,6 +150,13 @@ optional arguments: | |||||||
|                         refresh the pairs files in tests/testdata with the |                         refresh the pairs files in tests/testdata with the | ||||||
|                         latest data from the exchange. Use it if you want to |                         latest data from the exchange. Use it if you want to | ||||||
|                         run your backtesting with up-to-date data. |                         run your backtesting with up-to-date data. | ||||||
|  |   --strategy-list STRATEGY_LIST [STRATEGY_LIST ...] | ||||||
|  |                         Provide a commaseparated list of strategies to | ||||||
|  |                         backtest Please note that ticker-interval needs to be | ||||||
|  |                         set either in config or via command line. When using | ||||||
|  |                         this together with --export trades, the strategy-name | ||||||
|  |                         is injected into the filename (so backtest-data.json | ||||||
|  |                         becomes backtest-data-DefaultStrategy.json | ||||||
|   --export EXPORT       export backtest results, argument are: trades Example |   --export EXPORT       export backtest results, argument are: trades Example | ||||||
|                         --export=trades |                         --export=trades | ||||||
|   --export-filename PATH |   --export-filename PATH | ||||||
| @@ -151,6 +169,7 @@ optional arguments: | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| ### How to use --refresh-pairs-cached parameter? | ### How to use --refresh-pairs-cached parameter? | ||||||
|  |  | ||||||
| The first time your run Backtesting, it will take the pairs you have  | The first time your run Backtesting, it will take the pairs you have  | ||||||
| set in your config file and download data from Bittrex.  | set in your config file and download data from Bittrex.  | ||||||
|  |  | ||||||
| @@ -162,7 +181,6 @@ to come back to the previous version.** | |||||||
| To test your strategy with latest data, we recommend continuing using  | To test your strategy with latest data, we recommend continuing using  | ||||||
| the parameter `-l` or `--live`. | the parameter `-l` or `--live`. | ||||||
|  |  | ||||||
|  |  | ||||||
| ## Hyperopt commands | ## Hyperopt commands | ||||||
|  |  | ||||||
| To optimize your strategy, you can use hyperopt parameter hyperoptimization | To optimize your strategy, you can use hyperopt parameter hyperoptimization | ||||||
| @@ -194,10 +212,11 @@ optional arguments: | |||||||
| ``` | ``` | ||||||
|  |  | ||||||
| ## A parameter missing in the configuration? | ## A parameter missing in the configuration? | ||||||
|  |  | ||||||
| All parameters for `main.py`, `backtesting`, `hyperopt` are referenced | All parameters for `main.py`, `backtesting`, `hyperopt` are referenced | ||||||
| in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L84) | in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L84) | ||||||
|  |  | ||||||
| ## Next step | ## Next step | ||||||
| The optimal strategy of the bot will change with time depending of the |  | ||||||
| market trends. The next step is to  | The optimal strategy of the bot will change with time depending of the market trends. The next step is to  | ||||||
| [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md). | [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md). | ||||||
|   | |||||||
| @@ -142,6 +142,16 @@ class Arguments(object): | |||||||
|             action='store_true', |             action='store_true', | ||||||
|             dest='refresh_pairs', |             dest='refresh_pairs', | ||||||
|         ) |         ) | ||||||
|  |         parser.add_argument( | ||||||
|  |             '--strategy-list', | ||||||
|  |             help='Provide a commaseparated list of strategies to backtest ' | ||||||
|  |                  'Please note that ticker-interval needs to be set either in config ' | ||||||
|  |                  'or via command line. When using this together with --export trades, ' | ||||||
|  |                  'the strategy-name is injected into the filename ' | ||||||
|  |                  '(so backtest-data.json becomes backtest-data-DefaultStrategy.json', | ||||||
|  |             nargs='+', | ||||||
|  |             dest='strategy_list', | ||||||
|  |         ) | ||||||
|         parser.add_argument( |         parser.add_argument( | ||||||
|             '--export', |             '--export', | ||||||
|             help='export backtest results, argument are: trades\ |             help='export backtest results, argument are: trades\ | ||||||
|   | |||||||
| @@ -187,6 +187,14 @@ class Configuration(object): | |||||||
|             config.update({'refresh_pairs': True}) |             config.update({'refresh_pairs': True}) | ||||||
|             logger.info('Parameter -r/--refresh-pairs-cached detected ...') |             logger.info('Parameter -r/--refresh-pairs-cached detected ...') | ||||||
|  |  | ||||||
|  |         if 'strategy_list' in self.args and self.args.strategy_list: | ||||||
|  |             config.update({'strategy_list': self.args.strategy_list}) | ||||||
|  |             logger.info('Using strategy list of %s Strategies', len(self.args.strategy_list)) | ||||||
|  |  | ||||||
|  |         if 'ticker_interval' in self.args and self.args.ticker_interval: | ||||||
|  |             config.update({'ticker_interval': self.args.ticker_interval}) | ||||||
|  |             logger.info('Overriding ticker interval with Command line argument') | ||||||
|  |  | ||||||
|         # If --export is used we add it to the configuration |         # If --export is used we add it to the configuration | ||||||
|         if 'export' in self.args and self.args.export: |         if 'export' in self.args and self.args.export: | ||||||
|             config.update({'export': self.args.export}) |             config.update({'export': self.args.export}) | ||||||
|   | |||||||
| @@ -6,7 +6,9 @@ This module contains the backtesting logic | |||||||
| import logging | import logging | ||||||
| import operator | import operator | ||||||
| from argparse import Namespace | from argparse import Namespace | ||||||
|  | from copy import deepcopy | ||||||
| from datetime import datetime, timedelta | from datetime import datetime, timedelta | ||||||
|  | from pathlib import Path | ||||||
| from typing import Any, Dict, List, NamedTuple, Optional, Tuple | from typing import Any, Dict, List, NamedTuple, Optional, Tuple | ||||||
|  |  | ||||||
| import arrow | import arrow | ||||||
| @@ -52,13 +54,9 @@ class Backtesting(object): | |||||||
|     backtesting = Backtesting(config) |     backtesting = Backtesting(config) | ||||||
|     backtesting.start() |     backtesting.start() | ||||||
|     """ |     """ | ||||||
|  |  | ||||||
|     def __init__(self, config: Dict[str, Any]) -> None: |     def __init__(self, config: Dict[str, Any]) -> None: | ||||||
|         self.config = config |         self.config = config | ||||||
|         self.strategy: IStrategy = StrategyResolver(self.config).strategy |  | ||||||
|         self.ticker_interval = self.strategy.ticker_interval |  | ||||||
|         self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe |  | ||||||
|         self.advise_buy = self.strategy.advise_buy |  | ||||||
|         self.advise_sell = self.strategy.advise_sell |  | ||||||
|  |  | ||||||
|         # Reset keys for backtesting |         # Reset keys for backtesting | ||||||
|         self.config['exchange']['key'] = '' |         self.config['exchange']['key'] = '' | ||||||
| @@ -66,9 +64,36 @@ class Backtesting(object): | |||||||
|         self.config['exchange']['password'] = '' |         self.config['exchange']['password'] = '' | ||||||
|         self.config['exchange']['uid'] = '' |         self.config['exchange']['uid'] = '' | ||||||
|         self.config['dry_run'] = True |         self.config['dry_run'] = True | ||||||
|  |         self.strategylist: List[IStrategy] = [] | ||||||
|  |         if self.config.get('strategy_list', None): | ||||||
|  |             # Force one interval | ||||||
|  |             self.ticker_interval = str(self.config.get('ticker_interval')) | ||||||
|  |             for strat in list(self.config['strategy_list']): | ||||||
|  |                 stratconf = deepcopy(self.config) | ||||||
|  |                 stratconf['strategy'] = strat | ||||||
|  |                 self.strategylist.append(StrategyResolver(stratconf).strategy) | ||||||
|  |  | ||||||
|  |         else: | ||||||
|  |             # only one strategy | ||||||
|  |             strat = StrategyResolver(self.config).strategy | ||||||
|  |  | ||||||
|  |             self.strategylist.append(StrategyResolver(self.config).strategy) | ||||||
|  |         # Load one strategy | ||||||
|  |         self._set_strategy(self.strategylist[0]) | ||||||
|  |  | ||||||
|         self.exchange = Exchange(self.config) |         self.exchange = Exchange(self.config) | ||||||
|         self.fee = self.exchange.get_fee() |         self.fee = self.exchange.get_fee() | ||||||
|  |  | ||||||
|  |     def _set_strategy(self, strategy): | ||||||
|  |         """ | ||||||
|  |         Load strategy into backtesting | ||||||
|  |         """ | ||||||
|  |         self.strategy = strategy | ||||||
|  |         self.ticker_interval = self.config.get('ticker_interval') | ||||||
|  |         self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe | ||||||
|  |         self.advise_buy = strategy.advise_buy | ||||||
|  |         self.advise_sell = strategy.advise_sell | ||||||
|  |  | ||||||
|     @staticmethod |     @staticmethod | ||||||
|     def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]: |     def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]: | ||||||
|         """ |         """ | ||||||
| @@ -132,7 +157,32 @@ class Backtesting(object): | |||||||
|             tabular_data.append([reason.value,  count]) |             tabular_data.append([reason.value,  count]) | ||||||
|         return tabulate(tabular_data, headers=headers, tablefmt="pipe") |         return tabulate(tabular_data, headers=headers, tablefmt="pipe") | ||||||
|  |  | ||||||
|     def _store_backtest_result(self, recordfilename: Optional[str], results: DataFrame) -> None: |     def _generate_text_table_strategy(self, all_results: dict) -> str: | ||||||
|  |         """ | ||||||
|  |         Generate summary table per strategy | ||||||
|  |         """ | ||||||
|  |         stake_currency = str(self.config.get('stake_currency')) | ||||||
|  |  | ||||||
|  |         floatfmt = ('s', 'd', '.2f', '.2f', '.8f', 'd', '.1f', '.1f') | ||||||
|  |         tabular_data = [] | ||||||
|  |         headers = ['Strategy', 'buy count', 'avg profit %', 'cum profit %', | ||||||
|  |                    'total profit ' + stake_currency, 'avg duration', 'profit', 'loss'] | ||||||
|  |         for strategy, results in all_results.items(): | ||||||
|  |             tabular_data.append([ | ||||||
|  |                 strategy, | ||||||
|  |                 len(results.index), | ||||||
|  |                 results.profit_percent.mean() * 100.0, | ||||||
|  |                 results.profit_percent.sum() * 100.0, | ||||||
|  |                 results.profit_abs.sum(), | ||||||
|  |                 str(timedelta( | ||||||
|  |                     minutes=round(results.trade_duration.mean()))) if not results.empty else '0:00', | ||||||
|  |                 len(results[results.profit_abs > 0]), | ||||||
|  |                 len(results[results.profit_abs < 0]) | ||||||
|  |             ]) | ||||||
|  |         return tabulate(tabular_data, headers=headers, floatfmt=floatfmt, tablefmt="pipe") | ||||||
|  |  | ||||||
|  |     def _store_backtest_result(self, recordfilename: str, results: DataFrame, | ||||||
|  |                                strategyname: Optional[str] = None) -> None: | ||||||
|  |  | ||||||
|         records = [(t.pair, t.profit_percent, t.open_time.timestamp(), |         records = [(t.pair, t.profit_percent, t.open_time.timestamp(), | ||||||
|                     t.close_time.timestamp(), t.open_index - 1, t.trade_duration, |                     t.close_time.timestamp(), t.open_index - 1, t.trade_duration, | ||||||
| @@ -140,6 +190,11 @@ class Backtesting(object): | |||||||
|                    for index, t in results.iterrows()] |                    for index, t in results.iterrows()] | ||||||
|  |  | ||||||
|         if records: |         if records: | ||||||
|  |             if strategyname: | ||||||
|  |                 # Inject strategyname to filename | ||||||
|  |                 recname = Path(recordfilename) | ||||||
|  |                 recordfilename = str(Path.joinpath( | ||||||
|  |                     recname.parent, f'{recname.stem}-{strategyname}').with_suffix(recname.suffix)) | ||||||
|             logger.info('Dumping backtest results to %s', recordfilename) |             logger.info('Dumping backtest results to %s', recordfilename) | ||||||
|             file_dump_json(recordfilename, records) |             file_dump_json(recordfilename, records) | ||||||
|  |  | ||||||
| @@ -307,62 +362,55 @@ class Backtesting(object): | |||||||
|         else: |         else: | ||||||
|             logger.info('Ignoring max_open_trades (--disable-max-market-positions was used) ...') |             logger.info('Ignoring max_open_trades (--disable-max-market-positions was used) ...') | ||||||
|             max_open_trades = 0 |             max_open_trades = 0 | ||||||
|  |         all_results = {} | ||||||
|  |  | ||||||
|         preprocessed = self.tickerdata_to_dataframe(data) |         for strat in self.strategylist: | ||||||
|  |             logger.info("Running backtesting for Strategy %s", strat.get_strategy_name()) | ||||||
|  |             self._set_strategy(strat) | ||||||
|  |  | ||||||
|         # Print timeframe |             # need to reprocess data every time to populate signals | ||||||
|         min_date, max_date = self.get_timeframe(preprocessed) |             preprocessed = self.tickerdata_to_dataframe(data) | ||||||
|         logger.info( |  | ||||||
|             'Measuring data from %s up to %s (%s days)..', |  | ||||||
|             min_date.isoformat(), |  | ||||||
|             max_date.isoformat(), |  | ||||||
|             (max_date - min_date).days |  | ||||||
|         ) |  | ||||||
|  |  | ||||||
|         # Execute backtest and print results |             # Print timeframe | ||||||
|         results = self.backtest( |             min_date, max_date = self.get_timeframe(preprocessed) | ||||||
|             { |             logger.info( | ||||||
|                 'stake_amount': self.config.get('stake_amount'), |                 'Measuring data from %s up to %s (%s days)..', | ||||||
|                 'processed': preprocessed, |                 min_date.isoformat(), | ||||||
|                 'max_open_trades': max_open_trades, |                 max_date.isoformat(), | ||||||
|                 'position_stacking': self.config.get('position_stacking', False), |                 (max_date - min_date).days | ||||||
|             } |  | ||||||
|         ) |  | ||||||
|  |  | ||||||
|         if self.config.get('export', False): |  | ||||||
|             self._store_backtest_result(self.config.get('exportfilename'), results) |  | ||||||
|  |  | ||||||
|         logger.info( |  | ||||||
|             '\n' + '=' * 49 + |  | ||||||
|             ' BACKTESTING REPORT ' + |  | ||||||
|             '=' * 50 + '\n' |  | ||||||
|             '%s', |  | ||||||
|             self._generate_text_table( |  | ||||||
|                 data, |  | ||||||
|                 results |  | ||||||
|             ) |             ) | ||||||
|         ) |  | ||||||
|         # logger.info( |  | ||||||
|         #     results[['sell_reason']].groupby('sell_reason').count() |  | ||||||
|         # ) |  | ||||||
|  |  | ||||||
|         logger.info( |             # Execute backtest and print results | ||||||
|             '\n' + |             all_results[self.strategy.get_strategy_name()] = self.backtest( | ||||||
|             ' SELL READON STATS '.center(119, '=') + |                 { | ||||||
|             '\n%s \n', |                     'stake_amount': self.config.get('stake_amount'), | ||||||
|             self._generate_text_table_sell_reason(data, results) |                     'processed': preprocessed, | ||||||
|  |                     'max_open_trades': max_open_trades, | ||||||
|         ) |                     'position_stacking': self.config.get('position_stacking', False), | ||||||
|  |                 } | ||||||
|         logger.info( |  | ||||||
|             '\n' + |  | ||||||
|             ' LEFT OPEN TRADES REPORT '.center(119, '=') + |  | ||||||
|             '\n%s', |  | ||||||
|             self._generate_text_table( |  | ||||||
|                 data, |  | ||||||
|                 results.loc[results.open_at_end] |  | ||||||
|             ) |             ) | ||||||
|         ) |  | ||||||
|  |         for strategy, results in all_results.items(): | ||||||
|  |  | ||||||
|  |             if self.config.get('export', False): | ||||||
|  |                 self._store_backtest_result(self.config['exportfilename'], results, | ||||||
|  |                                             strategy if len(self.strategylist) > 1 else None) | ||||||
|  |  | ||||||
|  |             print(f"Result for strategy {strategy}") | ||||||
|  |             print(' BACKTESTING REPORT '.center(119, '=')) | ||||||
|  |             print(self._generate_text_table(data, results)) | ||||||
|  |  | ||||||
|  |             print(' SELL REASON STATS '.center(119, '=')) | ||||||
|  |             print(self._generate_text_table_sell_reason(data, results)) | ||||||
|  |  | ||||||
|  |             print(' LEFT OPEN TRADES REPORT '.center(119, '=')) | ||||||
|  |             print(self._generate_text_table(data, results.loc[results.open_at_end])) | ||||||
|  |             print() | ||||||
|  |         if len(all_results) > 1: | ||||||
|  |             # Print Strategy summary table | ||||||
|  |             print(' Strategy Summary '.center(119, '=')) | ||||||
|  |             print(self._generate_text_table_strategy(all_results)) | ||||||
|  |             print('\nFor more details, please look at the detail tables above') | ||||||
|  |  | ||||||
|  |  | ||||||
| def setup_configuration(args: Namespace) -> Dict[str, Any]: | def setup_configuration(args: Namespace) -> Dict[str, Any]: | ||||||
|   | |||||||
| @@ -406,6 +406,50 @@ def test_generate_text_table_sell_reason(default_conf, mocker): | |||||||
|         data={'ETH/BTC': {}}, results=results) == result_str |         data={'ETH/BTC': {}}, results=results) == result_str | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_generate_text_table_strategyn(default_conf, mocker): | ||||||
|  |     """ | ||||||
|  |     Test Backtesting.generate_text_table_sell_reason() method | ||||||
|  |     """ | ||||||
|  |     patch_exchange(mocker) | ||||||
|  |     backtesting = Backtesting(default_conf) | ||||||
|  |     results = {} | ||||||
|  |     results['ETH/BTC'] = pd.DataFrame( | ||||||
|  |         { | ||||||
|  |             'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'], | ||||||
|  |             'profit_percent': [0.1, 0.2, 0.3], | ||||||
|  |             'profit_abs': [0.2, 0.4, 0.5], | ||||||
|  |             'trade_duration': [10, 30, 10], | ||||||
|  |             'profit': [2, 0, 0], | ||||||
|  |             'loss': [0, 0, 1], | ||||||
|  |             'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS] | ||||||
|  |         } | ||||||
|  |     ) | ||||||
|  |     results['LTC/BTC'] = pd.DataFrame( | ||||||
|  |         { | ||||||
|  |             'pair': ['LTC/BTC', 'LTC/BTC', 'LTC/BTC'], | ||||||
|  |             'profit_percent': [0.4, 0.2, 0.3], | ||||||
|  |             'profit_abs': [0.4, 0.4, 0.5], | ||||||
|  |             'trade_duration': [15, 30, 15], | ||||||
|  |             'profit': [4, 1, 0], | ||||||
|  |             'loss': [0, 0, 1], | ||||||
|  |             'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS] | ||||||
|  |         } | ||||||
|  |     ) | ||||||
|  |  | ||||||
|  |     result_str = ( | ||||||
|  |         '| Strategy   |   buy count |   avg profit % |   cum profit % ' | ||||||
|  |         '|   total profit BTC | avg duration   |   profit |   loss |\n' | ||||||
|  |         '|:-----------|------------:|---------------:|---------------:' | ||||||
|  |         '|-------------------:|:---------------|---------:|-------:|\n' | ||||||
|  |         '| ETH/BTC    |           3 |          20.00 |          60.00 ' | ||||||
|  |         '|         1.10000000 | 0:17:00        |        3 |      0 |\n' | ||||||
|  |         '| LTC/BTC    |           3 |          30.00 |          90.00 ' | ||||||
|  |         '|         1.30000000 | 0:20:00        |        3 |      0 |' | ||||||
|  |     ) | ||||||
|  |     print(backtesting._generate_text_table_strategy(all_results=results)) | ||||||
|  |     assert backtesting._generate_text_table_strategy(all_results=results) == result_str | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_backtesting_start(default_conf, mocker, caplog) -> None: | def test_backtesting_start(default_conf, mocker, caplog) -> None: | ||||||
|     def get_timeframe(input1, input2): |     def get_timeframe(input1, input2): | ||||||
|         return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59) |         return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59) | ||||||
| @@ -654,6 +698,18 @@ def test_backtest_record(default_conf, fee, mocker): | |||||||
|     records = records[0] |     records = records[0] | ||||||
|     # Ensure records are of correct type |     # Ensure records are of correct type | ||||||
|     assert len(records) == 4 |     assert len(records) == 4 | ||||||
|  |  | ||||||
|  |     # reset test to test with strategy name | ||||||
|  |     names = [] | ||||||
|  |     records = [] | ||||||
|  |     backtesting._store_backtest_result("backtest-result.json", results, "DefStrat") | ||||||
|  |     assert len(results) == 4 | ||||||
|  |     # Assert file_dump_json was only called once | ||||||
|  |     assert names == ['backtest-result-DefStrat.json'] | ||||||
|  |     records = records[0] | ||||||
|  |     # Ensure records are of correct type | ||||||
|  |     assert len(records) == 4 | ||||||
|  |  | ||||||
|     # ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117) |     # ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117) | ||||||
|     # Below follows just a typecheck of the schema/type of trade-records |     # Below follows just a typecheck of the schema/type of trade-records | ||||||
|     oix = None |     oix = None | ||||||
| @@ -686,15 +742,6 @@ def test_backtest_start_live(default_conf, mocker, caplog): | |||||||
|         read_data=json.dumps(default_conf) |         read_data=json.dumps(default_conf) | ||||||
|     )) |     )) | ||||||
|  |  | ||||||
|     args = MagicMock() |  | ||||||
|     args.ticker_interval = 1 |  | ||||||
|     args.level = 10 |  | ||||||
|     args.live = True |  | ||||||
|     args.datadir = None |  | ||||||
|     args.export = None |  | ||||||
|     args.strategy = 'DefaultStrategy' |  | ||||||
|     args.timerange = '-100'  # needed due to MagicMock malleability |  | ||||||
|  |  | ||||||
|     args = [ |     args = [ | ||||||
|         '--config', 'config.json', |         '--config', 'config.json', | ||||||
|         '--strategy', 'DefaultStrategy', |         '--strategy', 'DefaultStrategy', | ||||||
| @@ -725,3 +772,60 @@ def test_backtest_start_live(default_conf, mocker, caplog): | |||||||
|  |  | ||||||
|     for line in exists: |     for line in exists: | ||||||
|         assert log_has(line, caplog.record_tuples) |         assert log_has(line, caplog.record_tuples) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_backtest_start_multi_strat(default_conf, mocker, caplog): | ||||||
|  |     default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] | ||||||
|  |     mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', | ||||||
|  |                  new=lambda s, n, i: _load_pair_as_ticks(n, i)) | ||||||
|  |     patch_exchange(mocker) | ||||||
|  |     backtestmock = MagicMock() | ||||||
|  |     mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest', backtestmock) | ||||||
|  |     gen_table_mock = MagicMock() | ||||||
|  |     mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table', gen_table_mock) | ||||||
|  |     gen_strattable_mock = MagicMock() | ||||||
|  |     mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table_strategy', | ||||||
|  |                  gen_strattable_mock) | ||||||
|  |     mocker.patch('freqtrade.configuration.open', mocker.mock_open( | ||||||
|  |         read_data=json.dumps(default_conf) | ||||||
|  |     )) | ||||||
|  |  | ||||||
|  |     args = [ | ||||||
|  |         '--config', 'config.json', | ||||||
|  |         '--datadir', 'freqtrade/tests/testdata', | ||||||
|  |         'backtesting', | ||||||
|  |         '--ticker-interval', '1m', | ||||||
|  |         '--live', | ||||||
|  |         '--timerange', '-100', | ||||||
|  |         '--enable-position-stacking', | ||||||
|  |         '--disable-max-market-positions', | ||||||
|  |         '--strategy-list', | ||||||
|  |         'DefaultStrategy', | ||||||
|  |         'TestStrategy', | ||||||
|  |     ] | ||||||
|  |     args = get_args(args) | ||||||
|  |     start(args) | ||||||
|  |     # 2 backtests, 4 tables | ||||||
|  |     assert backtestmock.call_count == 2 | ||||||
|  |     assert gen_table_mock.call_count == 4 | ||||||
|  |     assert gen_strattable_mock.call_count == 1 | ||||||
|  |  | ||||||
|  |     # check the logs, that will contain the backtest result | ||||||
|  |     exists = [ | ||||||
|  |         'Parameter -i/--ticker-interval detected ...', | ||||||
|  |         'Using ticker_interval: 1m ...', | ||||||
|  |         'Parameter -l/--live detected ...', | ||||||
|  |         'Ignoring max_open_trades (--disable-max-market-positions was used) ...', | ||||||
|  |         'Parameter --timerange detected: -100 ...', | ||||||
|  |         'Using data folder: freqtrade/tests/testdata ...', | ||||||
|  |         'Using stake_currency: BTC ...', | ||||||
|  |         'Using stake_amount: 0.001 ...', | ||||||
|  |         'Downloading data for all pairs in whitelist ...', | ||||||
|  |         'Measuring data from 2017-11-14T19:31:00+00:00 up to 2017-11-14T22:58:00+00:00 (0 days)..', | ||||||
|  |         'Parameter --enable-position-stacking detected ...', | ||||||
|  |         'Running backtesting for Strategy DefaultStrategy', | ||||||
|  |         'Running backtesting for Strategy TestStrategy', | ||||||
|  |     ] | ||||||
|  |  | ||||||
|  |     for line in exists: | ||||||
|  |         assert log_has(line, caplog.record_tuples) | ||||||
|   | |||||||
| @@ -132,7 +132,11 @@ def test_parse_args_backtesting_custom() -> None: | |||||||
|         'backtesting', |         'backtesting', | ||||||
|         '--live', |         '--live', | ||||||
|         '--ticker-interval', '1m', |         '--ticker-interval', '1m', | ||||||
|         '--refresh-pairs-cached'] |         '--refresh-pairs-cached', | ||||||
|  |         '--strategy-list', | ||||||
|  |         'DefaultStrategy', | ||||||
|  |         'TestStrategy' | ||||||
|  |         ] | ||||||
|     call_args = Arguments(args, '').get_parsed_arg() |     call_args = Arguments(args, '').get_parsed_arg() | ||||||
|     assert call_args.config == 'test_conf.json' |     assert call_args.config == 'test_conf.json' | ||||||
|     assert call_args.live is True |     assert call_args.live is True | ||||||
| @@ -141,6 +145,8 @@ def test_parse_args_backtesting_custom() -> None: | |||||||
|     assert call_args.func is not None |     assert call_args.func is not None | ||||||
|     assert call_args.ticker_interval == '1m' |     assert call_args.ticker_interval == '1m' | ||||||
|     assert call_args.refresh_pairs is True |     assert call_args.refresh_pairs is True | ||||||
|  |     assert type(call_args.strategy_list) is list | ||||||
|  |     assert len(call_args.strategy_list) == 2 | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_parse_args_hyperopt_custom() -> None: | def test_parse_args_hyperopt_custom() -> None: | ||||||
|   | |||||||
| @@ -292,6 +292,61 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non | |||||||
|     ) |     ) | ||||||
|  |  | ||||||
|  |  | ||||||
|  | def test_setup_configuration_with_stratlist(mocker, default_conf, caplog) -> None: | ||||||
|  |     """ | ||||||
|  |     Test setup_configuration() function | ||||||
|  |     """ | ||||||
|  |     mocker.patch('freqtrade.configuration.open', mocker.mock_open( | ||||||
|  |         read_data=json.dumps(default_conf) | ||||||
|  |     )) | ||||||
|  |  | ||||||
|  |     arglist = [ | ||||||
|  |         '--config', 'config.json', | ||||||
|  |         'backtesting', | ||||||
|  |         '--ticker-interval', '1m', | ||||||
|  |         '--export', '/bar/foo', | ||||||
|  |         '--strategy-list', | ||||||
|  |         'DefaultStrategy', | ||||||
|  |         'TestStrategy' | ||||||
|  |     ] | ||||||
|  |  | ||||||
|  |     args = Arguments(arglist, '').get_parsed_arg() | ||||||
|  |  | ||||||
|  |     configuration = Configuration(args) | ||||||
|  |     config = configuration.get_config() | ||||||
|  |     assert 'max_open_trades' in config | ||||||
|  |     assert 'stake_currency' in config | ||||||
|  |     assert 'stake_amount' in config | ||||||
|  |     assert 'exchange' in config | ||||||
|  |     assert 'pair_whitelist' in config['exchange'] | ||||||
|  |     assert 'datadir' in config | ||||||
|  |     assert log_has( | ||||||
|  |         'Using data folder: {} ...'.format(config['datadir']), | ||||||
|  |         caplog.record_tuples | ||||||
|  |     ) | ||||||
|  |     assert 'ticker_interval' in config | ||||||
|  |     assert log_has('Parameter -i/--ticker-interval detected ...', caplog.record_tuples) | ||||||
|  |     assert log_has( | ||||||
|  |         'Using ticker_interval: 1m ...', | ||||||
|  |         caplog.record_tuples | ||||||
|  |     ) | ||||||
|  |  | ||||||
|  |     assert 'strategy_list' in config | ||||||
|  |     assert log_has('Using strategy list of 2 Strategies', caplog.record_tuples) | ||||||
|  |  | ||||||
|  |     assert 'position_stacking' not in config | ||||||
|  |  | ||||||
|  |     assert 'use_max_market_positions' not in config | ||||||
|  |  | ||||||
|  |     assert 'timerange' not in config | ||||||
|  |  | ||||||
|  |     assert 'export' in config | ||||||
|  |     assert log_has( | ||||||
|  |         'Parameter --export detected: {} ...'.format(config['export']), | ||||||
|  |         caplog.record_tuples | ||||||
|  |     ) | ||||||
|  |  | ||||||
|  |  | ||||||
| def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None: | def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None: | ||||||
|     mocker.patch('freqtrade.configuration.open', mocker.mock_open( |     mocker.patch('freqtrade.configuration.open', mocker.mock_open( | ||||||
|         read_data=json.dumps(default_conf) |         read_data=json.dumps(default_conf) | ||||||
|   | |||||||
		Reference in New Issue
	
	Block a user