Merge pull request #3807 from freqtrade/hyperopt_disablecontinue
Hyperopt disablecontinue
This commit is contained in:
Matthias
GitHub
commit
c410599a52
@ -27,9 +27,9 @@ class MyAwesomeHyperOpt2(MyAwesomeHyperOpt):
|
||||
and then quickly switch between hyperopt classes, running optimization process with hyperopt class you need in each particular case:
|
||||
|
||||
```
|
||||
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt ...
|
||||
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt --strategy MyAwesomeStrategy ...
|
||||
or
|
||||
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt2 ...
|
||||
$ freqtrade hyperopt --hyperopt MyAwesomeHyperOpt2 --strategy MyAwesomeStrategy ...
|
||||
```
|
||||
|
||||
## Creating and using a custom loss function
|
||||
|
||||
@ -303,7 +303,7 @@ usage: freqtrade hyperopt [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH]
|
||||
[--spaces {all,buy,sell,roi,stoploss,trailing,default} [{all,buy,sell,roi,stoploss,trailing,default} ...]]
|
||||
[--dmmp] [--print-all] [--no-color] [--print-json]
|
||||
[-j JOBS] [--random-state INT] [--min-trades INT]
|
||||
[--continue] [--hyperopt-loss NAME]
|
||||
[--hyperopt-loss NAME]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
@ -349,9 +349,6 @@ optional arguments:
|
||||
reproducible hyperopt results.
|
||||
--min-trades INT Set minimal desired number of trades for evaluations
|
||||
in the hyperopt optimization path (default: 1).
|
||||
--continue Continue hyperopt from previous runs. By default,
|
||||
temporary files will be removed and hyperopt will
|
||||
start from scratch.
|
||||
--hyperopt-loss NAME Specify the class name of the hyperopt loss function
|
||||
class (IHyperOptLoss). Different functions can
|
||||
generate completely different results, since the
|
||||
|
||||
@ -140,13 +140,7 @@ Since hyperopt uses Bayesian search, running for too many epochs may not produce
|
||||
It's therefore recommended to run between 500-1000 epochs over and over until you hit at least 10.000 epochs in total (or are satisfied with the result). You can best judge by looking at the results - if the bot keeps discovering better strategies, it's best to keep on going.
|
||||
|
||||
```bash
|
||||
freqtrade hyperopt -e 1000
|
||||
```
|
||||
|
||||
or if you want intermediate result to see
|
||||
|
||||
```bash
|
||||
for i in {1..100}; do freqtrade hyperopt -e 1000; done
|
||||
freqtrade hyperopt --hyperop SampleHyperopt --strategy SampleStrategy -e 1000
|
||||
```
|
||||
|
||||
### Why does it take a long time to run hyperopt?
|
||||
|
||||
@ -102,7 +102,7 @@ There you have two different types of indicators: 1. `guards` and 2. `triggers`.
|
||||
1. Guards are conditions like "never buy if ADX < 10", or never buy if current price is over EMA10.
|
||||
2. Triggers are ones that actually trigger buy in specific moment, like "buy when EMA5 crosses over EMA10" or "buy when close price touches lower Bollinger band".
|
||||
|
||||
Hyperoptimization will, for each eval round, pick one trigger and possibly
|
||||
Hyper-optimization will, for each epoch round, pick one trigger and possibly
|
||||
multiple guards. The constructed strategy will be something like
|
||||
"*buy exactly when close price touches lower Bollinger band, BUT only if
|
||||
ADX > 10*".
|
||||
@ -229,7 +229,7 @@ Because hyperopt tries a lot of combinations to find the best parameters it will
|
||||
We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
|
||||
|
||||
```bash
|
||||
freqtrade hyperopt --config config.json --hyperopt <hyperoptname> -e 500 --spaces all
|
||||
freqtrade hyperopt --config config.json --hyperopt <hyperoptname> --strategy <strategyname> -e 500 --spaces all
|
||||
```
|
||||
|
||||
Use `<hyperoptname>` as the name of the custom hyperopt used.
|
||||
@ -240,10 +240,8 @@ running at least several thousand evaluations.
|
||||
The `--spaces all` option determines that all possible parameters should be optimized. Possibilities are listed below.
|
||||
|
||||
!!! Note
|
||||
By default, hyperopt will erase previous results and start from scratch. Continuation can be archived by using `--continue`.
|
||||
|
||||
!!! Warning
|
||||
When switching parameters or changing configuration options, make sure to not use the argument `--continue` so temporary results can be removed.
|
||||
Hyperopt will store hyperopt results with the timestamp of the hyperopt start time.
|
||||
Reading commands (`hyperopt-list`, `hyperopt-show`) can use `--hyperopt-filename <filename>` to read and display older hyperopt results.
|
||||
|
||||
### Execute Hyperopt with different historical data source
|
||||
|
||||
@ -253,11 +251,11 @@ uses data from directory `user_data/data`.
|
||||
|
||||
### Running Hyperopt with Smaller Testset
|
||||
|
||||
Use the `--timerange` argument to change how much of the testset you want to use.
|
||||
Use the `--timerange` argument to change how much of the test-set you want to use.
|
||||
For example, to use one month of data, pass the following parameter to the hyperopt call:
|
||||
|
||||
```bash
|
||||
freqtrade hyperopt --timerange 20180401-20180501
|
||||
freqtrade hyperopt --hyperopt <hyperoptname> --strategy <strategyname> --timerange 20180401-20180501
|
||||
```
|
||||
|
||||
### Running Hyperopt using methods from a strategy
|
||||
@ -318,7 +316,7 @@ The initial state for generation of these random values (random state) is contro
|
||||
|
||||
If you have not set this value explicitly in the command line options, Hyperopt seeds the random state with some random value for you. The random state value for each Hyperopt run is shown in the log, so you can copy and paste it into the `--random-state` command line option to repeat the set of the initial random epochs used.
|
||||
|
||||
If you have not changed anything in the command line options, configuration, timerange, Strategy and Hyperopt classes, historical data and the Loss Function -- you should obtain same hyperoptimization results with same random state value used.
|
||||
If you have not changed anything in the command line options, configuration, timerange, Strategy and Hyperopt classes, historical data and the Loss Function -- you should obtain same hyper-optimization results with same random state value used.
|
||||
|
||||
## Understand the Hyperopt Result
|
||||
|
||||
@ -371,7 +369,7 @@ By default, hyperopt prints colorized results -- epochs with positive profit are
|
||||
You can use the `--print-all` command line option if you would like to see all results in the hyperopt output, not only the best ones. When `--print-all` is used, current best results are also colorized by default -- they are printed in bold (bright) style. This can also be switched off with the `--no-color` command line option.
|
||||
|
||||
!!! Note "Windows and color output"
|
||||
Windows does not support color-output nativly, therefore it is automatically disabled. To have color-output for hyperopt running under windows, please consider using WSL.
|
||||
Windows does not support color-output natively, therefore it is automatically disabled. To have color-output for hyperopt running under windows, please consider using WSL.
|
||||
|
||||
### Understand Hyperopt ROI results
|
||||
|
||||
@ -494,7 +492,7 @@ Override the `trailing_space()` method and define the desired range in it if you
|
||||
|
||||
## Show details of Hyperopt results
|
||||
|
||||
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` subcommands. The usage of these subcommands is described in the [Utils](utils.md#list-hyperopt-results) chapter.
|
||||
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter.
|
||||
|
||||
## Validate backtesting results
|
||||
|
||||
|
||||
@ -423,7 +423,7 @@ freqtrade test-pairlist --config config.json --quote USDT BTC
|
||||
|
||||
## List Hyperopt results
|
||||
|
||||
You can list the hyperoptimization epochs the Hyperopt module evaluated previously with the `hyperopt-list` subcommand.
|
||||
You can list the hyperoptimization epochs the Hyperopt module evaluated previously with the `hyperopt-list` sub-command.
|
||||
|
||||
```
|
||||
usage: freqtrade hyperopt-list [-h] [-v] [--logfile FILE] [-V] [-c PATH]
|
||||
@ -432,10 +432,11 @@ usage: freqtrade hyperopt-list [-h] [-v] [--logfile FILE] [-V] [-c PATH]
|
||||
[--max-trades INT] [--min-avg-time FLOAT]
|
||||
[--max-avg-time FLOAT] [--min-avg-profit FLOAT]
|
||||
[--max-avg-profit FLOAT]
|
||||
[--min-total-profit FLOAT] [--max-total-profit FLOAT]
|
||||
[--min-total-profit FLOAT]
|
||||
[--max-total-profit FLOAT]
|
||||
[--min-objective FLOAT] [--max-objective FLOAT]
|
||||
[--no-color] [--print-json] [--no-details]
|
||||
[--export-csv FILE]
|
||||
[--hyperopt-filename PATH] [--export-csv FILE]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
@ -443,24 +444,27 @@ optional arguments:
|
||||
--profitable Select only profitable epochs.
|
||||
--min-trades INT Select epochs with more than INT trades.
|
||||
--max-trades INT Select epochs with less than INT trades.
|
||||
--min-avg-time FLOAT Select epochs on above average time.
|
||||
--max-avg-time FLOAT Select epochs on under average time.
|
||||
--min-avg-time FLOAT Select epochs above average time.
|
||||
--max-avg-time FLOAT Select epochs below average time.
|
||||
--min-avg-profit FLOAT
|
||||
Select epochs on above average profit.
|
||||
Select epochs above average profit.
|
||||
--max-avg-profit FLOAT
|
||||
Select epochs on below average profit.
|
||||
Select epochs below average profit.
|
||||
--min-total-profit FLOAT
|
||||
Select epochs on above total profit.
|
||||
Select epochs above total profit.
|
||||
--max-total-profit FLOAT
|
||||
Select epochs on below total profit.
|
||||
Select epochs below total profit.
|
||||
--min-objective FLOAT
|
||||
Select epochs on above objective (- is added by default).
|
||||
Select epochs above objective.
|
||||
--max-objective FLOAT
|
||||
Select epochs on below objective (- is added by default).
|
||||
Select epochs below objective.
|
||||
--no-color Disable colorization of hyperopt results. May be
|
||||
useful if you are redirecting output to a file.
|
||||
--print-json Print best result detailization in JSON format.
|
||||
--print-json Print output in JSON format.
|
||||
--no-details Do not print best epoch details.
|
||||
--hyperopt-filename FILENAME
|
||||
Hyperopt result filename.Example: `--hyperopt-
|
||||
filename=hyperopt_results_2020-09-27_16-20-48.pickle`
|
||||
--export-csv FILE Export to CSV-File. This will disable table print.
|
||||
Example: --export-csv hyperopt.csv
|
||||
|
||||
@ -481,6 +485,10 @@ Common arguments:
|
||||
Path to userdata directory.
|
||||
```
|
||||
|
||||
!!! Note
|
||||
`hyperopt-list` will automatically use the latest available hyperopt results file.
|
||||
You can override this using the `--hyperopt-filename` argument, and specify another, available filename (without path!).
|
||||
|
||||
### Examples
|
||||
|
||||
List all results, print details of the best result at the end:
|
||||
@ -501,17 +509,41 @@ You can show the details of any hyperoptimization epoch previously evaluated by
|
||||
usage: freqtrade hyperopt-show [-h] [-v] [--logfile FILE] [-V] [-c PATH]
|
||||
[-d PATH] [--userdir PATH] [--best]
|
||||
[--profitable] [-n INT] [--print-json]
|
||||
[--no-header]
|
||||
[--hyperopt-filename PATH] [--no-header]
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--best Select only best epochs.
|
||||
--profitable Select only profitable epochs.
|
||||
-n INT, --index INT Specify the index of the epoch to print details for.
|
||||
--print-json Print best result detailization in JSON format.
|
||||
--print-json Print output in JSON format.
|
||||
--hyperopt-filename FILENAME
|
||||
Hyperopt result filename.Example: `--hyperopt-
|
||||
filename=hyperopt_results_2020-09-27_16-20-48.pickle`
|
||||
--no-header Do not print epoch details header.
|
||||
|
||||
Common arguments:
|
||||
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
|
||||
--logfile FILE Log to the file specified. Special values are:
|
||||
'syslog', 'journald'. See the documentation for more
|
||||
details.
|
||||
-V, --version show program's version number and exit
|
||||
-c PATH, --config PATH
|
||||
Specify configuration file (default:
|
||||
`userdir/config.json` or `config.json` whichever
|
||||
exists). Multiple --config options may be used. Can be
|
||||
set to `-` to read config from stdin.
|
||||
-d PATH, --datadir PATH
|
||||
Path to directory with historical backtesting data.
|
||||
--userdir PATH, --user-data-dir PATH
|
||||
Path to userdata directory.
|
||||
|
||||
```
|
||||
|
||||
!!! Note
|
||||
`hyperopt-show` will automatically use the latest available hyperopt results file.
|
||||
You can override this using the `--hyperopt-filename` argument, and specify another, available filename (without path!).
|
||||
|
||||
### Examples
|
||||
|
||||
Print details for the epoch 168 (the number of the epoch is shown by the `hyperopt-list` subcommand or by Hyperopt itself during hyperoptimization run):
|
||||
|
||||
@ -26,7 +26,7 @@ ARGS_HYPEROPT = ARGS_COMMON_OPTIMIZE + ["hyperopt", "hyperopt_path",
|
||||
"use_max_market_positions", "print_all",
|
||||
"print_colorized", "print_json", "hyperopt_jobs",
|
||||
"hyperopt_random_state", "hyperopt_min_trades",
|
||||
"hyperopt_continue", "hyperopt_loss"]
|
||||
"hyperopt_loss"]
|
||||
|
||||
ARGS_EDGE = ARGS_COMMON_OPTIMIZE + ["stoploss_range"]
|
||||
|
||||
@ -75,10 +75,10 @@ ARGS_HYPEROPT_LIST = ["hyperopt_list_best", "hyperopt_list_profitable",
|
||||
"hyperopt_list_min_total_profit", "hyperopt_list_max_total_profit",
|
||||
"hyperopt_list_min_objective", "hyperopt_list_max_objective",
|
||||
"print_colorized", "print_json", "hyperopt_list_no_details",
|
||||
"export_csv"]
|
||||
"hyperoptexportfilename", "export_csv"]
|
||||
|
||||
ARGS_HYPEROPT_SHOW = ["hyperopt_list_best", "hyperopt_list_profitable", "hyperopt_show_index",
|
||||
"print_json", "hyperopt_show_no_header"]
|
||||
"print_json", "hyperoptexportfilename", "hyperopt_show_no_header"]
|
||||
|
||||
NO_CONF_REQURIED = ["convert-data", "convert-trade-data", "download-data", "list-timeframes",
|
||||
"list-markets", "list-pairs", "list-strategies", "list-data",
|
||||
|
||||
@ -252,13 +252,6 @@ AVAILABLE_CLI_OPTIONS = {
|
||||
metavar='INT',
|
||||
default=1,
|
||||
),
|
||||
"hyperopt_continue": Arg(
|
||||
"--continue",
|
||||
help="Continue hyperopt from previous runs. "
|
||||
"By default, temporary files will be removed and hyperopt will start from scratch.",
|
||||
default=False,
|
||||
action='store_true',
|
||||
),
|
||||
"hyperopt_loss": Arg(
|
||||
'--hyperopt-loss',
|
||||
help='Specify the class name of the hyperopt loss function class (IHyperOptLoss). '
|
||||
@ -270,6 +263,12 @@ AVAILABLE_CLI_OPTIONS = {
|
||||
metavar='NAME',
|
||||
default=constants.DEFAULT_HYPEROPT_LOSS,
|
||||
),
|
||||
"hyperoptexportfilename": Arg(
|
||||
'--hyperopt-filename',
|
||||
help='Hyperopt result filename.'
|
||||
'Example: `--hyperopt-filename=hyperopt_results_2020-09-27_16-20-48.pickle`',
|
||||
metavar='FILENAME',
|
||||
),
|
||||
# List exchanges
|
||||
"print_one_column": Arg(
|
||||
'-1', '--one-column',
|
||||
|
||||
@ -7,6 +7,7 @@ from colorama import init as colorama_init
|
||||
from freqtrade.configuration import setup_utils_configuration
|
||||
from freqtrade.exceptions import OperationalException
|
||||
from freqtrade.state import RunMode
|
||||
from freqtrade.data.btanalysis import get_latest_hyperopt_file
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@ -40,8 +41,9 @@ def start_hyperopt_list(args: Dict[str, Any]) -> None:
|
||||
'filter_max_objective': config.get('hyperopt_list_max_objective', None),
|
||||
}
|
||||
|
||||
results_file = (config['user_data_dir'] /
|
||||
'hyperopt_results' / 'hyperopt_results.pickle')
|
||||
results_file = get_latest_hyperopt_file(
|
||||
config['user_data_dir'] / 'hyperopt_results',
|
||||
config.get('hyperoptexportfilename'))
|
||||
|
||||
# Previous evaluations
|
||||
epochs = Hyperopt.load_previous_results(results_file)
|
||||
@ -80,8 +82,10 @@ def start_hyperopt_show(args: Dict[str, Any]) -> None:
|
||||
|
||||
print_json = config.get('print_json', False)
|
||||
no_header = config.get('hyperopt_show_no_header', False)
|
||||
results_file = (config['user_data_dir'] /
|
||||
'hyperopt_results' / 'hyperopt_results.pickle')
|
||||
results_file = get_latest_hyperopt_file(
|
||||
config['user_data_dir'] / 'hyperopt_results',
|
||||
config.get('hyperoptexportfilename'))
|
||||
|
||||
n = config.get('hyperopt_show_index', -1)
|
||||
|
||||
filteroptions = {
|
||||
|
||||
@ -263,6 +263,9 @@ class Configuration:
|
||||
self._args_to_config(config, argname='hyperopt_path',
|
||||
logstring='Using additional Hyperopt lookup path: {}')
|
||||
|
||||
self._args_to_config(config, argname='hyperoptexportfilename',
|
||||
logstring='Using hyperopt file: {}')
|
||||
|
||||
self._args_to_config(config, argname='epochs',
|
||||
logstring='Parameter --epochs detected ... '
|
||||
'Will run Hyperopt with for {} epochs ...'
|
||||
@ -295,9 +298,6 @@ class Configuration:
|
||||
self._args_to_config(config, argname='hyperopt_min_trades',
|
||||
logstring='Parameter --min-trades detected: {}')
|
||||
|
||||
self._args_to_config(config, argname='hyperopt_continue',
|
||||
logstring='Hyperopt continue: {}')
|
||||
|
||||
self._args_to_config(config, argname='hyperopt_loss',
|
||||
logstring='Using Hyperopt loss class name: {}')
|
||||
|
||||
|
||||
@ -21,10 +21,11 @@ BT_DATA_COLUMNS = ["pair", "profit_percent", "open_date", "close_date", "index",
|
||||
"open_rate", "close_rate", "open_at_end", "sell_reason"]
|
||||
|
||||
|
||||
def get_latest_backtest_filename(directory: Union[Path, str]) -> str:
|
||||
def get_latest_optimize_filename(directory: Union[Path, str], variant: str) -> str:
|
||||
"""
|
||||
Get latest backtest export based on '.last_result.json'.
|
||||
:param directory: Directory to search for last result
|
||||
:param variant: 'backtest' or 'hyperopt' - the method to return
|
||||
:return: string containing the filename of the latest backtest result
|
||||
:raises: ValueError in the following cases:
|
||||
* Directory does not exist
|
||||
@ -44,10 +45,57 @@ def get_latest_backtest_filename(directory: Union[Path, str]) -> str:
|
||||
with filename.open() as file:
|
||||
data = json_load(file)
|
||||
|
||||
if 'latest_backtest' not in data:
|
||||
if f'latest_{variant}' not in data:
|
||||
raise ValueError(f"Invalid '{LAST_BT_RESULT_FN}' format.")
|
||||
|
||||
return data['latest_backtest']
|
||||
return data[f'latest_{variant}']
|
||||
|
||||
|
||||
def get_latest_backtest_filename(directory: Union[Path, str]) -> str:
|
||||
"""
|
||||
Get latest backtest export based on '.last_result.json'.
|
||||
:param directory: Directory to search for last result
|
||||
:return: string containing the filename of the latest backtest result
|
||||
:raises: ValueError in the following cases:
|
||||
* Directory does not exist
|
||||
* `directory/.last_result.json` does not exist
|
||||
* `directory/.last_result.json` has the wrong content
|
||||
"""
|
||||
return get_latest_optimize_filename(directory, 'backtest')
|
||||
|
||||
|
||||
def get_latest_hyperopt_filename(directory: Union[Path, str]) -> str:
|
||||
"""
|
||||
Get latest hyperopt export based on '.last_result.json'.
|
||||
:param directory: Directory to search for last result
|
||||
:return: string containing the filename of the latest hyperopt result
|
||||
:raises: ValueError in the following cases:
|
||||
* Directory does not exist
|
||||
* `directory/.last_result.json` does not exist
|
||||
* `directory/.last_result.json` has the wrong content
|
||||
"""
|
||||
try:
|
||||
return get_latest_optimize_filename(directory, 'hyperopt')
|
||||
except ValueError:
|
||||
# Return default (legacy) pickle filename
|
||||
return 'hyperopt_results.pickle'
|
||||
|
||||
|
||||
def get_latest_hyperopt_file(directory: Union[Path, str], predef_filename: str = None) -> Path:
|
||||
"""
|
||||
Get latest hyperopt export based on '.last_result.json'.
|
||||
:param directory: Directory to search for last result
|
||||
:return: string containing the filename of the latest hyperopt result
|
||||
:raises: ValueError in the following cases:
|
||||
* Directory does not exist
|
||||
* `directory/.last_result.json` does not exist
|
||||
* `directory/.last_result.json` has the wrong content
|
||||
"""
|
||||
if isinstance(directory, str):
|
||||
directory = Path(directory)
|
||||
if predef_filename:
|
||||
return directory / predef_filename
|
||||
return directory / get_latest_hyperopt_filename(directory)
|
||||
|
||||
|
||||
def load_backtest_stats(filename: Union[Path, str]) -> Dict[str, Any]:
|
||||
|
||||
@ -10,6 +10,7 @@ import logging
|
||||
import random
|
||||
import warnings
|
||||
from collections import OrderedDict
|
||||
from datetime import datetime
|
||||
from math import ceil
|
||||
from operator import itemgetter
|
||||
from pathlib import Path
|
||||
@ -25,16 +26,15 @@ from joblib import (Parallel, cpu_count, delayed, dump, load,
|
||||
wrap_non_picklable_objects)
|
||||
from pandas import DataFrame, isna, json_normalize
|
||||
|
||||
from freqtrade.constants import DATETIME_PRINT_FORMAT
|
||||
from freqtrade.constants import DATETIME_PRINT_FORMAT, LAST_BT_RESULT_FN
|
||||
from freqtrade.data.converter import trim_dataframe
|
||||
from freqtrade.data.history import get_timerange
|
||||
from freqtrade.exceptions import OperationalException
|
||||
from freqtrade.misc import plural, round_dict
|
||||
from freqtrade.misc import file_dump_json, plural, round_dict
|
||||
from freqtrade.optimize.backtesting import Backtesting
|
||||
# Import IHyperOpt and IHyperOptLoss to allow unpickling classes from these modules
|
||||
from freqtrade.optimize.hyperopt_interface import IHyperOpt # noqa: F401
|
||||
from freqtrade.optimize.hyperopt_loss_interface import \
|
||||
IHyperOptLoss # noqa: F401
|
||||
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F401
|
||||
from freqtrade.resolvers.hyperopt_resolver import (HyperOptLossResolver,
|
||||
HyperOptResolver)
|
||||
from freqtrade.strategy import IStrategy
|
||||
@ -77,19 +77,16 @@ class Hyperopt:
|
||||
|
||||
self.custom_hyperoptloss = HyperOptLossResolver.load_hyperoptloss(self.config)
|
||||
self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
|
||||
|
||||
time_now = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
|
||||
self.results_file = (self.config['user_data_dir'] /
|
||||
'hyperopt_results' / 'hyperopt_results.pickle')
|
||||
'hyperopt_results' / f'hyperopt_results_{time_now}.pickle')
|
||||
self.data_pickle_file = (self.config['user_data_dir'] /
|
||||
'hyperopt_results' / 'hyperopt_tickerdata.pkl')
|
||||
self.total_epochs = config.get('epochs', 0)
|
||||
|
||||
self.current_best_loss = 100
|
||||
|
||||
if not self.config.get('hyperopt_continue'):
|
||||
self.clean_hyperopt()
|
||||
else:
|
||||
logger.info("Continuing on previous hyperopt results.")
|
||||
self.clean_hyperopt()
|
||||
|
||||
self.num_epochs_saved = 0
|
||||
|
||||
@ -165,6 +162,9 @@ class Hyperopt:
|
||||
self.num_epochs_saved = num_epochs
|
||||
logger.debug(f"{self.num_epochs_saved} {plural(self.num_epochs_saved, 'epoch')} "
|
||||
f"saved to '{self.results_file}'.")
|
||||
# Store hyperopt filename
|
||||
latest_filename = Path.joinpath(self.results_file.parent, LAST_BT_RESULT_FN)
|
||||
file_dump_json(latest_filename, {'latest_hyperopt': str(self.results_file.name)})
|
||||
|
||||
@staticmethod
|
||||
def _read_results(results_file: Path) -> List:
|
||||
@ -657,8 +657,6 @@ class Hyperopt:
|
||||
self.backtesting.strategy.dp = None # type: ignore
|
||||
IStrategy.dp = None # type: ignore
|
||||
|
||||
self.epochs = self.load_previous_results(self.results_file)
|
||||
|
||||
cpus = cpu_count()
|
||||
logger.info(f"Found {cpus} CPU cores. Let's make them scream!")
|
||||
config_jobs = self.config.get('hyperopt_jobs', -1)
|
||||
|
||||
@ -15,6 +15,7 @@ from freqtrade.data.btanalysis import (BT_DATA_COLUMNS,
|
||||
create_cum_profit,
|
||||
extract_trades_of_period,
|
||||
get_latest_backtest_filename,
|
||||
get_latest_hyperopt_file,
|
||||
load_backtest_data, load_trades,
|
||||
load_trades_from_db)
|
||||
from freqtrade.data.history import load_data, load_pair_history
|
||||
@ -43,6 +44,17 @@ def test_get_latest_backtest_filename(testdatadir, mocker):
|
||||
get_latest_backtest_filename(testdatadir)
|
||||
|
||||
|
||||
def test_get_latest_hyperopt_file(testdatadir, mocker):
|
||||
res = get_latest_hyperopt_file(testdatadir / 'does_not_exist', 'testfile.pickle')
|
||||
assert res == testdatadir / 'does_not_exist/testfile.pickle'
|
||||
|
||||
res = get_latest_hyperopt_file(testdatadir.parent)
|
||||
assert res == testdatadir.parent / "hyperopt_results.pickle"
|
||||
|
||||
res = get_latest_hyperopt_file(str(testdatadir.parent))
|
||||
assert res == testdatadir.parent / "hyperopt_results.pickle"
|
||||
|
||||
|
||||
def test_load_backtest_data_old_format(testdatadir):
|
||||
|
||||
filename = testdatadir / "backtest-result_test.json"
|
||||
|
||||
@ -5,7 +5,7 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from copy import deepcopy
|
||||
from typing import Dict, List
|
||||
from unittest.mock import MagicMock, PropertyMock
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
@ -81,13 +81,14 @@ def create_results(mocker, hyperopt, testdatadir) -> List[Dict]:
|
||||
|
||||
mocker.patch.object(Path, "is_file", MagicMock(return_value=False))
|
||||
stat_mock = MagicMock()
|
||||
stat_mock.st_size = PropertyMock(return_value=1)
|
||||
mocker.patch.object(Path, "stat", MagicMock(return_value=False))
|
||||
stat_mock.st_size = 1
|
||||
mocker.patch.object(Path, "stat", MagicMock(return_value=stat_mock))
|
||||
|
||||
mocker.patch.object(Path, "unlink", MagicMock(return_value=True))
|
||||
mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None)
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
|
||||
return [{'loss': 1, 'result': 'foo', 'params': {}}]
|
||||
return [{'loss': 1, 'result': 'foo', 'params': {}, 'is_best': True}]
|
||||
|
||||
|
||||
def test_setup_hyperopt_configuration_without_arguments(mocker, default_conf, caplog) -> None:
|
||||
@ -497,6 +498,7 @@ def test_no_log_if_loss_does_not_improve(hyperopt, caplog) -> None:
|
||||
def test_save_results_saves_epochs(mocker, hyperopt, testdatadir, caplog) -> None:
|
||||
epochs = create_results(mocker, hyperopt, testdatadir)
|
||||
mock_dump = mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None)
|
||||
mock_dump_json = mocker.patch('freqtrade.optimize.hyperopt.file_dump_json', return_value=None)
|
||||
results_file = testdatadir / 'optimize' / 'ut_results.pickle'
|
||||
|
||||
caplog.set_level(logging.DEBUG)
|
||||
@ -505,6 +507,7 @@ def test_save_results_saves_epochs(mocker, hyperopt, testdatadir, caplog) -> Non
|
||||
hyperopt._save_results()
|
||||
assert log_has(f"1 epoch saved to '{results_file}'.", caplog)
|
||||
mock_dump.assert_called_once()
|
||||
mock_dump_json.assert_called_once()
|
||||
|
||||
hyperopt.epochs = epochs + epochs
|
||||
hyperopt._save_results()
|
||||
@ -521,6 +524,28 @@ def test_read_results_returns_epochs(mocker, hyperopt, testdatadir, caplog) -> N
|
||||
mock_load.assert_called_once()
|
||||
|
||||
|
||||
def test_load_previous_results(mocker, hyperopt, testdatadir, caplog) -> None:
|
||||
epochs = create_results(mocker, hyperopt, testdatadir)
|
||||
mock_load = mocker.patch('freqtrade.optimize.hyperopt.load', return_value=epochs)
|
||||
mocker.patch.object(Path, 'is_file', MagicMock(return_value=True))
|
||||
statmock = MagicMock()
|
||||
statmock.st_size = 5
|
||||
# mocker.patch.object(Path, 'stat', MagicMock(return_value=statmock))
|
||||
|
||||
results_file = testdatadir / 'optimize' / 'ut_results.pickle'
|
||||
|
||||
hyperopt_epochs = hyperopt.load_previous_results(results_file)
|
||||
|
||||
assert hyperopt_epochs == epochs
|
||||
mock_load.assert_called_once()
|
||||
|
||||
del epochs[0]['is_best']
|
||||
mock_load = mocker.patch('freqtrade.optimize.hyperopt.load', return_value=epochs)
|
||||
|
||||
with pytest.raises(OperationalException):
|
||||
hyperopt.load_previous_results(results_file)
|
||||
|
||||
|
||||
def test_roi_table_generation(hyperopt) -> None:
|
||||
params = {
|
||||
'roi_t1': 5,
|
||||
@ -536,6 +561,8 @@ def test_roi_table_generation(hyperopt) -> None:
|
||||
|
||||
def test_start_calls_optimizer(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -839,19 +866,10 @@ def test_clean_hyperopt(mocker, hyperopt_conf, caplog):
|
||||
assert log_has(f"Removing `{h.data_pickle_file}`.", caplog)
|
||||
|
||||
|
||||
def test_continue_hyperopt(mocker, hyperopt_conf, caplog):
|
||||
patch_exchange(mocker)
|
||||
hyperopt_conf.update({'hyperopt_continue': True})
|
||||
mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
|
||||
unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
|
||||
Hyperopt(hyperopt_conf)
|
||||
|
||||
assert unlinkmock.call_count == 0
|
||||
assert log_has("Continuing on previous hyperopt results.", caplog)
|
||||
|
||||
|
||||
def test_print_json_spaces_all(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -907,6 +925,7 @@ def test_print_json_spaces_all(mocker, hyperopt_conf, capsys) -> None:
|
||||
|
||||
def test_print_json_spaces_default(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -954,6 +973,7 @@ def test_print_json_spaces_default(mocker, hyperopt_conf, capsys) -> None:
|
||||
|
||||
def test_print_json_spaces_roi_stoploss(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -1000,6 +1020,7 @@ def test_print_json_spaces_roi_stoploss(mocker, hyperopt_conf, capsys) -> None:
|
||||
|
||||
def test_simplified_interface_roi_stoploss(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -1052,6 +1073,7 @@ def test_simplified_interface_roi_stoploss(mocker, hyperopt_conf, capsys) -> Non
|
||||
|
||||
def test_simplified_interface_all_failed(mocker, hyperopt_conf) -> None:
|
||||
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -1078,6 +1100,7 @@ def test_simplified_interface_all_failed(mocker, hyperopt_conf) -> None:
|
||||
|
||||
def test_simplified_interface_buy(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -1130,6 +1153,7 @@ def test_simplified_interface_buy(mocker, hyperopt_conf, capsys) -> None:
|
||||
|
||||
def test_simplified_interface_sell(mocker, hyperopt_conf, capsys) -> None:
|
||||
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
@ -1188,6 +1212,7 @@ def test_simplified_interface_sell(mocker, hyperopt_conf, capsys) -> None:
|
||||
])
|
||||
def test_simplified_interface_failed(mocker, hyperopt_conf, method, space) -> None:
|
||||
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
|
||||
mocker.patch('freqtrade.optimize.hyperopt.file_dump_json')
|
||||
mocker.patch('freqtrade.optimize.backtesting.Backtesting.load_bt_data',
|
||||
MagicMock(return_value=(MagicMock(), None)))
|
||||
mocker.patch(
|
||||
|
||||
@ -1139,9 +1139,9 @@ def test_telegram_logs(default_conf, update, mocker) -> None:
|
||||
context = MagicMock()
|
||||
context.args = []
|
||||
telegram._logs(update=update, context=context)
|
||||
# Called at least 3 times. Exact times will change with unrelated changes to setup messages
|
||||
# Called at least 2 times. Exact times will change with unrelated changes to setup messages
|
||||
# Therefore we don't test for this explicitly.
|
||||
assert msg_mock.call_count > 3
|
||||
assert msg_mock.call_count >= 2
|
||||
|
||||
|
||||
def test_edge_disabled(default_conf, update, mocker) -> None:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user