Merge branch 'develop' of https://github.com/freqtrade/freqtrade into docker-compose

docs/backtesting.md

@@ -7,7 +7,7 @@ Backtesting.
 
 To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the `freqtrade download-data` command.
 
-If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes.
+If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes for 30 days.
 Exchange and pairs will come from `config.json` (if specified using `-c/--config`). Otherwise `--exchange` becomes mandatory.
 
 Alternatively, a `pairs.json` file can be used.
@@ -37,6 +37,10 @@ This will download ticker data for all the currency pairs you defined in `pairs.
 - Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
 - To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
 
+!!! Tip Updating existing data
+    If you already have backtesting data available in your data-directory and would like to refresh this data up to today, use `--days xx` with a number slightly higher than the missing number of days. Freqtrade will keep the available data and only download the missing data.
+    Be carefull though: If the number is too small (which would result in a few missing days), the whole dataset will be removed and only xx days will be downloaded.
+
 ## Test your strategy with Backtesting
 
 Now you have good Buy and Sell strategies and some historic data, you want to test it against

docs/bot-usage.md

@@ -184,10 +184,6 @@ optional arguments:
                         Specify max_open_trades to use.
   --stake_amount STAKE_AMOUNT
                         Specify stake_amount.
-  -r, --refresh-pairs-cached
-                        Refresh the pairs files in tests/testdata with the
-                        latest data from the exchange. Use it if you want to
-                        run your optimization commands with up-to-date data.
   --eps, --enable-position-stacking
                         Allow buying the same pair multiple times (position
                         stacking).
@@ -245,10 +241,6 @@ optional arguments:
                         Specify max_open_trades to use.
   --stake_amount STAKE_AMOUNT
                         Specify stake_amount.
-  -r, --refresh-pairs-cached
-                        Refresh the pairs files in tests/testdata with the
-                        latest data from the exchange. Use it if you want to
-                        run your optimization commands with up-to-date data.
   --customhyperopt NAME
                         Specify hyperopt class name (default:
                         `DefaultHyperOpts`).
@@ -310,10 +302,6 @@ optional arguments:
                         Specify max_open_trades to use.
   --stake_amount STAKE_AMOUNT
                         Specify stake_amount.
-  -r, --refresh-pairs-cached
-                        Refresh the pairs files in tests/testdata with the
-                        latest data from the exchange. Use it if you want to
-                        run your optimization commands with up-to-date data.
   --stoplosses STOPLOSS_RANGE
                         Defines a range of stoploss against which edge will
                         assess the strategy the format is "min,max,step"

docs/data-analysis.md

@@ -91,7 +91,8 @@ df.groupby("pair")["sell_reason"].value_counts()
 
 ### Load multiple configuration files
 
-This option can be useful to inspect the results of passing in multiple configs
+This option can be useful to inspect the results of passing in multiple configs.
+This will also run through the whole Configuration initialization, so the configuration is completely initialized to be passed to other methods.
 
 ``` python
 import json
@@ -101,7 +102,16 @@ from freqtrade.configuration import Configuration
 config = Configuration.from_files(["config1.json", "config2.json"])
 
 # Show the config in memory
-print(json.dumps(config, indent=1))
+print(json.dumps(config['original_config'], indent=2))
+```
+
+For Interactive environments, have an additional configuration specifying `user_data_dir` and pass this in last, so you don't have to change directories while running the bot.
+Best avoid relative paths, since this starts at the storage location of the jupyter notebook, unless the directory is changed.
+
+``` json
+{
+    "user_data_dir": "~/.freqtrade/"
+}
 ```
 
 ### Load exchange data to a pandas dataframe

docs/deprecated.md

@@ -4,7 +4,7 @@ This page contains description of the command line arguments, configuration para
 and the bot features that were declared as DEPRECATED by the bot development team
 and are no longer supported. Please avoid their usage in your configuration.
 
-## Deprecated
+## Removed features
 
 ### the `--refresh-pairs-cached` command line option
 
@@ -12,9 +12,7 @@ and are no longer supported. Please avoid their usage in your configuration.
 Since this leads to much confusion, and slows down backtesting (while not being part of backtesting) this has been singled out 
 as a seperate freqtrade subcommand `freqtrade download-data`.
 
-This command line option was deprecated in `2019.7-dev` and will be removed after the next release.
+This command line option was deprecated in 2019.7-dev (develop branch) and removed in 2019.9 (master branch).
-
-## Removed features
 
 ### The **--dynamic-whitelist** command line option
 

freqtrade/configuration/arguments.py

@@ -2,11 +2,11 @@
 This module contains the argument manager class
 """
 import argparse
-from typing import List, Optional
 from pathlib import Path
+from typing import Any, Dict, List, Optional
 
-from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
 from freqtrade import constants
+from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
 
 ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir", "user_data_dir"]
 
@@ -15,7 +15,7 @@ ARGS_STRATEGY = ["strategy", "strategy_path"]
 ARGS_MAIN = ARGS_COMMON + ARGS_STRATEGY + ["db_url", "sd_notify"]
 
 ARGS_COMMON_OPTIMIZE = ["ticker_interval", "timerange",
-                        "max_open_trades", "stake_amount", "refresh_pairs"]
+                        "max_open_trades", "stake_amount"]
 
 ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_positions",
                                         "strategy_list", "export", "exportfilename"]
@@ -41,10 +41,10 @@ ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit", "db_
 ARGS_PLOT_PROFIT = ["pairs", "timerange", "export", "exportfilename", "db_url",
                     "trade_source", "ticker_interval"]
 
-NO_CONF_REQURIED = ["download-data", "plot-dataframe", "plot-profit"]
+NO_CONF_REQURIED = ["create-userdir", "download-data", "plot-dataframe", "plot-profit"]
 
 
-class Arguments(object):
+class Arguments:
     """
     Arguments Class. Manage the arguments received by the cli
     """
@@ -57,7 +57,7 @@ class Arguments(object):
         self._build_args(optionlist=ARGS_MAIN)
         self._build_subcommands()
 
-    def get_parsed_arg(self) -> argparse.Namespace:
+    def get_parsed_arg(self) -> Dict[str, Any]:
         """
         Return the list of arguments
         :return: List[str] List of arguments
@@ -66,7 +66,7 @@ class Arguments(object):
             self._load_args()
             self._parsed_arg = self._parse_args()
 
-        return self._parsed_arg
+        return vars(self._parsed_arg)
 
     def _parse_args(self) -> argparse.Namespace:
         """

freqtrade/configuration/check_exchange.py

@@ -27,6 +27,14 @@ def check_exchange(config: Dict[str, Any], check_for_bad: bool = True) -> bool:
     logger.info("Checking exchange...")
 
     exchange = config.get('exchange', {}).get('name').lower()
+    if not exchange:
+        raise OperationalException(
+            f'This command requires a configured exchange. You should either use '
+            f'`--exchange <exchange_name>` or specify a configuration file via `--config`.\n'
+            f'The following exchanges are supported by ccxt: '
+            f'{", ".join(available_exchanges())}'
+        )
+
     if not is_exchange_available(exchange):
         raise OperationalException(
                 f'Exchange "{exchange}" is not supported by ccxt '

freqtrade/configuration/cli_options.py

@@ -107,13 +107,6 @@ AVAILABLE_CLI_OPTIONS = {
         help='Specify stake_amount.',
         type=float,
     ),
-    "refresh_pairs": Arg(
-        '-r', '--refresh-pairs-cached',
-        help='Refresh the pairs files in tests/testdata with the latest data from the '
-        'exchange. Use it if you want to run your optimization commands with '
-        'up-to-date data.',
-        action='store_true',
-    ),
     # Backtesting
     "position_stacking": Arg(
         '--eps', '--enable-position-stacking',

freqtrade/configuration/configuration.py

@@ -3,7 +3,6 @@ This module contains the configuration class
 """
 import logging
 import warnings
-from argparse import Namespace
 from copy import deepcopy
 from pathlib import Path
 from typing import Any, Callable, Dict, List, Optional
@@ -22,13 +21,13 @@ from freqtrade.state import RunMode
 logger = logging.getLogger(__name__)
 
 
-class Configuration(object):
+class Configuration:
     """
     Class to read and init the bot configuration
     Reuse this class for the bot, backtesting, hyperopt and every script that required configuration
     """
 
-    def __init__(self, args: Namespace, runmode: RunMode = None) -> None:
+    def __init__(self, args: Dict[str, Any], runmode: RunMode = None) -> None:
         self.args = args
         self.config: Optional[Dict[str, Any]] = None
         self.runmode = runmode
@@ -50,9 +49,16 @@ class Configuration(object):
         and merging their contents.
         Files are loaded in sequence, parameters in later configuration files
         override the same parameter from an earlier file (last definition wins).
+        Runs through the whole Configuration initialization, so all expected config entries
+        are available to interactive environments.
         :param files: List of file paths
         :return: configuration dictionary
         """
+        c = Configuration({"config": files}, RunMode.OTHER)
+        return c.get_config()
+
+    def load_from_files(self, files: List[str]) -> Dict[str, Any]:
+
         # Keep this method as staticmethod, so it can be used from interactive environments
         config: Dict[str, Any] = {}
 
@@ -82,7 +88,10 @@ class Configuration(object):
         :return: Configuration dictionary
         """
         # Load all configs
-        config: Dict[str, Any] = Configuration.from_files(self.args.config)
+        config: Dict[str, Any] = self.load_from_files(self.args["config"])
+
+        # Keep a copy of the original configuration file
+        config['original_config'] = deepcopy(config)
 
         self._process_common_options(config)
 
@@ -107,13 +116,10 @@ class Configuration(object):
         the -v/--verbose, --logfile options
         """
         # Log level
-        if 'verbosity' in self.args and self.args.verbosity:
+        config.update({'verbosity': self.args.get("verbosity", 0)})
-            config.update({'verbosity': self.args.verbosity})
-        else:
-            config.update({'verbosity': 0})
 
-        if 'logfile' in self.args and self.args.logfile:
+        if 'logfile' in self.args and self.args["logfile"]:
-            config.update({'logfile': self.args.logfile})
+            config.update({'logfile': self.args["logfile"]})
 
         setup_logging(config)
 
@@ -122,15 +128,15 @@ class Configuration(object):
         self._process_logging_options(config)
 
         # Set strategy if not specified in config and or if it's non default
-        if self.args.strategy != constants.DEFAULT_STRATEGY or not config.get('strategy'):
+        if self.args.get("strategy") != constants.DEFAULT_STRATEGY or not config.get('strategy'):
-            config.update({'strategy': self.args.strategy})
+            config.update({'strategy': self.args.get("strategy")})
 
         self._args_to_config(config, argname='strategy_path',
                              logstring='Using additional Strategy lookup path: {}')
 
-        if ('db_url' in self.args and self.args.db_url and
+        if ('db_url' in self.args and self.args["db_url"] and
-                self.args.db_url != constants.DEFAULT_DB_PROD_URL):
+                self.args["db_url"] != constants.DEFAULT_DB_PROD_URL):
-            config.update({'db_url': self.args.db_url})
+            config.update({'db_url': self.args["db_url"]})
             logger.info('Parameter --db-url detected ...')
 
         if config.get('dry_run', False):
@@ -153,7 +159,7 @@ class Configuration(object):
             config['max_open_trades'] = float('inf')
 
         # Support for sd_notify
-        if 'sd_notify' in self.args and self.args.sd_notify:
+        if 'sd_notify' in self.args and self.args["sd_notify"]:
             config['internals'].update({'sd_notify': True})
 
     def _process_datadir_options(self, config: Dict[str, Any]) -> None:
@@ -162,12 +168,12 @@ class Configuration(object):
         --user-data, --datadir
         """
         # Check exchange parameter here - otherwise `datadir` might be wrong.
-        if "exchange" in self.args and self.args.exchange:
+        if "exchange" in self.args and self.args["exchange"]:
-            config['exchange']['name'] = self.args.exchange
+            config['exchange']['name'] = self.args["exchange"]
             logger.info(f"Using exchange {config['exchange']['name']}")
 
-        if 'user_data_dir' in self.args and self.args.user_data_dir:
+        if 'user_data_dir' in self.args and self.args["user_data_dir"]:
-            config.update({'user_data_dir': self.args.user_data_dir})
+            config.update({'user_data_dir': self.args["user_data_dir"]})
         elif 'user_data_dir' not in config:
             # Default to cwd/user_data (legacy option ...)
             config.update({'user_data_dir': str(Path.cwd() / "user_data")})
@@ -176,10 +182,7 @@ class Configuration(object):
         config['user_data_dir'] = create_userdata_dir(config['user_data_dir'], create_dir=False)
         logger.info('Using user-data directory: %s ...', config['user_data_dir'])
 
-        if 'datadir' in self.args and self.args.datadir:
+        config.update({'datadir': create_datadir(config, self.args.get("datadir", None))})
-            config.update({'datadir': create_datadir(config, self.args.datadir)})
-        else:
-            config.update({'datadir': create_datadir(config, None)})
         logger.info('Using data directory: %s ...', config.get('datadir'))
 
     def _process_optimize_options(self, config: Dict[str, Any]) -> None:
@@ -192,12 +195,12 @@ class Configuration(object):
         self._args_to_config(config, argname='position_stacking',
                              logstring='Parameter --enable-position-stacking detected ...')
 
-        if 'use_max_market_positions' in self.args and not self.args.use_max_market_positions:
+        if 'use_max_market_positions' in self.args and not self.args["use_max_market_positions"]:
             config.update({'use_max_market_positions': False})
             logger.info('Parameter --disable-max-market-positions detected ...')
             logger.info('max_open_trades set to unlimited ...')
-        elif 'max_open_trades' in self.args and self.args.max_open_trades:
+        elif 'max_open_trades' in self.args and self.args["max_open_trades"]:
-            config.update({'max_open_trades': self.args.max_open_trades})
+            config.update({'max_open_trades': self.args["max_open_trades"]})
             logger.info('Parameter --max_open_trades detected, '
                         'overriding max_open_trades to: %s ...', config.get('max_open_trades'))
         else:
@@ -212,12 +215,8 @@ class Configuration(object):
 
         self._process_datadir_options(config)
 
-        self._args_to_config(config, argname='refresh_pairs',
-                             logstring='Parameter -r/--refresh-pairs-cached detected ...',
-                             deprecated_msg='-r/--refresh-pairs-cached will be removed soon.')
-
         self._args_to_config(config, argname='strategy_list',
-                             logstring='Using strategy list of {} Strategies', logfun=len)
+                             logstring='Using strategy list of {} strategies', logfun=len)
 
         self._args_to_config(config, argname='ticker_interval',
                              logstring='Overriding ticker interval with Command line argument')
@@ -229,16 +228,16 @@ class Configuration(object):
                              logstring='Storing backtest results to {} ...')
 
         # Edge section:
-        if 'stoploss_range' in self.args and self.args.stoploss_range:
+        if 'stoploss_range' in self.args and self.args["stoploss_range"]:
-            txt_range = eval(self.args.stoploss_range)
+            txt_range = eval(self.args["stoploss_range"])
             config['edge'].update({'stoploss_range_min': txt_range[0]})
             config['edge'].update({'stoploss_range_max': txt_range[1]})
             config['edge'].update({'stoploss_range_step': txt_range[2]})
-            logger.info('Parameter --stoplosses detected: %s ...', self.args.stoploss_range)
+            logger.info('Parameter --stoplosses detected: %s ...', self.args["stoploss_range"])
 
         # Hyperopt section
         self._args_to_config(config, argname='hyperopt',
-                             logstring='Using Hyperopt file {}')
+                             logstring='Using Hyperopt class name: {}')
 
         self._args_to_config(config, argname='hyperopt_path',
                              logstring='Using additional Hyperopt lookup path: {}')
@@ -254,7 +253,7 @@ class Configuration(object):
         self._args_to_config(config, argname='print_all',
                              logstring='Parameter --print-all detected ...')
 
-        if 'print_colorized' in self.args and not self.args.print_colorized:
+        if 'print_colorized' in self.args and not self.args["print_colorized"]:
             logger.info('Parameter --no-color detected ...')
             config.update({'print_colorized': False})
         else:
@@ -276,7 +275,7 @@ class Configuration(object):
                              logstring='Hyperopt continue: {}')
 
         self._args_to_config(config, argname='hyperopt_loss',
-                             logstring='Using loss function: {}')
+                             logstring='Using Hyperopt loss class name: {}')
 
     def _process_plot_options(self, config: Dict[str, Any]) -> None:
 
@@ -324,9 +323,9 @@ class Configuration(object):
                         sample: logfun=len (prints the length of the found
                         configuration instead of the content)
         """
-        if argname in self.args and getattr(self.args, argname):
+        if argname in self.args and self.args[argname]:
 
-            config.update({argname: getattr(self.args, argname)})
+            config.update({argname: self.args[argname]})
             if logfun:
                 logger.info(logstring.format(logfun(config[argname])))
             else:
@@ -346,8 +345,8 @@ class Configuration(object):
         if "pairs" in config:
             return
 
-        if "pairs_file" in self.args and self.args.pairs_file:
+        if "pairs_file" in self.args and self.args["pairs_file"]:
-            pairs_file = Path(self.args.pairs_file)
+            pairs_file = Path(self.args["pairs_file"])
             logger.info(f'Reading pairs file "{pairs_file}".')
             # Download pairs from the pairs file if no config is specified
             # or if pairs file is specified explicitely
@@ -358,7 +357,7 @@ class Configuration(object):
                 config['pairs'].sort()
             return
 
-        if "config" in self.args and self.args.config:
+        if "config" in self.args and self.args["config"]:
             logger.info("Using pairlist from configuration.")
             config['pairs'] = config.get('exchange', {}).get('pair_whitelist')
         else:

freqtrade/configuration/timerange.py

@@ -7,7 +7,7 @@ from typing import Optional
 import arrow
 
 
-class TimeRange():
+class TimeRange:
     """
     object defining timerange inputs.
     [start/stop]type defines if [start/stop]ts shall be used.

freqtrade/data/dataprovider.py

@@ -17,7 +17,7 @@ from freqtrade.state import RunMode
 logger = logging.getLogger(__name__)
 
 
-class DataProvider():
+class DataProvider:
 
     def __init__(self, config: dict, exchange: Exchange) -> None:
         self._config = config
@@ -65,7 +65,6 @@ class DataProvider():
         """
         return load_pair_history(pair=pair,
                                  ticker_interval=ticker_interval or self._config['ticker_interval'],
-                                 refresh_pairs=False,
                                  datadir=Path(self._config['datadir'])
                                  )
 

freqtrade/data/history.py

@@ -129,8 +129,7 @@ def load_pair_history(pair: str,
     else:
         logger.warning(
             f'No history data for pair: "{pair}", interval: {ticker_interval}. '
-            'Use --refresh-pairs-cached option or `freqtrade download-data` '
+            'Use `freqtrade download-data` to download the data'
-            'script to download the data'
         )
         return None
 
@@ -142,24 +141,16 @@ def load_data(datadir: Path,
               exchange: Optional[Exchange] = None,
               timerange: TimeRange = TimeRange(None, None, 0, 0),
               fill_up_missing: bool = True,
-              live: bool = False
               ) -> Dict[str, DataFrame]:
     """
-    Loads ticker history data for a list of pairs the given parameters
+    Loads ticker history data for a list of pairs
     :return: dict(<pair>:<tickerlist>)
+    TODO: refresh_pairs is still used by edge to keep the data uptodate.
+        This should be replaced in the future. Instead, writing the current candles to disk
+        from dataprovider should be implemented, as this would avoid loading ohlcv data twice.
+        exchange and refresh_pairs are then not needed here nor in load_pair_history.
     """
     result: Dict[str, DataFrame] = {}
-    if live:
-        if exchange:
-            logger.info('Live: Downloading data for all defined pairs ...')
-            exchange.refresh_latest_ohlcv([(pair, ticker_interval) for pair in pairs])
-            result = {key[0]: value for key, value in exchange._klines.items() if value is not None}
-        else:
-            raise OperationalException(
-                "Exchange needs to be initialized when using live data."
-            )
-    else:
-        logger.info('Using local backtesting data ...')
 
     for pair in pairs:
         hist = load_pair_history(pair=pair, ticker_interval=ticker_interval,

freqtrade/edge/__init__.py

@@ -28,7 +28,7 @@ class PairInfo(NamedTuple):
     avg_trade_duration: float
 
 
-class Edge():
+class Edge:
     """
     Calculates Win Rate, Risk Reward Ratio, Expectancy
     against historical data for a give set of markets and a strategy

freqtrade/exchange/exchange.py

@@ -30,9 +30,6 @@ BAD_EXCHANGES = {
     "bitmex": "Various reasons",
     "bitstamp": "Does not provide history. "
                 "Details in https://github.com/freqtrade/freqtrade/issues/1983",
-    "kraken": "TEMPORARY: Balance does not report free balance, so freqtrade will not know "
-              "if enough balance is available."
-              "Details in https://github.com/freqtrade/freqtrade/issues/1687#issuecomment-528509266"
     }
 
 
@@ -72,7 +69,7 @@ def retrier(f):
     return wrapper
 
 
-class Exchange(object):
+class Exchange:
 
     _config: Dict = {}
     _params: Dict = {}

freqtrade/exchange/kraken.py

@@ -2,7 +2,11 @@
 import logging
 from typing import Dict
 
+import ccxt
+
+from freqtrade import OperationalException, TemporaryError
 from freqtrade.exchange import Exchange
+from freqtrade.exchange.exchange import retrier
 
 logger = logging.getLogger(__name__)
 
@@ -10,3 +14,33 @@ logger = logging.getLogger(__name__)
 class Kraken(Exchange):
 
     _params: Dict = {"trading_agreement": "agree"}
+
+    @retrier
+    def get_balances(self) -> dict:
+        if self._config['dry_run']:
+            return {}
+
+        try:
+            balances = self._api.fetch_balance()
+            # Remove additional info from ccxt results
+            balances.pop("info", None)
+            balances.pop("free", None)
+            balances.pop("total", None)
+            balances.pop("used", None)
+
+            orders = self._api.fetch_open_orders()
+            order_list = [(x["symbol"].split("/")[0 if x["side"] == "sell" else 1],
+                           x["remaining"],
+                           # Don't remove the below comment, this can be important for debuggung
+                           # x["side"], x["amount"],
+                           ) for x in orders]
+            for bal in balances:
+                balances[bal]['used'] = sum(order[1] for order in order_list if order[0] == bal)
+                balances[bal]['free'] = balances[bal]['total'] - balances[bal]['used']
+
+            return balances
+        except (ccxt.NetworkError, ccxt.ExchangeError) as e:
+            raise TemporaryError(
+                f'Could not get balance due to {e.__class__.__name__}. Message: {e}') from e
+        except ccxt.BaseError as e:
+            raise OperationalException(e) from e

freqtrade/freqtradebot.py

@@ -29,7 +29,7 @@ from freqtrade.wallets import Wallets
 logger = logging.getLogger(__name__)
 
 
-class FreqtradeBot(object):
+class FreqtradeBot:
     """
     Freqtrade is the main class of the bot.
     This is from here the bot start its logic.

freqtrade/main.py

@@ -11,7 +11,6 @@ if sys.version_info < (3, 6):
 
 # flake8: noqa E402
 import logging
-from argparse import Namespace
 from typing import Any, List
 
 from freqtrade import OperationalException
@@ -32,12 +31,12 @@ def main(sysargv: List[str] = None) -> None:
     worker = None
     try:
         arguments = Arguments(sysargv)
-        args: Namespace = arguments.get_parsed_arg()
+        args = arguments.get_parsed_arg()
 
         # A subcommand has been issued.
         # Means if Backtesting or Hyperopt have been called we exit the bot
-        if hasattr(args, 'func'):
+        if 'func' in args:
-            args.func(args)
+            args['func'](args)
             # TODO: fetch return_code as returned by the command function here
             return_code = 0
         else:

freqtrade/optimize/__init__.py

@@ -1,5 +1,4 @@
 import logging
-from argparse import Namespace
 from typing import Any, Dict
 
 from filelock import FileLock, Timeout
@@ -12,7 +11,7 @@ from freqtrade.utils import setup_utils_configuration
 logger = logging.getLogger(__name__)
 
 
-def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
+def setup_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
     """
     Prepare the configuration for the Hyperopt module
     :param args: Cli args from Arguments()
@@ -28,7 +27,7 @@ def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
     return config
 
 
-def start_backtesting(args: Namespace) -> None:
+def start_backtesting(args: Dict[str, Any]) -> None:
     """
     Start Backtesting script
     :param args: Cli args from Arguments()
@@ -47,7 +46,7 @@ def start_backtesting(args: Namespace) -> None:
     backtesting.start()
 
 
-def start_hyperopt(args: Namespace) -> None:
+def start_hyperopt(args: Dict[str, Any]) -> None:
     """
     Start hyperopt script
     :param args: Cli args from Arguments()
@@ -85,7 +84,7 @@ def start_hyperopt(args: Namespace) -> None:
         # Same in Edge and Backtesting start() functions.
 
 
-def start_edge(args: Namespace) -> None:
+def start_edge(args: Dict[str, Any]) -> None:
     """
     Start Edge script
     :param args: Cli args from Arguments()

freqtrade/optimize/backtesting.py

@@ -44,7 +44,7 @@ class BacktestResult(NamedTuple):
     sell_reason: SellType
 
 
-class Backtesting(object):
+class Backtesting:
     """
     Backtesting class, this class contains all the logic to run a backtest
 
@@ -95,8 +95,6 @@ class Backtesting(object):
         Load strategy into backtesting
         """
         self.strategy = strategy
-        self.advise_buy = strategy.advise_buy
-        self.advise_sell = strategy.advise_sell
         # Set stoploss_on_exchange to false for backtesting,
         # since a "perfect" stoploss-sell is assumed anyway
         # And the regular "stoploss" function would not apply to that case
@@ -219,8 +217,8 @@ class Backtesting(object):
         for pair, pair_data in processed.items():
             pair_data['buy'], pair_data['sell'] = 0, 0  # cleanup from previous run
 
-            ticker_data = self.advise_sell(
+            ticker_data = self.strategy.advise_sell(
-                self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
+                self.strategy.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
 
             # to avoid using data from future, we buy/sell with signal from previous candle
             ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
@@ -239,14 +237,16 @@ class Backtesting(object):
             stake_amount: float, max_open_trades: int) -> Optional[BacktestResult]:
 
         trade = Trade(
+            pair=pair,
             open_rate=buy_row.open,
             open_date=buy_row.date,
             stake_amount=stake_amount,
             amount=stake_amount / buy_row.open,
             fee_open=self.fee,
-            fee_close=self.fee
+            fee_close=self.fee,
+            is_open=True,
         )
-
+        logger.debug(f"{pair} - Backtesting emulates creation of new trade: {trade}.")
         # calculate win/lose forwards from buy point
         for sell_row in partial_ticker:
             if max_open_trades > 0:
@@ -289,7 +289,7 @@ class Backtesting(object):
         if partial_ticker:
             # no sell condition found - trade stil open at end of backtest period
             sell_row = partial_ticker[-1]
-            btr = BacktestResult(pair=pair,
+            bt_res = BacktestResult(pair=pair,
                                     profit_percent=trade.calc_profit_percent(rate=sell_row.open),
                                     profit_abs=trade.calc_profit(rate=sell_row.open),
                                     open_time=buy_row.date,
@@ -303,9 +303,11 @@ class Backtesting(object):
                                     close_rate=sell_row.open,
                                     sell_reason=SellType.FORCE_SELL
                                     )
-            logger.debug('Force_selling still open trade %s with %s perc - %s', btr.pair,
+            logger.debug(f"{pair} - Force selling still open trade, "
-                         btr.profit_percent, btr.profit_abs)
+                         f"profit percent: {bt_res.profit_percent}, "
-            return btr
+                         f"profit abs: {bt_res.profit_abs}")
+
+            return bt_res
         return None
 
     def backtest(self, args: Dict) -> DataFrame:
@@ -384,6 +386,8 @@ class Backtesting(object):
                                                          max_open_trades)
 
                 if trade_entry:
+                    logger.debug(f"{pair} - Locking pair till "
+                                 f"close_time={trade_entry.close_time}")
                     lock_pair_until[pair] = trade_entry.close_time
                     trades.append(trade_entry)
                 else:
@@ -410,8 +414,6 @@ class Backtesting(object):
             datadir=Path(self.config['datadir']),
             pairs=pairs,
             ticker_interval=self.ticker_interval,
-            refresh_pairs=self.config.get('refresh_pairs', False),
-            exchange=self.exchange,
             timerange=timerange,
         )
 

freqtrade/optimize/edge_cli.py

@@ -16,7 +16,7 @@ from freqtrade.resolvers import StrategyResolver
 logger = logging.getLogger(__name__)
 
 
-class EdgeCli(object):
+class EdgeCli:
     """
     EdgeCli class, this class contains all the logic to run edge backtesting
 
@@ -39,7 +39,8 @@ class EdgeCli(object):
         self.strategy = StrategyResolver(self.config).strategy
 
         self.edge = Edge(config, self.exchange, self.strategy)
-        self.edge._refresh_pairs = self.config.get('refresh_pairs', False)
+        # Set refresh_pairs to false for edge-cli (it must be true for edge)
+        self.edge._refresh_pairs = False
 
         self.timerange = TimeRange.parse_timerange(None if self.config.get(
             'timerange') is None else str(self.config.get('timerange')))

freqtrade/optimize/hyperopt.py

@@ -49,10 +49,11 @@ class Hyperopt:
     """
     def __init__(self, config: Dict[str, Any]) -> None:
         self.config = config
-        self.backtesting = Backtesting(self.config)
 
         self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
 
+        self.backtesting = Backtesting(self.config)
+
         self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
         self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
 
@@ -73,11 +74,15 @@ class Hyperopt:
         self.trials: List = []
 
         # Populate functions here (hasattr is slow so should not be run during "regular" operations)
+        if hasattr(self.custom_hyperopt, 'populate_indicators'):
+            self.backtesting.strategy.advise_indicators = \
+                    self.custom_hyperopt.populate_indicators  # type: ignore
         if hasattr(self.custom_hyperopt, 'populate_buy_trend'):
-            self.backtesting.advise_buy = self.custom_hyperopt.populate_buy_trend  # type: ignore
+            self.backtesting.strategy.advise_buy = \
-
+                    self.custom_hyperopt.populate_buy_trend  # type: ignore
         if hasattr(self.custom_hyperopt, 'populate_sell_trend'):
-            self.backtesting.advise_sell = self.custom_hyperopt.populate_sell_trend  # type: ignore
+            self.backtesting.strategy.advise_sell = \
+                    self.custom_hyperopt.populate_sell_trend  # type: ignore
 
         # Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
         if self.config.get('use_max_market_positions', True):
@@ -109,7 +114,9 @@ class Hyperopt:
                 p.unlink()
 
     def get_args(self, params):
-        dimensions = self.hyperopt_space()
+
+        dimensions = self.dimensions
+
         # Ensure the number of dimensions match
         # the number of parameters in the list x.
         if len(params) != len(dimensions):
@@ -255,13 +262,16 @@ class Hyperopt:
         """
         params = self.get_args(_params)
         if self.has_space('roi'):
-            self.backtesting.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params)
+            self.backtesting.strategy.minimal_roi = \
+                    self.custom_hyperopt.generate_roi_table(params)
 
         if self.has_space('buy'):
-            self.backtesting.advise_buy = self.custom_hyperopt.buy_strategy_generator(params)
+            self.backtesting.strategy.advise_buy = \
+                    self.custom_hyperopt.buy_strategy_generator(params)
 
         if self.has_space('sell'):
-            self.backtesting.advise_sell = self.custom_hyperopt.sell_strategy_generator(params)
+            self.backtesting.strategy.advise_sell = \
+                    self.custom_hyperopt.sell_strategy_generator(params)
 
         if self.has_space('stoploss'):
             self.backtesting.strategy.stoploss = params['stoploss']
@@ -322,9 +332,9 @@ class Hyperopt:
                 f'Total profit {total_profit: 11.8f} {stake_cur} '
                 f'({profit: 7.2f}Σ%). Avg duration {duration:5.1f} mins.')
 
-    def get_optimizer(self, cpu_count) -> Optimizer:
+    def get_optimizer(self, dimensions, cpu_count) -> Optimizer:
         return Optimizer(
-            self.hyperopt_space(),
+            dimensions,
             base_estimator="ET",
             acq_optimizer="auto",
             n_initial_points=INITIAL_POINTS,
@@ -352,8 +362,6 @@ class Hyperopt:
             datadir=Path(self.config['datadir']),
             pairs=self.config['exchange']['pair_whitelist'],
             ticker_interval=self.backtesting.ticker_interval,
-            refresh_pairs=self.config.get('refresh_pairs', False),
-            exchange=self.backtesting.exchange,
             timerange=timerange
         )
 
@@ -370,9 +378,6 @@ class Hyperopt:
             (max_date - min_date).days
         )
 
-        self.backtesting.strategy.advise_indicators = \
-            self.custom_hyperopt.populate_indicators  # type: ignore
-
         preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data)
 
         dump(preprocessed, self.tickerdata_pickle)
@@ -387,7 +392,8 @@ class Hyperopt:
         config_jobs = self.config.get('hyperopt_jobs', -1)
         logger.info(f'Number of parallel jobs set as: {config_jobs}')
 
-        opt = self.get_optimizer(config_jobs)
+        self.dimensions = self.hyperopt_space()
+        self.opt = self.get_optimizer(self.dimensions, config_jobs)
 
         if self.config.get('print_colorized', False):
             colorama_init(autoreset=True)
@@ -398,9 +404,9 @@ class Hyperopt:
                 logger.info(f'Effective number of parallel workers used: {jobs}')
                 EVALS = max(self.total_epochs // jobs, 1)
                 for i in range(EVALS):
-                    asked = opt.ask(n_points=jobs)
+                    asked = self.opt.ask(n_points=jobs)
                     f_val = self.run_optimizer_parallel(parallel, asked)
-                    opt.tell(asked, [v['loss'] for v in f_val])
+                    self.opt.tell(asked, [v['loss'] for v in f_val])
                     for j in range(jobs):
                         current = i * jobs + j
                         val = f_val[j]

freqtrade/persistence.py

@@ -1,7 +1,6 @@
 """
 This module contains the class to persist trades into SQLite
 """
-
 import logging
 from datetime import datetime
 from decimal import Decimal
@@ -19,8 +18,10 @@ from sqlalchemy.pool import StaticPool
 
 from freqtrade import OperationalException
 
+
 logger = logging.getLogger(__name__)
 
+
 _DECL_BASE: Any = declarative_base()
 _SQL_DOCS_URL = 'http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls'
 
@@ -209,7 +210,7 @@ class Trade(_DECL_BASE):
     ticker_interval = Column(Integer, nullable=True)
 
     def __repr__(self):
-        open_since = arrow.get(self.open_date).humanize() if self.is_open else 'closed'
+        open_since = self.open_date.strftime('%Y-%m-%d %H:%M:%S') if self.is_open else 'closed'
 
         return (f'Trade(id={self.id}, pair={self.pair}, amount={self.amount:.8f}, '
                 f'open_rate={self.open_rate:.8f}, open_since={open_since})')
@@ -250,7 +251,6 @@ class Trade(_DECL_BASE):
         :param initial: Called to initiate stop_loss.
             Skips everything if self.stop_loss is already set.
         """
-
         if initial and not (self.stop_loss is None or self.stop_loss == 0):
             # Don't modify if called with initial and nothing to do
             return
@@ -259,7 +259,7 @@ class Trade(_DECL_BASE):
 
         # no stop loss assigned yet
         if not self.stop_loss:
-            logger.debug("assigning new stop loss")
+            logger.debug(f"{self.pair} - Assigning new stoploss...")
             self.stop_loss = new_loss
             self.stop_loss_pct = -1 * abs(stoploss)
             self.initial_stop_loss = new_loss
@@ -269,21 +269,20 @@ class Trade(_DECL_BASE):
         # evaluate if the stop loss needs to be updated
         else:
             if new_loss > self.stop_loss:  # stop losses only walk up, never down!
+                logger.debug(f"{self.pair} - Adjusting stoploss...")
                 self.stop_loss = new_loss
                 self.stop_loss_pct = -1 * abs(stoploss)
                 self.stoploss_last_update = datetime.utcnow()
-                logger.debug("adjusted stop loss")
             else:
-                logger.debug("keeping current stop loss")
+                logger.debug(f"{self.pair} - Keeping current stoploss...")
 
         logger.debug(
-            f"{self.pair} - current price {current_price:.8f}, "
+            f"{self.pair} - Stoploss adjusted. current_price={current_price:.8f}, "
-            f"bought at {self.open_rate:.8f} and calculated "
+            f"open_rate={self.open_rate:.8f}, max_rate={self.max_rate:.8f}, "
-            f"stop loss is at: {self.initial_stop_loss:.8f} initial "
+            f"initial_stop_loss={self.initial_stop_loss:.8f}, "
-            f"stop at {self.stop_loss:.8f}. "
+            f"stop_loss={self.stop_loss:.8f}. "
-            f"trailing stop loss saved us: "
+            f"Trailing stoploss saved us: "
-            f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f} "
+            f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f}.")
-            f"and max observed rate was {self.max_rate:.8f}")
 
     def update(self, order: Dict) -> None:
         """
@@ -331,23 +330,18 @@ class Trade(_DECL_BASE):
             self
         )
 
-    def calc_open_trade_price(
+    def calc_open_trade_price(self, fee: Optional[float] = None) -> float:
-            self,
-            fee: Optional[float] = None) -> float:
         """
         Calculate the open_rate including fee.
         :param fee: fee to use on the open rate (optional).
         If rate is not set self.fee will be used
         :return: Price in of the open trade incl. Fees
         """
-
         buy_trade = (Decimal(self.amount) * Decimal(self.open_rate))
         fees = buy_trade * Decimal(fee or self.fee_open)
         return float(buy_trade + fees)
 
-    def calc_close_trade_price(
+    def calc_close_trade_price(self, rate: Optional[float] = None,
-            self,
-            rate: Optional[float] = None,
                                fee: Optional[float] = None) -> float:
         """
         Calculate the close_rate including fee
@@ -357,7 +351,6 @@ class Trade(_DECL_BASE):
         If rate is not set self.close_rate will be used
         :return: Price in BTC of the open trade
         """
-
         if rate is None and not self.close_rate:
             return 0.0
 
@@ -365,9 +358,7 @@ class Trade(_DECL_BASE):
         fees = sell_trade * Decimal(fee or self.fee_close)
         return float(sell_trade - fees)
 
-    def calc_profit(
+    def calc_profit(self, rate: Optional[float] = None,
-            self,
-            rate: Optional[float] = None,
                     fee: Optional[float] = None) -> float:
         """
         Calculate the absolute profit in stake currency between Close and Open trade
@@ -385,9 +376,7 @@ class Trade(_DECL_BASE):
         profit = close_trade_price - open_trade_price
         return float(f"{profit:.8f}")
 
-    def calc_profit_percent(
+    def calc_profit_percent(self, rate: Optional[float] = None,
-            self,
-            rate: Optional[float] = None,
                             fee: Optional[float] = None) -> float:
         """
         Calculates the profit in percentage (including fee).
@@ -396,7 +385,6 @@ class Trade(_DECL_BASE):
         :param fee: fee to use on the close rate (optional).
         :return: profit in percentage as float
         """
-
         open_trade_price = self.calc_open_trade_price()
         close_trade_price = self.calc_close_trade_price(
             rate=(rate or self.close_rate),
@@ -436,8 +424,8 @@ class Trade(_DECL_BASE):
                and trade.initial_stop_loss_pct != desired_stoploss):
                 # Stoploss value got changed
 
-                logger.info(f"Stoploss for {trade} needs adjustment.")
+                logger.info(f"Stoploss for {trade} needs adjustment...")
                 # Force reset of stoploss
                 trade.stop_loss = None
                 trade.adjust_stop_loss(trade.open_rate, desired_stoploss)
-                logger.info(f"new stoploss: {trade.stop_loss}, ")
+                logger.info(f"New stoploss: {trade.stop_loss}.")

freqtrade/plot/plot_utils.py

@@ -1,18 +1,18 @@
-from argparse import Namespace
+from typing import Any, Dict
+
 from freqtrade import OperationalException
 from freqtrade.state import RunMode
 from freqtrade.utils import setup_utils_configuration
 
 
-def validate_plot_args(args: Namespace):
+def validate_plot_args(args: Dict[str, Any]):
-    args_tmp = vars(args)
+    if not args.get('datadir') and not args.get('config'):
-    if not args_tmp.get('datadir') and not args_tmp.get('config'):
         raise OperationalException(
             "You need to specify either `--datadir` or `--config` "
             "for plot-profit and plot-dataframe.")
 
 
-def start_plot_dataframe(args: Namespace) -> None:
+def start_plot_dataframe(args: Dict[str, Any]) -> None:
     """
     Entrypoint for dataframe plotting
     """
@@ -24,7 +24,7 @@ def start_plot_dataframe(args: Namespace) -> None:
     load_and_plot_trades(config)
 
 
-def start_plot_profit(args: Namespace) -> None:
+def start_plot_profit(args: Dict[str, Any]) -> None:
     """
     Entrypoint for plot_profit
     """

freqtrade/resolvers/hyperopt_resolver.py

@@ -38,11 +38,11 @@ class HyperOptResolver(IResolver):
         IHyperOpt.ticker_interval = str(config['ticker_interval'])
 
         if not hasattr(self.hyperopt, 'populate_buy_trend'):
-            logger.warning("Custom Hyperopt does not provide populate_buy_trend. "
+            logger.warning("Hyperopt class does not provide populate_buy_trend() method. "
-                           "Using populate_buy_trend from DefaultStrategy.")
+                           "Using populate_buy_trend from the strategy.")
         if not hasattr(self.hyperopt, 'populate_sell_trend'):
-            logger.warning("Custom Hyperopt does not provide populate_sell_trend. "
+            logger.warning("Hyperopt class does not provide populate_sell_trend() method. "
-                           "Using populate_sell_trend from DefaultStrategy.")
+                           "Using populate_sell_trend from the strategy.")
 
     def _load_hyperopt(
             self, hyperopt_name: str, config: Dict, extra_dir: Optional[str] = None) -> IHyperOpt:

freqtrade/resolvers/iresolver.py

@@ -12,7 +12,7 @@ from typing import Any, List, Optional, Tuple, Type, Union
 logger = logging.getLogger(__name__)
 
 
-class IResolver(object):
+class IResolver:
     """
     This class contains all the logic to load custom classes
     """

freqtrade/rpc/fiat_convert.py

@@ -15,7 +15,7 @@ from freqtrade.constants import SUPPORTED_FIAT
 logger = logging.getLogger(__name__)
 
 
-class CryptoFiat(object):
+class CryptoFiat:
     """
     Object to describe what is the price of Crypto-currency in a FIAT
     """
@@ -60,7 +60,7 @@ class CryptoFiat(object):
         return self._expiration - time.time() <= 0
 
 
-class CryptoToFiatConverter(object):
+class CryptoToFiatConverter:
     """
     Main class to initiate Crypto to FIAT.
     This object contains a list of pair Crypto, FIAT
@@ -104,7 +104,7 @@ class CryptoToFiatConverter(object):
         :return: float, value in fiat of the crypto-currency amount
         """
         if crypto_symbol == fiat_symbol:
-            return crypto_amount
+            return float(crypto_amount)
         price = self.get_price(crypto_symbol=crypto_symbol, fiat_symbol=fiat_symbol)
         return float(crypto_amount) * float(price)
 

freqtrade/rpc/rpc.py

@@ -54,7 +54,7 @@ class RPCException(Exception):
         }
 
 
-class RPC(object):
+class RPC:
     """
     RPC class can be used to have extra feature, like bot data, and access to DB data
     """

freqtrade/rpc/rpc_manager.py

@@ -9,7 +9,7 @@ from freqtrade.rpc import RPC, RPCMessageType
 logger = logging.getLogger(__name__)
 
 
-class RPCManager(object):
+class RPCManager:
     """
     Class to manage RPC objects (Telegram, Slack, ...)
     """

freqtrade/strategy/default_strategy.py

@@ -4,14 +4,16 @@ import talib.abstract as ta
 from pandas import DataFrame
 
 import freqtrade.vendor.qtpylib.indicators as qtpylib
-from freqtrade.indicator_helpers import fishers_inverse
 from freqtrade.strategy.interface import IStrategy
 
 
 class DefaultStrategy(IStrategy):
     """
     Default Strategy provided by freqtrade bot.
-    You can override it with your own strategy
+    Please do not modify this strategy, it's  intended for internal use only.
+    Please look at the SampleStrategy in the user_data/strategy directory
+    or strategy repository https://github.com/freqtrade/freqtrade-strategies
+    for samples and inspiration.
     """
     INTERFACE_VERSION = 2
 
@@ -74,67 +76,25 @@ class DefaultStrategy(IStrategy):
         # ADX
         dataframe['adx'] = ta.ADX(dataframe)
 
-        # Awesome oscillator
-        dataframe['ao'] = qtpylib.awesome_oscillator(dataframe)
-        """
-        # Commodity Channel Index: values Oversold:<-100, Overbought:>100
-        dataframe['cci'] = ta.CCI(dataframe)
-        """
         # MACD
         macd = ta.MACD(dataframe)
         dataframe['macd'] = macd['macd']
         dataframe['macdsignal'] = macd['macdsignal']
         dataframe['macdhist'] = macd['macdhist']
 
-        # MFI
-        dataframe['mfi'] = ta.MFI(dataframe)
-
         # Minus Directional Indicator / Movement
-        dataframe['minus_dm'] = ta.MINUS_DM(dataframe)
         dataframe['minus_di'] = ta.MINUS_DI(dataframe)
 
         # Plus Directional Indicator / Movement
-        dataframe['plus_dm'] = ta.PLUS_DM(dataframe)
         dataframe['plus_di'] = ta.PLUS_DI(dataframe)
-        dataframe['minus_di'] = ta.MINUS_DI(dataframe)
 
-        """
-        # ROC
-        dataframe['roc'] = ta.ROC(dataframe)
-        """
         # RSI
         dataframe['rsi'] = ta.RSI(dataframe)
 
-        # Inverse Fisher transform on RSI, values [-1.0, 1.0] (https://goo.gl/2JGGoy)
-        dataframe['fisher_rsi'] = fishers_inverse(dataframe['rsi'])
-
-        # Inverse Fisher transform on RSI normalized, value [0.0, 100.0] (https://goo.gl/2JGGoy)
-        dataframe['fisher_rsi_norma'] = 50 * (dataframe['fisher_rsi'] + 1)
-
-        # Stoch
-        stoch = ta.STOCH(dataframe)
-        dataframe['slowd'] = stoch['slowd']
-        dataframe['slowk'] = stoch['slowk']
-
         # Stoch fast
         stoch_fast = ta.STOCHF(dataframe)
         dataframe['fastd'] = stoch_fast['fastd']
         dataframe['fastk'] = stoch_fast['fastk']
-        """
-        # Stoch RSI
-        stoch_rsi = ta.STOCHRSI(dataframe)
-        dataframe['fastd_rsi'] = stoch_rsi['fastd']
-        dataframe['fastk_rsi'] = stoch_rsi['fastk']
-        """
-
-        # Overlap Studies
-        # ------------------------------------
-
-        # Previous Bollinger bands
-        # Because ta.BBANDS implementation is broken with small numbers, it actually
-        # returns middle band for all the three bands. Switch to qtpylib.bollinger_bands
-        # and use middle band instead.
-        dataframe['blower'] = ta.BBANDS(dataframe, nbdevup=2, nbdevdn=2)['lowerband']
 
         # Bollinger bands
         bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
@@ -143,88 +103,11 @@ class DefaultStrategy(IStrategy):
         dataframe['bb_upperband'] = bollinger['upper']
 
         # EMA - Exponential Moving Average
-        dataframe['ema3'] = ta.EMA(dataframe, timeperiod=3)
-        dataframe['ema5'] = ta.EMA(dataframe, timeperiod=5)
         dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10)
-        dataframe['ema50'] = ta.EMA(dataframe, timeperiod=50)
-        dataframe['ema100'] = ta.EMA(dataframe, timeperiod=100)
-
-        # SAR Parabol
-        dataframe['sar'] = ta.SAR(dataframe)
 
         # SMA - Simple Moving Average
         dataframe['sma'] = ta.SMA(dataframe, timeperiod=40)
 
-        # TEMA - Triple Exponential Moving Average
-        dataframe['tema'] = ta.TEMA(dataframe, timeperiod=9)
-
-        # Cycle Indicator
-        # ------------------------------------
-        # Hilbert Transform Indicator - SineWave
-        hilbert = ta.HT_SINE(dataframe)
-        dataframe['htsine'] = hilbert['sine']
-        dataframe['htleadsine'] = hilbert['leadsine']
-
-        # Pattern Recognition - Bullish candlestick patterns
-        # ------------------------------------
-        """
-        # Hammer: values [0, 100]
-        dataframe['CDLHAMMER'] = ta.CDLHAMMER(dataframe)
-        # Inverted Hammer: values [0, 100]
-        dataframe['CDLINVERTEDHAMMER'] = ta.CDLINVERTEDHAMMER(dataframe)
-        # Dragonfly Doji: values [0, 100]
-        dataframe['CDLDRAGONFLYDOJI'] = ta.CDLDRAGONFLYDOJI(dataframe)
-        # Piercing Line: values [0, 100]
-        dataframe['CDLPIERCING'] = ta.CDLPIERCING(dataframe) # values [0, 100]
-        # Morningstar: values [0, 100]
-        dataframe['CDLMORNINGSTAR'] = ta.CDLMORNINGSTAR(dataframe) # values [0, 100]
-        # Three White Soldiers: values [0, 100]
-        dataframe['CDL3WHITESOLDIERS'] = ta.CDL3WHITESOLDIERS(dataframe) # values [0, 100]
-        """
-
-        # Pattern Recognition - Bearish candlestick patterns
-        # ------------------------------------
-        """
-        # Hanging Man: values [0, 100]
-        dataframe['CDLHANGINGMAN'] = ta.CDLHANGINGMAN(dataframe)
-        # Shooting Star: values [0, 100]
-        dataframe['CDLSHOOTINGSTAR'] = ta.CDLSHOOTINGSTAR(dataframe)
-        # Gravestone Doji: values [0, 100]
-        dataframe['CDLGRAVESTONEDOJI'] = ta.CDLGRAVESTONEDOJI(dataframe)
-        # Dark Cloud Cover: values [0, 100]
-        dataframe['CDLDARKCLOUDCOVER'] = ta.CDLDARKCLOUDCOVER(dataframe)
-        # Evening Doji Star: values [0, 100]
-        dataframe['CDLEVENINGDOJISTAR'] = ta.CDLEVENINGDOJISTAR(dataframe)
-        # Evening Star: values [0, 100]
-        dataframe['CDLEVENINGSTAR'] = ta.CDLEVENINGSTAR(dataframe)
-        """
-
-        # Pattern Recognition - Bullish/Bearish candlestick patterns
-        # ------------------------------------
-        """
-        # Three Line Strike: values [0, -100, 100]
-        dataframe['CDL3LINESTRIKE'] = ta.CDL3LINESTRIKE(dataframe)
-        # Spinning Top: values [0, -100, 100]
-        dataframe['CDLSPINNINGTOP'] = ta.CDLSPINNINGTOP(dataframe) # values [0, -100, 100]
-        # Engulfing: values [0, -100, 100]
-        dataframe['CDLENGULFING'] = ta.CDLENGULFING(dataframe) # values [0, -100, 100]
-        # Harami: values [0, -100, 100]
-        dataframe['CDLHARAMI'] = ta.CDLHARAMI(dataframe) # values [0, -100, 100]
-        # Three Outside Up/Down: values [0, -100, 100]
-        dataframe['CDL3OUTSIDE'] = ta.CDL3OUTSIDE(dataframe) # values [0, -100, 100]
-        # Three Inside Up/Down: values [0, -100, 100]
-        dataframe['CDL3INSIDE'] = ta.CDL3INSIDE(dataframe) # values [0, -100, 100]
-        """
-
-        # Chart type
-        # ------------------------------------
-        # Heikinashi stategy
-        heikinashi = qtpylib.heikinashi(dataframe)
-        dataframe['ha_open'] = heikinashi['open']
-        dataframe['ha_close'] = heikinashi['close']
-        dataframe['ha_high'] = heikinashi['high']
-        dataframe['ha_low'] = heikinashi['low']
-
         return dataframe
 
     def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

freqtrade/strategy/interface.py

@@ -202,7 +202,6 @@ class IStrategy(ABC):
         :param metadata: Metadata dictionary with additional data (e.g. 'pair')
         :return: DataFrame with ticker data and indicator data
         """
-
         pair = str(metadata.get('pair'))
 
         # Test if seen this pair and last candle before.
@@ -292,7 +291,6 @@ class IStrategy(ABC):
         :param force_stoploss: Externally provided stoploss
         :return: True if trade should be sold, False otherwise
         """
-
         # Set current rate to low for backtesting sell
         current_rate = low or rate
         current_profit = trade.calc_profit_percent(current_rate)
@@ -304,6 +302,8 @@ class IStrategy(ABC):
                                               force_stoploss=force_stoploss, high=high)
 
         if stoplossflag.sell_flag:
+            logger.debug(f"{trade.pair} - Stoploss hit. sell_flag=True, "
+                         f"sell_type={stoplossflag.sell_type}")
             return stoplossflag
 
         # Set current rate to high for backtesting sell
@@ -312,22 +312,31 @@ class IStrategy(ABC):
         experimental = self.config.get('experimental', {})
 
         if buy and experimental.get('ignore_roi_if_buy_signal', False):
-            logger.debug('Buy signal still active - not selling.')
+            # This one is noisy, commented out
+            # logger.debug(f"{trade.pair} - Buy signal still active. sell_flag=False")
             return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
 
         # Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee)
         if self.min_roi_reached(trade=trade, current_profit=current_profit, current_time=date):
-            logger.debug('Required profit reached. Selling..')
+            logger.debug(f"{trade.pair} - Required profit reached. sell_flag=True, "
+                         f"sell_type=SellType.ROI")
             return SellCheckTuple(sell_flag=True, sell_type=SellType.ROI)
 
         if experimental.get('sell_profit_only', False):
-            logger.debug('Checking if trade is profitable..')
+            # This one is noisy, commented out
+            # logger.debug(f"{trade.pair} - Checking if trade is profitable...")
             if trade.calc_profit(rate=rate) <= 0:
+                # This one is noisy, commented out
+                # logger.debug(f"{trade.pair} - Trade is not profitable. sell_flag=False")
                 return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
+
         if sell and not buy and experimental.get('use_sell_signal', False):
-            logger.debug('Sell signal received. Selling..')
+            logger.debug(f"{trade.pair} - Sell signal received. sell_flag=True, "
+                         f"sell_type=SellType.SELL_SIGNAL")
             return SellCheckTuple(sell_flag=True, sell_type=SellType.SELL_SIGNAL)
 
+        # This one is noisy, commented out...
+        # logger.debug(f"{trade.pair} - No sell signal. sell_flag=False")
         return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
 
     def stop_loss_reached(self, current_rate: float, trade: Trade,
@@ -338,7 +347,6 @@ class IStrategy(ABC):
         decides to sell or not
         :param current_profit: current profit in percent
         """
-
         trailing_stop = self.config.get('trailing_stop', False)
         stop_loss_value = force_stoploss if force_stoploss else self.stoploss
 
@@ -359,7 +367,7 @@ class IStrategy(ABC):
                 if 'trailing_stop_positive' in self.config and high_profit > sl_offset:
                     # Ignore mypy error check in configuration that this is a float
                     stop_loss_value = self.config.get('trailing_stop_positive')  # type: ignore
-                    logger.debug(f"using positive stop loss: {stop_loss_value} "
+                    logger.debug(f"{trade.pair} - Using positive stoploss: {stop_loss_value} "
                                  f"offset: {sl_offset:.4g} profit: {current_profit:.4f}%")
 
                 trade.adjust_stop_loss(high or current_rate, stop_loss_value)
@@ -369,20 +377,20 @@ class IStrategy(ABC):
             (trade.stop_loss >= current_rate) and
                 (not self.order_types.get('stoploss_on_exchange'))):
 
-            selltype = SellType.STOP_LOSS
+            sell_type = SellType.STOP_LOSS
 
             # If initial stoploss is not the same as current one then it is trailing.
             if trade.initial_stop_loss != trade.stop_loss:
-                selltype = SellType.TRAILING_STOP_LOSS
+                sell_type = SellType.TRAILING_STOP_LOSS
                 logger.debug(
-                    f"HIT STOP: current price at {current_rate:.6f}, "
+                    f"{trade.pair} - HIT STOP: current price at {current_rate:.6f}, "
                     f"stoploss is {trade.stop_loss:.6f}, "
                     f"initial stoploss was at {trade.initial_stop_loss:.6f}, "
                     f"trade opened at {trade.open_rate:.6f}")
-                logger.debug(f"trailing stop saved {trade.stop_loss - trade.initial_stop_loss:.6f}")
+                logger.debug(f"{trade.pair} - Trailing stop saved "
+                             f"{trade.stop_loss - trade.initial_stop_loss:.6f}")
 
-            logger.debug('Stop loss hit.')
+            return SellCheckTuple(sell_flag=True, sell_type=sell_type)
-            return SellCheckTuple(sell_flag=True, sell_type=selltype)
 
         return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
 

freqtrade/utils.py

@@ -1,11 +1,11 @@
 import logging
 import sys
-from argparse import Namespace
 from pathlib import Path
 from typing import Any, Dict, List
 
 import arrow
 
+from freqtrade import OperationalException
 from freqtrade.configuration import Configuration, TimeRange
 from freqtrade.configuration.directory_operations import create_userdata_dir
 from freqtrade.data.history import refresh_backtest_ohlcv_data
@@ -16,7 +16,7 @@ from freqtrade.state import RunMode
 logger = logging.getLogger(__name__)
 
 
-def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
+def setup_utils_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
     """
     Prepare the configuration for utils subcommands
     :param args: Cli args from Arguments()
@@ -33,34 +33,34 @@ def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any
     return config
 
 
-def start_list_exchanges(args: Namespace) -> None:
+def start_list_exchanges(args: Dict[str, Any]) -> None:
     """
     Print available exchanges
     :param args: Cli args from Arguments()
     :return: None
     """
 
-    if args.print_one_column:
+    if args['print_one_column']:
         print('\n'.join(available_exchanges()))
     else:
         print(f"Exchanges supported by ccxt and available for Freqtrade: "
               f"{', '.join(available_exchanges())}")
 
 
-def start_create_userdir(args: Namespace) -> None:
+def start_create_userdir(args: Dict[str, Any]) -> None:
     """
     Create "user_data" directory to contain user data strategies, hyperopts, ...)
     :param args: Cli args from Arguments()
     :return: None
     """
-    if "user_data_dir" in args and args.user_data_dir:
+    if "user_data_dir" in args and args["user_data_dir"]:
-        create_userdata_dir(args.user_data_dir, create_dir=True)
+        create_userdata_dir(args["user_data_dir"], create_dir=True)
     else:
         logger.warning("`create-userdir` requires --userdir to be set.")
         sys.exit(1)
 
 
-def start_download_data(args: Namespace) -> None:
+def start_download_data(args: Dict[str, Any]) -> None:
     """
     Download data (former download_backtest_data.py script)
     """
@@ -71,6 +71,11 @@ def start_download_data(args: Namespace) -> None:
         time_since = arrow.utcnow().shift(days=-config['days']).strftime("%Y%m%d")
         timerange = TimeRange.parse_timerange(f'{time_since}-')
 
+    if 'pairs' not in config:
+        raise OperationalException(
+            "Downloading data requires a list of pairs. "
+            "Please check the documentation on how to configure this.")
+
     dl_path = Path(config['datadir'])
     logger.info(f'About to download pairs: {config["pairs"]}, '
                 f'intervals: {config["timeframes"]} to {dl_path}')

freqtrade/wallets.py

@@ -17,7 +17,7 @@ class Wallet(NamedTuple):
     total: float = 0
 
 
-class Wallets(object):
+class Wallets:
 
     def __init__(self, config: dict, exchange: Exchange) -> None:
         self._config = config

freqtrade/worker.py

@@ -4,27 +4,26 @@ Main Freqtrade worker class.
 import logging
 import time
 import traceback
-from argparse import Namespace
+from typing import Any, Callable, Dict, Optional
-from typing import Any, Callable, Optional
+
 import sdnotify
 
-from freqtrade import (constants, OperationalException, TemporaryError,
+from freqtrade import (OperationalException, TemporaryError, __version__,
-                       __version__)
+                       constants)
 from freqtrade.configuration import Configuration
 from freqtrade.freqtradebot import FreqtradeBot
-from freqtrade.state import State
 from freqtrade.rpc import RPCMessageType
-
+from freqtrade.state import State
 
 logger = logging.getLogger(__name__)
 
 
-class Worker(object):
+class Worker:
     """
     Freqtradebot worker class
     """
 
-    def __init__(self, args: Namespace, config=None) -> None:
+    def __init__(self, args: Dict[str, Any], config=None) -> None:
         """
         Init all variables and objects the bot needs to work
         """

requirements-common.txt

@@ -1,12 +1,12 @@
 # requirements without requirements installable via conda
 # mainly used for Raspberry pi installs
-ccxt==1.18.1115
+ccxt==1.18.1180
 SQLAlchemy==1.3.8
-python-telegram-bot==12.0.0
+python-telegram-bot==12.1.1
-arrow==0.14.6
+arrow==0.15.2
 cachetools==3.1.1
 requests==2.22.0
-urllib3==1.25.3
+urllib3==1.25.5
 wrapt==1.11.2
 scikit-learn==0.21.3
 joblib==0.13.2

requirements-dev.txt

@@ -7,7 +7,7 @@ flake8==3.7.8
 flake8-type-annotations==0.1.0
 flake8-tidy-imports==2.0.0
 mypy==0.720
-pytest==5.1.2
+pytest==5.1.3
 pytest-asyncio==0.10.0
 pytest-cov==2.7.1
 pytest-mock==1.10.4

setup.py

@@ -46,7 +46,7 @@ setup(name='freqtrade',
       long_description=readme_long,
       long_description_content_type="text/markdown",
       url='https://github.com/freqtrade/freqtrade',
-      author='gcarq and contributors',
+      author='Freqtrade Team',
       author_email='michael.egger@tsn.at',
       license='GPLv3',
       packages=['freqtrade'],

tests/conftest.py

@@ -117,7 +117,7 @@ def patch_freqtradebot(mocker, config) -> None:
     """
     mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
     persistence.init(config['db_url'])
-    patch_exchange(mocker, None)
+    patch_exchange(mocker)
     mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
     mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
 

tests/data/test_converter.py

@@ -24,7 +24,6 @@ def test_parse_ticker_dataframe(ticker_history_list, caplog):
 def test_ohlcv_fill_up_missing_data(testdatadir, caplog):
     data = load_pair_history(datadir=testdatadir,
                              ticker_interval='1m',
-                             refresh_pairs=False,
                              pair='UNITTEST/BTC',
                              fill_up_missing=False)
     caplog.set_level(logging.DEBUG)

tests/data/test_dataprovider.py

@@ -45,7 +45,6 @@ def test_historic_ohlcv(mocker, default_conf, ticker_history):
     data = dp.historic_ohlcv("UNITTEST/BTC", "5m")
     assert isinstance(data, DataFrame)
     assert historymock.call_count == 1
-    assert historymock.call_args_list[0][1]["refresh_pairs"] is False
     assert historymock.call_args_list[0][1]["ticker_interval"] == "5m"
 
 

tests/data/test_history.py

@@ -74,8 +74,7 @@ def test_load_data_7min_ticker(mocker, caplog, default_conf, testdatadir) -> Non
     assert ld is None
     assert log_has(
         'No history data for pair: "UNITTEST/BTC", interval: 7m. '
-        'Use --refresh-pairs-cached option or `freqtrade download-data` '
+        'Use `freqtrade download-data` to download the data', caplog
-        'script to download the data', caplog
     )
 
 
@@ -105,13 +104,11 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog,
     # do not download a new pair if refresh_pairs isn't set
     history.load_pair_history(datadir=testdatadir,
                               ticker_interval='1m',
-                              refresh_pairs=False,
                               pair='MEME/BTC')
     assert os.path.isfile(file) is False
     assert log_has(
         'No history data for pair: "MEME/BTC", interval: 1m. '
-        'Use --refresh-pairs-cached option or `freqtrade download-data` '
+        'Use `freqtrade download-data` to download the data', caplog
-        'script to download the data', caplog
     )
 
     # download a new pair if refresh_pairs is set
@@ -134,31 +131,6 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog,
     _clean_test_file(file)
 
 
-def test_load_data_live(default_conf, mocker, caplog, testdatadir) -> None:
-    refresh_mock = MagicMock()
-    mocker.patch("freqtrade.exchange.Exchange.refresh_latest_ohlcv", refresh_mock)
-    exchange = get_patched_exchange(mocker, default_conf)
-
-    history.load_data(datadir=testdatadir, ticker_interval='5m',
-                      pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
-                      live=True,
-                      exchange=exchange)
-    assert refresh_mock.call_count == 1
-    assert len(refresh_mock.call_args_list[0][0][0]) == 2
-    assert log_has('Live: Downloading data for all defined pairs ...', caplog)
-
-
-def test_load_data_live_noexchange(default_conf, mocker, caplog, testdatadir) -> None:
-
-    with pytest.raises(OperationalException,
-                       match=r'Exchange needs to be initialized when using live data.'):
-        history.load_data(datadir=testdatadir, ticker_interval='5m',
-                          pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
-                          exchange=None,
-                          live=True,
-                          )
-
-
 def test_testdata_path(testdatadir) -> None:
     assert str(Path('tests') / 'testdata') in str(testdatadir)
 
@@ -349,7 +321,6 @@ def test_load_partial_missing(testdatadir, caplog) -> None:
     start = arrow.get('2018-01-01T00:00:00')
     end = arrow.get('2018-01-11T00:00:00')
     tickerdata = history.load_data(testdatadir, '5m', ['UNITTEST/BTC'],
-                                   refresh_pairs=False,
                                    timerange=TimeRange('date', 'date',
                                                        start.timestamp, end.timestamp))
     # timedifference in 5 minutes
@@ -364,7 +335,7 @@ def test_load_partial_missing(testdatadir, caplog) -> None:
     start = arrow.get('2018-01-10T00:00:00')
     end = arrow.get('2018-02-20T00:00:00')
     tickerdata = history.load_data(datadir=testdatadir, ticker_interval='5m',
-                                   pairs=['UNITTEST/BTC'], refresh_pairs=False,
+                                   pairs=['UNITTEST/BTC'],
                                    timerange=TimeRange('date', 'date',
                                                        start.timestamp, end.timestamp))
     # timedifference in 5 minutes

tests/exchange/test_exchange.py

@@ -867,7 +867,7 @@ def test_get_balance_dry_run(default_conf, mocker):
 @pytest.mark.parametrize("exchange_name", EXCHANGES)
 def test_get_balance_prod(default_conf, mocker, exchange_name):
     api_mock = MagicMock()
-    api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4}})
+    api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4, 'total': 123.4}})
     default_conf['dry_run'] = False
 
     exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
@@ -883,6 +883,7 @@ def test_get_balance_prod(default_conf, mocker, exchange_name):
     with pytest.raises(TemporaryError, match=r'.*balance due to malformed exchange response:.*'):
         exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
         mocker.patch('freqtrade.exchange.Exchange.get_balances', MagicMock(return_value={}))
+        mocker.patch('freqtrade.exchange.Kraken.get_balances', MagicMock(return_value={}))
         exchange.get_balance(currency='BTC')
 
 
@@ -1363,9 +1364,7 @@ def test_get_order(default_conf, mocker, exchange_name):
 
 @pytest.mark.parametrize("exchange_name", EXCHANGES)
 def test_name(default_conf, mocker, exchange_name):
-    mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
+    exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
-    default_conf['exchange']['name'] = exchange_name
-    exchange = Exchange(default_conf)
 
     assert exchange.name == exchange_name.title()
     assert exchange.id == exchange_name

tests/exchange/test_kraken.py

@@ -4,6 +4,7 @@ from random import randint
 from unittest.mock import MagicMock
 
 from tests.conftest import get_patched_exchange
+from tests.exchange.test_exchange import ccxt_exceptionhandlers
 
 
 def test_buy_kraken_trading_agreement(default_conf, mocker):
@@ -67,3 +68,84 @@ def test_sell_kraken_trading_agreement(default_conf, mocker):
     assert api_mock.create_order.call_args[0][3] == 1
     assert api_mock.create_order.call_args[0][4] is None
     assert api_mock.create_order.call_args[0][5] == {'trading_agreement': 'agree'}
+
+
+def test_get_balances_prod(default_conf, mocker):
+    balance_item = {
+        'free': None,
+        'total': 10.0,
+        'used': 0.0
+    }
+
+    api_mock = MagicMock()
+    api_mock.fetch_balance = MagicMock(return_value={
+        '1ST': balance_item.copy(),
+        '2ST': balance_item.copy(),
+        '3ST': balance_item.copy(),
+        '4ST': balance_item.copy(),
+    })
+    kraken_open_orders = [{'symbol': '1ST/EUR',
+                           'type': 'limit',
+                           'side': 'sell',
+                           'price': 20,
+                           'cost': 0.0,
+                           'amount': 1.0,
+                           'filled': 0.0,
+                           'average': 0.0,
+                           'remaining': 1.0,
+                           },
+                          {'status': 'open',
+                           'symbol': '2ST/EUR',
+                           'type': 'limit',
+                           'side': 'sell',
+                           'price': 20.0,
+                           'cost': 0.0,
+                           'amount': 2.0,
+                           'filled': 0.0,
+                           'average': 0.0,
+                           'remaining': 2.0,
+                           },
+                          {'status': 'open',
+                           'symbol': '2ST/USD',
+                           'type': 'limit',
+                           'side': 'sell',
+                           'price': 20.0,
+                           'cost': 0.0,
+                           'amount': 2.0,
+                           'filled': 0.0,
+                           'average': 0.0,
+                           'remaining': 2.0,
+                           },
+                          {'status': 'open',
+                           'symbol': 'BTC/3ST',
+                           'type': 'limit',
+                           'side': 'buy',
+                           'price': 20,
+                           'cost': 0.0,
+                           'amount': 3.0,
+                           'filled': 0.0,
+                           'average': 0.0,
+                           'remaining': 3.0,
+                           }]
+    api_mock.fetch_open_orders = MagicMock(return_value=kraken_open_orders)
+    default_conf['dry_run'] = False
+    exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
+    balances = exchange.get_balances()
+    assert len(balances) == 4
+    assert balances['1ST']['free'] == 9.0
+    assert balances['1ST']['total'] == 10.0
+    assert balances['1ST']['used'] == 1.0
+
+    assert balances['2ST']['free'] == 6.0
+    assert balances['2ST']['total'] == 10.0
+    assert balances['2ST']['used'] == 4.0
+
+    assert balances['3ST']['free'] == 7.0
+    assert balances['3ST']['total'] == 10.0
+    assert balances['3ST']['used'] == 3.0
+
+    assert balances['4ST']['free'] == 10.0
+    assert balances['4ST']['total'] == 10.0
+    assert balances['4ST']['used'] == 0.0
+    ccxt_exceptionhandlers(mocker, default_conf, api_mock, "kraken",
+                           "get_balances", "fetch_balance")

tests/optimize/test_backtest_detail.py

@@ -291,8 +291,8 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
     patch_exchange(mocker)
     frame = _build_backtest_dataframe(data.data)
     backtesting = Backtesting(default_conf)
-    backtesting.advise_buy = lambda a, m: frame
+    backtesting.strategy.advise_buy = lambda a, m: frame
-    backtesting.advise_sell = lambda a, m: frame
+    backtesting.strategy.advise_sell = lambda a, m: frame
     caplog.set_level(logging.DEBUG)
 
     pair = "UNITTEST/BTC"

tests/optimize/test_backtesting.py

@@ -188,16 +188,12 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
     assert 'position_stacking' not in config
     assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
 
-    assert 'refresh_pairs' not in config
-    assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' not in config
     assert 'export' not in config
     assert 'runmode' in config
     assert config['runmode'] == RunMode.BACKTEST
 
 
-@pytest.mark.filterwarnings("ignore:DEPRECATED")
 def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
     patched_configuration_load_config_file(mocker, default_conf)
     mocker.patch(
@@ -213,7 +209,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
         '--ticker-interval', '1m',
         '--enable-position-stacking',
         '--disable-max-market-positions',
-        '--refresh-pairs-cached',
         '--timerange', ':100',
         '--export', '/bar/foo',
         '--export-filename', 'foo_bar.json'
@@ -240,9 +235,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
     assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
     assert log_has('max_open_trades set to unlimited ...', caplog)
 
-    assert 'refresh_pairs' in config
-    assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' in config
     assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
 
@@ -313,8 +305,8 @@ def test_backtesting_init(mocker, default_conf, order_types) -> None:
     assert backtesting.config == default_conf
     assert backtesting.ticker_interval == '5m'
     assert callable(backtesting.strategy.tickerdata_to_dataframe)
-    assert callable(backtesting.advise_buy)
+    assert callable(backtesting.strategy.advise_buy)
-    assert callable(backtesting.advise_sell)
+    assert callable(backtesting.strategy.advise_sell)
     assert isinstance(backtesting.strategy.dp, DataProvider)
     get_fee.assert_called()
     assert backtesting.fee == 0.5
@@ -603,7 +595,7 @@ def test_processed(default_conf, mocker, testdatadir) -> None:
     cols = dataframe.columns
     # assert the dataframe got some of the indicator columns
     for col in ['close', 'high', 'low', 'open', 'date',
-                'ema50', 'ao', 'macd', 'plus_dm']:
+                'ema10', 'rsi', 'fastd', 'plus_di']:
         assert col in cols
 
 
@@ -627,8 +619,8 @@ def test_backtest_clash_buy_sell(mocker, default_conf, testdatadir):
 
     backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
     backtesting = Backtesting(default_conf)
-    backtesting.advise_buy = fun  # Override
+    backtesting.strategy.advise_buy = fun  # Override
-    backtesting.advise_sell = fun  # Override
+    backtesting.strategy.advise_sell = fun  # Override
     results = backtesting.backtest(backtest_conf)
     assert results.empty
 
@@ -642,8 +634,8 @@ def test_backtest_only_sell(mocker, default_conf, testdatadir):
 
     backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
     backtesting = Backtesting(default_conf)
-    backtesting.advise_buy = fun  # Override
+    backtesting.strategy.advise_buy = fun  # Override
-    backtesting.advise_sell = fun  # Override
+    backtesting.strategy.advise_sell = fun  # Override
     results = backtesting.backtest(backtest_conf)
     assert results.empty
 
@@ -657,8 +649,8 @@ def test_backtest_alternate_buy_sell(default_conf, fee, mocker, testdatadir):
     default_conf['experimental'] = {"use_sell_signal": True}
     default_conf['ticker_interval'] = '1m'
     backtesting = Backtesting(default_conf)
-    backtesting.advise_buy = _trend_alternate  # Override
+    backtesting.strategy.advise_buy = _trend_alternate  # Override
-    backtesting.advise_sell = _trend_alternate  # Override
+    backtesting.strategy.advise_sell = _trend_alternate  # Override
     results = backtesting.backtest(backtest_conf)
     backtesting._store_backtest_result("test_.json", results)
     # 200 candles in backtest data
@@ -700,8 +692,8 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair, testdatadir)
     default_conf['ticker_interval'] = '5m'
 
     backtesting = Backtesting(default_conf)
-    backtesting.advise_buy = _trend_alternate_hold  # Override
+    backtesting.strategy.advise_buy = _trend_alternate_hold  # Override
-    backtesting.advise_sell = _trend_alternate_hold  # Override
+    backtesting.strategy.advise_sell = _trend_alternate_hold  # Override
 
     data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
     min_date, max_date = get_timeframe(data_processed)

tests/optimize/test_edge_cli.py

@@ -3,8 +3,6 @@
 
 from unittest.mock import MagicMock
 
-import pytest
-
 from freqtrade.edge import PairInfo
 from freqtrade.optimize import setup_configuration, start_edge
 from freqtrade.optimize.edge_cli import EdgeCli
@@ -35,14 +33,10 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
     assert 'ticker_interval' in config
     assert not log_has_re('Parameter -i/--ticker-interval detected .*', caplog)
 
-    assert 'refresh_pairs' not in config
-    assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' not in config
     assert 'stoploss_range' not in config
 
 
-@pytest.mark.filterwarnings("ignore:DEPRECATED")
 def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> None:
     patched_configuration_load_config_file(mocker, edge_conf)
     mocker.patch(
@@ -56,7 +50,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
         '--datadir', '/foo/bar',
         'edge',
         '--ticker-interval', '1m',
-        '--refresh-pairs-cached',
         '--timerange', ':100',
         '--stoplosses=-0.01,-0.10,-0.001'
     ]
@@ -74,8 +67,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
     assert log_has('Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
                    caplog)
 
-    assert 'refresh_pairs' in config
-    assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
     assert 'timerange' in config
     assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
 

tests/optimize/test_hyperopt.py

@@ -35,12 +35,10 @@ def hyperopt_results():
     return pd.DataFrame(
         {
             'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
-            'profit_percent': [0.1, 0.2, 0.3],
+            'profit_percent': [-0.1, 0.2, 0.3],
-            'profit_abs': [0.2, 0.4, 0.5],
+            'profit_abs': [-0.2, 0.4, 0.6],
             'trade_duration': [10, 30, 10],
-            'profit': [2, 0, 0],
+            'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI]
-            'loss': [0, 0, 1],
-            'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
         }
     )
 
@@ -88,15 +86,11 @@ def test_setup_hyperopt_configuration_without_arguments(mocker, default_conf, ca
     assert 'position_stacking' not in config
     assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
 
-    assert 'refresh_pairs' not in config
-    assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' not in config
     assert 'runmode' in config
     assert config['runmode'] == RunMode.HYPEROPT
 
 
-@pytest.mark.filterwarnings("ignore:DEPRECATED")
 def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
     patched_configuration_load_config_file(mocker, default_conf)
     mocker.patch(
@@ -110,7 +104,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
         'hyperopt',
         '--ticker-interval', '1m',
         '--timerange', ':100',
-        '--refresh-pairs-cached',
         '--enable-position-stacking',
         '--disable-max-market-positions',
         '--epochs', '1000',
@@ -139,9 +132,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
     assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
     assert log_has('max_open_trades set to unlimited ...', caplog)
 
-    assert 'refresh_pairs' in config
-    assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' in config
     assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
 
@@ -168,10 +158,10 @@ def test_hyperoptresolver(mocker, default_conf, caplog) -> None:
     x = HyperOptResolver(default_conf, ).hyperopt
     assert not hasattr(x, 'populate_buy_trend')
     assert not hasattr(x, 'populate_sell_trend')
-    assert log_has("Custom Hyperopt does not provide populate_sell_trend. "
+    assert log_has("Hyperopt class does not provide populate_sell_trend() method. "
-                   "Using populate_sell_trend from DefaultStrategy.", caplog)
+                   "Using populate_sell_trend from the strategy.", caplog)
-    assert log_has("Custom Hyperopt does not provide populate_buy_trend. "
+    assert log_has("Hyperopt class does not provide populate_buy_trend() method. "
-                   "Using populate_buy_trend from DefaultStrategy.", caplog)
+                   "Using populate_buy_trend from the strategy.", caplog)
     assert hasattr(x, "ticker_interval")
 
 
@@ -417,8 +407,8 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
     assert dumper.called
     # Should be called twice, once for tickerdata, once to save evaluations
     assert dumper.call_count == 2
-    assert hasattr(hyperopt.backtesting, "advise_sell")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
-    assert hasattr(hyperopt.backtesting, "advise_buy")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
     assert hasattr(hyperopt, "max_open_trades")
     assert hyperopt.max_open_trades == default_conf['max_open_trades']
     assert hasattr(hyperopt, "position_stacking")
@@ -560,6 +550,7 @@ def test_generate_optimizer(mocker, default_conf) -> None:
     }
 
     hyperopt = Hyperopt(default_conf)
+    hyperopt.dimensions = hyperopt.hyperopt_space()
     generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values()))
     assert generate_optimizer_value == response_expected
 
@@ -710,8 +701,8 @@ def test_simplified_interface_roi_stoploss(mocker, default_conf, caplog, capsys)
     assert dumper.called
     # Should be called twice, once for tickerdata, once to save evaluations
     assert dumper.call_count == 2
-    assert hasattr(hyperopt.backtesting, "advise_sell")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
-    assert hasattr(hyperopt.backtesting, "advise_buy")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
     assert hasattr(hyperopt, "max_open_trades")
     assert hyperopt.max_open_trades == default_conf['max_open_trades']
     assert hasattr(hyperopt, "position_stacking")
@@ -784,8 +775,8 @@ def test_simplified_interface_buy(mocker, default_conf, caplog, capsys) -> None:
     assert dumper.called
     # Should be called twice, once for tickerdata, once to save evaluations
     assert dumper.call_count == 2
-    assert hasattr(hyperopt.backtesting, "advise_sell")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
-    assert hasattr(hyperopt.backtesting, "advise_buy")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
     assert hasattr(hyperopt, "max_open_trades")
     assert hyperopt.max_open_trades == default_conf['max_open_trades']
     assert hasattr(hyperopt, "position_stacking")
@@ -829,8 +820,8 @@ def test_simplified_interface_sell(mocker, default_conf, caplog, capsys) -> None
     assert dumper.called
     # Should be called twice, once for tickerdata, once to save evaluations
     assert dumper.call_count == 2
-    assert hasattr(hyperopt.backtesting, "advise_sell")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
-    assert hasattr(hyperopt.backtesting, "advise_buy")
+    assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
     assert hasattr(hyperopt, "max_open_trades")
     assert hyperopt.max_open_trades == default_conf['max_open_trades']
     assert hasattr(hyperopt, "position_stacking")

tests/rpc/test_fiat_convert.py

@@ -210,3 +210,10 @@ def test_convert_amount(mocker):
         fiat_symbol="BTC"
     )
     assert result == 1.23
+
+    result = fiat_convert.convert_amount(
+        crypto_amount="1.23",
+        crypto_symbol="BTC",
+        fiat_symbol="BTC"
+    )
+    assert result == 1.23

tests/rpc/test_rpc.py

@@ -28,9 +28,9 @@ def prec_satoshi(a, b) -> float:
 # Unit tests
 def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
     mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)

tests/rpc/test_rpc_telegram.py

@@ -90,7 +90,7 @@ def test_cleanup(default_conf, mocker) -> None:
 
 
 def test_authorized_only(default_conf, mocker, caplog) -> None:
-    patch_exchange(mocker, None)
+    patch_exchange(mocker)
 
     chat = Chat(0, 0)
     update = Update(randint(1, 100))
@@ -108,7 +108,7 @@ def test_authorized_only(default_conf, mocker, caplog) -> None:
 
 
 def test_authorized_only_unauthorized(default_conf, mocker, caplog) -> None:
-    patch_exchange(mocker, None)
+    patch_exchange(mocker)
     chat = Chat(0xdeadbeef, 0)
     update = Update(randint(1, 100))
     update.message = Message(randint(1, 100), 0, datetime.utcnow(), chat)
@@ -728,13 +728,12 @@ def test_forcesell_handle(default_conf, update, ticker, fee,
     mocker.patch('freqtrade.rpc.rpc.CryptoToFiatConverter._find_price', return_value=15000.0)
     rpc_mock = mocker.patch('freqtrade.rpc.telegram.Telegram.send_msg', MagicMock())
     mocker.patch('freqtrade.rpc.telegram.Telegram._init', MagicMock())
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets),
-        validate_pairs=MagicMock(return_value={})
     )
 
     freqtradebot = FreqtradeBot(default_conf)
@@ -781,13 +780,12 @@ def test_forcesell_down_handle(default_conf, update, ticker, fee,
                  return_value=15000.0)
     rpc_mock = mocker.patch('freqtrade.rpc.telegram.Telegram.send_msg', MagicMock())
     mocker.patch('freqtrade.rpc.telegram.Telegram._init', MagicMock())
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets),
-        validate_pairs=MagicMock(return_value={})
     )
 
     freqtradebot = FreqtradeBot(default_conf)
@@ -843,7 +841,6 @@ def test_forcesell_all_handle(default_conf, update, ticker, fee, markets, mocker
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets),
-        validate_pairs=MagicMock(return_value={})
     )
     default_conf['max_open_trades'] = 4
     freqtradebot = FreqtradeBot(default_conf)
@@ -927,11 +924,10 @@ def test_forcebuy_handle(default_conf, update, markets, mocker) -> None:
     mocker.patch('freqtrade.rpc.rpc.CryptoToFiatConverter._find_price', return_value=15000.0)
     mocker.patch('freqtrade.rpc.telegram.Telegram._send_msg', MagicMock())
     mocker.patch('freqtrade.rpc.telegram.Telegram._init', MagicMock())
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         markets=PropertyMock(markets),
-        validate_pairs=MagicMock(return_value={})
         )
     fbuy_mock = MagicMock(return_value=None)
     mocker.patch('freqtrade.rpc.RPC._rpc_forcebuy', fbuy_mock)
@@ -967,11 +963,10 @@ def test_forcebuy_handle_exception(default_conf, update, markets, mocker) -> Non
     mocker.patch('freqtrade.rpc.rpc.CryptoToFiatConverter._find_price', return_value=15000.0)
     rpc_mock = mocker.patch('freqtrade.rpc.telegram.Telegram._send_msg', MagicMock())
     mocker.patch('freqtrade.rpc.telegram.Telegram._init', MagicMock())
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         markets=PropertyMock(markets),
-        validate_pairs=MagicMock(return_value={})
     )
     freqtradebot = FreqtradeBot(default_conf)
     patch_get_signal(freqtradebot, (True, False))
@@ -998,7 +993,6 @@ def test_performance_handle(default_conf, update, ticker, fee,
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(markets),
-        validate_pairs=MagicMock(return_value={})
     )
     mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
     freqtradebot = FreqtradeBot(default_conf)

tests/test_arguments.py

@@ -12,48 +12,48 @@ def test_parse_args_none() -> None:
     arguments = Arguments([])
     assert isinstance(arguments, Arguments)
     x = arguments.get_parsed_arg()
-    assert isinstance(x, argparse.Namespace)
+    assert isinstance(x, dict)
     assert isinstance(arguments.parser, argparse.ArgumentParser)
 
 
 def test_parse_args_defaults() -> None:
     args = Arguments([]).get_parsed_arg()
-    assert args.config == ['config.json']
+    assert args["config"] == ['config.json']
-    assert args.strategy_path is None
+    assert args["strategy_path"] is None
-    assert args.datadir is None
+    assert args["datadir"] is None
-    assert args.verbosity == 0
+    assert args["verbosity"] == 0
 
 
 def test_parse_args_config() -> None:
     args = Arguments(['-c', '/dev/null']).get_parsed_arg()
-    assert args.config == ['/dev/null']
+    assert args["config"] == ['/dev/null']
 
     args = Arguments(['--config', '/dev/null']).get_parsed_arg()
-    assert args.config == ['/dev/null']
+    assert args["config"] == ['/dev/null']
 
     args = Arguments(['--config', '/dev/null',
                       '--config', '/dev/zero'],).get_parsed_arg()
-    assert args.config == ['/dev/null', '/dev/zero']
+    assert args["config"] == ['/dev/null', '/dev/zero']
 
 
 def test_parse_args_db_url() -> None:
     args = Arguments(['--db-url', 'sqlite:///test.sqlite']).get_parsed_arg()
-    assert args.db_url == 'sqlite:///test.sqlite'
+    assert args["db_url"] == 'sqlite:///test.sqlite'
 
 
 def test_parse_args_verbose() -> None:
     args = Arguments(['-v']).get_parsed_arg()
-    assert args.verbosity == 1
+    assert args["verbosity"] == 1
 
     args = Arguments(['--verbose']).get_parsed_arg()
-    assert args.verbosity == 1
+    assert args["verbosity"] == 1
 
 
 def test_common_scripts_options() -> None:
     args = Arguments(['download-data', '-p', 'ETH/BTC', 'XRP/BTC']).get_parsed_arg()
 
-    assert args.pairs == ['ETH/BTC', 'XRP/BTC']
+    assert args["pairs"] == ['ETH/BTC', 'XRP/BTC']
-    assert hasattr(args, "func")
+    assert "func" in args
 
 
 def test_parse_args_version() -> None:
@@ -68,7 +68,7 @@ def test_parse_args_invalid() -> None:
 
 def test_parse_args_strategy() -> None:
     args = Arguments(['--strategy', 'SomeStrategy']).get_parsed_arg()
-    assert args.strategy == 'SomeStrategy'
+    assert args["strategy"] == 'SomeStrategy'
 
 
 def test_parse_args_strategy_invalid() -> None:
@@ -78,7 +78,7 @@ def test_parse_args_strategy_invalid() -> None:
 
 def test_parse_args_strategy_path() -> None:
     args = Arguments(['--strategy-path', '/some/path']).get_parsed_arg()
-    assert args.strategy_path == '/some/path'
+    assert args["strategy_path"] == '/some/path'
 
 
 def test_parse_args_strategy_path_invalid() -> None:
@@ -99,20 +99,18 @@ def test_parse_args_backtesting_custom() -> None:
         '-c', 'test_conf.json',
         'backtesting',
         '--ticker-interval', '1m',
-        '--refresh-pairs-cached',
         '--strategy-list',
         'DefaultStrategy',
         'SampleStrategy'
         ]
     call_args = Arguments(args).get_parsed_arg()
-    assert call_args.config == ['test_conf.json']
+    assert call_args["config"] == ['test_conf.json']
-    assert call_args.verbosity == 0
+    assert call_args["verbosity"] == 0
-    assert call_args.subparser == 'backtesting'
+    assert call_args["subparser"] == 'backtesting'
-    assert call_args.func is not None
+    assert call_args["func"] is not None
-    assert call_args.ticker_interval == '1m'
+    assert call_args["ticker_interval"] == '1m'
-    assert call_args.refresh_pairs is True
+    assert type(call_args["strategy_list"]) is list
-    assert type(call_args.strategy_list) is list
+    assert len(call_args["strategy_list"]) == 2
-    assert len(call_args.strategy_list) == 2
 
 
 def test_parse_args_hyperopt_custom() -> None:
@@ -123,12 +121,13 @@ def test_parse_args_hyperopt_custom() -> None:
         '--spaces', 'buy'
     ]
     call_args = Arguments(args).get_parsed_arg()
-    assert call_args.config == ['test_conf.json']
+    assert call_args["config"] == ['test_conf.json']
-    assert call_args.epochs == 20
+    assert call_args["epochs"] == 20
-    assert call_args.verbosity == 0
+    assert call_args["verbosity"] == 0
-    assert call_args.subparser == 'hyperopt'
+    assert call_args["subparser"] == 'hyperopt'
-    assert call_args.spaces == ['buy']
+    assert call_args["spaces"] == ['buy']
-    assert call_args.func is not None
+    assert call_args["func"] is not None
+    assert callable(call_args["func"])
 
 
 def test_download_data_options() -> None:
@@ -139,12 +138,12 @@ def test_download_data_options() -> None:
         '--days', '30',
         '--exchange', 'binance'
     ]
-    args = Arguments(args).get_parsed_arg()
+    pargs = Arguments(args).get_parsed_arg()
 
-    assert args.pairs_file == 'file_with_pairs'
+    assert pargs["pairs_file"] == 'file_with_pairs'
-    assert args.datadir == 'datadir/directory'
+    assert pargs["datadir"] == 'datadir/directory'
-    assert args.days == 30
+    assert pargs["days"] == 30
-    assert args.exchange == 'binance'
+    assert pargs["exchange"] == 'binance'
 
 
 def test_plot_dataframe_options() -> None:
@@ -158,10 +157,10 @@ def test_plot_dataframe_options() -> None:
     ]
     pargs = Arguments(args).get_parsed_arg()
 
-    assert pargs.indicators1 == ["sma10", "sma100"]
+    assert pargs["indicators1"] == ["sma10", "sma100"]
-    assert pargs.indicators2 == ["macd", "fastd", "fastk"]
+    assert pargs["indicators2"] == ["macd", "fastd", "fastk"]
-    assert pargs.plot_limit == 30
+    assert pargs["plot_limit"] == 30
-    assert pargs.pairs == ["UNITTEST/BTC"]
+    assert pargs["pairs"] == ["UNITTEST/BTC"]
 
 
 def test_plot_profit_options() -> None:
@@ -173,9 +172,9 @@ def test_plot_profit_options() -> None:
     ]
     pargs = Arguments(args).get_parsed_arg()
 
-    assert pargs.trade_source == "DB"
+    assert pargs["trade_source"] == "DB"
-    assert pargs.pairs == ["UNITTEST/BTC"]
+    assert pargs["pairs"] == ["UNITTEST/BTC"]
-    assert pargs.db_url == "sqlite:///whatever.sqlite"
+    assert pargs["db_url"] == "sqlite:///whatever.sqlite"
 
 
 def test_check_int_positive() -> None:

tests/test_configuration.py

@@ -143,12 +143,10 @@ def test_from_config(default_conf, mocker, caplog) -> None:
     conf2['exchange']['pair_whitelist'] += ['NANO/BTC']
     conf2['fiat_display_currency'] = "EUR"
     config_files = [conf1, conf2]
+    mocker.patch('freqtrade.configuration.configuration.create_datadir', lambda c, x: x)
 
     configsmock = MagicMock(side_effect=config_files)
-    mocker.patch(
+    mocker.patch('freqtrade.configuration.configuration.load_config_file', configsmock)
-        'freqtrade.configuration.configuration.load_config_file',
-        configsmock
-    )
 
     validated_conf = Configuration.from_files(['test_conf.json', 'test2_conf.json'])
 
@@ -161,6 +159,25 @@ def test_from_config(default_conf, mocker, caplog) -> None:
     assert validated_conf['fiat_display_currency'] == "EUR"
     assert 'internals' in validated_conf
     assert log_has('Validating configuration ...', caplog)
+    assert isinstance(validated_conf['user_data_dir'], Path)
+
+
+def test_print_config(default_conf, mocker, caplog) -> None:
+    conf1 = deepcopy(default_conf)
+    # Delete non-json elements from default_conf
+    del conf1['user_data_dir']
+    config_files = [conf1]
+
+    configsmock = MagicMock(side_effect=config_files)
+    mocker.patch('freqtrade.configuration.configuration.create_datadir', lambda c, x: x)
+    mocker.patch('freqtrade.configuration.configuration.load_config_file', configsmock)
+
+    validated_conf = Configuration.from_files(['test_conf.json'])
+
+    assert isinstance(validated_conf['user_data_dir'], Path)
+    assert "user_data_dir" in validated_conf
+    assert "original_config" in validated_conf
+    assert isinstance(json.dumps(validated_conf['original_config']), str)
 
 
 def test_load_config_max_open_trades_minus_one(default_conf, mocker, caplog) -> None:
@@ -341,14 +358,10 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
     assert 'position_stacking' not in config
     assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
 
-    assert 'refresh_pairs' not in config
-    assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
-
     assert 'timerange' not in config
     assert 'export' not in config
 
 
-@pytest.mark.filterwarnings("ignore:DEPRECATED")
 def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> None:
     patched_configuration_load_config_file(mocker, default_conf)
     mocker.patch(
@@ -368,7 +381,6 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
         '--ticker-interval', '1m',
         '--enable-position-stacking',
         '--disable-max-market-positions',
-        '--refresh-pairs-cached',
         '--timerange', ':100',
         '--export', '/bar/foo'
     ]
@@ -398,8 +410,6 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
     assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
     assert log_has('max_open_trades set to unlimited ...', caplog)
 
-    assert 'refresh_pairs'in config
-    assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
     assert 'timerange' in config
     assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
 
@@ -440,7 +450,7 @@ def test_setup_configuration_with_stratlist(mocker, default_conf, caplog) -> Non
                    caplog)
 
     assert 'strategy_list' in config
-    assert log_has('Using strategy list of 2 Strategies', caplog)
+    assert log_has('Using strategy list of 2 strategies', caplog)
 
     assert 'position_stacking' not in config
 
@@ -529,6 +539,13 @@ def test_check_exchange(default_conf, caplog) -> None:
     default_conf['runmode'] = RunMode.PLOT
     assert check_exchange(default_conf)
 
+    # Test no exchange...
+    default_conf.get('exchange').update({'name': ''})
+    default_conf['runmode'] = RunMode.OTHER
+    with pytest.raises(OperationalException,
+                       match=r'This command requires a configured exchange.*'):
+        check_exchange(default_conf)
+
 
 def test_cli_verbose_with_params(default_conf, mocker, caplog) -> None:
     patched_configuration_load_config_file(mocker, default_conf)
@@ -871,7 +888,7 @@ def test_pairlist_resolving_fallback(mocker):
 
     args = Arguments(arglist).get_parsed_arg()
     # Fix flaky tests if config.json exists
-    args.config = None
+    args["config"] = None
 
     configuration = Configuration(args)
     config = configuration.get_config()

tests/test_freqtradebot.py

@@ -1590,6 +1590,8 @@ def test_update_trade_state(mocker, default_conf, limit_buy_order, caplog) -> No
     Trade.session = MagicMock()
     trade.open_order_id = '123'
     trade.open_fee = 0.001
+    # Add datetime explicitly since sqlalchemy defaults apply only once written to database
+    trade.open_date = arrow.utcnow().datetime
     freqtrade.update_trade_state(trade)
     # Test amount not modified by fee-logic
     assert not log_has_re(r'Applying fee to .*', caplog)
@@ -1823,7 +1825,8 @@ def test_handle_trade_roi(default_conf, ticker, limit_buy_order,
     # if ROI is reached we must sell
     patch_get_signal(freqtrade, value=(False, True))
     assert freqtrade.handle_trade(trade)
-    assert log_has('Required profit reached. Selling..', caplog)
+    assert log_has("ETH/BTC - Required profit reached. sell_flag=True, sell_type=SellType.ROI",
+                   caplog)
 
 
 def test_handle_trade_experimental(
@@ -1853,7 +1856,8 @@ def test_handle_trade_experimental(
 
     patch_get_signal(freqtrade, value=(False, True))
     assert freqtrade.handle_trade(trade)
-    assert log_has('Sell signal received. Selling..', caplog)
+    assert log_has("ETH/BTC - Sell signal received. sell_flag=True, sell_type=SellType.SELL_SIGNAL",
+                   caplog)
 
 
 def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order,
@@ -2132,6 +2136,7 @@ def test_check_handle_timedout_exception(default_conf, ticker, mocker, caplog) -
     )
     freqtrade = FreqtradeBot(default_conf)
 
+    open_date = arrow.utcnow().shift(minutes=-601)
     trade_buy = Trade(
         pair='ETH/BTC',
         open_rate=0.00001099,
@@ -2141,16 +2146,18 @@ def test_check_handle_timedout_exception(default_conf, ticker, mocker, caplog) -
         fee_open=0.0,
         fee_close=0.0,
         stake_amount=1,
-        open_date=arrow.utcnow().shift(minutes=-601).datetime,
+        open_date=open_date.datetime,
         is_open=True
     )
 
     Trade.session.add(trade_buy)
 
     freqtrade.check_handle_timedout()
-    assert log_has_re(r'Cannot query order for Trade\(id=1, pair=ETH/BTC, amount=90.99181073, '
+    assert log_has_re(r"Cannot query order for Trade\(id=1, pair=ETH/BTC, amount=90.99181073, "
-                      r'open_rate=0.00001099, open_since=10 hours ago\) due to Traceback \(most '
+                      r"open_rate=0.00001099, open_since="
-                      r'recent call last\):\n.*', caplog)
+                      f"{open_date.strftime('%Y-%m-%d %H:%M:%S')}"
+                      r"\) due to Traceback \(most recent call last\):\n*",
+                      caplog)
 
 
 def test_handle_timedout_limit_buy(mocker, default_conf) -> None:
@@ -2200,9 +2207,9 @@ def test_handle_timedout_limit_sell(mocker, default_conf) -> None:
 
 def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, markets, mocker) -> None:
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2246,9 +2253,9 @@ def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, markets, moc
 
 def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, markets, mocker) -> None:
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2295,9 +2302,9 @@ def test_execute_sell_down_stoploss_on_exchange_dry_run(default_conf, ticker, fe
                                                         ticker_sell_down,
                                                         markets, mocker) -> None:
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2352,9 +2359,9 @@ def test_execute_sell_sloe_cancel_exception(mocker, default_conf, ticker, fee,
     freqtrade = get_patched_freqtradebot(mocker, default_conf)
     mocker.patch('freqtrade.exchange.Exchange.cancel_order', side_effect=InvalidOrderException())
     sellmock = MagicMock()
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets),
@@ -2383,9 +2390,9 @@ def test_execute_sell_with_stoploss_on_exchange(default_conf,
 
     default_conf['exchange']['name'] = 'binance'
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2438,9 +2445,9 @@ def test_may_execute_sell_after_stoploss_on_exchange_hit(default_conf,
                                                          markets, mocker) -> None:
     default_conf['exchange']['name'] = 'binance'
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2502,9 +2509,9 @@ def test_may_execute_sell_after_stoploss_on_exchange_hit(default_conf,
 def test_execute_sell_market_order(default_conf, ticker, fee,
                                    ticker_sell_up, markets, mocker) -> None:
     rpc_mock = patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2678,9 +2685,9 @@ def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, fee, marke
 
 def test_locked_pairs(default_conf, ticker, fee, ticker_sell_down, markets, mocker, caplog) -> None:
     patch_RPCManager(mocker)
+    patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
-        _load_markets=MagicMock(return_value={}),
         get_ticker=ticker,
         get_fee=fee,
         markets=PropertyMock(return_value=markets)
@@ -2793,8 +2800,9 @@ def test_trailing_stop_loss(default_conf, limit_buy_order, fee, markets, caplog,
     # Sell as trailing-stop is reached
     assert freqtrade.handle_trade(trade) is True
     assert log_has(
-        f'HIT STOP: current price at 0.000012, stop loss is 0.000015, '
+        f"ETH/BTC - HIT STOP: current price at 0.000012, "
-        f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog)
+        f"stoploss is 0.000015, "
+        f"initial stoploss was at 0.000010, trade opened at 0.000011", caplog)
     assert trade.sell_reason == SellType.TRAILING_STOP_LOSS.value
 
 
@@ -2836,8 +2844,8 @@ def test_trailing_stop_loss_positive(default_conf, limit_buy_order, fee, markets
                  }))
     # stop-loss not reached, adjusted stoploss
     assert freqtrade.handle_trade(trade) is False
-    assert log_has(f'using positive stop loss: 0.01 offset: 0 profit: 0.2666%', caplog)
+    assert log_has(f"ETH/BTC - Using positive stoploss: 0.01 offset: 0 profit: 0.2666%", caplog)
-    assert log_has(f'adjusted stop loss', caplog)
+    assert log_has(f"ETH/BTC - Adjusting stoploss...", caplog)
     assert trade.stop_loss == 0.0000138501
 
     mocker.patch('freqtrade.exchange.Exchange.get_ticker',
@@ -2849,9 +2857,9 @@ def test_trailing_stop_loss_positive(default_conf, limit_buy_order, fee, markets
     # Lower price again (but still positive)
     assert freqtrade.handle_trade(trade) is True
     assert log_has(
-        f'HIT STOP: current price at {buy_price + 0.000002:.6f}, '
+        f"ETH/BTC - HIT STOP: current price at {buy_price + 0.000002:.6f}, "
-        f'stop loss is {trade.stop_loss:.6f}, '
+        f"stoploss is {trade.stop_loss:.6f}, "
-        f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog)
+        f"initial stoploss was at 0.000010, trade opened at 0.000011", caplog)
 
 
 def test_trailing_stop_loss_offset(default_conf, limit_buy_order, fee,
@@ -2894,8 +2902,9 @@ def test_trailing_stop_loss_offset(default_conf, limit_buy_order, fee,
                  }))
     # stop-loss not reached, adjusted stoploss
     assert freqtrade.handle_trade(trade) is False
-    assert log_has(f'using positive stop loss: 0.01 offset: 0.011 profit: 0.2666%', caplog)
+    assert log_has(f"ETH/BTC - Using positive stoploss: 0.01 offset: 0.011 profit: 0.2666%",
-    assert log_has(f'adjusted stop loss', caplog)
+                   caplog)
+    assert log_has(f"ETH/BTC - Adjusting stoploss...", caplog)
     assert trade.stop_loss == 0.0000138501
 
     mocker.patch('freqtrade.exchange.Exchange.get_ticker',
@@ -2907,9 +2916,9 @@ def test_trailing_stop_loss_offset(default_conf, limit_buy_order, fee,
     # Lower price again (but still positive)
     assert freqtrade.handle_trade(trade) is True
     assert log_has(
-        f'HIT STOP: current price at {buy_price + 0.000002:.6f}, '
+        f"ETH/BTC - HIT STOP: current price at {buy_price + 0.000002:.6f}, "
-        f'stop loss is {trade.stop_loss:.6f}, '
+        f"stoploss is {trade.stop_loss:.6f}, "
-        f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog)
+        f"initial stoploss was at 0.000010, trade opened at 0.000011", caplog)
     assert trade.sell_reason == SellType.TRAILING_STOP_LOSS.value
 
 
@@ -2960,7 +2969,7 @@ def test_tsl_only_offset_reached(default_conf, limit_buy_order, fee,
     # stop-loss should not be adjusted as offset is not reached yet
     assert freqtrade.handle_trade(trade) is False
 
-    assert not log_has(f'adjusted stop loss', caplog)
+    assert not log_has(f"ETH/BTC - Adjusting stoploss...", caplog)
     assert trade.stop_loss == 0.0000098910
 
     # price rises above the offset (rises 12% when the offset is 5.5%)
@@ -2972,8 +2981,9 @@ def test_tsl_only_offset_reached(default_conf, limit_buy_order, fee,
                  }))
 
     assert freqtrade.handle_trade(trade) is False
-    assert log_has(f'using positive stop loss: 0.05 offset: 0.055 profit: 0.1218%', caplog)
+    assert log_has(f"ETH/BTC - Using positive stoploss: 0.05 offset: 0.055 profit: 0.1218%",
-    assert log_has(f'adjusted stop loss', caplog)
+                   caplog)
+    assert log_has(f"ETH/BTC - Adjusting stoploss...", caplog)
     assert trade.stop_loss == 0.0000117705
 
 
@@ -3342,8 +3352,8 @@ def test_order_book_bid_strategy2(mocker, default_conf, order_book_l2, markets)
     default_conf['telegram']['enabled'] = False
 
     freqtrade = FreqtradeBot(default_conf)
-    # ordrebook shall be used even if tickers would be lower.
+    # orderbook shall be used even if tickers would be lower.
-    assert freqtrade.get_target_bid('ETH/BTC', ) != 0.042
+    assert freqtrade.get_target_bid('ETH/BTC') != 0.042
     assert ticker_mock.call_count == 0
 
 

tests/test_main.py

@@ -27,11 +27,12 @@ def test_parse_args_backtesting(mocker) -> None:
         main(['backtesting'])
     assert backtesting_mock.call_count == 1
     call_args = backtesting_mock.call_args[0][0]
-    assert call_args.config == ['config.json']
+    assert call_args["config"] == ['config.json']
-    assert call_args.verbosity == 0
+    assert call_args["verbosity"] == 0
-    assert call_args.subparser == 'backtesting'
+    assert call_args["subparser"] == 'backtesting'
-    assert call_args.func is not None
+    assert call_args["func"] is not None
-    assert call_args.ticker_interval is None
+    assert callable(call_args["func"])
+    assert call_args["ticker_interval"] is None
 
 
 def test_main_start_hyperopt(mocker) -> None:
@@ -42,10 +43,11 @@ def test_main_start_hyperopt(mocker) -> None:
         main(['hyperopt'])
     assert hyperopt_mock.call_count == 1
     call_args = hyperopt_mock.call_args[0][0]
-    assert call_args.config == ['config.json']
+    assert call_args["config"] == ['config.json']
-    assert call_args.verbosity == 0
+    assert call_args["verbosity"] == 0
-    assert call_args.subparser == 'hyperopt'
+    assert call_args["subparser"] == 'hyperopt'
-    assert call_args.func is not None
+    assert call_args["func"] is not None
+    assert callable(call_args["func"])
 
 
 def test_main_fatal_exception(mocker, default_conf, caplog) -> None:

tests/test_plotting.py

@@ -344,7 +344,7 @@ def test_start_plot_profit_error(mocker):
     argsp = get_args(args)
     # Make sure we use no config. Details: #2241
     # not resetting config causes random failures if config.json exists
-    argsp.config = []
+    argsp["config"] = []
     with pytest.raises(OperationalException):
         start_plot_profit(argsp)
 

tests/test_utils.py

@@ -1,12 +1,14 @@
 import re
+from pathlib import Path
 from unittest.mock import MagicMock, PropertyMock
 
 import pytest
 
+from freqtrade import OperationalException
 from freqtrade.state import RunMode
-from tests.conftest import get_args, log_has, patch_exchange
 from freqtrade.utils import (setup_utils_configuration, start_create_userdir,
                              start_download_data, start_list_exchanges)
+from tests.conftest import get_args, log_has, patch_exchange
 
 
 def test_setup_utils_configuration():
@@ -103,3 +105,37 @@ def test_download_data_no_markets(mocker, caplog):
     start_download_data(get_args(args))
     assert dl_mock.call_args[1]['timerange'].starttype == "date"
     assert log_has("Pairs [ETH/BTC,XRP/BTC] not available on exchange binance.", caplog)
+
+
+def test_download_data_no_exchange(mocker, caplog):
+    mocker.patch('freqtrade.utils.refresh_backtest_ohlcv_data',
+                 MagicMock(return_value=["ETH/BTC", "XRP/BTC"]))
+    patch_exchange(mocker)
+    mocker.patch(
+        'freqtrade.exchange.Exchange.markets', PropertyMock(return_value={})
+    )
+    args = [
+        "download-data",
+        ]
+    with pytest.raises(OperationalException,
+                       match=r"This command requires a configured exchange.*"):
+        start_download_data(get_args(args))
+
+
+def test_download_data_no_pairs(mocker, caplog):
+    mocker.patch.object(Path, "exists", MagicMock(return_value=False))
+
+    mocker.patch('freqtrade.utils.refresh_backtest_ohlcv_data',
+                 MagicMock(return_value=["ETH/BTC", "XRP/BTC"]))
+    patch_exchange(mocker)
+    mocker.patch(
+        'freqtrade.exchange.Exchange.markets', PropertyMock(return_value={})
+    )
+    args = [
+        "download-data",
+        "--exchange",
+        "binance",
+    ]
+    with pytest.raises(OperationalException,
+                       match=r"Downloading data requires a list of pairs\..*"):
+        start_download_data(get_args(args))