Merge pull request #2067 from freqtrade/align_userdata

Align userdata usage

docs/backtesting.md

@@ -248,7 +248,7 @@ All listed Strategies need to be in the same directory.
 freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strategy-list Strategy001 Strategy002 --export trades
 ```
 
-This will save the results to `user_data/backtest_data/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
+This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
 There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
 Detailed output for all strategies one after the other will be available, so make sure to scroll up.
 

docs/bot-usage.md

@@ -9,38 +9,43 @@ This page explains the different parameters of the bot and how to run it.
 ## Bot commands
 
 ```
-usage: freqtrade [-h] [-v] [--logfile FILE] [--version] [-c PATH] [-d PATH]
+usage: freqtrade [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH]
-                 [-s NAME] [--strategy-path PATH] [--db-url PATH]
+                 [--userdir PATH] [-s NAME] [--strategy-path PATH]
-                 [--sd-notify]
+                 [--db-url PATH] [--sd-notify]
-                 {backtesting,edge,hyperopt} ...
+                 {backtesting,edge,hyperopt,create-userdir,list-exchanges} ...
 
 Free, open source crypto trading bot
 
 positional arguments:
-  {backtesting,edge,hyperopt}
+  {backtesting,edge,hyperopt,create-userdir,list-exchanges}
     backtesting         Backtesting module.
     edge                Edge module.
     hyperopt            Hyperopt module.
+    create-userdir      Create user-data directory.
+    list-exchanges      Print available exchanges.
 
 optional arguments:
   -h, --help            show this help message and exit
   -v, --verbose         Verbose mode (-vv for more, -vvv to get all messages).
-  --logfile FILE        Log to the file specified
+  --logfile FILE        Log to the file specified.
   -V, --version         show program's version number and exit
   -c PATH, --config PATH
-                        Specify configuration file (default: None). Multiple
+                        Specify configuration file (default: `config.json`).
-                        --config options may be used. Can be set to '-' to
+                        Multiple --config options may be used. Can be set to
-                        read config from stdin.
+                        `-` to read config from stdin.
   -d PATH, --datadir PATH
-                        Path to backtest data.
+                        Path to directory with historical backtesting data.
+  --userdir PATH, --user-data-dir PATH
+                        Path to userdata directory.
   -s NAME, --strategy NAME
                         Specify strategy class name (default:
-                        DefaultStrategy).
+                        `DefaultStrategy`).
   --strategy-path PATH  Specify additional strategy lookup path.
-  --db-url PATH         Override trades database URL, this is useful if
+  --db-url PATH         Override trades database URL, this is useful in custom
-                        dry_run is enabled or in custom deployments (default:
+                        deployments (default: `sqlite:///tradesv3.sqlite` for
-                        None).
+                        Live Run mode, `sqlite://` for Dry Run).
   --sd-notify           Notify systemd service manager.
+
 ```
 
 ### How to specify which configuration file be used?
@@ -85,6 +90,29 @@ of your configuration in the project issues or in the Internet.
 See more details on this technique with examples in the documentation page on
 [configuration](configuration.md).
 
+### Where to store custom data
+
+Freqtrade allows the creation of a user-data directory using `freqtrade create-userdir --userdir someDirectory`.
+This directory will look as follows:
+
+```
+user_data/
+├── backtest_results
+├── data
+├── hyperopts
+├── hyperopts_results
+├── plot
+└── strategies
+```
+
+You can add the entry "user_data_dir" setting to your configuration, to always point your bot to this directory.
+Alternatively, pass in `--userdir` to every command.
+The bot will fail to start if the directory does not exist, but will create necessary subdirectories.
+
+This directory should contain your custom strategies, custom hyperopts and hyperopt loss functions, backtesting historical data (downloaded using either backtesting command or the download script) and plot outputs.
+
+It is recommended to use version control to keep track of changes to your strategies.
+
 ### How to use **--strategy**?
 
 This parameter will allow you to load your custom strategy class.
@@ -179,8 +207,8 @@ optional arguments:
   --export-filename PATH
                         Save backtest results to this filename requires
                         --export to be set as well Example --export-
-                        filename=user_data/backtest_data/backtest_today.json
+                        filename=user_data/backtest_results/backtest_today.json
-                        (default: user_data/backtest_data/backtest-
+                        (default: user_data/backtest_results/backtest-
                         result.json)
 ```
 

docs/configuration.md

@@ -91,6 +91,7 @@ Mandatory parameters are marked as **Required**.
 | `internals.process_throttle_secs` | 5 | **Required.** Set the process throttle. Value in second.
 | `internals.sd_notify` | false | Enables use of the sd_notify protocol to tell systemd service manager about changes in the bot state and issue keep-alive pings. See [here](installation.md#7-optional-configure-freqtrade-as-a-systemd-service) for more details.
 | `logfile` | | Specify Logfile. Uses a rolling strategy of 10 files, with 1Mb per file.
+| `user_data_dir` | cwd()/user_data | Directory containing user data. Defaults to `./user_data/`.
 
 ### Parameters in the strategy
 

docs/data-analysis.md

@@ -11,7 +11,7 @@ You can analyze the results of backtests and trading history easily using Jupyte
 ```python
 from freqtrade.data.btanalysis import load_backtest_data
 # Load backtest results
-df = load_backtest_data("user_data/backtest_data/backtest-result.json")
+df = load_backtest_data("user_data/backtest_results/backtest-result.json")
 
 # Show value-counts per pair
 df.groupby("pair")["sell_reason"].value_counts()

docs/plotting.md

@@ -64,7 +64,7 @@ python3 scripts/plot_dataframe.py --db-url sqlite:///tradesv3.dry_run.sqlite -p
 To plot trades from a backtesting result, use `--export-filename <filename>`
 
 ``` bash
-python3 scripts/plot_dataframe.py --export-filename user_data/backtest_data/backtest-result.json -p BTC/ETH
+python3 scripts/plot_dataframe.py --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
 ```
 
 To plot a custom strategy the strategy should have first be backtested.

freqtrade/configuration/arguments.py

@@ -7,7 +7,7 @@ from typing import List, Optional
 from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
 from freqtrade import constants
 
-ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir"]
+ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir", "user_data_dir"]
 
 ARGS_STRATEGY = ["strategy", "strategy_path"]
 
@@ -30,6 +30,8 @@ ARGS_EDGE = ARGS_COMMON_OPTIMIZE + ["stoploss_range"]
 
 ARGS_LIST_EXCHANGES = ["print_one_column"]
 
+ARGS_CREATE_USERDIR = ["user_data_dir"]
+
 ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "exchange", "timeframes", "erase"]
 
 ARGS_PLOT_DATAFRAME = (ARGS_COMMON + ARGS_STRATEGY +
@@ -98,7 +100,7 @@ class Arguments(object):
         :return: None
         """
         from freqtrade.optimize import start_backtesting, start_hyperopt, start_edge
-        from freqtrade.utils import start_download_data, start_list_exchanges
+        from freqtrade.utils import start_create_userdir, start_download_data, start_list_exchanges
 
         subparsers = self.parser.add_subparsers(dest='subparser')
 
@@ -117,6 +119,11 @@ class Arguments(object):
         hyperopt_cmd.set_defaults(func=start_hyperopt)
         self._build_args(optionlist=ARGS_HYPEROPT, parser=hyperopt_cmd)
 
+        create_userdir_cmd = subparsers.add_parser('create-userdir',
+                                                   help="Create user-data directory.")
+        create_userdir_cmd.set_defaults(func=start_create_userdir)
+        self._build_args(optionlist=ARGS_CREATE_USERDIR, parser=create_userdir_cmd)
+
         # Add list-exchanges subcommand
         list_exchanges_cmd = subparsers.add_parser(
             'list-exchanges',

freqtrade/configuration/cli_options.py

@@ -55,7 +55,12 @@ AVAILABLE_CLI_OPTIONS = {
     ),
     "datadir": Arg(
         '-d', '--datadir',
-        help='Path to backtest data.',
+        help='Path to directory with historical backtesting data.',
+        metavar='PATH',
+    ),
+    "user_data_dir": Arg(
+        '--userdir', '--user-data-dir',
+        help='Path to userdata directory.',
         metavar='PATH',
     ),
     # Main options
@@ -146,9 +151,9 @@ AVAILABLE_CLI_OPTIONS = {
         '--export-filename',
         help='Save backtest results to the file with this filename (default: `%(default)s`). '
         'Requires `--export` to be set as well. '
-        'Example: `--export-filename=user_data/backtest_data/backtest_today.json`',
+        'Example: `--export-filename=user_data/backtest_results/backtest_today.json`',
         metavar='PATH',
-        default=os.path.join('user_data', 'backtest_data',
+        default=os.path.join('user_data', 'backtest_results',
                              'backtest-result.json'),
     ),
     # Edge

freqtrade/configuration/configuration.py

@@ -7,11 +7,12 @@ from argparse import Namespace
 from pathlib import Path
 from typing import Any, Callable, Dict, List, Optional
 
-from freqtrade import constants, OperationalException
+from freqtrade import OperationalException, constants
 from freqtrade.configuration.check_exchange import check_exchange
-from freqtrade.configuration.create_datadir import create_datadir
+from freqtrade.configuration.config_validation import (
-from freqtrade.configuration.config_validation import (validate_config_schema,
+    validate_config_consistency, validate_config_schema)
-                                                       validate_config_consistency)
+from freqtrade.configuration.directory_operations import (create_datadir,
+                                                          create_userdata_dir)
 from freqtrade.configuration.load_config import load_config_file
 from freqtrade.loggers import setup_logging
 from freqtrade.misc import deep_merge_dicts, json_load
@@ -115,7 +116,9 @@ class Configuration(object):
 
         setup_logging(config)
 
-    def _process_strategy_options(self, config: Dict[str, Any]) -> None:
+    def _process_common_options(self, config: Dict[str, Any]) -> None:
+
+        self._process_logging_options(config)
 
         # Set strategy if not specified in config and or if it's non default
         if self.args.strategy != constants.DEFAULT_STRATEGY or not config.get('strategy'):
@@ -124,11 +127,6 @@ class Configuration(object):
         self._args_to_config(config, argname='strategy_path',
                              logstring='Using additional Strategy lookup path: {}')
 
-    def _process_common_options(self, config: Dict[str, Any]) -> None:
-
-        self._process_logging_options(config)
-        self._process_strategy_options(config)
-
         if ('db_url' in self.args and self.args.db_url and
                 self.args.db_url != constants.DEFAULT_DB_PROD_URL):
             config.update({'db_url': self.args.db_url})
@@ -159,9 +157,19 @@ class Configuration(object):
 
     def _process_datadir_options(self, config: Dict[str, Any]) -> None:
         """
-        Extract information for sys.argv and load datadir configuration:
+        Extract information for sys.argv and load directory configurations
-        the --datadir option
+        --user-data, --datadir
         """
+        if 'user_data_dir' in self.args and self.args.user_data_dir:
+            config.update({'user_data_dir': self.args.user_data_dir})
+        elif 'user_data_dir' not in config:
+            # Default to cwd/user_data (legacy option ...)
+            config.update({'user_data_dir': str(Path.cwd() / "user_data")})
+
+        # reset to user_data_dir so this contains the absolute path.
+        config['user_data_dir'] = create_userdata_dir(config['user_data_dir'], create_dir=False)
+        logger.info('Using user-data directory: %s ...', config['user_data_dir'])
+
         if 'datadir' in self.args and self.args.datadir:
             config.update({'datadir': create_datadir(config, self.args.datadir)})
         else:
@@ -357,7 +365,6 @@ class Configuration(object):
         else:
             # Fall back to /dl_path/pairs.json
             pairs_file = Path(config['datadir']) / config['exchange']['name'].lower() / "pairs.json"
-            print(config['datadir'])
             if pairs_file.exists():
                 with pairs_file.open('r') as f:
                     config['pairs'] = json_load(f)

freqtrade/configuration/create_datadir.py

@@ -1,20 +0,0 @@
-import logging
-from typing import Any, Dict, Optional
-from pathlib import Path
-
-
-logger = logging.getLogger(__name__)
-
-
-def create_datadir(config: Dict[str, Any], datadir: Optional[str] = None) -> str:
-
-    folder = Path(datadir) if datadir else Path('user_data/data')
-    if not datadir:
-        # set datadir
-        exchange_name = config.get('exchange', {}).get('name').lower()
-        folder = folder.joinpath(exchange_name)
-
-    if not folder.is_dir():
-        folder.mkdir(parents=True)
-        logger.info(f'Created data directory: {datadir}')
-    return str(folder)

freqtrade/configuration/directory_operations.py

@@ -0,0 +1,50 @@
+import logging
+from typing import Any, Dict, Optional
+from pathlib import Path
+
+from freqtrade import OperationalException
+
+logger = logging.getLogger(__name__)
+
+
+def create_datadir(config: Dict[str, Any], datadir: Optional[str] = None) -> str:
+
+    folder = Path(datadir) if datadir else Path(f"{config['user_data_dir']}/data")
+    if not datadir:
+        # set datadir
+        exchange_name = config.get('exchange', {}).get('name').lower()
+        folder = folder.joinpath(exchange_name)
+
+    if not folder.is_dir():
+        folder.mkdir(parents=True)
+        logger.info(f'Created data directory: {datadir}')
+    return str(folder)
+
+
+def create_userdata_dir(directory: str, create_dir=False) -> Path:
+    """
+    Create userdata directory structure.
+    if create_dir is True, then the parent-directory will be created if it does not exist.
+    Sub-directories will always be created if the parent directory exists.
+    Raises OperationalException if given a non-existing directory.
+    :param directory: Directory to check
+    :param create_dir: Create directory if it does not exist.
+    :return: Path object containing the directory
+    """
+    sub_dirs = ["backtest_results", "data", "hyperopts", "hyperopt_results", "plot", "strategies", ]
+    folder = Path(directory)
+    if not folder.is_dir():
+        if create_dir:
+            folder.mkdir(parents=True)
+            logger.info(f'Created user-data directory: {folder}')
+        else:
+            raise OperationalException(
+                f"Directory `{folder}` does not exist. "
+                "Please use `freqtrade create-userdir` to create a user directory")
+
+    # Create required subdirectories
+    for f in sub_dirs:
+        subfolder = folder / f
+        if not subfolder.is_dir():
+            subfolder.mkdir(parents=False)
+    return folder

freqtrade/optimize/__init__.py

@@ -64,14 +64,14 @@ def start_hyperopt(args: Namespace) -> None:
     :return: None
     """
     # Import here to avoid loading hyperopt module when it's not used
-    from freqtrade.optimize.hyperopt import Hyperopt, HYPEROPT_LOCKFILE
+    from freqtrade.optimize.hyperopt import Hyperopt
 
     # Initialize configuration
     config = setup_configuration(args, RunMode.HYPEROPT)
 
     logger.info('Starting freqtrade in Hyperopt mode')
 
-    lock = FileLock(HYPEROPT_LOCKFILE)
+    lock = FileLock(Hyperopt.get_lock_filename(config))
 
     try:
         with lock.acquire(timeout=1):

freqtrade/optimize/hyperopt.py

@@ -5,7 +5,6 @@ This module contains the hyperopt logic
 """
 
 import logging
-import os
 import sys
 
 from collections import OrderedDict
@@ -36,9 +35,6 @@ logger = logging.getLogger(__name__)
 
 INITIAL_POINTS = 30
 MAX_LOSS = 100000  # just a big enough number to be bad result in loss optimization
-TICKERDATA_PICKLE = os.path.join('user_data', 'hyperopt_tickerdata.pkl')
-TRIALSDATA_PICKLE = os.path.join('user_data', 'hyperopt_results.pickle')
-HYPEROPT_LOCKFILE = os.path.join('user_data', 'hyperopt.lock')
 
 
 class Hyperopt(Backtesting):
@@ -56,7 +52,12 @@ class Hyperopt(Backtesting):
         self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
         self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
 
+        self.trials_file = (self.config['user_data_dir'] /
+                            'hyperopt_results' / 'hyperopt_results.pickle')
+        self.tickerdata_pickle = (self.config['user_data_dir'] /
+                                  'hyperopt_results' / 'hyperopt_tickerdata.pkl')
         self.total_epochs = config.get('epochs', 0)
+
         self.current_best_loss = 100
 
         if not self.config.get('hyperopt_continue'):
@@ -65,7 +66,6 @@ class Hyperopt(Backtesting):
             logger.info("Continuing on previous hyperopt results.")
 
         # Previous evaluations
-        self.trials_file = TRIALSDATA_PICKLE
         self.trials: List = []
 
         # Populate functions here (hasattr is slow so should not be run during "regular" operations)
@@ -89,11 +89,16 @@ class Hyperopt(Backtesting):
                 self.config['experimental'] = {}
             self.config['experimental']['use_sell_signal'] = True
 
+    @staticmethod
+    def get_lock_filename(config) -> str:
+
+        return str(config['user_data_dir'] / 'hyperopt.lock')
+
     def clean_hyperopt(self):
         """
         Remove hyperopt pickle files to restart hyperopt.
         """
-        for f in [TICKERDATA_PICKLE, TRIALSDATA_PICKLE]:
+        for f in [self.tickerdata_pickle, self.trials_file]:
             p = Path(f)
             if p.is_file():
                 logger.info(f"Removing `{p}`.")
@@ -126,7 +131,7 @@ class Hyperopt(Backtesting):
         """
         logger.info('Reading Trials from \'%s\'', self.trials_file)
         trials = load(self.trials_file)
-        os.remove(self.trials_file)
+        self.trials_file.unlink()
         return trials
 
     def log_trials_result(self) -> None:
@@ -255,7 +260,7 @@ class Hyperopt(Backtesting):
         if self.has_space('stoploss'):
             self.strategy.stoploss = params['stoploss']
 
-        processed = load(TICKERDATA_PICKLE)
+        processed = load(self.tickerdata_pickle)
 
         min_date, max_date = get_timeframe(processed)
 
@@ -327,7 +332,7 @@ class Hyperopt(Backtesting):
 
     def load_previous_results(self):
         """ read trials file if we have one """
-        if os.path.exists(self.trials_file) and os.path.getsize(self.trials_file) > 0:
+        if self.trials_file.is_file() and self.trials_file.stat().st_size > 0:
             self.trials = self.read_trials()
             logger.info(
                 'Loaded %d previous evaluations from disk.',
@@ -364,7 +369,7 @@ class Hyperopt(Backtesting):
 
         preprocessed = self.strategy.tickerdata_to_dataframe(data)
 
-        dump(preprocessed, TICKERDATA_PICKLE)
+        dump(preprocessed, self.tickerdata_pickle)
 
         # We don't need exchange instance anymore while running hyperopt
         self.exchange = None  # type: ignore

freqtrade/plot/plotting.py

@@ -308,7 +308,7 @@ def generate_plot_filename(pair, ticker_interval) -> str:
     return file_name
 
 
-def store_plot_file(fig, filename: str, auto_open: bool = False) -> None:
+def store_plot_file(fig, filename: str, directory: Path, auto_open: bool = False) -> None:
     """
     Generate a plot html file from pre populated fig plotly object
     :param fig: Plotly Figure to plot
@@ -316,9 +316,9 @@ def store_plot_file(fig, filename: str, auto_open: bool = False) -> None:
     :param ticker_interval: Used as part of the filename
     :return: None
     """
+    directory.mkdir(parents=True, exist_ok=True)
 
-    Path("user_data/plots").mkdir(parents=True, exist_ok=True)
+    _filename = directory.joinpath(filename)
-    _filename = Path('user_data/plots').joinpath(filename)
     plot(fig, filename=str(_filename),
          auto_open=auto_open)
     logger.info(f"Stored plot as {_filename}")

freqtrade/resolvers/hyperopt_resolver.py

@@ -31,7 +31,8 @@ class HyperOptResolver(IResolver):
 
         # Verify the hyperopt is in the configuration, otherwise fallback to the default hyperopt
         hyperopt_name = config.get('hyperopt') or DEFAULT_HYPEROPT
-        self.hyperopt = self._load_hyperopt(hyperopt_name, extra_dir=config.get('hyperopt_path'))
+        self.hyperopt = self._load_hyperopt(hyperopt_name, config,
+                                            extra_dir=config.get('hyperopt_path'))
 
         # Assign ticker_interval to be used in hyperopt
         self.hyperopt.__class__.ticker_interval = str(config['ticker_interval'])
@@ -44,17 +45,18 @@ class HyperOptResolver(IResolver):
                            "Using populate_sell_trend from DefaultStrategy.")
 
     def _load_hyperopt(
-            self, hyperopt_name: str, extra_dir: Optional[str] = None) -> IHyperOpt:
+            self, hyperopt_name: str, config: Dict, extra_dir: Optional[str] = None) -> IHyperOpt:
         """
         Search and loads the specified hyperopt.
         :param hyperopt_name: name of the module to import
+        :param config: configuration dictionary
         :param extra_dir: additional directory to search for the given hyperopt
         :return: HyperOpt instance or None
         """
         current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
 
         abs_paths = [
-            Path.cwd().joinpath('user_data/hyperopts'),
+            config['user_data_dir'].joinpath('hyperopts'),
             current_path,
         ]
 
@@ -79,7 +81,7 @@ class HyperOptLossResolver(IResolver):
 
     __slots__ = ['hyperoptloss']
 
-    def __init__(self, config: Optional[Dict] = None) -> None:
+    def __init__(self, config: Dict = None) -> None:
         """
         Load the custom class from config parameter
         :param config: configuration dictionary or None
@@ -89,7 +91,7 @@ class HyperOptLossResolver(IResolver):
         # Verify the hyperopt is in the configuration, otherwise fallback to the default hyperopt
         hyperopt_name = config.get('hyperopt_loss') or DEFAULT_HYPEROPT_LOSS
         self.hyperoptloss = self._load_hyperoptloss(
-            hyperopt_name, extra_dir=config.get('hyperopt_path'))
+            hyperopt_name, config, extra_dir=config.get('hyperopt_path'))
 
         # Assign ticker_interval to be used in hyperopt
         self.hyperoptloss.__class__.ticker_interval = str(config['ticker_interval'])
@@ -99,17 +101,19 @@ class HyperOptLossResolver(IResolver):
                 f"Found hyperopt {hyperopt_name} does not implement `hyperopt_loss_function`.")
 
     def _load_hyperoptloss(
-            self, hyper_loss_name: str, extra_dir: Optional[str] = None) -> IHyperOptLoss:
+            self, hyper_loss_name: str, config: Dict,
+            extra_dir: Optional[str] = None) -> IHyperOptLoss:
         """
         Search and loads the specified hyperopt loss class.
         :param hyper_loss_name: name of the module to import
+        :param config: configuration dictionary
         :param extra_dir: additional directory to search for the given hyperopt
         :return: HyperOptLoss instance or None
         """
         current_path = Path(__file__).parent.parent.joinpath('optimize').resolve()
 
         abs_paths = [
-            Path.cwd().joinpath('user_data/hyperopts'),
+            config['user_data_dir'].joinpath('hyperopts'),
             current_path,
         ]
 

freqtrade/resolvers/iresolver.py

@@ -58,7 +58,7 @@ class IResolver(object):
             if not str(entry).endswith('.py'):
                 logger.debug('Ignoring %s', entry)
                 continue
-            module_path = Path.resolve(directory.joinpath(entry))
+            module_path = entry.resolve()
             obj = IResolver._get_valid_object(
                 object_type, module_path, object_name
             )

freqtrade/resolvers/pairlist_resolver.py

@@ -25,21 +25,22 @@ class PairListResolver(IResolver):
         Load the custom class from config parameter
         :param config: configuration dictionary or None
         """
-        self.pairlist = self._load_pairlist(pairlist_name, kwargs={'freqtrade': freqtrade,
+        self.pairlist = self._load_pairlist(pairlist_name, config, kwargs={'freqtrade': freqtrade,
                                                                            'config': config})
 
     def _load_pairlist(
-            self, pairlist_name: str, kwargs: dict) -> IPairList:
+            self, pairlist_name: str, config: dict, kwargs: dict) -> IPairList:
         """
         Search and loads the specified pairlist.
         :param pairlist_name: name of the module to import
+        :param config: configuration dictionary
         :param extra_dir: additional directory to search for the given pairlist
         :return: PairList instance or None
         """
         current_path = Path(__file__).parent.parent.joinpath('pairlist').resolve()
 
         abs_paths = [
-            Path.cwd().joinpath('user_data/pairlist'),
+            config['user_data_dir'].joinpath('pairlist'),
             current_path,
         ]
 

freqtrade/resolvers/strategy_resolver.py

@@ -123,7 +123,7 @@ class StrategyResolver(IResolver):
         current_path = Path(__file__).parent.parent.joinpath('strategy').resolve()
 
         abs_paths = [
-            Path.cwd().joinpath('user_data/strategies'),
+            config['user_data_dir'].joinpath('strategies'),
             current_path,
         ]
 

freqtrade/tests/conftest.py

@@ -239,6 +239,7 @@ def default_conf():
         },
         "initial_state": "running",
         "db_url": "sqlite://",
+        "user_data_dir": Path("user_data"),
         "verbosity": 3,
     }
     return configuration

freqtrade/tests/optimize/test_hyperopt.py

@@ -1,12 +1,13 @@
 # pragma pylint: disable=missing-docstring,W0212,C0103
 import os
 from datetime import datetime
-from unittest.mock import MagicMock
+from unittest.mock import MagicMock, PropertyMock
 
 import pandas as pd
 import pytest
 from arrow import Arrow
 from filelock import Timeout
+from pathlib import Path
 
 from freqtrade import DependencyException, OperationalException
 from freqtrade.data.converter import parse_ticker_dataframe
@@ -14,8 +15,7 @@ from freqtrade.data.history import load_tickerdata_file
 from freqtrade.optimize import setup_configuration, start_hyperopt
 from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
 from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss
-from freqtrade.optimize.hyperopt import (HYPEROPT_LOCKFILE, TICKERDATA_PICKLE,
+from freqtrade.optimize.hyperopt import Hyperopt
-                                         Hyperopt)
 from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
 from freqtrade.state import RunMode
 from freqtrade.strategy.interface import SellType
@@ -54,11 +54,14 @@ def create_trials(mocker, hyperopt) -> None:
       - we might have a pickle'd file so make sure that we return
         false when looking for it
     """
-    hyperopt.trials_file = os.path.join('freqtrade', 'tests', 'optimize', 'ut_trials.pickle')
+    hyperopt.trials_file = Path('freqtrade/tests/optimize/ut_trials.pickle')
 
-    mocker.patch('freqtrade.optimize.hyperopt.os.path.exists', return_value=False)
+    mocker.patch.object(Path, "is_file", MagicMock(return_value=False))
-    mocker.patch('freqtrade.optimize.hyperopt.os.path.getsize', return_value=1)
+    stat_mock = MagicMock()
-    mocker.patch('freqtrade.optimize.hyperopt.os.remove', return_value=True)
+    stat_mock.st_size = PropertyMock(return_value=1)
+    mocker.patch.object(Path, "stat", MagicMock(return_value=False))
+
+    mocker.patch.object(Path, "unlink", MagicMock(return_value=True))
     mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None)
 
     return [{'loss': 1, 'result': 'foo', 'params': {}}]
@@ -264,7 +267,7 @@ def test_start_failure(mocker, default_conf, caplog) -> None:
 
 
 def test_start_filelock(mocker, default_conf, caplog) -> None:
-    start_mock = MagicMock(side_effect=Timeout(HYPEROPT_LOCKFILE))
+    start_mock = MagicMock(side_effect=Timeout(Hyperopt.get_lock_filename(default_conf)))
     patched_configuration_load_config_file(mocker, default_conf)
     mocker.patch('freqtrade.optimize.hyperopt.Hyperopt.start', start_mock)
     patch_exchange(mocker)
@@ -597,10 +600,10 @@ def test_clean_hyperopt(mocker, default_conf, caplog):
                          })
     mocker.patch("freqtrade.optimize.hyperopt.Path.is_file", MagicMock(return_value=True))
     unlinkmock = mocker.patch("freqtrade.optimize.hyperopt.Path.unlink", MagicMock())
-    Hyperopt(default_conf)
+    h = Hyperopt(default_conf)
 
     assert unlinkmock.call_count == 2
-    assert log_has(f"Removing `{TICKERDATA_PICKLE}`.", caplog)
+    assert log_has(f"Removing `{h.tickerdata_pickle}`.", caplog)
 
 
 def test_continue_hyperopt(mocker, default_conf, caplog):

freqtrade/tests/strategy/test_strategy.py

@@ -60,62 +60,65 @@ def test_search_strategy():
     assert s is None
 
 
-def test_load_strategy(result):
+def test_load_strategy(default_conf, result):
-    resolver = StrategyResolver({'strategy': 'TestStrategy'})
+    default_conf.update({'strategy': 'TestStrategy'})
+    resolver = StrategyResolver(default_conf)
     assert 'adx' in resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
 
 
-def test_load_strategy_base64(result, caplog):
+def test_load_strategy_base64(result, caplog, default_conf):
     with open("user_data/strategies/test_strategy.py", "rb") as file:
         encoded_string = urlsafe_b64encode(file.read()).decode("utf-8")
-    resolver = StrategyResolver({'strategy': 'TestStrategy:{}'.format(encoded_string)})
+    default_conf.update({'strategy': 'TestStrategy:{}'.format(encoded_string)})
+
+    resolver = StrategyResolver(default_conf)
     assert 'adx' in resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
     # Make sure strategy was loaded from base64 (using temp directory)!!
     assert log_has_re(r"Using resolved strategy TestStrategy from '"
                       + tempfile.gettempdir() + r"/.*/TestStrategy\.py'\.\.\.", caplog)
 
 
-def test_load_strategy_invalid_directory(result, caplog):
+def test_load_strategy_invalid_directory(result, caplog, default_conf):
-    resolver = StrategyResolver()
+    resolver = StrategyResolver(default_conf)
     extra_dir = Path.cwd() / 'some/path'
-    resolver._load_strategy('TestStrategy', config={}, extra_dir=extra_dir)
+    resolver._load_strategy('TestStrategy', config=default_conf, extra_dir=extra_dir)
 
     assert log_has_re(r'Path .*' + r'some.*path.*' + r'.* does not exist', caplog)
 
     assert 'adx' in resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
 
 
-def test_load_not_found_strategy():
+def test_load_not_found_strategy(default_conf):
-    strategy = StrategyResolver()
+    strategy = StrategyResolver(default_conf)
     with pytest.raises(OperationalException,
                        match=r"Impossible to load Strategy 'NotFoundStrategy'. "
                              r"This class does not exist or contains Python code errors."):
-        strategy._load_strategy(strategy_name='NotFoundStrategy', config={})
+        strategy._load_strategy(strategy_name='NotFoundStrategy', config=default_conf)
 
 
-def test_load_staticmethod_importerror(mocker, caplog):
+def test_load_staticmethod_importerror(mocker, caplog, default_conf):
     mocker.patch("freqtrade.resolvers.strategy_resolver.import_strategy", Mock(
         side_effect=TypeError("can't pickle staticmethod objects")))
     with pytest.raises(OperationalException,
                        match=r"Impossible to load Strategy 'DefaultStrategy'. "
                              r"This class does not exist or contains Python code errors."):
-        StrategyResolver()
+        StrategyResolver(default_conf)
     assert log_has_re(r".*Error: can't pickle staticmethod objects", caplog)
 
 
-def test_strategy(result):
+def test_strategy(result, default_conf):
-    config = {'strategy': 'DefaultStrategy'}
+    default_conf.update({'strategy': 'DefaultStrategy'})
 
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
     metadata = {'pair': 'ETH/BTC'}
     assert resolver.strategy.minimal_roi[0] == 0.04
-    assert config["minimal_roi"]['0'] == 0.04
+    assert default_conf["minimal_roi"]['0'] == 0.04
 
     assert resolver.strategy.stoploss == -0.10
-    assert config['stoploss'] == -0.10
+    assert default_conf['stoploss'] == -0.10
 
     assert resolver.strategy.ticker_interval == '5m'
-    assert config['ticker_interval'] == '5m'
+    assert default_conf['ticker_interval'] == '5m'
 
     df_indicators = resolver.strategy.advise_indicators(result, metadata=metadata)
     assert 'adx' in df_indicators
@@ -127,54 +130,54 @@ def test_strategy(result):
     assert 'sell' in dataframe.columns
 
 
-def test_strategy_override_minimal_roi(caplog):
+def test_strategy_override_minimal_roi(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'minimal_roi': {
             "0": 0.5
         }
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.minimal_roi[0] == 0.5
     assert log_has("Override strategy 'minimal_roi' with value in config file: {'0': 0.5}.", caplog)
 
 
-def test_strategy_override_stoploss(caplog):
+def test_strategy_override_stoploss(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'stoploss': -0.5
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.stoploss == -0.5
     assert log_has("Override strategy 'stoploss' with value in config file: -0.5.", caplog)
 
 
-def test_strategy_override_trailing_stop(caplog):
+def test_strategy_override_trailing_stop(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'trailing_stop': True
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.trailing_stop
     assert isinstance(resolver.strategy.trailing_stop, bool)
     assert log_has("Override strategy 'trailing_stop' with value in config file: True.", caplog)
 
 
-def test_strategy_override_trailing_stop_positive(caplog):
+def test_strategy_override_trailing_stop_positive(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'trailing_stop_positive': -0.1,
         'trailing_stop_positive_offset': -0.2
 
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.trailing_stop_positive == -0.1
     assert log_has("Override strategy 'trailing_stop_positive' with value in config file: -0.1.",
@@ -185,15 +188,15 @@ def test_strategy_override_trailing_stop_positive(caplog):
                    caplog)
 
 
-def test_strategy_override_ticker_interval(caplog):
+def test_strategy_override_ticker_interval(caplog, default_conf):
     caplog.set_level(logging.INFO)
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'ticker_interval': 60,
         'stake_currency': 'ETH'
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.ticker_interval == 60
     assert resolver.strategy.stake_currency == 'ETH'
@@ -201,21 +204,21 @@ def test_strategy_override_ticker_interval(caplog):
                    caplog)
 
 
-def test_strategy_override_process_only_new_candles(caplog):
+def test_strategy_override_process_only_new_candles(caplog, default_conf):
     caplog.set_level(logging.INFO)
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'process_only_new_candles': True
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.process_only_new_candles
     assert log_has("Override strategy 'process_only_new_candles' with value in config file: True.",
                    caplog)
 
 
-def test_strategy_override_order_types(caplog):
+def test_strategy_override_order_types(caplog, default_conf):
     caplog.set_level(logging.INFO)
 
     order_types = {
@@ -224,12 +227,11 @@ def test_strategy_override_order_types(caplog):
         'stoploss': 'limit',
         'stoploss_on_exchange': True,
     }
-
+    default_conf.update({
-    config = {
         'strategy': 'DefaultStrategy',
         'order_types': order_types
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.order_types
     for method in ['buy', 'sell', 'stoploss', 'stoploss_on_exchange']:
@@ -239,18 +241,18 @@ def test_strategy_override_order_types(caplog):
                    " {'buy': 'market', 'sell': 'limit', 'stoploss': 'limit',"
                    " 'stoploss_on_exchange': True}.", caplog)
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'order_types': {'buy': 'market'}
-    }
+    })
     # Raise error for invalid configuration
     with pytest.raises(ImportError,
                        match=r"Impossible to load Strategy 'DefaultStrategy'. "
                              r"Order-types mapping is incomplete."):
-        StrategyResolver(config)
+        StrategyResolver(default_conf)
 
 
-def test_strategy_override_order_tif(caplog):
+def test_strategy_override_order_tif(caplog, default_conf):
     caplog.set_level(logging.INFO)
 
     order_time_in_force = {
@@ -258,11 +260,11 @@ def test_strategy_override_order_tif(caplog):
         'sell': 'gtc',
     }
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'order_time_in_force': order_time_in_force
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.order_time_in_force
     for method in ['buy', 'sell']:
@@ -271,61 +273,61 @@ def test_strategy_override_order_tif(caplog):
     assert log_has("Override strategy 'order_time_in_force' with value in config file:"
                    " {'buy': 'fok', 'sell': 'gtc'}.", caplog)
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'order_time_in_force': {'buy': 'fok'}
-    }
+    })
     # Raise error for invalid configuration
     with pytest.raises(ImportError,
                        match=r"Impossible to load Strategy 'DefaultStrategy'. "
                              r"Order-time-in-force mapping is incomplete."):
-        StrategyResolver(config)
+        StrategyResolver(default_conf)
 
 
-def test_strategy_override_use_sell_signal(caplog):
+def test_strategy_override_use_sell_signal(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
     assert not resolver.strategy.use_sell_signal
     assert isinstance(resolver.strategy.use_sell_signal, bool)
     # must be inserted to configuration
-    assert 'use_sell_signal' in config['experimental']
+    assert 'use_sell_signal' in default_conf['experimental']
-    assert not config['experimental']['use_sell_signal']
+    assert not default_conf['experimental']['use_sell_signal']
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'experimental': {
             'use_sell_signal': True,
         },
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.use_sell_signal
     assert isinstance(resolver.strategy.use_sell_signal, bool)
     assert log_has("Override strategy 'use_sell_signal' with value in config file: True.", caplog)
 
 
-def test_strategy_override_use_sell_profit_only(caplog):
+def test_strategy_override_use_sell_profit_only(caplog, default_conf):
     caplog.set_level(logging.INFO)
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
     assert not resolver.strategy.sell_profit_only
     assert isinstance(resolver.strategy.sell_profit_only, bool)
     # must be inserted to configuration
-    assert 'sell_profit_only' in config['experimental']
+    assert 'sell_profit_only' in default_conf['experimental']
-    assert not config['experimental']['sell_profit_only']
+    assert not default_conf['experimental']['sell_profit_only']
 
-    config = {
+    default_conf.update({
         'strategy': 'DefaultStrategy',
         'experimental': {
             'sell_profit_only': True,
         },
-    }
+    })
-    resolver = StrategyResolver(config)
+    resolver = StrategyResolver(default_conf)
 
     assert resolver.strategy.sell_profit_only
     assert isinstance(resolver.strategy.sell_profit_only, bool)
@@ -333,10 +335,11 @@ def test_strategy_override_use_sell_profit_only(caplog):
 
 
 @pytest.mark.filterwarnings("ignore:deprecated")
-def test_deprecate_populate_indicators(result):
+def test_deprecate_populate_indicators(result, default_conf):
     default_location = path.join(path.dirname(path.realpath(__file__)))
-    resolver = StrategyResolver({'strategy': 'TestStrategyLegacy',
+    default_conf.update({'strategy': 'TestStrategyLegacy',
                          'strategy_path': default_location})
+    resolver = StrategyResolver(default_conf)
     with warnings.catch_warnings(record=True) as w:
         # Cause all warnings to always be triggered.
         warnings.simplefilter("always")
@@ -366,10 +369,11 @@ def test_deprecate_populate_indicators(result):
 
 
 @pytest.mark.filterwarnings("ignore:deprecated")
-def test_call_deprecated_function(result, monkeypatch):
+def test_call_deprecated_function(result, monkeypatch, default_conf):
     default_location = path.join(path.dirname(path.realpath(__file__)))
-    resolver = StrategyResolver({'strategy': 'TestStrategyLegacy',
+    default_conf.update({'strategy': 'TestStrategyLegacy',
                          'strategy_path': default_location})
+    resolver = StrategyResolver(default_conf)
     metadata = {'pair': 'ETH/BTC'}
 
     # Make sure we are using a legacy function

freqtrade/tests/test_configuration.py

@@ -13,7 +13,8 @@ from freqtrade import OperationalException, constants
 from freqtrade.configuration import Arguments, Configuration, validate_config_consistency
 from freqtrade.configuration.check_exchange import check_exchange
 from freqtrade.configuration.config_validation import validate_config_schema
-from freqtrade.configuration.create_datadir import create_datadir
+from freqtrade.configuration.directory_operations import (create_datadir,
+                                                          create_userdata_dir)
 from freqtrade.configuration.load_config import load_config_file
 from freqtrade.constants import DEFAULT_DB_DRYRUN_URL, DEFAULT_DB_PROD_URL
 from freqtrade.loggers import _set_loggers
@@ -52,6 +53,7 @@ def test_load_config_incorrect_stake_amount(default_conf) -> None:
 
 
 def test_load_config_file(default_conf, mocker, caplog) -> None:
+    del default_conf['user_data_dir']
     file_mock = mocker.patch('freqtrade.configuration.load_config.open', mocker.mock_open(
         read_data=json.dumps(default_conf)
     ))
@@ -331,6 +333,7 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
     assert 'exchange' in config
     assert 'pair_whitelist' in config['exchange']
     assert 'datadir' in config
+    assert 'user_data_dir' in config
     assert log_has('Using data directory: {} ...'.format(config['datadir']), caplog)
     assert 'ticker_interval' in config
     assert not log_has('Parameter -i/--ticker-interval detected ...', caplog)
@@ -355,11 +358,15 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
         'freqtrade.configuration.configuration.create_datadir',
         lambda c, x: x
     )
-
+    mocker.patch(
+        'freqtrade.configuration.configuration.create_userdata_dir',
+        lambda x, *args, **kwargs: Path(x)
+    )
     arglist = [
         '--config', 'config.json',
         '--strategy', 'DefaultStrategy',
         '--datadir', '/foo/bar',
+        '--userdir', "/tmp/freqtrade",
         'backtesting',
         '--ticker-interval', '1m',
         '--live',
@@ -380,7 +387,10 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
     assert 'exchange' in config
     assert 'pair_whitelist' in config['exchange']
     assert 'datadir' in config
-    assert log_has('Using data directory: {} ...'.format(config['datadir']), caplog)
+    assert log_has('Using data directory: {} ...'.format("/foo/bar"), caplog)
+    assert log_has('Using user-data directory: {} ...'.format("/tmp/freqtrade"), caplog)
+    assert 'user_data_dir' in config
+
     assert 'ticker_interval' in config
     assert log_has('Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
                    caplog)
@@ -615,6 +625,35 @@ def test_create_datadir(mocker, default_conf, caplog) -> None:
     assert log_has('Created data directory: /foo/bar', caplog)
 
 
+def test_create_userdata_dir(mocker, default_conf, caplog) -> None:
+    mocker.patch.object(Path, "is_dir", MagicMock(return_value=False))
+    md = mocker.patch.object(Path, 'mkdir', MagicMock())
+
+    x = create_userdata_dir('/tmp/bar', create_dir=True)
+    assert md.call_count == 7
+    assert md.call_args[1]['parents'] is False
+    assert log_has('Created user-data directory: /tmp/bar', caplog)
+    assert isinstance(x, Path)
+    assert str(x) == "/tmp/bar"
+
+
+def test_create_userdata_dir_exists(mocker, default_conf, caplog) -> None:
+    mocker.patch.object(Path, "is_dir", MagicMock(return_value=True))
+    md = mocker.patch.object(Path, 'mkdir', MagicMock())
+
+    create_userdata_dir('/tmp/bar')
+    assert md.call_count == 0
+
+
+def test_create_userdata_dir_exists_exception(mocker, default_conf, caplog) -> None:
+    mocker.patch.object(Path, "is_dir", MagicMock(return_value=False))
+    md = mocker.patch.object(Path, 'mkdir', MagicMock())
+
+    with pytest.raises(OperationalException, match=r'Directory `/tmp/bar` does not exist.*'):
+        create_userdata_dir('/tmp/bar',  create_dir=False)
+    assert md.call_count == 0
+
+
 def test_validate_tsl(default_conf):
     default_conf['trailing_stop'] = True
     default_conf['trailing_stop_positive'] = 0

freqtrade/tests/test_plotting.py

@@ -1,5 +1,6 @@
 
 from copy import deepcopy
+from pathlib import Path
 from unittest.mock import MagicMock
 
 import plotly.graph_objects as go
@@ -209,7 +210,8 @@ def test_generate_Plot_filename():
 def test_generate_plot_file(mocker, caplog):
     fig = generage_empty_figure()
     plot_mock = mocker.patch("freqtrade.plot.plotting.plot", MagicMock())
-    store_plot_file(fig, filename="freqtrade-plot-UNITTEST_BTC-5m.html")
+    store_plot_file(fig, filename="freqtrade-plot-UNITTEST_BTC-5m.html",
+                    directory=Path("user_data/plots"))
 
     assert plot_mock.call_count == 1
     assert plot_mock.call_args[0][0] == fig

freqtrade/tests/test_utils.py

@@ -6,8 +6,8 @@ import pytest
 
 from freqtrade.state import RunMode
 from freqtrade.tests.conftest import get_args, log_has, patch_exchange
-from freqtrade.utils import (setup_utils_configuration, start_download_data,
+from freqtrade.utils import (setup_utils_configuration, start_create_userdir,
-                             start_list_exchanges)
+                             start_download_data, start_list_exchanges)
 
 
 def test_setup_utils_configuration():
@@ -47,6 +47,29 @@ def test_list_exchanges(capsys):
     assert re.search(r"^bittrex$", captured.out, re.MULTILINE)
 
 
+def test_create_datadir_failed(caplog):
+
+    args = [
+        "create-userdir",
+    ]
+    with pytest.raises(SystemExit):
+        start_create_userdir(get_args(args))
+    assert log_has("`create-userdir` requires --userdir to be set.", caplog)
+
+
+def test_create_datadir(caplog, mocker):
+    cud = mocker.patch("freqtrade.utils.create_userdata_dir", MagicMock())
+    args = [
+        "create-userdir",
+        "--userdir",
+        "/temp/freqtrade/test"
+    ]
+    start_create_userdir(get_args(args))
+
+    assert cud.call_count == 1
+    assert len(caplog.record_tuples) == 0
+
+
 def test_download_data(mocker, markets, caplog):
     dl_mock = mocker.patch('freqtrade.utils.download_pair_history', MagicMock())
     patch_exchange(mocker)

freqtrade/utils.py

@@ -7,6 +7,7 @@ from typing import Any, Dict
 import arrow
 
 from freqtrade.configuration import Configuration, TimeRange
+from freqtrade.configuration.directory_operations import create_userdata_dir
 from freqtrade.data.history import download_pair_history
 from freqtrade.exchange import available_exchanges
 from freqtrade.resolvers import ExchangeResolver
@@ -46,6 +47,19 @@ def start_list_exchanges(args: Namespace) -> None:
               f"{', '.join(available_exchanges())}")
 
 
+def start_create_userdir(args: Namespace) -> None:
+    """
+    Create "user_data" directory to contain user data strategies, hyperopts, ...)
+    :param args: Cli args from Arguments()
+    :return: None
+    """
+    if "user_data_dir" in args and args.user_data_dir:
+        create_userdata_dir(args.user_data_dir, create_dir=True)
+    else:
+        logger.warning("`create-userdir` requires --userdir to be set.")
+        sys.exit(1)
+
+
 def start_download_data(args: Namespace) -> None:
     """
     Download data (former download_backtest_data.py script)

scripts/plot_dataframe.py

@@ -63,7 +63,8 @@ def analyse_and_plot_pairs(config: Dict[str, Any]):
             indicators2=config["indicators2"].split(",")
         )
 
-        store_plot_file(fig, generate_plot_filename(pair, config['ticker_interval']))
+        store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
+                        directory=config['user_data_dir'] / "plot")
 
     logger.info('End of ploting process %s plots generated', pair_counter)
 

scripts/plot_profit.py

@@ -32,7 +32,8 @@ def plot_profit(config: Dict[str, Any]) -> None:
     # Create an average close price of all the pairs that were involved.
     # this could be useful to gauge the overall market trend
     fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
-    store_plot_file(fig, filename='freqtrade-profit-plot.html', auto_open=True)
+    store_plot_file(fig, filename='freqtrade-profit-plot.html',
+                    directory=config['user_data_dir'] / "plot", auto_open=True)
 
 
 def plot_parse_args(args: List[str]) -> Dict[str, Any]: