Merge pull request #2176 from freqtrade/plot_commands

Move Plot scripts to freqtrade subcommands

docs/plotting.md

@@ -2,9 +2,9 @@
 
 This page explains how to plot prices, indicators and profits.
 
-## Installation
+## Installation / Setup
 
-Plotting scripts use Plotly library. Install/upgrade it with:
+Plotting modules use the Plotly library. You can install / upgrade this by running the following command:
 
 ``` bash
 pip install -U -r requirements-plot.txt
@@ -12,90 +12,172 @@ pip install -U -r requirements-plot.txt
 
 ## Plot price and indicators
 
-Usage for the price plotter:
+The `freqtrade plot-dataframe` subcommand shows an interactive graph with three subplots:
+
+* Main plot with candlestics and indicators following price (sma/ema)
+* Volume bars
+* Additional indicators as specified by `--indicators2`
+
+![plot-dataframe](assets/plot-dataframe.png)
+
+Possible arguments:
+
+```
+usage: freqtrade plot-dataframe [-h] [-p PAIRS [PAIRS ...]]
+                                [--indicators1 INDICATORS1 [INDICATORS1 ...]]
+                                [--indicators2 INDICATORS2 [INDICATORS2 ...]]
+                                [--plot-limit INT] [--db-url PATH]
+                                [--trade-source {DB,file}] [--export EXPORT]
+                                [--export-filename PATH]
+                                [--timerange TIMERANGE]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
+                        Show profits for only these pairs. Pairs are space-
+                        separated.
+  --indicators1 INDICATORS1 [INDICATORS1 ...]
+                        Set indicators from your strategy you want in the
+                        first row of the graph. Space-separated list. Example:
+                        `ema3 ema5`. Default: `['sma', 'ema3', 'ema5']`.
+  --indicators2 INDICATORS2 [INDICATORS2 ...]
+                        Set indicators from your strategy you want in the
+                        third row of the graph. Space-separated list. Example:
+                        `fastd fastk`. Default: `['macd', 'macdsignal']`.
+  --plot-limit INT      Specify tick limit for plotting. Notice: too high
+                        values cause huge files. Default: 750.
+  --db-url PATH         Override trades database URL, this is useful in custom
+                        deployments (default: `sqlite:///tradesv3.sqlite` for
+                        Live Run mode, `sqlite://` for Dry Run).
+  --trade-source {DB,file}
+                        Specify the source for trades (Can be DB or file
+                        (backtest file)) Default: file
+  --export EXPORT       Export backtest results, argument are: trades.
+                        Example: `--export=trades`
+  --export-filename PATH
+                        Save backtest results to the file with this filename
+                        (default: `user_data/backtest_results/backtest-
+                        result.json`). Requires `--export` to be set as well.
+                        Example: `--export-filename=user_data/backtest_results
+                        /backtest_today.json`
+  --timerange TIMERANGE
+                        Specify what timerange of data to use.
 
-``` bash
-python3 script/plot_dataframe.py [-h] [-p pairs]
 ```
 
-Example
+Example:
 
 ``` bash
-python3 scripts/plot_dataframe.py -p BTC/ETH
+freqtrade plot-dataframe -p BTC/ETH
 ```
 
-The `-p` pairs argument can be used to specify pairs you would like to plot.
+The `-p/--pairs` argument can be used to specify pairs you would like to plot.
+
+!!! Note
+    The `freqtrade plot-dataframe` subcommand generates one plot-file per pair.
 
 Specify custom indicators.
 Use `--indicators1` for the main plot and `--indicators2` for the subplot below (if values are in a different range than prices).
 
+!!! tip
+    You will almost certainly want to specify a custom strategy! This can be done by adding `-s Classname` / `--strategy ClassName` to the command.
+
 ``` bash
-python3 scripts/plot_dataframe.py -p BTC/ETH --indicators1 sma,ema --indicators2 macd
+freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --indicators1 sma ema --indicators2 macd
 ```
 
-### Advanced use
+### Further usage examples
 
-To plot multiple pairs, separate them with a comma:
+To plot multiple pairs, separate them with a space:
 
 ``` bash
-python3 scripts/plot_dataframe.py -p BTC/ETH,XRP/ETH
+freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH XRP/ETH
 ```
 
-To plot a timerange (to zoom in):
+To plot a timerange (to zoom in)
 
 ``` bash
-python3 scripts/plot_dataframe.py -p BTC/ETH --timerange=20180801-20180805
+freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --timerange=20180801-20180805
 ```
 
-To plot trades stored in a database use `--db-url` argument:
+To plot trades stored in a database use `--db-url` in combination with `--trade-source DB`:
 
 ``` bash
-python3 scripts/plot_dataframe.py --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB
+freqtrade --strategy AwesomeStrategy plot-dataframe --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB
 ```
 
 To plot trades from a backtesting result, use `--export-filename <filename>`
 
 ``` bash
-python3 scripts/plot_dataframe.py --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
+freqtrade --strategy AwesomeStrategy plot-dataframe --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
-```
-
-To plot a custom strategy the strategy should have first be backtested.
-The results may then be plotted with the -s argument:
-
-``` bash
-python3 scripts/plot_dataframe.py -s Strategy_Name -p BTC/ETH --datadir user_data/data/<exchange_name>/
 ```
 
 ## Plot profit
 
-The profit plotter shows a picture with three plots:
+![plot-profit](assets/plot-profit.png)
+
+The `freqtrade plot-profit` subcommand shows an interactive graph with three plots:
 
 1) Average closing price for all pairs
 2) The summarized profit made by backtesting.
-   Note that this is not the real-world profit, but
+   Note that this is not the real-world profit, but more of an estimate.
-   more of an estimate.
+3) Profit for each individual pair
-3) Each pair individually profit
 
-The first graph is good to get a grip of how the overall market
+The first graph is good to get a grip of how the overall market progresses.
-progresses.
 
-The second graph will show how your algorithm works or doesn't.
+The second graph will show if your algorithm works or doesn't.
-Perhaps you want an algorithm that steadily makes small profits,
+Perhaps you want an algorithm that steadily makes small profits, or one that acts less often, but makes big swings.
-or one that acts less seldom, but makes big swings.
 
-The third graph can be useful to spot outliers, events in pairs
+The third graph can be useful to spot outliers, events in pairs that cause profit spikes.
-that makes profit spikes.
 
-Usage for the profit plotter:
+Possible options for the `freqtrade plot-profit` subcommand:
+
+```
+usage: freqtrade plot-profit [-h] [-p PAIRS [PAIRS ...]]
+                             [--timerange TIMERANGE] [--export EXPORT]
+                             [--export-filename PATH] [--db-url PATH]
+                             [--trade-source {DB,file}]
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
+                        Show profits for only these pairs. Pairs are space-
+                        separated.
+  --timerange TIMERANGE
+                        Specify what timerange of data to use.
+  --export EXPORT       Export backtest results, argument are: trades.
+                        Example: `--export=trades`
+  --export-filename PATH
+                        Save backtest results to the file with this filename
+                        (default: `user_data/backtest_results/backtest-
+                        result.json`). Requires `--export` to be set as well.
+                        Example: `--export-filename=user_data/backtest_results
+                        /backtest_today.json`
+  --db-url PATH         Override trades database URL, this is useful in custom
+                        deployments (default: `sqlite:///tradesv3.sqlite` for
+                        Live Run mode, `sqlite://` for Dry Run).
+  --trade-source {DB,file}
+                        Specify the source for trades (Can be DB or file
+                        (backtest file)) Default: file
 
-``` bash
-python3 script/plot_profit.py [-h] [-p pair] [--datadir directory] [--ticker_interval num]
 ```
 
-The `-p` pair argument, can be used to plot a single pair
+The `-p/--pairs`  argument, can be used to limit the pairs that are considered for this calculation.
 
-Example
+Examples:
+
+Use custom backtest-export file
 
 ``` bash
-python3 scripts/plot_profit.py --datadir ../freqtrade/freqtrade/tests/testdata-20171221/ -p LTC/BTC
+freqtrade plot-profit  -p LTC/BTC --export-filename user_data/backtest_results/backtest-result-Strategy005.json
+```
+
+Use custom database
+
+``` bash
+freqtrade plot-profit  -p LTC/BTC --db-url sqlite:///tradesv3.sqlite --trade-source DB
+```
+
+``` bash
+freqtrade plot-profit --datadir ../freqtrade/freqtrade/tests/testdata-20171221/ -p LTC/BTC
 ```

freqtrade/configuration/arguments.py

@@ -34,15 +34,13 @@ ARGS_CREATE_USERDIR = ["user_data_dir"]
 
 ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "exchange", "timeframes", "erase"]
 
-ARGS_PLOT_DATAFRAME = (ARGS_COMMON + ARGS_STRATEGY +
+ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit", "db_url",
-                       ["pairs", "indicators1", "indicators2", "plot_limit", "db_url",
+                       "trade_source", "export", "exportfilename", "timerange", "ticker_interval"]
-                        "trade_source", "export", "exportfilename", "timerange",
-                        "refresh_pairs"])
 
-ARGS_PLOT_PROFIT = (ARGS_COMMON + ARGS_STRATEGY +
+ARGS_PLOT_PROFIT = ["pairs", "timerange", "export", "exportfilename", "db_url",
-                    ["pairs", "timerange", "export", "exportfilename", "db_url", "trade_source"])
+                    "trade_source", "ticker_interval"]
 
-NO_CONF_REQURIED = ["start_download_data"]
+NO_CONF_REQURIED = ["download-data", "plot-dataframe", "plot-profit"]
 
 
 class Arguments(object):
@@ -81,8 +79,7 @@ class Arguments(object):
         # (see https://bugs.python.org/issue16399)
         # Allow no-config for certain commands (like downloading / plotting)
         if (not self._no_default_config and parsed_arg.config is None
-                and not (hasattr(parsed_arg, 'func')
+                and not ('subparser' in parsed_arg and parsed_arg.subparser in NO_CONF_REQURIED)):
-                         and parsed_arg.func.__name__ in NO_CONF_REQURIED)):
             parsed_arg.config = [constants.DEFAULT_CONFIG]
 
         return parsed_arg
@@ -119,6 +116,7 @@ class Arguments(object):
         hyperopt_cmd.set_defaults(func=start_hyperopt)
         self._build_args(optionlist=ARGS_HYPEROPT, parser=hyperopt_cmd)
 
+        # add create-userdir subcommand
         create_userdir_cmd = subparsers.add_parser('create-userdir',
                                                    help="Create user-data directory.")
         create_userdir_cmd.set_defaults(func=start_create_userdir)
@@ -139,3 +137,20 @@ class Arguments(object):
         )
         download_data_cmd.set_defaults(func=start_download_data)
         self._build_args(optionlist=ARGS_DOWNLOAD_DATA, parser=download_data_cmd)
+
+        # Add Plotting subcommand
+        from freqtrade.plot.plot_utils import start_plot_dataframe, start_plot_profit
+        plot_dataframe_cmd = subparsers.add_parser(
+            'plot-dataframe',
+            help='Plot candles with indicators.'
+        )
+        plot_dataframe_cmd.set_defaults(func=start_plot_dataframe)
+        self._build_args(optionlist=ARGS_PLOT_DATAFRAME, parser=plot_dataframe_cmd)
+
+        # Plot profit
+        plot_profit_cmd = subparsers.add_parser(
+            'plot-profit',
+            help='Generate plot showing profits.'
+        )
+        plot_profit_cmd.set_defaults(func=start_plot_profit)
+        self._build_args(optionlist=ARGS_PLOT_PROFIT, parser=plot_profit_cmd)

freqtrade/configuration/check_exchange.py

@@ -5,6 +5,7 @@ from freqtrade import OperationalException
 from freqtrade.exchange import (available_exchanges, get_exchange_bad_reason,
                                 is_exchange_available, is_exchange_bad,
                                 is_exchange_officially_supported)
+from freqtrade.state import RunMode
 
 logger = logging.getLogger(__name__)
 
@@ -19,6 +20,10 @@ def check_exchange(config: Dict[str, Any], check_for_bad: bool = True) -> bool:
              raises an exception if the exchange if not supported by ccxt
              and thus is not known for the Freqtrade at all.
     """
+
+    if config['runmode'] in [RunMode.PLOT] and not config.get('exchange', {}).get('name'):
+        # Skip checking exchange in plot mode, since it requires no exchange
+        return True
     logger.info("Checking exchange...")
 
     exchange = config.get('exchange', {}).get('name').lower()

freqtrade/configuration/cli_options.py

@@ -292,14 +292,16 @@ AVAILABLE_CLI_OPTIONS = {
     "indicators1": Arg(
         '--indicators1',
         help='Set indicators from your strategy you want in the first row of the graph. '
-        'Comma-separated list. Example: `ema3,ema5`. Default: `%(default)s`.',
+        'Space-separated list. Example: `ema3 ema5`. Default: `%(default)s`.',
-        default='sma,ema3,ema5',
+        default=['sma', 'ema3', 'ema5'],
+        nargs='+',
     ),
     "indicators2": Arg(
         '--indicators2',
         help='Set indicators from your strategy you want in the third row of the graph. '
-        'Comma-separated list. Example: `fastd,fastk`. Default: `%(default)s`.',
+        'Space-separated list. Example: `fastd fastk`. Default: `%(default)s`.',
-        default='macd,macdsignal',
+        default=['macd', 'macdsignal'],
+        nargs='+',
     ),
     "plot_limit": Arg(
         '--plot-limit',

freqtrade/data/btanalysis.py

@@ -112,16 +112,16 @@ def load_trades_from_db(db_url: str) -> pd.DataFrame:
     return trades
 
 
-def load_trades(config) -> pd.DataFrame:
+def load_trades(source: str, db_url: str, exportfilename: str) -> pd.DataFrame:
     """
     Based on configuration option "trade_source":
     * loads data from DB (using `db_url`)
     * loads data from backtestfile (using `exportfilename`)
     """
-    if config["trade_source"] == "DB":
+    if source == "DB":
-        return load_trades_from_db(config["db_url"])
+        return load_trades_from_db(db_url)
-    elif config["trade_source"] == "file":
+    elif source == "file":
-        return load_backtest_data(Path(config["exportfilename"]))
+        return load_backtest_data(Path(exportfilename))
 
 
 def extract_trades_of_period(dataframe: pd.DataFrame, trades: pd.DataFrame) -> pd.DataFrame:
@@ -157,7 +157,8 @@ def create_cum_profit(df: pd.DataFrame, trades: pd.DataFrame, col_name: str) ->
     :param trades: DataFrame containing trades (requires columns close_time and profitperc)
     :return: Returns df with one additional column, col_name, containing the cumulative profit.
     """
-    df[col_name] = trades.set_index('close_time')['profitperc'].cumsum()
+    # Use groupby/sum().cumsum() to avoid errors when multiple trades sold at the same candle.
+    df[col_name] = trades.groupby('close_time')['profitperc'].sum().cumsum()
     # Set first value to 0
     df.loc[df.iloc[0].name, col_name] = 0
     # FFill to get continuous

freqtrade/plot/plot_utils.py

@@ -0,0 +1,26 @@
+from argparse import Namespace
+
+from freqtrade.state import RunMode
+from freqtrade.utils import setup_utils_configuration
+
+
+def start_plot_dataframe(args: Namespace) -> None:
+    """
+    Entrypoint for dataframe plotting
+    """
+    # Import here to avoid errors if plot-dependencies are not installed.
+    from freqtrade.plot.plotting import analyse_and_plot_pairs
+    config = setup_utils_configuration(args, RunMode.PLOT)
+
+    analyse_and_plot_pairs(config)
+
+
+def start_plot_profit(args: Namespace) -> None:
+    """
+    Entrypoint for plot_profit
+    """
+    # Import here to avoid errors if plot-dependencies are not installed.
+    from freqtrade.plot.plotting import plot_profit
+    config = setup_utils_configuration(args, RunMode.PLOT)
+
+    plot_profit(config)

freqtrade/plot/plotting.py

@@ -1,15 +1,15 @@
 import logging
 from pathlib import Path
-from typing import Dict, List, Optional
+from typing import Any, Dict, List
 
 import pandas as pd
 
 from freqtrade.configuration import TimeRange
 from freqtrade.data import history
 from freqtrade.data.btanalysis import (combine_tickers_with_mean,
-                                       create_cum_profit, load_trades)
+                                       create_cum_profit,
-from freqtrade.exchange import Exchange
+                                       extract_trades_of_period, load_trades)
-from freqtrade.resolvers import ExchangeResolver, StrategyResolver
+from freqtrade.resolvers import StrategyResolver
 
 logger = logging.getLogger(__name__)
 
@@ -19,23 +19,16 @@ try:
     from plotly.offline import plot
     import plotly.graph_objects as go
 except ImportError:
-    logger.exception("Module plotly not found \n Please install using `pip install plotly`")
+    logger.exception("Module plotly not found \n Please install using `pip3 install plotly`")
     exit(1)
 
 
 def init_plotscript(config):
     """
     Initialize objects needed for plotting
-    :return: Dict with tickers, trades, pairs and strategy
+    :return: Dict with tickers, trades and pairs
     """
-    exchange: Optional[Exchange] = None
 
-    # Exchange is only needed when downloading data!
-    if config.get("refresh_pairs", False):
-        exchange = ExchangeResolver(config.get('exchange', {}).get('name'),
-                                    config).exchange
-
-    strategy = StrategyResolver(config).strategy
     if "pairs" in config:
         pairs = config["pairs"]
     else:
@@ -47,17 +40,18 @@ def init_plotscript(config):
     tickers = history.load_data(
         datadir=Path(str(config.get("datadir"))),
         pairs=pairs,
-        ticker_interval=config['ticker_interval'],
+        ticker_interval=config.get('ticker_interval', '5m'),
-        refresh_pairs=config.get('refresh_pairs', False),
         timerange=timerange,
-        exchange=exchange,
     )
 
-    trades = load_trades(config)
+    trades = load_trades(config['trade_source'],
+                         db_url=config.get('db_url'),
+                         exportfilename=config.get('exportfilename'),
+                         )
+
     return {"tickers": tickers,
             "trades": trades,
             "pairs": pairs,
-            "strategy": strategy,
             }
 
 
@@ -280,8 +274,15 @@ def generate_profit_graph(pairs: str, tickers: Dict[str, pd.DataFrame],
         name='Avg close price',
     )
 
-    fig = make_subplots(rows=3, cols=1, shared_xaxes=True, row_width=[1, 1, 1])
+    fig = make_subplots(rows=3, cols=1, shared_xaxes=True,
-    fig['layout'].update(title="Profit plot")
+                        row_width=[1, 1, 1],
+                        vertical_spacing=0.05,
+                        subplot_titles=["AVG Close Price", "Combined Profit", "Profit per pair"])
+    fig['layout'].update(title="Freqtrade Profit plot")
+    fig['layout']['yaxis1'].update(title='Price')
+    fig['layout']['yaxis2'].update(title='Profit')
+    fig['layout']['yaxis3'].update(title='Profit')
+    fig['layout']['xaxis']['rangeslider'].update(visible=False)
 
     fig.add_trace(avgclose, 1, 1)
     fig = add_profit(fig, 2, df_comb, 'cum_profit', 'Profit')
@@ -321,3 +322,67 @@ def store_plot_file(fig, filename: str, directory: Path, auto_open: bool = False
     plot(fig, filename=str(_filename),
          auto_open=auto_open)
     logger.info(f"Stored plot as {_filename}")
+
+
+def analyse_and_plot_pairs(config: Dict[str, Any]):
+    """
+    From configuration provided
+    - Initializes plot-script
+    - Get tickers data
+    - Generate Dafaframes populated with indicators and signals based on configured strategy
+    - Load trades excecuted during the selected period
+    - Generate Plotly plot objects
+    - Generate plot files
+    :return: None
+    """
+    strategy = StrategyResolver(config).strategy
+
+    plot_elements = init_plotscript(config)
+    trades = plot_elements['trades']
+
+    pair_counter = 0
+    for pair, data in plot_elements["tickers"].items():
+        pair_counter += 1
+        logger.info("analyse pair %s", pair)
+        tickers = {}
+        tickers[pair] = data
+
+        dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
+
+        trades_pair = trades.loc[trades['pair'] == pair]
+        trades_pair = extract_trades_of_period(dataframe, trades_pair)
+
+        fig = generate_candlestick_graph(
+            pair=pair,
+            data=dataframe,
+            trades=trades_pair,
+            indicators1=config["indicators1"],
+            indicators2=config["indicators2"],
+        )
+
+        store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
+                        directory=config['user_data_dir'] / "plot")
+
+    logger.info('End of plotting process. %s plots generated', pair_counter)
+
+
+def plot_profit(config: Dict[str, Any]) -> None:
+    """
+    Plots the total profit for all pairs.
+    Note, the profit calculation isn't realistic.
+    But should be somewhat proportional, and therefor useful
+    in helping out to find a good algorithm.
+    """
+    plot_elements = init_plotscript(config)
+    trades = load_trades(config['trade_source'],
+                         db_url=str(config.get('db_url')),
+                         exportfilename=str(config.get('exportfilename')),
+                         )
+    # Filter trades to relevant pairs
+    trades = trades[trades['pair'].isin(plot_elements["pairs"])]
+
+    # Create an average close price of all the pairs that were involved.
+    # this could be useful to gauge the overall market trend
+    fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
+    store_plot_file(fig, filename='freqtrade-profit-plot.html',
+                    directory=config['user_data_dir'] / "plot", auto_open=True)

freqtrade/state.py

@@ -25,4 +25,5 @@ class RunMode(Enum):
     BACKTEST = "backtest"
     EDGE = "edge"
     HYPEROPT = "hyperopt"
+    PLOT = "plot"
     OTHER = "other"  # Used for plotting scripts and test

freqtrade/tests/data/test_btanalysis.py

@@ -89,17 +89,20 @@ def test_load_trades(default_conf, mocker):
     db_mock = mocker.patch("freqtrade.data.btanalysis.load_trades_from_db", MagicMock())
     bt_mock = mocker.patch("freqtrade.data.btanalysis.load_backtest_data", MagicMock())
 
-    default_conf['trade_source'] = "DB"
+    load_trades("DB",
-    load_trades(default_conf)
+                db_url=default_conf.get('db_url'),
+                exportfilename=default_conf.get('exportfilename'),
+                )
 
     assert db_mock.call_count == 1
     assert bt_mock.call_count == 0
 
     db_mock.reset_mock()
     bt_mock.reset_mock()
-    default_conf['trade_source'] = "file"
     default_conf['exportfilename'] = "testfile.json"
-    load_trades(default_conf)
+    load_trades("file",
+                db_url=default_conf.get('db_url'),
+                exportfilename=default_conf.get('exportfilename'),)
 
     assert db_mock.call_count == 0
     assert bt_mock.call_count == 1

freqtrade/tests/test_arguments.py

@@ -4,7 +4,6 @@ import argparse
 import pytest
 
 from freqtrade.configuration import Arguments
-from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
 from freqtrade.configuration.cli_options import check_int_positive
 
 
@@ -149,20 +148,35 @@ def test_download_data_options() -> None:
 
 def test_plot_dataframe_options() -> None:
     args = [
-        '--indicators1', 'sma10,sma100',
+        '-c', 'config.json.example',
-        '--indicators2', 'macd,fastd,fastk',
+        'plot-dataframe',
+        '--indicators1', 'sma10', 'sma100',
+        '--indicators2', 'macd', 'fastd', 'fastk',
         '--plot-limit', '30',
         '-p', 'UNITTEST/BTC',
     ]
-    arguments = Arguments(args, '')
+    pargs = Arguments(args, '').get_parsed_arg()
-    arguments._build_args(ARGS_PLOT_DATAFRAME)
+
-    pargs = arguments._parse_args()
+    assert pargs.indicators1 == ["sma10", "sma100"]
-    assert pargs.indicators1 == "sma10,sma100"
+    assert pargs.indicators2 == ["macd", "fastd", "fastk"]
-    assert pargs.indicators2 == "macd,fastd,fastk"
     assert pargs.plot_limit == 30
     assert pargs.pairs == ["UNITTEST/BTC"]
 
 
+def test_plot_profit_options() -> None:
+    args = [
+        'plot-profit',
+        '-p', 'UNITTEST/BTC',
+        '--trade-source', 'DB',
+        "--db-url", "sqlite:///whatever.sqlite",
+    ]
+    pargs = Arguments(args, '').get_parsed_arg()
+
+    assert pargs.trade_source == "DB"
+    assert pargs.pairs == ["UNITTEST/BTC"]
+    assert pargs.db_url == "sqlite:///whatever.sqlite"
+
+
 def test_check_int_positive() -> None:
     assert check_int_positive("3") == 3
     assert check_int_positive("1") == 1

freqtrade/tests/test_configuration.py

@@ -479,6 +479,7 @@ def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None:
 
 def test_check_exchange(default_conf, caplog) -> None:
     # Test an officially supported by Freqtrade team exchange
+    default_conf['runmode'] = RunMode.DRY_RUN
     default_conf.get('exchange').update({'name': 'BITTREX'})
     assert check_exchange(default_conf)
     assert log_has_re(r"Exchange .* is officially supported by the Freqtrade development team\.",
@@ -523,6 +524,11 @@ def test_check_exchange(default_conf, caplog) -> None:
     ):
         check_exchange(default_conf)
 
+    # Test no exchange...
+    default_conf.get('exchange').update({'name': ''})
+    default_conf['runmode'] = RunMode.PLOT
+    assert check_exchange(default_conf)
+
 
 def test_cli_verbose_with_params(default_conf, mocker, caplog) -> None:
     patched_configuration_load_config_file(mocker, default_conf)

freqtrade/tests/test_plotting.py

@@ -9,13 +9,15 @@ from plotly.subplots import make_subplots
 from freqtrade.configuration import TimeRange
 from freqtrade.data import history
 from freqtrade.data.btanalysis import create_cum_profit, load_backtest_data
+from freqtrade.plot.plot_utils import start_plot_dataframe, start_plot_profit
 from freqtrade.plot.plotting import (add_indicators, add_profit,
+                                     analyse_and_plot_pairs,
                                      generate_candlestick_graph,
                                      generate_plot_filename,
                                      generate_profit_graph, init_plotscript,
-                                     plot_trades, store_plot_file)
+                                     plot_profit, plot_trades, store_plot_file)
 from freqtrade.strategy.default_strategy import DefaultStrategy
-from freqtrade.tests.conftest import log_has, log_has_re
+from freqtrade.tests.conftest import get_args, log_has, log_has_re
 
 
 def fig_generating_mock(fig, *args, **kwargs):
@@ -49,7 +51,6 @@ def test_init_plotscript(default_conf, mocker):
     assert "tickers" in ret
     assert "trades" in ret
     assert "pairs" in ret
-    assert "strategy" in ret
 
     default_conf['pairs'] = ["POWR/BTC", "XLM/BTC"]
     ret = init_plotscript(default_conf)
@@ -257,7 +258,11 @@ def test_generate_profit_graph():
     fig = generate_profit_graph(pairs, tickers, trades)
     assert isinstance(fig, go.Figure)
 
-    assert fig.layout.title.text == "Profit plot"
+    assert fig.layout.title.text == "Freqtrade Profit plot"
+    assert fig.layout.yaxis.title.text == "Price"
+    assert fig.layout.yaxis2.title.text == "Profit"
+    assert fig.layout.yaxis3.title.text == "Profit"
+
     figure = fig.layout.figure
     assert len(figure.data) == 4
 
@@ -270,3 +275,85 @@ def test_generate_profit_graph():
     for pair in pairs:
         profit_pair = find_trace_in_fig_data(figure.data, f"Profit {pair}")
         assert isinstance(profit_pair, go.Scattergl)
+
+
+def test_start_plot_dataframe(mocker):
+    aup = mocker.patch("freqtrade.plot.plotting.analyse_and_plot_pairs", MagicMock())
+    args = [
+        "--config", "config.json.example",
+        "plot-dataframe",
+        "--pairs", "ETH/BTC"
+    ]
+    start_plot_dataframe(get_args(args))
+
+    assert aup.call_count == 1
+    called_config = aup.call_args_list[0][0][0]
+    assert "pairs" in called_config
+    assert called_config['pairs'] == ["ETH/BTC"]
+
+
+def test_analyse_and_plot_pairs(default_conf, mocker, caplog):
+    default_conf['trade_source'] = 'file'
+    default_conf["datadir"] = history.make_testdata_path(None)
+    default_conf['exportfilename'] = str(
+        history.make_testdata_path(None) / "backtest-result_test.json")
+    default_conf['indicators1'] = ["sma5", "ema10"]
+    default_conf['indicators2'] = ["macd"]
+    default_conf['pairs'] = ["ETH/BTC", "LTC/BTC"]
+
+    candle_mock = MagicMock()
+    store_mock = MagicMock()
+    mocker.patch.multiple(
+        "freqtrade.plot.plotting",
+        generate_candlestick_graph=candle_mock,
+        store_plot_file=store_mock
+        )
+    analyse_and_plot_pairs(default_conf)
+
+    # Both mocks should be called once per pair
+    assert candle_mock.call_count == 2
+    assert store_mock.call_count == 2
+
+    assert candle_mock.call_args_list[0][1]['indicators1'] == ['sma5', 'ema10']
+    assert candle_mock.call_args_list[0][1]['indicators2'] == ['macd']
+
+    assert log_has("End of plotting process. 2 plots generated", caplog)
+
+
+def test_start_plot_profit(mocker):
+    aup = mocker.patch("freqtrade.plot.plotting.plot_profit", MagicMock())
+    args = [
+        "--config", "config.json.example",
+        "plot-profit",
+        "--pairs", "ETH/BTC"
+    ]
+    start_plot_profit(get_args(args))
+
+    assert aup.call_count == 1
+    called_config = aup.call_args_list[0][0][0]
+    assert "pairs" in called_config
+    assert called_config['pairs'] == ["ETH/BTC"]
+
+
+def test_plot_profit(default_conf, mocker, caplog):
+    default_conf['trade_source'] = 'file'
+    default_conf["datadir"] = history.make_testdata_path(None)
+    default_conf['exportfilename'] = str(
+        history.make_testdata_path(None) / "backtest-result_test.json")
+    default_conf['pairs'] = ["ETH/BTC", "LTC/BTC"]
+
+    profit_mock = MagicMock()
+    store_mock = MagicMock()
+    mocker.patch.multiple(
+        "freqtrade.plot.plotting",
+        generate_profit_graph=profit_mock,
+        store_plot_file=store_mock
+    )
+    plot_profit(default_conf)
+
+    # Plot-profit generates one combined plot
+    assert profit_mock.call_count == 1
+    assert store_mock.call_count == 1
+
+    assert profit_mock.call_args_list[0][0][0] == default_conf['pairs']
+    assert store_mock.call_args_list[0][1]['auto_open'] is True

freqtrade/utils.py

@@ -92,6 +92,3 @@ def start_download_data(args: Namespace) -> None:
         if pairs_not_available:
             logger.info(f"Pairs [{','.join(pairs_not_available)}] not available "
                         f"on exchange {config['exchange']['name']}.")
-
-    # configuration.resolve_pairs_list()
-    print(config)

mkdocs.yml

@@ -14,12 +14,12 @@ nav:
     - Backtesting: backtesting.md
     - Hyperopt: hyperopt.md
     - Edge positioning: edge.md
-    - Plotting: plotting.md
-    - Deprecated features: deprecated.md
     - FAQ: faq.md
     - Data Analysis: data-analysis.md
+    - Plotting: plotting.md
     - SQL Cheatsheet: sql_cheatsheet.md
     - Sandbox testing: sandbox-testing.md
+    - Deprecated features: deprecated.md
     - Contributors guide: developer.md
 theme:
   name: material

scripts/plot_dataframe.py

@@ -1,100 +1,11 @@
 #!/usr/bin/env python3
-"""
-Script to display when the bot will buy on specific pair(s)
 
-Use `python plot_dataframe.py --help` to display the command line arguments
-
-Indicators recommended
-Row 1: sma, ema3, ema5, ema10, ema50
-Row 3: macd, rsi, fisher_rsi, mfi, slowd, slowk, fastd, fastk
-
-Example of usage:
-> python3 scripts/plot_dataframe.py --pairs BTC/EUR,XRP/BTC -d user_data/data/
-  --indicators1 sma,ema3 --indicators2 fastk,fastd
-"""
-import logging
 import sys
-from typing import Any, Dict, List
-
-from freqtrade.configuration import Arguments
-from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
-from freqtrade.data.btanalysis import extract_trades_of_period
-from freqtrade.optimize import setup_configuration
-from freqtrade.plot.plotting import (init_plotscript, generate_candlestick_graph,
-                                     store_plot_file,
-                                     generate_plot_filename)
-from freqtrade.state import RunMode
-
-logger = logging.getLogger(__name__)
 
 
-def analyse_and_plot_pairs(config: Dict[str, Any]):
+print("This script has been integrated into freqtrade "
-    """
+      "and its functionality is available by calling `freqtrade plot-dataframe`.")
-    From arguments provided in cli:
+print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
-    -Initialise backtest env
+      "for details.")
-    -Get tickers data
-    -Generate Dafaframes populated with indicators and signals
-    -Load trades excecuted on same periods
-    -Generate Plotly plot objects
-    -Generate plot files
-    :return: None
-    """
-    plot_elements = init_plotscript(config)
-    trades = plot_elements['trades']
-    strategy = plot_elements["strategy"]
 
-    pair_counter = 0
+sys.exit(1)
-    for pair, data in plot_elements["tickers"].items():
-        pair_counter += 1
-        logger.info("analyse pair %s", pair)
-        tickers = {}
-        tickers[pair] = data
-
-        dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
-
-        trades_pair = trades.loc[trades['pair'] == pair]
-        trades_pair = extract_trades_of_period(dataframe, trades_pair)
-
-        fig = generate_candlestick_graph(
-            pair=pair,
-            data=dataframe,
-            trades=trades_pair,
-            indicators1=config["indicators1"].split(","),
-            indicators2=config["indicators2"].split(",")
-        )
-
-        store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
-                        directory=config['user_data_dir'] / "plot")
-
-    logger.info('End of ploting process %s plots generated', pair_counter)
-
-
-def plot_parse_args(args: List[str]) -> Dict[str, Any]:
-    """
-    Parse args passed to the script
-    :param args: Cli arguments
-    :return: args: Array with all arguments
-    """
-    arguments = Arguments(args, 'Graph dataframe')
-    arguments._build_args(optionlist=ARGS_PLOT_DATAFRAME)
-    parsed_args = arguments._parse_args()
-
-    # Load the configuration
-    config = setup_configuration(parsed_args, RunMode.OTHER)
-    return config
-
-
-def main(sysargv: List[str]) -> None:
-    """
-    This function will initiate the bot and start the trading loop.
-    :return: None
-    """
-    logger.info('Starting Plot Dataframe')
-    analyse_and_plot_pairs(
-        plot_parse_args(sysargv)
-    )
-    exit()
-
-
-if __name__ == '__main__':
-    main(sys.argv[1:])

scripts/plot_profit.py

@@ -1,66 +1,11 @@
 #!/usr/bin/env python3
-"""
-Script to display profits
 
-Use `python plot_profit.py --help` to display the command line arguments
-"""
-import logging
 import sys
-from typing import Any, Dict, List
-
-from freqtrade.configuration import Arguments
-from freqtrade.configuration.arguments import ARGS_PLOT_PROFIT
-from freqtrade.optimize import setup_configuration
-from freqtrade.plot.plotting import init_plotscript, generate_profit_graph, store_plot_file
-from freqtrade.state import RunMode
-
-logger = logging.getLogger(__name__)
 
 
-def plot_profit(config: Dict[str, Any]) -> None:
+print("This script has been integrated into freqtrade "
-    """
+      "and its functionality is available by calling `freqtrade plot-profit`.")
-    Plots the total profit for all pairs.
+print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
-    Note, the profit calculation isn't realistic.
+      "for details.")
-    But should be somewhat proportional, and therefor useful
-    in helping out to find a good algorithm.
-    """
-    plot_elements = init_plotscript(config)
-    trades = plot_elements['trades']
-    # Filter trades to relevant pairs
-    trades = trades[trades['pair'].isin(plot_elements["pairs"])]
 
-    # Create an average close price of all the pairs that were involved.
+sys.exit(1)
-    # this could be useful to gauge the overall market trend
-    fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
-    store_plot_file(fig, filename='freqtrade-profit-plot.html',
-                    directory=config['user_data_dir'] / "plot", auto_open=True)
-
-
-def plot_parse_args(args: List[str]) -> Dict[str, Any]:
-    """
-    Parse args passed to the script
-    :param args: Cli arguments
-    :return: args: Array with all arguments
-    """
-    arguments = Arguments(args, 'Graph profits')
-    arguments._build_args(optionlist=ARGS_PLOT_PROFIT)
-    parsed_args = arguments._parse_args()
-
-    # Load the configuration
-    config = setup_configuration(parsed_args, RunMode.OTHER)
-    return config
-
-
-def main(sysargv: List[str]) -> None:
-    """
-    This function will initiate the bot and start the trading loop.
-    :return: None
-    """
-    logger.info('Starting Plot Dataframe')
-    plot_profit(
-        plot_parse_args(sysargv)
-    )
-
-
-if __name__ == '__main__':
-    main(sys.argv[1:])