whitelist conflict resolved with develop branch

docs/hyperopt.md

@@ -1,4 +1,5 @@
 # Hyperopt
+
 This page explains how to tune your strategy by finding the optimal 
 parameters, a process called hyperparameter optimization. The bot uses several 
 algorithms included in the `scikit-optimize` package to accomplish this. The
@@ -8,17 +9,20 @@ and still take a long time.
 *Note:* Hyperopt will crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133)
 
 ## Table of Contents
+
 - [Prepare your Hyperopt](#prepare-hyperopt)
 - [Configure your Guards and Triggers](#configure-your-guards-and-triggers)
 - [Solving a Mystery](#solving-a-mystery)
 - [Adding New Indicators](#adding-new-indicators)
 - [Execute Hyperopt](#execute-hyperopt)
-- [Understand the hyperopts result](#understand-the-backtesting-result)
+- [Understand the hyperopt result](#understand-the-hyperopt-result)
 
 ## Prepare Hyperopting
+
 We recommend you start by taking a look at `hyperopt.py` file located in [freqtrade/optimize](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py)
  
 ### Configure your Guards and Triggers
+
 There are two places you need to change to add a new buy strategy for testing:
 - Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L231-L264).
 - Inside [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L213-L224) 
@@ -113,11 +117,12 @@ When you want to test an indicator that isn't used by the bot currently, remembe
 add it to the `populate_indicators()` method in `hyperopt.py`.
 
 ## Execute Hyperopt
-Once you have updated your hyperopt configuration you can run it. 
-Because hyperopt tries a lot of combination to find the best parameters
-it will take time you will have the result (more than 30 mins).
 
-We strongly recommend to use `screen` to prevent any connection loss.
+Once you have updated your hyperopt configuration you can run it.
+Because hyperopt tries a lot of combinations to find the best parameters it will take time you will have the result (more than 30 mins).
+
+We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
+
 ```bash
 python3 ./freqtrade/main.py -c config.json hyperopt -e 5000
 ```
@@ -126,11 +131,13 @@ The `-e` flag will set how many evaluations hyperopt will do. We recommend
 running at least several thousand evaluations.
 
 ### Execute Hyperopt with Different Ticker-Data Source
+
 If you would like to hyperopt parameters using an alternate ticker data that
 you have on-disk, use the `--datadir PATH` option. Default hyperopt will
 use data from directory `user_data/data`.
 
 ### Running Hyperopt with Smaller Testset
+
 Use the `--timerange` argument to change how much of the testset
 you want to use. The last N ticks/timeframes will be used.
 Example:
@@ -140,6 +147,7 @@ python3 ./freqtrade/main.py hyperopt --timerange -200
 ```
 
 ### Running Hyperopt with Smaller Search Space
+
 Use the `--spaces` argument to limit the search space used by hyperopt.
 Letting Hyperopt optimize everything is a huuuuge search space. Often it 
 might make more sense to start by just searching for initial buy algorithm. 
@@ -154,7 +162,8 @@ Legal values are:
 - `stoploss`: search for the best stoploss value
 - space-separated list of any of the above values for example `--spaces roi stoploss`
 
-## Understand the Hyperopts Result 
+## Understand the Hyperopt Result
+
 Once Hyperopt is completed you can use the result to create a new strategy.
 Given the following result from hyperopt:
 
@@ -166,22 +175,24 @@ with values:
 ```
 
 You should understand this result like:
+
 - The buy trigger that worked best was `bb_lower`.
 - You should not use ADX because `adx-enabled: False`) 
 - You should **consider** using the RSI indicator (`rsi-enabled: True` and the best value is `29.0` (`rsi-value: 29.0`)
 
 You have to look inside your strategy file into `buy_strategy_generator()` 
-method, what those values match to.   
+method, what those values match to.
   
-So for example you had `rsi-value: 29.0` so we would look 
+So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
-at `rsi`-block, that translates to the following code block:
+
 ```
 (dataframe['rsi'] < 29.0)
 ```
   
 Translating your whole hyperopt result as the new buy-signal 
 would then look like:
-```
+
+```python
 def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
     dataframe.loc[
         (
@@ -192,6 +203,39 @@ def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
     return dataframe
 ```
 
+### Understand Hyperopt ROI results
+
+If you are optimizing ROI, you're result will look as follows and include a ROI table.
+
+```
+Best result:
+   135 trades. Avg profit  0.57%. Total profit  0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins.
+with values:
+{'adx-value': 44, 'rsi-value': 29, 'adx-enabled': False, 'rsi-enabled': True, 'trigger': 'bb_lower', 'roi_t1': 40, 'roi_t2': 57, 'roi_t3': 21, 'roi_p1': 0.03634636907306948, 'roi_p2': 0.055237357937802885, 'roi_p3': 0.015163796015548354, 'stoploss': -0.37996664668703606}
+ROI table:
+{0: 0.10674752302642071, 21: 0.09158372701087236, 78: 0.03634636907306948, 118: 0}
+```
+
+This would translate to the following ROI table:
+
+``` python
+ minimal_roi = {
+        "118": 0,
+        "78": 0.0363463,
+        "21": 0.0915,
+        "0": 0.106
+    }
+```
+
+### Validate backtest result
+
+Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.
+To archive the same results (number of trades, ...) than during hyperopt, please use the command line flag `--disable-max-market-positions`.
+This setting is the default for hyperopt for speed reasons. You can overwrite this in the configuration by setting `"position_stacking"=false` or by changing the relevant line in your hyperopt file [here](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L283).
+
+Dry/live runs will **NOT** use position stacking - therefore it does make sense to also validate the strategy without this as it's closer to reality. 
+
 ## Next Step
+
 Now you have a perfect bot and want to control it from Telegram. Your
 next step is to learn the [Telegram usage](https://github.com/freqtrade/freqtrade/blob/develop/docs/telegram-usage.md).

freqtrade/edge/__init__.py

@@ -38,7 +38,7 @@ class Edge():
         self.strategy: IStrategy = StrategyResolver(self.config).strategy
         self.ticker_interval = self.strategy.ticker_interval
         self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
-        self.get_timeframe = Backtesting.get_timeframe
+        self.get_timeframe = optimize.get_timeframe
         self.advise_sell = self.strategy.advise_sell
         self.advise_buy = self.strategy.advise_buy
 

freqtrade/exchange/__init__.py

@@ -375,6 +375,8 @@ class Exchange(object):
     def get_ticker(self, pair: str, refresh: Optional[bool] = True) -> dict:
         if refresh or pair not in self._cached_ticker.keys():
             try:
+                if pair not in self._api.markets:
+                    raise DependencyException(f"Pair {pair} not available")
                 data = self._api.fetch_ticker(pair)
                 try:
                     self._cached_ticker[pair] = {

freqtrade/freqtradebot.py

@@ -61,6 +61,7 @@ class FreqtradeBot(object):
         if self.config.get('edge', {}).get('enabled', False):
             self.edge = Edge(self.config, self.exchange)
 
+        self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist']
         self._init_modules()
 
     def _init_modules(self) -> None:
@@ -114,11 +115,8 @@ class FreqtradeBot(object):
                 constants.PROCESS_THROTTLE_SECS
             )
 
-            nb_assets = self.config.get('dynamic_whitelist', None)
-
             self._throttle(func=self._process,
-                           min_secs=min_secs,
+                           min_secs=min_secs)
-                           nb_assets=nb_assets)
         return state
 
     def _startup_messages(self) -> None:
@@ -169,15 +167,15 @@ class FreqtradeBot(object):
         time.sleep(duration)
         return result
 
-    def _process(self, nb_assets: Optional[int] = 0) -> bool:
+    def _process(self) -> bool:
         """
         Queries the persistence layer for open trades and handles them,
         otherwise a new trade is created.
-        :param: nb_assets: the maximum number of pairs to be traded at the same time
         :return: True if one or more trades has been created or closed, False otherwise
         """
         state_changed = False
         try:
+            nb_assets = self.config.get('dynamic_whitelist', None)
             # Refresh whitelist based on wallet maintenance
             sanitized_list = self._refresh_whitelist(
                 self._gen_pair_whitelist(
@@ -186,8 +184,7 @@ class FreqtradeBot(object):
             )
 
             # Keep only the subsets of pairs wanted (up to nb_assets)
-            final_list = sanitized_list[:nb_assets] if nb_assets else sanitized_list
+            self.active_pair_whitelist = sanitized_list[:nb_assets] if nb_assets else sanitized_list
-            self.config['exchange']['pair_whitelist'] = final_list
 
             # Calculating Edge positiong
             # Should be called before refresh_tickers
@@ -197,11 +194,20 @@ class FreqtradeBot(object):
                 self.edge.calculate()
 
             # Refreshing candles
-            self.exchange.refresh_tickers(final_list, self.strategy.ticker_interval)
+            self.exchange.refresh_tickers(self.active_pair_whitelist, self.strategy.ticker_interval)
+
 
             # Query trades from persistence layer
             trades = Trade.query.filter(Trade.is_open.is_(True)).all()
 
+            # Extend active-pair whitelist with pairs from open trades
+            # ensures that tickers are downloaded for open trades
+            self.active_pair_whitelist.extend([trade.pair for trade in trades
+                                               if trade.pair not in self.active_pair_whitelist])
+
+            # Refreshing candles
+            self.exchange.refresh_tickers(self.active_pair_whitelist, self.strategy.ticker_interval)
+
             # First process current opened trades
             for trade in trades:
                 state_changed |= self.process_maybe_execute_sell(trade)
@@ -389,7 +395,7 @@ class FreqtradeBot(object):
         :return: True if a trade object has been created and persisted, False otherwise
         """
         interval = self.strategy.ticker_interval
-        whitelist = copy.deepcopy(self.config['exchange']['pair_whitelist'])
+        whitelist = copy.deepcopy(self.active_pair_whitelist)
 
         # Remove currently opened and latest pairs from whitelist
         for trade in Trade.query.filter(Trade.is_open.is_(True)).all():

freqtrade/optimize/__init__.py

@@ -10,8 +10,12 @@ except ImportError:
     _UJSON = False
 import logging
 import os
+from datetime import datetime
 from typing import Optional, List, Dict, Tuple, Any
+import operator
+
 import arrow
+from pandas import DataFrame
 
 from freqtrade import misc, constants, OperationalException
 from freqtrade.exchange import Exchange
@@ -59,6 +63,42 @@ def trim_tickerlist(tickerlist: List[Dict], timerange: TimeRange) -> List[Dict]:
     return tickerlist[start_index:stop_index]
 
 
+def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
+    """
+    Get the maximum timeframe for the given backtest data
+    :param data: dictionary with preprocessed backtesting data
+    :return: tuple containing min_date, max_date
+    """
+    timeframe = [
+        (arrow.get(frame['date'].min()), arrow.get(frame['date'].max()))
+        for frame in data.values()
+    ]
+    return min(timeframe, key=operator.itemgetter(0))[0], \
+        max(timeframe, key=operator.itemgetter(1))[1]
+
+
+def validate_backtest_data(data: Dict[str, DataFrame], min_date: datetime,
+                           max_date: datetime, ticker_interval_mins: int) -> bool:
+    """
+    Validates preprocessed backtesting data for missing values and shows warnings about it that.
+
+    :param data: dictionary with preprocessed backtesting data
+    :param min_date: start-date of the data
+    :param max_date: end-date of the data
+    :param ticker_interval_mins: ticker interval in minutes
+    """
+    # total difference in minutes / interval-minutes
+    expected_frames = int((max_date - min_date).total_seconds() // 60 // ticker_interval_mins)
+    found_missing = False
+    for pair, df in data.items():
+        dflen = len(df)
+        if dflen < expected_frames:
+            found_missing = True
+            logger.warning("%s has missing frames: expected %s, got %s, that's %s missing values",
+                           pair, expected_frames, dflen, expected_frames - dflen)
+    return found_missing
+
+
 def load_tickerdata_file(
         datadir: str, pair: str,
         ticker_interval: str,
@@ -113,6 +153,14 @@ def load_data(datadir: str,
     for pair in pairs:
         pairdata = load_tickerdata_file(datadir, pair, ticker_interval, timerange=timerange)
         if pairdata:
+            if timerange.starttype == 'date' and pairdata[0][0] > timerange.startts * 1000:
+                logger.warning('Missing data at start for pair %s, data starts at %s',
+                               pair,
+                               arrow.get(pairdata[0][0] // 1000).strftime('%Y-%m-%d %H:%M:%S'))
+            if timerange.stoptype == 'date' and pairdata[-1][0] < timerange.stopts * 1000:
+                logger.warning('Missing data at end for pair %s, data ends at %s',
+                               pair,
+                               arrow.get(pairdata[-1][0] // 1000).strftime('%Y-%m-%d %H:%M:%S'))
             result[pair] = pairdata
         else:
             logger.warning(

freqtrade/optimize/backtesting.py

@@ -4,14 +4,12 @@
 This module contains the backtesting logic
 """
 import logging
-import operator
 from argparse import Namespace
 from copy import deepcopy
 from datetime import datetime, timedelta
 from pathlib import Path
-from typing import Any, Dict, List, NamedTuple, Optional, Tuple
+from typing import Any, Dict, List, NamedTuple, Optional
 
-import arrow
 from pandas import DataFrame
 from tabulate import tabulate
 
@@ -88,24 +86,9 @@ class Backtesting(object):
         """
         self.strategy = strategy
         self.ticker_interval = self.config.get('ticker_interval')
-        self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe
         self.advise_buy = strategy.advise_buy
         self.advise_sell = strategy.advise_sell
 
-    @staticmethod
-    def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
-        """
-        Get the maximum timeframe for the given backtest data
-        :param data: dictionary with preprocessed backtesting data
-        :return: tuple containing min_date, max_date
-        """
-        timeframe = [
-            (arrow.get(frame['date'].min()), arrow.get(frame['date'].max()))
-            for frame in data.values()
-        ]
-        return min(timeframe, key=operator.itemgetter(0))[0], \
-            max(timeframe, key=operator.itemgetter(1))[1]
-
     def _generate_text_table(self, data: Dict[str, Dict], results: DataFrame,
                              skip_nan: bool = False) -> str:
         """
@@ -371,10 +354,12 @@ class Backtesting(object):
             self._set_strategy(strat)
 
             # need to reprocess data every time to populate signals
-            preprocessed = self.tickerdata_to_dataframe(data)
+            preprocessed = self.strategy.tickerdata_to_dataframe(data)
 
-            # Print timeframe
+            min_date, max_date = optimize.get_timeframe(preprocessed)
-            min_date, max_date = self.get_timeframe(preprocessed)
+            # Validate dataframe for missing values
+            optimize.validate_backtest_data(preprocessed, min_date, max_date,
+                                            constants.TICKER_INTERVAL_MINUTES[self.ticker_interval])
             logger.info(
                 'Measuring data from %s up to %s (%s days)..',
                 min_date.isoformat(),

freqtrade/optimize/hyperopt.py

@@ -352,7 +352,7 @@ class Hyperopt(Backtesting):
 
         if self.has_space('buy'):
             self.strategy.advise_indicators = Hyperopt.populate_indicators  # type: ignore
-        dump(self.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
+        dump(self.strategy.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
         self.exchange = None  # type: ignore
         self.load_previous_results()
 

freqtrade/rpc/rpc.py

@@ -10,10 +10,10 @@ from typing import Dict, Any, List, Optional
 
 import arrow
 import sqlalchemy as sql
-from numpy import mean, nan_to_num
+from numpy import mean, nan_to_num, NAN
 from pandas import DataFrame
 
-from freqtrade import TemporaryError
+from freqtrade import TemporaryError, DependencyException
 from freqtrade.fiat_convert import CryptoToFiatConverter
 from freqtrade.misc import shorten_date
 from freqtrade.persistence import Trade
@@ -93,7 +93,10 @@ class RPC(object):
                 if trade.open_order_id:
                     order = self._freqtrade.exchange.get_order(trade.open_order_id, trade.pair)
                 # calculate profit and send message to user
-                current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                try:
+                    current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                except DependencyException:
+                    current_rate = NAN
                 current_profit = trade.calc_profit_percent(current_rate)
                 fmt_close_profit = (f'{round(trade.close_profit * 100, 2):.2f}%'
                                     if trade.close_profit else None)
@@ -122,7 +125,10 @@ class RPC(object):
             trades_list = []
             for trade in trades:
                 # calculate profit and send message to user
-                current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                try:
+                    current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                except DependencyException:
+                    current_rate = NAN
                 trade_perc = (100 * trade.calc_profit_percent(current_rate))
                 trades_list.append([
                     trade.id,
@@ -207,7 +213,10 @@ class RPC(object):
                 profit_closed_percent.append(profit_percent)
             else:
                 # Get current rate
-                current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                try:
+                    current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
+                except DependencyException:
+                    current_rate = NAN
                 profit_percent = trade.calc_profit_percent(rate=current_rate)
 
             profit_all_coin.append(
@@ -275,7 +284,7 @@ class RPC(object):
                         rate = 1.0 / self._freqtrade.exchange.get_ticker('BTC/USDT', False)['bid']
                     else:
                         rate = self._freqtrade.exchange.get_ticker(coin + '/BTC', False)['bid']
-                except TemporaryError:
+                except (TemporaryError, DependencyException):
                     continue
             est_btc: float = rate * balance['total']
             total = total + est_btc

freqtrade/tests/exchange/test_exchange.py

@@ -572,6 +572,7 @@ def test_get_ticker(default_conf, mocker):
         'last': 0.0001,
     }
     api_mock.fetch_ticker = MagicMock(return_value=tick)
+    api_mock.markets = {'ETH/BTC': {}}
     exchange = get_patched_exchange(mocker, default_conf, api_mock)
     # retrieve original ticker
     ticker = exchange.get_ticker(pair='ETH/BTC')
@@ -614,6 +615,9 @@ def test_get_ticker(default_conf, mocker):
     exchange = get_patched_exchange(mocker, default_conf, api_mock)
     exchange.get_ticker(pair='ETH/BTC', refresh=True)
 
+    with pytest.raises(DependencyException, match=r'Pair XRP/ETH not available'):
+        exchange.get_ticker(pair='XRP/ETH', refresh=True)
+
 
 def test_get_history(default_conf, mocker, caplog):
     exchange = get_patched_exchange(mocker, default_conf)

freqtrade/tests/optimize/test_backtesting.py

@@ -89,7 +89,7 @@ def simple_backtest(config, contour, num_results, mocker) -> None:
     backtesting = Backtesting(config)
 
     data = load_data_test(contour)
-    processed = backtesting.tickerdata_to_dataframe(data)
+    processed = backtesting.strategy.tickerdata_to_dataframe(data)
     assert isinstance(processed, dict)
     results = backtesting.backtest(
         {
@@ -119,13 +119,13 @@ def _load_pair_as_ticks(pair, tickfreq):
 
 # FIX: fixturize this?
 def _make_backtest_conf(mocker, conf=None, pair='UNITTEST/BTC', record=None):
-    data = optimize.load_data(None, ticker_interval='8m', pairs=[pair])
+    data = optimize.load_data(None, ticker_interval='1m', pairs=[pair])
     data = trim_dictlist(data, -201)
     patch_exchange(mocker)
     backtesting = Backtesting(conf)
     return {
         'stake_amount': conf['stake_amount'],
-        'processed': backtesting.tickerdata_to_dataframe(data),
+        'processed': backtesting.strategy.tickerdata_to_dataframe(data),
         'max_open_trades': 10,
         'position_stacking': False,
         'record': record
@@ -313,7 +313,7 @@ def test_backtesting_init(mocker, default_conf) -> None:
     backtesting = Backtesting(default_conf)
     assert backtesting.config == default_conf
     assert backtesting.ticker_interval == '5m'
-    assert callable(backtesting.tickerdata_to_dataframe)
+    assert callable(backtesting.strategy.tickerdata_to_dataframe)
     assert callable(backtesting.advise_buy)
     assert callable(backtesting.advise_sell)
     get_fee.assert_called()
@@ -327,7 +327,7 @@ def test_tickerdata_to_dataframe(default_conf, mocker) -> None:
     tickerlist = {'UNITTEST/BTC': tick}
 
     backtesting = Backtesting(default_conf)
-    data = backtesting.tickerdata_to_dataframe(tickerlist)
+    data = backtesting.strategy.tickerdata_to_dataframe(tickerlist)
     assert len(data['UNITTEST/BTC']) == 99
 
     # Load strategy to compare the result between Backtesting function and strategy are the same
@@ -336,22 +336,6 @@ def test_tickerdata_to_dataframe(default_conf, mocker) -> None:
     assert data['UNITTEST/BTC'].equals(data2['UNITTEST/BTC'])
 
 
-def test_get_timeframe(default_conf, mocker) -> None:
-    patch_exchange(mocker)
-    backtesting = Backtesting(default_conf)
-
-    data = backtesting.tickerdata_to_dataframe(
-        optimize.load_data(
-            None,
-            ticker_interval='1m',
-            pairs=['UNITTEST/BTC']
-        )
-    )
-    min_date, max_date = backtesting.get_timeframe(data)
-    assert min_date.isoformat() == '2017-11-04T23:02:00+00:00'
-    assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
-
-
 def test_generate_text_table(default_conf, mocker):
     patch_exchange(mocker)
     backtesting = Backtesting(default_conf)
@@ -451,21 +435,21 @@ def test_generate_text_table_strategyn(default_conf, mocker):
 
 
 def test_backtesting_start(default_conf, mocker, caplog) -> None:
-    def get_timeframe(input1, input2):
+    def get_timeframe(input1):
         return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
 
     mocker.patch('freqtrade.optimize.load_data', mocked_load_data)
+    mocker.patch('freqtrade.optimize.get_timeframe', get_timeframe)
     mocker.patch('freqtrade.exchange.Exchange.refresh_tickers', MagicMock())
     patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.optimize.backtesting.Backtesting',
         backtest=MagicMock(),
         _generate_text_table=MagicMock(return_value='1'),
-        get_timeframe=get_timeframe,
     )
 
     default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
-    default_conf['ticker_interval'] = 1
+    default_conf['ticker_interval'] = "1m"
     default_conf['live'] = False
     default_conf['datadir'] = None
     default_conf['export'] = None
@@ -486,17 +470,17 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
 
 
 def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
-    def get_timeframe(input1, input2):
+    def get_timeframe(input1):
         return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
 
     mocker.patch('freqtrade.optimize.load_data', MagicMock(return_value={}))
+    mocker.patch('freqtrade.optimize.get_timeframe', get_timeframe)
     mocker.patch('freqtrade.exchange.Exchange.refresh_tickers', MagicMock())
     patch_exchange(mocker)
     mocker.patch.multiple(
         'freqtrade.optimize.backtesting.Backtesting',
         backtest=MagicMock(),
         _generate_text_table=MagicMock(return_value='1'),
-        get_timeframe=get_timeframe,
     )
 
     default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
@@ -520,7 +504,7 @@ def test_backtest(default_conf, fee, mocker) -> None:
     pair = 'UNITTEST/BTC'
     data = optimize.load_data(None, ticker_interval='5m', pairs=['UNITTEST/BTC'])
     data = trim_dictlist(data, -200)
-    data_processed = backtesting.tickerdata_to_dataframe(data)
+    data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
     results = backtesting.backtest(
         {
             'stake_amount': default_conf['stake_amount'],
@@ -571,7 +555,7 @@ def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None:
     results = backtesting.backtest(
         {
             'stake_amount': default_conf['stake_amount'],
-            'processed': backtesting.tickerdata_to_dataframe(data),
+            'processed': backtesting.strategy.tickerdata_to_dataframe(data),
             'max_open_trades': 1,
             'position_stacking': False
         }
@@ -585,7 +569,7 @@ def test_processed(default_conf, mocker) -> None:
     backtesting = Backtesting(default_conf)
 
     dict_of_tickerrows = load_data_test('raise')
-    dataframes = backtesting.tickerdata_to_dataframe(dict_of_tickerrows)
+    dataframes = backtesting.strategy.tickerdata_to_dataframe(dict_of_tickerrows)
     dataframe = dataframes['UNITTEST/BTC']
     cols = dataframe.columns
     # assert the dataframe got some of the indicator columns

freqtrade/tests/optimize/test_hyperopt.py

@@ -194,7 +194,7 @@ def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
     default_conf.update({'spaces': 'all'})
 
     hyperopt = Hyperopt(default_conf)
-    hyperopt.tickerdata_to_dataframe = MagicMock()
+    hyperopt.strategy.tickerdata_to_dataframe = MagicMock()
 
     hyperopt.start()
     parallel.assert_called_once()
@@ -242,7 +242,7 @@ def test_has_space(hyperopt):
 def test_populate_indicators(hyperopt) -> None:
     tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
     tickerlist = {'UNITTEST/BTC': tick}
-    dataframes = hyperopt.tickerdata_to_dataframe(tickerlist)
+    dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
     dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
 
     # Check if some indicators are generated. We will not test all of them
@@ -254,7 +254,7 @@ def test_populate_indicators(hyperopt) -> None:
 def test_buy_strategy_generator(hyperopt) -> None:
     tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
     tickerlist = {'UNITTEST/BTC': tick}
-    dataframes = hyperopt.tickerdata_to_dataframe(tickerlist)
+    dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
     dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
 
     populate_buy_trend = hyperopt.buy_strategy_generator(

freqtrade/tests/optimize/test_optimize.py

@@ -7,7 +7,7 @@ from shutil import copyfile
 
 import arrow
 
-from freqtrade import optimize
+from freqtrade import optimize, constants
 from freqtrade.arguments import TimeRange
 from freqtrade.misc import file_dump_json
 from freqtrade.optimize.__init__ import (download_backtesting_testdata,
@@ -15,7 +15,8 @@ from freqtrade.optimize.__init__ import (download_backtesting_testdata,
                                          load_cached_data_for_updating,
                                          load_tickerdata_file,
                                          make_testdata_path, trim_tickerlist)
-from freqtrade.tests.conftest import get_patched_exchange, log_has
+from freqtrade.strategy.default_strategy import DefaultStrategy
+from freqtrade.tests.conftest import get_patched_exchange, log_has, patch_exchange
 
 # Change this if modifying UNITTEST/BTC testdatafile
 _BTC_UNITTEST_LENGTH = 13681
@@ -322,6 +323,38 @@ def test_load_tickerdata_file() -> None:
     assert _BTC_UNITTEST_LENGTH == len(tickerdata)
 
 
+def test_load_partial_missing(caplog) -> None:
+    # Make sure we start fresh - test missing data at start
+    start = arrow.get('2018-01-01T00:00:00')
+    end = arrow.get('2018-01-11T00:00:00')
+    tickerdata = optimize.load_data(None, '5m', ['UNITTEST/BTC'],
+                                    refresh_pairs=False,
+                                    timerange=TimeRange('date', 'date',
+                                                        start.timestamp, end.timestamp))
+    # timedifference in 5 minutes
+    td = ((end - start).total_seconds() // 60 // 5) + 1
+    assert td != len(tickerdata['UNITTEST/BTC'])
+    start_real = arrow.get(tickerdata['UNITTEST/BTC'][0][0] / 1000)
+    assert log_has(f'Missing data at start for pair '
+                   f'UNITTEST/BTC, data starts at {start_real.strftime("%Y-%m-%d %H:%M:%S")}',
+                   caplog.record_tuples)
+    # Make sure we start fresh - test missing data at end
+    caplog.clear()
+    start = arrow.get('2018-01-10T00:00:00')
+    end = arrow.get('2018-02-20T00:00:00')
+    tickerdata = optimize.load_data(None, '5m', ['UNITTEST/BTC'],
+                                    refresh_pairs=False,
+                                    timerange=TimeRange('date', 'date',
+                                                        start.timestamp, end.timestamp))
+    # timedifference in 5 minutes
+    td = ((end - start).total_seconds() // 60 // 5) + 1
+    assert td != len(tickerdata['UNITTEST/BTC'])
+    end_real = arrow.get(tickerdata['UNITTEST/BTC'][-1][0] / 1000)
+    assert log_has(f'Missing data at end for pair '
+                   f'UNITTEST/BTC, data ends at {end_real.strftime("%Y-%m-%d %H:%M:%S")}',
+                   caplog.record_tuples)
+
+
 def test_init(default_conf, mocker) -> None:
     exchange = get_patched_exchange(mocker, default_conf)
     assert {} == optimize.load_data(
@@ -433,3 +466,61 @@ def test_file_dump_json() -> None:
 
     # Remove the file
     _clean_test_file(file)
+
+
+def test_get_timeframe(default_conf, mocker) -> None:
+    patch_exchange(mocker)
+    strategy = DefaultStrategy(default_conf)
+
+    data = strategy.tickerdata_to_dataframe(
+        optimize.load_data(
+            None,
+            ticker_interval='1m',
+            pairs=['UNITTEST/BTC']
+        )
+    )
+    min_date, max_date = optimize.get_timeframe(data)
+    assert min_date.isoformat() == '2017-11-04T23:02:00+00:00'
+    assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
+
+
+def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None:
+    patch_exchange(mocker)
+    strategy = DefaultStrategy(default_conf)
+
+    data = strategy.tickerdata_to_dataframe(
+        optimize.load_data(
+            None,
+            ticker_interval='1m',
+            pairs=['UNITTEST/BTC']
+        )
+    )
+    min_date, max_date = optimize.get_timeframe(data)
+    caplog.clear()
+    assert optimize.validate_backtest_data(data, min_date, max_date,
+                                           constants.TICKER_INTERVAL_MINUTES["1m"])
+    assert len(caplog.record_tuples) == 1
+    assert log_has(
+        "UNITTEST/BTC has missing frames: expected 14396, got 13680, that's 716 missing values",
+        caplog.record_tuples)
+
+
+def test_validate_backtest_data(default_conf, mocker, caplog) -> None:
+    patch_exchange(mocker)
+    strategy = DefaultStrategy(default_conf)
+
+    timerange = TimeRange('index', 'index', 200, 250)
+    data = strategy.tickerdata_to_dataframe(
+        optimize.load_data(
+            None,
+            ticker_interval='5m',
+            pairs=['UNITTEST/BTC'],
+            timerange=timerange
+        )
+    )
+
+    min_date, max_date = optimize.get_timeframe(data)
+    caplog.clear()
+    assert not optimize.validate_backtest_data(data, min_date, max_date,
+                                               constants.TICKER_INTERVAL_MINUTES["5m"])
+    assert len(caplog.record_tuples) == 0

freqtrade/tests/rpc/test_rpc.py

@@ -5,8 +5,9 @@ from datetime import datetime
 from unittest.mock import MagicMock, ANY
 
 import pytest
+from numpy import isnan
 
-from freqtrade import TemporaryError
+from freqtrade import TemporaryError, DependencyException
 from freqtrade.fiat_convert import CryptoToFiatConverter
 from freqtrade.freqtradebot import FreqtradeBot
 from freqtrade.persistence import Trade
@@ -61,6 +62,27 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
         'open_order': '(limit buy rem=0.00000000)'
     } == results[0]
 
+    mocker.patch('freqtrade.exchange.Exchange.get_ticker',
+                 MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
+    # invalidate ticker cache
+    rpc._freqtrade.exchange._cached_ticker = {}
+    results = rpc._rpc_trade_status()
+    assert isnan(results[0]['current_profit'])
+    assert isnan(results[0]['current_rate'])
+    assert {
+        'trade_id': 1,
+        'pair': 'ETH/BTC',
+        'market_url': 'https://bittrex.com/Market/Index?MarketName=BTC-ETH',
+        'date': ANY,
+        'open_rate': 1.099e-05,
+        'close_rate': None,
+        'current_rate': ANY,
+        'amount': 90.99181074,
+        'close_profit': None,
+        'current_profit': ANY,
+        'open_order': '(limit buy rem=0.00000000)'
+    } == results[0]
+
 
 def test_rpc_status_table(default_conf, ticker, fee, markets, mocker) -> None:
     patch_coinmarketcap(mocker)
@@ -87,6 +109,15 @@ def test_rpc_status_table(default_conf, ticker, fee, markets, mocker) -> None:
     assert 'ETH/BTC' in result['Pair'].all()
     assert '-0.59%' in result['Profit'].all()
 
+    mocker.patch('freqtrade.exchange.Exchange.get_ticker',
+                 MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
+    # invalidate ticker cache
+    rpc._freqtrade.exchange._cached_ticker = {}
+    result = rpc._rpc_status_table()
+    assert 'just now' in result['Since'].all()
+    assert 'ETH/BTC' in result['Pair'].all()
+    assert 'nan%' in result['Profit'].all()
+
 
 def test_rpc_daily_profit(default_conf, update, ticker, fee,
                           limit_buy_order, limit_sell_order, markets, mocker) -> None:
@@ -208,6 +239,20 @@ def test_rpc_trade_statistics(default_conf, ticker, ticker_sell_up, fee,
     assert stats['best_pair'] == 'ETH/BTC'
     assert prec_satoshi(stats['best_rate'], 6.2)
 
+    # Test non-available pair
+    mocker.patch('freqtrade.exchange.Exchange.get_ticker',
+                 MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
+    # invalidate ticker cache
+    rpc._freqtrade.exchange._cached_ticker = {}
+    stats = rpc._rpc_trade_statistics(stake_currency, fiat_display_currency)
+    assert stats['trade_count'] == 2
+    assert stats['first_trade_date'] == 'just now'
+    assert stats['latest_trade_date'] == 'just now'
+    assert stats['avg_duration'] == '0:00:00'
+    assert stats['best_pair'] == 'ETH/BTC'
+    assert prec_satoshi(stats['best_rate'], 6.2)
+    assert isnan(stats['profit_all_coin'])
+
 
 # Test that rpc_trade_statistics can handle trades that lacks
 # trade.open_rate (it is set to None)

freqtrade/tests/test_freqtradebot.py

@@ -764,6 +764,52 @@ def test_process_trade_handling(
     assert result is False
 
 
+def test_process_trade_no_whitelist_pair(
+        default_conf, ticker, limit_buy_order, markets, fee, mocker) -> None:
+    """ Test _process with trade not in pair list """
+    patch_RPCManager(mocker)
+    patch_exchange(mocker)
+    mocker.patch.multiple(
+        'freqtrade.exchange.Exchange',
+        get_ticker=ticker,
+        get_markets=markets,
+        buy=MagicMock(return_value={'id': limit_buy_order['id']}),
+        get_order=MagicMock(return_value=limit_buy_order),
+        get_fee=fee,
+    )
+    freqtrade = FreqtradeBot(default_conf)
+    patch_get_signal(freqtrade)
+    pair = 'NOCLUE/BTC'
+    # create open trade not in whitelist
+    Trade.session.add(Trade(
+        pair=pair,
+        stake_amount=0.001,
+        fee_open=fee.return_value,
+        fee_close=fee.return_value,
+        is_open=True,
+        amount=20,
+        open_rate=0.01,
+        exchange='bittrex',
+    ))
+    Trade.session.add(Trade(
+        pair='ETH/BTC',
+        stake_amount=0.001,
+        fee_open=fee.return_value,
+        fee_close=fee.return_value,
+        is_open=True,
+        amount=12,
+        open_rate=0.001,
+        exchange='bittrex',
+    ))
+
+    assert pair not in freqtrade.active_pair_whitelist
+    result = freqtrade._process()
+    assert pair in freqtrade.active_pair_whitelist
+    # Make sure each pair is only in the list once
+    assert len(freqtrade.active_pair_whitelist) == len(set(freqtrade.active_pair_whitelist))
+    assert result is True
+
+
 def test_balance_fully_ask_side(mocker, default_conf) -> None:
     default_conf['bid_strategy']['ask_last_balance'] = 0.0
     freqtrade = get_patched_freqtradebot(mocker, default_conf)

requirements.txt

@@ -1,5 +1,5 @@
-ccxt==1.17.411
+ccxt==1.17.455
-SQLAlchemy==1.2.12
+SQLAlchemy==1.2.13
 python-telegram-bot==11.1.0
 arrow==0.12.1
 cachetools==2.1.0
@@ -12,7 +12,7 @@ scipy==1.1.0
 jsonschema==2.6.0
 numpy==1.15.3
 TA-Lib==0.4.17
-pytest==3.9.2
+pytest==3.9.3
 pytest-mock==1.10.0
 pytest-asyncio==0.9.0
 pytest-cov==2.6.0

scripts/plot_dataframe.py

@@ -1,381 +0,0 @@
-#!/usr/bin/env python3
-"""
-Script to display when the bot will buy a specific pair
-
-Mandatory Cli parameters:
--p / --pair: pair to examine
-
-Option but recommended
--s / --strategy: strategy to use
-
-
-Optional Cli parameters
--d / --datadir: path to pair backtest data
---timerange: specify what timerange of data to use.
--l / --live: Live, to download the latest ticker for the pair
--db / --db-url: Show trades stored in database
-
-
-Indicators recommended
-Row 1: sma, ema3, ema5, ema10, ema50
-Row 3: macd, rsi, fisher_rsi, mfi, slowd, slowk, fastd, fastk
-
-Example of usage:
-> python3 scripts/plot_dataframe.py --pair BTC/EUR -d user_data/data/ --indicators1 sma,ema3
---indicators2 fastk,fastd
-"""
-import json
-import logging
-import sys
-from argparse import Namespace
-from pathlib import Path
-from typing import Dict, List, Any
-
-import pandas as pd
-import plotly.graph_objs as go
-import pytz
-
-from plotly import tools
-from plotly.offline import plot
-
-import freqtrade.optimize as optimize
-from freqtrade import persistence
-from freqtrade.arguments import Arguments, TimeRange
-from freqtrade.exchange import Exchange
-from freqtrade.optimize.backtesting import setup_configuration
-from freqtrade.persistence import Trade
-from freqtrade.strategy.resolver import StrategyResolver
-
-logger = logging.getLogger(__name__)
-_CONF: Dict[str, Any] = {}
-
-timeZone = pytz.UTC
-
-
-def load_trades(args: Namespace, pair: str, timerange: TimeRange) -> pd.DataFrame:
-    trades: pd.DataFrame = pd.DataFrame()
-    if args.db_url:
-        persistence.init(_CONF)
-        columns = ["pair", "profit", "opents", "closets", "open_rate", "close_rate", "duration"]
-
-        for x in Trade.query.all():
-            print("date: {}".format(x.open_date))
-
-        trades = pd.DataFrame([(t.pair, t.calc_profit(),
-                                t.open_date.replace(tzinfo=timeZone),
-                                t.close_date.replace(tzinfo=timeZone) if t.close_date else None,
-                                t.open_rate, t.close_rate,
-                                t.close_date.timestamp() - t.open_date.timestamp() if t.close_date else None)
-                               for t in Trade.query.filter(Trade.pair.is_(pair)).all()],
-                              columns=columns)
-
-    elif args.exportfilename:
-        file = Path(args.exportfilename)
-        # must align with columns in backtest.py
-        columns = ["pair", "profit", "opents", "closets", "index", "duration",
-                   "open_rate", "close_rate", "open_at_end", "sell_reason"]
-        with file.open() as f:
-            data = json.load(f)
-            trades = pd.DataFrame(data, columns=columns)
-        trades = trades.loc[trades["pair"] == pair]
-        if timerange:
-            if timerange.starttype == 'date':
-                trades = trades.loc[trades["opents"] >= timerange.startts]
-            if timerange.stoptype == 'date':
-                trades = trades.loc[trades["opents"] <= timerange.stopts]
-
-        trades['opents'] = pd.to_datetime(trades['opents'],
-                                          unit='s',
-                                          utc=True,
-                                          infer_datetime_format=True)
-        trades['closets'] = pd.to_datetime(trades['closets'],
-                                           unit='s',
-                                           utc=True,
-                                           infer_datetime_format=True)
-    return trades
-
-
-def plot_analyzed_dataframe(args: Namespace) -> None:
-    """
-    Calls analyze() and plots the returned dataframe
-    :return: None
-    """
-    global _CONF
-
-    # Load the configuration
-    _CONF.update(setup_configuration(args))
-
-    print(_CONF)
-    # Set the pair to audit
-    pair = args.pair
-
-    if pair is None:
-        logger.critical('Parameter --pair mandatory;. E.g --pair ETH/BTC')
-        exit()
-
-    if '/' not in pair:
-        logger.critical('--pair format must be XXX/YYY')
-        exit()
-
-    # Set timerange to use
-    timerange = Arguments.parse_timerange(args.timerange)
-
-    # Load the strategy
-    try:
-        strategy = StrategyResolver(_CONF).strategy
-        exchange = Exchange(_CONF)
-    except AttributeError:
-        logger.critical(
-            'Impossible to load the strategy. Please check the file "user_data/strategies/%s.py"',
-            args.strategy
-        )
-        exit()
-
-    # Set the ticker to use
-    tick_interval = strategy.ticker_interval
-
-    # Load pair tickers
-    tickers = {}
-    if args.live:
-        logger.info('Downloading pair.')
-        exchange.refresh_tickers([pair], tick_interval)
-        tickers[pair] = exchange.klines[pair]
-    else:
-        tickers = optimize.load_data(
-            datadir=_CONF.get("datadir"),
-            pairs=[pair],
-            ticker_interval=tick_interval,
-            refresh_pairs=_CONF.get('refresh_pairs', False),
-            timerange=timerange,
-            exchange=Exchange(_CONF)
-        )
-
-        # No ticker found, or impossible to download
-        if tickers == {}:
-            exit()
-
-    # Get trades already made from the DB
-    trades = load_trades(args, pair, timerange)
-
-    dataframes = strategy.tickerdata_to_dataframe(tickers)
-
-    dataframe = dataframes[pair]
-    dataframe = strategy.advise_buy(dataframe, {'pair': pair})
-    dataframe = strategy.advise_sell(dataframe, {'pair': pair})
-
-    if len(dataframe.index) > args.plot_limit:
-        logger.warning('Ticker contained more than %s candles as defined '
-                       'with --plot-limit, clipping.', args.plot_limit)
-    dataframe = dataframe.tail(args.plot_limit)
-
-    trades = trades.loc[trades['opents'] >= dataframe.iloc[0]['date']]
-    fig = generate_graph(
-        pair=pair,
-        trades=trades,
-        data=dataframe,
-        args=args
-    )
-
-    plot(fig, filename=str(Path('user_data').joinpath('freqtrade-plot.html')))
-
-
-def generate_graph(pair, trades: pd.DataFrame, data: pd.DataFrame, args) -> tools.make_subplots:
-    """
-    Generate the graph from the data generated by Backtesting or from DB
-    :param pair: Pair to Display on the graph
-    :param trades: All trades created
-    :param data: Dataframe
-    :param args: sys.argv that contrains the two params indicators1, and indicators2
-    :return: None
-    """
-
-    # Define the graph
-    fig = tools.make_subplots(
-        rows=3,
-        cols=1,
-        shared_xaxes=True,
-        row_width=[1, 1, 4],
-        vertical_spacing=0.0001,
-    )
-    fig['layout'].update(title=pair)
-    fig['layout']['yaxis1'].update(title='Price')
-    fig['layout']['yaxis2'].update(title='Volume')
-    fig['layout']['yaxis3'].update(title='Other')
-
-    # Common information
-    candles = go.Candlestick(
-        x=data.date,
-        open=data.open,
-        high=data.high,
-        low=data.low,
-        close=data.close,
-        name='Price'
-    )
-
-    df_buy = data[data['buy'] == 1]
-    buys = go.Scattergl(
-        x=df_buy.date,
-        y=df_buy.close,
-        mode='markers',
-        name='buy',
-        marker=dict(
-            symbol='triangle-up-dot',
-            size=9,
-            line=dict(width=1),
-            color='green',
-        )
-    )
-    df_sell = data[data['sell'] == 1]
-    sells = go.Scattergl(
-        x=df_sell.date,
-        y=df_sell.close,
-        mode='markers',
-        name='sell',
-        marker=dict(
-            symbol='triangle-down-dot',
-            size=9,
-            line=dict(width=1),
-            color='red',
-        )
-    )
-
-    trade_buys = go.Scattergl(
-        x=trades["opents"],
-        y=trades["open_rate"],
-        mode='markers',
-        name='trade_buy',
-        marker=dict(
-            symbol='square-open',
-            size=11,
-            line=dict(width=2),
-            color='green'
-        )
-    )
-    trade_sells = go.Scattergl(
-        x=trades["closets"],
-        y=trades["close_rate"],
-        mode='markers',
-        name='trade_sell',
-        marker=dict(
-            symbol='square-open',
-            size=11,
-            line=dict(width=2),
-            color='red'
-        )
-    )
-
-    # Row 1
-    fig.append_trace(candles, 1, 1)
-
-    if 'bb_lowerband' in data and 'bb_upperband' in data:
-        bb_lower = go.Scatter(
-            x=data.date,
-            y=data.bb_lowerband,
-            name='BB lower',
-            line={'color': 'rgba(255,255,255,0)'},
-        )
-        bb_upper = go.Scatter(
-            x=data.date,
-            y=data.bb_upperband,
-            name='BB upper',
-            fill="tonexty",
-            fillcolor="rgba(0,176,246,0.2)",
-            line={'color': 'rgba(255,255,255,0)'},
-        )
-        fig.append_trace(bb_lower, 1, 1)
-        fig.append_trace(bb_upper, 1, 1)
-
-    fig = generate_row(fig=fig, row=1, raw_indicators=args.indicators1, data=data)
-    fig.append_trace(buys, 1, 1)
-    fig.append_trace(sells, 1, 1)
-    fig.append_trace(trade_buys, 1, 1)
-    fig.append_trace(trade_sells, 1, 1)
-
-    # Row 2
-    volume = go.Bar(
-        x=data['date'],
-        y=data['volume'],
-        name='Volume'
-    )
-    fig.append_trace(volume, 2, 1)
-
-    # Row 3
-    fig = generate_row(fig=fig, row=3, raw_indicators=args.indicators2, data=data)
-
-    return fig
-
-
-def generate_row(fig, row, raw_indicators, data) -> tools.make_subplots:
-    """
-    Generator all the indicator selected by the user for a specific row
-    """
-    for indicator in raw_indicators.split(','):
-        if indicator in data:
-            scattergl = go.Scattergl(
-                x=data['date'],
-                y=data[indicator],
-                name=indicator
-            )
-            fig.append_trace(scattergl, row, 1)
-        else:
-            logger.info(
-                'Indicator "%s" ignored. Reason: This indicator is not found '
-                'in your strategy.',
-                indicator
-            )
-
-    return fig
-
-
-def plot_parse_args(args: List[str]) -> Namespace:
-    """
-    Parse args passed to the script
-    :param args: Cli arguments
-    :return: args: Array with all arguments
-    """
-    arguments = Arguments(args, 'Graph dataframe')
-    arguments.scripts_options()
-    arguments.parser.add_argument(
-        '--indicators1',
-        help='Set indicators from your strategy you want in the first row of the graph. Separate '
-             'them with a coma. E.g: ema3,ema5 (default: %(default)s)',
-        type=str,
-        default='sma,ema3,ema5',
-        dest='indicators1',
-    )
-
-    arguments.parser.add_argument(
-        '--indicators2',
-        help='Set indicators from your strategy you want in the third row of the graph. Separate '
-             'them with a coma. E.g: fastd,fastk (default: %(default)s)',
-        type=str,
-        default='macd',
-        dest='indicators2',
-    )
-    arguments.parser.add_argument(
-        '--plot-limit',
-        help='Specify tick limit for plotting - too high values cause huge files - '
-             'Default: %(default)s',
-        dest='plot_limit',
-        default=750,
-        type=int,
-    )
-    arguments.common_args_parser()
-    arguments.optimizer_shared_options(arguments.parser)
-    arguments.backtesting_options(arguments.parser)
-    return arguments.parse_args()
-
-
-def main(sysargv: List[str]) -> None:
-    """
-    This function will initiate the bot and start the trading loop.
-    :return: None
-    """
-    logger.info('Starting Plot Dataframe')
-    plot_analyzed_dataframe(
-        plot_parse_args(sysargv)
-    )
-
-
-if __name__ == '__main__':
-    main(sys.argv[1:])