Merge pull request #1390 from freqtrade/feat/dynamic_provider

Dynamic Pairlist provider

CONTRIBUTING.md

@@ -72,6 +72,10 @@ pip3.6 install mypy
 mypy freqtrade
 ```
 
+## Getting started
+
+Best start by reading the [documentation](https://github.com/freqtrade/freqtrade/blob/develop/docs/index.md) to get a feel for what is possible with the bot, or head straight to the [Developer-documentation](https://github.com/freqtrade/freqtrade/blob/develop/docs/developer.md) (WIP) which should help you getting started.
+
 ## (Core)-Committer Guide
 
 ### Process: Pull Requests

config_full.json.example

@@ -39,6 +39,13 @@
         "stoploss": "market",
         "stoploss_on_exchange": "false"
     },
+    "pairlist": {
+        "method": "VolumePairList",
+        "config": {
+            "number_assets": 20,
+            "sort_key": "quoteVolume"
+        }
+    },
     "exchange": {
         "name": "bittrex",
         "key": "your_exchange_key",

docs/bot-usage.md

@@ -36,7 +36,7 @@ optional arguments:
   --strategy-path PATH  specify additional strategy lookup path
   --dynamic-whitelist [INT]
                         dynamically generate and update whitelist based on 24h
-                        BaseVolume (default: 20)
+                        BaseVolume (default: 20) DEPRECATED
   --db-url PATH         Override trades database URL, this is useful if
                         dry_run is enabled or in custom deployments (default:
                         sqlite:///tradesv3.sqlite)
@@ -89,6 +89,8 @@ This is very simple. Copy paste your strategy file into the folder
 
 ### How to use --dynamic-whitelist?
 
+> Dynamic-whitelist is deprecated. Please move your configurations to the configuration as outlined [here](docs/configuration.md#Dynamic-Pairlists)
+
 Per default `--dynamic-whitelist` will retrieve the 20 currencies based
 on BaseVolume. This value can be changed when you run the script.
 

docs/configuration.md

@@ -34,8 +34,8 @@ The table below will list all configuration parameters.
 | `bid_strategy.ask_last_balance` | 0.0 | Yes | Set the bidding price. More information below.
 | `bid_strategy.use_order_book` | false | No | Allows buying of pair using the rates in Order Book Bids.
 | `bid_strategy.order_book_top` | 0 | No | Bot will use the top N rate in Order Book Bids. Ie. a value of 2 will allow the bot to pick the 2nd bid rate in Order Book Bids.
-| `bid_strategy.check_depth_of_market.enabled` | false | No | Does not buy if the % difference of buy orders and sell orders is met in Order Book.
+| `bid_strategy. check_depth_of_market.enabled` | false | No | Does not buy if the % difference of buy orders and sell orders is met in Order Book.
-| `bid_strategy.check_depth_of_market.bids_to_ask_delta` | 0 | No | The % difference of buy orders and sell orders found in Order Book. A value lesser than 1 means sell orders is greater, while value greater than 1 means buy orders is higher.
+| `bid_strategy. check_depth_of_market.bids_to_ask_delta` | 0 | No | The % difference of buy orders and sell orders found in Order Book. A value lesser than 1 means sell orders is greater, while value greater than 1 means buy orders is higher.
 | `ask_strategy.use_order_book` | false | No | Allows selling of open traded pair using the rates in Order Book Asks.
 | `ask_strategy.order_book_min` | 0 | No | Bot will scan from the top min to max Order Book Asks searching for a profitable rate.
 | `ask_strategy.order_book_max` | 0 | No | Bot will scan from the top min to max Order Book Asks searching for a profitable rate.
@@ -52,6 +52,8 @@ The table below will list all configuration parameters.
 | `experimental.use_sell_signal` | false | No | Use your sell strategy in addition of the `minimal_roi`.
 | `experimental.sell_profit_only` | false | No | waits until you have made a positive profit before taking a sell decision.
 | `experimental.ignore_roi_if_buy_signal` | false | No | Does not sell if the buy-signal is still active. Takes preference over `minimal_roi` and `use_sell_signal`
+| `pairlist.method` | StaticPairList | No | Use Static whitelist. [More information below](#dynamic-pairlists).
+| `pairlist.config` | None | No | Additional configuration for dynamic pairlists. [More information below](#dynamic-pairlists).
 | `telegram.enabled` | true | Yes | Enable or not the usage of Telegram.
 | `telegram.token` | token | No | Your Telegram bot token. Only required if `telegram.enabled` is `true`.
 | `telegram.chat_id` | chat_id | No | Your personal Telegram account id. Only required if `telegram.enabled` is `true`.
@@ -147,7 +149,7 @@ This can be set in the configuration or in the strategy. Configuration overwrite
 If this is configured, all 4 values (`"buy"`, `"sell"`, `"stoploss"`, `"stoploss_on_exchange"`) need to be present, otherwise the bot warn about it and will fail to start.
 The below is the default which is used if this is not configured in either Strategy or configuration.
 
-``` json
+``` python
     "order_types": {
         "buy": "limit",
         "sell": "limit",
@@ -211,13 +213,38 @@ creating trades.
 Once you will be happy with your bot performance, you can switch it to
 production mode.
 
+### Dynamic Pairlists
+
+Dynamic pairlists select pairs for you based on the logic configured. 
+The bot runs against all pairs (with that stake) on the exchange, and a number of assets (`number_assets`) is selected based on the selected criteria.
+
+By *default*, a Static Pairlist is used (configured as `"pair_whitelist"` under the `"exchange"` section of this configuration).
+
+#### Available Pairlist methods
+
+* `"StaticPairList"`
+  * uses configuration from `exchange.pair_whitelist` and `exchange.pair_blacklist`
+* `"VolumePairList"`
+  * Formerly available as `--dynamic-whitelist [<number_assets>]`
+  * Selects `number_assets` top pairs based on `sort_key`, which can be one of `askVolume`, `bidVolume` and `quoteVolume`, defaults to `quoteVolume`.
+
+```json
+"pairlist": {
+        "method": "VolumePairList",
+        "config": {
+            "number_assets": 20,
+            "sort_key": "quoteVolume"
+        }
+    },
+```
+
 ## Switch to production mode
 
 In production mode, the bot will engage your money. Be careful a wrong
 strategy can lose all your money. Be aware of what you are doing when
 you run it in production mode.
 
-### To switch your bot in production mode:
+### To switch your bot in production mode
 
 1. Edit your `config.json`  file
 

docs/developer.md

@@ -0,0 +1,70 @@
+# Development Help
+
+This page is intended for developers of FreqTrade, people who want to contribute to the FreqTrade codebase or documentation, or people who want to understand the source code of the application they're running.
+
+All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE) where you can ask questions.
+
+
+## Module 
+
+### Dynamic Pairlist
+
+You have a great idea for a new pair selection algorithm you would like to try out? Great.
+Hopefully you also want to contribute this back upstream.
+
+Whatever your motivations are - This should get you off the ground in trying to develop a new Pairlist provider.
+
+First of all, have a look at the [VolumePairList](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/pairlist/VolumePairList.py) provider, and best copy this file with a name of your new Pairlist Provider.
+
+This is a simple provider, which however serves as a good example on how to start developing.
+
+Next, modify the classname of the provider (ideally align this with the Filename).
+
+The base-class provides the an instance of the bot (`self._freqtrade`), as well as the configuration (`self._config`), and initiates both `_blacklist` and `_whitelist`.
+
+```python
+        self._freqtrade = freqtrade
+        self._config = config
+        self._whitelist = self._config['exchange']['pair_whitelist']
+        self._blacklist = self._config['exchange'].get('pair_blacklist', [])
+```
+
+
+Now, let's step through the methods which require actions:
+
+#### configuration
+
+Configuration for PairListProvider is done in the bot configuration file in the element `"pairlist"`.
+This Pairlist-object may contain a `"config"` dict with additional configurations for the configured pairlist.
+By convention, `"number_assets"` is used to specify the maximum number of pairs to keep in the whitelist. Please follow this to ensure a consistent user experience.
+
+Additional elements can be configured as needed. `VolumePairList` uses `"sort_key"` to specify the sorting value - however feel free to specify whatever is necessary for your great algorithm to be successfull and dynamic.
+
+#### short_desc
+
+Returns a description used for Telegram messages.
+This should contain the name of the Provider, as well as a short description containing the number of assets. Please follow the format `"PairlistName - top/bottom X pairs"`.
+
+#### refresh_pairlist
+
+Override this method and run all calculations needed in this method.
+This is called with each iteration of the bot - so consider implementing caching for compute/network heavy calculations.
+
+Assign the resulting whiteslist to `self._whitelist` and `self._blacklist` respectively. These will then be used to run the bot in this iteration. Pairs with open trades will be added to the whitelist to have the sell-methods run correctly.
+
+Please also run `self._validate_whitelist(pairs)` and to check and remove pairs with inactive markets. This function is available in the Parent class (`StaticPairList`) and should ideally not be overwritten.
+
+##### sample
+
+``` python
+    def refresh_pairlist(self) -> None:
+        # Generate dynamic whitelist
+        pairs = self._gen_pair_whitelist(self._config['stake_currency'], self._sort_key)
+        # Validate whitelist to only have active market pairs
+        self._whitelist = self._validate_whitelist(pairs)[:self._number_pairs]
+```
+
+#### _gen_pair_whitelist
+
+This is a simple method used by `VolumePairList` - however serves as a good example.
+It implements caching (`@cached(TTLCache(maxsize=1, ttl=1800))`) as well as a configuration option to allow different (but similar) strategies to work with the same PairListProvider.

docs/index.md

@@ -35,4 +35,5 @@ Pull-request. Do not hesitate to reach us on
 	- [Run tests & Check PEP8 compliance](https://github.com/freqtrade/freqtrade/blob/develop/CONTRIBUTING.md)
 - [FAQ](https://github.com/freqtrade/freqtrade/blob/develop/docs/faq.md)
     - [SQL cheatsheet](https://github.com/freqtrade/freqtrade/blob/develop/docs/sql_cheatsheet.md)
-- [Sandbox Testing](https://github.com/freqtrade/freqtrade/blob/develop/docs/sandbox-testing.md))
+- [Sandbox Testing](https://github.com/freqtrade/freqtrade/blob/develop/docs/sandbox-testing.md)
+- [Developer Docs](https://github.com/freqtrade/freqtrade/blob/develop/docs/developer.md)

freqtrade/arguments.py

@@ -115,7 +115,8 @@ class Arguments(object):
         self.parser.add_argument(
             '--dynamic-whitelist',
             help='dynamically generate and update whitelist'
-                 ' based on 24h BaseVolume (default: %(const)s)',
+                 ' based on 24h BaseVolume (default: %(const)s)'
+                 ' DEPRECATED.',
             dest='dynamic_whitelist',
             const=constants.DYNAMIC_WHITELIST,
             type=int,

freqtrade/configuration.py

@@ -110,10 +110,14 @@ class Configuration(object):
 
         # Add dynamic_whitelist if found
         if 'dynamic_whitelist' in self.args and self.args.dynamic_whitelist:
-            config.update({'dynamic_whitelist': self.args.dynamic_whitelist})
+            # Update to volumePairList (the previous default)
-            logger.info(
+            config['pairlist'] = {'method': 'VolumePairList',
-                'Parameter --dynamic-whitelist detected. '
+                                  'config': {'number_assets': self.args.dynamic_whitelist}
-                'Using dynamically generated whitelist. '
+                                  }
+            logger.warning(
+                'Parameter --dynamic-whitelist has been deprecated, '
+                'and will be completely replaced by the whitelist dict in the future. '
+                'For now: using dynamically generated whitelist based on VolumePairList. '
                 '(not applicable with Backtesting and Hyperopt)'
             )
 

freqtrade/constants.py

@@ -15,7 +15,7 @@ DEFAULT_DB_DRYRUN_URL = 'sqlite://'
 UNLIMITED_STAKE_AMOUNT = 'unlimited'
 REQUIRED_ORDERTYPES = ['buy', 'sell', 'stoploss', 'stoploss_on_exchange']
 ORDERTYPE_POSSIBILITIES = ['limit', 'market']
-
+AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList']
 
 TICKER_INTERVAL_MINUTES = {
     '1m': 1,
@@ -124,6 +124,14 @@ CONF_SCHEMA = {
                 'ignore_roi_if_buy_signal_true': {'type': 'boolean'}
             }
         },
+        'pairlist': {
+            'type': 'object',
+            'properties': {
+                'method': {'type': 'string',  'enum': AVAILABLE_PAIRLISTS},
+                'config': {'type': 'object'}
+            },
+            'required': ['method']
+        },
         'telegram': {
             'type': 'object',
             'properties': {

freqtrade/freqtradebot.py

@@ -12,8 +12,6 @@ from typing import Any, Callable, Dict, List, Optional
 import arrow
 from requests.exceptions import RequestException
 
-from cachetools import TTLCache, cached
-
 from freqtrade import (DependencyException, OperationalException,
                        TemporaryError, __version__, constants, persistence)
 from freqtrade.exchange import Exchange
@@ -21,7 +19,7 @@ from freqtrade.wallets import Wallets
 from freqtrade.edge import Edge
 from freqtrade.persistence import Trade
 from freqtrade.rpc import RPCManager, RPCMessageType
-from freqtrade.resolvers import StrategyResolver
+from freqtrade.resolvers import StrategyResolver, PairListResolver
 from freqtrade.state import State
 from freqtrade.strategy.interface import SellType, IStrategy
 from freqtrade.exchange.exchange_helpers import order_book_to_dataframe
@@ -59,6 +57,8 @@ class FreqtradeBot(object):
         self.persistence = None
         self.exchange = Exchange(self.config)
         self.wallets = Wallets(self.exchange)
+        pairlistname = self.config.get('pairlist', {}).get('method', 'StaticPairList')
+        self.pairlists = PairListResolver(pairlistname, self, self.config).pairlist
 
         # Initializing Edge only if enabled
         self.edge = Edge(self.config, self.exchange, self.strategy) if \
@@ -108,7 +108,7 @@ class FreqtradeBot(object):
             })
             logger.info('Changing state to: %s', state.name)
             if state == State.RUNNING:
-                self.rpc.startup_messages(self.config)
+                self.rpc.startup_messages(self.config, self.pairlists)
 
         if state == State.STOPPED:
             time.sleep(1)
@@ -146,16 +146,9 @@ class FreqtradeBot(object):
         """
         state_changed = False
         try:
-            nb_assets = self.config.get('dynamic_whitelist', None)
+            # Refresh whitelist
-            # Refresh whitelist based on wallet maintenance
+            self.pairlists.refresh_pairlist()
-            sanitized_list = self._refresh_whitelist(
+            self.active_pair_whitelist = self.pairlists.whitelist
-                self._gen_pair_whitelist(
-                    self.config['stake_currency']
-                ) if nb_assets else self.config['exchange']['pair_whitelist']
-            )
-
-            # Keep only the subsets of pairs wanted (up to nb_assets)
-            self.active_pair_whitelist = sanitized_list[:nb_assets] if nb_assets else sanitized_list
 
             # Calculating Edge positiong
             # Should be called before refresh_tickers
@@ -203,63 +196,6 @@ class FreqtradeBot(object):
             self.state = State.STOPPED
         return state_changed
 
-    @cached(TTLCache(maxsize=1, ttl=1800))
-    def _gen_pair_whitelist(self, base_currency: str, key: str = 'quoteVolume') -> List[str]:
-        """
-        Updates the whitelist with with a dynamically generated list
-        :param base_currency: base currency as str
-        :param key: sort key (defaults to 'quoteVolume')
-        :return: List of pairs
-        """
-
-        if not self.exchange.exchange_has('fetchTickers'):
-            raise OperationalException(
-                'Exchange does not support dynamic whitelist.'
-                'Please edit your config and restart the bot'
-            )
-
-        tickers = self.exchange.get_tickers()
-        # check length so that we make sure that '/' is actually in the string
-        tickers = [v for k, v in tickers.items()
-                   if len(k.split('/')) == 2 and k.split('/')[1] == base_currency]
-
-        sorted_tickers = sorted(tickers, reverse=True, key=lambda t: t[key])
-        pairs = [s['symbol'] for s in sorted_tickers]
-        return pairs
-
-    def _refresh_whitelist(self, whitelist: List[str]) -> List[str]:
-        """
-        Check available markets and remove pair from whitelist if necessary
-        :param whitelist: the sorted list (based on BaseVolume) of pairs the user might want to
-        trade
-        :return: the list of pairs the user wants to trade without the one unavailable or
-        black_listed
-        """
-        sanitized_whitelist = whitelist
-        markets = self.exchange.get_markets()
-
-        markets = [m for m in markets if m['quote'] == self.config['stake_currency']]
-        known_pairs = set()
-        for market in markets:
-            pair = market['symbol']
-            # pair is not int the generated dynamic market, or in the blacklist ... ignore it
-            if pair not in whitelist or pair in self.config['exchange'].get('pair_blacklist', []):
-                continue
-            # else the pair is valid
-            known_pairs.add(pair)
-            # Market is not active
-            if not market['active']:
-                sanitized_whitelist.remove(pair)
-                logger.info(
-                    'Ignoring %s from whitelist. Market is not active.',
-                    pair
-                )
-
-        # We need to remove pairs that are unknown
-        final_list = [x for x in sanitized_whitelist if x in known_pairs]
-
-        return final_list
-
     def get_target_bid(self, pair: str, ticker: Dict[str, float]) -> float:
         """
         Calculates bid target between current ask price and last price

freqtrade/pairlist/IPairList.py

@@ -0,0 +1,91 @@
+"""
+Static List provider
+
+Provides lists as configured in config.json
+
+ """
+import logging
+from abc import ABC, abstractmethod
+from typing import List
+
+logger = logging.getLogger(__name__)
+
+
+class IPairList(ABC):
+
+    def __init__(self, freqtrade, config: dict) -> None:
+        self._freqtrade = freqtrade
+        self._config = config
+        self._whitelist = self._config['exchange']['pair_whitelist']
+        self._blacklist = self._config['exchange'].get('pair_blacklist', [])
+
+    @property
+    def name(self) -> str:
+        """
+        Gets name of the class
+        -> no need to overwrite in subclasses
+        """
+        return self.__class__.__name__
+
+    @property
+    def whitelist(self) -> List[str]:
+        """
+        Has the current whitelist
+        -> no need to overwrite in subclasses
+        """
+        return self._whitelist
+
+    @property
+    def blacklist(self) -> List[str]:
+        """
+        Has the current blacklist
+        -> no need to overwrite in subclasses
+        """
+        return self._blacklist
+
+    @abstractmethod
+    def short_desc(self) -> str:
+        """
+        Short whitelist method description - used for startup-messages
+        -> Please overwrite in subclasses
+        """
+
+    @abstractmethod
+    def refresh_pairlist(self) -> None:
+        """
+        Refreshes pairlists and assigns them to self._whitelist and self._blacklist respectively
+        -> Please overwrite in subclasses
+        """
+
+    def _validate_whitelist(self, whitelist: List[str]) -> List[str]:
+        """
+        Check available markets and remove pair from whitelist if necessary
+        :param whitelist: the sorted list (based on BaseVolume) of pairs the user might want to
+        trade
+        :return: the list of pairs the user wants to trade without the one unavailable or
+        black_listed
+        """
+        sanitized_whitelist = whitelist
+        markets = self._freqtrade.exchange.get_markets()
+
+        # Filter to markets in stake currency
+        markets = [m for m in markets if m['quote'] == self._config['stake_currency']]
+        known_pairs = set()
+
+        for market in markets:
+            pair = market['symbol']
+            # pair is not int the generated dynamic market, or in the blacklist ... ignore it
+            if pair not in whitelist or pair in self.blacklist:
+                continue
+            # else the pair is valid
+            known_pairs.add(pair)
+            # Market is not active
+            if not market['active']:
+                sanitized_whitelist.remove(pair)
+                logger.info(
+                    'Ignoring %s from whitelist. Market is not active.',
+                    pair
+                )
+
+        # We need to remove pairs that are unknown
+        return [x for x in sanitized_whitelist if x in known_pairs]

freqtrade/pairlist/StaticPairList.py

@@ -0,0 +1,30 @@
+"""
+Static List provider
+
+Provides lists as configured in config.json
+
+ """
+import logging
+
+from freqtrade.pairlist.IPairList import IPairList
+
+logger = logging.getLogger(__name__)
+
+
+class StaticPairList(IPairList):
+
+    def __init__(self, freqtrade, config: dict) -> None:
+        super().__init__(freqtrade, config)
+
+    def short_desc(self) -> str:
+        """
+        Short whitelist method description - used for startup-messages
+        -> Please overwrite in subclasses
+        """
+        return f"{self.name}: {self.whitelist}"
+
+    def refresh_pairlist(self) -> None:
+        """
+        Refreshes pairlists and assigns them to self._whitelist and self._blacklist respectively
+        """
+        self._whitelist = self._validate_whitelist(self._config['exchange']['pair_whitelist'])

freqtrade/pairlist/VolumePairList.py

@@ -0,0 +1,75 @@
+"""
+Static List provider
+
+Provides lists as configured in config.json
+
+ """
+import logging
+from typing import List
+from cachetools import TTLCache, cached
+
+from freqtrade.pairlist.IPairList import IPairList
+from freqtrade import OperationalException
+logger = logging.getLogger(__name__)
+
+SORT_VALUES = ['askVolume', 'bidVolume', 'quoteVolume']
+
+
+class VolumePairList(IPairList):
+
+    def __init__(self, freqtrade, config: dict) -> None:
+        super().__init__(freqtrade, config)
+        self._whitelistconf = self._config.get('pairlist', {}).get('config')
+        if 'number_assets' not in self._whitelistconf:
+            raise OperationalException(
+                f'`number_assets` not specified. Please check your configuration '
+                'for "pairlist.config.number_assets"')
+        self._number_pairs = self._whitelistconf['number_assets']
+        self._sort_key = self._whitelistconf.get('sort_key', 'quoteVolume')
+
+        if not self._freqtrade.exchange.exchange_has('fetchTickers'):
+            raise OperationalException(
+                'Exchange does not support dynamic whitelist.'
+                'Please edit your config and restart the bot'
+            )
+        if not self._validate_keys(self._sort_key):
+            raise OperationalException(
+                f'key {self._sort_key} not in {SORT_VALUES}')
+
+    def _validate_keys(self, key):
+        return key in SORT_VALUES
+
+    def short_desc(self) -> str:
+        """
+        Short whitelist method description - used for startup-messages
+        -> Please overwrite in subclasses
+        """
+        return f"{self.name} - top {self._whitelistconf['number_assets']} volume pairs."
+
+    def refresh_pairlist(self) -> None:
+        """
+        Refreshes pairlists and assigns them to self._whitelist and self._blacklist respectively
+        -> Please overwrite in subclasses
+        """
+        # Generate dynamic whitelist
+        pairs = self._gen_pair_whitelist(self._config['stake_currency'], self._sort_key)
+        # Validate whitelist to only have active market pairs
+        self._whitelist = self._validate_whitelist(pairs)[:self._number_pairs]
+
+    @cached(TTLCache(maxsize=1, ttl=1800))
+    def _gen_pair_whitelist(self, base_currency: str, key: str) -> List[str]:
+        """
+        Updates the whitelist with with a dynamically generated list
+        :param base_currency: base currency as str
+        :param key: sort key (defaults to 'quoteVolume')
+        :return: List of pairs
+        """
+
+        tickers = self._freqtrade.exchange.get_tickers()
+        # check length so that we make sure that '/' is actually in the string
+        tickers = [v for k, v in tickers.items()
+                   if len(k.split('/')) == 2 and k.split('/')[1] == base_currency]
+
+        sorted_tickers = sorted(tickers, reverse=True, key=lambda t: t[key])
+        pairs = [s['symbol'] for s in sorted_tickers]
+        return pairs

freqtrade/resolvers/__init__.py

@@ -1,3 +1,4 @@
 from freqtrade.resolvers.iresolver import IResolver  # noqa: F401
 from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver  # noqa: F401
+from freqtrade.resolvers.pairlist_resolver import PairListResolver  # noqa: F401
 from freqtrade.resolvers.strategy_resolver import StrategyResolver  # noqa: F401

freqtrade/resolvers/hyperopt_resolver.py

@@ -52,11 +52,14 @@ class HyperOptResolver(IResolver):
             abs_paths.insert(0, Path(extra_dir))
 
         for _path in abs_paths:
-            hyperopt = self._search_object(directory=_path, object_type=IHyperOpt,
+            try:
-                                           object_name=hyperopt_name)
+                hyperopt = self._search_object(directory=_path, object_type=IHyperOpt,
-            if hyperopt:
+                                               object_name=hyperopt_name)
-                logger.info('Using resolved hyperopt %s from \'%s\'', hyperopt_name, _path)
+                if hyperopt:
-                return hyperopt
+                    logger.info('Using resolved hyperopt %s from \'%s\'', hyperopt_name, _path)
+                    return hyperopt
+            except FileNotFoundError:
+                logger.warning('Path "%s" does not exist', _path.relative_to(Path.cwd()))
 
         raise ImportError(
             "Impossible to load Hyperopt '{}'. This class does not exist"

freqtrade/resolvers/pairlist_resolver.py

@@ -0,0 +1,59 @@
+# pragma pylint: disable=attribute-defined-outside-init
+
+"""
+This module load custom hyperopts
+"""
+import logging
+from pathlib import Path
+
+from freqtrade.pairlist.IPairList import IPairList
+from freqtrade.resolvers import IResolver
+
+logger = logging.getLogger(__name__)
+
+
+class PairListResolver(IResolver):
+    """
+    This class contains all the logic to load custom hyperopt class
+    """
+
+    __slots__ = ['pairlist']
+
+    def __init__(self, pairlist_name: str, freqtrade, config: dict) -> None:
+        """
+        Load the custom class from config parameter
+        :param config: configuration dictionary or None
+        """
+        self.pairlist = self._load_pairlist(pairlist_name, kwargs={'freqtrade': freqtrade,
+                                                                   'config': config})
+
+    def _load_pairlist(
+            self, pairlist_name: str, kwargs: dict) -> IPairList:
+        """
+        Search and loads the specified pairlist.
+        :param pairlist_name: name of the module to import
+        :param extra_dir: additional directory to search for the given pairlist
+        :return: PairList instance or None
+        """
+        current_path = Path(__file__).parent.parent.joinpath('pairlist').resolve()
+
+        abs_paths = [
+            current_path.parent.parent.joinpath('user_data/pairlist'),
+            current_path,
+        ]
+
+        for _path in abs_paths:
+            try:
+                pairlist = self._search_object(directory=_path, object_type=IPairList,
+                                               object_name=pairlist_name,
+                                               kwargs=kwargs)
+                if pairlist:
+                    logger.info('Using resolved pairlist %s from \'%s\'', pairlist_name, _path)
+                    return pairlist
+            except FileNotFoundError:
+                logger.warning('Path "%s" does not exist', _path.relative_to(Path.cwd()))
+
+        raise ImportError(
+            "Impossible to load Pairlist '{}'. This class does not exist"
+            " or contains Python code errors".format(pairlist_name)
+        )

freqtrade/rpc/rpc.py

@@ -446,7 +446,8 @@ class RPC(object):
 
     def _rpc_whitelist(self) -> Dict:
         """ Returns the currently active whitelist"""
-        res = {'method': self._freqtrade.config.get('dynamic_whitelist', 0) or 'static',
+        res = {'method': self._freqtrade.pairlists.name,
+               'length': len(self._freqtrade.pairlists.whitelist),
                'whitelist': self._freqtrade.active_pair_whitelist
                }
         return res

freqtrade/rpc/rpc_manager.py

@@ -52,7 +52,7 @@ class RPCManager(object):
             logger.debug('Forwarding message to rpc.%s', mod.name)
             mod.send_msg(msg)
 
-    def startup_messages(self, config) -> None:
+    def startup_messages(self, config, pairlist) -> None:
         if config.get('dry_run', False):
             self.send_msg({
                 'type': RPCMessageType.WARNING_NOTIFICATION,
@@ -72,14 +72,8 @@ class RPCManager(object):
                       f'*Ticker Interval:* `{ticker_interval}`\n'
                       f'*Strategy:* `{strategy_name}`'
         })
-        if config.get('dynamic_whitelist', False):
-            top_pairs = 'top volume ' + str(config.get('dynamic_whitelist', 20))
-            specific_pairs = ''
-        else:
-            top_pairs = 'whitelisted'
-            specific_pairs = '\n' + ', '.join(config['exchange'].get('pair_whitelist', ''))
         self.send_msg({
             'type': RPCMessageType.STATUS_NOTIFICATION,
-            'status': f'Searching for {top_pairs} {stake_currency} pairs to buy and sell...'
+            'status': f'Searching for {stake_currency} pairs to buy and sell '
-                      f'{specific_pairs}'
+                      f'based on {pairlist.short_desc()}'
         })

freqtrade/rpc/telegram.py

@@ -448,10 +448,8 @@ class Telegram(RPC):
         """
         try:
             whitelist = self._rpc_whitelist()
-            if whitelist['method'] == 'static':
+
-                message = f"Using static whitelist with `{len(whitelist['whitelist'])}` pairs \n"
+            message = f"Using whitelist `{whitelist['method']}` with {whitelist['length']} pairs\n"
-            else:
-                message = f"Dynamic whitelist with `{whitelist['method']}` pairs\n"
             message += f"`{', '.join(whitelist['whitelist'])}`"
 
             logger.debug(message)

freqtrade/tests/pairlist/test_pairlist.py

@@ -0,0 +1,170 @@
+# pragma pylint: disable=missing-docstring,C0103,protected-access
+
+from unittest.mock import MagicMock
+
+from freqtrade import OperationalException
+from freqtrade.constants import AVAILABLE_PAIRLISTS
+from freqtrade.resolvers import PairListResolver
+from freqtrade.tests.conftest import get_patched_freqtradebot
+import pytest
+
+# whitelist, blacklist
+
+
+@pytest.fixture(scope="function")
+def whitelist_conf(default_conf):
+    default_conf['stake_currency'] = 'BTC'
+    default_conf['exchange']['pair_whitelist'] = [
+        'ETH/BTC',
+        'TKN/BTC',
+        'TRST/BTC',
+        'SWT/BTC',
+        'BCC/BTC'
+    ]
+    default_conf['exchange']['pair_blacklist'] = [
+        'BLK/BTC'
+    ]
+    default_conf['pairlist'] = {'method': 'StaticPairList',
+                                'config': {'number_assets': 3}
+                                }
+
+    return default_conf
+
+
+def test_load_pairlist_noexist(mocker, markets, default_conf):
+    freqtradebot = get_patched_freqtradebot(mocker, default_conf)
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
+    with pytest.raises(ImportError,
+                       match=r"Impossible to load Pairlist 'NonexistingPairList'."
+                             r" This class does not exist or contains Python code errors"):
+        PairListResolver('NonexistingPairList', freqtradebot, default_conf).pairlist
+
+
+def test_refresh_market_pair_not_in_whitelist(mocker, markets, whitelist_conf):
+
+    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
+
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
+    freqtradebot.pairlists.refresh_pairlist()
+    # List ordered by BaseVolume
+    whitelist = ['ETH/BTC', 'TKN/BTC']
+    # Ensure all except those in whitelist are removed
+    assert set(whitelist) == set(freqtradebot.pairlists.whitelist)
+    # Ensure config dict hasn't been changed
+    assert (whitelist_conf['exchange']['pair_whitelist'] ==
+            freqtradebot.config['exchange']['pair_whitelist'])
+
+
+def test_refresh_pairlists(mocker, markets, whitelist_conf):
+    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
+
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
+    freqtradebot.pairlists.refresh_pairlist()
+    # List ordered by BaseVolume
+    whitelist = ['ETH/BTC', 'TKN/BTC']
+    # Ensure all except those in whitelist are removed
+    assert set(whitelist) == set(freqtradebot.pairlists.whitelist)
+    assert whitelist_conf['exchange']['pair_blacklist'] == freqtradebot.pairlists.blacklist
+
+
+def test_refresh_pairlist_dynamic(mocker, markets, tickers, whitelist_conf):
+    whitelist_conf['pairlist'] = {'method': 'VolumePairList',
+                                  'config': {'number_assets': 5}
+                                  }
+    mocker.patch.multiple(
+        'freqtrade.exchange.Exchange',
+        get_markets=markets,
+        get_tickers=tickers,
+        exchange_has=MagicMock(return_value=True)
+    )
+    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
+
+    # argument: use the whitelist dynamically by exchange-volume
+    whitelist = ['ETH/BTC', 'TKN/BTC']
+    freqtradebot.pairlists.refresh_pairlist()
+
+    assert whitelist == freqtradebot.pairlists.whitelist
+
+    whitelist_conf['pairlist'] = {'method': 'VolumePairList',
+                                  'config': {}
+                                  }
+    with pytest.raises(OperationalException,
+                       match=r'`number_assets` not specified. Please check your configuration '
+                             r'for "pairlist.config.number_assets"'):
+        PairListResolver('VolumePairList', freqtradebot, whitelist_conf).pairlist
+
+
+def test_VolumePairList_refresh_empty(mocker, markets_empty, whitelist_conf):
+    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets_empty)
+
+    # argument: use the whitelist dynamically by exchange-volume
+    whitelist = []
+    whitelist_conf['exchange']['pair_whitelist'] = []
+    freqtradebot.pairlists.refresh_pairlist()
+    pairslist = whitelist_conf['exchange']['pair_whitelist']
+
+    assert set(whitelist) == set(pairslist)
+
+
+def test_VolumePairList_whitelist_gen(mocker, whitelist_conf, markets, tickers) -> None:
+    whitelist_conf['pairlist']['method'] = 'VolumePairList'
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
+    freqtrade = get_patched_freqtradebot(mocker, whitelist_conf)
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
+    mocker.patch('freqtrade.exchange.Exchange.get_tickers', tickers)
+
+    # Test to retrieved BTC sorted on quoteVolume (default)
+    whitelist = freqtrade.pairlists._gen_pair_whitelist(base_currency='BTC', key='quoteVolume')
+    assert whitelist == ['ETH/BTC', 'TKN/BTC', 'BLK/BTC', 'LTC/BTC']
+
+    # Test to retrieve BTC sorted on bidVolume
+    whitelist = freqtrade.pairlists._gen_pair_whitelist(base_currency='BTC', key='bidVolume')
+    assert whitelist == ['LTC/BTC', 'TKN/BTC', 'ETH/BTC', 'BLK/BTC']
+
+    # Test with USDT sorted on quoteVolume (default)
+    whitelist = freqtrade.pairlists._gen_pair_whitelist(base_currency='USDT', key='quoteVolume')
+    assert whitelist == ['TKN/USDT', 'ETH/USDT', 'LTC/USDT', 'BLK/USDT']
+
+    # Test with ETH (our fixture does not have ETH, so result should be empty)
+    whitelist = freqtrade.pairlists._gen_pair_whitelist(base_currency='ETH', key='quoteVolume')
+    assert whitelist == []
+
+
+def test_gen_pair_whitelist_not_supported(mocker, default_conf, tickers) -> None:
+    default_conf['pairlist'] = {'method': 'VolumePairList',
+                                'config': {'number_assets': 10}
+                                }
+    mocker.patch('freqtrade.exchange.Exchange.get_tickers', tickers)
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=False))
+
+    with pytest.raises(OperationalException):
+        get_patched_freqtradebot(mocker, default_conf)
+
+
+@pytest.mark.parametrize("pairlist", AVAILABLE_PAIRLISTS)
+def test_pairlist_class(mocker, whitelist_conf, markets, pairlist):
+    whitelist_conf['pairlist']['method'] = pairlist
+    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
+    freqtrade = get_patched_freqtradebot(mocker, whitelist_conf)
+
+    assert freqtrade.pairlists.name == pairlist
+    assert pairlist in freqtrade.pairlists.short_desc()
+    assert isinstance(freqtrade.pairlists.whitelist, list)
+    assert isinstance(freqtrade.pairlists.blacklist, list)
+
+    whitelist = ['ETH/BTC', 'TKN/BTC']
+    new_whitelist = freqtrade.pairlists._validate_whitelist(whitelist)
+
+    assert set(whitelist) == set(new_whitelist)
+
+    whitelist = ['ETH/BTC', 'TKN/BTC', 'TRX/ETH']
+    new_whitelist = freqtrade.pairlists._validate_whitelist(whitelist)
+    # TRX/ETH was removed
+    assert set(['ETH/BTC', 'TKN/BTC']) == set(new_whitelist)
+
+    whitelist = ['ETH/BTC', 'TKN/BTC', 'BLK/BTC']
+    new_whitelist = freqtrade.pairlists._validate_whitelist(whitelist)
+    # BLK/BTC is in blacklist ...
+    assert set(['ETH/BTC', 'TKN/BTC']) == set(new_whitelist)

freqtrade/tests/rpc/test_rpc.py

@@ -655,18 +655,22 @@ def test_rpc_whitelist(mocker, default_conf) -> None:
     freqtradebot = FreqtradeBot(default_conf)
     rpc = RPC(freqtradebot)
     ret = rpc._rpc_whitelist()
-    assert ret['method'] == 'static'
+    assert ret['method'] == 'StaticPairList'
     assert ret['whitelist'] == default_conf['exchange']['pair_whitelist']
 
 
 def test_rpc_whitelist_dynamic(mocker, default_conf) -> None:
     patch_coinmarketcap(mocker)
     patch_exchange(mocker)
-    default_conf['dynamic_whitelist'] = 4
+    default_conf['pairlist'] = {'method': 'VolumePairList',
+                                'config': {'number_assets': 4}
+                                }
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
     mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
 
     freqtradebot = FreqtradeBot(default_conf)
     rpc = RPC(freqtradebot)
     ret = rpc._rpc_whitelist()
-    assert ret['method'] == 4
+    assert ret['method'] == 'VolumePairList'
+    assert ret['length'] == 4
     assert ret['whitelist'] == default_conf['exchange']['pair_whitelist']

freqtrade/tests/rpc/test_rpc_manager.py

@@ -121,15 +121,17 @@ def test_startupmessages_telegram_enabled(mocker, default_conf, caplog) -> None:
 
     freqtradebot = get_patched_freqtradebot(mocker, default_conf)
     rpc_manager = RPCManager(freqtradebot)
-    rpc_manager.startup_messages(default_conf)
+    rpc_manager.startup_messages(default_conf, freqtradebot.pairlists)
 
     assert telegram_mock.call_count == 3
     assert "*Exchange:* `bittrex`" in telegram_mock.call_args_list[1][0][0]['status']
 
     telegram_mock.reset_mock()
     default_conf['dry_run'] = True
-    default_conf['dynamic_whitelist'] = 20
+    default_conf['whitelist'] = {'method': 'VolumePairList',
+                                 'config': {'number_assets': 20}
+                                 }
 
-    rpc_manager.startup_messages(default_conf)
+    rpc_manager.startup_messages(default_conf,  freqtradebot.pairlists)
     assert telegram_mock.call_count == 3
     assert "Dry run is enabled." in telegram_mock.call_args_list[0][0][0]['status']

freqtrade/tests/rpc/test_rpc_telegram.py

@@ -1025,7 +1025,7 @@ def test_whitelist_static(default_conf, update, mocker) -> None:
 
     telegram._whitelist(bot=MagicMock(), update=update)
     assert msg_mock.call_count == 1
-    assert ('Using static whitelist with `4` pairs \n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
+    assert ('Using whitelist `StaticPairList` with 4 pairs\n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
             in msg_mock.call_args_list[0][0][0])
 
 
@@ -1037,14 +1037,17 @@ def test_whitelist_dynamic(default_conf, update, mocker) -> None:
         _init=MagicMock(),
         _send_msg=msg_mock
     )
-    default_conf['dynamic_whitelist'] = 4
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
+    default_conf['pairlist'] = {'method': 'VolumePairList',
+                                'config': {'number_assets': 4}
+                                }
     freqtradebot = get_patched_freqtradebot(mocker, default_conf)
 
     telegram = Telegram(freqtradebot)
 
     telegram._whitelist(bot=MagicMock(), update=update)
     assert msg_mock.call_count == 1
-    assert ('Dynamic whitelist with `4` pairs\n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
+    assert ('Using whitelist `VolumePairList` with 4 pairs\n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
             in msg_mock.call_args_list[0][0][0])
 
 

freqtrade/tests/test_acl_pair.py

@@ -1,87 +0,0 @@
-# pragma pylint: disable=missing-docstring,C0103,protected-access
-
-from unittest.mock import MagicMock
-
-from freqtrade.tests.conftest import get_patched_freqtradebot
-
-import pytest
-
-# whitelist, blacklist, filtering, all of that will
-# eventually become some rules to run on a generic ACL engine
-# perhaps try to anticipate that by using some python package
-
-
-@pytest.fixture(scope="function")
-def whitelist_conf(default_conf):
-    default_conf['stake_currency'] = 'BTC'
-    default_conf['exchange']['pair_whitelist'] = [
-        'ETH/BTC',
-        'TKN/BTC',
-        'TRST/BTC',
-        'SWT/BTC',
-        'BCC/BTC'
-    ]
-    default_conf['exchange']['pair_blacklist'] = [
-        'BLK/BTC'
-    ]
-
-    return default_conf
-
-
-def test_refresh_market_pair_not_in_whitelist(mocker, markets, whitelist_conf):
-
-    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
-
-    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
-    refreshedwhitelist = freqtradebot._refresh_whitelist(
-        whitelist_conf['exchange']['pair_whitelist'] + ['XXX/BTC']
-    )
-    # List ordered by BaseVolume
-    whitelist = ['ETH/BTC', 'TKN/BTC']
-    # Ensure all except those in whitelist are removed
-    assert whitelist == refreshedwhitelist
-
-
-def test_refresh_whitelist(mocker, markets, whitelist_conf):
-    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
-
-    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets)
-    refreshedwhitelist = freqtradebot._refresh_whitelist(
-        whitelist_conf['exchange']['pair_whitelist'])
-
-    # List ordered by BaseVolume
-    whitelist = ['ETH/BTC', 'TKN/BTC']
-    # Ensure all except those in whitelist are removed
-    assert whitelist == refreshedwhitelist
-
-
-def test_refresh_whitelist_dynamic(mocker, markets, tickers, whitelist_conf):
-    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
-    mocker.patch.multiple(
-        'freqtrade.exchange.Exchange',
-        get_markets=markets,
-        get_tickers=tickers,
-        exchange_has=MagicMock(return_value=True)
-    )
-
-    # argument: use the whitelist dynamically by exchange-volume
-    whitelist = ['ETH/BTC', 'TKN/BTC']
-
-    refreshedwhitelist = freqtradebot._refresh_whitelist(
-        freqtradebot._gen_pair_whitelist(whitelist_conf['stake_currency'])
-    )
-
-    assert whitelist == refreshedwhitelist
-
-
-def test_refresh_whitelist_dynamic_empty(mocker, markets_empty, whitelist_conf):
-    freqtradebot = get_patched_freqtradebot(mocker, whitelist_conf)
-    mocker.patch('freqtrade.exchange.Exchange.get_markets', markets_empty)
-
-    # argument: use the whitelist dynamically by exchange-volume
-    whitelist = []
-    whitelist_conf['exchange']['pair_whitelist'] = []
-    freqtradebot._refresh_whitelist(whitelist)
-    pairslist = whitelist_conf['exchange']['pair_whitelist']
-
-    assert set(whitelist) == set(pairslist)

freqtrade/tests/test_arguments.py

@@ -17,7 +17,8 @@ def test_parse_args_none() -> None:
 def test_parse_args_defaults() -> None:
     args = Arguments([], '').get_parsed_arg()
     assert args.config == 'config.json'
-    assert args.dynamic_whitelist is None
+    assert args.strategy_path is None
+    assert args.datadir is None
     assert args.loglevel == 0
 
 

freqtrade/tests/test_configuration.py

@@ -102,7 +102,7 @@ def test_load_config(default_conf, mocker) -> None:
 
     assert validated_conf.get('strategy') == 'DefaultStrategy'
     assert validated_conf.get('strategy_path') is None
-    assert 'dynamic_whitelist' not in validated_conf
+    assert 'edge' not in validated_conf
 
 
 def test_load_config_with_params(default_conf, mocker) -> None:
@@ -119,7 +119,8 @@ def test_load_config_with_params(default_conf, mocker) -> None:
     configuration = Configuration(args)
     validated_conf = configuration.load_config()
 
-    assert validated_conf.get('dynamic_whitelist') == 10
+    assert validated_conf.get('pairlist', {}).get('method') == 'VolumePairList'
+    assert validated_conf.get('pairlist', {}).get('config').get('number_assets') == 10
     assert validated_conf.get('strategy') == 'TestStrategy'
     assert validated_conf.get('strategy_path') == '/some/path'
     assert validated_conf.get('db_url') == 'sqlite:///someurl'
@@ -132,7 +133,6 @@ def test_load_config_with_params(default_conf, mocker) -> None:
     ))
 
     arglist = [
-        '--dynamic-whitelist', '10',
         '--strategy', 'TestStrategy',
         '--strategy-path', '/some/path'
     ]
@@ -151,7 +151,6 @@ def test_load_config_with_params(default_conf, mocker) -> None:
     ))
 
     arglist = [
-        '--dynamic-whitelist', '10',
         '--strategy', 'TestStrategy',
         '--strategy-path', '/some/path'
     ]
@@ -194,8 +193,9 @@ def test_show_info(default_conf, mocker, caplog) -> None:
     configuration.get_config()
 
     assert log_has(
-        'Parameter --dynamic-whitelist detected. '
+        'Parameter --dynamic-whitelist has been deprecated, '
-        'Using dynamically generated whitelist. '
+        'and will be completely replaced by the whitelist dict in the future. '
+        'For now: using dynamically generated whitelist based on VolumePairList. '
         '(not applicable with Backtesting and Hyperopt)',
         caplog.record_tuples
     )

freqtrade/tests/test_freqtradebot.py

@@ -136,37 +136,6 @@ def test_throttle_with_assets(mocker, default_conf) -> None:
     assert result == -1
 
 
-def test_gen_pair_whitelist(mocker, default_conf, tickers) -> None:
-    freqtrade = get_patched_freqtradebot(mocker, default_conf)
-    mocker.patch('freqtrade.exchange.Exchange.get_tickers', tickers)
-    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
-
-    # Test to retrieved BTC sorted on quoteVolume (default)
-    whitelist = freqtrade._gen_pair_whitelist(base_currency='BTC')
-    assert whitelist == ['ETH/BTC', 'TKN/BTC', 'BLK/BTC', 'LTC/BTC']
-
-    # Test to retrieve BTC sorted on bidVolume
-    whitelist = freqtrade._gen_pair_whitelist(base_currency='BTC', key='bidVolume')
-    assert whitelist == ['LTC/BTC', 'TKN/BTC', 'ETH/BTC', 'BLK/BTC']
-
-    # Test with USDT sorted on quoteVolume (default)
-    whitelist = freqtrade._gen_pair_whitelist(base_currency='USDT')
-    assert whitelist == ['TKN/USDT', 'ETH/USDT', 'LTC/USDT', 'BLK/USDT']
-
-    # Test with ETH (our fixture does not have ETH, so result should be empty)
-    whitelist = freqtrade._gen_pair_whitelist(base_currency='ETH')
-    assert whitelist == []
-
-
-def test_gen_pair_whitelist_not_supported(mocker, default_conf, tickers) -> None:
-    freqtrade = get_patched_freqtradebot(mocker, default_conf)
-    mocker.patch('freqtrade.exchange.Exchange.get_tickers', tickers)
-    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=False))
-
-    with pytest.raises(OperationalException):
-        freqtrade._gen_pair_whitelist(base_currency='BTC')
-
-
 def test_get_trade_stake_amount(default_conf, ticker, limit_buy_order, fee, mocker) -> None:
     patch_RPCManager(mocker)
     patch_exchange(mocker)
@@ -248,7 +217,7 @@ def test_edge_called_in_process(mocker, edge_conf) -> None:
 
     patch_exchange(mocker)
     freqtrade = FreqtradeBot(edge_conf)
-    freqtrade._refresh_whitelist = _refresh_whitelist
+    freqtrade.pairlists._validate_whitelist = _refresh_whitelist
     patch_get_signal(freqtrade)
     freqtrade._process()
     assert freqtrade.active_pair_whitelist == ['NEO/BTC', 'LTC/BTC']
@@ -2603,6 +2572,9 @@ def test_order_book_ask_strategy(default_conf, limit_buy_order, limit_sell_order
 
 
 def test_startup_messages(default_conf, mocker):
-    default_conf['dynamic_whitelist'] = 20
+    default_conf['pairlist'] = {'method': 'VolumePairList',
+                                'config': {'number_assets': 20}
+                                }
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
     freqtrade = get_patched_freqtradebot(mocker, default_conf)
     assert freqtrade.state is State.RUNNING