updating the forked repo

CONTRIBUTING.md

@@ -11,7 +11,7 @@ Few pointers for contributions:
 - Create your PR against the `develop` branch, not `master`.
 - New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100).
 
-If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE)
+If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg)
 or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR.
 
 ## Getting started

README.md

@@ -141,7 +141,7 @@ Accounts having BNB accounts use this to pay for fees - if your first trade happ
 For any questions not covered by the documentation or for further
 information about the bot, we encourage you to join our slack channel.
 
-- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE).
+- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg).
 
 ### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue)
 
@@ -172,7 +172,7 @@ to understand the requirements before sending your pull-requests.
 Coding is not a neccessity to contribute - maybe start with improving our documentation?
 Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
 
-**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
+**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
 
 **Important:** Always create your PR against the `develop` branch, not `master`.
 

docs/backtesting.md

@@ -123,11 +123,12 @@ python scripts/download_backtest_data.py --exchange binance
 
 This will download ticker data for all the currency pairs you defined in `pairs.json`.
 
-- To use a different folder than the exchange specific default, use `--export user_data/data/some_directory`.
+- To use a different folder than the exchange specific default, use `--datadir user_data/data/some_directory`.
 - To change the exchange used to download the tickers, use `--exchange`. Default is `bittrex`.
 - To use `pairs.json` from some other folder, use `--pairs-file some_other_dir/pairs.json`.
 - To download ticker data for only 10 days, use `--days 10`.
 - Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
+- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with other options.
 
 For help about backtesting usage, please refer to [Backtesting commands](#backtesting-commands).
 

docs/configuration.md

@@ -131,17 +131,11 @@ If it is not set in either Strategy or Configuration, a default of 1000% `{"0":
 
 ### Understand stoploss
 
-The `stoploss` configuration parameter is loss in percentage that should trigger a sale.
+Go to the [stoploss documentation](stoploss.md) for more details.
-For example, value `-0.10` will cause immediate sell if the
-profit dips below -10% for a given trade. This parameter is optional.
-
-Most of the strategy files already include the optimal `stoploss`
-value. This parameter is optional. If you use it in the configuration file, it will take over the
-`stoploss` value from the strategy file.
 
 ### Understand trailing stoploss
 
-Go to the [trailing stoploss Documentation](stoploss.md) for details on trailing stoploss.
+Go to the [trailing stoploss Documentation](stoploss.md#trailing-stop-loss) for details on trailing stoploss.
 
 ### Understand initial_state
 
@@ -303,6 +297,25 @@ This configuration enables binance, as well as rate limiting to avoid bans from
     Optimal settings for rate limiting depend on the exchange and the size of the whitelist, so an ideal parameter will vary on many other settings.
     We try to provide sensible defaults per exchange where possible, if you encounter bans please make sure that `"enableRateLimit"` is enabled and increase the `"rateLimit"` parameter step by step.
 
+#### Advanced FreqTrade Exchange configuration
+
+Advanced options can be configured using the `_ft_has_params` setting, which will override Defaults and exchange-specific behaviours.
+
+Available options are listed in the exchange-class as `_ft_has_default`.
+
+For example, to test the order type `FOK` with Kraken, and modify candle_limit to 200 (so you only get 200 candles per call):
+
+```json
+"exchange": {
+    "name": "kraken",
+    "_ft_has_params": {
+        "order_time_in_force": ["gtc", "fok"],
+        "ohlcv_candle_limit": 200
+        }
+```
+
+!!! Warning
+    Please make sure to fully understand the impacts of these settings before modifying them.
 
 ### What values can be used for fiat_display_currency?
 

docs/developer.md

@@ -2,7 +2,7 @@
 
 This page is intended for developers of FreqTrade, people who want to contribute to the FreqTrade codebase or documentation, or people who want to understand the source code of the application they're running.
 
-All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE) where you can ask questions.
+All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) where you can ask questions.
 
 ## Documentation
 
@@ -81,6 +81,51 @@ Please also run `self._validate_whitelist(pairs)` and to check and remove pairs
 This is a simple method used by `VolumePairList` - however serves as a good example.
 It implements caching (`@cached(TTLCache(maxsize=1, ttl=1800))`) as well as a configuration option to allow different (but similar) strategies to work with the same PairListProvider.
 
+## Implement a new Exchange (WIP)
+
+!!! Note
+    This section is a Work in Progress and is not a complete guide on how to test a new exchange with FreqTrade.
+
+Most exchanges supported by CCXT should work out of the box.
+
+### Stoploss On Exchange
+
+Check if the new exchange supports Stoploss on Exchange orders through their API.
+
+Since CCXT does not provide unification for Stoploss On Exchange yet, we'll need to implement the exchange-specific parameters ourselfs. Best look at `binance.py` for an example implementation of this. You'll need to dig through the documentation of the Exchange's API on how exactly this can be done. [CCXT Issues](https://github.com/ccxt/ccxt/issues) may also provide great help, since others may have implemented something similar for their projects.
+
+### Incomplete candles
+
+While fetching OHLCV data, we're may end up getting incomplete candles (Depending on the exchange).
+To demonstrate this, we'll use daily candles (`"1d"`) to keep things simple.
+We query the api (`ct.fetch_ohlcv()`) for the timeframe and look at the date of the last entry. If this entry changes or shows the date of a "incomplete" candle, then we should drop this since having incomplete candles is problematic because indicators assume that only complete candles are passed to them, and will generate a lot of false buy signals. By default, we're therefore removing the last candle assuming it's incomplete.
+
+To check how the new exchange behaves, you can use the following snippet:
+
+``` python
+import ccxt
+from datetime import datetime
+from freqtrade.data.converter import parse_ticker_dataframe
+ct = ccxt.binance()
+timeframe = "1d"
+pair = "XLM/BTC"  # Make sure to use a pair that exists on that exchange!
+raw = ct.fetch_ohlcv(pair, timeframe=timeframe)
+
+# convert to dataframe
+df1 = parse_ticker_dataframe(raw, timeframe, drop_incomplete=False)
+
+print(df1["date"].tail(1))
+print(datetime.utcnow())
+```
+
+``` output
+19   2019-06-08 00:00:00+00:00
+2019-06-09 12:30:27.873327
+```
+
+The output will show the last entry from the Exchange as well as the current UTC date.
+If the day shows the same day, then the last candle can be assumed as incomplete and should be dropped (leave the setting `"ohlcv_partial_candle"` from the exchange-class untouched / True). Otherwise, set `"ohlcv_partial_candle"` to `False` to not drop Candles (shown in the example above).
+
 ## Creating a release
 
 This part of the documentation is aimed at maintainers, and shows how to create a release.

docs/index.md

@@ -64,7 +64,7 @@ To run this bot we recommend you a cloud instance with a minimum of:
 Help / Slack
 For any questions not covered by the documentation or for further information about the bot, we encourage you to join our Slack channel.
 
-Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE) to join Slack channel.
+Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) to join Slack channel.
 
 ## Ready to try?
 

docs/stoploss.md

@@ -1,4 +1,13 @@
-# Stop Loss support
+# Stop Loss
+
+The `stoploss` configuration parameter is loss in percentage that should trigger a sale.
+For example, value `-0.10` will cause immediate sell if the profit dips below -10% for a given trade. This parameter is optional.
+
+Most of the strategy files already include the optimal `stoploss`
+value. This parameter is optional. If you use it in the configuration file, it will take over the
+`stoploss` value from the strategy file.
+
+## Stop Loss support
 
 At this stage the bot contains the following stoploss support modes:
 
@@ -16,13 +25,12 @@ In case of stoploss on exchange there is another parameter called `stoploss_on_e
 !!! Note
     Stoploss on exchange is only supported for Binance as of now.
 
-
 ## Static Stop Loss
 
 This is very simple, basically you define a stop loss of x in your strategy file or alternative in the configuration, which
 will overwrite the strategy definition. This will basically try to sell your asset, the second the loss exceeds the defined loss.
 
-## Trail Stop Loss
+## Trailing Stop Loss
 
 The initial value for this stop loss, is defined in your strategy or configuration. Just as you would define your Stop Loss normally.
 To enable this Feauture all you have to do is to define the configuration element:
@@ -63,3 +71,13 @@ The 0.01 would translate to a 1% stop loss, once you hit 1.1% profit.
 You should also make sure to have this value (`trailing_stop_positive_offset`) lower than your minimal ROI, otherwise minimal ROI will apply first and sell your trade.
 
 If `"trailing_only_offset_is_reached": true` then the trailing stoploss is only activated once the offset is reached. Until then, the stoploss remains at the configured`stoploss`.
+
+## Changing stoploss on open trades
+
+A stoploss on an open trade can be changed by changing the value in the configuration or strategy and use the `/reload_conf` command (alternatively, completely stopping and restarting the bot also works).
+
+The new stoploss value will be applied to open trades (and corresponding log-messages will be generated).
+
+### Limitations
+
+Stoploss values cannot be changed if `trailing_stop` is enabled and the stoploss has already been adjusted, or if [Edge](edge.md) is enabled (since Edge would recalculate stoploss based on the current market situation).

docs/strategy-customization.md

@@ -218,9 +218,12 @@ stoploss = -0.10
 ```
 
 This would signify a stoploss of -10%.
+
+For the full documentation on stoploss features, look at the dedicated [stoploss page](stoploss.md).
+
 If your exchange supports it, it's recommended to also set `"stoploss_on_exchange"` in the order dict, so your stoploss is on the exchange and cannot be missed for network-problems (or other problems).
 
-For more information on order_types please look [here](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md#understand-order_types).
+For more information on order_types please look [here](configuration.md#understand-order_types).
 
 ### Ticker interval
 
@@ -298,6 +301,18 @@ if self.dp:
 !!! Warning Warning in hyperopt
     This option cannot currently be used during hyperopt.
 
+#### Orderbook
+
+``` python
+if self.dp:
+    if self.dp.runmode in ('live', 'dry_run'):
+        ob = self.dp.orderbook(metadata['pair'], 1)
+        dataframe['best_bid'] = ob['bids'][0][0]
+        dataframe['best_ask'] = ob['asks'][0][0]
+```
+!Warning The order book is not part of the historic data which means backtesting and hyperopt will not work if this
+ method is used.
+
 #### Available Pairs
 
 ``` python
@@ -306,6 +321,7 @@ if self.dp:
         print(f"available {pair}, {ticker}")
 ```
 
+
 #### Get data for non-tradeable pairs
 
 Data for additional, informative pairs (reference pairs) can be beneficial for some strategies.
@@ -394,7 +410,7 @@ To get additional Ideas for strategies, head over to our [strategy repository](h
 Feel free to use any of them as inspiration for your own strategies.
 We're happy to accept Pull Requests containing new Strategies to that repo.
 
-We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE) which is a great place to get and/or share ideas.
+We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) which is a great place to get and/or share ideas.
 
 ## Next step
 

freqtrade/arguments.py

@@ -47,7 +47,7 @@ class Arguments(object):
 
         return self.parsed_arg
 
-    def parse_args(self) -> argparse.Namespace:
+    def parse_args(self, no_default_config: bool = False) -> argparse.Namespace:
         """
         Parses given arguments and returns an argparse Namespace instance.
         """
@@ -55,7 +55,7 @@ class Arguments(object):
 
         # Workaround issue in argparse with action='append' and default value
         # (see https://bugs.python.org/issue16399)
-        if parsed_arg.config is None:
+        if parsed_arg.config is None and not no_default_config:
             parsed_arg.config = [constants.DEFAULT_CONFIG]
 
         return parsed_arg
@@ -427,26 +427,24 @@ class Arguments(object):
             default=None
         )
 
-    def testdata_dl_options(self) -> None:
+    def download_data_options(self) -> None:
         """
         Parses given arguments for testdata download
         """
         self.parser.add_argument(
-            '--pairs-file',
+            '-v', '--verbose',
-            help='File containing a list of pairs to download.',
+            help='Verbose mode (-vv for more, -vvv to get all messages).',
-            dest='pairs_file',
+            action='count',
-            default=None,
+            dest='loglevel',
-            metavar='PATH',
+            default=0,
         )
-
         self.parser.add_argument(
-            '--export',
+            '--logfile',
-            help='Export files to given dir.',
+            help='Log to the file specified',
-            dest='export',
+            dest='logfile',
-            default=None,
+            type=str,
-            metavar='PATH',
+            metavar='FILE',
         )
-
         self.parser.add_argument(
             '-c', '--config',
             help='Specify configuration file (default: %(default)s). '
@@ -456,35 +454,39 @@ class Arguments(object):
             type=str,
             metavar='PATH',
         )
-
+        self.parser.add_argument(
+            '-d', '--datadir',
+            help='Path to backtest data.',
+            dest='datadir',
+            metavar='PATH',
+        )
+        self.parser.add_argument(
+            '--pairs-file',
+            help='File containing a list of pairs to download.',
+            dest='pairs_file',
+            metavar='FILE',
+        )
         self.parser.add_argument(
             '--days',
             help='Download data for given number of days.',
             dest='days',
-            type=int,
+            type=Arguments.check_int_positive,
             metavar='INT',
-            default=None
         )
-
         self.parser.add_argument(
             '--exchange',
             help='Exchange name (default: %(default)s). Only valid if no config is provided.',
             dest='exchange',
-            type=str,
-            default='bittrex'
         )
-
         self.parser.add_argument(
             '-t', '--timeframes',
             help='Specify which tickers to download. Space separated list. \
                   Default: %(default)s.',
             choices=['1m', '3m', '5m', '15m', '30m', '1h', '2h', '4h',
                      '6h', '8h', '12h', '1d', '3d', '1w'],
-            default=['1m', '5m'],
             nargs='+',
             dest='timeframes',
         )
-
         self.parser.add_argument(
             '--erase',
             help='Clean all existing data for the selected exchange/pairs/timeframes.',

freqtrade/configuration.py

@@ -13,7 +13,8 @@ from jsonschema import Draft4Validator, validators
 from jsonschema.exceptions import ValidationError, best_match
 
 from freqtrade import OperationalException, constants
-from freqtrade.exchange import is_exchange_supported, supported_exchanges
+from freqtrade.exchange import (is_exchange_bad, is_exchange_available,
+                                is_exchange_officially_supported, available_exchanges)
 from freqtrade.misc import deep_merge_dicts
 from freqtrade.state import RunMode
 
@@ -122,12 +123,11 @@ class Configuration(object):
 
         return conf
 
-    def _load_common_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
+    def _load_logging_config(self, config: Dict[str, Any]) -> None:
         """
-        Extract information for sys.argv and load common configuration
+        Extract information for sys.argv and load logging configuration:
-        :return: configuration as dictionary
+        the --loglevel, --logfile options
         """
-
         # Log level
         if 'loglevel' in self.args and self.args.loglevel:
             config.update({'verbosity': self.args.loglevel})
@@ -153,6 +153,13 @@ class Configuration(object):
         set_loggers(config['verbosity'])
         logger.info('Verbosity set to %s', config['verbosity'])
 
+    def _load_common_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
+        """
+        Extract information for sys.argv and load common configuration
+        :return: configuration as dictionary
+        """
+        self._load_logging_config(config)
+
         # Support for sd_notify
         if self.args.sd_notify:
             config['internals'].update({'sd_notify': True})
@@ -228,6 +235,17 @@ class Configuration(object):
             else:
                 logger.info(logstring.format(config[argname]))
 
+    def _load_datadir_config(self, config: Dict[str, Any]) -> None:
+        """
+        Extract information for sys.argv and load datadir configuration:
+        the --datadir option
+        """
+        if 'datadir' in self.args and self.args.datadir:
+            config.update({'datadir': self._create_datadir(config, self.args.datadir)})
+        else:
+            config.update({'datadir': self._create_datadir(config, None)})
+        logger.info('Using data folder: %s ...', config.get('datadir'))
+
     def _load_optimize_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
         """
         Extract information for sys.argv and load Optimize configuration
@@ -263,11 +281,7 @@ class Configuration(object):
         self._args_to_config(config, argname='timerange',
                              logstring='Parameter --timerange detected: {} ...')
 
-        if 'datadir' in self.args and self.args.datadir:
+        self._load_datadir_config(config)
-            config.update({'datadir': self._create_datadir(config, self.args.datadir)})
-        else:
-            config.update({'datadir': self._create_datadir(config, None)})
-        logger.info('Using data folder: %s ...', config.get('datadir'))
 
         self._args_to_config(config, argname='refresh_pairs',
                              logstring='Parameter -r/--refresh-pairs-cached detected ...')
@@ -375,22 +389,40 @@ class Configuration(object):
 
         return self.config
 
-    def check_exchange(self, config: Dict[str, Any]) -> bool:
+    def check_exchange(self, config: Dict[str, Any], check_for_bad: bool = True) -> bool:
         """
         Check if the exchange name in the config file is supported by Freqtrade
-        :return: True or raised an exception if the exchange if not supported
+        :param check_for_bad: if True, check the exchange against the list of known 'bad'
+                              exchanges
+        :return: False if exchange is 'bad', i.e. is known to work with the bot with
+                 critical issues or does not work at all, crashes, etc. True otherwise.
+                 raises an exception if the exchange if not supported by ccxt
+                 and thus is not known for the Freqtrade at all.
         """
+        logger.info("Checking exchange...")
+
         exchange = config.get('exchange', {}).get('name').lower()
-        if not is_exchange_supported(exchange):
+        if not is_exchange_available(exchange):
-
-            exception_msg = f'Exchange "{exchange}" not supported.\n' \
-                            f'The following exchanges are supported: ' \
-                            f'{", ".join(supported_exchanges())}'
-
-            logger.critical(exception_msg)
             raise OperationalException(
-                exception_msg
+                    f'Exchange "{exchange}" is not supported by ccxt '
+                    f'and therefore not available for the bot.\n'
+                    f'The following exchanges are supported by ccxt: '
+                    f'{", ".join(available_exchanges())}'
             )
 
-        logger.debug('Exchange "%s" supported', exchange)
+        if check_for_bad and is_exchange_bad(exchange):
+            logger.warning(f'Exchange "{exchange}" is known to not work with the bot yet. '
+                           f'Use it only for development and testing purposes.')
+            return False
+
+        if is_exchange_officially_supported(exchange):
+            logger.info(f'Exchange "{exchange}" is officially supported '
+                        f'by the Freqtrade development team.')
+        else:
+            logger.warning(f'Exchange "{exchange}" is supported by ccxt '
+                           f'and therefore available for the bot but not officially supported '
+                           f'by the Freqtrade development team. '
+                           f'It may work flawlessly (please report back) or have serious issues. '
+                           f'Use it at your own discretion.')
+
         return True

freqtrade/data/converter.py

@@ -10,14 +10,16 @@ from pandas import DataFrame, to_datetime
 logger = logging.getLogger(__name__)
 
 
-def parse_ticker_dataframe(ticker: list, ticker_interval: str,
+def parse_ticker_dataframe(ticker: list, ticker_interval: str, *,
-                           fill_missing: bool = True) -> DataFrame:
+                           fill_missing: bool = True,
+                           drop_incomplete: bool = True) -> DataFrame:
     """
     Converts a ticker-list (format ccxt.fetch_ohlcv) to a Dataframe
     :param ticker: ticker list, as returned by exchange.async_get_candle_history
     :param ticker_interval: ticker_interval (e.g. 5m). Used to fill up eventual missing data
     :param fill_missing: fill up missing candles with 0 candles
                          (see ohlcv_fill_up_missing_data for details)
+    :param drop_incomplete: Drop the last candle of the dataframe, assuming it's incomplete
     :return: DataFrame
     """
     logger.debug("Parsing tickerlist to dataframe")
@@ -43,7 +45,9 @@ def parse_ticker_dataframe(ticker: list, ticker_interval: str,
         'close': 'last',
         'volume': 'max',
     })
-    frame.drop(frame.tail(1).index, inplace=True)     # eliminate partial candle
+    # eliminate partial candle
+    if drop_incomplete:
+        frame.drop(frame.tail(1).index, inplace=True)
         logger.debug('Dropping last candle')
 
     if fill_missing:

freqtrade/data/dataprovider.py

@@ -85,8 +85,7 @@ class DataProvider(object):
         """
         return latest orderbook data
         """
-        # TODO: Implement me
+        return self._exchange.get_order_book(pair, max)
-        pass
 
     @property
     def runmode(self) -> RunMode:

freqtrade/data/history.py

@@ -81,10 +81,20 @@ def load_pair_history(pair: str,
                       timerange: TimeRange = TimeRange(None, None, 0, 0),
                       refresh_pairs: bool = False,
                       exchange: Optional[Exchange] = None,
-                      fill_up_missing: bool = True
+                      fill_up_missing: bool = True,
+                      drop_incomplete: bool = True
                       ) -> DataFrame:
     """
     Loads cached ticker history for the given pair.
+    :param pair: Pair to load data for
+    :param ticker_interval: Ticker-interval (e.g. "5m")
+    :param datadir: Path to the data storage location.
+    :param timerange: Limit data to be loaded to this timerange
+    :param refresh_pairs: Refresh pairs from exchange.
+        (Note: Requires exchange to be passed as well.)
+    :param exchange: Exchange object (needed when using "refresh_pairs")
+    :param fill_up_missing: Fill missing values with "No action"-candles
+    :param drop_incomplete: Drop last candle assuming it may be incomplete.
     :return: DataFrame with ohlcv data
     """
 
@@ -106,7 +116,9 @@ def load_pair_history(pair: str,
             logger.warning('Missing data at end for pair %s, data ends at %s',
                            pair,
                            arrow.get(pairdata[-1][0] // 1000).strftime('%Y-%m-%d %H:%M:%S'))
-        return parse_ticker_dataframe(pairdata, ticker_interval, fill_up_missing)
+        return parse_ticker_dataframe(pairdata, ticker_interval,
+                                      fill_missing=fill_up_missing,
+                                      drop_incomplete=drop_incomplete)
     else:
         logger.warning(
             f'No history data for pair: "{pair}", interval: {ticker_interval}. '

freqtrade/edge/__init__.py

@@ -46,10 +46,6 @@ class Edge():
         self.config = config
         self.exchange = exchange
         self.strategy = strategy
-        self.ticker_interval = self.strategy.ticker_interval
-        self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
-        self.advise_sell = self.strategy.advise_sell
-        self.advise_buy = self.strategy.advise_buy
 
         self.edge_config = self.config.get('edge', {})
         self._cached_pairs: Dict[str, Any] = {}  # Keeps a list of pairs
@@ -100,7 +96,7 @@ class Edge():
         data = history.load_data(
             datadir=Path(self.config['datadir']) if self.config.get('datadir') else None,
             pairs=pairs,
-            ticker_interval=self.ticker_interval,
+            ticker_interval=self.strategy.ticker_interval,
             refresh_pairs=self._refresh_pairs,
             exchange=self.exchange,
             timerange=self._timerange
@@ -112,7 +108,7 @@ class Edge():
             logger.critical("No data found. Edge is stopped ...")
             return False
 
-        preprocessed = self.tickerdata_to_dataframe(data)
+        preprocessed = self.strategy.tickerdata_to_dataframe(data)
 
         # Print timeframe
         min_date, max_date = history.get_timeframe(preprocessed)
@@ -130,8 +126,8 @@ class Edge():
             pair_data = pair_data.sort_values(by=['date'])
             pair_data = pair_data.reset_index(drop=True)
 
-            ticker_data = self.advise_sell(
+            ticker_data = self.strategy.advise_sell(
-                self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
+                self.strategy.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
 
             trades += self._find_trades_for_stoploss_range(ticker_data, pair, self._stoploss_range)
 

freqtrade/exchange/__init__.py

@@ -1,6 +1,8 @@
 from freqtrade.exchange.exchange import Exchange  # noqa: F401
-from freqtrade.exchange.exchange import (is_exchange_supported,  # noqa: F401
+from freqtrade.exchange.exchange import (is_exchange_bad,  # noqa: F401
-                                         supported_exchanges)
+                                         is_exchange_available,
+                                         is_exchange_officially_supported,
+                                         available_exchanges)
 from freqtrade.exchange.exchange import (timeframe_to_seconds,  # noqa: F401
                                          timeframe_to_minutes,
                                          timeframe_to_msecs)

freqtrade/exchange/exchange.py

@@ -2,23 +2,24 @@
 """
 Cryptocurrency Exchanges support
 """
-import logging
+import asyncio
 import inspect
-from random import randint
+import logging
-from typing import List, Dict, Tuple, Any, Optional
+from copy import deepcopy
 from datetime import datetime
-from math import floor, ceil
+from math import ceil, floor
+from random import randint
+from typing import Any, Dict, List, Optional, Tuple
 
 import arrow
-import asyncio
 import ccxt
 import ccxt.async_support as ccxt_async
 from pandas import DataFrame
 
-from freqtrade import (constants, DependencyException, OperationalException,
+from freqtrade import (DependencyException, InvalidOrderException,
-                       TemporaryError, InvalidOrderException)
+                       OperationalException, TemporaryError, constants)
 from freqtrade.data.converter import parse_ticker_dataframe
-
+from freqtrade.misc import deep_merge_dicts
 
 logger = logging.getLogger(__name__)
 
@@ -68,12 +69,15 @@ class Exchange(object):
     _params: Dict = {}
 
     # Dict to specify which options each exchange implements
-    # TODO: this should be merged with attributes from subclasses
+    # This defines defaults, which can be selectively overridden by subclasses using _ft_has
-    # To avoid having to copy/paste this to all subclasses.
+    # or by specifying them in the configuration.
-    _ft_has: Dict = {
+    _ft_has_default: Dict = {
         "stoploss_on_exchange": False,
         "order_time_in_force": ["gtc"],
+        "ohlcv_candle_limit": 500,
+        "ohlcv_partial_candle": True,
     }
+    _ft_has: Dict = {}
 
     def __init__(self, config: dict) -> None:
         """
@@ -100,6 +104,19 @@ class Exchange(object):
             logger.info('Instance is running with dry_run enabled')
 
         exchange_config = config['exchange']
+
+        # Deep merge ft_has with default ft_has options
+        self._ft_has = deep_merge_dicts(self._ft_has, deepcopy(self._ft_has_default))
+        if exchange_config.get("_ft_has_params"):
+            self._ft_has = deep_merge_dicts(exchange_config.get("_ft_has_params"),
+                                            self._ft_has)
+            logger.info("Overriding exchange._ft_has with config params, result: %s", self._ft_has)
+
+        # Assign this directly for easy access
+        self._ohlcv_candle_limit = self._ft_has['ohlcv_candle_limit']
+        self._ohlcv_partial_candle = self._ft_has['ohlcv_partial_candle']
+
+        # Initialize ccxt objects
         self._api: ccxt.Exchange = self._init_ccxt(
             exchange_config, ccxt_kwargs=exchange_config.get('ccxt_config'))
         self._api_async: ccxt_async.Exchange = self._init_ccxt(
@@ -139,8 +156,8 @@ class Exchange(object):
         # Find matching class for the given exchange name
         name = exchange_config['name']
 
-        if not is_exchange_supported(name, ccxt_module):
+        if not is_exchange_available(name, ccxt_module):
-            raise OperationalException(f'Exchange {name} is not supported')
+            raise OperationalException(f'Exchange {name} is not supported by ccxt')
 
         ex_config = {
             'apiKey': exchange_config.get('key'),
@@ -506,10 +523,8 @@ class Exchange(object):
     async def _async_get_history(self, pair: str,
                                  ticker_interval: str,
                                  since_ms: int) -> List:
-        # Assume exchange returns 500 candles
-        _LIMIT = 500
 
-        one_call = timeframe_to_msecs(ticker_interval) * _LIMIT
+        one_call = timeframe_to_msecs(ticker_interval) * self._ohlcv_candle_limit
         logger.debug(
             "one_call: %s msecs (%s)",
             one_call,
@@ -566,7 +581,8 @@ class Exchange(object):
                 self._pairs_last_refresh_time[(pair, ticker_interval)] = ticks[-1][0] // 1000
             # keeping parsed dataframe in cache
             self._klines[(pair, ticker_interval)] = parse_ticker_dataframe(
-                ticks, ticker_interval, fill_missing=True)
+                ticks, ticker_interval, fill_missing=True,
+                drop_incomplete=self._ohlcv_partial_candle)
         return tickers
 
     def _now_is_time_to_refresh(self, pair: str, ticker_interval: str) -> bool:
@@ -706,11 +722,19 @@ class Exchange(object):
             raise OperationalException(e)
 
 
-def is_exchange_supported(exchange: str, ccxt_module=None) -> bool:
+def is_exchange_bad(exchange: str) -> bool:
-    return exchange in supported_exchanges(ccxt_module)
+    return exchange in ['bitmex']
 
 
-def supported_exchanges(ccxt_module=None) -> List[str]:
+def is_exchange_available(exchange: str, ccxt_module=None) -> bool:
+    return exchange in available_exchanges(ccxt_module)
+
+
+def is_exchange_officially_supported(exchange: str) -> bool:
+    return exchange in ['bittrex', 'binance']
+
+
+def available_exchanges(ccxt_module=None) -> List[str]:
     return ccxt_module.exchanges if ccxt_module is not None else ccxt.exchanges
 
 

freqtrade/freqtradebot.py

@@ -90,6 +90,16 @@ class FreqtradeBot(object):
         self.rpc.cleanup()
         persistence.cleanup()
 
+    def startup(self) -> None:
+        """
+        Called on startup and after reloading the bot - triggers notifications and
+        performs startup tasks
+        """
+        self.rpc.startup_messages(self.config, self.pairlists)
+        if not self.edge:
+            # Adjust stoploss if it was changed
+            Trade.stoploss_reinitialization(self.strategy.stoploss)
+
     def process(self) -> bool:
         """
         Queries the persistence layer for open trades and handles them,

freqtrade/misc.py

@@ -117,6 +117,8 @@ def format_ms_time(date: int) -> str:
 
 def deep_merge_dicts(source, destination):
     """
+    Values from Source override destination, destination is returned (and modified!!)
+    Sample:
     >>> a = { 'first' : { 'rows' : { 'pass' : 'dog', 'number' : '1' } } }
     >>> b = { 'first' : { 'rows' : { 'fail' : 'cat', 'number' : '5' } } }
     >>> merge(b, a) == { 'first' : { 'rows' : { 'pass' : 'dog', 'fail' : 'cat', 'number' : '5' } } }

freqtrade/optimize/backtesting.py

@@ -72,18 +72,16 @@ class Backtesting(object):
             IStrategy.dp = self.dataprovider
 
         if self.config.get('strategy_list', None):
-            # Force one interval
-            self.ticker_interval = str(self.config.get('ticker_interval'))
-            self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
             for strat in list(self.config['strategy_list']):
                 stratconf = deepcopy(self.config)
                 stratconf['strategy'] = strat
                 self.strategylist.append(StrategyResolver(stratconf).strategy)
 
         else:
-            # only one strategy
+            # No strategy list specified, only one strategy
             self.strategylist.append(StrategyResolver(self.config).strategy)
-        # Load one strategy
+
+        # Load one (first) strategy
         self._set_strategy(self.strategylist[0])
 
     def _set_strategy(self, strategy):
@@ -94,7 +92,6 @@ class Backtesting(object):
 
         self.ticker_interval = self.config.get('ticker_interval')
         self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
-        self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe
         self.advise_buy = strategy.advise_buy
         self.advise_sell = strategy.advise_sell
         # Set stoploss_on_exchange to false for backtesting,
@@ -235,10 +232,9 @@ class Backtesting(object):
 
     def _get_sell_trade_entry(
             self, pair: str, buy_row: DataFrame,
-            partial_ticker: List, trade_count_lock: Dict, args: Dict) -> Optional[BacktestResult]:
+            partial_ticker: List, trade_count_lock: Dict,
+            stake_amount: float, max_open_trades: int) -> Optional[BacktestResult]:
 
-        stake_amount = args['stake_amount']
-        max_open_trades = args.get('max_open_trades', 0)
         trade = Trade(
             open_rate=buy_row.open,
             open_date=buy_row.date,
@@ -254,8 +250,7 @@ class Backtesting(object):
                 # Increase trade_count_lock for every iteration
                 trade_count_lock[sell_row.date] = trade_count_lock.get(sell_row.date, 0) + 1
 
-            buy_signal = sell_row.buy
+            sell = self.strategy.should_sell(trade, sell_row.open, sell_row.date, sell_row.buy,
-            sell = self.strategy.should_sell(trade, sell_row.open, sell_row.date, buy_signal,
                                              sell_row.sell, low=sell_row.low, high=sell_row.high)
             if sell.sell_flag:
 
@@ -328,6 +323,7 @@ class Backtesting(object):
         :return: DataFrame
         """
         processed = args['processed']
+        stake_amount = args['stake_amount']
         max_open_trades = args.get('max_open_trades', 0)
         position_stacking = args.get('position_stacking', False)
         start_date = args['start_date']
@@ -378,7 +374,8 @@ class Backtesting(object):
                     trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1
 
                 trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][indexes[pair]:],
-                                                         trade_count_lock, args)
+                                                         trade_count_lock, stake_amount,
+                                                         max_open_trades)
 
                 if trade_entry:
                     lock_pair_until[pair] = trade_entry.close_time

freqtrade/optimize/edge_cli.py

@@ -6,6 +6,7 @@ This module contains the edge backtesting interface
 import logging
 from typing import Dict, Any
 from tabulate import tabulate
+from freqtrade import constants
 from freqtrade.edge import Edge
 
 from freqtrade.arguments import Arguments
@@ -32,6 +33,7 @@ class EdgeCli(object):
         self.config['exchange']['secret'] = ''
         self.config['exchange']['password'] = ''
         self.config['exchange']['uid'] = ''
+        self.config['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT
         self.config['dry_run'] = True
         self.exchange = Exchange(self.config)
         self.strategy = StrategyResolver(self.config).strategy

freqtrade/optimize/hyperopt.py

@@ -48,7 +48,6 @@ class Hyperopt(Backtesting):
     """
     def __init__(self, config: Dict[str, Any]) -> None:
         super().__init__(config)
-        self.config = config
         self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
 
         # set TARGET_TRADES to suit your number concurrent trades so its realistic
@@ -325,7 +324,9 @@ class Hyperopt(Backtesting):
             self.strategy.advise_indicators = \
                 self.custom_hyperopt.populate_indicators  # type: ignore
 
-        dump(self.strategy.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
+        preprocessed = self.strategy.tickerdata_to_dataframe(data)
+
+        dump(preprocessed, TICKERDATA_PICKLE)
 
         # We don't need exchange instance anymore while running hyperopt
         self.exchange = None  # type: ignore

freqtrade/persistence.py

@@ -422,3 +422,22 @@ class Trade(_DECL_BASE):
         Query trades from persistence layer
         """
         return Trade.query.filter(Trade.is_open.is_(True)).all()
+
+    @staticmethod
+    def stoploss_reinitialization(desired_stoploss):
+        """
+        Adjust initial Stoploss to desired stoploss for all open trades.
+        """
+        for trade in Trade.get_open_trades():
+            logger.info("Found open trade: %s", trade)
+
+            # skip case if trailing-stop changed the stoploss already.
+            if (trade.stop_loss == trade.initial_stop_loss
+               and trade.initial_stop_loss_pct != desired_stoploss):
+                # Stoploss value got changed
+
+                logger.info(f"Stoploss for {trade} needs adjustment.")
+                # Force reset of stoploss
+                trade.stop_loss = None
+                trade.adjust_stop_loss(trade.open_rate, desired_stoploss)
+                logger.info(f"new stoploss: {trade.stop_loss}, ")

freqtrade/strategy/interface.py

@@ -308,14 +308,16 @@ class IStrategy(ABC):
 
         if trailing_stop:
             # trailing stoploss handling
-
             sl_offset = self.config.get('trailing_stop_positive_offset') or 0.0
             tsl_only_offset = self.config.get('trailing_only_offset_is_reached', False)
 
+            # Make sure current_profit is calculated using high for backtesting.
+            high_profit = current_profit if not high else trade.calc_profit_percent(high)
+
             # Don't update stoploss if trailing_only_offset_is_reached is true.
-            if not (tsl_only_offset and current_profit < sl_offset):
+            if not (tsl_only_offset and high_profit < sl_offset):
                 # Specific handling for trailing_stop_positive
-                if 'trailing_stop_positive' in self.config and current_profit > sl_offset:
+                if 'trailing_stop_positive' in self.config and high_profit > sl_offset:
                     # Ignore mypy error check in configuration that this is a float
                     stop_loss_value = self.config.get('trailing_stop_positive')  # type: ignore
                     logger.debug(f"using positive stop loss: {stop_loss_value} "
@@ -329,8 +331,9 @@ class IStrategy(ABC):
                 (not self.order_types.get('stoploss_on_exchange'))):
 
             selltype = SellType.STOP_LOSS
-            # If Trailing stop (and max-rate did move above open rate)
+
-            if trailing_stop and trade.open_rate != trade.max_rate:
+            # If initial stoploss is not the same as current one then it is trailing.
+            if trade.initial_stop_loss != trade.stop_loss:
                 selltype = SellType.TRAILING_STOP_LOSS
                 logger.debug(
                     f"HIT STOP: current price at {current_rate:.6f}, "
@@ -377,6 +380,7 @@ class IStrategy(ABC):
         :param metadata: Additional information, like the currently traded pair
         :return: a Dataframe with all mandatory indicators for the strategies
         """
+        logger.debug(f"Populating indicators for pair {metadata.get('pair')}.")
         if self._populate_fun_len == 2:
             warnings.warn("deprecated - check out the Sample strategy to see "
                           "the current function headers!", DeprecationWarning)
@@ -392,6 +396,7 @@ class IStrategy(ABC):
         :param pair: Additional information, like the currently traded pair
         :return: DataFrame with buy column
         """
+        logger.debug(f"Populating buy signals for pair {metadata.get('pair')}.")
         if self._buy_fun_len == 2:
             warnings.warn("deprecated - check out the Sample strategy to see "
                           "the current function headers!", DeprecationWarning)
@@ -407,6 +412,7 @@ class IStrategy(ABC):
         :param pair: Additional information, like the currently traded pair
         :return: DataFrame with sell column
         """
+        logger.debug(f"Populating sell signals for pair {metadata.get('pair')}.")
         if self._sell_fun_len == 2:
             warnings.warn("deprecated - check out the Sample strategy to see "
                           "the current function headers!", DeprecationWarning)

freqtrade/tests/conftest.py

@@ -2,6 +2,7 @@
 import json
 import logging
 import re
+from copy import deepcopy
 from datetime import datetime
 from functools import reduce
 from unittest.mock import MagicMock, PropertyMock
@@ -649,7 +650,7 @@ def ticker_history_list():
 
 @pytest.fixture
 def ticker_history(ticker_history_list):
-    return parse_ticker_dataframe(ticker_history_list, "5m", True)
+    return parse_ticker_dataframe(ticker_history_list, "5m", fill_missing=True)
 
 
 @pytest.fixture
@@ -854,7 +855,7 @@ def tickers():
 @pytest.fixture
 def result():
     with open('freqtrade/tests/testdata/UNITTEST_BTC-1m.json') as data_file:
-        return parse_ticker_dataframe(json.load(data_file), '1m', True)
+        return parse_ticker_dataframe(json.load(data_file), '1m', fill_missing=True)
 
 # FIX:
 # Create an fixture/function
@@ -952,9 +953,10 @@ def buy_order_fee():
 
 @pytest.fixture(scope="function")
 def edge_conf(default_conf):
-    default_conf['max_open_trades'] = -1
+    conf = deepcopy(default_conf)
-    default_conf['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT
+    conf['max_open_trades'] = -1
-    default_conf['edge'] = {
+    conf['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT
+    conf['edge'] = {
         "enabled": True,
         "process_throttle_secs": 1800,
         "calculate_since_number_of_days": 14,
@@ -970,7 +972,7 @@ def edge_conf(default_conf):
         "remove_pumps": False
     }
 
-    return default_conf
+    return conf
 
 
 @pytest.fixture

freqtrade/tests/data/test_converter.py

@@ -96,3 +96,50 @@ def test_ohlcv_fill_up_missing_data2(caplog):
 
     assert log_has(f"Missing data fillup: before: {len(data)} - after: {len(data2)}",
                    caplog.record_tuples)
+
+
+def test_ohlcv_drop_incomplete(caplog):
+    ticker_interval = '1d'
+    ticks = [[
+            1559750400000,  # 2019-06-04
+            8.794e-05,  # open
+            8.948e-05,  # high
+            8.794e-05,  # low
+            8.88e-05,  # close
+            2255,  # volume (in quote currency)
+        ],
+        [
+            1559836800000,  # 2019-06-05
+            8.88e-05,
+            8.942e-05,
+            8.88e-05,
+            8.893e-05,
+            9911,
+        ],
+        [
+            1559923200000,  # 2019-06-06
+            8.891e-05,
+            8.893e-05,
+            8.875e-05,
+            8.877e-05,
+            2251
+        ],
+        [
+            1560009600000,  # 2019-06-07
+            8.877e-05,
+            8.883e-05,
+            8.895e-05,
+            8.817e-05,
+            123551
+     ]
+    ]
+    caplog.set_level(logging.DEBUG)
+    data = parse_ticker_dataframe(ticks, ticker_interval, fill_missing=False, drop_incomplete=False)
+    assert len(data) == 4
+    assert not log_has("Dropping last candle", caplog.record_tuples)
+
+    # Drop last candle
+    data = parse_ticker_dataframe(ticks, ticker_interval, fill_missing=False, drop_incomplete=True)
+    assert len(data) == 3
+
+    assert log_has("Dropping last candle", caplog.record_tuples)

freqtrade/tests/exchange/test_exchange.py

@@ -1435,3 +1435,30 @@ def test_stoploss_limit_order_dry_run(default_conf, mocker):
     assert order['type'] == order_type
     assert order['price'] == 220
     assert order['amount'] == 1
+
+
+def test_merge_ft_has_dict(default_conf, mocker):
+    mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=MagicMock()))
+    mocker.patch('freqtrade.exchange.Exchange._load_async_markets', MagicMock())
+    mocker.patch('freqtrade.exchange.Exchange.validate_pairs', MagicMock())
+    mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
+    ex = Exchange(default_conf)
+    assert ex._ft_has == Exchange._ft_has_default
+
+    ex = Kraken(default_conf)
+    assert ex._ft_has == Exchange._ft_has_default
+
+    # Binance defines different values
+    ex = Binance(default_conf)
+    assert ex._ft_has != Exchange._ft_has_default
+    assert ex._ft_has['stoploss_on_exchange']
+    assert ex._ft_has['order_time_in_force'] == ['gtc', 'fok', 'ioc']
+
+    conf = copy.deepcopy(default_conf)
+    conf['exchange']['_ft_has_params'] = {"DeadBeef": 20,
+                                          "stoploss_on_exchange": False}
+    # Use settings from configuration (overriding stoploss_on_exchange)
+    ex = Binance(conf)
+    assert ex._ft_has != Exchange._ft_has_default
+    assert not ex._ft_has['stoploss_on_exchange']
+    assert ex._ft_has['DeadBeef'] == 20

freqtrade/tests/optimize/__init__.py

@@ -29,6 +29,10 @@ class BTContainer(NamedTuple):
     trades: List[BTrade]
     profit_perc: float
     trailing_stop: bool = False
+    trailing_only_offset_is_reached: bool = False
+    trailing_stop_positive: float = None
+    trailing_stop_positive_offset: float = 0.0
+    use_sell_signal: bool = False
 
 
 def _get_frame_time_from_offset(offset):

freqtrade/tests/optimize/test_backtest_detail.py

@@ -14,6 +14,21 @@ from freqtrade.tests.optimize import (BTContainer, BTrade,
                                       _get_frame_time_from_offset,
                                       tests_ticker_interval)
 
+# Test 0 Sell signal sell
+# Test with Stop-loss at 1%
+# TC0: Sell signal in candle 3
+tc0 = BTContainer(data=[
+    # D  O     H     L     C     V    B  S
+    [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
+    [1, 5000, 5025, 4975, 4987, 6172, 0, 0],  # enter trade (signal on last candle)
+    [2, 4987, 5012, 4986, 4600, 6172, 0, 0],  # exit with stoploss hit
+    [3, 5010, 5000, 4980, 5010, 6172, 0, 1],
+    [4, 5010, 4987, 4977, 4995, 6172, 0, 0],
+    [5, 4995, 4995, 4995, 4950, 6172, 0, 0]],
+    stop_loss=-0.01, roi=1, profit_perc=0.002, use_sell_signal=True,
+    trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=4)]
+)
+
 # Test 1 Minus 8% Close
 # Test with Stop-loss at 1%
 # TC1: Stop-Loss Triggered 1% loss
@@ -146,7 +161,7 @@ tc8 = BTContainer(data=[
 # Test 9 - trailing_stop should raise - high and low in same candle.
 # Candle Data for test 9
 # Set stop-loss at 10%, ROI at 10% (should not apply)
-# TC9: Trailing stoploss - stoploss should be adjusted candle 2
+# TC9: Trailing stoploss - stoploss should be adjusted candle 3
 tc9 = BTContainer(data=[
     # D   O     H     L     C    V    B  S
     [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@@ -158,7 +173,59 @@ tc9 = BTContainer(data=[
     trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
 )
 
+# Test 10 - trailing_stop should raise so candle 3 causes a stoploss
+# without applying trailing_stop_positive since stoploss_offset is at 10%.
+# Set stop-loss at 10%, ROI at 10% (should not apply)
+# TC10: Trailing stoploss - stoploss should be adjusted candle 2
+tc10 = BTContainer(data=[
+    # D   O     H     L     C    V    B  S
+    [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
+    [1, 5000, 5050, 4950, 5100, 6172, 0, 0],
+    [2, 5100, 5251, 5100, 5100, 6172, 0, 0],
+    [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
+    [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
+    stop_loss=-0.10, roi=0.10, profit_perc=-0.1, trailing_stop=True,
+    trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.10,
+    trailing_stop_positive=0.03,
+    trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=4)]
+)
+
+# Test 11 - trailing_stop should raise so candle 3 causes a stoploss
+# applying a positive trailing stop of 3% since stop_positive_offset is reached.
+# Set stop-loss at 10%, ROI at 10% (should not apply)
+# TC11: Trailing stoploss - stoploss should be adjusted candle 2,
+tc11 = BTContainer(data=[
+    # D   O     H     L     C    V    B  S
+    [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
+    [1, 5000, 5050, 4950, 5100, 6172, 0, 0],
+    [2, 5100, 5251, 5100, 5100, 6172, 0, 0],
+    [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
+    [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
+    stop_loss=-0.10, roi=0.10, profit_perc=0.019, trailing_stop=True,
+    trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.05,
+    trailing_stop_positive=0.03,
+    trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
+)
+
+# Test 12 - trailing_stop should raise in candle 2 and cause a stoploss in the same candle
+# applying a positive trailing stop of 3% since stop_positive_offset is reached.
+# Set stop-loss at 10%, ROI at 10% (should not apply)
+# TC12: Trailing stoploss - stoploss should be adjusted candle 2,
+tc12 = BTContainer(data=[
+    # D   O     H     L     C    V    B  S
+    [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
+    [1, 5000, 5050, 4950, 5100, 6172, 0, 0],
+    [2, 5100, 5251, 4650, 5100, 6172, 0, 0],
+    [3, 4850, 5050, 4650, 4750, 6172, 0, 0],
+    [4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
+    stop_loss=-0.10, roi=0.10, profit_perc=0.019, trailing_stop=True,
+    trailing_only_offset_is_reached=True, trailing_stop_positive_offset=0.05,
+    trailing_stop_positive=0.03,
+    trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=2)]
+)
+
 TESTS = [
+    tc0,
     tc1,
     tc2,
     tc3,
@@ -168,6 +235,9 @@ TESTS = [
     tc7,
     tc8,
     tc9,
+    tc10,
+    tc11,
+    tc12,
 ]
 
 
@@ -180,6 +250,13 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
     default_conf["minimal_roi"] = {"0": data.roi}
     default_conf["ticker_interval"] = tests_ticker_interval
     default_conf["trailing_stop"] = data.trailing_stop
+    default_conf["trailing_only_offset_is_reached"] = data.trailing_only_offset_is_reached
+    # Only add this to configuration If it's necessary
+    if data.trailing_stop_positive:
+        default_conf["trailing_stop_positive"] = data.trailing_stop_positive
+    default_conf["trailing_stop_positive_offset"] = data.trailing_stop_positive_offset
+    default_conf["experimental"] = {"use_sell_signal": data.use_sell_signal}
+
     mocker.patch("freqtrade.exchange.Exchange.get_fee", MagicMock(return_value=0.0))
     patch_exchange(mocker)
     frame = _build_backtest_dataframe(data.data)

freqtrade/tests/optimize/test_edge_cli.py

@@ -117,8 +117,10 @@ def test_start(mocker, fee, edge_conf, caplog) -> None:
 
 def test_edge_init(mocker, edge_conf) -> None:
     patch_exchange(mocker)
+    edge_conf['stake_amount'] = 20
     edge_cli = EdgeCli(edge_conf)
     assert edge_cli.config == edge_conf
+    assert edge_cli.config['stake_amount'] == 'unlimited'
     assert callable(edge_cli.edge.calculate)
 
 

freqtrade/tests/optimize/test_hyperopt.py

@@ -372,20 +372,21 @@ def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
     )
     patch_exchange(mocker)
 
-    default_conf.update({'config': 'config.json.example'})
+    default_conf.update({'config': 'config.json.example',
-    default_conf.update({'epochs': 1})
+                         'epochs': 1,
-    default_conf.update({'timerange': None})
+                         'timerange': None,
-    default_conf.update({'spaces': 'all'})
+                         'spaces': 'all',
-    default_conf.update({'hyperopt_jobs': 1})
+                         'hyperopt_jobs': 1, })
 
     hyperopt = Hyperopt(default_conf)
     hyperopt.strategy.tickerdata_to_dataframe = MagicMock()
 
     hyperopt.start()
     parallel.assert_called_once()
-
+    assert log_has('Best result:\nfoo result\nwith values:\n', caplog.record_tuples)
-    assert 'Best result:\nfoo result\nwith values:\n\n' in caplog.text
     assert dumper.called
+    # Should be called twice, once for tickerdata, once to save evaluations
+    assert dumper.call_count == 2
 
 
 def test_format_results(hyperopt):

freqtrade/tests/strategy/test_interface.py

@@ -111,7 +111,7 @@ def test_tickerdata_to_dataframe(default_conf) -> None:
 
     timerange = TimeRange(None, 'line', 0, -100)
     tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m', timerange=timerange)
-    tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', True)}
+    tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', fill_missing=True)}
     data = strategy.tickerdata_to_dataframe(tickerlist)
     assert len(data['UNITTEST/BTC']) == 102       # partial candle was removed
 

freqtrade/tests/strategy/test_strategy.py

@@ -63,15 +63,14 @@ def test_search_strategy():
 
 def test_load_strategy(result):
     resolver = StrategyResolver({'strategy': 'TestStrategy'})
-    metadata = {'pair': 'ETH/BTC'}
+    assert 'adx' in resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
-    assert 'adx' in resolver.strategy.advise_indicators(result, metadata=metadata)
 
 
 def test_load_strategy_byte64(result):
     with open("freqtrade/tests/strategy/test_strategy.py", "r") as file:
         encoded_string = urlsafe_b64encode(file.read().encode("utf-8")).decode("utf-8")
     resolver = StrategyResolver({'strategy': 'TestStrategy:{}'.format(encoded_string)})
-    assert 'adx' in resolver.strategy.advise_indicators(result, 'ETH/BTC')
+    assert 'adx' in resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
 
 
 def test_load_strategy_invalid_directory(result, caplog):
@@ -371,7 +370,7 @@ def test_deprecate_populate_indicators(result):
     with warnings.catch_warnings(record=True) as w:
         # Cause all warnings to always be triggered.
         warnings.simplefilter("always")
-        indicators = resolver.strategy.advise_indicators(result, 'ETH/BTC')
+        indicators = resolver.strategy.advise_indicators(result, {'pair': 'ETH/BTC'})
         assert len(w) == 1
         assert issubclass(w[-1].category, DeprecationWarning)
         assert "deprecated - check out the Sample strategy to see the current function headers!" \
@@ -380,7 +379,7 @@ def test_deprecate_populate_indicators(result):
     with warnings.catch_warnings(record=True) as w:
         # Cause all warnings to always be triggered.
         warnings.simplefilter("always")
-        resolver.strategy.advise_buy(indicators, 'ETH/BTC')
+        resolver.strategy.advise_buy(indicators, {'pair': 'ETH/BTC'})
         assert len(w) == 1
         assert issubclass(w[-1].category, DeprecationWarning)
         assert "deprecated - check out the Sample strategy to see the current function headers!" \
@@ -389,7 +388,7 @@ def test_deprecate_populate_indicators(result):
     with warnings.catch_warnings(record=True) as w:
         # Cause all warnings to always be triggered.
         warnings.simplefilter("always")
-        resolver.strategy.advise_sell(indicators, 'ETH_BTC')
+        resolver.strategy.advise_sell(indicators, {'pair': 'ETH_BTC'})
         assert len(w) == 1
         assert issubclass(w[-1].category, DeprecationWarning)
         assert "deprecated - check out the Sample strategy to see the current function headers!" \

freqtrade/tests/test_arguments.py

@@ -170,18 +170,18 @@ def test_parse_args_hyperopt_custom() -> None:
     assert call_args.func is not None
 
 
-def test_testdata_dl_options() -> None:
+def test_download_data_options() -> None:
     args = [
         '--pairs-file', 'file_with_pairs',
-        '--export', 'export/folder',
+        '--datadir', 'datadir/folder',
         '--days', '30',
         '--exchange', 'binance'
     ]
     arguments = Arguments(args, '')
-    arguments.testdata_dl_options()
+    arguments.download_data_options()
     args = arguments.parse_args()
     assert args.pairs_file == 'file_with_pairs'
-    assert args.export == 'export/folder'
+    assert args.datadir == 'datadir/folder'
     assert args.days == 30
     assert args.exchange == 'binance'
 

freqtrade/tests/test_configuration.py

@@ -15,7 +15,7 @@ from freqtrade.arguments import Arguments
 from freqtrade.configuration import Configuration, set_loggers
 from freqtrade.constants import DEFAULT_DB_DRYRUN_URL, DEFAULT_DB_PROD_URL
 from freqtrade.state import RunMode
-from freqtrade.tests.conftest import log_has
+from freqtrade.tests.conftest import log_has, log_has_re
 
 
 @pytest.fixture(scope="function")
@@ -470,21 +470,52 @@ def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None:
 def test_check_exchange(default_conf, caplog) -> None:
     configuration = Configuration(Namespace())
 
-    # Test a valid exchange
+    # Test an officially supported by Freqtrade team exchange
     default_conf.get('exchange').update({'name': 'BITTREX'})
     assert configuration.check_exchange(default_conf)
+    assert log_has_re(r"Exchange .* is officially supported by the Freqtrade development team\.",
+                      caplog.record_tuples)
+    caplog.clear()
 
-    # Test a valid exchange
+    # Test an officially supported by Freqtrade team exchange
     default_conf.get('exchange').update({'name': 'binance'})
     assert configuration.check_exchange(default_conf)
+    assert log_has_re(r"Exchange .* is officially supported by the Freqtrade development team\.",
+                      caplog.record_tuples)
+    caplog.clear()
 
-    # Test a invalid exchange
+    # Test an available exchange, supported by ccxt
+    default_conf.get('exchange').update({'name': 'kraken'})
+    assert configuration.check_exchange(default_conf)
+    assert log_has_re(r"Exchange .* is supported by ccxt and .* not officially supported "
+                      r"by the Freqtrade development team\. .*",
+                      caplog.record_tuples)
+    caplog.clear()
+
+    # Test a 'bad' exchange, which known to have serious problems
+    default_conf.get('exchange').update({'name': 'bitmex'})
+    assert not configuration.check_exchange(default_conf)
+    assert log_has_re(r"Exchange .* is known to not work with the bot yet\. "
+                      r"Use it only for development and testing purposes\.",
+                      caplog.record_tuples)
+    caplog.clear()
+
+    # Test a 'bad' exchange with check_for_bad=False
+    default_conf.get('exchange').update({'name': 'bitmex'})
+    assert configuration.check_exchange(default_conf, False)
+    assert log_has_re(r"Exchange .* is supported by ccxt and .* not officially supported "
+                      r"by the Freqtrade development team\. .*",
+                      caplog.record_tuples)
+    caplog.clear()
+
+    # Test an invalid exchange
     default_conf.get('exchange').update({'name': 'unknown_exchange'})
     configuration.config = default_conf
 
     with pytest.raises(
         OperationalException,
-        match=r'.*Exchange "unknown_exchange" not supported.*'
+        match=r'.*Exchange "unknown_exchange" is not supported by ccxt '
+              r'and therefore not available for the bot.*'
     ):
         configuration.check_exchange(default_conf)
 

freqtrade/tests/test_freqtradebot.py

@@ -105,6 +105,7 @@ def test_cleanup(mocker, default_conf, caplog) -> None:
 def test_worker_running(mocker, default_conf, caplog) -> None:
     mock_throttle = MagicMock()
     mocker.patch('freqtrade.worker.Worker._throttle', mock_throttle)
+    mocker.patch('freqtrade.persistence.Trade.stoploss_reinitialization', MagicMock())
 
     worker = get_patched_worker(mocker, default_conf)
 
@@ -2484,9 +2485,9 @@ def test_trailing_stop_loss(default_conf, limit_buy_order, fee, markets, caplog,
     mocker.patch.multiple(
         'freqtrade.exchange.Exchange',
         get_ticker=MagicMock(return_value={
-            'bid': 0.00000102,
+            'bid': 0.00001099,
-            'ask': 0.00000103,
+            'ask': 0.00001099,
-            'last': 0.00000102
+            'last': 0.00001099
         }),
         buy=MagicMock(return_value={'id': limit_buy_order['id']}),
         get_fee=fee,
@@ -2498,15 +2499,33 @@ def test_trailing_stop_loss(default_conf, limit_buy_order, fee, markets, caplog,
     freqtrade.strategy.min_roi_reached = MagicMock(return_value=False)
 
     freqtrade.create_trade()
-
     trade = Trade.query.first()
-    trade.update(limit_buy_order)
+    assert freqtrade.handle_trade(trade) is False
-    trade.max_rate = trade.open_rate * 1.003
+
+    # Raise ticker above buy price
+    mocker.patch('freqtrade.exchange.Exchange.get_ticker',
+                 MagicMock(return_value={
+                     'bid': 0.00001099 * 1.5,
+                     'ask': 0.00001099 * 1.5,
+                     'last': 0.00001099 * 1.5
+                 }))
+
+    # Stoploss should be adjusted
+    assert freqtrade.handle_trade(trade) is False
+
+    # Price fell
+    mocker.patch('freqtrade.exchange.Exchange.get_ticker',
+                 MagicMock(return_value={
+                     'bid': 0.00001099 * 1.1,
+                     'ask': 0.00001099 * 1.1,
+                     'last': 0.00001099 * 1.1
+                 }))
+
     caplog.set_level(logging.DEBUG)
     # Sell as trailing-stop is reached
     assert freqtrade.handle_trade(trade) is True
     assert log_has(
-        f'HIT STOP: current price at 0.000001, stop loss is {trade.stop_loss:.6f}, '
+        f'HIT STOP: current price at 0.000012, stop loss is 0.000015, '
         f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog.record_tuples)
     assert trade.sell_reason == SellType.TRAILING_STOP_LOSS.value
 
@@ -3126,10 +3145,27 @@ def test_get_sell_rate(default_conf, mocker, ticker, order_book_l2) -> None:
     assert rate == 0.043936
 
 
-def test_startup_messages(default_conf, mocker):
+def test_startup_state(default_conf, mocker):
     default_conf['pairlist'] = {'method': 'VolumePairList',
                                 'config': {'number_assets': 20}
                                 }
     mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
     worker = get_patched_worker(mocker, default_conf)
     assert worker.state is State.RUNNING
+
+
+def test_startup_trade_reinit(default_conf, edge_conf, mocker):
+
+    mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
+    reinit_mock = MagicMock()
+    mocker.patch('freqtrade.persistence.Trade.stoploss_reinitialization', reinit_mock)
+
+    ftbot = get_patched_freqtradebot(mocker, default_conf)
+    ftbot.startup()
+    assert reinit_mock.call_count == 1
+
+    reinit_mock.reset_mock()
+
+    ftbot = get_patched_freqtradebot(mocker, edge_conf)
+    ftbot.startup()
+    assert reinit_mock.call_count == 0

freqtrade/tests/test_persistence.py

@@ -777,3 +777,63 @@ def test_to_json(default_conf, fee):
                       'stop_loss_pct': None,
                       'initial_stop_loss': None,
                       'initial_stop_loss_pct': None}
+
+
+def test_stoploss_reinitialization(default_conf, fee):
+    init(default_conf['db_url'])
+    trade = Trade(
+        pair='ETH/BTC',
+        stake_amount=0.001,
+        fee_open=fee.return_value,
+        open_date=arrow.utcnow().shift(hours=-2).datetime,
+        amount=10,
+        fee_close=fee.return_value,
+        exchange='bittrex',
+        open_rate=1,
+        max_rate=1,
+    )
+
+    trade.adjust_stop_loss(trade.open_rate, 0.05, True)
+    assert trade.stop_loss == 0.95
+    assert trade.stop_loss_pct == -0.05
+    assert trade.initial_stop_loss == 0.95
+    assert trade.initial_stop_loss_pct == -0.05
+    Trade.session.add(trade)
+
+    # Lower stoploss
+    Trade.stoploss_reinitialization(0.06)
+
+    trades = Trade.get_open_trades()
+    assert len(trades) == 1
+    trade_adj = trades[0]
+    assert trade_adj.stop_loss == 0.94
+    assert trade_adj.stop_loss_pct == -0.06
+    assert trade_adj.initial_stop_loss == 0.94
+    assert trade_adj.initial_stop_loss_pct == -0.06
+
+    # Raise stoploss
+    Trade.stoploss_reinitialization(0.04)
+
+    trades = Trade.get_open_trades()
+    assert len(trades) == 1
+    trade_adj = trades[0]
+    assert trade_adj.stop_loss == 0.96
+    assert trade_adj.stop_loss_pct == -0.04
+    assert trade_adj.initial_stop_loss == 0.96
+    assert trade_adj.initial_stop_loss_pct == -0.04
+
+    # Trailing stoploss (move stoplos up a bit)
+    trade.adjust_stop_loss(1.02, 0.04)
+    assert trade_adj.stop_loss == 0.9792
+    assert trade_adj.initial_stop_loss == 0.96
+
+    Trade.stoploss_reinitialization(0.04)
+
+    trades = Trade.get_open_trades()
+    assert len(trades) == 1
+    trade_adj = trades[0]
+    # Stoploss should not change in this case.
+    assert trade_adj.stop_loss == 0.9792
+    assert trade_adj.stop_loss_pct == -0.04
+    assert trade_adj.initial_stop_loss == 0.96
+    assert trade_adj.initial_stop_loss_pct == -0.04

freqtrade/worker.py

@@ -91,7 +91,7 @@ class Worker(object):
             })
             logger.info('Changing state to: %s', state.name)
             if state == State.RUNNING:
-                self.freqtrade.rpc.startup_messages(self._config, self.freqtrade.pairlists)
+                self.freqtrade.startup()
 
         if state == State.STOPPED:
             # Ping systemd watchdog before sleeping in the stopped state

requirements-common.txt

@@ -1,9 +1,9 @@
 # requirements without requirements installable via conda
 # mainly used for Raspberry pi installs
-ccxt==1.18.578
+ccxt==1.18.667
-SQLAlchemy==1.3.3
+SQLAlchemy==1.3.4
 python-telegram-bot==11.1.0
-arrow==0.13.2
+arrow==0.14.2
 cachetools==3.1.1
 requests==2.22.0
 urllib3==1.24.2  # pyup: ignore
@@ -23,10 +23,10 @@ filelock==3.0.12
 py_find_1st==1.1.3
 
 #Load ticker files 30% faster
-python-rapidjson==0.7.1
+python-rapidjson==0.7.2
 
 # Notify systemd
 sdnotify==0.3.2
 
 # Api server
-flask==1.0.2
+flask==1.0.3

requirements-dev.txt

@@ -4,9 +4,9 @@
 flake8==3.7.7
 flake8-type-annotations==0.1.0
 flake8-tidy-imports==2.0.0
-pytest==4.5.0
+pytest==4.6.2
 pytest-mock==1.10.4
 pytest-asyncio==0.10.0
 pytest-cov==2.7.1
-coveralls==1.7.0
+coveralls==1.8.0
 mypy==0.701

requirements-plot.txt

@@ -1,5 +1,5 @@
 # Include all requirements to run the bot.
 -r requirements.txt
 
-plotly==3.9.0
+plotly==3.10.0
 

requirements.txt

@@ -1,6 +1,6 @@
 # Load common requirements
 -r requirements-common.txt
 
-numpy==1.16.3
+numpy==1.16.4
 pandas==0.24.2
 scipy==1.3.0

scripts/download_backtest_data.py

@@ -1,55 +1,66 @@
 #!/usr/bin/env python3
 """
-This script generates json data
+This script generates json files with pairs history data
 """
+import arrow
 import json
 import sys
 from pathlib import Path
-import arrow
+from typing import Any, Dict, List
-from typing import Any, Dict
 
-from freqtrade.arguments import Arguments
+from freqtrade.arguments import Arguments, TimeRange
-from freqtrade.arguments import TimeRange
+from freqtrade.configuration import Configuration
-from freqtrade.exchange import Exchange
 from freqtrade.data.history import download_pair_history
-from freqtrade.configuration import Configuration, set_loggers
+from freqtrade.exchange import Exchange
 from freqtrade.misc import deep_merge_dicts
 
 import logging
-logging.basicConfig(
+
-    level=logging.INFO,
+logger = logging.getLogger('download_backtest_data')
-    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
-)
-set_loggers(0)
 
 DEFAULT_DL_PATH = 'user_data/data'
 
 arguments = Arguments(sys.argv[1:], 'download utility')
-arguments.testdata_dl_options()
+arguments.download_data_options()
-args = arguments.parse_args()
 
-timeframes = args.timeframes
+# Do not read the default config if config is not specified
+# in the command line options explicitely
+args = arguments.parse_args(no_default_config=True)
+
+# Use bittrex as default exchange
+exchange_name = args.exchange or 'bittrex'
+
+pairs: List = []
+
+configuration = Configuration(args)
+config: Dict[str, Any] = {}
 
 if args.config:
-    configuration = Configuration(args)
-
-    config: Dict[str, Any] = {}
     # Now expecting a list of config filenames here, not a string
     for path in args.config:
-        print(f"Using config: {path}...")
+        logger.info(f"Using config: {path}...")
         # Merge config options, overwriting old values
         config = deep_merge_dicts(configuration._load_config_file(path), config)
 
     config['stake_currency'] = ''
     # Ensure we do not use Exchange credentials
+    config['exchange']['dry_run'] = True
     config['exchange']['key'] = ''
     config['exchange']['secret'] = ''
+
+    pairs = config['exchange']['pair_whitelist']
+
+    if config.get('ticker_interval'):
+        timeframes = args.timeframes or [config.get('ticker_interval')]
+    else:
+        timeframes = args.timeframes or ['1m', '5m']
+
 else:
     config = {
         'stake_currency': '',
         'dry_run': True,
         'exchange': {
-            'name': args.exchange,
+            'name': exchange_name,
             'key': '',
             'secret': '',
             'pair_whitelist': [],
@@ -59,56 +70,72 @@ else:
             }
         }
     }
+    timeframes = args.timeframes or ['1m', '5m']
 
+configuration._load_logging_config(config)
 
-dl_path = Path(DEFAULT_DL_PATH).joinpath(config['exchange']['name'])
+if args.config and args.exchange:
-if args.export:
+    logger.warning("The --exchange option is ignored, "
-    dl_path = Path(args.export)
+                   "using exchange settings from the configuration file.")
 
-if not dl_path.is_dir():
+# Check if the exchange set by the user is supported
-    sys.exit(f'Directory {dl_path} does not exist.')
+configuration.check_exchange(config)
+
+configuration._load_datadir_config(config)
+
+dl_path = Path(config['datadir'])
 
 pairs_file = Path(args.pairs_file) if args.pairs_file else dl_path.joinpath('pairs.json')
-if not pairs_file.exists():
-    sys.exit(f'No pairs file found with path {pairs_file}.')
 
-with pairs_file.open() as file:
+if not pairs or args.pairs_file:
-    PAIRS = list(set(json.load(file)))
+    logger.info(f'Reading pairs file "{pairs_file}".')
+    # Download pairs from the pairs file if no config is specified
+    # or if pairs file is specified explicitely
+    if not pairs_file.exists():
+        sys.exit(f'No pairs file found with path "{pairs_file}".')
 
-PAIRS.sort()
+    with pairs_file.open() as file:
+        pairs = list(set(json.load(file)))
 
+    pairs.sort()
 
 timerange = TimeRange()
 if args.days:
     time_since = arrow.utcnow().shift(days=-args.days).strftime("%Y%m%d")
     timerange = arguments.parse_timerange(f'{time_since}-')
 
+logger.info(f'About to download pairs: {pairs}, intervals: {timeframes} to {dl_path}')
 
-print(f'About to download pairs: {PAIRS} to {dl_path}')
-
-# Init exchange
-exchange = Exchange(config)
 pairs_not_available = []
 
-for pair in PAIRS:
+try:
+    # Init exchange
+    exchange = Exchange(config)
+
+    for pair in pairs:
         if pair not in exchange._api.markets:
             pairs_not_available.append(pair)
-        print(f"skipping pair {pair}")
+            logger.info(f"Skipping pair {pair}...")
             continue
         for ticker_interval in timeframes:
             pair_print = pair.replace('/', '_')
             filename = f'{pair_print}-{ticker_interval}.json'
             dl_file = dl_path.joinpath(filename)
             if args.erase and dl_file.exists():
-            print(f'Deleting existing data for pair {pair}, interval {ticker_interval}')
+                logger.info(
+                    f'Deleting existing data for pair {pair}, interval {ticker_interval}.')
                 dl_file.unlink()
 
-        print(f'downloading pair {pair}, interval {ticker_interval}')
+            logger.info(f'Downloading pair {pair}, interval {ticker_interval}.')
             download_pair_history(datadir=dl_path, exchange=exchange,
-                              pair=pair,
+                                  pair=pair, ticker_interval=str(ticker_interval),
-                              ticker_interval=ticker_interval,
                                   timerange=timerange)
 
+except KeyboardInterrupt:
+    sys.exit("SIGINT received, aborting ...")
 
-if pairs_not_available:
+finally:
-    print(f"Pairs [{','.join(pairs_not_available)}] not availble.")
+    if pairs_not_available:
+        logger.info(
+            f"Pairs [{','.join(pairs_not_available)}] not available "
+            f"on exchange {config['exchange']['name']}.")

user_data/strategies/test_strategy.py

@@ -44,8 +44,8 @@ class TestStrategy(IStrategy):
 
     # trailing stoploss
     trailing_stop = False
-    trailing_stop_positive = 0.01
+    # trailing_stop_positive = 0.01
-    trailing_stop_positive_offset = 0.0  # Disabled / not configured
+    # trailing_stop_positive_offset = 0.0  # Disabled / not configured
 
     # Optimal ticker interval for the strategy
     ticker_interval = '5m'
@@ -253,6 +253,17 @@ class TestStrategy(IStrategy):
         dataframe['ha_low'] = heikinashi['low']
         """
 
+        # Retrieve best bid and best ask
+        # ------------------------------------
+        """
+        # first check if dataprovider is available 
+        if self.dp:
+            if self.dp.runmode in ('live', 'dry_run'):
+                ob = self.dp.orderbook(metadata['pair'], 1)
+                dataframe['best_bid'] = ob['bids'][0][0]
+                dataframe['best_ask'] = ob['asks'][0][0]
+        """
+        
         return dataframe
 
     def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: