adelorenzo
/
stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into feat/new_args_system

This commit is contained in:
hroff-1902 2019-10-23 22:45:06 +03:00 committed by GitHub
parent 1c503f39b2 a135eaa993
commit 4ce278a06e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
20 changed files with 785 additions and 73 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/backtesting.md
View File

@ -201,6 +201,8 @@ Since backtesting lacks some detailed information about what happens within a ca
Taking these assumptions, backtesting tries to mirror real trading as closely as possible. However, backtesting will **never** replace running a strategy in dry-run mode.
Also, keep in mind that past results don't guarantee future success.
In addition to the above assumptions, strategy authors should carefully read the [Common Mistakes](strategy-customization.md#common-mistakes-when-developing-strategies) section, to avoid using data in backtesting which is not available in real market conditions.
### Further backtest-result analysis
To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).

21
docs/strategy-customization.md
View File

@ -60,8 +60,7 @@ file as reference.**
!!! Warning Using future data
Since backtesting passes the full time interval to the `populate_*()` methods, the strategy author
needs to take care to avoid having the strategy utilize data from the future.
Samples for usage of future data are `dataframe.shift(-1)`, `dataframe.resample("1h")` (this uses the left border of the interval, so moves data from an hour to the start of the hour).
They all use data which is not available during regular operations, so these strategies will perform well during backtesting, but will fail / perform badly in dry-runs.
Some common patterns for this are listed in the [Common Mistakes](#common-mistakes-when-developing-strategies) section of this document.
### Customize Indicators
@ -399,10 +398,10 @@ def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
Printing more than a few rows is also possible (simply use `print(dataframe)` instead of `print(dataframe.tail())`), however not recommended, as that will be very verbose (~500 lines per pair every 5 seconds).
### Where is the default strategy?
### Where can i find a strategy template?
The default buy strategy is located in the file
[freqtrade/default_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/strategy/default_strategy.py).
The strategy template is located in the file
[user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py).
### Specify custom strategy location
@ -412,6 +411,18 @@ If you want to use a strategy from a different directory you can pass `--strateg
freqtrade trade --strategy AwesomeStrategy --strategy-path /some/directory
```
### Common mistakes when developing strategies
Backtesting analyzes the whole time-range at once for performance reasons. Because of this, strategy authors need to make sure that strategies do not look-ahead into the future.
This is a common pain-point, which can cause huge differences between backtesting and dry/live run methods, since they all use data which is not available during dry/live runs, so these strategies will perform well during backtesting, but will fail / perform badly in real conditions.
The following lists some common patterns which should be avoided to prevent frustration:
- don't use `shift(-1)`. This uses data from the future, which is not available.
- don't use `.iloc[-1]` or any other absolute position in the dataframe, this will be different between dry-run and backtesting.
- don't use `dataframe['volume'].mean()`. This uses the full DataFrame for backtesting, including data from the future. Use `dataframe['volume'].rolling(<window>).mean()` instead
- don't use `.resample('1h')`. This uses the left border of the interval, so moves data from an hour to the start of the hour. Use `.resample('1h', label='right')` instead.
### Further strategy ideas
To get additional Ideas for strategies, head over to our [strategy repository](https://github.com/freqtrade/freqtrade-strategies). Feel free to use them as they are - but results will depend on the current market situation, pairs used etc. - therefore please backtest the strategy for your exchange/desired pairs first, evaluate carefully, use at your own risk.

70
docs/utils.md
View File

@ -54,3 +54,73 @@ Timeframes available for the exchange `binance`: 1m, 3m, 5m, 15m, 30m, 1h, 2h, 4
```
$ for i in `freqtrade list-exchanges -1`; do freqtrade list-timeframes --exchange $i; done
```
## List pairs/list markets
The `list-pairs` and `list-markets` subcommands allow to see the pairs/markets available on exchange.
Pairs are markets with the '/' character between the base currency part and the quote currency part in the market symbol.
For example, in the 'ETH/BTC' pair 'ETH' is the base currency, while 'BTC' is the quote currency.
For pairs traded by Freqtrade the pair quote currency is defined by the value of the `stake_currency` configuration setting.
You can print info about any pair/market with these subcommands - and you can filter output by quote-currency using `--quote BTC`, or by base-currency using `--base ETH` options correspondingly.
These subcommands have same usage and same set of available options:
```
usage: freqtrade list-markets [-h] [--exchange EXCHANGE] [--print-list]
[--print-json] [-1] [--print-csv]
[--base BASE_CURRENCY [BASE_CURRENCY ...]]
[--quote QUOTE_CURRENCY [QUOTE_CURRENCY ...]]
[-a]
usage: freqtrade list-pairs [-h] [--exchange EXCHANGE] [--print-list]
[--print-json] [-1] [--print-csv]
[--base BASE_CURRENCY [BASE_CURRENCY ...]]
[--quote QUOTE_CURRENCY [QUOTE_CURRENCY ...]] [-a]
optional arguments:
-h, --help show this help message and exit
--exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
config is provided.
--print-list Print list of pairs or market symbols. By default data
is printed in the tabular format.
--print-json Print list of pairs or market symbols in JSON format.
-1, --one-column Print output in one column.
--print-csv Print exchange pair or market data in the csv format.
--base BASE_CURRENCY [BASE_CURRENCY ...]
Specify base currency(-ies). Space-separated list.
--quote QUOTE_CURRENCY [QUOTE_CURRENCY ...]
Specify quote currency(-ies). Space-separated list.
-a, --all Print all pairs or market symbols. By default only
active ones are shown.
```
By default, only active pairs/markets are shown. Active pairs/markets are those that can currently be traded
on the exchange. The see the list of all pairs/markets (not only the active ones), use the `-a`/`-all` option.
Pairs/markets are sorted by its symbol string in the printed output.
### Examples
* Print the list of active pairs with quote currency USD on exchange, specified in the default
configuration file (i.e. pairs on the "Bittrex" exchange) in JSON format:
```
$ freqtrade list-pairs --quote USD --print-json
```
* Print the list of all pairs on the exchange, specified in the `config_binance.json` configuration file
(i.e. on the "Binance" exchange) with base currencies BTC or ETH and quote currencies USDT or USD, as the
human-readable list with summary:
```
$ freqtrade -c config_binance.json list-pairs --all --base BTC ETH --quote USDT USD --print-list
```
* Print all markets on exchange "Kraken", in the tabular format:
```
$ freqtrade list-markets --exchange kraken --all
```

26
freqtrade/configuration/arguments.py
View File

@ -2,6 +2,7 @@
This module contains the argument manager class
"""
import argparse
from functools import partial
from pathlib import Path
from typing import Any, Dict, List, Optional
@ -33,6 +34,9 @@ ARGS_LIST_EXCHANGES = ["print_one_column", "list_exchanges_all"]
ARGS_LIST_TIMEFRAMES = ["exchange", "print_one_column"]
ARGS_LIST_PAIRS = ["exchange", "print_list", "list_pairs_print_json", "print_one_column",
"print_csv", "base_currencies", "quote_currencies", "list_pairs_all"]
ARGS_CREATE_USERDIR = ["user_data_dir"]
ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "download_trades", "exchange",
@ -45,7 +49,8 @@ ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit",
ARGS_PLOT_PROFIT = ["pairs", "timerange", "export", "exportfilename", "db_url",
"trade_source", "ticker_interval"]
NO_CONF_REQURIED = ["download-data", "list-timeframes", "plot-dataframe", "plot-profit"]
NO_CONF_REQURIED = ["download-data", "list-timeframes", "list-markets", "list-pairs",
"plot-dataframe", "plot-profit"]
NO_CONF_ALLOWED = ["create-userdir", "list-exchanges"]
@ -111,7 +116,8 @@ class Arguments:
from freqtrade.optimize import start_backtesting, start_hyperopt, start_edge
from freqtrade.utils import (start_create_userdir, start_download_data,
start_list_exchanges, start_list_timeframes, start_trading)
start_list_exchanges, start_list_markets,
start_list_timeframes, start_trading)
from freqtrade.plot.plot_utils import start_plot_dataframe, start_plot_profit
subparsers = self.parser.add_subparsers(dest='command',
@ -170,6 +176,22 @@ class Arguments:
list_timeframes_cmd.set_defaults(func=start_list_timeframes)
self._build_args(optionlist=ARGS_LIST_TIMEFRAMES, parser=list_timeframes_cmd)
# Add list-markets subcommand
list_markets_cmd = subparsers.add_parser(
'list-markets',
help='Print markets on exchange.'
)
list_markets_cmd.set_defaults(func=partial(start_list_markets, pairs_only=False))
self._build_args(optionlist=ARGS_LIST_PAIRS, parser=list_markets_cmd)
# Add list-pairs subcommand
list_pairs_cmd = subparsers.add_parser(
'list-pairs',
help='Print pairs on exchange.'
)
list_pairs_cmd.set_defaults(func=partial(start_list_markets, pairs_only=True))
self._build_args(optionlist=ARGS_LIST_PAIRS, parser=list_pairs_cmd)
# Add download-data subcommand
download_data_cmd = subparsers.add_parser(
'download-data',

36
freqtrade/configuration/cli_options.py
View File

@ -255,6 +255,42 @@ AVAILABLE_CLI_OPTIONS = {
help='Print all exchanges known to the ccxt library.',
action='store_true',
),
# List pairs / markets
"list_pairs_all": Arg(
'-a', '--all',
help='Print all pairs or market symbols. By default only active '
'ones are shown.',
action='store_true',
),
"print_list": Arg(
'--print-list',
help='Print list of pairs or market symbols. By default data is '
'printed in the tabular format.',
action='store_true',
),
"list_pairs_print_json": Arg(
'--print-json',
help='Print list of pairs or market symbols in JSON format.',
action='store_true',
default=False,
),
"print_csv": Arg(
'--print-csv',
help='Print exchange pair or market data in the csv format.',
action='store_true',
),
"quote_currencies": Arg(
'--quote',
help='Specify quote currency(-ies). Space-separated list.',
nargs='+',
metavar='QUOTE_CURRENCY',
),
"base_currencies": Arg(
'--base',
help='Specify base currency(-ies). Space-separated list.',
nargs='+',
metavar='BASE_CURRENCY',
),
# Script options
"pairs": Arg(
'-p', '--pairs',

2
freqtrade/exchange/__init__.py
View File

@ -10,5 +10,7 @@ from freqtrade.exchange.exchange import (timeframe_to_seconds, # noqa: F401
timeframe_to_msecs,
timeframe_to_next_date,
timeframe_to_prev_date)
from freqtrade.exchange.exchange import (market_is_active, # noqa: F401
symbol_is_pair)
from freqtrade.exchange.kraken import Kraken # noqa: F401
from freqtrade.exchange.binance import Binance # noqa: F401

69
freqtrade/exchange/exchange.py
View File

@ -22,6 +22,7 @@ from freqtrade import (DependencyException, InvalidOrderException,
from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.misc import deep_merge_dicts
logger = logging.getLogger(__name__)
@ -165,7 +166,7 @@ class Exchange:
}
_ft_has: Dict = {}
def __init__(self, config: dict) -> None:
def __init__(self, config: dict, validate: bool = True) -> None:
"""
Initializes this module with the given config,
it does basic validation whether the specified exchange and pairs are valid.
@ -216,19 +217,21 @@ class Exchange:
logger.info('Using Exchange "%s"', self.name)
# Check if timeframe is available
self.validate_timeframes(config.get('ticker_interval'))
if validate:
# Check if timeframe is available
self.validate_timeframes(config.get('ticker_interval'))
# Initial markets load
self._load_markets()
# Check if all pairs are available
self.validate_pairs(config['exchange']['pair_whitelist'])
self.validate_ordertypes(config.get('order_types', {}))
self.validate_order_time_in_force(config.get('order_time_in_force', {}))
# Converts the interval provided in minutes in config to seconds
self.markets_refresh_interval: int = exchange_config.get(
"markets_refresh_interval", 60) * 60
# Initial markets load
self._load_markets()
# Check if all pairs are available
self.validate_pairs(config['exchange']['pair_whitelist'])
self.validate_ordertypes(config.get('order_types', {}))
self.validate_order_time_in_force(config.get('order_time_in_force', {}))
def __del__(self):
"""
@ -293,6 +296,28 @@ class Exchange:
self._load_markets()
return self._api.markets
def get_markets(self, base_currencies: List[str] = None, quote_currencies: List[str] = None,
pairs_only: bool = False, active_only: bool = False) -> Dict:
"""
Return exchange ccxt markets, filtered out by base currency and quote currency
if this was requested in parameters.
TODO: consider moving it to the Dataprovider
"""
markets = self.markets
if not markets:
raise OperationalException("Markets were not loaded.")
if base_currencies:
markets = {k: v for k, v in markets.items() if v['base'] in base_currencies}
if quote_currencies:
markets = {k: v for k, v in markets.items() if v['quote'] in quote_currencies}
if pairs_only:
markets = {k: v for k, v in markets.items() if symbol_is_pair(v['symbol'])}
if active_only:
markets = {k: v for k, v in markets.items() if market_is_active(v)}
return markets
def klines(self, pair_interval: Tuple[str, str], copy=True) -> DataFrame:
if pair_interval in self._klines:
return self._klines[pair_interval].copy() if copy else self._klines[pair_interval]
@ -1074,3 +1099,27 @@ def timeframe_to_next_date(timeframe: str, date: datetime = None) -> datetime:
new_timestamp = ccxt.Exchange.round_timeframe(timeframe, date.timestamp() * 1000,
ROUND_UP) // 1000
return datetime.fromtimestamp(new_timestamp, tz=timezone.utc)
def symbol_is_pair(market_symbol: str, base_currency: str = None, quote_currency: str = None):
"""
Check if the market symbol is a pair, i.e. that its symbol consists of the base currency and the
quote currency separated by '/' character. If base_currency and/or quote_currency is passed,
it also checks that the symbol contains appropriate base and/or quote currency part before
and after the separating character correspondingly.
"""
symbol_parts = market_symbol.split('/')
return (len(symbol_parts) == 2 and
(symbol_parts[0] == base_currency if base_currency else len(symbol_parts[0]) > 0) and
(symbol_parts[1] == quote_currency if quote_currency else len(symbol_parts[1]) > 0))
def market_is_active(market):
"""
Return True if the market is active.
"""
# "It's active, if the active flag isn't explicitly set to false. If it's missing or
# true then it's true. If it's undefined, then it's most likely true, but not 100% )"
# See https://github.com/ccxt/ccxt/issues/4874,
# https://github.com/ccxt/ccxt/issues/4075#issuecomment-434760520
return market.get('active', True) is not False

4
freqtrade/misc.py
View File

@ -123,3 +123,7 @@ def round_dict(d, n):
Rounds float values in the dict to n digits after the decimal point.
"""
return {k: (round(v, n) if isinstance(v, float) else v) for k, v in d.items()}
def plural(num, singular: str, plural: str = None) -> str:
return singular if (num == 1 or num == -1) else plural or singular + 's'

5
freqtrade/pairlist/IPairList.py
View File

@ -8,6 +8,9 @@ import logging
from abc import ABC, abstractmethod
from typing import List
from freqtrade.exchange import market_is_active
logger = logging.getLogger(__name__)
@ -77,7 +80,7 @@ class IPairList(ABC):
continue
# Check if market is active
market = markets[pair]
if not market['active']:
if not market_is_active(market):
logger.info(f"Ignoring {pair} from whitelist. Market is not active.")
continue
sanitized_whitelist.add(pair)

9
freqtrade/resolvers/exchange_resolver.py
View File

@ -17,7 +17,7 @@ class ExchangeResolver(IResolver):
__slots__ = ['exchange']
def __init__(self, exchange_name: str, config: dict) -> None:
def __init__(self, exchange_name: str, config: dict, validate: bool = True) -> None:
"""
Load the custom class from config parameter
:param config: configuration dictionary
@ -26,12 +26,13 @@ class ExchangeResolver(IResolver):
exchange_name = MAP_EXCHANGE_CHILDCLASS.get(exchange_name, exchange_name)
exchange_name = exchange_name.title()
try:
self.exchange = self._load_exchange(exchange_name, kwargs={'config': config})
self.exchange = self._load_exchange(exchange_name, kwargs={'config': config,
'validate': validate})
except ImportError:
logger.info(
f"No {exchange_name} specific subclass found. Using the generic class instead.")
if not hasattr(self, "exchange"):
self.exchange = Exchange(config)
self.exchange = Exchange(config, validate=validate)
def _load_exchange(
self, exchange_name: str, kwargs: dict) -> Exchange:
@ -45,7 +46,7 @@ class ExchangeResolver(IResolver):
try:
ex_class = getattr(exchanges, exchange_name)
exchange = ex_class(kwargs['config'])
exchange = ex_class(**kwargs)
if exchange:
logger.info(f"Using resolved exchange '{exchange_name}'...")
return exchange

56
freqtrade/rpc/api_server.py
View File

@ -2,7 +2,7 @@ import logging
import threading
from datetime import date, datetime
from ipaddress import IPv4Address
from typing import Dict
from typing import Dict, Callable, Any
from arrow import Arrow
from flask import Flask, jsonify, request
@ -34,41 +34,45 @@ class ArrowJSONEncoder(JSONEncoder):
return JSONEncoder.default(self, obj)
# Type should really be Callable[[ApiServer, Any], Any], but that will create a circular dependency
def require_login(func: Callable[[Any, Any], Any]):
def func_wrapper(obj, *args, **kwargs):
auth = request.authorization
if auth and obj.check_auth(auth.username, auth.password):
return func(obj, *args, **kwargs)
else:
return jsonify({"error": "Unauthorized"}), 401
return func_wrapper
# Type should really be Callable[[ApiServer], Any], but that will create a circular dependency
def rpc_catch_errors(func: Callable[[Any], Any]):
def func_wrapper(obj, *args, **kwargs):
try:
return func(obj, *args, **kwargs)
except RPCException as e:
logger.exception("API Error calling %s: %s", func.__name__, e)
return obj.rest_error(f"Error querying {func.__name__}: {e}")
return func_wrapper
class ApiServer(RPC):
"""
This class runs api server and provides rpc.rpc functionality to it
This class starts a none blocking thread the api server runs within
This class starts a non-blocking thread the api server runs within
"""
def rpc_catch_errors(func):
def func_wrapper(self, *args, **kwargs):
try:
return func(self, *args, **kwargs)
except RPCException as e:
logger.exception("API Error calling %s: %s", func.__name__, e)
return self.rest_error(f"Error querying {func.__name__}: {e}")
return func_wrapper
def check_auth(self, username, password):
return (username == self._config['api_server'].get('username') and
password == self._config['api_server'].get('password'))
def require_login(func):
def func_wrapper(self, *args, **kwargs):
auth = request.authorization
if auth and self.check_auth(auth.username, auth.password):
return func(self, *args, **kwargs)
else:
return jsonify({"error": "Unauthorized"}), 401
return func_wrapper
def __init__(self, freqtrade) -> None:
"""
Init the api server, and init the super class RPC

96
freqtrade/utils.py
View File

@ -1,9 +1,13 @@
import logging
import sys
from collections import OrderedDict
from pathlib import Path
from typing import Any, Dict, List
import arrow
import csv
import rapidjson
from tabulate import tabulate
from freqtrade import OperationalException
from freqtrade.configuration import Configuration, TimeRange
@ -11,7 +15,9 @@ from freqtrade.configuration.directory_operations import create_userdata_dir
from freqtrade.data.history import (convert_trades_to_ohlcv,
refresh_backtest_ohlcv_data,
refresh_backtest_trades_data)
from freqtrade.exchange import available_exchanges, ccxt_exchanges
from freqtrade.exchange import (available_exchanges, ccxt_exchanges, market_is_active,
symbol_is_pair)
from freqtrade.misc import plural
from freqtrade.resolvers import ExchangeResolver
from freqtrade.state import RunMode
@ -133,10 +139,94 @@ def start_list_timeframes(args: Dict[str, Any]) -> None:
config['ticker_interval'] = None
# Init exchange
exchange = ExchangeResolver(config['exchange']['name'], config).exchange
exchange = ExchangeResolver(config['exchange']['name'], config, validate=False).exchange
if args['print_one_column']:
print('\n'.join(exchange.timeframes))
else:
print(f"Timeframes available for the exchange `{config['exchange']['name']}`: "
f"{', '.join(exchange.timeframes)}")
f"{', '.join(exchange.timeframes)}.")
def start_list_markets(args: Dict[str, Any], pairs_only: bool = False) -> None:
"""
Print pairs/markets on the exchange
:param args: Cli args from Arguments()
:param pairs_only: if True print only pairs, otherwise print all instruments (markets)
:return: None
"""
config = setup_utils_configuration(args, RunMode.OTHER)
# Init exchange
exchange = ExchangeResolver(config['exchange']['name'], config, validate=False).exchange
# By default only active pairs/markets are to be shown
active_only = not args.get('list_pairs_all', False)
base_currencies = args.get('base_currencies', [])
quote_currencies = args.get('quote_currencies', [])
try:
pairs = exchange.get_markets(base_currencies=base_currencies,
quote_currencies=quote_currencies,
pairs_only=pairs_only,
active_only=active_only)
# Sort the pairs/markets by symbol
pairs = OrderedDict(sorted(pairs.items()))
except Exception as e:
raise OperationalException(f"Cannot get markets. Reason: {e}") from e
else:
summary_str = ((f"Exchange {exchange.name} has {len(pairs)} ") +
("active " if active_only else "") +
(plural(len(pairs), "pair" if pairs_only else "market")) +
(f" with {', '.join(base_currencies)} as base "
f"{plural(len(base_currencies), 'currency', 'currencies')}"
if base_currencies else "") +
(" and" if base_currencies and quote_currencies else "") +
(f" with {', '.join(quote_currencies)} as quote "
f"{plural(len(quote_currencies), 'currency', 'currencies')}"
if quote_currencies else ""))
headers = ["Id", "Symbol", "Base", "Quote", "Active",
*(['Is pair'] if not pairs_only else [])]
tabular_data = []
for _, v in pairs.items():
tabular_data.append({'Id': v['id'], 'Symbol': v['symbol'],
'Base': v['base'], 'Quote': v['quote'],
'Active': market_is_active(v),
**({'Is pair': symbol_is_pair(v['symbol'])}
if not pairs_only else {})})
if (args.get('print_one_column', False) or
args.get('list_pairs_print_json', False) or
args.get('print_csv', False)):
# Print summary string in the log in case of machine-readable
# regular formats.
logger.info(f"{summary_str}.")
else:
# Print empty string separating leading logs and output in case of
# human-readable formats.
print()
if len(pairs):
if args.get('print_list', False):
# print data as a list, with human-readable summary
print(f"{summary_str}: {', '.join(pairs.keys())}.")
elif args.get('print_one_column', False):
print('\n'.join(pairs.keys()))
elif args.get('list_pairs_print_json', False):
print(rapidjson.dumps(list(pairs.keys()), default=str))
elif args.get('print_csv', False):
writer = csv.DictWriter(sys.stdout, fieldnames=headers)
writer.writeheader()
writer.writerows(tabular_data)
else:
# print data as a table, with the human-readable summary
print(f"{summary_str}:")
print(tabulate(tabular_data, headers='keys', tablefmt='pipe'))
elif not (args.get('print_one_column', False) or
args.get('list_pairs_print_json', False) or
args.get('print_csv', False)):
print(f"{summary_str}.")

4
requirements-common.txt
View File

@ -1,8 +1,8 @@
# requirements without requirements installable via conda
# mainly used for Raspberry pi installs
ccxt==1.18.1260
ccxt==1.18.1306
SQLAlchemy==1.3.10
python-telegram-bot==12.1.1
python-telegram-bot==12.2.0
arrow==0.15.2
cachetools==3.1.1
requests==2.22.0

4
requirements-dev.txt
View File

@ -6,8 +6,8 @@
coveralls==1.8.2
flake8==3.7.8
flake8-type-annotations==0.1.0
flake8-tidy-imports==2.0.0
mypy==0.730
flake8-tidy-imports==3.0.0
mypy==0.740
pytest==5.2.1
pytest-asyncio==0.10.0
pytest-cov==2.8.1

2
requirements-plot.txt
View File

@ -1,5 +1,5 @@
# Include all requirements to run the bot.
-r requirements.txt
plotly==4.1.1
plotly==4.2.1

4
requirements.txt
View File

@ -1,5 +1,5 @@
# Load common requirements
-r requirements-common.txt
numpy==1.17.2
pandas==0.25.1
numpy==1.17.3
pandas==0.25.2

47
tests/conftest.py
View File

@ -319,7 +319,8 @@ def markets():
'symbol': 'TKN/BTC',
'base': 'TKN',
'quote': 'BTC',
'active': True,
# According to ccxt, markets without active item set are also active
# 'active': True,
'precision': {
'price': 8,
'amount': 8,
@ -510,6 +511,50 @@ def markets():
}
},
'info': {},
},
'LTC/USD': {
'id': 'USD-LTC',
'symbol': 'LTC/USD',
'base': 'LTC',
'quote': 'USD',
'active': True,
'precision': {
'amount': 8,
'price': 8
},
'limits': {
'amount': {
'min': 0.06646786,
'max': None
},
'price': {
'min': 1e-08,
'max': None
}
},
'info': {},
},
'XLTCUSDT': {
'id': 'xLTCUSDT',
'symbol': 'XLTCUSDT',
'base': 'LTC',
'quote': 'USDT',
'active': True,
'precision': {
'amount': 8,
'price': 8
},
'limits': {
'amount': {
'min': 0.06646786,
'max': None
},
'price': {
'min': 1e-08,
'max': None
}
},
'info': {},
}
}

124
tests/exchange/test_exchange.py
View File

@ -18,7 +18,9 @@ from freqtrade.exchange.exchange import (API_RETRY_COUNT, timeframe_to_minutes,
timeframe_to_msecs,
timeframe_to_next_date,
timeframe_to_prev_date,
timeframe_to_seconds)
timeframe_to_seconds,
symbol_is_pair,
market_is_active)
from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from tests.conftest import get_patched_exchange, log_has, log_has_re
@ -929,17 +931,17 @@ def test_get_balances_prod(default_conf, mocker, exchange_name):
def test_get_tickers(default_conf, mocker, exchange_name):
api_mock = MagicMock()
tick = {'ETH/BTC': {
'symbol': 'ETH/BTC',
'bid': 0.5,
'ask': 1,
'last': 42,
}, 'BCH/BTC': {
'symbol': 'BCH/BTC',
'bid': 0.6,
'ask': 0.5,
'last': 41,
}
}
'symbol': 'ETH/BTC',
'bid': 0.5,
'ask': 1,
'last': 42,
}, 'BCH/BTC': {
'symbol': 'BCH/BTC',
'bid': 0.6,
'ask': 0.5,
'last': 41,
}
}
api_mock.fetch_tickers = MagicMock(return_value=tick)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
# retrieve original ticker
@ -1693,6 +1695,74 @@ def test_get_valid_pair_combination(default_conf, mocker, markets):
ex.get_valid_pair_combination("NOPAIR", "ETH")
@pytest.mark.parametrize(
"base_currencies, quote_currencies, pairs_only, active_only, expected_keys", [
# Testing markets (in conftest.py):
# 'BLK/BTC': 'active': True
# 'BTT/BTC': 'active': True
# 'ETH/BTC': 'active': True
# 'ETH/USDT': 'active': True
# 'LTC/BTC': 'active': False
# 'LTC/USD': 'active': True
# 'LTC/USDT': 'active': True
# 'NEO/BTC': 'active': False
# 'TKN/BTC': 'active' not set
# 'XLTCUSDT': 'active': True, not a pair
# 'XRP/BTC': 'active': False
# all markets
([], [], False, False,
['BLK/BTC', 'BTT/BTC', 'ETH/BTC', 'ETH/USDT', 'LTC/BTC', 'LTC/USD',
'LTC/USDT', 'NEO/BTC', 'TKN/BTC', 'XLTCUSDT', 'XRP/BTC']),
# active markets
([], [], False, True,
['BLK/BTC', 'BTT/BTC', 'ETH/BTC', 'ETH/USDT', 'LTC/USD', 'LTC/USDT',
'TKN/BTC', 'XLTCUSDT']),
# all pairs
([], [], True, False,
['BLK/BTC', 'BTT/BTC', 'ETH/BTC', 'ETH/USDT', 'LTC/BTC', 'LTC/USD',
'LTC/USDT', 'NEO/BTC', 'TKN/BTC', 'XRP/BTC']),
# active pairs
([], [], True, True,
['BLK/BTC', 'BTT/BTC', 'ETH/BTC', 'ETH/USDT', 'LTC/USD', 'LTC/USDT', 'TKN/BTC']),
# all markets, base=ETH, LTC
(['ETH', 'LTC'], [], False, False,
['ETH/BTC', 'ETH/USDT', 'LTC/BTC', 'LTC/USD', 'LTC/USDT', 'XLTCUSDT']),
# all markets, base=LTC
(['LTC'], [], False, False,
['LTC/BTC', 'LTC/USD', 'LTC/USDT', 'XLTCUSDT']),
# all markets, quote=USDT
([], ['USDT'], False, False,
['ETH/USDT', 'LTC/USDT', 'XLTCUSDT']),
# all markets, quote=USDT, USD
([], ['USDT', 'USD'], False, False,
['ETH/USDT', 'LTC/USD', 'LTC/USDT', 'XLTCUSDT']),
# all markets, base=LTC, quote=USDT
(['LTC'], ['USDT'], False, False,
['LTC/USDT', 'XLTCUSDT']),
# all pairs, base=LTC, quote=USDT
(['LTC'], ['USDT'], True, False,
['LTC/USDT']),
# all markets, base=LTC, quote=USDT, NONEXISTENT
(['LTC'], ['USDT', 'NONEXISTENT'], False, False,
['LTC/USDT', 'XLTCUSDT']),
# all markets, base=LTC, quote=NONEXISTENT
(['LTC'], ['NONEXISTENT'], False, False,
[]),
])
def test_get_markets(default_conf, mocker, markets,
base_currencies, quote_currencies, pairs_only, active_only,
expected_keys):
mocker.patch.multiple('freqtrade.exchange.Exchange',
_init_ccxt=MagicMock(return_value=MagicMock()),
_load_async_markets=MagicMock(),
validate_pairs=MagicMock(),
validate_timeframes=MagicMock(),
markets=PropertyMock(return_value=markets))
ex = Exchange(default_conf)
pairs = ex.get_markets(base_currencies, quote_currencies, pairs_only, active_only)
assert sorted(pairs.keys()) == sorted(expected_keys)
def test_timeframe_to_minutes():
assert timeframe_to_minutes("5m") == 5
assert timeframe_to_minutes("10m") == 10
@ -1762,3 +1832,33 @@ def test_timeframe_to_next_date():
date = datetime.now(tz=timezone.utc)
assert timeframe_to_next_date("5m") > date
@pytest.mark.parametrize("market_symbol,base_currency,quote_currency,expected_result", [
("BTC/USDT", None, None, True),
("USDT/BTC", None, None, True),
("BTCUSDT", None, None, False),
("BTC/USDT", None, "USDT", True),
("USDT/BTC", None, "USDT", False),
("BTCUSDT", None, "USDT", False),
("BTC/USDT", "BTC", None, True),
("USDT/BTC", "BTC", None, False),
("BTCUSDT", "BTC", None, False),
("BTC/USDT", "BTC", "USDT", True),
("BTC/USDT", "USDT", "BTC", False),
("BTC/USDT", "BTC", "USD", False),
("BTCUSDT", "BTC", "USDT", False),
("BTC/", None, None, False),
("/USDT", None, None, False),
])
def test_symbol_is_pair(market_symbol, base_currency, quote_currency, expected_result) -> None:
assert symbol_is_pair(market_symbol, base_currency, quote_currency) == expected_result
@pytest.mark.parametrize("market,expected_result", [
({'symbol': 'ETH/BTC', 'active': True}, True),
({'symbol': 'ETH/BTC', 'active': False}, False),
({'symbol': 'ETH/BTC', }, True),
])
def test_market_is_active(market, expected_result) -> None:
assert market_is_active(market) == expected_result

34
tests/test_misc.py
View File

@ -7,7 +7,7 @@ from unittest.mock import MagicMock
from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.data.history import pair_data_filename
from freqtrade.misc import (datesarray_to_datetimearray, file_dump_json,
file_load_json, format_ms_time, shorten_date)
file_load_json, format_ms_time, plural, shorten_date)
def test_shorten_date() -> None:
@ -69,3 +69,35 @@ def test_format_ms_time() -> None:
# Date 2017-12-13 08:02:01
date_in_epoch_ms = 1513152121000
assert format_ms_time(date_in_epoch_ms) == res.astimezone(None).strftime('%Y-%m-%dT%H:%M:%S')
def test_plural() -> None:
assert plural(0, "page") == "pages"
assert plural(0.0, "page") == "pages"
assert plural(1, "page") == "page"
assert plural(1.0, "page") == "page"
assert plural(2, "page") == "pages"
assert plural(2.0, "page") == "pages"
assert plural(-1, "page") == "page"
assert plural(-1.0, "page") == "page"
assert plural(-2, "page") == "pages"
assert plural(-2.0, "page") == "pages"
assert plural(0.5, "page") == "pages"
assert plural(1.5, "page") == "pages"
assert plural(-0.5, "page") == "pages"
assert plural(-1.5, "page") == "pages"
assert plural(0, "ox", "oxen") == "oxen"
assert plural(0.0, "ox", "oxen") == "oxen"
assert plural(1, "ox", "oxen") == "ox"
assert plural(1.0, "ox", "oxen") == "ox"
assert plural(2, "ox", "oxen") == "oxen"
assert plural(2.0, "ox", "oxen") == "oxen"
assert plural(-1, "ox", "oxen") == "ox"
assert plural(-1.0, "ox", "oxen") == "ox"
assert plural(-2, "ox", "oxen") == "oxen"
assert plural(-2.0, "ox", "oxen") == "oxen"
assert plural(0.5, "ox", "oxen") == "oxen"
assert plural(1.5, "ox", "oxen") == "oxen"
assert plural(-0.5, "ox", "oxen") == "oxen"
assert plural(-1.5, "ox", "oxen") == "oxen"

243
tests/test_utils.py
View File

@ -8,7 +8,7 @@ from freqtrade import OperationalException
from freqtrade.state import RunMode
from freqtrade.utils import (setup_utils_configuration, start_create_userdir,
start_download_data, start_list_exchanges,
start_list_timeframes)
start_list_markets, start_list_timeframes)
from tests.conftest import get_args, log_has, patch_exchange
@ -164,6 +164,247 @@ def test_list_timeframes(mocker, capsys):
assert re.search(r"^1d$", captured.out, re.MULTILINE)
def test_list_markets(mocker, markets, capsys):
api_mock = MagicMock()
api_mock.markets = markets
patch_exchange(mocker, api_mock=api_mock)
# Test with no --config
args = [
"list-markets",
]
pargs = get_args(args)
pargs['config'] = None
with pytest.raises(OperationalException,
match=r"This command requires a configured exchange.*"):
start_list_markets(pargs, False)
# Test with --config config.json.example
args = [
'--config', 'config.json.example',
"list-markets",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 8 active markets: "
"BLK/BTC, BTT/BTC, ETH/BTC, ETH/USDT, LTC/USD, LTC/USDT, TKN/BTC, XLTCUSDT.\n"
in captured.out)
patch_exchange(mocker, api_mock=api_mock, id="binance")
# Test with --exchange
args = [
"list-markets",
"--exchange", "binance"
]
pargs = get_args(args)
pargs['config'] = None
start_list_markets(pargs, False)
captured = capsys.readouterr()
assert re.match("\nExchange Binance has 8 active markets:\n",
captured.out)
patch_exchange(mocker, api_mock=api_mock, id="bittrex")
# Test with --all: all markets
args = [
'--config', 'config.json.example',
"list-markets", "--all",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 11 markets: "
"BLK/BTC, BTT/BTC, ETH/BTC, ETH/USDT, LTC/BTC, LTC/USD, LTC/USDT, NEO/BTC, "
"TKN/BTC, XLTCUSDT, XRP/BTC.\n"
in captured.out)
# Test list-pairs subcommand: active pairs
args = [
'--config', 'config.json.example',
"list-pairs",
"--print-list",
]
start_list_markets(get_args(args), True)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 7 active pairs: "
"BLK/BTC, BTT/BTC, ETH/BTC, ETH/USDT, LTC/USD, LTC/USDT, TKN/BTC.\n"
in captured.out)
# Test list-pairs subcommand with --all: all pairs
args = [
'--config', 'config.json.example',
"list-pairs", "--all",
"--print-list",
]
start_list_markets(get_args(args), True)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 10 pairs: "
"BLK/BTC, BTT/BTC, ETH/BTC, ETH/USDT, LTC/BTC, LTC/USD, LTC/USDT, NEO/BTC, "
"TKN/BTC, XRP/BTC.\n"
in captured.out)
# active markets, base=ETH, LTC
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "ETH", "LTC",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 5 active markets with ETH, LTC as base currencies: "
"ETH/BTC, ETH/USDT, LTC/USD, LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active markets, base=LTC
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "LTC",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 3 active markets with LTC as base currency: "
"LTC/USD, LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active markets, quote=USDT, USD
args = [
'--config', 'config.json.example',
"list-markets",
"--quote", "USDT", "USD",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 4 active markets with USDT, USD as quote currencies: "
"ETH/USDT, LTC/USD, LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active markets, quote=USDT
args = [
'--config', 'config.json.example',
"list-markets",
"--quote", "USDT",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 3 active markets with USDT as quote currency: "
"ETH/USDT, LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active markets, base=LTC, quote=USDT
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "LTC", "--quote", "USDT",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 2 active markets with LTC as base currency and "
"with USDT as quote currency: LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active pairs, base=LTC, quote=USDT
args = [
'--config', 'config.json.example',
"list-pairs",
"--base", "LTC", "--quote", "USDT",
"--print-list",
]
start_list_markets(get_args(args), True)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 1 active pair with LTC as base currency and "
"with USDT as quote currency: LTC/USDT.\n"
in captured.out)
# active markets, base=LTC, quote=USDT, NONEXISTENT
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "LTC", "--quote", "USDT", "NONEXISTENT",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 2 active markets with LTC as base currency and "
"with USDT, NONEXISTENT as quote currencies: LTC/USDT, XLTCUSDT.\n"
in captured.out)
# active markets, base=LTC, quote=NONEXISTENT
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "LTC", "--quote", "NONEXISTENT",
"--print-list",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 0 active markets with LTC as base currency and "
"with NONEXISTENT as quote currency.\n"
in captured.out)
# Test tabular output
args = [
'--config', 'config.json.example',
"list-markets",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 8 active markets:\n"
in captured.out)
# Test tabular output, no markets found
args = [
'--config', 'config.json.example',
"list-markets",
"--base", "LTC", "--quote", "NONEXISTENT",
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Exchange Bittrex has 0 active markets with LTC as base currency and "
"with NONEXISTENT as quote currency.\n"
in captured.out)
# Test --print-json
args = [
'--config', 'config.json.example',
"list-markets",
"--print-json"
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ('["BLK/BTC","BTT/BTC","ETH/BTC","ETH/USDT","LTC/USD","LTC/USDT","TKN/BTC","XLTCUSDT"]'
in captured.out)
# Test --print-csv
args = [
'--config', 'config.json.example',
"list-markets",
"--print-csv"
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert ("Id,Symbol,Base,Quote,Active,Is pair" in captured.out)
assert ("blkbtc,BLK/BTC,BLK,BTC,True,True" in captured.out)
assert ("BTTBTC,BTT/BTC,BTT,BTC,True,True" in captured.out)
# Test --one-column
args = [
'--config', 'config.json.example',
"list-markets",
"--one-column"
]
start_list_markets(get_args(args), False)
captured = capsys.readouterr()
assert re.search(r"^BLK/BTC$", captured.out, re.MULTILINE)
assert re.search(r"^BTT/BTC$", captured.out, re.MULTILINE)
def test_create_datadir_failed(caplog):
args = [