async branch updated to reflect develop branch changes

This commit is contained in:
misagh 2018-08-02 16:48:21 +02:00
commit 2ec2f1abce
25 changed files with 543 additions and 113 deletions

View File

@ -24,7 +24,7 @@ hesitate to read the source code and understand the mechanism of this bot.
## Exchange marketplaces supported
- [X] [Bittrex](https://bittrex.com/)
- [X] [Binance](https://www.binance.com/)
- [X] [Binance](https://www.binance.com/) ([*Note for binance users](#a-note-on-binance))
- [ ] [113 others to tests](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_
## Features
@ -152,6 +152,13 @@ The project is currently setup in two main branches:
- `develop` - This branch has often new features, but might also cause breaking changes.
- `master` - This branch contains the latest stable release. The bot 'should' be stable on this branch, and is generally well tested.
- `feat/*` - These are feature branches, which are beeing worked on heavily. Please don't use these unless you want to test a specific feature.
## A note on Binance
For Binance, please add `"BNB/<STAKE>"` to your blacklist to avoid issues.
Accounts having BNB accounts use this to pay for fees - if your first trade happens to be on `BNB`, further trades will consume this position and make the initial BNB order unsellable as the expected amount is not there anymore.
## Support

View File

@ -17,6 +17,7 @@
"name": "bittrex",
"key": "your_exchange_key",
"secret": "your_exchange_secret",
"ccxt_rate_limit": true,
"pair_whitelist": [
"ETH/BTC",
"LTC/BTC",

View File

@ -26,6 +26,7 @@
"name": "bittrex",
"key": "your_exchange_key",
"secret": "your_exchange_secret",
"ccxt_rate_limit": true,
"pair_whitelist": [
"ETH/BTC",
"LTC/BTC",

View File

@ -151,7 +151,7 @@ cp freqtrade/tests/testdata/pairs.json user_data/data/binance
Then run:
```bash
python scripts/download_backtest_data --exchange binance
python scripts/download_backtest_data.py --exchange binance
```
This will download ticker data for all the currency pairs you defined in `pairs.json`.
@ -238,6 +238,31 @@ On the other hand, if you set a too high `minimal_roi` like `"0": 0.55`
profit. Hence, keep in mind that your performance is a mix of your
strategies, your configuration, and the crypto-currency you have set up.
## Backtesting multiple strategies
To backtest multiple strategies, a list of Strategies can be provided.
This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this should give a nice runtime boost.
All listed Strategies need to be in the same folder.
``` bash
freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strategy-list Strategy001 Strategy002 --export trades
```
This will save the results to `user_data/backtest_data/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up.
```
=================================================== Strategy Summary ====================================================
| Strategy | buy count | avg profit % | cum profit % | total profit ETH | avg duration | profit | loss |
|:-----------|------------:|---------------:|---------------:|-------------------:|:----------------|---------:|-------:|
| Strategy1 | 19 | -0.76 | -14.39 | -0.01440287 | 15:48:00 | 15 | 4 |
| Strategy2 | 6 | -2.73 | -16.40 | -0.01641299 | 1 day, 14:12:00 | 3 | 3 |
```
## Next step
Great, your strategy is profitable. What if the bot can give your the

View File

@ -1,13 +1,15 @@
# Bot usage
This page explains the difference parameters of the bot and how to run
it.
This page explains the difference parameters of the bot and how to run it.
## Table of Contents
- [Bot commands](#bot-commands)
- [Backtesting commands](#backtesting-commands)
- [Hyperopt commands](#hyperopt-commands)
## Bot commands
```
usage: freqtrade [-h] [-v] [--version] [-c PATH] [-d PATH] [-s NAME]
[--strategy-path PATH] [--dynamic-whitelist [INT]]
@ -41,6 +43,7 @@ optional arguments:
```
### How to use a different config file?
The bot allows you to select which config file you want to use. Per
default, the bot will load the file `./config.json`
@ -49,6 +52,7 @@ python3 ./freqtrade/main.py -c path/far/far/away/config.json
```
### How to use --strategy?
This parameter will allow you to load your custom strategy class.
Per default without `--strategy` or `-s` the bot will load the
`DefaultStrategy` included with the bot (`freqtrade/strategy/default_strategy.py`).
@ -60,6 +64,7 @@ To load a strategy, simply pass the class name (e.g.: `CustomStrategy`) in this
**Example:**
In `user_data/strategies` you have a file `my_awesome_strategy.py` which has
a strategy class called `AwesomeStrategy` to load it:
```bash
python3 ./freqtrade/main.py --strategy AwesomeStrategy
```
@ -70,6 +75,7 @@ message the reason (File not found, or errors in your code).
Learn more about strategy file in [optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).
### How to use --strategy-path?
This parameter allows you to add an additional strategy lookup path, which gets
checked before the default locations (The passed path must be a folder!):
```bash
@ -77,21 +83,25 @@ python3 ./freqtrade/main.py --strategy AwesomeStrategy --strategy-path /some/fol
```
#### How to install a strategy?
This is very simple. Copy paste your strategy file into the folder
`user_data/strategies` or use `--strategy-path`. And voila, the bot is ready to use it.
### How to use --dynamic-whitelist?
Per default `--dynamic-whitelist` will retrieve the 20 currencies based
on BaseVolume. This value can be changed when you run the script.
**By Default**
Get the 20 currencies based on BaseVolume.
```bash
python3 ./freqtrade/main.py --dynamic-whitelist
```
**Customize the number of currencies to retrieve**
Get the 30 currencies based on BaseVolume.
```bash
python3 ./freqtrade/main.py --dynamic-whitelist 30
```
@ -102,6 +112,7 @@ negative value (e.g -2), `--dynamic-whitelist` will use the default
value (20).
### How to use --db-url?
When you run the bot in Dry-run mode, per default no transactions are
stored in a database. If you want to store your bot actions in a DB
using `--db-url`. This can also be used to specify a custom database
@ -111,14 +122,14 @@ in production mode. Example command:
python3 ./freqtrade/main.py -c config.json --db-url sqlite:///tradesv3.dry_run.sqlite
```
## Backtesting commands
Backtesting also uses the config specified via `-c/--config`.
```
usage: main.py backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp]
usage: freqtrade backtesting [-h] [-i TICKER_INTERVAL] [--eps] [--dmmp]
[--timerange TIMERANGE] [-l] [-r]
[--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]]
[--export EXPORT] [--export-filename PATH]
optional arguments:
@ -139,6 +150,13 @@ optional arguments:
refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your backtesting with up-to-date data.
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
Provide a commaseparated list of strategies to
backtest Please note that ticker-interval needs to be
set either in config or via command line. When using
this together with --export trades, the strategy-name
is injected into the filename (so backtest-data.json
becomes backtest-data-DefaultStrategy.json
--export EXPORT export backtest results, argument are: trades Example
--export=trades
--export-filename PATH
@ -151,6 +169,7 @@ optional arguments:
```
### How to use --refresh-pairs-cached parameter?
The first time your run Backtesting, it will take the pairs you have
set in your config file and download data from Bittrex.
@ -162,7 +181,6 @@ to come back to the previous version.**
To test your strategy with latest data, we recommend continuing using
the parameter `-l` or `--live`.
## Hyperopt commands
To optimize your strategy, you can use hyperopt parameter hyperoptimization
@ -194,10 +212,11 @@ optional arguments:
```
## A parameter missing in the configuration?
All parameters for `main.py`, `backtesting`, `hyperopt` are referenced
in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L84)
## Next step
The optimal strategy of the bot will change with time depending of the
market trends. The next step is to
The optimal strategy of the bot will change with time depending of the market trends. The next step is to
[optimize your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md).

View File

@ -197,6 +197,33 @@ you run it in production mode.
```
If you have not your Bittrex API key yet, [see our tutorial](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md).
### Embedding Strategies
FreqTrade provides you with with an easy way to embed the strategy into your configuration file.
This is done by utilizing BASE64 encoding and providing this string at the strategy configuration field,
in your chosen config file.
##### Encoding a string as BASE64
This is a quick example, how to generate the BASE64 string in python
```python
from base64 import urlsafe_b64encode
with open(file, 'r') as f:
content = f.read()
content = urlsafe_b64encode(content.encode('utf-8'))
```
The variable 'content', will contain the strategy file in a BASE64 encoded form. Which can now be set in your configurations file as following
```json
"strategy": "NameOfStrategy:BASE64String"
```
Please ensure that 'NameOfStrategy' is identical to the strategy name!
## Next step
Now you have configured your config.json, the next step is to [start your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md).

View File

@ -142,6 +142,16 @@ class Arguments(object):
action='store_true',
dest='refresh_pairs',
)
parser.add_argument(
'--strategy-list',
help='Provide a commaseparated list of strategies to backtest '
'Please note that ticker-interval needs to be set either in config '
'or via command line. When using this together with --export trades, '
'the strategy-name is injected into the filename '
'(so backtest-data.json becomes backtest-data-DefaultStrategy.json',
nargs='+',
dest='strategy_list',
)
parser.add_argument(
'--export',
help='export backtest results, argument are: trades\

View File

@ -187,6 +187,14 @@ class Configuration(object):
config.update({'refresh_pairs': True})
logger.info('Parameter -r/--refresh-pairs-cached detected ...')
if 'strategy_list' in self.args and self.args.strategy_list:
config.update({'strategy_list': self.args.strategy_list})
logger.info('Using strategy list of %s Strategies', len(self.args.strategy_list))
if 'ticker_interval' in self.args and self.args.ticker_interval:
config.update({'ticker_interval': self.args.ticker_interval})
logger.info('Overriding ticker interval with Command line argument')
# If --export is used we add it to the configuration
if 'export' in self.args and self.args.export:
config.update({'export': self.args.export})

View File

@ -95,8 +95,7 @@ class Exchange(object):
'secret': exchange_config.get('secret'),
'password': exchange_config.get('password'),
'uid': exchange_config.get('uid', ''),
# 'enableRateLimit': True,
'enableRateLimit': False,
'enableRateLimit': exchange_config.get('ccxt_rate_limit', True)
})
except (KeyError, AttributeError):
raise OperationalException(f'Exchange {name} is not supported')
@ -334,17 +333,17 @@ class Exchange(object):
logger.info("returning cached ticker-data for %s", pair)
return self._cached_ticker[pair]
async def async_get_tickers_history(self, pairs, tick_interval) -> List[Tuple[str, List]]:
async def async_get_candles_history(self, pairs, tick_interval) -> List[Tuple[str, List]]:
# COMMENTED CODE IS FOR DISCUSSION: where should we close the loop on async ?
# loop = asyncio.new_event_loop()
# asyncio.set_event_loop(loop)
input_coroutines = [self.async_get_ticker_history(
input_coroutines = [self.async_get_candle_history(
symbol, tick_interval) for symbol in pairs]
tickers = await asyncio.gather(*input_coroutines, return_exceptions=True)
# await self._api_async.close()
return tickers
async def async_get_ticker_history(self, pair: str, tick_interval: str,
async def async_get_candle_history(self, pair: str, tick_interval: str,
since_ms: Optional[int] = None) -> Tuple[str, List]:
try:
# fetch ohlcv asynchronously
@ -369,14 +368,14 @@ class Exchange(object):
"""
# TODO: maybe add since_ms to use async in the download-script?
# TODO: only refresh once per interval ? *may require this to move to freqtradebot.py
# TODO@ Add tests for this and the async stuff above
# TODO: Add tests for this and the async stuff above
logger.debug("Refreshing klines for %d pairs", len(pair_list))
datatups = asyncio.get_event_loop().run_until_complete(
self.async_get_tickers_history(pair_list, ticker_interval))
self.async_get_candles_history(pair_list, ticker_interval))
return {pair: data for (pair, data) in datatups}
@retrier
def get_ticker_history(self, pair: str, tick_interval: str,
def get_candle_history(self, pair: str, tick_interval: str,
since_ms: Optional[int] = None) -> List[Dict]:
try:
# last item should be in the time interval [now - tick_interval, now]

View File

@ -10,7 +10,7 @@ logger = logging.getLogger(__name__)
def parse_ticker_dataframe(ticker: list) -> DataFrame:
"""
Analyses the trend for the given ticker history
:param ticker: See exchange.get_ticker_history
:param ticker: See exchange.get_candle_history
:return: DataFrame
"""
cols = ['date', 'open', 'high', 'low', 'close', 'volume']

View File

@ -510,7 +510,6 @@ class FreqtradeBot(object):
(buy, sell) = (False, False)
experimental = self.config.get('experimental', {})
if experimental.get('use_sell_signal') or experimental.get('ignore_roi_if_buy_signal'):
# ticker = self.exchange.get_ticker_history(trade.pair, self.strategy.ticker_interval)
ticker = self._klines.get(trade.pair)
(buy, sell) = self.strategy.get_signal(trade.pair, self.strategy.ticker_interval,
ticker)

View File

@ -219,7 +219,7 @@ def download_backtesting_testdata(datadir: str,
logger.debug("Current Start: %s", misc.format_ms_time(data[1][0]) if data else 'None')
logger.debug("Current End: %s", misc.format_ms_time(data[-1][0]) if data else 'None')
new_data = exchange.get_ticker_history(pair=pair, tick_interval=tick_interval,
new_data = exchange.get_candle_history(pair=pair, tick_interval=tick_interval,
since_ms=since_ms)
data.extend(new_data)

View File

@ -6,7 +6,9 @@ This module contains the backtesting logic
import logging
import operator
from argparse import Namespace
from copy import deepcopy
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
import arrow
@ -52,13 +54,9 @@ class Backtesting(object):
backtesting = Backtesting(config)
backtesting.start()
"""
def __init__(self, config: Dict[str, Any]) -> None:
self.config = config
self.strategy: IStrategy = StrategyResolver(self.config).strategy
self.ticker_interval = self.strategy.ticker_interval
self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
self.advise_buy = self.strategy.advise_buy
self.advise_sell = self.strategy.advise_sell
# Reset keys for backtesting
self.config['exchange']['key'] = ''
@ -66,9 +64,36 @@ class Backtesting(object):
self.config['exchange']['password'] = ''
self.config['exchange']['uid'] = ''
self.config['dry_run'] = True
self.strategylist: List[IStrategy] = []
if self.config.get('strategy_list', None):
# Force one interval
self.ticker_interval = str(self.config.get('ticker_interval'))
for strat in list(self.config['strategy_list']):
stratconf = deepcopy(self.config)
stratconf['strategy'] = strat
self.strategylist.append(StrategyResolver(stratconf).strategy)
else:
# only one strategy
strat = StrategyResolver(self.config).strategy
self.strategylist.append(StrategyResolver(self.config).strategy)
# Load one strategy
self._set_strategy(self.strategylist[0])
self.exchange = Exchange(self.config)
self.fee = self.exchange.get_fee()
def _set_strategy(self, strategy):
"""
Load strategy into backtesting
"""
self.strategy = strategy
self.ticker_interval = self.config.get('ticker_interval')
self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe
self.advise_buy = strategy.advise_buy
self.advise_sell = strategy.advise_sell
@staticmethod
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
"""
@ -132,7 +157,32 @@ class Backtesting(object):
tabular_data.append([reason.value, count])
return tabulate(tabular_data, headers=headers, tablefmt="pipe")
def _store_backtest_result(self, recordfilename: Optional[str], results: DataFrame) -> None:
def _generate_text_table_strategy(self, all_results: dict) -> str:
"""
Generate summary table per strategy
"""
stake_currency = str(self.config.get('stake_currency'))
floatfmt = ('s', 'd', '.2f', '.2f', '.8f', 'd', '.1f', '.1f')
tabular_data = []
headers = ['Strategy', 'buy count', 'avg profit %', 'cum profit %',
'total profit ' + stake_currency, 'avg duration', 'profit', 'loss']
for strategy, results in all_results.items():
tabular_data.append([
strategy,
len(results.index),
results.profit_percent.mean() * 100.0,
results.profit_percent.sum() * 100.0,
results.profit_abs.sum(),
str(timedelta(
minutes=round(results.trade_duration.mean()))) if not results.empty else '0:00',
len(results[results.profit_abs > 0]),
len(results[results.profit_abs < 0])
])
return tabulate(tabular_data, headers=headers, floatfmt=floatfmt, tablefmt="pipe")
def _store_backtest_result(self, recordfilename: str, results: DataFrame,
strategyname: Optional[str] = None) -> None:
records = [(t.pair, t.profit_percent, t.open_time.timestamp(),
t.close_time.timestamp(), t.open_index - 1, t.trade_duration,
@ -140,6 +190,11 @@ class Backtesting(object):
for index, t in results.iterrows()]
if records:
if strategyname:
# Inject strategyname to filename
recname = Path(recordfilename)
recordfilename = str(Path.joinpath(
recname.parent, f'{recname.stem}-{strategyname}').with_suffix(recname.suffix))
logger.info('Dumping backtest results to %s', recordfilename)
file_dump_json(recordfilename, records)
@ -283,7 +338,7 @@ class Backtesting(object):
if self.config.get('live'):
logger.info('Downloading data for all pairs in whitelist ...')
for pair in pairs:
data[pair] = self.exchange.get_ticker_history(pair, self.ticker_interval)
data[pair] = self.exchange.get_candle_history(pair, self.ticker_interval)
else:
logger.info('Using local backtesting data (using whitelist in given config) ...')
@ -307,62 +362,55 @@ class Backtesting(object):
else:
logger.info('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
max_open_trades = 0
all_results = {}
preprocessed = self.tickerdata_to_dataframe(data)
for strat in self.strategylist:
logger.info("Running backtesting for Strategy %s", strat.get_strategy_name())
self._set_strategy(strat)
# Print timeframe
min_date, max_date = self.get_timeframe(preprocessed)
logger.info(
'Measuring data from %s up to %s (%s days)..',
min_date.isoformat(),
max_date.isoformat(),
(max_date - min_date).days
)
# need to reprocess data every time to populate signals
preprocessed = self.tickerdata_to_dataframe(data)
# Execute backtest and print results
results = self.backtest(
{
'stake_amount': self.config.get('stake_amount'),
'processed': preprocessed,
'max_open_trades': max_open_trades,
'position_stacking': self.config.get('position_stacking', False),
}
)
if self.config.get('export', False):
self._store_backtest_result(self.config.get('exportfilename'), results)
logger.info(
'\n' + '=' * 49 +
' BACKTESTING REPORT ' +
'=' * 50 + '\n'
'%s',
self._generate_text_table(
data,
results
# Print timeframe
min_date, max_date = self.get_timeframe(preprocessed)
logger.info(
'Measuring data from %s up to %s (%s days)..',
min_date.isoformat(),
max_date.isoformat(),
(max_date - min_date).days
)
)
# logger.info(
# results[['sell_reason']].groupby('sell_reason').count()
# )
logger.info(
'\n' +
' SELL READON STATS '.center(119, '=') +
'\n%s \n',
self._generate_text_table_sell_reason(data, results)
)
logger.info(
'\n' +
' LEFT OPEN TRADES REPORT '.center(119, '=') +
'\n%s',
self._generate_text_table(
data,
results.loc[results.open_at_end]
# Execute backtest and print results
all_results[self.strategy.get_strategy_name()] = self.backtest(
{
'stake_amount': self.config.get('stake_amount'),
'processed': preprocessed,
'max_open_trades': max_open_trades,
'position_stacking': self.config.get('position_stacking', False),
}
)
)
for strategy, results in all_results.items():
if self.config.get('export', False):
self._store_backtest_result(self.config['exportfilename'], results,
strategy if len(self.strategylist) > 1 else None)
print(f"Result for strategy {strategy}")
print(' BACKTESTING REPORT '.center(119, '='))
print(self._generate_text_table(data, results))
print(' SELL REASON STATS '.center(119, '='))
print(self._generate_text_table_sell_reason(data, results))
print(' LEFT OPEN TRADES REPORT '.center(119, '='))
print(self._generate_text_table(data, results.loc[results.open_at_end]))
print()
if len(all_results) > 1:
# Print Strategy summary table
print(' Strategy Summary '.center(119, '='))
print(self._generate_text_table_strategy(all_results))
print('\nFor more details, please look at the detail tables above')
def setup_configuration(args: Namespace) -> Dict[str, Any]:

View File

@ -7,7 +7,10 @@ import importlib.util
import inspect
import logging
import os
import tempfile
from base64 import urlsafe_b64decode
from collections import OrderedDict
from pathlib import Path
from typing import Dict, Optional, Type
from freqtrade import constants
@ -87,6 +90,22 @@ class StrategyResolver(object):
# Add extra strategy directory on top of search paths
abs_paths.insert(0, extra_dir)
if ":" in strategy_name:
logger.info("loading base64 endocded strategy")
strat = strategy_name.split(":")
if len(strat) == 2:
temp = Path(tempfile.mkdtemp("freq", "strategy"))
name = strat[0] + ".py"
temp.joinpath(name).write_text(urlsafe_b64decode(strat[1]).decode('utf-8'))
temp.joinpath("__init__.py").touch()
strategy_name = os.path.splitext(name)[0]
# register temp path with the bot
abs_paths.insert(0, str(temp.resolve()))
for path in abs_paths:
try:
strategy = self._search_strategy(path, strategy_name=strategy_name, config=config)

View File

@ -624,7 +624,7 @@ def make_fetch_ohlcv_mock(data):
return fetch_ohlcv_mock
def test_get_ticker_history(default_conf, mocker):
def test_get_candle_history(default_conf, mocker):
api_mock = MagicMock()
tick = [
[
@ -641,7 +641,7 @@ def test_get_ticker_history(default_conf, mocker):
exchange = get_patched_exchange(mocker, default_conf, api_mock)
# retrieve original ticker
ticks = exchange.get_ticker_history('ETH/BTC', default_conf['ticker_interval'])
ticks = exchange.get_candle_history('ETH/BTC', default_conf['ticker_interval'])
assert ticks[0][0] == 1511686200000
assert ticks[0][1] == 1
assert ticks[0][2] == 2
@ -663,7 +663,7 @@ def test_get_ticker_history(default_conf, mocker):
api_mock.fetch_ohlcv = MagicMock(side_effect=make_fetch_ohlcv_mock(new_tick))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
ticks = exchange.get_ticker_history('ETH/BTC', default_conf['ticker_interval'])
ticks = exchange.get_candle_history('ETH/BTC', default_conf['ticker_interval'])
assert ticks[0][0] == 1511686210000
assert ticks[0][1] == 6
assert ticks[0][2] == 7
@ -672,16 +672,16 @@ def test_get_ticker_history(default_conf, mocker):
assert ticks[0][5] == 10
ccxt_exceptionhandlers(mocker, default_conf, api_mock,
"get_ticker_history", "fetch_ohlcv",
"get_candle_history", "fetch_ohlcv",
pair='ABCD/BTC', tick_interval=default_conf['ticker_interval'])
with pytest.raises(OperationalException, match=r'Exchange .* does not support.*'):
api_mock.fetch_ohlcv = MagicMock(side_effect=ccxt.NotSupported)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.get_ticker_history(pair='ABCD/BTC', tick_interval=default_conf['ticker_interval'])
exchange.get_candle_history(pair='ABCD/BTC', tick_interval=default_conf['ticker_interval'])
def test_get_ticker_history_sort(default_conf, mocker):
def test_get_candle_history_sort(default_conf, mocker):
api_mock = MagicMock()
# GDAX use-case (real data from GDAX)
@ -704,7 +704,7 @@ def test_get_ticker_history_sort(default_conf, mocker):
exchange = get_patched_exchange(mocker, default_conf, api_mock)
# Test the ticker history sort
ticks = exchange.get_ticker_history('ETH/BTC', default_conf['ticker_interval'])
ticks = exchange.get_candle_history('ETH/BTC', default_conf['ticker_interval'])
assert ticks[0][0] == 1527830400000
assert ticks[0][1] == 0.07649
assert ticks[0][2] == 0.07651
@ -737,7 +737,7 @@ def test_get_ticker_history_sort(default_conf, mocker):
api_mock.fetch_ohlcv = MagicMock(side_effect=make_fetch_ohlcv_mock(tick))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
# Test the ticker history sort
ticks = exchange.get_ticker_history('ETH/BTC', default_conf['ticker_interval'])
ticks = exchange.get_candle_history('ETH/BTC', default_conf['ticker_interval'])
assert ticks[0][0] == 1527827700000
assert ticks[0][1] == 0.07659999
assert ticks[0][2] == 0.0766

View File

@ -110,7 +110,7 @@ def mocked_load_data(datadir, pairs=[], ticker_interval='0m', refresh_pairs=Fals
return pairdata
# use for mock freqtrade.exchange.get_ticker_history'
# use for mock freqtrade.exchange.get_candle_history'
def _load_pair_as_ticks(pair, tickfreq):
ticks = optimize.load_data(None, ticker_interval=tickfreq, pairs=[pair])
ticks = trim_dictlist(ticks, -201)
@ -406,12 +406,56 @@ def test_generate_text_table_sell_reason(default_conf, mocker):
data={'ETH/BTC': {}}, results=results) == result_str
def test_generate_text_table_strategyn(default_conf, mocker):
"""
Test Backtesting.generate_text_table_sell_reason() method
"""
patch_exchange(mocker)
backtesting = Backtesting(default_conf)
results = {}
results['ETH/BTC'] = pd.DataFrame(
{
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
'profit_percent': [0.1, 0.2, 0.3],
'profit_abs': [0.2, 0.4, 0.5],
'trade_duration': [10, 30, 10],
'profit': [2, 0, 0],
'loss': [0, 0, 1],
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
}
)
results['LTC/BTC'] = pd.DataFrame(
{
'pair': ['LTC/BTC', 'LTC/BTC', 'LTC/BTC'],
'profit_percent': [0.4, 0.2, 0.3],
'profit_abs': [0.4, 0.4, 0.5],
'trade_duration': [15, 30, 15],
'profit': [4, 1, 0],
'loss': [0, 0, 1],
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
}
)
result_str = (
'| Strategy | buy count | avg profit % | cum profit % '
'| total profit BTC | avg duration | profit | loss |\n'
'|:-----------|------------:|---------------:|---------------:'
'|-------------------:|:---------------|---------:|-------:|\n'
'| ETH/BTC | 3 | 20.00 | 60.00 '
'| 1.10000000 | 0:17:00 | 3 | 0 |\n'
'| LTC/BTC | 3 | 30.00 | 90.00 '
'| 1.30000000 | 0:20:00 | 3 | 0 |'
)
print(backtesting._generate_text_table_strategy(all_results=results))
assert backtesting._generate_text_table_strategy(all_results=results) == result_str
def test_backtesting_start(default_conf, mocker, caplog) -> None:
def get_timeframe(input1, input2):
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
mocker.patch('freqtrade.optimize.load_data', mocked_load_data)
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history')
mocker.patch('freqtrade.exchange.Exchange.get_candle_history')
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.optimize.backtesting.Backtesting',
@ -446,7 +490,7 @@ def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
mocker.patch('freqtrade.optimize.load_data', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history')
mocker.patch('freqtrade.exchange.Exchange.get_candle_history')
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.optimize.backtesting.Backtesting',
@ -654,6 +698,18 @@ def test_backtest_record(default_conf, fee, mocker):
records = records[0]
# Ensure records are of correct type
assert len(records) == 4
# reset test to test with strategy name
names = []
records = []
backtesting._store_backtest_result("backtest-result.json", results, "DefStrat")
assert len(results) == 4
# Assert file_dump_json was only called once
assert names == ['backtest-result-DefStrat.json']
records = records[0]
# Ensure records are of correct type
assert len(records) == 4
# ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117)
# Below follows just a typecheck of the schema/type of trade-records
oix = None
@ -677,7 +733,7 @@ def test_backtest_record(default_conf, fee, mocker):
def test_backtest_start_live(default_conf, mocker, caplog):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history',
mocker.patch('freqtrade.exchange.Exchange.get_candle_history',
new=lambda s, n, i: _load_pair_as_ticks(n, i))
patch_exchange(mocker)
mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest', MagicMock())
@ -686,15 +742,6 @@ def test_backtest_start_live(default_conf, mocker, caplog):
read_data=json.dumps(default_conf)
))
args = MagicMock()
args.ticker_interval = 1
args.level = 10
args.live = True
args.datadir = None
args.export = None
args.strategy = 'DefaultStrategy'
args.timerange = '-100' # needed due to MagicMock malleability
args = [
'--config', 'config.json',
'--strategy', 'DefaultStrategy',
@ -725,3 +772,60 @@ def test_backtest_start_live(default_conf, mocker, caplog):
for line in exists:
assert log_has(line, caplog.record_tuples)
def test_backtest_start_multi_strat(default_conf, mocker, caplog):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
mocker.patch('freqtrade.exchange.Exchange.get_candle_history',
new=lambda s, n, i: _load_pair_as_ticks(n, i))
patch_exchange(mocker)
backtestmock = MagicMock()
mocker.patch('freqtrade.optimize.backtesting.Backtesting.backtest', backtestmock)
gen_table_mock = MagicMock()
mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table', gen_table_mock)
gen_strattable_mock = MagicMock()
mocker.patch('freqtrade.optimize.backtesting.Backtesting._generate_text_table_strategy',
gen_strattable_mock)
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(default_conf)
))
args = [
'--config', 'config.json',
'--datadir', 'freqtrade/tests/testdata',
'backtesting',
'--ticker-interval', '1m',
'--live',
'--timerange', '-100',
'--enable-position-stacking',
'--disable-max-market-positions',
'--strategy-list',
'DefaultStrategy',
'TestStrategy',
]
args = get_args(args)
start(args)
# 2 backtests, 4 tables
assert backtestmock.call_count == 2
assert gen_table_mock.call_count == 4
assert gen_strattable_mock.call_count == 1
# check the logs, that will contain the backtest result
exists = [
'Parameter -i/--ticker-interval detected ...',
'Using ticker_interval: 1m ...',
'Parameter -l/--live detected ...',
'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
'Parameter --timerange detected: -100 ...',
'Using data folder: freqtrade/tests/testdata ...',
'Using stake_currency: BTC ...',
'Using stake_amount: 0.001 ...',
'Downloading data for all pairs in whitelist ...',
'Measuring data from 2017-11-14T19:31:00+00:00 up to 2017-11-14T22:58:00+00:00 (0 days)..',
'Parameter --enable-position-stacking detected ...',
'Running backtesting for Strategy DefaultStrategy',
'Running backtesting for Strategy TestStrategy',
]
for line in exists:
assert log_has(line, caplog.record_tuples)

View File

@ -53,7 +53,7 @@ def _clean_test_file(file: str) -> None:
def test_load_data_30min_ticker(ticker_history, mocker, caplog, default_conf) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-30m.json')
_backup_file(file, copy_file=True)
optimize.load_data(None, pairs=['UNITTEST/BTC'], ticker_interval='30m')
@ -63,7 +63,7 @@ def test_load_data_30min_ticker(ticker_history, mocker, caplog, default_conf) ->
def test_load_data_5min_ticker(ticker_history, mocker, caplog, default_conf) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-5m.json')
_backup_file(file, copy_file=True)
@ -74,7 +74,7 @@ def test_load_data_5min_ticker(ticker_history, mocker, caplog, default_conf) ->
def test_load_data_1min_ticker(ticker_history, mocker, caplog) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-1m.json')
_backup_file(file, copy_file=True)
optimize.load_data(None, ticker_interval='1m', pairs=['UNITTEST/BTC'])
@ -87,7 +87,7 @@ def test_load_data_with_new_pair_1min(ticker_history, mocker, caplog, default_co
"""
Test load_data() with 1 min ticker
"""
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
exchange = get_patched_exchange(mocker, default_conf)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-1m.json')
@ -118,7 +118,7 @@ def test_testdata_path() -> None:
def test_download_pairs(ticker_history, mocker, default_conf) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
exchange = get_patched_exchange(mocker, default_conf)
file1_1 = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-1m.json')
file1_5 = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-5m.json')
@ -261,7 +261,7 @@ def test_load_cached_data_for_updating(mocker) -> None:
def test_download_pairs_exception(ticker_history, mocker, caplog, default_conf) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
mocker.patch('freqtrade.optimize.__init__.download_backtesting_testdata',
side_effect=BaseException('File Error'))
exchange = get_patched_exchange(mocker, default_conf)
@ -279,7 +279,7 @@ def test_download_pairs_exception(ticker_history, mocker, caplog, default_conf)
def test_download_backtesting_testdata(ticker_history, mocker, default_conf) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=ticker_history)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=ticker_history)
exchange = get_patched_exchange(mocker, default_conf)
# Download a 1 min ticker file
@ -304,7 +304,7 @@ def test_download_backtesting_testdata2(mocker, default_conf) -> None:
[1509836580000, 0.00161, 0.00161, 0.00161, 0.00161, 82.390199]
]
json_dump_mock = mocker.patch('freqtrade.misc.file_dump_json', return_value=None)
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=tick)
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=tick)
exchange = get_patched_exchange(mocker, default_conf)
download_backtesting_testdata(None, exchange, pair="UNITTEST/BTC", tick_interval='1m')
download_backtesting_testdata(None, exchange, pair="UNITTEST/BTC", tick_interval='3m')

View File

@ -88,7 +88,7 @@ def test_get_signal_old_dataframe(default_conf, mocker, caplog):
def test_get_signal_handles_exceptions(mocker, default_conf):
mocker.patch('freqtrade.exchange.Exchange.get_ticker_history', return_value=MagicMock())
mocker.patch('freqtrade.exchange.Exchange.get_candle_history', return_value=MagicMock())
exchange = get_patched_exchange(mocker, default_conf)
mocker.patch.object(
_STRATEGY, 'analyze_ticker',

View File

@ -1,5 +1,6 @@
# pragma pylint: disable=missing-docstring, protected-access, C0103
import logging
from base64 import urlsafe_b64encode
from os import path
import warnings
@ -63,6 +64,13 @@ def test_load_strategy(result):
assert 'adx' in resolver.strategy.advise_indicators(result, metadata=metadata)
def test_load_strategy_byte64(result):
with open("freqtrade/tests/strategy/test_strategy.py", "r") as file:
encoded_string = urlsafe_b64encode(file.read().encode("utf-8")).decode("utf-8")
resolver = StrategyResolver({'strategy': 'TestStrategy:{}'.format(encoded_string)})
assert 'adx' in resolver.strategy.advise_indicators(result, 'ETH/BTC')
def test_load_strategy_invalid_directory(result, caplog):
resolver = StrategyResolver()
extra_dir = path.join('some', 'path')

View File

@ -132,7 +132,11 @@ def test_parse_args_backtesting_custom() -> None:
'backtesting',
'--live',
'--ticker-interval', '1m',
'--refresh-pairs-cached']
'--refresh-pairs-cached',
'--strategy-list',
'DefaultStrategy',
'TestStrategy'
]
call_args = Arguments(args, '').get_parsed_arg()
assert call_args.config == 'test_conf.json'
assert call_args.live is True
@ -141,6 +145,8 @@ def test_parse_args_backtesting_custom() -> None:
assert call_args.func is not None
assert call_args.ticker_interval == '1m'
assert call_args.refresh_pairs is True
assert type(call_args.strategy_list) is list
assert len(call_args.strategy_list) == 2
def test_parse_args_hyperopt_custom() -> None:

View File

@ -292,6 +292,61 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non
)
def test_setup_configuration_with_stratlist(mocker, default_conf, caplog) -> None:
"""
Test setup_configuration() function
"""
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(default_conf)
))
arglist = [
'--config', 'config.json',
'backtesting',
'--ticker-interval', '1m',
'--export', '/bar/foo',
'--strategy-list',
'DefaultStrategy',
'TestStrategy'
]
args = Arguments(arglist, '').get_parsed_arg()
configuration = Configuration(args)
config = configuration.get_config()
assert 'max_open_trades' in config
assert 'stake_currency' in config
assert 'stake_amount' in config
assert 'exchange' in config
assert 'pair_whitelist' in config['exchange']
assert 'datadir' in config
assert log_has(
'Using data folder: {} ...'.format(config['datadir']),
caplog.record_tuples
)
assert 'ticker_interval' in config
assert log_has('Parameter -i/--ticker-interval detected ...', caplog.record_tuples)
assert log_has(
'Using ticker_interval: 1m ...',
caplog.record_tuples
)
assert 'strategy_list' in config
assert log_has('Using strategy list of 2 Strategies', caplog.record_tuples)
assert 'position_stacking' not in config
assert 'use_max_market_positions' not in config
assert 'timerange' not in config
assert 'export' in config
assert log_has(
'Parameter --export detected: {} ...'.format(config['export']),
caplog.record_tuples
)
def test_hyperopt_with_arguments(mocker, default_conf, caplog) -> None:
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(default_conf)

View File

@ -43,10 +43,11 @@ def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None:
:return: None
"""
freqtrade.strategy.get_signal = lambda e, s, t: value
freqtrade.exchange.get_ticker_history = lambda p, i: None
freqtrade.exchange.get_candle_history = lambda p, i: None
freqtrade.exchange.refresh_tickers = lambda pl, i: {}
def patch_RPCManager(mocker) -> MagicMock:
"""
This function mock RPC manager to avoid repeating this code in almost every tests
@ -545,7 +546,7 @@ def test_create_trade_no_signal(default_conf, fee, mocker) -> None:
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
validate_pairs=MagicMock(),
get_ticker_history=MagicMock(return_value=20),
get_candle_history=MagicMock(return_value=20),
get_balance=MagicMock(return_value=20),
get_fee=fee,
)

View File

@ -1,4 +1,4 @@
ccxt==1.17.49
ccxt==1.17.63
SQLAlchemy==1.2.10
python-telegram-bot==10.1.0
arrow==0.12.1
@ -12,7 +12,7 @@ scipy==1.1.0
jsonschema==2.6.0
numpy==1.15.0
TA-Lib==0.4.17
pytest==3.6.4
pytest==3.7.0
pytest-mock==1.10.0
pytest-cov==2.5.1
pytest-asyncio==0.9.0

View File

@ -0,0 +1,93 @@
import os
import sys
root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
sys.path.append(root + '/python')
import ccxt # noqa: E402
def style(s, style):
return style + s + '\033[0m'
def green(s):
return style(s, '\033[92m')
def blue(s):
return style(s, '\033[94m')
def yellow(s):
return style(s, '\033[93m')
def red(s):
return style(s, '\033[91m')
def pink(s):
return style(s, '\033[95m')
def bold(s):
return style(s, '\033[1m')
def underline(s):
return style(s, '\033[4m')
def dump(*args):
print(' '.join([str(arg) for arg in args]))
def print_supported_exchanges():
dump('Supported exchanges:', green(', '.join(ccxt.exchanges)))
try:
id = sys.argv[1] # get exchange id from command line arguments
# check if the exchange is supported by ccxt
exchange_found = id in ccxt.exchanges
if exchange_found:
dump('Instantiating', green(id), 'exchange')
# instantiate the exchange by id
exchange = getattr(ccxt, id)({
# 'proxy':'https://cors-anywhere.herokuapp.com/',
})
# load all markets from the exchange
markets = exchange.load_markets()
# output a list of all market symbols
dump(green(id), 'has', len(exchange.symbols), 'symbols:', exchange.symbols)
tuples = list(ccxt.Exchange.keysort(markets).items())
# debug
for (k, v) in tuples:
print(v)
# output a table of all markets
dump(pink('{:<15} {:<15} {:<15} {:<15}'.format('id', 'symbol', 'base', 'quote')))
for (k, v) in tuples:
dump('{:<15} {:<15} {:<15} {:<15}'.format(v['id'], v['symbol'], v['base'], v['quote']))
else:
dump('Exchange ' + red(id) + ' not found')
print_supported_exchanges()
except Exception as e:
dump('[' + type(e).__name__ + ']', str(e))
dump("Usage: python " + sys.argv[0], green('id'))
print_supported_exchanges()

View File

@ -138,7 +138,7 @@ def plot_analyzed_dataframe(args: Namespace) -> None:
tickers = {}
if args.live:
logger.info('Downloading pair.')
tickers[pair] = exchange.get_ticker_history(pair, tick_interval)
tickers[pair] = exchange.get_candle_history(pair, tick_interval)
else:
tickers = optimize.load_data(
datadir=_CONF.get("datadir"),