adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into support_multiple_ticker

Browse Source
This commit is contained in:
Jean-Baptiste LE STANG 2018-01-20 19:25:47 +01:00
commit 36797cda30
22 changed files with 767 additions and 253 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
README.md
View File

@ -136,8 +136,8 @@ to understand the requirements before sending your pull-requests.
### Bot commands
```bash
usage: main.py [-h] [-c PATH] [-v] [--version] [--dynamic-whitelist [INT]]
[--dry-run-db]
usage: main.py [-h] [-v] [--version] [-c PATH] [--dry-run-db] [--datadir PATH]
[--dynamic-whitelist [INT]]
{backtesting,hyperopt} ...
Simple High Frequency Trading Bot for crypto currencies
@ -149,16 +149,17 @@ positional arguments:
optional arguments:
-h, --help show this help message and exit
-c PATH, --config PATH
specify configuration file (default: config.json)
-v, --verbose be verbose
--version show program's version number and exit
--dynamic-whitelist [INT]
dynamically generate and update whitelist based on 24h
BaseVolume (Default 20 currencies)
-c PATH, --config PATH
specify configuration file (default: config.json)
--dry-run-db Force dry run to use a local DB
"tradesv3.dry_run.sqlite" instead of memory DB. Work
only if dry_run is enabled.
--datadir PATH path to backtest data (default freqdata/tests/testdata
--dynamic-whitelist [INT]
dynamically generate and update whitelist based on 24h
BaseVolume (Default 20 currencies)
```
More details on:
- [How to run the bot](https://github.com/gcarq/freqtrade/blob/develop/docs/bot-usage.md#bot-commands)

42
docs/backtesting.md
View File

@ -51,15 +51,49 @@ python3 ./freqtrade/main.py backtesting --realistic-simulation --live
python3 ./freqtrade/main.py backtesting --datadir freqtrade/tests/testdata-20180101
```
**Exporting trades to file**
```bash
freqtrade backtesting --export trades
```
**Running backtest with smaller testset**
Use the `--timerange` argument to change how much of the testset
you want to use. The last N ticks/timeframes will be used.
Example:
```bash
python3 ./freqtrade/main.py backtesting --timerange=-200
```
***Advanced use of timerange***
Doing `--timerange=-200` will get the last 200 timeframes
from your inputdata. You can also specify specific dates,
or a range span indexed by start and stop.
The full timerange specification:
- Use last 123 tickframes of data: `--timerange=-123`
- Use first 123 tickframes of data: `--timerange=123-`
- Use tickframes from line 123 through 456: `--timerange=123-456`
Incoming feature, not implemented yet:
- `--timerange=-20180131`
- `--timerange=20180101-`
- `--timerange=20180101-20181231`
**Update testdata directory**
To update your testdata directory, or download into another testdata directory:
```bash
mkdir freqtrade/tests/testdata-20180113
cp freqtrade/tests/testdata/pairs.json freqtrade/tests/testdata-20180113
cd freqtrade/tests/testdata-20180113
mkdir -p user_data/data/testdata-20180113
cp freqtrade/tests/testdata/pairs.json user_data/data-20180113
cd user_data/data-20180113
```
Possibly edit pairs.json file to include/exclude pairs
python download_backtest_data.py -p pairs.json
```bash
python freqtrade/tests/testdata/download_backtest_data.py -p pairs.json
```
The script will read your pairs.json file, and download ticker data

1
docs/configuration.md
View File

@ -30,6 +30,7 @@ The table below will list all configuration parameters.
| `exchange.pair_whitelist` | [] | No | List of currency to use by the bot. Can be overrided with `--dynamic-whitelist` param.
| `exchange.pair_blacklist` | [] | No | List of currency the bot must avoid. Useful when using `--dynamic-whitelist` param.
| `experimental.use_sell_signal` | false | No | Use your sell strategy in addition of the `minimal_roi`.
| `experimental.sell_profit_only` | false | No | waits until you have made a positive profit before taking a sell decision.
| `telegram.enabled` | true | Yes | Enable or not the usage of Telegram.
| `telegram.token` | token | No | Your Telegram bot token. Only required is `enable` is `true`.
| `telegram.chat_id` | chat_id | No | Your personal Telegram account id. Only required is `enable` is `true`.

10
docs/hyperopt.md
View File

@ -168,6 +168,16 @@ If you would like to learn parameters using an alternate ticke-data that
you have on-disk, use the --datadir PATH option. Default hyperopt will
use data from directory freqtrade/tests/testdata.
### Running hyperopt with smaller testset
Use the --timeperiod argument to change how much of the testset
you want to use. The last N ticks/timeframes will be used.
Example:
```bash
python3 ./freqtrade/main.py hyperopt --timeperiod -200
```
### Hyperopt with MongoDB
Hyperopt with MongoDB, is like Hyperopt under steroids. As you saw by
executing the previous command is the execution takes a long time.

48
docs/plotting.md Normal file
View File

@ -0,0 +1,48 @@
# Plotting
This page explains how to plot prices, indicator, profits.
## Table of Contents
- [Plot price and indicators](#plot-price-and-indicators)
- [Plot profit](#plot-profit)
## Plot price and indicators
Usage for the price plotter:
script/plot_dataframe.py [-h] [-p pair]
Example
```
python script/plot_dataframe.py -p BTC_ETH,BTC_LTC
```
The -p pair argument, can be used to specify what
pair you would like to plot.
## Plot profit
The profit plotter show a picture with three plots:
1) Average closing price for all pairs
2) The summarized profit made by backtesting.
Note that this is not the real-world profit, but
more of an estimate.
3) Each pair individually profit
The first graph is good to get a grip of how the overall market
progresses.
The second graph will show how you algorithm works or doesnt.
Perhaps you want an algorithm that steadily makes small profits,
or one that acts less seldom, but makes big swings.
The third graph can be useful to spot outliers, events in pairs
that makes profit spikes.
Usage for the profit plotter:
script/plot_profit.py [-h] [-p pair] [--datadir directory] [--ticker_interval num]
The -p pair argument, can be used to plot a single pair
Example
```
python python scripts/plot_profit.py --datadir ../freqtrade/freqtrade/tests/testdata-20171221/ -p BTC_LTC
```

20
freqtrade/analyze.py
View File

@ -281,36 +281,36 @@ def analyze_ticker(ticker_history: List[Dict]) -> DataFrame:
return dataframe
def get_signal(pair: str, signal: SignalType, interval: int) -> bool:
def get_signal(pair: str, interval: int) -> (bool, bool):
"""
Calculates current signal based several technical analysis indicators
:param pair: pair in format BTC_ANT or BTC-ANT
:return: True if pair is good for buying, False otherwise
:return: (True, False) if pair is good for buying and not for selling
"""
ticker_hist = get_ticker_history(pair, interval)
if not ticker_hist:
logger.warning('Empty ticker history for pair %s', pair)
return False
return (False, False)
try:
dataframe = analyze_ticker(ticker_hist)
except ValueError as ex:
logger.warning('Unable to analyze ticker for pair %s: %s', pair, str(ex))
return False
return (False, False)
except Exception as ex:
logger.exception('Unexpected error when analyzing ticker for pair %s: %s', pair, str(ex))
return False
return (False, False)
if dataframe.empty:
return False
return (False, False)
latest = dataframe.iloc[-1]
# Check if dataframe is out of date
signal_date = arrow.get(latest['date'])
if signal_date < arrow.now() - timedelta(minutes=10):
return False
return (False, False)
result = latest[signal.value] == 1
logger.debug('%s_trigger: %s (pair=%s, signal=%s)', signal.value, latest['date'], pair, result)
return result
(buy, sell) = latest[SignalType.BUY.value] == 1, latest[SignalType.SELL.value] == 1
logger.debug('trigger: %s (pair=%s) buy=%s sell=%s', latest['date'], pair, str(buy), str(sell))
return (buy, sell)

52
freqtrade/main.py
View File

@ -14,7 +14,7 @@ from cachetools import cached, TTLCache
from freqtrade import (DependencyException, OperationalException, __version__,
exchange, persistence, rpc)
from freqtrade.analyze import SignalType, get_signal
from freqtrade.analyze import get_signal
from freqtrade.fiat_convert import CryptoToFiatConverter
from freqtrade.misc import (State, get_state, load_config, parse_args,
throttle, update_state)
@ -129,9 +129,17 @@ def check_handle_timedout(timeoutvalue: int) -> None:
timeoutthreashold = arrow.utcnow().shift(minutes=-timeoutvalue).datetime
for trade in Trade.query.filter(Trade.open_order_id.isnot(None)).all():
order = exchange.get_order(trade.open_order_id)
try:
order = exchange.get_order(trade.open_order_id)
except requests.exceptions.RequestException:
logger.info('Cannot query order for %s due to %s', trade, traceback.format_exc())
continue
ordertime = arrow.get(order['opened'])
# Check if trade is still actually open
if int(order['remaining']) == 0:
continue
if order['type'] == "LIMIT_BUY" and ordertime < timeoutthreashold:
# Buy timeout - cancel order
exchange.cancel_order(trade.open_order_id)
@ -140,6 +148,8 @@ def check_handle_timedout(timeoutvalue: int) -> None:
Trade.session.delete(trade)
Trade.session.flush()
logger.info('Buy order timeout for %s.', trade)
rpc.send_msg('*Timeout:* Unfilled buy order for {} cancelled'.format(
trade.pair.replace('_', '/')))
else:
# if trade is partially complete, edit the stake details for the trade
# and close the order
@ -147,6 +157,8 @@ def check_handle_timedout(timeoutvalue: int) -> None:
trade.stake_amount = trade.amount * trade.open_rate
trade.open_order_id = None
logger.info('Partial buy order timeout for %s.', trade)
rpc.send_msg('*Timeout:* Remaining buy order for {} cancelled'.format(
trade.pair.replace('_', '/')))
elif order['type'] == "LIMIT_SELL" and ordertime < timeoutthreashold:
# Sell timeout - cancel order and update trade
if order['remaining'] == order['amount']:
@ -157,6 +169,8 @@ def check_handle_timedout(timeoutvalue: int) -> None:
trade.close_date = None
trade.is_open = True
trade.open_order_id = None
rpc.send_msg('*Timeout:* Unfilled sell order for {} cancelled'.format(
trade.pair.replace('_', '/')))
logger.info('Sell order timeout for %s.', trade)
return True
else:
@ -247,24 +261,27 @@ def handle_trade(trade: Trade, interval: int) -> bool:
logger.debug('Handling %s ...', trade)
current_rate = exchange.get_ticker(trade.pair)['bid']
# Check if minimal roi has been reached
if min_roi_reached(trade, current_rate, datetime.utcnow()):
(buy, sell) = (False, False)
if _CONF.get('experimental', {}).get('use_sell_signal'):
(buy, sell) = get_signal(trade.pair)
# Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee)
if not buy and min_roi_reached(trade, current_rate, datetime.utcnow()):
logger.debug('Executing sell due to ROI ...')
execute_sell(trade, current_rate)
return True
# Experimental: Check if sell signal has been enabled and triggered
if _CONF.get('experimental', {}).get('use_sell_signal'):
# Experimental: Check if the trade is profitable before selling it (avoid selling at loss)
if _CONF.get('experimental', {}).get('sell_profit_only'):
logger.debug('Checking if trade is profitable ...')
if trade.calc_profit(rate=current_rate) <= 0:
return False
logger.debug('Checking sell_signal ...')
if get_signal(trade.pair, SignalType.SELL, interval):
logger.debug('Executing sell due to sell signal ...')
execute_sell(trade, current_rate)
return True
# Experimental: Check if the trade is profitable before selling it (avoid selling at loss)
if _CONF.get('experimental', {}).get('sell_profit_only', False):
logger.debug('Checking if trade is profitable ...')
if not buy and trade.calc_profit(rate=current_rate) <= 0:
return False
if sell and not buy:
logger.debug('Executing sell due to sell signal ...')
execute_sell(trade, current_rate)
return True
return False
@ -305,7 +322,8 @@ def create_trade(stake_amount: float, interval: int) -> bool:
# Pick pair based on StochRSI buy signals
for _pair in whitelist:
if get_signal(_pair, SignalType.BUY, interval):
(buy, sell) = get_signal(_pair)
if buy and not sell:
pair = _pair
break
else:

175
freqtrade/misc.py
View File

@ -4,6 +4,7 @@ import json
import logging
import time
import os
import re
from typing import Any, Callable, Dict, List
from jsonschema import Draft4Validator, validate
@ -115,6 +116,14 @@ def common_args_parser(description: str):
type=str,
metavar='PATH',
)
parser.add_argument(
'--datadir',
help='path to backtest data (default freqdata/tests/testdata)',
dest='datadir',
default=os.path.join('freqtrade', 'tests', 'testdata'),
type=str,
metavar='PATH',
)
return parser
@ -131,14 +140,6 @@ def parse_args(args: List[str], description: str):
action='store_true',
dest='dry_run_db',
)
parser.add_argument(
'-dd', '--datadir',
help='path to backtest data (default freqdata/tests/testdata',
dest='datadir',
default=os.path.join('freqtrade', 'tests', 'testdata'),
type=str,
metavar='PATH',
)
parser.add_argument(
'--dynamic-whitelist',
help='dynamically generate and update whitelist \
@ -154,6 +155,113 @@ def parse_args(args: List[str], description: str):
return parser.parse_args(args)
def backtesting_options(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
'-l', '--live',
action='store_true',
dest='live',
help='using live data',
)
parser.add_argument(
'-i', '--ticker-interval',
help='specify ticker interval in minutes (1, 5, 30, 60, 1440)',
dest='ticker_interval',
default=5,
type=int,
metavar='INT',
)
parser.add_argument(
'--realistic-simulation',
help='uses max_open_trades from config to simulate real world limitations',
action='store_true',
dest='realistic_simulation',
)
parser.add_argument(
'-r', '--refresh-pairs-cached',
help='refresh the pairs files in tests/testdata with the latest data from Bittrex. \
Use it if you want to run your backtesting with up-to-date data.',
action='store_true',
dest='refresh_pairs',
)
parser.add_argument(
'--export',
help='Export backtest results, argument are: trades\
Example --export=trades',
type=str,
default=None,
dest='export',
)
parser.add_argument(
'--timerange',
help='Specify what timerange of data to use.',
default=None,
type=str,
dest='timerange',
)
def hyperopt_options(parser: argparse.ArgumentParser) -> None:
parser.add_argument(
'-e', '--epochs',
help='specify number of epochs (default: 100)',
dest='epochs',
default=100,
type=int,
metavar='INT',
)
parser.add_argument(
'--use-mongodb',
help='parallelize evaluations with mongodb (requires mongod in PATH)',
dest='mongodb',
action='store_true',
)
parser.add_argument(
'-i', '--ticker-interval',
help='specify ticker interval in minutes (default: 5)',
dest='ticker_interval',
default=5,
type=int,
metavar='INT',
)
parser.add_argument(
'--timerange',
help='Specify what timerange of data to use.',
default=None,
type=str,
dest='timerange',
)
def parse_timerange(text):
if text is None:
return None
syntax = [('^-(\d{8})$', (None, 'date')),
('^(\d{8})-$', ('date', None)),
('^(\d{8})-(\d{8})$', ('date', 'date')),
('^(-\d+)$', (None, 'line')),
('^(\d+)-$', ('line', None)),
('^(\d+)-(\d+)$', ('index', 'index'))]
for rex, stype in syntax:
# Apply the regular expression to text
m = re.match(rex, text)
if m: # Regex has matched
rvals = m.groups()
n = 0
start = None
stop = None
if stype[0]:
start = rvals[n]
if stype[0] != 'date':
start = int(start)
n += 1
if stype[1]:
stop = rvals[n]
if stype[1] != 'date':
stop = int(stop)
return (stype, start, stop)
raise Exception('Incorrect syntax for timerange "%s"' % text)
def build_subcommands(parser: argparse.ArgumentParser) -> None:
""" Builds and attaches all subcommands """
from freqtrade.optimize import backtesting, hyperopt
@ -163,59 +271,12 @@ def build_subcommands(parser: argparse.ArgumentParser) -> None:
# Add backtesting subcommand
backtesting_cmd = subparsers.add_parser('backtesting', help='backtesting module')
backtesting_cmd.set_defaults(func=backtesting.start)
backtesting_cmd.add_argument(
'-l', '--live',
action='store_true',
dest='live',
help='using live data',
)
backtesting_cmd.add_argument(
'-i', '--ticker-interval',
help='specify ticker interval in minutes (1, 5, 30, 60, 1440)',
dest='ticker_interval',
default=5,
type=int,
metavar='INT',
)
backtesting_cmd.add_argument(
'--realistic-simulation',
help='uses max_open_trades from config to simulate real world limitations',
action='store_true',
dest='realistic_simulation',
)
backtesting_cmd.add_argument(
'-r', '--refresh-pairs-cached',
help='refresh the pairs files in tests/testdata with the latest data from Bittrex. \
Use it if you want to run your backtesting with up-to-date data.',
action='store_true',
dest='refresh_pairs',
)
backtesting_options(backtesting_cmd)
# Add hyperopt subcommand
hyperopt_cmd = subparsers.add_parser('hyperopt', help='hyperopt module')
hyperopt_cmd.set_defaults(func=hyperopt.start)
hyperopt_cmd.add_argument(
'-e', '--epochs',
help='specify number of epochs (default: 100)',
dest='epochs',
default=100,
type=int,
metavar='INT',
)
hyperopt_cmd.add_argument(
'--use-mongodb',
help='parallelize evaluations with mongodb (requires mongod in PATH)',
dest='mongodb',
action='store_true',
)
hyperopt_cmd.add_argument(
'-i', '--ticker-interval',
help='specify ticker interval in minutes (default: 5)',
dest='ticker_interval',
default=5,
type=int,
metavar='INT',
)
hyperopt_options(hyperopt_cmd)
# Required json-schema for user specified config

32
freqtrade/optimize/__init__.py
View File

@ -8,11 +8,25 @@ from pandas import DataFrame
from freqtrade.exchange import get_ticker_history
from freqtrade.optimize.hyperopt_conf import hyperopt_optimize_conf
from freqtrade.analyze import populate_indicators, parse_ticker_dataframe
from freqtrade import misc
logger = logging.getLogger(__name__)
def load_tickerdata_file(datadir, pair, ticker_interval):
def trim_tickerlist(tickerlist, timerange):
(stype, start, stop) = timerange
if stype == (None, 'line'):
return tickerlist[stop:]
elif stype == ('line', None):
return tickerlist[0:start]
elif stype == ('index', 'index'):
return tickerlist[start:stop]
else:
return tickerlist
def load_tickerdata_file(datadir, pair, ticker_interval,
timerange=None):
"""
Load a pair from file,
:return dict OR empty if unsuccesful
@ -30,11 +44,13 @@ def load_tickerdata_file(datadir, pair, ticker_interval):
# Read the file, load the json
with open(file) as tickerdata:
pairdata = json.load(tickerdata)
if timerange:
pairdata = trim_tickerlist(pairdata, timerange)
return pairdata
def load_data(datadir: str, ticker_interval: int, pairs: Optional[List[str]] = None,
refresh_pairs: Optional[bool] = False) -> Dict[str, List]:
refresh_pairs: Optional[bool] = False, timerange=None) -> Dict[str, List]:
"""
Loads ticker history data for the given parameters
:param ticker_interval: ticker interval in minutes
@ -51,16 +67,21 @@ def load_data(datadir: str, ticker_interval: int, pairs: Optional[List[str]] = N
download_pairs(datadir, _pairs, ticker_interval)
for pair in _pairs:
pairdata = load_tickerdata_file(datadir, pair, ticker_interval)
pairdata = load_tickerdata_file(datadir, pair, ticker_interval, timerange=timerange)
if not pairdata:
# download the tickerdata from exchange
download_backtesting_testdata(datadir, pair=pair, interval=ticker_interval)
# and retry reading the pair
pairdata = load_tickerdata_file(datadir, pair, ticker_interval)
pairdata = load_tickerdata_file(datadir, pair, ticker_interval, timerange=timerange)
result[pair] = pairdata
return result
def tickerdata_to_dataframe(data):
preprocessed = preprocess(data)
return preprocessed
def preprocess(tickerdata: Dict[str, List]) -> Dict[str, DataFrame]:
"""Creates a dataframe and populates indicators for given ticker data"""
return {pair: populate_indicators(parse_ticker_dataframe(pair_data))
@ -126,7 +147,6 @@ def download_backtesting_testdata(datadir: str, pair: str, interval: int = 5) ->
logger.debug("New End: {}".format(data[-1:][0]['T']))
data = sorted(data, key=lambda data: data['T'])
with open(filename, "wt") as fp:
json.dump(data, fp)
misc.file_dump_json(filename, data)
return True

139
freqtrade/optimize/backtesting.py
View File

@ -13,7 +13,6 @@ from freqtrade import exchange
from freqtrade.analyze import populate_buy_trend, populate_sell_trend
from freqtrade.exchange import Bittrex
from freqtrade.main import min_roi_reached
from freqtrade.optimize import preprocess
from freqtrade.persistence import Trade
logger = logging.getLogger(__name__)
@ -67,17 +66,60 @@ def generate_text_table(
return tabulate(tabular_data, headers=headers, floatfmt=floatfmt)
def backtest(stake_amount: float, processed: Dict[str, DataFrame],
max_open_trades: int = 0, realistic: bool = True, sell_profit_only: bool = False,
stoploss: int = -1.00, use_sell_signal: bool = False) -> DataFrame:
def get_trade_entry(pair, row, ticker, trade_count_lock, args):
stake_amount = args['stake_amount']
max_open_trades = args.get('max_open_trades', 0)
sell_profit_only = args.get('sell_profit_only', False)
stoploss = args.get('stoploss', -1)
use_sell_signal = args.get('use_sell_signal', False)
trade = Trade(open_rate=row.close,
open_date=row.date,
stake_amount=stake_amount,
amount=stake_amount / row.open,
fee=exchange.get_fee()
)
# calculate win/lose forwards from buy point
sell_subset = ticker[row.Index + 1:][['close', 'date', 'sell']]
for row2 in sell_subset.itertuples(index=True):
if max_open_trades > 0:
# Increase trade_count_lock for every iteration
trade_count_lock[row2.date] = trade_count_lock.get(row2.date, 0) + 1
current_profit_percent = trade.calc_profit_percent(rate=row2.close)
if (sell_profit_only and current_profit_percent < 0):
continue
if min_roi_reached(trade, row2.close, row2.date) or \
(row2.sell == 1 and use_sell_signal) or \
current_profit_percent <= stoploss:
current_profit_btc = trade.calc_profit(rate=row2.close)
return row2, (pair,
current_profit_percent,
current_profit_btc,
row2.Index - row.Index,
current_profit_btc > 0,
current_profit_btc < 0
)
def backtest(args) -> DataFrame:
"""
Implements backtesting functionality
:param stake_amount: btc amount to use for each trade
:param processed: a processed dictionary with format {pair, data}
:param max_open_trades: maximum number of concurrent trades (default: 0, disabled)
:param realistic: do we try to simulate realistic trades? (default: True)
:param args: a dict containing:
stake_amount: btc amount to use for each trade
processed: a processed dictionary with format {pair, data}
max_open_trades: maximum number of concurrent trades (default: 0, disabled)
realistic: do we try to simulate realistic trades? (default: True)
sell_profit_only: sell if profit only
use_sell_signal: act on sell-signal
stoploss: use stoploss
:return: DataFrame
"""
processed = args['processed']
max_open_trades = args.get('max_open_trades', 0)
realistic = args.get('realistic', True)
record = args.get('record', None)
records = []
trades = []
trade_count_lock: dict = {}
exchange._API = Bittrex({'key': '', 'secret': ''})
@ -100,41 +142,25 @@ def backtest(stake_amount: float, processed: Dict[str, DataFrame],
# Increase lock
trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1
trade = Trade(
open_rate=row.close,
open_date=row.date,
stake_amount=stake_amount,
amount=stake_amount / row.open,
fee=exchange.get_fee()
)
# calculate win/lose forwards from buy point
sell_subset = ticker[row.Index + 1:][['close', 'date', 'sell']]
for row2 in sell_subset.itertuples(index=True):
if max_open_trades > 0:
# Increase trade_count_lock for every iteration
trade_count_lock[row2.date] = trade_count_lock.get(row2.date, 0) + 1
current_profit_percent = trade.calc_profit_percent(rate=row2.close)
if (sell_profit_only and current_profit_percent < 0):
continue
if min_roi_reached(trade, row2.close, row2.date) or \
(row2.sell == 1 and use_sell_signal) or \
current_profit_percent <= stoploss:
current_profit_btc = trade.calc_profit(rate=row2.close)
lock_pair_until = row2.Index
trades.append(
(
pair,
current_profit_percent,
current_profit_btc,
row2.Index - row.Index,
current_profit_btc > 0,
current_profit_btc < 0
)
)
break
ret = get_trade_entry(pair, row, ticker,
trade_count_lock, args)
if ret:
row2, trade_entry = ret
lock_pair_until = row2.Index
trades.append(trade_entry)
if record:
# Note, need to be json.dump friendly
# record a tuple of pair, current_profit_percent,
# entry-date, duration
records.append((pair, trade_entry[1],
row.date.strftime('%s'),
row2.date.strftime('%s'),
row.Index, trade_entry[3]))
# For now export inside backtest(), maybe change so that backtest()
# returns a tuple like: (dataframe, records, logs, etc)
if record and record.find('trades') >= 0:
logger.info('Dumping backtest results')
misc.file_dump_json('backtest-result.json', records)
labels = ['currency', 'profit_percent', 'profit_BTC', 'duration', 'profit', 'loss']
return DataFrame.from_records(trades, columns=labels)
@ -167,6 +193,10 @@ def start(args):
logger.info('Using stake_currency: %s ...', config['stake_currency'])
logger.info('Using stake_amount: %s ...', config['stake_amount'])
timerange = misc.parse_timerange(args.timerange)
data = optimize.load_data(args.datadir, pairs=pairs, ticker_interval=args.ticker_interval,
refresh_pairs=args.refresh_pairs,
timerange=timerange)
max_open_trades = 0
if args.realistic_simulation:
logger.info('Using max_open_trades: %s ...', config['max_open_trades'])
@ -176,21 +206,22 @@ def start(args):
from freqtrade import main
main._CONF = config
preprocessed = preprocess(data)
preprocessed = optimize.tickerdata_to_dataframe(data)
# Print timeframe
min_date, max_date = get_timeframe(preprocessed)
logger.info('Measuring data from %s up to %s ...', min_date.isoformat(), max_date.isoformat())
# Execute backtest and print results
results = backtest(
stake_amount=config['stake_amount'],
processed=preprocessed,
max_open_trades=max_open_trades,
realistic=args.realistic_simulation,
sell_profit_only=config.get('experimental', {}).get('sell_profit_only', False),
stoploss=config.get('stoploss'),
use_sell_signal=config.get('experimental', {}).get('use_sell_signal', False)
)
sell_profit_only = config.get('experimental', {}).get('sell_profit_only', False)
use_sell_signal = config.get('experimental', {}).get('use_sell_signal', False)
results = backtest({'stake_amount': config['stake_amount'],
'processed': preprocessed,
'max_open_trades': max_open_trades,
'realistic': args.realistic_simulation,
'sell_profit_only': sell_profit_only,
'use_sell_signal': use_sell_signal,
'stoploss': config.get('stoploss'),
'record': args.export
})
logger.info(
'\n==================================== BACKTESTING REPORT ====================================\n%s', # noqa
generate_text_table(data, results, config['stake_currency'], args.ticker_interval)

13
freqtrade/optimize/hyperopt.py
View File

@ -15,7 +15,7 @@ from hyperopt import STATUS_FAIL, STATUS_OK, Trials, fmin, hp, space_eval, tpe
from hyperopt.mongoexp import MongoTrials
from pandas import DataFrame
from freqtrade import main # noqa
from freqtrade import main, misc # noqa
from freqtrade import exchange, optimize
from freqtrade.exchange import Bittrex
from freqtrade.misc import load_config
@ -164,7 +164,9 @@ def optimizer(params):
from freqtrade.optimize import backtesting
backtesting.populate_buy_trend = buy_strategy_generator(params)
results = backtest(OPTIMIZE_CONFIG['stake_amount'], PROCESSED, stoploss=params['stoploss'])
results = backtest({'stake_amount': OPTIMIZE_CONFIG['stake_amount'],
'processed': PROCESSED,
'stoploss': params['stoploss']})
result_explanation = format_results(results)
total_profit = results.profit_percent.sum()
@ -273,8 +275,11 @@ def start(args):
logger.info('Using config: %s ...', args.config)
config = load_config(args.config)
pairs = config['exchange']['pair_whitelist']
PROCESSED = optimize.preprocess(optimize.load_data(
args.datadir, pairs=pairs, ticker_interval=args.ticker_interval))
timerange = misc.parse_timerange(args.timerange)
data = optimize.load_data(args.datadir, pairs=pairs,
ticker_interval=args.ticker_interval,
timerange=timerange)
PROCESSED = optimize.tickerdata_to_dataframe(data)
if args.mongodb:
logger.info('Using mongodb ...')

18
freqtrade/rpc/telegram.py
View File

@ -241,20 +241,27 @@ def _daily(bot: Bot, update: Update) -> None:
.order_by(Trade.close_date)\
.all()
curdayprofit = sum(trade.calc_profit() for trade in trades)
profit_days[profitday] = format(curdayprofit, '.8f')
profit_days[profitday] = {
'amount': format(curdayprofit, '.8f'),
'trades': len(trades)
}
stats = [
[
key,
'{value:.8f} {symbol}'.format(value=float(value), symbol=_CONF['stake_currency']),
'{value:.8f} {symbol}'.format(
value=float(value['amount']),
symbol=_CONF['stake_currency']
),
'{value:.3f} {symbol}'.format(
value=_FIAT_CONVERT.convert_amount(
value,
value['amount'],
_CONF['stake_currency'],
_CONF['fiat_display_currency']
),
symbol=_CONF['fiat_display_currency']
)
),
'{value} trade{s}'.format(value=value['trades'], s='' if value['trades'] < 2 else 's'),
]
for key, value in profit_days.items()
]
@ -262,7 +269,8 @@ def _daily(bot: Bot, update: Update) -> None:
headers=[
'Day',
'Profit {}'.format(_CONF['stake_currency']),
'Profit {}'.format(_CONF['fiat_display_currency'])
'Profit {}'.format(_CONF['fiat_display_currency']),
'# Trades'
],
tablefmt='simple')

52
freqtrade/tests/optimize/test_backtesting.py
View File

@ -11,6 +11,13 @@ from freqtrade.optimize.backtesting import backtest, generate_text_table, get_ti
import freqtrade.optimize.backtesting as backtesting
def trim_dictlist(dl, num):
new = {}
for pair, pair_data in dl.items():
new[pair] = pair_data[num:]
return new
def test_generate_text_table():
results = pd.DataFrame(
{
@ -43,8 +50,11 @@ def test_backtest(default_conf, mocker):
exchange._API = Bittrex({'key': '', 'secret': ''})
data = optimize.load_data(None, ticker_interval=5, pairs=['BTC_ETH'])
results = backtest(default_conf['stake_amount'],
optimize.preprocess(data), 10, True)
data = trim_dictlist(data, -200)
results = backtest({'stake_amount': default_conf['stake_amount'],
'processed': optimize.preprocess(data),
'max_open_trades': 10,
'realistic': True})
assert not results.empty
@ -54,21 +64,17 @@ def test_backtest_1min_ticker_interval(default_conf, mocker):
# Run a backtesting for an exiting 5min ticker_interval
data = optimize.load_data(None, ticker_interval=1, pairs=['BTC_UNITEST'])
results = backtest(default_conf['stake_amount'],
optimize.preprocess(data), 1, True)
data = trim_dictlist(data, -200)
results = backtest({'stake_amount': default_conf['stake_amount'],
'processed': optimize.preprocess(data),
'max_open_trades': 1,
'realistic': True})
assert not results.empty
def trim_dictlist(dl, num):
new = {}
for pair, pair_data in dl.items():
new[pair] = pair_data[num:]
return new
def load_data_test(what):
data = optimize.load_data(None, ticker_interval=1, pairs=['BTC_UNITEST'])
data = trim_dictlist(data, -100)
timerange = ((None, 'line'), None, -100)
data = optimize.load_data(None, ticker_interval=1, pairs=['BTC_UNITEST'], timerange=timerange)
pair = data['BTC_UNITEST']
datalen = len(pair)
# Depending on the what parameter we now adjust the
@ -113,7 +119,10 @@ def simple_backtest(config, contour, num_results):
data = load_data_test(contour)
processed = optimize.preprocess(data)
assert isinstance(processed, dict)
results = backtest(config['stake_amount'], processed, 1, True)
results = backtest({'stake_amount': config['stake_amount'],
'processed': processed,
'max_open_trades': 1,
'realistic': True})
# results :: <class 'pandas.core.frame.DataFrame'>
assert len(results) == num_results
@ -125,8 +134,11 @@ def simple_backtest(config, contour, num_results):
def test_backtest2(default_conf, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
data = optimize.load_data(None, ticker_interval=5, pairs=['BTC_ETH'])
results = backtest(default_conf['stake_amount'],
optimize.preprocess(data), 10, True)
data = trim_dictlist(data, -200)
results = backtest({'stake_amount': default_conf['stake_amount'],
'processed': optimize.preprocess(data),
'max_open_trades': 10,
'realistic': True})
assert not results.empty
@ -149,10 +161,10 @@ def test_backtest_pricecontours(default_conf, mocker):
simple_backtest(default_conf, contour, numres)
def mocked_load_data(datadir, pairs=[], ticker_interval=0, refresh_pairs=False):
tickerdata = optimize.load_tickerdata_file(datadir, 'BTC_UNITEST', 1)
def mocked_load_data(datadir, pairs=[], ticker_interval=0, refresh_pairs=False, timerange=None):
tickerdata = optimize.load_tickerdata_file(datadir, 'BTC_UNITEST', 1, timerange=timerange)
pairdata = {'BTC_UNITEST': tickerdata}
return trim_dictlist(pairdata, -100)
return pairdata
def test_backtest_start(default_conf, mocker, caplog):
@ -166,6 +178,8 @@ def test_backtest_start(default_conf, mocker, caplog):
args.level = 10
args.live = False
args.datadir = None
args.export = None
args.timerange = '-100' # needed due to MagicMock malleability
backtesting.start(args)
# check the logs, that will contain the backtest result
exists = ['Using max_open_trades: 1 ...',

22
freqtrade/tests/optimize/test_hyperopt.py
View File

@ -54,6 +54,7 @@ def create_trials(mocker):
def test_start_calls_fmin(mocker):
trials = create_trials(mocker)
mocker.patch('freqtrade.optimize.tickerdata_to_dataframe')
mocker.patch('freqtrade.optimize.hyperopt.TRIALS', return_value=trials)
mocker.patch('freqtrade.optimize.hyperopt.sorted',
return_value=trials.results)
@ -61,7 +62,8 @@ def test_start_calls_fmin(mocker):
mocker.patch('freqtrade.optimize.load_data')
mock_fmin = mocker.patch('freqtrade.optimize.hyperopt.fmin', return_value={})
args = mocker.Mock(epochs=1, config='config.json.example', mongodb=False)
args = mocker.Mock(epochs=1, config='config.json.example', mongodb=False,
timerange=None)
start(args)
mock_fmin.assert_called_once()
@ -70,11 +72,12 @@ def test_start_calls_fmin(mocker):
def test_start_uses_mongotrials(mocker):
mock_mongotrials = mocker.patch('freqtrade.optimize.hyperopt.MongoTrials',
return_value=create_trials(mocker))
mocker.patch('freqtrade.optimize.preprocess')
mocker.patch('freqtrade.optimize.tickerdata_to_dataframe')
mocker.patch('freqtrade.optimize.load_data')
mocker.patch('freqtrade.optimize.hyperopt.fmin', return_value={})
args = mocker.Mock(epochs=1, config='config.json.example', mongodb=True)
args = mocker.Mock(epochs=1, config='config.json.example', mongodb=True,
timerange=None)
start(args)
mock_mongotrials.assert_called_once()
@ -125,11 +128,12 @@ def test_fmin_best_results(mocker, caplog):
}
mocker.patch('freqtrade.optimize.hyperopt.MongoTrials', return_value=create_trials(mocker))
mocker.patch('freqtrade.optimize.preprocess')
mocker.patch('freqtrade.optimize.tickerdata_to_dataframe')
mocker.patch('freqtrade.optimize.load_data')
mocker.patch('freqtrade.optimize.hyperopt.fmin', return_value=fmin_result)
args = mocker.Mock(epochs=1, config='config.json.example')
args = mocker.Mock(epochs=1, config='config.json.example',
timerange=None)
start(args)
exists = [
@ -147,11 +151,12 @@ def test_fmin_best_results(mocker, caplog):
def test_fmin_throw_value_error(mocker, caplog):
mocker.patch('freqtrade.optimize.hyperopt.MongoTrials', return_value=create_trials(mocker))
mocker.patch('freqtrade.optimize.preprocess')
mocker.patch('freqtrade.optimize.tickerdata_to_dataframe')
mocker.patch('freqtrade.optimize.load_data')
mocker.patch('freqtrade.optimize.hyperopt.fmin', side_effect=ValueError())
args = mocker.Mock(epochs=1, config='config.json.example')
args = mocker.Mock(epochs=1, config='config.json.example',
timerange=None)
start(args)
exists = [
@ -185,7 +190,8 @@ def test_resuming_previous_hyperopt_results_succeeds(mocker):
return_value={})
args = mocker.Mock(epochs=1,
config='config.json.example',
mongodb=False)
mongodb=False,
timerange=None)
start(args)

19
freqtrade/tests/optimize/test_optimize.py
View File

@ -130,21 +130,24 @@ def test_download_pairs(default_conf, ticker_history, mocker):
_backup_file(file2_1)
_backup_file(file2_5)
assert os.path.isfile(file1_1) is False
assert os.path.isfile(file2_1) is False
assert download_pairs(None, pairs=['BTC-MEME', 'BTC-CFI'], ticker_interval=1) is True
assert os.path.isfile(file1_1) is True
assert os.path.isfile(file1_5) is False
assert os.path.isfile(file2_1) is True
assert os.path.isfile(file2_5) is False
# clean files freshly downloaded
_clean_test_file(file1_1)
_clean_test_file(file2_1)
assert os.path.isfile(file1_5) is False
assert os.path.isfile(file2_5) is False
assert download_pairs(None, pairs=['BTC-MEME', 'BTC-CFI'], ticker_interval=5) is True
assert os.path.isfile(file1_1) is False
assert os.path.isfile(file1_5) is True
assert os.path.isfile(file2_1) is False
assert os.path.isfile(file2_5) is True
# clean files freshly downloaded
@ -199,3 +202,11 @@ def test_load_tickerdata_file():
assert not load_tickerdata_file(None, 'BTC_UNITEST', 7)
tickerdata = load_tickerdata_file(None, 'BTC_UNITEST', 1)
assert _btc_unittest_length == len(tickerdata)
def test_tickerdata_to_dataframe():
timerange = ((None, 'line'), None, -100)
tick = load_tickerdata_file(None, 'BTC_UNITEST', 1, timerange=timerange)
tickerlist = {'BTC_UNITEST': tick}
data = optimize.tickerdata_to_dataframe(tickerlist)
assert 100 == len(data['BTC_UNITEST'])

44
freqtrade/tests/rpc/test_rpc_telegram.py
View File

@ -77,7 +77,7 @@ def test_authorized_only_exception(default_conf, mocker):
def test_status_handle(default_conf, update, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
@ -112,7 +112,7 @@ def test_status_handle(default_conf, update, ticker, mocker):
def test_status_table_handle(default_conf, update, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple(
@ -154,7 +154,7 @@ def test_status_table_handle(default_conf, update, ticker, mocker):
def test_profit_handle(
default_conf, update, ticker, ticker_sell_up, limit_buy_order, limit_sell_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
@ -210,7 +210,7 @@ def test_profit_handle(
def test_forcesell_handle(default_conf, update, ticker, ticker_sell_up, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
_CONF=default_conf,
@ -247,7 +247,7 @@ def test_forcesell_handle(default_conf, update, ticker, ticker_sell_up, mocker):
def test_forcesell_down_handle(default_conf, update, ticker, ticker_sell_down, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
_CONF=default_conf,
@ -308,7 +308,7 @@ def test_exec_forcesell_open_orders(default_conf, ticker, mocker):
def test_forcesell_all_handle(default_conf, update, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
_CONF=default_conf,
@ -339,7 +339,7 @@ def test_forcesell_all_handle(default_conf, update, ticker, mocker):
def test_forcesell_handle_invalid(default_conf, update, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, True))
msg_mock = MagicMock()
mocker.patch.multiple('freqtrade.rpc.telegram',
_CONF=default_conf,
@ -376,7 +376,7 @@ def test_forcesell_handle_invalid(default_conf, update, mocker):
def test_performance_handle(
default_conf, update, ticker, limit_buy_order, limit_sell_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
@ -410,7 +410,7 @@ def test_performance_handle(
def test_daily_handle(
default_conf, update, ticker, limit_buy_order, limit_sell_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
@ -448,6 +448,28 @@ def test_daily_handle(
assert str(datetime.utcnow().date()) in msg_mock.call_args_list[0][0][0]
assert str(' 0.00006217 BTC') in msg_mock.call_args_list[0][0][0]
assert str(' 0.933 USD') in msg_mock.call_args_list[0][0][0]
assert str(' 1 trade') in msg_mock.call_args_list[0][0][0]
assert str(' 0 trade') in msg_mock.call_args_list[0][0][0]
# Reset msg_mock
msg_mock.reset_mock()
# Add two other trades
create_trade(0.001, int(default_conf['ticker_interval']))
create_trade(0.001, int(default_conf['ticker_interval']))
trades = Trade.query.all()
for trade in trades:
trade.update(limit_buy_order)
trade.update(limit_sell_order)
trade.close_date = datetime.utcnow()
trade.is_open = False
update.message.text = '/daily 1'
_daily(bot=MagicMock(), update=update)
assert str(' 0.00018651 BTC') in msg_mock.call_args_list[0][0][0]
assert str(' 2.798 USD') in msg_mock.call_args_list[0][0][0]
assert str(' 3 trades') in msg_mock.call_args_list[0][0][0]
# Try invalid data
msg_mock.reset_mock()
@ -460,7 +482,7 @@ def test_daily_handle(
def test_count_handle(default_conf, update, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
msg_mock = MagicMock()
mocker.patch.multiple(
'freqtrade.rpc.telegram',
@ -492,7 +514,7 @@ def test_count_handle(default_conf, update, ticker, mocker):
def test_performance_handle_invalid(default_conf, update, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, True))
msg_mock = MagicMock()
mocker.patch.multiple('freqtrade.rpc.telegram',
_CONF=default_conf,

20
freqtrade/tests/test_analyze.py
View File

@ -6,7 +6,7 @@ import arrow
import pytest
from pandas import DataFrame
from freqtrade.analyze import (SignalType, get_signal, parse_ticker_dataframe,
from freqtrade.analyze import (get_signal, parse_ticker_dataframe,
populate_buy_trend, populate_indicators,
populate_sell_trend)
@ -40,30 +40,30 @@ def test_returns_latest_buy_signal(mocker):
mocker.patch('freqtrade.analyze.get_ticker_history', return_value=MagicMock())
mocker.patch(
'freqtrade.analyze.analyze_ticker',
return_value=DataFrame([{'buy': 1, 'date': arrow.utcnow()}])
return_value=DataFrame([{'buy': 1, 'sell': 0, 'date': arrow.utcnow()}])
)
assert get_signal('BTC-ETH', SignalType.BUY, 5)
assert get_signal('BTC-ETH', 5) == (True, False)
mocker.patch(
'freqtrade.analyze.analyze_ticker',
return_value=DataFrame([{'buy': 0, 'date': arrow.utcnow()}])
return_value=DataFrame([{'buy': 0, 'sell': 1, 'date': arrow.utcnow()}])
)
assert not get_signal('BTC-ETH', SignalType.BUY, 5)
assert get_signal('BTC-ETH',5) == (False, True)
def test_returns_latest_sell_signal(mocker):
mocker.patch('freqtrade.analyze.get_ticker_history', return_value=MagicMock())
mocker.patch(
'freqtrade.analyze.analyze_ticker',
return_value=DataFrame([{'sell': 1, 'date': arrow.utcnow()}])
return_value=DataFrame([{'sell': 1, 'buy': 0, 'date': arrow.utcnow()}])
)
assert get_signal('BTC-ETH', SignalType.SELL, 5)
assert get_signal('BTC-ETH', 5) == (False, True)
mocker.patch(
'freqtrade.analyze.analyze_ticker',
return_value=DataFrame([{'sell': 0, 'date': arrow.utcnow()}])
return_value=DataFrame([{'sell': 0, 'buy': 1, 'date': arrow.utcnow()}])
)
assert not get_signal('BTC-ETH', SignalType.SELL, 5)
assert get_signal('BTC-ETH', 5) == (True, False)
def test_get_signal_handles_exceptions(mocker):
@ -71,4 +71,4 @@ def test_get_signal_handles_exceptions(mocker):
mocker.patch('freqtrade.analyze.analyze_ticker',
side_effect=Exception('invalid ticker history '))
assert not get_signal('BTC-ETH', SignalType.BUY, 5)
assert get_signal('BTC-ETH', 5) == (False, False)

117
freqtrade/tests/test_main.py
View File

@ -10,7 +10,6 @@ from sqlalchemy import create_engine
import freqtrade.main as main
from freqtrade import DependencyException, OperationalException
from freqtrade.analyze import SignalType
from freqtrade.exchange import Exchanges
from freqtrade.main import (_process, check_handle_timedout, create_trade,
execute_sell, get_target_bid, handle_trade, init)
@ -52,7 +51,7 @@ def test_main_start_hyperopt(mocker):
def test_process_trade_creation(default_conf, ticker, limit_buy_order, health, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -82,7 +81,7 @@ def test_process_trade_creation(default_conf, ticker, limit_buy_order, health, m
def test_process_exchange_failures(default_conf, ticker, health, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
sleep_mock = mocker.patch('time.sleep', side_effect=lambda _: None)
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -99,7 +98,7 @@ def test_process_operational_exception(default_conf, ticker, health, mocker):
msg_mock = MagicMock()
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=msg_mock)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -117,8 +116,7 @@ def test_process_operational_exception(default_conf, ticker, health, mocker):
def test_process_trade_handling(default_conf, ticker, limit_buy_order, health, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.main.get_signal',
side_effect=lambda *args: False if args[1] == SignalType.SELL else True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -140,7 +138,7 @@ def test_process_trade_handling(default_conf, ticker, limit_buy_order, health, m
def test_create_trade(default_conf, ticker, limit_buy_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -171,7 +169,7 @@ def test_create_trade(default_conf, ticker, limit_buy_order, mocker):
def test_create_trade_minimal_amount(default_conf, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
buy_mock = mocker.patch(
'freqtrade.main.exchange.buy', MagicMock(return_value='mocked_limit_buy')
)
@ -187,7 +185,7 @@ def test_create_trade_minimal_amount(default_conf, ticker, mocker):
def test_create_trade_no_stake_amount(default_conf, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -200,7 +198,7 @@ def test_create_trade_no_stake_amount(default_conf, ticker, mocker):
def test_create_trade_no_pairs(default_conf, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -216,7 +214,7 @@ def test_create_trade_no_pairs(default_conf, ticker, mocker):
def test_create_trade_no_pairs_after_blacklist(default_conf, ticker, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -233,7 +231,7 @@ def test_create_trade_no_pairs_after_blacklist(default_conf, ticker, mocker):
def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -256,6 +254,7 @@ def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, mocker):
trade.update(limit_buy_order)
assert trade.is_open is True
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval'])) is True
assert trade.open_order_id == 'mocked_limit_sell'
@ -268,11 +267,57 @@ def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, mocker):
assert trade.close_date is not None
def test_handle_overlpapping_signals(default_conf, ticker, mocker, caplog):
default_conf.update({'experimental': {'use_sell_signal': True}})
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, True))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
buy=MagicMock(return_value='mocked_limit_buy'))
mocker.patch('freqtrade.main.min_roi_reached', return_value=False)
init(default_conf, create_engine('sqlite://'))
create_trade(0.001, int(default_conf['ticker_interval']))
# Buy and Sell triggering, so doing nothing ...
trades = Trade.query.all()
assert len(trades) == 0
# Buy is triggering, so buying ...
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
create_trade(0.001, int(default_conf['ticker_interval']))
trades = Trade.query.all()
assert len(trades) == 1
assert trades[0].is_open is True
# Buy and Sell are not triggering, so doing nothing ...
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, False))
assert handle_trade(trades[0], int(default_conf['ticker_interval'])) is False
trades = Trade.query.all()
assert len(trades) == 1
assert trades[0].is_open is True
# Buy and Sell are triggering, so doing nothing ...
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, True))
assert handle_trade(trades[0], int(default_conf['ticker_interval'])) is False
trades = Trade.query.all()
assert len(trades) == 1
assert trades[0].is_open is True
# Sell is triggering, guess what : we are Selling!
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
trades = Trade.query.all()
assert handle_trade(trades[0], int(default_conf['ticker_interval'])) is True
def test_handle_trade_roi(default_conf, ticker, mocker, caplog):
default_conf.update({'experimental': {'use_sell_signal': True}})
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -291,13 +336,12 @@ def test_handle_trade_roi(default_conf, ticker, mocker, caplog):
# we might just want to check if we are in a sell condition without
# executing
# if ROI is reached we must sell
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, interval=int(default_conf['ticker_interval']))
assert ('freqtrade', logging.DEBUG, 'Executing sell due to ROI ...') in caplog.record_tuples
# if ROI is reached we must sell even if sell-signal is not signalled
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, interval=int(default_conf['ticker_interval']))
assert ('freqtrade', logging.DEBUG, 'Executing sell due to ROI ...') in caplog.record_tuples
@ -305,7 +349,7 @@ def test_handle_trade_experimental(default_conf, ticker, mocker, caplog):
default_conf.update({'experimental': {'use_sell_signal': True}})
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -319,11 +363,10 @@ def test_handle_trade_experimental(default_conf, ticker, mocker, caplog):
trade = Trade.query.first()
trade.is_open = True
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, False))
value_returned = handle_trade(trade, int(default_conf['ticker_interval']))
assert ('freqtrade', logging.DEBUG, 'Checking sell_signal ...') in caplog.record_tuples
assert value_returned is False
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval']))
s = 'Executing sell due to sell signal ...'
assert ('freqtrade', logging.DEBUG, s) in caplog.record_tuples
@ -331,7 +374,7 @@ def test_handle_trade_experimental(default_conf, ticker, mocker, caplog):
def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -356,7 +399,8 @@ def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order, mo
def test_check_handle_timedout_buy(default_conf, ticker, limit_buy_order_old, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
cancel_order_mock = MagicMock()
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -381,6 +425,7 @@ def test_check_handle_timedout_buy(default_conf, ticker, limit_buy_order_old, mo
# check it does cancel buy orders over the time limit
check_handle_timedout(600)
assert cancel_order_mock.call_count == 1
assert rpc_mock.call_count == 1
trades = Trade.query.filter(Trade.open_order_id.is_(trade_buy.open_order_id)).all()
assert len(trades) == 0
@ -388,7 +433,8 @@ def test_check_handle_timedout_buy(default_conf, ticker, limit_buy_order_old, mo
def test_check_handle_timedout_sell(default_conf, ticker, limit_sell_order_old, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
cancel_order_mock = MagicMock()
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -414,6 +460,7 @@ def test_check_handle_timedout_sell(default_conf, ticker, limit_sell_order_old,
# check it does cancel sell orders over the time limit
check_handle_timedout(600)
assert cancel_order_mock.call_count == 1
assert rpc_mock.call_count == 1
assert trade_sell.is_open is True
@ -421,7 +468,8 @@ def test_check_handle_timedout_partial(default_conf, ticker, limit_buy_order_old
mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
cancel_order_mock = MagicMock()
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
get_ticker=ticker,
@ -447,6 +495,7 @@ def test_check_handle_timedout_partial(default_conf, ticker, limit_buy_order_old
# note this is for a partially-complete buy order
check_handle_timedout(600)
assert cancel_order_mock.call_count == 1
assert rpc_mock.call_count == 1
trades = Trade.query.filter(Trade.open_order_id.is_(trade_buy.open_order_id)).all()
assert len(trades) == 1
assert trades[0].amount == 23.0
@ -470,7 +519,7 @@ def test_balance_bigger_last_ask(mocker):
def test_execute_sell_up(default_conf, ticker, ticker_sell_up, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
@ -503,7 +552,7 @@ def test_execute_sell_up(default_conf, ticker, ticker_sell_up, mocker):
def test_execute_sell_down(default_conf, ticker, ticker_sell_down, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.rpc.telegram',
@ -540,7 +589,7 @@ def test_execute_sell_down(default_conf, ticker, ticker_sell_down, mocker):
def test_execute_sell_without_conf_sell_down(default_conf, ticker, ticker_sell_down, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s,: (True, False))
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
@ -572,7 +621,7 @@ def test_execute_sell_without_conf_sell_down(default_conf, ticker, ticker_sell_d
def test_execute_sell_without_conf_sell_up(default_conf, ticker, ticker_sell_up, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch('freqtrade.rpc.init', MagicMock())
rpc_mock = mocker.patch('freqtrade.main.rpc.send_msg', MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
@ -609,7 +658,7 @@ def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.min_roi_reached', return_value=False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -625,6 +674,7 @@ def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, mocker):
trade = Trade.query.first()
trade.update(limit_buy_order)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval'])) is True
@ -636,7 +686,7 @@ def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.min_roi_reached', return_value=False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -652,6 +702,7 @@ def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, mocker):
trade = Trade.query.first()
trade.update(limit_buy_order)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval'])) is True
@ -663,7 +714,7 @@ def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.min_roi_reached', return_value=False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -679,6 +730,7 @@ def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, mocker):
trade = Trade.query.first()
trade.update(limit_buy_order)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval'])) is False
@ -690,7 +742,7 @@ def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, mocker):
mocker.patch.dict('freqtrade.main._CONF', default_conf)
mocker.patch('freqtrade.main.min_roi_reached', return_value=False)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s, t, i: True)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (True, False))
mocker.patch.multiple('freqtrade.rpc', init=MagicMock(), send_msg=MagicMock())
mocker.patch.multiple('freqtrade.main.exchange',
validate_pairs=MagicMock(),
@ -706,4 +758,5 @@ def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, mocker):
trade = Trade.query.first()
trade.update(limit_buy_order)
mocker.patch('freqtrade.main.get_signal', side_effect=lambda s: (False, True))
assert handle_trade(trade, int(default_conf['ticker_interval'])) is True

18
freqtrade/tests/test_misc.py
View File

@ -5,10 +5,11 @@ import time
from copy import deepcopy
import pytest
from unittest.mock import MagicMock
from jsonschema import ValidationError
from freqtrade.misc import (common_args_parser, load_config, parse_args,
throttle)
throttle, file_dump_json, parse_timerange)
def test_throttle():
@ -133,6 +134,21 @@ def test_parse_args_hyperopt_custom(mocker):
assert call_args.func is not None
def test_file_dump_json(default_conf, mocker):
file_open = mocker.patch('freqtrade.misc.open', MagicMock())
json_dump = mocker.patch('json.dump', MagicMock())
file_dump_json('somefile', [1, 2, 3])
assert file_open.call_count == 1
assert json_dump.call_count == 1
def test_parse_timerange_incorrect():
assert ((None, 'line'), None, -200) == parse_timerange('-200')
assert (('line', None), 200, None) == parse_timerange('200-')
with pytest.raises(Exception, match=r'Incorrect syntax.*'):
parse_timerange('-')
def test_load_config(default_conf, mocker):
file_mock = mocker.patch('freqtrade.misc.open', mocker.mock_open(
read_data=json.dumps(default_conf)

4
requirements.txt
View File

@ -1,7 +1,7 @@
python-bittrex==0.2.2
SQLAlchemy==1.2.1
python-telegram-bot==9.0.0
arrow==0.12.0
arrow==0.12.1
cachetools==2.0.1
requests==2.18.4
urllib3==1.22
@ -11,7 +11,7 @@ scikit-learn==0.19.1
scipy==1.0.0
jsonschema==2.6.0
numpy==1.14.0
TA-Lib==0.4.15
TA-Lib==0.4.16
pytest==3.3.2
pytest-mock==1.6.3
pytest-cov==2.5.1

2
scripts/plot_dataframe.py
View File

@ -28,7 +28,7 @@ def plot_parse_args(args ):
return parser.parse_args(args)
def plot_analyzed_dataframe(args) -> None:
def plot_analyzed_dataframe(args):
"""
Calls analyze() and plots the returned dataframe
:param pair: pair as str

155
scripts/plot_profit.py Executable file
View File

@ -0,0 +1,155 @@
#!/usr/bin/env python3
import sys
import argparse
import json
import matplotlib.pyplot as plt
import numpy as np
import freqtrade.optimize as optimize
import freqtrade.misc as misc
import freqtrade.exchange as exchange
import freqtrade.analyze as analyze
def plot_parse_args(args ):
parser = misc.common_args_parser('Graph utility')
# FIX: perhaps delete those backtesting options that are not feasible (shows up in -h)
misc.backtesting_options(parser)
parser.add_argument(
'-p', '--pair',
help = 'Show profits for only this pairs. Pairs are comma-separated.',
dest = 'pair',
default = None
)
return parser.parse_args(args)
# data:: [ pair, profit-%, enter, exit, time, duration]
# data:: ['BTC_XMR', 0.00537847, '1511176800', '1511178000', 5057, 1]
# FIX: make use of the enter/exit dates to insert the
# profit more precisely into the pg array
def make_profit_array(data, px, filter_pairs=[]):
pg = np.zeros(px)
# Go through the trades
# and make an total profit
# array
for trade in data:
pair = trade[0]
if filter_pairs and pair not in filter_pairs:
continue
profit = trade[1]
tim = trade[4]
dur = trade[5]
pg[tim+dur-1] += profit
# rewrite the pg array to go from
# total profits at each timeframe
# to accumulated profits
pa = 0
for x in range(0,len(pg)):
p = pg[x] # Get current total percent
pa += p # Add to the accumulated percent
pg[x] = pa # write back to save memory
return pg
def plot_profit(args) -> None:
"""
Plots the total profit for all pairs.
Note, the profit calculation isn't realistic.
But should be somewhat proportional, and therefor useful
in helping out to find a good algorithm.
"""
# We need to use the same pairs, same tick_interval
# and same timeperiod as used in backtesting
# to match the tickerdata against the profits-results
filter_pairs = args.pair
config = misc.load_config(args.config)
pairs = config['exchange']['pair_whitelist']
if filter_pairs:
filter_pairs = filter_pairs.split(',')
pairs = list(set(pairs) & set(filter_pairs))
print('Filter, keep pairs %s' % pairs)
tickers = optimize.load_data(args.datadir, pairs=pairs,
ticker_interval=args.ticker_interval,
refresh_pairs=False)
dataframes = optimize.preprocess(tickers)
# Make an average close price of all the pairs that was involved.
# this could be useful to gauge the overall market trend
# FIX: since the dataframes are of unequal length,
# andor has different dates, we need to merge them
# But we dont have the date information in the
# backtesting results, this is needed to match the dates
# For now, assume the dataframes are aligned.
max_x = 0
for pair, pair_data in dataframes.items():
n = len(pair_data['close'])
max_x = max(max_x, n)
# if max_x != n:
# raise Exception('Please rerun script. Input data has different lengths %s'
# %('Different pair length: %s <=> %s' %(max_x, n)))
print('max_x: %s' %(max_x))
# We are essentially saying:
# array <- sum dataframes[*]['close'] / num_items dataframes
# FIX: there should be some onliner numpy/panda for this
avgclose = np.zeros(max_x)
num = 0
for pair, pair_data in dataframes.items():
close = pair_data['close']
maxprice = max(close) # Normalize price to [0,1]
print('Pair %s has length %s' %(pair, len(close)))
for x in range(0, len(close)):
avgclose[x] += close[x] / maxprice
# avgclose += close
num += 1
avgclose /= num
# Load the profits results
# And make an profits-growth array
filename = 'backtest-result.json'
with open(filename) as file:
data = json.load(file)
pg = make_profit_array(data, max_x, filter_pairs)
#
# Plot the pairs average close prices, and total profit growth
#
fig, (ax1, ax2, ax3) = plt.subplots(3, sharex=True)
fig.suptitle('total profit')
ax1.plot(avgclose, label='avgclose')
ax2.plot(pg, label='profit')
ax1.legend(loc='upper left')
ax2.legend(loc='upper left')
# FIX if we have one line pair in paris
# then skip the plotting of the third graph,
# or change what we plot
# In third graph, we plot each profit separately
for pair in pairs:
pg = make_profit_array(data, max_x, pair)
ax3.plot(pg, label=pair)
ax3.legend(loc='upper left')
# black background to easier see multiple colors
ax3.set_facecolor('black')
# Fine-tune figure; make subplots close to each other and hide x ticks for
# all but bottom plot.
fig.subplots_adjust(hspace=0)
plt.setp([a.get_xticklabels() for a in fig.axes[:-1]], visible=False)
plt.show()
if __name__ == '__main__':
args = plot_parse_args(sys.argv[1:])
plot_profit(args)