whitelist conflict resolved with develop branch
This commit is contained in:
misagh
commit
080ecae332
|
|
@ -1,4 +1,5 @@
|
|||
# Hyperopt
|
||||
|
||||
This page explains how to tune your strategy by finding the optimal
|
||||
parameters, a process called hyperparameter optimization. The bot uses several
|
||||
algorithms included in the `scikit-optimize` package to accomplish this. The
|
||||
|
|
@ -8,17 +9,20 @@ and still take a long time.
|
|||
*Note:* Hyperopt will crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133)
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Prepare your Hyperopt](#prepare-hyperopt)
|
||||
- [Configure your Guards and Triggers](#configure-your-guards-and-triggers)
|
||||
- [Solving a Mystery](#solving-a-mystery)
|
||||
- [Adding New Indicators](#adding-new-indicators)
|
||||
- [Execute Hyperopt](#execute-hyperopt)
|
||||
- [Understand the hyperopts result](#understand-the-backtesting-result)
|
||||
- [Understand the hyperopt result](#understand-the-hyperopt-result)
|
||||
|
||||
## Prepare Hyperopting
|
||||
|
||||
We recommend you start by taking a look at `hyperopt.py` file located in [freqtrade/optimize](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py)
|
||||
|
||||
### Configure your Guards and Triggers
|
||||
|
||||
There are two places you need to change to add a new buy strategy for testing:
|
||||
- Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L231-L264).
|
||||
- Inside [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L213-L224)
|
||||
|
|
@ -113,11 +117,12 @@ When you want to test an indicator that isn't used by the bot currently, remembe
|
|||
add it to the `populate_indicators()` method in `hyperopt.py`.
|
||||
|
||||
## Execute Hyperopt
|
||||
Once you have updated your hyperopt configuration you can run it.
|
||||
Because hyperopt tries a lot of combination to find the best parameters
|
||||
it will take time you will have the result (more than 30 mins).
|
||||
|
||||
We strongly recommend to use `screen` to prevent any connection loss.
|
||||
Once you have updated your hyperopt configuration you can run it.
|
||||
Because hyperopt tries a lot of combinations to find the best parameters it will take time you will have the result (more than 30 mins).
|
||||
|
||||
We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
|
||||
|
||||
```bash
|
||||
python3 ./freqtrade/main.py -c config.json hyperopt -e 5000
|
||||
```
|
||||
|
|
@ -126,11 +131,13 @@ The `-e` flag will set how many evaluations hyperopt will do. We recommend
|
|||
running at least several thousand evaluations.
|
||||
|
||||
### Execute Hyperopt with Different Ticker-Data Source
|
||||
|
||||
If you would like to hyperopt parameters using an alternate ticker data that
|
||||
you have on-disk, use the `--datadir PATH` option. Default hyperopt will
|
||||
use data from directory `user_data/data`.
|
||||
|
||||
### Running Hyperopt with Smaller Testset
|
||||
|
||||
Use the `--timerange` argument to change how much of the testset
|
||||
you want to use. The last N ticks/timeframes will be used.
|
||||
Example:
|
||||
|
|
@ -140,6 +147,7 @@ python3 ./freqtrade/main.py hyperopt --timerange -200
|
|||
```
|
||||
|
||||
### Running Hyperopt with Smaller Search Space
|
||||
|
||||
Use the `--spaces` argument to limit the search space used by hyperopt.
|
||||
Letting Hyperopt optimize everything is a huuuuge search space. Often it
|
||||
might make more sense to start by just searching for initial buy algorithm.
|
||||
|
|
@ -154,7 +162,8 @@ Legal values are:
|
|||
- `stoploss`: search for the best stoploss value
|
||||
- space-separated list of any of the above values for example `--spaces roi stoploss`
|
||||
|
||||
## Understand the Hyperopts Result
|
||||
## Understand the Hyperopt Result
|
||||
|
||||
Once Hyperopt is completed you can use the result to create a new strategy.
|
||||
Given the following result from hyperopt:
|
||||
|
||||
|
|
@ -166,22 +175,24 @@ with values:
|
|||
```
|
||||
|
||||
You should understand this result like:
|
||||
|
||||
- The buy trigger that worked best was `bb_lower`.
|
||||
- You should not use ADX because `adx-enabled: False`)
|
||||
- You should **consider** using the RSI indicator (`rsi-enabled: True` and the best value is `29.0` (`rsi-value: 29.0`)
|
||||
|
||||
You have to look inside your strategy file into `buy_strategy_generator()`
|
||||
method, what those values match to.
|
||||
method, what those values match to.
|
||||
|
||||
So for example you had `rsi-value: 29.0` so we would look
|
||||
at `rsi`-block, that translates to the following code block:
|
||||
So for example you had `rsi-value: 29.0` so we would look at `rsi`-block, that translates to the following code block:
|
||||
|
||||
```
|
||||
(dataframe['rsi'] < 29.0)
|
||||
```
|
||||
|
||||
Translating your whole hyperopt result as the new buy-signal
|
||||
would then look like:
|
||||
```
|
||||
|
||||
```python
|
||||
def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
|
||||
dataframe.loc[
|
||||
(
|
||||
|
|
@ -192,6 +203,39 @@ def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
|
|||
return dataframe
|
||||
```
|
||||
|
||||
### Understand Hyperopt ROI results
|
||||
|
||||
If you are optimizing ROI, you're result will look as follows and include a ROI table.
|
||||
|
||||
```
|
||||
Best result:
|
||||
135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins.
|
||||
with values:
|
||||
{'adx-value': 44, 'rsi-value': 29, 'adx-enabled': False, 'rsi-enabled': True, 'trigger': 'bb_lower', 'roi_t1': 40, 'roi_t2': 57, 'roi_t3': 21, 'roi_p1': 0.03634636907306948, 'roi_p2': 0.055237357937802885, 'roi_p3': 0.015163796015548354, 'stoploss': -0.37996664668703606}
|
||||
ROI table:
|
||||
{0: 0.10674752302642071, 21: 0.09158372701087236, 78: 0.03634636907306948, 118: 0}
|
||||
```
|
||||
|
||||
This would translate to the following ROI table:
|
||||
|
||||
``` python
|
||||
minimal_roi = {
|
||||
"118": 0,
|
||||
"78": 0.0363463,
|
||||
"21": 0.0915,
|
||||
"0": 0.106
|
||||
}
|
||||
```
|
||||
|
||||
### Validate backtest result
|
||||
|
||||
Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.
|
||||
To archive the same results (number of trades, ...) than during hyperopt, please use the command line flag `--disable-max-market-positions`.
|
||||
This setting is the default for hyperopt for speed reasons. You can overwrite this in the configuration by setting `"position_stacking"=false` or by changing the relevant line in your hyperopt file [here](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L283).
|
||||
|
||||
Dry/live runs will **NOT** use position stacking - therefore it does make sense to also validate the strategy without this as it's closer to reality.
|
||||
|
||||
## Next Step
|
||||
|
||||
Now you have a perfect bot and want to control it from Telegram. Your
|
||||
next step is to learn the [Telegram usage](https://github.com/freqtrade/freqtrade/blob/develop/docs/telegram-usage.md).
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ class Edge():
|
|||
self.strategy: IStrategy = StrategyResolver(self.config).strategy
|
||||
self.ticker_interval = self.strategy.ticker_interval
|
||||
self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
|
||||
self.get_timeframe = Backtesting.get_timeframe
|
||||
self.get_timeframe = optimize.get_timeframe
|
||||
self.advise_sell = self.strategy.advise_sell
|
||||
self.advise_buy = self.strategy.advise_buy
|
||||
|
||||
|
|
|
|||
|
|
@ -375,6 +375,8 @@ class Exchange(object):
|
|||
def get_ticker(self, pair: str, refresh: Optional[bool] = True) -> dict:
|
||||
if refresh or pair not in self._cached_ticker.keys():
|
||||
try:
|
||||
if pair not in self._api.markets:
|
||||
raise DependencyException(f"Pair {pair} not available")
|
||||
data = self._api.fetch_ticker(pair)
|
||||
try:
|
||||
self._cached_ticker[pair] = {
|
||||
|
|
|
|||
|
|
@ -61,6 +61,7 @@ class FreqtradeBot(object):
|
|||
if self.config.get('edge', {}).get('enabled', False):
|
||||
self.edge = Edge(self.config, self.exchange)
|
||||
|
||||
self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist']
|
||||
self._init_modules()
|
||||
|
||||
def _init_modules(self) -> None:
|
||||
|
|
@ -114,11 +115,8 @@ class FreqtradeBot(object):
|
|||
constants.PROCESS_THROTTLE_SECS
|
||||
)
|
||||
|
||||
nb_assets = self.config.get('dynamic_whitelist', None)
|
||||
|
||||
self._throttle(func=self._process,
|
||||
min_secs=min_secs,
|
||||
nb_assets=nb_assets)
|
||||
min_secs=min_secs)
|
||||
return state
|
||||
|
||||
def _startup_messages(self) -> None:
|
||||
|
|
@ -169,15 +167,15 @@ class FreqtradeBot(object):
|
|||
time.sleep(duration)
|
||||
return result
|
||||
|
||||
def _process(self, nb_assets: Optional[int] = 0) -> bool:
|
||||
def _process(self) -> bool:
|
||||
"""
|
||||
Queries the persistence layer for open trades and handles them,
|
||||
otherwise a new trade is created.
|
||||
:param: nb_assets: the maximum number of pairs to be traded at the same time
|
||||
:return: True if one or more trades has been created or closed, False otherwise
|
||||
"""
|
||||
state_changed = False
|
||||
try:
|
||||
nb_assets = self.config.get('dynamic_whitelist', None)
|
||||
# Refresh whitelist based on wallet maintenance
|
||||
sanitized_list = self._refresh_whitelist(
|
||||
self._gen_pair_whitelist(
|
||||
|
|
@ -186,8 +184,7 @@ class FreqtradeBot(object):
|
|||
)
|
||||
|
||||
# Keep only the subsets of pairs wanted (up to nb_assets)
|
||||
final_list = sanitized_list[:nb_assets] if nb_assets else sanitized_list
|
||||
self.config['exchange']['pair_whitelist'] = final_list
|
||||
self.active_pair_whitelist = sanitized_list[:nb_assets] if nb_assets else sanitized_list
|
||||
|
||||
# Calculating Edge positiong
|
||||
# Should be called before refresh_tickers
|
||||
|
|
@ -197,11 +194,20 @@ class FreqtradeBot(object):
|
|||
self.edge.calculate()
|
||||
|
||||
# Refreshing candles
|
||||
self.exchange.refresh_tickers(final_list, self.strategy.ticker_interval)
|
||||
self.exchange.refresh_tickers(self.active_pair_whitelist, self.strategy.ticker_interval)
|
||||
|
||||
|
||||
# Query trades from persistence layer
|
||||
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
|
||||
|
||||
# Extend active-pair whitelist with pairs from open trades
|
||||
# ensures that tickers are downloaded for open trades
|
||||
self.active_pair_whitelist.extend([trade.pair for trade in trades
|
||||
if trade.pair not in self.active_pair_whitelist])
|
||||
|
||||
# Refreshing candles
|
||||
self.exchange.refresh_tickers(self.active_pair_whitelist, self.strategy.ticker_interval)
|
||||
|
||||
# First process current opened trades
|
||||
for trade in trades:
|
||||
state_changed |= self.process_maybe_execute_sell(trade)
|
||||
|
|
@ -389,7 +395,7 @@ class FreqtradeBot(object):
|
|||
:return: True if a trade object has been created and persisted, False otherwise
|
||||
"""
|
||||
interval = self.strategy.ticker_interval
|
||||
whitelist = copy.deepcopy(self.config['exchange']['pair_whitelist'])
|
||||
whitelist = copy.deepcopy(self.active_pair_whitelist)
|
||||
|
||||
# Remove currently opened and latest pairs from whitelist
|
||||
for trade in Trade.query.filter(Trade.is_open.is_(True)).all():
|
||||
|
|
|
|||
|
|
@ -10,8 +10,12 @@ except ImportError:
|
|||
_UJSON = False
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
from typing import Optional, List, Dict, Tuple, Any
|
||||
import operator
|
||||
|
||||
import arrow
|
||||
from pandas import DataFrame
|
||||
|
||||
from freqtrade import misc, constants, OperationalException
|
||||
from freqtrade.exchange import Exchange
|
||||
|
|
@ -59,6 +63,42 @@ def trim_tickerlist(tickerlist: List[Dict], timerange: TimeRange) -> List[Dict]:
|
|||
return tickerlist[start_index:stop_index]
|
||||
|
||||
|
||||
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
|
||||
"""
|
||||
Get the maximum timeframe for the given backtest data
|
||||
:param data: dictionary with preprocessed backtesting data
|
||||
:return: tuple containing min_date, max_date
|
||||
"""
|
||||
timeframe = [
|
||||
(arrow.get(frame['date'].min()), arrow.get(frame['date'].max()))
|
||||
for frame in data.values()
|
||||
]
|
||||
return min(timeframe, key=operator.itemgetter(0))[0], \
|
||||
max(timeframe, key=operator.itemgetter(1))[1]
|
||||
|
||||
|
||||
def validate_backtest_data(data: Dict[str, DataFrame], min_date: datetime,
|
||||
max_date: datetime, ticker_interval_mins: int) -> bool:
|
||||
"""
|
||||
Validates preprocessed backtesting data for missing values and shows warnings about it that.
|
||||
|
||||
:param data: dictionary with preprocessed backtesting data
|
||||
:param min_date: start-date of the data
|
||||
:param max_date: end-date of the data
|
||||
:param ticker_interval_mins: ticker interval in minutes
|
||||
"""
|
||||
# total difference in minutes / interval-minutes
|
||||
expected_frames = int((max_date - min_date).total_seconds() // 60 // ticker_interval_mins)
|
||||
found_missing = False
|
||||
for pair, df in data.items():
|
||||
dflen = len(df)
|
||||
if dflen < expected_frames:
|
||||
found_missing = True
|
||||
logger.warning("%s has missing frames: expected %s, got %s, that's %s missing values",
|
||||
pair, expected_frames, dflen, expected_frames - dflen)
|
||||
return found_missing
|
||||
|
||||
|
||||
def load_tickerdata_file(
|
||||
datadir: str, pair: str,
|
||||
ticker_interval: str,
|
||||
|
|
@ -113,6 +153,14 @@ def load_data(datadir: str,
|
|||
for pair in pairs:
|
||||
pairdata = load_tickerdata_file(datadir, pair, ticker_interval, timerange=timerange)
|
||||
if pairdata:
|
||||
if timerange.starttype == 'date' and pairdata[0][0] > timerange.startts * 1000:
|
||||
logger.warning('Missing data at start for pair %s, data starts at %s',
|
||||
pair,
|
||||
arrow.get(pairdata[0][0] // 1000).strftime('%Y-%m-%d %H:%M:%S'))
|
||||
if timerange.stoptype == 'date' and pairdata[-1][0] < timerange.stopts * 1000:
|
||||
logger.warning('Missing data at end for pair %s, data ends at %s',
|
||||
pair,
|
||||
arrow.get(pairdata[-1][0] // 1000).strftime('%Y-%m-%d %H:%M:%S'))
|
||||
result[pair] = pairdata
|
||||
else:
|
||||
logger.warning(
|
||||
|
|
|
|||
|
|
@ -4,14 +4,12 @@
|
|||
This module contains the backtesting logic
|
||||
"""
|
||||
import logging
|
||||
import operator
|
||||
from argparse import Namespace
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
||||
from typing import Any, Dict, List, NamedTuple, Optional
|
||||
|
||||
import arrow
|
||||
from pandas import DataFrame
|
||||
from tabulate import tabulate
|
||||
|
||||
|
|
@ -88,24 +86,9 @@ class Backtesting(object):
|
|||
"""
|
||||
self.strategy = strategy
|
||||
self.ticker_interval = self.config.get('ticker_interval')
|
||||
self.tickerdata_to_dataframe = strategy.tickerdata_to_dataframe
|
||||
self.advise_buy = strategy.advise_buy
|
||||
self.advise_sell = strategy.advise_sell
|
||||
|
||||
@staticmethod
|
||||
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
|
||||
"""
|
||||
Get the maximum timeframe for the given backtest data
|
||||
:param data: dictionary with preprocessed backtesting data
|
||||
:return: tuple containing min_date, max_date
|
||||
"""
|
||||
timeframe = [
|
||||
(arrow.get(frame['date'].min()), arrow.get(frame['date'].max()))
|
||||
for frame in data.values()
|
||||
]
|
||||
return min(timeframe, key=operator.itemgetter(0))[0], \
|
||||
max(timeframe, key=operator.itemgetter(1))[1]
|
||||
|
||||
def _generate_text_table(self, data: Dict[str, Dict], results: DataFrame,
|
||||
skip_nan: bool = False) -> str:
|
||||
"""
|
||||
|
|
@ -371,10 +354,12 @@ class Backtesting(object):
|
|||
self._set_strategy(strat)
|
||||
|
||||
# need to reprocess data every time to populate signals
|
||||
preprocessed = self.tickerdata_to_dataframe(data)
|
||||
preprocessed = self.strategy.tickerdata_to_dataframe(data)
|
||||
|
||||
# Print timeframe
|
||||
min_date, max_date = self.get_timeframe(preprocessed)
|
||||
min_date, max_date = optimize.get_timeframe(preprocessed)
|
||||
# Validate dataframe for missing values
|
||||
optimize.validate_backtest_data(preprocessed, min_date, max_date,
|
||||
constants.TICKER_INTERVAL_MINUTES[self.ticker_interval])
|
||||
logger.info(
|
||||
'Measuring data from %s up to %s (%s days)..',
|
||||
min_date.isoformat(),
|
||||
|
|
|
|||
|
|
@ -352,7 +352,7 @@ class Hyperopt(Backtesting):
|
|||
|
||||
if self.has_space('buy'):
|
||||
self.strategy.advise_indicators = Hyperopt.populate_indicators # type: ignore
|
||||
dump(self.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
|
||||
dump(self.strategy.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
|
||||
self.exchange = None # type: ignore
|
||||
self.load_previous_results()
|
||||
|
||||
|
|
|
|||
|
|
@ -10,10 +10,10 @@ from typing import Dict, Any, List, Optional
|
|||
|
||||
import arrow
|
||||
import sqlalchemy as sql
|
||||
from numpy import mean, nan_to_num
|
||||
from numpy import mean, nan_to_num, NAN
|
||||
from pandas import DataFrame
|
||||
|
||||
from freqtrade import TemporaryError
|
||||
from freqtrade import TemporaryError, DependencyException
|
||||
from freqtrade.fiat_convert import CryptoToFiatConverter
|
||||
from freqtrade.misc import shorten_date
|
||||
from freqtrade.persistence import Trade
|
||||
|
|
@ -93,7 +93,10 @@ class RPC(object):
|
|||
if trade.open_order_id:
|
||||
order = self._freqtrade.exchange.get_order(trade.open_order_id, trade.pair)
|
||||
# calculate profit and send message to user
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
try:
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
except DependencyException:
|
||||
current_rate = NAN
|
||||
current_profit = trade.calc_profit_percent(current_rate)
|
||||
fmt_close_profit = (f'{round(trade.close_profit * 100, 2):.2f}%'
|
||||
if trade.close_profit else None)
|
||||
|
|
@ -122,7 +125,10 @@ class RPC(object):
|
|||
trades_list = []
|
||||
for trade in trades:
|
||||
# calculate profit and send message to user
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
try:
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
except DependencyException:
|
||||
current_rate = NAN
|
||||
trade_perc = (100 * trade.calc_profit_percent(current_rate))
|
||||
trades_list.append([
|
||||
trade.id,
|
||||
|
|
@ -207,7 +213,10 @@ class RPC(object):
|
|||
profit_closed_percent.append(profit_percent)
|
||||
else:
|
||||
# Get current rate
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
try:
|
||||
current_rate = self._freqtrade.exchange.get_ticker(trade.pair, False)['bid']
|
||||
except DependencyException:
|
||||
current_rate = NAN
|
||||
profit_percent = trade.calc_profit_percent(rate=current_rate)
|
||||
|
||||
profit_all_coin.append(
|
||||
|
|
@ -275,7 +284,7 @@ class RPC(object):
|
|||
rate = 1.0 / self._freqtrade.exchange.get_ticker('BTC/USDT', False)['bid']
|
||||
else:
|
||||
rate = self._freqtrade.exchange.get_ticker(coin + '/BTC', False)['bid']
|
||||
except TemporaryError:
|
||||
except (TemporaryError, DependencyException):
|
||||
continue
|
||||
est_btc: float = rate * balance['total']
|
||||
total = total + est_btc
|
||||
|
|
|
|||
|
|
@ -572,6 +572,7 @@ def test_get_ticker(default_conf, mocker):
|
|||
'last': 0.0001,
|
||||
}
|
||||
api_mock.fetch_ticker = MagicMock(return_value=tick)
|
||||
api_mock.markets = {'ETH/BTC': {}}
|
||||
exchange = get_patched_exchange(mocker, default_conf, api_mock)
|
||||
# retrieve original ticker
|
||||
ticker = exchange.get_ticker(pair='ETH/BTC')
|
||||
|
|
@ -614,6 +615,9 @@ def test_get_ticker(default_conf, mocker):
|
|||
exchange = get_patched_exchange(mocker, default_conf, api_mock)
|
||||
exchange.get_ticker(pair='ETH/BTC', refresh=True)
|
||||
|
||||
with pytest.raises(DependencyException, match=r'Pair XRP/ETH not available'):
|
||||
exchange.get_ticker(pair='XRP/ETH', refresh=True)
|
||||
|
||||
|
||||
def test_get_history(default_conf, mocker, caplog):
|
||||
exchange = get_patched_exchange(mocker, default_conf)
|
||||
|
|
|
|||
|
|
@ -89,7 +89,7 @@ def simple_backtest(config, contour, num_results, mocker) -> None:
|
|||
backtesting = Backtesting(config)
|
||||
|
||||
data = load_data_test(contour)
|
||||
processed = backtesting.tickerdata_to_dataframe(data)
|
||||
processed = backtesting.strategy.tickerdata_to_dataframe(data)
|
||||
assert isinstance(processed, dict)
|
||||
results = backtesting.backtest(
|
||||
{
|
||||
|
|
@ -119,13 +119,13 @@ def _load_pair_as_ticks(pair, tickfreq):
|
|||
|
||||
# FIX: fixturize this?
|
||||
def _make_backtest_conf(mocker, conf=None, pair='UNITTEST/BTC', record=None):
|
||||
data = optimize.load_data(None, ticker_interval='8m', pairs=[pair])
|
||||
data = optimize.load_data(None, ticker_interval='1m', pairs=[pair])
|
||||
data = trim_dictlist(data, -201)
|
||||
patch_exchange(mocker)
|
||||
backtesting = Backtesting(conf)
|
||||
return {
|
||||
'stake_amount': conf['stake_amount'],
|
||||
'processed': backtesting.tickerdata_to_dataframe(data),
|
||||
'processed': backtesting.strategy.tickerdata_to_dataframe(data),
|
||||
'max_open_trades': 10,
|
||||
'position_stacking': False,
|
||||
'record': record
|
||||
|
|
@ -313,7 +313,7 @@ def test_backtesting_init(mocker, default_conf) -> None:
|
|||
backtesting = Backtesting(default_conf)
|
||||
assert backtesting.config == default_conf
|
||||
assert backtesting.ticker_interval == '5m'
|
||||
assert callable(backtesting.tickerdata_to_dataframe)
|
||||
assert callable(backtesting.strategy.tickerdata_to_dataframe)
|
||||
assert callable(backtesting.advise_buy)
|
||||
assert callable(backtesting.advise_sell)
|
||||
get_fee.assert_called()
|
||||
|
|
@ -327,7 +327,7 @@ def test_tickerdata_to_dataframe(default_conf, mocker) -> None:
|
|||
tickerlist = {'UNITTEST/BTC': tick}
|
||||
|
||||
backtesting = Backtesting(default_conf)
|
||||
data = backtesting.tickerdata_to_dataframe(tickerlist)
|
||||
data = backtesting.strategy.tickerdata_to_dataframe(tickerlist)
|
||||
assert len(data['UNITTEST/BTC']) == 99
|
||||
|
||||
# Load strategy to compare the result between Backtesting function and strategy are the same
|
||||
|
|
@ -336,22 +336,6 @@ def test_tickerdata_to_dataframe(default_conf, mocker) -> None:
|
|||
assert data['UNITTEST/BTC'].equals(data2['UNITTEST/BTC'])
|
||||
|
||||
|
||||
def test_get_timeframe(default_conf, mocker) -> None:
|
||||
patch_exchange(mocker)
|
||||
backtesting = Backtesting(default_conf)
|
||||
|
||||
data = backtesting.tickerdata_to_dataframe(
|
||||
optimize.load_data(
|
||||
None,
|
||||
ticker_interval='1m',
|
||||
pairs=['UNITTEST/BTC']
|
||||
)
|
||||
)
|
||||
min_date, max_date = backtesting.get_timeframe(data)
|
||||
assert min_date.isoformat() == '2017-11-04T23:02:00+00:00'
|
||||
assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
|
||||
|
||||
|
||||
def test_generate_text_table(default_conf, mocker):
|
||||
patch_exchange(mocker)
|
||||
backtesting = Backtesting(default_conf)
|
||||
|
|
@ -451,21 +435,21 @@ def test_generate_text_table_strategyn(default_conf, mocker):
|
|||
|
||||
|
||||
def test_backtesting_start(default_conf, mocker, caplog) -> None:
|
||||
def get_timeframe(input1, input2):
|
||||
def get_timeframe(input1):
|
||||
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
|
||||
|
||||
mocker.patch('freqtrade.optimize.load_data', mocked_load_data)
|
||||
mocker.patch('freqtrade.optimize.get_timeframe', get_timeframe)
|
||||
mocker.patch('freqtrade.exchange.Exchange.refresh_tickers', MagicMock())
|
||||
patch_exchange(mocker)
|
||||
mocker.patch.multiple(
|
||||
'freqtrade.optimize.backtesting.Backtesting',
|
||||
backtest=MagicMock(),
|
||||
_generate_text_table=MagicMock(return_value='1'),
|
||||
get_timeframe=get_timeframe,
|
||||
)
|
||||
|
||||
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
|
||||
default_conf['ticker_interval'] = 1
|
||||
default_conf['ticker_interval'] = "1m"
|
||||
default_conf['live'] = False
|
||||
default_conf['datadir'] = None
|
||||
default_conf['export'] = None
|
||||
|
|
@ -486,17 +470,17 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
|
|||
|
||||
|
||||
def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
|
||||
def get_timeframe(input1, input2):
|
||||
def get_timeframe(input1):
|
||||
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
|
||||
|
||||
mocker.patch('freqtrade.optimize.load_data', MagicMock(return_value={}))
|
||||
mocker.patch('freqtrade.optimize.get_timeframe', get_timeframe)
|
||||
mocker.patch('freqtrade.exchange.Exchange.refresh_tickers', MagicMock())
|
||||
patch_exchange(mocker)
|
||||
mocker.patch.multiple(
|
||||
'freqtrade.optimize.backtesting.Backtesting',
|
||||
backtest=MagicMock(),
|
||||
_generate_text_table=MagicMock(return_value='1'),
|
||||
get_timeframe=get_timeframe,
|
||||
)
|
||||
|
||||
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
|
||||
|
|
@ -520,7 +504,7 @@ def test_backtest(default_conf, fee, mocker) -> None:
|
|||
pair = 'UNITTEST/BTC'
|
||||
data = optimize.load_data(None, ticker_interval='5m', pairs=['UNITTEST/BTC'])
|
||||
data = trim_dictlist(data, -200)
|
||||
data_processed = backtesting.tickerdata_to_dataframe(data)
|
||||
data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
|
||||
results = backtesting.backtest(
|
||||
{
|
||||
'stake_amount': default_conf['stake_amount'],
|
||||
|
|
@ -571,7 +555,7 @@ def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None:
|
|||
results = backtesting.backtest(
|
||||
{
|
||||
'stake_amount': default_conf['stake_amount'],
|
||||
'processed': backtesting.tickerdata_to_dataframe(data),
|
||||
'processed': backtesting.strategy.tickerdata_to_dataframe(data),
|
||||
'max_open_trades': 1,
|
||||
'position_stacking': False
|
||||
}
|
||||
|
|
@ -585,7 +569,7 @@ def test_processed(default_conf, mocker) -> None:
|
|||
backtesting = Backtesting(default_conf)
|
||||
|
||||
dict_of_tickerrows = load_data_test('raise')
|
||||
dataframes = backtesting.tickerdata_to_dataframe(dict_of_tickerrows)
|
||||
dataframes = backtesting.strategy.tickerdata_to_dataframe(dict_of_tickerrows)
|
||||
dataframe = dataframes['UNITTEST/BTC']
|
||||
cols = dataframe.columns
|
||||
# assert the dataframe got some of the indicator columns
|
||||
|
|
|
|||
|
|
@ -194,7 +194,7 @@ def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
|
|||
default_conf.update({'spaces': 'all'})
|
||||
|
||||
hyperopt = Hyperopt(default_conf)
|
||||
hyperopt.tickerdata_to_dataframe = MagicMock()
|
||||
hyperopt.strategy.tickerdata_to_dataframe = MagicMock()
|
||||
|
||||
hyperopt.start()
|
||||
parallel.assert_called_once()
|
||||
|
|
@ -242,7 +242,7 @@ def test_has_space(hyperopt):
|
|||
def test_populate_indicators(hyperopt) -> None:
|
||||
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
|
||||
tickerlist = {'UNITTEST/BTC': tick}
|
||||
dataframes = hyperopt.tickerdata_to_dataframe(tickerlist)
|
||||
dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
|
||||
dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
|
||||
|
||||
# Check if some indicators are generated. We will not test all of them
|
||||
|
|
@ -254,7 +254,7 @@ def test_populate_indicators(hyperopt) -> None:
|
|||
def test_buy_strategy_generator(hyperopt) -> None:
|
||||
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
|
||||
tickerlist = {'UNITTEST/BTC': tick}
|
||||
dataframes = hyperopt.tickerdata_to_dataframe(tickerlist)
|
||||
dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
|
||||
dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
|
||||
|
||||
populate_buy_trend = hyperopt.buy_strategy_generator(
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@ from shutil import copyfile
|
|||
|
||||
import arrow
|
||||
|
||||
from freqtrade import optimize
|
||||
from freqtrade import optimize, constants
|
||||
from freqtrade.arguments import TimeRange
|
||||
from freqtrade.misc import file_dump_json
|
||||
from freqtrade.optimize.__init__ import (download_backtesting_testdata,
|
||||
|
|
@ -15,7 +15,8 @@ from freqtrade.optimize.__init__ import (download_backtesting_testdata,
|
|||
load_cached_data_for_updating,
|
||||
load_tickerdata_file,
|
||||
make_testdata_path, trim_tickerlist)
|
||||
from freqtrade.tests.conftest import get_patched_exchange, log_has
|
||||
from freqtrade.strategy.default_strategy import DefaultStrategy
|
||||
from freqtrade.tests.conftest import get_patched_exchange, log_has, patch_exchange
|
||||
|
||||
# Change this if modifying UNITTEST/BTC testdatafile
|
||||
_BTC_UNITTEST_LENGTH = 13681
|
||||
|
|
@ -322,6 +323,38 @@ def test_load_tickerdata_file() -> None:
|
|||
assert _BTC_UNITTEST_LENGTH == len(tickerdata)
|
||||
|
||||
|
||||
def test_load_partial_missing(caplog) -> None:
|
||||
# Make sure we start fresh - test missing data at start
|
||||
start = arrow.get('2018-01-01T00:00:00')
|
||||
end = arrow.get('2018-01-11T00:00:00')
|
||||
tickerdata = optimize.load_data(None, '5m', ['UNITTEST/BTC'],
|
||||
refresh_pairs=False,
|
||||
timerange=TimeRange('date', 'date',
|
||||
start.timestamp, end.timestamp))
|
||||
# timedifference in 5 minutes
|
||||
td = ((end - start).total_seconds() // 60 // 5) + 1
|
||||
assert td != len(tickerdata['UNITTEST/BTC'])
|
||||
start_real = arrow.get(tickerdata['UNITTEST/BTC'][0][0] / 1000)
|
||||
assert log_has(f'Missing data at start for pair '
|
||||
f'UNITTEST/BTC, data starts at {start_real.strftime("%Y-%m-%d %H:%M:%S")}',
|
||||
caplog.record_tuples)
|
||||
# Make sure we start fresh - test missing data at end
|
||||
caplog.clear()
|
||||
start = arrow.get('2018-01-10T00:00:00')
|
||||
end = arrow.get('2018-02-20T00:00:00')
|
||||
tickerdata = optimize.load_data(None, '5m', ['UNITTEST/BTC'],
|
||||
refresh_pairs=False,
|
||||
timerange=TimeRange('date', 'date',
|
||||
start.timestamp, end.timestamp))
|
||||
# timedifference in 5 minutes
|
||||
td = ((end - start).total_seconds() // 60 // 5) + 1
|
||||
assert td != len(tickerdata['UNITTEST/BTC'])
|
||||
end_real = arrow.get(tickerdata['UNITTEST/BTC'][-1][0] / 1000)
|
||||
assert log_has(f'Missing data at end for pair '
|
||||
f'UNITTEST/BTC, data ends at {end_real.strftime("%Y-%m-%d %H:%M:%S")}',
|
||||
caplog.record_tuples)
|
||||
|
||||
|
||||
def test_init(default_conf, mocker) -> None:
|
||||
exchange = get_patched_exchange(mocker, default_conf)
|
||||
assert {} == optimize.load_data(
|
||||
|
|
@ -433,3 +466,61 @@ def test_file_dump_json() -> None:
|
|||
|
||||
# Remove the file
|
||||
_clean_test_file(file)
|
||||
|
||||
|
||||
def test_get_timeframe(default_conf, mocker) -> None:
|
||||
patch_exchange(mocker)
|
||||
strategy = DefaultStrategy(default_conf)
|
||||
|
||||
data = strategy.tickerdata_to_dataframe(
|
||||
optimize.load_data(
|
||||
None,
|
||||
ticker_interval='1m',
|
||||
pairs=['UNITTEST/BTC']
|
||||
)
|
||||
)
|
||||
min_date, max_date = optimize.get_timeframe(data)
|
||||
assert min_date.isoformat() == '2017-11-04T23:02:00+00:00'
|
||||
assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
|
||||
|
||||
|
||||
def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None:
|
||||
patch_exchange(mocker)
|
||||
strategy = DefaultStrategy(default_conf)
|
||||
|
||||
data = strategy.tickerdata_to_dataframe(
|
||||
optimize.load_data(
|
||||
None,
|
||||
ticker_interval='1m',
|
||||
pairs=['UNITTEST/BTC']
|
||||
)
|
||||
)
|
||||
min_date, max_date = optimize.get_timeframe(data)
|
||||
caplog.clear()
|
||||
assert optimize.validate_backtest_data(data, min_date, max_date,
|
||||
constants.TICKER_INTERVAL_MINUTES["1m"])
|
||||
assert len(caplog.record_tuples) == 1
|
||||
assert log_has(
|
||||
"UNITTEST/BTC has missing frames: expected 14396, got 13680, that's 716 missing values",
|
||||
caplog.record_tuples)
|
||||
|
||||
|
||||
def test_validate_backtest_data(default_conf, mocker, caplog) -> None:
|
||||
patch_exchange(mocker)
|
||||
strategy = DefaultStrategy(default_conf)
|
||||
|
||||
timerange = TimeRange('index', 'index', 200, 250)
|
||||
data = strategy.tickerdata_to_dataframe(
|
||||
optimize.load_data(
|
||||
None,
|
||||
ticker_interval='5m',
|
||||
pairs=['UNITTEST/BTC'],
|
||||
timerange=timerange
|
||||
)
|
||||
)
|
||||
|
||||
min_date, max_date = optimize.get_timeframe(data)
|
||||
caplog.clear()
|
||||
assert not optimize.validate_backtest_data(data, min_date, max_date,
|
||||
constants.TICKER_INTERVAL_MINUTES["5m"])
|
||||
assert len(caplog.record_tuples) == 0
|
||||
|
|
|
|||
|
|
@ -5,8 +5,9 @@ from datetime import datetime
|
|||
from unittest.mock import MagicMock, ANY
|
||||
|
||||
import pytest
|
||||
from numpy import isnan
|
||||
|
||||
from freqtrade import TemporaryError
|
||||
from freqtrade import TemporaryError, DependencyException
|
||||
from freqtrade.fiat_convert import CryptoToFiatConverter
|
||||
from freqtrade.freqtradebot import FreqtradeBot
|
||||
from freqtrade.persistence import Trade
|
||||
|
|
@ -61,6 +62,27 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
|
|||
'open_order': '(limit buy rem=0.00000000)'
|
||||
} == results[0]
|
||||
|
||||
mocker.patch('freqtrade.exchange.Exchange.get_ticker',
|
||||
MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
|
||||
# invalidate ticker cache
|
||||
rpc._freqtrade.exchange._cached_ticker = {}
|
||||
results = rpc._rpc_trade_status()
|
||||
assert isnan(results[0]['current_profit'])
|
||||
assert isnan(results[0]['current_rate'])
|
||||
assert {
|
||||
'trade_id': 1,
|
||||
'pair': 'ETH/BTC',
|
||||
'market_url': 'https://bittrex.com/Market/Index?MarketName=BTC-ETH',
|
||||
'date': ANY,
|
||||
'open_rate': 1.099e-05,
|
||||
'close_rate': None,
|
||||
'current_rate': ANY,
|
||||
'amount': 90.99181074,
|
||||
'close_profit': None,
|
||||
'current_profit': ANY,
|
||||
'open_order': '(limit buy rem=0.00000000)'
|
||||
} == results[0]
|
||||
|
||||
|
||||
def test_rpc_status_table(default_conf, ticker, fee, markets, mocker) -> None:
|
||||
patch_coinmarketcap(mocker)
|
||||
|
|
@ -87,6 +109,15 @@ def test_rpc_status_table(default_conf, ticker, fee, markets, mocker) -> None:
|
|||
assert 'ETH/BTC' in result['Pair'].all()
|
||||
assert '-0.59%' in result['Profit'].all()
|
||||
|
||||
mocker.patch('freqtrade.exchange.Exchange.get_ticker',
|
||||
MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
|
||||
# invalidate ticker cache
|
||||
rpc._freqtrade.exchange._cached_ticker = {}
|
||||
result = rpc._rpc_status_table()
|
||||
assert 'just now' in result['Since'].all()
|
||||
assert 'ETH/BTC' in result['Pair'].all()
|
||||
assert 'nan%' in result['Profit'].all()
|
||||
|
||||
|
||||
def test_rpc_daily_profit(default_conf, update, ticker, fee,
|
||||
limit_buy_order, limit_sell_order, markets, mocker) -> None:
|
||||
|
|
@ -208,6 +239,20 @@ def test_rpc_trade_statistics(default_conf, ticker, ticker_sell_up, fee,
|
|||
assert stats['best_pair'] == 'ETH/BTC'
|
||||
assert prec_satoshi(stats['best_rate'], 6.2)
|
||||
|
||||
# Test non-available pair
|
||||
mocker.patch('freqtrade.exchange.Exchange.get_ticker',
|
||||
MagicMock(side_effect=DependencyException(f"Pair 'ETH/BTC' not available")))
|
||||
# invalidate ticker cache
|
||||
rpc._freqtrade.exchange._cached_ticker = {}
|
||||
stats = rpc._rpc_trade_statistics(stake_currency, fiat_display_currency)
|
||||
assert stats['trade_count'] == 2
|
||||
assert stats['first_trade_date'] == 'just now'
|
||||
assert stats['latest_trade_date'] == 'just now'
|
||||
assert stats['avg_duration'] == '0:00:00'
|
||||
assert stats['best_pair'] == 'ETH/BTC'
|
||||
assert prec_satoshi(stats['best_rate'], 6.2)
|
||||
assert isnan(stats['profit_all_coin'])
|
||||
|
||||
|
||||
# Test that rpc_trade_statistics can handle trades that lacks
|
||||
# trade.open_rate (it is set to None)
|
||||
|
|
|
|||
|
|
@ -764,6 +764,52 @@ def test_process_trade_handling(
|
|||
assert result is False
|
||||
|
||||
|
||||
def test_process_trade_no_whitelist_pair(
|
||||
default_conf, ticker, limit_buy_order, markets, fee, mocker) -> None:
|
||||
""" Test _process with trade not in pair list """
|
||||
patch_RPCManager(mocker)
|
||||
patch_exchange(mocker)
|
||||
mocker.patch.multiple(
|
||||
'freqtrade.exchange.Exchange',
|
||||
get_ticker=ticker,
|
||||
get_markets=markets,
|
||||
buy=MagicMock(return_value={'id': limit_buy_order['id']}),
|
||||
get_order=MagicMock(return_value=limit_buy_order),
|
||||
get_fee=fee,
|
||||
)
|
||||
freqtrade = FreqtradeBot(default_conf)
|
||||
patch_get_signal(freqtrade)
|
||||
pair = 'NOCLUE/BTC'
|
||||
# create open trade not in whitelist
|
||||
Trade.session.add(Trade(
|
||||
pair=pair,
|
||||
stake_amount=0.001,
|
||||
fee_open=fee.return_value,
|
||||
fee_close=fee.return_value,
|
||||
is_open=True,
|
||||
amount=20,
|
||||
open_rate=0.01,
|
||||
exchange='bittrex',
|
||||
))
|
||||
Trade.session.add(Trade(
|
||||
pair='ETH/BTC',
|
||||
stake_amount=0.001,
|
||||
fee_open=fee.return_value,
|
||||
fee_close=fee.return_value,
|
||||
is_open=True,
|
||||
amount=12,
|
||||
open_rate=0.001,
|
||||
exchange='bittrex',
|
||||
))
|
||||
|
||||
assert pair not in freqtrade.active_pair_whitelist
|
||||
result = freqtrade._process()
|
||||
assert pair in freqtrade.active_pair_whitelist
|
||||
# Make sure each pair is only in the list once
|
||||
assert len(freqtrade.active_pair_whitelist) == len(set(freqtrade.active_pair_whitelist))
|
||||
assert result is True
|
||||
|
||||
|
||||
def test_balance_fully_ask_side(mocker, default_conf) -> None:
|
||||
default_conf['bid_strategy']['ask_last_balance'] = 0.0
|
||||
freqtrade = get_patched_freqtradebot(mocker, default_conf)
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
ccxt==1.17.411
|
||||
SQLAlchemy==1.2.12
|
||||
ccxt==1.17.455
|
||||
SQLAlchemy==1.2.13
|
||||
python-telegram-bot==11.1.0
|
||||
arrow==0.12.1
|
||||
cachetools==2.1.0
|
||||
|
|
@ -12,7 +12,7 @@ scipy==1.1.0
|
|||
jsonschema==2.6.0
|
||||
numpy==1.15.3
|
||||
TA-Lib==0.4.17
|
||||
pytest==3.9.2
|
||||
pytest==3.9.3
|
||||
pytest-mock==1.10.0
|
||||
pytest-asyncio==0.9.0
|
||||
pytest-cov==2.6.0
|
||||
|
|
|
|||
|
|
@ -1,381 +0,0 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to display when the bot will buy a specific pair
|
||||
|
||||
Mandatory Cli parameters:
|
||||
-p / --pair: pair to examine
|
||||
|
||||
Option but recommended
|
||||
-s / --strategy: strategy to use
|
||||
|
||||
|
||||
Optional Cli parameters
|
||||
-d / --datadir: path to pair backtest data
|
||||
--timerange: specify what timerange of data to use.
|
||||
-l / --live: Live, to download the latest ticker for the pair
|
||||
-db / --db-url: Show trades stored in database
|
||||
|
||||
|
||||
Indicators recommended
|
||||
Row 1: sma, ema3, ema5, ema10, ema50
|
||||
Row 3: macd, rsi, fisher_rsi, mfi, slowd, slowk, fastd, fastk
|
||||
|
||||
Example of usage:
|
||||
> python3 scripts/plot_dataframe.py --pair BTC/EUR -d user_data/data/ --indicators1 sma,ema3
|
||||
--indicators2 fastk,fastd
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
from argparse import Namespace
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Any
|
||||
|
||||
import pandas as pd
|
||||
import plotly.graph_objs as go
|
||||
import pytz
|
||||
|
||||
from plotly import tools
|
||||
from plotly.offline import plot
|
||||
|
||||
import freqtrade.optimize as optimize
|
||||
from freqtrade import persistence
|
||||
from freqtrade.arguments import Arguments, TimeRange
|
||||
from freqtrade.exchange import Exchange
|
||||
from freqtrade.optimize.backtesting import setup_configuration
|
||||
from freqtrade.persistence import Trade
|
||||
from freqtrade.strategy.resolver import StrategyResolver
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
_CONF: Dict[str, Any] = {}
|
||||
|
||||
timeZone = pytz.UTC
|
||||
|
||||
|
||||
def load_trades(args: Namespace, pair: str, timerange: TimeRange) -> pd.DataFrame:
|
||||
trades: pd.DataFrame = pd.DataFrame()
|
||||
if args.db_url:
|
||||
persistence.init(_CONF)
|
||||
columns = ["pair", "profit", "opents", "closets", "open_rate", "close_rate", "duration"]
|
||||
|
||||
for x in Trade.query.all():
|
||||
print("date: {}".format(x.open_date))
|
||||
|
||||
trades = pd.DataFrame([(t.pair, t.calc_profit(),
|
||||
t.open_date.replace(tzinfo=timeZone),
|
||||
t.close_date.replace(tzinfo=timeZone) if t.close_date else None,
|
||||
t.open_rate, t.close_rate,
|
||||
t.close_date.timestamp() - t.open_date.timestamp() if t.close_date else None)
|
||||
for t in Trade.query.filter(Trade.pair.is_(pair)).all()],
|
||||
columns=columns)
|
||||
|
||||
elif args.exportfilename:
|
||||
file = Path(args.exportfilename)
|
||||
# must align with columns in backtest.py
|
||||
columns = ["pair", "profit", "opents", "closets", "index", "duration",
|
||||
"open_rate", "close_rate", "open_at_end", "sell_reason"]
|
||||
with file.open() as f:
|
||||
data = json.load(f)
|
||||
trades = pd.DataFrame(data, columns=columns)
|
||||
trades = trades.loc[trades["pair"] == pair]
|
||||
if timerange:
|
||||
if timerange.starttype == 'date':
|
||||
trades = trades.loc[trades["opents"] >= timerange.startts]
|
||||
if timerange.stoptype == 'date':
|
||||
trades = trades.loc[trades["opents"] <= timerange.stopts]
|
||||
|
||||
trades['opents'] = pd.to_datetime(trades['opents'],
|
||||
unit='s',
|
||||
utc=True,
|
||||
infer_datetime_format=True)
|
||||
trades['closets'] = pd.to_datetime(trades['closets'],
|
||||
unit='s',
|
||||
utc=True,
|
||||
infer_datetime_format=True)
|
||||
return trades
|
||||
|
||||
|
||||
def plot_analyzed_dataframe(args: Namespace) -> None:
|
||||
"""
|
||||
Calls analyze() and plots the returned dataframe
|
||||
:return: None
|
||||
"""
|
||||
global _CONF
|
||||
|
||||
# Load the configuration
|
||||
_CONF.update(setup_configuration(args))
|
||||
|
||||
print(_CONF)
|
||||
# Set the pair to audit
|
||||
pair = args.pair
|
||||
|
||||
if pair is None:
|
||||
logger.critical('Parameter --pair mandatory;. E.g --pair ETH/BTC')
|
||||
exit()
|
||||
|
||||
if '/' not in pair:
|
||||
logger.critical('--pair format must be XXX/YYY')
|
||||
exit()
|
||||
|
||||
# Set timerange to use
|
||||
timerange = Arguments.parse_timerange(args.timerange)
|
||||
|
||||
# Load the strategy
|
||||
try:
|
||||
strategy = StrategyResolver(_CONF).strategy
|
||||
exchange = Exchange(_CONF)
|
||||
except AttributeError:
|
||||
logger.critical(
|
||||
'Impossible to load the strategy. Please check the file "user_data/strategies/%s.py"',
|
||||
args.strategy
|
||||
)
|
||||
exit()
|
||||
|
||||
# Set the ticker to use
|
||||
tick_interval = strategy.ticker_interval
|
||||
|
||||
# Load pair tickers
|
||||
tickers = {}
|
||||
if args.live:
|
||||
logger.info('Downloading pair.')
|
||||
exchange.refresh_tickers([pair], tick_interval)
|
||||
tickers[pair] = exchange.klines[pair]
|
||||
else:
|
||||
tickers = optimize.load_data(
|
||||
datadir=_CONF.get("datadir"),
|
||||
pairs=[pair],
|
||||
ticker_interval=tick_interval,
|
||||
refresh_pairs=_CONF.get('refresh_pairs', False),
|
||||
timerange=timerange,
|
||||
exchange=Exchange(_CONF)
|
||||
)
|
||||
|
||||
# No ticker found, or impossible to download
|
||||
if tickers == {}:
|
||||
exit()
|
||||
|
||||
# Get trades already made from the DB
|
||||
trades = load_trades(args, pair, timerange)
|
||||
|
||||
dataframes = strategy.tickerdata_to_dataframe(tickers)
|
||||
|
||||
dataframe = dataframes[pair]
|
||||
dataframe = strategy.advise_buy(dataframe, {'pair': pair})
|
||||
dataframe = strategy.advise_sell(dataframe, {'pair': pair})
|
||||
|
||||
if len(dataframe.index) > args.plot_limit:
|
||||
logger.warning('Ticker contained more than %s candles as defined '
|
||||
'with --plot-limit, clipping.', args.plot_limit)
|
||||
dataframe = dataframe.tail(args.plot_limit)
|
||||
|
||||
trades = trades.loc[trades['opents'] >= dataframe.iloc[0]['date']]
|
||||
fig = generate_graph(
|
||||
pair=pair,
|
||||
trades=trades,
|
||||
data=dataframe,
|
||||
args=args
|
||||
)
|
||||
|
||||
plot(fig, filename=str(Path('user_data').joinpath('freqtrade-plot.html')))
|
||||
|
||||
|
||||
def generate_graph(pair, trades: pd.DataFrame, data: pd.DataFrame, args) -> tools.make_subplots:
|
||||
"""
|
||||
Generate the graph from the data generated by Backtesting or from DB
|
||||
:param pair: Pair to Display on the graph
|
||||
:param trades: All trades created
|
||||
:param data: Dataframe
|
||||
:param args: sys.argv that contrains the two params indicators1, and indicators2
|
||||
:return: None
|
||||
"""
|
||||
|
||||
# Define the graph
|
||||
fig = tools.make_subplots(
|
||||
rows=3,
|
||||
cols=1,
|
||||
shared_xaxes=True,
|
||||
row_width=[1, 1, 4],
|
||||
vertical_spacing=0.0001,
|
||||
)
|
||||
fig['layout'].update(title=pair)
|
||||
fig['layout']['yaxis1'].update(title='Price')
|
||||
fig['layout']['yaxis2'].update(title='Volume')
|
||||
fig['layout']['yaxis3'].update(title='Other')
|
||||
|
||||
# Common information
|
||||
candles = go.Candlestick(
|
||||
x=data.date,
|
||||
open=data.open,
|
||||
high=data.high,
|
||||
low=data.low,
|
||||
close=data.close,
|
||||
name='Price'
|
||||
)
|
||||
|
||||
df_buy = data[data['buy'] == 1]
|
||||
buys = go.Scattergl(
|
||||
x=df_buy.date,
|
||||
y=df_buy.close,
|
||||
mode='markers',
|
||||
name='buy',
|
||||
marker=dict(
|
||||
symbol='triangle-up-dot',
|
||||
size=9,
|
||||
line=dict(width=1),
|
||||
color='green',
|
||||
)
|
||||
)
|
||||
df_sell = data[data['sell'] == 1]
|
||||
sells = go.Scattergl(
|
||||
x=df_sell.date,
|
||||
y=df_sell.close,
|
||||
mode='markers',
|
||||
name='sell',
|
||||
marker=dict(
|
||||
symbol='triangle-down-dot',
|
||||
size=9,
|
||||
line=dict(width=1),
|
||||
color='red',
|
||||
)
|
||||
)
|
||||
|
||||
trade_buys = go.Scattergl(
|
||||
x=trades["opents"],
|
||||
y=trades["open_rate"],
|
||||
mode='markers',
|
||||
name='trade_buy',
|
||||
marker=dict(
|
||||
symbol='square-open',
|
||||
size=11,
|
||||
line=dict(width=2),
|
||||
color='green'
|
||||
)
|
||||
)
|
||||
trade_sells = go.Scattergl(
|
||||
x=trades["closets"],
|
||||
y=trades["close_rate"],
|
||||
mode='markers',
|
||||
name='trade_sell',
|
||||
marker=dict(
|
||||
symbol='square-open',
|
||||
size=11,
|
||||
line=dict(width=2),
|
||||
color='red'
|
||||
)
|
||||
)
|
||||
|
||||
# Row 1
|
||||
fig.append_trace(candles, 1, 1)
|
||||
|
||||
if 'bb_lowerband' in data and 'bb_upperband' in data:
|
||||
bb_lower = go.Scatter(
|
||||
x=data.date,
|
||||
y=data.bb_lowerband,
|
||||
name='BB lower',
|
||||
line={'color': 'rgba(255,255,255,0)'},
|
||||
)
|
||||
bb_upper = go.Scatter(
|
||||
x=data.date,
|
||||
y=data.bb_upperband,
|
||||
name='BB upper',
|
||||
fill="tonexty",
|
||||
fillcolor="rgba(0,176,246,0.2)",
|
||||
line={'color': 'rgba(255,255,255,0)'},
|
||||
)
|
||||
fig.append_trace(bb_lower, 1, 1)
|
||||
fig.append_trace(bb_upper, 1, 1)
|
||||
|
||||
fig = generate_row(fig=fig, row=1, raw_indicators=args.indicators1, data=data)
|
||||
fig.append_trace(buys, 1, 1)
|
||||
fig.append_trace(sells, 1, 1)
|
||||
fig.append_trace(trade_buys, 1, 1)
|
||||
fig.append_trace(trade_sells, 1, 1)
|
||||
|
||||
# Row 2
|
||||
volume = go.Bar(
|
||||
x=data['date'],
|
||||
y=data['volume'],
|
||||
name='Volume'
|
||||
)
|
||||
fig.append_trace(volume, 2, 1)
|
||||
|
||||
# Row 3
|
||||
fig = generate_row(fig=fig, row=3, raw_indicators=args.indicators2, data=data)
|
||||
|
||||
return fig
|
||||
|
||||
|
||||
def generate_row(fig, row, raw_indicators, data) -> tools.make_subplots:
|
||||
"""
|
||||
Generator all the indicator selected by the user for a specific row
|
||||
"""
|
||||
for indicator in raw_indicators.split(','):
|
||||
if indicator in data:
|
||||
scattergl = go.Scattergl(
|
||||
x=data['date'],
|
||||
y=data[indicator],
|
||||
name=indicator
|
||||
)
|
||||
fig.append_trace(scattergl, row, 1)
|
||||
else:
|
||||
logger.info(
|
||||
'Indicator "%s" ignored. Reason: This indicator is not found '
|
||||
'in your strategy.',
|
||||
indicator
|
||||
)
|
||||
|
||||
return fig
|
||||
|
||||
|
||||
def plot_parse_args(args: List[str]) -> Namespace:
|
||||
"""
|
||||
Parse args passed to the script
|
||||
:param args: Cli arguments
|
||||
:return: args: Array with all arguments
|
||||
"""
|
||||
arguments = Arguments(args, 'Graph dataframe')
|
||||
arguments.scripts_options()
|
||||
arguments.parser.add_argument(
|
||||
'--indicators1',
|
||||
help='Set indicators from your strategy you want in the first row of the graph. Separate '
|
||||
'them with a coma. E.g: ema3,ema5 (default: %(default)s)',
|
||||
type=str,
|
||||
default='sma,ema3,ema5',
|
||||
dest='indicators1',
|
||||
)
|
||||
|
||||
arguments.parser.add_argument(
|
||||
'--indicators2',
|
||||
help='Set indicators from your strategy you want in the third row of the graph. Separate '
|
||||
'them with a coma. E.g: fastd,fastk (default: %(default)s)',
|
||||
type=str,
|
||||
default='macd',
|
||||
dest='indicators2',
|
||||
)
|
||||
arguments.parser.add_argument(
|
||||
'--plot-limit',
|
||||
help='Specify tick limit for plotting - too high values cause huge files - '
|
||||
'Default: %(default)s',
|
||||
dest='plot_limit',
|
||||
default=750,
|
||||
type=int,
|
||||
)
|
||||
arguments.common_args_parser()
|
||||
arguments.optimizer_shared_options(arguments.parser)
|
||||
arguments.backtesting_options(arguments.parser)
|
||||
return arguments.parse_args()
|
||||
|
||||
|
||||
def main(sysargv: List[str]) -> None:
|
||||
"""
|
||||
This function will initiate the bot and start the trading loop.
|
||||
:return: None
|
||||
"""
|
||||
logger.info('Starting Plot Dataframe')
|
||||
plot_analyzed_dataframe(
|
||||
plot_parse_args(sysargv)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main(sys.argv[1:])
|
||||
Loading…
Reference in New Issue