conflict with develop resolved

This commit is contained in:
Misagh 2019-04-05 20:23:15 +02:00
commit 4b2eb22989
23 changed files with 546 additions and 403 deletions

View File

@ -79,7 +79,7 @@ prevent unintended disclosure of sensitive private data when you publish example
of your configuration in the project issues or in the Internet. of your configuration in the project issues or in the Internet.
See more details on this technique with examples in the documentation page on See more details on this technique with examples in the documentation page on
[configuration](bot-configuration.md). [configuration](configuration.md).
### How to use **--strategy**? ### How to use **--strategy**?

View File

@ -62,7 +62,7 @@ If you have updated the buy strategy, ie. changed the contents of
#### Sell optimization #### Sell optimization
Similar to the buy-signal above, sell-signals can also be optimized. Similar to the buy-signal above, sell-signals can also be optimized.
Place the corresponding settings into the following methods Place the corresponding settings into the following methods
* Inside `sell_indicator_space()` - the parameters hyperopt shall be optimizing. * Inside `sell_indicator_space()` - the parameters hyperopt shall be optimizing.
@ -163,7 +163,7 @@ running at least several thousand evaluations.
The `--spaces all` flag determines that all possible parameters should be optimized. Possibilities are listed below. The `--spaces all` flag determines that all possible parameters should be optimized. Possibilities are listed below.
!!! Warning !!! Warning
When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file. When switching parameters or changing configuration options, the file `user_data/hyperopt_results.pickle` should be removed. It's used to be able to continue interrupted calculations, but does not detect changes to settings or the hyperopt file.
### Execute Hyperopt with Different Ticker-Data Source ### Execute Hyperopt with Different Ticker-Data Source

View File

@ -1,5 +1,5 @@
# SQL Helper # SQL Helper
This page constains some help if you want to edit your sqlite db. This page contains some help if you want to edit your sqlite db.
## Install sqlite3 ## Install sqlite3
**Ubuntu/Debian installation** **Ubuntu/Debian installation**
@ -66,11 +66,11 @@ SELECT * FROM trades;
## Fix trade still open after a manual sell on the exchange ## Fix trade still open after a manual sell on the exchange
!!! Warning !!! Warning
Manually selling on the exchange should not be done by default, since the bot does not detect this and will try to sell anyway. Manually selling a pair on the exchange will not be detected by the bot and it will try to sell anyway. Whenever possible, forcesell <tradeid> should be used to accomplish the same thing.
/foresell <tradeid> should accomplish the same thing. It is strongly advised to backup your database file before making any manual changes.
!!! Note !!! Note
This should not be necessary after /forcesell, as forcesell orders are closed automatically by the bot on the next iteration. This should not be necessary after /forcesell, as forcesell orders are closed automatically by the bot on the next iteration.
```sql ```sql
UPDATE trades UPDATE trades

View File

@ -1,13 +1,13 @@
# Telegram usage # Telegram usage
This page explains how to command your bot with Telegram.
## Prerequisite ## Prerequisite
To control your bot with Telegram, you need first to To control your bot with Telegram, you need first to
[set up a Telegram bot](installation.md) [set up a Telegram bot](installation.md)
and add your Telegram API keys into your config file. and add your Telegram API keys into your config file.
## Telegram commands ## Telegram commands
Per default, the Telegram bot shows predefined commands. Some commands Per default, the Telegram bot shows predefined commands. Some commands
are only available by sending them to the bot. The table below list the are only available by sending them to the bot. The table below list the
official commands. You can ask at any moment for help with `/help`. official commands. You can ask at any moment for help with `/help`.
@ -65,16 +65,14 @@ Once all positions are sold, run `/stop` to completely stop the bot.
For each open trade, the bot will send you the following message. For each open trade, the bot will send you the following message.
> **Trade ID:** `123` > **Trade ID:** `123` `(since 1 days ago)`
> **Current Pair:** CVC/BTC > **Current Pair:** CVC/BTC
> **Open Since:** `1 days ago` > **Open Since:** `1 days ago`
> **Amount:** `26.64180098` > **Amount:** `26.64180098`
> **Open Rate:** `0.00007489` > **Open Rate:** `0.00007489`
> **Close Rate:** `None` > **Current Rate:** `0.00007489`
> **Current Rate:** `0.00007489` > **Current Profit:** `12.95%`
> **Close Profit:** `None` > **Stoploss:** `0.00007389 (-0.02%)`
> **Current Profit:** `12.95%`
> **Open Order:** `None`
### /status table ### /status table
@ -99,18 +97,18 @@ current max
Return a summary of your profit/loss and performance. Return a summary of your profit/loss and performance.
> **ROI:** Close trades > **ROI:** Close trades
> ∙ `0.00485701 BTC (258.45%)` > ∙ `0.00485701 BTC (258.45%)`
> ∙ `62.968 USD` > ∙ `62.968 USD`
> **ROI:** All trades > **ROI:** All trades
> ∙ `0.00255280 BTC (143.43%)` > ∙ `0.00255280 BTC (143.43%)`
> ∙ `33.095 EUR` > ∙ `33.095 EUR`
> >
> **Total Trade Count:** `138` > **Total Trade Count:** `138`
> **First Trade opened:** `3 days ago` > **First Trade opened:** `3 days ago`
> **Latest Trade opened:** `2 minutes ago` > **Latest Trade opened:** `2 minutes ago`
> **Avg. Duration:** `2:33:45` > **Avg. Duration:** `2:33:45`
> **Best Performing:** `PAY/BTC: 50.23%` > **Best Performing:** `PAY/BTC: 50.23%`
### /forcesell <trade_id> ### /forcesell <trade_id>
@ -128,26 +126,26 @@ Note that for this to work, `forcebuy_enable` needs to be set to true.
Return the performance of each crypto-currency the bot has sold. Return the performance of each crypto-currency the bot has sold.
> Performance: > Performance:
> 1. `RCN/BTC 57.77%` > 1. `RCN/BTC 57.77%`
> 2. `PAY/BTC 56.91%` > 2. `PAY/BTC 56.91%`
> 3. `VIB/BTC 47.07%` > 3. `VIB/BTC 47.07%`
> 4. `SALT/BTC 30.24%` > 4. `SALT/BTC 30.24%`
> 5. `STORJ/BTC 27.24%` > 5. `STORJ/BTC 27.24%`
> ... > ...
### /balance ### /balance
Return the balance of all crypto-currency your have on the exchange. Return the balance of all crypto-currency your have on the exchange.
> **Currency:** BTC > **Currency:** BTC
> **Available:** 3.05890234 > **Available:** 3.05890234
> **Balance:** 3.05890234 > **Balance:** 3.05890234
> **Pending:** 0.0 > **Pending:** 0.0
> **Currency:** CVC > **Currency:** CVC
> **Available:** 86.64180098 > **Available:** 86.64180098
> **Balance:** 86.64180098 > **Balance:** 86.64180098
> **Pending:** 0.0 > **Pending:** 0.0
### /daily <n> ### /daily <n>

View File

@ -1,7 +1,5 @@
# Webhook usage # Webhook usage
This page explains how to configure your bot to talk to webhooks.
## Configuration ## Configuration
Enable webhooks by adding a webhook-section to your configuration file, and setting `webhook.enabled` to `true`. Enable webhooks by adding a webhook-section to your configuration file, and setting `webhook.enabled` to `true`.
@ -39,32 +37,30 @@ Different payloads can be configured for different events. Not all fields are ne
The fields in `webhook.webhookbuy` are filled when the bot executes a buy. Parameters are filled using string.format. The fields in `webhook.webhookbuy` are filled when the bot executes a buy. Parameters are filled using string.format.
Possible parameters are: Possible parameters are:
* exchange * `exchange`
* pair * `pair`
* limit * `limit`
* stake_amount * `stake_amount`
* stake_amount_fiat * `stake_currency`
* stake_currency * `fiat_currency`
* fiat_currency
### Webhooksell ### Webhooksell
The fields in `webhook.webhooksell` are filled when the bot sells a trade. Parameters are filled using string.format. The fields in `webhook.webhooksell` are filled when the bot sells a trade. Parameters are filled using string.format.
Possible parameters are: Possible parameters are:
* exchange * `exchange`
* pair * `pair`
* gain * `gain`
* limit * `limit`
* amount * `amount`
* open_rate * `open_rate`
* current_rate * `current_rate`
* profit_amount * `profit_amount`
* profit_percent * `profit_percent`
* profit_fiat * `stake_currency`
* stake_currency * `fiat_currency`
* fiat_currency * `sell_reason`
* sell_reason
### Webhookstatus ### Webhookstatus

View File

@ -203,6 +203,22 @@ class Edge():
return self._final_pairs return self._final_pairs
def accepted_pairs(self) -> list:
"""
return a list of accepted pairs along with their winrate, expectancy and stoploss
"""
final = []
for pair, info in self._cached_pairs.items():
if info.expectancy > float(self.edge_config.get('minimum_expectancy', 0.2)) and \
info.winrate > float(self.edge_config.get('minimum_winrate', 0.60)):
final.append({
'Pair': pair,
'Winrate': info.winrate,
'Expectancy': info.expectancy,
'Stoploss': info.stoploss,
})
return final
def _fill_calculable_fields(self, result: DataFrame) -> DataFrame: def _fill_calculable_fields(self, result: DataFrame) -> DataFrame:
""" """
The result frame contains a number of columns that are calculable The result frame contains a number of columns that are calculable

View File

@ -4,14 +4,12 @@ Freqtrade is the main module of this bot. It contains the class Freqtrade()
import copy import copy
import logging import logging
import time
import traceback import traceback
from datetime import datetime from datetime import datetime
from typing import Any, Callable, Dict, List, Optional, Tuple from typing import Any, Dict, List, Optional, Tuple
import arrow import arrow
from requests.exceptions import RequestException from requests.exceptions import RequestException
import sdnotify
from freqtrade import (DependencyException, OperationalException, InvalidOrderException, from freqtrade import (DependencyException, OperationalException, InvalidOrderException,
TemporaryError, __version__, constants, persistence) TemporaryError, __version__, constants, persistence)
@ -42,20 +40,14 @@ class FreqtradeBot(object):
to get the config dict. to get the config dict.
""" """
logger.info( logger.info('Starting freqtrade %s', __version__)
'Starting freqtrade %s',
__version__,
)
# Init bot states # Init bot state
self.state = State.STOPPED self.state = State.STOPPED
# Init objects # Init objects
self.config = config self.config = config
self._sd_notify = sdnotify.SystemdNotifier() if \
self.config.get('internals', {}).get('sd_notify', False) else None
self.strategy: IStrategy = StrategyResolver(self.config).strategy self.strategy: IStrategy = StrategyResolver(self.config).strategy
self.rpc: RPCManager = RPCManager(self) self.rpc: RPCManager = RPCManager(self)
@ -79,29 +71,12 @@ class FreqtradeBot(object):
self.config.get('edge', {}).get('enabled', False) else None self.config.get('edge', {}).get('enabled', False) else None
self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist'] self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist']
self._init_modules()
# Tell the systemd that we completed initialization phase
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def _init_modules(self) -> None:
"""
Initializes all modules and updates the config
:return: None
"""
# Initialize all modules
persistence.init(self.config) persistence.init(self.config)
# Set initial application state # Set initial bot state from config
initial_state = self.config.get('initial_state') initial_state = self.config.get('initial_state')
self.state = State[initial_state.upper()] if initial_state else State.STOPPED
if initial_state:
self.state = State[initial_state.upper()]
else:
self.state = State.STOPPED
def cleanup(self) -> None: def cleanup(self) -> None:
""" """
@ -113,130 +88,50 @@ class FreqtradeBot(object):
self.rpc.cleanup() self.rpc.cleanup()
persistence.cleanup() persistence.cleanup()
def stopping(self) -> None: def process(self) -> bool:
# Tell systemd that we are exiting now
if self._sd_notify:
logger.debug("sd_notify: STOPPING=1")
self._sd_notify.notify("STOPPING=1")
def reconfigure(self) -> None:
# Tell systemd that we initiated reconfiguring
if self._sd_notify:
logger.debug("sd_notify: RELOADING=1")
self._sd_notify.notify("RELOADING=1")
def worker(self, old_state: State = None) -> State:
"""
Trading routine that must be run at each loop
:param old_state: the previous service state from the previous call
:return: current service state
"""
# Log state transition
state = self.state
if state != old_state:
self.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'{state.name.lower()}'
})
logger.info('Changing state to: %s', state.name)
if state == State.RUNNING:
self.rpc.startup_messages(self.config, self.pairlists)
throttle_secs = self.config.get('internals', {}).get(
'process_throttle_secs',
constants.PROCESS_THROTTLE_SECS
)
if state == State.STOPPED:
# Ping systemd watchdog before sleeping in the stopped state
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: STOPPED.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: STOPPED.")
time.sleep(throttle_secs)
elif state == State.RUNNING:
# Ping systemd watchdog before throttling
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: RUNNING.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: RUNNING.")
self._throttle(func=self._process, min_secs=throttle_secs)
return state
def _throttle(self, func: Callable[..., Any], min_secs: float, *args, **kwargs) -> Any:
"""
Throttles the given callable that it
takes at least `min_secs` to finish execution.
:param func: Any callable
:param min_secs: minimum execution time in seconds
:return: Any
"""
start = time.time()
result = func(*args, **kwargs)
end = time.time()
duration = max(min_secs - (end - start), 0.0)
logger.debug('Throttling %s for %.2f seconds', func.__name__, duration)
time.sleep(duration)
return result
def _process(self) -> bool:
""" """
Queries the persistence layer for open trades and handles them, Queries the persistence layer for open trades and handles them,
otherwise a new trade is created. otherwise a new trade is created.
:return: True if one or more trades has been created or closed, False otherwise :return: True if one or more trades has been created or closed, False otherwise
""" """
state_changed = False state_changed = False
try:
# Check whether markets have to be reloaded
self.exchange._reload_markets()
# Refresh whitelist # Check whether markets have to be reloaded
self.pairlists.refresh_pairlist() self.exchange._reload_markets()
self.active_pair_whitelist = self.pairlists.whitelist
# Calculating Edge positioning # Refresh whitelist
if self.edge: self.pairlists.refresh_pairlist()
self.edge.calculate() self.active_pair_whitelist = self.pairlists.whitelist
self.active_pair_whitelist = self.edge.adjust(self.active_pair_whitelist)
# Query trades from persistence layer # Calculating Edge positioning
trades = Trade.get_open_trades() if self.edge:
self.edge.calculate()
self.active_pair_whitelist = self.edge.adjust(self.active_pair_whitelist)
# Extend active-pair whitelist with pairs from open trades # Query trades from persistence layer
# It ensures that tickers are downloaded for open trades trades = Trade.get_open_trades()
self._extend_whitelist_with_trades(self.active_pair_whitelist, trades)
# Refreshing candles # Extend active-pair whitelist with pairs from open trades
self.dataprovider.refresh(self._create_pair_whitelist(self.active_pair_whitelist), # It ensures that tickers are downloaded for open trades
self.strategy.informative_pairs()) self._extend_whitelist_with_trades(self.active_pair_whitelist, trades)
# First process current opened trades # Refreshing candles
for trade in trades: self.dataprovider.refresh(self._create_pair_whitelist(self.active_pair_whitelist),
state_changed |= self.process_maybe_execute_sell(trade) self.strategy.informative_pairs())
# Then looking for buy opportunities # First process current opened trades
if len(trades) < self.config['max_open_trades']: for trade in trades:
state_changed = self.process_maybe_execute_buy() state_changed |= self.process_maybe_execute_sell(trade)
if 'unfilledtimeout' in self.config: # Then looking for buy opportunities
# Check and handle any timed out open orders if len(trades) < self.config['max_open_trades']:
self.check_handle_timedout() state_changed = self.process_maybe_execute_buy()
Trade.session.flush()
if 'unfilledtimeout' in self.config:
# Check and handle any timed out open orders
self.check_handle_timedout()
Trade.session.flush()
except TemporaryError as error:
logger.warning(f"Error: {error}, retrying in {constants.RETRY_TIMEOUT} seconds...")
time.sleep(constants.RETRY_TIMEOUT)
except OperationalException:
tb = traceback.format_exc()
hint = 'Issue `/start` if you think it is safe to restart.'
self.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'OperationalException:\n```\n{tb}```{hint}'
})
logger.exception('OperationalException. Stopping trader ...')
self.state = State.STOPPED
return state_changed return state_changed
def _extend_whitelist_with_trades(self, whitelist: List[str], trades: List[Any]): def _extend_whitelist_with_trades(self, whitelist: List[str], trades: List[Any]):
@ -356,6 +251,10 @@ class FreqtradeBot(object):
interval = self.strategy.ticker_interval interval = self.strategy.ticker_interval
whitelist = copy.deepcopy(self.active_pair_whitelist) whitelist = copy.deepcopy(self.active_pair_whitelist)
if not whitelist:
logger.warning("Whitelist is empty.")
return False
# Remove currently opened and latest pairs from whitelist # Remove currently opened and latest pairs from whitelist
for trade in Trade.get_open_trades(): for trade in Trade.get_open_trades():
if trade.pair in whitelist: if trade.pair in whitelist:
@ -363,7 +262,8 @@ class FreqtradeBot(object):
logger.debug('Ignoring %s in pair whitelist', trade.pair) logger.debug('Ignoring %s in pair whitelist', trade.pair)
if not whitelist: if not whitelist:
raise DependencyException('No currency pairs in whitelist') logger.info("No currency pair in whitelist, but checking to sell open trades.")
return False
# running get_signal on historical data fetched # running get_signal on historical data fetched
for _pair in whitelist: for _pair in whitelist:

View File

@ -10,10 +10,9 @@ from typing import List
from freqtrade import OperationalException from freqtrade import OperationalException
from freqtrade.arguments import Arguments from freqtrade.arguments import Arguments
from freqtrade.configuration import Configuration, set_loggers from freqtrade.configuration import set_loggers
from freqtrade.freqtradebot import FreqtradeBot from freqtrade.worker import Worker
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType
logger = logging.getLogger('freqtrade') logger = logging.getLogger('freqtrade')
@ -27,7 +26,7 @@ def main(sysargv: List[str]) -> None:
sysargv, sysargv,
'Free, open source crypto trading bot' 'Free, open source crypto trading bot'
) )
args = arguments.get_parsed_arg() args: Namespace = arguments.get_parsed_arg()
# A subcommand has been issued. # A subcommand has been issued.
# Means if Backtesting or Hyperopt have been called we exit the bot # Means if Backtesting or Hyperopt have been called we exit the bot
@ -35,20 +34,12 @@ def main(sysargv: List[str]) -> None:
args.func(args) args.func(args)
return return
freqtrade = None worker = None
return_code = 1 return_code = 1
try: try:
# Load and validate configuration # Load and run worker
config = Configuration(args, None).get_config() worker = Worker(args)
worker.run()
# Init the bot
freqtrade = FreqtradeBot(config)
state = None
while True:
state = freqtrade.worker(old_state=state)
if state == State.RELOAD_CONF:
freqtrade = reconfigure(freqtrade, args)
except KeyboardInterrupt: except KeyboardInterrupt:
logger.info('SIGINT received, aborting ...') logger.info('SIGINT received, aborting ...')
@ -59,34 +50,11 @@ def main(sysargv: List[str]) -> None:
except BaseException: except BaseException:
logger.exception('Fatal exception!') logger.exception('Fatal exception!')
finally: finally:
if freqtrade: if worker:
freqtrade.stopping() worker.exit()
freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'process died'
})
freqtrade.cleanup()
sys.exit(return_code) sys.exit(return_code)
def reconfigure(freqtrade: FreqtradeBot, args: Namespace) -> FreqtradeBot:
"""
Cleans up current instance, reloads the configuration and returns the new instance
"""
freqtrade.reconfigure()
# Clean up current modules
freqtrade.cleanup()
# Create new instance
freqtrade = FreqtradeBot(Configuration(args, None).get_config())
freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'config reloaded'
})
return freqtrade
if __name__ == '__main__': if __name__ == '__main__':
set_loggers() set_loggers()
main(sys.argv[1:]) main(sys.argv[1:])

View File

@ -210,6 +210,32 @@ class Backtesting(object):
logger.info('Dumping backtest results to %s', recordfilename) logger.info('Dumping backtest results to %s', recordfilename)
file_dump_json(recordfilename, records) file_dump_json(recordfilename, records)
def _get_ticker_list(self, processed) -> Dict[str, DataFrame]:
"""
Helper function to convert a processed tickerlist into a list for performance reasons.
Used by backtest() - so keep this optimized for performance.
"""
headers = ['date', 'buy', 'open', 'close', 'sell', 'low', 'high']
ticker: Dict = {}
# Create ticker dict
for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
# to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
ticker_data.loc[:, 'sell'] = ticker_data['sell'].shift(1)
ticker_data.drop(ticker_data.head(1).index, inplace=True)
# Convert from Pandas to list for performance reasons
# (Looping Pandas is slow.)
ticker[pair] = [x for x in ticker_data.itertuples()]
return ticker
def _get_sell_trade_entry( def _get_sell_trade_entry(
self, pair: str, buy_row: DataFrame, self, pair: str, buy_row: DataFrame,
partial_ticker: List, trade_count_lock: Dict, args: Dict) -> Optional[BacktestResult]: partial_ticker: List, trade_count_lock: Dict, args: Dict) -> Optional[BacktestResult]:
@ -304,7 +330,6 @@ class Backtesting(object):
position_stacking: do we allow position stacking? (default: False) position_stacking: do we allow position stacking? (default: False)
:return: DataFrame :return: DataFrame
""" """
headers = ['date', 'buy', 'open', 'close', 'sell', 'low', 'high']
processed = args['processed'] processed = args['processed']
max_open_trades = args.get('max_open_trades', 0) max_open_trades = args.get('max_open_trades', 0)
position_stacking = args.get('position_stacking', False) position_stacking = args.get('position_stacking', False)
@ -312,54 +337,50 @@ class Backtesting(object):
end_date = args['end_date'] end_date = args['end_date']
trades = [] trades = []
trade_count_lock: Dict = {} trade_count_lock: Dict = {}
ticker: Dict = {}
pairs = []
# Create ticker dict
for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell( # Dict of ticker-lists for performance (looping lists is a lot faster than dataframes)
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy() ticker: Dict = self._get_ticker_list(processed)
# to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
ticker_data.loc[:, 'sell'] = ticker_data['sell'].shift(1)
ticker_data.drop(ticker_data.head(1).index, inplace=True)
# Convert from Pandas to list for performance reasons
# (Looping Pandas is slow.)
ticker[pair] = [x for x in ticker_data.itertuples()]
pairs.append(pair)
lock_pair_until: Dict = {} lock_pair_until: Dict = {}
# Indexes per pair, so some pairs are allowed to have a missing start.
indexes: Dict = {}
tmp = start_date + timedelta(minutes=self.ticker_interval_mins) tmp = start_date + timedelta(minutes=self.ticker_interval_mins)
index = 0
# Loop timerange and test per pair # Loop timerange and get candle for each pair at that point in time
while tmp < end_date: while tmp < end_date:
# print(f"time: {tmp}")
for i, pair in enumerate(ticker): for i, pair in enumerate(ticker):
if pair not in indexes:
indexes[pair] = 0
try: try:
row = ticker[pair][index] row = ticker[pair][indexes[pair]]
except IndexError: except IndexError:
# missing Data for one pair ... # missing Data for one pair at the end.
# Warnings for this are shown by `validate_backtest_data` # Warnings for this are shown by `validate_backtest_data`
continue continue
# Waits until the time-counter reaches the start of the data for this pair.
if row.date > tmp.datetime:
continue
indexes[pair] += 1
if row.buy == 0 or row.sell == 1: if row.buy == 0 or row.sell == 1:
continue # skip rows where no buy signal or that would immediately sell off continue # skip rows where no buy signal or that would immediately sell off
if not position_stacking: if (not position_stacking and pair in lock_pair_until
if pair in lock_pair_until and row.date <= lock_pair_until[pair]: and row.date <= lock_pair_until[pair]):
continue # without positionstacking, we can only have one open trade per pair.
continue
if max_open_trades > 0: if max_open_trades > 0:
# Check if max_open_trades has already been reached for the given date # Check if max_open_trades has already been reached for the given date
if not trade_count_lock.get(row.date, 0) < max_open_trades: if not trade_count_lock.get(row.date, 0) < max_open_trades:
continue continue
trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1 trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1
trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][index + 1:], trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][indexes[pair]:],
trade_count_lock, args) trade_count_lock, args)
if trade_entry: if trade_entry:
@ -367,11 +388,10 @@ class Backtesting(object):
trades.append(trade_entry) trades.append(trade_entry)
else: else:
# Set lock_pair_until to end of testing period if trade could not be closed # Set lock_pair_until to end of testing period if trade could not be closed
# This happens only if the buy-signal was with the last candle lock_pair_until[pair] = end_date.datetime
lock_pair_until[pair] = end_date
# Move time one configured time_interval ahead.
tmp += timedelta(minutes=self.ticker_interval_mins) tmp += timedelta(minutes=self.ticker_interval_mins)
index += 1
return DataFrame.from_records(trades, columns=BacktestResult._fields) return DataFrame.from_records(trades, columns=BacktestResult._fields)
def start(self) -> None: def start(self) -> None:

View File

@ -103,16 +103,23 @@ class RPC(object):
results.append(dict( results.append(dict(
trade_id=trade.id, trade_id=trade.id,
pair=trade.pair, pair=trade.pair,
base_currency=self._freqtrade.config['stake_currency'],
date=arrow.get(trade.open_date), date=arrow.get(trade.open_date),
open_rate=trade.open_rate, open_rate=trade.open_rate,
close_rate=trade.close_rate, close_rate=trade.close_rate,
current_rate=current_rate, current_rate=current_rate,
amount=round(trade.amount, 8), amount=round(trade.amount, 8),
stake_amount=round(trade.stake_amount, 8),
close_profit=fmt_close_profit, close_profit=fmt_close_profit,
current_profit=round(current_profit * 100, 2), current_profit=round(current_profit * 100, 2),
stop_loss=trade.stop_loss, stop_loss=trade.stop_loss,
stop_loss_pct=(trade.stop_loss_pct * 100)
if trade.stop_loss_pct else None,
initial_stop_loss=trade.initial_stop_loss,
initial_stop_loss_pct=(trade.initial_stop_loss_pct * 100)
if trade.initial_stop_loss_pct else None,
open_order='({} {} rem={:.8f})'.format( open_order='({} {} rem={:.8f})'.format(
order['type'], order['side'], order['remaining'] order['type'], order['side'], order['remaining']
) if order else None, ) if order else None,
)) ))
return results return results
@ -481,13 +488,4 @@ class RPC(object):
""" Returns information related to Edge """ """ Returns information related to Edge """
if not self._freqtrade.edge: if not self._freqtrade.edge:
raise RPCException(f'Edge is not enabled.') raise RPCException(f'Edge is not enabled.')
return self._freqtrade.edge.accepted_pairs()
return [
{
'Pair': k,
'Winrate': v.winrate,
'Expectancy': v.expectancy,
'Stoploss': v.stoploss,
}
for k, v in self._freqtrade.edge._cached_pairs.items()
]

View File

@ -197,16 +197,25 @@ class Telegram(RPC):
messages = [] messages = []
for r in results: for r in results:
lines = [ lines = [
"*Trade ID:* `{trade_id}` (since `{date}`)", "*Trade ID:* `{trade_id}` `(since {date})`",
"*Current Pair:* {pair}", "*Current Pair:* {pair}",
"*Amount:* `{amount}`", "*Amount:* `{amount} ({stake_amount} {base_currency})`",
"*Open Rate:* `{open_rate:.8f}`", "*Open Rate:* `{open_rate:.8f}`",
"*Close Rate:* `{close_rate}`" if r['close_rate'] else "", "*Close Rate:* `{close_rate}`" if r['close_rate'] else "",
"*Current Rate:* `{current_rate:.8f}`", "*Current Rate:* `{current_rate:.8f}`",
"*Close Profit:* `{close_profit}`" if r['close_profit'] else "", "*Close Profit:* `{close_profit}`" if r['close_profit'] else "",
"*Current Profit:* `{current_profit:.2f}%`", "*Current Profit:* `{current_profit:.2f}%`",
"*Stoploss:* `{stop_loss:.8f}`",
"*Open Order:* `{open_order}`" if r['open_order'] else "", # Adding initial stoploss only if it is different from stoploss
"*Initial Stoploss:* `{initial_stop_loss:.8f}` " +
("`({initial_stop_loss_pct:.2f}%)`" if r['initial_stop_loss_pct'] else "")
if r['stop_loss'] != r['initial_stop_loss'] else "",
# Adding stoploss and stoploss percentage only if it is not None
"*Stoploss:* `{stop_loss:.8f}` " +
("`({stop_loss_pct:.2f}%)`" if r['stop_loss_pct'] else ""),
"*Open Order:* `{open_order}`" if r['open_order'] else ""
] ]
messages.append("\n".join(filter(None, lines)).format(**r)) messages.append("\n".join(filter(None, lines)).format(**r))
@ -502,7 +511,6 @@ class Telegram(RPC):
""" """
try: try:
edge_pairs = self._rpc_edge() edge_pairs = self._rpc_edge()
print(edge_pairs)
edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple') edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple')
message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>' message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML) self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML)

View File

@ -16,6 +16,7 @@ from freqtrade.edge import Edge, PairInfo
from freqtrade.exchange import Exchange from freqtrade.exchange import Exchange
from freqtrade.freqtradebot import FreqtradeBot from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.resolvers import ExchangeResolver from freqtrade.resolvers import ExchangeResolver
from freqtrade.worker import Worker
logging.getLogger('').setLevel(logging.INFO) logging.getLogger('').setLevel(logging.INFO)
@ -87,7 +88,7 @@ def get_patched_edge(mocker, config) -> Edge:
# Functions for recurrent object patching # Functions for recurrent object patching
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot: def patch_freqtradebot(mocker, config) -> None:
""" """
This function patch _init_modules() to not call dependencies This function patch _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches :param mocker: a Mocker object to apply patches
@ -100,9 +101,17 @@ def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
patch_freqtradebot(mocker, config)
return FreqtradeBot(config) return FreqtradeBot(config)
def get_patched_worker(mocker, config) -> Worker:
patch_freqtradebot(mocker, config)
return Worker(args=None, config=config)
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def patch_coinmarketcap(mocker) -> None: def patch_coinmarketcap(mocker) -> None:
""" """

View File

@ -122,8 +122,8 @@ def test_edge_results(edge_conf, mocker, caplog, data) -> None:
for c, trade in enumerate(data.trades): for c, trade in enumerate(data.trades):
res = results.iloc[c] res = results.iloc[c]
assert res.exit_type == trade.sell_reason assert res.exit_type == trade.sell_reason
assert res.open_time == _get_frame_time_from_offset(trade.open_tick) assert arrow.get(res.open_time) == _get_frame_time_from_offset(trade.open_tick)
assert res.close_time == _get_frame_time_from_offset(trade.close_tick) assert arrow.get(res.close_time) == _get_frame_time_from_offset(trade.close_tick)
def test_adjust(mocker, edge_conf): def test_adjust(mocker, edge_conf):

View File

@ -33,7 +33,7 @@ class BTContainer(NamedTuple):
def _get_frame_time_from_offset(offset): def _get_frame_time_from_offset(offset):
return ticker_start_time.shift(minutes=(offset * TICKER_INTERVAL_MINUTES[tests_ticker_interval]) return ticker_start_time.shift(minutes=(offset * TICKER_INTERVAL_MINUTES[tests_ticker_interval])
).datetime.replace(tzinfo=None) ).datetime
def _build_backtest_dataframe(ticker_with_signals): def _build_backtest_dataframe(ticker_with_signals):

View File

@ -685,25 +685,32 @@ def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
assert len(results.loc[results.open_at_end]) == 0 assert len(results.loc[results.open_at_end]) == 0
def test_backtest_multi_pair(default_conf, fee, mocker): @pytest.mark.parametrize("pair", ['ADA/BTC', 'LTC/BTC'])
@pytest.mark.parametrize("tres", [0, 20, 30])
def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
def _trend_alternate_hold(dataframe=None, metadata=None): def _trend_alternate_hold(dataframe=None, metadata=None):
""" """
Buy every 8th candle - sell every other 8th -2 (hold on to pairs a bit) Buy every xth candle - sell every other xth -2 (hold on to pairs a bit)
""" """
multi = 8 if metadata['pair'] in('ETH/BTC', 'LTC/BTC'):
multi = 20
else:
multi = 18
dataframe['buy'] = np.where(dataframe.index % multi == 0, 1, 0) dataframe['buy'] = np.where(dataframe.index % multi == 0, 1, 0)
dataframe['sell'] = np.where((dataframe.index + multi - 2) % multi == 0, 1, 0) dataframe['sell'] = np.where((dataframe.index + multi - 2) % multi == 0, 1, 0)
if metadata['pair'] in('ETH/BTC', 'LTC/BTC'):
dataframe['buy'] = dataframe['buy'].shift(-4)
dataframe['sell'] = dataframe['sell'].shift(-4)
return dataframe return dataframe
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker) patch_exchange(mocker)
pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC'] pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC']
data = history.load_data(datadir=None, ticker_interval='5m', pairs=pairs) data = history.load_data(datadir=None, ticker_interval='5m', pairs=pairs)
# Only use 500 lines to increase performance
data = trim_dictlist(data, -500) data = trim_dictlist(data, -500)
# Remove data for one pair from the beginning of the data
data[pair] = data[pair][tres:]
# We need to enable sell-signal - otherwise it sells on ROI!! # We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True} default_conf['experimental'] = {"use_sell_signal": True}
default_conf['ticker_interval'] = '5m' default_conf['ticker_interval'] = '5m'

View File

@ -51,14 +51,19 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
assert { assert {
'trade_id': 1, 'trade_id': 1,
'pair': 'ETH/BTC', 'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': ANY, 'date': ANY,
'open_rate': 1.099e-05, 'open_rate': 1.099e-05,
'close_rate': None, 'close_rate': None,
'current_rate': 1.098e-05, 'current_rate': 1.098e-05,
'amount': 90.99181074, 'amount': 90.99181074,
'stake_amount': 0.001,
'close_profit': None, 'close_profit': None,
'current_profit': -0.59, 'current_profit': -0.59,
'stop_loss': 0.0, 'stop_loss': 0.0,
'initial_stop_loss': 0.0,
'initial_stop_loss_pct': None,
'stop_loss_pct': None,
'open_order': '(limit buy rem=0.00000000)' 'open_order': '(limit buy rem=0.00000000)'
} == results[0] } == results[0]
@ -72,14 +77,19 @@ def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None:
assert { assert {
'trade_id': 1, 'trade_id': 1,
'pair': 'ETH/BTC', 'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': ANY, 'date': ANY,
'open_rate': 1.099e-05, 'open_rate': 1.099e-05,
'close_rate': None, 'close_rate': None,
'current_rate': ANY, 'current_rate': ANY,
'amount': 90.99181074, 'amount': 90.99181074,
'stake_amount': 0.001,
'close_profit': None, 'close_profit': None,
'current_profit': ANY, 'current_profit': ANY,
'stop_loss': 0.0, 'stop_loss': 0.0,
'initial_stop_loss': 0.0,
'initial_stop_loss_pct': None,
'stop_loss_pct': None,
'open_order': '(limit buy rem=0.00000000)' 'open_order': '(limit buy rem=0.00000000)'
} == results[0] } == results[0]

View File

@ -189,14 +189,19 @@ def test_status(default_conf, update, mocker, fee, ticker, markets) -> None:
_rpc_trade_status=MagicMock(return_value=[{ _rpc_trade_status=MagicMock(return_value=[{
'trade_id': 1, 'trade_id': 1,
'pair': 'ETH/BTC', 'pair': 'ETH/BTC',
'base_currency': 'BTC',
'date': arrow.utcnow(), 'date': arrow.utcnow(),
'open_rate': 1.099e-05, 'open_rate': 1.099e-05,
'close_rate': None, 'close_rate': None,
'current_rate': 1.098e-05, 'current_rate': 1.098e-05,
'amount': 90.99181074, 'amount': 90.99181074,
'stake_amount': 90.99181074,
'close_profit': None, 'close_profit': None,
'current_profit': -0.59, 'current_profit': -0.59,
'initial_stop_loss': 1.098e-05,
'stop_loss': 1.099e-05, 'stop_loss': 1.099e-05,
'initial_stop_loss_pct': -0.05,
'stop_loss_pct': -0.01,
'open_order': '(limit buy rem=0.00000000)' 'open_order': '(limit buy rem=0.00000000)'
}]), }]),
_status_table=status_table, _status_table=status_table,

View File

@ -21,12 +21,13 @@ from freqtrade.state import State
from freqtrade.strategy.interface import SellCheckTuple, SellType from freqtrade.strategy.interface import SellCheckTuple, SellType
from freqtrade.tests.conftest import (log_has, log_has_re, patch_edge, from freqtrade.tests.conftest import (log_has, log_has_re, patch_edge,
patch_exchange, patch_wallet) patch_exchange, patch_wallet)
from freqtrade.worker import Worker
# Functions for recurrent object patching # Functions for recurrent object patching
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot: def patch_freqtradebot(mocker, config) -> None:
""" """
This function patch _init_modules() to not call dependencies This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches :param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot :param config: Config to pass to the bot
:return: None :return: None
@ -35,9 +36,29 @@ def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock()) mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
patch_exchange(mocker) patch_exchange(mocker)
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
"""
This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot
:return: FreqtradeBot
"""
patch_freqtradebot(mocker, config)
return FreqtradeBot(config) return FreqtradeBot(config)
def get_patched_worker(mocker, config) -> Worker:
"""
This function patches _init_modules() to not call dependencies
:param mocker: a Mocker object to apply patches
:param config: Config to pass to the bot
:return: Worker
"""
patch_freqtradebot(mocker, config)
return Worker(args=None, config=config)
def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None: def patch_get_signal(freqtrade: FreqtradeBot, value=(True, False)) -> None:
""" """
:param mocker: mocker to patch IStrategy class :param mocker: mocker to patch IStrategy class
@ -61,7 +82,7 @@ def patch_RPCManager(mocker) -> MagicMock:
# Unit tests # Unit tests
def test_freqtradebot(mocker, default_conf, markets) -> None: def test_freqtradebot_state(mocker, default_conf, markets) -> None:
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets)) mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets))
freqtrade = get_patched_freqtradebot(mocker, default_conf) freqtrade = get_patched_freqtradebot(mocker, default_conf)
assert freqtrade.state is State.RUNNING assert freqtrade.state is State.RUNNING
@ -71,6 +92,16 @@ def test_freqtradebot(mocker, default_conf, markets) -> None:
assert freqtrade.state is State.STOPPED assert freqtrade.state is State.STOPPED
def test_worker_state(mocker, default_conf, markets) -> None:
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets))
worker = get_patched_worker(mocker, default_conf)
assert worker.state is State.RUNNING
default_conf.pop('initial_state')
worker = Worker(args=None, config=default_conf)
assert worker.state is State.STOPPED
def test_cleanup(mocker, default_conf, caplog) -> None: def test_cleanup(mocker, default_conf, caplog) -> None:
mock_cleanup = MagicMock() mock_cleanup = MagicMock()
mocker.patch('freqtrade.persistence.cleanup', mock_cleanup) mocker.patch('freqtrade.persistence.cleanup', mock_cleanup)
@ -82,28 +113,28 @@ def test_cleanup(mocker, default_conf, caplog) -> None:
def test_worker_running(mocker, default_conf, caplog) -> None: def test_worker_running(mocker, default_conf, caplog) -> None:
mock_throttle = MagicMock() mock_throttle = MagicMock()
mocker.patch('freqtrade.freqtradebot.FreqtradeBot._throttle', mock_throttle) mocker.patch('freqtrade.worker.Worker._throttle', mock_throttle)
freqtrade = get_patched_freqtradebot(mocker, default_conf) worker = get_patched_worker(mocker, default_conf)
state = freqtrade.worker(old_state=None) state = worker._worker(old_state=None)
assert state is State.RUNNING assert state is State.RUNNING
assert log_has('Changing state to: RUNNING', caplog.record_tuples) assert log_has('Changing state to: RUNNING', caplog.record_tuples)
assert mock_throttle.call_count == 1 assert mock_throttle.call_count == 1
# Check strategy is loaded, and received a dataprovider object # Check strategy is loaded, and received a dataprovider object
assert freqtrade.strategy assert worker.freqtrade.strategy
assert freqtrade.strategy.dp assert worker.freqtrade.strategy.dp
assert isinstance(freqtrade.strategy.dp, DataProvider) assert isinstance(worker.freqtrade.strategy.dp, DataProvider)
def test_worker_stopped(mocker, default_conf, caplog) -> None: def test_worker_stopped(mocker, default_conf, caplog) -> None:
mock_throttle = MagicMock() mock_throttle = MagicMock()
mocker.patch('freqtrade.freqtradebot.FreqtradeBot._throttle', mock_throttle) mocker.patch('freqtrade.worker.Worker._throttle', mock_throttle)
mock_sleep = mocker.patch('time.sleep', return_value=None) mock_sleep = mocker.patch('time.sleep', return_value=None)
freqtrade = get_patched_freqtradebot(mocker, default_conf) worker = get_patched_worker(mocker, default_conf)
freqtrade.state = State.STOPPED worker.state = State.STOPPED
state = freqtrade.worker(old_state=State.RUNNING) state = worker._worker(old_state=State.RUNNING)
assert state is State.STOPPED assert state is State.STOPPED
assert log_has('Changing state to: STOPPED', caplog.record_tuples) assert log_has('Changing state to: STOPPED', caplog.record_tuples)
assert mock_throttle.call_count == 0 assert mock_throttle.call_count == 0
@ -115,17 +146,17 @@ def test_throttle(mocker, default_conf, caplog) -> None:
return 42 return 42
caplog.set_level(logging.DEBUG) caplog.set_level(logging.DEBUG)
freqtrade = get_patched_freqtradebot(mocker, default_conf) worker = get_patched_worker(mocker, default_conf)
start = time.time() start = time.time()
result = freqtrade._throttle(throttled_func, min_secs=0.1) result = worker._throttle(throttled_func, min_secs=0.1)
end = time.time() end = time.time()
assert result == 42 assert result == 42
assert end - start > 0.1 assert end - start > 0.1
assert log_has('Throttling throttled_func for 0.10 seconds', caplog.record_tuples) assert log_has('Throttling throttled_func for 0.10 seconds', caplog.record_tuples)
result = freqtrade._throttle(throttled_func, min_secs=-1) result = worker._throttle(throttled_func, min_secs=-1)
assert result == 42 assert result == 42
@ -133,12 +164,12 @@ def test_throttle_with_assets(mocker, default_conf) -> None:
def throttled_func(nb_assets=-1): def throttled_func(nb_assets=-1):
return nb_assets return nb_assets
freqtrade = get_patched_freqtradebot(mocker, default_conf) worker = get_patched_worker(mocker, default_conf)
result = freqtrade._throttle(throttled_func, min_secs=0.1, nb_assets=666) result = worker._throttle(throttled_func, min_secs=0.1, nb_assets=666)
assert result == 666 assert result == 666
result = freqtrade._throttle(throttled_func, min_secs=0.1) result = worker._throttle(throttled_func, min_secs=0.1)
assert result == -1 assert result == -1
@ -224,7 +255,7 @@ def test_edge_called_in_process(mocker, edge_conf) -> None:
freqtrade = FreqtradeBot(edge_conf) freqtrade = FreqtradeBot(edge_conf)
freqtrade.pairlists._validate_whitelist = _refresh_whitelist freqtrade.pairlists._validate_whitelist = _refresh_whitelist
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
freqtrade._process() freqtrade.process()
assert freqtrade.active_pair_whitelist == ['NEO/BTC', 'LTC/BTC'] assert freqtrade.active_pair_whitelist == ['NEO/BTC', 'LTC/BTC']
@ -551,8 +582,7 @@ def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_ord
freqtrade = FreqtradeBot(default_conf) freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
result = freqtrade.create_trade() assert not freqtrade.create_trade()
assert result is False
def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order, def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order,
@ -573,11 +603,12 @@ def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order,
freqtrade = FreqtradeBot(default_conf) freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
assert freqtrade.create_trade() is False assert not freqtrade.create_trade()
assert freqtrade._get_trade_stake_amount('ETH/BTC') is None assert freqtrade._get_trade_stake_amount('ETH/BTC') is None
def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, markets, mocker) -> None: def test_create_trade_no_pairs_let(default_conf, ticker, limit_buy_order, fee,
markets, mocker, caplog) -> None:
patch_RPCManager(mocker) patch_RPCManager(mocker)
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch.multiple(
@ -589,18 +620,17 @@ def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, marke
) )
default_conf['exchange']['pair_whitelist'] = ["ETH/BTC"] default_conf['exchange']['pair_whitelist'] = ["ETH/BTC"]
default_conf['exchange']['pair_blacklist'] = ["ETH/BTC"]
freqtrade = FreqtradeBot(default_conf) freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
freqtrade.create_trade() assert freqtrade.create_trade()
assert not freqtrade.create_trade()
with pytest.raises(DependencyException, match=r'.*No currency pairs in whitelist.*'): assert log_has("No currency pair in whitelist, but checking to sell open trades.",
freqtrade.create_trade() caplog.record_tuples)
def test_create_trade_no_pairs_after_blacklist(default_conf, ticker, def test_create_trade_no_pairs_in_whitelist(default_conf, ticker, limit_buy_order, fee,
limit_buy_order, fee, markets, mocker) -> None: markets, mocker, caplog) -> None:
patch_RPCManager(mocker) patch_RPCManager(mocker)
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch.multiple(
@ -610,15 +640,12 @@ def test_create_trade_no_pairs_after_blacklist(default_conf, ticker,
get_fee=fee, get_fee=fee,
markets=PropertyMock(return_value=markets) markets=PropertyMock(return_value=markets)
) )
default_conf['exchange']['pair_whitelist'] = ["ETH/BTC"] default_conf['exchange']['pair_whitelist'] = []
default_conf['exchange']['pair_blacklist'] = ["ETH/BTC"]
freqtrade = FreqtradeBot(default_conf) freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
freqtrade.create_trade() assert not freqtrade.create_trade()
assert log_has("Whitelist is empty.", caplog.record_tuples)
with pytest.raises(DependencyException, match=r'.*No currency pairs in whitelist.*'):
freqtrade.create_trade()
def test_create_trade_no_signal(default_conf, fee, mocker) -> None: def test_create_trade_no_signal(default_conf, fee, mocker) -> None:
@ -658,7 +685,7 @@ def test_process_trade_creation(default_conf, ticker, limit_buy_order,
trades = Trade.query.filter(Trade.is_open.is_(True)).all() trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert not trades assert not trades
result = freqtrade._process() result = freqtrade.process()
assert result is True assert result is True
trades = Trade.query.filter(Trade.is_open.is_(True)).all() trades = Trade.query.filter(Trade.is_open.is_(True)).all()
@ -689,10 +716,10 @@ def test_process_exchange_failures(default_conf, ticker, markets, mocker) -> Non
) )
sleep_mock = mocker.patch('time.sleep', side_effect=lambda _: None) sleep_mock = mocker.patch('time.sleep', side_effect=lambda _: None)
freqtrade = FreqtradeBot(default_conf) worker = Worker(args=None, config=default_conf)
patch_get_signal(freqtrade) patch_get_signal(worker.freqtrade)
result = freqtrade._process() result = worker._process()
assert result is False assert result is False
assert sleep_mock.has_calls() assert sleep_mock.has_calls()
@ -706,14 +733,14 @@ def test_process_operational_exception(default_conf, ticker, markets, mocker) ->
markets=PropertyMock(return_value=markets), markets=PropertyMock(return_value=markets),
buy=MagicMock(side_effect=OperationalException) buy=MagicMock(side_effect=OperationalException)
) )
freqtrade = FreqtradeBot(default_conf) worker = Worker(args=None, config=default_conf)
patch_get_signal(freqtrade) patch_get_signal(worker.freqtrade)
assert freqtrade.state == State.RUNNING assert worker.state == State.RUNNING
result = freqtrade._process() result = worker._process()
assert result is False assert result is False
assert freqtrade.state == State.STOPPED assert worker.state == State.STOPPED
assert 'OperationalException' in msg_mock.call_args_list[-1][0][0]['status'] assert 'OperationalException' in msg_mock.call_args_list[-1][0][0]['status']
@ -734,18 +761,18 @@ def test_process_trade_handling(
trades = Trade.query.filter(Trade.is_open.is_(True)).all() trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert not trades assert not trades
result = freqtrade._process() result = freqtrade.process()
assert result is True assert result is True
trades = Trade.query.filter(Trade.is_open.is_(True)).all() trades = Trade.query.filter(Trade.is_open.is_(True)).all()
assert len(trades) == 1 assert len(trades) == 1
result = freqtrade._process() result = freqtrade.process()
assert result is False assert result is False
def test_process_trade_no_whitelist_pair( def test_process_trade_no_whitelist_pair(
default_conf, ticker, limit_buy_order, markets, fee, mocker) -> None: default_conf, ticker, limit_buy_order, markets, fee, mocker) -> None:
""" Test _process with trade not in pair list """ """ Test process with trade not in pair list """
patch_RPCManager(mocker) patch_RPCManager(mocker)
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch.multiple(
@ -782,7 +809,7 @@ def test_process_trade_no_whitelist_pair(
)) ))
assert pair not in freqtrade.active_pair_whitelist assert pair not in freqtrade.active_pair_whitelist
result = freqtrade._process() result = freqtrade.process()
assert pair in freqtrade.active_pair_whitelist assert pair in freqtrade.active_pair_whitelist
# Make sure each pair is only in the list once # Make sure each pair is only in the list once
assert len(freqtrade.active_pair_whitelist) == len(set(freqtrade.active_pair_whitelist)) assert len(freqtrade.active_pair_whitelist) == len(set(freqtrade.active_pair_whitelist))
@ -812,7 +839,7 @@ def test_process_informative_pairs_added(default_conf, ticker, markets, mocker)
freqtrade.strategy.informative_pairs = inf_pairs freqtrade.strategy.informative_pairs = inf_pairs
# patch_get_signal(freqtrade) # patch_get_signal(freqtrade)
freqtrade._process() freqtrade.process()
assert inf_pairs.call_count == 1 assert inf_pairs.call_count == 1
assert refresh_mock.call_count == 1 assert refresh_mock.call_count == 1
assert ("BTC/ETH", "1m") in refresh_mock.call_args[0][0] assert ("BTC/ETH", "1m") in refresh_mock.call_args[0][0]
@ -3083,5 +3110,5 @@ def test_startup_messages(default_conf, mocker):
'config': {'number_assets': 20} 'config': {'number_assets': 20}
} }
mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True)) mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=True))
freqtrade = get_patched_freqtradebot(mocker, default_conf) worker = get_patched_worker(mocker, default_conf)
assert freqtrade.state is State.RUNNING assert worker.state is State.RUNNING

View File

@ -7,8 +7,8 @@ import pytest
from freqtrade import OperationalException from freqtrade import OperationalException
from freqtrade.arguments import Arguments from freqtrade.arguments import Arguments
from freqtrade.freqtradebot import FreqtradeBot from freqtrade.worker import Worker
from freqtrade.main import main, reconfigure from freqtrade.main import main
from freqtrade.state import State from freqtrade.state import State
from freqtrade.tests.conftest import log_has, patch_exchange from freqtrade.tests.conftest import log_has, patch_exchange
@ -43,17 +43,14 @@ def test_main_start_hyperopt(mocker) -> None:
def test_main_fatal_exception(mocker, default_conf, caplog) -> None: def test_main_fatal_exception(mocker, default_conf, caplog) -> None:
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
'freqtrade.freqtradebot.FreqtradeBot', mocker.patch('freqtrade.worker.Worker._worker', MagicMock(side_effect=Exception))
_init_modules=MagicMock(),
worker=MagicMock(side_effect=Exception),
cleanup=MagicMock(),
)
mocker.patch( mocker.patch(
'freqtrade.configuration.Configuration._load_config_file', 'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf lambda *args, **kwargs: default_conf
) )
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example'] args = ['-c', 'config.json.example']
@ -66,17 +63,14 @@ def test_main_fatal_exception(mocker, default_conf, caplog) -> None:
def test_main_keyboard_interrupt(mocker, default_conf, caplog) -> None: def test_main_keyboard_interrupt(mocker, default_conf, caplog) -> None:
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
'freqtrade.freqtradebot.FreqtradeBot', mocker.patch('freqtrade.worker.Worker._worker', MagicMock(side_effect=KeyboardInterrupt))
_init_modules=MagicMock(),
worker=MagicMock(side_effect=KeyboardInterrupt),
cleanup=MagicMock(),
)
mocker.patch( mocker.patch(
'freqtrade.configuration.Configuration._load_config_file', 'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf lambda *args, **kwargs: default_conf
) )
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example'] args = ['-c', 'config.json.example']
@ -89,17 +83,17 @@ def test_main_keyboard_interrupt(mocker, default_conf, caplog) -> None:
def test_main_operational_exception(mocker, default_conf, caplog) -> None: def test_main_operational_exception(mocker, default_conf, caplog) -> None:
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
'freqtrade.freqtradebot.FreqtradeBot', mocker.patch(
_init_modules=MagicMock(), 'freqtrade.worker.Worker._worker',
worker=MagicMock(side_effect=OperationalException('Oh snap!')), MagicMock(side_effect=OperationalException('Oh snap!'))
cleanup=MagicMock(),
) )
mocker.patch( mocker.patch(
'freqtrade.configuration.Configuration._load_config_file', 'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf lambda *args, **kwargs: default_conf
) )
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
args = ['-c', 'config.json.example'] args = ['-c', 'config.json.example']
@ -112,21 +106,18 @@ def test_main_operational_exception(mocker, default_conf, caplog) -> None:
def test_main_reload_conf(mocker, default_conf, caplog) -> None: def test_main_reload_conf(mocker, default_conf, caplog) -> None:
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
'freqtrade.freqtradebot.FreqtradeBot', mocker.patch('freqtrade.worker.Worker._worker', MagicMock(return_value=State.RELOAD_CONF))
_init_modules=MagicMock(),
worker=MagicMock(return_value=State.RELOAD_CONF),
cleanup=MagicMock(),
)
mocker.patch( mocker.patch(
'freqtrade.configuration.Configuration._load_config_file', 'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf lambda *args, **kwargs: default_conf
) )
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
# Raise exception as side effect to avoid endless loop # Raise exception as side effect to avoid endless loop
reconfigure_mock = mocker.patch( reconfigure_mock = mocker.patch(
'freqtrade.main.reconfigure', MagicMock(side_effect=Exception) 'freqtrade.main.Worker._reconfigure', MagicMock(side_effect=Exception)
) )
with pytest.raises(SystemExit): with pytest.raises(SystemExit):
@ -138,19 +129,21 @@ def test_main_reload_conf(mocker, default_conf, caplog) -> None:
def test_reconfigure(mocker, default_conf) -> None: def test_reconfigure(mocker, default_conf) -> None:
patch_exchange(mocker) patch_exchange(mocker)
mocker.patch.multiple( mocker.patch('freqtrade.freqtradebot.FreqtradeBot.cleanup', MagicMock())
'freqtrade.freqtradebot.FreqtradeBot', mocker.patch(
_init_modules=MagicMock(), 'freqtrade.worker.Worker._worker',
worker=MagicMock(side_effect=OperationalException('Oh snap!')), MagicMock(side_effect=OperationalException('Oh snap!'))
cleanup=MagicMock(),
) )
mocker.patch( mocker.patch(
'freqtrade.configuration.Configuration._load_config_file', 'freqtrade.configuration.Configuration._load_config_file',
lambda *args, **kwargs: default_conf lambda *args, **kwargs: default_conf
) )
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
mocker.patch('freqtrade.freqtradebot.persistence.init', MagicMock())
freqtrade = FreqtradeBot(default_conf) args = Arguments(['-c', 'config.json.example'], '').get_parsed_arg()
worker = Worker(args=args, config=default_conf)
freqtrade = worker.freqtrade
# Renew mock to return modified data # Renew mock to return modified data
conf = deepcopy(default_conf) conf = deepcopy(default_conf)
@ -160,11 +153,10 @@ def test_reconfigure(mocker, default_conf) -> None:
lambda *args, **kwargs: conf lambda *args, **kwargs: conf
) )
worker._config = conf
# reconfigure should return a new instance # reconfigure should return a new instance
freqtrade2 = reconfigure( worker._reconfigure()
freqtrade, freqtrade2 = worker.freqtrade
Arguments(['-c', 'config.json.example'], '').get_parsed_arg()
)
# Verify we have a new instance with the new config # Verify we have a new instance with the new config
assert freqtrade is not freqtrade2 assert freqtrade is not freqtrade2

188
freqtrade/worker.py Executable file
View File

@ -0,0 +1,188 @@
"""
Main Freqtrade worker class.
"""
import logging
import time
import traceback
from argparse import Namespace
from typing import Any, Callable, Optional
import sdnotify
from freqtrade import (constants, OperationalException, TemporaryError,
__version__)
from freqtrade.configuration import Configuration
from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType
logger = logging.getLogger(__name__)
class Worker(object):
"""
Freqtradebot worker class
"""
def __init__(self, args: Namespace, config=None) -> None:
"""
Init all variables and objects the bot needs to work
"""
logger.info('Starting worker %s', __version__)
self._args = args
self._config = config
self._init(False)
# Tell systemd that we completed initialization phase
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def _init(self, reconfig: bool):
"""
Also called from the _reconfigure() method (with reconfig=True).
"""
if reconfig or self._config is None:
# Load configuration
self._config = Configuration(self._args, None).get_config()
# Init the instance of the bot
self.freqtrade = FreqtradeBot(self._config)
self._throttle_secs = self._config.get('internals', {}).get(
'process_throttle_secs',
constants.PROCESS_THROTTLE_SECS
)
self._sd_notify = sdnotify.SystemdNotifier() if \
self._config.get('internals', {}).get('sd_notify', False) else None
@property
def state(self) -> State:
return self.freqtrade.state
@state.setter
def state(self, value: State):
self.freqtrade.state = value
def run(self):
state = None
while True:
state = self._worker(old_state=state)
if state == State.RELOAD_CONF:
self.freqtrade = self._reconfigure()
def _worker(self, old_state: State, throttle_secs: Optional[float] = None) -> State:
"""
Trading routine that must be run at each loop
:param old_state: the previous service state from the previous call
:return: current service state
"""
state = self.freqtrade.state
if throttle_secs is None:
throttle_secs = self._throttle_secs
# Log state transition
if state != old_state:
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'{state.name.lower()}'
})
logger.info('Changing state to: %s', state.name)
if state == State.RUNNING:
self.freqtrade.rpc.startup_messages(self._config, self.freqtrade.pairlists)
if state == State.STOPPED:
# Ping systemd watchdog before sleeping in the stopped state
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: STOPPED.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: STOPPED.")
time.sleep(throttle_secs)
elif state == State.RUNNING:
# Ping systemd watchdog before throttling
if self._sd_notify:
logger.debug("sd_notify: WATCHDOG=1\\nSTATUS=State: RUNNING.")
self._sd_notify.notify("WATCHDOG=1\nSTATUS=State: RUNNING.")
self._throttle(func=self._process, min_secs=throttle_secs)
return state
def _throttle(self, func: Callable[..., Any], min_secs: float, *args, **kwargs) -> Any:
"""
Throttles the given callable that it
takes at least `min_secs` to finish execution.
:param func: Any callable
:param min_secs: minimum execution time in seconds
:return: Any
"""
start = time.time()
result = func(*args, **kwargs)
end = time.time()
duration = max(min_secs - (end - start), 0.0)
logger.debug('Throttling %s for %.2f seconds', func.__name__, duration)
time.sleep(duration)
return result
def _process(self) -> bool:
state_changed = False
try:
state_changed = self.freqtrade.process()
except TemporaryError as error:
logger.warning(f"Error: {error}, retrying in {constants.RETRY_TIMEOUT} seconds...")
time.sleep(constants.RETRY_TIMEOUT)
except OperationalException:
tb = traceback.format_exc()
hint = 'Issue `/start` if you think it is safe to restart.'
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': f'OperationalException:\n```\n{tb}```{hint}'
})
logger.exception('OperationalException. Stopping trader ...')
self.freqtrade.state = State.STOPPED
# TODO: The return value of _process() is not used apart tests
# and should (could) be eliminated later. See PR #1689.
# state_changed = True
return state_changed
def _reconfigure(self):
"""
Cleans up current freqtradebot instance, reloads the configuration and
replaces it with the new instance
"""
# Tell systemd that we initiated reconfiguration
if self._sd_notify:
logger.debug("sd_notify: RELOADING=1")
self._sd_notify.notify("RELOADING=1")
# Clean up current freqtrade modules
self.freqtrade.cleanup()
# Load and validate config and create new instance of the bot
self._init(True)
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'config reloaded'
})
# Tell systemd that we completed reconfiguration
if self._sd_notify:
logger.debug("sd_notify: READY=1")
self._sd_notify.notify("READY=1")
def exit(self):
# Tell systemd that we are exiting now
if self._sd_notify:
logger.debug("sd_notify: STOPPING=1")
self._sd_notify.notify("STOPPING=1")
if self.freqtrade:
self.freqtrade.rpc.send_msg({
'type': RPCMessageType.STATUS_NOTIFICATION,
'status': 'process died'
})
self.freqtrade.cleanup()

View File

@ -3,11 +3,12 @@ nav:
- About: index.md - About: index.md
- Installation: installation.md - Installation: installation.md
- Configuration: configuration.md - Configuration: configuration.md
- Start the bot: bot-usage.md
- Stoploss: stoploss.md
- Custom Strategy: bot-optimization.md - Custom Strategy: bot-optimization.md
- Telegram: telegram-usage.md - Stoploss: stoploss.md
- Web Hook: webhook-config.md - Start the bot: bot-usage.md
- Control the bot:
- Telegram: telegram-usage.md
- Web Hook: webhook-config.md
- Backtesting: backtesting.md - Backtesting: backtesting.md
- Hyperopt: hyperopt.md - Hyperopt: hyperopt.md
- Edge positioning: edge.md - Edge positioning: edge.md

View File

@ -9,4 +9,4 @@ pytest-mock==1.10.3
pytest-asyncio==0.10.0 pytest-asyncio==0.10.0
pytest-cov==2.6.1 pytest-cov==2.6.1
coveralls==1.7.0 coveralls==1.7.0
mypy==0.670 mypy==0.700

View File

@ -1,5 +1,5 @@
ccxt==1.18.425 ccxt==1.18.432
SQLAlchemy==1.3.1 SQLAlchemy==1.3.2
python-telegram-bot==11.1.0 python-telegram-bot==11.1.0
arrow==0.13.1 arrow==0.13.1
cachetools==3.1.0 cachetools==3.1.0