adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'feat/short' into funding-fee

Browse Source
This commit is contained in:
Sam Germain 2021-10-03 02:15:54 -06:00
commit 70db228f24
40 changed files with 1897 additions and 1323 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
config_examples/config_full.example.json
View File

@ -149,7 +149,9 @@
},
"sell_fill": "on",
"buy_cancel": "on",
"sell_cancel": "on"
"sell_cancel": "on",
"protection_trigger": "off",
"protection_trigger_global": "on"
},
"reload": true,
"balance_dust_level": 0.01

55
docs/data-download.md
View File

@ -204,6 +204,61 @@ It'll also remove original jsongz data files (`--erase` parameter).
freqtrade convert-trade-data --format-from jsongz --format-to json --datadir ~/.freqtrade/data/kraken --erase
```
### Sub-command trades to ohlcv
When you need to use `--dl-trades` (kraken only) to download data, conversion of trades data to ohlcv data is the last step.
This command will allow you to repeat this last step for additional timeframes without re-downloading the data.
```
usage: freqtrade trades-to-ohlcv [-h] [-v] [--logfile FILE] [-V] [-c PATH]
[-d PATH] [--userdir PATH]
[-p PAIRS [PAIRS ...]]
[-t {1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} [{1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} ...]]
[--exchange EXCHANGE]
[--data-format-ohlcv {json,jsongz,hdf5}]
[--data-format-trades {json,jsongz,hdf5}]
optional arguments:
-h, --help show this help message and exit
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Limit command to these pairs. Pairs are space-
separated.
-t {1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} [{1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} ...], --timeframes {1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} [{1m,3m,5m,15m,30m,1h,2h,4h,6h,8h,12h,1d,3d,1w,2w,1M,1y} ...]
Specify which tickers to download. Space-separated
list. Default: `1m 5m`.
--exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
config is provided.
--data-format-ohlcv {json,jsongz,hdf5}
Storage format for downloaded candle (OHLCV) data.
(default: `json`).
--data-format-trades {json,jsongz,hdf5}
Storage format for downloaded trades data. (default:
`jsongz`).
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
-c PATH, --config PATH
Specify configuration file (default:
`userdir/config.json` or `config.json` whichever
exists). Multiple --config options may be used. Can be
set to `-` to read config from stdin.
-d PATH, --datadir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
```
#### Example trade-to-ohlcv conversion
``` bash
freqtrade trades-to-ohlcv --exchange kraken -t 5m 1h 1d --pairs BTC/EUR ETH/EUR
```
### Sub-command list-data
You can get a list of downloaded data using the `list-data` sub-command.

19
docs/docker_quickstart.md
View File

@ -109,6 +109,7 @@ All freqtrade arguments will be available by running `docker-compose run --rm fr
!!! Warning "`docker-compose` for trade commands"
Trade commands (`freqtrade trade <...>`) should not be ran via `docker-compose run` - but should use `docker-compose up -d` instead.
This makes sure that the container is properly started (including port forwardings) and will make sure that the container will restart after a system reboot.
If you intend to use freqUI, please also ensure to adjust the [configuration accordingly](rest-api.md#configuration-with-docker), otherwise the UI will not be available.
!!! Note "`docker-compose run --rm`"
Including `--rm` will remove the container after completion, and is highly recommended for all modes except trading mode (running with `freqtrade trade` command).
@ -149,6 +150,24 @@ You'll then also need to modify the `docker-compose.yml` file and uncomment the
You can then run `docker-compose build` to build the docker image, and run it using the commands described above.
### Troubleshooting
#### Docker on Windows
* Error: `"Timestamp for this request is outside of the recvWindow."`
* The market api requests require a synchronized clock but the time in the docker container shifts a bit over time into the past.
To fix this issue temporarily you need to run `wsl --shutdown` and restart docker again (a popup on windows 10 will ask you to do so).
A permanent solution is either to host the docker container on a linux host or restart the wsl from time to time with the scheduler.
```
taskkill /IM "Docker Desktop.exe" /F
wsl --shutdown
start "" "C:\Program Files\Docker\Docker\Docker Desktop.exe"
```
!!! Warning
Due to the above, we do not recommend the usage of docker on windows for production setups, but only for experimentation, datadownload and backtesting.
Best use a linux-VPS for running freqtrade reliably.
## Plotting with docker-compose
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your docker-compose.yml file.

2
docs/requirements-docs.txt
View File

@ -1,4 +1,4 @@
mkdocs==1.2.2
mkdocs-material==7.2.6
mkdocs-material==7.3.0
mdx_truly_sane_lists==1.2
pymdown-extensions==8.2

30
docs/strategy-advanced.md
View File

@ -731,3 +731,33 @@ The variable 'content', will contain the strategy file in a BASE64 encoded form.
```
Please ensure that 'NameOfStrategy' is identical to the strategy name!
## Performance warning
When executing a strategy, one can sometimes be greeted by the following in the logs
> PerformanceWarning: DataFrame is highly fragmented.
This is a warning from [`pandas`](https://github.com/pandas-dev/pandas) and as the warning continues to say:
use `pd.concat(axis=1)`.
This can have slight performance implications, which are usually only visible during hyperopt (when optimizing an indicator).
For example:
```python
for val in self.buy_ema_short.range:
dataframe[f'ema_short_{val}'] = ta.EMA(dataframe, timeperiod=val)
```
should be rewritten to
```python
frames = [dataframe]
for val in self.buy_ema_short.range:
frames.append({
f'ema_short_{val}': ta.EMA(dataframe, timeperiod=val)
})
# Append columns to existing dataframe
merged_frame = pd.concat(frames, axis=1)
```

12
docs/strategy-customization.md
View File

@ -122,6 +122,16 @@ def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame
Look into the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_strategy.py).
Then uncomment indicators you need.
#### Indicator libraries
Out of the box, freqtrade installs the following technical libraries:
* [ta-lib](http://mrjbq7.github.io/ta-lib/)
* [pandas-ta](https://twopirllc.github.io/pandas-ta/)
* [technical](https://github.com/freqtrade/technical/)
Additional technical libraries can be installed as necessary, or custom indicators may be written / invented by the strategy author.
### Strategy startup period
Most indicators have an instable startup period, in which they are either not available, or the calculation is incorrect. This can lead to inconsistencies, since Freqtrade does not know how long this instable period should be.
@ -942,6 +952,8 @@ Printing more than a few rows is also possible (simply use `print(dataframe)` i
## Common mistakes when developing strategies
### Peeking into the future while backtesting
Backtesting analyzes the whole time-range at once for performance reasons. Because of this, strategy authors need to make sure that strategies do not look-ahead into the future.
This is a common pain-point, which can cause huge differences between backtesting and dry/live run methods, since they all use data which is not available during dry/live runs, so these strategies will perform well during backtesting, but will fail / perform badly in real conditions.

5
docs/telegram-usage.md
View File

@ -93,7 +93,9 @@ Example configuration showing the different settings:
"buy_cancel": "silent",
"sell_cancel": "on",
"buy_fill": "off",
"sell_fill": "off"
"sell_fill": "off",
"protection_trigger": "off",
"protection_trigger_global": "on"
},
"reload": true,
"balance_dust_level": 0.01
@ -103,6 +105,7 @@ Example configuration showing the different settings:
`buy` notifications are sent when the order is placed, while `buy_fill` notifications are sent when the order is filled on the exchange.
`sell` notifications are sent when the order is placed, while `sell_fill` notifications are sent when the order is filled on the exchange.
`*_fill` notifications are off by default and must be explicitly enabled.
`protection_trigger` notifications are sent when a protection triggers and `protection_trigger_global` notifications trigger when global protections are triggered.
`balance_dust_level` will define what the `/balance` command takes as "dust" - Currencies with a balance below this will be shown.

4
freqtrade/commands/__init__.py
View File

@ -8,8 +8,8 @@ Note: Be careful with file-scoped imports in these subfiles.
"""
from freqtrade.commands.arguments import Arguments
from freqtrade.commands.build_config_commands import start_new_config
from freqtrade.commands.data_commands import (start_convert_data, start_download_data,
start_list_data)
from freqtrade.commands.data_commands import (start_convert_data, start_convert_trades,
start_download_data, start_list_data)
from freqtrade.commands.deploy_commands import (start_create_userdir, start_install_ui,
start_new_strategy)
from freqtrade.commands.hyperopt_commands import start_hyperopt_list, start_hyperopt_show

29
freqtrade/commands/arguments.py
View File

@ -58,6 +58,8 @@ ARGS_BUILD_STRATEGY = ["user_data_dir", "strategy", "template"]
ARGS_CONVERT_DATA = ["pairs", "format_from", "format_to", "erase"]
ARGS_CONVERT_DATA_OHLCV = ARGS_CONVERT_DATA + ["timeframes"]
ARGS_CONVERT_TRADES = ["pairs", "timeframes", "exchange", "dataformat_ohlcv", "dataformat_trades"]
ARGS_LIST_DATA = ["exchange", "dataformat_ohlcv", "pairs"]
ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "new_pairs_days", "timerange",
@ -91,7 +93,7 @@ ARGS_HYPEROPT_SHOW = ["hyperopt_list_best", "hyperopt_list_profitable", "hyperop
NO_CONF_REQURIED = ["convert-data", "convert-trade-data", "download-data", "list-timeframes",
"list-markets", "list-pairs", "list-strategies", "list-data",
"hyperopt-list", "hyperopt-show",
"plot-dataframe", "plot-profit", "show-trades"]
"plot-dataframe", "plot-profit", "show-trades", "trades-to-ohlcv"]
NO_CONF_ALLOWED = ["create-userdir", "list-exchanges", "new-strategy"]
@ -169,14 +171,14 @@ class Arguments:
self.parser = argparse.ArgumentParser(description='Free, open source crypto trading bot')
self._build_args(optionlist=['version'], parser=self.parser)
from freqtrade.commands import (start_backtesting, start_convert_data, start_create_userdir,
start_download_data, start_edge, start_hyperopt,
start_hyperopt_list, start_hyperopt_show, start_install_ui,
start_list_data, start_list_exchanges, start_list_markets,
start_list_strategies, start_list_timeframes,
start_new_config, start_new_strategy, start_plot_dataframe,
start_plot_profit, start_show_trades, start_test_pairlist,
start_trading, start_webserver)
from freqtrade.commands import (start_backtesting, start_convert_data, start_convert_trades,
start_create_userdir, start_download_data, start_edge,
start_hyperopt, start_hyperopt_list, start_hyperopt_show,
start_install_ui, start_list_data, start_list_exchanges,
start_list_markets, start_list_strategies,
start_list_timeframes, start_new_config, start_new_strategy,
start_plot_dataframe, start_plot_profit, start_show_trades,
start_test_pairlist, start_trading, start_webserver)
subparsers = self.parser.add_subparsers(dest='command',
# Use custom message when no subhandler is added
@ -236,6 +238,15 @@ class Arguments:
convert_trade_data_cmd.set_defaults(func=partial(start_convert_data, ohlcv=False))
self._build_args(optionlist=ARGS_CONVERT_DATA, parser=convert_trade_data_cmd)
# Add trades-to-ohlcv subcommand
convert_trade_data_cmd = subparsers.add_parser(
'trades-to-ohlcv',
help='Convert trade data to OHLCV data.',
parents=[_common_parser],
)
convert_trade_data_cmd.set_defaults(func=start_convert_trades)
self._build_args(optionlist=ARGS_CONVERT_TRADES, parser=convert_trade_data_cmd)
# Add list-data subcommand
list_data_cmd = subparsers.add_parser(
'list-data',

4
freqtrade/commands/cli_options.py
View File

@ -381,12 +381,12 @@ AVAILABLE_CLI_OPTIONS = {
),
"dataformat_ohlcv": Arg(
'--data-format-ohlcv',
help='Storage format for downloaded candle (OHLCV) data. (default: `%(default)s`).',
help='Storage format for downloaded candle (OHLCV) data. (default: `json`).',
choices=constants.AVAILABLE_DATAHANDLERS,
),
"dataformat_trades": Arg(
'--data-format-trades',
help='Storage format for downloaded trades data. (default: `%(default)s`).',
help='Storage format for downloaded trades data. (default: `jsongz`).',
choices=constants.AVAILABLE_DATAHANDLERS,
),
"exchange": Arg(

35
freqtrade/commands/data_commands.py
View File

@ -89,6 +89,41 @@ def start_download_data(args: Dict[str, Any]) -> None:
f"on exchange {exchange.name}.")
def start_convert_trades(args: Dict[str, Any]) -> None:
config = setup_utils_configuration(args, RunMode.UTIL_EXCHANGE)
timerange = TimeRange()
# Remove stake-currency to skip checks which are not relevant for datadownload
config['stake_currency'] = ''
if 'pairs' not in config:
raise OperationalException(
"Downloading data requires a list of pairs. "
"Please check the documentation on how to configure this.")
# Init exchange
exchange = ExchangeResolver.load_exchange(config['exchange']['name'], config, validate=False)
# Manual validations of relevant settings
if not config['exchange'].get('skip_pair_validation', False):
exchange.validate_pairs(config['pairs'])
expanded_pairs = expand_pairlist(config['pairs'], list(exchange.markets))
logger.info(f"About to Convert pairs: {expanded_pairs}, "
f"intervals: {config['timeframes']} to {config['datadir']}")
for timeframe in config['timeframes']:
exchange.validate_timeframes(timeframe)
# Convert downloaded trade data to different timeframes
convert_trades_to_ohlcv(
pairs=expanded_pairs, timeframes=config['timeframes'],
datadir=config['datadir'], timerange=timerange, erase=bool(config.get('erase')),
data_format_ohlcv=config['dataformat_ohlcv'],
data_format_trades=config['dataformat_trades'],
)
def start_convert_data(args: Dict[str, Any], ohlcv: bool = True) -> None:
"""
Convert data from one format to another

11
freqtrade/constants.py
View File

@ -110,7 +110,7 @@ CONF_SCHEMA = {
},
'tradable_balance_ratio': {
'type': 'number',
'minimum': 0.1,
'minimum': 0.0,
'maximum': 1,
'default': 0.99
},
@ -284,6 +284,15 @@ CONF_SCHEMA = {
'enum': TELEGRAM_SETTING_OPTIONS,
'default': 'off'
},
'protection_trigger': {
'type': 'string',
'enum': TELEGRAM_SETTING_OPTIONS,
'default': 'off'
},
'protection_trigger_global': {
'type': 'string',
'enum': TELEGRAM_SETTING_OPTIONS,
},
}
},
'reload': {'type': 'boolean'},

2
freqtrade/data/dataprovider.py
View File

@ -149,6 +149,8 @@ class DataProvider:
Clear pair dataframe cache.
"""
self.__cached_pairs = {}
self.__cached_pairs_backtesting = {}
self.__slice_index = 0
# Exchange functions

2
freqtrade/enums/rpcmessagetype.py
View File

@ -11,6 +11,8 @@ class RPCMessageType(Enum):
SELL = 'sell'
SELL_FILL = 'sell_fill'
SELL_CANCEL = 'sell_cancel'
PROTECTION_TRIGGER = 'protection_trigger'
PROTECTION_TRIGGER_GLOBAL = 'protection_trigger_global'
def __repr__(self):
return self.value

24
freqtrade/freqtradebot.py
View File

@ -152,7 +152,7 @@ class FreqtradeBot(LoggingMixin):
# Only update open orders on startup
# This will update the database after the initial migration
self.update_open_orders()
self.startup_update_open_orders()
def process(self) -> None:
"""
@ -259,7 +259,7 @@ class FreqtradeBot(LoggingMixin):
)
trade.funding_fees = funding_fees
def update_open_orders(self):
def startup_update_open_orders(self):
"""
Updates open orders based on order list kept in the database.
Mainly updates the state of orders - but may also close trades
@ -1275,7 +1275,7 @@ class FreqtradeBot(LoggingMixin):
'exchange': trade.exchange.capitalize(),
'pair': trade.pair,
'gain': gain,
'limit': profit_rate,
'limit': profit_rate or 0,
'order_type': order_type,
'amount': trade.amount,
'open_rate': trade.open_rate,
@ -1284,7 +1284,7 @@ class FreqtradeBot(LoggingMixin):
'profit_ratio': profit_ratio,
'sell_reason': trade.sell_reason,
'open_date': trade.open_date,
'close_date': trade.close_date,
'close_date': trade.close_date or datetime.now(timezone.utc),
'stake_currency': self.config['stake_currency'],
'fiat_currency': self.config.get('fiat_display_currency', None),
'reason': reason,
@ -1350,8 +1350,7 @@ class FreqtradeBot(LoggingMixin):
if not trade.is_open:
if not stoploss_order and not trade.open_order_id:
self._notify_exit(trade, '', True)
self.protections.stop_per_pair(trade.pair)
self.protections.global_stop()
self.handle_protections(trade.pair)
self.wallets.update()
elif not trade.open_order_id:
# Buy fill
@ -1359,6 +1358,19 @@ class FreqtradeBot(LoggingMixin):
return False
def handle_protections(self, pair: str) -> None:
prot_trig = self.protections.stop_per_pair(pair)
if prot_trig:
msg = {'type': RPCMessageType.PROTECTION_TRIGGER, }
msg.update(prot_trig.to_json())
self.rpc.send_msg(msg)
prot_trig_glb = self.protections.global_stop()
if prot_trig_glb:
msg = {'type': RPCMessageType.PROTECTION_TRIGGER_GLOBAL, }
msg.update(prot_trig_glb.to_json())
self.rpc.send_msg(msg)
def apply_fee_conditional(self, trade: Trade, trade_base_currency: str,
amount: float, fee_abs: float) -> float:
"""

59
freqtrade/optimize/backtesting.py
View File

@ -87,18 +87,7 @@ class Backtesting:
"configuration or as cli argument `--timeframe 5m`")
self.timeframe = str(self.config.get('timeframe'))
self.timeframe_min = timeframe_to_minutes(self.timeframe)
# Load detail timeframe if specified
self.timeframe_detail = str(self.config.get('timeframe_detail', ''))
if self.timeframe_detail:
self.timeframe_detail_min = timeframe_to_minutes(self.timeframe_detail)
if self.timeframe_min <= self.timeframe_detail_min:
raise OperationalException(
"Detail timeframe must be smaller than strategy timeframe.")
else:
self.timeframe_detail_min = 0
self.detail_data: Dict[str, DataFrame] = {}
self.init_backtest_detail()
self.pairlists = PairListManager(self.exchange, self.config)
if 'VolumePairList' in self.pairlists.name_list:
raise OperationalException("VolumePairList not allowed for backtesting.")
@ -121,14 +110,6 @@ class Backtesting:
else:
self.fee = self.exchange.get_fee(symbol=self.pairlists.whitelist[0])
Trade.use_db = False
Trade.reset_trades()
PairLocks.timeframe = self.config['timeframe']
PairLocks.use_db = False
PairLocks.reset_locks()
self.wallets = Wallets(self.config, self.exchange, log=False)
self.timerange = TimeRange.parse_timerange(
None if self.config.get('timerange') is None else str(self.config.get('timerange')))
@ -144,6 +125,7 @@ class Backtesting:
self.progress = BTProgress()
self.abort = False
self.init_backtest()
def __del__(self):
self.cleanup()
@ -153,6 +135,28 @@ class Backtesting:
PairLocks.use_db = True
Trade.use_db = True
def init_backtest_detail(self):
# Load detail timeframe if specified
self.timeframe_detail = str(self.config.get('timeframe_detail', ''))
if self.timeframe_detail:
self.timeframe_detail_min = timeframe_to_minutes(self.timeframe_detail)
if self.timeframe_min <= self.timeframe_detail_min:
raise OperationalException(
"Detail timeframe must be smaller than strategy timeframe.")
else:
self.timeframe_detail_min = 0
self.detail_data: Dict[str, DataFrame] = {}
def init_backtest(self):
self.prepare_backtest(False)
self.wallets = Wallets(self.config, self.exchange, log=False)
self.progress = BTProgress()
self.abort = False
def _set_strategy(self, strategy: IStrategy):
"""
Load strategy into backtesting
@ -232,7 +236,8 @@ class Backtesting:
Trade.reset_trades()
self.rejected_trades = 0
self.dataprovider.clear_cache()
self._load_protections(self.strategy)
if enable_protections:
self._load_protections(self.strategy)
def check_abort(self):
"""
@ -365,7 +370,7 @@ class Backtesting:
trade, sell_row[OPEN_IDX], sell_candle_time, # type: ignore
enter=enter, exit_=exit_,
low=sell_row[LOW_IDX], high=sell_row[HIGH_IDX]
)
)
if sell.sell_flag:
trade.close_date = sell_candle_time
@ -397,14 +402,14 @@ class Backtesting:
detail_data = detail_data.loc[
(detail_data['date'] >= sell_candle_time) &
(detail_data['date'] < sell_candle_end)
]
].copy()
if len(detail_data) == 0:
# Fall back to "regular" data if no detail data was found for this candle
return self._get_sell_trade_entry_for_candle(trade, sell_row)
detail_data['enter_long'] = sell_row[LONG_IDX]
detail_data['exit_long'] = sell_row[ELONG_IDX]
detail_data['enter_short'] = sell_row[SHORT_IDX]
detail_data['exit_short'] = sell_row[ESHORT_IDX]
detail_data.loc[:, 'enter_long'] = sell_row[LONG_IDX]
detail_data.loc[:, 'exit_long'] = sell_row[ELONG_IDX]
detail_data.loc[:, 'enter_long'] = sell_row[LONG_IDX]
detail_data.loc[:, 'exit_long'] = sell_row[ELONG_IDX]
headers = ['date', 'open', 'high', 'low', 'close', 'enter_long', 'exit_long',
'enter_short', 'exit_short']
for det_row in detail_data[headers].values.tolist():

4
freqtrade/persistence/pairlock_middleware.py
View File

@ -30,7 +30,8 @@ class PairLocks():
PairLocks.locks = []
@staticmethod
def lock_pair(pair: str, until: datetime, reason: str = None, *, now: datetime = None) -> None:
def lock_pair(pair: str, until: datetime, reason: str = None, *,
now: datetime = None) -> PairLock:
"""
Create PairLock from now to "until".
Uses database by default, unless PairLocks.use_db is set to False,
@ -52,6 +53,7 @@ class PairLocks():
PairLock.query.session.commit()
else:
PairLocks.locks.append(lock)
return lock
@staticmethod
def get_pair_locks(pair: Optional[str], now: Optional[datetime] = None) -> List[PairLock]:

23
freqtrade/plugins/protectionmanager.py
View File

@ -6,6 +6,7 @@ from datetime import datetime, timezone
from typing import Dict, List, Optional
from freqtrade.persistence import PairLocks
from freqtrade.persistence.models import PairLock
from freqtrade.plugins.protections import IProtection
from freqtrade.resolvers import ProtectionResolver
@ -43,30 +44,28 @@ class ProtectionManager():
"""
return [{p.name: p.short_desc()} for p in self._protection_handlers]
def global_stop(self, now: Optional[datetime] = None) -> bool:
def global_stop(self, now: Optional[datetime] = None) -> Optional[PairLock]:
if not now:
now = datetime.now(timezone.utc)
result = False
result = None
for protection_handler in self._protection_handlers:
if protection_handler.has_global_stop:
result, until, reason = protection_handler.global_stop(now)
lock, until, reason = protection_handler.global_stop(now)
# Early stopping - first positive result blocks further trades
if result and until:
if lock and until:
if not PairLocks.is_global_lock(until):
PairLocks.lock_pair('*', until, reason, now=now)
result = True
result = PairLocks.lock_pair('*', until, reason, now=now)
return result
def stop_per_pair(self, pair, now: Optional[datetime] = None) -> bool:
def stop_per_pair(self, pair, now: Optional[datetime] = None) -> Optional[PairLock]:
if not now:
now = datetime.now(timezone.utc)
result = False
result = None
for protection_handler in self._protection_handlers:
if protection_handler.has_local_stop:
result, until, reason = protection_handler.stop_per_pair(pair, now)
if result and until:
lock, until, reason = protection_handler.stop_per_pair(pair, now)
if lock and until:
if not PairLocks.is_pair_locked(pair, until):
PairLocks.lock_pair(pair, until, reason, now=now)
result = True
result = PairLocks.lock_pair(pair, until, reason, now=now)
return result

23
freqtrade/rpc/api_server/api_backtest.py
View File

@ -4,6 +4,7 @@ from copy import deepcopy
from fastapi import APIRouter, BackgroundTasks, Depends
from freqtrade.configuration.config_validation import validate_config_consistency
from freqtrade.enums import BacktestState
from freqtrade.exceptions import DependencyException
from freqtrade.rpc.api_server.api_schemas import BacktestRequest, BacktestResponse
@ -42,38 +43,40 @@ async def api_start_backtest(bt_settings: BacktestRequest, background_tasks: Bac
# Reload strategy
lastconfig = ApiServer._bt_last_config
strat = StrategyResolver.load_strategy(btconfig)
validate_config_consistency(btconfig)
if (
not ApiServer._bt
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timeframe_detail') != btconfig.get('timeframe_detail')
or lastconfig.get('dry_run_wallet') != btconfig.get('dry_run_wallet', 0)
or lastconfig.get('timerange') != btconfig['timerange']
):
from freqtrade.optimize.backtesting import Backtesting
ApiServer._bt = Backtesting(btconfig)
if ApiServer._bt.timeframe_detail:
ApiServer._bt.load_bt_data_detail()
else:
ApiServer._bt.config = btconfig
ApiServer._bt.init_backtest()
# Only reload data if timeframe changed.
if (
not ApiServer._bt_data
or not ApiServer._bt_timerange
or lastconfig.get('stake_amount') != btconfig.get('stake_amount')
or lastconfig.get('enable_protections') != btconfig.get('enable_protections')
or lastconfig.get('protections') != btconfig.get('protections', [])
or lastconfig.get('timeframe') != strat.timeframe
or lastconfig.get('timerange') != btconfig['timerange']
):
lastconfig['timerange'] = btconfig['timerange']
lastconfig['protections'] = btconfig.get('protections', [])
lastconfig['enable_protections'] = btconfig.get('enable_protections')
lastconfig['dry_run_wallet'] = btconfig.get('dry_run_wallet')
lastconfig['timeframe'] = strat.timeframe
ApiServer._bt_data, ApiServer._bt_timerange = ApiServer._bt.load_bt_data()
lastconfig['timerange'] = btconfig['timerange']
lastconfig['timeframe'] = strat.timeframe
lastconfig['protections'] = btconfig.get('protections', [])
lastconfig['enable_protections'] = btconfig.get('enable_protections')
lastconfig['dry_run_wallet'] = btconfig.get('dry_run_wallet')
ApiServer._bt.abort = False
min_date, max_date = ApiServer._bt.backtest_one_strategy(
strat, ApiServer._bt_data, ApiServer._bt_timerange)
ApiServer._bt.results = generate_backtest_stats(
ApiServer._bt_data, ApiServer._bt.all_results,
min_date=min_date, max_date=max_date)

76
freqtrade/rpc/telegram.py
View File

@ -260,6 +260,50 @@ class Telegram(RPCHandler):
return message
def compose_message(self, msg: Dict[str, Any], msg_type: RPCMessageType) -> str:
if msg_type == RPCMessageType.BUY:
message = self._format_buy_msg(msg)
elif msg_type in (RPCMessageType.BUY_CANCEL, RPCMessageType.SELL_CANCEL):
msg['message_side'] = 'buy' if msg_type == RPCMessageType.BUY_CANCEL else 'sell'
message = ("\N{WARNING SIGN} *{exchange}:* "
"Cancelling open {message_side} Order for {pair} (#{trade_id}). "
"Reason: {reason}.".format(**msg))
elif msg_type == RPCMessageType.BUY_FILL:
message = ("\N{LARGE CIRCLE} *{exchange}:* "
"Buy order for {pair} (#{trade_id}) filled "
"for {open_rate}.".format(**msg))
elif msg_type == RPCMessageType.SELL_FILL:
message = ("\N{LARGE CIRCLE} *{exchange}:* "
"Sell order for {pair} (#{trade_id}) filled "
"for {close_rate}.".format(**msg))
elif msg_type == RPCMessageType.SELL:
message = self._format_sell_msg(msg)
elif msg_type == RPCMessageType.PROTECTION_TRIGGER:
message = (
"*Protection* triggered due to {reason}. "
"`{pair}` will be locked until `{lock_end_time}`."
).format(**msg)
elif msg_type == RPCMessageType.PROTECTION_TRIGGER_GLOBAL:
message = (
"*Protection* triggered due to {reason}. "
"*All pairs* will be locked until `{lock_end_time}`."
).format(**msg)
elif msg_type == RPCMessageType.STATUS:
message = '*Status:* `{status}`'.format(**msg)
elif msg_type == RPCMessageType.WARNING:
message = '\N{WARNING SIGN} *Warning:* `{status}`'.format(**msg)
elif msg_type == RPCMessageType.STARTUP:
message = '{status}'.format(**msg)
else:
raise NotImplementedError('Unknown message type: {}'.format(msg_type))
return message
def send_msg(self, msg: Dict[str, Any]) -> None:
""" Send a message to telegram channel """
@ -284,37 +328,7 @@ class Telegram(RPCHandler):
# Notification disabled
return
if msg_type == RPCMessageType.BUY:
message = self._format_buy_msg(msg)
elif msg_type in (RPCMessageType.BUY_CANCEL, RPCMessageType.SELL_CANCEL):
msg['message_side'] = 'buy' if msg_type == RPCMessageType.BUY_CANCEL else 'sell'
message = ("\N{WARNING SIGN} *{exchange}:* "
"Cancelling open {message_side} Order for {pair} (#{trade_id}). "
"Reason: {reason}.".format(**msg))
elif msg_type == RPCMessageType.BUY_FILL:
message = ("\N{LARGE CIRCLE} *{exchange}:* "
"Buy order for {pair} (#{trade_id}) filled "
"for {open_rate}.".format(**msg))
elif msg_type == RPCMessageType.SELL_FILL:
message = ("\N{LARGE CIRCLE} *{exchange}:* "
"Sell order for {pair} (#{trade_id}) filled "
"for {close_rate}.".format(**msg))
elif msg_type == RPCMessageType.SELL:
message = self._format_sell_msg(msg)
elif msg_type == RPCMessageType.STATUS:
message = '*Status:* `{status}`'.format(**msg)
elif msg_type == RPCMessageType.WARNING:
message = '\N{WARNING SIGN} *Warning:* `{status}`'.format(**msg)
elif msg_type == RPCMessageType.STARTUP:
message = '{status}'.format(**msg)
else:
raise NotImplementedError('Unknown message type: {}'.format(msg_type))
message = self.compose_message(msg, msg_type)
self._send_msg(message, disable_notification=(noti == 'silent'))

2
freqtrade/templates/subtemplates/exchange_kucoin.j2
View File

@ -4,7 +4,7 @@
"secret": "{{ exchange_secret }}",
"password": "{{ exchange_key_password }}",
"ccxt_config": {
"enableRateLimit": true
"enableRateLimit": true,
"rateLimit": 200
},
"ccxt_async_config": {

4
requirements-dev.txt
View File

@ -18,10 +18,10 @@ isort==5.9.3
time-machine==2.4.0
# Convert jupyter notebooks to markdown documents
nbconvert==6.1.0
nbconvert==6.2.0
# mypy types
types-cachetools==4.2.0
types-filelock==0.1.5
types-requests==2.25.6
types-requests==2.25.9
types-tabulate==0.8.2

2
requirements-hyperopt.txt
View File

@ -8,4 +8,4 @@ scikit-optimize==0.8.1
filelock==3.0.12
joblib==1.0.1
psutil==5.8.0
progressbar2==3.53.2
progressbar2==3.53.3

7
requirements.txt
View File

@ -1,16 +1,17 @@
numpy==1.21.2
pandas==1.3.3
pandas-ta==0.3.14b
ccxt==1.56.30
ccxt==1.57.3
# Pin cryptography for now due to rust build errors with piwheels
cryptography==3.4.8
aiohttp==3.7.4.post0
SQLAlchemy==1.4.23
SQLAlchemy==1.4.25
python-telegram-bot==13.7
arrow==1.1.1
cachetools==4.2.2
requests==2.26.0
urllib3==1.26.6
urllib3==1.26.7
wrapt==1.12.1
jsonschema==3.2.0
TA-Lib==0.4.21

2
scripts/rest_client.py
View File

@ -312,7 +312,7 @@ class FtRestClient():
:param limit: Limit result to the last n candles.
:return: json object
"""
return self._get("available_pairs", params={
return self._get("pair_candles", params={
"pair": pair,
"timeframe": timeframe,
"limit": limit,

1
setup.py
View File

@ -54,6 +54,7 @@ setup(
'wrapt',
'jsonschema',
'TA-Lib',
'pandas-ta',
'technical',
'tabulate',
'pycoingecko',

37
tests/commands/test_commands.py
View File

@ -8,12 +8,12 @@ from zipfile import ZipFile
import arrow
import pytest
from freqtrade.commands import (start_convert_data, start_create_userdir, start_download_data,
start_hyperopt_list, start_hyperopt_show, start_install_ui,
start_list_data, start_list_exchanges, start_list_markets,
start_list_strategies, start_list_timeframes, start_new_strategy,
start_show_trades, start_test_pairlist, start_trading,
start_webserver)
from freqtrade.commands import (start_convert_data, start_convert_trades, start_create_userdir,
start_download_data, start_hyperopt_list, start_hyperopt_show,
start_install_ui, start_list_data, start_list_exchanges,
start_list_markets, start_list_strategies, start_list_timeframes,
start_new_strategy, start_show_trades, start_test_pairlist,
start_trading, start_webserver)
from freqtrade.commands.deploy_commands import (clean_ui_subdir, download_and_install_ui,
get_ui_download_url, read_ui_version)
from freqtrade.configuration import setup_utils_configuration
@ -208,11 +208,10 @@ def test_list_timeframes(mocker, capsys):
assert re.search(r"^1d$", captured.out, re.MULTILINE)
def test_list_markets(mocker, markets, capsys):
def test_list_markets(mocker, markets_static, capsys):
api_mock = MagicMock()
api_mock.markets = markets
patch_exchange(mocker, api_mock=api_mock, id='bittrex')
patch_exchange(mocker, api_mock=api_mock, id='bittrex', mock_markets=markets_static)
# Test with no --config
args = [
@ -237,7 +236,7 @@ def test_list_markets(mocker, markets, capsys):
"TKN/BTC, XLTCUSDT, XRP/BTC.\n"
in captured.out)
patch_exchange(mocker, api_mock=api_mock, id="binance")
patch_exchange(mocker, api_mock=api_mock, id="binance", mock_markets=markets_static)
# Test with --exchange
args = [
"list-markets",
@ -250,7 +249,7 @@ def test_list_markets(mocker, markets, capsys):
assert re.match("\nExchange Binance has 10 active markets:\n",
captured.out)
patch_exchange(mocker, api_mock=api_mock, id="bittrex")
patch_exchange(mocker, api_mock=api_mock, id="bittrex", mock_markets=markets_static)
# Test with --all: all markets
args = [
"list-markets", "--all",
@ -760,6 +759,22 @@ def test_download_data_trades(mocker, caplog):
assert convert_mock.call_count == 1
def test_start_convert_trades(mocker, caplog):
convert_mock = mocker.patch('freqtrade.commands.data_commands.convert_trades_to_ohlcv',
MagicMock(return_value=[]))
patch_exchange(mocker)
mocker.patch(
'freqtrade.exchange.Exchange.markets', PropertyMock(return_value={})
)
args = [
"trades-to-ohlcv",
"--exchange", "kraken",
"--pairs", "ETH/BTC", "XRP/BTC",
]
start_convert_trades(get_args(args))
assert convert_mock.call_count == 1
def test_start_list_strategies(mocker, caplog, capsys):
args = [

268
tests/conftest.py
View File

@ -27,6 +27,8 @@ from freqtrade.resolvers import ExchangeResolver
from freqtrade.worker import Worker
from tests.conftest_trades import (leverage_trade, mock_trade_1, mock_trade_2, mock_trade_3,
mock_trade_4, mock_trade_5, mock_trade_6, short_trade)
from tests.conftest_trades_usdt import (mock_trade_usdt_1, mock_trade_usdt_2, mock_trade_usdt_3,
mock_trade_usdt_4, mock_trade_usdt_5, mock_trade_usdt_6)
logging.getLogger('').setLevel(logging.INFO)
@ -102,8 +104,10 @@ def patch_exchange(
mocker.patch('freqtrade.exchange.Exchange.precisionMode', PropertyMock(return_value=2))
if mock_markets:
if isinstance(mock_markets, bool):
mock_markets = get_markets()
mocker.patch('freqtrade.exchange.Exchange.markets',
PropertyMock(return_value=get_markets()))
PropertyMock(return_value=mock_markets))
if mock_supported_modes:
mocker.patch(
@ -279,6 +283,7 @@ def create_mock_trades_with_leverage(fee, use_db: bool = True):
Trade.query.session.add(trade)
else:
LocalTrade.add_bt_trade(trade)
# Simulate dry_run entries
trade = mock_trade_1(fee)
add_trade(trade)
@ -303,6 +308,40 @@ def create_mock_trades_with_leverage(fee, use_db: bool = True):
trade = leverage_trade(fee)
add_trade(trade)
if use_db:
Trade.query.session.flush()
def create_mock_trades_usdt(fee, use_db: bool = True):
"""
Create some fake trades ...
"""
def add_trade(trade):
if use_db:
Trade.query.session.add(trade)
else:
LocalTrade.add_bt_trade(trade)
# Simulate dry_run entries
trade = mock_trade_usdt_1(fee)
add_trade(trade)
trade = mock_trade_usdt_2(fee)
add_trade(trade)
trade = mock_trade_usdt_3(fee)
add_trade(trade)
trade = mock_trade_usdt_4(fee)
add_trade(trade)
trade = mock_trade_usdt_5(fee)
add_trade(trade)
trade = mock_trade_usdt_6(fee)
add_trade(trade)
if use_db:
Trade.query.session.flush()
@ -343,6 +382,11 @@ def default_conf(testdatadir):
return get_default_conf(testdatadir)
@pytest.fixture(scope="function")
def default_conf_usdt(testdatadir):
return get_default_conf_usdt(testdatadir)
def get_default_conf(testdatadir):
""" Returns validated configuration suitable for most tests """
configuration = {
@ -417,6 +461,32 @@ def get_default_conf(testdatadir):
return configuration
def get_default_conf_usdt(testdatadir):
configuration = get_default_conf(testdatadir)
configuration.update({
"stake_amount": 60.0,
"stake_currency": "USDT",
"exchange": {
"name": "binance",
"enabled": True,
"key": "key",
"secret": "secret",
"pair_whitelist": [
"ETH/USDT",
"LTC/USDT",
"XRP/USDT",
"NEO/USDT",
"TKN/USDT",
],
"pair_blacklist": [
"DOGE/USDT",
"HOT/USDT",
]
},
})
return configuration
@pytest.fixture
def update():
_update = Update(0)
@ -456,12 +526,41 @@ def ticker_sell_down():
})
@pytest.fixture
def ticker_usdt():
return MagicMock(return_value={
'bid': 2.0,
'ask': 2.02,
'last': 2.0,
})
@pytest.fixture
def ticker_usdt_sell_up():
return MagicMock(return_value={
'bid': 2.2,
'ask': 2.3,
'last': 2.2,
})
@pytest.fixture
def ticker_usdt_sell_down():
return MagicMock(return_value={
'bid': 2.01,
'ask': 2.0,
'last': 2.01,
})
@pytest.fixture
def markets():
return get_markets()
def get_markets():
# See get_markets_static() for immutable markets and do not modify them unless absolutely
# necessary!
return {
'ETH/BTC': {
'id': 'ethbtc',
@ -701,6 +800,81 @@ def get_markets():
},
'info': {},
},
'XRP/USDT': {
'id': 'xrpusdt',
'symbol': 'XRP/USDT',
'base': 'XRP',
'quote': 'USDT',
'active': True,
'precision': {
'price': 8,
'amount': 8,
'cost': 8,
},
'lot': 0.00000001,
'limits': {
'amount': {
'min': 0.01,
'max': 1000,
},
'price': 500000,
'cost': {
'min': 0.0001,
'max': 500000,
},
},
'info': {},
},
'NEO/USDT': {
'id': 'neousdt',
'symbol': 'NEO/USDT',
'base': 'NEO',
'quote': 'USDT',
'active': True,
'precision': {
'price': 8,
'amount': 8,
'cost': 8,
},
'lot': 0.00000001,
'limits': {
'amount': {
'min': 0.01,
'max': 1000,
},
'price': 500000,
'cost': {
'min': 0.0001,
'max': 500000,
},
},
'info': {},
},
'TKN/USDT': {
'id': 'tknusdt',
'symbol': 'TKN/USDT',
'base': 'TKN',
'quote': 'USDT',
'active': True,
'precision': {
'price': 8,
'amount': 8,
'cost': 8,
},
'lot': 0.00000001,
'limits': {
'amount': {
'min': 0.01,
'max': 1000,
},
'price': 500000,
'cost': {
'min': 0.0001,
'max': 500000,
},
},
'info': {},
},
'LTC/USD': {
'id': 'USD-LTC',
'symbol': 'LTC/USD',
@ -778,11 +952,22 @@ def get_markets():
@pytest.fixture
def shitcoinmarkets(markets):
def markets_static():
# These markets are used in some tests that would need adaptation should anything change in
# market list. Do not modify this list without a good reason! Do not modify market parameters
# of listed pairs in get_markets() without a good reason either!
static_markets = ['BLK/BTC', 'BTT/BTC', 'ETH/BTC', 'ETH/USDT', 'LTC/BTC', 'LTC/ETH', 'LTC/USD',
'LTC/USDT', 'NEO/BTC', 'TKN/BTC', 'XLTCUSDT', 'XRP/BTC']
all_markets = get_markets()
return {m: all_markets[m] for m in static_markets}
@pytest.fixture
def shitcoinmarkets(markets_static):
"""
Fixture with shitcoin markets - used to test filters in pairlists
"""
shitmarkets = deepcopy(markets)
shitmarkets = deepcopy(markets_static)
shitmarkets.update({
'HOT/BTC': {
'id': 'HOTBTC',
@ -1624,27 +1809,34 @@ def result(testdatadir):
@pytest.fixture(scope="function")
def trades_for_order():
return [{'info': {'id': 34567,
'orderId': 123456,
'price': '0.24544100',
'qty': '8.00000000',
'commission': '0.00800000',
'commissionAsset': 'LTC',
'time': 1521663363189,
'isBuyer': True,
'isMaker': False,
'isBestMatch': True},
'timestamp': 1521663363189,
'datetime': '2018-03-21T20:16:03.189Z',
'symbol': 'LTC/ETH',
'id': '34567',
'order': '123456',
'type': None,
'side': 'buy',
'price': 0.245441,
'cost': 1.963528,
'amount': 8.0,
'fee': {'cost': 0.008, 'currency': 'LTC'}}]
return [{
'info': {
'id': 34567,
'orderId': 123456,
'price': '2.0',
'qty': '8.00000000',
'commission': '0.00800000',
'commissionAsset': 'LTC',
'time': 1521663363189,
'isBuyer': True,
'isMaker': False,
'isBestMatch': True
},
'timestamp': 1521663363189,
'datetime': '2018-03-21T20:16:03.189Z',
'symbol': 'LTC/USDT',
'id': '34567',
'order': '123456',
'type': None,
'side': 'buy',
'price': 2.0,
'cost': 16.0,
'amount': 8.0,
'fee': {
'cost': 0.008,
'currency': 'LTC'
}
}]
@pytest.fixture(scope="function")
@ -1788,14 +1980,6 @@ def trades_for_order2():
'fee': {'cost': 0.004, 'currency': 'LTC'}}]
@pytest.fixture(scope="function")
def trades_for_order3(trades_for_order2):
# Different fee currencies for each trade
trades_for_order = deepcopy(trades_for_order2)
trades_for_order[0]['fee'] = {'cost': 0.02, 'currency': 'BNB'}
return trades_for_order
@pytest.fixture
def buy_order_fee():
return {
@ -1917,6 +2101,22 @@ def open_trade():
)
@pytest.fixture(scope="function")
def open_trade_usdt():
return Trade(
pair='ADA/USDT',
open_rate=2.0,
exchange='binance',
open_order_id='123456789',
amount=30.0,
fee_open=0.0,
fee_close=0.0,
stake_amount=60.0,
open_date=arrow.utcnow().shift(minutes=-601).datetime,
is_open=True
)
@pytest.fixture
def saved_hyperopt_results():
hyperopt_res = [
@ -2060,7 +2260,7 @@ def saved_hyperopt_results():
@pytest.fixture(scope='function')
def limit_buy_order_usdt_open():
return {
'id': 'mocked_limit_buy',
'id': 'mocked_limit_buy_usdt',
'type': 'limit',
'side': 'buy',
'symbol': 'mocked',
@ -2087,7 +2287,7 @@ def limit_buy_order_usdt(limit_buy_order_usdt_open):
@pytest.fixture
def limit_sell_order_usdt_open():
return {
'id': 'mocked_limit_sell',
'id': 'mocked_limit_sell_usdt',
'type': 'limit',
'side': 'sell',
'pair': 'mocked',

305
tests/conftest_trades_usdt.py Normal file
View File

@ -0,0 +1,305 @@
from datetime import datetime, timedelta, timezone
from freqtrade.persistence.models import Order, Trade
MOCK_TRADE_COUNT = 6
def mock_order_usdt_1():
return {
'id': '1234',
'symbol': 'ADA/USDT',
'status': 'closed',
'side': 'buy',
'type': 'limit',
'price': 2.0,
'amount': 10.0,
'filled': 10.0,
'remaining': 0.0,
}
def mock_trade_usdt_1(fee):
trade = Trade(
pair='ADA/USDT',
stake_amount=20.0,
amount=10.0,
amount_requested=10.0,
fee_open=fee.return_value,
fee_close=fee.return_value,
is_open=True,
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=17),
open_rate=2.0,
exchange='binance',
open_order_id='dry_run_buy_12345',
strategy='StrategyTestV2',
timeframe=5,
)
o = Order.parse_from_ccxt_object(mock_order_usdt_1(), 'ADA/USDT', 'buy')
trade.orders.append(o)
return trade
def mock_order_usdt_2():
return {
'id': '1235',
'symbol': 'ETC/USDT',
'status': 'closed',
'side': 'buy',
'type': 'limit',
'price': 2.0,
'amount': 100.0,
'filled': 100.0,
'remaining': 0.0,
}
def mock_order_usdt_2_sell():
return {
'id': '12366',
'symbol': 'ETC/USDT',
'status': 'closed',
'side': 'sell',
'type': 'limit',
'price': 2.05,
'amount': 100.0,
'filled': 100.0,
'remaining': 0.0,
}
def mock_trade_usdt_2(fee):
"""
Closed trade...
"""
trade = Trade(
pair='ETC/USDT',
stake_amount=200.0,
amount=100.0,
amount_requested=100.0,
fee_open=fee.return_value,
fee_close=fee.return_value,
open_rate=2.0,
close_rate=2.05,
close_profit=5.0,
close_profit_abs=3.9875,
exchange='binance',
is_open=False,
open_order_id='dry_run_sell_12345',
strategy='StrategyTestV2',
timeframe=5,
sell_reason='sell_signal',
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=20),
close_date=datetime.now(tz=timezone.utc) - timedelta(minutes=2),
)
o = Order.parse_from_ccxt_object(mock_order_usdt_2(), 'ETC/USDT', 'buy')
trade.orders.append(o)
o = Order.parse_from_ccxt_object(mock_order_usdt_2_sell(), 'ETC/USDT', 'sell')
trade.orders.append(o)
return trade
def mock_order_usdt_3():
return {
'id': '41231a12a',
'symbol': 'XRP/USDT',
'status': 'closed',
'side': 'buy',
'type': 'limit',
'price': 1.0,
'amount': 30.0,
'filled': 30.0,
'remaining': 0.0,
}
def mock_order_usdt_3_sell():
return {
'id': '41231a666a',
'symbol': 'XRP/USDT',
'status': 'closed',
'side': 'sell',
'type': 'stop_loss_limit',
'price': 1.1,
'average': 1.1,
'amount': 30.0,
'filled': 30.0,
'remaining': 0.0,
}
def mock_trade_usdt_3(fee):
"""
Closed trade
"""
trade = Trade(
pair='XRP/USDT',
stake_amount=30.0,
amount=30.0,
amount_requested=30.0,
fee_open=fee.return_value,
fee_close=fee.return_value,
open_rate=1.0,
close_rate=1.1,
close_profit=10.0,
close_profit_abs=9.8425,
exchange='binance',
is_open=False,
strategy='StrategyTestV2',
timeframe=5,
sell_reason='roi',
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=20),
close_date=datetime.now(tz=timezone.utc),
)
o = Order.parse_from_ccxt_object(mock_order_usdt_3(), 'XRP/USDT', 'buy')
trade.orders.append(o)
o = Order.parse_from_ccxt_object(mock_order_usdt_3_sell(), 'XRP/USDT', 'sell')
trade.orders.append(o)
return trade
def mock_order_usdt_4():
return {
'id': 'prod_buy_12345',
'symbol': 'ETC/USDT',
'status': 'open',
'side': 'buy',
'type': 'limit',
'price': 2.0,
'amount': 10.0,
'filled': 0.0,
'remaining': 30.0,
}
def mock_trade_usdt_4(fee):
"""
Simulate prod entry
"""
trade = Trade(
pair='ETC/USDT',
stake_amount=20.0,
amount=10.0,
amount_requested=10.01,
fee_open=fee.return_value,
fee_close=fee.return_value,
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=14),
is_open=True,
open_rate=2.0,
exchange='binance',
open_order_id='prod_buy_12345',
strategy='StrategyTestV2',
timeframe=5,
)
o = Order.parse_from_ccxt_object(mock_order_usdt_4(), 'ETC/USDT', 'buy')
trade.orders.append(o)
return trade
def mock_order_usdt_5():
return {
'id': 'prod_buy_3455',
'symbol': 'XRP/USDT',
'status': 'closed',
'side': 'buy',
'type': 'limit',
'price': 2.0,
'amount': 10.0,
'filled': 10.0,
'remaining': 0.0,
}
def mock_order_usdt_5_stoploss():
return {
'id': 'prod_stoploss_3455',
'symbol': 'XRP/USDT',
'status': 'open',
'side': 'sell',
'type': 'stop_loss_limit',
'price': 2.0,
'amount': 10.0,
'filled': 0.0,
'remaining': 30.0,
}
def mock_trade_usdt_5(fee):
"""
Simulate prod entry with stoploss
"""
trade = Trade(
pair='XRP/USDT',
stake_amount=20.0,
amount=10.0,
amount_requested=10.01,
fee_open=fee.return_value,
fee_close=fee.return_value,
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=12),
is_open=True,
open_rate=2.0,
exchange='binance',
strategy='SampleStrategy',
stoploss_order_id='prod_stoploss_3455',
timeframe=5,
)
o = Order.parse_from_ccxt_object(mock_order_usdt_5(), 'XRP/USDT', 'buy')
trade.orders.append(o)
o = Order.parse_from_ccxt_object(mock_order_usdt_5_stoploss(), 'XRP/USDT', 'stoploss')
trade.orders.append(o)
return trade
def mock_order_usdt_6():
return {
'id': 'prod_buy_6',
'symbol': 'LTC/USDT',
'status': 'closed',
'side': 'buy',
'type': 'limit',
'price': 10.0,
'amount': 2.0,
'filled': 2.0,
'remaining': 0.0,
}
def mock_order_usdt_6_sell():
return {
'id': 'prod_sell_6',
'symbol': 'LTC/USDT',
'status': 'open',
'side': 'sell',
'type': 'limit',
'price': 12.0,
'amount': 2.0,
'filled': 0.0,
'remaining': 2.0,
}
def mock_trade_usdt_6(fee):
"""
Simulate prod entry with open sell order
"""
trade = Trade(
pair='LTC/USDT',
stake_amount=20.0,
amount=2.0,
amount_requested=2.0,
open_date=datetime.now(tz=timezone.utc) - timedelta(minutes=5),
fee_open=fee.return_value,
fee_close=fee.return_value,
is_open=True,
open_rate=10.0,
exchange='binance',
strategy='SampleStrategy',
open_order_id="prod_sell_6",
timeframe=5,
)
o = Order.parse_from_ccxt_object(mock_order_usdt_6(), 'LTC/USDT', 'buy')
trade.orders.append(o)
o = Order.parse_from_ccxt_object(mock_order_usdt_6_sell(), 'LTC/USDT', 'sell')
trade.orders.append(o)
return trade

4
tests/exchange/test_exchange.py
View File

@ -2806,7 +2806,7 @@ def test_get_valid_pair_combination(default_conf, mocker, markets):
(['LTC'], ['NONEXISTENT'], False, False,
[]),
])
def test_get_markets(default_conf, mocker, markets,
def test_get_markets(default_conf, mocker, markets_static,
base_currencies, quote_currencies, pairs_only, active_only,
expected_keys):
mocker.patch.multiple('freqtrade.exchange.Exchange',
@ -2814,7 +2814,7 @@ def test_get_markets(default_conf, mocker, markets,
_load_async_markets=MagicMock(),
validate_pairs=MagicMock(),
validate_timeframes=MagicMock(),
markets=PropertyMock(return_value=markets))
markets=PropertyMock(return_value=markets_static))
ex = Exchange(default_conf)
pairs = ex.get_markets(base_currencies, quote_currencies, pairs_only, active_only)
assert sorted(pairs.keys()) == sorted(expected_keys)

5
tests/exchange/test_kraken.py
View File

@ -327,5 +327,8 @@ def test_fill_leverage_brackets_kraken(default_conf, mocker):
'LTC/USDT': [1],
'LTC/USD': [1],
'XLTCUSDT': [1],
'LTC/ETH': [1]
'LTC/ETH': [1],
'NEO/USDT': [1],
'TKN/USDT': [1],
'XRP/USDT': [1]
}

27
tests/optimize/conftest.py
View File

@ -39,16 +39,25 @@ def hyperopt(hyperopt_conf, mocker):
def hyperopt_results():
return pd.DataFrame(
{
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
'profit_ratio': [-0.1, 0.2, 0.3],
'profit_abs': [-0.2, 0.4, 0.6],
'trade_duration': [10, 30, 10],
'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI],
'pair': ['ETH/USDT', 'ETH/USDT', 'ETH/USDT', 'ETH/USDT'],
'profit_ratio': [-0.1, 0.2, -0.1, 0.3],
'profit_abs': [-0.2, 0.4, -0.2, 0.6],
'trade_duration': [10, 30, 10, 10],
'amount': [0.1, 0.1, 0.1, 0.1],
'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.STOP_LOSS, SellType.ROI],
'open_date':
[
datetime(2019, 1, 1, 9, 15, 0),
datetime(2019, 1, 2, 8, 55, 0),
datetime(2019, 1, 3, 9, 15, 0),
datetime(2019, 1, 4, 9, 15, 0),
],
'close_date':
[
datetime(2019, 1, 1, 9, 26, 3, 478039),
datetime(2019, 2, 1, 9, 26, 3, 478039),
datetime(2019, 3, 1, 9, 26, 3, 478039)
]
datetime(2019, 1, 1, 9, 25, 0),
datetime(2019, 1, 2, 9, 25, 0),
datetime(2019, 1, 3, 9, 25, 0),
datetime(2019, 1, 4, 9, 25, 0),
],
}
)

92
tests/optimize/test_hyperoptloss.py
View File

@ -35,6 +35,7 @@ def test_hyperoptlossresolver_wrongname(default_conf) -> None:
def test_loss_calculation_prefer_correct_trade_count(hyperopt_conf, hyperopt_results) -> None:
hyperopt_conf.update({'hyperopt_loss': "ShortTradeDurHyperOptLoss"})
hl = HyperOptLossResolver.load_hyperoptloss(hyperopt_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, 600,
datetime(2019, 1, 1), datetime(2019, 5, 1))
@ -50,6 +51,7 @@ def test_loss_calculation_prefer_shorter_trades(hyperopt_conf, hyperopt_results)
resultsb = hyperopt_results.copy()
resultsb.loc[1, 'trade_duration'] = 20
hyperopt_conf.update({'hyperopt_loss': "ShortTradeDurHyperOptLoss"})
hl = HyperOptLossResolver.load_hyperoptloss(hyperopt_conf)
longer = hl.hyperopt_loss_function(hyperopt_results, 100,
datetime(2019, 1, 1), datetime(2019, 5, 1))
@ -64,6 +66,7 @@ def test_loss_calculation_has_limited_profit(hyperopt_conf, hyperopt_results) ->
results_under = hyperopt_results.copy()
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
hyperopt_conf.update({'hyperopt_loss': "ShortTradeDurHyperOptLoss"})
hl = HyperOptLossResolver.load_hyperoptloss(hyperopt_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, 600,
datetime(2019, 1, 1), datetime(2019, 5, 1))
@ -75,91 +78,28 @@ def test_loss_calculation_has_limited_profit(hyperopt_conf, hyperopt_results) ->
assert under > correct
def test_sharpe_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
results_over = hyperopt_results.copy()
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': 'SharpeHyperOptLoss'})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
assert over < correct
assert under > correct
def test_sharpe_loss_daily_prefers_higher_profits(default_conf, hyperopt_results) -> None:
results_over = hyperopt_results.copy()
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': 'SharpeHyperOptLossDaily'})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
assert over < correct
assert under > correct
def test_sortino_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
results_over = hyperopt_results.copy()
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': 'SortinoHyperOptLoss'})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
assert over < correct
assert under > correct
def test_sortino_loss_daily_prefers_higher_profits(default_conf, hyperopt_results) -> None:
results_over = hyperopt_results.copy()
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': 'SortinoHyperOptLossDaily'})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
assert over < correct
assert under > correct
def test_onlyprofit_loss_prefers_higher_profits(default_conf, hyperopt_results) -> None:
@pytest.mark.parametrize('lossfunction', [
"OnlyProfitHyperOptLoss",
"SortinoHyperOptLoss",
"SortinoHyperOptLossDaily",
"SharpeHyperOptLoss",
"SharpeHyperOptLossDaily",
])
def test_loss_functions_better_profits(default_conf, hyperopt_results, lossfunction) -> None:
results_over = hyperopt_results.copy()
results_over['profit_abs'] = hyperopt_results['profit_abs'] * 2
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_abs'] = hyperopt_results['profit_abs'] / 2
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': 'OnlyProfitHyperOptLoss'})
default_conf.update({'hyperopt_loss': lossfunction})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1))
over = hl.hyperopt_loss_function(results_over, len(hyperopt_results),
over = hl.hyperopt_loss_function(results_over, len(results_over),
datetime(2019, 1, 1), datetime(2019, 5, 1))
under = hl.hyperopt_loss_function(results_under, len(hyperopt_results),
under = hl.hyperopt_loss_function(results_under, len(results_under),
datetime(2019, 1, 1), datetime(2019, 5, 1))
assert over < correct
assert under > correct

4
tests/plugins/test_pairlist.py
View File

@ -131,9 +131,9 @@ def test_load_pairlist_noexist(mocker, markets, default_conf):
default_conf, {}, 1)
def test_load_pairlist_verify_multi(mocker, markets, default_conf):
def test_load_pairlist_verify_multi(mocker, markets_static, default_conf):
freqtrade = get_patched_freqtradebot(mocker, default_conf)
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets))
mocker.patch('freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets_static))
plm = PairListManager(freqtrade.exchange, default_conf)
# Call different versions one after the other, should always consider what was passed in
# and have no side-effects (therefore the same check multiple times)

4
tests/plugins/test_protections.py
View File

@ -125,7 +125,7 @@ def test_stoploss_guard(mocker, default_conf, fee, caplog):
# Test 5m after lock-period - this should try and relock the pair, but end-time
# should be the previous end-time
end_time = PairLocks.get_pair_longest_lock('*').lock_end_time + timedelta(minutes=5)
assert freqtrade.protections.global_stop(end_time)
freqtrade.protections.global_stop(end_time)
assert not PairLocks.is_global_lock(end_time)
@ -182,7 +182,7 @@ def test_stoploss_guard_perpair(mocker, default_conf, fee, caplog, only_per_pair
min_ago_open=180, min_ago_close=30, profit_rate=0.9,
))
assert freqtrade.protections.stop_per_pair(pair)
freqtrade.protections.stop_per_pair(pair)
assert freqtrade.protections.global_stop() != only_per_pair
assert PairLocks.is_pair_locked(pair)
assert PairLocks.is_global_lock() != only_per_pair

2
tests/rpc/test_rpc.py
View File

@ -1015,7 +1015,7 @@ def test_rpc_blacklist(mocker, default_conf) -> None:
assert len(ret['blacklist']) == 4
assert ret['blacklist'] == default_conf['exchange']['pair_blacklist']
assert ret['blacklist'] == ['DOGE/BTC', 'HOT/BTC', 'ETH/BTC', 'XRP/.*']
assert ret['blacklist_expanded'] == ['ETH/BTC', 'XRP/BTC']
assert ret['blacklist_expanded'] == ['ETH/BTC', 'XRP/BTC', 'XRP/USDT']
assert 'errors' in ret
assert isinstance(ret['errors'], dict)

2
tests/rpc/test_rpc_apiserver.py
View File

@ -937,7 +937,7 @@ def test_api_blacklist(botclient, mocker):
data='{"blacklist": ["XRP/.*"]}')
assert_response(rc)
assert rc.json() == {"blacklist": ["DOGE/BTC", "HOT/BTC", "ETH/BTC", "XRP/.*"],
"blacklist_expanded": ["ETH/BTC", "XRP/BTC"],
"blacklist_expanded": ["ETH/BTC", "XRP/BTC", "XRP/USDT"],
"length": 4,
"method": ["StaticPairList"],
"errors": {},

28
tests/rpc/test_rpc_telegram.py
View File

@ -1313,6 +1313,34 @@ def test_send_msg_buy_cancel_notification(default_conf, mocker) -> None:
'Reason: cancelled due to timeout.')
def test_send_msg_protection_notification(default_conf, mocker, time_machine) -> None:
default_conf['telegram']['notification_settings']['protection_trigger'] = 'on'
telegram, _, msg_mock = get_telegram_testobject(mocker, default_conf)
time_machine.move_to("2021-09-01 05:00:00 +00:00")
lock = PairLocks.lock_pair('ETH/BTC', arrow.utcnow().shift(minutes=6).datetime, 'randreason')
msg = {
'type': RPCMessageType.PROTECTION_TRIGGER,
}
msg.update(lock.to_json())
telegram.send_msg(msg)
assert (msg_mock.call_args[0][0] == "*Protection* triggered due to randreason. "
"`ETH/BTC` will be locked until `2021-09-01 05:10:00`.")
msg_mock.reset_mock()
# Test global protection
msg = {
'type': RPCMessageType.PROTECTION_TRIGGER_GLOBAL,
}
lock = PairLocks.lock_pair('*', arrow.utcnow().shift(minutes=100).datetime, 'randreason')
msg.update(lock.to_json())
telegram.send_msg(msg)
assert (msg_mock.call_args[0][0] == "*Protection* triggered due to randreason. "
"*All pairs* will be locked until `2021-09-01 06:45:00`.")
def test_send_msg_buy_fill_notification(default_conf, mocker) -> None:
default_conf['telegram']['notification_settings']['buy_fill'] = 'on'

4
tests/strategy/test_strategy_helpers.py
View File

@ -9,13 +9,13 @@ from freqtrade.strategy import (merge_informative_pair, stoploss_from_absolute,
timeframe_to_minutes)
def generate_test_data(timeframe: str, size: int):
def generate_test_data(timeframe: str, size: int, start: str = '2020-07-05'):
np.random.seed(42)
tf_mins = timeframe_to_minutes(timeframe)
base = np.random.normal(20, 2, size=size)
date = pd.period_range('2020-07-05', periods=size, freq=f'{tf_mins}min').to_timestamp()
date = pd.date_range(start, periods=size, freq=f'{tf_mins}min', tz='UTC')
df = pd.DataFrame({
'date': date,
'open': base,

1997
tests/test_freqtradebot.py
View File

File diff suppressed because it is too large Load Diff