Merge branch 'develop' into feat/short

This commit is contained in:
Matthias 2021-10-30 19:45:19 +02:00
commit c094ac5762
22 changed files with 367 additions and 179 deletions

View File

@ -11,7 +11,7 @@ Otherwise `--exchange` becomes mandatory.
You can use a relative timerange (`--days 20`) or an absolute starting point (`--timerange 20200101-`). For incremental downloads, the relative approach should be used. You can use a relative timerange (`--days 20`) or an absolute starting point (`--timerange 20200101-`). For incremental downloads, the relative approach should be used.
!!! Tip "Tip: Updating existing data" !!! Tip "Tip: Updating existing data"
If you already have backtesting data available in your data-directory and would like to refresh this data up to today, do not use `--days` or `--timerange` parameters. Freqtrade will keep the available data and only download the missing data. If you already have backtesting data available in your data-directory and would like to refresh this data up to today, freqtrade will automatically calculate the data missing for the existing pairs and the download will occur from the latest available point until "now", neither --days or --timerange parameters are required. Freqtrade will keep the available data and only download the missing data.
If you are updating existing data after inserting new pairs that you have no data for, use `--new-pairs-days xx` parameter. Specified number of days will be downloaded for new pairs while old pairs will be updated with missing data only. If you are updating existing data after inserting new pairs that you have no data for, use `--new-pairs-days xx` parameter. Specified number of days will be downloaded for new pairs while old pairs will be updated with missing data only.
If you use `--days xx` parameter alone - data for specified number of days will be downloaded for _all_ pairs. Be careful, if specified number of days is smaller than gap between now and last downloaded candle - freqtrade will delete all existing data to avoid gaps in candle data. If you use `--days xx` parameter alone - data for specified number of days will be downloaded for _all_ pairs. Be careful, if specified number of days is smaller than gap between now and last downloaded candle - freqtrade will delete all existing data to avoid gaps in candle data.

View File

@ -54,6 +54,21 @@ you can't say much from few trades.
Yes. You can edit your config and use the `/reload_config` command to reload the configuration. The bot will stop, reload the configuration and strategy and will restart with the new configuration and strategy. Yes. You can edit your config and use the `/reload_config` command to reload the configuration. The bot will stop, reload the configuration and strategy and will restart with the new configuration and strategy.
### Why does my bot not sell everything it bought?
This is called "coin dust" and can happen on all exchanges.
It happens because many exchanges subtract fees from the "receiving currency" - so you buy 100 COIN - but you only get 99.9 COIN.
As COIN is trading in full lot sizes (1COIN steps), you cannot sell 0.9 COIN (or 99.9 COIN) - but you need to round down to 99 COIN.
This is not a bot-problem, but will also happen while manual trading.
While freqtrade can handle this (it'll sell 99 COIN), fees are often below the minimum tradable lot-size (you can only trade full COIN, not 0.9 COIN).
Leaving the dust (0.9 COIN) on the exchange makes usually sense, as the next time freqtrade buys COIN, it'll eat into the remaining small balance, this time selling everything it bought, and therefore slowly declining the dust balance (although it most likely will never reach exactly 0).
Where possible (e.g. on binance), the use of the exchange's dedicated fee currency will fix this.
On binance, it's sufficient to have BNB in your account, and have "Pay fees in BNB" enabled in your profile. Your BNB balance will slowly decline (as it's used to pay fees) - but you'll no longer encounter dust (Freqtrade will include the fees in the profit calculations).
Other exchanges don't offer such possibilities, where it's simply something you'll have to accept or move to a different exchange.
### I want to use incomplete candles ### I want to use incomplete candles
Freqtrade will not provide incomplete candles to strategies. Using incomplete candles will lead to repainting and consequently to strategies with "ghost" buys, which are impossible to both backtest, and verify after they happened. Freqtrade will not provide incomplete candles to strategies. Using incomplete candles will lead to repainting and consequently to strategies with "ghost" buys, which are impossible to both backtest, and verify after they happened.

View File

@ -116,7 +116,7 @@ optional arguments:
ShortTradeDurHyperOptLoss, OnlyProfitHyperOptLoss, ShortTradeDurHyperOptLoss, OnlyProfitHyperOptLoss,
SharpeHyperOptLoss, SharpeHyperOptLossDaily, SharpeHyperOptLoss, SharpeHyperOptLossDaily,
SortinoHyperOptLoss, SortinoHyperOptLossDaily, SortinoHyperOptLoss, SortinoHyperOptLossDaily,
MaxDrawDownHyperOptLoss CalmarHyperOptLoss, MaxDrawDownHyperOptLoss
--disable-param-export --disable-param-export
Disable automatic hyperopt parameter export. Disable automatic hyperopt parameter export.
--ignore-missing-spaces, --ignore-unparameterized-spaces --ignore-missing-spaces, --ignore-unparameterized-spaces
@ -524,6 +524,7 @@ Currently, the following loss functions are builtin:
* `SortinoHyperOptLoss` - optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation. * `SortinoHyperOptLoss` - optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation.
* `SortinoHyperOptLossDaily` - optimizes Sortino Ratio calculated on **daily** trade returns relative to **downside** standard deviation. * `SortinoHyperOptLossDaily` - optimizes Sortino Ratio calculated on **daily** trade returns relative to **downside** standard deviation.
* `MaxDrawDownHyperOptLoss` - Optimizes Maximum drawdown. * `MaxDrawDownHyperOptLoss` - Optimizes Maximum drawdown.
* `CalmarHyperOptLoss` - Optimizes Calmar Ratio calculated on trade returns relative to max drawdown.
Creation of a custom loss function is covered in the [Advanced Hyperopt](advanced-hyperopt.md) part of the documentation. Creation of a custom loss function is covered in the [Advanced Hyperopt](advanced-hyperopt.md) part of the documentation.

View File

@ -312,7 +312,7 @@ Currently this is `pair`, which can be accessed using `metadata['pair']` - and w
The Metadata-dict should not be modified and does not persist information across multiple calls. The Metadata-dict should not be modified and does not persist information across multiple calls.
Instead, have a look at the section [Storing information](strategy-advanced.md#Storing-information) Instead, have a look at the section [Storing information](strategy-advanced.md#Storing-information)
## Additional data (informative_pairs) ## Informative Pairs
### Get data for non-tradeable pairs ### Get data for non-tradeable pairs
@ -341,6 +341,133 @@ A full sample can be found [in the DataProvider section](#complete-data-provider
*** ***
### Informative pairs decorator (`@informative()`)
In most common case it is possible to easily define informative pairs by using a decorator. All decorated `populate_indicators_*` methods run in isolation,
not having access to data from other informative pairs, in the end all informative dataframes are merged and passed to main `populate_indicators()` method.
When hyperopting, use of hyperoptable parameter `.value` attribute is not supported. Please use `.range` attribute. See [optimizing an indicator parameter](hyperopt.md#optimizing-an-indicator-parameter)
for more information.
??? info "Full documentation"
``` python
def informative(timeframe: str, asset: str = '',
fmt: Optional[Union[str, Callable[[KwArg(str)], str]]] = None,
ffill: bool = True) -> Callable[[PopulateIndicators], PopulateIndicators]:
"""
A decorator for populate_indicators_Nn(self, dataframe, metadata), allowing these functions to
define informative indicators.
Example usage:
@informative('1h')
def populate_indicators_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
:param timeframe: Informative timeframe. Must always be equal or higher than strategy timeframe.
:param asset: Informative asset, for example BTC, BTC/USDT, ETH/BTC. Do not specify to use
current pair.
:param fmt: Column format (str) or column formatter (callable(name, asset, timeframe)). When not
specified, defaults to:
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {column}_{timeframe} if asset is not specified.
Format string supports these format variables:
* {asset} - full name of the asset, for example 'BTC/USDT'.
* {base} - base currency in lower case, for example 'eth'.
* {BASE} - same as {base}, except in upper case.
* {quote} - quote currency in lower case, for example 'usdt'.
* {QUOTE} - same as {quote}, except in upper case.
* {column} - name of dataframe column.
* {timeframe} - timeframe of informative dataframe.
:param ffill: ffill dataframe after merging informative pair.
"""
```
??? Example "Fast and easy way to define informative pairs"
Most of the time we do not need power and flexibility offered by `merge_informative_pair()`, therefore we can use a decorator to quickly define informative pairs.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, informative
class AwesomeStrategy(IStrategy):
# This method is not required.
# def informative_pairs(self): ...
# Define informative upper timeframe for each pair. Decorators can be stacked on same
# method. Available in populate_indicators as 'rsi_30m' and 'rsi_1h'.
@informative('30m')
@informative('1h')
def populate_indicators_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/STAKE informative pair. Available in populate_indicators and other methods as
# 'btc_rsi_1h'. Current stake currency should be specified as {stake} format variable
# instead of hardcoding actual stake currency. Available in populate_indicators and other
# methods as 'btc_usdt_rsi_1h' (when stake currency is USDT).
@informative('1h', 'BTC/{stake}')
def populate_indicators_btc_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/ETH informative pair. You must specify quote currency if it is different from
# stake currency. Available in populate_indicators and other methods as 'eth_btc_rsi_1h'.
@informative('1h', 'ETH/BTC')
def populate_indicators_eth_btc_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/STAKE informative pair. A custom formatter may be specified for formatting
# column names. A callable `fmt(**kwargs) -> str` may be specified, to implement custom
# formatting. Available in populate_indicators and other methods as 'rsi_upper'.
@informative('1h', 'BTC/{stake}', '{column}')
def populate_indicators_btc_1h_2(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi_upper'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# Strategy timeframe indicators for current pair.
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
# Informative pairs are available in this method.
dataframe['rsi_less'] = dataframe['rsi'] < dataframe['rsi_1h']
return dataframe
```
!!! Note
Do not use `@informative` decorator if you need to use data of one informative pair when generating another informative pair. Instead, define informative pairs
manually as described [in the DataProvider section](#complete-data-provider-sample).
!!! Note
Use string formatting when accessing informative dataframes of other pairs. This will allow easily changing stake currency in config without having to adjust strategy code.
``` python
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
stake = self.config['stake_currency']
dataframe.loc[
(
(dataframe[f'btc_{stake}_rsi_1h'] < 35)
&
(dataframe['volume'] > 0)
),
['buy', 'buy_tag']] = (1, 'buy_signal_rsi')
return dataframe
```
Alternatively column renaming may be used to remove stake currency from column names: `@informative('1h', 'BTC/{stake}', fmt='{base}_{column}_{timeframe}')`.
!!! Warning "Duplicate method names"
Methods tagged with `@informative()` decorator must always have unique names! Re-using same name (for example when copy-pasting already defined informative method)
will overwrite previously defined method and not produce any errors due to limitations of Python programming language. In such cases you will find that indicators
created in earlier-defined methods are not available in the dataframe. Carefully review method names and make sure they are unique!
## Additional data (DataProvider) ## Additional data (DataProvider)
The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy. The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy.
@ -686,131 +813,6 @@ In some situations it may be confusing to deal with stops relative to current ra
``` ```
### *@informative()*
``` python
def informative(timeframe: str, asset: str = '',
fmt: Optional[Union[str, Callable[[KwArg(str)], str]]] = None,
ffill: bool = True) -> Callable[[PopulateIndicators], PopulateIndicators]:
"""
A decorator for populate_indicators_Nn(self, dataframe, metadata), allowing these functions to
define informative indicators.
Example usage:
@informative('1h')
def populate_indicators_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
:param timeframe: Informative timeframe. Must always be equal or higher than strategy timeframe.
:param asset: Informative asset, for example BTC, BTC/USDT, ETH/BTC. Do not specify to use
current pair.
:param fmt: Column format (str) or column formatter (callable(name, asset, timeframe)). When not
specified, defaults to:
* {base}_{quote}_{column}_{timeframe} if asset is specified.
* {column}_{timeframe} if asset is not specified.
Format string supports these format variables:
* {asset} - full name of the asset, for example 'BTC/USDT'.
* {base} - base currency in lower case, for example 'eth'.
* {BASE} - same as {base}, except in upper case.
* {quote} - quote currency in lower case, for example 'usdt'.
* {QUOTE} - same as {quote}, except in upper case.
* {column} - name of dataframe column.
* {timeframe} - timeframe of informative dataframe.
:param ffill: ffill dataframe after merging informative pair.
"""
```
In most common case it is possible to easily define informative pairs by using a decorator. All decorated `populate_indicators_*` methods run in isolation,
not having access to data from other informative pairs, in the end all informative dataframes are merged and passed to main `populate_indicators()` method.
When hyperopting, use of hyperoptable parameter `.value` attribute is not supported. Please use `.range` attribute. See [optimizing an indicator parameter](hyperopt.md#optimizing-an-indicator-parameter)
for more information.
??? Example "Fast and easy way to define informative pairs"
Most of the time we do not need power and flexibility offered by `merge_informative_pair()`, therefore we can use a decorator to quickly define informative pairs.
``` python
from datetime import datetime
from freqtrade.persistence import Trade
from freqtrade.strategy import IStrategy, informative
class AwesomeStrategy(IStrategy):
# This method is not required.
# def informative_pairs(self): ...
# Define informative upper timeframe for each pair. Decorators can be stacked on same
# method. Available in populate_indicators as 'rsi_30m' and 'rsi_1h'.
@informative('30m')
@informative('1h')
def populate_indicators_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/STAKE informative pair. Available in populate_indicators and other methods as
# 'btc_rsi_1h'. Current stake currency should be specified as {stake} format variable
# instead of hardcoding actual stake currency. Available in populate_indicators and other
# methods as 'btc_usdt_rsi_1h' (when stake currency is USDT).
@informative('1h', 'BTC/{stake}')
def populate_indicators_btc_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/ETH informative pair. You must specify quote currency if it is different from
# stake currency. Available in populate_indicators and other methods as 'eth_btc_rsi_1h'.
@informative('1h', 'ETH/BTC')
def populate_indicators_eth_btc_1h(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
# Define BTC/STAKE informative pair. A custom formatter may be specified for formatting
# column names. A callable `fmt(**kwargs) -> str` may be specified, to implement custom
# formatting. Available in populate_indicators and other methods as 'rsi_upper'.
@informative('1h', 'BTC/{stake}', '{column}')
def populate_indicators_btc_1h_2(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['rsi_upper'] = ta.RSI(dataframe, timeperiod=14)
return dataframe
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
# Strategy timeframe indicators for current pair.
dataframe['rsi'] = ta.RSI(dataframe, timeperiod=14)
# Informative pairs are available in this method.
dataframe['rsi_less'] = dataframe['rsi'] < dataframe['rsi_1h']
return dataframe
```
!!! Note
Do not use `@informative` decorator if you need to use data of one informative pair when generating another informative pair. Instead, define informative pairs
manually as described [in the DataProvider section](#complete-data-provider-sample).
!!! Note
Use string formatting when accessing informative dataframes of other pairs. This will allow easily changing stake currency in config without having to adjust strategy code.
``` python
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
stake = self.config['stake_currency']
dataframe.loc[
(
(dataframe[f'btc_{stake}_rsi_1h'] < 35)
&
(dataframe['volume'] > 0)
),
['buy', 'buy_tag']] = (1, 'buy_signal_rsi')
return dataframe
```
Alternatively column renaming may be used to remove stake currency from column names: `@informative('1h', 'BTC/{stake}', fmt='{base}_{column}_{timeframe}')`.
!!! Warning "Duplicate method names"
Methods tagged with `@informative()` decorator must always have unique names! Re-using same name (for example when copy-pasting already defined informative method)
will overwrite previously defined method and not produce any errors due to limitations of Python programming language. In such cases you will find that indicators
created in earlier-defined methods are not available in the dataframe. Carefully review method names and make sure they are unique!
## Additional data (Wallets) ## Additional data (Wallets)
The strategy provides access to the `Wallets` object. This contains the current balances on the exchange. The strategy provides access to the `Wallets` object. This contains the current balances on the exchange.
@ -894,7 +896,8 @@ Sometimes it may be desired to lock a pair after certain events happen (e.g. mul
Freqtrade has an easy method to do this from within the strategy, by calling `self.lock_pair(pair, until, [reason])`. Freqtrade has an easy method to do this from within the strategy, by calling `self.lock_pair(pair, until, [reason])`.
`until` must be a datetime object in the future, after which trading will be re-enabled for that pair, while `reason` is an optional string detailing why the pair was locked. `until` must be a datetime object in the future, after which trading will be re-enabled for that pair, while `reason` is an optional string detailing why the pair was locked.
Locks can also be lifted manually, by calling `self.unlock_pair(pair)`. Locks can also be lifted manually, by calling `self.unlock_pair(pair)` or `self.unlock_reason(<reason>)` - providing reason the pair was locked with.
`self.unlock_reason(<reason>)` will unlock all pairs currently locked with the provided reason.
To verify if a pair is currently locked, use `self.is_pair_locked(pair)`. To verify if a pair is currently locked, use `self.is_pair_locked(pair)`.

View File

@ -171,7 +171,7 @@ official commands. You can ask at any moment for help with `/help`.
| `/profit [<n>]` | Display a summary of your profit/loss from close trades and some stats about your performance, over the last n days (all trades by default) | `/profit [<n>]` | Display a summary of your profit/loss from close trades and some stats about your performance, over the last n days (all trades by default)
| `/forcesell <trade_id>` | Instantly sells the given trade (Ignoring `minimum_roi`). | `/forcesell <trade_id>` | Instantly sells the given trade (Ignoring `minimum_roi`).
| `/forcesell all` | Instantly sells all open trades (Ignoring `minimum_roi`). | `/forcesell all` | Instantly sells all open trades (Ignoring `minimum_roi`).
| `/forcebuy <pair> [rate]` | Instantly buys the given pair. Rate is optional. (`forcebuy_enable` must be set to True) | `/forcebuy <pair> [rate]` | Instantly buys the given pair. Rate is optional and only applies to limit orders. (`forcebuy_enable` must be set to True)
| `/performance` | Show performance of each finished trade grouped by pair | `/performance` | Show performance of each finished trade grouped by pair
| `/balance` | Show account balance per currency | `/balance` | Show account balance per currency
| `/daily <n>` | Shows profit or loss per day, over the last n days (n defaults to 7) | `/daily <n>` | Shows profit or loss per day, over the last n days (n defaults to 7)

View File

@ -25,6 +25,7 @@ ORDERTIF_POSSIBILITIES = ['gtc', 'fok', 'ioc']
HYPEROPT_LOSS_BUILTIN = ['ShortTradeDurHyperOptLoss', 'OnlyProfitHyperOptLoss', HYPEROPT_LOSS_BUILTIN = ['ShortTradeDurHyperOptLoss', 'OnlyProfitHyperOptLoss',
'SharpeHyperOptLoss', 'SharpeHyperOptLossDaily', 'SharpeHyperOptLoss', 'SharpeHyperOptLossDaily',
'SortinoHyperOptLoss', 'SortinoHyperOptLossDaily', 'SortinoHyperOptLoss', 'SortinoHyperOptLossDaily',
'CalmarHyperOptLoss',
'MaxDrawDownHyperOptLoss'] 'MaxDrawDownHyperOptLoss']
AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList', AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList',
'AgeFilter', 'OffsetFilter', 'PerformanceFilter', 'AgeFilter', 'OffsetFilter', 'PerformanceFilter',
@ -55,7 +56,6 @@ ENV_VAR_PREFIX = 'FREQTRADE__'
NON_OPEN_EXCHANGE_STATES = ('cancelled', 'canceled', 'closed', 'expired') NON_OPEN_EXCHANGE_STATES = ('cancelled', 'canceled', 'closed', 'expired')
# Define decimals per coin for outputs # Define decimals per coin for outputs
# Only used for outputs. # Only used for outputs.
DECIMAL_PER_COIN_FALLBACK = 3 # Should be low to avoid listing all possible FIAT's DECIMAL_PER_COIN_FALLBACK = 3 # Should be low to avoid listing all possible FIAT's
@ -69,7 +69,6 @@ DUST_PER_COIN = {
'ETH': 0.01 'ETH': 0.01
} }
# Source files with destination directories within user-directory # Source files with destination directories within user-directory
USER_DATA_FILES = { USER_DATA_FILES = {
'sample_strategy.py': USERPATH_STRATEGIES, 'sample_strategy.py': USERPATH_STRATEGIES,

View File

@ -0,0 +1,64 @@
"""
CalmarHyperOptLoss
This module defines the alternative HyperOptLoss class which can be used for
Hyperoptimization.
"""
from datetime import datetime
from math import sqrt as msqrt
from typing import Any, Dict
from pandas import DataFrame
from freqtrade.data.btanalysis import calculate_max_drawdown
from freqtrade.optimize.hyperopt import IHyperOptLoss
class CalmarHyperOptLoss(IHyperOptLoss):
"""
Defines the loss function for hyperopt.
This implementation uses the Calmar Ratio calculation.
"""
@staticmethod
def hyperopt_loss_function(
results: DataFrame,
trade_count: int,
min_date: datetime,
max_date: datetime,
config: Dict,
processed: Dict[str, DataFrame],
backtest_stats: Dict[str, Any],
*args,
**kwargs
) -> float:
"""
Objective function, returns smaller number for more optimal results.
Uses Calmar Ratio calculation.
"""
total_profit = backtest_stats["profit_total"]
days_period = (max_date - min_date).days
# adding slippage of 0.1% per trade
total_profit = total_profit - 0.0005
expected_returns_mean = total_profit.sum() / days_period * 100
# calculate max drawdown
try:
_, _, _, high_val, low_val = calculate_max_drawdown(
results, value_col="profit_abs"
)
max_drawdown = (high_val - low_val) / high_val
except ValueError:
max_drawdown = 0
if max_drawdown != 0:
calmar_ratio = expected_returns_mean / max_drawdown * msqrt(365)
else:
# Define high (negative) calmar ratio to be clear that this is NOT optimal.
calmar_ratio = -20.0
# print(expected_returns_mean, max_drawdown, calmar_ratio)
return -calmar_ratio

View File

@ -1,4 +1,3 @@
import io import io
import logging import logging
from copy import deepcopy from copy import deepcopy
@ -64,7 +63,8 @@ class HyperoptTools():
'export_time': datetime.now(timezone.utc), 'export_time': datetime.now(timezone.utc),
} }
logger.info(f"Dumping parameters to {filename}") logger.info(f"Dumping parameters to {filename}")
rapidjson.dump(final_params, filename.open('w'), indent=2, with filename.open('w') as f:
rapidjson.dump(final_params, f, indent=2,
default=hyperopt_serializer, default=hyperopt_serializer,
number_mode=rapidjson.NM_NATIVE | rapidjson.NM_NAN number_mode=rapidjson.NM_NATIVE | rapidjson.NM_NAN
) )

View File

@ -7,11 +7,15 @@ class SKDecimal(Integer):
def __init__(self, low, high, decimals=3, prior="uniform", base=10, transform=None, def __init__(self, low, high, decimals=3, prior="uniform", base=10, transform=None,
name=None, dtype=np.int64): name=None, dtype=np.int64):
self.decimals = decimals self.decimals = decimals
_low = int(low * pow(10, self.decimals))
_high = int(high * pow(10, self.decimals)) self.pow_dot_one = pow(0.1, self.decimals)
self.pow_ten = pow(10, self.decimals)
_low = int(low * self.pow_ten)
_high = int(high * self.pow_ten)
# trunc to precision to avoid points out of space # trunc to precision to avoid points out of space
self.low_orig = round(_low * pow(0.1, self.decimals), self.decimals) self.low_orig = round(_low * self.pow_dot_one, self.decimals)
self.high_orig = round(_high * pow(0.1, self.decimals), self.decimals) self.high_orig = round(_high * self.pow_dot_one, self.decimals)
super().__init__(_low, _high, prior, base, transform, name, dtype) super().__init__(_low, _high, prior, base, transform, name, dtype)
@ -25,9 +29,9 @@ class SKDecimal(Integer):
return self.low_orig <= point <= self.high_orig return self.low_orig <= point <= self.high_orig
def transform(self, Xt): def transform(self, Xt):
aa = [int(x * pow(10, self.decimals)) for x in Xt] return super().transform([int(v * self.pow_ten) for v in Xt])
return super().transform(aa)
def inverse_transform(self, Xt): def inverse_transform(self, Xt):
res = super().inverse_transform(Xt) res = super().inverse_transform(Xt)
return [round(x * pow(0.1, self.decimals), self.decimals) for x in res] # equivalent to [round(x * pow(0.1, self.decimals), self.decimals) for x in res]
return [int(v) / self.pow_ten for v in res]

View File

@ -1123,7 +1123,7 @@ class PairLock(_DECL_BASE):
lock_time = self.lock_time.strftime(DATETIME_PRINT_FORMAT) lock_time = self.lock_time.strftime(DATETIME_PRINT_FORMAT)
lock_end_time = self.lock_end_time.strftime(DATETIME_PRINT_FORMAT) lock_end_time = self.lock_end_time.strftime(DATETIME_PRINT_FORMAT)
return (f'PairLock(id={self.id}, pair={self.pair}, lock_time={lock_time}, ' return (f'PairLock(id={self.id}, pair={self.pair}, lock_time={lock_time}, '
f'lock_end_time={lock_end_time})') f'lock_end_time={lock_end_time}, reason={self.reason}, active={self.active})')
@staticmethod @staticmethod
def query_pair_locks(pair: Optional[str], now: datetime) -> Query: def query_pair_locks(pair: Optional[str], now: datetime) -> Query:
@ -1132,7 +1132,6 @@ class PairLock(_DECL_BASE):
:param pair: Pair to check for. Returns all current locks if pair is empty :param pair: Pair to check for. Returns all current locks if pair is empty
:param now: Datetime object (generated via datetime.now(timezone.utc)). :param now: Datetime object (generated via datetime.now(timezone.utc)).
""" """
filters = [PairLock.lock_end_time > now, filters = [PairLock.lock_end_time > now,
# Only active locks # Only active locks
PairLock.active.is_(True), ] PairLock.active.is_(True), ]

View File

@ -103,6 +103,36 @@ class PairLocks():
if PairLocks.use_db: if PairLocks.use_db:
PairLock.query.session.commit() PairLock.query.session.commit()
@staticmethod
def unlock_reason(reason: str, now: Optional[datetime] = None) -> None:
"""
Release all locks for this reason.
:param reason: Which reason to unlock
:param now: Datetime object (generated via datetime.now(timezone.utc)).
defaults to datetime.now(timezone.utc)
"""
if not now:
now = datetime.now(timezone.utc)
if PairLocks.use_db:
# used in live modes
logger.info(f"Releasing all locks with reason '{reason}':")
filters = [PairLock.lock_end_time > now,
PairLock.active.is_(True),
PairLock.reason == reason
]
locks = PairLock.query.filter(*filters)
for lock in locks:
logger.info(f"Releasing lock for {lock.pair} with reason '{reason}'.")
lock.active = False
PairLock.query.session.commit()
else:
# used in backtesting mode; don't show log messages for speed
locks = PairLocks.get_pair_locks(None)
for lock in locks:
if lock.reason == reason:
lock.active = False
@staticmethod @staticmethod
def is_global_lock(now: Optional[datetime] = None) -> bool: def is_global_lock(now: Optional[datetime] = None) -> bool:
""" """
@ -128,7 +158,9 @@ class PairLocks():
@staticmethod @staticmethod
def get_all_locks() -> List[PairLock]: def get_all_locks() -> List[PairLock]:
"""
Return all locks, also locks with expired end date
"""
if PairLocks.use_db: if PairLocks.use_db:
return PairLock.query.all() return PairLock.query.all()
else: else:

View File

@ -91,7 +91,7 @@ class IResolver:
logger.debug(f"Searching for {cls.object_type.__name__} {object_name} in '{directory}'") logger.debug(f"Searching for {cls.object_type.__name__} {object_name} in '{directory}'")
for entry in directory.iterdir(): for entry in directory.iterdir():
# Only consider python files # Only consider python files
if not str(entry).endswith('.py'): if entry.suffix != '.py':
logger.debug('Ignoring %s', entry) logger.debug('Ignoring %s', entry)
continue continue
if entry.is_symlink() and not entry.is_file(): if entry.is_symlink() and not entry.is_file():
@ -169,7 +169,7 @@ class IResolver:
objects = [] objects = []
for entry in directory.iterdir(): for entry in directory.iterdir():
# Only consider python files # Only consider python files
if not str(entry).endswith('.py'): if entry.suffix != '.py':
logger.debug('Ignoring %s', entry) logger.debug('Ignoring %s', entry)
continue continue
module_path = entry.resolve() module_path = entry.resolve()

View File

@ -56,17 +56,21 @@ class StrategyResolver(IResolver):
if strategy._ft_params_from_file: if strategy._ft_params_from_file:
# Set parameters from Hyperopt results file # Set parameters from Hyperopt results file
params = strategy._ft_params_from_file params = strategy._ft_params_from_file
strategy.minimal_roi = params.get('roi', strategy.minimal_roi) strategy.minimal_roi = params.get('roi', getattr(strategy, 'minimal_roi', {}))
strategy.stoploss = params.get('stoploss', {}).get('stoploss', strategy.stoploss) strategy.stoploss = params.get('stoploss', {}).get(
'stoploss', getattr(strategy, 'stoploss', -0.1))
trailing = params.get('trailing', {}) trailing = params.get('trailing', {})
strategy.trailing_stop = trailing.get('trailing_stop', strategy.trailing_stop) strategy.trailing_stop = trailing.get(
strategy.trailing_stop_positive = trailing.get('trailing_stop_positive', 'trailing_stop', getattr(strategy, 'trailing_stop', False))
strategy.trailing_stop_positive) strategy.trailing_stop_positive = trailing.get(
'trailing_stop_positive', getattr(strategy, 'trailing_stop_positive', None))
strategy.trailing_stop_positive_offset = trailing.get( strategy.trailing_stop_positive_offset = trailing.get(
'trailing_stop_positive_offset', strategy.trailing_stop_positive_offset) 'trailing_stop_positive_offset',
getattr(strategy, 'trailing_stop_positive_offset', 0))
strategy.trailing_only_offset_is_reached = trailing.get( strategy.trailing_only_offset_is_reached = trailing.get(
'trailing_only_offset_is_reached', strategy.trailing_only_offset_is_reached) 'trailing_only_offset_is_reached',
getattr(strategy, 'trailing_only_offset_is_reached', 0.0))
# Set attributes # Set attributes
# Check if we need to override configuration # Check if we need to override configuration

View File

@ -1033,7 +1033,8 @@ class Telegram(RPCHandler):
:return: None :return: None
""" """
forcebuy_text = ("*/forcebuy <pair> [<rate>]:* `Instantly buys the given pair. " forcebuy_text = ("*/forcebuy <pair> [<rate>]:* `Instantly buys the given pair. "
"Optionally takes a rate at which to buy.` \n") "Optionally takes a rate at which to buy "
"(only applies to limit orders).` \n")
message = ("*/start:* `Starts the trader`\n" message = ("*/start:* `Starts the trader`\n"
"*/stop:* `Stops the trader`\n" "*/stop:* `Stops the trader`\n"
"*/status <trade_id>|[table]:* `Lists all open trades`\n" "*/status <trade_id>|[table]:* `Lists all open trades`\n"

View File

@ -381,7 +381,8 @@ class HyperStrategyMixin(object):
if filename.is_file(): if filename.is_file():
logger.info(f"Loading parameters from file {filename}") logger.info(f"Loading parameters from file {filename}")
try: try:
params = json_load(filename.open('r')) with filename.open('r') as f:
params = json_load(f)
if params.get('strategy_name') != self.__class__.__name__: if params.get('strategy_name') != self.__class__.__name__:
raise OperationalException('Invalid parameter file provided.') raise OperationalException('Invalid parameter file provided.')
return params return params

View File

@ -65,9 +65,9 @@ class IStrategy(ABC, HyperStrategyMixin):
_populate_fun_len: int = 0 _populate_fun_len: int = 0
_buy_fun_len: int = 0 _buy_fun_len: int = 0
_sell_fun_len: int = 0 _sell_fun_len: int = 0
_ft_params_from_file: Dict = {} _ft_params_from_file: Dict
# associated minimal roi # associated minimal roi
minimal_roi: Dict minimal_roi: Dict = {}
# associated stoploss # associated stoploss
stoploss: float stoploss: float
@ -462,6 +462,15 @@ class IStrategy(ABC, HyperStrategyMixin):
""" """
PairLocks.unlock_pair(pair, datetime.now(timezone.utc)) PairLocks.unlock_pair(pair, datetime.now(timezone.utc))
def unlock_reason(self, reason: str) -> None:
"""
Unlocks all pairs previously locked using lock_pair with specified reason.
Not used by freqtrade itself, but intended to be used if users lock pairs
manually from within the strategy, to allow an easy way to unlock pairs.
:param reason: Unlock pairs to allow trading again
"""
PairLocks.unlock_reason(reason, datetime.now(timezone.utc))
def is_pair_locked(self, pair: str, candle_date: datetime = None) -> bool: def is_pair_locked(self, pair: str, candle_date: datetime = None) -> bool:
""" """
Checks if a pair is currently locked Checks if a pair is currently locked

View File

@ -1,18 +1,18 @@
numpy==1.21.2 numpy==1.21.3
pandas==1.3.4 pandas==1.3.4
pandas-ta==0.3.14b pandas-ta==0.3.14b
ccxt==1.58.47 ccxt==1.59.2
# Pin cryptography for now due to rust build errors with piwheels # Pin cryptography for now due to rust build errors with piwheels
cryptography==35.0.0 cryptography==35.0.0
aiohttp==3.7.4.post0 aiohttp==3.7.4.post0
SQLAlchemy==1.4.25 SQLAlchemy==1.4.26
python-telegram-bot==13.7 python-telegram-bot==13.7
arrow==1.2.0 arrow==1.2.1
cachetools==4.2.2 cachetools==4.2.2
requests==2.26.0 requests==2.26.0
urllib3==1.26.7 urllib3==1.26.7
jsonschema==4.1.0 jsonschema==4.1.2
TA-Lib==0.4.21 TA-Lib==0.4.21
technical==1.3.0 technical==1.3.0
tabulate==0.8.9 tabulate==0.8.9
@ -41,7 +41,7 @@ psutil==5.8.0
colorama==0.4.4 colorama==0.4.4
# Building config files interactively # Building config files interactively
questionary==1.10.0 questionary==1.10.0
prompt-toolkit==3.0.20 prompt-toolkit==3.0.21
#Futures #Futures
schedule==1.1.0 schedule==1.1.0

View File

@ -209,7 +209,8 @@ def test_export_params(tmpdir):
assert filename.is_file() assert filename.is_file()
content = rapidjson.load(filename.open('r')) with filename.open('r') as f:
content = rapidjson.load(f)
assert content['strategy_name'] == CURRENT_TEST_STRATEGY assert content['strategy_name'] == CURRENT_TEST_STRATEGY
assert 'params' in content assert 'params' in content
assert "buy" in content["params"] assert "buy" in content["params"]

View File

@ -85,6 +85,8 @@ def test_loss_calculation_has_limited_profit(hyperopt_conf, hyperopt_results) ->
"SharpeHyperOptLoss", "SharpeHyperOptLoss",
"SharpeHyperOptLossDaily", "SharpeHyperOptLossDaily",
"MaxDrawDownHyperOptLoss", "MaxDrawDownHyperOptLoss",
"CalmarHyperOptLoss",
]) ])
def test_loss_functions_better_profits(default_conf, hyperopt_results, lossfunction) -> None: def test_loss_functions_better_profits(default_conf, hyperopt_results, lossfunction) -> None:
results_over = hyperopt_results.copy() results_over = hyperopt_results.copy()
@ -96,11 +98,32 @@ def test_loss_functions_better_profits(default_conf, hyperopt_results, lossfunct
default_conf.update({'hyperopt_loss': lossfunction}) default_conf.update({'hyperopt_loss': lossfunction})
hl = HyperOptLossResolver.load_hyperoptloss(default_conf) hl = HyperOptLossResolver.load_hyperoptloss(default_conf)
correct = hl.hyperopt_loss_function(hyperopt_results, len(hyperopt_results), correct = hl.hyperopt_loss_function(
datetime(2019, 1, 1), datetime(2019, 5, 1)) hyperopt_results,
over = hl.hyperopt_loss_function(results_over, len(results_over), trade_count=len(hyperopt_results),
datetime(2019, 1, 1), datetime(2019, 5, 1)) min_date=datetime(2019, 1, 1),
under = hl.hyperopt_loss_function(results_under, len(results_under), max_date=datetime(2019, 5, 1),
datetime(2019, 1, 1), datetime(2019, 5, 1)) config=default_conf,
processed=None,
backtest_stats={'profit_total': hyperopt_results['profit_abs'].sum()}
)
over = hl.hyperopt_loss_function(
results_over,
trade_count=len(results_over),
min_date=datetime(2019, 1, 1),
max_date=datetime(2019, 5, 1),
config=default_conf,
processed=None,
backtest_stats={'profit_total': results_over['profit_abs'].sum()}
)
under = hl.hyperopt_loss_function(
results_under,
trade_count=len(results_under),
min_date=datetime(2019, 1, 1),
max_date=datetime(2019, 5, 1),
config=default_conf,
processed=None,
backtest_stats={'profit_total': results_under['profit_abs'].sum()}
)
assert over < correct assert over < correct
assert under > correct assert under > correct

View File

@ -116,3 +116,28 @@ def test_PairLocks_getlongestlock(use_db):
PairLocks.reset_locks() PairLocks.reset_locks()
PairLocks.use_db = True PairLocks.use_db = True
@pytest.mark.parametrize('use_db', (False, True))
@pytest.mark.usefixtures("init_persistence")
def test_PairLocks_reason(use_db):
PairLocks.timeframe = '5m'
PairLocks.use_db = use_db
# No lock should be present
if use_db:
assert len(PairLock.query.all()) == 0
assert PairLocks.use_db == use_db
PairLocks.lock_pair('XRP/USDT', arrow.utcnow().shift(minutes=4).datetime, 'TestLock1')
PairLocks.lock_pair('ETH/USDT', arrow.utcnow().shift(minutes=4).datetime, 'TestLock2')
assert PairLocks.is_pair_locked('XRP/USDT')
assert PairLocks.is_pair_locked('ETH/USDT')
PairLocks.unlock_reason('TestLock1')
assert not PairLocks.is_pair_locked('XRP/USDT')
assert PairLocks.is_pair_locked('ETH/USDT')
PairLocks.reset_locks()
PairLocks.use_db = True

View File

@ -633,6 +633,13 @@ def test_is_pair_locked(default_conf):
strategy.unlock_pair(pair) strategy.unlock_pair(pair)
assert not strategy.is_pair_locked(pair) assert not strategy.is_pair_locked(pair)
# Lock with reason
reason = "TestLockR"
strategy.lock_pair(pair, arrow.now(timezone.utc).shift(minutes=4).datetime, reason)
assert strategy.is_pair_locked(pair)
strategy.unlock_reason(reason)
assert not strategy.is_pair_locked(pair)
pair = 'BTC/USDT' pair = 'BTC/USDT'
# Lock until 14:30 # Lock until 14:30
lock_time = datetime(2020, 5, 1, 14, 30, 0, tzinfo=timezone.utc) lock_time = datetime(2020, 5, 1, 14, 30, 0, tzinfo=timezone.utc)

View File

@ -62,8 +62,8 @@ def test_load_strategy(default_conf, result):
def test_load_strategy_base64(result, caplog, default_conf): def test_load_strategy_base64(result, caplog, default_conf):
with (Path(__file__).parents[2] / 'freqtrade/templates/sample_strategy.py').open("rb") as file: filepath = Path(__file__).parents[2] / 'freqtrade/templates/sample_strategy.py'
encoded_string = urlsafe_b64encode(file.read()).decode("utf-8") encoded_string = urlsafe_b64encode(filepath.read_bytes()).decode("utf-8")
default_conf.update({'strategy': 'SampleStrategy:{}'.format(encoded_string)}) default_conf.update({'strategy': 'SampleStrategy:{}'.format(encoded_string)})
strategy = StrategyResolver.load_strategy(default_conf) strategy = StrategyResolver.load_strategy(default_conf)