Merge pull request #2 from freqtrade/develop

update develop from base repo
This commit is contained in:
wr0ngc0degen 2021-04-25 05:50:03 +02:00 committed by GitHub
commit 869a45a031
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 312 additions and 112 deletions

View File

@ -11,7 +11,16 @@ Per default, the bot loads the configuration from the `config.json` file, locate
You can specify a different configuration file used by the bot with the `-c/--config` command line option. You can specify a different configuration file used by the bot with the `-c/--config` command line option.
In some advanced use cases, multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream. Multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
!!! Tip "Use multiple configuration files to keep secrets secret"
You can use a 2nd configuration file containing your secrets. That way you can share your "primary" configuration file, while still keeping your API keys for yourself.
``` bash
freqtrade trade --config user_data/config.json --config user_data/config-private.json <...>
```
The 2nd file should only specify what you intend to override.
If a key is in more than one of the configurations, then the "last specified configuration" wins (in the above example, `config-private.json`).
If you used the [Quick start](installation.md/#quick-start) method for installing If you used the [Quick start](installation.md/#quick-start) method for installing
the bot, the installation script should have already created the default configuration file (`config.json`) for you. the bot, the installation script should have already created the default configuration file (`config.json`) for you.
@ -518,16 +527,27 @@ API Keys are usually only required for live trading (trading for real money, bot
**Insert your Exchange API key (change them by fake api keys):** **Insert your Exchange API key (change them by fake api keys):**
```json ```json
{
"exchange": { "exchange": {
"name": "bittrex", "name": "bittrex",
"key": "af8ddd35195e9dc500b9a6f799f6f5c93d89193b", "key": "af8ddd35195e9dc500b9a6f799f6f5c93d89193b",
"secret": "08a9dc6db3d7b53e1acebd9275677f4b0a04f1a5", "secret": "08a9dc6db3d7b53e1acebd9275677f4b0a04f1a5",
... //"password": "", // Optional, not needed by all exchanges)
// ...
}
//...
} }
``` ```
You should also make sure to read the [Exchanges](exchanges.md) section of the documentation to be aware of potential configuration details specific to your exchange. You should also make sure to read the [Exchanges](exchanges.md) section of the documentation to be aware of potential configuration details specific to your exchange.
!!! Hint "Keep your secrets secret"
To keep your secrets secret, we recommend to use a 2nd configuration for your API keys.
Simply use the above snippet in a new configuration file (e.g. `config-private.json`) and keep your settings in this file.
You can then start the bot with `freqtrade trade --config user_data/config.json --config user_data/config-private.json <...>` to have your keys loaded.
**NEVER** share your private configuration file or your exchange keys with anyone!
### Using proxy with Freqtrade ### Using proxy with Freqtrade
To use a proxy with freqtrade, add the kwarg `"aiohttp_trust_env"=true` to the `"ccxt_async_kwargs"` dict in the exchange section of the configuration. To use a proxy with freqtrade, add the kwarg `"aiohttp_trust_env"=true` to the `"ccxt_async_kwargs"` dict in the exchange section of the configuration.

View File

@ -165,11 +165,22 @@ Rarely you may also need to create a [nested class](advanced-hyperopt.md#overrid
!!! Tip "Quickly optimize ROI, stoploss and trailing stoploss" !!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything in your strategy. You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything in your strategy.
```python ``` bash
# Have a working strategy at hand. # Have a working strategy at hand.
freqtrade hyperopt --hyperopt-loss SharpeHyperOptLossDaily --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100 freqtrade hyperopt --hyperopt-loss SharpeHyperOptLossDaily --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100
``` ```
### Hyperopt execution logic
Hyperopt will first load your data into memory and will then run `populate_indicators()` once per Pair to generate all indicators.
Hyperopt will then spawn into different processes (number of processors, or `-j <n>`), and run backtesting over and over again, changing the parameters that are part of the `--spaces` defined.
For every new set of parameters, freqtrade will run first `populate_buy_trend()` followed by `populate_sell_trend()`, and then run the regular backtesting process to simulate trades.
After backtesting, the results are passed into the [loss function](#loss-functions), which will evaluate if this result was better or worse than previous results.
Based on the loss function result, hyperopt will determine the next set of parameters to try in the next round of backtesting.
### Configure your Guards and Triggers ### Configure your Guards and Triggers
There are two places you need to change in your strategy file to add a new buy hyperopt for testing: There are two places you need to change in your strategy file to add a new buy hyperopt for testing:
@ -187,60 +198,54 @@ There you have two different types of indicators: 1. `guards` and 2. `triggers`.
However, this guide will make this distinction to make it clear that signals should not be "sticking". However, this guide will make this distinction to make it clear that signals should not be "sticking".
Sticking signals are signals that are active for multiple candles. This can lead into buying a signal late (right before the signal disappears - which means that the chance of success is a lot lower than right at the beginning). Sticking signals are signals that are active for multiple candles. This can lead into buying a signal late (right before the signal disappears - which means that the chance of success is a lot lower than right at the beginning).
Hyper-optimization will, for each epoch round, pick one trigger and possibly Hyper-optimization will, for each epoch round, pick one trigger and possibly multiple guards.
multiple guards. The constructed strategy will be something like "*buy exactly when close price touches lower Bollinger band, BUT only if
ADX > 10*".
```python
from freqtrade.strategy import IntParameter, IStrategy
class MyAwesomeStrategy(IStrategy):
# If parameter is prefixed with `buy_` or `sell_` then specifying `space` parameter is optional
# and space is inferred from parameter name.
buy_adx_min = IntParameter(0, 100, default=10)
def populate_buy_trend(self, dataframe: 'DataFrame', metadata: dict) -> 'DataFrame':
dataframe.loc[
(
(dataframe['adx'] > self.buy_adx_min.value)
), 'buy'] = 1
return dataframe
```
#### Sell optimization #### Sell optimization
Similar to the buy-signal above, sell-signals can also be optimized. Similar to the buy-signal above, sell-signals can also be optimized.
Place the corresponding settings into the following methods Place the corresponding settings into the following methods
* Define the parameters at the class level hyperopt shall be optimizing. * Define the parameters at the class level hyperopt shall be optimizing, either naming them `sell_*`, or by explicitly defining `space='sell'`.
* Within `populate_sell_trend()` - use defined parameter values instead of raw constants. * Within `populate_sell_trend()` - use defined parameter values instead of raw constants.
The configuration and rules are the same than for buy signals. The configuration and rules are the same than for buy signals.
## Solving a Mystery
Let's say you are curious: should you use MACD crossings or lower Bollinger Bands to trigger your buys.
And you also wonder should you use RSI or ADX to help with those buy decisions.
If you decide to use RSI or ADX, which values should I use for them?
So let's use hyperparameter optimization to solve this mystery.
### Defining indicators to be used
We start by calculating the indicators our strategy is going to use.
``` python ``` python
class MyAwesomeStrategy(IStrategy): class MyAwesomeStrategy(IStrategy):
# There is no strict parameter naming scheme. If you do not use `buy_` or `sell_` prefixes -
# please specify to which space parameter belongs using `space` parameter. Possible values:
# 'buy' or 'sell'.
adx_max = IntParameter(0, 100, default=50, space='sell')
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe.loc[ """
( Generate all indicators used by the strategy
(dataframe['adx'] < self.adx_max.value) """
), 'buy'] = 1 dataframe['adx'] = ta.ADX(dataframe)
dataframe['rsi'] = ta.RSI(dataframe)
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['macdhist'] = macd['macdhist']
bollinger = ta.BBANDS(dataframe, timeperiod=20, nbdevup=2.0, nbdevdn=2.0)
dataframe['bb_lowerband'] = boll['lowerband']
dataframe['bb_middleband'] = boll['middleband']
dataframe['bb_upperband'] = boll['upperband']
return dataframe return dataframe
``` ```
## Solving a Mystery ### Hyperoptable parameters
Let's say you are curious: should you use MACD crossings or lower Bollinger We continue to define hyperoptable parameters:
Bands to trigger your buys. And you also wonder should you use RSI or ADX to
help with those buy decisions. If you decide to use RSI or ADX, which values
should I use for them? So let's use hyperparameter optimization to solve this
mystery.
We will start by defining hyperoptable parameters:
```python ```python
class MyAwesomeStrategy(IStrategy): class MyAwesomeStrategy(IStrategy):
@ -288,7 +293,7 @@ So let's write the buy strategy using these values:
``` ```
Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations. Hyperopt will now call `populate_buy_trend()` many times (`epochs`) with different value combinations.
It will use the given historical data and make buys based on the buy signals generated with the above function. It will use the given historical data and simulate buys based on the buy signals generated with the above function.
Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured [loss function](#loss-functions)). Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured [loss function](#loss-functions)).
!!! Note !!! Note
@ -314,6 +319,90 @@ There are four parameter types each suited for different purposes.
!!! Warning !!! Warning
Hyperoptable parameters cannot be used in `populate_indicators` - as hyperopt does not recalculate indicators for each epoch, so the starting value would be used in this case. Hyperoptable parameters cannot be used in `populate_indicators` - as hyperopt does not recalculate indicators for each epoch, so the starting value would be used in this case.
### Optimizing an indicator parameter
Assuming you have a simple strategy in mind - a EMA cross strategy (2 Moving averages crossing) - and you'd like to find the ideal parameters for this strategy.
``` python
from pandas import DataFrame
from functools import reduce
import talib.abstract as ta
from freqtrade.strategy import IStrategy
from freqtrade.strategy import CategoricalParameter, DecimalParameter, IntParameter
import freqtrade.vendor.qtpylib.indicators as qtpylib
class MyAwesomeStrategy(IStrategy):
stoploss = -0.05
timeframe = '15m'
# Define the parameter spaces
buy_ema_short = IntParameter(3, 50, default=5)
buy_ema_long = IntParameter(15, 200, default=50)
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""Generate all indicators used by the strategy"""
# Calculate all ema_short values
for val in self.buy_ema_short.range:
dataframe[f'ema_short_{val}'] = ta.EMA(dataframe, timeperiod=val)
# Calculate all ema_long values
for val in self.buy_ema_long.range:
dataframe[f'ema_long_{val}'] = ta.EMA(dataframe, timeperiod=val)
return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = []
conditions.append(qtpylib.crossed_above(
dataframe[f'ema_short_{self.buy_ema_short.value}'], dataframe[f'ema_long_{self.buy_ema_long.value}']
))
# Check that volume is not 0
conditions.append(dataframe['volume'] > 0)
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
conditions = []
conditions.append(qtpylib.crossed_above(
dataframe[f'ema_long_{self.buy_ema_long.value}'], dataframe[f'ema_short_{self.buy_ema_short.value}']
))
# Check that volume is not 0
conditions.append(dataframe['volume'] > 0)
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'sell'] = 1
return dataframe
```
Breaking it down:
Using `self.buy_ema_short.range` will return a range object containing all entries between the Parameters low and high value.
In this case (`IntParameter(3, 50, default=5)`), the loop would run for all numbers between 3 and 50 (`[3, 4, 5, ... 49, 50]`).
By using this in a loop, hyperopt will generate 48 new columns (`['buy_ema_3', 'buy_ema_4', ... , 'buy_ema_50']`).
Hyperopt itself will then use the selected value to create the buy and sell signals
While this strategy is most likely too simple to provide consistent profit, it should serve as an example how optimize indicator parameters.
!!! Note
`self.buy_ema_short.range` will act differently between hyperopt and other modes. For hyperopt, the above example may generate 48 new columns, however for all other modes (backtesting, dry/live), it will only generate the column for the selected value. You should therefore avoid using the resulting column with explicit values (values other than `self.buy_ema_short.value`).
??? Hint "Performance tip"
By doing the calculation of all possible indicators in `populate_indicators()`, the calculation of the indicator happens only once for every parameter.
While this may slow down the hyperopt startup speed, the overall performance will increase as the Hyperopt execution itself may pick the same value for multiple epochs (changing other values).
You should however try to use space ranges as small as possible. Every new column will require more memory, and every possibility hyperopt can try will increase the search space.
## Loss-functions ## Loss-functions
Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results. Each hyperparameter tuning requires a target. This is usually defined as a loss function (sometimes also called objective function), which should decrease for more desirable results, and increase for bad results.
@ -606,6 +695,16 @@ number).
You can also enable position stacking in the configuration file by explicitly setting You can also enable position stacking in the configuration file by explicitly setting
`"position_stacking"=true`. `"position_stacking"=true`.
## Out of Memory errors
As hyperopt consumes a lot of memory (the complete data needs to be in memory once per parallel backtesting process), it's likely that you run into "out of memory" errors.
To combat these, you have multiple options:
* reduce the amount of pairs
* reduce the timerange used (`--timerange <timerange>`)
* reduce the number of parallel processes (`-j <n>`)
* Increase the memory of your machine
## Show details of Hyperopt results ## Show details of Hyperopt results
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter. After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter.

View File

@ -195,4 +195,18 @@ graph.show(renderer="browser")
``` ```
## Plot average profit per trade as distribution graph
```python
import plotly.figure_factory as ff
hist_data = [trades.profit_ratio]
group_labels = ['profit_ratio'] # name of the dataset
fig = ff.create_distplot(hist_data, group_labels,bin_size=0.01)
fig.show()
```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data. Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

View File

@ -75,8 +75,6 @@ class Configuration:
# Normalize config # Normalize config
if 'internals' not in config: if 'internals' not in config:
config['internals'] = {} config['internals'] = {}
# TODO: This can be deleted along with removal of deprecated
# experimental settings
if 'ask_strategy' not in config: if 'ask_strategy' not in config:
config['ask_strategy'] = {} config['ask_strategy'] = {}

View File

@ -89,7 +89,7 @@ class HDF5DataHandler(IDataHandler):
if timerange.starttype == 'date': if timerange.starttype == 'date':
where.append(f"date >= Timestamp({timerange.startts * 1e9})") where.append(f"date >= Timestamp({timerange.startts * 1e9})")
if timerange.stoptype == 'date': if timerange.stoptype == 'date':
where.append(f"date < Timestamp({timerange.stopts * 1e9})") where.append(f"date <= Timestamp({timerange.stopts * 1e9})")
pairdata = pd.read_hdf(filename, key=key, mode="r", where=where) pairdata = pd.read_hdf(filename, key=key, mode="r", where=where)

View File

@ -363,7 +363,6 @@ class Exchange:
invalid_pairs = [] invalid_pairs = []
for pair in extended_pairs: for pair in extended_pairs:
# Note: ccxt has BaseCurrency/QuoteCurrency format for pairs # Note: ccxt has BaseCurrency/QuoteCurrency format for pairs
# TODO: add a support for having coins in BTC/USDT format
if self.markets and pair not in self.markets: if self.markets and pair not in self.markets:
raise OperationalException( raise OperationalException(
f'Pair {pair} is not available on {self.name}. ' f'Pair {pair} is not available on {self.name}. '

View File

@ -473,8 +473,7 @@ class FreqtradeBot(LoggingMixin):
(buy, sell) = self.strategy.get_signal(pair, self.strategy.timeframe, analyzed_df) (buy, sell) = self.strategy.get_signal(pair, self.strategy.timeframe, analyzed_df)
if buy and not sell: if buy and not sell:
stake_amount = self.wallets.get_trade_stake_amount(pair, self.get_free_open_trades(), stake_amount = self.wallets.get_trade_stake_amount(pair, self.edge)
self.edge)
if not stake_amount: if not stake_amount:
logger.debug(f"Stake amount is 0, ignoring possible trade for {pair}.") logger.debug(f"Stake amount is 0, ignoring possible trade for {pair}.")
return False return False

View File

@ -273,11 +273,9 @@ class Backtesting:
return None return None
def _enter_trade(self, pair: str, row: List, max_open_trades: int, def _enter_trade(self, pair: str, row: List) -> Optional[LocalTrade]:
open_trade_count: int) -> Optional[LocalTrade]:
try: try:
stake_amount = self.wallets.get_trade_stake_amount( stake_amount = self.wallets.get_trade_stake_amount(pair, None)
pair, max_open_trades - open_trade_count, None)
except DependencyException: except DependencyException:
return None return None
min_stake_amount = self.exchange.get_min_pair_stake_amount(pair, row[OPEN_IDX], -0.05) min_stake_amount = self.exchange.get_min_pair_stake_amount(pair, row[OPEN_IDX], -0.05)
@ -354,7 +352,7 @@ class Backtesting:
data: Dict = self._get_ohlcv_as_lists(processed) data: Dict = self._get_ohlcv_as_lists(processed)
# Indexes per pair, so some pairs are allowed to have a missing start. # Indexes per pair, so some pairs are allowed to have a missing start.
indexes: Dict = {} indexes: Dict = defaultdict(int)
tmp = start_date + timedelta(minutes=self.timeframe_min) tmp = start_date + timedelta(minutes=self.timeframe_min)
open_trades: Dict[str, List[LocalTrade]] = defaultdict(list) open_trades: Dict[str, List[LocalTrade]] = defaultdict(list)
@ -365,9 +363,6 @@ class Backtesting:
open_trade_count_start = open_trade_count open_trade_count_start = open_trade_count
for i, pair in enumerate(data): for i, pair in enumerate(data):
if pair not in indexes:
indexes[pair] = 0
try: try:
row = data[pair][indexes[pair]] row = data[pair][indexes[pair]]
except IndexError: except IndexError:
@ -388,7 +383,7 @@ class Backtesting:
and tmp != end_date and tmp != end_date
and row[BUY_IDX] == 1 and row[SELL_IDX] != 1 and row[BUY_IDX] == 1 and row[SELL_IDX] != 1
and not PairLocks.is_pair_locked(pair, row[DATE_IDX])): and not PairLocks.is_pair_locked(pair, row[DATE_IDX])):
trade = self._enter_trade(pair, row, max_open_trades, open_trade_count_start) trade = self._enter_trade(pair, row)
if trade: if trade:
# TODO: hacky workaround to avoid opening > max_open_trades # TODO: hacky workaround to avoid opening > max_open_trades
# This emulates previous behaviour - not sure if this is correct # This emulates previous behaviour - not sure if this is correct

View File

@ -607,8 +607,7 @@ class RPC:
raise RPCException(f'position for {pair} already open - id: {trade.id}') raise RPCException(f'position for {pair} already open - id: {trade.id}')
# gen stake amount # gen stake amount
stakeamount = self._freqtrade.wallets.get_trade_stake_amount( stakeamount = self._freqtrade.wallets.get_trade_stake_amount(pair)
pair, self._freqtrade.get_free_open_trades())
# execute buy # execute buy
if self._freqtrade.execute_buy(pair, stakeamount, price, forcebuy=True): if self._freqtrade.execute_buy(pair, stakeamount, price, forcebuy=True):

View File

@ -5,7 +5,7 @@ This module defines a base class for auto-hyperoptable strategies.
import logging import logging
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from contextlib import suppress from contextlib import suppress
from typing import Any, Iterator, Optional, Sequence, Tuple, Union from typing import Any, Dict, Iterator, Optional, Sequence, Tuple, Union
with suppress(ImportError): with suppress(ImportError):
@ -13,6 +13,7 @@ with suppress(ImportError):
from freqtrade.optimize.space import SKDecimal from freqtrade.optimize.space import SKDecimal
from freqtrade.exceptions import OperationalException from freqtrade.exceptions import OperationalException
from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -25,6 +26,7 @@ class BaseParameter(ABC):
category: Optional[str] category: Optional[str]
default: Any default: Any
value: Any value: Any
hyperopt: bool = False
def __init__(self, *, default: Any, space: Optional[str] = None, def __init__(self, *, default: Any, space: Optional[str] = None,
optimize: bool = True, load: bool = True, **kwargs): optimize: bool = True, load: bool = True, **kwargs):
@ -121,6 +123,20 @@ class IntParameter(NumericParameter):
""" """
return Integer(low=self.low, high=self.high, name=name, **self._space_params) return Integer(low=self.low, high=self.high, name=name, **self._space_params)
@property
def range(self):
"""
Get each value in this space as list.
Returns a List from low to high (inclusive) in Hyperopt mode.
Returns a List with 1 item (`value`) in "non-hyperopt" mode, to avoid
calculating 100ds of indicators.
"""
if self.hyperopt:
# Scikit-optimize ranges are "inclusive", while python's "range" is exclusive
return range(self.low, self.high + 1)
else:
return range(self.value, self.value + 1)
class RealParameter(NumericParameter): class RealParameter(NumericParameter):
default: float default: float
@ -227,12 +243,11 @@ class HyperStrategyMixin(object):
strategy logic. strategy logic.
""" """
def __init__(self, *args, **kwargs): def __init__(self, config: Dict[str, Any], *args, **kwargs):
""" """
Initialize hyperoptable strategy mixin. Initialize hyperoptable strategy mixin.
""" """
self._load_params(getattr(self, 'buy_params', None)) self._load_hyper_params(config.get('runmode') == RunMode.HYPEROPT)
self._load_params(getattr(self, 'sell_params', None))
def enumerate_parameters(self, category: str = None) -> Iterator[Tuple[str, BaseParameter]]: def enumerate_parameters(self, category: str = None) -> Iterator[Tuple[str, BaseParameter]]:
""" """
@ -254,18 +269,29 @@ class HyperStrategyMixin(object):
(attr_name.startswith(category + '_') and attr.category is None)): (attr_name.startswith(category + '_') and attr.category is None)):
yield attr_name, attr yield attr_name, attr
def _load_params(self, params: dict) -> None: def _load_hyper_params(self, hyperopt: bool = False) -> None:
"""
Load Hyperoptable parameters
"""
self._load_params(getattr(self, 'buy_params', None), 'buy', hyperopt)
self._load_params(getattr(self, 'sell_params', None), 'sell', hyperopt)
def _load_params(self, params: dict, space: str, hyperopt: bool = False) -> None:
""" """
Set optimizeable parameter values. Set optimizeable parameter values.
:param params: Dictionary with new parameter values. :param params: Dictionary with new parameter values.
""" """
if not params: if not params:
return logger.info(f"No params for {space} found, using default values.")
for attr_name, attr in self.enumerate_parameters(): for attr_name, attr in self.enumerate_parameters():
if attr_name in params: attr.hyperopt = hyperopt
if params and attr_name in params:
if attr.load: if attr.load:
attr.value = params[attr_name] attr.value = params[attr_name]
logger.info(f'Strategy Parameter: {attr_name} = {attr.value}') logger.info(f'Strategy Parameter: {attr_name} = {attr.value}')
else: else:
logger.warning(f'Parameter "{attr_name}" exists, but is disabled. ' logger.warning(f'Parameter "{attr_name}" exists, but is disabled. '
f'Default value "{attr.value}" used.') f'Default value "{attr.value}" used.')
else:
logger.info(f'Strategy Parameter(default): {attr_name} = {attr.value}')

View File

@ -282,6 +282,28 @@
"graph.show(renderer=\"browser\")\n" "graph.show(renderer=\"browser\")\n"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plot average profit per trade as distribution graph"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import plotly.figure_factory as ff\n",
"\n",
"hist_data = [trades.profit_ratio]\n",
"group_labels = ['profit_ratio'] # name of the dataset\n",
"\n",
"fig = ff.create_distplot(hist_data, group_labels,bin_size=0.01)\n",
"fig.show()\n"
]
},
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},

View File

@ -130,14 +130,13 @@ class Wallets:
def get_all_balances(self) -> Dict[str, Any]: def get_all_balances(self) -> Dict[str, Any]:
return self._wallets return self._wallets
def _get_available_stake_amount(self) -> float: def _get_available_stake_amount(self, val_tied_up: float) -> float:
""" """
Return the total currently available balance in stake currency, Return the total currently available balance in stake currency,
respecting tradable_balance_ratio. respecting tradable_balance_ratio.
Calculated as Calculated as
(<open_trade stakes> + free amount) * tradable_balance_ratio - <open_trade stakes> (<open_trade stakes> + free amount) * tradable_balance_ratio - <open_trade stakes>
""" """
val_tied_up = Trade.total_open_trades_stakes()
# Ensure <tradable_balance_ratio>% is used from the overall balance # Ensure <tradable_balance_ratio>% is used from the overall balance
# Otherwise we'd risk lowering stakes with each open trade. # Otherwise we'd risk lowering stakes with each open trade.
@ -146,26 +145,26 @@ class Wallets:
self._config['tradable_balance_ratio']) - val_tied_up self._config['tradable_balance_ratio']) - val_tied_up
return available_amount return available_amount
def _calculate_unlimited_stake_amount(self, free_open_trades: int) -> float: def _calculate_unlimited_stake_amount(self, available_amount: float,
val_tied_up: float) -> float:
""" """
Calculate stake amount for "unlimited" stake amount Calculate stake amount for "unlimited" stake amount
:return: 0 if max number of trades reached, else stake_amount to use. :return: 0 if max number of trades reached, else stake_amount to use.
""" """
if not free_open_trades: if self._config['max_open_trades'] == 0:
return 0 return 0
available_amount = self._get_available_stake_amount() possible_stake = (available_amount + val_tied_up) / self._config['max_open_trades']
# Theoretical amount can be above available amount - therefore limit to available amount!
return min(possible_stake, available_amount)
return available_amount / free_open_trades def _check_available_stake_amount(self, stake_amount: float, available_amount: float) -> float:
def _check_available_stake_amount(self, stake_amount: float) -> float:
""" """
Check if stake amount can be fulfilled with the available balance Check if stake amount can be fulfilled with the available balance
for the stake currency for the stake currency
:return: float: Stake amount :return: float: Stake amount
:raise: DependencyException if balance is lower than stake-amount :raise: DependencyException if balance is lower than stake-amount
""" """
available_amount = self._get_available_stake_amount()
if self._config['amend_last_stake_amount']: if self._config['amend_last_stake_amount']:
# Remaining amount needs to be at least stake_amount * last_stake_amount_min_ratio # Remaining amount needs to be at least stake_amount * last_stake_amount_min_ratio
@ -183,7 +182,7 @@ class Wallets:
return stake_amount return stake_amount
def get_trade_stake_amount(self, pair: str, free_open_trades: int, edge=None) -> float: def get_trade_stake_amount(self, pair: str, edge=None) -> float:
""" """
Calculate stake amount for the trade Calculate stake amount for the trade
:return: float: Stake amount :return: float: Stake amount
@ -192,17 +191,20 @@ class Wallets:
stake_amount: float stake_amount: float
# Ensure wallets are uptodate. # Ensure wallets are uptodate.
self.update() self.update()
val_tied_up = Trade.total_open_trades_stakes()
available_amount = self._get_available_stake_amount(val_tied_up)
if edge: if edge:
stake_amount = edge.stake_amount( stake_amount = edge.stake_amount(
pair, pair,
self.get_free(self._config['stake_currency']), self.get_free(self._config['stake_currency']),
self.get_total(self._config['stake_currency']), self.get_total(self._config['stake_currency']),
Trade.total_open_trades_stakes() val_tied_up
) )
else: else:
stake_amount = self._config['stake_amount'] stake_amount = self._config['stake_amount']
if stake_amount == UNLIMITED_STAKE_AMOUNT: if stake_amount == UNLIMITED_STAKE_AMOUNT:
stake_amount = self._calculate_unlimited_stake_amount(free_open_trades) stake_amount = self._calculate_unlimited_stake_amount(
available_amount, val_tied_up)
return self._check_available_stake_amount(stake_amount) return self._check_available_stake_amount(stake_amount, available_amount)

View File

@ -6,6 +6,7 @@ import pandas as pd
import pytest import pytest
from freqtrade.optimize.hyperopt import Hyperopt from freqtrade.optimize.hyperopt import Hyperopt
from freqtrade.state import RunMode
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from tests.conftest import patch_exchange from tests.conftest import patch_exchange
@ -15,6 +16,7 @@ def hyperopt_conf(default_conf):
hyperconf = deepcopy(default_conf) hyperconf = deepcopy(default_conf)
hyperconf.update({ hyperconf.update({
'datadir': Path(default_conf['datadir']), 'datadir': Path(default_conf['datadir']),
'runmode': RunMode.HYPEROPT,
'hyperopt': 'DefaultHyperOpt', 'hyperopt': 'DefaultHyperOpt',
'hyperopt_loss': 'ShortTradeDurHyperOptLoss', 'hyperopt_loss': 'ShortTradeDurHyperOptLoss',
'hyperopt_path': str(Path(__file__).parent / 'hyperopts'), 'hyperopt_path': str(Path(__file__).parent / 'hyperopts'),

View File

@ -457,12 +457,13 @@ def test_backtesting_pairlist_list(default_conf, mocker, caplog, testdatadir, ti
Backtesting(default_conf) Backtesting(default_conf)
def test_backtest__enter_trade(default_conf, fee, mocker, testdatadir) -> None: def test_backtest__enter_trade(default_conf, fee, mocker) -> None:
default_conf['ask_strategy']['use_sell_signal'] = False default_conf['ask_strategy']['use_sell_signal'] = False
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
mocker.patch("freqtrade.exchange.Exchange.get_min_pair_stake_amount", return_value=0.00001) mocker.patch("freqtrade.exchange.Exchange.get_min_pair_stake_amount", return_value=0.00001)
patch_exchange(mocker) patch_exchange(mocker)
default_conf['stake_amount'] = 'unlimited' default_conf['stake_amount'] = 'unlimited'
default_conf['max_open_trades'] = 2
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
pair = 'UNITTEST/BTC' pair = 'UNITTEST/BTC'
row = [ row = [
@ -474,24 +475,30 @@ def test_backtest__enter_trade(default_conf, fee, mocker, testdatadir) -> None:
0.00099, # Low 0.00099, # Low
0.0012, # High 0.0012, # High
] ]
trade = backtesting._enter_trade(pair, row=row, max_open_trades=2, open_trade_count=0) trade = backtesting._enter_trade(pair, row=row)
assert isinstance(trade, LocalTrade) assert isinstance(trade, LocalTrade)
assert trade.stake_amount == 495 assert trade.stake_amount == 495
trade = backtesting._enter_trade(pair, row=row, max_open_trades=2, open_trade_count=2) # Fake 2 trades, so there's not enough amount for the next trade left.
LocalTrade.trades_open.append(trade)
LocalTrade.trades_open.append(trade)
trade = backtesting._enter_trade(pair, row=row)
assert trade is None assert trade is None
LocalTrade.trades_open.pop()
trade = backtesting._enter_trade(pair, row=row)
assert trade is not None
# Stake-amount too high! # Stake-amount too high!
mocker.patch("freqtrade.exchange.Exchange.get_min_pair_stake_amount", return_value=600.0) mocker.patch("freqtrade.exchange.Exchange.get_min_pair_stake_amount", return_value=600.0)
trade = backtesting._enter_trade(pair, row=row, max_open_trades=2, open_trade_count=0) trade = backtesting._enter_trade(pair, row=row)
assert trade is None assert trade is None
# Stake-amount too high! # Stake-amount throwing error
mocker.patch("freqtrade.wallets.Wallets.get_trade_stake_amount", mocker.patch("freqtrade.wallets.Wallets.get_trade_stake_amount",
side_effect=DependencyException) side_effect=DependencyException)
trade = backtesting._enter_trade(pair, row=row, max_open_trades=2, open_trade_count=0) trade = backtesting._enter_trade(pair, row=row)
assert trade is None assert trade is None

View File

@ -21,6 +21,7 @@ from freqtrade.optimize.hyperopt_tools import HyperoptTools
from freqtrade.optimize.space import SKDecimal from freqtrade.optimize.space import SKDecimal
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.strategy.hyper import IntParameter
from tests.conftest import (get_args, log_has, log_has_re, patch_exchange, from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patched_configuration_load_config_file) patched_configuration_load_config_file)
@ -1103,6 +1104,14 @@ def test_in_strategy_auto_hyperopt(mocker, hyperopt_conf, tmpdir) -> None:
}) })
hyperopt = Hyperopt(hyperopt_conf) hyperopt = Hyperopt(hyperopt_conf)
assert isinstance(hyperopt.custom_hyperopt, HyperOptAuto) assert isinstance(hyperopt.custom_hyperopt, HyperOptAuto)
assert isinstance(hyperopt.backtesting.strategy.buy_rsi, IntParameter)
assert hyperopt.backtesting.strategy.buy_rsi.hyperopt is True
assert hyperopt.backtesting.strategy.buy_rsi.value == 35
buy_rsi_range = hyperopt.backtesting.strategy.buy_rsi.range
assert isinstance(buy_rsi_range, range)
# Range from 0 - 50 (inclusive)
assert len(list(buy_rsi_range)) == 51
hyperopt.start() hyperopt.start()

View File

@ -588,6 +588,14 @@ def test_hyperopt_parameters():
intpar = IntParameter(low=0, high=5, default=1, space='buy') intpar = IntParameter(low=0, high=5, default=1, space='buy')
assert intpar.value == 1 assert intpar.value == 1
assert isinstance(intpar.get_space(''), Integer) assert isinstance(intpar.get_space(''), Integer)
assert isinstance(intpar.range, range)
assert len(list(intpar.range)) == 1
# Range contains ONLY the default / value.
assert list(intpar.range) == [intpar.value]
intpar.hyperopt = True
assert len(list(intpar.range)) == 6
assert list(intpar.range) == [0, 1, 2, 3, 4, 5]
fltpar = RealParameter(low=0.0, high=5.5, default=1.0, space='buy') fltpar = RealParameter(low=0.0, high=5.5, default=1.0, space='buy')
assert isinstance(fltpar.get_space(''), Real) assert isinstance(fltpar.get_space(''), Real)

View File

@ -160,8 +160,7 @@ def test_get_trade_stake_amount(default_conf, ticker, mocker) -> None:
freqtrade = FreqtradeBot(default_conf) freqtrade = FreqtradeBot(default_conf)
result = freqtrade.wallets.get_trade_stake_amount( result = freqtrade.wallets.get_trade_stake_amount('ETH/BTC')
'ETH/BTC', freqtrade.get_free_open_trades())
assert result == default_conf['stake_amount'] assert result == default_conf['stake_amount']
@ -197,14 +196,12 @@ def test_check_available_stake_amount(default_conf, ticker, mocker, fee, limit_b
if expected[i] is not None: if expected[i] is not None:
limit_buy_order_open['id'] = str(i) limit_buy_order_open['id'] = str(i)
result = freqtrade.wallets.get_trade_stake_amount('ETH/BTC', result = freqtrade.wallets.get_trade_stake_amount('ETH/BTC')
freqtrade.get_free_open_trades())
assert pytest.approx(result) == expected[i] assert pytest.approx(result) == expected[i]
freqtrade.execute_buy('ETH/BTC', result) freqtrade.execute_buy('ETH/BTC', result)
else: else:
with pytest.raises(DependencyException): with pytest.raises(DependencyException):
freqtrade.wallets.get_trade_stake_amount('ETH/BTC', freqtrade.wallets.get_trade_stake_amount('ETH/BTC')
freqtrade.get_free_open_trades())
def test_edge_called_in_process(mocker, edge_conf) -> None: def test_edge_called_in_process(mocker, edge_conf) -> None:
@ -230,9 +227,9 @@ def test_edge_overrides_stake_amount(mocker, edge_conf) -> None:
freqtrade = FreqtradeBot(edge_conf) freqtrade = FreqtradeBot(edge_conf)
assert freqtrade.wallets.get_trade_stake_amount( assert freqtrade.wallets.get_trade_stake_amount(
'NEO/BTC', freqtrade.get_free_open_trades(), freqtrade.edge) == (999.9 * 0.5 * 0.01) / 0.20 'NEO/BTC', freqtrade.edge) == (999.9 * 0.5 * 0.01) / 0.20
assert freqtrade.wallets.get_trade_stake_amount( assert freqtrade.wallets.get_trade_stake_amount(
'LTC/BTC', freqtrade.get_free_open_trades(), freqtrade.edge) == (999.9 * 0.5 * 0.01) / 0.21 'LTC/BTC', freqtrade.edge) == (999.9 * 0.5 * 0.01) / 0.21
def test_edge_overrides_stoploss(limit_buy_order, fee, caplog, mocker, edge_conf) -> None: def test_edge_overrides_stoploss(limit_buy_order, fee, caplog, mocker, edge_conf) -> None:
@ -448,8 +445,7 @@ def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order_open,
patch_get_signal(freqtrade) patch_get_signal(freqtrade)
assert not freqtrade.create_trade('ETH/BTC') assert not freqtrade.create_trade('ETH/BTC')
assert freqtrade.wallets.get_trade_stake_amount('ETH/BTC', freqtrade.get_free_open_trades(), assert freqtrade.wallets.get_trade_stake_amount('ETH/BTC', freqtrade.edge) == 0
freqtrade.edge) == 0
def test_enter_positions_no_pairs_left(default_conf, ticker, limit_buy_order_open, fee, def test_enter_positions_no_pairs_left(default_conf, ticker, limit_buy_order_open, fee,

View File

@ -177,8 +177,7 @@ def test_forcebuy_last_unlimited(default_conf, ticker, fee, limit_buy_order, moc
trades = Trade.query.all() trades = Trade.query.all()
assert len(trades) == 4 assert len(trades) == 4
assert freqtrade.wallets.get_trade_stake_amount( assert freqtrade.wallets.get_trade_stake_amount('XRP/BTC') == result1
'XRP/BTC', freqtrade.get_free_open_trades()) == result1
rpc._rpc_forcebuy('TKN/BTC', None) rpc._rpc_forcebuy('TKN/BTC', None)
@ -199,8 +198,7 @@ def test_forcebuy_last_unlimited(default_conf, ticker, fee, limit_buy_order, moc
# One trade sold # One trade sold
assert len(trades) == 4 assert len(trades) == 4
# stake-amount should now be reduced, since one trade was sold at a loss. # stake-amount should now be reduced, since one trade was sold at a loss.
assert freqtrade.wallets.get_trade_stake_amount( assert freqtrade.wallets.get_trade_stake_amount('XRP/BTC') < result1
'XRP/BTC', freqtrade.get_free_open_trades()) < result1
# Validate that balance of sold trade is not in dry-run balances anymore. # Validate that balance of sold trade is not in dry-run balances anymore.
bals2 = freqtrade.wallets.get_all_balances() bals2 = freqtrade.wallets.get_all_balances()
assert bals != bals2 assert bals != bals2

View File

@ -118,16 +118,17 @@ def test_get_trade_stake_amount_no_stake_amount(default_conf, mocker) -> None:
freqtrade = get_patched_freqtradebot(mocker, default_conf) freqtrade = get_patched_freqtradebot(mocker, default_conf)
with pytest.raises(DependencyException, match=r'.*stake amount.*'): with pytest.raises(DependencyException, match=r'.*stake amount.*'):
freqtrade.wallets.get_trade_stake_amount('ETH/BTC', freqtrade.get_free_open_trades()) freqtrade.wallets.get_trade_stake_amount('ETH/BTC')
@pytest.mark.parametrize("balance_ratio,result1", [ @pytest.mark.parametrize("balance_ratio,result1,result2", [
(1, 50), (1, 50, 66.66666),
(0.99, 49.5), (0.99, 49.5, 66.0),
(0.50, 25), (0.50, 25, 33.3333),
]) ])
def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_ratio, result1, def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_ratio, result1,
limit_buy_order_open, fee, mocker) -> None: result2, limit_buy_order_open,
fee, mocker) -> None:
mocker.patch.multiple( mocker.patch.multiple(
'freqtrade.exchange.Exchange', 'freqtrade.exchange.Exchange',
fetch_ticker=ticker, fetch_ticker=ticker,
@ -144,22 +145,28 @@ def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_r
freqtrade = get_patched_freqtradebot(mocker, conf) freqtrade = get_patched_freqtradebot(mocker, conf)
# no open trades, order amount should be 'balance / max_open_trades' # no open trades, order amount should be 'balance / max_open_trades'
result = freqtrade.wallets.get_trade_stake_amount('ETH/USDT', freqtrade.get_free_open_trades()) result = freqtrade.wallets.get_trade_stake_amount('ETH/USDT')
assert result == result1 assert result == result1
# create one trade, order amount should be 'balance / (max_open_trades - num_open_trades)' # create one trade, order amount should be 'balance / (max_open_trades - num_open_trades)'
freqtrade.execute_buy('ETH/USDT', result) freqtrade.execute_buy('ETH/USDT', result)
result = freqtrade.wallets.get_trade_stake_amount('LTC/USDDT', freqtrade.get_free_open_trades()) result = freqtrade.wallets.get_trade_stake_amount('LTC/USDT')
assert result == result1 assert result == result1
# create 2 trades, order amount should be None # create 2 trades, order amount should be None
freqtrade.execute_buy('LTC/BTC', result) freqtrade.execute_buy('LTC/BTC', result)
result = freqtrade.wallets.get_trade_stake_amount('XRP/USDT', freqtrade.get_free_open_trades()) result = freqtrade.wallets.get_trade_stake_amount('XRP/USDT')
assert result == 0 assert result == 0
freqtrade.config['max_open_trades'] = 3
freqtrade.config['dry_run_wallet'] = 200
freqtrade.wallets.start_cap = 200
result = freqtrade.wallets.get_trade_stake_amount('XRP/USDT')
assert round(result, 4) == round(result2, 4)
# set max_open_trades = None, so do not trade # set max_open_trades = None, so do not trade
freqtrade.config['max_open_trades'] = 0 freqtrade.config['max_open_trades'] = 0
result = freqtrade.wallets.get_trade_stake_amount('NEO/USDT', freqtrade.get_free_open_trades()) result = freqtrade.wallets.get_trade_stake_amount('NEO/USDT')
assert result == 0 assert result == 0