adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial commit for Dynamic ROI functionality

Browse Source
This commit is contained in:
werkkrew 2021-03-13 10:36:40 -05:00
parent 441d3fad39
commit f6a60c0f18
8 changed files with 240 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

72
docs/configuration.md
View File

@ -53,6 +53,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `cancel_open_orders_on_exit` | Cancel open orders when the `/stop` RPC command is issued, `Ctrl+C` is pressed or the bot dies unexpectedly. When set to `true`, this allows you to use `/stop` to cancel unfilled and partially filled orders in the event of a market crash. It does not impact open positions. <br>*Defaults to `false`.* <br> **Datatype:** Boolean
| `process_only_new_candles` | Enable processing of indicators only when new candles arrive. If false each loop populates the indicators, this will mean the same candle is processed many times creating system load but can be useful of your strategy depends on tick data not only candle. [Strategy Override](#parameters-in-the-strategy). <br>*Defaults to `false`.* <br> **Datatype:** Boolean
| `minimal_roi` | **Required.** Set the threshold as ratio the bot will use to sell a trade. [More information below](#understand-minimal_roi). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Dict
| `dynamic_roi` | Set the parameters which govern the dynamic ROI functionality. [More information below](#understand-dynamic_roi). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Dict
| `stoploss` | **Required.** Value as ratio of the stoploss used by the bot. More details in the [stoploss documentation](stoploss.md). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Float (as ratio)
| `trailing_stop` | Enables trailing stoploss (based on `stoploss` in either configuration or strategy file). More details in the [stoploss documentation](stoploss.md#trailing-stop-loss). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Boolean
| `trailing_stop_positive` | Changes stoploss once profit has been reached. More details in the [stoploss documentation](stoploss.md#trailing-stop-loss-custom-positive-loss). [Strategy Override](#parameters-in-the-strategy). <br> **Datatype:** Float
@ -132,6 +133,7 @@ The following parameters can be set in either configuration file or strategy.
Values set in the configuration file always overwrite values set in the strategy.
* `minimal_roi`
* `dynamic_roi`
* `timeframe`
* `stoploss`
* `trailing_stop`
@ -249,6 +251,76 @@ If it is not set in either Strategy or Configuration, a default of 1000% `{"0":
!!! Note "Special case to forcesell after a specific time"
A special case presents using `"<N>": -1` as ROI. This forces the bot to sell a trade after N Minutes, no matter if it's positive or negative, so represents a time-limited force-sell.
### Understand dynamic_roi
The `dynamic_roi` configuration parameters enable and control the functionality of dyanamic ROI. In a nutshell, dynamic ROI either works along side the existing ROI table, or moves it to a different method entirely.
There are 3 different types of dynamic ROI algorithms available, `linear`, `exponential`, and `connect`. The `linear` and `exponential` types will override the standard `minimal_roi` table, while the `connect` type works along side the standard table and requires one to be defined to properly work.
* The `linear` type will decay from the `start` value to the `end` value in a straight line over `decay-time` (in minutes). Formula: `f(t) = start - (rate * t)` where `rate = (start -end) / decay-time`
* The `exponential` type will decay from the `start` value in an exponential curve with `decay-rate` and end at minimum value of `end`. Formula: `f(t) = start * e^(-decay-rate*t)`
* The `connect` type will "connect the dots" on the minimal_roi table with straight lines returning the values between the points.
Exapmple Linear Config:
```json
"dynamic_roi": {
"enabled": true,
"type": "linear",
"decay-time": 720,
"start": 0.10,
"end": 0
}
```
Exapmple Linear Config for strategy:
```python
dynamic_roi = {
'enabled': True,
'type': 'linear',
'decay-time': 720,
'start': 0.10,
'end': 0
}
```
Exapmple Exponential Config:
```json
"dynamic_roi": {
"enabled": true,
"type": "exponential",
"decay-rate": 0.015,
"start": 0.10,
"end": 0
}
```
Example Exponential Config for strategy:
```python
dynamic_roi = {
'enabled': True,
'type': 'exponential',
'decay-rate': 0.015,
'start': 0.10,
'end': 0
}
```
Example Connect Config:
```json
"dynamic_roi": {
"enabled": true,
"type": "connect"
}
```
Example Connect Config for strategy:
```python
dynamic_roi = {
'enabled': True,
'type': 'connect'
}
```
### Understand forcebuy_enable
The `forcebuy_enable` configuration parameter enables the usage of forcebuy commands via Telegram and REST API.

49
docs/hyperopt.md
View File

@ -87,7 +87,7 @@ optional arguments:
Starting balance, used for backtesting / hyperopt and
dry-runs.
-e INT, --epochs INT Specify number of epochs (default: 100).
--spaces {all,buy,sell,roi,stoploss,trailing,default} [{all,buy,sell,roi,stoploss,trailing,default} ...]
--spaces {all,buy,sell,roi,stoploss,trailing,dynamic-roi,default} [{all,buy,sell,roi,stoploss,trailing,dynamic-roi,default} ...]
Specify which parameters to hyperopt. Space-separated
list.
--print-all Print all results, not only the best ones.
@ -180,12 +180,13 @@ Optional in hyperopt - can also be loaded from a strategy (recommended):
Rarely you may also need to override:
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
* `dynamic_roi_space` - for custom dynamic ROI optimization (if you need the ranges for the dynamic ROI parameters in the optimization hyperspace that differ from default)
* `generate_roi_table` - for custom ROI optimization (if you need the ranges for the values in the ROI table that differ from default or the number of entries (steps) in the ROI table which differs from the default 4 steps)
* `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
* `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)
!!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
You can quickly optimize the spaces `roi`, `stoploss` and `trailing` without changing anything (i.e. without creation of a "complete" Hyperopt class with dimensions, parameters, triggers and guards, as described in this document) from the default hyperopt template by relying on your strategy to do most of the calculations.
You can quickly optimize the spaces `roi`, `stoploss`, `trailing` and `dynamic-roi` without changing anything (i.e. without creation of a "complete" Hyperopt class with dimensions, parameters, triggers and guards, as described in this document) from the default hyperopt template by relying on your strategy to do most of the calculations.
```python
# Have a working strategy at hand.
@ -401,10 +402,11 @@ Legal values are:
* `roi`: just optimize the minimal profit table for your strategy
* `stoploss`: search for the best stoploss value
* `trailing`: search for the best trailing stop values
* `default`: `all` except `trailing`
* `dynamic-roi`: search for the best dynamic roi values
* `default`: `all` except `trailing` and `dynamic-roi`
* space-separated list of any of the above values for example `--spaces roi stoploss`
The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy.
The default Hyperopt Search Space, used when no `--space` command line option is specified, does not include the `trailing` or `dynamic-roi` hyperspace. We recommend you to run optimization for the `trailing` hyperspace separately, when the best parameters for other hyperspaces were found, validated and pasted into your custom strategy. For `dynamic-roi` this is best run alone after an optimal `roi` space has been found.
### Position stacking and disabling max market positions
@ -611,6 +613,45 @@ If you are optimizing trailing stop values, Freqtrade creates the 'trailing' opt
Override the `trailing_space()` method and define the desired range in it if you need values of the trailing stop parameters to vary in other ranges during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt_advanced.py).
### Understand Hyperopt Dynamic ROI results
If you are optimizing dynamic ROI values (i.e. if optimization search-space contains 'all' or 'dynamic-roi'), your result will look as follows and include dynamic ROI parameters:
```
Best result:
* 1/10: 1263 trades. 1140/90/33 Wins/Draws/Losses. Avg profit 0.77%. Median profit 1.43%. Total profit 489.21273292 USD ( 976.86Σ%). Avg duration 851.3 min. Objective: -89.68666
# Dynamic ROI table:
dynamic_roi = {
'decay-rate': 0.02356,
'decay-time': 909,
'enabled': True,
'end': 0.002,
'start': 0.07778,
'type': 'connect'
}
```
In order to use these best dynamic ROI parameters found by Hyperopt in backtesting and for live trades/dry-run, copy-paste the output into your strategy or transpose the values into your configuration file.
**Note:** The hyperopt output will include every parameter available to the dynamic ROI table, but depending on the type, only certain ones are needed. It is fine to copy them all over but you can clean the list up if you so desire.
#### Default Dynamic ROI Search Space
If you are optimizing dynamic ROI values, Freqtrade creates the 'dynamic-roi' optimization hyperspace for you. By default, the `enabled` parameter will try both True and False values. The value the `type` vary between `linear`, `exponential`, and `connect`.
Other values have default ranges of:
| Param | Range |
|------------|-------------|
| decay-time | 180..1440 |
| decay-rate | 0.001..0.03 |
| start | 0.05..0.25 |
| end | 0..0.005 |
Override the `dynamic_roi_space()` method and define the desired range in it if you want values of the dynamic ROI parameters to vary in other ranges during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/templates/sample_hyperopt_advanced.py).
## Show details of Hyperopt results
After you run Hyperopt for the desired amount of epochs, you can later list all results for analysis, select only best or profitable once, and show the details for any of the epochs previously evaluated. This can be done with the `hyperopt-list` and `hyperopt-show` sub-commands. The usage of these sub-commands is described in the [Utils](utils.md#list-hyperopt-results) chapter.

2
freqtrade/commands/cli_options.py
View File

@ -211,7 +211,7 @@ AVAILABLE_CLI_OPTIONS = {
"spaces": Arg(
'--spaces',
help='Specify which parameters to hyperopt. Space-separated list.',
choices=['all', 'buy', 'sell', 'roi', 'stoploss', 'trailing', 'default'],
choices=['all', 'buy', 'sell', 'roi', 'stoploss', 'trailing', 'dynamic-roi', 'default'],
nargs='+',
default='default',
),

12
freqtrade/constants.py
View File

@ -45,6 +45,7 @@ USERPATH_NOTEBOOKS = 'notebooks'
TELEGRAM_SETTING_OPTIONS = ['on', 'off', 'silent']
DYNAMIC_ROI_TYPES = ['linear', 'exponential', 'connect']
# Define decimals per coin for outputs
# Only used for outputs.
@ -125,6 +126,17 @@ CONF_SCHEMA = {
},
'minProperties': 1
},
'dynamic_roi': {
'type': 'object',
'properties': {
'enabled': {'type': 'boolean'},
'type': {'type': 'string', 'enum': DYNAMIC_ROI_TYPES},
'decay-rate': {'type': 'number', 'minimum': 0.0001, 'maximum': 0.9},
'decay-time': {'type': 'integer', 'minimum': 1},
'start': {'type': 'number', 'minimum': 0.0, 'maximum': 1.0},
'end': {'type': 'number', 'minimum': 0.0}
}
},
'amount_reserve_percent': {'type': 'number', 'minimum': 0.0, 'maximum': 0.5},
'stoploss': {'type': 'number', 'maximum': 0, 'exclusiveMaximum': True},
'trailing_stop': {'type': 'boolean'},

22
freqtrade/optimize/hyperopt.py
View File

@ -191,6 +191,8 @@ class Hyperopt:
for p in self.hyperopt_space('sell')}
if self.has_space('roi'):
result['roi'] = self.custom_hyperopt.generate_roi_table(params)
if self.has_space('dynamic-roi'):
result['dynamic-roi'] = self.custom_hyperopt.generate_dynamic_roi_table(params)
if self.has_space('stoploss'):
result['stoploss'] = {p.name: params.get(p.name)
for p in self.hyperopt_space('stoploss')}
@ -217,7 +219,7 @@ class Hyperopt:
if print_json:
result_dict: Dict = {}
for s in ['buy', 'sell', 'roi', 'stoploss', 'trailing']:
for s in ['buy', 'sell', 'roi', 'dynamic-roi', 'stoploss', 'trailing']:
Hyperopt._params_update_for_json(result_dict, params, s)
print(rapidjson.dumps(result_dict, default=str, number_mode=rapidjson.NM_NATIVE))
@ -225,6 +227,7 @@ class Hyperopt:
Hyperopt._params_pretty_print(params, 'buy', "Buy hyperspace params:")
Hyperopt._params_pretty_print(params, 'sell', "Sell hyperspace params:")
Hyperopt._params_pretty_print(params, 'roi', "ROI table:")
Hyperopt._params_pretty_print(params, 'dynamic-roi', "Dynamic ROI table:")
Hyperopt._params_pretty_print(params, 'stoploss', "Stoploss:")
Hyperopt._params_pretty_print(params, 'trailing', "Trailing stop:")
@ -245,6 +248,8 @@ class Hyperopt:
result_dict['minimal_roi'] = OrderedDict(
(str(k), v) for k, v in space_params.items()
)
elif space == 'dynamic-roi':
result_dict['dynamic_roi'] = space_params
else: # 'stoploss', 'trailing'
result_dict.update(space_params)
@ -269,6 +274,9 @@ class Hyperopt:
for k, v in space_params.items():
params_result += f'{k} = {v}\n'
elif space == 'dynamic-roi':
params_result += f"dynamic_roi = {pformat(space_params, indent=4)}"
params_result = params_result.replace("}", "\n}").replace("{", "{\n ")
else:
params_result += f"{space}_params = {pformat(space_params, indent=4)}"
params_result = params_result.replace("}", "\n}").replace("{", "{\n ")
@ -466,8 +474,8 @@ class Hyperopt:
"""
Tell if the space value is contained in the configuration
"""
# The 'trailing' space is not included in the 'default' set of spaces
if space == 'trailing':
# The 'trailing' and 'dynamic-roi' space is not included in the 'default' set of spaces
if space == 'trailing' or space == 'dynamic-roi':
return any(s in self.config['spaces'] for s in [space, 'all'])
else:
return any(s in self.config['spaces'] for s in [space, 'all', 'default'])
@ -493,6 +501,10 @@ class Hyperopt:
logger.debug("Hyperopt has 'roi' space")
spaces += self.custom_hyperopt.roi_space()
if space == 'dynamic-roi' or (space is None and self.has_space('dynamic-roi')):
logger.debug("Hyperopt has 'dynamic-roi' space")
spaces += self.custom_hyperopt.dynamic_roi_space()
if space == 'stoploss' or (space is None and self.has_space('stoploss')):
logger.debug("Hyperopt has 'stoploss' space")
spaces += self.custom_hyperopt.stoploss_space()
@ -515,6 +527,10 @@ class Hyperopt:
self.backtesting.strategy.minimal_roi = ( # type: ignore
self.custom_hyperopt.generate_roi_table(params_dict))
if self.has_space('dynamic-roi'):
self.backtesting.strategy.dynamic_roi = ( # type: ignore
self.custom_hyperopt.generate_dynamic_roi_table(params_dict))
if self.has_space('buy'):
self.backtesting.strategy.advise_buy = ( # type: ignore
self.custom_hyperopt.buy_strategy_generator(params_dict))

33
freqtrade/optimize/hyperopt_interface.py
View File

@ -216,6 +216,39 @@ class IHyperOpt(ABC):
Categorical([True, False], name='trailing_only_offset_is_reached'),
]
@staticmethod
def generate_dynamic_roi_table(params: Dict) -> Dict[int, float]:
"""
Create a dynamic_roi table.
"""
dynamic_roi_table = {
'enabled': params['enabled'],
'type': params['type'],
'decay-rate': params['decay-rate'],
'decay-time': params['decay-time'],
'start': params['start'],
'end': params['end']
}
return dynamic_roi_table
@staticmethod
def dynamic_roi_space() -> List[Dimension]:
"""
Create a dynamic ROI space.
You may override it in your custom Hyperopt class.
"""
return [
Categorical([True, False], name='enabled'),
Categorical(['linear', 'exponential', 'connect'], name='type'),
Real(0.001, 0.03, name='decay-rate'),
Integer(180, 1440, name='decay-time'),
Real(0.05, 0.25, name='start'),
Real(0, 0.005, name='end')
]
# This is needed for proper unpickling the class attribute ticker_interval
# which is set to the actual value by the resolver.
# Why do I still need such shamanic mantras in modern python?

36
freqtrade/strategy/interface.py
View File

@ -634,6 +634,42 @@ class IStrategy(ABC):
:param trade_dur: trade duration in minutes
:return: minimal ROI entry value or None if none proper ROI entry was found.
"""
dynamic_roi = self.dynamic_roi
minimal_roi = self.minimal_roi
# if the dynamic_roi dict is defined and enabled, use it, otherwise fallback to default functionality
if dynamic_roi and dynamic_roi['enabled']:
# linear decay: f(t) = start - (rate * t)
if dynamic_roi['type'] == 'linear':
rate = (dynamic_roi['start'] - dynamic_roi['end']) / dynamic_roi['decay-time']
min_roi = max(dynamic_roi['end'], dynamic_roi['start'] - (rate * trade_dur))
# exponential decay: f(t) = start * e^(-rate*t)
elif dynamic_roi['type'] == 'exponential':
min_roi = max(dynamic_roi['end'], dynamic_roi['start'] * np.exp(-dynamic_roi['decay-rate']*trade_dur))
elif dynamic_roi['type'] == 'connect':
# connect the points in the defined table with lines
past_roi = list(filter(lambda x: x <= trade_dur, minimal_roi.keys()))
next_roi = list(filter(lambda x: x > trade_dur, minimal_roi.keys()))
if not past_roi:
return None, None
current_entry = max(past_roi)
if not next_roi:
return current_entry, minimal_roi[current_entry]
# use the slope-intercept formula between the two points in the roi table we are between
else:
next_entry = min(next_roi)
# y = mx + b
x1, y1 = current_entry, minimal_roi[current_entry]
x2, y2 = next_entry, minimal_roi[next_entry]
m = (y1-y2)/(x1-x2)
b = (x1*y2 - x2*y1)/(x1-x2)
min_roi = (m * trade_dur) + b
else:
min_roi = 0
return trade_dur, min_roi
# Default ROI table functionality
else:
# Get highest entry in ROI dict where key <= trade-duration
roi_list = list(filter(lambda x: x <= trade_dur, self.minimal_roi.keys()))
if not roi_list:

16
freqtrade/templates/sample_hyperopt_advanced.py
View File

@ -268,6 +268,22 @@ class AdvancedSampleHyperOpt(IHyperOpt):
Categorical([True, False], name='trailing_only_offset_is_reached'),
]
@staticmethod
def dynamic_roi_space() -> List[Dimension]:
"""
Create a dynamic ROI space.
You may override it in your custom Hyperopt class.
"""
return [
Categorical([True, False], name='enabled'),
Categorical(['linear', 'exponential', 'connect'], name='type'),
Real(0.001, 0.03, name='decay-rate'),
Integer(180, 1440, name='decay-time'),
Real(0.05, 0.25, name='start'),
Real(0, 0.005, name='end')
]
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators.