adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into pr/jonny07/5674

Browse Source
This commit is contained in:
Matthias 2021-10-09 15:29:24 +02:00
commit fa9484a06b
19 changed files with 148 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@ -53,7 +53,7 @@ Please find the complete documentation on our [website](https://www.freqtrade.io
- [x] **Dry-run**: Run the bot without paying money.
- [x] **Backtesting**: Run a simulation of your buy/sell strategy.
- [x] **Strategy Optimization by machine learning**: Use machine learning to optimize your buy/sell strategy parameters with real exchange data.
- [x] **Edge position sizing** Calculate your win rate, risk reward ratio, the best stoploss and adjust your position size before taking a position for each specific market. [Learn more](https://www.freqtrade.io/en/latest/edge/).
- [x] **Edge position sizing** Calculate your win rate, risk reward ratio, the best stoploss and adjust your position size before taking a position for each specific market. [Learn more](https://www.freqtrade.io/en/stable/edge/).
- [x] **Whitelist crypto-currencies**: Select which crypto-currency you want to trade or use dynamic whitelists.
- [x] **Blacklist crypto-currencies**: Select which crypto-currency you want to avoid.
- [x] **Manageable via Telegram**: Manage the bot with Telegram.
@ -66,12 +66,12 @@ Please find the complete documentation on our [website](https://www.freqtrade.io
Freqtrade provides a Linux/macOS script to install all dependencies and help you to configure the bot.
```bash
git clone -b develop https://github.com/freqtrade/freqtrade.git
git clone -b develop https://github.com/freqtrade/freqtrade.git
cd freqtrade
./setup.sh --install
```
For any other type of installation please refer to [Installation doc](https://www.freqtrade.io/en/latest/installation/).
For any other type of installation please refer to [Installation doc](https://www.freqtrade.io/en/stable/installation/).
## Basic Usage

6
docker-compose.yml
View File

@ -15,10 +15,10 @@ services:
volumes:
- "./user_data:/freqtrade/user_data"
# Expose api on port 8080 (localhost only)
# Please read the https://www.freqtrade.io/en/latest/rest-api/ documentation
# Please read the https://www.freqtrade.io/en/stable/rest-api/ documentation
# before enabling this.
# ports:
# - "127.0.0.1:8080:8080"
ports:
- "127.0.0.1:8080:8080"
# Default command used when running `docker compose up`
command: >
trade

43
docs/docker_quickstart.md
View File

@ -182,27 +182,9 @@ You'll then also need to modify the `docker-compose.yml` file and uncomment the
dockerfile: "./Dockerfile.<yourextension>"
```
You can then run `docker-compose build` to build the docker image, and run it using the commands described above.
You can then run `docker-compose build --pull` to build the docker image, and run it using the commands described above.
### Troubleshooting
#### Docker on Windows
* Error: `"Timestamp for this request is outside of the recvWindow."`
* The market api requests require a synchronized clock but the time in the docker container shifts a bit over time into the past.
To fix this issue temporarily you need to run `wsl --shutdown` and restart docker again (a popup on windows 10 will ask you to do so).
A permanent solution is either to host the docker container on a linux host or restart the wsl from time to time with the scheduler.
```
taskkill /IM "Docker Desktop.exe" /F
wsl --shutdown
start "" "C:\Program Files\Docker\Docker\Docker Desktop.exe"
```
!!! Warning
Due to the above, we do not recommend the usage of docker on windows for production setups, but only for experimentation, datadownload and backtesting.
Best use a linux-VPS for running freqtrade reliably.
## Plotting with docker-compose
### Plotting with docker-compose
Commands `freqtrade plot-profit` and `freqtrade plot-dataframe` ([Documentation](plotting.md)) are available by changing the image to `*_plot` in your docker-compose.yml file.
You can then use these commands as follows:
@ -213,7 +195,7 @@ docker-compose run --rm freqtrade plot-dataframe --strategy AwesomeStrategy -p B
The output will be stored in the `user_data/plot` directory, and can be opened with any modern browser.
## Data analysis using docker compose
### Data analysis using docker compose
Freqtrade provides a docker-compose file which starts up a jupyter lab server.
You can run this server using the following command:
@ -230,3 +212,22 @@ Since part of this image is built on your machine, it is recommended to rebuild
``` bash
docker-compose -f docker/docker-compose-jupyter.yml build --no-cache
```
## Troubleshooting
### Docker on Windows
* Error: `"Timestamp for this request is outside of the recvWindow."`
* The market api requests require a synchronized clock but the time in the docker container shifts a bit over time into the past.
To fix this issue temporarily you need to run `wsl --shutdown` and restart docker again (a popup on windows 10 will ask you to do so).
A permanent solution is either to host the docker container on a linux host or restart the wsl from time to time with the scheduler.
``` bash
taskkill /IM "Docker Desktop.exe" /F
wsl --shutdown
start "" "C:\Program Files\Docker\Docker\Docker Desktop.exe"
```
!!! Warning
Due to the above, we do not recommend the usage of docker on windows for production setups, but only for experimentation, datadownload and backtesting.
Best use a linux-VPS for running freqtrade reliably.

6
docs/faq.md
View File

@ -54,9 +54,11 @@ you can't say much from few trades.
Yes. You can edit your config and use the `/reload_config` command to reload the configuration. The bot will stop, reload the configuration and strategy and will restart with the new configuration and strategy.
### I want to improve the bot with a new strategy
### I want to use incomplete candles
That's great. We have a nice backtesting and hyperoptimization setup. See the tutorial [here|Testing-new-strategies-with-Hyperopt](bot-usage.md#hyperopt-commands).
Freqtrade will not provide incomplete candles to strategies. Using incomplete candles will lead to repainting and consequently to strategies with "ghost" buys, which are impossible to both backtest, and verify after they happened.
You can use "current" market data by using the [dataprovider](strategy-customization.md#orderbookpair-maximum)'s orderbook or ticker methods - which however cannot be used during backtesting.
### Is there a setting to only SELL the coins being held and not perform anymore BUYS?

18
docs/hyperopt.md
View File

@ -60,7 +60,7 @@ optional arguments:
Specify what timerange of data to use.
--data-format-ohlcv {json,jsongz,hdf5}
Storage format for downloaded candle (OHLCV) data.
(default: `None`).
(default: `json`).
--max-open-trades INT
Override the value of the `max_open_trades`
configuration setting.
@ -114,7 +114,8 @@ optional arguments:
Hyperopt-loss-functions are:
ShortTradeDurHyperOptLoss, OnlyProfitHyperOptLoss,
SharpeHyperOptLoss, SharpeHyperOptLossDaily,
SortinoHyperOptLoss, SortinoHyperOptLossDaily
SortinoHyperOptLoss, SortinoHyperOptLossDaily,
MaxDrawDownHyperOptLoss
--disable-param-export
Disable automatic hyperopt parameter export.
@ -512,12 +513,13 @@ This class should be in its own file within the `user_data/hyperopts/` directory
Currently, the following loss functions are builtin:
* `ShortTradeDurHyperOptLoss` (default legacy Freqtrade hyperoptimization loss function) - Mostly for short trade duration and avoiding losses.
* `OnlyProfitHyperOptLoss` (which takes only amount of profit into consideration)
* `SharpeHyperOptLoss` (optimizes Sharpe Ratio calculated on trade returns relative to standard deviation)
* `SharpeHyperOptLossDaily` (optimizes Sharpe Ratio calculated on **daily** trade returns relative to standard deviation)
* `SortinoHyperOptLoss` (optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation)
* `SortinoHyperOptLossDaily` (optimizes Sortino Ratio calculated on **daily** trade returns relative to **downside** standard deviation)
* `ShortTradeDurHyperOptLoss` - (default legacy Freqtrade hyperoptimization loss function) - Mostly for short trade duration and avoiding losses.
* `OnlyProfitHyperOptLoss` - takes only amount of profit into consideration.
* `SharpeHyperOptLoss` - optimizes Sharpe Ratio calculated on trade returns relative to standard deviation.
* `SharpeHyperOptLossDaily` - optimizes Sharpe Ratio calculated on **daily** trade returns relative to standard deviation.
* `SortinoHyperOptLoss` - optimizes Sortino Ratio calculated on trade returns relative to **downside** standard deviation.
* `SortinoHyperOptLossDaily` - optimizes Sortino Ratio calculated on **daily** trade returns relative to **downside** standard deviation.
* `MaxDrawDownHyperOptLoss` - Optimizes Maximum drawdown.
Creation of a custom loss function is covered in the [Advanced Hyperopt](advanced-hyperopt.md) part of the documentation.

7
docs/installation.md
View File

@ -113,6 +113,13 @@ git checkout develop
You may later switch between branches at any time with the `git checkout stable`/`git checkout develop` commands.
??? Note "Install from pypi"
An alternative way to install Freqtrade is from [pypi](https://pypi.org/project/freqtrade/). The downside is that this method requires ta-lib to be correctly installed beforehand, and is therefore currently not the recommended way to install Freqtrade.
``` bash
pip install freqtrade
```
------
## Script Installation

2
docs/rest-api.md
View File

@ -78,7 +78,7 @@ If you run your bot using docker, you'll need to have the bot listen to incoming
},
```
Uncomment the following from your docker-compose file:
Make sure that the following 2 lines are available in your docker-compose file:
```yml
ports:

3
freqtrade/commands/build_config_commands.py
View File

@ -163,7 +163,8 @@ def ask_user_config() -> Dict[str, Any]:
{
"type": "text",
"name": "api_server_listen_addr",
"message": "Insert Api server Listen Address (best left untouched default!)",
"message": ("Insert Api server Listen Address (0.0.0.0 for docker, "
"otherwise best left untouched)"),
"default": "127.0.0.1",
"when": lambda x: x['api_server']
},

3
freqtrade/constants.py
View File

@ -24,7 +24,8 @@ ORDERTYPE_POSSIBILITIES = ['limit', 'market']
ORDERTIF_POSSIBILITIES = ['gtc', 'fok', 'ioc']
HYPEROPT_LOSS_BUILTIN = ['ShortTradeDurHyperOptLoss', 'OnlyProfitHyperOptLoss',
'SharpeHyperOptLoss', 'SharpeHyperOptLossDaily',
'SortinoHyperOptLoss', 'SortinoHyperOptLossDaily']
'SortinoHyperOptLoss', 'SortinoHyperOptLossDaily',
'MaxDrawDownHyperOptLoss']
AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList',
'AgeFilter', 'OffsetFilter', 'PerformanceFilter',
'PrecisionFilter', 'PriceFilter', 'RangeStabilityFilter',

4
freqtrade/exchange/exchange.py
View File

@ -480,7 +480,7 @@ class Exchange:
if startup_candles + 5 > candle_limit:
raise OperationalException(
f"This strategy requires {startup_candles} candles to start. "
f"{self.name} only provides {candle_limit} for {timeframe}.")
f"{self.name} only provides {candle_limit - 5} for {timeframe}.")
def exchange_has(self, endpoint: str) -> bool:
"""
@ -1058,7 +1058,7 @@ class Exchange:
ticker_rate = ticker[conf_strategy['price_side']]
if ticker['last'] and ticker_rate:
if side == 'buy' and ticker_rate > ticker['last']:
balance = conf_strategy['ask_last_balance']
balance = conf_strategy.get('ask_last_balance', 0.0)
ticker_rate = ticker_rate + balance * (ticker['last'] - ticker_rate)
elif side == 'sell' and ticker_rate < ticker['last']:
balance = conf_strategy.get('bid_last_balance', 0.0)

41
freqtrade/optimize/hyperopt_loss_max_drawdown.py Normal file
View File

@ -0,0 +1,41 @@
"""
MaxDrawDownHyperOptLoss
This module defines the alternative HyperOptLoss class which can be used for
Hyperoptimization.
"""
from datetime import datetime
from pandas import DataFrame
from freqtrade.data.btanalysis import calculate_max_drawdown
from freqtrade.optimize.hyperopt import IHyperOptLoss
class MaxDrawDownHyperOptLoss(IHyperOptLoss):
"""
Defines the loss function for hyperopt.
This implementation optimizes for max draw down and profit
Less max drawdown more profit -> Lower return value
"""
@staticmethod
def hyperopt_loss_function(results: DataFrame, trade_count: int,
min_date: datetime, max_date: datetime,
*args, **kwargs) -> float:
"""
Objective function.
Uses profit ratio weighted max_drawdown when drawdown is available.
Otherwise directly optimizes profit ratio.
"""
total_profit = results['profit_abs'].sum()
try:
max_drawdown = calculate_max_drawdown(results, value_col='profit_abs')
except ValueError:
# No losing trade, therefore no drawdown.
return -total_profit
return -total_profit / max_drawdown[0]

5
freqtrade/rpc/api_server/api_schemas.py
View File

@ -347,3 +347,8 @@ class BacktestResponse(BaseModel):
trade_count: Optional[float]
# TODO: Properly type backtestresult...
backtest_result: Optional[Dict[str, Any]]
class SysInfo(BaseModel):
cpu_pct: List[float]
ram_pct: float

8
freqtrade/rpc/api_server/api_v1.py
View File

@ -18,7 +18,8 @@ from freqtrade.rpc.api_server.api_schemas import (AvailablePairs, Balances, Blac
OpenTradeSchema, PairHistory, PerformanceEntry,
Ping, PlotConfig, Profit, ResultMsg, ShowConfig,
Stats, StatusMsg, StrategyListResponse,
StrategyResponse, Version, WhitelistResponse)
StrategyResponse, SysInfo, Version,
WhitelistResponse)
from freqtrade.rpc.api_server.deps import get_config, get_rpc, get_rpc_optional
from freqtrade.rpc.rpc import RPCException
@ -259,3 +260,8 @@ def list_available_pairs(timeframe: Optional[str] = None, stake_currency: Option
'pair_interval': pair_interval,
}
return result
@router.get('/sysinfo', response_model=SysInfo, tags=['info'])
def sysinfo():
return RPC._rpc_sysinfo()

8
freqtrade/rpc/rpc.py
View File

@ -8,6 +8,7 @@ from math import isnan
from typing import Any, Dict, List, Optional, Tuple, Union
import arrow
import psutil
from numpy import NAN, inf, int64, mean
from pandas import DataFrame
@ -870,3 +871,10 @@ class RPC:
'subplots' not in self._freqtrade.strategy.plot_config):
self._freqtrade.strategy.plot_config['subplots'] = {}
return self._freqtrade.strategy.plot_config
@staticmethod
def _rpc_sysinfo() -> Dict[str, Any]:
return {
"cpu_pct": psutil.cpu_percent(interval=1, percpu=True),
"ram_pct": psutil.virtual_memory().percent
}

1
requirements.txt
View File

@ -36,6 +36,7 @@ fastapi==0.68.1
uvicorn==0.15.0
pyjwt==2.1.0
aiofiles==0.7.0
psutil==5.8.0
# Support for colorized terminal output
colorama==0.4.4

7
scripts/rest_client.py
View File

@ -334,6 +334,13 @@ class FtRestClient():
"timerange": timerange if timerange else '',
})
def sysinfo(self):
"""Provides system information (CPU, RAM usage)
:return: json object
"""
return self._get("sysinfo")
def add_arguments():
parser = argparse.ArgumentParser()

12
tests/exchange/test_exchange.py
View File

@ -1832,6 +1832,7 @@ def test_fetch_l2_order_book_exception(default_conf, mocker, exchange_name):
('ask', 20, 19, 10, 0.3, 17), # Between ask and last
('ask', 5, 6, 10, 1.0, 5), # last bigger than ask
('ask', 5, 6, 10, 0.5, 5), # last bigger than ask
('ask', 20, 19, 10, None, 20), # ask_last_balance missing
('ask', 10, 20, None, 0.5, 10), # last not available - uses ask
('ask', 4, 5, None, 0.5, 4), # last not available - uses ask
('ask', 4, 5, None, 1, 4), # last not available - uses ask
@ -1842,6 +1843,7 @@ def test_fetch_l2_order_book_exception(default_conf, mocker, exchange_name):
('bid', 21, 20, 10, 0.7, 13), # Between bid and last
('bid', 21, 20, 10, 0.3, 17), # Between bid and last
('bid', 6, 5, 10, 1.0, 5), # last bigger than bid
('bid', 21, 20, 10, None, 20), # ask_last_balance missing
('bid', 6, 5, 10, 0.5, 5), # last bigger than bid
('bid', 21, 20, None, 0.5, 20), # last not available - uses bid
('bid', 6, 5, None, 0.5, 5), # last not available - uses bid
@ -1851,7 +1853,10 @@ def test_fetch_l2_order_book_exception(default_conf, mocker, exchange_name):
def test_get_buy_rate(mocker, default_conf, caplog, side, ask, bid,
last, last_ab, expected) -> None:
caplog.set_level(logging.DEBUG)
default_conf['bid_strategy']['ask_last_balance'] = last_ab
if last_ab is None:
del default_conf['bid_strategy']['ask_last_balance']
else:
default_conf['bid_strategy']['ask_last_balance'] = last_ab
default_conf['bid_strategy']['price_side'] = side
exchange = get_patched_exchange(mocker, default_conf)
mocker.patch('freqtrade.exchange.Exchange.fetch_ticker',
@ -1876,6 +1881,7 @@ def test_get_buy_rate(mocker, default_conf, caplog, side, ask, bid,
('bid', 12.0, 11.2, 10.5, 1.0, 11.2), # Last smaller than bid - uses bid
('bid', 12.0, 11.2, 10.5, 0.5, 11.2), # Last smaller than bid - uses bid
('bid', 0.003, 0.002, 0.005, 0.0, 0.002),
('bid', 0.003, 0.002, 0.005, None, 0.002),
('ask', 12.0, 11.0, 12.5, 0.0, 12.0), # full ask side
('ask', 12.0, 11.0, 12.5, 1.0, 12.5), # full last side
('ask', 12.0, 11.0, 12.5, 0.5, 12.25), # between bid and lat
@ -1886,13 +1892,15 @@ def test_get_buy_rate(mocker, default_conf, caplog, side, ask, bid,
('ask', 10.11, 11.2, 11.0, 0.0, 10.11),
('ask', 0.001, 0.002, 11.0, 0.0, 0.001),
('ask', 0.006, 1.0, 11.0, 0.0, 0.006),
('ask', 0.006, 1.0, 11.0, None, 0.006),
])
def test_get_sell_rate(default_conf, mocker, caplog, side, bid, ask,
last, last_ab, expected) -> None:
caplog.set_level(logging.DEBUG)
default_conf['ask_strategy']['price_side'] = side
default_conf['ask_strategy']['bid_last_balance'] = last_ab
if last_ab is not None:
default_conf['ask_strategy']['bid_last_balance'] = last_ab
mocker.patch('freqtrade.exchange.Exchange.fetch_ticker',
return_value={'ask': ask, 'bid': bid, 'last': last})
pair = "ETH/BTC"

5
tests/optimize/test_hyperoptloss.py
View File

@ -84,13 +84,14 @@ def test_loss_calculation_has_limited_profit(hyperopt_conf, hyperopt_results) ->
"SortinoHyperOptLossDaily",
"SharpeHyperOptLoss",
"SharpeHyperOptLossDaily",
"MaxDrawDownHyperOptLoss",
])
def test_loss_functions_better_profits(default_conf, hyperopt_results, lossfunction) -> None:
results_over = hyperopt_results.copy()
results_over['profit_abs'] = hyperopt_results['profit_abs'] * 2
results_over['profit_abs'] = hyperopt_results['profit_abs'] * 2 + 0.2
results_over['profit_ratio'] = hyperopt_results['profit_ratio'] * 2
results_under = hyperopt_results.copy()
results_under['profit_abs'] = hyperopt_results['profit_abs'] / 2
results_under['profit_abs'] = hyperopt_results['profit_abs'] / 2 - 0.2
results_under['profit_ratio'] = hyperopt_results['profit_ratio'] / 2
default_conf.update({'hyperopt_loss': lossfunction})

10
tests/rpc/test_rpc_apiserver.py
View File

@ -1271,6 +1271,16 @@ def test_list_available_pairs(botclient):
assert len(rc.json()['pair_interval']) == 1
def test_sysinfo(botclient):
ftbot, client = botclient
rc = client_get(client, f"{BASE_URI}/sysinfo")
assert_response(rc)
result = rc.json()
assert 'cpu_pct' in result
assert 'ram_pct' in result
def test_api_backtesting(botclient, mocker, fee, caplog):
ftbot, client = botclient
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)