adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into pr/jpribyl/3210-1

Browse Source
This commit is contained in:
Matthias
2020-05-16 13:09:38 +02:00
parent 22a08768f7 9d6a41aa7a
commit 2a1fcc0f06
61 changed files with 1459 additions and 413 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/configuration.md
View File

@@ -109,7 +109,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi
| `forcebuy_enable` | Enables the RPC Commands to force a buy. More information below. <br> **Datatype:** Boolean
| `strategy` | **Required** Defines Strategy class to use. Recommended to be set via `--strategy NAME`. <br> **Datatype:** ClassName
| `strategy_path` | Adds an additional strategy lookup path (must be a directory). <br> **Datatype:** String
| `internals.process_throttle_secs` | Set the process throttle. Value in second. <br>*Defaults to `5` seconds.* <br> **Datatype:** Positive Intege
| `internals.process_throttle_secs` | Set the process throttle. Value in second. <br>*Defaults to `5` seconds.* <br> **Datatype:** Positive Integer
| `internals.heartbeat_interval` | Print heartbeat message every N seconds. Set to 0 to disable heartbeat messages. <br>*Defaults to `60` seconds.* <br> **Datatype:** Positive Integer or 0
| `internals.sd_notify` | Enables use of the sd_notify protocol to tell systemd service manager about changes in the bot state and issue keep-alive pings. See [here](installation.md#7-optional-configure-freqtrade-as-a-systemd-service) for more details. <br> **Datatype:** Boolean
| `logfile` | Specifies logfile name. Uses a rolling strategy for log file rotation for 10 files with the 1MB limit per file. <br> **Datatype:** String

5
docs/docker.md
View File

@@ -22,6 +22,9 @@ Freqtrade provides an official Docker image on [Dockerhub](https://hub.docker.co
!!! Note
All below comands use relative directories and will have to be executed from the directory containing the `docker-compose.yml` file.
!!! Note "Docker on Raspberry"
If you're running freqtrade on a Raspberry PI, you must change the image from `freqtradeorg/freqtrade:master` to `freqtradeorg/freqtrade:master_pi` or `freqtradeorg/freqtrade:develop_pi`, otherwise the image will not work.
### Docker quick start
Create a new directory and place the [docker-compose file](https://github.com/freqtrade/freqtrade/blob/develop/docker-compose.yml) in this directory.
@@ -65,7 +68,7 @@ docker-compose up -d
#### Docker-compose logs
Logs will be written to `user_data/freqtrade.log`.
Logs will be written to `user_data/logs/freqtrade.log`.
Alternatively, you can check the latest logs using `docker-compose logs -f`.
#### Database

4
docs/installation.md
View File

@@ -248,14 +248,14 @@ git clone https://github.com/freqtrade/freqtrade.git
Install ta-lib according to the [ta-lib documentation](https://github.com/mrjbq7/ta-lib#windows).
As compiling from source on windows has heavy dependencies (requires a partial visual studio installation), there is also a repository of unofficial precompiled windows Wheels [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#ta-lib), which needs to be downloaded and installed using `pip install TA_Lib‑0.4.17‑cp36‑cp36m‑win32.whl` (make sure to use the version matching your python version)
As compiling from source on windows has heavy dependencies (requires a partial visual studio installation), there is also a repository of unofficial precompiled windows Wheels [here](https://www.lfd.uci.edu/~gohlke/pythonlibs/#ta-lib), which needs to be downloaded and installed using `pip install TA_Lib‑0.4.18‑cp38‑cp38‑win_amd64.whl` (make sure to use the version matching your python version)
```cmd
>cd \path\freqtrade-develop
>python -m venv .env
>.env\Scripts\activate.bat
REM optionally install ta-lib from wheel
REM >pip install TA_Lib‑0.4.17‑cp36‑cp36m‑win32.whl
REM >pip install TA_Lib‑0.4.18‑cp38‑cp38‑win_amd64.whl
>pip install -r requirements.txt
>pip install -e .
>freqtrade

2
docs/requirements-docs.txt
View File

@@ -1,2 +1,2 @@
mkdocs-material==5.1.3
mkdocs-material==5.1.6
mdx_truly_sane_lists==1.2

31
docs/rest-api.md
View File

@@ -11,6 +11,7 @@ Sample configuration:
"enabled": true,
"listen_ip_address": "127.0.0.1",
"listen_port": 8080,
"jwt_secret_key": "somethingrandom",
"username": "Freqtrader",
"password": "SuperSecret1!"
},
@@ -29,7 +30,7 @@ This should return the response:
{"status":"pong"}
```
All other endpoints return sensitive info and require authentication, so are not available through a web browser.
All other endpoints return sensitive info and require authentication and are therefore not available through a web browser.
To generate a secure password, either use a password manager, or use the below code snipped.
@@ -38,6 +39,9 @@ import secrets
secrets.token_hex()
```
!!! Hint
Use the same method to also generate a JWT secret key (`jwt_secret_key`).
### Configuration with docker
If you run your bot using docker, you'll need to have the bot listen to incomming connections. The security is then handled by docker.
@@ -202,3 +206,28 @@ whitelist
Show the current whitelist
:returns: json object
```
## Advanced API usage using JWT tokens
!!! Note
The below should be done in an application (a Freqtrade REST API client, which fetches info via API), and is not intended to be used on a regular basis.
Freqtrade's REST API also offers JWT (JSON Web Tokens).
You can login using the following command, and subsequently use the resulting access_token.
``` bash
> curl -X POST --user Freqtrader http://localhost:8080/api/v1/token/login
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1ODkxMTk2ODEsIm5iZiI6MTU4OTExOTY4MSwianRpIjoiMmEwYmY0NWUtMjhmOS00YTUzLTlmNzItMmM5ZWVlYThkNzc2IiwiZXhwIjoxNTg5MTIwNTgxLCJpZGVudGl0eSI6eyJ1IjoiRnJlcXRyYWRlciJ9LCJmcmVzaCI6ZmFsc2UsInR5cGUiOiJhY2Nlc3MifQ.qt6MAXYIa-l556OM7arBvYJ0SDI9J8bIk3_glDujF5g","refresh_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1ODkxMTk2ODEsIm5iZiI6MTU4OTExOTY4MSwianRpIjoiZWQ1ZWI3YjAtYjMwMy00YzAyLTg2N2MtNWViMjIxNWQ2YTMxIiwiZXhwIjoxNTkxNzExNjgxLCJpZGVudGl0eSI6eyJ1IjoiRnJlcXRyYWRlciJ9LCJ0eXBlIjoicmVmcmVzaCJ9.d1AT_jYICyTAjD0fiQAr52rkRqtxCjUGEMwlNuuzgNQ"}
> access_token="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1ODkxMTk2ODEsIm5iZiI6MTU4OTExOTY4MSwianRpIjoiMmEwYmY0NWUtMjhmOS00YTUzLTlmNzItMmM5ZWVlYThkNzc2IiwiZXhwIjoxNTg5MTIwNTgxLCJpZGVudGl0eSI6eyJ1IjoiRnJlcXRyYWRlciJ9LCJmcmVzaCI6ZmFsc2UsInR5cGUiOiJhY2Nlc3MifQ.qt6MAXYIa-l556OM7arBvYJ0SDI9J8bIk3_glDujF5g"
# Use access_token for authentication
> curl -X GET --header "Authorization: Bearer ${access_token}" http://localhost:8080/api/v1/count
```
Since the access token has a short timeout (15 min) - the `token/refresh` request should be used periodically to get a fresh access token:
``` bash
> curl -X POST --header "Authorization: Bearer ${refresh_token}"http://localhost:8080/api/v1/token/refresh
{"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1ODkxMTk5NzQsIm5iZiI6MTU4OTExOTk3NCwianRpIjoiMDBjNTlhMWUtMjBmYS00ZTk0LTliZjAtNWQwNTg2MTdiZDIyIiwiZXhwIjoxNTg5MTIwODc0LCJpZGVudGl0eSI6eyJ1IjoiRnJlcXRyYWRlciJ9LCJmcmVzaCI6ZmFsc2UsInR5cGUiOiJhY2Nlc3MifQ.1seHlII3WprjjclY6DpRhen0rqdF4j6jbvxIhUFaSbs"}
```

79
docs/sql_cheatsheet.md
View File

@@ -1,13 +1,20 @@
# SQL Helper
This page contains some help if you want to edit your sqlite db.
## Install sqlite3
**Ubuntu/Debian installation**
Sqlite3 is a terminal based sqlite application.
Feel free to use a visual Database editor like SqliteBrowser if you feel more comfortable with that.
### Ubuntu/Debian installation
```bash
sudo apt-get install sqlite3
```
## Open the DB
```bash
sqlite3
.open <filepath>
@@ -16,45 +23,61 @@ sqlite3
## Table structure
### List tables
```bash
.tables
```
### Display table structure
```bash
.schema <table_name>
```
### Trade table structure
```sql
CREATE TABLE trades (
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee_open FLOAT NOT NULL,
fee_close FLOAT NOT NULL,
open_rate FLOAT,
open_rate_requested FLOAT,
close_rate FLOAT,
close_rate_requested FLOAT,
close_profit FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
stop_loss FLOAT,
initial_stop_loss FLOAT,
stoploss_order_id VARCHAR,
stoploss_last_update DATETIME,
max_rate FLOAT,
sell_reason VARCHAR,
strategy VARCHAR,
ticker_interval INTEGER,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
CREATE TABLE trades
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee_open FLOAT NOT NULL,
fee_open_cost FLOAT,
fee_open_currency VARCHAR,
fee_close FLOAT NOT NULL,
fee_close_cost FLOAT,
fee_close_currency VARCHAR,
open_rate FLOAT,
open_rate_requested FLOAT,
open_trade_price FLOAT,
close_rate FLOAT,
close_rate_requested FLOAT,
close_profit FLOAT,
close_profit_abs FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
stop_loss FLOAT,
stop_loss_pct FLOAT,
initial_stop_loss FLOAT,
initial_stop_loss_pct FLOAT,
stoploss_order_id VARCHAR,
stoploss_last_update DATETIME,
max_rate FLOAT,
min_rate FLOAT,
sell_reason VARCHAR,
strategy VARCHAR,
ticker_interval INTEGER,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
);
CREATE INDEX ix_trades_stoploss_order_id ON trades (stoploss_order_id);
CREATE INDEX ix_trades_pair ON trades (pair);
CREATE INDEX ix_trades_is_open ON trades (is_open);
```
## Get all trades in the table

4
docs/strategy-advanced.md
View File

@@ -20,7 +20,7 @@ It applies a tight timeout for higher priced assets, while allowing more time to
The function must return either `True` (cancel order) or `False` (keep order alive).
``` python
from datetime import datetime, timestamp
from datetime import datetime, timedelta
from freqtrade.persistence import Trade
class Awesomestrategy(IStrategy):
@@ -59,7 +59,7 @@ class Awesomestrategy(IStrategy):
### Custom order timeout example (using additional data)
``` python
from datetime import datetime, timestamp
from datetime import datetime
from freqtrade.persistence import Trade
class Awesomestrategy(IStrategy):

179
docs/strategy-customization.md
View File

@@ -324,67 +324,14 @@ class Awesomestrategy(IStrategy):
!!! Note
If the data is pair-specific, make sure to use pair as one of the keys in the dictionary.
### Additional data (DataProvider)
***
The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy.
All methods return `None` in case of failure (do not raise an exception).
Please always check the mode of operation to select the correct method to get data (samples see below).
#### Possible options for DataProvider
- `available_pairs` - Property with tuples listing cached pairs with their intervals (pair, interval).
- `ohlcv(pair, timeframe)` - Currently cached candle (OHLCV) data for the pair, returns DataFrame or empty DataFrame.
- `historic_ohlcv(pair, timeframe)` - Returns historical data stored on disk.
- `get_pair_dataframe(pair, timeframe)` - This is a universal method, which returns either historical data (for backtesting) or cached live data (for the Dry-Run and Live-Run modes).
- `orderbook(pair, maximum)` - Returns latest orderbook data for the pair, a dict with bids/asks with a total of `maximum` entries.
- `market(pair)` - Returns market data for the pair: fees, limits, precisions, activity flag, etc. See [ccxt documentation](https://github.com/ccxt/ccxt/wiki/Manual#markets) for more details on Market data structure.
- `runmode` - Property containing the current runmode.
#### Example: fetch live / historical candle (OHLCV) data for the first informative pair
``` python
if self.dp:
inf_pair, inf_timeframe = self.informative_pairs()[0]
informative = self.dp.get_pair_dataframe(pair=inf_pair,
timeframe=inf_timeframe)
```
!!! Warning "Warning about backtesting"
Be carefull when using dataprovider in backtesting. `historic_ohlcv()` (and `get_pair_dataframe()`
for the backtesting runmode) provides the full time-range in one go,
so please be aware of it and make sure to not "look into the future" to avoid surprises when running in dry/live mode).
!!! Warning "Warning in hyperopt"
This option cannot currently be used during hyperopt.
#### Orderbook
``` python
if self.dp:
if self.dp.runmode.value in ('live', 'dry_run'):
ob = self.dp.orderbook(metadata['pair'], 1)
dataframe['best_bid'] = ob['bids'][0][0]
dataframe['best_ask'] = ob['asks'][0][0]
```
!!! Warning
The order book is not part of the historic data which means backtesting and hyperopt will not work if this
method is used.
#### Available Pairs
``` python
if self.dp:
for pair, timeframe in self.dp.available_pairs:
print(f"available {pair}, {timeframe}")
```
### Additional data (informative_pairs)
#### Get data for non-tradeable pairs
Data for additional, informative pairs (reference pairs) can be beneficial for some strategies.
Ohlcv data for these pairs will be downloaded as part of the regular whitelist refresh process and is available via `DataProvider` just as other pairs (see above).
Ohlcv data for these pairs will be downloaded as part of the regular whitelist refresh process and is available via `DataProvider` just as other pairs (see below).
These parts will **not** be traded unless they are also specified in the pair whitelist, or have been selected by Dynamic Whitelisting.
The pairs need to be specified as tuples in the format `("pair", "interval")`, with pair as the first and time interval as the second argument.
@@ -404,6 +351,125 @@ def informative_pairs(self):
It is however better to use resampling to longer time-intervals when possible
to avoid hammering the exchange with too many requests and risk being blocked.
***
### Additional data (DataProvider)
The strategy provides access to the `DataProvider`. This allows you to get additional data to use in your strategy.
All methods return `None` in case of failure (do not raise an exception).
Please always check the mode of operation to select the correct method to get data (samples see below).
#### Possible options for DataProvider
- [`available_pairs`](#available_pairs) - Property with tuples listing cached pairs with their intervals (pair, interval).
- [`current_whitelist()`](#current_whitelist) - Returns a current list of whitelisted pairs. Useful for accessing dynamic whitelists (ie. VolumePairlist)
- [`get_pair_dataframe(pair, timeframe)`](#get_pair_dataframepair-timeframe) - This is a universal method, which returns either historical data (for backtesting) or cached live data (for the Dry-Run and Live-Run modes).
- `historic_ohlcv(pair, timeframe)` - Returns historical data stored on disk.
- `market(pair)` - Returns market data for the pair: fees, limits, precisions, activity flag, etc. See [ccxt documentation](https://github.com/ccxt/ccxt/wiki/Manual#markets) for more details on the Market data structure.
- `ohlcv(pair, timeframe)` - Currently cached candle (OHLCV) data for the pair, returns DataFrame or empty DataFrame.
- [`orderbook(pair, maximum)`](#orderbookpair-maximum) - Returns latest orderbook data for the pair, a dict with bids/asks with a total of `maximum` entries.
- [`ticker(pair)`](#tickerpair) - Returns current ticker data for the pair. See [ccxt documentation](https://github.com/ccxt/ccxt/wiki/Manual#price-tickers) for more details on the Ticker data structure.
- `runmode` - Property containing the current runmode.
#### Example Usages:
#### *available_pairs*
``` python
if self.dp:
for pair, timeframe in self.dp.available_pairs:
print(f"available {pair}, {timeframe}")
```
#### *current_whitelist()*
Imagine you've developed a strategy that trades the `5m` timeframe using signals generated from a `1d` timeframe on the top 10 volume pairs by volume.
The strategy might look something like this:
*Scan through the top 10 pairs by volume using the `VolumePairList` every 5 minutes and use a 14 day ATR to buy and sell.*
Due to the limited available data, it's very difficult to resample our `5m` candles into daily candles for use in a 14 day ATR. Most exchanges limit us to just 500 candles which effectively gives us around 1.74 daily candles. We need 14 days at least!
Since we can't resample our data we will have to use an informative pair; and since our whitelist will be dynamic we don't know which pair(s) to use.
This is where calling `self.dp.current_whitelist()` comes in handy.
```python
class SampleStrategy(IStrategy):
# strategy init stuff...
ticker_interval = '5m'
# more strategy init stuff..
def informative_pairs(self):
# get access to all pairs available in whitelist.
pairs = self.dp.current_whitelist()
# Assign tf to each pair so they can be downloaded and cached for strategy.
informative_pairs = [(pair, '1d') for pair in pairs]
return informative_pairs
def populate_indicators(self, dataframe, metadata):
# Get the informative pair
informative = self.dp.get_pair_dataframe(pair=metadata['pair'], timeframe='1d')
# Get the 14 day ATR.
atr = ta.ATR(informative, timeperiod=14)
# Do other stuff
```
#### *get_pair_dataframe(pair, timeframe)*
``` python
# fetch live / historical candle (OHLCV) data for the first informative pair
if self.dp:
inf_pair, inf_timeframe = self.informative_pairs()[0]
informative = self.dp.get_pair_dataframe(pair=inf_pair,
timeframe=inf_timeframe)
```
!!! Warning "Warning about backtesting"
Be carefull when using dataprovider in backtesting. `historic_ohlcv()` (and `get_pair_dataframe()`
for the backtesting runmode) provides the full time-range in one go,
so please be aware of it and make sure to not "look into the future" to avoid surprises when running in dry/live mode).
!!! Warning "Warning in hyperopt"
This option cannot currently be used during hyperopt.
#### *orderbook(pair, maximum)*
``` python
if self.dp:
if self.dp.runmode.value in ('live', 'dry_run'):
ob = self.dp.orderbook(metadata['pair'], 1)
dataframe['best_bid'] = ob['bids'][0][0]
dataframe['best_ask'] = ob['asks'][0][0]
```
!!! Warning
The order book is not part of the historic data which means backtesting and hyperopt will not work if this
method is used.
#### *ticker(pair)*
``` python
if self.dp:
if self.dp.runmode.value in ('live', 'dry_run'):
ticker = self.dp.ticker(metadata['pair'])
dataframe['last_price'] = ticker['last']
dataframe['volume24h'] = ticker['quoteVolume']
dataframe['vwap'] = ticker['vwap']
```
!!! Warning
Although the ticker data structure is a part of the ccxt Unified Interface, the values returned by this method can
vary for different exchanges. For instance, many exchanges do not return `vwap` values, the FTX exchange
does not always fills in the `last` field (so it can be None), etc. So you need to carefully verify the ticker
data returned from the exchange and add appropriate error handling / defaults.
***
### Additional data (Wallets)
The strategy provides access to the `Wallets` object. This contains the current balances on the exchange.
@@ -426,6 +492,7 @@ if self.wallets:
- `get_used(asset)` - currently tied up balance (open orders)
- `get_total(asset)` - total available balance - sum of the 2 above
***
### Additional data (Trades)
A history of Trades can be retrieved in the strategy by querying the database.

45
docs/utils.md
View File

@@ -521,3 +521,48 @@ Prints JSON data with details for the last best epoch (i.e., the best of all epo
```
freqtrade hyperopt-show --best -n -1 --print-json --no-header
```
## Show trades
Print selected (or all) trades from database to screen.
```
usage: freqtrade show-trades [-h] [-v] [--logfile FILE] [-V] [-c PATH]
[-d PATH] [--userdir PATH] [--db-url PATH]
[--trade-ids TRADE_IDS [TRADE_IDS ...]]
[--print-json]
optional arguments:
-h, --help show this help message and exit
--db-url PATH Override trades database URL, this is useful in custom
deployments (default: `sqlite:///tradesv3.sqlite` for
Live Run mode, `sqlite:///tradesv3.dryrun.sqlite` for
Dry Run).
--trade-ids TRADE_IDS [TRADE_IDS ...]
Specify the list of trade ids.
--print-json Print output in JSON format.
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
-c PATH, --config PATH
Specify configuration file (default:
`userdir/config.json` or `config.json` whichever
exists). Multiple --config options may be used. Can be
set to `-` to read config from stdin.
-d PATH, --datadir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
```
### Examples
Print trades with id 2 and 3 as json
``` bash
freqtrade show-trades --db-url sqlite:///tradesv3.sqlite --trade-ids 2 3 --print-json
```