adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2329 from freqtrade/release_2019.9

Browse Source 
Release 2019.9
This commit is contained in:
Matthias 2019-10-01 20:04:18 +02:00 committed by GitHub
commit 6bbc0eefed
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
152 changed files with 2895 additions and 2122 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.coveragerc
View File

@ -1,6 +1,6 @@
[run] [run]
omit = omit =
scripts/* scripts/*
freqtrade/tests/*
freqtrade/vendor/* freqtrade/vendor/*
freqtrade/__main__.py freqtrade/__main__.py
tests/*

5
.gitignore vendored
View File

@ -1,12 +1,9 @@
# Freqtrade rules # Freqtrade rules
freqtrade/tests/testdata/*.json
hyperopt_conf.py
config*.json config*.json
*.sqlite *.sqlite
.hyperopt
logfile.txt logfile.txt
hyperopt_trials.pickle
user_data/* user_data/*
!user_data/strategy/sample_strategy.py
!user_data/notebooks !user_data/notebooks
user_data/notebooks/* user_data/notebooks/*
!user_data/notebooks/*example.ipynb !user_data/notebooks/*example.ipynb

8
.travis.yml
View File

@ -22,19 +22,19 @@ jobs:
include: include:
- stage: tests - stage: tests
script: script:
- pytest --random-order --cov=freqtrade --cov-config=.coveragerc freqtrade/tests/ - pytest --random-order --cov=freqtrade --cov-config=.coveragerc
# Allow failure for coveralls # Allow failure for coveralls
- coveralls || true - coveralls || true
name: pytest name: pytest
- script: - script:
- cp config.json.example config.json - cp config.json.example config.json
- freqtrade --datadir freqtrade/tests/testdata backtesting - freqtrade --datadir tests/testdata backtesting
name: backtest name: backtest
- script: - script:
- cp config.json.example config.json - cp config.json.example config.json
- freqtrade --datadir freqtrade/tests/testdata hyperopt -e 5 - freqtrade --datadir tests/testdata hyperopt -e 5
name: hyperopt name: hyperopt
- script: flake8 freqtrade scripts - script: flake8
name: flake8 name: flake8
- script: - script:
# Test Documentation boxes - # Test Documentation boxes -

8
CONTRIBUTING.md
View File

@ -11,7 +11,7 @@ Few pointers for contributions:
- Create your PR against the `develop` branch, not `master`. - Create your PR against the `develop` branch, not `master`.
- New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100). - New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100).
If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE)
or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR. or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR.
## Getting started ## Getting started
@ -28,19 +28,19 @@ make it pass. It means you have introduced a regression.
#### Test the whole project #### Test the whole project
```bash ```bash
pytest freqtrade pytest
``` ```
#### Test only one file #### Test only one file
```bash ```bash
pytest freqtrade/tests/test_<file_name>.py pytest tests/test_<file_name>.py
``` ```
#### Test only one method from one file #### Test only one method from one file
```bash ```bash
pytest freqtrade/tests/test_<file_name>.py::test_<method_name> pytest tests/test_<file_name>.py::test_<method_name>
``` ```
### 2. Test if your code is PEP8 compliant ### 2. Test if your code is PEP8 compliant

4
Dockerfile.pi
View File

@ -22,13 +22,13 @@ RUN tar -xzf /freqtrade/ta-lib-0.4.0-src.tar.gz \
ENV LD_LIBRARY_PATH /usr/local/lib ENV LD_LIBRARY_PATH /usr/local/lib
# Install berryconda # Install berryconda
RUN wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh \ RUN wget -q https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh \
&& bash ./Berryconda3-2.0.0-Linux-armv7l.sh -b \ && bash ./Berryconda3-2.0.0-Linux-armv7l.sh -b \
&& rm Berryconda3-2.0.0-Linux-armv7l.sh && rm Berryconda3-2.0.0-Linux-armv7l.sh
# Install dependencies # Install dependencies
COPY requirements-common.txt /freqtrade/ COPY requirements-common.txt /freqtrade/
RUN ~/berryconda3/bin/conda install -y numpy pandas scipy \ RUN ~/berryconda3/bin/conda install -y numpy pandas \
&& ~/berryconda3/bin/pip install -r requirements-common.txt --no-cache-dir && ~/berryconda3/bin/pip install -r requirements-common.txt --no-cache-dir
# Install and execute # Install and execute

1
MANIFEST.in
View File

@ -2,4 +2,3 @@ include LICENSE
include README.md include README.md
include config.json.example include config.json.example
recursive-include freqtrade *.py recursive-include freqtrade *.py
include freqtrade/tests/testdata/*.json

4
README.md
View File

@ -141,7 +141,7 @@ Accounts having BNB accounts use this to pay for fees - if your first trade happ
For any questions not covered by the documentation or for further For any questions not covered by the documentation or for further
information about the bot, we encourage you to join our slack channel. information about the bot, we encourage you to join our slack channel.
- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg). - [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE).
### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) ### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue)
@ -172,7 +172,7 @@ to understand the requirements before sending your pull-requests.
Coding is not a neccessity to contribute - maybe start with improving our documentation? Coding is not a neccessity to contribute - maybe start with improving our documentation?
Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase. Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it. **Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
**Important:** Always create your PR against the `develop` branch, not `master`. **Important:** Always create your PR against the `develop` branch, not `master`.

2
build_helpers/publish_docker.sh
View File

@ -23,7 +23,7 @@ if [ $? -ne 0 ]; then
fi fi
# Run backtest # Run backtest
docker run --rm -it -v $(pwd)/config.json.example:/freqtrade/config.json:ro freqtrade:${TAG} --datadir freqtrade/tests/testdata backtesting docker run --rm -it -v $(pwd)/config.json.example:/freqtrade/config.json:ro -v $(pwd)/tests:/tests freqtrade:${TAG} --datadir /tests/testdata backtesting
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "failed running backtest" echo "failed running backtest"

1
config_full.json.example
View File

@ -38,6 +38,7 @@
"order_types": { "order_types": {
"buy": "limit", "buy": "limit",
"sell": "limit", "sell": "limit",
"emergencysell": "market",
"stoploss": "market", "stoploss": "market",
"stoploss_on_exchange": false, "stoploss_on_exchange": false,
"stoploss_on_exchange_interval": 60 "stoploss_on_exchange_interval": 60

BIN
docs/assets/plot-dataframe.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 173 KiB

BIN
docs/assets/plot-profit.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 121 KiB

139
docs/backtesting.md
View File

@ -1,41 +1,9 @@
# Backtesting # Backtesting
This page explains how to validate your strategy performance by using This page explains how to validate your strategy performance by using Backtesting.
Backtesting.
## Getting data for backtesting and hyperopt Backtesting requires historic data to be available.
To learn how to get data for the pairs and exchange you're interested in, head over to the [Data Downloading](data-download.md) section of the documentation.
To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the `freqtrade download-data` command.
If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes.
Exchange and pairs will come from `config.json` (if specified using `-c/--config`). Otherwise `--exchange` becomes mandatory.
Alternatively, a `pairs.json` file can be used.
If you are using Binance for example:
- create a directory `user_data/data/binance` and copy `pairs.json` in that directory.
- update the `pairs.json` to contain the currency pairs you are interested in.
```bash
mkdir -p user_data/data/binance
cp freqtrade/tests/testdata/pairs.json user_data/data/binance
```
Then run:
```bash
freqtrade download-data --exchange binance
```
This will download ticker data for all the currency pairs you defined in `pairs.json`.
- To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
- To change the exchange used to download the tickers, please use a different configuration file (you'll probably need to adjust ratelimits etc.)
- To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
- To download ticker data for only 10 days, use `--days 10` (defaults to 30 days).
- Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
## Test your strategy with Backtesting ## Test your strategy with Backtesting
@ -43,18 +11,16 @@ Now you have good Buy and Sell strategies and some historic data, you want to te
real data. This is what we call real data. This is what we call
[backtesting](https://en.wikipedia.org/wiki/Backtesting). [backtesting](https://en.wikipedia.org/wiki/Backtesting).
Backtesting will use the crypto-currencies (pair) from your config file Backtesting will use the crypto-currencies (pairs) from your config file
and load static tickers located in and load ticker data from `user_data/data/<exchange>` by default.
[/freqtrade/tests/testdata](https://github.com/freqtrade/freqtrade/tree/develop/freqtrade/tests/testdata). If no data is available for the exchange / pair / ticker interval combination, backtesting will
If the 5 min and 1 min ticker for the crypto-currencies to test is not ask you to download them first using `freqtrade download-data`.
already in the `testdata` directory, backtesting will download them For details on downloading, please refer to the [Data Downloading](data-download.md) section in the documentation.
automatically. Testdata files will not be updated until you specify it.
The result of backtesting will confirm you if your bot has better odds of making a profit than a loss. The result of backtesting will confirm if your bot has better odds of making a profit than a loss.
The backtesting is very easy with freqtrade.
### Run a backtesting against the currencies listed in your config file ### Run a backtesting against the currencies listed in your config file
#### With 5 min tickers (Per default) #### With 5 min tickers (Per default)
```bash ```bash
@ -79,18 +45,18 @@ freqtrade backtesting --datadir user_data/data/bittrex-20180101
#### With a (custom) strategy file #### With a (custom) strategy file
```bash ```bash
freqtrade -s TestStrategy backtesting freqtrade -s SampleStrategy backtesting
``` ```
Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory. Where `-s SampleStrategy` refers to the class name within the strategy file `sample_strategy.py` found in the `freqtrade/user_data/strategies` directory.
#### Comparing multiple Strategies #### Comparing multiple Strategies
```bash ```bash
freqtrade backtesting --strategy-list TestStrategy1 AwesomeStrategy --ticker-interval 5m freqtrade backtesting --strategy-list SampleStrategy1 AwesomeStrategy --ticker-interval 5m
``` ```
Where `TestStrategy1` and `AwesomeStrategy` refer to class names of strategies. Where `SampleStrategy1` and `AwesomeStrategy` refer to class names of strategies.
#### Exporting trades to file #### Exporting trades to file
@ -103,36 +69,35 @@ The exported trades can be used for [further analysis](#further-backtest-result-
#### Exporting trades to file specifying a custom filename #### Exporting trades to file specifying a custom filename
```bash ```bash
freqtrade backtesting --export trades --export-filename=backtest_teststrategy.json freqtrade backtesting --export trades --export-filename=backtest_samplestrategy.json
``` ```
#### Running backtest with smaller testset #### Running backtest with smaller testset by using timerange
Use the `--timerange` argument to change how much of the testset Use the `--timerange` argument to change how much of the testset you want to use.
you want to use. The last N ticks/timeframes will be used.
Example:
For example, running backtesting with the `--timerange=20190501-` option will use all available data starting with May 1st, 2019 from your inputdata.
```bash ```bash
freqtrade backtesting --timerange=-200 freqtrade backtesting --timerange=20190501-
``` ```
#### Advanced use of timerange You can also specify particular dates or a range span indexed by start and stop.
Doing `--timerange=-200` will get the last 200 timeframes
from your inputdata. You can also specify specific dates,
or a range span indexed by start and stop.
The full timerange specification: The full timerange specification:
- Use last 123 tickframes of data: `--timerange=-123`
- Use first 123 tickframes of data: `--timerange=123-`
- Use tickframes from line 123 through 456: `--timerange=123-456`
- Use tickframes till 2018/01/31: `--timerange=-20180131` - Use tickframes till 2018/01/31: `--timerange=-20180131`
- Use tickframes since 2018/01/31: `--timerange=20180131-` - Use tickframes since 2018/01/31: `--timerange=20180131-`
- Use tickframes since 2018/01/31 till 2018/03/01 : `--timerange=20180131-20180301` - Use tickframes since 2018/01/31 till 2018/03/01 : `--timerange=20180131-20180301`
- Use tickframes between POSIX timestamps 1527595200 1527618600: - Use tickframes between POSIX timestamps 1527595200 1527618600:
`--timerange=1527595200-1527618600` `--timerange=1527595200-1527618600`
- Use last 123 tickframes of data: `--timerange=-123`
- Use first 123 tickframes of data: `--timerange=123-`
- Use tickframes from line 123 through 456: `--timerange=123-456`
!!! warning
Be carefull when using non-date functions - these do not allow you to specify precise dates, so if you updated the test-data it will probably use a different dataset.
## Understand the backtesting result ## Understand the backtesting result
@ -178,11 +143,12 @@ A backtesting result will look like that:
| TOTAL | 2 | 0.78 | 1.57 | 0.00007855 | 0.78 | 4:00:00 | 2 | 0 | | TOTAL | 2 | 0.78 | 1.57 | 0.00007855 | 0.78 | 4:00:00 | 2 | 0 |
``` ```
The 1st table will contain all trades the bot made. The 1st table contains all trades the bot made, including "left open trades".
The 2nd table will contain a recap of sell reasons. The 2nd table contains a recap of sell reasons.
The 3rd table will contain all trades the bot had to `forcesell` at the end of the backtest period to present a full picture. The 3rd table contains all trades the bot had to `forcesell` at the end of the backtest period to present a full picture.
This is necessary to simulate realistic behaviour, since the backtest period has to end at some point, while realistically, you could leave the bot running forever.
These trades are also included in the first table, but are extracted separately for clarity. These trades are also included in the first table, but are extracted separately for clarity.
The last line will give you the overall performance of your strategy, The last line will give you the overall performance of your strategy,
@ -192,22 +158,16 @@ here:
| TOTAL | 429 | 0.36 | 152.41 | 0.00762792 | 76.20 | 4:12:00 | 186 | 243 | | TOTAL | 429 | 0.36 | 152.41 | 0.00762792 | 76.20 | 4:12:00 | 186 | 243 |
``` ```
We understand the bot has made `429` trades for an average duration of The bot has made `429` trades for an average duration of `4:12:00`, with a performance of `76.20%` (profit), that means it has
`4:12:00`, with a performance of `76.20%` (profit), that means it has
earned a total of `0.00762792 BTC` starting with a capital of 0.01 BTC. earned a total of `0.00762792 BTC` starting with a capital of 0.01 BTC.
The column `avg profit %` shows the average profit for all trades made while the column `cum profit %` sums all the profits/losses. The column `avg profit %` shows the average profit for all trades made while the column `cum profit %` sums up all the profits/losses.
The column `tot profit %` shows instead the total profit % in relation to allocated capital The column `tot profit %` shows instead the total profit % in relation to allocated capital (`max_open_trades * stake_amount`).
(`max_open_trades * stake_amount`). In the above results we have `max_open_trades=2 stake_amount=0.005` in config In the above results we have `max_open_trades=2` and `stake_amount=0.005` in config so `tot_profit %` will be `(76.20/100) * (0.005 * 2) =~ 0.00762792 BTC`.
so `(76.20/100) * (0.005 * 2) =~ 0.00762792 BTC`.
As you will see your strategy performance will be influenced by your buy Your strategy performance is influenced by your buy strategy, your sell strategy, and also by the `minimal_roi` and `stop_loss` you have set.
strategy, your sell strategy, and also by the `minimal_roi` and
`stop_loss` you have set.
As for an example if your minimal_roi is only `"0": 0.01`. You cannot For example, if your `minimal_roi` is only `"0": 0.01` you cannot expect the bot to make more profit than 1% (because it will sell every time a trade reaches 1%).
expect the bot to make more profit than 1% (because it will sell every
time a trade will reach 1%).
```json ```json
"minimal_roi": { "minimal_roi": {
@ -216,22 +176,33 @@ time a trade will reach 1%).
``` ```
On the other hand, if you set a too high `minimal_roi` like `"0": 0.55` On the other hand, if you set a too high `minimal_roi` like `"0": 0.55`
(55%), there is a lot of chance that the bot will never reach this (55%), there is almost no chance that the bot will ever reach this profit.
profit. Hence, keep in mind that your performance is a mix of your Hence, keep in mind that your performance is an integral mix of all different elements of the strategy, your configuration, and the crypto-currency pairs you have set up.
strategies, your configuration, and the crypto-currency you have set up.
### Assumptions made by backtesting
Since backtesting lacks some detailed information about what happens within a candle, it needs to take a few assumptions:
- Buys happen at open-price
- Low happens before high for stoploss, protecting capital first.
- ROI sells are compared to high - but the ROI value is used (e.g. ROI = 2%, high=5% - so the sell will be at 2%)
- Stoploss sells happen exactly at stoploss price, even if low was lower
- Trailing stoploss
- High happens first - adjusting stoploss
- Low uses the adjusted stoploss (so sells with large high-low difference are backtested correctly)
- Sell-reason does not explain if a trade was positive or negative, just what triggered the sell (this can look odd if negative ROI values are used)
### Further backtest-result analysis ### Further backtest-result analysis
To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file). To further analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis as shown in our [data analysis](data-analysis.md#backtesting) backtesting section. You can then load the trades to perform further analysis as shown in our [data analysis](data-analysis.md#backtesting) backtesting section.
## Backtesting multiple strategies ## Backtesting multiple strategies
To backtest multiple strategies, a list of Strategies can be provided. To compare multiple strategies, a list of Strategies can be provided to backtesting.
This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple This is limited to 1 ticker-interval per run, however, data is only loaded once from disk so if you have multiple
strategies you'd like to compare, this should give a nice runtime boost. strategies you'd like to compare, this will give a nice runtime boost.
All listed Strategies need to be in the same directory. All listed Strategies need to be in the same directory.
@ -241,7 +212,7 @@ freqtrade backtesting --timerange 20180401-20180410 --ticker-interval 5m --strat
This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename. This will save the results to `user_data/backtest_results/backtest-result-<strategy>.json`, injecting the strategy-name into the target filename.
There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table). There will be an additional table comparing win/losses of the different strategies (identical to the "Total" row in the first table).
Detailed output for all strategies one after the other will be available, so make sure to scroll up. Detailed output for all strategies one after the other will be available, so make sure to scroll up to see the details per strategy.
``` ```
=========================================================== Strategy Summary =========================================================== =========================================================== Strategy Summary ===========================================================

12
docs/bot-usage.md
View File

@ -184,10 +184,6 @@ optional arguments:
Specify max_open_trades to use. Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT --stake_amount STAKE_AMOUNT
Specify stake_amount. Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--eps, --enable-position-stacking --eps, --enable-position-stacking
Allow buying the same pair multiple times (position Allow buying the same pair multiple times (position
stacking). stacking).
@ -245,10 +241,6 @@ optional arguments:
Specify max_open_trades to use. Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT --stake_amount STAKE_AMOUNT
Specify stake_amount. Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--customhyperopt NAME --customhyperopt NAME
Specify hyperopt class name (default: Specify hyperopt class name (default:
`DefaultHyperOpts`). `DefaultHyperOpts`).
@ -310,10 +302,6 @@ optional arguments:
Specify max_open_trades to use. Specify max_open_trades to use.
--stake_amount STAKE_AMOUNT --stake_amount STAKE_AMOUNT
Specify stake_amount. Specify stake_amount.
-r, --refresh-pairs-cached
Refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your optimization commands with up-to-date data.
--stoplosses STOPLOSS_RANGE --stoplosses STOPLOSS_RANGE
Defines a range of stoploss against which edge will Defines a range of stoploss against which edge will
assess the strategy the format is "min,max,step" assess the strategy the format is "min,max,step"

68
docs/configuration.md
View File

@ -1,14 +1,15 @@
# Configure the bot # Configure the bot
This page explains how to configure the bot. Freqtrade has many configurable features and possibilities.
By default, these settings are configured via the configuration file (see below).
## The Freqtrade configuration file ## The Freqtrade configuration file
The bot uses a set of configuration parameters during its operation that all together conform the bot configuration. It normally reads its configuration from a file (Freqtrade configuration file). The bot uses a set of configuration parameters during its operation that all together conform the bot configuration. It normally reads its configuration from a file (Freqtrade configuration file).
Per default, the bot loads configuration from the `config.json` file located in the current working directory. Per default, the bot loads the configuration from the `config.json` file, located in the current working directory.
You can change the name of the configuration file used by the bot with the `-c/--config` command line option. You can specify a different configuration file used by the bot with the `-c/--config` command line option.
In some advanced use cases, multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream. In some advanced use cases, multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
@ -22,19 +23,26 @@ The Freqtrade configuration file is to be written in the JSON format.
Additionally to the standard JSON syntax, you may use one-line `// ...` and multi-line `/* ... */` comments in your configuration files and trailing commas in the lists of parameters. Additionally to the standard JSON syntax, you may use one-line `// ...` and multi-line `/* ... */` comments in your configuration files and trailing commas in the lists of parameters.
Do not worry if you are not familiar with JSON format -- simply open the configuration file with an editor of your choice, make some changes to the parameters you need, save your changes and, finally, restart the bot or, if it was previously stopped, run it again with the changes you made to the configuration. The bot validates syntax of the configuration file at startup and will warn you if you made any errors editing it. Do not worry if you are not familiar with JSON format -- simply open the configuration file with an editor of your choice, make some changes to the parameters you need, save your changes and, finally, restart the bot or, if it was previously stopped, run it again with the changes you made to the configuration. The bot validates syntax of the configuration file at startup and will warn you if you made any errors editing it, pointing out problematic lines.
## Configuration parameters ## Configuration parameters
The table below will list all configuration parameters available. The table below will list all configuration parameters available.
Mandatory parameters are marked as **Required**. Freqtrade can also load many options via command line (CLI) arguments (check out the commands `--help` output for details).
The prevelance for all Options is as follows:
- CLI arguments override any other option
- Configuration files are used in sequence (last file wins), and override Strategy configurations.
- Strategy configurations are only used if they are not set via configuration or via command line arguments. These options are market with [Strategy Override](#parameters-in-the-strategy) in the below table.
Mandatory parameters are marked as **Required**, which means that they are required to be set in one of the possible ways.
| Command | Default | Description | | Command | Default | Description |
|----------|---------|-------------| |----------|---------|-------------|
| `max_open_trades` | 3 | **Required.** Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades) | `max_open_trades` | 3 | **Required.** Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades)
| `stake_currency` | BTC | **Required.** Crypto-currency used for trading. [Strategy Override](#parameters-in-the-strategy). | `stake_currency` | BTC | **Required.** Crypto-currency used for trading.
| `stake_amount` | 0.05 | **Required.** Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to `"unlimited"` to allow the bot to use all available balance. [Strategy Override](#parameters-in-the-strategy). | `stake_amount` | 0.05 | **Required.** Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to `"unlimited"` to allow the bot to use all available balance.
| `amount_reserve_percent` | 0.05 | Reserve some amount in min pair stake amount. Default is 5%. The bot will reserve `amount_reserve_percent` + stop-loss value when calculating min pair stake amount in order to avoid possible trade refusals. | `amount_reserve_percent` | 0.05 | Reserve some amount in min pair stake amount. Default is 5%. The bot will reserve `amount_reserve_percent` + stop-loss value when calculating min pair stake amount in order to avoid possible trade refusals.
| `ticker_interval` | [1m, 5m, 15m, 30m, 1h, 1d, ...] | The ticker interval to use (1min, 5 min, 15 min, 30 min, 1 hour or 1 day). Default is 5 minutes. [Strategy Override](#parameters-in-the-strategy). | `ticker_interval` | [1m, 5m, 15m, 30m, 1h, 1d, ...] | The ticker interval to use (1min, 5 min, 15 min, 30 min, 1 hour or 1 day). Default is 5 minutes. [Strategy Override](#parameters-in-the-strategy).
| `fiat_display_currency` | USD | **Required.** Fiat currency used to show your profits. More information below. | `fiat_display_currency` | USD | **Required.** Fiat currency used to show your profits. More information below.
@ -61,8 +69,9 @@ Mandatory parameters are marked as **Required**.
| `order_time_in_force` | None | Configure time in force for buy and sell orders. [More information below](#understand-order_time_in_force). [Strategy Override](#parameters-in-the-strategy). | `order_time_in_force` | None | Configure time in force for buy and sell orders. [More information below](#understand-order_time_in_force). [Strategy Override](#parameters-in-the-strategy).
| `exchange.name` | | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). | `exchange.name` | | **Required.** Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename).
| `exchange.sandbox` | false | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details. | `exchange.sandbox` | false | Use the 'sandbox' version of the exchange, where the exchange provides a sandbox for risk-free integration. See [here](sandbox-testing.md) in more details.
| `exchange.key` | '' | API key to use for the exchange. Only required when you are in production mode. | `exchange.key` | '' | API key to use for the exchange. Only required when you are in production mode. ***Keep it in secrete, do not disclose publicly.***
| `exchange.secret` | '' | API secret to use for the exchange. Only required when you are in production mode. | `exchange.secret` | '' | API secret to use for the exchange. Only required when you are in production mode. ***Keep it in secrete, do not disclose publicly.***
| `exchange.password` | '' | API password to use for the exchange. Only required when you are in production mode and for exchanges that use password for API requests. ***Keep it in secrete, do not disclose publicly.***
| `exchange.pair_whitelist` | [] | List of pairs to use by the bot for trading and to check for potential trades during backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)). | `exchange.pair_whitelist` | [] | List of pairs to use by the bot for trading and to check for potential trades during backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)).
| `exchange.pair_blacklist` | [] | List of pairs the bot must absolutely avoid for trading and backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)). | `exchange.pair_blacklist` | [] | List of pairs the bot must absolutely avoid for trading and backtesting. Can be overriden by dynamic pairlists (see [below](#dynamic-pairlists)).
| `exchange.ccxt_config` | None | Additional CCXT parameters passed to the regular ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation) | `exchange.ccxt_config` | None | Additional CCXT parameters passed to the regular ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation)
@ -76,8 +85,8 @@ Mandatory parameters are marked as **Required**.
| `pairlist.method` | StaticPairList | Use static or dynamic volume-based pairlist. [More information below](#dynamic-pairlists). | `pairlist.method` | StaticPairList | Use static or dynamic volume-based pairlist. [More information below](#dynamic-pairlists).
| `pairlist.config` | None | Additional configuration for dynamic pairlists. [More information below](#dynamic-pairlists). | `pairlist.config` | None | Additional configuration for dynamic pairlists. [More information below](#dynamic-pairlists).
| `telegram.enabled` | true | **Required.** Enable or not the usage of Telegram. | `telegram.enabled` | true | **Required.** Enable or not the usage of Telegram.
| `telegram.token` | token | Your Telegram bot token. Only required if `telegram.enabled` is `true`. | `telegram.token` | token | Your Telegram bot token. Only required if `telegram.enabled` is `true`. ***Keep it in secrete, do not disclose publicly.***
| `telegram.chat_id` | chat_id | Your personal Telegram account id. Only required if `telegram.enabled` is `true`. | `telegram.chat_id` | chat_id | Your personal Telegram account id. Only required if `telegram.enabled` is `true`. ***Keep it in secrete, do not disclose publicly.***
| `webhook.enabled` | false | Enable usage of Webhook notifications | `webhook.enabled` | false | Enable usage of Webhook notifications
| `webhook.url` | false | URL for the webhook. Only required if `webhook.enabled` is `true`. See the [webhook documentation](webhook-config.md) for more details. | `webhook.url` | false | URL for the webhook. Only required if `webhook.enabled` is `true`. See the [webhook documentation](webhook-config.md) for more details.
| `webhook.webhookbuy` | false | Payload to send on buy. Only required if `webhook.enabled` is `true`. See the [webhook documentationV](webhook-config.md) for more details. | `webhook.webhookbuy` | false | Payload to send on buy. Only required if `webhook.enabled` is `true`. See the [webhook documentationV](webhook-config.md) for more details.
@ -98,8 +107,6 @@ Mandatory parameters are marked as **Required**.
The following parameters can be set in either configuration file or strategy. The following parameters can be set in either configuration file or strategy.
Values set in the configuration file always overwrite values set in the strategy. Values set in the configuration file always overwrite values set in the strategy.
* `stake_currency`
* `stake_amount`
* `ticker_interval` * `ticker_interval`
* `minimal_roi` * `minimal_roi`
* `stoploss` * `stoploss`
@ -191,19 +198,20 @@ end up paying more then would probably have been necessary.
### Understand order_types ### Understand order_types
The `order_types` configuration parameter contains a dict mapping order-types to The `order_types` configuration parameter maps actions (`buy`, `sell`, `stoploss`) to order-types (`market`, `limit`, ...) as well as configures stoploss to be on the exchange and defines stoploss on exchange update interval in seconds.
market-types as well as stoploss on or off exchange type and stoploss on exchange
update interval in seconds. This allows to buy using limit orders, sell using
limit-orders, and create stoploss orders using market. It also allows to set the
stoploss "on exchange" which means stoploss order would be placed immediately once
the buy order is fulfilled. In case stoploss on exchange and `trailing_stop` are
both set, then the bot will use `stoploss_on_exchange_interval` to check it periodically
and update it if necessary (e.x. in case of trailing stoploss).
This can be set in the configuration file or in the strategy.
Values set in the configuration file overwrites values set in the strategy.
If this is configured, all 4 values (`buy`, `sell`, `stoploss` and This allows to buy using limit orders, sell using
`stoploss_on_exchange`) need to be present, otherwise the bot will warn about it and fail to start. limit-orders, and create stoplosses using using market orders. It also allows to set the
stoploss "on exchange" which means stoploss order would be placed immediately once
the buy order is fulfilled.
If `stoploss_on_exchange` and `trailing_stop` are both set, then the bot will use `stoploss_on_exchange_interval` to check and update the stoploss on exchange periodically.
`order_types` can be set in the configuration file or in the strategy.
`order_types` set in the configuration file overwrites values set in the strategy as a whole, so you need to configure the whole `order_types` dictionary in one place.
If this is configured, the following 4 values (`buy`, `sell`, `stoploss` and
`stoploss_on_exchange`) need to be present, otherwise the bot will fail to start.
`emergencysell` is an optional value, which defaults to `market` and is used when creating stoploss on exchange orders fails.
The below is the default which is used if this is not configured in either strategy or configuration file. The below is the default which is used if this is not configured in either strategy or configuration file.
Syntax for Strategy: Syntax for Strategy:
@ -212,6 +220,7 @@ Syntax for Strategy:
order_types = { order_types = {
"buy": "limit", "buy": "limit",
"sell": "limit", "sell": "limit",
"emergencysell": "market",
"stoploss": "market", "stoploss": "market",
"stoploss_on_exchange": False, "stoploss_on_exchange": False,
"stoploss_on_exchange_interval": 60 "stoploss_on_exchange_interval": 60
@ -224,6 +233,7 @@ Configuration:
"order_types": { "order_types": {
"buy": "limit", "buy": "limit",
"sell": "limit", "sell": "limit",
"emergencysell": "market",
"stoploss": "market", "stoploss": "market",
"stoploss_on_exchange": false, "stoploss_on_exchange": false,
"stoploss_on_exchange_interval": 60 "stoploss_on_exchange_interval": 60
@ -238,11 +248,13 @@ Configuration:
!!! Note !!! Note
Stoploss on exchange interval is not mandatory. Do not change its value if you are Stoploss on exchange interval is not mandatory. Do not change its value if you are
unsure of what you are doing. For more information about how stoploss works please unsure of what you are doing. For more information about how stoploss works please
read [the stoploss documentation](stoploss.md). refer to [the stoploss documentation](stoploss.md).
!!! Note !!! Note
In case of stoploss on exchange if the stoploss is cancelled manually then If `stoploss_on_exchange` is enabled and the stoploss is cancelled manually on the exchange, then the bot will create a new order.
the bot would recreate one.
!!! Warning stoploss_on_exchange failures
If stoploss on exchange creation fails for some reason, then an "emergency sell" is initiated. By default, this will sell the asset using a market order. The order-type for the emergency-sell can be changed by setting the `emergencysell` value in the `order_types` dictionary - however this is not advised.
### Understand order_time_in_force ### Understand order_time_in_force

16
docs/data-analysis.md
View File

@ -91,7 +91,8 @@ df.groupby("pair")["sell_reason"].value_counts()
### Load multiple configuration files ### Load multiple configuration files
This option can be useful to inspect the results of passing in multiple configs This option can be useful to inspect the results of passing in multiple configs.
This will also run through the whole Configuration initialization, so the configuration is completely initialized to be passed to other methods.
``` python ``` python
import json import json
@ -101,7 +102,16 @@ from freqtrade.configuration import Configuration
config = Configuration.from_files(["config1.json", "config2.json"]) config = Configuration.from_files(["config1.json", "config2.json"])
# Show the config in memory # Show the config in memory
print(json.dumps(config, indent=1)) print(json.dumps(config['original_config'], indent=2))
```
For Interactive environments, have an additional configuration specifying `user_data_dir` and pass this in last, so you don't have to change directories while running the bot.
Best avoid relative paths, since this starts at the storage location of the jupyter notebook, unless the directory is changed.
``` json
{
"user_data_dir": "~/.freqtrade/"
}
``` ```
### Load exchange data to a pandas dataframe ### Load exchange data to a pandas dataframe
@ -139,7 +149,7 @@ You can override strategy settings as demonstrated below.
# Define some constants # Define some constants
ticker_interval = "5m" ticker_interval = "5m"
# Name of the strategy class # Name of the strategy class
strategy_name = 'TestStrategy' strategy_name = 'SampleStrategy'
# Path to user data # Path to user data
user_data_dir = 'user_data' user_data_dir = 'user_data'
# Location of the strategy # Location of the strategy

62
docs/data-download.md Normal file
View File

@ -0,0 +1,62 @@
# Data Downloading
## Getting data for backtesting and hyperopt
To download data (candles / OHLCV) needed for backtesting and hyperoptimization use the `freqtrade download-data` command.
If no additional parameter is specified, freqtrade will download data for `"1m"` and `"5m"` timeframes for the last 30 days.
Exchange and pairs will come from `config.json` (if specified using `-c/--config`).
Otherwise `--exchange` becomes mandatory.
!!! Tip Updating existing data
If you already have backtesting data available in your data-directory and would like to refresh this data up to today, use `--days xx` with a number slightly higher than the missing number of days. Freqtrade will keep the available data and only download the missing data.
Be carefull though: If the number is too small (which would result in a few missing days), the whole dataset will be removed and only xx days will be downloaded.
### Pairs file
In alternative to the whitelist from `config.json`, a `pairs.json` file can be used.
If you are using Binance for example:
- create a directory `user_data/data/binance` and copy or create the `pairs.json` file in that directory.
- update the `pairs.json` file to contain the currency pairs you are interested in.
```bash
mkdir -p user_data/data/binance
cp freqtrade/tests/testdata/pairs.json user_data/data/binance
```
The format of the `pairs.json` file is a simple json list.
Mixing different stake-currencies is allowed for this file, since it's only used for downloading.
``` json
[
"ETH/BTC",
"ETH/USDT",
"BTC/USDT",
"XRP/ETH"
]
```
### start download
Then run:
```bash
freqtrade download-data --exchange binance
```
This will download ticker data for all the currency pairs you defined in `pairs.json`.
### Other Notes
- To use a different directory than the exchange specific default, use `--datadir user_data/data/some_directory`.
- To change the exchange used to download the tickers, please use a different configuration file (you'll probably need to adjust ratelimits etc.)
- To use `pairs.json` from some other directory, use `--pairs-file some_other_dir/pairs.json`.
- To download ticker data for only 10 days, use `--days 10` (defaults to 30 days).
- Use `--timeframes` to specify which tickers to download. Default is `--timeframes 1m 5m` which will download 1-minute and 5-minute tickers.
- To use exchange, timeframe and list of pairs as defined in your configuration file, use the `-c/--config` option. With this, the script uses the whitelist defined in the config as the list of currency pairs to download data for and does not require the pairs.json file. You can combine `-c/--config` with most other options.
## Next step
Great, you now have backtest data downloaded, so you can now start [backtesting](backtesting.md) your strategy.

6
docs/deprecated.md
View File

@ -4,7 +4,7 @@ This page contains description of the command line arguments, configuration para
and the bot features that were declared as DEPRECATED by the bot development team and the bot features that were declared as DEPRECATED by the bot development team
and are no longer supported. Please avoid their usage in your configuration. and are no longer supported. Please avoid their usage in your configuration.
## Deprecated ## Removed features
### the `--refresh-pairs-cached` command line option ### the `--refresh-pairs-cached` command line option
@ -12,9 +12,7 @@ and are no longer supported. Please avoid their usage in your configuration.
Since this leads to much confusion, and slows down backtesting (while not being part of backtesting) this has been singled out Since this leads to much confusion, and slows down backtesting (while not being part of backtesting) this has been singled out
as a seperate freqtrade subcommand `freqtrade download-data`. as a seperate freqtrade subcommand `freqtrade download-data`.
This command line option was deprecated in `2019.7-dev` and will be removed after the next release. This command line option was deprecated in 2019.7-dev (develop branch) and removed in 2019.9 (master branch).
## Removed features
### The **--dynamic-whitelist** command line option ### The **--dynamic-whitelist** command line option

4
docs/developer.md
View File

@ -2,7 +2,7 @@
This page is intended for developers of FreqTrade, people who want to contribute to the FreqTrade codebase or documentation, or people who want to understand the source code of the application they're running. This page is intended for developers of FreqTrade, people who want to contribute to the FreqTrade codebase or documentation, or people who want to understand the source code of the application they're running.
All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) where you can ask questions. All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel in [slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) where you can ask questions.
## Documentation ## Documentation
@ -30,7 +30,7 @@ These are available from `conftest.py` and can be imported in any test module.
A sample check looks as follows: A sample check looks as follows:
``` python ``` python
from freqtrade.tests.conftest import log_has, log_has_re from tests.conftest import log_has, log_has_re
def test_method_to_test(caplog): def test_method_to_test(caplog):
method_to_test() method_to_test()

65
docs/hyperopt.md
View File

@ -6,6 +6,9 @@ algorithms included in the `scikit-optimize` package to accomplish this. The
search will burn all your CPU cores, make your laptop sound like a fighter jet search will burn all your CPU cores, make your laptop sound like a fighter jet
and still take a long time. and still take a long time.
Hyperopt requires historic data to be available, just as backtesting does.
To learn how to get data for the pairs and exchange you're interrested in, head over to the [Data Downloading](data-download.md) section of the documentation.
!!! Bug !!! Bug
Hyperopt will crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133) Hyperopt will crash when used with only 1 CPU Core as found out in [Issue #1133](https://github.com/freqtrade/freqtrade/issues/1133)
@ -372,35 +375,42 @@ Buy hyperspace params:
'rsi-enabled': True, 'rsi-enabled': True,
'trigger': 'bb_lower'} 'trigger': 'bb_lower'}
ROI table: ROI table:
{ 0: 0.10674752302642071, { 0: 0.10674,
21: 0.09158372701087236, 21: 0.09158,
78: 0.03634636907306948, 78: 0.03634,
118: 0} 118: 0}
``` ```
This would translate to the following ROI table: In order to use this best ROI table found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `minimal_roi` attribute of your custom strategy:
``` python ```
# Minimal ROI designed for the strategy.
# This attribute will be overridden if the config file contains "minimal_roi"
minimal_roi = { minimal_roi = {
"118": 0, 0: 0.10674,
"78": 0.0363, 21: 0.09158,
"21": 0.0915, 78: 0.03634,
"0": 0.106 118: 0
} }
``` ```
As stated in the comment, you can also use it as the value of the `minimal_roi` setting in the configuration file.
If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps) with the values that can vary in the following ranges: #### Default ROI Search Space
| # | minutes | ROI percentage | If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps). Hyperopt implements adaptive ranges for ROI tables with ranges for values in the ROI steps that depend on the ticker_interval used. By default the values can vary in the following ranges (for some of the most used ticker intervals, values are rounded to 5 digits after the decimal point):
|---|---|---|
| 1 | always 0 | 0.03...0.31 |
| 2 | 10...40 | 0.02...0.11 |
| 3 | 20...100 | 0.01...0.04 |
| 4 | 30...220 | always 0 |
This structure of the ROI table is sufficient in most cases. Override the `roi_space()` method defining the ranges desired if you need components of the ROI tables to vary in other ranges. | # step | 1m | | 5m | | 1h | | 1d | |
|---|---|---|---|---|---|---|---|---|
| 1 | 0 | 0.01161...0.11992 | 0 | 0.03...0.31 | 0 | 0.06883...0.71124 | 0 | 0.12178...1.25835 |
| 2 | 2...8 | 0.00774...0.04255 | 10...40 | 0.02...0.11 | 120...480 | 0.04589...0.25238 | 2880...11520 | 0.08118...0.44651 |
| 3 | 4...20 | 0.00387...0.01547 | 20...100 | 0.01...0.04 | 240...1200 | 0.02294...0.09177 | 5760...28800 | 0.04059...0.16237 |
| 4 | 6...44 | 0.0 | 30...220 | 0.0 | 360...2640 | 0.0 | 8640...63360 | 0.0 |
Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization in these methods if you need a different structure of the ROI table or other amount of rows (steps) in the ROI tables. These ranges should be sufficient in most cases. The minutes in the steps (ROI dict keys) are scaled linearly depending on the ticker interval used. The ROI values in the steps (ROI dict values) are scaled logarithmically depending on the ticker interval used.
If you have the `generate_roi_table()` and `roi_space()` methods in your custom hyperopt file, remove them in order to utilize these adaptive ROI tables and the ROI hyperoptimization space generated by Freqtrade by default.
Override the `roi_space()` method if you need components of the ROI tables to vary in other ranges. Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization if you need a different structure of the ROI tables or other amount of rows (steps). A sample for these methods can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_advanced.py).
### Understand Hyperopt Stoploss results ### Understand Hyperopt Stoploss results
@ -417,12 +427,25 @@ Buy hyperspace params:
'adx-enabled': False, 'adx-enabled': False,
'rsi-enabled': True, 'rsi-enabled': True,
'trigger': 'bb_lower'} 'trigger': 'bb_lower'}
Stoploss: -0.37996664668703606 Stoploss: -0.27996
``` ```
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.5...-0.02, which is sufficient in most cases. In order to use this best stoploss value found by Hyperopt in backtesting and for live trades/dry-run, copy-paste it as the value of the `stoploss` attribute of your custom strategy:
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization. ```
# Optimal stoploss designed for the strategy
# This attribute will be overridden if the config file contains "stoploss"
stoploss = -0.27996
```
As stated in the comment, you can also use it as the value of the `stoploss` setting in the configuration file.
#### Default Stoploss Search Space
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.35...-0.02, which is sufficient in most cases.
If you have the `stoploss_space()` method in your custom hyperopt file, remove it in order to utilize Stoploss hyperoptimization space generated by Freqtrade by default.
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization. A sample for this method can be found in [user_data/hyperopts/sample_hyperopt_advanced.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_advanced.py).
### Validate backtesting results ### Validate backtesting results

2
docs/index.md
View File

@ -64,7 +64,7 @@ To run this bot we recommend you a cloud instance with a minimum of:
Help / Slack Help / Slack
For any questions not covered by the documentation or for further information about the bot, we encourage you to join our Slack channel. For any questions not covered by the documentation or for further information about the bot, we encourage you to join our Slack channel.
Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) to join Slack channel. Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) to join Slack channel.
## Ready to try? ## Ready to try?

11
docs/installation.md
View File

@ -99,8 +99,8 @@ sudo apt-get install build-essential git
Before installing FreqTrade on a Raspberry Pi running the official Raspbian Image, make sure you have at least Python 3.6 installed. The default image only provides Python 3.5. Probably the easiest way to get a recent version of python is [miniconda](https://repo.continuum.io/miniconda/). Before installing FreqTrade on a Raspberry Pi running the official Raspbian Image, make sure you have at least Python 3.6 installed. The default image only provides Python 3.5. Probably the easiest way to get a recent version of python is [miniconda](https://repo.continuum.io/miniconda/).
The following assumes that miniconda3 is installed and available in your environment. Last miniconda3 installation file use python 3.4, we will update to python 3.6 on this installation. The following assumes that miniconda3 is installed and available in your environment. Since the last miniconda3 installation file uses python 3.4, we will update to python 3.6 on this installation.
It's recommended to use (mini)conda for this as installation/compilation of `numpy`, `scipy` and `pandas` takes a long time. It's recommended to use (mini)conda for this as installation/compilation of `numpy` and `pandas` takes a long time.
Additional package to install on your Raspbian, `libffi-dev` required by cryptography (from python-telegram-bot). Additional package to install on your Raspbian, `libffi-dev` required by cryptography (from python-telegram-bot).
@ -109,13 +109,17 @@ conda config --add channels rpi
conda install python=3.6 conda install python=3.6
conda create -n freqtrade python=3.6 conda create -n freqtrade python=3.6
conda activate freqtrade conda activate freqtrade
conda install scipy pandas numpy conda install pandas numpy
sudo apt install libffi-dev sudo apt install libffi-dev
python3 -m pip install -r requirements-common.txt python3 -m pip install -r requirements-common.txt
python3 -m pip install -e . python3 -m pip install -e .
``` ```
!!! Note
This does not install hyperopt dependencies. To install these, please use `python3 -m pip install -e .[hyperopt]`.
We do not advise to run hyperopt on a Raspberry Pi, since this is a very resource-heavy operation, which should be done on powerful machine.
### Common ### Common
#### 1. Install TA-Lib #### 1. Install TA-Lib
@ -175,7 +179,6 @@ cp config.json.example config.json
``` bash ``` bash
python3 -m pip install --upgrade pip python3 -m pip install --upgrade pip
python3 -m pip install -r requirements.txt
python3 -m pip install -e . python3 -m pip install -e .
``` ```

164
docs/plotting.md
View File

@ -2,9 +2,9 @@
This page explains how to plot prices, indicators and profits. This page explains how to plot prices, indicators and profits.
## Installation ## Installation / Setup
Plotting scripts use Plotly library. Install/upgrade it with: Plotting modules use the Plotly library. You can install / upgrade this by running the following command:
``` bash ``` bash
pip install -U -r requirements-plot.txt pip install -U -r requirements-plot.txt
@ -12,90 +12,172 @@ pip install -U -r requirements-plot.txt
## Plot price and indicators ## Plot price and indicators
Usage for the price plotter: The `freqtrade plot-dataframe` subcommand shows an interactive graph with three subplots:
* Main plot with candlestics and indicators following price (sma/ema)
* Volume bars
* Additional indicators as specified by `--indicators2`
![plot-dataframe](assets/plot-dataframe.png)
Possible arguments:
```
usage: freqtrade plot-dataframe [-h] [-p PAIRS [PAIRS ...]]
[--indicators1 INDICATORS1 [INDICATORS1 ...]]
[--indicators2 INDICATORS2 [INDICATORS2 ...]]
[--plot-limit INT] [--db-url PATH]
[--trade-source {DB,file}] [--export EXPORT]
[--export-filename PATH]
[--timerange TIMERANGE]
optional arguments:
-h, --help show this help message and exit
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Show profits for only these pairs. Pairs are space-
separated.
--indicators1 INDICATORS1 [INDICATORS1 ...]
Set indicators from your strategy you want in the
first row of the graph. Space-separated list. Example:
`ema3 ema5`. Default: `['sma', 'ema3', 'ema5']`.
--indicators2 INDICATORS2 [INDICATORS2 ...]
Set indicators from your strategy you want in the
third row of the graph. Space-separated list. Example:
`fastd fastk`. Default: `['macd', 'macdsignal']`.
--plot-limit INT Specify tick limit for plotting. Notice: too high
values cause huge files. Default: 750.
--db-url PATH Override trades database URL, this is useful in custom
deployments (default: `sqlite:///tradesv3.sqlite` for
Live Run mode, `sqlite://` for Dry Run).
--trade-source {DB,file}
Specify the source for trades (Can be DB or file
(backtest file)) Default: file
--export EXPORT Export backtest results, argument are: trades.
Example: `--export=trades`
--export-filename PATH
Save backtest results to the file with this filename
(default: `user_data/backtest_results/backtest-
result.json`). Requires `--export` to be set as well.
Example: `--export-filename=user_data/backtest_results
/backtest_today.json`
--timerange TIMERANGE
Specify what timerange of data to use.
``` bash
python3 script/plot_dataframe.py [-h] [-p pairs]
``` ```
Example Example:
``` bash ``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH freqtrade plot-dataframe -p BTC/ETH
``` ```
The `-p` pairs argument can be used to specify pairs you would like to plot. The `-p/--pairs` argument can be used to specify pairs you would like to plot.
!!! Note
The `freqtrade plot-dataframe` subcommand generates one plot-file per pair.
Specify custom indicators. Specify custom indicators.
Use `--indicators1` for the main plot and `--indicators2` for the subplot below (if values are in a different range than prices). Use `--indicators1` for the main plot and `--indicators2` for the subplot below (if values are in a different range than prices).
!!! tip
You will almost certainly want to specify a custom strategy! This can be done by adding `-s Classname` / `--strategy ClassName` to the command.
``` bash ``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH --indicators1 sma,ema --indicators2 macd freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --indicators1 sma ema --indicators2 macd
``` ```
### Advanced use ### Further usage examples
To plot multiple pairs, separate them with a comma: To plot multiple pairs, separate them with a space:
``` bash ``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH,XRP/ETH freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH XRP/ETH
``` ```
To plot a timerange (to zoom in): To plot a timerange (to zoom in)
``` bash ``` bash
python3 scripts/plot_dataframe.py -p BTC/ETH --timerange=20180801-20180805 freqtrade --strategy AwesomeStrategy plot-dataframe -p BTC/ETH --timerange=20180801-20180805
``` ```
To plot trades stored in a database use `--db-url` argument: To plot trades stored in a database use `--db-url` in combination with `--trade-source DB`:
``` bash ``` bash
python3 scripts/plot_dataframe.py --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB freqtrade --strategy AwesomeStrategy plot-dataframe --db-url sqlite:///tradesv3.dry_run.sqlite -p BTC/ETH --trade-source DB
``` ```
To plot trades from a backtesting result, use `--export-filename <filename>` To plot trades from a backtesting result, use `--export-filename <filename>`
``` bash ``` bash
python3 scripts/plot_dataframe.py --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH freqtrade --strategy AwesomeStrategy plot-dataframe --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
```
To plot a custom strategy the strategy should have first be backtested.
The results may then be plotted with the -s argument:
``` bash
python3 scripts/plot_dataframe.py -s Strategy_Name -p BTC/ETH --datadir user_data/data/<exchange_name>/
``` ```
## Plot profit ## Plot profit
The profit plotter shows a picture with three plots: ![plot-profit](assets/plot-profit.png)
The `freqtrade plot-profit` subcommand shows an interactive graph with three plots:
1) Average closing price for all pairs 1) Average closing price for all pairs
2) The summarized profit made by backtesting. 2) The summarized profit made by backtesting.
Note that this is not the real-world profit, but Note that this is not the real-world profit, but more of an estimate.
more of an estimate. 3) Profit for each individual pair
3) Each pair individually profit
The first graph is good to get a grip of how the overall market The first graph is good to get a grip of how the overall market progresses.
progresses.
The second graph will show how your algorithm works or doesn't. The second graph will show if your algorithm works or doesn't.
Perhaps you want an algorithm that steadily makes small profits, Perhaps you want an algorithm that steadily makes small profits, or one that acts less often, but makes big swings.
or one that acts less seldom, but makes big swings.
The third graph can be useful to spot outliers, events in pairs The third graph can be useful to spot outliers, events in pairs that cause profit spikes.
that makes profit spikes.
Usage for the profit plotter: Possible options for the `freqtrade plot-profit` subcommand:
```
usage: freqtrade plot-profit [-h] [-p PAIRS [PAIRS ...]]
[--timerange TIMERANGE] [--export EXPORT]
[--export-filename PATH] [--db-url PATH]
[--trade-source {DB,file}]
optional arguments:
-h, --help show this help message and exit
-p PAIRS [PAIRS ...], --pairs PAIRS [PAIRS ...]
Show profits for only these pairs. Pairs are space-
separated.
--timerange TIMERANGE
Specify what timerange of data to use.
--export EXPORT Export backtest results, argument are: trades.
Example: `--export=trades`
--export-filename PATH
Save backtest results to the file with this filename
(default: `user_data/backtest_results/backtest-
result.json`). Requires `--export` to be set as well.
Example: `--export-filename=user_data/backtest_results
/backtest_today.json`
--db-url PATH Override trades database URL, this is useful in custom
deployments (default: `sqlite:///tradesv3.sqlite` for
Live Run mode, `sqlite://` for Dry Run).
--trade-source {DB,file}
Specify the source for trades (Can be DB or file
(backtest file)) Default: file
``` bash
python3 script/plot_profit.py [-h] [-p pair] [--datadir directory] [--ticker_interval num]
``` ```
The `-p` pair argument, can be used to plot a single pair The `-p/--pairs` argument, can be used to limit the pairs that are considered for this calculation.
Example Examples:
Use custom backtest-export file
``` bash ``` bash
python3 scripts/plot_profit.py --datadir ../freqtrade/freqtrade/tests/testdata-20171221/ -p LTC/BTC freqtrade plot-profit -p LTC/BTC --export-filename user_data/backtest_results/backtest-result-Strategy005.json
```
Use custom database
``` bash
freqtrade plot-profit -p LTC/BTC --db-url sqlite:///tradesv3.sqlite --trade-source DB
```
``` bash
freqtrade plot-profit --datadir user_data/data/binance_save/ -p LTC/BTC
``` ```

2
docs/requirements-docs.txt
View File

@ -1 +1 @@
mkdocs-material==4.4.0 mkdocs-material==4.4.2

24
docs/strategy-customization.md
View File

@ -24,7 +24,7 @@ strategy file will be updated on Github. Put your custom strategy file
into the directory `user_data/strategies`. into the directory `user_data/strategies`.
Best copy the test-strategy and modify this copy to avoid having bot-updates override your changes. Best copy the test-strategy and modify this copy to avoid having bot-updates override your changes.
`cp user_data/strategies/test_strategy.py user_data/strategies/awesome-strategy.py` `cp user_data/strategies/sample_strategy.py user_data/strategies/awesome-strategy.py`
### Anatomy of a strategy ### Anatomy of a strategy
@ -36,14 +36,19 @@ A strategy file contains all the information needed to build a good strategy:
- Minimal ROI recommended - Minimal ROI recommended
- Stoploss strongly recommended - Stoploss strongly recommended
The bot also include a sample strategy called `TestStrategy` you can update: `user_data/strategies/test_strategy.py`. The bot also include a sample strategy called `SampleStrategy` you can update: `user_data/strategies/sample_strategy.py`.
You can test it with the parameter: `--strategy TestStrategy` You can test it with the parameter: `--strategy SampleStrategy`
Additionally, there is an attribute called `INTERFACE_VERSION`, which defines the version of the strategy interface the bot should use.
The current version is 2 - which is also the default when it's not set explicitly in the strategy.
Future versions will require this to be set.
```bash ```bash
freqtrade --strategy AwesomeStrategy freqtrade --strategy AwesomeStrategy
``` ```
**For the following section we will use the [user_data/strategies/test_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py) **For the following section we will use the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py)
file as reference.** file as reference.**
!!! Note Strategies and Backtesting !!! Note Strategies and Backtesting
@ -109,9 +114,8 @@ def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame
return dataframe return dataframe
``` ```
!!! Note "Want more indicator examples?" !!! Note "Want more indicator examples?"
Look into the [user_data/strategies/test_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/test_strategy.py).<br/> Look into the [user_data/strategies/sample_strategy.py](https://github.com/freqtrade/freqtrade/blob/develop/user_data/strategies/sample_strategy.py).
Then uncomment indicators you need. Then uncomment indicators you need.
### Buy signal rules ### Buy signal rules
@ -122,7 +126,7 @@ It's important to always return the dataframe without removing/modifying the col
This will method will also define a new column, `"buy"`, which needs to contain 1 for buys, and 0 for "no action". This will method will also define a new column, `"buy"`, which needs to contain 1 for buys, and 0 for "no action".
Sample from `user_data/strategies/test_strategy.py`: Sample from `user_data/strategies/sample_strategy.py`:
```python ```python
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -152,7 +156,7 @@ It's important to always return the dataframe without removing/modifying the col
This will method will also define a new column, `"sell"`, which needs to contain 1 for sells, and 0 for "no action". This will method will also define a new column, `"sell"`, which needs to contain 1 for sells, and 0 for "no action".
Sample from `user_data/strategies/test_strategy.py`: Sample from `user_data/strategies/sample_strategy.py`:
```python ```python
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -220,7 +224,7 @@ This would signify a stoploss of -10%.
For the full documentation on stoploss features, look at the dedicated [stoploss page](stoploss.md). For the full documentation on stoploss features, look at the dedicated [stoploss page](stoploss.md).
If your exchange supports it, it's recommended to also set `"stoploss_on_exchange"` in the order dict, so your stoploss is on the exchange and cannot be missed for network-problems (or other problems). If your exchange supports it, it's recommended to also set `"stoploss_on_exchange"` in the order_types dictionary, so your stoploss is on the exchange and cannot be missed due to network problems, high load or other reasons.
For more information on order_types please look [here](configuration.md#understand-order_types). For more information on order_types please look [here](configuration.md#understand-order_types).
@ -407,7 +411,7 @@ To get additional Ideas for strategies, head over to our [strategy repository](h
Feel free to use any of them as inspiration for your own strategies. Feel free to use any of them as inspiration for your own strategies.
We're happy to accept Pull Requests containing new Strategies to that repo. We're happy to accept Pull Requests containing new Strategies to that repo.
We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LWEyODBiNzkzNzcyNzU0MWYyYzE5NjIyOTQxMzBmMGUxOTIzM2YyN2Y4NWY1YTEwZDgwYTRmMzE2NmM5ZmY2MTg) which is a great place to get and/or share ideas. We also got a *strategy-sharing* channel in our [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) which is a great place to get and/or share ideas.
## Next step ## Next step

11
environment.yml
View File

@ -9,25 +9,26 @@ dependencies:
- wheel - wheel
- numpy - numpy
- pandas - pandas
- scipy
- SQLAlchemy - SQLAlchemy
- scikit-learn
- arrow - arrow
- requests - requests
- urllib3 - urllib3
- wrapt - wrapt
- joblib
- jsonschema - jsonschema
- tabulate - tabulate
- python-rapidjson - python-rapidjson
- filelock
- flask - flask
- python-dotenv - python-dotenv
- cachetools - cachetools
- scikit-optimize
- python-telegram-bot - python-telegram-bot
# Optional for plotting # Optional for plotting
- plotly - plotly
# Optional for hyperopt
- scipy
- scikit-optimize
- scikit-learn
- filelock
- joblib
# Optional for development # Optional for development
- flake8 - flake8
- pytest - pytest

15
freqtrade/__init__.py
View File

@ -1,5 +1,16 @@
""" FreqTrade bot """ """ FreqTrade bot """
__version__ = '2019.8-1' __version__ = '2019.9'
if __version__ == 'develop':
try:
import subprocess
__version__ = 'develop-' + subprocess.check_output(
['git', 'log', '--format="%h"', '-n 1'],
stderr=subprocess.DEVNULL).decode("utf-8").rstrip().strip('"')
except Exception:
# git not available, ignore
pass
class DependencyException(Exception): class DependencyException(Exception):
@ -11,7 +22,7 @@ class DependencyException(Exception):
class OperationalException(Exception): class OperationalException(Exception):
""" """
Requires manual intervention. Requires manual intervention and will usually stop the bot.
This happens when an exchange returns an unexpected error during runtime This happens when an exchange returns an unexpected error during runtime
or given configuration is invalid. or given configuration is invalid.
""" """

56
freqtrade/configuration/arguments.py
View File

@ -2,10 +2,11 @@
This module contains the argument manager class This module contains the argument manager class
""" """
import argparse import argparse
from typing import List, Optional from pathlib import Path
from typing import Any, Dict, List, Optional
from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
from freqtrade import constants from freqtrade import constants
from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir", "user_data_dir"] ARGS_COMMON = ["verbosity", "logfile", "version", "config", "datadir", "user_data_dir"]
@ -14,7 +15,7 @@ ARGS_STRATEGY = ["strategy", "strategy_path"]
ARGS_MAIN = ARGS_COMMON + ARGS_STRATEGY + ["db_url", "sd_notify"] ARGS_MAIN = ARGS_COMMON + ARGS_STRATEGY + ["db_url", "sd_notify"]
ARGS_COMMON_OPTIMIZE = ["ticker_interval", "timerange", ARGS_COMMON_OPTIMIZE = ["ticker_interval", "timerange",
"max_open_trades", "stake_amount", "refresh_pairs"] "max_open_trades", "stake_amount"]
ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_positions", ARGS_BACKTEST = ARGS_COMMON_OPTIMIZE + ["position_stacking", "use_max_market_positions",
"strategy_list", "export", "exportfilename"] "strategy_list", "export", "exportfilename"]
@ -34,33 +35,29 @@ ARGS_CREATE_USERDIR = ["user_data_dir"]
ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "exchange", "timeframes", "erase"] ARGS_DOWNLOAD_DATA = ["pairs", "pairs_file", "days", "exchange", "timeframes", "erase"]
ARGS_PLOT_DATAFRAME = (ARGS_COMMON + ARGS_STRATEGY + ARGS_PLOT_DATAFRAME = ["pairs", "indicators1", "indicators2", "plot_limit", "db_url",
["pairs", "indicators1", "indicators2", "plot_limit", "db_url", "trade_source", "export", "exportfilename", "timerange", "ticker_interval"]
"trade_source", "export", "exportfilename", "timerange",
"refresh_pairs"])
ARGS_PLOT_PROFIT = (ARGS_COMMON + ARGS_STRATEGY + ARGS_PLOT_PROFIT = ["pairs", "timerange", "export", "exportfilename", "db_url",
["pairs", "timerange", "export", "exportfilename", "db_url", "trade_source"]) "trade_source", "ticker_interval"]
NO_CONF_REQURIED = ["start_download_data"] NO_CONF_REQURIED = ["create-userdir", "download-data", "plot-dataframe", "plot-profit"]
class Arguments(object): class Arguments:
""" """
Arguments Class. Manage the arguments received by the cli Arguments Class. Manage the arguments received by the cli
""" """
def __init__(self, args: Optional[List[str]], description: str, def __init__(self, args: Optional[List[str]]) -> None:
no_default_config: bool = False) -> None:
self.args = args self.args = args
self._parsed_arg: Optional[argparse.Namespace] = None self._parsed_arg: Optional[argparse.Namespace] = None
self.parser = argparse.ArgumentParser(description=description) self.parser = argparse.ArgumentParser(description='Free, open source crypto trading bot')
self._no_default_config = no_default_config
def _load_args(self) -> None: def _load_args(self) -> None:
self._build_args(optionlist=ARGS_MAIN) self._build_args(optionlist=ARGS_MAIN)
self._build_subcommands() self._build_subcommands()
def get_parsed_arg(self) -> argparse.Namespace: def get_parsed_arg(self) -> Dict[str, Any]:
""" """
Return the list of arguments Return the list of arguments
:return: List[str] List of arguments :return: List[str] List of arguments
@ -69,7 +66,7 @@ class Arguments(object):
self._load_args() self._load_args()
self._parsed_arg = self._parse_args() self._parsed_arg = self._parse_args()
return self._parsed_arg return vars(self._parsed_arg)
def _parse_args(self) -> argparse.Namespace: def _parse_args(self) -> argparse.Namespace:
""" """
@ -77,12 +74,13 @@ class Arguments(object):
""" """
parsed_arg = self.parser.parse_args(self.args) parsed_arg = self.parser.parse_args(self.args)
# When no config is provided, but a config exists, use that configuration!
# Workaround issue in argparse with action='append' and default value # Workaround issue in argparse with action='append' and default value
# (see https://bugs.python.org/issue16399) # (see https://bugs.python.org/issue16399)
# Allow no-config for certain commands (like downloading / plotting) # Allow no-config for certain commands (like downloading / plotting)
if (not self._no_default_config and parsed_arg.config is None if (parsed_arg.config is None and ((Path.cwd() / constants.DEFAULT_CONFIG).is_file() or
and not (hasattr(parsed_arg, 'func') not ('subparser' in parsed_arg and parsed_arg.subparser in NO_CONF_REQURIED))):
and parsed_arg.func.__name__ in NO_CONF_REQURIED)):
parsed_arg.config = [constants.DEFAULT_CONFIG] parsed_arg.config = [constants.DEFAULT_CONFIG]
return parsed_arg return parsed_arg
@ -119,6 +117,7 @@ class Arguments(object):
hyperopt_cmd.set_defaults(func=start_hyperopt) hyperopt_cmd.set_defaults(func=start_hyperopt)
self._build_args(optionlist=ARGS_HYPEROPT, parser=hyperopt_cmd) self._build_args(optionlist=ARGS_HYPEROPT, parser=hyperopt_cmd)
# add create-userdir subcommand
create_userdir_cmd = subparsers.add_parser('create-userdir', create_userdir_cmd = subparsers.add_parser('create-userdir',
help="Create user-data directory.") help="Create user-data directory.")
create_userdir_cmd.set_defaults(func=start_create_userdir) create_userdir_cmd.set_defaults(func=start_create_userdir)
@ -139,3 +138,20 @@ class Arguments(object):
) )
download_data_cmd.set_defaults(func=start_download_data) download_data_cmd.set_defaults(func=start_download_data)
self._build_args(optionlist=ARGS_DOWNLOAD_DATA, parser=download_data_cmd) self._build_args(optionlist=ARGS_DOWNLOAD_DATA, parser=download_data_cmd)
# Add Plotting subcommand
from freqtrade.plot.plot_utils import start_plot_dataframe, start_plot_profit
plot_dataframe_cmd = subparsers.add_parser(
'plot-dataframe',
help='Plot candles with indicators.'
)
plot_dataframe_cmd.set_defaults(func=start_plot_dataframe)
self._build_args(optionlist=ARGS_PLOT_DATAFRAME, parser=plot_dataframe_cmd)
# Plot profit
plot_profit_cmd = subparsers.add_parser(
'plot-profit',
help='Generate plot showing profits.'
)
plot_profit_cmd.set_defaults(func=start_plot_profit)
self._build_args(optionlist=ARGS_PLOT_PROFIT, parser=plot_profit_cmd)

13
freqtrade/configuration/check_exchange.py
View File

@ -5,6 +5,7 @@ from freqtrade import OperationalException
from freqtrade.exchange import (available_exchanges, get_exchange_bad_reason, from freqtrade.exchange import (available_exchanges, get_exchange_bad_reason,
is_exchange_available, is_exchange_bad, is_exchange_available, is_exchange_bad,
is_exchange_officially_supported) is_exchange_officially_supported)
from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -19,9 +20,21 @@ def check_exchange(config: Dict[str, Any], check_for_bad: bool = True) -> bool:
raises an exception if the exchange if not supported by ccxt raises an exception if the exchange if not supported by ccxt
and thus is not known for the Freqtrade at all. and thus is not known for the Freqtrade at all.
""" """
if config['runmode'] in [RunMode.PLOT] and not config.get('exchange', {}).get('name'):
# Skip checking exchange in plot mode, since it requires no exchange
return True
logger.info("Checking exchange...") logger.info("Checking exchange...")
exchange = config.get('exchange', {}).get('name').lower() exchange = config.get('exchange', {}).get('name').lower()
if not exchange:
raise OperationalException(
f'This command requires a configured exchange. You should either use '
f'`--exchange <exchange_name>` or specify a configuration file via `--config`.\n'
f'The following exchanges are supported by ccxt: '
f'{", ".join(available_exchanges())}'
)
if not is_exchange_available(exchange): if not is_exchange_available(exchange):
raise OperationalException( raise OperationalException(
f'Exchange "{exchange}" is not supported by ccxt ' f'Exchange "{exchange}" is not supported by ccxt '

17
freqtrade/configuration/cli_options.py
View File

@ -107,13 +107,6 @@ AVAILABLE_CLI_OPTIONS = {
help='Specify stake_amount.', help='Specify stake_amount.',
type=float, type=float,
), ),
"refresh_pairs": Arg(
'-r', '--refresh-pairs-cached',
help='Refresh the pairs files in tests/testdata with the latest data from the '
'exchange. Use it if you want to run your optimization commands with '
'up-to-date data.',
action='store_true',
),
# Backtesting # Backtesting
"position_stacking": Arg( "position_stacking": Arg(
'--eps', '--enable-position-stacking', '--eps', '--enable-position-stacking',
@ -292,14 +285,16 @@ AVAILABLE_CLI_OPTIONS = {
"indicators1": Arg( "indicators1": Arg(
'--indicators1', '--indicators1',
help='Set indicators from your strategy you want in the first row of the graph. ' help='Set indicators from your strategy you want in the first row of the graph. '
'Comma-separated list. Example: `ema3,ema5`. Default: `%(default)s`.', 'Space-separated list. Example: `ema3 ema5`. Default: `%(default)s`.',
default='sma,ema3,ema5', default=['sma', 'ema3', 'ema5'],
nargs='+',
), ),
"indicators2": Arg( "indicators2": Arg(
'--indicators2', '--indicators2',
help='Set indicators from your strategy you want in the third row of the graph. ' help='Set indicators from your strategy you want in the third row of the graph. '
'Comma-separated list. Example: `fastd,fastk`. Default: `%(default)s`.', 'Space-separated list. Example: `fastd fastk`. Default: `%(default)s`.',
default='macd,macdsignal', default=['macd', 'macdsignal'],
nargs='+',
), ),
"plot_limit": Arg( "plot_limit": Arg(
'--plot-limit', '--plot-limit',

85
freqtrade/configuration/configuration.py
View File

@ -3,7 +3,6 @@ This module contains the configuration class
""" """
import logging import logging
import warnings import warnings
from argparse import Namespace
from copy import deepcopy from copy import deepcopy
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Dict, List, Optional from typing import Any, Callable, Dict, List, Optional
@ -22,13 +21,13 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class Configuration(object): class Configuration:
""" """
Class to read and init the bot configuration Class to read and init the bot configuration
Reuse this class for the bot, backtesting, hyperopt and every script that required configuration Reuse this class for the bot, backtesting, hyperopt and every script that required configuration
""" """
def __init__(self, args: Namespace, runmode: RunMode = None) -> None: def __init__(self, args: Dict[str, Any], runmode: RunMode = None) -> None:
self.args = args self.args = args
self.config: Optional[Dict[str, Any]] = None self.config: Optional[Dict[str, Any]] = None
self.runmode = runmode self.runmode = runmode
@ -50,9 +49,16 @@ class Configuration(object):
and merging their contents. and merging their contents.
Files are loaded in sequence, parameters in later configuration files Files are loaded in sequence, parameters in later configuration files
override the same parameter from an earlier file (last definition wins). override the same parameter from an earlier file (last definition wins).
Runs through the whole Configuration initialization, so all expected config entries
are available to interactive environments.
:param files: List of file paths :param files: List of file paths
:return: configuration dictionary :return: configuration dictionary
""" """
c = Configuration({"config": files}, RunMode.OTHER)
return c.get_config()
def load_from_files(self, files: List[str]) -> Dict[str, Any]:
# Keep this method as staticmethod, so it can be used from interactive environments # Keep this method as staticmethod, so it can be used from interactive environments
config: Dict[str, Any] = {} config: Dict[str, Any] = {}
@ -82,7 +88,10 @@ class Configuration(object):
:return: Configuration dictionary :return: Configuration dictionary
""" """
# Load all configs # Load all configs
config: Dict[str, Any] = Configuration.from_files(self.args.config) config: Dict[str, Any] = self.load_from_files(self.args["config"])
# Keep a copy of the original configuration file
config['original_config'] = deepcopy(config)
self._process_common_options(config) self._process_common_options(config)
@ -107,13 +116,10 @@ class Configuration(object):
the -v/--verbose, --logfile options the -v/--verbose, --logfile options
""" """
# Log level # Log level
if 'verbosity' in self.args and self.args.verbosity: config.update({'verbosity': self.args.get("verbosity", 0)})
config.update({'verbosity': self.args.verbosity})
else:
config.update({'verbosity': 0})
if 'logfile' in self.args and self.args.logfile: if 'logfile' in self.args and self.args["logfile"]:
config.update({'logfile': self.args.logfile}) config.update({'logfile': self.args["logfile"]})
setup_logging(config) setup_logging(config)
@ -122,15 +128,15 @@ class Configuration(object):
self._process_logging_options(config) self._process_logging_options(config)
# Set strategy if not specified in config and or if it's non default # Set strategy if not specified in config and or if it's non default
if self.args.strategy != constants.DEFAULT_STRATEGY or not config.get('strategy'): if self.args.get("strategy") != constants.DEFAULT_STRATEGY or not config.get('strategy'):
config.update({'strategy': self.args.strategy}) config.update({'strategy': self.args.get("strategy")})
self._args_to_config(config, argname='strategy_path', self._args_to_config(config, argname='strategy_path',
logstring='Using additional Strategy lookup path: {}') logstring='Using additional Strategy lookup path: {}')
if ('db_url' in self.args and self.args.db_url and if ('db_url' in self.args and self.args["db_url"] and
self.args.db_url != constants.DEFAULT_DB_PROD_URL): self.args["db_url"] != constants.DEFAULT_DB_PROD_URL):
config.update({'db_url': self.args.db_url}) config.update({'db_url': self.args["db_url"]})
logger.info('Parameter --db-url detected ...') logger.info('Parameter --db-url detected ...')
if config.get('dry_run', False): if config.get('dry_run', False):
@ -153,7 +159,7 @@ class Configuration(object):
config['max_open_trades'] = float('inf') config['max_open_trades'] = float('inf')
# Support for sd_notify # Support for sd_notify
if 'sd_notify' in self.args and self.args.sd_notify: if 'sd_notify' in self.args and self.args["sd_notify"]:
config['internals'].update({'sd_notify': True}) config['internals'].update({'sd_notify': True})
def _process_datadir_options(self, config: Dict[str, Any]) -> None: def _process_datadir_options(self, config: Dict[str, Any]) -> None:
@ -162,12 +168,12 @@ class Configuration(object):
--user-data, --datadir --user-data, --datadir
""" """
# Check exchange parameter here - otherwise `datadir` might be wrong. # Check exchange parameter here - otherwise `datadir` might be wrong.
if "exchange" in self.args and self.args.exchange: if "exchange" in self.args and self.args["exchange"]:
config['exchange']['name'] = self.args.exchange config['exchange']['name'] = self.args["exchange"]
logger.info(f"Using exchange {config['exchange']['name']}") logger.info(f"Using exchange {config['exchange']['name']}")
if 'user_data_dir' in self.args and self.args.user_data_dir: if 'user_data_dir' in self.args and self.args["user_data_dir"]:
config.update({'user_data_dir': self.args.user_data_dir}) config.update({'user_data_dir': self.args["user_data_dir"]})
elif 'user_data_dir' not in config: elif 'user_data_dir' not in config:
# Default to cwd/user_data (legacy option ...) # Default to cwd/user_data (legacy option ...)
config.update({'user_data_dir': str(Path.cwd() / "user_data")}) config.update({'user_data_dir': str(Path.cwd() / "user_data")})
@ -176,10 +182,7 @@ class Configuration(object):
config['user_data_dir'] = create_userdata_dir(config['user_data_dir'], create_dir=False) config['user_data_dir'] = create_userdata_dir(config['user_data_dir'], create_dir=False)
logger.info('Using user-data directory: %s ...', config['user_data_dir']) logger.info('Using user-data directory: %s ...', config['user_data_dir'])
if 'datadir' in self.args and self.args.datadir: config.update({'datadir': create_datadir(config, self.args.get("datadir", None))})
config.update({'datadir': create_datadir(config, self.args.datadir)})
else:
config.update({'datadir': create_datadir(config, None)})
logger.info('Using data directory: %s ...', config.get('datadir')) logger.info('Using data directory: %s ...', config.get('datadir'))
def _process_optimize_options(self, config: Dict[str, Any]) -> None: def _process_optimize_options(self, config: Dict[str, Any]) -> None:
@ -192,12 +195,12 @@ class Configuration(object):
self._args_to_config(config, argname='position_stacking', self._args_to_config(config, argname='position_stacking',
logstring='Parameter --enable-position-stacking detected ...') logstring='Parameter --enable-position-stacking detected ...')
if 'use_max_market_positions' in self.args and not self.args.use_max_market_positions: if 'use_max_market_positions' in self.args and not self.args["use_max_market_positions"]:
config.update({'use_max_market_positions': False}) config.update({'use_max_market_positions': False})
logger.info('Parameter --disable-max-market-positions detected ...') logger.info('Parameter --disable-max-market-positions detected ...')
logger.info('max_open_trades set to unlimited ...') logger.info('max_open_trades set to unlimited ...')
elif 'max_open_trades' in self.args and self.args.max_open_trades: elif 'max_open_trades' in self.args and self.args["max_open_trades"]:
config.update({'max_open_trades': self.args.max_open_trades}) config.update({'max_open_trades': self.args["max_open_trades"]})
logger.info('Parameter --max_open_trades detected, ' logger.info('Parameter --max_open_trades detected, '
'overriding max_open_trades to: %s ...', config.get('max_open_trades')) 'overriding max_open_trades to: %s ...', config.get('max_open_trades'))
else: else:
@ -212,12 +215,8 @@ class Configuration(object):
self._process_datadir_options(config) self._process_datadir_options(config)
self._args_to_config(config, argname='refresh_pairs',
logstring='Parameter -r/--refresh-pairs-cached detected ...',
deprecated_msg='-r/--refresh-pairs-cached will be removed soon.')
self._args_to_config(config, argname='strategy_list', self._args_to_config(config, argname='strategy_list',
logstring='Using strategy list of {} Strategies', logfun=len) logstring='Using strategy list of {} strategies', logfun=len)
self._args_to_config(config, argname='ticker_interval', self._args_to_config(config, argname='ticker_interval',
logstring='Overriding ticker interval with Command line argument') logstring='Overriding ticker interval with Command line argument')
@ -229,16 +228,16 @@ class Configuration(object):
logstring='Storing backtest results to {} ...') logstring='Storing backtest results to {} ...')
# Edge section: # Edge section:
if 'stoploss_range' in self.args and self.args.stoploss_range: if 'stoploss_range' in self.args and self.args["stoploss_range"]:
txt_range = eval(self.args.stoploss_range) txt_range = eval(self.args["stoploss_range"])
config['edge'].update({'stoploss_range_min': txt_range[0]}) config['edge'].update({'stoploss_range_min': txt_range[0]})
config['edge'].update({'stoploss_range_max': txt_range[1]}) config['edge'].update({'stoploss_range_max': txt_range[1]})
config['edge'].update({'stoploss_range_step': txt_range[2]}) config['edge'].update({'stoploss_range_step': txt_range[2]})
logger.info('Parameter --stoplosses detected: %s ...', self.args.stoploss_range) logger.info('Parameter --stoplosses detected: %s ...', self.args["stoploss_range"])
# Hyperopt section # Hyperopt section
self._args_to_config(config, argname='hyperopt', self._args_to_config(config, argname='hyperopt',
logstring='Using Hyperopt file {}') logstring='Using Hyperopt class name: {}')
self._args_to_config(config, argname='hyperopt_path', self._args_to_config(config, argname='hyperopt_path',
logstring='Using additional Hyperopt lookup path: {}') logstring='Using additional Hyperopt lookup path: {}')
@ -254,7 +253,7 @@ class Configuration(object):
self._args_to_config(config, argname='print_all', self._args_to_config(config, argname='print_all',
logstring='Parameter --print-all detected ...') logstring='Parameter --print-all detected ...')
if 'print_colorized' in self.args and not self.args.print_colorized: if 'print_colorized' in self.args and not self.args["print_colorized"]:
logger.info('Parameter --no-color detected ...') logger.info('Parameter --no-color detected ...')
config.update({'print_colorized': False}) config.update({'print_colorized': False})
else: else:
@ -276,7 +275,7 @@ class Configuration(object):
logstring='Hyperopt continue: {}') logstring='Hyperopt continue: {}')
self._args_to_config(config, argname='hyperopt_loss', self._args_to_config(config, argname='hyperopt_loss',
logstring='Using loss function: {}') logstring='Using Hyperopt loss class name: {}')
def _process_plot_options(self, config: Dict[str, Any]) -> None: def _process_plot_options(self, config: Dict[str, Any]) -> None:
@ -324,9 +323,9 @@ class Configuration(object):
sample: logfun=len (prints the length of the found sample: logfun=len (prints the length of the found
configuration instead of the content) configuration instead of the content)
""" """
if argname in self.args and getattr(self.args, argname): if argname in self.args and self.args[argname]:
config.update({argname: getattr(self.args, argname)}) config.update({argname: self.args[argname]})
if logfun: if logfun:
logger.info(logstring.format(logfun(config[argname]))) logger.info(logstring.format(logfun(config[argname])))
else: else:
@ -346,8 +345,8 @@ class Configuration(object):
if "pairs" in config: if "pairs" in config:
return return
if "pairs_file" in self.args and self.args.pairs_file: if "pairs_file" in self.args and self.args["pairs_file"]:
pairs_file = Path(self.args.pairs_file) pairs_file = Path(self.args["pairs_file"])
logger.info(f'Reading pairs file "{pairs_file}".') logger.info(f'Reading pairs file "{pairs_file}".')
# Download pairs from the pairs file if no config is specified # Download pairs from the pairs file if no config is specified
# or if pairs file is specified explicitely # or if pairs file is specified explicitely
@ -358,7 +357,7 @@ class Configuration(object):
config['pairs'].sort() config['pairs'].sort()
return return
if "config" in self.args and self.args.config: if "config" in self.args and self.args["config"]:
logger.info("Using pairlist from configuration.") logger.info("Using pairlist from configuration.")
config['pairs'] = config.get('exchange', {}).get('pair_whitelist') config['pairs'] = config.get('exchange', {}).get('pair_whitelist')
else: else:

2
freqtrade/configuration/timerange.py
View File

@ -7,7 +7,7 @@ from typing import Optional
import arrow import arrow
class TimeRange(): class TimeRange:
""" """
object defining timerange inputs. object defining timerange inputs.
[start/stop]type defines if [start/stop]ts shall be used. [start/stop]type defines if [start/stop]ts shall be used.

2
freqtrade/constants.py
View File

@ -22,6 +22,7 @@ ORDERTYPE_POSSIBILITIES = ['limit', 'market']
ORDERTIF_POSSIBILITIES = ['gtc', 'fok', 'ioc'] ORDERTIF_POSSIBILITIES = ['gtc', 'fok', 'ioc']
AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList'] AVAILABLE_PAIRLISTS = ['StaticPairList', 'VolumePairList']
DRY_RUN_WALLET = 999.9 DRY_RUN_WALLET = 999.9
MATH_CLOSE_PREC = 1e-14 # Precision used for float comparisons
TICKER_INTERVALS = [ TICKER_INTERVALS = [
'1m', '3m', '5m', '15m', '30m', '1m', '3m', '5m', '15m', '30m',
@ -121,6 +122,7 @@ CONF_SCHEMA = {
'properties': { 'properties': {
'buy': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES}, 'buy': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'sell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES}, 'sell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'emergencysell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'stoploss': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES}, 'stoploss': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'stoploss_on_exchange': {'type': 'boolean'}, 'stoploss_on_exchange': {'type': 'boolean'},
'stoploss_on_exchange_interval': {'type': 'number'} 'stoploss_on_exchange_interval': {'type': 'number'}

2
freqtrade/data/__init__.py
View File

@ -2,7 +2,7 @@
Module to handle data operations for freqtrade Module to handle data operations for freqtrade
""" """
# limit what's imported when using `from freqtrad.data import *`` # limit what's imported when using `from freqtrade.data import *`
__all__ = [ __all__ = [
'converter' 'converter'
] ]

13
freqtrade/data/btanalysis.py
View File

@ -112,16 +112,16 @@ def load_trades_from_db(db_url: str) -> pd.DataFrame:
return trades return trades
def load_trades(config) -> pd.DataFrame: def load_trades(source: str, db_url: str, exportfilename: str) -> pd.DataFrame:
""" """
Based on configuration option "trade_source": Based on configuration option "trade_source":
* loads data from DB (using `db_url`) * loads data from DB (using `db_url`)
* loads data from backtestfile (using `exportfilename`) * loads data from backtestfile (using `exportfilename`)
""" """
if config["trade_source"] == "DB": if source == "DB":
return load_trades_from_db(config["db_url"]) return load_trades_from_db(db_url)
elif config["trade_source"] == "file": elif source == "file":
return load_backtest_data(Path(config["exportfilename"])) return load_backtest_data(Path(exportfilename))
def extract_trades_of_period(dataframe: pd.DataFrame, trades: pd.DataFrame) -> pd.DataFrame: def extract_trades_of_period(dataframe: pd.DataFrame, trades: pd.DataFrame) -> pd.DataFrame:
@ -157,7 +157,8 @@ def create_cum_profit(df: pd.DataFrame, trades: pd.DataFrame, col_name: str) ->
:param trades: DataFrame containing trades (requires columns close_time and profitperc) :param trades: DataFrame containing trades (requires columns close_time and profitperc)
:return: Returns df with one additional column, col_name, containing the cumulative profit. :return: Returns df with one additional column, col_name, containing the cumulative profit.
""" """
df[col_name] = trades.set_index('close_time')['profitperc'].cumsum() # Use groupby/sum().cumsum() to avoid errors when multiple trades sold at the same candle.
df[col_name] = trades.groupby('close_time')['profitperc'].sum().cumsum()
# Set first value to 0 # Set first value to 0
df.loc[df.iloc[0].name, col_name] = 0 df.loc[df.iloc[0].name, col_name] = 0
# FFill to get continuous # FFill to get continuous

6
freqtrade/data/dataprovider.py
View File

@ -17,7 +17,7 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class DataProvider(): class DataProvider:
def __init__(self, config: dict, exchange: Exchange) -> None: def __init__(self, config: dict, exchange: Exchange) -> None:
self._config = config self._config = config
@ -65,9 +65,7 @@ class DataProvider():
""" """
return load_pair_history(pair=pair, return load_pair_history(pair=pair,
ticker_interval=ticker_interval or self._config['ticker_interval'], ticker_interval=ticker_interval or self._config['ticker_interval'],
refresh_pairs=False, datadir=Path(self._config['datadir'])
datadir=Path(self._config['datadir']) if self._config.get(
'datadir') else None
) )
def get_pair_dataframe(self, pair: str, ticker_interval: str = None) -> DataFrame: def get_pair_dataframe(self, pair: str, ticker_interval: str = None) -> DataFrame:

74
freqtrade/data/history.py
View File

@ -57,11 +57,11 @@ def trim_tickerlist(tickerlist: List[Dict], timerange: TimeRange) -> List[Dict]:
return tickerlist[start_index:stop_index] return tickerlist[start_index:stop_index]
def load_tickerdata_file(datadir: Optional[Path], pair: str, ticker_interval: str, def load_tickerdata_file(datadir: Path, pair: str, ticker_interval: str,
timerange: Optional[TimeRange] = None) -> Optional[list]: timerange: Optional[TimeRange] = None) -> Optional[list]:
""" """
Load a pair from file, either .json.gz or .json Load a pair from file, either .json.gz or .json
:return: tickerlist or None if unsuccesful :return: tickerlist or None if unsuccessful
""" """
filename = pair_data_filename(datadir, pair, ticker_interval) filename = pair_data_filename(datadir, pair, ticker_interval)
pairdata = misc.file_load_json(filename) pairdata = misc.file_load_json(filename)
@ -73,7 +73,7 @@ def load_tickerdata_file(datadir: Optional[Path], pair: str, ticker_interval: st
return pairdata return pairdata
def store_tickerdata_file(datadir: Optional[Path], pair: str, def store_tickerdata_file(datadir: Path, pair: str,
ticker_interval: str, data: list, is_zip: bool = False): ticker_interval: str, data: list, is_zip: bool = False):
""" """
Stores tickerdata to file Stores tickerdata to file
@ -84,7 +84,7 @@ def store_tickerdata_file(datadir: Optional[Path], pair: str,
def load_pair_history(pair: str, def load_pair_history(pair: str,
ticker_interval: str, ticker_interval: str,
datadir: Optional[Path], datadir: Path,
timerange: TimeRange = TimeRange(None, None, 0, 0), timerange: TimeRange = TimeRange(None, None, 0, 0),
refresh_pairs: bool = False, refresh_pairs: bool = False,
exchange: Optional[Exchange] = None, exchange: Optional[Exchange] = None,
@ -129,37 +129,28 @@ def load_pair_history(pair: str,
else: else:
logger.warning( logger.warning(
f'No history data for pair: "{pair}", interval: {ticker_interval}. ' f'No history data for pair: "{pair}", interval: {ticker_interval}. '
'Use --refresh-pairs-cached option or `freqtrade download-data` ' 'Use `freqtrade download-data` to download the data'
'script to download the data'
) )
return None return None
def load_data(datadir: Optional[Path], def load_data(datadir: Path,
ticker_interval: str, ticker_interval: str,
pairs: List[str], pairs: List[str],
refresh_pairs: bool = False, refresh_pairs: bool = False,
exchange: Optional[Exchange] = None, exchange: Optional[Exchange] = None,
timerange: TimeRange = TimeRange(None, None, 0, 0), timerange: TimeRange = TimeRange(None, None, 0, 0),
fill_up_missing: bool = True, fill_up_missing: bool = True,
live: bool = False
) -> Dict[str, DataFrame]: ) -> Dict[str, DataFrame]:
""" """
Loads ticker history data for a list of pairs the given parameters Loads ticker history data for a list of pairs
:return: dict(<pair>:<tickerlist>) :return: dict(<pair>:<tickerlist>)
TODO: refresh_pairs is still used by edge to keep the data uptodate.
This should be replaced in the future. Instead, writing the current candles to disk
from dataprovider should be implemented, as this would avoid loading ohlcv data twice.
exchange and refresh_pairs are then not needed here nor in load_pair_history.
""" """
result: Dict[str, DataFrame] = {} result: Dict[str, DataFrame] = {}
if live:
if exchange:
logger.info('Live: Downloading data for all defined pairs ...')
exchange.refresh_latest_ohlcv([(pair, ticker_interval) for pair in pairs])
result = {key[0]: value for key, value in exchange._klines.items() if value is not None}
else:
raise OperationalException(
"Exchange needs to be initialized when using live data."
)
else:
logger.info('Using local backtesting data ...')
for pair in pairs: for pair in pairs:
hist = load_pair_history(pair=pair, ticker_interval=ticker_interval, hist = load_pair_history(pair=pair, ticker_interval=ticker_interval,
@ -172,19 +163,13 @@ def load_data(datadir: Optional[Path],
return result return result
def make_testdata_path(datadir: Optional[Path]) -> Path: def pair_data_filename(datadir: Path, pair: str, ticker_interval: str) -> Path:
"""Return the path where testdata files are stored"""
return datadir or (Path(__file__).parent.parent / "tests" / "testdata").resolve()
def pair_data_filename(datadir: Optional[Path], pair: str, ticker_interval: str) -> Path:
path = make_testdata_path(datadir)
pair_s = pair.replace("/", "_") pair_s = pair.replace("/", "_")
filename = path.joinpath(f'{pair_s}-{ticker_interval}.json') filename = datadir.joinpath(f'{pair_s}-{ticker_interval}.json')
return filename return filename
def load_cached_data_for_updating(datadir: Optional[Path], pair: str, ticker_interval: str, def load_cached_data_for_updating(datadir: Path, pair: str, ticker_interval: str,
timerange: Optional[TimeRange]) -> Tuple[List[Any], timerange: Optional[TimeRange]) -> Tuple[List[Any],
Optional[int]]: Optional[int]]:
""" """
@ -224,7 +209,7 @@ def load_cached_data_for_updating(datadir: Optional[Path], pair: str, ticker_int
return (data, since_ms) return (data, since_ms)
def download_pair_history(datadir: Optional[Path], def download_pair_history(datadir: Path,
exchange: Optional[Exchange], exchange: Optional[Exchange],
pair: str, pair: str,
ticker_interval: str = '5m', ticker_interval: str = '5m',
@ -280,6 +265,35 @@ def download_pair_history(datadir: Optional[Path],
return False return False
def refresh_backtest_ohlcv_data(exchange: Exchange, pairs: List[str], timeframes: List[str],
dl_path: Path, timerange: TimeRange,
erase=False) -> List[str]:
"""
Refresh stored ohlcv data for backtesting and hyperopt operations.
Used by freqtrade download-data
:return: Pairs not available
"""
pairs_not_available = []
for pair in pairs:
if pair not in exchange.markets:
pairs_not_available.append(pair)
logger.info(f"Skipping pair {pair}...")
continue
for ticker_interval in timeframes:
dl_file = pair_data_filename(dl_path, pair, ticker_interval)
if erase and dl_file.exists():
logger.info(
f'Deleting existing data for pair {pair}, interval {ticker_interval}.')
dl_file.unlink()
logger.info(f'Downloading pair {pair}, interval {ticker_interval}.')
download_pair_history(datadir=dl_path, exchange=exchange,
pair=pair, ticker_interval=str(ticker_interval),
timerange=timerange)
return pairs_not_available
def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]: def get_timeframe(data: Dict[str, DataFrame]) -> Tuple[arrow.Arrow, arrow.Arrow]:
""" """
Get the maximum timeframe for the given backtest data Get the maximum timeframe for the given backtest data

4
freqtrade/edge/__init__.py
View File

@ -28,7 +28,7 @@ class PairInfo(NamedTuple):
avg_trade_duration: float avg_trade_duration: float
class Edge(): class Edge:
""" """
Calculates Win Rate, Risk Reward Ratio, Expectancy Calculates Win Rate, Risk Reward Ratio, Expectancy
against historical data for a give set of markets and a strategy against historical data for a give set of markets and a strategy
@ -93,7 +93,7 @@ class Edge():
logger.info('Using local backtesting data (using whitelist in given config) ...') logger.info('Using local backtesting data (using whitelist in given config) ...')
data = history.load_data( data = history.load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None, datadir=Path(self.config['datadir']),
pairs=pairs, pairs=pairs,
ticker_interval=self.strategy.ticker_interval, ticker_interval=self.strategy.ticker_interval,
refresh_pairs=self._refresh_pairs, refresh_pairs=self._refresh_pairs,

56
freqtrade/exchange/binance.py
View File

@ -2,6 +2,10 @@
import logging import logging
from typing import Dict from typing import Dict
import ccxt
from freqtrade import (DependencyException, InvalidOrderException,
OperationalException, TemporaryError)
from freqtrade.exchange import Exchange from freqtrade.exchange import Exchange
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -25,3 +29,55 @@ class Binance(Exchange):
limit = min(list(filter(lambda x: limit <= x, limit_range))) limit = min(list(filter(lambda x: limit <= x, limit_range)))
return super().get_order_book(pair, limit) return super().get_order_book(pair, limit)
def stoploss_limit(self, pair: str, amount: float, stop_price: float, rate: float) -> Dict:
"""
creates a stoploss limit order.
this stoploss-limit is binance-specific.
It may work with a limited number of other exchanges, but this has not been tested yet.
"""
ordertype = "stop_loss_limit"
stop_price = self.symbol_price_prec(pair, stop_price)
# Ensure rate is less than stop price
if stop_price <= rate:
raise OperationalException(
'In stoploss limit order, stop price should be more than limit price')
if self._config['dry_run']:
dry_order = self.dry_run_order(
pair, ordertype, "sell", amount, stop_price)
return dry_order
try:
params = self._params.copy()
params.update({'stopPrice': stop_price})
amount = self.symbol_amount_prec(pair, amount)
rate = self.symbol_price_prec(pair, rate)
order = self._api.create_order(pair, ordertype, 'sell',
amount, rate, params)
logger.info('stoploss limit order added for %s. '
'stop price: %s. limit: %s', pair, stop_price, rate)
return order
except ccxt.InsufficientFunds as e:
raise DependencyException(
f'Insufficient funds to create {ordertype} sell order on market {pair}.'
f'Tried to sell amount {amount} at rate {rate}. '
f'Message: {e}') from e
except ccxt.InvalidOrder as e:
# Errors:
# `binance Order would trigger immediately.`
raise InvalidOrderException(
f'Could not create {ordertype} sell order on market {pair}. '
f'Tried to sell amount {amount} at rate {rate}. '
f'Message: {e}') from e
except (ccxt.NetworkError, ccxt.ExchangeError) as e:
raise TemporaryError(
f'Could not place sell order due to {e.__class__.__name__}. Message: {e}') from e
except ccxt.BaseError as e:
raise OperationalException(e) from e

55
freqtrade/exchange/exchange.py
View File

@ -14,6 +14,7 @@ from typing import Any, Dict, List, Optional, Tuple
import arrow import arrow
import ccxt import ccxt
import ccxt.async_support as ccxt_async import ccxt.async_support as ccxt_async
from ccxt.base.decimal_to_precision import ROUND_UP, ROUND_DOWN
from pandas import DataFrame from pandas import DataFrame
from freqtrade import (DependencyException, InvalidOrderException, from freqtrade import (DependencyException, InvalidOrderException,
@ -68,7 +69,7 @@ def retrier(f):
return wrapper return wrapper
class Exchange(object): class Exchange:
_config: Dict = {} _config: Dict = {}
_params: Dict = {} _params: Dict = {}
@ -320,7 +321,7 @@ class Exchange(object):
if (order_types.get("stoploss_on_exchange") if (order_types.get("stoploss_on_exchange")
and not self._ft_has.get("stoploss_on_exchange", False)): and not self._ft_has.get("stoploss_on_exchange", False)):
raise OperationalException( raise OperationalException(
'On exchange stoploss is not supported for %s.' % self.name f'On exchange stoploss is not supported for {self.name}.'
) )
def validate_order_time_in_force(self, order_time_in_force: Dict) -> None: def validate_order_time_in_force(self, order_time_in_force: Dict) -> None:
@ -366,7 +367,7 @@ class Exchange(object):
def dry_run_order(self, pair: str, ordertype: str, side: str, amount: float, def dry_run_order(self, pair: str, ordertype: str, side: str, amount: float,
rate: float, params: Dict = {}) -> Dict[str, Any]: rate: float, params: Dict = {}) -> Dict[str, Any]:
order_id = f'dry_run_{side}_{randint(0, 10**6)}' order_id = f'dry_run_{side}_{randint(0, 10**6)}'
dry_order = { # TODO: additional entry should be added for stoploss limit dry_order = {
"id": order_id, "id": order_id,
'pair': pair, 'pair': pair,
'price': rate, 'price': rate,
@ -381,6 +382,7 @@ class Exchange(object):
"info": {} "info": {}
} }
self._store_dry_order(dry_order) self._store_dry_order(dry_order)
# Copy order and close it - so the returned order is open unless it's a market order
return dry_order return dry_order
def _store_dry_order(self, dry_order: Dict) -> None: def _store_dry_order(self, dry_order: Dict) -> None:
@ -391,6 +393,8 @@ class Exchange(object):
"filled": closed_order["amount"], "filled": closed_order["amount"],
"remaining": 0 "remaining": 0
}) })
if closed_order["type"] in ["stop_loss_limit"]:
closed_order["info"].update({"stopPrice": closed_order["price"]})
self._dry_run_open_orders[closed_order["id"]] = closed_order self._dry_run_open_orders[closed_order["id"]] = closed_order
def create_order(self, pair: str, ordertype: str, side: str, amount: float, def create_order(self, pair: str, ordertype: str, side: str, amount: float,
@ -450,30 +454,14 @@ class Exchange(object):
def stoploss_limit(self, pair: str, amount: float, stop_price: float, rate: float) -> Dict: def stoploss_limit(self, pair: str, amount: float, stop_price: float, rate: float) -> Dict:
""" """
creates a stoploss limit order. creates a stoploss limit order.
NOTICE: it is not supported by all exchanges. only binance is tested for now. Since ccxt does not unify stoploss-limit orders yet, this needs to be implemented in each
TODO: implementation maybe needs to be moved to the binance subclass exchange's subclass.
The exception below should never raise, since we disallow
starting the bot in validate_ordertypes()
Note: Changes to this interface need to be applied to all sub-classes too.
""" """
ordertype = "stop_loss_limit"
stop_price = self.symbol_price_prec(pair, stop_price) raise OperationalException(f"stoploss_limit is not implemented for {self.name}.")
# Ensure rate is less than stop price
if stop_price <= rate:
raise OperationalException(
'In stoploss limit order, stop price should be more than limit price')
if self._config['dry_run']:
dry_order = self.dry_run_order(
pair, ordertype, "sell", amount, stop_price)
return dry_order
params = self._params.copy()
params.update({'stopPrice': stop_price})
order = self.create_order(pair, ordertype, 'sell', amount, rate, params)
logger.info('stoploss limit order added for %s. '
'stop price: %s. limit: %s', pair, stop_price, rate)
return order
@retrier @retrier
def get_balance(self, currency: str) -> float: def get_balance(self, currency: str) -> float:
@ -824,11 +812,9 @@ def timeframe_to_prev_date(timeframe: str, date: datetime = None) -> datetime:
""" """
if not date: if not date:
date = datetime.now(timezone.utc) date = datetime.now(timezone.utc)
timeframe_secs = timeframe_to_seconds(timeframe)
# Get offset based on timerame_secs new_timestamp = ccxt.Exchange.round_timeframe(timeframe, date.timestamp() * 1000,
offset = date.timestamp() % timeframe_secs ROUND_DOWN) // 1000
# Subtract seconds passed since last offset
new_timestamp = date.timestamp() - offset
return datetime.fromtimestamp(new_timestamp, tz=timezone.utc) return datetime.fromtimestamp(new_timestamp, tz=timezone.utc)
@ -839,9 +825,8 @@ def timeframe_to_next_date(timeframe: str, date: datetime = None) -> datetime:
:param date: date to use. Defaults to utcnow() :param date: date to use. Defaults to utcnow()
:returns: date of next candle (with utc timezone) :returns: date of next candle (with utc timezone)
""" """
prevdate = timeframe_to_prev_date(timeframe, date) if not date:
timeframe_secs = timeframe_to_seconds(timeframe) date = datetime.now(timezone.utc)
new_timestamp = ccxt.Exchange.round_timeframe(timeframe, date.timestamp() * 1000,
# Add one interval to previous candle ROUND_UP) // 1000
new_timestamp = prevdate.timestamp() + timeframe_secs
return datetime.fromtimestamp(new_timestamp, tz=timezone.utc) return datetime.fromtimestamp(new_timestamp, tz=timezone.utc)

34
freqtrade/exchange/kraken.py
View File

@ -2,7 +2,11 @@
import logging import logging
from typing import Dict from typing import Dict
import ccxt
from freqtrade import OperationalException, TemporaryError
from freqtrade.exchange import Exchange from freqtrade.exchange import Exchange
from freqtrade.exchange.exchange import retrier
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -10,3 +14,33 @@ logger = logging.getLogger(__name__)
class Kraken(Exchange): class Kraken(Exchange):
_params: Dict = {"trading_agreement": "agree"} _params: Dict = {"trading_agreement": "agree"}
@retrier
def get_balances(self) -> dict:
if self._config['dry_run']:
return {}
try:
balances = self._api.fetch_balance()
# Remove additional info from ccxt results
balances.pop("info", None)
balances.pop("free", None)
balances.pop("total", None)
balances.pop("used", None)
orders = self._api.fetch_open_orders()
order_list = [(x["symbol"].split("/")[0 if x["side"] == "sell" else 1],
x["remaining"],
# Don't remove the below comment, this can be important for debuggung
# x["side"], x["amount"],
) for x in orders]
for bal in balances:
balances[bal]['used'] = sum(order[1] for order in order_list if order[0] == bal)
balances[bal]['free'] = balances[bal]['total'] - balances[bal]['used']
return balances
except (ccxt.NetworkError, ccxt.ExchangeError) as e:
raise TemporaryError(
f'Could not get balance due to {e.__class__.__name__}. Message: {e}') from e
except ccxt.BaseError as e:
raise OperationalException(e) from e

119
freqtrade/freqtradebot.py
View File

@ -6,6 +6,7 @@ import copy
import logging import logging
import traceback import traceback
from datetime import datetime from datetime import datetime
from math import isclose
from typing import Any, Dict, List, Optional, Tuple from typing import Any, Dict, List, Optional, Tuple
import arrow import arrow
@ -21,7 +22,7 @@ from freqtrade.exchange import timeframe_to_minutes, timeframe_to_next_date
from freqtrade.persistence import Trade from freqtrade.persistence import Trade
from freqtrade.rpc import RPCManager, RPCMessageType from freqtrade.rpc import RPCManager, RPCMessageType
from freqtrade.resolvers import ExchangeResolver, StrategyResolver, PairListResolver from freqtrade.resolvers import ExchangeResolver, StrategyResolver, PairListResolver
from freqtrade.state import State, RunMode from freqtrade.state import State
from freqtrade.strategy.interface import SellType, IStrategy from freqtrade.strategy.interface import SellType, IStrategy
from freqtrade.wallets import Wallets from freqtrade.wallets import Wallets
@ -29,7 +30,7 @@ from freqtrade.wallets import Wallets
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class FreqtradeBot(object): class FreqtradeBot:
""" """
Freqtrade is the main class of the bot. Freqtrade is the main class of the bot.
This is from here the bot start its logic. This is from here the bot start its logic.
@ -79,12 +80,6 @@ class FreqtradeBot(object):
persistence.init(self.config.get('db_url', None), persistence.init(self.config.get('db_url', None),
clean_open_orders=self.config.get('dry_run', False)) clean_open_orders=self.config.get('dry_run', False))
# Stoploss on exchange does not make sense, therefore we need to disable that.
if (self.dataprovider.runmode == RunMode.DRY_RUN and
self.strategy.order_types.get('stoploss_on_exchange', False)):
logger.info("Disabling stoploss_on_exchange during dry-run.")
self.strategy.order_types['stoploss_on_exchange'] = False
config['order_types']['stoploss_on_exchange'] = False
# Set initial bot state from config # Set initial bot state from config
initial_state = self.config.get('initial_state') initial_state = self.config.get('initial_state')
self.state = State[initial_state.upper()] if initial_state else State.STOPPED self.state = State[initial_state.upper()] if initial_state else State.STOPPED
@ -216,7 +211,7 @@ class FreqtradeBot(object):
if stake_amount == constants.UNLIMITED_STAKE_AMOUNT: if stake_amount == constants.UNLIMITED_STAKE_AMOUNT:
open_trades = len(Trade.get_open_trades()) open_trades = len(Trade.get_open_trades())
if open_trades >= self.config['max_open_trades']: if open_trades >= self.config['max_open_trades']:
logger.warning('Can\'t open a new trade: max number of trades is reached') logger.warning("Can't open a new trade: max number of trades is reached")
return None return None
return available_amount / (self.config['max_open_trades'] - open_trades) return available_amount / (self.config['max_open_trades'] - open_trades)
@ -351,8 +346,8 @@ class FreqtradeBot(object):
min_stake_amount = self._get_min_pair_stake_amount(pair_s, buy_limit_requested) min_stake_amount = self._get_min_pair_stake_amount(pair_s, buy_limit_requested)
if min_stake_amount is not None and min_stake_amount > stake_amount: if min_stake_amount is not None and min_stake_amount > stake_amount:
logger.warning( logger.warning(
f'Can\'t open a new trade for {pair_s}: stake amount ' f"Can't open a new trade for {pair_s}: stake amount "
f'is too small ({stake_amount} < {min_stake_amount})' f"is too small ({stake_amount} < {min_stake_amount})"
) )
return False return False
@ -516,7 +511,7 @@ class FreqtradeBot(object):
trade.pair.startswith(exectrade['fee']['currency'])): trade.pair.startswith(exectrade['fee']['currency'])):
fee_abs += exectrade['fee']['cost'] fee_abs += exectrade['fee']['cost']
if amount != order_amount: if not isclose(amount, order_amount, abs_tol=constants.MATH_CLOSE_PREC):
logger.warning(f"Amount {amount} does not match amount {trade.amount}") logger.warning(f"Amount {amount} does not match amount {trade.amount}")
raise OperationalException("Half bought? Amounts don't match") raise OperationalException("Half bought? Amounts don't match")
real_amount = amount - fee_abs real_amount = amount - fee_abs
@ -541,7 +536,7 @@ class FreqtradeBot(object):
# Try update amount (binance-fix) # Try update amount (binance-fix)
try: try:
new_amount = self.get_real_amount(trade, order) new_amount = self.get_real_amount(trade, order)
if order['amount'] != new_amount: if not isclose(order['amount'], new_amount, abs_tol=constants.MATH_CLOSE_PREC):
order['amount'] = new_amount order['amount'] = new_amount
# Fee was applied, so set to 0 # Fee was applied, so set to 0
trade.fee_open = 0 trade.fee_open = 0
@ -617,6 +612,33 @@ class FreqtradeBot(object):
logger.debug('Found no sell signal for %s.', trade) logger.debug('Found no sell signal for %s.', trade)
return False return False
def create_stoploss_order(self, trade: Trade, stop_price: float, rate: float) -> bool:
"""
Abstracts creating stoploss orders from the logic.
Handles errors and updates the trade database object.
Force-sells the pair (using EmergencySell reason) in case of Problems creating the order.
:return: True if the order succeeded, and False in case of problems.
"""
# Limit price threshold: As limit price should always be below price
LIMIT_PRICE_PCT = 0.99
try:
stoploss_order = self.exchange.stoploss_limit(pair=trade.pair, amount=trade.amount,
stop_price=stop_price,
rate=rate * LIMIT_PRICE_PCT)
trade.stoploss_order_id = str(stoploss_order['id'])
return True
except InvalidOrderException as e:
trade.stoploss_order_id = None
logger.error(f'Unable to place a stoploss order on exchange. {e}')
logger.warning('Selling the trade forcefully')
self.execute_sell(trade, trade.stop_loss, sell_reason=SellType.EMERGENCY_SELL)
except DependencyException:
trade.stoploss_order_id = None
logger.exception('Unable to place a stoploss order on exchange.')
return False
def handle_stoploss_on_exchange(self, trade: Trade) -> bool: def handle_stoploss_on_exchange(self, trade: Trade) -> bool:
""" """
Check if trade is fulfilled in which case the stoploss Check if trade is fulfilled in which case the stoploss
@ -635,49 +657,25 @@ class FreqtradeBot(object):
except InvalidOrderException as exception: except InvalidOrderException as exception:
logger.warning('Unable to fetch stoploss order: %s', exception) logger.warning('Unable to fetch stoploss order: %s', exception)
# If trade open order id does not exist: buy order is fulfilled
buy_order_fulfilled = not trade.open_order_id
# Limit price threshold: As limit price should always be below price
limit_price_pct = 0.99
# If buy order is fulfilled but there is no stoploss, we add a stoploss on exchange # If buy order is fulfilled but there is no stoploss, we add a stoploss on exchange
if (buy_order_fulfilled and not stoploss_order): if (not trade.open_order_id and not stoploss_order):
if self.edge:
stoploss = self.edge.stoploss(pair=trade.pair) stoploss = self.edge.stoploss(pair=trade.pair) if self.edge else self.strategy.stoploss
else:
stoploss = self.strategy.stoploss
stop_price = trade.open_rate * (1 + stoploss) stop_price = trade.open_rate * (1 + stoploss)
# limit price should be less than stop price. if self.create_stoploss_order(trade=trade, stop_price=stop_price, rate=stop_price):
limit_price = stop_price * limit_price_pct
try:
stoploss_order_id = self.exchange.stoploss_limit(
pair=trade.pair, amount=trade.amount, stop_price=stop_price, rate=limit_price
)['id']
trade.stoploss_order_id = str(stoploss_order_id)
trade.stoploss_last_update = datetime.now() trade.stoploss_last_update = datetime.now()
return False return False
except DependencyException as exception:
trade.stoploss_order_id = None
logger.warning('Unable to place a stoploss order on exchange: %s', exception)
# If stoploss order is canceled for some reason we add it # If stoploss order is canceled for some reason we add it
if stoploss_order and stoploss_order['status'] == 'canceled': if stoploss_order and stoploss_order['status'] == 'canceled':
try: if self.create_stoploss_order(trade=trade, stop_price=trade.stop_loss,
stoploss_order_id = self.exchange.stoploss_limit( rate=trade.stop_loss):
pair=trade.pair, amount=trade.amount,
stop_price=trade.stop_loss, rate=trade.stop_loss * limit_price_pct
)['id']
trade.stoploss_order_id = str(stoploss_order_id)
return False return False
except DependencyException as exception: else:
trade.stoploss_order_id = None trade.stoploss_order_id = None
logger.warning('Stoploss order was cancelled, ' logger.warning('Stoploss order was cancelled, but unable to recreate one.')
'but unable to recreate one: %s', exception)
# We check if stoploss order is fulfilled # We check if stoploss order is fulfilled
if stoploss_order and stoploss_order['status'] == 'closed': if stoploss_order and stoploss_order['status'] == 'closed':
@ -686,7 +684,7 @@ class FreqtradeBot(object):
# Lock pair for one candle to prevent immediate rebuys # Lock pair for one candle to prevent immediate rebuys
self.strategy.lock_pair(trade.pair, self.strategy.lock_pair(trade.pair,
timeframe_to_next_date(self.config['ticker_interval'])) timeframe_to_next_date(self.config['ticker_interval']))
self._notify_sell(trade) self._notify_sell(trade, "stoploss")
return True return True
# Finally we check if stoploss on exchange should be moved up because of trailing. # Finally we check if stoploss on exchange should be moved up because of trailing.
@ -720,16 +718,12 @@ class FreqtradeBot(object):
logger.exception(f"Could not cancel stoploss order {order['id']} " logger.exception(f"Could not cancel stoploss order {order['id']} "
f"for pair {trade.pair}") f"for pair {trade.pair}")
try: # Create new stoploss order
# creating the new one if self.create_stoploss_order(trade=trade, stop_price=trade.stop_loss,
stoploss_order_id = self.exchange.stoploss_limit( rate=trade.stop_loss):
pair=trade.pair, amount=trade.amount, return False
stop_price=trade.stop_loss, rate=trade.stop_loss * 0.99 else:
)['id'] logger.warning(f"Could not create trailing stoploss order "
trade.stoploss_order_id = str(stoploss_order_id)
except DependencyException:
trade.stoploss_order_id = None
logger.exception(f"Could not create trailing stoploss order "
f"for pair {trade.pair}.") f"for pair {trade.pair}.")
def _check_and_execute_sell(self, trade: Trade, sell_rate: float, def _check_and_execute_sell(self, trade: Trade, sell_rate: float,
@ -883,9 +877,14 @@ class FreqtradeBot(object):
except InvalidOrderException: except InvalidOrderException:
logger.exception(f"Could not cancel stoploss order {trade.stoploss_order_id}") logger.exception(f"Could not cancel stoploss order {trade.stoploss_order_id}")
ordertype = self.strategy.order_types[sell_type]
if sell_reason == SellType.EMERGENCY_SELL:
# Emergencysells (default to market!)
ordertype = self.strategy.order_types.get("emergencysell", "market")
# Execute sell and update trade record # Execute sell and update trade record
order = self.exchange.sell(pair=str(trade.pair), order = self.exchange.sell(pair=str(trade.pair),
ordertype=self.strategy.order_types[sell_type], ordertype=ordertype,
amount=trade.amount, rate=limit, amount=trade.amount, rate=limit,
time_in_force=self.strategy.order_time_in_force['sell'] time_in_force=self.strategy.order_time_in_force['sell']
) )
@ -901,9 +900,9 @@ class FreqtradeBot(object):
# Lock pair for one candle to prevent immediate rebuys # Lock pair for one candle to prevent immediate rebuys
self.strategy.lock_pair(trade.pair, timeframe_to_next_date(self.config['ticker_interval'])) self.strategy.lock_pair(trade.pair, timeframe_to_next_date(self.config['ticker_interval']))
self._notify_sell(trade) self._notify_sell(trade, ordertype)
def _notify_sell(self, trade: Trade): def _notify_sell(self, trade: Trade, order_type: str):
""" """
Sends rpc notification when a sell occured. Sends rpc notification when a sell occured.
""" """
@ -920,7 +919,7 @@ class FreqtradeBot(object):
'pair': trade.pair, 'pair': trade.pair,
'gain': gain, 'gain': gain,
'limit': trade.close_rate_requested, 'limit': trade.close_rate_requested,
'order_type': self.strategy.order_types['sell'], 'order_type': order_type,
'amount': trade.amount, 'amount': trade.amount,
'open_rate': trade.open_rate, 'open_rate': trade.open_rate,
'current_rate': current_rate, 'current_rate': current_rate,

12
freqtrade/main.py
View File

@ -11,7 +11,6 @@ if sys.version_info < (3, 6):
# flake8: noqa E402 # flake8: noqa E402
import logging import logging
from argparse import Namespace
from typing import Any, List from typing import Any, List
from freqtrade import OperationalException from freqtrade import OperationalException
@ -31,16 +30,13 @@ def main(sysargv: List[str] = None) -> None:
return_code: Any = 1 return_code: Any = 1
worker = None worker = None
try: try:
arguments = Arguments( arguments = Arguments(sysargv)
sysargv, args = arguments.get_parsed_arg()
'Free, open source crypto trading bot'
)
args: Namespace = arguments.get_parsed_arg()
# A subcommand has been issued. # A subcommand has been issued.
# Means if Backtesting or Hyperopt have been called we exit the bot # Means if Backtesting or Hyperopt have been called we exit the bot
if hasattr(args, 'func'): if 'func' in args:
args.func(args) args['func'](args)
# TODO: fetch return_code as returned by the command function here # TODO: fetch return_code as returned by the command function here
return_code = 0 return_code = 0
else: else:

7
freqtrade/misc.py
View File

@ -114,3 +114,10 @@ def deep_merge_dicts(source, destination):
destination[key] = value destination[key] = value
return destination return destination
def round_dict(d, n):
"""
Rounds float values in the dict to n digits after the decimal point.
"""
return {k: (round(v, n) if isinstance(v, float) else v) for k, v in d.items()}

29
freqtrade/optimize/__init__.py
View File

@ -1,10 +1,7 @@
import logging import logging
from argparse import Namespace
from typing import Any, Dict from typing import Any, Dict
from filelock import FileLock, Timeout from freqtrade import DependencyException, constants, OperationalException
from freqtrade import DependencyException, constants
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.utils import setup_utils_configuration from freqtrade.utils import setup_utils_configuration
@ -12,7 +9,7 @@ from freqtrade.utils import setup_utils_configuration
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]: def setup_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
""" """
Prepare the configuration for the Hyperopt module Prepare the configuration for the Hyperopt module
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
@ -25,20 +22,10 @@ def setup_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]:
raise DependencyException('stake amount could not be "%s" for backtesting' % raise DependencyException('stake amount could not be "%s" for backtesting' %
constants.UNLIMITED_STAKE_AMOUNT) constants.UNLIMITED_STAKE_AMOUNT)
if method == RunMode.HYPEROPT:
# Special cases for Hyperopt
if config.get('strategy') and config.get('strategy') != 'DefaultStrategy':
logger.error("Please don't use --strategy for hyperopt.")
logger.error(
"Read the documentation at "
"https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md "
"to understand how to configure hyperopt.")
raise DependencyException("--strategy configured but not supported for hyperopt")
return config return config
def start_backtesting(args: Namespace) -> None: def start_backtesting(args: Dict[str, Any]) -> None:
""" """
Start Backtesting script Start Backtesting script
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
@ -57,15 +44,19 @@ def start_backtesting(args: Namespace) -> None:
backtesting.start() backtesting.start()
def start_hyperopt(args: Namespace) -> None: def start_hyperopt(args: Dict[str, Any]) -> None:
""" """
Start hyperopt script Start hyperopt script
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
:return: None :return: None
""" """
# Import here to avoid loading hyperopt module when it's not used # Import here to avoid loading hyperopt module when it's not used
try:
from filelock import FileLock, Timeout
from freqtrade.optimize.hyperopt import Hyperopt from freqtrade.optimize.hyperopt import Hyperopt
except ImportError as e:
raise OperationalException(
f"{e}. Please ensure that the hyperopt dependencies are installed.") from e
# Initialize configuration # Initialize configuration
config = setup_configuration(args, RunMode.HYPEROPT) config = setup_configuration(args, RunMode.HYPEROPT)
@ -95,7 +86,7 @@ def start_hyperopt(args: Namespace) -> None:
# Same in Edge and Backtesting start() functions. # Same in Edge and Backtesting start() functions.
def start_edge(args: Namespace) -> None: def start_edge(args: Dict[str, Any]) -> None:
""" """
Start Edge script Start Edge script
:param args: Cli args from Arguments() :param args: Cli args from Arguments()

42
freqtrade/optimize/backtesting.py
View File

@ -44,7 +44,7 @@ class BacktestResult(NamedTuple):
sell_reason: SellType sell_reason: SellType
class Backtesting(object): class Backtesting:
""" """
Backtesting class, this class contains all the logic to run a backtest Backtesting class, this class contains all the logic to run a backtest
@ -81,6 +81,12 @@ class Backtesting(object):
# No strategy list specified, only one strategy # No strategy list specified, only one strategy
self.strategylist.append(StrategyResolver(self.config).strategy) self.strategylist.append(StrategyResolver(self.config).strategy)
if "ticker_interval" not in self.config:
raise OperationalException("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`")
self.ticker_interval = str(self.config.get('ticker_interval'))
self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
# Load one (first) strategy # Load one (first) strategy
self._set_strategy(self.strategylist[0]) self._set_strategy(self.strategylist[0])
@ -89,14 +95,6 @@ class Backtesting(object):
Load strategy into backtesting Load strategy into backtesting
""" """
self.strategy = strategy self.strategy = strategy
if "ticker_interval" not in self.config:
raise OperationalException("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`")
self.ticker_interval = self.config.get('ticker_interval')
self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
self.advise_buy = strategy.advise_buy
self.advise_sell = strategy.advise_sell
# Set stoploss_on_exchange to false for backtesting, # Set stoploss_on_exchange to false for backtesting,
# since a "perfect" stoploss-sell is assumed anyway # since a "perfect" stoploss-sell is assumed anyway
# And the regular "stoploss" function would not apply to that case # And the regular "stoploss" function would not apply to that case
@ -219,8 +217,8 @@ class Backtesting(object):
for pair, pair_data in processed.items(): for pair, pair_data in processed.items():
pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run pair_data['buy'], pair_data['sell'] = 0, 0 # cleanup from previous run
ticker_data = self.advise_sell( ticker_data = self.strategy.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy() self.strategy.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
# to avoid using data from future, we buy/sell with signal from previous candle # to avoid using data from future, we buy/sell with signal from previous candle
ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1) ticker_data.loc[:, 'buy'] = ticker_data['buy'].shift(1)
@ -239,14 +237,16 @@ class Backtesting(object):
stake_amount: float, max_open_trades: int) -> Optional[BacktestResult]: stake_amount: float, max_open_trades: int) -> Optional[BacktestResult]:
trade = Trade( trade = Trade(
pair=pair,
open_rate=buy_row.open, open_rate=buy_row.open,
open_date=buy_row.date, open_date=buy_row.date,
stake_amount=stake_amount, stake_amount=stake_amount,
amount=stake_amount / buy_row.open, amount=stake_amount / buy_row.open,
fee_open=self.fee, fee_open=self.fee,
fee_close=self.fee fee_close=self.fee,
is_open=True,
) )
logger.debug(f"{pair} - Backtesting emulates creation of new trade: {trade}.")
# calculate win/lose forwards from buy point # calculate win/lose forwards from buy point
for sell_row in partial_ticker: for sell_row in partial_ticker:
if max_open_trades > 0: if max_open_trades > 0:
@ -289,7 +289,7 @@ class Backtesting(object):
if partial_ticker: if partial_ticker:
# no sell condition found - trade stil open at end of backtest period # no sell condition found - trade stil open at end of backtest period
sell_row = partial_ticker[-1] sell_row = partial_ticker[-1]
btr = BacktestResult(pair=pair, bt_res = BacktestResult(pair=pair,
profit_percent=trade.calc_profit_percent(rate=sell_row.open), profit_percent=trade.calc_profit_percent(rate=sell_row.open),
profit_abs=trade.calc_profit(rate=sell_row.open), profit_abs=trade.calc_profit(rate=sell_row.open),
open_time=buy_row.date, open_time=buy_row.date,
@ -303,9 +303,11 @@ class Backtesting(object):
close_rate=sell_row.open, close_rate=sell_row.open,
sell_reason=SellType.FORCE_SELL sell_reason=SellType.FORCE_SELL
) )
logger.debug('Force_selling still open trade %s with %s perc - %s', btr.pair, logger.debug(f"{pair} - Force selling still open trade, "
btr.profit_percent, btr.profit_abs) f"profit percent: {bt_res.profit_percent}, "
return btr f"profit abs: {bt_res.profit_abs}")
return bt_res
return None return None
def backtest(self, args: Dict) -> DataFrame: def backtest(self, args: Dict) -> DataFrame:
@ -384,6 +386,8 @@ class Backtesting(object):
max_open_trades) max_open_trades)
if trade_entry: if trade_entry:
logger.debug(f"{pair} - Locking pair till "
f"close_time={trade_entry.close_time}")
lock_pair_until[pair] = trade_entry.close_time lock_pair_until[pair] = trade_entry.close_time
trades.append(trade_entry) trades.append(trade_entry)
else: else:
@ -407,11 +411,9 @@ class Backtesting(object):
timerange = TimeRange.parse_timerange(None if self.config.get( timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange'))) 'timerange') is None else str(self.config.get('timerange')))
data = history.load_data( data = history.load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None, datadir=Path(self.config['datadir']),
pairs=pairs, pairs=pairs,
ticker_interval=self.ticker_interval, ticker_interval=self.ticker_interval,
refresh_pairs=self.config.get('refresh_pairs', False),
exchange=self.exchange,
timerange=timerange, timerange=timerange,
) )

5
freqtrade/optimize/edge_cli.py
View File

@ -16,7 +16,7 @@ from freqtrade.resolvers import StrategyResolver
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class EdgeCli(object): class EdgeCli:
""" """
EdgeCli class, this class contains all the logic to run edge backtesting EdgeCli class, this class contains all the logic to run edge backtesting
@ -39,7 +39,8 @@ class EdgeCli(object):
self.strategy = StrategyResolver(self.config).strategy self.strategy = StrategyResolver(self.config).strategy
self.edge = Edge(config, self.exchange, self.strategy) self.edge = Edge(config, self.exchange, self.strategy)
self.edge._refresh_pairs = self.config.get('refresh_pairs', False) # Set refresh_pairs to false for edge-cli (it must be true for edge)
self.edge._refresh_pairs = False
self.timerange = TimeRange.parse_timerange(None if self.config.get( self.timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange'))) 'timerange') is None else str(self.config.get('timerange')))

94
freqtrade/optimize/hyperopt.py
View File

@ -24,8 +24,10 @@ from skopt.space import Dimension
from freqtrade.configuration import TimeRange from freqtrade.configuration import TimeRange
from freqtrade.data.history import load_data, get_timeframe from freqtrade.data.history import load_data, get_timeframe
from freqtrade.misc import round_dict
from freqtrade.optimize.backtesting import Backtesting from freqtrade.optimize.backtesting import Backtesting
# Import IHyperOptLoss to allow users import from this file # Import IHyperOpt and IHyperOptLoss to allow unpickling classes from these modules
from freqtrade.optimize.hyperopt_interface import IHyperOpt # noqa: F4
from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F4 from freqtrade.optimize.hyperopt_loss_interface import IHyperOptLoss # noqa: F4
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver
@ -34,6 +36,11 @@ logger = logging.getLogger(__name__)
INITIAL_POINTS = 30 INITIAL_POINTS = 30
# Keep no more than 2*SKOPT_MODELS_MAX_NUM models
# in the skopt models list
SKOPT_MODELS_MAX_NUM = 10
MAX_LOSS = 100000 # just a big enough number to be bad result in loss optimization MAX_LOSS = 100000 # just a big enough number to be bad result in loss optimization
@ -47,10 +54,11 @@ class Hyperopt:
""" """
def __init__(self, config: Dict[str, Any]) -> None: def __init__(self, config: Dict[str, Any]) -> None:
self.config = config self.config = config
self.backtesting = Backtesting(self.config)
self.custom_hyperopt = HyperOptResolver(self.config).hyperopt self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
self.backtesting = Backtesting(self.config)
self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss self.custom_hyperoptloss = HyperOptLossResolver(self.config).hyperoptloss
self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function self.calculate_loss = self.custom_hyperoptloss.hyperopt_loss_function
@ -71,11 +79,15 @@ class Hyperopt:
self.trials: List = [] self.trials: List = []
# Populate functions here (hasattr is slow so should not be run during "regular" operations) # Populate functions here (hasattr is slow so should not be run during "regular" operations)
if hasattr(self.custom_hyperopt, 'populate_indicators'):
self.backtesting.strategy.advise_indicators = \
self.custom_hyperopt.populate_indicators # type: ignore
if hasattr(self.custom_hyperopt, 'populate_buy_trend'): if hasattr(self.custom_hyperopt, 'populate_buy_trend'):
self.backtesting.advise_buy = self.custom_hyperopt.populate_buy_trend # type: ignore self.backtesting.strategy.advise_buy = \
self.custom_hyperopt.populate_buy_trend # type: ignore
if hasattr(self.custom_hyperopt, 'populate_sell_trend'): if hasattr(self.custom_hyperopt, 'populate_sell_trend'):
self.backtesting.advise_sell = self.custom_hyperopt.populate_sell_trend # type: ignore self.backtesting.strategy.advise_sell = \
self.custom_hyperopt.populate_sell_trend # type: ignore
# Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set # Use max_open_trades for hyperopt as well, except --disable-max-market-positions is set
if self.config.get('use_max_market_positions', True): if self.config.get('use_max_market_positions', True):
@ -83,7 +95,7 @@ class Hyperopt:
else: else:
logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...') logger.debug('Ignoring max_open_trades (--disable-max-market-positions was used) ...')
self.max_open_trades = 0 self.max_open_trades = 0
self.position_stacking = self.config.get('position_stacking', False), self.position_stacking = self.config.get('position_stacking', False)
if self.has_space('sell'): if self.has_space('sell'):
# Make sure experimental is enabled # Make sure experimental is enabled
@ -107,7 +119,9 @@ class Hyperopt:
p.unlink() p.unlink()
def get_args(self, params): def get_args(self, params):
dimensions = self.hyperopt_space()
dimensions = self.dimensions
# Ensure the number of dimensions match # Ensure the number of dimensions match
# the number of parameters in the list x. # the number of parameters in the list x.
if len(params) != len(dimensions): if len(params) != len(dimensions):
@ -124,14 +138,14 @@ class Hyperopt:
Save hyperopt trials to file Save hyperopt trials to file
""" """
if self.trials: if self.trials:
logger.info('Saving %d evaluations to \'%s\'', len(self.trials), self.trials_file) logger.info("Saving %d evaluations to '%s'", len(self.trials), self.trials_file)
dump(self.trials, self.trials_file) dump(self.trials, self.trials_file)
def read_trials(self) -> List: def read_trials(self) -> List:
""" """
Read hyperopt trials file Read hyperopt trials file
""" """
logger.info('Reading Trials from \'%s\'', self.trials_file) logger.info("Reading Trials from '%s'", self.trials_file)
trials = load(self.trials_file) trials = load(self.trials_file)
self.trials_file.unlink() self.trials_file.unlink()
return trials return trials
@ -178,9 +192,11 @@ class Hyperopt:
indent=4) indent=4)
if self.has_space('roi'): if self.has_space('roi'):
print("ROI table:") print("ROI table:")
pprint(self.custom_hyperopt.generate_roi_table(params), indent=4) # Round printed values to 5 digits after the decimal point
pprint(round_dict(self.custom_hyperopt.generate_roi_table(params), 5), indent=4)
if self.has_space('stoploss'): if self.has_space('stoploss'):
print(f"Stoploss: {params.get('stoploss')}") # Also round to 5 digits after the decimal point
print(f"Stoploss: {round(params.get('stoploss'), 5)}")
def log_results(self, results) -> None: def log_results(self, results) -> None:
""" """
@ -244,20 +260,24 @@ class Hyperopt:
spaces += self.custom_hyperopt.stoploss_space() spaces += self.custom_hyperopt.stoploss_space()
return spaces return spaces
def generate_optimizer(self, _params: Dict) -> Dict: def generate_optimizer(self, _params: Dict, iteration=None) -> Dict:
""" """
Used Optimize function. Called once per epoch to optimize whatever is configured. Used Optimize function. Called once per epoch to optimize whatever is configured.
Keep this function as optimized as possible! Keep this function as optimized as possible!
""" """
params = self.get_args(_params) params = self.get_args(_params)
if self.has_space('roi'): if self.has_space('roi'):
self.backtesting.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params) self.backtesting.strategy.minimal_roi = \
self.custom_hyperopt.generate_roi_table(params)
if self.has_space('buy'): if self.has_space('buy'):
self.backtesting.advise_buy = self.custom_hyperopt.buy_strategy_generator(params) self.backtesting.strategy.advise_buy = \
self.custom_hyperopt.buy_strategy_generator(params)
if self.has_space('sell'): if self.has_space('sell'):
self.backtesting.advise_sell = self.custom_hyperopt.sell_strategy_generator(params) self.backtesting.strategy.advise_sell = \
self.custom_hyperopt.sell_strategy_generator(params)
if self.has_space('stoploss'): if self.has_space('stoploss'):
self.backtesting.strategy.stoploss = params['stoploss'] self.backtesting.strategy.stoploss = params['stoploss']
@ -318,9 +338,9 @@ class Hyperopt:
f'Total profit {total_profit: 11.8f} {stake_cur} ' f'Total profit {total_profit: 11.8f} {stake_cur} '
f'({profit: 7.2f}Σ%). Avg duration {duration:5.1f} mins.') f'({profit: 7.2f}Σ%). Avg duration {duration:5.1f} mins.')
def get_optimizer(self, cpu_count) -> Optimizer: def get_optimizer(self, dimensions, cpu_count) -> Optimizer:
return Optimizer( return Optimizer(
self.hyperopt_space(), dimensions,
base_estimator="ET", base_estimator="ET",
acq_optimizer="auto", acq_optimizer="auto",
n_initial_points=INITIAL_POINTS, n_initial_points=INITIAL_POINTS,
@ -328,9 +348,26 @@ class Hyperopt:
random_state=self.config.get('hyperopt_random_state', None) random_state=self.config.get('hyperopt_random_state', None)
) )
def run_optimizer_parallel(self, parallel, asked) -> List: def fix_optimizer_models_list(self):
"""
WORKAROUND: Since skopt is not actively supported, this resolves problems with skopt
memory usage, see also: https://github.com/scikit-optimize/scikit-optimize/pull/746
This may cease working when skopt updates if implementation of this intrinsic
part changes.
"""
n = len(self.opt.models) - SKOPT_MODELS_MAX_NUM
# Keep no more than 2*SKOPT_MODELS_MAX_NUM models in the skopt models list,
# remove the old ones. These are actually of no use, the current model
# from the estimator is the only one used in the skopt optimizer.
# Freqtrade code also does not inspect details of the models.
if n >= SKOPT_MODELS_MAX_NUM:
logger.debug(f"Fixing skopt models list, removing {n} old items...")
del self.opt.models[0:n]
def run_optimizer_parallel(self, parallel, asked, i) -> List:
return parallel(delayed( return parallel(delayed(
wrap_non_picklable_objects(self.generate_optimizer))(v) for v in asked) wrap_non_picklable_objects(self.generate_optimizer))(v, i) for v in asked)
def load_previous_results(self): def load_previous_results(self):
""" read trials file if we have one """ """ read trials file if we have one """
@ -345,11 +382,9 @@ class Hyperopt:
timerange = TimeRange.parse_timerange(None if self.config.get( timerange = TimeRange.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange'))) 'timerange') is None else str(self.config.get('timerange')))
data = load_data( data = load_data(
datadir=Path(self.config['datadir']) if self.config.get('datadir') else None, datadir=Path(self.config['datadir']),
pairs=self.config['exchange']['pair_whitelist'], pairs=self.config['exchange']['pair_whitelist'],
ticker_interval=self.backtesting.ticker_interval, ticker_interval=self.backtesting.ticker_interval,
refresh_pairs=self.config.get('refresh_pairs', False),
exchange=self.backtesting.exchange,
timerange=timerange timerange=timerange
) )
@ -366,9 +401,6 @@ class Hyperopt:
(max_date - min_date).days (max_date - min_date).days
) )
self.backtesting.strategy.advise_indicators = \
self.custom_hyperopt.populate_indicators # type: ignore
preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data) preprocessed = self.backtesting.strategy.tickerdata_to_dataframe(data)
dump(preprocessed, self.tickerdata_pickle) dump(preprocessed, self.tickerdata_pickle)
@ -379,11 +411,12 @@ class Hyperopt:
self.load_previous_results() self.load_previous_results()
cpus = cpu_count() cpus = cpu_count()
logger.info(f'Found {cpus} CPU cores. Let\'s make them scream!') logger.info(f"Found {cpus} CPU cores. Let's make them scream!")
config_jobs = self.config.get('hyperopt_jobs', -1) config_jobs = self.config.get('hyperopt_jobs', -1)
logger.info(f'Number of parallel jobs set as: {config_jobs}') logger.info(f'Number of parallel jobs set as: {config_jobs}')
opt = self.get_optimizer(config_jobs) self.dimensions = self.hyperopt_space()
self.opt = self.get_optimizer(self.dimensions, config_jobs)
if self.config.get('print_colorized', False): if self.config.get('print_colorized', False):
colorama_init(autoreset=True) colorama_init(autoreset=True)
@ -394,9 +427,10 @@ class Hyperopt:
logger.info(f'Effective number of parallel workers used: {jobs}') logger.info(f'Effective number of parallel workers used: {jobs}')
EVALS = max(self.total_epochs // jobs, 1) EVALS = max(self.total_epochs // jobs, 1)
for i in range(EVALS): for i in range(EVALS):
asked = opt.ask(n_points=jobs) asked = self.opt.ask(n_points=jobs)
f_val = self.run_optimizer_parallel(parallel, asked) f_val = self.run_optimizer_parallel(parallel, asked, i)
opt.tell(asked, [v['loss'] for v in f_val]) self.opt.tell(asked, [v['loss'] for v in f_val])
self.fix_optimizer_models_list()
for j in range(jobs): for j in range(jobs):
current = i * jobs + j current = i * jobs + j
val = f_val[j] val = f_val[j]

140
freqtrade/optimize/hyperopt_interface.py
View File

@ -2,6 +2,8 @@
IHyperOpt interface IHyperOpt interface
This module defines the interface to apply for hyperopts This module defines the interface to apply for hyperopts
""" """
import logging
import math
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import Dict, Any, Callable, List from typing import Dict, Any, Callable, List
@ -9,19 +11,37 @@ from typing import Dict, Any, Callable, List
from pandas import DataFrame from pandas import DataFrame
from skopt.space import Dimension, Integer, Real from skopt.space import Dimension, Integer, Real
from freqtrade import OperationalException
from freqtrade.exchange import timeframe_to_minutes
from freqtrade.misc import round_dict
logger = logging.getLogger(__name__)
def _format_exception_message(method: str, space: str) -> str:
return (f"The '{space}' space is included into the hyperoptimization "
f"but {method}() method is not found in your "
f"custom Hyperopt class. You should either implement this "
f"method or remove the '{space}' space from hyperoptimization.")
class IHyperOpt(ABC): class IHyperOpt(ABC):
""" """
Interface for freqtrade hyperopts Interface for freqtrade hyperopts
Defines the mandatory structure must follow any custom strategies Defines the mandatory structure must follow any custom hyperopts
Attributes you can use: Class attributes you can use:
minimal_roi -> Dict: Minimal ROI designed for the strategy
stoploss -> float: optimal stoploss designed for the strategy
ticker_interval -> int: value of the ticker interval to use for the strategy ticker_interval -> int: value of the ticker interval to use for the strategy
""" """
ticker_interval: str ticker_interval: str
def __init__(self, config: dict) -> None:
self.config = config
# Assign ticker_interval to be used in hyperopt
IHyperOpt.ticker_interval = str(config['ticker_interval'])
@staticmethod @staticmethod
@abstractmethod @abstractmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
@ -32,32 +52,32 @@ class IHyperOpt(ABC):
""" """
@staticmethod @staticmethod
@abstractmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable: def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a buy strategy generator. Create a buy strategy generator.
""" """
raise OperationalException(_format_exception_message('buy_strategy_generator', 'buy'))
@staticmethod @staticmethod
@abstractmethod
def sell_strategy_generator(params: Dict[str, Any]) -> Callable: def sell_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a sell strategy generator. Create a sell strategy generator.
""" """
raise OperationalException(_format_exception_message('sell_strategy_generator', 'sell'))
@staticmethod @staticmethod
@abstractmethod
def indicator_space() -> List[Dimension]: def indicator_space() -> List[Dimension]:
""" """
Create an indicator space. Create an indicator space.
""" """
raise OperationalException(_format_exception_message('indicator_space', 'buy'))
@staticmethod @staticmethod
@abstractmethod
def sell_indicator_space() -> List[Dimension]: def sell_indicator_space() -> List[Dimension]:
""" """
Create a sell indicator space. Create a sell indicator space.
""" """
raise OperationalException(_format_exception_message('sell_indicator_space', 'sell'))
@staticmethod @staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]: def generate_roi_table(params: Dict) -> Dict[int, float]:
@ -75,6 +95,83 @@ class IHyperOpt(ABC):
return roi_table return roi_table
@staticmethod
def roi_space() -> List[Dimension]:
"""
Create a ROI space.
Defines values to search for each ROI steps.
This method implements adaptive roi hyperspace with varied
ranges for parameters which automatically adapts to the
ticker interval used.
It's used by Freqtrade by default, if no custom roi_space method is defined.
"""
# Default scaling coefficients for the roi hyperspace. Can be changed
# to adjust resulting ranges of the ROI tables.
# Increase if you need wider ranges in the roi hyperspace, decrease if shorter
# ranges are needed.
roi_t_alpha = 1.0
roi_p_alpha = 1.0
ticker_interval_mins = timeframe_to_minutes(IHyperOpt.ticker_interval)
# We define here limits for the ROI space parameters automagically adapted to the
# ticker_interval used by the bot:
#
# * 'roi_t' (limits for the time intervals in the ROI tables) components
# are scaled linearly.
# * 'roi_p' (limits for the ROI value steps) components are scaled logarithmically.
#
# The scaling is designed so that it maps exactly to the legacy Freqtrade roi_space()
# method for the 5m ticker interval.
roi_t_scale = ticker_interval_mins / 5
roi_p_scale = math.log1p(ticker_interval_mins) / math.log1p(5)
roi_limits = {
'roi_t1_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t1_max': int(120 * roi_t_scale * roi_t_alpha),
'roi_t2_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t2_max': int(60 * roi_t_scale * roi_t_alpha),
'roi_t3_min': int(10 * roi_t_scale * roi_t_alpha),
'roi_t3_max': int(40 * roi_t_scale * roi_t_alpha),
'roi_p1_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p1_max': 0.04 * roi_p_scale * roi_p_alpha,
'roi_p2_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p2_max': 0.07 * roi_p_scale * roi_p_alpha,
'roi_p3_min': 0.01 * roi_p_scale * roi_p_alpha,
'roi_p3_max': 0.20 * roi_p_scale * roi_p_alpha,
}
logger.debug(f"Using roi space limits: {roi_limits}")
p = {
'roi_t1': roi_limits['roi_t1_min'],
'roi_t2': roi_limits['roi_t2_min'],
'roi_t3': roi_limits['roi_t3_min'],
'roi_p1': roi_limits['roi_p1_min'],
'roi_p2': roi_limits['roi_p2_min'],
'roi_p3': roi_limits['roi_p3_min'],
}
logger.info(f"Min roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}")
p = {
'roi_t1': roi_limits['roi_t1_max'],
'roi_t2': roi_limits['roi_t2_max'],
'roi_t3': roi_limits['roi_t3_max'],
'roi_p1': roi_limits['roi_p1_max'],
'roi_p2': roi_limits['roi_p2_max'],
'roi_p3': roi_limits['roi_p3_max'],
}
logger.info(f"Max roi table: {round_dict(IHyperOpt.generate_roi_table(p), 5)}")
return [
Integer(roi_limits['roi_t1_min'], roi_limits['roi_t1_max'], name='roi_t1'),
Integer(roi_limits['roi_t2_min'], roi_limits['roi_t2_max'], name='roi_t2'),
Integer(roi_limits['roi_t3_min'], roi_limits['roi_t3_max'], name='roi_t3'),
Real(roi_limits['roi_p1_min'], roi_limits['roi_p1_max'], name='roi_p1'),
Real(roi_limits['roi_p2_min'], roi_limits['roi_p2_max'], name='roi_p2'),
Real(roi_limits['roi_p3_min'], roi_limits['roi_p3_max'], name='roi_p3'),
]
@staticmethod @staticmethod
def stoploss_space() -> List[Dimension]: def stoploss_space() -> List[Dimension]:
""" """
@ -84,22 +181,17 @@ class IHyperOpt(ABC):
You may override it in your custom Hyperopt class. You may override it in your custom Hyperopt class.
""" """
return [ return [
Real(-0.5, -0.02, name='stoploss'), Real(-0.35, -0.02, name='stoploss'),
] ]
@staticmethod # This is needed for proper unpickling the class attribute ticker_interval
def roi_space() -> List[Dimension]: # which is set to the actual value by the resolver.
""" # Why do I still need such shamanic mantras in modern python?
Create a ROI space. def __getstate__(self):
state = self.__dict__.copy()
state['ticker_interval'] = self.ticker_interval
return state
Defines values to search for each ROI steps. def __setstate__(self, state):
You may override it in your custom Hyperopt class. self.__dict__.update(state)
""" IHyperOpt.ticker_interval = state['ticker_interval']
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]

52
freqtrade/persistence.py
View File

@ -1,7 +1,6 @@
""" """
This module contains the class to persist trades into SQLite This module contains the class to persist trades into SQLite
""" """
import logging import logging
from datetime import datetime from datetime import datetime
from decimal import Decimal from decimal import Decimal
@ -19,8 +18,10 @@ from sqlalchemy.pool import StaticPool
from freqtrade import OperationalException from freqtrade import OperationalException
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_DECL_BASE: Any = declarative_base() _DECL_BASE: Any = declarative_base()
_SQL_DOCS_URL = 'http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls' _SQL_DOCS_URL = 'http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls'
@ -48,8 +49,8 @@ def init(db_url: str, clean_open_orders: bool = False) -> None:
try: try:
engine = create_engine(db_url, **kwargs) engine = create_engine(db_url, **kwargs)
except NoSuchModuleError: except NoSuchModuleError:
raise OperationalException(f'Given value for db_url: \'{db_url}\' ' raise OperationalException(f"Given value for db_url: '{db_url}' "
f'is no valid database URL! (See {_SQL_DOCS_URL})') f"is no valid database URL! (See {_SQL_DOCS_URL})")
session = scoped_session(sessionmaker(bind=engine, autoflush=True, autocommit=True)) session = scoped_session(sessionmaker(bind=engine, autoflush=True, autocommit=True))
Trade.session = session() Trade.session = session()
@ -209,7 +210,7 @@ class Trade(_DECL_BASE):
ticker_interval = Column(Integer, nullable=True) ticker_interval = Column(Integer, nullable=True)
def __repr__(self): def __repr__(self):
open_since = arrow.get(self.open_date).humanize() if self.is_open else 'closed' open_since = self.open_date.strftime('%Y-%m-%d %H:%M:%S') if self.is_open else 'closed'
return (f'Trade(id={self.id}, pair={self.pair}, amount={self.amount:.8f}, ' return (f'Trade(id={self.id}, pair={self.pair}, amount={self.amount:.8f}, '
f'open_rate={self.open_rate:.8f}, open_since={open_since})') f'open_rate={self.open_rate:.8f}, open_since={open_since})')
@ -250,7 +251,6 @@ class Trade(_DECL_BASE):
:param initial: Called to initiate stop_loss. :param initial: Called to initiate stop_loss.
Skips everything if self.stop_loss is already set. Skips everything if self.stop_loss is already set.
""" """
if initial and not (self.stop_loss is None or self.stop_loss == 0): if initial and not (self.stop_loss is None or self.stop_loss == 0):
# Don't modify if called with initial and nothing to do # Don't modify if called with initial and nothing to do
return return
@ -259,7 +259,7 @@ class Trade(_DECL_BASE):
# no stop loss assigned yet # no stop loss assigned yet
if not self.stop_loss: if not self.stop_loss:
logger.debug("assigning new stop loss") logger.debug(f"{self.pair} - Assigning new stoploss...")
self.stop_loss = new_loss self.stop_loss = new_loss
self.stop_loss_pct = -1 * abs(stoploss) self.stop_loss_pct = -1 * abs(stoploss)
self.initial_stop_loss = new_loss self.initial_stop_loss = new_loss
@ -269,21 +269,20 @@ class Trade(_DECL_BASE):
# evaluate if the stop loss needs to be updated # evaluate if the stop loss needs to be updated
else: else:
if new_loss > self.stop_loss: # stop losses only walk up, never down! if new_loss > self.stop_loss: # stop losses only walk up, never down!
logger.debug(f"{self.pair} - Adjusting stoploss...")
self.stop_loss = new_loss self.stop_loss = new_loss
self.stop_loss_pct = -1 * abs(stoploss) self.stop_loss_pct = -1 * abs(stoploss)
self.stoploss_last_update = datetime.utcnow() self.stoploss_last_update = datetime.utcnow()
logger.debug("adjusted stop loss")
else: else:
logger.debug("keeping current stop loss") logger.debug(f"{self.pair} - Keeping current stoploss...")
logger.debug( logger.debug(
f"{self.pair} - current price {current_price:.8f}, " f"{self.pair} - Stoploss adjusted. current_price={current_price:.8f}, "
f"bought at {self.open_rate:.8f} and calculated " f"open_rate={self.open_rate:.8f}, max_rate={self.max_rate:.8f}, "
f"stop loss is at: {self.initial_stop_loss:.8f} initial " f"initial_stop_loss={self.initial_stop_loss:.8f}, "
f"stop at {self.stop_loss:.8f}. " f"stop_loss={self.stop_loss:.8f}. "
f"trailing stop loss saved us: " f"Trailing stoploss saved us: "
f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f} " f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f}.")
f"and max observed rate was {self.max_rate:.8f}")
def update(self, order: Dict) -> None: def update(self, order: Dict) -> None:
""" """
@ -331,23 +330,18 @@ class Trade(_DECL_BASE):
self self
) )
def calc_open_trade_price( def calc_open_trade_price(self, fee: Optional[float] = None) -> float:
self,
fee: Optional[float] = None) -> float:
""" """
Calculate the open_rate including fee. Calculate the open_rate including fee.
:param fee: fee to use on the open rate (optional). :param fee: fee to use on the open rate (optional).
If rate is not set self.fee will be used If rate is not set self.fee will be used
:return: Price in of the open trade incl. Fees :return: Price in of the open trade incl. Fees
""" """
buy_trade = (Decimal(self.amount) * Decimal(self.open_rate)) buy_trade = (Decimal(self.amount) * Decimal(self.open_rate))
fees = buy_trade * Decimal(fee or self.fee_open) fees = buy_trade * Decimal(fee or self.fee_open)
return float(buy_trade + fees) return float(buy_trade + fees)
def calc_close_trade_price( def calc_close_trade_price(self, rate: Optional[float] = None,
self,
rate: Optional[float] = None,
fee: Optional[float] = None) -> float: fee: Optional[float] = None) -> float:
""" """
Calculate the close_rate including fee Calculate the close_rate including fee
@ -357,7 +351,6 @@ class Trade(_DECL_BASE):
If rate is not set self.close_rate will be used If rate is not set self.close_rate will be used
:return: Price in BTC of the open trade :return: Price in BTC of the open trade
""" """
if rate is None and not self.close_rate: if rate is None and not self.close_rate:
return 0.0 return 0.0
@ -365,9 +358,7 @@ class Trade(_DECL_BASE):
fees = sell_trade * Decimal(fee or self.fee_close) fees = sell_trade * Decimal(fee or self.fee_close)
return float(sell_trade - fees) return float(sell_trade - fees)
def calc_profit( def calc_profit(self, rate: Optional[float] = None,
self,
rate: Optional[float] = None,
fee: Optional[float] = None) -> float: fee: Optional[float] = None) -> float:
""" """
Calculate the absolute profit in stake currency between Close and Open trade Calculate the absolute profit in stake currency between Close and Open trade
@ -385,9 +376,7 @@ class Trade(_DECL_BASE):
profit = close_trade_price - open_trade_price profit = close_trade_price - open_trade_price
return float(f"{profit:.8f}") return float(f"{profit:.8f}")
def calc_profit_percent( def calc_profit_percent(self, rate: Optional[float] = None,
self,
rate: Optional[float] = None,
fee: Optional[float] = None) -> float: fee: Optional[float] = None) -> float:
""" """
Calculates the profit in percentage (including fee). Calculates the profit in percentage (including fee).
@ -396,7 +385,6 @@ class Trade(_DECL_BASE):
:param fee: fee to use on the close rate (optional). :param fee: fee to use on the close rate (optional).
:return: profit in percentage as float :return: profit in percentage as float
""" """
open_trade_price = self.calc_open_trade_price() open_trade_price = self.calc_open_trade_price()
close_trade_price = self.calc_close_trade_price( close_trade_price = self.calc_close_trade_price(
rate=(rate or self.close_rate), rate=(rate or self.close_rate),
@ -436,8 +424,8 @@ class Trade(_DECL_BASE):
and trade.initial_stop_loss_pct != desired_stoploss): and trade.initial_stop_loss_pct != desired_stoploss):
# Stoploss value got changed # Stoploss value got changed
logger.info(f"Stoploss for {trade} needs adjustment.") logger.info(f"Stoploss for {trade} needs adjustment...")
# Force reset of stoploss # Force reset of stoploss
trade.stop_loss = None trade.stop_loss = None
trade.adjust_stop_loss(trade.open_rate, desired_stoploss) trade.adjust_stop_loss(trade.open_rate, desired_stoploss)
logger.info(f"new stoploss: {trade.stop_loss}, ") logger.info(f"New stoploss: {trade.stop_loss}.")

36
freqtrade/plot/plot_utils.py Normal file
View File

@ -0,0 +1,36 @@
from typing import Any, Dict
from freqtrade import OperationalException
from freqtrade.state import RunMode
from freqtrade.utils import setup_utils_configuration
def validate_plot_args(args: Dict[str, Any]):
if not args.get('datadir') and not args.get('config'):
raise OperationalException(
"You need to specify either `--datadir` or `--config` "
"for plot-profit and plot-dataframe.")
def start_plot_dataframe(args: Dict[str, Any]) -> None:
"""
Entrypoint for dataframe plotting
"""
# Import here to avoid errors if plot-dependencies are not installed.
from freqtrade.plot.plotting import load_and_plot_trades
validate_plot_args(args)
config = setup_utils_configuration(args, RunMode.PLOT)
load_and_plot_trades(config)
def start_plot_profit(args: Dict[str, Any]) -> None:
"""
Entrypoint for plot_profit
"""
# Import here to avoid errors if plot-dependencies are not installed.
from freqtrade.plot.plotting import plot_profit
validate_plot_args(args)
config = setup_utils_configuration(args, RunMode.PLOT)
plot_profit(config)

104
freqtrade/plot/plotting.py
View File

@ -1,15 +1,14 @@
import logging import logging
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional from typing import Any, Dict, List
import pandas as pd import pandas as pd
from freqtrade.configuration import TimeRange from freqtrade.configuration import TimeRange
from freqtrade.data import history from freqtrade.data import history
from freqtrade.data.btanalysis import (combine_tickers_with_mean, from freqtrade.data.btanalysis import (combine_tickers_with_mean,
create_cum_profit, load_trades) create_cum_profit,
from freqtrade.exchange import Exchange extract_trades_of_period, load_trades)
from freqtrade.resolvers import ExchangeResolver, StrategyResolver from freqtrade.resolvers import StrategyResolver
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -19,23 +18,16 @@ try:
from plotly.offline import plot from plotly.offline import plot
import plotly.graph_objects as go import plotly.graph_objects as go
except ImportError: except ImportError:
logger.exception("Module plotly not found \n Please install using `pip install plotly`") logger.exception("Module plotly not found \n Please install using `pip3 install plotly`")
exit(1) exit(1)
def init_plotscript(config): def init_plotscript(config):
""" """
Initialize objects needed for plotting Initialize objects needed for plotting
:return: Dict with tickers, trades, pairs and strategy :return: Dict with tickers, trades and pairs
""" """
exchange: Optional[Exchange] = None
# Exchange is only needed when downloading data!
if config.get("refresh_pairs", False):
exchange = ExchangeResolver(config.get('exchange', {}).get('name'),
config).exchange
strategy = StrategyResolver(config).strategy
if "pairs" in config: if "pairs" in config:
pairs = config["pairs"] pairs = config["pairs"]
else: else:
@ -47,17 +39,18 @@ def init_plotscript(config):
tickers = history.load_data( tickers = history.load_data(
datadir=Path(str(config.get("datadir"))), datadir=Path(str(config.get("datadir"))),
pairs=pairs, pairs=pairs,
ticker_interval=config['ticker_interval'], ticker_interval=config.get('ticker_interval', '5m'),
refresh_pairs=config.get('refresh_pairs', False),
timerange=timerange, timerange=timerange,
exchange=exchange,
) )
trades = load_trades(config) trades = load_trades(config['trade_source'],
db_url=config.get('db_url'),
exportfilename=config.get('exportfilename'),
)
return {"tickers": tickers, return {"tickers": tickers,
"trades": trades, "trades": trades,
"pairs": pairs, "pairs": pairs,
"strategy": strategy,
} }
@ -280,8 +273,15 @@ def generate_profit_graph(pairs: str, tickers: Dict[str, pd.DataFrame],
name='Avg close price', name='Avg close price',
) )
fig = make_subplots(rows=3, cols=1, shared_xaxes=True, row_width=[1, 1, 1]) fig = make_subplots(rows=3, cols=1, shared_xaxes=True,
fig['layout'].update(title="Profit plot") row_width=[1, 1, 1],
vertical_spacing=0.05,
subplot_titles=["AVG Close Price", "Combined Profit", "Profit per pair"])
fig['layout'].update(title="Freqtrade Profit plot")
fig['layout']['yaxis1'].update(title='Price')
fig['layout']['yaxis2'].update(title='Profit')
fig['layout']['yaxis3'].update(title='Profit')
fig['layout']['xaxis']['rangeslider'].update(visible=False)
fig.add_trace(avgclose, 1, 1) fig.add_trace(avgclose, 1, 1)
fig = add_profit(fig, 2, df_comb, 'cum_profit', 'Profit') fig = add_profit(fig, 2, df_comb, 'cum_profit', 'Profit')
@ -321,3 +321,65 @@ def store_plot_file(fig, filename: str, directory: Path, auto_open: bool = False
plot(fig, filename=str(_filename), plot(fig, filename=str(_filename),
auto_open=auto_open) auto_open=auto_open)
logger.info(f"Stored plot as {_filename}") logger.info(f"Stored plot as {_filename}")
def load_and_plot_trades(config: Dict[str, Any]):
"""
From configuration provided
- Initializes plot-script
- Get tickers data
- Generate Dafaframes populated with indicators and signals based on configured strategy
- Load trades excecuted during the selected period
- Generate Plotly plot objects
- Generate plot files
:return: None
"""
strategy = StrategyResolver(config).strategy
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
pair_counter = 0
for pair, data in plot_elements["tickers"].items():
pair_counter += 1
logger.info("analyse pair %s", pair)
tickers = {}
tickers[pair] = data
dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
trades_pair = trades.loc[trades['pair'] == pair]
trades_pair = extract_trades_of_period(dataframe, trades_pair)
fig = generate_candlestick_graph(
pair=pair,
data=dataframe,
trades=trades_pair,
indicators1=config["indicators1"],
indicators2=config["indicators2"],
)
store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
directory=config['user_data_dir'] / "plot")
logger.info('End of plotting process. %s plots generated', pair_counter)
def plot_profit(config: Dict[str, Any]) -> None:
"""
Plots the total profit for all pairs.
Note, the profit calculation isn't realistic.
But should be somewhat proportional, and therefor useful
in helping out to find a good algorithm.
"""
plot_elements = init_plotscript(config)
trades = load_trades(config['trade_source'],
db_url=str(config.get('db_url')),
exportfilename=str(config.get('exportfilename')),
)
# Filter trades to relevant pairs
trades = trades[trades['pair'].isin(plot_elements["pairs"])]
# Create an average close price of all the pairs that were involved.
# this could be useful to gauge the overall market trend
fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
store_plot_file(fig, filename='freqtrade-profit-plot.html',
directory=config['user_data_dir'] / "plot", auto_open=True)

13
freqtrade/resolvers/hyperopt_resolver.py
View File

@ -34,15 +34,12 @@ class HyperOptResolver(IResolver):
self.hyperopt = self._load_hyperopt(hyperopt_name, config, self.hyperopt = self._load_hyperopt(hyperopt_name, config,
extra_dir=config.get('hyperopt_path')) extra_dir=config.get('hyperopt_path'))
# Assign ticker_interval to be used in hyperopt
self.hyperopt.__class__.ticker_interval = str(config['ticker_interval'])
if not hasattr(self.hyperopt, 'populate_buy_trend'): if not hasattr(self.hyperopt, 'populate_buy_trend'):
logger.warning("Custom Hyperopt does not provide populate_buy_trend. " logger.warning("Hyperopt class does not provide populate_buy_trend() method. "
"Using populate_buy_trend from DefaultStrategy.") "Using populate_buy_trend from the strategy.")
if not hasattr(self.hyperopt, 'populate_sell_trend'): if not hasattr(self.hyperopt, 'populate_sell_trend'):
logger.warning("Custom Hyperopt does not provide populate_sell_trend. " logger.warning("Hyperopt class does not provide populate_sell_trend() method. "
"Using populate_sell_trend from DefaultStrategy.") "Using populate_sell_trend from the strategy.")
def _load_hyperopt( def _load_hyperopt(
self, hyperopt_name: str, config: Dict, extra_dir: Optional[str] = None) -> IHyperOpt: self, hyperopt_name: str, config: Dict, extra_dir: Optional[str] = None) -> IHyperOpt:
@ -65,7 +62,7 @@ class HyperOptResolver(IResolver):
abs_paths.insert(0, Path(extra_dir).resolve()) abs_paths.insert(0, Path(extra_dir).resolve())
hyperopt = self._load_object(paths=abs_paths, object_type=IHyperOpt, hyperopt = self._load_object(paths=abs_paths, object_type=IHyperOpt,
object_name=hyperopt_name) object_name=hyperopt_name, kwargs={'config': config})
if hyperopt: if hyperopt:
return hyperopt return hyperopt
raise OperationalException( raise OperationalException(

2
freqtrade/resolvers/iresolver.py
View File

@ -12,7 +12,7 @@ from typing import Any, List, Optional, Tuple, Type, Union
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class IResolver(object): class IResolver:
""" """
This class contains all the logic to load custom classes This class contains all the logic to load custom classes
""" """

12
freqtrade/resolvers/strategy_resolver.py
View File

@ -13,7 +13,6 @@ from typing import Dict, Optional
from freqtrade import constants, OperationalException from freqtrade import constants, OperationalException
from freqtrade.resolvers import IResolver from freqtrade.resolvers import IResolver
from freqtrade.strategy import import_strategy
from freqtrade.strategy.interface import IStrategy from freqtrade.strategy.interface import IStrategy
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -153,13 +152,12 @@ class StrategyResolver(IResolver):
strategy._populate_fun_len = len(getfullargspec(strategy.populate_indicators).args) strategy._populate_fun_len = len(getfullargspec(strategy.populate_indicators).args)
strategy._buy_fun_len = len(getfullargspec(strategy.populate_buy_trend).args) strategy._buy_fun_len = len(getfullargspec(strategy.populate_buy_trend).args)
strategy._sell_fun_len = len(getfullargspec(strategy.populate_sell_trend).args) strategy._sell_fun_len = len(getfullargspec(strategy.populate_sell_trend).args)
if any([x == 2 for x in [strategy._populate_fun_len,
strategy._buy_fun_len,
strategy._sell_fun_len]]):
strategy.INTERFACE_VERSION = 1
try: return strategy
return import_strategy(strategy, config=config)
except TypeError as e:
logger.warning(
f"Impossible to load strategy '{strategy_name}'. "
f"Error: {e}")
raise OperationalException( raise OperationalException(
f"Impossible to load Strategy '{strategy_name}'. This class does not exist " f"Impossible to load Strategy '{strategy_name}'. This class does not exist "

6
freqtrade/rpc/fiat_convert.py
View File

@ -15,7 +15,7 @@ from freqtrade.constants import SUPPORTED_FIAT
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class CryptoFiat(object): class CryptoFiat:
""" """
Object to describe what is the price of Crypto-currency in a FIAT Object to describe what is the price of Crypto-currency in a FIAT
""" """
@ -60,7 +60,7 @@ class CryptoFiat(object):
return self._expiration - time.time() <= 0 return self._expiration - time.time() <= 0
class CryptoToFiatConverter(object): class CryptoToFiatConverter:
""" """
Main class to initiate Crypto to FIAT. Main class to initiate Crypto to FIAT.
This object contains a list of pair Crypto, FIAT This object contains a list of pair Crypto, FIAT
@ -104,7 +104,7 @@ class CryptoToFiatConverter(object):
:return: float, value in fiat of the crypto-currency amount :return: float, value in fiat of the crypto-currency amount
""" """
if crypto_symbol == fiat_symbol: if crypto_symbol == fiat_symbol:
return crypto_amount return float(crypto_amount)
price = self.get_price(crypto_symbol=crypto_symbol, fiat_symbol=fiat_symbol) price = self.get_price(crypto_symbol=crypto_symbol, fiat_symbol=fiat_symbol)
return float(crypto_amount) * float(price) return float(crypto_amount) * float(price)

8
freqtrade/rpc/rpc.py
View File

@ -54,7 +54,7 @@ class RPCException(Exception):
} }
class RPC(object): class RPC:
""" """
RPC class can be used to have extra feature, like bot data, and access to DB data RPC class can be used to have extra feature, like bot data, and access to DB data
""" """
@ -294,9 +294,9 @@ class RPC(object):
total = total + est_btc total = total + est_btc
output.append({ output.append({
'currency': coin, 'currency': coin,
'available': balance['free'], 'free': balance['free'] if balance['free'] is not None else 0,
'balance': balance['total'], 'balance': balance['total'] if balance['total'] is not None else 0,
'pending': balance['used'], 'used': balance['used'] if balance['used'] is not None else 0,
'est_btc': est_btc, 'est_btc': est_btc,
}) })
if total == 0.0: if total == 0.0:

5
freqtrade/rpc/rpc_manager.py
View File

@ -9,7 +9,7 @@ from freqtrade.rpc import RPC, RPCMessageType
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class RPCManager(object): class RPCManager:
""" """
Class to manage RPC objects (Telegram, Slack, ...) Class to manage RPC objects (Telegram, Slack, ...)
""" """
@ -56,7 +56,10 @@ class RPCManager(object):
logger.info('Sending rpc message: %s', msg) logger.info('Sending rpc message: %s', msg)
for mod in self.registered_modules: for mod in self.registered_modules:
logger.debug('Forwarding message to rpc.%s', mod.name) logger.debug('Forwarding message to rpc.%s', mod.name)
try:
mod.send_msg(msg) mod.send_msg(msg)
except NotImplementedError:
logger.error(f"Message type {msg['type']} not implemented by handler {mod.name}.")
def startup_messages(self, config, pairlist) -> None: def startup_messages(self, config, pairlist) -> None:
if config.get('dry_run', False): if config.get('dry_run', False):

131
freqtrade/rpc/telegram.py
View File

@ -4,12 +4,12 @@
This module manage Telegram communication This module manage Telegram communication
""" """
import logging import logging
from typing import Any, Callable, Dict, List from typing import Any, Callable, Dict
from tabulate import tabulate from tabulate import tabulate
from telegram import Bot, ParseMode, ReplyKeyboardMarkup, Update from telegram import ParseMode, ReplyKeyboardMarkup, Update
from telegram.error import NetworkError, TelegramError from telegram.error import NetworkError, TelegramError
from telegram.ext import CommandHandler, Updater from telegram.ext import CommandHandler, Updater, CallbackContext
from freqtrade.__init__ import __version__ from freqtrade.__init__ import __version__
from freqtrade.rpc import RPC, RPCException, RPCMessageType from freqtrade.rpc import RPC, RPCException, RPCMessageType
@ -31,7 +31,7 @@ def authorized_only(command_handler: Callable[..., None]) -> Callable[..., Any]:
""" """
def wrapper(self, *args, **kwargs): def wrapper(self, *args, **kwargs):
""" Decorator logic """ """ Decorator logic """
update = kwargs.get('update') or args[1] update = kwargs.get('update') or args[0]
# Reject unauthorized messages # Reject unauthorized messages
chat_id = int(self._config['telegram']['chat_id']) chat_id = int(self._config['telegram']['chat_id'])
@ -79,7 +79,8 @@ class Telegram(RPC):
registers all known command handlers registers all known command handlers
and starts polling for message updates and starts polling for message updates
""" """
self._updater = Updater(token=self._config['telegram']['token'], workers=0) self._updater = Updater(token=self._config['telegram']['token'], workers=0,
use_context=True)
# Register command handler and start telegram message polling # Register command handler and start telegram message polling
handles = [ handles = [
@ -96,7 +97,7 @@ class Telegram(RPC):
CommandHandler('reload_conf', self._reload_conf), CommandHandler('reload_conf', self._reload_conf),
CommandHandler('stopbuy', self._stopbuy), CommandHandler('stopbuy', self._stopbuy),
CommandHandler('whitelist', self._whitelist), CommandHandler('whitelist', self._whitelist),
CommandHandler('blacklist', self._blacklist, pass_args=True), CommandHandler('blacklist', self._blacklist),
CommandHandler('edge', self._edge), CommandHandler('edge', self._edge),
CommandHandler('help', self._help), CommandHandler('help', self._help),
CommandHandler('version', self._version), CommandHandler('version', self._version),
@ -175,7 +176,7 @@ class Telegram(RPC):
self._send_msg(message) self._send_msg(message)
@authorized_only @authorized_only
def _status(self, bot: Bot, update: Update) -> None: def _status(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /status. Handler for /status.
Returns the current TradeThread status Returns the current TradeThread status
@ -184,11 +185,8 @@ class Telegram(RPC):
:return: None :return: None
""" """
# Check if additional parameters are passed if 'table' in context.args:
params = update.message.text.replace('/status', '').split(' ') \ self._status_table(update, context)
if update.message.text else []
if 'table' in params:
self._status_table(bot, update)
return return
try: try:
@ -221,13 +219,13 @@ class Telegram(RPC):
messages.append("\n".join([l for l in lines if l]).format(**r)) messages.append("\n".join([l for l in lines if l]).format(**r))
for msg in messages: for msg in messages:
self._send_msg(msg, bot=bot) self._send_msg(msg)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _status_table(self, bot: Bot, update: Update) -> None: def _status_table(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /status table. Handler for /status table.
Returns the current TradeThread status in table format Returns the current TradeThread status in table format
@ -240,10 +238,10 @@ class Telegram(RPC):
message = tabulate(df_statuses, headers='keys', tablefmt='simple') message = tabulate(df_statuses, headers='keys', tablefmt='simple')
self._send_msg(f"<pre>{message}</pre>", parse_mode=ParseMode.HTML) self._send_msg(f"<pre>{message}</pre>", parse_mode=ParseMode.HTML)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _daily(self, bot: Bot, update: Update) -> None: def _daily(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /daily <n> Handler for /daily <n>
Returns a daily profit (in BTC) over the last n days. Returns a daily profit (in BTC) over the last n days.
@ -254,8 +252,8 @@ class Telegram(RPC):
stake_cur = self._config['stake_currency'] stake_cur = self._config['stake_currency']
fiat_disp_cur = self._config.get('fiat_display_currency', '') fiat_disp_cur = self._config.get('fiat_display_currency', '')
try: try:
timescale = int(update.message.text.replace('/daily', '').strip()) timescale = int(context.args[0])
except (TypeError, ValueError): except (TypeError, ValueError, IndexError):
timescale = 7 timescale = 7
try: try:
stats = self._rpc_daily_profit( stats = self._rpc_daily_profit(
@ -272,12 +270,12 @@ class Telegram(RPC):
], ],
tablefmt='simple') tablefmt='simple')
message = f'<b>Daily Profit over the last {timescale} days</b>:\n<pre>{stats_tab}</pre>' message = f'<b>Daily Profit over the last {timescale} days</b>:\n<pre>{stats_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML) self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _profit(self, bot: Bot, update: Update) -> None: def _profit(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /profit. Handler for /profit.
Returns a cumulative profit statistics. Returns a cumulative profit statistics.
@ -317,12 +315,12 @@ class Telegram(RPC):
f"*Latest Trade opened:* `{latest_trade_date}`\n" \ f"*Latest Trade opened:* `{latest_trade_date}`\n" \
f"*Avg. Duration:* `{avg_duration}`\n" \ f"*Avg. Duration:* `{avg_duration}`\n" \
f"*Best Performing:* `{best_pair}: {best_rate:.2f}%`" f"*Best Performing:* `{best_pair}: {best_rate:.2f}%`"
self._send_msg(markdown_msg, bot=bot) self._send_msg(markdown_msg)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _balance(self, bot: Bot, update: Update) -> None: def _balance(self, update: Update, context: CallbackContext) -> None:
""" Handler for /balance """ """ Handler for /balance """
try: try:
result = self._rpc_balance(self._config.get('fiat_display_currency', '')) result = self._rpc_balance(self._config.get('fiat_display_currency', ''))
@ -330,16 +328,16 @@ class Telegram(RPC):
for currency in result['currencies']: for currency in result['currencies']:
if currency['est_btc'] > 0.0001: if currency['est_btc'] > 0.0001:
curr_output = "*{currency}:*\n" \ curr_output = "*{currency}:*\n" \
"\t`Available: {available: .8f}`\n" \ "\t`Available: {free: .8f}`\n" \
"\t`Balance: {balance: .8f}`\n" \ "\t`Balance: {balance: .8f}`\n" \
"\t`Pending: {pending: .8f}`\n" \ "\t`Pending: {used: .8f}`\n" \
"\t`Est. BTC: {est_btc: .8f}`\n".format(**currency) "\t`Est. BTC: {est_btc: .8f}`\n".format(**currency)
else: else:
curr_output = "*{currency}:* not showing <1$ amount \n".format(**currency) curr_output = "*{currency}:* not showing <1$ amount \n".format(**currency)
# Handle overflowing messsage length # Handle overflowing messsage length
if len(output + curr_output) >= MAX_TELEGRAM_MESSAGE_LENGTH: if len(output + curr_output) >= MAX_TELEGRAM_MESSAGE_LENGTH:
self._send_msg(output, bot=bot) self._send_msg(output)
output = curr_output output = curr_output
else: else:
output += curr_output output += curr_output
@ -347,12 +345,12 @@ class Telegram(RPC):
output += "\n*Estimated Value*:\n" \ output += "\n*Estimated Value*:\n" \
"\t`BTC: {total: .8f}`\n" \ "\t`BTC: {total: .8f}`\n" \
"\t`{symbol}: {value: .2f}`\n".format(**result) "\t`{symbol}: {value: .2f}`\n".format(**result)
self._send_msg(output, bot=bot) self._send_msg(output)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _start(self, bot: Bot, update: Update) -> None: def _start(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /start. Handler for /start.
Starts TradeThread Starts TradeThread
@ -361,10 +359,10 @@ class Telegram(RPC):
:return: None :return: None
""" """
msg = self._rpc_start() msg = self._rpc_start()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot) self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only @authorized_only
def _stop(self, bot: Bot, update: Update) -> None: def _stop(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /stop. Handler for /stop.
Stops TradeThread Stops TradeThread
@ -373,10 +371,10 @@ class Telegram(RPC):
:return: None :return: None
""" """
msg = self._rpc_stop() msg = self._rpc_stop()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot) self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only @authorized_only
def _reload_conf(self, bot: Bot, update: Update) -> None: def _reload_conf(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /reload_conf. Handler for /reload_conf.
Triggers a config file reload Triggers a config file reload
@ -385,10 +383,10 @@ class Telegram(RPC):
:return: None :return: None
""" """
msg = self._rpc_reload_conf() msg = self._rpc_reload_conf()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot) self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only @authorized_only
def _stopbuy(self, bot: Bot, update: Update) -> None: def _stopbuy(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /stop_buy. Handler for /stop_buy.
Sets max_open_trades to 0 and gracefully sells all open trades Sets max_open_trades to 0 and gracefully sells all open trades
@ -397,10 +395,10 @@ class Telegram(RPC):
:return: None :return: None
""" """
msg = self._rpc_stopbuy() msg = self._rpc_stopbuy()
self._send_msg('Status: `{status}`'.format(**msg), bot=bot) self._send_msg('Status: `{status}`'.format(**msg))
@authorized_only @authorized_only
def _forcesell(self, bot: Bot, update: Update) -> None: def _forcesell(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /forcesell <id>. Handler for /forcesell <id>.
Sells the given trade at current price Sells the given trade at current price
@ -409,16 +407,16 @@ class Telegram(RPC):
:return: None :return: None
""" """
trade_id = update.message.text.replace('/forcesell', '').strip() trade_id = context.args[0] if len(context.args) > 0 else None
try: try:
msg = self._rpc_forcesell(trade_id) msg = self._rpc_forcesell(trade_id)
self._send_msg('Forcesell Result: `{result}`'.format(**msg), bot=bot) self._send_msg('Forcesell Result: `{result}`'.format(**msg))
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _forcebuy(self, bot: Bot, update: Update) -> None: def _forcebuy(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /forcebuy <asset> <price>. Handler for /forcebuy <asset> <price>.
Buys a pair trade at the given or current price Buys a pair trade at the given or current price
@ -427,16 +425,15 @@ class Telegram(RPC):
:return: None :return: None
""" """
message = update.message.text.replace('/forcebuy', '').strip().split() pair = context.args[0]
pair = message[0] price = float(context.args[1]) if len(context.args) > 1 else None
price = float(message[1]) if len(message) > 1 else None
try: try:
self._rpc_forcebuy(pair, price) self._rpc_forcebuy(pair, price)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _performance(self, bot: Bot, update: Update) -> None: def _performance(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /performance. Handler for /performance.
Shows a performance statistic from finished trades Shows a performance statistic from finished trades
@ -455,10 +452,10 @@ class Telegram(RPC):
message = '<b>Performance:</b>\n{}'.format(stats) message = '<b>Performance:</b>\n{}'.format(stats)
self._send_msg(message, parse_mode=ParseMode.HTML) self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _count(self, bot: Bot, update: Update) -> None: def _count(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /count. Handler for /count.
Returns the number of trades running Returns the number of trades running
@ -475,10 +472,10 @@ class Telegram(RPC):
logger.debug(message) logger.debug(message)
self._send_msg(message, parse_mode=ParseMode.HTML) self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _whitelist(self, bot: Bot, update: Update) -> None: def _whitelist(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /whitelist Handler for /whitelist
Shows the currently active whitelist Shows the currently active whitelist
@ -492,17 +489,17 @@ class Telegram(RPC):
logger.debug(message) logger.debug(message)
self._send_msg(message) self._send_msg(message)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _blacklist(self, bot: Bot, update: Update, args: List[str]) -> None: def _blacklist(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /blacklist Handler for /blacklist
Shows the currently active blacklist Shows the currently active blacklist
""" """
try: try:
blacklist = self._rpc_blacklist(args) blacklist = self._rpc_blacklist(context.args)
message = f"Blacklist contains {blacklist['length']} pairs\n" message = f"Blacklist contains {blacklist['length']} pairs\n"
message += f"`{', '.join(blacklist['blacklist'])}`" message += f"`{', '.join(blacklist['blacklist'])}`"
@ -510,10 +507,10 @@ class Telegram(RPC):
logger.debug(message) logger.debug(message)
self._send_msg(message) self._send_msg(message)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _edge(self, bot: Bot, update: Update) -> None: def _edge(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /edge Handler for /edge
Shows information related to Edge Shows information related to Edge
@ -522,12 +519,12 @@ class Telegram(RPC):
edge_pairs = self._rpc_edge() edge_pairs = self._rpc_edge()
edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple') edge_pairs_tab = tabulate(edge_pairs, headers='keys', tablefmt='simple')
message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>' message = f'<b>Edge only validated following pairs:</b>\n<pre>{edge_pairs_tab}</pre>'
self._send_msg(message, bot=bot, parse_mode=ParseMode.HTML) self._send_msg(message, parse_mode=ParseMode.HTML)
except RPCException as e: except RPCException as e:
self._send_msg(str(e), bot=bot) self._send_msg(str(e))
@authorized_only @authorized_only
def _help(self, bot: Bot, update: Update) -> None: def _help(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /help. Handler for /help.
Show commands of the bot Show commands of the bot
@ -559,10 +556,10 @@ class Telegram(RPC):
"*/help:* `This help message`\n" \ "*/help:* `This help message`\n" \
"*/version:* `Show version`" "*/version:* `Show version`"
self._send_msg(message, bot=bot) self._send_msg(message)
@authorized_only @authorized_only
def _version(self, bot: Bot, update: Update) -> None: def _version(self, update: Update, context: CallbackContext) -> None:
""" """
Handler for /version. Handler for /version.
Show version information Show version information
@ -570,10 +567,9 @@ class Telegram(RPC):
:param update: message update :param update: message update
:return: None :return: None
""" """
self._send_msg('*Version:* `{}`'.format(__version__), bot=bot) self._send_msg('*Version:* `{}`'.format(__version__))
def _send_msg(self, msg: str, bot: Bot = None, def _send_msg(self, msg: str, parse_mode: ParseMode = ParseMode.MARKDOWN) -> None:
parse_mode: ParseMode = ParseMode.MARKDOWN) -> None:
""" """
Send given markdown message Send given markdown message
:param msg: message :param msg: message
@ -581,7 +577,6 @@ class Telegram(RPC):
:param parse_mode: telegram parse mode :param parse_mode: telegram parse mode
:return: None :return: None
""" """
bot = bot or self._updater.bot
keyboard = [['/daily', '/profit', '/balance'], keyboard = [['/daily', '/profit', '/balance'],
['/status', '/status table', '/performance'], ['/status', '/status table', '/performance'],
@ -591,7 +586,7 @@ class Telegram(RPC):
try: try:
try: try:
bot.send_message( self._updater.bot.send_message(
self._config['telegram']['chat_id'], self._config['telegram']['chat_id'],
text=msg, text=msg,
parse_mode=parse_mode, parse_mode=parse_mode,
@ -604,7 +599,7 @@ class Telegram(RPC):
'Telegram NetworkError: %s! Trying one more time.', 'Telegram NetworkError: %s! Trying one more time.',
network_err.message network_err.message
) )
bot.send_message( self._updater.bot.send_message(
self._config['telegram']['chat_id'], self._config['telegram']['chat_id'],
text=msg, text=msg,
parse_mode=parse_mode, parse_mode=parse_mode,

4
freqtrade/rpc/webhook.py
View File

@ -43,7 +43,9 @@ class Webhook(RPC):
valuedict = self._config['webhook'].get('webhookbuy', None) valuedict = self._config['webhook'].get('webhookbuy', None)
elif msg['type'] == RPCMessageType.SELL_NOTIFICATION: elif msg['type'] == RPCMessageType.SELL_NOTIFICATION:
valuedict = self._config['webhook'].get('webhooksell', None) valuedict = self._config['webhook'].get('webhooksell', None)
elif msg['type'] == RPCMessageType.STATUS_NOTIFICATION: elif msg['type'] in(RPCMessageType.STATUS_NOTIFICATION,
RPCMessageType.CUSTOM_NOTIFICATION,
RPCMessageType.WARNING_NOTIFICATION):
valuedict = self._config['webhook'].get('webhookstatus', None) valuedict = self._config['webhook'].get('webhookstatus', None)
else: else:
raise NotImplementedError('Unknown message type: {}'.format(msg['type'])) raise NotImplementedError('Unknown message type: {}'.format(msg['type']))

1
freqtrade/state.py
View File

@ -25,4 +25,5 @@ class RunMode(Enum):
BACKTEST = "backtest" BACKTEST = "backtest"
EDGE = "edge" EDGE = "edge"
HYPEROPT = "hyperopt" HYPEROPT = "hyperopt"
PLOT = "plot"
OTHER = "other" # Used for plotting scripts and test OTHER = "other" # Used for plotting scripts and test

46
freqtrade/strategy/__init__.py
View File

@ -1,45 +1 @@
import logging from freqtrade.strategy.interface import IStrategy # noqa: F401
import sys
from copy import deepcopy
from freqtrade.strategy.interface import IStrategy
# Import Default-Strategy to have hyperopt correctly resolve
from freqtrade.strategy.default_strategy import DefaultStrategy # noqa: F401
logger = logging.getLogger(__name__)
def import_strategy(strategy: IStrategy, config: dict) -> IStrategy:
"""
Imports given Strategy instance to global scope
of freqtrade.strategy and returns an instance of it
"""
# Copy all attributes from base class and class
comb = {**strategy.__class__.__dict__, **strategy.__dict__}
# Delete '_abc_impl' from dict as deepcopy fails on 3.7 with
# `TypeError: can't pickle _abc_data objects``
# This will only apply to python 3.7
if sys.version_info.major == 3 and sys.version_info.minor == 7 and '_abc_impl' in comb:
del comb['_abc_impl']
attr = deepcopy(comb)
# Adjust module name
attr['__module__'] = 'freqtrade.strategy'
name = strategy.__class__.__name__
clazz = type(name, (IStrategy,), attr)
logger.debug(
'Imported strategy %s.%s as %s.%s',
strategy.__module__, strategy.__class__.__name__,
clazz.__module__, strategy.__class__.__name__,
)
# Modify global scope to declare class
globals()[name] = clazz
return clazz(config)

126
freqtrade/strategy/default_strategy.py
View File

@ -4,15 +4,18 @@ import talib.abstract as ta
from pandas import DataFrame from pandas import DataFrame
import freqtrade.vendor.qtpylib.indicators as qtpylib import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.indicator_helpers import fishers_inverse
from freqtrade.strategy.interface import IStrategy from freqtrade.strategy.interface import IStrategy
class DefaultStrategy(IStrategy): class DefaultStrategy(IStrategy):
""" """
Default Strategy provided by freqtrade bot. Default Strategy provided by freqtrade bot.
You can override it with your own strategy Please do not modify this strategy, it's intended for internal use only.
Please look at the SampleStrategy in the user_data/strategy directory
or strategy repository https://github.com/freqtrade/freqtrade-strategies
for samples and inspiration.
""" """
INTERFACE_VERSION = 2
# Minimal ROI designed for the strategy # Minimal ROI designed for the strategy
minimal_roi = { minimal_roi = {
@ -73,67 +76,25 @@ class DefaultStrategy(IStrategy):
# ADX # ADX
dataframe['adx'] = ta.ADX(dataframe) dataframe['adx'] = ta.ADX(dataframe)
# Awesome oscillator
dataframe['ao'] = qtpylib.awesome_oscillator(dataframe)
"""
# Commodity Channel Index: values Oversold:<-100, Overbought:>100
dataframe['cci'] = ta.CCI(dataframe)
"""
# MACD # MACD
macd = ta.MACD(dataframe) macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd'] dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal'] dataframe['macdsignal'] = macd['macdsignal']
dataframe['macdhist'] = macd['macdhist'] dataframe['macdhist'] = macd['macdhist']
# MFI
dataframe['mfi'] = ta.MFI(dataframe)
# Minus Directional Indicator / Movement # Minus Directional Indicator / Movement
dataframe['minus_dm'] = ta.MINUS_DM(dataframe)
dataframe['minus_di'] = ta.MINUS_DI(dataframe) dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Plus Directional Indicator / Movement # Plus Directional Indicator / Movement
dataframe['plus_dm'] = ta.PLUS_DM(dataframe)
dataframe['plus_di'] = ta.PLUS_DI(dataframe) dataframe['plus_di'] = ta.PLUS_DI(dataframe)
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
"""
# ROC
dataframe['roc'] = ta.ROC(dataframe)
"""
# RSI # RSI
dataframe['rsi'] = ta.RSI(dataframe) dataframe['rsi'] = ta.RSI(dataframe)
# Inverse Fisher transform on RSI, values [-1.0, 1.0] (https://goo.gl/2JGGoy)
dataframe['fisher_rsi'] = fishers_inverse(dataframe['rsi'])
# Inverse Fisher transform on RSI normalized, value [0.0, 100.0] (https://goo.gl/2JGGoy)
dataframe['fisher_rsi_norma'] = 50 * (dataframe['fisher_rsi'] + 1)
# Stoch
stoch = ta.STOCH(dataframe)
dataframe['slowd'] = stoch['slowd']
dataframe['slowk'] = stoch['slowk']
# Stoch fast # Stoch fast
stoch_fast = ta.STOCHF(dataframe) stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd'] dataframe['fastd'] = stoch_fast['fastd']
dataframe['fastk'] = stoch_fast['fastk'] dataframe['fastk'] = stoch_fast['fastk']
"""
# Stoch RSI
stoch_rsi = ta.STOCHRSI(dataframe)
dataframe['fastd_rsi'] = stoch_rsi['fastd']
dataframe['fastk_rsi'] = stoch_rsi['fastk']
"""
# Overlap Studies
# ------------------------------------
# Previous Bollinger bands
# Because ta.BBANDS implementation is broken with small numbers, it actually
# returns middle band for all the three bands. Switch to qtpylib.bollinger_bands
# and use middle band instead.
dataframe['blower'] = ta.BBANDS(dataframe, nbdevup=2, nbdevdn=2)['lowerband']
# Bollinger bands # Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2) bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
@ -142,88 +103,11 @@ class DefaultStrategy(IStrategy):
dataframe['bb_upperband'] = bollinger['upper'] dataframe['bb_upperband'] = bollinger['upper']
# EMA - Exponential Moving Average # EMA - Exponential Moving Average
dataframe['ema3'] = ta.EMA(dataframe, timeperiod=3)
dataframe['ema5'] = ta.EMA(dataframe, timeperiod=5)
dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10) dataframe['ema10'] = ta.EMA(dataframe, timeperiod=10)
dataframe['ema50'] = ta.EMA(dataframe, timeperiod=50)
dataframe['ema100'] = ta.EMA(dataframe, timeperiod=100)
# SAR Parabol
dataframe['sar'] = ta.SAR(dataframe)
# SMA - Simple Moving Average # SMA - Simple Moving Average
dataframe['sma'] = ta.SMA(dataframe, timeperiod=40) dataframe['sma'] = ta.SMA(dataframe, timeperiod=40)
# TEMA - Triple Exponential Moving Average
dataframe['tema'] = ta.TEMA(dataframe, timeperiod=9)
# Cycle Indicator
# ------------------------------------
# Hilbert Transform Indicator - SineWave
hilbert = ta.HT_SINE(dataframe)
dataframe['htsine'] = hilbert['sine']
dataframe['htleadsine'] = hilbert['leadsine']
# Pattern Recognition - Bullish candlestick patterns
# ------------------------------------
"""
# Hammer: values [0, 100]
dataframe['CDLHAMMER'] = ta.CDLHAMMER(dataframe)
# Inverted Hammer: values [0, 100]
dataframe['CDLINVERTEDHAMMER'] = ta.CDLINVERTEDHAMMER(dataframe)
# Dragonfly Doji: values [0, 100]
dataframe['CDLDRAGONFLYDOJI'] = ta.CDLDRAGONFLYDOJI(dataframe)
# Piercing Line: values [0, 100]
dataframe['CDLPIERCING'] = ta.CDLPIERCING(dataframe) # values [0, 100]
# Morningstar: values [0, 100]
dataframe['CDLMORNINGSTAR'] = ta.CDLMORNINGSTAR(dataframe) # values [0, 100]
# Three White Soldiers: values [0, 100]
dataframe['CDL3WHITESOLDIERS'] = ta.CDL3WHITESOLDIERS(dataframe) # values [0, 100]
"""
# Pattern Recognition - Bearish candlestick patterns
# ------------------------------------
"""
# Hanging Man: values [0, 100]
dataframe['CDLHANGINGMAN'] = ta.CDLHANGINGMAN(dataframe)
# Shooting Star: values [0, 100]
dataframe['CDLSHOOTINGSTAR'] = ta.CDLSHOOTINGSTAR(dataframe)
# Gravestone Doji: values [0, 100]
dataframe['CDLGRAVESTONEDOJI'] = ta.CDLGRAVESTONEDOJI(dataframe)
# Dark Cloud Cover: values [0, 100]
dataframe['CDLDARKCLOUDCOVER'] = ta.CDLDARKCLOUDCOVER(dataframe)
# Evening Doji Star: values [0, 100]
dataframe['CDLEVENINGDOJISTAR'] = ta.CDLEVENINGDOJISTAR(dataframe)
# Evening Star: values [0, 100]
dataframe['CDLEVENINGSTAR'] = ta.CDLEVENINGSTAR(dataframe)
"""
# Pattern Recognition - Bullish/Bearish candlestick patterns
# ------------------------------------
"""
# Three Line Strike: values [0, -100, 100]
dataframe['CDL3LINESTRIKE'] = ta.CDL3LINESTRIKE(dataframe)
# Spinning Top: values [0, -100, 100]
dataframe['CDLSPINNINGTOP'] = ta.CDLSPINNINGTOP(dataframe) # values [0, -100, 100]
# Engulfing: values [0, -100, 100]
dataframe['CDLENGULFING'] = ta.CDLENGULFING(dataframe) # values [0, -100, 100]
# Harami: values [0, -100, 100]
dataframe['CDLHARAMI'] = ta.CDLHARAMI(dataframe) # values [0, -100, 100]
# Three Outside Up/Down: values [0, -100, 100]
dataframe['CDL3OUTSIDE'] = ta.CDL3OUTSIDE(dataframe) # values [0, -100, 100]
# Three Inside Up/Down: values [0, -100, 100]
dataframe['CDL3INSIDE'] = ta.CDL3INSIDE(dataframe) # values [0, -100, 100]
"""
# Chart type
# ------------------------------------
# Heikinashi stategy
heikinashi = qtpylib.heikinashi(dataframe)
dataframe['ha_open'] = heikinashi['open']
dataframe['ha_close'] = heikinashi['close']
dataframe['ha_high'] = heikinashi['high']
dataframe['ha_low'] = heikinashi['low']
return dataframe return dataframe
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

42
freqtrade/strategy/interface.py
View File

@ -39,6 +39,7 @@ class SellType(Enum):
TRAILING_STOP_LOSS = "trailing_stop_loss" TRAILING_STOP_LOSS = "trailing_stop_loss"
SELL_SIGNAL = "sell_signal" SELL_SIGNAL = "sell_signal"
FORCE_SELL = "force_sell" FORCE_SELL = "force_sell"
EMERGENCY_SELL = "emergency_sell"
NONE = "" NONE = ""
@ -60,6 +61,11 @@ class IStrategy(ABC):
stoploss -> float: optimal stoploss designed for the strategy stoploss -> float: optimal stoploss designed for the strategy
ticker_interval -> str: value of the ticker interval to use for the strategy ticker_interval -> str: value of the ticker interval to use for the strategy
""" """
# Strategy interface version
# Default to version 2
# Version 1 is the initial interface without metadata dict
# Version 2 populate_* include metadata dict
INTERFACE_VERSION: int = 2
_populate_fun_len: int = 0 _populate_fun_len: int = 0
_buy_fun_len: int = 0 _buy_fun_len: int = 0
@ -196,7 +202,6 @@ class IStrategy(ABC):
:param metadata: Metadata dictionary with additional data (e.g. 'pair') :param metadata: Metadata dictionary with additional data (e.g. 'pair')
:return: DataFrame with ticker data and indicator data :return: DataFrame with ticker data and indicator data
""" """
pair = str(metadata.get('pair')) pair = str(metadata.get('pair'))
# Test if seen this pair and last candle before. # Test if seen this pair and last candle before.
@ -286,7 +291,6 @@ class IStrategy(ABC):
:param force_stoploss: Externally provided stoploss :param force_stoploss: Externally provided stoploss
:return: True if trade should be sold, False otherwise :return: True if trade should be sold, False otherwise
""" """
# Set current rate to low for backtesting sell # Set current rate to low for backtesting sell
current_rate = low or rate current_rate = low or rate
current_profit = trade.calc_profit_percent(current_rate) current_profit = trade.calc_profit_percent(current_rate)
@ -298,6 +302,8 @@ class IStrategy(ABC):
force_stoploss=force_stoploss, high=high) force_stoploss=force_stoploss, high=high)
if stoplossflag.sell_flag: if stoplossflag.sell_flag:
logger.debug(f"{trade.pair} - Stoploss hit. sell_flag=True, "
f"sell_type={stoplossflag.sell_type}")
return stoplossflag return stoplossflag
# Set current rate to high for backtesting sell # Set current rate to high for backtesting sell
@ -306,22 +312,31 @@ class IStrategy(ABC):
experimental = self.config.get('experimental', {}) experimental = self.config.get('experimental', {})
if buy and experimental.get('ignore_roi_if_buy_signal', False): if buy and experimental.get('ignore_roi_if_buy_signal', False):
logger.debug('Buy signal still active - not selling.') # This one is noisy, commented out
# logger.debug(f"{trade.pair} - Buy signal still active. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE) return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
# Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee) # Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee)
if self.min_roi_reached(trade=trade, current_profit=current_profit, current_time=date): if self.min_roi_reached(trade=trade, current_profit=current_profit, current_time=date):
logger.debug('Required profit reached. Selling..') logger.debug(f"{trade.pair} - Required profit reached. sell_flag=True, "
f"sell_type=SellType.ROI")
return SellCheckTuple(sell_flag=True, sell_type=SellType.ROI) return SellCheckTuple(sell_flag=True, sell_type=SellType.ROI)
if experimental.get('sell_profit_only', False): if experimental.get('sell_profit_only', False):
logger.debug('Checking if trade is profitable..') # This one is noisy, commented out
# logger.debug(f"{trade.pair} - Checking if trade is profitable...")
if trade.calc_profit(rate=rate) <= 0: if trade.calc_profit(rate=rate) <= 0:
# This one is noisy, commented out
# logger.debug(f"{trade.pair} - Trade is not profitable. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE) return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
if sell and not buy and experimental.get('use_sell_signal', False): if sell and not buy and experimental.get('use_sell_signal', False):
logger.debug('Sell signal received. Selling..') logger.debug(f"{trade.pair} - Sell signal received. sell_flag=True, "
f"sell_type=SellType.SELL_SIGNAL")
return SellCheckTuple(sell_flag=True, sell_type=SellType.SELL_SIGNAL) return SellCheckTuple(sell_flag=True, sell_type=SellType.SELL_SIGNAL)
# This one is noisy, commented out...
# logger.debug(f"{trade.pair} - No sell signal. sell_flag=False")
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE) return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
def stop_loss_reached(self, current_rate: float, trade: Trade, def stop_loss_reached(self, current_rate: float, trade: Trade,
@ -332,7 +347,6 @@ class IStrategy(ABC):
decides to sell or not decides to sell or not
:param current_profit: current profit in percent :param current_profit: current profit in percent
""" """
trailing_stop = self.config.get('trailing_stop', False) trailing_stop = self.config.get('trailing_stop', False)
stop_loss_value = force_stoploss if force_stoploss else self.stoploss stop_loss_value = force_stoploss if force_stoploss else self.stoploss
@ -353,7 +367,7 @@ class IStrategy(ABC):
if 'trailing_stop_positive' in self.config and high_profit > sl_offset: if 'trailing_stop_positive' in self.config and high_profit > sl_offset:
# Ignore mypy error check in configuration that this is a float # Ignore mypy error check in configuration that this is a float
stop_loss_value = self.config.get('trailing_stop_positive') # type: ignore stop_loss_value = self.config.get('trailing_stop_positive') # type: ignore
logger.debug(f"using positive stop loss: {stop_loss_value} " logger.debug(f"{trade.pair} - Using positive stoploss: {stop_loss_value} "
f"offset: {sl_offset:.4g} profit: {current_profit:.4f}%") f"offset: {sl_offset:.4g} profit: {current_profit:.4f}%")
trade.adjust_stop_loss(high or current_rate, stop_loss_value) trade.adjust_stop_loss(high or current_rate, stop_loss_value)
@ -363,20 +377,20 @@ class IStrategy(ABC):
(trade.stop_loss >= current_rate) and (trade.stop_loss >= current_rate) and
(not self.order_types.get('stoploss_on_exchange'))): (not self.order_types.get('stoploss_on_exchange'))):
selltype = SellType.STOP_LOSS sell_type = SellType.STOP_LOSS
# If initial stoploss is not the same as current one then it is trailing. # If initial stoploss is not the same as current one then it is trailing.
if trade.initial_stop_loss != trade.stop_loss: if trade.initial_stop_loss != trade.stop_loss:
selltype = SellType.TRAILING_STOP_LOSS sell_type = SellType.TRAILING_STOP_LOSS
logger.debug( logger.debug(
f"HIT STOP: current price at {current_rate:.6f}, " f"{trade.pair} - HIT STOP: current price at {current_rate:.6f}, "
f"stoploss is {trade.stop_loss:.6f}, " f"stoploss is {trade.stop_loss:.6f}, "
f"initial stoploss was at {trade.initial_stop_loss:.6f}, " f"initial stoploss was at {trade.initial_stop_loss:.6f}, "
f"trade opened at {trade.open_rate:.6f}") f"trade opened at {trade.open_rate:.6f}")
logger.debug(f"trailing stop saved {trade.stop_loss - trade.initial_stop_loss:.6f}") logger.debug(f"{trade.pair} - Trailing stop saved "
f"{trade.stop_loss - trade.initial_stop_loss:.6f}")
logger.debug('Stop loss hit.') return SellCheckTuple(sell_flag=True, sell_type=sell_type)
return SellCheckTuple(sell_flag=True, sell_type=selltype)
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE) return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)

69
freqtrade/tests/exchange/test_kraken.py
View File

@ -1,69 +0,0 @@
# pragma pylint: disable=missing-docstring, C0103, bad-continuation, global-statement
# pragma pylint: disable=protected-access
from random import randint
from unittest.mock import MagicMock
from freqtrade.tests.conftest import get_patched_exchange
def test_buy_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'limit'
time_in_force = 'ioc'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.buy(pair='ETH/BTC', ordertype=order_type,
amount=1, rate=200, time_in_force=time_in_force)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'timeInForce': 'ioc',
'trading_agreement': 'agree'}
def test_sell_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
assert api_mock.create_order.call_args[0][5] == {'trading_agreement': 'agree'}

181
freqtrade/tests/test_arguments.py
View File

@ -1,181 +0,0 @@
# pragma pylint: disable=missing-docstring, C0103
import argparse
import pytest
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
from freqtrade.configuration.cli_options import check_int_positive
# Parse common command-line-arguments. Used for all tools
def test_parse_args_none() -> None:
arguments = Arguments([], '')
assert isinstance(arguments, Arguments)
assert isinstance(arguments.parser, argparse.ArgumentParser)
def test_parse_args_defaults() -> None:
args = Arguments([], '').get_parsed_arg()
assert args.config == ['config.json']
assert args.strategy_path is None
assert args.datadir is None
assert args.verbosity == 0
def test_parse_args_config() -> None:
args = Arguments(['-c', '/dev/null'], '').get_parsed_arg()
assert args.config == ['/dev/null']
args = Arguments(['--config', '/dev/null'], '').get_parsed_arg()
assert args.config == ['/dev/null']
args = Arguments(['--config', '/dev/null',
'--config', '/dev/zero'],
'').get_parsed_arg()
assert args.config == ['/dev/null', '/dev/zero']
def test_parse_args_db_url() -> None:
args = Arguments(['--db-url', 'sqlite:///test.sqlite'], '').get_parsed_arg()
assert args.db_url == 'sqlite:///test.sqlite'
def test_parse_args_verbose() -> None:
args = Arguments(['-v'], '').get_parsed_arg()
assert args.verbosity == 1
args = Arguments(['--verbose'], '').get_parsed_arg()
assert args.verbosity == 1
def test_common_scripts_options() -> None:
args = Arguments(['download-data', '-p', 'ETH/BTC', 'XRP/BTC'], '').get_parsed_arg()
assert args.pairs == ['ETH/BTC', 'XRP/BTC']
assert hasattr(args, "func")
def test_parse_args_version() -> None:
with pytest.raises(SystemExit, match=r'0'):
Arguments(['--version'], '').get_parsed_arg()
def test_parse_args_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['-c'], '').get_parsed_arg()
def test_parse_args_strategy() -> None:
args = Arguments(['--strategy', 'SomeStrategy'], '').get_parsed_arg()
assert args.strategy == 'SomeStrategy'
def test_parse_args_strategy_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['--strategy'], '').get_parsed_arg()
def test_parse_args_strategy_path() -> None:
args = Arguments(['--strategy-path', '/some/path'], '').get_parsed_arg()
assert args.strategy_path == '/some/path'
def test_parse_args_strategy_path_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['--strategy-path'], '').get_parsed_arg()
def test_parse_args_backtesting_invalid() -> None:
with pytest.raises(SystemExit, match=r'2'):
Arguments(['backtesting --ticker-interval'], '').get_parsed_arg()
with pytest.raises(SystemExit, match=r'2'):
Arguments(['backtesting --ticker-interval', 'abc'], '').get_parsed_arg()
def test_parse_args_backtesting_custom() -> None:
args = [
'-c', 'test_conf.json',
'backtesting',
'--ticker-interval', '1m',
'--refresh-pairs-cached',
'--strategy-list',
'DefaultStrategy',
'TestStrategy'
]
call_args = Arguments(args, '').get_parsed_arg()
assert call_args.config == ['test_conf.json']
assert call_args.verbosity == 0
assert call_args.subparser == 'backtesting'
assert call_args.func is not None
assert call_args.ticker_interval == '1m'
assert call_args.refresh_pairs is True
assert type(call_args.strategy_list) is list
assert len(call_args.strategy_list) == 2
def test_parse_args_hyperopt_custom() -> None:
args = [
'-c', 'test_conf.json',
'hyperopt',
'--epochs', '20',
'--spaces', 'buy'
]
call_args = Arguments(args, '').get_parsed_arg()
assert call_args.config == ['test_conf.json']
assert call_args.epochs == 20
assert call_args.verbosity == 0
assert call_args.subparser == 'hyperopt'
assert call_args.spaces == ['buy']
assert call_args.func is not None
def test_download_data_options() -> None:
args = [
'--datadir', 'datadir/directory',
'download-data',
'--pairs-file', 'file_with_pairs',
'--days', '30',
'--exchange', 'binance'
]
args = Arguments(args, '').get_parsed_arg()
assert args.pairs_file == 'file_with_pairs'
assert args.datadir == 'datadir/directory'
assert args.days == 30
assert args.exchange == 'binance'
def test_plot_dataframe_options() -> None:
args = [
'--indicators1', 'sma10,sma100',
'--indicators2', 'macd,fastd,fastk',
'--plot-limit', '30',
'-p', 'UNITTEST/BTC',
]
arguments = Arguments(args, '')
arguments._build_args(ARGS_PLOT_DATAFRAME)
pargs = arguments._parse_args()
assert pargs.indicators1 == "sma10,sma100"
assert pargs.indicators2 == "macd,fastd,fastk"
assert pargs.plot_limit == 30
assert pargs.pairs == ["UNITTEST/BTC"]
def test_check_int_positive() -> None:
assert check_int_positive("3") == 3
assert check_int_positive("1") == 1
assert check_int_positive("100") == 100
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("-2")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("0")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("3.5")
with pytest.raises(argparse.ArgumentTypeError):
check_int_positive("DeadBeef")

54
freqtrade/utils.py
View File

@ -1,14 +1,14 @@
import logging import logging
import sys import sys
from argparse import Namespace
from pathlib import Path from pathlib import Path
from typing import Any, Dict from typing import Any, Dict, List
import arrow import arrow
from freqtrade import OperationalException
from freqtrade.configuration import Configuration, TimeRange from freqtrade.configuration import Configuration, TimeRange
from freqtrade.configuration.directory_operations import create_userdata_dir from freqtrade.configuration.directory_operations import create_userdata_dir
from freqtrade.data.history import download_pair_history from freqtrade.data.history import refresh_backtest_ohlcv_data
from freqtrade.exchange import available_exchanges from freqtrade.exchange import available_exchanges
from freqtrade.resolvers import ExchangeResolver from freqtrade.resolvers import ExchangeResolver
from freqtrade.state import RunMode from freqtrade.state import RunMode
@ -16,7 +16,7 @@ from freqtrade.state import RunMode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any]: def setup_utils_configuration(args: Dict[str, Any], method: RunMode) -> Dict[str, Any]:
""" """
Prepare the configuration for utils subcommands Prepare the configuration for utils subcommands
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
@ -33,34 +33,34 @@ def setup_utils_configuration(args: Namespace, method: RunMode) -> Dict[str, Any
return config return config
def start_list_exchanges(args: Namespace) -> None: def start_list_exchanges(args: Dict[str, Any]) -> None:
""" """
Print available exchanges Print available exchanges
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
:return: None :return: None
""" """
if args.print_one_column: if args['print_one_column']:
print('\n'.join(available_exchanges())) print('\n'.join(available_exchanges()))
else: else:
print(f"Exchanges supported by ccxt and available for Freqtrade: " print(f"Exchanges supported by ccxt and available for Freqtrade: "
f"{', '.join(available_exchanges())}") f"{', '.join(available_exchanges())}")
def start_create_userdir(args: Namespace) -> None: def start_create_userdir(args: Dict[str, Any]) -> None:
""" """
Create "user_data" directory to contain user data strategies, hyperopts, ...) Create "user_data" directory to contain user data strategies, hyperopts, ...)
:param args: Cli args from Arguments() :param args: Cli args from Arguments()
:return: None :return: None
""" """
if "user_data_dir" in args and args.user_data_dir: if "user_data_dir" in args and args["user_data_dir"]:
create_userdata_dir(args.user_data_dir, create_dir=True) create_userdata_dir(args["user_data_dir"], create_dir=True)
else: else:
logger.warning("`create-userdir` requires --userdir to be set.") logger.warning("`create-userdir` requires --userdir to be set.")
sys.exit(1) sys.exit(1)
def start_download_data(args: Namespace) -> None: def start_download_data(args: Dict[str, Any]) -> None:
""" """
Download data (former download_backtest_data.py script) Download data (former download_backtest_data.py script)
""" """
@ -71,43 +71,29 @@ def start_download_data(args: Namespace) -> None:
time_since = arrow.utcnow().shift(days=-config['days']).strftime("%Y%m%d") time_since = arrow.utcnow().shift(days=-config['days']).strftime("%Y%m%d")
timerange = TimeRange.parse_timerange(f'{time_since}-') timerange = TimeRange.parse_timerange(f'{time_since}-')
if 'pairs' not in config:
raise OperationalException(
"Downloading data requires a list of pairs. "
"Please check the documentation on how to configure this.")
dl_path = Path(config['datadir']) dl_path = Path(config['datadir'])
logger.info(f'About to download pairs: {config["pairs"]}, ' logger.info(f'About to download pairs: {config["pairs"]}, '
f'intervals: {config["timeframes"]} to {dl_path}') f'intervals: {config["timeframes"]} to {dl_path}')
pairs_not_available = [] pairs_not_available: List[str] = []
try: try:
# Init exchange # Init exchange
exchange = ExchangeResolver(config['exchange']['name'], config).exchange exchange = ExchangeResolver(config['exchange']['name'], config).exchange
for pair in config["pairs"]: pairs_not_available = refresh_backtest_ohlcv_data(
if pair not in exchange.markets: exchange, pairs=config["pairs"], timeframes=config["timeframes"],
pairs_not_available.append(pair) dl_path=Path(config['datadir']), timerange=timerange, erase=config.get("erase"))
logger.info(f"Skipping pair {pair}...")
continue
for ticker_interval in config["timeframes"]:
pair_print = pair.replace('/', '_')
filename = f'{pair_print}-{ticker_interval}.json'
dl_file = dl_path.joinpath(filename)
if config.get("erase") and dl_file.exists():
logger.info(
f'Deleting existing data for pair {pair}, interval {ticker_interval}.')
dl_file.unlink()
logger.info(f'Downloading pair {pair}, interval {ticker_interval}.')
download_pair_history(datadir=dl_path, exchange=exchange,
pair=pair, ticker_interval=str(ticker_interval),
timerange=timerange)
except KeyboardInterrupt: except KeyboardInterrupt:
sys.exit("SIGINT received, aborting ...") sys.exit("SIGINT received, aborting ...")
finally: finally:
if pairs_not_available: if pairs_not_available:
logger.info( logger.info(f"Pairs [{','.join(pairs_not_available)}] not available "
f"Pairs [{','.join(pairs_not_available)}] not available "
f"on exchange {config['exchange']['name']}.") f"on exchange {config['exchange']['name']}.")
# configuration.resolve_pairs_list()
print(config)

2
freqtrade/wallets.py
View File

@ -17,7 +17,7 @@ class Wallet(NamedTuple):
total: float = 0 total: float = 0
class Wallets(object): class Wallets:
def __init__(self, config: dict, exchange: Exchange) -> None: def __init__(self, config: dict, exchange: Exchange) -> None:
self._config = config self._config = config

15
freqtrade/worker.py
View File

@ -4,27 +4,26 @@ Main Freqtrade worker class.
import logging import logging
import time import time
import traceback import traceback
from argparse import Namespace from typing import Any, Callable, Dict, Optional
from typing import Any, Callable, Optional
import sdnotify import sdnotify
from freqtrade import (constants, OperationalException, TemporaryError, from freqtrade import (OperationalException, TemporaryError, __version__,
__version__) constants)
from freqtrade.configuration import Configuration from freqtrade.configuration import Configuration
from freqtrade.freqtradebot import FreqtradeBot from freqtrade.freqtradebot import FreqtradeBot
from freqtrade.state import State
from freqtrade.rpc import RPCMessageType from freqtrade.rpc import RPCMessageType
from freqtrade.state import State
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class Worker(object): class Worker:
""" """
Freqtradebot worker class Freqtradebot worker class
""" """
def __init__(self, args: Namespace, config=None) -> None: def __init__(self, args: Dict[str, Any], config=None) -> None:
""" """
Init all variables and objects the bot needs to work Init all variables and objects the bot needs to work
""" """

5
mkdocs.yml
View File

@ -11,15 +11,16 @@ nav:
- Telegram: telegram-usage.md - Telegram: telegram-usage.md
- Web Hook: webhook-config.md - Web Hook: webhook-config.md
- REST API: rest-api.md - REST API: rest-api.md
- Data Downloading: data-download.md
- Backtesting: backtesting.md - Backtesting: backtesting.md
- Hyperopt: hyperopt.md - Hyperopt: hyperopt.md
- Edge positioning: edge.md - Edge positioning: edge.md
- Plotting: plotting.md
- Deprecated features: deprecated.md
- FAQ: faq.md - FAQ: faq.md
- Data Analysis: data-analysis.md - Data Analysis: data-analysis.md
- Plotting: plotting.md
- SQL Cheatsheet: sql_cheatsheet.md - SQL Cheatsheet: sql_cheatsheet.md
- Sandbox testing: sandbox-testing.md - Sandbox testing: sandbox-testing.md
- Deprecated features: deprecated.md
- Contributors guide: developer.md - Contributors guide: developer.md
theme: theme:
name: material name: material

16
requirements-common.txt
View File

@ -1,24 +1,18 @@
# requirements without requirements installable via conda # requirements without requirements installable via conda
# mainly used for Raspberry pi installs # mainly used for Raspberry pi installs
ccxt==1.18.1068 ccxt==1.18.1180
SQLAlchemy==1.3.7 SQLAlchemy==1.3.8
python-telegram-bot==11.1.0 python-telegram-bot==12.1.1
arrow==0.14.5 arrow==0.15.2
cachetools==3.1.1 cachetools==3.1.1
requests==2.22.0 requests==2.22.0
urllib3==1.25.3 urllib3==1.25.5
wrapt==1.11.2 wrapt==1.11.2
scikit-learn==0.21.3
joblib==0.13.2
jsonschema==3.0.2 jsonschema==3.0.2
TA-Lib==0.4.17 TA-Lib==0.4.17
tabulate==0.8.3 tabulate==0.8.3
coinmarketcap==5.0.3 coinmarketcap==5.0.3
# Required for hyperopt
scikit-optimize==0.5.2
filelock==3.0.12
# find first, C search in arrays # find first, C search in arrays
py_find_1st==1.1.4 py_find_1st==1.1.4

3
requirements-dev.txt
View File

@ -1,13 +1,14 @@
# Include all requirements to run the bot. # Include all requirements to run the bot.
-r requirements.txt -r requirements.txt
-r requirements-plot.txt -r requirements-plot.txt
-r requirements-hyperopt.txt
coveralls==1.8.2 coveralls==1.8.2
flake8==3.7.8 flake8==3.7.8
flake8-type-annotations==0.1.0 flake8-type-annotations==0.1.0
flake8-tidy-imports==2.0.0 flake8-tidy-imports==2.0.0
mypy==0.720 mypy==0.720
pytest==5.1.0 pytest==5.1.3
pytest-asyncio==0.10.0 pytest-asyncio==0.10.0
pytest-cov==2.7.1 pytest-cov==2.7.1
pytest-mock==1.10.4 pytest-mock==1.10.4

9
requirements-hyperopt.txt Normal file
View File

@ -0,0 +1,9 @@
# Include all requirements to run the bot.
# -r requirements.txt
# Required for hyperopt
scipy==1.3.1
scikit-learn==0.21.3
scikit-optimize==0.5.2
filelock==3.0.12
joblib==0.13.2

2
requirements-plot.txt
View File

@ -1,5 +1,5 @@
# Include all requirements to run the bot. # Include all requirements to run the bot.
-r requirements.txt -r requirements.txt
plotly==4.1.0 plotly==4.1.1

5
requirements.txt
View File

@ -1,6 +1,5 @@
# Load common requirements # Load common requirements
-r requirements-common.txt -r requirements-common.txt
numpy==1.17.0 numpy==1.17.2
pandas==0.25.0 pandas==0.25.1
scipy==1.3.1

99
scripts/plot_dataframe.py
View File

@ -1,100 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""
Script to display when the bot will buy on specific pair(s)
Use `python plot_dataframe.py --help` to display the command line arguments
Indicators recommended
Row 1: sma, ema3, ema5, ema10, ema50
Row 3: macd, rsi, fisher_rsi, mfi, slowd, slowk, fastd, fastk
Example of usage:
> python3 scripts/plot_dataframe.py --pairs BTC/EUR,XRP/BTC -d user_data/data/
--indicators1 sma,ema3 --indicators2 fastk,fastd
"""
import logging
import sys import sys
from typing import Any, Dict, List
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_DATAFRAME
from freqtrade.data.btanalysis import extract_trades_of_period
from freqtrade.optimize import setup_configuration
from freqtrade.plot.plotting import (init_plotscript, generate_candlestick_graph,
store_plot_file,
generate_plot_filename)
from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
def analyse_and_plot_pairs(config: Dict[str, Any]): print("This script has been integrated into freqtrade "
""" "and its functionality is available by calling `freqtrade plot-dataframe`.")
From arguments provided in cli: print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
-Initialise backtest env "for details.")
-Get tickers data
-Generate Dafaframes populated with indicators and signals
-Load trades excecuted on same periods
-Generate Plotly plot objects
-Generate plot files
:return: None
"""
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
strategy = plot_elements["strategy"]
pair_counter = 0 sys.exit(1)
for pair, data in plot_elements["tickers"].items():
pair_counter += 1
logger.info("analyse pair %s", pair)
tickers = {}
tickers[pair] = data
dataframe = strategy.analyze_ticker(tickers[pair], {'pair': pair})
trades_pair = trades.loc[trades['pair'] == pair]
trades_pair = extract_trades_of_period(dataframe, trades_pair)
fig = generate_candlestick_graph(
pair=pair,
data=dataframe,
trades=trades_pair,
indicators1=config["indicators1"].split(","),
indicators2=config["indicators2"].split(",")
)
store_plot_file(fig, filename=generate_plot_filename(pair, config['ticker_interval']),
directory=config['user_data_dir'] / "plot")
logger.info('End of ploting process %s plots generated', pair_counter)
def plot_parse_args(args: List[str]) -> Dict[str, Any]:
"""
Parse args passed to the script
:param args: Cli arguments
:return: args: Array with all arguments
"""
arguments = Arguments(args, 'Graph dataframe')
arguments._build_args(optionlist=ARGS_PLOT_DATAFRAME)
parsed_args = arguments._parse_args()
# Load the configuration
config = setup_configuration(parsed_args, RunMode.OTHER)
return config
def main(sysargv: List[str]) -> None:
"""
This function will initiate the bot and start the trading loop.
:return: None
"""
logger.info('Starting Plot Dataframe')
analyse_and_plot_pairs(
plot_parse_args(sysargv)
)
exit()
if __name__ == '__main__':
main(sys.argv[1:])

65
scripts/plot_profit.py
View File

@ -1,66 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""
Script to display profits
Use `python plot_profit.py --help` to display the command line arguments
"""
import logging
import sys import sys
from typing import Any, Dict, List
from freqtrade.configuration import Arguments
from freqtrade.configuration.arguments import ARGS_PLOT_PROFIT
from freqtrade.optimize import setup_configuration
from freqtrade.plot.plotting import init_plotscript, generate_profit_graph, store_plot_file
from freqtrade.state import RunMode
logger = logging.getLogger(__name__)
def plot_profit(config: Dict[str, Any]) -> None: print("This script has been integrated into freqtrade "
""" "and its functionality is available by calling `freqtrade plot-profit`.")
Plots the total profit for all pairs. print("Please check the documentation on https://www.freqtrade.io/en/latest/plotting/ "
Note, the profit calculation isn't realistic. "for details.")
But should be somewhat proportional, and therefor useful
in helping out to find a good algorithm.
"""
plot_elements = init_plotscript(config)
trades = plot_elements['trades']
# Filter trades to relevant pairs
trades = trades[trades['pair'].isin(plot_elements["pairs"])]
# Create an average close price of all the pairs that were involved. sys.exit(1)
# this could be useful to gauge the overall market trend
fig = generate_profit_graph(plot_elements["pairs"], plot_elements["tickers"], trades)
store_plot_file(fig, filename='freqtrade-profit-plot.html',
directory=config['user_data_dir'] / "plot", auto_open=True)
def plot_parse_args(args: List[str]) -> Dict[str, Any]:
"""
Parse args passed to the script
:param args: Cli arguments
:return: args: Array with all arguments
"""
arguments = Arguments(args, 'Graph profits')
arguments._build_args(optionlist=ARGS_PLOT_PROFIT)
parsed_args = arguments._parse_args()
# Load the configuration
config = setup_configuration(parsed_args, RunMode.OTHER)
return config
def main(sysargv: List[str]) -> None:
"""
This function will initiate the bot and start the trading loop.
:return: None
"""
logger.info('Starting Plot Dataframe')
plot_profit(
plot_parse_args(sysargv)
)
if __name__ == '__main__':
main(sys.argv[1:])

7
setup.cfg
View File

@ -2,9 +2,14 @@
#ignore = #ignore =
max-line-length = 100 max-line-length = 100
max-complexity = 12 max-complexity = 12
exclude =
.git,
__pycache__,
.eggs,
user_data,
[mypy] [mypy]
ignore_missing_imports = True ignore_missing_imports = True
[mypy-freqtrade.tests.*] [mypy-tests.*]
ignore_errors = True ignore_errors = True

33
setup.py
View File

@ -6,11 +6,25 @@ if version_info.major == 3 and version_info.minor < 6 or \
print('Your Python interpreter must be 3.6 or greater!') print('Your Python interpreter must be 3.6 or greater!')
exit(1) exit(1)
from freqtrade import __version__ from pathlib import Path # noqa: E402
from freqtrade import __version__ # noqa: E402
readme_file = Path(__file__).parent / "README.md"
readme_long = "Crypto Trading Bot"
if readme_file.is_file():
readme_long = (Path(__file__).parent / "README.md").read_text()
# Requirements used for submodules # Requirements used for submodules
api = ['flask'] api = ['flask']
plot = ['plotly>=4.0'] plot = ['plotly>=4.0']
hyperopt = [
'scipy',
'scikit-learn',
'scikit-optimize',
'filelock',
'joblib',
]
develop = [ develop = [
'coveralls', 'coveralls',
@ -31,13 +45,15 @@ jupyter = [
'ipykernel', 'ipykernel',
] ]
all_extra = api + plot + develop + jupyter all_extra = api + plot + develop + jupyter + hyperopt
setup(name='freqtrade', setup(name='freqtrade',
version=__version__, version=__version__,
description='Crypto Trading Bot', description='Crypto Trading Bot',
long_description=readme_long,
long_description_content_type="text/markdown",
url='https://github.com/freqtrade/freqtrade', url='https://github.com/freqtrade/freqtrade',
author='gcarq and contributors', author='Freqtrade Team',
author_email='michael.egger@tsn.at', author_email='michael.egger@tsn.at',
license='GPLv3', license='GPLv3',
packages=['freqtrade'], packages=['freqtrade'],
@ -45,7 +61,7 @@ setup(name='freqtrade',
tests_require=['pytest', 'pytest-mock', 'pytest-cov'], tests_require=['pytest', 'pytest-mock', 'pytest-cov'],
install_requires=[ install_requires=[
# from requirements-common.txt # from requirements-common.txt
'ccxt>=1.18', 'ccxt>=1.18.1080',
'SQLAlchemy', 'SQLAlchemy',
'python-telegram-bot', 'python-telegram-bot',
'arrow', 'arrow',
@ -53,14 +69,10 @@ setup(name='freqtrade',
'requests', 'requests',
'urllib3', 'urllib3',
'wrapt', 'wrapt',
'scikit-learn',
'joblib',
'jsonschema', 'jsonschema',
'TA-Lib', 'TA-Lib',
'tabulate', 'tabulate',
'coinmarketcap', 'coinmarketcap',
'scikit-optimize',
'filelock',
'py_find_1st', 'py_find_1st',
'python-rapidjson', 'python-rapidjson',
'sdnotify', 'sdnotify',
@ -68,15 +80,14 @@ setup(name='freqtrade',
# from requirements.txt # from requirements.txt
'numpy', 'numpy',
'pandas', 'pandas',
'scipy',
], ],
extras_require={ extras_require={
'api': api, 'api': api,
'dev': all_extra, 'dev': all_extra,
'plot': plot, 'plot': plot,
'all': all_extra,
'jupyter': jupyter, 'jupyter': jupyter,
'hyperopt': hyperopt,
'all': all_extra,
}, },
include_package_data=True, include_package_data=True,
zip_safe=False, zip_safe=False,

0
freqtrade/tests/__init__.py → tests/__init__.py
View File

0
freqtrade/tests/config_test_comments.json → tests/config_test_comments.json
View File

38
freqtrade/tests/conftest.py → tests/conftest.py
View File

@ -45,7 +45,7 @@ def log_has_re(line, logs):
def get_args(args): def get_args(args):
return Arguments(args, '').get_parsed_arg() return Arguments(args).get_parsed_arg()
def patched_configuration_load_config_file(mocker, config) -> None: def patched_configuration_load_config_file(mocker, config) -> None:
@ -117,7 +117,7 @@ def patch_freqtradebot(mocker, config) -> None:
""" """
mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock())
persistence.init(config['db_url']) persistence.init(config['db_url'])
patch_exchange(mocker, None) patch_exchange(mocker)
mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager._init', MagicMock())
mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock()) mocker.patch('freqtrade.freqtradebot.RPCManager.send_msg', MagicMock())
@ -182,7 +182,7 @@ def init_persistence(default_conf):
@pytest.fixture(scope="function") @pytest.fixture(scope="function")
def default_conf(): def default_conf(testdatadir):
""" Returns validated configuration suitable for most tests """ """ Returns validated configuration suitable for most tests """
configuration = { configuration = {
"max_open_trades": 1, "max_open_trades": 1,
@ -237,6 +237,7 @@ def default_conf():
"token": "token", "token": "token",
"chat_id": "0" "chat_id": "0"
}, },
"datadir": str(testdatadir),
"initial_state": "running", "initial_state": "running",
"db_url": "sqlite://", "db_url": "sqlite://",
"user_data_dir": Path("user_data"), "user_data_dir": Path("user_data"),
@ -890,8 +891,8 @@ def tickers():
@pytest.fixture @pytest.fixture
def result(): def result(testdatadir):
with Path('freqtrade/tests/testdata/UNITTEST_BTC-1m.json').open('r') as data_file: with (testdatadir / 'UNITTEST_BTC-1m.json').open('r') as data_file:
return parse_ticker_dataframe(json.load(data_file), '1m', pair="UNITTEST/BTC", return parse_ticker_dataframe(json.load(data_file), '1m', pair="UNITTEST/BTC",
fill_missing=True) fill_missing=True)
@ -1047,3 +1048,30 @@ def rpc_balance():
'used': 0.0 'used': 0.0
}, },
} }
@pytest.fixture
def testdatadir() -> Path:
"""Return the path where testdata files are stored"""
return (Path(__file__).parent / "testdata").resolve()
@pytest.fixture(scope="function")
def import_fails() -> None:
# Source of this test-method:
# https://stackoverflow.com/questions/2481511/mocking-importerror-in-python
import builtins
realimport = builtins.__import__
def mockedimport(name, *args, **kwargs):
if name in ["filelock"]:
raise ImportError(f"No module named '{name}'")
return realimport(name, *args, **kwargs)
builtins.__import__ = mockedimport
# Run test - then cleanup
yield
# restore previous importfunction
builtins.__import__ = realimport

0
freqtrade/tests/data/__init__.py → tests/data/__init__.py
View File

34
freqtrade/tests/data/test_btanalysis.py → tests/data/test_btanalysis.py
View File

@ -11,14 +11,13 @@ from freqtrade.data.btanalysis import (BT_DATA_COLUMNS,
extract_trades_of_period, extract_trades_of_period,
load_backtest_data, load_trades, load_backtest_data, load_trades,
load_trades_from_db) load_trades_from_db)
from freqtrade.data.history import (load_data, load_pair_history, from freqtrade.data.history import load_data, load_pair_history
make_testdata_path) from tests.test_persistence import create_mock_trades
from freqtrade.tests.test_persistence import create_mock_trades
def test_load_backtest_data(): def test_load_backtest_data(testdatadir):
filename = make_testdata_path(None) / "backtest-result_test.json" filename = testdatadir / "backtest-result_test.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
assert isinstance(bt_data, DataFrame) assert isinstance(bt_data, DataFrame)
assert list(bt_data.columns) == BT_DATA_COLUMNS + ["profitabs"] assert list(bt_data.columns) == BT_DATA_COLUMNS + ["profitabs"]
@ -52,12 +51,12 @@ def test_load_trades_db(default_conf, fee, mocker):
assert col in trades.columns assert col in trades.columns
def test_extract_trades_of_period(): def test_extract_trades_of_period(testdatadir):
pair = "UNITTEST/BTC" pair = "UNITTEST/BTC"
timerange = TimeRange(None, 'line', 0, -1000) timerange = TimeRange(None, 'line', 0, -1000)
data = load_pair_history(pair=pair, ticker_interval='1m', data = load_pair_history(pair=pair, ticker_interval='1m',
datadir=None, timerange=timerange) datadir=testdatadir, timerange=timerange)
# timerange = 2017-11-14 06:07 - 2017-11-14 22:58:00 # timerange = 2017-11-14 06:07 - 2017-11-14 22:58:00
trades = DataFrame( trades = DataFrame(
@ -89,25 +88,28 @@ def test_load_trades(default_conf, mocker):
db_mock = mocker.patch("freqtrade.data.btanalysis.load_trades_from_db", MagicMock()) db_mock = mocker.patch("freqtrade.data.btanalysis.load_trades_from_db", MagicMock())
bt_mock = mocker.patch("freqtrade.data.btanalysis.load_backtest_data", MagicMock()) bt_mock = mocker.patch("freqtrade.data.btanalysis.load_backtest_data", MagicMock())
default_conf['trade_source'] = "DB" load_trades("DB",
load_trades(default_conf) db_url=default_conf.get('db_url'),
exportfilename=default_conf.get('exportfilename'),
)
assert db_mock.call_count == 1 assert db_mock.call_count == 1
assert bt_mock.call_count == 0 assert bt_mock.call_count == 0
db_mock.reset_mock() db_mock.reset_mock()
bt_mock.reset_mock() bt_mock.reset_mock()
default_conf['trade_source'] = "file"
default_conf['exportfilename'] = "testfile.json" default_conf['exportfilename'] = "testfile.json"
load_trades(default_conf) load_trades("file",
db_url=default_conf.get('db_url'),
exportfilename=default_conf.get('exportfilename'),)
assert db_mock.call_count == 0 assert db_mock.call_count == 0
assert bt_mock.call_count == 1 assert bt_mock.call_count == 1
def test_combine_tickers_with_mean(): def test_combine_tickers_with_mean(testdatadir):
pairs = ["ETH/BTC", "XLM/BTC"] pairs = ["ETH/BTC", "XLM/BTC"]
tickers = load_data(datadir=None, tickers = load_data(datadir=testdatadir,
pairs=pairs, pairs=pairs,
ticker_interval='5m' ticker_interval='5m'
) )
@ -118,13 +120,13 @@ def test_combine_tickers_with_mean():
assert "mean" in df.columns assert "mean" in df.columns
def test_create_cum_profit(): def test_create_cum_profit(testdatadir):
filename = make_testdata_path(None) / "backtest-result_test.json" filename = testdatadir / "backtest-result_test.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
timerange = TimeRange.parse_timerange("20180110-20180112") timerange = TimeRange.parse_timerange("20180110-20180112")
df = load_pair_history(pair="POWR/BTC", ticker_interval='5m', df = load_pair_history(pair="POWR/BTC", ticker_interval='5m',
datadir=None, timerange=timerange) datadir=testdatadir, timerange=timerange)
cum_profits = create_cum_profit(df.set_index('date'), cum_profits = create_cum_profit(df.set_index('date'),
bt_data[bt_data["pair"] == 'POWR/BTC'], bt_data[bt_data["pair"] == 'POWR/BTC'],

7
freqtrade/tests/data/test_converter.py → tests/data/test_converter.py
View File

@ -3,7 +3,7 @@ import logging
from freqtrade.data.converter import parse_ticker_dataframe, ohlcv_fill_up_missing_data from freqtrade.data.converter import parse_ticker_dataframe, ohlcv_fill_up_missing_data
from freqtrade.data.history import load_pair_history, validate_backtest_data, get_timeframe from freqtrade.data.history import load_pair_history, validate_backtest_data, get_timeframe
from freqtrade.tests.conftest import log_has from tests.conftest import log_has
def test_dataframe_correct_columns(result): def test_dataframe_correct_columns(result):
@ -21,10 +21,9 @@ def test_parse_ticker_dataframe(ticker_history_list, caplog):
assert log_has('Parsing tickerlist to dataframe', caplog) assert log_has('Parsing tickerlist to dataframe', caplog)
def test_ohlcv_fill_up_missing_data(caplog): def test_ohlcv_fill_up_missing_data(testdatadir, caplog):
data = load_pair_history(datadir=None, data = load_pair_history(datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
refresh_pairs=False,
pair='UNITTEST/BTC', pair='UNITTEST/BTC',
fill_up_missing=False) fill_up_missing=False)
caplog.set_level(logging.DEBUG) caplog.set_level(logging.DEBUG)

4
freqtrade/tests/data/test_dataprovider.py → tests/data/test_dataprovider.py
View File

@ -4,7 +4,7 @@ from pandas import DataFrame
from freqtrade.data.dataprovider import DataProvider from freqtrade.data.dataprovider import DataProvider
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.tests.conftest import get_patched_exchange from tests.conftest import get_patched_exchange
def test_ohlcv(mocker, default_conf, ticker_history): def test_ohlcv(mocker, default_conf, ticker_history):
@ -45,8 +45,6 @@ def test_historic_ohlcv(mocker, default_conf, ticker_history):
data = dp.historic_ohlcv("UNITTEST/BTC", "5m") data = dp.historic_ohlcv("UNITTEST/BTC", "5m")
assert isinstance(data, DataFrame) assert isinstance(data, DataFrame)
assert historymock.call_count == 1 assert historymock.call_count == 1
assert historymock.call_args_list[0][1]["datadir"] is None
assert historymock.call_args_list[0][1]["refresh_pairs"] is False
assert historymock.call_args_list[0][1]["ticker_interval"] == "5m" assert historymock.call_args_list[0][1]["ticker_interval"] == "5m"

159
freqtrade/tests/data/test_history.py → tests/data/test_history.py
View File

@ -5,7 +5,7 @@ import os
import uuid import uuid
from pathlib import Path from pathlib import Path
from shutil import copyfile from shutil import copyfile
from unittest.mock import MagicMock from unittest.mock import MagicMock, PropertyMock
import arrow import arrow
import pytest import pytest
@ -16,13 +16,13 @@ from freqtrade.configuration import TimeRange
from freqtrade.data import history from freqtrade.data import history
from freqtrade.data.history import (download_pair_history, from freqtrade.data.history import (download_pair_history,
load_cached_data_for_updating, load_cached_data_for_updating,
load_tickerdata_file, make_testdata_path, load_tickerdata_file,
refresh_backtest_ohlcv_data,
trim_tickerlist) trim_tickerlist)
from freqtrade.exchange import timeframe_to_minutes from freqtrade.exchange import timeframe_to_minutes
from freqtrade.misc import file_dump_json from freqtrade.misc import file_dump_json
from freqtrade.strategy.default_strategy import DefaultStrategy from freqtrade.strategy.default_strategy import DefaultStrategy
from freqtrade.tests.conftest import (get_patched_exchange, log_has, from tests.conftest import get_patched_exchange, log_has, log_has_re, patch_exchange
patch_exchange)
# Change this if modifying UNITTEST/BTC testdatafile # Change this if modifying UNITTEST/BTC testdatafile
_BTC_UNITTEST_LENGTH = 13681 _BTC_UNITTEST_LENGTH = 13681
@ -59,8 +59,8 @@ def _clean_test_file(file: str) -> None:
os.rename(file_swp, file) os.rename(file_swp, file)
def test_load_data_30min_ticker(mocker, caplog, default_conf) -> None: def test_load_data_30min_ticker(mocker, caplog, default_conf, testdatadir) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='30m', datadir=None) ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='30m', datadir=testdatadir)
assert isinstance(ld, DataFrame) assert isinstance(ld, DataFrame)
assert not log_has( assert not log_has(
'Download history data for pair: "UNITTEST/BTC", interval: 30m ' 'Download history data for pair: "UNITTEST/BTC", interval: 30m '
@ -68,22 +68,21 @@ def test_load_data_30min_ticker(mocker, caplog, default_conf) -> None:
) )
def test_load_data_7min_ticker(mocker, caplog, default_conf) -> None: def test_load_data_7min_ticker(mocker, caplog, default_conf, testdatadir) -> None:
ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='7m', datadir=None) ld = history.load_pair_history(pair='UNITTEST/BTC', ticker_interval='7m', datadir=testdatadir)
assert not isinstance(ld, DataFrame) assert not isinstance(ld, DataFrame)
assert ld is None assert ld is None
assert log_has( assert log_has(
'No history data for pair: "UNITTEST/BTC", interval: 7m. ' 'No history data for pair: "UNITTEST/BTC", interval: 7m. '
'Use --refresh-pairs-cached option or `freqtrade download-data` ' 'Use `freqtrade download-data` to download the data', caplog
'script to download the data', caplog
) )
def test_load_data_1min_ticker(ticker_history, mocker, caplog) -> None: def test_load_data_1min_ticker(ticker_history, mocker, caplog, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history) mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history)
file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-1m.json') file = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'UNITTEST_BTC-1m.json')
_backup_file(file, copy_file=True) _backup_file(file, copy_file=True)
history.load_data(datadir=None, ticker_interval='1m', pairs=['UNITTEST/BTC']) history.load_data(datadir=testdatadir, ticker_interval='1m', pairs=['UNITTEST/BTC'])
assert os.path.isfile(file) is True assert os.path.isfile(file) is True
assert not log_has( assert not log_has(
'Download history data for pair: "UNITTEST/BTC", interval: 1m ' 'Download history data for pair: "UNITTEST/BTC", interval: 1m '
@ -92,7 +91,8 @@ def test_load_data_1min_ticker(ticker_history, mocker, caplog) -> None:
_clean_test_file(file) _clean_test_file(file)
def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, default_conf) -> None: def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog,
default_conf, testdatadir) -> None:
""" """
Test load_pair_history() with 1 min ticker Test load_pair_history() with 1 min ticker
""" """
@ -102,30 +102,28 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, defau
_backup_file(file) _backup_file(file)
# do not download a new pair if refresh_pairs isn't set # do not download a new pair if refresh_pairs isn't set
history.load_pair_history(datadir=None, history.load_pair_history(datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
refresh_pairs=False,
pair='MEME/BTC') pair='MEME/BTC')
assert os.path.isfile(file) is False assert os.path.isfile(file) is False
assert log_has( assert log_has(
'No history data for pair: "MEME/BTC", interval: 1m. ' 'No history data for pair: "MEME/BTC", interval: 1m. '
'Use --refresh-pairs-cached option or `freqtrade download-data` ' 'Use `freqtrade download-data` to download the data', caplog
'script to download the data', caplog
) )
# download a new pair if refresh_pairs is set # download a new pair if refresh_pairs is set
history.load_pair_history(datadir=None, history.load_pair_history(datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
refresh_pairs=True, refresh_pairs=True,
exchange=exchange, exchange=exchange,
pair='MEME/BTC') pair='MEME/BTC')
assert os.path.isfile(file) is True assert os.path.isfile(file) is True
assert log_has( assert log_has_re(
'Download history data for pair: "MEME/BTC", interval: 1m ' 'Download history data for pair: "MEME/BTC", interval: 1m '
'and store in None.', caplog 'and store in .*', caplog
) )
with pytest.raises(OperationalException, match=r'Exchange needs to be initialized when.*'): with pytest.raises(OperationalException, match=r'Exchange needs to be initialized when.*'):
history.load_pair_history(datadir=None, history.load_pair_history(datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
refresh_pairs=True, refresh_pairs=True,
exchange=None, exchange=None,
@ -133,33 +131,8 @@ def test_load_data_with_new_pair_1min(ticker_history_list, mocker, caplog, defau
_clean_test_file(file) _clean_test_file(file)
def test_load_data_live(default_conf, mocker, caplog) -> None: def test_testdata_path(testdatadir) -> None:
refresh_mock = MagicMock() assert str(Path('tests') / 'testdata') in str(testdatadir)
mocker.patch("freqtrade.exchange.Exchange.refresh_latest_ohlcv", refresh_mock)
exchange = get_patched_exchange(mocker, default_conf)
history.load_data(datadir=None, ticker_interval='5m',
pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
live=True,
exchange=exchange)
assert refresh_mock.call_count == 1
assert len(refresh_mock.call_args_list[0][0][0]) == 2
assert log_has('Live: Downloading data for all defined pairs ...', caplog)
def test_load_data_live_noexchange(default_conf, mocker, caplog) -> None:
with pytest.raises(OperationalException,
match=r'Exchange needs to be initialized when using live data.'):
history.load_data(datadir=None, ticker_interval='5m',
pairs=['UNITTEST/BTC', 'UNITTEST2/BTC'],
exchange=None,
live=True,
)
def test_testdata_path() -> None:
assert str(Path('freqtrade') / 'tests' / 'testdata') in str(make_testdata_path(None))
def test_load_cached_data_for_updating(mocker) -> None: def test_load_cached_data_for_updating(mocker) -> None:
@ -247,7 +220,7 @@ def test_load_cached_data_for_updating(mocker) -> None:
assert start_ts is None assert start_ts is None
def test_download_pair_history(ticker_history_list, mocker, default_conf) -> None: def test_download_pair_history(ticker_history_list, mocker, default_conf, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history_list) mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=ticker_history_list)
exchange = get_patched_exchange(mocker, default_conf) exchange = get_patched_exchange(mocker, default_conf)
file1_1 = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-1m.json') file1_1 = os.path.join(os.path.dirname(__file__), '..', 'testdata', 'MEME_BTC-1m.json')
@ -263,10 +236,10 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
assert os.path.isfile(file1_1) is False assert os.path.isfile(file1_1) is False
assert os.path.isfile(file2_1) is False assert os.path.isfile(file2_1) is False
assert download_pair_history(datadir=None, exchange=exchange, assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC', pair='MEME/BTC',
ticker_interval='1m') ticker_interval='1m')
assert download_pair_history(datadir=None, exchange=exchange, assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='CFI/BTC', pair='CFI/BTC',
ticker_interval='1m') ticker_interval='1m')
assert not exchange._pairs_last_refresh_time assert not exchange._pairs_last_refresh_time
@ -280,10 +253,10 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
assert os.path.isfile(file1_5) is False assert os.path.isfile(file1_5) is False
assert os.path.isfile(file2_5) is False assert os.path.isfile(file2_5) is False
assert download_pair_history(datadir=None, exchange=exchange, assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC', pair='MEME/BTC',
ticker_interval='5m') ticker_interval='5m')
assert download_pair_history(datadir=None, exchange=exchange, assert download_pair_history(datadir=testdatadir, exchange=exchange,
pair='CFI/BTC', pair='CFI/BTC',
ticker_interval='5m') ticker_interval='5m')
assert not exchange._pairs_last_refresh_time assert not exchange._pairs_last_refresh_time
@ -295,7 +268,7 @@ def test_download_pair_history(ticker_history_list, mocker, default_conf) -> Non
_clean_test_file(file2_5) _clean_test_file(file2_5)
def test_download_pair_history2(mocker, default_conf) -> None: def test_download_pair_history2(mocker, default_conf, testdatadir) -> None:
tick = [ tick = [
[1509836520000, 0.00162008, 0.00162008, 0.00162008, 0.00162008, 108.14853839], [1509836520000, 0.00162008, 0.00162008, 0.00162008, 0.00162008, 108.14853839],
[1509836580000, 0.00161, 0.00161, 0.00161, 0.00161, 82.390199] [1509836580000, 0.00161, 0.00161, 0.00161, 0.00161, 82.390199]
@ -303,12 +276,13 @@ def test_download_pair_history2(mocker, default_conf) -> None:
json_dump_mock = mocker.patch('freqtrade.misc.file_dump_json', return_value=None) json_dump_mock = mocker.patch('freqtrade.misc.file_dump_json', return_value=None)
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=tick) mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', return_value=tick)
exchange = get_patched_exchange(mocker, default_conf) exchange = get_patched_exchange(mocker, default_conf)
download_pair_history(None, exchange, pair="UNITTEST/BTC", ticker_interval='1m') download_pair_history(testdatadir, exchange, pair="UNITTEST/BTC", ticker_interval='1m')
download_pair_history(None, exchange, pair="UNITTEST/BTC", ticker_interval='3m') download_pair_history(testdatadir, exchange, pair="UNITTEST/BTC", ticker_interval='3m')
assert json_dump_mock.call_count == 2 assert json_dump_mock.call_count == 2
def test_download_backtesting_data_exception(ticker_history, mocker, caplog, default_conf) -> None: def test_download_backtesting_data_exception(ticker_history, mocker, caplog,
default_conf, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv', mocker.patch('freqtrade.exchange.Exchange.get_historic_ohlcv',
side_effect=Exception('File Error')) side_effect=Exception('File Error'))
@ -319,7 +293,7 @@ def test_download_backtesting_data_exception(ticker_history, mocker, caplog, def
_backup_file(file1_1) _backup_file(file1_1)
_backup_file(file1_5) _backup_file(file1_5)
assert not download_pair_history(datadir=None, exchange=exchange, assert not download_pair_history(datadir=testdatadir, exchange=exchange,
pair='MEME/BTC', pair='MEME/BTC',
ticker_interval='1m') ticker_interval='1m')
# clean files freshly downloaded # clean files freshly downloaded
@ -331,23 +305,22 @@ def test_download_backtesting_data_exception(ticker_history, mocker, caplog, def
) )
def test_load_tickerdata_file() -> None: def test_load_tickerdata_file(testdatadir) -> None:
# 7 does not exist in either format. # 7 does not exist in either format.
assert not load_tickerdata_file(None, 'UNITTEST/BTC', '7m') assert not load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '7m')
# 1 exists only as a .json # 1 exists only as a .json
tickerdata = load_tickerdata_file(None, 'UNITTEST/BTC', '1m') tickerdata = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
assert _BTC_UNITTEST_LENGTH == len(tickerdata) assert _BTC_UNITTEST_LENGTH == len(tickerdata)
# 8 .json is empty and will fail if it's loaded. .json.gz is a copy of 1.json # 8 .json is empty and will fail if it's loaded. .json.gz is a copy of 1.json
tickerdata = load_tickerdata_file(None, 'UNITTEST/BTC', '8m') tickerdata = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '8m')
assert _BTC_UNITTEST_LENGTH == len(tickerdata) assert _BTC_UNITTEST_LENGTH == len(tickerdata)
def test_load_partial_missing(caplog) -> None: def test_load_partial_missing(testdatadir, caplog) -> None:
# Make sure we start fresh - test missing data at start # Make sure we start fresh - test missing data at start
start = arrow.get('2018-01-01T00:00:00') start = arrow.get('2018-01-01T00:00:00')
end = arrow.get('2018-01-11T00:00:00') end = arrow.get('2018-01-11T00:00:00')
tickerdata = history.load_data(None, '5m', ['UNITTEST/BTC'], tickerdata = history.load_data(testdatadir, '5m', ['UNITTEST/BTC'],
refresh_pairs=False,
timerange=TimeRange('date', 'date', timerange=TimeRange('date', 'date',
start.timestamp, end.timestamp)) start.timestamp, end.timestamp))
# timedifference in 5 minutes # timedifference in 5 minutes
@ -361,8 +334,8 @@ def test_load_partial_missing(caplog) -> None:
caplog.clear() caplog.clear()
start = arrow.get('2018-01-10T00:00:00') start = arrow.get('2018-01-10T00:00:00')
end = arrow.get('2018-02-20T00:00:00') end = arrow.get('2018-02-20T00:00:00')
tickerdata = history.load_data(datadir=None, ticker_interval='5m', tickerdata = history.load_data(datadir=testdatadir, ticker_interval='5m',
pairs=['UNITTEST/BTC'], refresh_pairs=False, pairs=['UNITTEST/BTC'],
timerange=TimeRange('date', 'date', timerange=TimeRange('date', 'date',
start.timestamp, end.timestamp)) start.timestamp, end.timestamp))
# timedifference in 5 minutes # timedifference in 5 minutes
@ -501,13 +474,13 @@ def test_file_dump_json_tofile() -> None:
_clean_test_file(file) _clean_test_file(file)
def test_get_timeframe(default_conf, mocker) -> None: def test_get_timeframe(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
strategy = DefaultStrategy(default_conf) strategy = DefaultStrategy(default_conf)
data = strategy.tickerdata_to_dataframe( data = strategy.tickerdata_to_dataframe(
history.load_data( history.load_data(
datadir=None, datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
pairs=['UNITTEST/BTC'] pairs=['UNITTEST/BTC']
) )
@ -517,13 +490,13 @@ def test_get_timeframe(default_conf, mocker) -> None:
assert max_date.isoformat() == '2017-11-14T22:58:00+00:00' assert max_date.isoformat() == '2017-11-14T22:58:00+00:00'
def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None: def test_validate_backtest_data_warn(default_conf, mocker, caplog, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
strategy = DefaultStrategy(default_conf) strategy = DefaultStrategy(default_conf)
data = strategy.tickerdata_to_dataframe( data = strategy.tickerdata_to_dataframe(
history.load_data( history.load_data(
datadir=None, datadir=testdatadir,
ticker_interval='1m', ticker_interval='1m',
pairs=['UNITTEST/BTC'], pairs=['UNITTEST/BTC'],
fill_up_missing=False fill_up_missing=False
@ -539,14 +512,14 @@ def test_validate_backtest_data_warn(default_conf, mocker, caplog) -> None:
caplog) caplog)
def test_validate_backtest_data(default_conf, mocker, caplog) -> None: def test_validate_backtest_data(default_conf, mocker, caplog, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
strategy = DefaultStrategy(default_conf) strategy = DefaultStrategy(default_conf)
timerange = TimeRange('index', 'index', 200, 250) timerange = TimeRange('index', 'index', 200, 250)
data = strategy.tickerdata_to_dataframe( data = strategy.tickerdata_to_dataframe(
history.load_data( history.load_data(
datadir=None, datadir=testdatadir,
ticker_interval='5m', ticker_interval='5m',
pairs=['UNITTEST/BTC'], pairs=['UNITTEST/BTC'],
timerange=timerange timerange=timerange
@ -558,3 +531,43 @@ def test_validate_backtest_data(default_conf, mocker, caplog) -> None:
assert not history.validate_backtest_data(data['UNITTEST/BTC'], 'UNITTEST/BTC', assert not history.validate_backtest_data(data['UNITTEST/BTC'], 'UNITTEST/BTC',
min_date, max_date, timeframe_to_minutes('5m')) min_date, max_date, timeframe_to_minutes('5m'))
assert len(caplog.record_tuples) == 0 assert len(caplog.record_tuples) == 0
def test_refresh_backtest_ohlcv_data(mocker, default_conf, markets, caplog, testdatadir):
dl_mock = mocker.patch('freqtrade.data.history.download_pair_history', MagicMock())
mocker.patch(
'freqtrade.exchange.Exchange.markets', PropertyMock(return_value=markets)
)
mocker.patch.object(Path, "exists", MagicMock(return_value=True))
mocker.patch.object(Path, "unlink", MagicMock())
ex = get_patched_exchange(mocker, default_conf)
timerange = TimeRange.parse_timerange("20190101-20190102")
refresh_backtest_ohlcv_data(exchange=ex, pairs=["ETH/BTC", "XRP/BTC"],
timeframes=["1m", "5m"], dl_path=testdatadir,
timerange=timerange, erase=True
)
assert dl_mock.call_count == 4
assert dl_mock.call_args[1]['timerange'].starttype == 'date'
assert log_has("Downloading pair ETH/BTC, interval 1m.", caplog)
def test_download_data_no_markets(mocker, default_conf, caplog, testdatadir):
dl_mock = mocker.patch('freqtrade.data.history.download_pair_history', MagicMock())
mocker.patch(
'freqtrade.exchange.Exchange.markets', PropertyMock(return_value={})
)
ex = get_patched_exchange(mocker, default_conf)
timerange = TimeRange.parse_timerange("20190101-20190102")
unav_pairs = refresh_backtest_ohlcv_data(exchange=ex, pairs=["ETH/BTC", "XRP/BTC"],
timeframes=["1m", "5m"],
dl_path=testdatadir,
timerange=timerange, erase=False
)
assert dl_mock.call_count == 0
assert "ETH/BTC" in unav_pairs
assert "XRP/BTC" in unav_pairs
assert log_has("Skipping pair ETH/BTC...", caplog)

0
freqtrade/tests/edge/__init__.py → tests/edge/__init__.py
View File

10
freqtrade/tests/edge/test_edge.py → tests/edge/test_edge.py
View File

@ -14,9 +14,8 @@ from freqtrade import OperationalException
from freqtrade.data.converter import parse_ticker_dataframe from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.edge import Edge, PairInfo from freqtrade.edge import Edge, PairInfo
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import get_patched_freqtradebot, log_has from tests.conftest import get_patched_freqtradebot, log_has
from freqtrade.tests.optimize import (BTContainer, BTrade, from tests.optimize import (BTContainer, BTrade, _build_backtest_dataframe,
_build_backtest_dataframe,
_get_frame_time_from_offset) _get_frame_time_from_offset)
# Cases to be tested: # Cases to be tested:
@ -156,8 +155,6 @@ def test_edge_results(edge_conf, mocker, caplog, data) -> None:
trades = edge._find_trades_for_stoploss_range(frame, 'TEST/BTC', [data.stop_loss]) trades = edge._find_trades_for_stoploss_range(frame, 'TEST/BTC', [data.stop_loss])
results = edge._fill_calculable_fields(DataFrame(trades)) if trades else DataFrame() results = edge._fill_calculable_fields(DataFrame(trades)) if trades else DataFrame()
print(results)
assert len(trades) == len(data.trades) assert len(trades) == len(data.trades)
if not results.empty: if not results.empty:
@ -291,7 +288,6 @@ def mocked_load_data(datadir, pairs=[], ticker_interval='0m', refresh_pairs=Fals
def test_edge_process_downloaded_data(mocker, edge_conf): def test_edge_process_downloaded_data(mocker, edge_conf):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf) freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001)) mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', mocked_load_data) mocker.patch('freqtrade.data.history.load_data', mocked_load_data)
@ -303,7 +299,6 @@ def test_edge_process_downloaded_data(mocker, edge_conf):
def test_edge_process_no_data(mocker, edge_conf, caplog): def test_edge_process_no_data(mocker, edge_conf, caplog):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf) freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001)) mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', MagicMock(return_value={})) mocker.patch('freqtrade.data.history.load_data', MagicMock(return_value={}))
@ -316,7 +311,6 @@ def test_edge_process_no_data(mocker, edge_conf, caplog):
def test_edge_process_no_trades(mocker, edge_conf, caplog): def test_edge_process_no_trades(mocker, edge_conf, caplog):
edge_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, edge_conf) freqtrade = get_patched_freqtradebot(mocker, edge_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001)) mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.data.history.load_data', mocked_load_data) mocker.patch('freqtrade.data.history.load_data', mocked_load_data)

0
freqtrade/tests/exchange/__init__.py → tests/exchange/__init__.py
View File

92
tests/exchange/test_binance.py Normal file
View File

@ -0,0 +1,92 @@
from random import randint
from unittest.mock import MagicMock
import ccxt
import pytest
from freqtrade import (DependencyException, InvalidOrderException,
OperationalException, TemporaryError)
from tests.conftest import get_patched_exchange
def test_stoploss_limit_order(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'stop_loss_limit'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'stopPrice': 220}
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds("0 balance"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(InvalidOrderException):
api_mock.create_order = MagicMock(
side_effect=ccxt.InvalidOrder("binance Order would trigger immediately."))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError("No connection"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(OperationalException, match=r".*DeadBeef.*"):
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError("DeadBeef"))
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
def test_stoploss_limit_order_dry_run(default_conf, mocker):
api_mock = MagicMock()
order_type = 'stop_loss_limit'
default_conf['dry_run'] = True
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert 'type' in order
assert order['type'] == order_type
assert order['price'] == 220
assert order['amount'] == 1

112
freqtrade/tests/exchange/test_exchange.py → tests/exchange/test_exchange.py
View File

@ -20,7 +20,7 @@ from freqtrade.exchange.exchange import (API_RETRY_COUNT, timeframe_to_minutes,
timeframe_to_prev_date, timeframe_to_prev_date,
timeframe_to_seconds) timeframe_to_seconds)
from freqtrade.resolvers.exchange_resolver import ExchangeResolver from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from freqtrade.tests.conftest import get_patched_exchange, log_has, log_has_re from tests.conftest import get_patched_exchange, log_has, log_has_re
# Make sure to always keep one exchange here which is NOT subclassed!! # Make sure to always keep one exchange here which is NOT subclassed!!
EXCHANGES = ['bittrex', 'binance', 'kraken', ] EXCHANGES = ['bittrex', 'binance', 'kraken', ]
@ -101,18 +101,21 @@ def test_destroy(default_conf, mocker, caplog):
def test_init_exception(default_conf, mocker): def test_init_exception(default_conf, mocker):
default_conf['exchange']['name'] = 'wrong_exchange_name' default_conf['exchange']['name'] = 'wrong_exchange_name'
with pytest.raises( with pytest.raises(OperationalException,
OperationalException, match=f"Exchange {default_conf['exchange']['name']} is not supported"):
match='Exchange {} is not supported'.format(default_conf['exchange']['name'])):
Exchange(default_conf) Exchange(default_conf)
default_conf['exchange']['name'] = 'binance' default_conf['exchange']['name'] = 'binance'
with pytest.raises( with pytest.raises(OperationalException,
OperationalException, match=f"Exchange {default_conf['exchange']['name']} is not supported"):
match='Exchange {} is not supported'.format(default_conf['exchange']['name'])):
mocker.patch("ccxt.binance", MagicMock(side_effect=AttributeError)) mocker.patch("ccxt.binance", MagicMock(side_effect=AttributeError))
Exchange(default_conf) Exchange(default_conf)
with pytest.raises(OperationalException,
match=r"Initialization of ccxt failed. Reason: DeadBeef"):
mocker.patch("ccxt.binance", MagicMock(side_effect=ccxt.BaseError("DeadBeef")))
Exchange(default_conf)
def test_exchange_resolver(default_conf, mocker, caplog): def test_exchange_resolver(default_conf, mocker, caplog):
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=MagicMock())) mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=MagicMock()))
@ -864,7 +867,7 @@ def test_get_balance_dry_run(default_conf, mocker):
@pytest.mark.parametrize("exchange_name", EXCHANGES) @pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_get_balance_prod(default_conf, mocker, exchange_name): def test_get_balance_prod(default_conf, mocker, exchange_name):
api_mock = MagicMock() api_mock = MagicMock()
api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4}}) api_mock.fetch_balance = MagicMock(return_value={'BTC': {'free': 123.4, 'total': 123.4}})
default_conf['dry_run'] = False default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name) exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
@ -880,6 +883,7 @@ def test_get_balance_prod(default_conf, mocker, exchange_name):
with pytest.raises(TemporaryError, match=r'.*balance due to malformed exchange response:.*'): with pytest.raises(TemporaryError, match=r'.*balance due to malformed exchange response:.*'):
exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name) exchange = get_patched_exchange(mocker, default_conf, api_mock, id=exchange_name)
mocker.patch('freqtrade.exchange.Exchange.get_balances', MagicMock(return_value={})) mocker.patch('freqtrade.exchange.Exchange.get_balances', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Kraken.get_balances', MagicMock(return_value={}))
exchange.get_balance(currency='BTC') exchange.get_balance(currency='BTC')
@ -1032,7 +1036,6 @@ def test_get_historic_ohlcv(default_conf, mocker, caplog, exchange_name):
exchange._async_get_candle_history = Mock(wraps=mock_candle_hist) exchange._async_get_candle_history = Mock(wraps=mock_candle_hist)
# one_call calculation * 1.8 should do 2 calls # one_call calculation * 1.8 should do 2 calls
since = 5 * 60 * 500 * 1.8 since = 5 * 60 * 500 * 1.8
print(f"since = {since}")
ret = exchange.get_historic_ohlcv(pair, "5m", int((arrow.utcnow().timestamp - since) * 1000)) ret = exchange.get_historic_ohlcv(pair, "5m", int((arrow.utcnow().timestamp - since) * 1000))
assert exchange._async_get_candle_history.call_count == 2 assert exchange._async_get_candle_history.call_count == 2
@ -1337,7 +1340,6 @@ def test_get_order(default_conf, mocker, exchange_name):
order.myid = 123 order.myid = 123
exchange = get_patched_exchange(mocker, default_conf, id=exchange_name) exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
exchange._dry_run_open_orders['X'] = order exchange._dry_run_open_orders['X'] = order
print(exchange.get_order('X', 'TKN/BTC'))
assert exchange.get_order('X', 'TKN/BTC').myid == 123 assert exchange.get_order('X', 'TKN/BTC').myid == 123
with pytest.raises(InvalidOrderException, match=r'Tried to get an invalid dry-run-order.*'): with pytest.raises(InvalidOrderException, match=r'Tried to get an invalid dry-run-order.*'):
@ -1362,9 +1364,7 @@ def test_get_order(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES) @pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_name(default_conf, mocker, exchange_name): def test_name(default_conf, mocker, exchange_name):
mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={})) exchange = get_patched_exchange(mocker, default_conf, id=exchange_name)
default_conf['exchange']['name'] = exchange_name
exchange = Exchange(default_conf)
assert exchange.name == exchange_name.title() assert exchange.name == exchange_name.title()
assert exchange.id == exchange_name assert exchange.id == exchange_name
@ -1436,87 +1436,11 @@ def test_get_fee(default_conf, mocker, exchange_name):
'get_fee', 'calculate_fee') 'get_fee', 'calculate_fee')
def test_stoploss_limit_order(default_conf, mocker): def test_stoploss_limit_order_unsupported_exchange(default_conf, mocker):
api_mock = MagicMock() exchange = get_patched_exchange(mocker, default_conf, 'bittrex')
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6)) with pytest.raises(OperationalException, match=r"stoploss_limit is not implemented .*"):
order_type = 'stop_loss_limit'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'stopPrice': 220}
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds("0 balance"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200) exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(DependencyException):
api_mock.create_order = MagicMock(side_effect=ccxt.InvalidOrder("Order not found"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError("No connection"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
with pytest.raises(OperationalException):
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError("DeadBeef"))
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
def test_stoploss_limit_order_dry_run(default_conf, mocker):
api_mock = MagicMock()
order_type = 'stop_loss_limit'
default_conf['dry_run'] = True
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
with pytest.raises(OperationalException):
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=190, rate=200)
api_mock.create_order.reset_mock()
order = exchange.stoploss_limit(pair='ETH/BTC', amount=1, stop_price=220, rate=200)
assert 'id' in order
assert 'info' in order
assert 'type' in order
assert order['type'] == order_type
assert order['price'] == 220
assert order['amount'] == 1
def test_merge_ft_has_dict(default_conf, mocker): def test_merge_ft_has_dict(default_conf, mocker):
mocker.patch.multiple('freqtrade.exchange.Exchange', mocker.patch.multiple('freqtrade.exchange.Exchange',
@ -1604,7 +1528,7 @@ def test_timeframe_to_prev_date():
assert timeframe_to_prev_date(interval, date) == result assert timeframe_to_prev_date(interval, date) == result
date = datetime.now(tz=timezone.utc) date = datetime.now(tz=timezone.utc)
assert timeframe_to_prev_date("5m", date) < date assert timeframe_to_prev_date("5m") < date
def test_timeframe_to_next_date(): def test_timeframe_to_next_date():
@ -1629,4 +1553,4 @@ def test_timeframe_to_next_date():
assert timeframe_to_next_date(interval, date) == result assert timeframe_to_next_date(interval, date) == result
date = datetime.now(tz=timezone.utc) date = datetime.now(tz=timezone.utc)
assert timeframe_to_next_date("5m", date) > date assert timeframe_to_next_date("5m") > date

151
tests/exchange/test_kraken.py Normal file
View File

@ -0,0 +1,151 @@
# pragma pylint: disable=missing-docstring, C0103, bad-continuation, global-statement
# pragma pylint: disable=protected-access
from random import randint
from unittest.mock import MagicMock
from tests.conftest import get_patched_exchange
from tests.exchange.test_exchange import ccxt_exceptionhandlers
def test_buy_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'limit'
time_in_force = 'ioc'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.buy(pair='ETH/BTC', ordertype=order_type,
amount=1, rate=200, time_in_force=time_in_force)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
assert api_mock.create_order.call_args[0][5] == {'timeInForce': 'ioc',
'trading_agreement': 'agree'}
def test_sell_kraken_trading_agreement(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
order_type = 'market'
api_mock.options = {}
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
assert api_mock.create_order.call_args[0][5] == {'trading_agreement': 'agree'}
def test_get_balances_prod(default_conf, mocker):
balance_item = {
'free': None,
'total': 10.0,
'used': 0.0
}
api_mock = MagicMock()
api_mock.fetch_balance = MagicMock(return_value={
'1ST': balance_item.copy(),
'2ST': balance_item.copy(),
'3ST': balance_item.copy(),
'4ST': balance_item.copy(),
})
kraken_open_orders = [{'symbol': '1ST/EUR',
'type': 'limit',
'side': 'sell',
'price': 20,
'cost': 0.0,
'amount': 1.0,
'filled': 0.0,
'average': 0.0,
'remaining': 1.0,
},
{'status': 'open',
'symbol': '2ST/EUR',
'type': 'limit',
'side': 'sell',
'price': 20.0,
'cost': 0.0,
'amount': 2.0,
'filled': 0.0,
'average': 0.0,
'remaining': 2.0,
},
{'status': 'open',
'symbol': '2ST/USD',
'type': 'limit',
'side': 'sell',
'price': 20.0,
'cost': 0.0,
'amount': 2.0,
'filled': 0.0,
'average': 0.0,
'remaining': 2.0,
},
{'status': 'open',
'symbol': 'BTC/3ST',
'type': 'limit',
'side': 'buy',
'price': 20,
'cost': 0.0,
'amount': 3.0,
'filled': 0.0,
'average': 0.0,
'remaining': 3.0,
}]
api_mock.fetch_open_orders = MagicMock(return_value=kraken_open_orders)
default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
balances = exchange.get_balances()
assert len(balances) == 4
assert balances['1ST']['free'] == 9.0
assert balances['1ST']['total'] == 10.0
assert balances['1ST']['used'] == 1.0
assert balances['2ST']['free'] == 6.0
assert balances['2ST']['total'] == 10.0
assert balances['2ST']['used'] == 4.0
assert balances['3ST']['free'] == 7.0
assert balances['3ST']['total'] == 10.0
assert balances['3ST']['used'] == 3.0
assert balances['4ST']['free'] == 10.0
assert balances['4ST']['total'] == 10.0
assert balances['4ST']['used'] == 0.0
ccxt_exceptionhandlers(mocker, default_conf, api_mock, "kraken",
"get_balances", "fetch_balance")

0
freqtrade/tests/optimize/__init__.py → tests/optimize/__init__.py
View File

13
freqtrade/tests/optimize/test_backtest_detail.py → tests/optimize/test_backtest_detail.py
View File

@ -8,11 +8,9 @@ from pandas import DataFrame
from freqtrade.data.history import get_timeframe from freqtrade.data.history import get_timeframe
from freqtrade.optimize.backtesting import Backtesting from freqtrade.optimize.backtesting import Backtesting
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import patch_exchange from tests.conftest import patch_exchange
from freqtrade.tests.optimize import (BTContainer, BTrade, from tests.optimize import (BTContainer, BTrade, _build_backtest_dataframe,
_build_backtest_dataframe, _get_frame_time_from_offset, tests_ticker_interval)
_get_frame_time_from_offset,
tests_ticker_interval)
# Test 0: Sell with signal sell in candle 3 # Test 0: Sell with signal sell in candle 3
# Test with Stop-loss at 1% # Test with Stop-loss at 1%
@ -293,8 +291,8 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
patch_exchange(mocker) patch_exchange(mocker)
frame = _build_backtest_dataframe(data.data) frame = _build_backtest_dataframe(data.data)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
backtesting.advise_buy = lambda a, m: frame backtesting.strategy.advise_buy = lambda a, m: frame
backtesting.advise_sell = lambda a, m: frame backtesting.strategy.advise_sell = lambda a, m: frame
caplog.set_level(logging.DEBUG) caplog.set_level(logging.DEBUG)
pair = "UNITTEST/BTC" pair = "UNITTEST/BTC"
@ -310,7 +308,6 @@ def test_backtest_results(default_conf, fee, mocker, caplog, data) -> None:
'end_date': max_date, 'end_date': max_date,
} }
) )
print(results.T)
assert len(results) == len(data.trades) assert len(results) == len(data.trades)
assert round(results["profit_percent"].sum(), 3) == round(data.profit_perc, 3) assert round(results["profit_percent"].sum(), 3) == round(data.profit_perc, 3)

109
freqtrade/tests/optimize/test_backtesting.py → tests/optimize/test_backtesting.py
View File

@ -22,8 +22,7 @@ from freqtrade.optimize.backtesting import Backtesting
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.strategy.default_strategy import DefaultStrategy from freqtrade.strategy.default_strategy import DefaultStrategy
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_args, log_has, log_has_re, from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patch_exchange,
patched_configuration_load_config_file) patched_configuration_load_config_file)
@ -34,9 +33,9 @@ def trim_dictlist(dict_list, num):
return new return new
def load_data_test(what): def load_data_test(what, testdatadir):
timerange = TimeRange(None, 'line', 0, -101) timerange = TimeRange(None, 'line', 0, -101)
pair = history.load_tickerdata_file(None, ticker_interval='1m', pair = history.load_tickerdata_file(testdatadir, ticker_interval='1m',
pair='UNITTEST/BTC', timerange=timerange) pair='UNITTEST/BTC', timerange=timerange)
datalen = len(pair) datalen = len(pair)
@ -79,12 +78,12 @@ def load_data_test(what):
fill_missing=True)} fill_missing=True)}
def simple_backtest(config, contour, num_results, mocker) -> None: def simple_backtest(config, contour, num_results, mocker, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
config['ticker_interval'] = '1m' config['ticker_interval'] = '1m'
backtesting = Backtesting(config) backtesting = Backtesting(config)
data = load_data_test(contour) data = load_data_test(contour, testdatadir)
processed = backtesting.strategy.tickerdata_to_dataframe(data) processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(processed) min_date, max_date = get_timeframe(processed)
assert isinstance(processed, dict) assert isinstance(processed, dict)
@ -118,8 +117,8 @@ def _load_pair_as_ticks(pair, tickfreq):
# FIX: fixturize this? # FIX: fixturize this?
def _make_backtest_conf(mocker, conf=None, pair='UNITTEST/BTC', record=None): def _make_backtest_conf(mocker, datadir, conf=None, pair='UNITTEST/BTC', record=None):
data = history.load_data(datadir=None, ticker_interval='1m', pairs=[pair]) data = history.load_data(datadir=datadir, ticker_interval='1m', pairs=[pair])
data = trim_dictlist(data, -201) data = trim_dictlist(data, -201)
patch_exchange(mocker) patch_exchange(mocker)
backtesting = Backtesting(conf) backtesting = Backtesting(conf)
@ -189,16 +188,12 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert 'position_stacking' not in config assert 'position_stacking' not in config
assert not log_has('Parameter --enable-position-stacking detected ...', caplog) assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config assert 'timerange' not in config
assert 'export' not in config assert 'export' not in config
assert 'runmode' in config assert 'runmode' in config
assert config['runmode'] == RunMode.BACKTEST assert config['runmode'] == RunMode.BACKTEST
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None: def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
mocker.patch( mocker.patch(
@ -214,7 +209,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
'--ticker-interval', '1m', '--ticker-interval', '1m',
'--enable-position-stacking', '--enable-position-stacking',
'--disable-max-market-positions', '--disable-max-market-positions',
'--refresh-pairs-cached',
'--timerange', ':100', '--timerange', ':100',
'--export', '/bar/foo', '--export', '/bar/foo',
'--export-filename', 'foo_bar.json' '--export-filename', 'foo_bar.json'
@ -241,9 +235,6 @@ def test_setup_bt_configuration_with_arguments(mocker, default_conf, caplog) ->
assert log_has('Parameter --disable-max-market-positions detected ...', caplog) assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
assert log_has('max_open_trades set to unlimited ...', caplog) assert log_has('max_open_trades set to unlimited ...', caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog) assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
@ -314,8 +305,8 @@ def test_backtesting_init(mocker, default_conf, order_types) -> None:
assert backtesting.config == default_conf assert backtesting.config == default_conf
assert backtesting.ticker_interval == '5m' assert backtesting.ticker_interval == '5m'
assert callable(backtesting.strategy.tickerdata_to_dataframe) assert callable(backtesting.strategy.tickerdata_to_dataframe)
assert callable(backtesting.advise_buy) assert callable(backtesting.strategy.advise_buy)
assert callable(backtesting.advise_sell) assert callable(backtesting.strategy.advise_sell)
assert isinstance(backtesting.strategy.dp, DataProvider) assert isinstance(backtesting.strategy.dp, DataProvider)
get_fee.assert_called() get_fee.assert_called()
assert backtesting.fee == 0.5 assert backtesting.fee == 0.5
@ -330,7 +321,7 @@ def test_backtesting_init_no_ticker_interval(mocker, default_conf, caplog) -> No
patch_exchange(mocker) patch_exchange(mocker)
del default_conf['ticker_interval'] del default_conf['ticker_interval']
default_conf['strategy_list'] = ['DefaultStrategy', default_conf['strategy_list'] = ['DefaultStrategy',
'TestStrategy'] 'SampleStrategy']
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.5)) mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.5))
with pytest.raises(OperationalException): with pytest.raises(OperationalException):
@ -339,10 +330,10 @@ def test_backtesting_init_no_ticker_interval(mocker, default_conf, caplog) -> No
"or as cli argument `--ticker-interval 5m`", caplog) "or as cli argument `--ticker-interval 5m`", caplog)
def test_tickerdata_to_dataframe_bt(default_conf, mocker) -> None: def test_tickerdata_to_dataframe_bt(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
timerange = TimeRange(None, 'line', 0, -100) timerange = TimeRange(None, 'line', 0, -100)
tick = history.load_tickerdata_file(None, 'UNITTEST/BTC', '1m', timerange=timerange) tick = history.load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m', timerange=timerange)
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC", tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)} fill_missing=True)}
@ -452,11 +443,10 @@ def test_generate_text_table_strategyn(default_conf, mocker):
'| LTC/BTC | 3 | 30.00 | 90.00 ' '| LTC/BTC | 3 | 30.00 | 90.00 '
'| 1.30000000 | 45.00 | 0:20:00 | 3 | 0 |' '| 1.30000000 | 45.00 | 0:20:00 | 3 | 0 |'
) )
print(backtesting._generate_text_table_strategy(all_results=results))
assert backtesting._generate_text_table_strategy(all_results=results) == result_str assert backtesting._generate_text_table_strategy(all_results=results) == result_str
def test_backtesting_start(default_conf, mocker, caplog) -> None: def test_backtesting_start(default_conf, mocker, testdatadir, caplog) -> None:
def get_timeframe(input1): def get_timeframe(input1):
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59) return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
@ -472,7 +462,7 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
default_conf['ticker_interval'] = '1m' default_conf['ticker_interval'] = '1m'
default_conf['datadir'] = None default_conf['datadir'] = testdatadir
default_conf['export'] = None default_conf['export'] = None
default_conf['timerange'] = '-100' default_conf['timerange'] = '-100'
@ -489,7 +479,7 @@ def test_backtesting_start(default_conf, mocker, caplog) -> None:
assert log_has(line, caplog) assert log_has(line, caplog)
def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None: def test_backtesting_start_no_data(default_conf, mocker, caplog, testdatadir) -> None:
def get_timeframe(input1): def get_timeframe(input1):
return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59) return Arrow(2017, 11, 14, 21, 17), Arrow(2017, 11, 14, 22, 59)
@ -505,7 +495,7 @@ def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
default_conf['ticker_interval'] = "1m" default_conf['ticker_interval'] = "1m"
default_conf['datadir'] = None default_conf['datadir'] = testdatadir
default_conf['export'] = None default_conf['export'] = None
default_conf['timerange'] = '20180101-20180102' default_conf['timerange'] = '20180101-20180102'
@ -516,13 +506,13 @@ def test_backtesting_start_no_data(default_conf, mocker, caplog) -> None:
assert log_has('No data found. Terminating.', caplog) assert log_has('No data found. Terminating.', caplog)
def test_backtest(default_conf, fee, mocker) -> None: def test_backtest(default_conf, fee, mocker, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker) patch_exchange(mocker)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
pair = 'UNITTEST/BTC' pair = 'UNITTEST/BTC'
timerange = TimeRange(None, 'line', 0, -201) timerange = TimeRange(None, 'line', 0, -201)
data = history.load_data(datadir=None, ticker_interval='5m', pairs=['UNITTEST/BTC'], data = history.load_data(datadir=testdatadir, ticker_interval='5m', pairs=['UNITTEST/BTC'],
timerange=timerange) timerange=timerange)
data_processed = backtesting.strategy.tickerdata_to_dataframe(data) data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(data_processed) min_date, max_date = get_timeframe(data_processed)
@ -570,14 +560,14 @@ def test_backtest(default_conf, fee, mocker) -> None:
t["close_rate"], 6) < round(ln.iloc[0]["high"], 6)) t["close_rate"], 6) < round(ln.iloc[0]["high"], 6))
def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None: def test_backtest_1min_ticker_interval(default_conf, fee, mocker, testdatadir) -> None:
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker) patch_exchange(mocker)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
# Run a backtesting for an exiting 1min ticker_interval # Run a backtesting for an exiting 1min ticker_interval
timerange = TimeRange(None, 'line', 0, -200) timerange = TimeRange(None, 'line', 0, -200)
data = history.load_data(datadir=None, ticker_interval='1m', pairs=['UNITTEST/BTC'], data = history.load_data(datadir=testdatadir, ticker_interval='1m', pairs=['UNITTEST/BTC'],
timerange=timerange) timerange=timerange)
processed = backtesting.strategy.tickerdata_to_dataframe(data) processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(processed) min_date, max_date = get_timeframe(processed)
@ -595,21 +585,21 @@ def test_backtest_1min_ticker_interval(default_conf, fee, mocker) -> None:
assert len(results) == 1 assert len(results) == 1
def test_processed(default_conf, mocker) -> None: def test_processed(default_conf, mocker, testdatadir) -> None:
patch_exchange(mocker) patch_exchange(mocker)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
dict_of_tickerrows = load_data_test('raise') dict_of_tickerrows = load_data_test('raise', testdatadir)
dataframes = backtesting.strategy.tickerdata_to_dataframe(dict_of_tickerrows) dataframes = backtesting.strategy.tickerdata_to_dataframe(dict_of_tickerrows)
dataframe = dataframes['UNITTEST/BTC'] dataframe = dataframes['UNITTEST/BTC']
cols = dataframe.columns cols = dataframe.columns
# assert the dataframe got some of the indicator columns # assert the dataframe got some of the indicator columns
for col in ['close', 'high', 'low', 'open', 'date', for col in ['close', 'high', 'low', 'open', 'date',
'ema50', 'ao', 'macd', 'plus_dm']: 'ema10', 'rsi', 'fastd', 'plus_di']:
assert col in cols assert col in cols
def test_backtest_pricecontours(default_conf, fee, mocker) -> None: def test_backtest_pricecontours(default_conf, fee, mocker, testdatadir) -> None:
# TODO: Evaluate usefullness of this, the patterns and buy-signls are unrealistic # TODO: Evaluate usefullness of this, the patterns and buy-signls are unrealistic
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
tests = [['raise', 19], ['lower', 0], ['sine', 35]] tests = [['raise', 19], ['lower', 0], ['sine', 35]]
@ -617,49 +607,50 @@ def test_backtest_pricecontours(default_conf, fee, mocker) -> None:
default_conf['experimental'] = {"use_sell_signal": True} default_conf['experimental'] = {"use_sell_signal": True}
for [contour, numres] in tests: for [contour, numres] in tests:
simple_backtest(default_conf, contour, numres, mocker) simple_backtest(default_conf, contour, numres, mocker, testdatadir)
def test_backtest_clash_buy_sell(mocker, default_conf): def test_backtest_clash_buy_sell(mocker, default_conf, testdatadir):
# Override the default buy trend function in our default_strategy # Override the default buy trend function in our default_strategy
def fun(dataframe=None, pair=None): def fun(dataframe=None, pair=None):
buy_value = 1 buy_value = 1
sell_value = 1 sell_value = 1
return _trend(dataframe, buy_value, sell_value) return _trend(dataframe, buy_value, sell_value)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf) backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
backtesting.advise_buy = fun # Override backtesting.strategy.advise_buy = fun # Override
backtesting.advise_sell = fun # Override backtesting.strategy.advise_sell = fun # Override
results = backtesting.backtest(backtest_conf) results = backtesting.backtest(backtest_conf)
assert results.empty assert results.empty
def test_backtest_only_sell(mocker, default_conf): def test_backtest_only_sell(mocker, default_conf, testdatadir):
# Override the default buy trend function in our default_strategy # Override the default buy trend function in our default_strategy
def fun(dataframe=None, pair=None): def fun(dataframe=None, pair=None):
buy_value = 0 buy_value = 0
sell_value = 1 sell_value = 1
return _trend(dataframe, buy_value, sell_value) return _trend(dataframe, buy_value, sell_value)
backtest_conf = _make_backtest_conf(mocker, conf=default_conf) backtest_conf = _make_backtest_conf(mocker, conf=default_conf, datadir=testdatadir)
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
backtesting.advise_buy = fun # Override backtesting.strategy.advise_buy = fun # Override
backtesting.advise_sell = fun # Override backtesting.strategy.advise_sell = fun # Override
results = backtesting.backtest(backtest_conf) results = backtesting.backtest(backtest_conf)
assert results.empty assert results.empty
def test_backtest_alternate_buy_sell(default_conf, fee, mocker): def test_backtest_alternate_buy_sell(default_conf, fee, mocker, testdatadir):
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
mocker.patch('freqtrade.optimize.backtesting.file_dump_json', MagicMock()) mocker.patch('freqtrade.optimize.backtesting.file_dump_json', MagicMock())
backtest_conf = _make_backtest_conf(mocker, conf=default_conf, pair='UNITTEST/BTC') backtest_conf = _make_backtest_conf(mocker, conf=default_conf,
pair='UNITTEST/BTC', datadir=testdatadir)
# We need to enable sell-signal - otherwise it sells on ROI!! # We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True} default_conf['experimental'] = {"use_sell_signal": True}
default_conf['ticker_interval'] = '1m' default_conf['ticker_interval'] = '1m'
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
backtesting.advise_buy = _trend_alternate # Override backtesting.strategy.advise_buy = _trend_alternate # Override
backtesting.advise_sell = _trend_alternate # Override backtesting.strategy.advise_sell = _trend_alternate # Override
results = backtesting.backtest(backtest_conf) results = backtesting.backtest(backtest_conf)
backtesting._store_backtest_result("test_.json", results) backtesting._store_backtest_result("test_.json", results)
# 200 candles in backtest data # 200 candles in backtest data
@ -672,7 +663,7 @@ def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
@pytest.mark.parametrize("pair", ['ADA/BTC', 'LTC/BTC']) @pytest.mark.parametrize("pair", ['ADA/BTC', 'LTC/BTC'])
@pytest.mark.parametrize("tres", [0, 20, 30]) @pytest.mark.parametrize("tres", [0, 20, 30])
def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair): def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair, testdatadir):
def _trend_alternate_hold(dataframe=None, metadata=None): def _trend_alternate_hold(dataframe=None, metadata=None):
""" """
@ -690,7 +681,7 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
patch_exchange(mocker) patch_exchange(mocker)
pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC'] pairs = ['ADA/BTC', 'DASH/BTC', 'ETH/BTC', 'LTC/BTC', 'NXT/BTC']
data = history.load_data(datadir=None, ticker_interval='5m', pairs=pairs) data = history.load_data(datadir=testdatadir, ticker_interval='5m', pairs=pairs)
# Only use 500 lines to increase performance # Only use 500 lines to increase performance
data = trim_dictlist(data, -500) data = trim_dictlist(data, -500)
@ -701,8 +692,8 @@ def test_backtest_multi_pair(default_conf, fee, mocker, tres, pair):
default_conf['ticker_interval'] = '5m' default_conf['ticker_interval'] = '5m'
backtesting = Backtesting(default_conf) backtesting = Backtesting(default_conf)
backtesting.advise_buy = _trend_alternate_hold # Override backtesting.strategy.advise_buy = _trend_alternate_hold # Override
backtesting.advise_sell = _trend_alternate_hold # Override backtesting.strategy.advise_sell = _trend_alternate_hold # Override
data_processed = backtesting.strategy.tickerdata_to_dataframe(data) data_processed = backtesting.strategy.tickerdata_to_dataframe(data)
min_date, max_date = get_timeframe(data_processed) min_date, max_date = get_timeframe(data_processed)
@ -806,7 +797,7 @@ def test_backtest_record(default_conf, fee, mocker):
assert dur > 0 assert dur > 0
def test_backtest_start_timerange(default_conf, mocker, caplog): def test_backtest_start_timerange(default_conf, mocker, caplog, testdatadir):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
async def load_pairs(pair, timeframe, since): async def load_pairs(pair, timeframe, since):
@ -823,7 +814,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
args = [ args = [
'--config', 'config.json', '--config', 'config.json',
'--strategy', 'DefaultStrategy', '--strategy', 'DefaultStrategy',
'--datadir', 'freqtrade/tests/testdata', '--datadir', str(testdatadir),
'backtesting', 'backtesting',
'--ticker-interval', '1m', '--ticker-interval', '1m',
'--timerange', '-100', '--timerange', '-100',
@ -837,7 +828,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...', 'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
'Ignoring max_open_trades (--disable-max-market-positions was used) ...', 'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
'Parameter --timerange detected: -100 ...', 'Parameter --timerange detected: -100 ...',
'Using data directory: freqtrade/tests/testdata ...', f'Using data directory: {testdatadir} ...',
'Using stake_currency: BTC ...', 'Using stake_currency: BTC ...',
'Using stake_amount: 0.001 ...', 'Using stake_amount: 0.001 ...',
'Backtesting with data from 2017-11-14T21:17:00+00:00 ' 'Backtesting with data from 2017-11-14T21:17:00+00:00 '
@ -849,7 +840,7 @@ def test_backtest_start_timerange(default_conf, mocker, caplog):
assert log_has(line, caplog) assert log_has(line, caplog)
def test_backtest_start_multi_strat(default_conf, mocker, caplog): def test_backtest_start_multi_strat(default_conf, mocker, caplog, testdatadir):
default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC'] default_conf['exchange']['pair_whitelist'] = ['UNITTEST/BTC']
async def load_pairs(pair, timeframe, since): async def load_pairs(pair, timeframe, since):
@ -869,7 +860,7 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
args = [ args = [
'--config', 'config.json', '--config', 'config.json',
'--datadir', 'freqtrade/tests/testdata', '--datadir', str(testdatadir),
'backtesting', 'backtesting',
'--ticker-interval', '1m', '--ticker-interval', '1m',
'--timerange', '-100', '--timerange', '-100',
@ -877,7 +868,7 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
'--disable-max-market-positions', '--disable-max-market-positions',
'--strategy-list', '--strategy-list',
'DefaultStrategy', 'DefaultStrategy',
'TestStrategy', 'SampleStrategy',
] ]
args = get_args(args) args = get_args(args)
start_backtesting(args) start_backtesting(args)
@ -891,14 +882,14 @@ def test_backtest_start_multi_strat(default_conf, mocker, caplog):
'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...', 'Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
'Ignoring max_open_trades (--disable-max-market-positions was used) ...', 'Ignoring max_open_trades (--disable-max-market-positions was used) ...',
'Parameter --timerange detected: -100 ...', 'Parameter --timerange detected: -100 ...',
'Using data directory: freqtrade/tests/testdata ...', f'Using data directory: {testdatadir} ...',
'Using stake_currency: BTC ...', 'Using stake_currency: BTC ...',
'Using stake_amount: 0.001 ...', 'Using stake_amount: 0.001 ...',
'Backtesting with data from 2017-11-14T21:17:00+00:00 ' 'Backtesting with data from 2017-11-14T21:17:00+00:00 '
'up to 2017-11-14T22:58:00+00:00 (0 days)..', 'up to 2017-11-14T22:58:00+00:00 (0 days)..',
'Parameter --enable-position-stacking detected ...', 'Parameter --enable-position-stacking detected ...',
'Running backtesting for Strategy DefaultStrategy', 'Running backtesting for Strategy DefaultStrategy',
'Running backtesting for Strategy TestStrategy', 'Running backtesting for Strategy SampleStrategy',
] ]
for line in exists: for line in exists:

12
freqtrade/tests/optimize/test_edge_cli.py → tests/optimize/test_edge_cli.py
View File

@ -3,14 +3,11 @@
from unittest.mock import MagicMock from unittest.mock import MagicMock
import pytest
from freqtrade.edge import PairInfo from freqtrade.edge import PairInfo
from freqtrade.optimize import setup_configuration, start_edge from freqtrade.optimize import setup_configuration, start_edge
from freqtrade.optimize.edge_cli import EdgeCli from freqtrade.optimize.edge_cli import EdgeCli
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.tests.conftest import (get_args, log_has, log_has_re, from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patch_exchange,
patched_configuration_load_config_file) patched_configuration_load_config_file)
@ -36,14 +33,10 @@ def test_setup_configuration_without_arguments(mocker, default_conf, caplog) ->
assert 'ticker_interval' in config assert 'ticker_interval' in config
assert not log_has_re('Parameter -i/--ticker-interval detected .*', caplog) assert not log_has_re('Parameter -i/--ticker-interval detected .*', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config assert 'timerange' not in config
assert 'stoploss_range' not in config assert 'stoploss_range' not in config
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> None: def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, edge_conf) patched_configuration_load_config_file(mocker, edge_conf)
mocker.patch( mocker.patch(
@ -57,7 +50,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
'--datadir', '/foo/bar', '--datadir', '/foo/bar',
'edge', 'edge',
'--ticker-interval', '1m', '--ticker-interval', '1m',
'--refresh-pairs-cached',
'--timerange', ':100', '--timerange', ':100',
'--stoplosses=-0.01,-0.10,-0.001' '--stoplosses=-0.01,-0.10,-0.001'
] ]
@ -75,8 +67,6 @@ def test_setup_edge_configuration_with_arguments(mocker, edge_conf, caplog) -> N
assert log_has('Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...', assert log_has('Parameter -i/--ticker-interval detected ... Using ticker_interval: 1m ...',
caplog) caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog) assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)

312
freqtrade/tests/optimize/test_hyperopt.py → tests/optimize/test_hyperopt.py
View File

@ -1,26 +1,25 @@
# pragma pylint: disable=missing-docstring,W0212,C0103 # pragma pylint: disable=missing-docstring,W0212,C0103
import os
from datetime import datetime from datetime import datetime
from pathlib import Path
from unittest.mock import MagicMock, PropertyMock from unittest.mock import MagicMock, PropertyMock
import pandas as pd import pandas as pd
import pytest import pytest
from arrow import Arrow from arrow import Arrow
from filelock import Timeout from filelock import Timeout
from pathlib import Path
from freqtrade import DependencyException, OperationalException from freqtrade import OperationalException
from freqtrade.data.converter import parse_ticker_dataframe from freqtrade.data.converter import parse_ticker_dataframe
from freqtrade.data.history import load_tickerdata_file from freqtrade.data.history import load_tickerdata_file
from freqtrade.optimize import setup_configuration, start_hyperopt from freqtrade.optimize import setup_configuration, start_hyperopt
from freqtrade.optimize.default_hyperopt import DefaultHyperOpts from freqtrade.optimize.default_hyperopt import DefaultHyperOpts
from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss from freqtrade.optimize.default_hyperopt_loss import DefaultHyperOptLoss
from freqtrade.optimize.hyperopt import Hyperopt from freqtrade.optimize.hyperopt import Hyperopt
from freqtrade.resolvers.hyperopt_resolver import HyperOptResolver, HyperOptLossResolver from freqtrade.resolvers.hyperopt_resolver import (HyperOptLossResolver,
HyperOptResolver)
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_args, log_has, log_has_re, from tests.conftest import (get_args, log_has, log_has_re, patch_exchange,
patch_exchange,
patched_configuration_load_config_file) patched_configuration_load_config_file)
@ -36,25 +35,23 @@ def hyperopt_results():
return pd.DataFrame( return pd.DataFrame(
{ {
'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'], 'pair': ['ETH/BTC', 'ETH/BTC', 'ETH/BTC'],
'profit_percent': [0.1, 0.2, 0.3], 'profit_percent': [-0.1, 0.2, 0.3],
'profit_abs': [0.2, 0.4, 0.5], 'profit_abs': [-0.2, 0.4, 0.6],
'trade_duration': [10, 30, 10], 'trade_duration': [10, 30, 10],
'profit': [2, 0, 0], 'sell_reason': [SellType.STOP_LOSS, SellType.ROI, SellType.ROI]
'loss': [0, 0, 1],
'sell_reason': [SellType.ROI, SellType.ROI, SellType.STOP_LOSS]
} }
) )
# Functions for recurrent object patching # Functions for recurrent object patching
def create_trials(mocker, hyperopt) -> None: def create_trials(mocker, hyperopt, testdatadir) -> None:
""" """
When creating trials, mock the hyperopt Trials so that *by default* When creating trials, mock the hyperopt Trials so that *by default*
- we don't create any pickle'd files in the filesystem - we don't create any pickle'd files in the filesystem
- we might have a pickle'd file so make sure that we return - we might have a pickle'd file so make sure that we return
false when looking for it false when looking for it
""" """
hyperopt.trials_file = Path('freqtrade/tests/optimize/ut_trials.pickle') hyperopt.trials_file = testdatadir / 'optimize/ut_trials.pickle'
mocker.patch.object(Path, "is_file", MagicMock(return_value=False)) mocker.patch.object(Path, "is_file", MagicMock(return_value=False))
stat_mock = MagicMock() stat_mock = MagicMock()
@ -89,15 +86,11 @@ def test_setup_hyperopt_configuration_without_arguments(mocker, default_conf, ca
assert 'position_stacking' not in config assert 'position_stacking' not in config
assert not log_has('Parameter --enable-position-stacking detected ...', caplog) assert not log_has('Parameter --enable-position-stacking detected ...', caplog)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' not in config assert 'timerange' not in config
assert 'runmode' in config assert 'runmode' in config
assert config['runmode'] == RunMode.HYPEROPT assert config['runmode'] == RunMode.HYPEROPT
@pytest.mark.filterwarnings("ignore:DEPRECATED")
def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplog) -> None: def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplog) -> None:
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
mocker.patch( mocker.patch(
@ -111,7 +104,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
'hyperopt', 'hyperopt',
'--ticker-interval', '1m', '--ticker-interval', '1m',
'--timerange', ':100', '--timerange', ':100',
'--refresh-pairs-cached',
'--enable-position-stacking', '--enable-position-stacking',
'--disable-max-market-positions', '--disable-max-market-positions',
'--epochs', '1000', '--epochs', '1000',
@ -140,9 +132,6 @@ def test_setup_hyperopt_configuration_with_arguments(mocker, default_conf, caplo
assert log_has('Parameter --disable-max-market-positions detected ...', caplog) assert log_has('Parameter --disable-max-market-positions detected ...', caplog)
assert log_has('max_open_trades set to unlimited ...', caplog) assert log_has('max_open_trades set to unlimited ...', caplog)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog)
assert 'timerange' in config assert 'timerange' in config
assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog) assert log_has('Parameter --timerange detected: {} ...'.format(config['timerange']), caplog)
@ -164,15 +153,15 @@ def test_hyperoptresolver(mocker, default_conf, caplog) -> None:
delattr(hyperopts, 'populate_sell_trend') delattr(hyperopts, 'populate_sell_trend')
mocker.patch( mocker.patch(
'freqtrade.resolvers.hyperopt_resolver.HyperOptResolver._load_hyperopt', 'freqtrade.resolvers.hyperopt_resolver.HyperOptResolver._load_hyperopt',
MagicMock(return_value=hyperopts) MagicMock(return_value=hyperopts(default_conf))
) )
x = HyperOptResolver(default_conf, ).hyperopt x = HyperOptResolver(default_conf, ).hyperopt
assert not hasattr(x, 'populate_buy_trend') assert not hasattr(x, 'populate_buy_trend')
assert not hasattr(x, 'populate_sell_trend') assert not hasattr(x, 'populate_sell_trend')
assert log_has("Custom Hyperopt does not provide populate_sell_trend. " assert log_has("Hyperopt class does not provide populate_sell_trend() method. "
"Using populate_sell_trend from DefaultStrategy.", caplog) "Using populate_sell_trend from the strategy.", caplog)
assert log_has("Custom Hyperopt does not provide populate_buy_trend. " assert log_has("Hyperopt class does not provide populate_buy_trend() method. "
"Using populate_buy_trend from DefaultStrategy.", caplog) "Using populate_buy_trend from the strategy.", caplog)
assert hasattr(x, "ticker_interval") assert hasattr(x, "ticker_interval")
@ -201,6 +190,24 @@ def test_hyperoptlossresolver_wrongname(mocker, default_conf, caplog) -> None:
HyperOptLossResolver(default_conf, ).hyperopt HyperOptLossResolver(default_conf, ).hyperopt
def test_start_not_installed(mocker, default_conf, caplog, import_fails) -> None:
start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch('freqtrade.optimize.hyperopt.Hyperopt.start', start_mock)
patch_exchange(mocker)
args = [
'--config', 'config.json',
'hyperopt',
'--epochs', '5'
]
args = get_args(args)
with pytest.raises(OperationalException, match=r"Please ensure that the hyperopt dependencies"):
start_hyperopt(args)
def test_start(mocker, default_conf, caplog) -> None: def test_start(mocker, default_conf, caplog) -> None:
start_mock = MagicMock() start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
@ -215,9 +222,6 @@ def test_start(mocker, default_conf, caplog) -> None:
args = get_args(args) args = get_args(args)
start_hyperopt(args) start_hyperopt(args)
import pprint
pprint.pprint(caplog.record_tuples)
assert log_has('Starting freqtrade in Hyperopt mode', caplog) assert log_has('Starting freqtrade in Hyperopt mode', caplog)
assert start_mock.call_count == 1 assert start_mock.call_count == 1
@ -240,30 +244,9 @@ def test_start_no_data(mocker, default_conf, caplog) -> None:
args = get_args(args) args = get_args(args)
start_hyperopt(args) start_hyperopt(args)
import pprint
pprint.pprint(caplog.record_tuples)
assert log_has('No data found. Terminating.', caplog) assert log_has('No data found. Terminating.', caplog)
def test_start_failure(mocker, default_conf, caplog) -> None:
start_mock = MagicMock()
patched_configuration_load_config_file(mocker, default_conf)
mocker.patch('freqtrade.optimize.hyperopt.Hyperopt.start', start_mock)
patch_exchange(mocker)
args = [
'--config', 'config.json',
'--strategy', 'TestStrategy',
'hyperopt',
'--epochs', '5'
]
args = get_args(args)
with pytest.raises(DependencyException):
start_hyperopt(args)
assert log_has("Please don't use --strategy for hyperopt.", caplog)
def test_start_filelock(mocker, default_conf, caplog) -> None: def test_start_filelock(mocker, default_conf, caplog) -> None:
start_mock = MagicMock(side_effect=Timeout(Hyperopt.get_lock_filename(default_conf))) start_mock = MagicMock(side_effect=Timeout(Hyperopt.get_lock_filename(default_conf)))
patched_configuration_load_config_file(mocker, default_conf) patched_configuration_load_config_file(mocker, default_conf)
@ -374,23 +357,23 @@ def test_no_log_if_loss_does_not_improve(hyperopt, caplog) -> None:
assert caplog.record_tuples == [] assert caplog.record_tuples == []
def test_save_trials_saves_trials(mocker, hyperopt, caplog) -> None: def test_save_trials_saves_trials(mocker, hyperopt, testdatadir, caplog) -> None:
trials = create_trials(mocker, hyperopt) trials = create_trials(mocker, hyperopt, testdatadir)
mock_dump = mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None) mock_dump = mocker.patch('freqtrade.optimize.hyperopt.dump', return_value=None)
hyperopt.trials = trials hyperopt.trials = trials
hyperopt.save_trials() hyperopt.save_trials()
trials_file = os.path.join('freqtrade', 'tests', 'optimize', 'ut_trials.pickle') trials_file = testdatadir / 'optimize' / 'ut_trials.pickle'
assert log_has('Saving 1 evaluations to \'{}\''.format(trials_file), caplog) assert log_has(f"Saving 1 evaluations to '{trials_file}'", caplog)
mock_dump.assert_called_once() mock_dump.assert_called_once()
def test_read_trials_returns_trials_file(mocker, hyperopt, caplog) -> None: def test_read_trials_returns_trials_file(mocker, hyperopt, testdatadir, caplog) -> None:
trials = create_trials(mocker, hyperopt) trials = create_trials(mocker, hyperopt, testdatadir)
mock_load = mocker.patch('freqtrade.optimize.hyperopt.load', return_value=trials) mock_load = mocker.patch('freqtrade.optimize.hyperopt.load', return_value=trials)
hyperopt_trial = hyperopt.read_trials() hyperopt_trial = hyperopt.read_trials()
trials_file = os.path.join('freqtrade', 'tests', 'optimize', 'ut_trials.pickle') trials_file = testdatadir / 'optimize' / 'ut_trials.pickle'
assert log_has('Reading Trials from \'{}\''.format(trials_file), caplog) assert log_has(f"Reading Trials from '{trials_file}'", caplog)
assert hyperopt_trial == trials assert hyperopt_trial == trials
mock_load.assert_called_once() mock_load.assert_called_once()
@ -418,7 +401,8 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
parallel = mocker.patch( parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel', 'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}]) MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result',
'params': {'buy': {}, 'sell': {}, 'roi': {}, 'stoploss': 0.0}}])
) )
patch_exchange(mocker) patch_exchange(mocker)
@ -441,8 +425,8 @@ def test_start_calls_optimizer(mocker, default_conf, caplog, capsys) -> None:
assert dumper.called assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations # Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2 assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting, "advise_sell") assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting, "advise_buy") assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades") assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades'] assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking") assert hasattr(hyperopt, "position_stacking")
@ -484,8 +468,8 @@ def test_has_space(hyperopt):
assert hyperopt.has_space('buy') assert hyperopt.has_space('buy')
def test_populate_indicators(hyperopt) -> None: def test_populate_indicators(hyperopt, testdatadir) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m') tick = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC", tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)} fill_missing=True)}
dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist) dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist)
@ -498,8 +482,8 @@ def test_populate_indicators(hyperopt) -> None:
assert 'rsi' in dataframe assert 'rsi' in dataframe
def test_buy_strategy_generator(hyperopt) -> None: def test_buy_strategy_generator(hyperopt, testdatadir) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m') tick = load_tickerdata_file(testdatadir, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC", tickerlist = {'UNITTEST/BTC': parse_ticker_dataframe(tick, '1m', pair="UNITTEST/BTC",
fill_missing=True)} fill_missing=True)}
dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist) dataframes = hyperopt.backtesting.strategy.tickerdata_to_dataframe(tickerlist)
@ -584,6 +568,7 @@ def test_generate_optimizer(mocker, default_conf) -> None:
} }
hyperopt = Hyperopt(default_conf) hyperopt = Hyperopt(default_conf)
hyperopt.dimensions = hyperopt.hyperopt_space()
generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values())) generate_optimizer_value = hyperopt.generate_optimizer(list(optimizer_param.values()))
assert generate_optimizer_value == response_expected assert generate_optimizer_value == response_expected
@ -693,3 +678,200 @@ def test_print_json_spaces_roi_stoploss(mocker, default_conf, caplog, capsys) ->
assert dumper.called assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations # Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2 assert dumper.call_count == 2
def test_simplified_interface_roi_stoploss(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{
'loss': 1, 'results_explanation': 'foo result', 'params': {'stoploss': 0.0}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'roi stoploss',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
def test_simplified_interface_all_failed(mocker, default_conf, caplog, capsys) -> None:
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'all',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
with pytest.raises(OperationalException, match=r"The 'buy' space is included into *"):
hyperopt.start()
def test_simplified_interface_buy(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'buy',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
# TODO: sell_strategy_generator() is actually not called because
# run_optimizer_parallel() is mocked
del hyperopt.custom_hyperopt.__class__.sell_strategy_generator
del hyperopt.custom_hyperopt.__class__.sell_indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
def test_simplified_interface_sell(mocker, default_conf, caplog, capsys) -> None:
dumper = mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
parallel = mocker.patch(
'freqtrade.optimize.hyperopt.Hyperopt.run_optimizer_parallel',
MagicMock(return_value=[{'loss': 1, 'results_explanation': 'foo result', 'params': {}}])
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': 'sell',
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
# TODO: buy_strategy_generator() is actually not called because
# run_optimizer_parallel() is mocked
del hyperopt.custom_hyperopt.__class__.buy_strategy_generator
del hyperopt.custom_hyperopt.__class__.indicator_space
hyperopt.start()
parallel.assert_called_once()
out, err = capsys.readouterr()
assert 'Best result:\n\n* 1/1: foo result Objective: 1.00000\n' in out
assert dumper.called
# Should be called twice, once for tickerdata, once to save evaluations
assert dumper.call_count == 2
assert hasattr(hyperopt.backtesting.strategy, "advise_sell")
assert hasattr(hyperopt.backtesting.strategy, "advise_buy")
assert hasattr(hyperopt, "max_open_trades")
assert hyperopt.max_open_trades == default_conf['max_open_trades']
assert hasattr(hyperopt, "position_stacking")
@pytest.mark.parametrize("method,space", [
('buy_strategy_generator', 'buy'),
('indicator_space', 'buy'),
('sell_strategy_generator', 'sell'),
('sell_indicator_space', 'sell'),
])
def test_simplified_interface_failed(mocker, default_conf, caplog, capsys, method, space) -> None:
mocker.patch('freqtrade.optimize.hyperopt.dump', MagicMock())
mocker.patch('freqtrade.optimize.hyperopt.load_data', MagicMock())
mocker.patch(
'freqtrade.optimize.hyperopt.get_timeframe',
MagicMock(return_value=(datetime(2017, 12, 10), datetime(2017, 12, 13)))
)
patch_exchange(mocker)
default_conf.update({'config': 'config.json.example',
'epochs': 1,
'timerange': None,
'spaces': space,
'hyperopt_jobs': 1, })
hyperopt = Hyperopt(default_conf)
hyperopt.backtesting.strategy.tickerdata_to_dataframe = MagicMock()
hyperopt.custom_hyperopt.generate_roi_table = MagicMock(return_value={})
delattr(hyperopt.custom_hyperopt.__class__, method)
with pytest.raises(OperationalException, match=f"The '{space}' space is included into *"):
hyperopt.start()

0
freqtrade/tests/pairlist/__init__.py → tests/pairlist/__init__.py
View File

2
freqtrade/tests/pairlist/test_pairlist.py → tests/pairlist/test_pairlist.py
View File

@ -5,7 +5,7 @@ from unittest.mock import MagicMock, PropertyMock
from freqtrade import OperationalException from freqtrade import OperationalException
from freqtrade.constants import AVAILABLE_PAIRLISTS from freqtrade.constants import AVAILABLE_PAIRLISTS
from freqtrade.resolvers import PairListResolver from freqtrade.resolvers import PairListResolver
from freqtrade.tests.conftest import get_patched_freqtradebot from tests.conftest import get_patched_freqtradebot
import pytest import pytest
# whitelist, blacklist # whitelist, blacklist

0
freqtrade/tests/rpc/__init__.py → tests/rpc/__init__.py
View File

Some files were not shown because too many files have changed in this diff Show More