Merge branch 'develop' into fix/backtest_toomanyopen

This commit is contained in:
Matthias 2018-11-24 10:38:30 +01:00
commit 805f509498
47 changed files with 2615 additions and 301 deletions

View File

@ -2,50 +2,52 @@
## Contribute to freqtrade
Feel like our bot is missing a feature? We welcome your pull requests! Few pointers for contributions:
Feel like our bot is missing a feature? We welcome your pull requests!
Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
Few pointers for contributions:
- Create your PR against the `develop` branch, not `master`.
- New features need to contain unit tests and must be PEP8
conformant (max-line-length = 100).
- New features need to contain unit tests and must be PEP8 conformant (max-line-length = 100).
If you are unsure, discuss the feature on our [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE)
or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR.
**Before sending the PR:**
## Before sending the PR:
## 1. Run unit tests
### 1. Run unit tests
All unit tests must pass. If a unit test is broken, change your code to
make it pass. It means you have introduced a regression.
### Test the whole project
#### Test the whole project
```bash
pytest freqtrade
```
### Test only one file
#### Test only one file
```bash
pytest freqtrade/tests/test_<file_name>.py
```
### Test only one method from one file
#### Test only one method from one file
```bash
pytest freqtrade/tests/test_<file_name>.py::test_<method_name>
```
## 2. Test if your code is PEP8 compliant
### 2. Test if your code is PEP8 compliant
### Install packages
#### Install packages
```bash
pip3.6 install flake8 coveralls
```
### Run Flake8
#### Run Flake8
```bash
flake8 freqtrade
@ -56,15 +58,15 @@ To help with that, we encourage you to install the git pre-commit
hook that will warn you when you try to commit code that fails these checks.
Guide for installing them is [here](http://flake8.pycqa.org/en/latest/user/using-hooks.html).
## 3. Test if all type-hints are correct
### 3. Test if all type-hints are correct
### Install packages
#### Install packages
``` bash
pip3.6 install mypy
```
### Run mypy
#### Run mypy
``` bash
mypy freqtrade

View File

@ -62,6 +62,7 @@ hesitate to read the source code and understand the mechanism of this bot.
- [Requirements](#requirements)
- [Min hardware required](#min-hardware-required)
- [Software requirements](#software-requirements)
- [Wanna help?]
## Quick start
@ -190,10 +191,14 @@ in the bug reports.
### [Pull Requests](https://github.com/freqtrade/freqtrade/pulls)
Feel like our bot is missing a feature? We welcome your pull requests!
Please read our
[Contributing document](https://github.com/freqtrade/freqtrade/blob/develop/CONTRIBUTING.md)
to understand the requirements before sending your pull-requests.
Coding is not a neccessity to contribute - maybe start with improving our documentation?
Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase.
**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it.
**Important:** Always create your PR against the `develop` branch, not `master`.
@ -218,3 +223,4 @@ To run this bot we recommend you a cloud instance with a minimum of:
- [TA-Lib](https://mrjbq7.github.io/ta-lib/install.html)
- [virtualenv](https://virtualenv.pypa.io/en/stable/installation/) (Recommended)
- [Docker](https://www.docker.com/products/docker) (Recommended)

View File

@ -53,6 +53,21 @@
"sell_profit_only": false,
"ignore_roi_if_buy_signal": false
},
"edge": {
"enabled": false,
"process_throttle_secs": 3600,
"calculate_since_number_of_days": 7,
"total_capital_in_stake_currency": 0.5,
"allowed_risk": 0.01,
"stoploss_range_min": -0.01,
"stoploss_range_max": -0.1,
"stoploss_range_step": -0.01,
"minimum_winrate": 0.60,
"minimum_expectancy": 0.20,
"min_trade_number": 10,
"max_trade_duration_minute": 1440,
"remove_pumps": false
},
"telegram": {
"enabled": true,
"token": "your_telegram_token",

View File

@ -33,6 +33,11 @@
"order_book_min": 1,
"order_book_max": 9
},
"order_types": {
"buy": "limit",
"sell": "limit",
"stoploss": "market"
},
"exchange": {
"name": "bittrex",
"key": "your_exchange_key",
@ -59,6 +64,20 @@
],
"outdated_offset": 5
},
"edge": {
"enabled": false,
"process_throttle_secs": 3600,
"calculate_since_number_of_days": 2,
"allowed_risk": 0.01,
"stoploss_range_min": -0.01,
"stoploss_range_max": -0.1,
"stoploss_range_step": -0.01,
"minimum_winrate": 0.60,
"minimum_expectancy": 0.20,
"min_trade_number": 10,
"max_trade_duration_minute": 1440,
"remove_pumps": false
},
"experimental": {
"use_sell_signal": false,
"sell_profit_only": false,

View File

@ -204,6 +204,8 @@ optional arguments:
number)
--timerange TIMERANGE
specify what timerange of data to use.
--hyperopt PATH specify hyperopt file (default:
freqtrade/optimize/default_hyperopt.py)
-e INT, --epochs INT specify number of epochs (default: 100)
-s {all,buy,roi,stoploss} [{all,buy,roi,stoploss} ...], --spaces {all,buy,roi,stoploss} [{all,buy,roi,stoploss} ...]
Specify which parameters to hyperopt. Space separate
@ -211,6 +213,31 @@ optional arguments:
```
## Edge commands
To know your trade expectacny and winrate against historical data, you can use Edge.
```
usage: main.py edge [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE] [-r]
[--stoplosses STOPLOSS_RANGE]
optional arguments:
-h, --help show this help message and exit
-i TICKER_INTERVAL, --ticker-interval TICKER_INTERVAL
specify ticker interval (1m, 5m, 30m, 1h, 1d)
--timerange TIMERANGE
specify what timerange of data to use.
-r, --refresh-pairs-cached
refresh the pairs files in tests/testdata with the
latest data from the exchange. Use it if you want to
run your edge with up-to-date data.
--stoplosses STOPLOSS_RANGE
defines a range of stoploss against which edge will
assess the strategythe format is "min,max,step"
(without any space).example:
--stoplosses=-0.01,-0.1,-0.001
```
## A parameter missing in the configuration?
All parameters for `main.py`, `backtesting`, `hyperopt` are referenced

View File

@ -17,7 +17,7 @@ The table below will list all configuration parameters.
| Command | Default | Mandatory | Description |
|----------|---------|----------|-------------|
| `max_open_trades` | 3 | Yes | Number of trades open your bot will have.
| `max_open_trades` | 3 | Yes | Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades)
| `stake_currency` | BTC | Yes | Crypto-currency used for trading.
| `stake_amount` | 0.05 | Yes | Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to 'unlimited' to allow the bot to use all avaliable balance.
| `ticker_interval` | [1m, 5m, 30m, 1h, 1d] | No | The ticker interval to use (1min, 5 min, 30 min, 1 hour or 1 day). Default is 5 minutes
@ -39,6 +39,7 @@ The table below will list all configuration parameters.
| `ask_strategy.use_order_book` | false | No | Allows selling of open traded pair using the rates in Order Book Asks.
| `ask_strategy.order_book_min` | 0 | No | Bot will scan from the top min to max Order Book Asks searching for a profitable rate.
| `ask_strategy.order_book_max` | 0 | No | Bot will scan from the top min to max Order Book Asks searching for a profitable rate.
| `order_types` | None | No | Configure order-types depending on the action (`"buy"`, `"sell"`, `"stoploss"`).
| `exchange.name` | bittrex | Yes | Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename).
| `exchange.key` | key | No | API key to use for the exchange. Only required when you are in production mode.
| `exchange.secret` | secret | No | API secret to use for the exchange. Only required when you are in production mode.
@ -47,6 +48,7 @@ The table below will list all configuration parameters.
| `exchange.ccxt_rate_limit` | True | No | DEPRECATED!! Have CCXT handle Exchange rate limits. Depending on the exchange, having this to false can lead to temporary bans from the exchange.
| `exchange.ccxt_config` | None | No | Additional CCXT parameters passed to the regular ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation)
| `exchange.ccxt_async_config` | None | No | Additional CCXT parameters passed to the async ccxt instance. Parameters may differ from exchange to exchange and are documented in the [ccxt documentation](https://ccxt.readthedocs.io/en/latest/manual.html#instantiation)
| `edge` | false | No | Please refer to [edge configuration document](edge.md) for detailed explanation.
| `experimental.use_sell_signal` | false | No | Use your sell strategy in addition of the `minimal_roi`.
| `experimental.sell_profit_only` | false | No | waits until you have made a positive profit before taking a sell decision.
| `experimental.ignore_roi_if_buy_signal` | false | No | Does not sell if the buy-signal is still active. Takes preference over `minimal_roi` and `use_sell_signal`
@ -137,6 +139,22 @@ use the `last` price and values between those interpolate between ask and last
price. Using `ask` price will guarantee quick success in bid, but bot will also
end up paying more then would probably have been necessary.
### Understand order_types
`order_types` contains a dict mapping order-types to market-types. This allows to buy using limit orders, sell using limit-orders, and create stoploss orders using market.
This can be set in the configuration or in the strategy. Configuration overwrites strategy configurations.
If this is configured, all 3 values (`"buy"`, `"sell"` and `"stoploss"`) need to be present, otherwise the bot warn about it and will fail to start.
The below is the default which is used if this is not configured in either Strategy or configuration.
``` json
"order_types": {
"buy": "limit",
"sell": "limit",
"stoploss": "market"
},
```
### What values for exchange.name?
Freqtrade is based on [CCXT library](https://github.com/ccxt/ccxt) that supports 115 cryptocurrency

207
docs/edge.md Normal file
View File

@ -0,0 +1,207 @@
# Edge positioning
This page explains how to use Edge Positioning module in your bot in order to enter into a trade only if the trade has a reasonable win rate and risk reward ratio, and consequently adjust your position size and stoploss.
**NOTICE:** Edge positioning is not compatible with dynamic whitelist. it overrides dynamic whitelist.
**NOTICE2:** Edge won't consider anything else than buy/sell/stoploss signals. So trailing stoploss, ROI, and everything else will be ignored in its calculation.
## Table of Contents
- [Introduction](#introduction)
- [How does it work?](#how-does-it-work?)
- [Configurations](#configurations)
- [Running Edge independently](#running-edge-independently)
## Introduction
Trading is all about probability. No one can claim that he has a strategy working all the time. You have to assume that sometimes you lose.<br/><br/>
But it doesn't mean there is no rule, it only means rules should work "most of the time". Let's play a game: we toss a coin, heads: I give you 10$, tails: You give me 10$. Is it an interesting game ? no, it is quite boring, isn't it?<br/><br/>
But let's say the probability that we have heads is 80%, and the probability that we have tails is 20%. Now it is becoming interesting ...
That means 10$ x 80% versus 10$ x 20%. 8$ versus 2$. That means over time you will win 8$ risking only 2$ on each toss of coin.<br/><br/>
Let's complicate it more: you win 80% of the time but only 2$, I win 20% of the time but 8$. The calculation is: 80% * 2$ versus 20% * 8$. It is becoming boring again because overtime you win $1.6$ (80% x 2$) and me $1.6 (20% * 8$) too.<br/><br/>
The question is: How do you calculate that? how do you know if you wanna play?
The answer comes to two factors:
- Win Rate
- Risk Reward Ratio
### Win Rate
Means over X trades what is the percentage of winning trades to total number of trades (note that we don't consider how much you gained but only If you won or not).
`W = (Number of winning trades) / (Number of losing trades)`
### Risk Reward Ratio
Risk Reward Ratio is a formula used to measure the expected gains of a given investment against the risk of loss. It is basically what you potentially win divided by what you potentially lose:
`R = Profit / Loss`
Over time, on many trades, you can calculate your risk reward by dividing your average profit on winning trades by your average loss on losing trades:
`Average profit = (Sum of profits) / (Number of winning trades)`
`Average loss = (Sum of losses) / (Number of losing trades)`
`R = (Average profit) / (Average loss)`
### Expectancy
At this point we can combine W and R to create an expectancy ratio. This is a simple process of multiplying the risk reward ratio by the percentage of winning trades, and subtracting the percentage of losing trades, which is calculated as follows:
Expectancy Ratio = (Risk Reward Ratio x Win Rate) Loss Rate
So lets say your Win rate is 28% and your Risk Reward Ratio is 5:
`Expectancy = (5 * 0.28) - 0.72 = 0.68`
Superficially, this means that on average you expect this strategys trades to return .68 times the size of your losers. This is important for two reasons: First, it may seem obvious, but you know right away that you have a positive return. Second, you now have a number you can compare to other candidate systems to make decisions about which ones you employ.
It is important to remember that any system with an expectancy greater than 0 is profitable using past data. The key is finding one that will be profitable in the future.
You can also use this number to evaluate the effectiveness of modifications to this system.
**NOTICE:** It's important to keep in mind that Edge is testing your expectancy using historical data , there's no guarantee that you will have a similar edge in the future. It's still vital to do this testing in order to build confidence in your methodology, but be wary of "curve-fitting" your approach to the historical data as things are unlikely to play out the exact same way for future trades.
## How does it work?
If enabled in config, Edge will go through historical data with a range of stoplosses in order to find buy and sell/stoploss signals. It then calculates win rate and expectancy over X trades for each stoploss. Here is an example:
| Pair | Stoploss | Win Rate | Risk Reward Ratio | Expectancy |
|----------|:-------------:|-------------:|------------------:|-----------:|
| XZC/ETH | -0.03 | 0.52 |1.359670 | 0.228 |
| XZC/ETH | -0.01 | 0.50 |1.176384 | 0.088 |
| XZC/ETH | -0.02 | 0.51 |1.115941 | 0.079 |
The goal here is to find the best stoploss for the strategy in order to have the maximum expectancy. In the above example stoploss at 3% leads to the maximum expectancy according to historical data.
Edge then forces stoploss to your strategy dynamically.
### Position size
Edge dictates the stake amount for each trade to the bot according to the following factors:
- Allowed capital at risk
- Stoploss
Allowed capital at risk is calculated as follows:
**allowed capital at risk** = **total capital** X **allowed risk per trade**
**total capital** is your stake amount.
**Stoploss** is calculated as described above against historical data.
Your position size then will be:
**position size** = **allowed capital at risk** / **stoploss**
Example:
Let's say your stake amount is 3 ETH, you would allow 1% of risk for each trade. thus your allowed capital at risk would be **3 x 0.01 = 0.03 ETH**. Let's assume Edge has calculated that for **XLM/ETH** market your stoploss should be at 2%. So your position size will be **0.03 / 0.02= 1.5ETH**.<br/>
## Configurations
Edge has following configurations:
#### enabled
If true, then Edge will run periodically<br/>
(default to false)
#### process_throttle_secs
How often should Edge run in seconds? <br/>
(default to 3600 so one hour)
#### calculate_since_number_of_days
Number of days of data against which Edge calculates Win Rate, Risk Reward and Expectancy
Note that it downloads historical data so increasing this number would lead to slowing down the bot<br/>
(default to 7)
#### allowed_risk
Percentage of allowed risk per trade<br/>
(default to 0.01 [1%])
#### stoploss_range_min
Minimum stoploss <br/>
(default to -0.01)
#### stoploss_range_max
Maximum stoploss <br/>
(default to -0.10)
#### stoploss_range_step
As an example if this is set to -0.01 then Edge will test the strategy for [-0.01, -0,02, -0,03 ..., -0.09, -0.10] ranges.
Note than having a smaller step means having a bigger range which could lead to slow calculation. <br/>
if you set this parameter to -0.001, you then slow down the Edge calculation by a factor of 10. <br/>
(default to -0.01)
#### minimum_winrate
It filters pairs which don't have at least minimum_winrate.
This comes handy if you want to be conservative and don't comprise win rate in favor of risk reward ratio.<br/>
(default to 0.60)
#### minimum_expectancy
It filters paris which have an expectancy lower than this number .
Having an expectancy of 0.20 means if you put 10$ on a trade you expect a 12$ return.<br/>
(default to 0.20)
#### min_trade_number
When calculating W and R and E (expectancy) against historical data, you always want to have a minimum number of trades. The more this number is the more Edge is reliable. Having a win rate of 100% on a single trade doesn't mean anything at all. But having a win rate of 70% over past 100 trades means clearly something. <br/>
(default to 10, it is highly recommended not to decrease this number)
#### max_trade_duration_minute
Edge will filter out trades with long duration. If a trade is profitable after 1 month, it is hard to evaluate the strategy based on it. But if most of trades are profitable and they have maximum duration of 30 minutes, then it is clearly a good sign.<br/>
**NOTICE:** While configuring this value, you should take into consideration your ticker interval. as an example filtering out trades having duration less than one day for a strategy which has 4h interval does not make sense. default value is set assuming your strategy interval is relatively small (1m or 5m, etc).<br/>
(default to 1 day, 1440 = 60 * 24)
#### remove_pumps
Edge will remove sudden pumps in a given market while going through historical data. However, given that pumps happen very often in crypto markets, we recommend you keep this off.<br/>
(default to false)
## Running Edge independently
You can run Edge independently in order to see in details the result. Here is an example:
```bash
python3 ./freqtrade/main.py edge
```
An example of its output:
| pair | stoploss | win rate | risk reward ratio | required risk reward | expectancy | total number of trades | average duration (min) |
|:----------|-----------:|-----------:|--------------------:|-----------------------:|-------------:|-------------------------:|-------------------------:|
| AGI/BTC | -0.02 | 0.64 | 5.86 | 0.56 | 3.41 | 14 | 54 |
| NXS/BTC | -0.03 | 0.64 | 2.99 | 0.57 | 1.54 | 11 | 26 |
| LEND/BTC | -0.02 | 0.82 | 2.05 | 0.22 | 1.50 | 11 | 36 |
| VIA/BTC | -0.01 | 0.55 | 3.01 | 0.83 | 1.19 | 11 | 48 |
| MTH/BTC | -0.09 | 0.56 | 2.82 | 0.80 | 1.12 | 18 | 52 |
| ARDR/BTC | -0.04 | 0.42 | 3.14 | 1.40 | 0.73 | 12 | 42 |
| BCPT/BTC | -0.01 | 0.71 | 1.34 | 0.40 | 0.67 | 14 | 30 |
| WINGS/BTC | -0.02 | 0.56 | 1.97 | 0.80 | 0.65 | 27 | 42 |
| VIBE/BTC | -0.02 | 0.83 | 0.91 | 0.20 | 0.59 | 12 | 35 |
| MCO/BTC | -0.02 | 0.79 | 0.97 | 0.27 | 0.55 | 14 | 31 |
| GNT/BTC | -0.02 | 0.50 | 2.06 | 1.00 | 0.53 | 18 | 24 |
| HOT/BTC | -0.01 | 0.17 | 7.72 | 4.81 | 0.50 | 209 | 7 |
| SNM/BTC | -0.03 | 0.71 | 1.06 | 0.42 | 0.45 | 17 | 38 |
| APPC/BTC | -0.02 | 0.44 | 2.28 | 1.27 | 0.44 | 25 | 43 |
| NEBL/BTC | -0.03 | 0.63 | 1.29 | 0.58 | 0.44 | 19 | 59 |
### Update cached pairs with the latest data
```bash
python3 ./freqtrade/main.py edge --refresh-pairs-cached
```
### Precising stoploss range
```bash
python3 ./freqtrade/main.py edge --stoplosses=-0.01,-0.1,-0.001 #min,max,step
```
### Advanced use of timerange
```bash
python3 ./freqtrade/main.py edge --timerange=20181110-20181113
```
Doing --timerange=-200 will get the last 200 timeframes from your inputdata. You can also specify specific dates, or a range span indexed by start and stop.
The full timerange specification:
* Use last 123 tickframes of data: --timerange=-123
* Use first 123 tickframes of data: --timerange=123-
* Use tickframes from line 123 through 456: --timerange=123-456
* Use tickframes till 2018/01/31: --timerange=-20180131
* Use tickframes since 2018/01/31: --timerange=20180131-
* Use tickframes since 2018/01/31 till 2018/03/01 : --timerange=20180131-20180301
* Use tickframes between POSIX timestamps 1527595200 1527618600: --timerange=1527595200-1527618600

View File

@ -19,18 +19,27 @@ and still take a long time.
## Prepare Hyperopting
We recommend you start by taking a look at `hyperopt.py` file located in [freqtrade/optimize](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py)
Before we start digging in Hyperopt, we recommend you to take a look at
an example hyperopt file located into [user_data/hyperopts/](https://github.com/gcarq/freqtrade/blob/develop/user_data/hyperopts/test_hyperopt.py)
### Configure your Guards and Triggers
### 1. Install a Custom Hyperopt File
This is very simple. Put your hyperopt file into the folder
`user_data/hyperopts`.
There are two places you need to change to add a new buy strategy for testing:
- Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L231-L264).
- Inside [hyperopt_space()](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/optimize/hyperopt.py#L213-L224)
and the associated methods `indicator_space`, `roi_space`, `stoploss_space`.
Let assume you want a hyperopt file `awesome_hyperopt.py`:
1. Copy the file `user_data/hyperopts/sample_hyperopt.py` into `user_data/hyperopts/awesome_hyperopt.py`
There you have two different type of indicators: 1. `guards` and 2. `triggers`.
1. Guards are conditions like "never buy if ADX < 10", or "never buy if
current price is over EMA10".
### 2. Configure your Guards and Triggers
There are two places you need to change in your hyperopt file to add a
new buy hyperopt for testing:
- Inside [populate_buy_trend()](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/test_hyperopt.py#L230-L251).
- Inside [indicator_space()](https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/test_hyperopt.py#L207-L223).
There you have two different types of indicators: 1. `guards` and 2. `triggers`.
1. Guards are conditions like "never buy if ADX < 10", or never buy if
current price is over EMA10.
2. Triggers are ones that actually trigger buy in specific moment, like
"buy when EMA5 crosses over EMA10" or "buy when close price touches lower
bollinger band".
@ -124,9 +133,12 @@ Because hyperopt tries a lot of combinations to find the best parameters it will
We strongly recommend to use `screen` or `tmux` to prevent any connection loss.
```bash
python3 ./freqtrade/main.py -c config.json hyperopt -e 5000
python3 ./freqtrade/main.py -s <strategyname> --hyperopt <hyperoptname> -c config.json hyperopt -e 5000
```
Use `<strategyname>` and `<hyperoptname>` as the names of the custom strategy
(only required for generating sells) and the custom hyperopt used.
The `-e` flag will set how many evaluations hyperopt will do. We recommend
running at least several thousand evaluations.

View File

@ -21,10 +21,12 @@ Pull-request. Do not hesitate to reach us on
- [Bot commands](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#bot-commands)
- [Backtesting commands](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#backtesting-commands)
- [Hyperopt commands](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#hyperopt-commands)
- [Edge commands](https://github.com/mishaker/freqtrade/blob/develop/docs/bot-usage.md#edge-commands)
- [Bot Optimization](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md)
- [Change your strategy](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md#change-your-strategy)
- [Add more Indicator](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md#add-more-indicator)
- [Test your strategy with Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/backtesting.md)
- [Edge positioning](https://github.com/mishaker/freqtrade/blob/money_mgt/docs/edge.md)
- [Find optimal parameters with Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md)
- [Control the bot with telegram](https://github.com/freqtrade/freqtrade/blob/develop/docs/telegram-usage.md)
- [Receive notifications via webhook](https://github.com/freqtrade/freqtrade/blob/develop/docs/webhook-config.md)

View File

@ -104,6 +104,14 @@ class Arguments(object):
type=str,
metavar='PATH',
)
self.parser.add_argument(
'--customhyperopt',
help='specify hyperopt class name (default: %(default)s)',
dest='hyperopt',
default=constants.DEFAULT_HYPEROPT,
type=str,
metavar='NAME',
)
self.parser.add_argument(
'--dynamic-whitelist',
help='dynamically generate and update whitelist'
@ -128,6 +136,22 @@ class Arguments(object):
"""
Parses given arguments for Backtesting scripts.
"""
parser.add_argument(
'--eps', '--enable-position-stacking',
help='Allow buying the same pair multiple times (position stacking)',
action='store_true',
dest='position_stacking',
default=False
)
parser.add_argument(
'--dmmp', '--disable-max-market-positions',
help='Disable applying `max_open_trades` during backtest '
'(same as setting `max_open_trades` to a very high number)',
action='store_false',
dest='use_max_market_positions',
default=True
)
parser.add_argument(
'-l', '--live',
help='using live data',
@ -171,6 +195,27 @@ class Arguments(object):
metavar='PATH',
)
@staticmethod
def edge_options(parser: argparse.ArgumentParser) -> None:
"""
Parses given arguments for Backtesting scripts.
"""
parser.add_argument(
'-r', '--refresh-pairs-cached',
help='refresh the pairs files in tests/testdata with the latest data from the '
'exchange. Use it if you want to run your edge with up-to-date data.',
action='store_true',
dest='refresh_pairs',
)
parser.add_argument(
'--stoplosses',
help='defines a range of stoploss against which edge will assess the strategy '
'the format is "min,max,step" (without any space).'
'example: --stoplosses=-0.01,-0.1,-0.001',
type=str,
dest='stoploss_range',
)
@staticmethod
def optimizer_shared_options(parser: argparse.ArgumentParser) -> None:
"""
@ -184,6 +229,20 @@ class Arguments(object):
dest='ticker_interval',
type=str,
)
parser.add_argument(
'--timerange',
help='specify what timerange of data to use.',
default=None,
type=str,
dest='timerange',
)
@staticmethod
def hyperopt_options(parser: argparse.ArgumentParser) -> None:
"""
Parses given arguments for Hyperopt scripts.
"""
parser.add_argument(
'--eps', '--enable-position-stacking',
help='Allow buying the same pair multiple times (position stacking)',
@ -200,20 +259,6 @@ class Arguments(object):
dest='use_max_market_positions',
default=True
)
parser.add_argument(
'--timerange',
help='specify what timerange of data to use.',
default=None,
type=str,
dest='timerange',
)
@staticmethod
def hyperopt_options(parser: argparse.ArgumentParser) -> None:
"""
Parses given arguments for Hyperopt scripts.
"""
parser.add_argument(
'-e', '--epochs',
help='specify number of epochs (default: %(default)d)',
@ -237,7 +282,7 @@ class Arguments(object):
Builds and attaches all subcommands
:return: None
"""
from freqtrade.optimize import backtesting, hyperopt
from freqtrade.optimize import backtesting, hyperopt, edge_cli
subparsers = self.parser.add_subparsers(dest='subparser')
@ -247,6 +292,12 @@ class Arguments(object):
self.optimizer_shared_options(backtesting_cmd)
self.backtesting_options(backtesting_cmd)
# Add edge subcommand
edge_cmd = subparsers.add_parser('edge', help='edge module')
edge_cmd.set_defaults(func=edge_cli.start)
self.optimizer_shared_options(edge_cmd)
self.edge_options(edge_cmd)
# Add hyperopt subcommand
hyperopt_cmd = subparsers.add_parser('hyperopt', help='hyperopt module')
hyperopt_cmd.set_defaults(func=hyperopt.start)

View File

@ -33,6 +33,7 @@ class Configuration(object):
Class to read and init the bot configuration
Reuse this class for the bot, backtesting, hyperopt and every script that required configuration
"""
def __init__(self, args: Namespace) -> None:
self.args = args
self.config: Optional[Dict[str, Any]] = None
@ -52,12 +53,18 @@ class Configuration(object):
if self.args.strategy_path:
config.update({'strategy_path': self.args.strategy_path})
# Add the hyperopt file to use
config.update({'hyperopt': self.args.hyperopt})
# Load Common configuration
config = self._load_common_config(config)
# Load Backtesting
config = self._load_backtesting_config(config)
# Load Edge
config = self._load_edge_config(config)
# Load Hyperopt
config = self._load_hyperopt_config(config)
@ -130,6 +137,10 @@ class Configuration(object):
if config.get('forcebuy_enable', False):
logger.warning('`forcebuy` RPC message enabled.')
# Setting max_open_trades to infinite if -1
if config.get('max_open_trades') == -1:
config['max_open_trades'] = float('inf')
logger.info(f'Using DB: "{config["db_url"]}"')
# Check if the exchange set by the user is supported
@ -213,6 +224,32 @@ class Configuration(object):
return config
def _load_edge_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
"""
Extract information for sys.argv and load Edge configuration
:return: configuration as dictionary
"""
# If --timerange is used we add it to the configuration
if 'timerange' in self.args and self.args.timerange:
config.update({'timerange': self.args.timerange})
logger.info('Parameter --timerange detected: %s ...', self.args.timerange)
# If --timerange is used we add it to the configuration
if 'stoploss_range' in self.args and self.args.stoploss_range:
txt_range = eval(self.args.stoploss_range)
config['edge'].update({'stoploss_range_min': txt_range[0]})
config['edge'].update({'stoploss_range_max': txt_range[1]})
config['edge'].update({'stoploss_range_step': txt_range[2]})
logger.info('Parameter --stoplosses detected: %s ...', self.args.stoploss_range)
# If -r/--refresh-pairs-cached is used we add it to the configuration
if 'refresh_pairs' in self.args and self.args.refresh_pairs:
config.update({'refresh_pairs': True})
logger.info('Parameter -r/--refresh-pairs-cached detected ...')
return config
def _load_hyperopt_config(self, config: Dict[str, Any]) -> Dict[str, Any]:
"""
Extract information for sys.argv and load Hyperopt configuration

View File

@ -9,9 +9,12 @@ TICKER_INTERVAL = 5 # min
HYPEROPT_EPOCH = 100 # epochs
RETRY_TIMEOUT = 30 # sec
DEFAULT_STRATEGY = 'DefaultStrategy'
DEFAULT_HYPEROPT = 'DefaultHyperOpts'
DEFAULT_DB_PROD_URL = 'sqlite:///tradesv3.sqlite'
DEFAULT_DB_DRYRUN_URL = 'sqlite://'
UNLIMITED_STAKE_AMOUNT = 'unlimited'
REQUIRED_ORDERTYPES = ['buy', 'sell', 'stoploss']
ORDERTYPE_POSSIBILITIES = ['limit', 'market']
TICKER_INTERVAL_MINUTES = {
@ -37,13 +40,13 @@ SUPPORTED_FIAT = [
"KRW", "MXN", "MYR", "NOK", "NZD", "PHP", "PKR", "PLN",
"RUB", "SEK", "SGD", "THB", "TRY", "TWD", "ZAR", "USD",
"BTC", "XBT", "ETH", "XRP", "LTC", "BCH", "USDT"
]
]
# Required json-schema for user specified config
CONF_SCHEMA = {
'type': 'object',
'properties': {
'max_open_trades': {'type': 'integer', 'minimum': 0},
'max_open_trades': {'type': 'integer', 'minimum': -1},
'ticker_interval': {'type': 'string', 'enum': list(TICKER_INTERVAL_MINUTES.keys())},
'stake_currency': {'type': 'string', 'enum': ['BTC', 'XBT', 'ETH', 'USDT', 'EUR', 'USD']},
'stake_amount': {
@ -101,7 +104,17 @@ CONF_SCHEMA = {
'order_book_max': {'type': 'number', 'minimum': 1, 'maximum': 50}
}
},
'order_types': {
'type': 'object',
'properties': {
'buy': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'sell': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES},
'stoploss': {'type': 'string', 'enum': ORDERTYPE_POSSIBILITIES}
},
'required': ['buy', 'sell', 'stoploss']
},
'exchange': {'$ref': '#/definitions/exchange'},
'edge': {'$ref': '#/definitions/edge'},
'experimental': {
'type': 'object',
'properties': {
@ -170,6 +183,23 @@ CONF_SCHEMA = {
'ccxt_async_config': {'type': 'object'}
},
'required': ['name', 'key', 'secret', 'pair_whitelist']
},
'edge': {
'type': 'object',
'properties': {
"enabled": {'type': 'boolean'},
"process_throttle_secs": {'type': 'integer', 'minimum': 600},
"calculate_since_number_of_days": {'type': 'integer'},
"allowed_risk": {'type': 'number'},
"stoploss_range_min": {'type': 'number'},
"stoploss_range_max": {'type': 'number'},
"stoploss_range_step": {'type': 'number'},
"minimum_winrate": {'type': 'number'},
"minimum_expectancy": {'type': 'number'},
"min_trade_number": {'type': 'number'},
"max_trade_duration_minute": {'type': 'integer'},
"remove_pumps": {'type': 'boolean'}
}
}
},
'anyOf': [

414
freqtrade/edge/__init__.py Normal file
View File

@ -0,0 +1,414 @@
# pragma pylint: disable=W0603
""" Edge positioning package """
import logging
from typing import Any, Dict, NamedTuple
import arrow
import numpy as np
import utils_find_1st as utf1st
from pandas import DataFrame
import freqtrade.optimize as optimize
from freqtrade.arguments import Arguments
from freqtrade.arguments import TimeRange
from freqtrade.strategy.interface import SellType
logger = logging.getLogger(__name__)
class PairInfo(NamedTuple):
stoploss: float
winrate: float
risk_reward_ratio: float
required_risk_reward: float
expectancy: float
nb_trades: int
avg_trade_duration: float
class Edge():
"""
Calculates Win Rate, Risk Reward Ratio, Expectancy
against historical data for a give set of markets and a strategy
it then adjusts stoploss and position size accordingly
and force it into the strategy
Author: https://github.com/mishaker
"""
config: Dict = {}
_cached_pairs: Dict[str, Any] = {} # Keeps a list of pairs
def __init__(self, config: Dict[str, Any], exchange, strategy) -> None:
self.config = config
self.exchange = exchange
self.strategy = strategy
self.ticker_interval = self.strategy.ticker_interval
self.tickerdata_to_dataframe = self.strategy.tickerdata_to_dataframe
self.get_timeframe = optimize.get_timeframe
self.advise_sell = self.strategy.advise_sell
self.advise_buy = self.strategy.advise_buy
self.edge_config = self.config.get('edge', {})
self._cached_pairs: Dict[str, Any] = {} # Keeps a list of pairs
self._total_capital: float = self.config['stake_amount']
self._allowed_risk: float = self.edge_config.get('allowed_risk')
self._since_number_of_days: int = self.edge_config.get('calculate_since_number_of_days', 14)
self._last_updated: int = 0 # Timestamp of pairs last updated time
self._refresh_pairs = True
self._stoploss_range_min = float(self.edge_config.get('stoploss_range_min', -0.01))
self._stoploss_range_max = float(self.edge_config.get('stoploss_range_max', -0.05))
self._stoploss_range_step = float(self.edge_config.get('stoploss_range_step', -0.001))
# calculating stoploss range
self._stoploss_range = np.arange(
self._stoploss_range_min,
self._stoploss_range_max,
self._stoploss_range_step
)
self._timerange: TimeRange = Arguments.parse_timerange("%s-" % arrow.now().shift(
days=-1 * self._since_number_of_days).format('YYYYMMDD'))
self.fee = self.exchange.get_fee()
def calculate(self) -> bool:
pairs = self.config['exchange']['pair_whitelist']
heartbeat = self.edge_config.get('process_throttle_secs')
if (self._last_updated > 0) and (
self._last_updated + heartbeat > arrow.utcnow().timestamp):
return False
data: Dict[str, Any] = {}
logger.info('Using stake_currency: %s ...', self.config['stake_currency'])
logger.info('Using local backtesting data (using whitelist in given config) ...')
data = optimize.load_data(
self.config['datadir'],
pairs=pairs,
ticker_interval=self.ticker_interval,
refresh_pairs=self._refresh_pairs,
exchange=self.exchange,
timerange=self._timerange
)
if not data:
# Reinitializing cached pairs
self._cached_pairs = {}
logger.critical("No data found. Edge is stopped ...")
return False
preprocessed = self.tickerdata_to_dataframe(data)
# Print timeframe
min_date, max_date = self.get_timeframe(preprocessed)
logger.info(
'Measuring data from %s up to %s (%s days) ...',
min_date.isoformat(),
max_date.isoformat(),
(max_date - min_date).days
)
headers = ['date', 'buy', 'open', 'close', 'sell', 'high', 'low']
trades: list = []
for pair, pair_data in preprocessed.items():
# Sorting dataframe by date and reset index
pair_data = pair_data.sort_values(by=['date'])
pair_data = pair_data.reset_index(drop=True)
ticker_data = self.advise_sell(
self.advise_buy(pair_data, {'pair': pair}), {'pair': pair})[headers].copy()
trades += self._find_trades_for_stoploss_range(ticker_data, pair, self._stoploss_range)
# If no trade found then exit
if len(trades) == 0:
return False
# Fill missing, calculable columns, profit, duration , abs etc.
trades_df = self._fill_calculable_fields(DataFrame(trades))
self._cached_pairs = self._process_expectancy(trades_df)
self._last_updated = arrow.utcnow().timestamp
# Not a nice hack but probably simplest solution:
# When backtest load data it loads the delta between disk and exchange
# The problem is that exchange consider that recent.
# it is but it is incomplete (c.f. _async_get_candle_history)
# So it causes get_signal to exit cause incomplete ticker_hist
# A patch to that would be update _pairs_last_refresh_time of exchange
# so it will download again all pairs
# Another solution is to add new data to klines instead of reassigning it:
# self.klines[pair].update(data) instead of self.klines[pair] = data in exchange package.
# But that means indexing timestamp and having a verification so that
# there is no empty range between two timestaps (recently added and last
# one)
self.exchange._pairs_last_refresh_time = {}
return True
def stake_amount(self, pair: str) -> float:
stoploss = self._cached_pairs[pair].stoploss
allowed_capital_at_risk = round(self._total_capital * self._allowed_risk, 5)
position_size = abs(round((allowed_capital_at_risk / stoploss), 5))
return position_size
def stoploss(self, pair: str) -> float:
return self._cached_pairs[pair].stoploss
def adjust(self, pairs) -> list:
"""
Filters out and sorts "pairs" according to Edge calculated pairs
"""
final = []
for pair, info in self._cached_pairs.items():
if info.expectancy > float(self.edge_config.get('minimum_expectancy', 0.2)) and \
info.winrate > float(self.edge_config.get('minimum_winrate', 0.60)) and \
pair in pairs:
final.append(pair)
if final:
logger.info('Edge validated only %s', final)
else:
logger.info('Edge removed all pairs as no pair with minimum expectancy was found !')
return final
def _fill_calculable_fields(self, result: DataFrame) -> DataFrame:
"""
The result frame contains a number of columns that are calculable
from other columns. These are left blank till all rows are added,
to be populated in single vector calls.
Columns to be populated are:
- Profit
- trade duration
- profit abs
:param result Dataframe
:return: result Dataframe
"""
# stake and fees
# stake = 0.015
# 0.05% is 0.0005
# fee = 0.001
stake = self.config.get('stake_amount')
fee = self.fee
open_fee = fee / 2
close_fee = fee / 2
result['trade_duration'] = result['close_time'] - result['open_time']
result['trade_duration'] = result['trade_duration'].map(
lambda x: int(x.total_seconds() / 60))
# Spends, Takes, Profit, Absolute Profit
# Buy Price
result['buy_vol'] = stake / result['open_rate'] # How many target are we buying
result['buy_fee'] = stake * open_fee
result['buy_spend'] = stake + result['buy_fee'] # How much we're spending
# Sell price
result['sell_sum'] = result['buy_vol'] * result['close_rate']
result['sell_fee'] = result['sell_sum'] * close_fee
result['sell_take'] = result['sell_sum'] - result['sell_fee']
# profit_percent
result['profit_percent'] = (result['sell_take'] - result['buy_spend']) / result['buy_spend']
# Absolute profit
result['profit_abs'] = result['sell_take'] - result['buy_spend']
return result
def _process_expectancy(self, results: DataFrame) -> Dict[str, Any]:
"""
This calculates WinRate, Required Risk Reward, Risk Reward and Expectancy of all pairs
The calulation will be done per pair and per strategy.
"""
# Removing pairs having less than min_trades_number
min_trades_number = self.edge_config.get('min_trade_number', 10)
results = results.groupby(['pair', 'stoploss']).filter(lambda x: len(x) > min_trades_number)
###################################
# Removing outliers (Only Pumps) from the dataset
# The method to detect outliers is to calculate standard deviation
# Then every value more than (standard deviation + 2*average) is out (pump)
#
# Removing Pumps
if self.edge_config.get('remove_pumps', False):
results = results.groupby(['pair', 'stoploss']).apply(
lambda x: x[x['profit_abs'] < 2 * x['profit_abs'].std() + x['profit_abs'].mean()])
##########################################################################
# Removing trades having a duration more than X minutes (set in config)
max_trade_duration = self.edge_config.get('max_trade_duration_minute', 1440)
results = results[results.trade_duration < max_trade_duration]
#######################################################################
if results.empty:
return {}
groupby_aggregator = {
'profit_abs': [
('nb_trades', 'count'), # number of all trades
('profit_sum', lambda x: x[x > 0].sum()), # cumulative profit of all winning trades
('loss_sum', lambda x: abs(x[x < 0].sum())), # cumulative loss of all losing trades
('nb_win_trades', lambda x: x[x > 0].count()) # number of winning trades
],
'trade_duration': [('avg_trade_duration', 'mean')]
}
# Group by (pair and stoploss) by applying above aggregator
df = results.groupby(['pair', 'stoploss'])['profit_abs', 'trade_duration'].agg(
groupby_aggregator).reset_index(col_level=1)
# Dropping level 0 as we don't need it
df.columns = df.columns.droplevel(0)
# Calculating number of losing trades, average win and average loss
df['nb_loss_trades'] = df['nb_trades'] - df['nb_win_trades']
df['average_win'] = df['profit_sum'] / df['nb_win_trades']
df['average_loss'] = df['loss_sum'] / df['nb_loss_trades']
# Win rate = number of profitable trades / number of trades
df['winrate'] = df['nb_win_trades'] / df['nb_trades']
# risk_reward_ratio = average win / average loss
df['risk_reward_ratio'] = df['average_win'] / df['average_loss']
# required_risk_reward = (1 / winrate) - 1
df['required_risk_reward'] = (1 / df['winrate']) - 1
# expectancy = (risk_reward_ratio * winrate) - (lossrate)
df['expectancy'] = (df['risk_reward_ratio'] * df['winrate']) - (1 - df['winrate'])
# sort by expectancy and stoploss
df = df.sort_values(by=['expectancy', 'stoploss'], ascending=False).groupby(
'pair').first().sort_values(by=['expectancy'], ascending=False).reset_index()
final = {}
for x in df.itertuples():
final[x.pair] = PairInfo(
x.stoploss,
x.winrate,
x.risk_reward_ratio,
x.required_risk_reward,
x.expectancy,
x.nb_trades,
x.avg_trade_duration
)
# Returning a list of pairs in order of "expectancy"
return final
def _find_trades_for_stoploss_range(self, ticker_data, pair, stoploss_range):
buy_column = ticker_data['buy'].values
sell_column = ticker_data['sell'].values
date_column = ticker_data['date'].values
ohlc_columns = ticker_data[['open', 'high', 'low', 'close']].values
result: list = []
for stoploss in stoploss_range:
result += self._detect_next_stop_or_sell_point(
buy_column, sell_column, date_column, ohlc_columns, round(stoploss, 6), pair
)
return result
def _detect_next_stop_or_sell_point(self, buy_column, sell_column, date_column,
ohlc_columns, stoploss, pair, start_point=0):
"""
Iterate through ohlc_columns recursively in order to find the next trade
Next trade opens from the first buy signal noticed to
The sell or stoploss signal after it.
It then calls itself cutting OHLC, buy_column, sell_colum and date_column
Cut from (the exit trade index) + 1
Author: https://github.com/mishaker
"""
result: list = []
open_trade_index = utf1st.find_1st(buy_column, 1, utf1st.cmp_equal)
# return empty if we don't find trade entry (i.e. buy==1) or
# we find a buy but at the of array
if open_trade_index == -1 or open_trade_index == len(buy_column) - 1:
return []
else:
open_trade_index += 1 # when a buy signal is seen,
# trade opens in reality on the next candle
stop_price_percentage = stoploss + 1
open_price = ohlc_columns[open_trade_index, 0]
stop_price = (open_price * stop_price_percentage)
# Searching for the index where stoploss is hit
stop_index = utf1st.find_1st(
ohlc_columns[open_trade_index:, 2], stop_price, utf1st.cmp_smaller)
# If we don't find it then we assume stop_index will be far in future (infinite number)
if stop_index == -1:
stop_index = float('inf')
# Searching for the index where sell is hit
sell_index = utf1st.find_1st(sell_column[open_trade_index:], 1, utf1st.cmp_equal)
# If we don't find it then we assume sell_index will be far in future (infinite number)
if sell_index == -1:
sell_index = float('inf')
# Check if we don't find any stop or sell point (in that case trade remains open)
# It is not interesting for Edge to consider it so we simply ignore the trade
# And stop iterating there is no more entry
if stop_index == sell_index == float('inf'):
return []
if stop_index <= sell_index:
exit_index = open_trade_index + stop_index
exit_type = SellType.STOP_LOSS
exit_price = stop_price
elif stop_index > sell_index:
# if exit is SELL then we exit at the next candle
exit_index = open_trade_index + sell_index + 1
# check if we have the next candle
if len(ohlc_columns) - 1 < exit_index:
return []
exit_type = SellType.SELL_SIGNAL
exit_price = ohlc_columns[exit_index, 0]
trade = {'pair': pair,
'stoploss': stoploss,
'profit_percent': '',
'profit_abs': '',
'open_time': date_column[open_trade_index],
'close_time': date_column[exit_index],
'open_index': start_point + open_trade_index,
'close_index': start_point + exit_index,
'trade_duration': '',
'open_rate': round(open_price, 15),
'close_rate': round(exit_price, 15),
'exit_type': exit_type
}
result.append(trade)
# Calling again the same function recursively but giving
# it a view of exit_index till the end of array
return result + self._detect_next_stop_or_sell_point(
buy_column[exit_index:],
sell_column[exit_index:],
date_column[exit_index:],
ohlc_columns[exit_index:],
stoploss,
pair,
(start_point + exit_index)
)

View File

@ -102,7 +102,7 @@ class Exchange(object):
self.markets = self._load_markets()
# Check if all pairs are available
self.validate_pairs(config['exchange']['pair_whitelist'])
self.validate_ordertypes(config.get('order_types', {}))
if config.get('ticker_interval'):
# Check if timeframe is available
self.validate_timeframes(config['ticker_interval'])
@ -218,6 +218,15 @@ class Exchange(object):
raise OperationalException(
f'Invalid ticker {timeframe}, this Exchange supports {timeframes}')
def validate_ordertypes(self, order_types: Dict) -> None:
"""
Checks if order-types configured in strategy/config are supported
"""
if any(v == 'market' for k, v in order_types.items()):
if not self.exchange_has('createMarketOrder'):
raise OperationalException(
f'Exchange {self.name} does not support market orders.')
def exchange_has(self, endpoint: str) -> bool:
"""
Checks if exchange implements a specific API endpoint.
@ -249,14 +258,14 @@ class Exchange(object):
price = ceil(big_price) / pow(10, symbol_prec)
return price
def buy(self, pair: str, rate: float, amount: float) -> Dict:
def buy(self, pair: str, ordertype: str, amount: float, rate: float) -> Dict:
if self._conf['dry_run']:
order_id = f'dry_run_buy_{randint(0, 10**6)}'
self._dry_run_open_orders[order_id] = {
'pair': pair,
'price': rate,
'amount': amount,
'type': 'limit',
'type': ordertype,
'side': 'buy',
'remaining': 0.0,
'datetime': arrow.utcnow().isoformat(),
@ -268,9 +277,9 @@ class Exchange(object):
try:
# Set the precision for amount and price(rate) as accepted by the exchange
amount = self.symbol_amount_prec(pair, amount)
rate = self.symbol_price_prec(pair, rate)
rate = self.symbol_price_prec(pair, rate) if ordertype != 'market' else None
return self._api.create_limit_buy_order(pair, amount, rate)
return self._api.create_order(pair, ordertype, 'buy', amount, rate)
except ccxt.InsufficientFunds as e:
raise DependencyException(
f'Insufficient funds to create limit buy order on market {pair}.'
@ -287,14 +296,14 @@ class Exchange(object):
except ccxt.BaseError as e:
raise OperationalException(e)
def sell(self, pair: str, rate: float, amount: float) -> Dict:
def sell(self, pair: str, ordertype: str, amount: float, rate: float) -> Dict:
if self._conf['dry_run']:
order_id = f'dry_run_sell_{randint(0, 10**6)}'
self._dry_run_open_orders[order_id] = {
'pair': pair,
'price': rate,
'amount': amount,
'type': 'limit',
'type': ordertype,
'side': 'sell',
'remaining': 0.0,
'datetime': arrow.utcnow().isoformat(),
@ -305,9 +314,9 @@ class Exchange(object):
try:
# Set the precision for amount and price(rate) as accepted by the exchange
amount = self.symbol_amount_prec(pair, amount)
rate = self.symbol_price_prec(pair, rate)
rate = self.symbol_price_prec(pair, rate) if ordertype != 'market' else None
return self._api.create_limit_sell_order(pair, amount, rate)
return self._api.create_order(pair, ordertype, 'sell', amount, rate)
except ccxt.InsufficientFunds as e:
raise DependencyException(
f'Insufficient funds to create limit sell order on market {pair}.'

View File

@ -17,6 +17,8 @@ from cachetools import TTLCache, cached
from freqtrade import (DependencyException, OperationalException,
TemporaryError, __version__, constants, persistence)
from freqtrade.exchange import Exchange
from freqtrade.wallets import Wallets
from freqtrade.edge import Edge
from freqtrade.persistence import Trade
from freqtrade.rpc import RPCManager, RPCMessageType
from freqtrade.state import State
@ -24,6 +26,7 @@ from freqtrade.strategy.interface import SellType
from freqtrade.strategy.resolver import IStrategy, StrategyResolver
from freqtrade.exchange.exchange_helpers import order_book_to_dataframe
logger = logging.getLogger(__name__)
@ -54,6 +57,12 @@ class FreqtradeBot(object):
self.rpc: RPCManager = RPCManager(self)
self.persistence = None
self.exchange = Exchange(self.config)
self.wallets = Wallets(self.exchange)
# Initializing Edge only if enabled
self.edge = Edge(self.config, self.exchange, self.strategy) if \
self.config.get('edge', {}).get('enabled', False) else None
self.active_pair_whitelist: List[str] = self.config['exchange']['pair_whitelist']
self._init_modules()
@ -133,7 +142,7 @@ class FreqtradeBot(object):
f'*Strategy:* `{strategy_name}`'
})
if self.config.get('dynamic_whitelist', False):
top_pairs = 'top ' + str(self.config.get('dynamic_whitelist', 20))
top_pairs = 'top volume ' + str(self.config.get('dynamic_whitelist', 20))
specific_pairs = ''
else:
top_pairs = 'whitelisted'
@ -179,6 +188,14 @@ class FreqtradeBot(object):
# Keep only the subsets of pairs wanted (up to nb_assets)
self.active_pair_whitelist = sanitized_list[:nb_assets] if nb_assets else sanitized_list
# Calculating Edge positiong
# Should be called before refresh_tickers
# Otherwise it will override cached klines in exchange
# with delta value (klines only from last refresh_pairs)
if self.edge:
self.edge.calculate()
self.active_pair_whitelist = self.edge.adjust(self.active_pair_whitelist)
# Query trades from persistence layer
trades = Trade.query.filter(Trade.is_open.is_(True)).all()
@ -309,14 +326,20 @@ class FreqtradeBot(object):
return used_rate
def _get_trade_stake_amount(self) -> Optional[float]:
def _get_trade_stake_amount(self, pair) -> Optional[float]:
"""
Check if stake amount can be fulfilled with the available balance
for the stake currency
:return: float: Stake Amount
"""
stake_amount = self.config['stake_amount']
if self.edge:
stake_amount = self.edge.stake_amount(pair)
else:
stake_amount = self.config['stake_amount']
# TODO: should come from the wallet
avaliable_amount = self.exchange.get_balance(self.config['stake_currency'])
# avaliable_amount = self.wallets.wallets[self.config['stake_currency']].free
if stake_amount == constants.UNLIMITED_STAKE_AMOUNT:
open_trades = len(Trade.query.filter(Trade.is_open.is_(True)).all())
@ -373,15 +396,6 @@ class FreqtradeBot(object):
:return: True if a trade object has been created and persisted, False otherwise
"""
interval = self.strategy.ticker_interval
stake_amount = self._get_trade_stake_amount()
if not stake_amount:
return False
logger.info(
'Checking buy signals to create a new trade with stake_amount: %f ...',
stake_amount
)
whitelist = copy.deepcopy(self.active_pair_whitelist)
# Remove currently opened and latest pairs from whitelist
@ -394,10 +408,18 @@ class FreqtradeBot(object):
raise DependencyException('No currency pairs in whitelist')
# running get_signal on historical data fetched
# to find buy signals
for _pair in whitelist:
(buy, sell) = self.strategy.get_signal(_pair, interval, self.exchange.klines.get(_pair))
if buy and not sell:
stake_amount = self._get_trade_stake_amount(_pair)
if not stake_amount:
return False
logger.info(
'Buy signal found: about create a new trade with stake_amount: %f ...',
stake_amount
)
bidstrat_check_depth_of_market = self.config.get('bid_strategy', {}).\
get('check_depth_of_market', {})
if (bidstrat_check_depth_of_market.get('enabled', False)) and\
@ -454,7 +476,8 @@ class FreqtradeBot(object):
amount = stake_amount / buy_limit
order_id = self.exchange.buy(pair, buy_limit, amount)['id']
order_id = self.exchange.buy(pair=pair, ordertype=self.strategy.order_types['buy'],
amount=amount, rate=buy_limit)['id']
self.rpc.send_msg({
'type': RPCMessageType.BUY_NOTIFICATION,
@ -484,6 +507,10 @@ class FreqtradeBot(object):
)
Trade.session.add(trade)
Trade.session.flush()
# Updating wallets
self.wallets.update()
return True
def process_maybe_execute_buy(self) -> bool:
@ -528,7 +555,14 @@ class FreqtradeBot(object):
if trade.is_open and trade.open_order_id is None:
# Check if we can sell our current pair
return self.handle_trade(trade)
result = self.handle_trade(trade)
# Updating wallets if any trade occured
if result:
self.wallets.update()
return result
except DependencyException as exception:
logger.warning('Unable to sell trade: %s', exception)
return False
@ -624,10 +658,16 @@ class FreqtradeBot(object):
return False
def check_sell(self, trade: Trade, sell_rate: float, buy: bool, sell: bool) -> bool:
should_sell = self.strategy.should_sell(trade, sell_rate, datetime.utcnow(), buy, sell)
if self.edge:
stoploss = self.edge.stoploss(trade.pair)
should_sell = self.strategy.should_sell(
trade, sell_rate, datetime.utcnow(), buy, sell, force_stoploss=stoploss)
else:
should_sell = self.strategy.should_sell(trade, sell_rate, datetime.utcnow(), buy, sell)
if should_sell.sell_flag:
self.execute_sell(trade, sell_rate, should_sell.sell_type)
logger.info('excuted sell')
logger.info('executed sell, reason: %s', should_sell.sell_type)
return True
return False
@ -661,14 +701,17 @@ class FreqtradeBot(object):
# Check if trade is still actually open
if int(order['remaining']) == 0:
self.wallets.update()
continue
# Check if trade is still actually open
if order['status'] == 'open':
if order['side'] == 'buy' and ordertime < buy_timeoutthreashold:
self.handle_timedout_limit_buy(trade, order)
self.wallets.update()
elif order['side'] == 'sell' and ordertime < sell_timeoutthreashold:
self.handle_timedout_limit_sell(trade, order)
self.wallets.update()
# FIX: 20180110, why is cancel.order unconditionally here, whereas
# it is conditionally called in the
@ -735,8 +778,13 @@ class FreqtradeBot(object):
:param sellreason: Reason the sell was triggered
:return: None
"""
sell_type = 'sell'
if sell_reason in (SellType.STOP_LOSS, SellType.TRAILING_STOP_LOSS):
sell_type = 'stoploss'
# Execute sell and update trade record
order_id = self.exchange.sell(str(trade.pair), limit, trade.amount)['id']
order_id = self.exchange.sell(pair=str(trade.pair),
ordertype=self.strategy.order_types[sell_type],
amount=trade.amount, rate=limit)['id']
trade.open_order_id = order_id
trade.close_rate_requested = limit
trade.sell_reason = sell_reason.value

View File

@ -20,6 +20,7 @@ from pandas import DataFrame
from freqtrade import misc, constants, OperationalException
from freqtrade.exchange import Exchange
from freqtrade.arguments import TimeRange
from freqtrade.optimize.default_hyperopt import DefaultHyperOpts # noqa: F401
logger = logging.getLogger(__name__)

View File

@ -0,0 +1,130 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
import talib.abstract as ta
from pandas import DataFrame
from typing import Dict, Any, Callable, List
from functools import reduce
from skopt.space import Categorical, Dimension, Integer, Real
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.optimize.hyperopt_interface import IHyperOpt
class_name = 'DefaultHyperOpts'
class DefaultHyperOpts(IHyperOpt):
"""
Default hyperopt provided by freqtrade bot.
You can override it with your own hyperopt
"""
@staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe)
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['mfi'] = ta.MFI(dataframe)
dataframe['rsi'] = ta.RSI(dataframe)
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['sar'] = ta.SAR(dataframe)
return dataframe
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by hyperopt
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Buy strategy Hyperopt will build and use
"""
conditions = []
# GUARDS AND TRENDS
if 'mfi-enabled' in params and params['mfi-enabled']:
conditions.append(dataframe['mfi'] < params['mfi-value'])
if 'fastd-enabled' in params and params['fastd-enabled']:
conditions.append(dataframe['fastd'] < params['fastd-value'])
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
if params['trigger'] == 'sar_reversal':
conditions.append(qtpylib.crossed_above(
dataframe['close'], dataframe['sar']
))
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
return populate_buy_trend
@staticmethod
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(10, 25, name='mfi-value'),
Integer(15, 45, name='fastd-value'),
Integer(20, 50, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='mfi-enabled'),
Categorical([True, False], name='fastd-enabled'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal', 'sar_reversal'], name='trigger')
]
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss Value to search
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]

View File

@ -0,0 +1,106 @@
# pragma pylint: disable=missing-docstring, W0212, too-many-arguments
"""
This module contains the backtesting logic
"""
import logging
from argparse import Namespace
from typing import Dict, Any
from tabulate import tabulate
from freqtrade.edge import Edge
from freqtrade.configuration import Configuration
from freqtrade.arguments import Arguments
from freqtrade.exchange import Exchange
from freqtrade.strategy.resolver import StrategyResolver
logger = logging.getLogger(__name__)
class EdgeCli(object):
"""
Backtesting class, this class contains all the logic to run a backtest
To run a backtest:
backtesting = Backtesting(config)
backtesting.start()
"""
def __init__(self, config: Dict[str, Any]) -> None:
self.config = config
# Reset keys for edge
self.config['exchange']['key'] = ''
self.config['exchange']['secret'] = ''
self.config['exchange']['password'] = ''
self.config['exchange']['uid'] = ''
self.config['dry_run'] = True
self.exchange = Exchange(self.config)
self.strategy = StrategyResolver(self.config).strategy
self.edge = Edge(config, self.exchange, self.strategy)
self.edge._refresh_pairs = self.config.get('refresh_pairs', False)
self.timerange = Arguments.parse_timerange(None if self.config.get(
'timerange') is None else str(self.config.get('timerange')))
self.edge._timerange = self.timerange
def _generate_edge_table(self, results: dict) -> str:
floatfmt = ('s', '.10g', '.2f', '.2f', '.2f', '.2f', 'd', '.d')
tabular_data = []
headers = ['pair', 'stoploss', 'win rate', 'risk reward ratio',
'required risk reward', 'expectancy', 'total number of trades',
'average duration (min)']
for result in results.items():
if result[1].nb_trades > 0:
tabular_data.append([
result[0],
result[1].stoploss,
result[1].winrate,
result[1].risk_reward_ratio,
result[1].required_risk_reward,
result[1].expectancy,
result[1].nb_trades,
round(result[1].avg_trade_duration)
])
return tabulate(tabular_data, headers=headers, floatfmt=floatfmt, tablefmt="pipe")
def start(self) -> None:
self.edge.calculate()
print('') # blank like for readability
print(self._generate_edge_table(self.edge._cached_pairs))
def setup_configuration(args: Namespace) -> Dict[str, Any]:
"""
Prepare the configuration for the backtesting
:param args: Cli args from Arguments()
:return: Configuration
"""
configuration = Configuration(args)
config = configuration.get_config()
# Ensure we do not use Exchange credentials
config['exchange']['key'] = ''
config['exchange']['secret'] = ''
return config
def start(args: Namespace) -> None:
"""
Start Edge script
:param args: Cli args from Arguments()
:return: None
"""
# Initialize configuration
config = setup_configuration(args)
logger.info('Starting freqtrade in Edge mode')
# Initialize Edge object
edge_cli = EdgeCli(config)
edge_cli.start()

View File

@ -9,22 +9,21 @@ import multiprocessing
import os
import sys
from argparse import Namespace
from functools import reduce
from math import exp
from operator import itemgetter
from typing import Any, Callable, Dict, List
from typing import Any, Dict, List
import talib.abstract as ta
from pandas import DataFrame
from sklearn.externals.joblib import Parallel, delayed, dump, load
from joblib import Parallel, delayed, dump, load, wrap_non_picklable_objects
from skopt import Optimizer
from skopt.space import Categorical, Dimension, Integer, Real
from skopt.space import Dimension
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.arguments import Arguments
from freqtrade.configuration import Configuration
from freqtrade.optimize import load_data, get_timeframe
from freqtrade.optimize.backtesting import Backtesting
from freqtrade.optimize.hyperopt_resolver import HyperOptResolver
logger = logging.getLogger(__name__)
@ -42,6 +41,9 @@ class Hyperopt(Backtesting):
"""
def __init__(self, config: Dict[str, Any]) -> None:
super().__init__(config)
self.config = config
self.custom_hyperopt = HyperOptResolver(self.config).hyperopt
# set TARGET_TRADES to suit your number concurrent trades so its realistic
# to the number of days
self.target_trades = 600
@ -74,24 +76,6 @@ class Hyperopt(Backtesting):
arg_dict = {dim.name: value for dim, value in zip(dimensions, params)}
return arg_dict
@staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe)
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['mfi'] = ta.MFI(dataframe)
dataframe['rsi'] = ta.RSI(dataframe)
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['sar'] = ta.SAR(dataframe)
return dataframe
def save_trials(self) -> None:
"""
Save hyperopt trials to file
@ -121,7 +105,8 @@ class Hyperopt(Backtesting):
best_result['params']
)
if 'roi_t1' in best_result['params']:
logger.info('ROI table:\n%s', self.generate_roi_table(best_result['params']))
logger.info('ROI table:\n%s',
self.custom_hyperopt.generate_roi_table(best_result['params']))
def log_results(self, results) -> None:
"""
@ -149,59 +134,6 @@ class Hyperopt(Backtesting):
result = trade_loss + profit_loss + duration_loss
return result
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss search space
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(10, 25, name='mfi-value'),
Integer(15, 45, name='fastd-value'),
Integer(20, 50, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='mfi-enabled'),
Categorical([True, False], name='fastd-enabled'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal', 'sar_reversal'], name='trigger')
]
def has_space(self, space: str) -> bool:
"""
Tell if a space value is contained in the configuration
@ -216,61 +148,20 @@ class Hyperopt(Backtesting):
"""
spaces: List[Dimension] = []
if self.has_space('buy'):
spaces += Hyperopt.indicator_space()
spaces += self.custom_hyperopt.indicator_space()
if self.has_space('roi'):
spaces += Hyperopt.roi_space()
spaces += self.custom_hyperopt.roi_space()
if self.has_space('stoploss'):
spaces += Hyperopt.stoploss_space()
spaces += self.custom_hyperopt.stoploss_space()
return spaces
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by hyperopt
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Buy strategy Hyperopt will build and use
"""
conditions = []
# GUARDS AND TRENDS
if 'mfi-enabled' in params and params['mfi-enabled']:
conditions.append(dataframe['mfi'] < params['mfi-value'])
if 'fastd-enabled' in params and params['fastd-enabled']:
conditions.append(dataframe['fastd'] < params['fastd-value'])
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
if params['trigger'] == 'sar_reversal':
conditions.append(qtpylib.crossed_above(
dataframe['close'], dataframe['sar']
))
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
return populate_buy_trend
def generate_optimizer(self, _params) -> Dict:
def generate_optimizer(self, _params: Dict) -> Dict:
params = self.get_args(_params)
if self.has_space('roi'):
self.strategy.minimal_roi = self.generate_roi_table(params)
self.strategy.minimal_roi = self.custom_hyperopt.generate_roi_table(params)
if self.has_space('buy'):
self.advise_buy = self.buy_strategy_generator(params)
self.advise_buy = self.custom_hyperopt.buy_strategy_generator(params)
if self.has_space('stoploss'):
self.strategy.stoploss = params['stoploss']
@ -332,7 +223,8 @@ class Hyperopt(Backtesting):
)
def run_optimizer_parallel(self, parallel, asked) -> List:
return parallel(delayed(self.generate_optimizer)(v) for v in asked)
return parallel(delayed(
wrap_non_picklable_objects(self.generate_optimizer))(v) for v in asked)
def load_previous_results(self):
""" read trials file if we have one """
@ -354,7 +246,8 @@ class Hyperopt(Backtesting):
)
if self.has_space('buy'):
self.strategy.advise_indicators = Hyperopt.populate_indicators # type: ignore
self.strategy.advise_indicators = \
self.custom_hyperopt.populate_indicators # type: ignore
dump(self.strategy.tickerdata_to_dataframe(data), TICKERDATA_PICKLE)
self.exchange = None # type: ignore
self.load_previous_results()

View File

@ -0,0 +1,66 @@
"""
IHyperOpt interface
This module defines the interface to apply for hyperopts
"""
from abc import ABC, abstractmethod
from typing import Dict, Any, Callable, List
from pandas import DataFrame
from skopt.space import Dimension
class IHyperOpt(ABC):
"""
Interface for freqtrade hyperopts
Defines the mandatory structure must follow any custom strategies
Attributes you can use:
minimal_roi -> Dict: Minimal ROI designed for the strategy
stoploss -> float: optimal stoploss designed for the strategy
ticker_interval -> int: value of the ticker interval to use for the strategy
"""
@staticmethod
@abstractmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Populate indicators that will be used in the Buy and Sell strategy
:param dataframe: Raw data from the exchange and parsed by parse_ticker_dataframe()
:return: a Dataframe with all mandatory indicators for the strategies
"""
@staticmethod
@abstractmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Create a buy strategy generator
"""
@staticmethod
@abstractmethod
def indicator_space() -> List[Dimension]:
"""
Create an indicator space
"""
@staticmethod
@abstractmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Create an roi table
"""
@staticmethod
@abstractmethod
def stoploss_space() -> List[Dimension]:
"""
Create a stoploss space
"""
@staticmethod
@abstractmethod
def roi_space() -> List[Dimension]:
"""
Create a roi space
"""

View File

@ -0,0 +1,104 @@
# pragma pylint: disable=attribute-defined-outside-init
"""
This module load custom hyperopts
"""
import importlib.util
import inspect
import logging
import os
from typing import Optional, Dict, Type
from freqtrade.constants import DEFAULT_HYPEROPT
from freqtrade.optimize.hyperopt_interface import IHyperOpt
logger = logging.getLogger(__name__)
class HyperOptResolver(object):
"""
This class contains all the logic to load custom hyperopt class
"""
__slots__ = ['hyperopt']
def __init__(self, config: Optional[Dict] = None) -> None:
"""
Load the custom class from config parameter
:param config: configuration dictionary or None
"""
config = config or {}
# Verify the hyperopt is in the configuration, otherwise fallback to the default hyperopt
hyperopt_name = config.get('hyperopt') or DEFAULT_HYPEROPT
self.hyperopt = self._load_hyperopt(hyperopt_name, extra_dir=config.get('hyperopt_path'))
def _load_hyperopt(
self, hyperopt_name: str, extra_dir: Optional[str] = None) -> IHyperOpt:
"""
Search and loads the specified hyperopt.
:param hyperopt_name: name of the module to import
:param extra_dir: additional directory to search for the given hyperopt
:return: HyperOpt instance or None
"""
current_path = os.path.dirname(os.path.realpath(__file__))
abs_paths = [
os.path.join(current_path, '..', '..', 'user_data', 'hyperopts'),
current_path,
]
if extra_dir:
# Add extra hyperopt directory on top of search paths
abs_paths.insert(0, extra_dir)
for path in abs_paths:
hyperopt = self._search_hyperopt(path, hyperopt_name)
if hyperopt:
logger.info('Using resolved hyperopt %s from \'%s\'', hyperopt_name, path)
return hyperopt
raise ImportError(
"Impossible to load Hyperopt '{}'. This class does not exist"
" or contains Python code errors".format(hyperopt_name)
)
@staticmethod
def _get_valid_hyperopts(module_path: str, hyperopt_name: str) -> Optional[Type[IHyperOpt]]:
"""
Returns a list of all possible hyperopts for the given module_path
:param module_path: absolute path to the module
:param hyperopt_name: Class name of the hyperopt
:return: Tuple with (name, class) or None
"""
# Generate spec based on absolute path
spec = importlib.util.spec_from_file_location('user_data.hyperopts', module_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) # type: ignore # importlib does not use typehints
valid_hyperopts_gen = (
obj for name, obj in inspect.getmembers(module, inspect.isclass)
if hyperopt_name == name and IHyperOpt in obj.__bases__
)
return next(valid_hyperopts_gen, None)
@staticmethod
def _search_hyperopt(directory: str, hyperopt_name: str) -> Optional[IHyperOpt]:
"""
Search for the hyperopt_name in the given directory
:param directory: relative or absolute directory path
:return: name of the hyperopt class
"""
logger.debug('Searching for hyperopt %s in \'%s\'', hyperopt_name, directory)
for entry in os.listdir(directory):
# Only consider python files
if not entry.endswith('.py'):
logger.debug('Ignoring %s', entry)
continue
hyperopt = HyperOptResolver._get_valid_hyperopts(
os.path.abspath(os.path.join(directory, entry)), hyperopt_name
)
if hyperopt:
return hyperopt()
return None

View File

@ -410,7 +410,7 @@ class RPC(object):
raise RPCException(f'position for {pair} already open - id: {trade.id}')
# gen stake amount
stakeamount = self._freqtrade._get_trade_stake_amount()
stakeamount = self._freqtrade._get_trade_stake_amount(pair)
# execute buy
if self._freqtrade.execute_buy(pair, stakeamount, price):
@ -443,3 +443,10 @@ class RPC(object):
raise RPCException('trader is not running')
return Trade.query.filter(Trade.is_open.is_(True)).all()
def _rpc_whitelist(self) -> Dict:
""" Returns the currently active whitelist"""
res = {'method': self._freqtrade.config.get('dynamic_whitelist', 0) or 'static',
'whitelist': self._freqtrade.active_pair_whitelist
}
return res

View File

@ -91,6 +91,7 @@ class Telegram(RPC):
CommandHandler('daily', self._daily),
CommandHandler('count', self._count),
CommandHandler('reload_conf', self._reload_conf),
CommandHandler('whitelist', self._whitelist),
CommandHandler('help', self._help),
CommandHandler('version', self._version),
]
@ -438,6 +439,25 @@ class Telegram(RPC):
except RPCException as e:
self._send_msg(str(e), bot=bot)
@authorized_only
def _whitelist(self, bot: Bot, update: Update) -> None:
"""
Handler for /whitelist
Shows the currently active whitelist
"""
try:
whitelist = self._rpc_whitelist()
if whitelist['method'] == 'static':
message = f"Using static whitelist with `{len(whitelist['whitelist'])}` pairs \n"
else:
message = f"Dynamic whitelist with `{whitelist['method']}` pairs\n"
message += f"`{', '.join(whitelist['whitelist'])}`"
logger.debug(message)
self._send_msg(message)
except RPCException as e:
self._send_msg(str(e), bot=bot)
@authorized_only
def _help(self, bot: Bot, update: Update) -> None:
"""
@ -460,6 +480,7 @@ class Telegram(RPC):
"\n" \
"*/balance:* `Show account balance per currency`\n" \
"*/reload_conf:* `Reload configuration file` \n" \
"*/whitelist:* `Show current whitelist` \n" \
"*/help:* `This help message`\n" \
"*/version:* `Show version`"

View File

@ -28,6 +28,13 @@ class DefaultStrategy(IStrategy):
# Optimal ticker interval for the strategy
ticker_interval = '5m'
# Optional order type mapping
order_types = {
'buy': 'limit',
'sell': 'limit',
'stoploss': 'limit'
}
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame

View File

@ -70,6 +70,13 @@ class IStrategy(ABC):
# associated ticker interval
ticker_interval: str
# Optional order types
order_types: Dict = {
'buy': 'limit',
'sell': 'limit',
'stoploss': 'limit'
}
# run "populate_indicators" only for new candle
process_only_new_candles: bool = False
@ -203,17 +210,20 @@ class IStrategy(ABC):
return buy, sell
def should_sell(self, trade: Trade, rate: float, date: datetime, buy: bool,
sell: bool, low: float = None, high: float = None) -> SellCheckTuple:
sell: bool, low: float = None, high: float = None,
force_stoploss: float = 0) -> SellCheckTuple:
"""
This function evaluate if on the condition required to trigger a sell has been reached
if the threshold is reached and updates the trade record.
:return: True if trade should be sold, False otherwise
"""
# Set current rate to low for backtesting sell
current_rate = low or rate
current_profit = trade.calc_profit_percent(current_rate)
stoplossflag = self.stop_loss_reached(current_rate=current_rate, trade=trade,
current_time=date, current_profit=current_profit)
current_time=date, current_profit=current_profit,
force_stoploss=force_stoploss)
if stoplossflag.sell_flag:
return stoplossflag
# Set current rate to low for backtesting sell
@ -241,7 +251,7 @@ class IStrategy(ABC):
return SellCheckTuple(sell_flag=False, sell_type=SellType.NONE)
def stop_loss_reached(self, current_rate: float, trade: Trade, current_time: datetime,
current_profit: float) -> SellCheckTuple:
current_profit: float, force_stoploss: float) -> SellCheckTuple:
"""
Based on current profit of the trade and configured (trailing) stoploss,
decides to sell or not
@ -250,7 +260,8 @@ class IStrategy(ABC):
trailing_stop = self.config.get('trailing_stop', False)
trade.adjust_stop_loss(trade.open_rate, self.stoploss, initial=True)
trade.adjust_stop_loss(trade.open_rate, force_stoploss if force_stoploss
else self.stoploss, initial=True)
# evaluate if the stoploss was hit
if self.stoploss is not None and trade.stop_loss >= current_rate:

View File

@ -75,6 +75,19 @@ class StrategyResolver(object):
else:
config['process_only_new_candles'] = self.strategy.process_only_new_candles
if 'order_types' in config:
self.strategy.order_types = config['order_types']
logger.info(
"Override strategy 'order_types' with value in config file: %s.",
config['order_types']
)
else:
config['order_types'] = self.strategy.order_types
if not all(k in self.strategy.order_types for k in constants.REQUIRED_ORDERTYPES):
raise ImportError(f"Impossible to load Strategy '{self.strategy.__class__.__name__}'. "
f"Order-types mapping is incomplete.")
# Sort and apply type conversions
self.strategy.minimal_roi = OrderedDict(sorted(
{int(key): value for (key, value) in self.strategy.minimal_roi.items()}.items(),

View File

@ -4,6 +4,7 @@ import logging
from datetime import datetime
from functools import reduce
from typing import Dict, Optional
from collections import namedtuple
from unittest.mock import MagicMock, PropertyMock
import arrow
@ -12,6 +13,7 @@ from telegram import Chat, Message, Update
from freqtrade.exchange.exchange_helpers import parse_ticker_dataframe
from freqtrade.exchange import Exchange
from freqtrade.edge import Edge
from freqtrade.freqtradebot import FreqtradeBot
logging.getLogger('').setLevel(logging.INFO)
@ -28,6 +30,7 @@ def log_has(line, logs):
def patch_exchange(mocker, api_mock=None) -> None:
mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
mocker.patch('freqtrade.exchange.Exchange.validate_ordertypes', MagicMock())
mocker.patch('freqtrade.exchange.Exchange.name', PropertyMock(return_value="Bittrex"))
mocker.patch('freqtrade.exchange.Exchange.id', PropertyMock(return_value="bittrex"))
if api_mock:
@ -42,7 +45,32 @@ def get_patched_exchange(mocker, config, api_mock=None) -> Exchange:
return exchange
def patch_edge(mocker) -> None:
# "ETH/BTC",
# "LTC/BTC",
# "XRP/BTC",
# "NEO/BTC"
pair_info = namedtuple(
'pair_info',
'stoploss, winrate, risk_reward_ratio, required_risk_reward, expectancy')
mocker.patch('freqtrade.edge.Edge._cached_pairs', mocker.PropertyMock(
return_value={
'NEO/BTC': pair_info(-0.20, 0.66, 3.71, 0.50, 1.71),
'LTC/BTC': pair_info(-0.21, 0.66, 3.71, 0.50, 1.71),
}
))
mocker.patch('freqtrade.edge.Edge.stoploss', MagicMock(return_value=-0.20))
mocker.patch('freqtrade.edge.Edge.calculate', MagicMock(return_value=True))
def get_patched_edge(mocker, config) -> Edge:
patch_edge(mocker)
edge = Edge(config)
return edge
# Functions for recurrent object patching
def get_patched_freqtradebot(mocker, config) -> FreqtradeBot:
"""
This function patch _init_modules() to not call dependencies
@ -752,3 +780,23 @@ def buy_order_fee():
'status': 'closed',
'fee': None
}
@pytest.fixture(scope="function")
def edge_conf(default_conf):
default_conf['edge'] = {
"enabled": True,
"process_throttle_secs": 1800,
"calculate_since_number_of_days": 14,
"allowed_risk": 0.01,
"stoploss_range_min": -0.01,
"stoploss_range_max": -0.1,
"stoploss_range_step": -0.01,
"maximum_winrate": 0.80,
"minimum_expectancy": 0.20,
"min_trade_number": 15,
"max_trade_duration_minute": 1440,
"remove_pumps": False
}
return default_conf

View File

View File

@ -0,0 +1,310 @@
# pragma pylint: disable=missing-docstring, C0103, C0330
# pragma pylint: disable=protected-access, too-many-lines, invalid-name, too-many-arguments
import pytest
import logging
from freqtrade.tests.conftest import get_patched_freqtradebot
from freqtrade.edge import Edge, PairInfo
from pandas import DataFrame, to_datetime
from freqtrade.strategy.interface import SellType
from freqtrade.tests.optimize import (BTrade, BTContainer, _build_backtest_dataframe,
_get_frame_time_from_offset)
import arrow
import numpy as np
import math
from unittest.mock import MagicMock
# Cases to be tested:
# 1) Open trade should be removed from the end
# 2) Two complete trades within dataframe (with sell hit for all)
# 3) Entered, sl 1%, candle drops 8% => Trade closed, 1% loss
# 4) Entered, sl 3%, candle drops 4%, recovers to 1% => Trade closed, 3% loss
# 5) Stoploss and sell are hit. should sell on stoploss
####################################################################
ticker_start_time = arrow.get(2018, 10, 3)
ticker_interval_in_minute = 60
_ohlc = {'date': 0, 'buy': 1, 'open': 2, 'high': 3, 'low': 4, 'close': 5, 'sell': 6, 'volume': 7}
# Open trade should be removed from the end
tc0 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4975, 4987, 6172, 0, 1]], # enter trade (signal on last candle)
stop_loss=-0.99, roi=float('inf'), profit_perc=0.00,
trades=[]
)
# Two complete trades within dataframe(with sell hit for all)
tc1 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4975, 4987, 6172, 0, 1], # enter trade (signal on last candle)
[2, 5000, 5025, 4975, 4987, 6172, 0, 0], # exit at open
[3, 5000, 5025, 4975, 4987, 6172, 1, 0], # no action
[4, 5000, 5025, 4975, 4987, 6172, 0, 0], # should enter the trade
[5, 5000, 5025, 4975, 4987, 6172, 0, 1], # no action
[6, 5000, 5025, 4975, 4987, 6172, 0, 0], # should sell
],
stop_loss=-0.99, roi=float('inf'), profit_perc=0.00,
trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=2),
BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=4, close_tick=6)]
)
# 3) Entered, sl 1%, candle drops 8% => Trade closed, 1% loss
tc2 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4600, 4987, 6172, 0, 0], # enter trade, stoploss hit
[2, 5000, 5025, 4975, 4987, 6172, 0, 0],
],
stop_loss=-0.01, roi=float('inf'), profit_perc=-0.01,
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
)
# 4) Entered, sl 3 %, candle drops 4%, recovers to 1 % = > Trade closed, 3 % loss
tc3 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4800, 4987, 6172, 0, 0], # enter trade, stoploss hit
[2, 5000, 5025, 4975, 4987, 6172, 0, 0],
],
stop_loss=-0.03, roi=float('inf'), profit_perc=-0.03,
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
)
# 5) Stoploss and sell are hit. should sell on stoploss
tc4 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4800, 4987, 6172, 0, 1], # enter trade, stoploss hit, sell signal
[2, 5000, 5025, 4975, 4987, 6172, 0, 0],
],
stop_loss=-0.03, roi=float('inf'), profit_perc=-0.03,
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
)
TESTS = [
tc0,
tc1,
tc2,
tc3,
tc4
]
@pytest.mark.parametrize("data", TESTS)
def test_edge_results(edge_conf, mocker, caplog, data) -> None:
"""
run functional tests
"""
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
edge = Edge(edge_conf, freqtrade.exchange, freqtrade.strategy)
frame = _build_backtest_dataframe(data.data)
caplog.set_level(logging.DEBUG)
edge.fee = 0
trades = edge._find_trades_for_stoploss_range(frame, 'TEST/BTC', [data.stop_loss])
results = edge._fill_calculable_fields(DataFrame(trades)) if trades else DataFrame()
print(results)
assert len(trades) == len(data.trades)
if not results.empty:
assert round(results["profit_percent"].sum(), 3) == round(data.profit_perc, 3)
for c, trade in enumerate(data.trades):
res = results.iloc[c]
assert res.exit_type == trade.sell_reason
assert res.open_time == _get_frame_time_from_offset(trade.open_tick)
assert res.close_time == _get_frame_time_from_offset(trade.close_tick)
def test_adjust(mocker, default_conf):
freqtrade = get_patched_freqtradebot(mocker, default_conf)
edge = Edge(default_conf, freqtrade.exchange, freqtrade.strategy)
mocker.patch('freqtrade.edge.Edge._cached_pairs', mocker.PropertyMock(
return_value={
'E/F': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60),
'C/D': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60),
'N/O': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60)
}
))
pairs = ['A/B', 'C/D', 'E/F', 'G/H']
assert(edge.adjust(pairs) == ['E/F', 'C/D'])
def test_stoploss(mocker, default_conf):
freqtrade = get_patched_freqtradebot(mocker, default_conf)
edge = Edge(default_conf, freqtrade.exchange, freqtrade.strategy)
mocker.patch('freqtrade.edge.Edge._cached_pairs', mocker.PropertyMock(
return_value={
'E/F': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60),
'C/D': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60),
'N/O': PairInfo(-0.01, 0.66, 3.71, 0.50, 1.71, 10, 60)
}
))
assert edge.stoploss('E/F') == -0.01
def _validate_ohlc(buy_ohlc_sell_matrice):
for index, ohlc in enumerate(buy_ohlc_sell_matrice):
# if not high < open < low or not high < close < low
if not ohlc[3] >= ohlc[2] >= ohlc[4] or not ohlc[3] >= ohlc[5] >= ohlc[4]:
raise Exception('Line ' + str(index + 1) + ' of ohlc has invalid values!')
return True
def _build_dataframe(buy_ohlc_sell_matrice):
_validate_ohlc(buy_ohlc_sell_matrice)
tickers = []
for ohlc in buy_ohlc_sell_matrice:
ticker = {
'date': ticker_start_time.shift(
minutes=(
ohlc[0] *
ticker_interval_in_minute)).timestamp *
1000,
'buy': ohlc[1],
'open': ohlc[2],
'high': ohlc[3],
'low': ohlc[4],
'close': ohlc[5],
'sell': ohlc[6]}
tickers.append(ticker)
frame = DataFrame(tickers)
frame['date'] = to_datetime(frame['date'],
unit='ms',
utc=True,
infer_datetime_format=True)
return frame
def _time_on_candle(number):
return np.datetime64(ticker_start_time.shift(
minutes=(number * ticker_interval_in_minute)).timestamp * 1000, 'ms')
def test_edge_heartbeat_calculate(mocker, edge_conf):
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
edge = Edge(edge_conf, freqtrade.exchange, freqtrade.strategy)
heartbeat = edge_conf['edge']['process_throttle_secs']
# should not recalculate if heartbeat not reached
edge._last_updated = arrow.utcnow().timestamp - heartbeat + 1
assert edge.calculate() is False
def mocked_load_data(datadir, pairs=[], ticker_interval='0m', refresh_pairs=False,
timerange=None, exchange=None):
hz = 0.1
base = 0.001
ETHBTC = [
[
ticker_start_time.shift(minutes=(x * ticker_interval_in_minute)).timestamp * 1000,
math.sin(x * hz) / 1000 + base,
math.sin(x * hz) / 1000 + base + 0.0001,
math.sin(x * hz) / 1000 + base - 0.0001,
math.sin(x * hz) / 1000 + base,
123.45
] for x in range(0, 500)]
hz = 0.2
base = 0.002
LTCBTC = [
[
ticker_start_time.shift(minutes=(x * ticker_interval_in_minute)).timestamp * 1000,
math.sin(x * hz) / 1000 + base,
math.sin(x * hz) / 1000 + base + 0.0001,
math.sin(x * hz) / 1000 + base - 0.0001,
math.sin(x * hz) / 1000 + base,
123.45
] for x in range(0, 500)]
pairdata = {'NEO/BTC': ETHBTC, 'LTC/BTC': LTCBTC}
return pairdata
def test_edge_process_downloaded_data(mocker, default_conf):
default_conf['datadir'] = None
freqtrade = get_patched_freqtradebot(mocker, default_conf)
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.001))
mocker.patch('freqtrade.optimize.load_data', mocked_load_data)
edge = Edge(default_conf, freqtrade.exchange, freqtrade.strategy)
assert edge.calculate()
assert len(edge._cached_pairs) == 2
assert edge._last_updated <= arrow.utcnow().timestamp + 2
def test_process_expectancy(mocker, edge_conf):
edge_conf['edge']['min_trade_number'] = 2
freqtrade = get_patched_freqtradebot(mocker, edge_conf)
def get_fee():
return 0.001
freqtrade.exchange.get_fee = get_fee
edge = Edge(edge_conf, freqtrade.exchange, freqtrade.strategy)
trades = [
{'pair': 'TEST/BTC',
'stoploss': -0.9,
'profit_percent': '',
'profit_abs': '',
'open_time': np.datetime64('2018-10-03T00:05:00.000000000'),
'close_time': np.datetime64('2018-10-03T00:10:00.000000000'),
'open_index': 1,
'close_index': 1,
'trade_duration': '',
'open_rate': 17,
'close_rate': 17,
'exit_type': 'sell_signal'},
{'pair': 'TEST/BTC',
'stoploss': -0.9,
'profit_percent': '',
'profit_abs': '',
'open_time': np.datetime64('2018-10-03T00:20:00.000000000'),
'close_time': np.datetime64('2018-10-03T00:25:00.000000000'),
'open_index': 4,
'close_index': 4,
'trade_duration': '',
'open_rate': 20,
'close_rate': 20,
'exit_type': 'sell_signal'},
{'pair': 'TEST/BTC',
'stoploss': -0.9,
'profit_percent': '',
'profit_abs': '',
'open_time': np.datetime64('2018-10-03T00:30:00.000000000'),
'close_time': np.datetime64('2018-10-03T00:40:00.000000000'),
'open_index': 6,
'close_index': 7,
'trade_duration': '',
'open_rate': 26,
'close_rate': 34,
'exit_type': 'sell_signal'}
]
trades_df = DataFrame(trades)
trades_df = edge._fill_calculable_fields(trades_df)
final = edge._process_expectancy(trades_df)
assert len(final) == 1
assert 'TEST/BTC' in final
assert final['TEST/BTC'].stoploss == -0.9
assert round(final['TEST/BTC'].winrate, 10) == 0.3333333333
assert round(final['TEST/BTC'].risk_reward_ratio, 10) == 306.5384615384
assert round(final['TEST/BTC'].required_risk_reward, 10) == 2.0
assert round(final['TEST/BTC'].expectancy, 10) == 101.5128205128

View File

@ -355,6 +355,36 @@ def test_validate_timeframes_not_in_config(default_conf, mocker):
Exchange(default_conf)
def test_validate_order_types(default_conf, mocker):
api_mock = MagicMock()
type(api_mock).has = PropertyMock(return_value={'createMarketOrder': True})
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock))
mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
default_conf['order_types'] = {'buy': 'limit', 'sell': 'limit', 'stoploss': 'market'}
Exchange(default_conf)
type(api_mock).has = PropertyMock(return_value={'createMarketOrder': False})
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock))
default_conf['order_types'] = {'buy': 'limit', 'sell': 'limit', 'stoploss': 'market'}
with pytest.raises(OperationalException,
match=r'Exchange .* does not support market orders.'):
Exchange(default_conf)
def test_validate_order_types_not_in_config(default_conf, mocker):
api_mock = MagicMock()
mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock))
mocker.patch('freqtrade.exchange.Exchange._load_markets', MagicMock(return_value={}))
mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock())
conf = copy.deepcopy(default_conf)
Exchange(conf)
def test_exchange_has(default_conf, mocker):
exchange = get_patched_exchange(mocker, default_conf)
assert not exchange.exchange_has('ASDFASDF')
@ -373,7 +403,7 @@ def test_buy_dry_run(default_conf, mocker):
default_conf['dry_run'] = True
exchange = get_patched_exchange(mocker, default_conf)
order = exchange.buy(pair='ETH/BTC', rate=200, amount=1)
order = exchange.buy(pair='ETH/BTC', ordertype='limit', amount=1, rate=200)
assert 'id' in order
assert 'dry_run_buy_' in order['id']
@ -381,47 +411,64 @@ def test_buy_dry_run(default_conf, mocker):
def test_buy_prod(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
api_mock.create_limit_buy_order = MagicMock(return_value={
order_type = 'market'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
}
})
default_conf['dry_run'] = False
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
order = exchange.buy(pair='ETH/BTC', rate=200, amount=1)
order = exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
api_mock.create_order.reset_mock()
order_type = 'limit'
order = exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'buy'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_limit_buy_order = MagicMock(side_effect=ccxt.InsufficientFunds)
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.buy(pair='ETH/BTC', rate=200, amount=1)
exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(DependencyException):
api_mock.create_limit_buy_order = MagicMock(side_effect=ccxt.InvalidOrder)
api_mock.create_order = MagicMock(side_effect=ccxt.InvalidOrder)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.buy(pair='ETH/BTC', rate=200, amount=1)
exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_limit_buy_order = MagicMock(side_effect=ccxt.NetworkError)
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.buy(pair='ETH/BTC', rate=200, amount=1)
exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(OperationalException):
api_mock.create_limit_buy_order = MagicMock(side_effect=ccxt.BaseError)
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.buy(pair='ETH/BTC', rate=200, amount=1)
exchange.buy(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
def test_sell_dry_run(default_conf, mocker):
default_conf['dry_run'] = True
exchange = get_patched_exchange(mocker, default_conf)
order = exchange.sell(pair='ETH/BTC', rate=200, amount=1)
order = exchange.sell(pair='ETH/BTC', ordertype='limit', amount=1, rate=200)
assert 'id' in order
assert 'dry_run_sell_' in order['id']
@ -429,7 +476,8 @@ def test_sell_dry_run(default_conf, mocker):
def test_sell_prod(default_conf, mocker):
api_mock = MagicMock()
order_id = 'test_prod_sell_{}'.format(randint(0, 10 ** 6))
api_mock.create_limit_sell_order = MagicMock(return_value={
order_type = 'market'
api_mock.create_order = MagicMock(return_value={
'id': order_id,
'info': {
'foo': 'bar'
@ -438,32 +486,48 @@ def test_sell_prod(default_conf, mocker):
default_conf['dry_run'] = False
exchange = get_patched_exchange(mocker, default_conf, api_mock)
mocker.patch('freqtrade.exchange.Exchange.symbol_amount_prec', lambda s, x, y: y)
mocker.patch('freqtrade.exchange.Exchange.symbol_price_prec', lambda s, x, y: y)
order = exchange.sell(pair='ETH/BTC', rate=200, amount=1)
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert 'id' in order
assert 'info' in order
assert order['id'] == order_id
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] is None
api_mock.create_order.reset_mock()
order_type = 'limit'
order = exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
assert api_mock.create_order.call_args[0][0] == 'ETH/BTC'
assert api_mock.create_order.call_args[0][1] == order_type
assert api_mock.create_order.call_args[0][2] == 'sell'
assert api_mock.create_order.call_args[0][3] == 1
assert api_mock.create_order.call_args[0][4] == 200
# test exception handling
with pytest.raises(DependencyException):
api_mock.create_limit_sell_order = MagicMock(side_effect=ccxt.InsufficientFunds)
api_mock.create_order = MagicMock(side_effect=ccxt.InsufficientFunds)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.sell(pair='ETH/BTC', rate=200, amount=1)
exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(DependencyException):
api_mock.create_limit_sell_order = MagicMock(side_effect=ccxt.InvalidOrder)
api_mock.create_order = MagicMock(side_effect=ccxt.InvalidOrder)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.sell(pair='ETH/BTC', rate=200, amount=1)
exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(TemporaryError):
api_mock.create_limit_sell_order = MagicMock(side_effect=ccxt.NetworkError)
api_mock.create_order = MagicMock(side_effect=ccxt.NetworkError)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.sell(pair='ETH/BTC', rate=200, amount=1)
exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
with pytest.raises(OperationalException):
api_mock.create_limit_sell_order = MagicMock(side_effect=ccxt.BaseError)
api_mock.create_order = MagicMock(side_effect=ccxt.BaseError)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.sell(pair='ETH/BTC', rate=200, amount=1)
exchange.sell(pair='ETH/BTC', ordertype=order_type, amount=1, rate=200)
def test_get_balance_dry_run(default_conf, mocker):

View File

@ -31,8 +31,8 @@ class BTContainer(NamedTuple):
def _get_frame_time_from_offset(offset):
return ticker_start_time.shift(
minutes=(offset * TICKER_INTERVAL_MINUTES[tests_ticker_interval])).datetime
return ticker_start_time.shift(minutes=(offset * TICKER_INTERVAL_MINUTES[tests_ticker_interval])
).datetime.replace(tzinfo=None)
def _build_backtest_dataframe(ticker_with_signals):

View File

@ -1,4 +1,4 @@
# pragma pylint: disable=missing-docstring, W0212, line-too-long, C0103, unused-argument
# pragma pylint: disable=missing-docstring, W0212, line-too-long, C0103, C0330, unused-argument
import logging
from unittest.mock import MagicMock
@ -35,15 +35,15 @@ tc0 = BTContainer(data=[
# TC2: Stop-Loss Triggered 3% Loss
tc1 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4975, 4987, 6172, 0, 0], # enter trade (signal on last candle)
[2, 4987, 5012, 4962, 4975, 6172, 0, 0],
[3, 4975, 5000, 4800, 4962, 6172, 0, 0], # exit with stoploss hit
[4, 4962, 4987, 4937, 4950, 6172, 0, 0],
[5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
[0, 5000, 5025, 4975, 4987, 6172, 1, 0],
[1, 5000, 5025, 4975, 4987, 6172, 0, 0], # enter trade (signal on last candle)
[2, 4987, 5012, 4962, 4975, 6172, 0, 0],
[3, 4975, 5000, 4800, 4962, 6172, 0, 0], # exit with stoploss hit
[4, 4962, 4987, 4937, 4950, 6172, 0, 0],
[5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
stop_loss=-0.03, roi=1, profit_perc=-0.03,
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=3)]
)
)
# Test 3 Candle drops 4%, Recovers 1%.
@ -128,7 +128,7 @@ tc6 = BTContainer(data=[
[5, 4950, 4975, 4925, 4950, 6172, 0, 0]],
stop_loss=-0.02, roi=0.03, profit_perc=0.03,
trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=2)]
)
)
TESTS = [
tc0,

View File

@ -638,6 +638,7 @@ def test_backtest_only_sell(mocker, default_conf):
def test_backtest_alternate_buy_sell(default_conf, fee, mocker):
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
mocker.patch('freqtrade.optimize.backtesting.file_dump_json', MagicMock())
backtest_conf = _make_backtest_conf(mocker, conf=default_conf, pair='UNITTEST/BTC')
# We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True}

View File

@ -0,0 +1,131 @@
# pragma pylint: disable=missing-docstring, C0103, C0330
# pragma pylint: disable=protected-access, too-many-lines, invalid-name, too-many-arguments
from unittest.mock import MagicMock
import json
from typing import List
from freqtrade.edge import PairInfo
from freqtrade.arguments import Arguments
from freqtrade.optimize.edge_cli import (EdgeCli, setup_configuration, start)
from freqtrade.tests.conftest import log_has, patch_exchange
def get_args(args) -> List[str]:
return Arguments(args, '').get_parsed_arg()
def test_setup_configuration_without_arguments(mocker, default_conf, caplog) -> None:
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(default_conf)
))
args = [
'--config', 'config.json',
'--strategy', 'DefaultStrategy',
'edge'
]
config = setup_configuration(get_args(args))
assert 'max_open_trades' in config
assert 'stake_currency' in config
assert 'stake_amount' in config
assert 'exchange' in config
assert 'pair_whitelist' in config['exchange']
assert 'datadir' in config
assert log_has(
'Using data folder: {} ...'.format(config['datadir']),
caplog.record_tuples
)
assert 'ticker_interval' in config
assert not log_has('Parameter -i/--ticker-interval detected ...', caplog.record_tuples)
assert 'refresh_pairs' not in config
assert not log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog.record_tuples)
assert 'timerange' not in config
assert 'stoploss_range' not in config
def test_setup_configuration_with_arguments(mocker, edge_conf, caplog) -> None:
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(edge_conf)
))
args = [
'--config', 'config.json',
'--strategy', 'DefaultStrategy',
'--datadir', '/foo/bar',
'edge',
'--ticker-interval', '1m',
'--refresh-pairs-cached',
'--timerange', ':100',
'--stoplosses=-0.01,-0.10,-0.001'
]
config = setup_configuration(get_args(args))
assert 'max_open_trades' in config
assert 'stake_currency' in config
assert 'stake_amount' in config
assert 'exchange' in config
assert 'pair_whitelist' in config['exchange']
assert 'datadir' in config
assert log_has(
'Using data folder: {} ...'.format(config['datadir']),
caplog.record_tuples
)
assert 'ticker_interval' in config
assert log_has('Parameter -i/--ticker-interval detected ...', caplog.record_tuples)
assert log_has(
'Using ticker_interval: 1m ...',
caplog.record_tuples
)
assert 'refresh_pairs' in config
assert log_has('Parameter -r/--refresh-pairs-cached detected ...', caplog.record_tuples)
assert 'timerange' in config
assert log_has(
'Parameter --timerange detected: {} ...'.format(config['timerange']),
caplog.record_tuples
)
def test_start(mocker, fee, edge_conf, caplog) -> None:
start_mock = MagicMock()
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
patch_exchange(mocker)
mocker.patch('freqtrade.optimize.edge_cli.EdgeCli.start', start_mock)
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(edge_conf)
))
args = [
'--config', 'config.json',
'--strategy', 'DefaultStrategy',
'edge'
]
args = get_args(args)
start(args)
assert log_has(
'Starting freqtrade in Edge mode',
caplog.record_tuples
)
assert start_mock.call_count == 1
def test_edge_init(mocker, edge_conf) -> None:
patch_exchange(mocker)
edge_cli = EdgeCli(edge_conf)
assert edge_cli.config == edge_conf
assert callable(edge_cli.edge.calculate)
def test_generate_edge_table(edge_conf, mocker):
patch_exchange(mocker)
edge_cli = EdgeCli(edge_conf)
results = {}
results['ETH/BTC'] = PairInfo(-0.01, 0.60, 2, 1, 3, 10, 60)
assert edge_cli._generate_edge_table(results).count(':|') == 7
assert edge_cli._generate_edge_table(results).count('| ETH/BTC |') == 1
assert edge_cli._generate_edge_table(results).count(
'| risk reward ratio | required risk reward | expectancy |') == 1

View File

@ -176,7 +176,7 @@ def test_roi_table_generation(hyperopt) -> None:
'roi_p3': 3,
}
assert hyperopt.generate_roi_table(params) == {0: 6, 15: 3, 25: 1, 30: 0}
assert hyperopt.custom_hyperopt.generate_roi_table(params) == {0: 6, 15: 3, 25: 1, 30: 0}
def test_start_calls_optimizer(mocker, default_conf, caplog) -> None:
@ -244,7 +244,8 @@ def test_populate_indicators(hyperopt) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': tick}
dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
dataframe = hyperopt.custom_hyperopt.populate_indicators(dataframes['UNITTEST/BTC'],
{'pair': 'UNITTEST/BTC'})
# Check if some indicators are generated. We will not test all of them
assert 'adx' in dataframe
@ -256,9 +257,10 @@ def test_buy_strategy_generator(hyperopt) -> None:
tick = load_tickerdata_file(None, 'UNITTEST/BTC', '1m')
tickerlist = {'UNITTEST/BTC': tick}
dataframes = hyperopt.strategy.tickerdata_to_dataframe(tickerlist)
dataframe = hyperopt.populate_indicators(dataframes['UNITTEST/BTC'], {'pair': 'UNITTEST/BTC'})
dataframe = hyperopt.custom_hyperopt.populate_indicators(dataframes['UNITTEST/BTC'],
{'pair': 'UNITTEST/BTC'})
populate_buy_trend = hyperopt.buy_strategy_generator(
populate_buy_trend = hyperopt.custom_hyperopt.buy_strategy_generator(
{
'adx-value': 20,
'fastd-value': 20,

View File

@ -645,3 +645,28 @@ def test_rpcforcebuy_disabled(mocker, default_conf) -> None:
pair = 'ETH/BTC'
with pytest.raises(RPCException, match=r'Forcebuy not enabled.'):
rpc._rpc_forcebuy(pair, None)
def test_rpc_whitelist(mocker, default_conf) -> None:
patch_coinmarketcap(mocker)
patch_exchange(mocker)
mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
freqtradebot = FreqtradeBot(default_conf)
rpc = RPC(freqtradebot)
ret = rpc._rpc_whitelist()
assert ret['method'] == 'static'
assert ret['whitelist'] == default_conf['exchange']['pair_whitelist']
def test_rpc_whitelist_dynamic(mocker, default_conf) -> None:
patch_coinmarketcap(mocker)
patch_exchange(mocker)
default_conf['dynamic_whitelist'] = 4
mocker.patch('freqtrade.rpc.telegram.Telegram', MagicMock())
freqtradebot = FreqtradeBot(default_conf)
rpc = RPC(freqtradebot)
ret = rpc._rpc_whitelist()
assert ret['method'] == 4
assert ret['whitelist'] == default_conf['exchange']['pair_whitelist']

View File

@ -72,7 +72,8 @@ def test_init(default_conf, mocker, caplog) -> None:
message_str = "rpc.telegram is listening for following commands: [['status'], ['profit'], " \
"['balance'], ['start'], ['stop'], ['forcesell'], ['forcebuy'], " \
"['performance'], ['daily'], ['count'], ['reload_conf'], ['help'], ['version']]"
"['performance'], ['daily'], ['count'], ['reload_conf'], " \
"['whitelist'], ['help'], ['version']]"
assert log_has(message_str, caplog.record_tuples)
@ -1006,6 +1007,43 @@ def test_count_handle(default_conf, update, ticker, fee, markets, mocker) -> Non
assert msg in msg_mock.call_args_list[0][0][0]
def test_whitelist_static(default_conf, update, mocker) -> None:
patch_coinmarketcap(mocker)
msg_mock = MagicMock()
mocker.patch.multiple(
'freqtrade.rpc.telegram.Telegram',
_init=MagicMock(),
_send_msg=msg_mock
)
freqtradebot = get_patched_freqtradebot(mocker, default_conf)
telegram = Telegram(freqtradebot)
telegram._whitelist(bot=MagicMock(), update=update)
assert msg_mock.call_count == 1
assert ('Using static whitelist with `4` pairs \n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
in msg_mock.call_args_list[0][0][0])
def test_whitelist_dynamic(default_conf, update, mocker) -> None:
patch_coinmarketcap(mocker)
msg_mock = MagicMock()
mocker.patch.multiple(
'freqtrade.rpc.telegram.Telegram',
_init=MagicMock(),
_send_msg=msg_mock
)
default_conf['dynamic_whitelist'] = 4
freqtradebot = get_patched_freqtradebot(mocker, default_conf)
telegram = Telegram(freqtradebot)
telegram._whitelist(bot=MagicMock(), update=update)
assert msg_mock.call_count == 1
assert ('Dynamic whitelist with `4` pairs\n`ETH/BTC, LTC/BTC, XRP/BTC, NEO/BTC`'
in msg_mock.call_args_list[0][0][0])
def test_help_handle(default_conf, update, mocker) -> None:
patch_coinmarketcap(mocker)
msg_mock = MagicMock()

View File

@ -88,8 +88,8 @@ def test_load_strategy_invalid_directory(result, caplog):
def test_load_not_found_strategy():
strategy = StrategyResolver()
with pytest.raises(ImportError,
match=r'Impossible to load Strategy \'NotFoundStrategy\'.'
r' This class does not exist or contains Python code errors'):
match=r"Impossible to load Strategy 'NotFoundStrategy'."
r" This class does not exist or contains Python code errors"):
strategy._load_strategy(strategy_name='NotFoundStrategy', config={})
@ -182,6 +182,42 @@ def test_strategy_override_process_only_new_candles(caplog):
) in caplog.record_tuples
def test_strategy_override_order_types(caplog):
caplog.set_level(logging.INFO)
order_types = {
'buy': 'market',
'sell': 'limit',
'stoploss': 'limit'
}
config = {
'strategy': 'DefaultStrategy',
'order_types': order_types
}
resolver = StrategyResolver(config)
assert resolver.strategy.order_types
for method in ['buy', 'sell', 'stoploss']:
assert resolver.strategy.order_types[method] == order_types[method]
assert ('freqtrade.strategy.resolver',
logging.INFO,
"Override strategy 'order_types' with value in config file:"
" {'buy': 'market', 'sell': 'limit', 'stoploss': 'limit'}."
) in caplog.record_tuples
config = {
'strategy': 'DefaultStrategy',
'order_types': {'buy': 'market'}
}
# Raise error for invalid configuration
with pytest.raises(ImportError,
match=r"Impossible to load Strategy 'DefaultStrategy'. "
r"Order-types mapping is incomplete."):
StrategyResolver(config)
def test_deprecate_populate_indicators(result):
default_location = path.join(path.dirname(path.realpath(__file__)))
resolver = StrategyResolver({'strategy': 'TestStrategyLegacy',

View File

@ -64,6 +64,22 @@ def test_load_config_max_open_trades_zero(default_conf, mocker, caplog) -> None:
assert log_has('Validating configuration ...', caplog.record_tuples)
def test_load_config_max_open_trades_minus_one(default_conf, mocker, caplog) -> None:
default_conf['max_open_trades'] = -1
mocker.patch('freqtrade.configuration.open', mocker.mock_open(
read_data=json.dumps(default_conf)
))
args = Arguments([], '').get_parsed_arg()
configuration = Configuration(args)
validated_conf = configuration.load_config()
print(validated_conf)
assert validated_conf['max_open_trades'] > 999999999
assert validated_conf['max_open_trades'] == float('inf')
assert log_has('Validating configuration ...', caplog.record_tuples)
def test_load_config_file_exception(mocker) -> None:
mocker.patch(
'freqtrade.configuration.open',

View File

@ -18,7 +18,7 @@ from freqtrade.persistence import Trade
from freqtrade.rpc import RPCMessageType
from freqtrade.state import State
from freqtrade.strategy.interface import SellType, SellCheckTuple
from freqtrade.tests.conftest import log_has, patch_exchange
from freqtrade.tests.conftest import log_has, patch_exchange, patch_edge
# Functions for recurrent object patching
@ -177,7 +177,7 @@ def test_get_trade_stake_amount(default_conf, ticker, limit_buy_order, fee, mock
freqtrade = FreqtradeBot(default_conf)
result = freqtrade._get_trade_stake_amount()
result = freqtrade._get_trade_stake_amount('ETH/BTC')
assert result == default_conf['stake_amount']
@ -195,7 +195,7 @@ def test_get_trade_stake_amount_no_stake_amount(default_conf,
freqtrade = FreqtradeBot(default_conf)
with pytest.raises(DependencyException, match=r'.*stake amount.*'):
freqtrade._get_trade_stake_amount()
freqtrade._get_trade_stake_amount('ETH/BTC')
def test_get_trade_stake_amount_unlimited_amount(default_conf,
@ -224,28 +224,131 @@ def test_get_trade_stake_amount_unlimited_amount(default_conf,
patch_get_signal(freqtrade)
# no open trades, order amount should be 'balance / max_open_trades'
result = freqtrade._get_trade_stake_amount()
result = freqtrade._get_trade_stake_amount('ETH/BTC')
assert result == default_conf['stake_amount'] / conf['max_open_trades']
# create one trade, order amount should be 'balance / (max_open_trades - num_open_trades)'
freqtrade.create_trade()
result = freqtrade._get_trade_stake_amount()
result = freqtrade._get_trade_stake_amount('LTC/BTC')
assert result == default_conf['stake_amount'] / (conf['max_open_trades'] - 1)
# create 2 trades, order amount should be None
freqtrade.create_trade()
result = freqtrade._get_trade_stake_amount()
result = freqtrade._get_trade_stake_amount('XRP/BTC')
assert result is None
# set max_open_trades = None, so do not trade
conf['max_open_trades'] = 0
freqtrade = FreqtradeBot(conf)
result = freqtrade._get_trade_stake_amount()
result = freqtrade._get_trade_stake_amount('NEO/BTC')
assert result is None
def test_edge_called_in_process(mocker, edge_conf) -> None:
patch_RPCManager(mocker)
patch_edge(mocker)
def _refresh_whitelist(list):
return ['ETH/BTC', 'LTC/BTC', 'XRP/BTC', 'NEO/BTC']
patch_exchange(mocker)
freqtrade = FreqtradeBot(edge_conf)
freqtrade._refresh_whitelist = _refresh_whitelist
patch_get_signal(freqtrade)
freqtrade._process()
assert freqtrade.active_pair_whitelist == ['NEO/BTC', 'LTC/BTC']
def test_edge_overrides_stake_amount(mocker, edge_conf) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
patch_edge(mocker)
freqtrade = FreqtradeBot(edge_conf)
assert freqtrade._get_trade_stake_amount('NEO/BTC') == (0.001 * 0.01) / 0.20
assert freqtrade._get_trade_stake_amount('LTC/BTC') == (0.001 * 0.01) / 0.20
def test_edge_overrides_stoploss(limit_buy_order, fee, markets, caplog, mocker, edge_conf) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
patch_edge(mocker)
# Strategy stoploss is -0.1 but Edge imposes a stoploss at -0.2
# Thus, if price falls 21%, stoploss should be triggered
#
# mocking the ticker: price is falling ...
buy_price = limit_buy_order['price']
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_ticker=MagicMock(return_value={
'bid': buy_price * 0.79,
'ask': buy_price * 0.79,
'last': buy_price * 0.79
}),
buy=MagicMock(return_value={'id': limit_buy_order['id']}),
get_fee=fee,
get_markets=markets,
)
#############################################
# Create a trade with "limit_buy_order" price
freqtrade = FreqtradeBot(edge_conf)
freqtrade.active_pair_whitelist = ['NEO/BTC']
patch_get_signal(freqtrade)
freqtrade.strategy.min_roi_reached = lambda trade, current_profit, current_time: False
freqtrade.create_trade()
trade = Trade.query.first()
trade.update(limit_buy_order)
#############################################
# stoploss shoud be hit
assert freqtrade.handle_trade(trade) is True
assert log_has('executed sell, reason: SellType.STOP_LOSS', caplog.record_tuples)
assert trade.sell_reason == SellType.STOP_LOSS.value
def test_edge_should_ignore_strategy_stoploss(limit_buy_order, fee, markets,
mocker, edge_conf) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
patch_edge(mocker)
# Strategy stoploss is -0.1 but Edge imposes a stoploss at -0.2
# Thus, if price falls 15%, stoploss should not be triggered
#
# mocking the ticker: price is falling ...
buy_price = limit_buy_order['price']
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_ticker=MagicMock(return_value={
'bid': buy_price * 0.85,
'ask': buy_price * 0.85,
'last': buy_price * 0.85
}),
buy=MagicMock(return_value={'id': limit_buy_order['id']}),
get_fee=fee,
get_markets=markets,
)
#############################################
# Create a trade with "limit_buy_order" price
freqtrade = FreqtradeBot(edge_conf)
freqtrade.active_pair_whitelist = ['NEO/BTC']
patch_get_signal(freqtrade)
freqtrade.strategy.min_roi_reached = lambda trade, current_profit, current_time: False
freqtrade.create_trade()
trade = Trade.query.first()
trade.update(limit_buy_order)
#############################################
# stoploss shoud not be hit
assert freqtrade.handle_trade(trade) is False
def test_get_min_pair_stake_amount(mocker, default_conf) -> None:
patch_RPCManager(mocker)
patch_exchange(mocker)
@ -450,7 +553,7 @@ def test_create_trade_minimal_amount(default_conf, ticker, limit_buy_order,
patch_get_signal(freqtrade)
freqtrade.create_trade()
rate, amount = buy_mock.call_args[0][1], buy_mock.call_args[0][2]
rate, amount = buy_mock.call_args[1]['rate'], buy_mock.call_args[1]['amount']
assert rate * amount >= default_conf['stake_amount']
@ -494,7 +597,7 @@ def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order,
patch_get_signal(freqtrade)
assert freqtrade.create_trade() is False
assert freqtrade._get_trade_stake_amount() is None
assert freqtrade._get_trade_stake_amount('ETH/BTC') is None
def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, markets, mocker) -> None:
@ -593,7 +696,7 @@ def test_process_trade_creation(default_conf, ticker, limit_buy_order,
assert trade.amount == 90.99181073703367
assert log_has(
'Checking buy signals to create a new trade with stake_amount: 0.001000 ...',
'Buy signal found: about create a new trade with stake_amount: 0.001000 ...',
caplog.record_tuples
)
@ -760,10 +863,10 @@ def test_execute_buy(mocker, default_conf, fee, markets, limit_buy_order) -> Non
assert freqtrade.execute_buy(pair, stake_amount)
assert get_bid.call_count == 1
assert buy_mm.call_count == 1
call_args = buy_mm.call_args_list[0][0]
assert call_args[0] == pair
assert call_args[1] == bid
assert call_args[2] == stake_amount / bid
call_args = buy_mm.call_args_list[0][1]
assert call_args['pair'] == pair
assert call_args['rate'] == bid
assert call_args['amount'] == stake_amount / bid
# Test calling with price
fix_price = 0.06
@ -772,10 +875,10 @@ def test_execute_buy(mocker, default_conf, fee, markets, limit_buy_order) -> Non
assert get_bid.call_count == 1
assert buy_mm.call_count == 2
call_args = buy_mm.call_args_list[1][0]
assert call_args[0] == pair
assert call_args[1] == fix_price
assert call_args[2] == stake_amount / fix_price
call_args = buy_mm.call_args_list[1][1]
assert call_args['pair'] == pair
assert call_args['rate'] == fix_price
assert call_args['amount'] == stake_amount / fix_price
def test_process_maybe_execute_buy(mocker, default_conf) -> None:
@ -1547,7 +1650,7 @@ def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, fee, market
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
freqtrade.strategy.stop_loss_reached = \
lambda current_rate, trade, current_time, current_profit: SellCheckTuple(
lambda current_rate, trade, current_time, force_stoploss, current_profit: SellCheckTuple(
sell_flag=False, sell_type=SellType.NONE)
freqtrade.create_trade()
@ -1821,7 +1924,7 @@ def test_get_real_amount_quote(default_conf, trades_for_order, buy_order_fee, ca
exchange='binance',
open_rate=0.245441,
open_order_id="123456"
)
)
freqtrade = FreqtradeBot(default_conf)
patch_get_signal(freqtrade)
@ -2097,9 +2200,9 @@ def test_order_book_bid_strategy2(mocker, default_conf, order_book_l2, markets)
"""
patch_exchange(mocker)
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_markets=markets,
get_order_book=order_book_l2
'freqtrade.exchange.Exchange',
get_markets=markets,
get_order_book=order_book_l2
)
default_conf['exchange']['name'] = 'binance'
default_conf['bid_strategy']['use_order_book'] = True

View File

@ -0,0 +1,84 @@
# pragma pylint: disable=missing-docstring
from freqtrade.tests.conftest import get_patched_freqtradebot
from unittest.mock import MagicMock
def test_sync_wallet_at_boot(mocker, default_conf):
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value={
"BNT": {
"free": 1.0,
"used": 2.0,
"total": 3.0
},
"GAS": {
"free": 0.260739,
"used": 0.0,
"total": 0.260739
},
})
)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
assert len(freqtrade.wallets.wallets) == 2
assert freqtrade.wallets.wallets['BNT'].free == 1.0
assert freqtrade.wallets.wallets['BNT'].used == 2.0
assert freqtrade.wallets.wallets['BNT'].total == 3.0
assert freqtrade.wallets.wallets['GAS'].free == 0.260739
assert freqtrade.wallets.wallets['GAS'].used == 0.0
assert freqtrade.wallets.wallets['GAS'].total == 0.260739
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value={
"BNT": {
"free": 1.2,
"used": 1.9,
"total": 3.5
},
"GAS": {
"free": 0.270739,
"used": 0.1,
"total": 0.260439
},
})
)
freqtrade.wallets.update()
assert len(freqtrade.wallets.wallets) == 2
assert freqtrade.wallets.wallets['BNT'].free == 1.2
assert freqtrade.wallets.wallets['BNT'].used == 1.9
assert freqtrade.wallets.wallets['BNT'].total == 3.5
assert freqtrade.wallets.wallets['GAS'].free == 0.270739
assert freqtrade.wallets.wallets['GAS'].used == 0.1
assert freqtrade.wallets.wallets['GAS'].total == 0.260439
def test_sync_wallet_missing_data(mocker, default_conf):
mocker.patch.multiple(
'freqtrade.exchange.Exchange',
get_balances=MagicMock(return_value={
"BNT": {
"free": 1.0,
"used": 2.0,
"total": 3.0
},
"GAS": {
"free": 0.260739,
"total": 0.260739
},
})
)
freqtrade = get_patched_freqtradebot(mocker, default_conf)
assert len(freqtrade.wallets.wallets) == 2
assert freqtrade.wallets.wallets['BNT'].free == 1.0
assert freqtrade.wallets.wallets['BNT'].used == 2.0
assert freqtrade.wallets.wallets['BNT'].total == 3.0
assert freqtrade.wallets.wallets['GAS'].free == 0.260739
assert freqtrade.wallets.wallets['GAS'].used is None
assert freqtrade.wallets.wallets['GAS'].total == 0.260739

44
freqtrade/wallets.py Normal file
View File

@ -0,0 +1,44 @@
# pragma pylint: disable=W0603
""" Wallet """
import logging
from typing import Dict, Any, NamedTuple
from collections import namedtuple
from freqtrade.exchange import Exchange
logger = logging.getLogger(__name__)
class Wallet(NamedTuple):
exchange: str
currency: str
free: float = 0
used: float = 0
total: float = 0
class Wallets(object):
# wallet data structure
wallet = namedtuple(
'wallet',
['exchange', 'currency', 'free', 'used', 'total']
)
def __init__(self, exchange: Exchange) -> None:
self.exchange = exchange
self.wallets: Dict[str, Any] = {}
self.update()
def update(self) -> None:
balances = self.exchange.get_balances()
for currency in balances:
self.wallets[currency] = Wallet(
self.exchange.id,
currency,
balances[currency].get('free', None),
balances[currency].get('used', None),
balances[currency].get('total', None)
)
logger.info('Wallets synced ...')

View File

@ -1,18 +1,19 @@
ccxt==1.17.481
SQLAlchemy==1.2.13
ccxt==1.17.529
SQLAlchemy==1.2.14
python-telegram-bot==11.1.0
arrow==0.12.1
cachetools==3.0.0
requests==2.20.0
requests==2.20.1
urllib3==1.24.1
wrapt==1.10.11
pandas==0.23.4
scikit-learn==0.20.0
joblib==0.13.0
scipy==1.1.0
jsonschema==2.6.0
numpy==1.15.4
TA-Lib==0.4.17
pytest==3.10.0
pytest==4.0.0
pytest-mock==1.10.0
pytest-asyncio==0.9.0
pytest-cov==2.6.0
@ -24,3 +25,9 @@ scikit-optimize==0.5.2
# Required for plotting data
#plotly==3.1.1
# find first, C search in arrays
py_find_1st==1.1.3
#Load ticker files 30% faster
ujson==1.35

View File

@ -31,12 +31,15 @@ setup(name='freqtrade',
'pandas',
'scikit-learn',
'scipy',
'joblib',
'jsonschema',
'TA-Lib',
'tabulate',
'cachetools',
'coinmarketcap',
'scikit-optimize',
'ujson',
'py_find_1st'
],
include_package_data=True,
zip_safe=False,

View File

View File

@ -0,0 +1,139 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
import talib.abstract as ta
from pandas import DataFrame
from typing import Dict, Any, Callable, List
from functools import reduce
import numpy
from skopt.space import Categorical, Dimension, Integer, Real
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.optimize.hyperopt_interface import IHyperOpt
class_name = 'SampleHyperOpts'
# This class is a sample. Feel free to customize it.
class SampleHyperOpts(IHyperOpt):
"""
This is a test hyperopt to inspire you.
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md
You can:
- Rename the class name (Do not forget to update class_name)
- Add any methods you want to build your hyperopt
- Add any lib you need to build your hyperopt
You must keep:
- the prototype for the methods: populate_indicators, indicator_space, buy_strategy_generator,
roi_space, generate_roi_table, stoploss_space
"""
@staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe)
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['mfi'] = ta.MFI(dataframe)
dataframe['rsi'] = ta.RSI(dataframe)
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['sar'] = ta.SAR(dataframe)
return dataframe
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by hyperopt
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Buy strategy Hyperopt will build and use
"""
conditions = []
# GUARDS AND TRENDS
if 'mfi-enabled' in params and params['mfi-enabled']:
conditions.append(dataframe['mfi'] < params['mfi-value'])
if 'fastd-enabled' in params and params['fastd-enabled']:
conditions.append(dataframe['fastd'] < params['fastd-value'])
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
if params['trigger'] == 'sar_reversal':
conditions.append(qtpylib.crossed_above(
dataframe['close'], dataframe['sar']
))
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
return populate_buy_trend
@staticmethod
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(10, 25, name='mfi-value'),
Integer(15, 45, name='fastd-value'),
Integer(20, 50, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='mfi-enabled'),
Categorical([True, False], name='fastd-enabled'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal', 'sar_reversal'], name='trigger')
]
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss Value to search
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]

View File

@ -48,6 +48,13 @@ class TestStrategy(IStrategy):
# run "populate_indicators" only for new candle
ta_on_candle = False
# Optional order type mapping
order_types = {
'buy': 'limit',
'sell': 'limit',
'stoploss': 'market'
}
def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Adds several different TA indicators to the given DataFrame