diff --git a/README.md b/README.md index 24e01531c..929d40292 100644 --- a/README.md +++ b/README.md @@ -22,33 +22,10 @@ expect. We strongly recommend you to have coding and Python knowledge. Do not hesitate to read the source code and understand the mechanism of this bot. -## Table of Contents -- [Features](#features) -- [Quick start](#quick-start) -- [Documentations](https://github.com/freqtrade/freqtrade/blob/develop/docs/index.md) - - [Installation](https://github.com/freqtrade/freqtrade/blob/develop/docs/installation.md) - - [Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md) - - [Strategy Optimization](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md) - - [Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/backtesting.md) - - [Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md) -- [Support](#support) - - [Help](#help--slack) - - [Bugs](#bugs--issues) - - [Feature Requests](#feature-requests) - - [Pull Requests](#pull-requests) -- [Basic Usage](#basic-usage) - - [Bot commands](#bot-commands) - - [Telegram RPC commands](#telegram-rpc-commands) -- [Requirements](#requirements) - - [Min hardware required](#min-hardware-required) - - [Software requirements](#software-requirements) - -## Branches -The project is currently setup in two main branches: -- `develop` - This branch has often new features, but might also cause -breaking changes. -- `master` - This branch contains the latest stable release. The bot -'should' be stable on this branch, and is generally well tested. +## Exchange marketplaces supported +- [X] [Bittrex](https://bittrex.com/) +- [X] [Binance](https://www.binance.com/) +- [ ] [113 others to tests](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_ ## Features - [x] **Based on Python 3.6+**: For botting on any operating system - @@ -65,74 +42,50 @@ strategy parameters with real exchange data. - [x] **Daily summary of profit/loss**: Provide a daily summary of your profit/loss. - [x] **Performance status report**: Provide a performance status of your current trades. -### Exchange marketplaces supported -- [X] [Bittrex](https://bittrex.com/) -- [X] [Binance](https://www.binance.com/) -- [ ] [113 others to tests](https://github.com/ccxt/ccxt/). _(We cannot guarantee they will work)_ +## Table of Contents +- [Quick start](#quick-start) +- [Documentations](https://github.com/freqtrade/freqtrade/blob/develop/docs/index.md) + - [Installation](https://github.com/freqtrade/freqtrade/blob/develop/docs/installation.md) + - [Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md) + - [Strategy Optimization](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md) + - [Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/backtesting.md) + - [Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md) +- [Basic Usage](#basic-usage) + - [Bot commands](#bot-commands) + - [Telegram RPC commands](#telegram-rpc-commands) +- [Support](#support) + - [Help](#help--slack) + - [Bugs](#bugs--issues) + - [Feature Requests](#feature-requests) + - [Pull Requests](#pull-requests) +- [Requirements](#requirements) + - [Min hardware required](#min-hardware-required) + - [Software requirements](#software-requirements) ## Quick start -This quick start section is a very short explanation on how to test the -bot in dry-run. We invite you to read the -[bot documentation](https://github.com/freqtrade/freqtrade/blob/develop/docs/index.md) -to ensure you understand how the bot is working. - -### Easy installation -The script below will install all dependencies and help you to configure the bot. -```bash -./setup.sh --install -``` - -### Manual installation -The following steps are made for Linux/MacOS environment - -**1. Clone the repo** +Freqtrade provides a Linux/macOS script to install all dependencies and help you to configure the bot. ```bash git clone git@github.com:freqtrade/freqtrade.git git checkout develop cd freqtrade +./setup.sh --install ``` -**2. Create the config file** -Switch `"dry_run": true,` -```bash -cp config.json.example config.json -vi config.json -``` -**3. Build your docker image and run it** -```bash -docker build -t freqtrade . -docker run --rm -v /etc/localtime:/etc/localtime:ro -v `pwd`/config.json:/freqtrade/config.json -it freqtrade -``` +_Windows installation is explained in [Installation doc](https://github.com/freqtrade/freqtrade/blob/develop/docs/installation.md)_ -### Help / Slack -For any questions not covered by the documentation or for further -information about the bot, we encourage you to join our slack channel. -- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). +## Documentation +We invite you to read the bot documentation to ensure you understand how the bot is working. +- [Index](https://github.com/freqtrade/freqtrade/blob/develop/docs/index.md) +- [Installation](https://github.com/freqtrade/freqtrade/blob/develop/docs/installation.md) +- [Configuration](https://github.com/freqtrade/freqtrade/blob/develop/docs/configuration.md) +- [Bot usage](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md) + - [How to run the bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#bot-commands) + - [How to use Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#backtesting-commands) + - [How to use Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#hyperopt-commands) +- [Strategy Optimization](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md) +- [Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/backtesting.md) +- [Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md) -### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) -If you discover a bug in the bot, please -[search our issue tracker](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) -first. If it hasn't been reported, please -[create a new issue](https://github.com/freqtrade/freqtrade/issues/new) and -ensure you follow the template guide so that our team can assist you as -quickly as possible. - -### [Feature Requests](https://github.com/freqtrade/freqtrade/labels/enhancement) -Have you a great idea to improve the bot you want to share? Please, -first search if this feature was not [already discussed](https://github.com/freqtrade/freqtrade/labels/enhancement). -If it hasn't been requested, please -[create a new request](https://github.com/freqtrade/freqtrade/issues/new) -and ensure you follow the template guide so that it does not get lost -in the bug reports. - -### [Pull Requests](https://github.com/freqtrade/freqtrade/pulls) -Feel like our bot is missing a feature? We welcome your pull requests! -Please read our -[Contributing document](https://github.com/freqtrade/freqtrade/blob/develop/CONTRIBUTING.md) -to understand the requirements before sending your pull-requests. - -**Important:** Always create your PR against the `develop` branch, not -`master`. ## Basic Usage @@ -170,11 +123,7 @@ optional arguments: "tradesv3.dry_run.sqlite" instead of memory DB. Work only if dry_run is enabled. ``` -More details on: -- [How to run the bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#bot-commands) -- [How to use Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#backtesting-commands) -- [How to use Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md#hyperopt-commands) - + ### Telegram RPC commands Telegram is not mandatory. However, this is a great way to control your bot. More details on our @@ -193,6 +142,48 @@ bot. More details on our - `/help`: Show help message - `/version`: Show version + +## Development branches +The project is currently setup in two main branches: +- `develop` - This branch has often new features, but might also cause +breaking changes. +- `master` - This branch contains the latest stable release. The bot +'should' be stable on this branch, and is generally well tested. + + +## Support +### Help / Slack +For any questions not covered by the documentation or for further +information about the bot, we encourage you to join our slack channel. +- [Click here to join Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). + +### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) +If you discover a bug in the bot, please +[search our issue tracker](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) +first. If it hasn't been reported, please +[create a new issue](https://github.com/freqtrade/freqtrade/issues/new) and +ensure you follow the template guide so that our team can assist you as +quickly as possible. + +### [Feature Requests](https://github.com/freqtrade/freqtrade/labels/enhancement) +Have you a great idea to improve the bot you want to share? Please, +first search if this feature was not [already discussed](https://github.com/freqtrade/freqtrade/labels/enhancement). +If it hasn't been requested, please +[create a new request](https://github.com/freqtrade/freqtrade/issues/new) +and ensure you follow the template guide so that it does not get lost +in the bug reports. + +### [Pull Requests](https://github.com/freqtrade/freqtrade/pulls) +Feel like our bot is missing a feature? We welcome your pull requests! +Please read our +[Contributing document](https://github.com/freqtrade/freqtrade/blob/develop/CONTRIBUTING.md) +to understand the requirements before sending your pull-requests. + +**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/enQtMjQ5NTM0OTYzMzY3LWMxYzE3M2MxNDdjMGM3ZTYwNzFjMGIwZGRjNTc3ZGU3MGE3NzdmZGMwNmU3NDM5ZTNmM2Y3NjRiNzk4NmM4OGE). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it. + +**Important:** Always create your PR against the `develop` branch, not +`master`. + ## Requirements ### Min hardware required diff --git a/config.json.example b/config.json.example index d3dbeb52e..e8473e919 100644 --- a/config.json.example +++ b/config.json.example @@ -5,7 +5,11 @@ "fiat_display_currency": "USD", "ticker_interval" : "5m", "dry_run": false, - "unfilledtimeout": 600, + "trailing_stop": false, + "unfilledtimeout": { + "buy": 10, + "sell": 30 + }, "bid_strategy": { "ask_last_balance": 0.0 }, @@ -31,7 +35,8 @@ }, "experimental": { "use_sell_signal": false, - "sell_profit_only": false + "sell_profit_only": false, + "ignore_roi_if_buy_signal": false }, "telegram": { "enabled": true, diff --git a/config_full.json.example b/config_full.json.example index c17d22a15..4003b1c5c 100644 --- a/config_full.json.example +++ b/config_full.json.example @@ -5,6 +5,8 @@ "fiat_display_currency": "USD", "dry_run": false, "ticker_interval": "5m", + "trailing_stop": false, + "trailing_stop_positive": 0.005, "minimal_roi": { "40": 0.0, "30": 0.01, @@ -12,7 +14,10 @@ "0": 0.04 }, "stoploss": -0.10, - "unfilledtimeout": 600, + "unfilledtimeout": { + "buy": 10, + "sell": 30 + }, "bid_strategy": { "ask_last_balance": 0.0 }, @@ -38,7 +43,8 @@ }, "experimental": { "use_sell_signal": false, - "sell_profit_only": false + "sell_profit_only": false, + "ignore_roi_if_buy_signal": false }, "telegram": { "enabled": true, diff --git a/docs/backtesting.md b/docs/backtesting.md index 1efb46b43..172969ae2 100644 --- a/docs/backtesting.md +++ b/docs/backtesting.md @@ -70,6 +70,34 @@ Where `-s TestStrategy` refers to the class name within the strategy file `test_ python3 ./freqtrade/main.py backtesting --export trades ``` +The exported trades can be read using the following code for manual analysis, or can be used by the plotting script `plot_dataframe.py` in the scripts folder. + +``` python +import json +from pathlib import Path +import pandas as pd + +filename=Path('user_data/backtest_data/backtest-result.json') + +with filename.open() as file: + data = json.load(file) + +columns = ["pair", "profit", "opents", "closets", "index", "duration", + "open_rate", "close_rate", "open_at_end"] +df = pd.DataFrame(data, columns=columns) + +df['opents'] = pd.to_datetime(df['opents'], + unit='s', + utc=True, + infer_datetime_format=True + ) +df['closets'] = pd.to_datetime(df['closets'], + unit='s', + utc=True, + infer_datetime_format=True + ) +``` + #### Exporting trades to file specifying a custom filename ```bash diff --git a/docs/configuration.md b/docs/configuration.md index d5d53860b..dd16ef6b5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1,12 +1,15 @@ # Configure the bot + This page explains how to configure your `config.json` file. ## Table of Contents + - [Bot commands](#bot-commands) - [Backtesting commands](#backtesting-commands) - [Hyperopt commands](#hyperopt-commands) ## Setup config.json + We recommend to copy and use the `config.json.example` as a template for your bot configuration. @@ -16,13 +19,16 @@ The table below will list all configuration parameters. |----------|---------|----------|-------------| | `max_open_trades` | 3 | Yes | Number of trades open your bot will have. | `stake_currency` | BTC | Yes | Crypto-currency used for trading. -| `stake_amount` | 0.05 | Yes | Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. +| `stake_amount` | 0.05 | Yes | Amount of crypto-currency your bot will use for each trade. Per default, the bot will use (0.05 BTC x 3) = 0.15 BTC in total will be always engaged. Set it to 'unlimited' to allow the bot to use all avaliable balance. | `ticker_interval` | [1m, 5m, 30m, 1h, 1d] | No | The ticker interval to use (1min, 5 min, 30 min, 1 hour or 1 day). Default is 5 minutes | `fiat_display_currency` | USD | Yes | Fiat currency used to show your profits. More information below. | `dry_run` | true | Yes | Define if the bot must be in Dry-run or production mode. | `minimal_roi` | See below | No | Set the threshold in percent the bot will use to sell a trade. More information below. If set, this parameter will override `minimal_roi` from your strategy file. | `stoploss` | -0.10 | No | Value of the stoploss in percent used by the bot. More information below. If set, this parameter will override `stoploss` from your strategy file. -| `unfilledtimeout` | 0 | No | How long (in minutes) the bot will wait for an unfilled order to complete, after which the order will be cancelled. +| `trailing_stoploss` | false | No | Enables trailing stop-loss (based on `stoploss` in either configuration or strategy file). +| `trailing_stoploss_positve` | 0 | No | Changes stop-loss once profit has been reached. +| `unfilledtimeout.buy` | 10 | Yes | How long (in minutes) the bot will wait for an unfilled buy order to complete, after which the order will be cancelled. +| `unfilledtimeout.sell` | 10 | Yes | How long (in minutes) the bot will wait for an unfilled sell order to complete, after which the order will be cancelled. | `bid_strategy.ask_last_balance` | 0.0 | Yes | Set the bidding price. More information below. | `exchange.name` | bittrex | Yes | Name of the exchange class to use. [List below](#user-content-what-values-for-exchangename). | `exchange.key` | key | No | API key to use for the exchange. Only required when you are in production mode. @@ -31,6 +37,7 @@ The table below will list all configuration parameters. | `exchange.pair_blacklist` | [] | No | List of currency the bot must avoid. Useful when using `--dynamic-whitelist` param. | `experimental.use_sell_signal` | false | No | Use your sell strategy in addition of the `minimal_roi`. | `experimental.sell_profit_only` | false | No | waits until you have made a positive profit before taking a sell decision. +| `experimental.ignore_roi_if_buy_signal` | false | No | Does not sell if the buy-signal is still active. Takes preference over `minimal_roi` and `use_sell_signal` | `telegram.enabled` | true | Yes | Enable or not the usage of Telegram. | `telegram.token` | token | No | Your Telegram bot token. Only required if `telegram.enabled` is `true`. | `telegram.chat_id` | chat_id | No | Your personal Telegram account id. Only required if `telegram.enabled` is `true`. @@ -40,13 +47,22 @@ The table below will list all configuration parameters. | `strategy_path` | null | No | Adds an additional strategy lookup path (must be a folder). | `internals.process_throttle_secs` | 5 | Yes | Set the process throttle. Value in second. -The definition of each config parameters is in -[misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L205). +The definition of each config parameters is in [misc.py](https://github.com/freqtrade/freqtrade/blob/develop/freqtrade/misc.py#L205). + +### Understand stake_amount + +`stake_amount` is an amount of crypto-currency your bot will use for each trade. +The minimal value is 0.0005. If there is not enough crypto-currency in +the account an exception is generated. +To allow the bot to trade all the avaliable `stake_currency` in your account set `stake_amount` = `unlimited`. +In this case a trade amount is calclulated as `currency_balanse / (max_open_trades - current_open_trades)`. ### Understand minimal_roi + `minimal_roi` is a JSON object where the key is a duration in minutes and the value is the minimum ROI in percent. See the example below: + ``` "minimal_roi": { "40": 0.0, # Sell after 40 minutes if the profit is not negative @@ -61,6 +77,7 @@ value. This parameter is optional. If you use it, it will take over the `minimal_roi` value from the strategy file. ### Understand stoploss + `stoploss` is loss in percentage that should trigger a sale. For example value `-0.10` will cause immediate sell if the profit dips below -10% for a given trade. This parameter is optional. @@ -69,82 +86,100 @@ Most of the strategy files already include the optimal `stoploss` value. This parameter is optional. If you use it, it will take over the `stoploss` value from the strategy file. +### Understand trailing stoploss + +Go to the [trailing stoploss Documentation](stoploss.md) for details on trailing stoploss. + ### Understand initial_state + `initial_state` is an optional field that defines the initial application state. Possible values are `running` or `stopped`. (default=`running`) If the value is `stopped` the bot has to be started with `/start` first. ### Understand process_throttle_secs + `process_throttle_secs` is an optional field that defines in seconds how long the bot should wait before asking the strategy if we should buy or a sell an asset. After each wait period, the strategy is asked again for every opened trade wether or not we should sell, and for all the remaining pairs (either the dynamic list of pairs or the static list of pairs) if we should buy. ### Understand ask_last_balance + `ask_last_balance` sets the bidding price. Value `0.0` will use `ask` price, `1.0` will use the `last` price and values between those interpolate between ask and last price. Using `ask` price will guarantee quick success in bid, but bot will also end up paying more then would probably have been necessary. ### What values for exchange.name? + Freqtrade is based on [CCXT library](https://github.com/ccxt/ccxt) that supports 115 cryptocurrency exchange markets and trading APIs. The complete up-to-date list can be found in the [CCXT repo homepage](https://github.com/ccxt/ccxt/tree/master/python). However, the bot was tested with only Bittrex and Binance. The bot was tested with the following exchanges: + - [Bittrex](https://bittrex.com/): "bittrex" - [Binance](https://www.binance.com/): "binance" Feel free to test other exchanges and submit your PR to improve the bot. ### What values for fiat_display_currency? + `fiat_display_currency` set the base currency to use for the conversion from coin to fiat in Telegram. The valid values are: "AUD", "BRL", "CAD", "CHF", "CLP", "CNY", "CZK", "DKK", "EUR", "GBP", "HKD", "HUF", "IDR", "ILS", "INR", "JPY", "KRW", "MXN", "MYR", "NOK", "NZD", "PHP", "PKR", "PLN", "RUB", "SEK", "SGD", "THB", "TRY", "TWD", "ZAR", "USD". In addition to central bank currencies, a range of cryto currencies are supported. The valid values are: "BTC", "ETH", "XRP", "LTC", "BCH", "USDT". ## Switch to dry-run mode + We recommend starting the bot in dry-run mode to see how your bot will behave and how is the performance of your strategy. In Dry-run mode the bot does not engage your money. It only runs a live simulation without creating trades. ### To switch your bot in Dry-run mode: + 1. Edit your `config.json` file 2. Switch dry-run to true and specify db_url for a persistent db + ```json "dry_run": true, "db_url": "sqlite///tradesv3.dryrun.sqlite", ``` 3. Remove your Exchange API key (change them by fake api credentials) + ```json "exchange": { "name": "bittrex", "key": "key", "secret": "secret", ... -} +} ``` Once you will be happy with your bot performance, you can switch it to production mode. ## Switch to production mode + In production mode, the bot will engage your money. Be careful a wrong strategy can lose all your money. Be aware of what you are doing when you run it in production mode. ### To switch your bot in production mode: + 1. Edit your `config.json` file 2. Switch dry-run to false and don't forget to adapt your database URL if set + ```json "dry_run": false, ``` 3. Insert your Exchange API key (change them by fake api keys) + ```json "exchange": { "name": "bittrex", @@ -152,10 +187,10 @@ you run it in production mode. "secret": "08a9dc6db3d7b53e1acebd9275677f4b0a04f1a5", ... } + ``` -If you have not your Bittrex API key yet, -[see our tutorial](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md). +If you have not your Bittrex API key yet, [see our tutorial](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md). ## Next step -Now you have configured your config.json, the next step is to -[start your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md). + +Now you have configured your config.json, the next step is to [start your bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-usage.md). diff --git a/docs/index.md b/docs/index.md index afde2d5eb..fd6bf4378 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,5 @@ # freqtrade documentation + Welcome to freqtrade documentation. Please feel free to contribute to this documentation if you see it became outdated by sending us a Pull-request. Do not hesitate to reach us on @@ -6,6 +7,7 @@ Pull-request. Do not hesitate to reach us on if you do not find the answer to your questions. ## Table of Contents + - [Pre-requisite](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md) - [Setup your Bittrex account](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md#setup-your-bittrex-account) - [Setup your Telegram bot](https://github.com/freqtrade/freqtrade/blob/develop/docs/pre-requisite.md#setup-your-telegram-bot) diff --git a/docs/installation.md b/docs/installation.md index 87d3dc5cf..e5724a7dc 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -8,6 +8,7 @@ To understand how to set up the bot please read the [Bot Configuration](https:// * [Table of Contents](#table-of-contents) * [Easy Installation - Linux Script](#easy-installation---linux-script) +* [Manual installation](#manual-installation) * [Automatic Installation - Docker](#automatic-installation---docker) * [Custom Linux MacOS Installation](#custom-installation) - [Requirements](#requirements) @@ -55,6 +56,28 @@ Reset parameter will hard reset your branch (only if you are on `master` or `dev Config parameter is a `config.json` configurator. This script will ask you questions to setup your bot and create your `config.json`. + +## Manual installation - Linux/MacOS +The following steps are made for Linux/MacOS environment + +**1. Clone the repo** +```bash +git clone git@github.com:freqtrade/freqtrade.git +git checkout develop +cd freqtrade +``` +**2. Create the config file** +Switch `"dry_run": true,` +```bash +cp config.json.example config.json +vi config.json +``` +**3. Build your docker image and run it** +```bash +docker build -t freqtrade . +docker run --rm -v /etc/localtime:/etc/localtime:ro -v `pwd`/config.json:/freqtrade/config.json -it freqtrade +``` + ------ ## Automatic Installation - Docker diff --git a/docs/stoploss.md b/docs/stoploss.md new file mode 100644 index 000000000..db4433a02 --- /dev/null +++ b/docs/stoploss.md @@ -0,0 +1,48 @@ +# Stop Loss support + +At this stage the bot contains the following stoploss support modes: + +1. static stop loss, defined in either the strategy or configuration +2. trailing stop loss, defined in the configuration +3. trailing stop loss, custom positive loss, defined in configuration + +## Static Stop Loss + +This is very simple, basically you define a stop loss of x in your strategy file or alternative in the configuration, which +will overwrite the strategy definition. This will basically try to sell your asset, the second the loss exceeds the defined loss. + +## Trail Stop Loss + +The initial value for this stop loss, is defined in your strategy or configuration. Just as you would define your Stop Loss normally. +To enable this Feauture all you have to do is to define the configuration element: + +``` json +"trailing_stop" : True +``` + +This will now activate an algorithm, which automatically moves your stop loss up every time the price of your asset increases. + +For example, simplified math, + +* you buy an asset at a price of 100$ +* your stop loss is defined at 2% +* which means your stop loss, gets triggered once your asset dropped below 98$ +* assuming your asset now increases to 102$ +* your stop loss, will now be 2% of 102$ or 99.96$ +* now your asset drops in value to 101$, your stop loss, will still be 99.96$ + +basically what this means is that your stop loss will be adjusted to be always be 2% of the highest observed price + +### Custom positive loss + +Due to demand, it is possible to have a default stop loss, when you are in the red with your buy, but once your buy turns positive, +the system will utilize a new stop loss, which can be a different value. For example your default stop loss is 5%, but once you are in the +black, it will be changed to be only a 1% stop loss + +This can be configured in the main configuration file and requires `"trailing_stop": true` to be set to true. + +``` json + "trailing_stop_positive": 0.01, +``` + +The 0.01 would translate to a 1% stop loss, once you hit profit. diff --git a/freqtrade/__init__.py b/freqtrade/__init__.py index 7cf0fa996..ac00264f0 100644 --- a/freqtrade/__init__.py +++ b/freqtrade/__init__.py @@ -1,5 +1,5 @@ """ FreqTrade bot """ -__version__ = '0.17.0' +__version__ = '0.17.1' class DependencyException(BaseException): diff --git a/freqtrade/analyze.py b/freqtrade/analyze.py index cf2d5519b..bf6b89b17 100644 --- a/freqtrade/analyze.py +++ b/freqtrade/analyze.py @@ -98,6 +98,13 @@ class Analyze(object): """ return self.strategy.ticker_interval + def get_stoploss(self) -> float: + """ + Return stoploss to use + :return: Strategy stoploss value to use + """ + return self.strategy.stoploss + def analyze_ticker(self, ticker_history: List[Dict]) -> DataFrame: """ Parses the given ticker history and returns a populated DataFrame @@ -172,33 +179,79 @@ class Analyze(object): if the threshold is reached and updates the trade record. :return: True if trade should be sold, False otherwise """ + current_profit = trade.calc_profit_percent(rate) + if self.stop_loss_reached(current_rate=rate, trade=trade, current_time=date): + return True + + experimental = self.config.get('experimental', {}) + + if buy and experimental.get('ignore_roi_if_buy_signal', False): + logger.debug('Buy signal still active - not selling.') + return False + # Check if minimal roi has been reached and no longer in buy conditions (avoiding a fee) - if self.min_roi_reached(trade=trade, current_rate=rate, current_time=date): + if self.min_roi_reached(trade=trade, current_profit=current_profit, current_time=date): logger.debug('Required profit reached. Selling..') return True - # Experimental: Check if the trade is profitable before selling it (avoid selling at loss) - if self.config.get('experimental', {}).get('sell_profit_only', False): + if experimental.get('sell_profit_only', False): logger.debug('Checking if trade is profitable..') if trade.calc_profit(rate=rate) <= 0: return False - - if sell and not buy and self.config.get('experimental', {}).get('use_sell_signal', False): + if sell and not buy and experimental.get('use_sell_signal', False): logger.debug('Sell signal received. Selling..') return True return False - def min_roi_reached(self, trade: Trade, current_rate: float, current_time: datetime) -> bool: + def stop_loss_reached(self, current_rate: float, trade: Trade, current_time: datetime) -> bool: + """ + Based on current profit of the trade and configured (trailing) stoploss, + decides to sell or not + """ + + current_profit = trade.calc_profit_percent(current_rate) + trailing_stop = self.config.get('trailing_stop', False) + + trade.adjust_stop_loss(trade.open_rate, self.strategy.stoploss, initial=True) + + # evaluate if the stoploss was hit + if self.strategy.stoploss is not None and trade.stop_loss >= current_rate: + + if trailing_stop: + logger.debug( + f"HIT STOP: current price at {current_rate:.6f}, " + f"stop loss is {trade.stop_loss:.6f}, " + f"initial stop loss was at {trade.initial_stop_loss:.6f}, " + f"trade opened at {trade.open_rate:.6f}") + logger.debug(f"trailing stop saved {trade.stop_loss - trade.initial_stop_loss:.6f}") + + logger.debug('Stop loss hit.') + return True + + # update the stop loss afterwards, after all by definition it's supposed to be hanging + if trailing_stop: + + # check if we have a special stop loss for positive condition + # and if profit is positive + stop_loss_value = self.strategy.stoploss + if 'trailing_stop_positive' in self.config and current_profit > 0: + + # Ignore mypy error check in configuration that this is a float + stop_loss_value = self.config.get('trailing_stop_positive') # type: ignore + logger.debug(f"using positive stop loss mode: {stop_loss_value} " + f"since we have profit {current_profit}") + + trade.adjust_stop_loss(current_rate, stop_loss_value) + + return False + + def min_roi_reached(self, trade: Trade, current_profit: float, current_time: datetime) -> bool: """ Based an earlier trade and current price and ROI configuration, decides whether bot should sell :return True if bot should sell at current rate """ - current_profit = trade.calc_profit_percent(current_rate) - if self.strategy.stoploss is not None and current_profit < self.strategy.stoploss: - logger.debug('Stop loss hit.') - return True # Check if time matches and current rate is above threshold time_diff = (current_time.timestamp() - trade.open_date.timestamp()) / 60 diff --git a/freqtrade/arguments.py b/freqtrade/arguments.py index b392fb53e..8ce7c546d 100644 --- a/freqtrade/arguments.py +++ b/freqtrade/arguments.py @@ -262,17 +262,15 @@ class Arguments(object): stop: int = 0 if stype[0]: starts = rvals[index] - if stype[0] == 'date': - start = int(starts) if len(starts) == 10 \ - else arrow.get(starts, 'YYYYMMDD').timestamp + if stype[0] == 'date' and len(starts) == 8: + start = arrow.get(starts, 'YYYYMMDD').timestamp else: start = int(starts) index += 1 if stype[1]: stops = rvals[index] - if stype[1] == 'date': - stop = int(stops) if len(stops) == 10 \ - else arrow.get(stops, 'YYYYMMDD').timestamp + if stype[1] == 'date' and len(stops) == 8: + stop = arrow.get(stops, 'YYYYMMDD').timestamp else: stop = int(stops) return TimeRange(stype[0], stype[1], start, stop) @@ -336,3 +334,10 @@ class Arguments(object): nargs='+', dest='timeframes', ) + + self.parser.add_argument( + '--erase', + help='Clean all existing data for the selected exchange/pairs/timeframes', + dest='erase', + action='store_true' + ) diff --git a/freqtrade/constants.py b/freqtrade/constants.py index 5be01f977..ec7765455 100644 --- a/freqtrade/constants.py +++ b/freqtrade/constants.py @@ -11,6 +11,8 @@ RETRY_TIMEOUT = 30 # sec DEFAULT_STRATEGY = 'DefaultStrategy' DEFAULT_DB_PROD_URL = 'sqlite:///tradesv3.sqlite' DEFAULT_DB_DRYRUN_URL = 'sqlite://' +UNLIMITED_STAKE_AMOUNT = 'unlimited' + TICKER_INTERVAL_MINUTES = { '1m': 1, @@ -44,7 +46,11 @@ CONF_SCHEMA = { 'max_open_trades': {'type': 'integer', 'minimum': 0}, 'ticker_interval': {'type': 'string', 'enum': list(TICKER_INTERVAL_MINUTES.keys())}, 'stake_currency': {'type': 'string', 'enum': ['BTC', 'ETH', 'USDT', 'EUR', 'USD']}, - 'stake_amount': {'type': 'number', 'minimum': 0.0005}, + 'stake_amount': { + "type": ["number", "string"], + "minimum": 0.0005, + "pattern": UNLIMITED_STAKE_AMOUNT + }, 'fiat_display_currency': {'type': 'string', 'enum': SUPPORTED_FIAT}, 'dry_run': {'type': 'boolean'}, 'minimal_roi': { @@ -55,7 +61,15 @@ CONF_SCHEMA = { 'minProperties': 1 }, 'stoploss': {'type': 'number', 'maximum': 0, 'exclusiveMaximum': True}, - 'unfilledtimeout': {'type': 'integer', 'minimum': 0}, + 'trailing_stop': {'type': 'boolean'}, + 'trailing_stop_positive': {'type': 'number', 'minimum': 0, 'maximum': 1}, + 'unfilledtimeout': { + 'type': 'object', + 'properties': { + 'buy': {'type': 'number', 'minimum': 3}, + 'sell': {'type': 'number', 'minimum': 10} + } + }, 'bid_strategy': { 'type': 'object', 'properties': { @@ -73,7 +87,8 @@ CONF_SCHEMA = { 'type': 'object', 'properties': { 'use_sell_signal': {'type': 'boolean'}, - 'sell_profit_only': {'type': 'boolean'} + 'sell_profit_only': {'type': 'boolean'}, + "ignore_roi_if_buy_signal_true": {'type': 'boolean'} } }, 'telegram': { diff --git a/freqtrade/freqtradebot.py b/freqtrade/freqtradebot.py index 02375ca80..530535710 100644 --- a/freqtrade/freqtradebot.py +++ b/freqtrade/freqtradebot.py @@ -160,7 +160,7 @@ class FreqtradeBot(object): if 'unfilledtimeout' in self.config: # Check and handle any timed out open orders - self.check_handle_timedout(self.config['unfilledtimeout']) + self.check_handle_timedout() Trade.session.flush() except TemporaryError as error: @@ -244,14 +244,69 @@ class FreqtradeBot(object): balance = self.config['bid_strategy']['ask_last_balance'] return ticker['ask'] + balance * (ticker['last'] - ticker['ask']) + def _get_trade_stake_amount(self) -> Optional[float]: + stake_amount = self.config['stake_amount'] + avaliable_amount = self.exchange.get_balance(self.config['stake_currency']) + + if stake_amount == constants.UNLIMITED_STAKE_AMOUNT: + open_trades = len(Trade.query.filter(Trade.is_open.is_(True)).all()) + if open_trades >= self.config['max_open_trades']: + logger.warning('Can\'t open a new trade: max number of trades is reached') + return None + return avaliable_amount / (self.config['max_open_trades'] - open_trades) + + # Check if stake_amount is fulfilled + if avaliable_amount < stake_amount: + raise DependencyException( + 'Available balance(%f %s) is lower than stake amount(%f %s)' % ( + avaliable_amount, self.config['stake_currency'], + stake_amount, self.config['stake_currency']) + ) + + return stake_amount + + def _get_min_pair_stake_amount(self, pair: str, price: float) -> Optional[float]: + markets = self.exchange.get_markets() + markets = [m for m in markets if m['symbol'] == pair] + if not markets: + raise ValueError(f'Can\'t get market information for symbol {pair}') + + market = markets[0] + + if 'limits' not in market: + return None + + min_stake_amounts = [] + limits = market['limits'] + if ('cost' in limits and 'min' in limits['cost'] + and limits['cost']['min'] is not None): + min_stake_amounts.append(limits['cost']['min']) + + if ('amount' in limits and 'min' in limits['amount'] + and limits['amount']['min'] is not None): + min_stake_amounts.append(limits['amount']['min'] * price) + + if not min_stake_amounts: + return None + + amount_reserve_percent = 1 - 0.05 # reserve 5% + stoploss + if self.analyze.get_stoploss() is not None: + amount_reserve_percent += self.analyze.get_stoploss() + # it should not be more than 50% + amount_reserve_percent = max(amount_reserve_percent, 0.5) + return min(min_stake_amounts)/amount_reserve_percent + def create_trade(self) -> bool: """ Checks the implemented trading indicator(s) for a randomly picked pair, if one pair triggers the buy_signal a new trade record gets created :return: True if a trade object has been created and persisted, False otherwise """ - stake_amount = self.config['stake_amount'] interval = self.analyze.get_ticker_interval() + stake_amount = self._get_trade_stake_amount() + + if not stake_amount: + return False stake_currency = self.config['stake_currency'] fiat_currency = self.config['fiat_display_currency'] exc_name = self.exchange.name @@ -261,10 +316,6 @@ class FreqtradeBot(object): stake_amount ) whitelist = copy.deepcopy(self.config['exchange']['pair_whitelist']) - # Check if stake_amount is fulfilled - if self.exchange.get_balance(stake_currency) < stake_amount: - raise DependencyException( - f'stake amount is not fulfilled (currency={stake_currency})') # Remove currently opened and latest pairs from whitelist for trade in Trade.query.filter(Trade.is_open.is_(True)).all(): @@ -285,8 +336,18 @@ class FreqtradeBot(object): return False pair_s = pair.replace('_', '/') pair_url = self.exchange.get_pair_detail_url(pair) + # Calculate amount buy_limit = self.get_target_bid(self.exchange.get_ticker(pair)) + + min_stake_amount = self._get_min_pair_stake_amount(pair_s, buy_limit) + if min_stake_amount is not None and min_stake_amount > stake_amount: + logger.warning( + f'Can\'t open a new trade for {pair_s}: stake amount' + f' is too small ({stake_amount} < {min_stake_amount})' + ) + return False + amount = stake_amount / buy_limit order_id = self.exchange.buy(pair, buy_limit, amount)['id'] @@ -423,8 +484,8 @@ with limit `{buy_limit:.8f} ({stake_amount:.6f} \ current_rate = self.exchange.get_ticker(trade.pair)['bid'] (buy, sell) = (False, False) - - if self.config.get('experimental', {}).get('use_sell_signal'): + experimental = self.config.get('experimental', {}) + if experimental.get('use_sell_signal') or experimental.get('ignore_roi_if_buy_signal'): (buy, sell) = self.analyze.get_signal(self.exchange, trade.pair, self.analyze.get_ticker_interval()) @@ -434,13 +495,16 @@ with limit `{buy_limit:.8f} ({stake_amount:.6f} \ logger.info('Found no sell signals for whitelisted currencies. Trying again..') return False - def check_handle_timedout(self, timeoutvalue: int) -> None: + def check_handle_timedout(self) -> None: """ Check if any orders are timed out and cancel if neccessary :param timeoutvalue: Number of minutes until order is considered timed out :return: None """ - timeoutthreashold = arrow.utcnow().shift(minutes=-timeoutvalue).datetime + buy_timeout = self.config['unfilledtimeout']['buy'] + sell_timeout = self.config['unfilledtimeout']['sell'] + buy_timeoutthreashold = arrow.utcnow().shift(minutes=-buy_timeout).datetime + sell_timeoutthreashold = arrow.utcnow().shift(minutes=-sell_timeout).datetime for trade in Trade.query.filter(Trade.open_order_id.isnot(None)).all(): try: @@ -463,10 +527,12 @@ with limit `{buy_limit:.8f} ({stake_amount:.6f} \ if int(order['remaining']) == 0: continue - if order['side'] == 'buy' and ordertime < timeoutthreashold: - self.handle_timedout_limit_buy(trade, order) - elif order['side'] == 'sell' and ordertime < timeoutthreashold: - self.handle_timedout_limit_sell(trade, order) + # Check if trade is still actually open + if order['status'] == 'open': + if order['side'] == 'buy' and ordertime < buy_timeoutthreashold: + self.handle_timedout_limit_buy(trade, order) + elif order['side'] == 'sell' and ordertime < sell_timeoutthreashold: + self.handle_timedout_limit_sell(trade, order) # FIX: 20180110, why is cancel.order unconditionally here, whereas # it is conditionally called in the diff --git a/freqtrade/optimize/backtesting.py b/freqtrade/optimize/backtesting.py index ffb808a24..316fba508 100644 --- a/freqtrade/optimize/backtesting.py +++ b/freqtrade/optimize/backtesting.py @@ -14,6 +14,7 @@ from pandas import DataFrame from tabulate import tabulate import freqtrade.optimize as optimize +from freqtrade import constants, DependencyException from freqtrade.exchange import Exchange from freqtrade.analyze import Analyze from freqtrade.arguments import Arguments @@ -37,6 +38,8 @@ class BacktestResult(NamedTuple): close_index: int trade_duration: float open_at_end: bool + open_rate: float + close_rate: float class Backtesting(object): @@ -115,11 +118,10 @@ class Backtesting(object): def _store_backtest_result(self, recordfilename: Optional[str], results: DataFrame) -> None: - records = [(trade_entry.pair, trade_entry.profit_percent, - trade_entry.open_time.timestamp(), - trade_entry.close_time.timestamp(), - trade_entry.open_index - 1, trade_entry.trade_duration) - for index, trade_entry in results.iterrows()] + records = [(t.pair, t.profit_percent, t.open_time.timestamp(), + t.close_time.timestamp(), t.open_index - 1, t.trade_duration, + t.open_rate, t.close_rate, t.open_at_end) + for index, t in results.iterrows()] if records: logger.info('Dumping backtest results to %s', recordfilename) @@ -158,7 +160,9 @@ class Backtesting(object): trade_duration=(sell_row.date - buy_row.date).seconds // 60, open_index=buy_row.Index, close_index=sell_row.Index, - open_at_end=False + open_at_end=False, + open_rate=buy_row.close, + close_rate=sell_row.close ) if partial_ticker: # no sell condition found - trade stil open at end of backtest period @@ -171,7 +175,9 @@ class Backtesting(object): trade_duration=(sell_row.date - buy_row.date).seconds // 60, open_index=buy_row.Index, close_index=sell_row.Index, - open_at_end=True + open_at_end=True, + open_rate=buy_row.close, + close_rate=sell_row.close ) logger.debug('Force_selling still open trade %s with %s perc - %s', btr.pair, btr.profit_percent, btr.profit_abs) @@ -341,6 +347,10 @@ def setup_configuration(args: Namespace) -> Dict[str, Any]: config['exchange']['key'] = '' config['exchange']['secret'] = '' + if config['stake_amount'] == constants.UNLIMITED_STAKE_AMOUNT: + raise DependencyException('stake amount could not be "%s" for backtesting' % + constants.UNLIMITED_STAKE_AMOUNT) + return config diff --git a/freqtrade/persistence.py b/freqtrade/persistence.py index 7fd8fdeb9..aa33f7063 100644 --- a/freqtrade/persistence.py +++ b/freqtrade/persistence.py @@ -21,7 +21,6 @@ from freqtrade import OperationalException logger = logging.getLogger(__name__) -_CONF = {} _DECL_BASE: Any = declarative_base() @@ -33,9 +32,7 @@ def init(config: Dict) -> None: :param config: config to use :return: None """ - _CONF.update(config) - - db_url = _CONF.get('db_url', None) + db_url = config.get('db_url', None) kwargs = {} # Take care of thread ownership if in-memory db @@ -61,7 +58,7 @@ def init(config: Dict) -> None: check_migrate(engine) # Clean dry_run DB if the db is not in-memory - if _CONF.get('dry_run', False) and db_url != 'sqlite://': + if config.get('dry_run', False) and db_url != 'sqlite://': clean_dry_run_db() @@ -69,6 +66,10 @@ def has_column(columns, searchname: str) -> bool: return len(list(filter(lambda x: x["name"] == searchname, columns))) == 1 +def get_column_def(columns, column: str, default: str) -> str: + return default if not has_column(columns, column) else column + + def check_migrate(engine) -> None: """ Checks if migration is necessary and migrates if necessary @@ -76,18 +77,32 @@ def check_migrate(engine) -> None: inspector = inspect(engine) cols = inspector.get_columns('trades') + tabs = inspector.get_table_names() + table_back_name = 'trades_bak' + for i, table_back_name in enumerate(tabs): + table_back_name = f'trades_bak{i}' + logger.info(f'trying {table_back_name}') + + # Check for latest column + if not has_column(cols, 'max_rate'): + open_rate_requested = get_column_def(cols, 'open_rate_requested', 'null') + close_rate_requested = get_column_def(cols, 'close_rate_requested', 'null') + stop_loss = get_column_def(cols, 'stop_loss', '0.0') + initial_stop_loss = get_column_def(cols, 'initial_stop_loss', '0.0') + max_rate = get_column_def(cols, 'max_rate', '0.0') - if not has_column(cols, 'fee_open'): # Schema migration necessary - engine.execute("alter table trades rename to trades_bak") + engine.execute(f"alter table trades rename to {table_back_name}") # let SQLAlchemy create the schema as required _DECL_BASE.metadata.create_all(engine) # Copy data back - following the correct schema - engine.execute("""insert into trades + engine.execute(f"""insert into trades (id, exchange, pair, is_open, fee_open, fee_close, open_rate, open_rate_requested, close_rate, close_rate_requested, close_profit, - stake_amount, amount, open_date, close_date, open_order_id) + stake_amount, amount, open_date, close_date, open_order_id, + stop_loss, initial_stop_loss, max_rate + ) select id, lower(exchange), case when instr(pair, '_') != 0 then @@ -97,21 +112,18 @@ def check_migrate(engine) -> None: end pair, is_open, fee fee_open, fee fee_close, - open_rate, null open_rate_requested, close_rate, - null close_rate_requested, close_profit, - stake_amount, amount, open_date, close_date, open_order_id - from trades_bak + open_rate, {open_rate_requested} open_rate_requested, close_rate, + {close_rate_requested} close_rate_requested, close_profit, + stake_amount, amount, open_date, close_date, open_order_id, + {stop_loss} stop_loss, {initial_stop_loss} initial_stop_loss, + {max_rate} max_rate + from {table_back_name} """) # Reread columns - the above recreated the table! inspector = inspect(engine) cols = inspector.get_columns('trades') - if not has_column(cols, 'open_rate_requested'): - engine.execute("alter table trades add open_rate_requested float") - if not has_column(cols, 'close_rate_requested'): - engine.execute("alter table trades add close_rate_requested float") - def cleanup() -> None: """ @@ -154,6 +166,12 @@ class Trade(_DECL_BASE): open_date = Column(DateTime, nullable=False, default=datetime.utcnow) close_date = Column(DateTime) open_order_id = Column(String) + # absolute value of the stop loss + stop_loss = Column(Float, nullable=True, default=0.0) + # absolute value of the initial stop loss + initial_stop_loss = Column(Float, nullable=True, default=0.0) + # absolute value of the highest reached price + max_rate = Column(Float, nullable=True, default=0.0) def __repr__(self): return 'Trade(id={}, pair={}, amount={:.8f}, open_rate={:.8f}, open_since={})'.format( @@ -164,6 +182,45 @@ class Trade(_DECL_BASE): arrow.get(self.open_date).humanize() if self.is_open else 'closed' ) + def adjust_stop_loss(self, current_price: float, stoploss: float, initial: bool = False): + """this adjusts the stop loss to it's most recently observed setting""" + + if initial and not (self.stop_loss is None or self.stop_loss == 0): + # Don't modify if called with initial and nothing to do + return + + new_loss = float(current_price * (1 - abs(stoploss))) + + # keeping track of the highest observed rate for this trade + if self.max_rate is None: + self.max_rate = current_price + else: + if current_price > self.max_rate: + self.max_rate = current_price + + # no stop loss assigned yet + if not self.stop_loss: + logger.debug("assigning new stop loss") + self.stop_loss = new_loss + self.initial_stop_loss = new_loss + + # evaluate if the stop loss needs to be updated + else: + if new_loss > self.stop_loss: # stop losses only walk up, never down! + self.stop_loss = new_loss + logger.debug("adjusted stop loss") + else: + logger.debug("keeping current stop loss") + + logger.debug( + f"{self.pair} - current price {current_price:.8f}, " + f"bought at {self.open_rate:.8f} and calculated " + f"stop loss is at: {self.initial_stop_loss:.8f} initial " + f"stop at {self.stop_loss:.8f}. " + f"trailing stop loss saved us: " + f"{float(self.stop_loss) - float(self.initial_stop_loss):.8f} " + f"and max observed rate was {self.max_rate:.8f}") + def update(self, order: Dict) -> None: """ Updates this entity with amount and actual open/close rates. diff --git a/freqtrade/strategy/resolver.py b/freqtrade/strategy/resolver.py index 3c7836291..0dcd3fc6a 100644 --- a/freqtrade/strategy/resolver.py +++ b/freqtrade/strategy/resolver.py @@ -72,7 +72,7 @@ class StrategyResolver(object): """ current_path = os.path.dirname(os.path.realpath(__file__)) abs_paths = [ - os.path.join(current_path, '..', '..', 'user_data', 'strategies'), + os.path.join(os.getcwd(), 'user_data', 'strategies'), current_path, ] @@ -81,10 +81,13 @@ class StrategyResolver(object): abs_paths.insert(0, extra_dir) for path in abs_paths: - strategy = self._search_strategy(path, strategy_name) - if strategy: - logger.info('Using resolved strategy %s from \'%s\'', strategy_name, path) - return import_strategy(strategy) + try: + strategy = self._search_strategy(path, strategy_name) + if strategy: + logger.info('Using resolved strategy %s from \'%s\'', strategy_name, path) + return import_strategy(strategy) + except FileNotFoundError: + logger.warning('Path "%s" does not exist', path) raise ImportError( "Impossible to load Strategy '{}'. This class does not exist" diff --git a/freqtrade/tests/conftest.py b/freqtrade/tests/conftest.py index ce22cd193..4877fe556 100644 --- a/freqtrade/tests/conftest.py +++ b/freqtrade/tests/conftest.py @@ -100,7 +100,10 @@ def default_conf(): "0": 0.04 }, "stoploss": -0.10, - "unfilledtimeout": 600, + "unfilledtimeout": { + "buy": 10, + "sell": 30 + }, "bid_strategy": { "ask_last_balance": 0.0 }, @@ -189,7 +192,10 @@ def markets(): 'max': 1000, }, 'price': 500000, - 'cost': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, }, 'info': '', }, @@ -211,7 +217,10 @@ def markets(): 'max': 1000, }, 'price': 500000, - 'cost': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, }, 'info': '', }, @@ -233,7 +242,85 @@ def markets(): 'max': 1000, }, 'price': 500000, - 'cost': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, + }, + 'info': '', + }, + { + 'id': 'ltcbtc', + 'symbol': 'LTC/BTC', + 'base': 'LTC', + 'quote': 'BTC', + 'active': False, + 'precision': { + 'price': 8, + 'amount': 8, + 'cost': 8, + }, + 'lot': 0.00000001, + 'limits': { + 'amount': { + 'min': 0.01, + 'max': 1000, + }, + 'price': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, + }, + 'info': '', + }, + { + 'id': 'xrpbtc', + 'symbol': 'XRP/BTC', + 'base': 'XRP', + 'quote': 'BTC', + 'active': False, + 'precision': { + 'price': 8, + 'amount': 8, + 'cost': 8, + }, + 'lot': 0.00000001, + 'limits': { + 'amount': { + 'min': 0.01, + 'max': 1000, + }, + 'price': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, + }, + 'info': '', + }, + { + 'id': 'neobtc', + 'symbol': 'NEO/BTC', + 'base': 'NEO', + 'quote': 'BTC', + 'active': False, + 'precision': { + 'price': 8, + 'amount': 8, + 'cost': 8, + }, + 'lot': 0.00000001, + 'limits': { + 'amount': { + 'min': 0.01, + 'max': 1000, + }, + 'price': 500000, + 'cost': { + 'min': 1, + 'max': 500000, + }, }, 'info': '', } diff --git a/freqtrade/tests/exchange/test_exchange.py b/freqtrade/tests/exchange/test_exchange.py index 3c68e8493..6cf7f11b0 100644 --- a/freqtrade/tests/exchange/test_exchange.py +++ b/freqtrade/tests/exchange/test_exchange.py @@ -14,13 +14,29 @@ from freqtrade.exchange import Exchange, API_RETRY_COUNT from freqtrade.tests.conftest import log_has, get_patched_exchange +def ccxt_exceptionhandlers(mocker, default_conf, api_mock, fun, mock_ccxt_fun, **kwargs): + """Function to test ccxt exception handling """ + + with pytest.raises(TemporaryError): + api_mock.__dict__[mock_ccxt_fun] = MagicMock(side_effect=ccxt.NetworkError) + exchange = get_patched_exchange(mocker, default_conf, api_mock) + getattr(exchange, fun)(**kwargs) + assert api_mock.__dict__[mock_ccxt_fun].call_count == API_RETRY_COUNT + 1 + + with pytest.raises(OperationalException): + api_mock.__dict__[mock_ccxt_fun] = MagicMock(side_effect=ccxt.BaseError) + exchange = get_patched_exchange(mocker, default_conf, api_mock) + getattr(exchange, fun)(**kwargs) + assert api_mock.__dict__[mock_ccxt_fun].call_count == 1 + + def test_init(default_conf, mocker, caplog): caplog.set_level(logging.INFO) get_patched_exchange(mocker, default_conf) assert log_has('Instance is running with dry_run enabled', caplog.record_tuples) -def test_init_exception(default_conf): +def test_init_exception(default_conf, mocker): default_conf['exchange']['name'] = 'wrong_exchange_name' with pytest.raises( @@ -28,6 +44,13 @@ def test_init_exception(default_conf): match='Exchange {} is not supported'.format(default_conf['exchange']['name'])): Exchange(default_conf) + default_conf['exchange']['name'] = 'binance' + with pytest.raises( + OperationalException, + match='Exchange {} is not supported'.format(default_conf['exchange']['name'])): + mocker.patch("ccxt.binance", MagicMock(side_effect=AttributeError)) + Exchange(default_conf) + def test_validate_pairs(default_conf, mocker): api_mock = MagicMock() @@ -97,6 +120,20 @@ def test_validate_pairs_stake_exception(default_conf, mocker, caplog): Exchange(conf) +def test_exchangehas(default_conf, mocker): + exchange = get_patched_exchange(mocker, default_conf) + assert not exchange.exchange_has('ASDFASDF') + api_mock = MagicMock() + + type(api_mock).has = PropertyMock(return_value={'deadbeef': True}) + exchange = get_patched_exchange(mocker, default_conf, api_mock) + assert exchange.exchange_has("deadbeef") + + type(api_mock).has = PropertyMock(return_value={'deadbeef': False}) + exchange = get_patched_exchange(mocker, default_conf, api_mock) + assert not exchange.exchange_has("deadbeef") + + def test_buy_dry_run(default_conf, mocker): default_conf['dry_run'] = True exchange = get_patched_exchange(mocker, default_conf) @@ -216,6 +253,11 @@ def test_get_balance_prod(default_conf, mocker): exchange.get_balance(currency='BTC') + with pytest.raises(TemporaryError, match=r'.*balance due to malformed exchange response:.*'): + exchange = get_patched_exchange(mocker, default_conf, api_mock) + mocker.patch('freqtrade.exchange.Exchange.get_balances', MagicMock(return_value={})) + exchange.get_balance(currency='BTC') + def test_get_balances_dry_run(default_conf, mocker): default_conf['dry_run'] = True @@ -243,17 +285,8 @@ def test_get_balances_prod(default_conf, mocker): assert exchange.get_balances()['1ST']['total'] == 10.0 assert exchange.get_balances()['1ST']['used'] == 0.0 - with pytest.raises(TemporaryError): - api_mock.fetch_balance = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_balances() - assert api_mock.fetch_balance.call_count == API_RETRY_COUNT + 1 - - with pytest.raises(OperationalException): - api_mock.fetch_balance = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_balances() - assert api_mock.fetch_balance.call_count == 1 + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + "get_balances", "fetch_balance") def test_get_tickers(default_conf, mocker): @@ -282,15 +315,8 @@ def test_get_tickers(default_conf, mocker): assert tickers['BCH/BTC']['bid'] == 0.6 assert tickers['BCH/BTC']['ask'] == 0.5 - with pytest.raises(TemporaryError): # test retrier - api_mock.fetch_tickers = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_tickers() - - with pytest.raises(OperationalException): - api_mock.fetch_tickers = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_tickers() + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + "get_tickers", "fetch_tickers") with pytest.raises(OperationalException): api_mock.fetch_tickers = MagicMock(side_effect=ccxt.NotSupported) @@ -345,15 +371,9 @@ def test_get_ticker(default_conf, mocker): exchange.get_ticker(pair='ETH/BTC', refresh=False) assert api_mock.fetch_ticker.call_count == 0 - with pytest.raises(TemporaryError): # test retrier - api_mock.fetch_ticker = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_ticker(pair='ETH/BTC', refresh=True) - - with pytest.raises(OperationalException): - api_mock.fetch_ticker = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_ticker(pair='ETH/BTC', refresh=True) + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + "get_ticker", "fetch_ticker", + pair='ETH/BTC', refresh=True) api_mock.fetch_ticker = MagicMock(return_value={}) exchange = get_patched_exchange(mocker, default_conf, api_mock) @@ -416,17 +436,14 @@ def test_get_ticker_history(default_conf, mocker): assert ticks[0][4] == 9 assert ticks[0][5] == 10 - with pytest.raises(TemporaryError): # test retrier - api_mock.fetch_ohlcv = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - # new symbol to get around cache - exchange.get_ticker_history('ABCD/BTC', default_conf['ticker_interval']) + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + "get_ticker_history", "fetch_ohlcv", + pair='ABCD/BTC', tick_interval=default_conf['ticker_interval']) - with pytest.raises(OperationalException): - api_mock.fetch_ohlcv = MagicMock(side_effect=ccxt.BaseError) + with pytest.raises(OperationalException, match=r'Exchange .* does not support.*'): + api_mock.fetch_ohlcv = MagicMock(side_effect=ccxt.NotSupported) exchange = get_patched_exchange(mocker, default_conf, api_mock) - # new symbol to get around cache - exchange.get_ticker_history('EFGH/BTC', default_conf['ticker_interval']) + exchange.get_ticker_history(pair='ABCD/BTC', tick_interval=default_conf['ticker_interval']) def test_get_ticker_history_sort(default_conf, mocker): @@ -510,30 +527,20 @@ def test_cancel_order_dry_run(default_conf, mocker): # Ensure that if not dry_run, we should call API def test_cancel_order(default_conf, mocker): default_conf['dry_run'] = False - # mocker.patch.dict('freqtrade.exchange.._CONF', default_conf) api_mock = MagicMock() api_mock.cancel_order = MagicMock(return_value=123) exchange = get_patched_exchange(mocker, default_conf, api_mock) assert exchange.cancel_order(order_id='_', pair='TKN/BTC') == 123 - with pytest.raises(TemporaryError): - api_mock.cancel_order = MagicMock(side_effect=ccxt.NetworkError) - - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.cancel_order(order_id='_', pair='TKN/BTC') - assert api_mock.cancel_order.call_count == API_RETRY_COUNT + 1 - with pytest.raises(DependencyException): api_mock.cancel_order = MagicMock(side_effect=ccxt.InvalidOrder) exchange = get_patched_exchange(mocker, default_conf, api_mock) exchange.cancel_order(order_id='_', pair='TKN/BTC') assert api_mock.cancel_order.call_count == API_RETRY_COUNT + 1 - with pytest.raises(OperationalException): - api_mock.cancel_order = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.cancel_order(order_id='_', pair='TKN/BTC') - assert api_mock.cancel_order.call_count == 1 + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + "cancel_order", "cancel_order", + order_id='_', pair='TKN/BTC') def test_get_order(default_conf, mocker): @@ -551,23 +558,15 @@ def test_get_order(default_conf, mocker): exchange = get_patched_exchange(mocker, default_conf, api_mock) assert exchange.get_order('X', 'TKN/BTC') == 456 - with pytest.raises(TemporaryError): - api_mock.fetch_order = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_order(order_id='_', pair='TKN/BTC') - assert api_mock.fetch_order.call_count == API_RETRY_COUNT + 1 - with pytest.raises(DependencyException): api_mock.fetch_order = MagicMock(side_effect=ccxt.InvalidOrder) exchange = get_patched_exchange(mocker, default_conf, api_mock) exchange.get_order(order_id='_', pair='TKN/BTC') assert api_mock.fetch_order.call_count == API_RETRY_COUNT + 1 - with pytest.raises(OperationalException): - api_mock.fetch_order = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_order(order_id='_', pair='TKN/BTC') - assert api_mock.fetch_order.call_count == 1 + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + 'get_order', 'fetch_order', + order_id='_', pair='TKN/BTC') def test_name(default_conf, mocker): @@ -652,19 +651,12 @@ def test_get_trades_for_order(default_conf, mocker): assert len(orders) == 1 assert orders[0]['price'] == 165 - # test Exceptions - with pytest.raises(OperationalException): - api_mock = MagicMock() - api_mock.fetch_my_trades = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_trades_for_order(order_id, 'LTC/BTC', since) + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + 'get_trades_for_order', 'fetch_my_trades', + order_id=order_id, pair='LTC/BTC', since=since) - with pytest.raises(TemporaryError): - api_mock = MagicMock() - api_mock.fetch_my_trades = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_trades_for_order(order_id, 'LTC/BTC', since) - assert api_mock.fetch_my_trades.call_count == API_RETRY_COUNT + 1 + mocker.patch('freqtrade.exchange.Exchange.exchange_has', MagicMock(return_value=False)) + assert exchange.get_trades_for_order(order_id, 'LTC/BTC', since) == [] def test_get_markets(default_conf, mocker, markets): @@ -673,24 +665,13 @@ def test_get_markets(default_conf, mocker, markets): exchange = get_patched_exchange(mocker, default_conf, api_mock) ret = exchange.get_markets() assert isinstance(ret, list) - assert len(ret) == 3 + assert len(ret) == 6 assert ret[0]["id"] == "ethbtc" assert ret[0]["symbol"] == "ETH/BTC" - # test Exceptions - with pytest.raises(OperationalException): - api_mock = MagicMock() - api_mock.fetch_markets = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_markets() - - with pytest.raises(TemporaryError): - api_mock = MagicMock() - api_mock.fetch_markets = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_markets() - assert api_mock.fetch_markets.call_count == API_RETRY_COUNT + 1 + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + 'get_markets', 'fetch_markets') def test_get_fee(default_conf, mocker): @@ -705,19 +686,8 @@ def test_get_fee(default_conf, mocker): assert exchange.get_fee() == 0.025 - # test Exceptions - with pytest.raises(OperationalException): - api_mock = MagicMock() - api_mock.calculate_fee = MagicMock(side_effect=ccxt.BaseError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_fee() - - with pytest.raises(TemporaryError): - api_mock = MagicMock() - api_mock.calculate_fee = MagicMock(side_effect=ccxt.NetworkError) - exchange = get_patched_exchange(mocker, default_conf, api_mock) - exchange.get_fee() - assert api_mock.calculate_fee.call_count == API_RETRY_COUNT + 1 + ccxt_exceptionhandlers(mocker, default_conf, api_mock, + 'get_fee', 'calculate_fee') def test_get_amount_lots(default_conf, mocker): diff --git a/freqtrade/tests/optimize/test_backtesting.py b/freqtrade/tests/optimize/test_backtesting.py index 65aa00a70..49a6370bb 100644 --- a/freqtrade/tests/optimize/test_backtesting.py +++ b/freqtrade/tests/optimize/test_backtesting.py @@ -3,6 +3,7 @@ import json import math import random +import pytest from copy import deepcopy from typing import List from unittest.mock import MagicMock @@ -11,7 +12,7 @@ import numpy as np import pandas as pd from arrow import Arrow -from freqtrade import optimize +from freqtrade import optimize, constants, DependencyException from freqtrade.analyze import Analyze from freqtrade.arguments import Arguments, TimeRange from freqtrade.optimize.backtesting import Backtesting, start, setup_configuration @@ -268,6 +269,28 @@ def test_setup_configuration_with_arguments(mocker, default_conf, caplog) -> Non ) +def test_setup_configuration_unlimited_stake_amount(mocker, default_conf, caplog) -> None: + """ + Test setup_configuration() function + """ + + conf = deepcopy(default_conf) + conf['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT + + mocker.patch('freqtrade.configuration.open', mocker.mock_open( + read_data=json.dumps(conf) + )) + + args = [ + '--config', 'config.json', + '--strategy', 'DefaultStrategy', + 'backtesting' + ] + + with pytest.raises(DependencyException, match=r'.*stake amount.*'): + setup_configuration(get_args(args)) + + def test_start(mocker, fee, default_conf, caplog) -> None: """ Test start() function @@ -604,9 +627,13 @@ def test_backtest_record(default_conf, fee, mocker): Arrow(2017, 11, 14, 22, 10, 00).datetime, Arrow(2017, 11, 14, 22, 43, 00).datetime, Arrow(2017, 11, 14, 22, 58, 00).datetime], + "open_rate": [0.002543, 0.003003, 0.003089, 0.003214], + "close_rate": [0.002546, 0.003014, 0.003103, 0.003217], "open_index": [1, 119, 153, 185], "close_index": [118, 151, 184, 199], - "trade_duration": [123, 34, 31, 14]}) + "trade_duration": [123, 34, 31, 14], + "open_at_end": [False, False, False, True] + }) backtesting._store_backtest_result("backtest-result.json", results) assert len(results) == 4 # Assert file_dump_json was only called once @@ -617,12 +644,16 @@ def test_backtest_record(default_conf, fee, mocker): # ('UNITTEST/BTC', 0.00331158, '1510684320', '1510691700', 0, 117) # Below follows just a typecheck of the schema/type of trade-records oix = None - for (pair, profit, date_buy, date_sell, buy_index, dur) in records: + for (pair, profit, date_buy, date_sell, buy_index, dur, + openr, closer, open_at_end) in records: assert pair == 'UNITTEST/BTC' - isinstance(profit, float) + assert isinstance(profit, float) # FIX: buy/sell should be converted to ints - isinstance(date_buy, str) - isinstance(date_sell, str) + assert isinstance(date_buy, float) + assert isinstance(date_sell, float) + assert isinstance(openr, float) + assert isinstance(closer, float) + assert isinstance(open_at_end, bool) isinstance(buy_index, pd._libs.tslib.Timestamp) if oix: assert buy_index > oix diff --git a/freqtrade/tests/rpc/test_rpc.py b/freqtrade/tests/rpc/test_rpc.py index cc3a78a0e..11db7ffb3 100644 --- a/freqtrade/tests/rpc/test_rpc.py +++ b/freqtrade/tests/rpc/test_rpc.py @@ -25,7 +25,7 @@ def prec_satoshi(a, b) -> float: # Unit tests -def test_rpc_trade_status(default_conf, ticker, fee, mocker) -> None: +def test_rpc_trade_status(default_conf, ticker, fee, markets, mocker) -> None: """ Test rpc_trade_status() method """ @@ -36,7 +36,8 @@ def test_rpc_trade_status(default_conf, ticker, fee, mocker) -> None: 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -71,7 +72,7 @@ def test_rpc_trade_status(default_conf, ticker, fee, mocker) -> None: assert trade.find('[ETH/BTC]') >= 0 -def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None: +def test_rpc_status_table(default_conf, ticker, fee, markets, mocker) -> None: """ Test rpc_status_table() method """ @@ -82,7 +83,8 @@ def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None: 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -104,7 +106,7 @@ def test_rpc_status_table(default_conf, ticker, fee, mocker) -> None: def test_rpc_daily_profit(default_conf, update, ticker, fee, - limit_buy_order, limit_sell_order, mocker) -> None: + limit_buy_order, limit_sell_order, markets, mocker) -> None: """ Test rpc_daily_profit() method """ @@ -115,7 +117,8 @@ def test_rpc_daily_profit(default_conf, update, ticker, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -155,7 +158,7 @@ def test_rpc_daily_profit(default_conf, update, ticker, fee, def test_rpc_trade_statistics(default_conf, ticker, ticker_sell_up, fee, - limit_buy_order, limit_sell_order, mocker) -> None: + limit_buy_order, limit_sell_order, markets, mocker) -> None: """ Test rpc_trade_statistics() method """ @@ -170,7 +173,8 @@ def test_rpc_trade_statistics(default_conf, ticker, ticker_sell_up, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -230,7 +234,7 @@ def test_rpc_trade_statistics(default_conf, ticker, ticker_sell_up, fee, # Test that rpc_trade_statistics can handle trades that lacks # trade.open_rate (it is set to None) -def test_rpc_trade_statistics_closed(mocker, default_conf, ticker, fee, +def test_rpc_trade_statistics_closed(mocker, default_conf, ticker, fee, markets, ticker_sell_up, limit_buy_order, limit_sell_order): """ Test rpc_trade_statistics() method @@ -246,7 +250,8 @@ def test_rpc_trade_statistics_closed(mocker, default_conf, ticker, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -386,7 +391,7 @@ def test_rpc_stop(mocker, default_conf) -> None: assert freqtradebot.state == State.STOPPED -def test_rpc_forcesell(default_conf, ticker, fee, mocker) -> None: +def test_rpc_forcesell(default_conf, ticker, fee, mocker, markets) -> None: """ Test rpc_forcesell() method """ @@ -408,6 +413,7 @@ def test_rpc_forcesell(default_conf, ticker, fee, mocker) -> None: } ), get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -489,7 +495,7 @@ def test_rpc_forcesell(default_conf, ticker, fee, mocker) -> None: def test_performance_handle(default_conf, ticker, limit_buy_order, fee, - limit_sell_order, mocker) -> None: + limit_sell_order, markets, mocker) -> None: """ Test rpc_performance() method """ @@ -501,7 +507,8 @@ def test_performance_handle(default_conf, ticker, limit_buy_order, fee, validate_pairs=MagicMock(), get_balances=MagicMock(return_value=ticker), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -527,7 +534,7 @@ def test_performance_handle(default_conf, ticker, limit_buy_order, fee, assert prec_satoshi(res[0]['profit'], 6.2) -def test_rpc_count(mocker, default_conf, ticker, fee) -> None: +def test_rpc_count(mocker, default_conf, ticker, fee, markets) -> None: """ Test rpc_count() method """ @@ -540,6 +547,7 @@ def test_rpc_count(mocker, default_conf, ticker, fee) -> None: get_balances=MagicMock(return_value=ticker), get_ticker=ticker, get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) diff --git a/freqtrade/tests/rpc/test_rpc_telegram.py b/freqtrade/tests/rpc/test_rpc_telegram.py index 0a7d9690f..b2cca9b9a 100644 --- a/freqtrade/tests/rpc/test_rpc_telegram.py +++ b/freqtrade/tests/rpc/test_rpc_telegram.py @@ -185,7 +185,7 @@ def test_authorized_only_exception(default_conf, mocker, caplog) -> None: ) -def test_status(default_conf, update, mocker, fee, ticker) -> None: +def test_status(default_conf, update, mocker, fee, ticker, markets) -> None: """ Test _status() method """ @@ -202,6 +202,7 @@ def test_status(default_conf, update, mocker, fee, ticker) -> None: get_ticker=ticker, get_pair_detail_url=MagicMock(), get_fee=fee, + get_markets=markets ) msg_mock = MagicMock() status_table = MagicMock() @@ -230,7 +231,7 @@ def test_status(default_conf, update, mocker, fee, ticker) -> None: assert status_table.call_count == 1 -def test_status_handle(default_conf, update, ticker, fee, mocker) -> None: +def test_status_handle(default_conf, update, ticker, fee, markets, mocker) -> None: """ Test _status() method """ @@ -241,6 +242,7 @@ def test_status_handle(default_conf, update, ticker, fee, mocker) -> None: validate_pairs=MagicMock(), get_ticker=ticker, get_fee=fee, + get_markets=markets ) msg_mock = MagicMock() status_table = MagicMock() @@ -276,7 +278,7 @@ def test_status_handle(default_conf, update, ticker, fee, mocker) -> None: assert '[ETH/BTC]' in msg_mock.call_args_list[0][0][0] -def test_status_table_handle(default_conf, update, ticker, fee, mocker) -> None: +def test_status_table_handle(default_conf, update, ticker, fee, markets, mocker) -> None: """ Test _status_table() method """ @@ -288,6 +290,7 @@ def test_status_table_handle(default_conf, update, ticker, fee, mocker) -> None: get_ticker=ticker, buy=MagicMock(return_value={'id': 'mocked_order_id'}), get_fee=fee, + get_markets=markets ) msg_mock = MagicMock() mocker.patch.multiple( @@ -329,7 +332,7 @@ def test_status_table_handle(default_conf, update, ticker, fee, mocker) -> None: def test_daily_handle(default_conf, update, ticker, limit_buy_order, fee, - limit_sell_order, mocker) -> None: + limit_sell_order, markets, mocker) -> None: """ Test _daily() method """ @@ -343,7 +346,8 @@ def test_daily_handle(default_conf, update, ticker, limit_buy_order, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) msg_mock = MagicMock() mocker.patch.multiple( @@ -441,7 +445,7 @@ def test_daily_wrong_input(default_conf, update, ticker, mocker) -> None: def test_profit_handle(default_conf, update, ticker, ticker_sell_up, fee, - limit_buy_order, limit_sell_order, mocker) -> None: + limit_buy_order, limit_sell_order, markets, mocker) -> None: """ Test _profit() method """ @@ -452,7 +456,8 @@ def test_profit_handle(default_conf, update, ticker, ticker_sell_up, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) msg_mock = MagicMock() mocker.patch.multiple( @@ -705,7 +710,8 @@ def test_reload_conf_handle(default_conf, update, mocker) -> None: assert 'Reloading config' in msg_mock.call_args_list[0][0][0] -def test_forcesell_handle(default_conf, update, ticker, fee, ticker_sell_up, mocker) -> None: +def test_forcesell_handle(default_conf, update, ticker, fee, + ticker_sell_up, markets, mocker) -> None: """ Test _forcesell() method """ @@ -718,7 +724,8 @@ def test_forcesell_handle(default_conf, update, ticker, fee, ticker_sell_up, moc 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -745,7 +752,8 @@ def test_forcesell_handle(default_conf, update, ticker, fee, ticker_sell_up, moc assert '0.919 USD' in rpc_mock.call_args_list[-1][0][0] -def test_forcesell_down_handle(default_conf, update, ticker, fee, ticker_sell_down, mocker) -> None: +def test_forcesell_down_handle(default_conf, update, ticker, fee, + ticker_sell_down, markets, mocker) -> None: """ Test _forcesell() method """ @@ -758,7 +766,8 @@ def test_forcesell_down_handle(default_conf, update, ticker, fee, ticker_sell_do 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -789,7 +798,7 @@ def test_forcesell_down_handle(default_conf, update, ticker, fee, ticker_sell_do assert '-0.824 USD' in rpc_mock.call_args_list[-1][0][0] -def test_forcesell_all_handle(default_conf, update, ticker, fee, mocker) -> None: +def test_forcesell_all_handle(default_conf, update, ticker, fee, markets, mocker) -> None: """ Test _forcesell() method """ @@ -803,7 +812,8 @@ def test_forcesell_all_handle(default_conf, update, ticker, fee, mocker) -> None 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtradebot = FreqtradeBot(default_conf) @@ -867,7 +877,7 @@ def test_forcesell_handle_invalid(default_conf, update, mocker) -> None: def test_performance_handle(default_conf, update, ticker, fee, - limit_buy_order, limit_sell_order, mocker) -> None: + limit_buy_order, limit_sell_order, markets, mocker) -> None: """ Test _performance() method """ @@ -883,7 +893,8 @@ def test_performance_handle(default_conf, update, ticker, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) mocker.patch('freqtrade.freqtradebot.RPCManager', MagicMock()) freqtradebot = FreqtradeBot(default_conf) @@ -931,7 +942,7 @@ def test_performance_handle_invalid(default_conf, update, mocker) -> None: assert 'not running' in msg_mock.call_args_list[0][0][0] -def test_count_handle(default_conf, update, ticker, fee, mocker) -> None: +def test_count_handle(default_conf, update, ticker, fee, markets, mocker) -> None: """ Test _count() method """ @@ -947,7 +958,8 @@ def test_count_handle(default_conf, update, ticker, fee, mocker) -> None: 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - buy=MagicMock(return_value={'id': 'mocked_order_id'}) + buy=MagicMock(return_value={'id': 'mocked_order_id'}), + get_markets=markets ) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) freqtradebot = FreqtradeBot(default_conf) diff --git a/freqtrade/tests/strategy/test_strategy.py b/freqtrade/tests/strategy/test_strategy.py index 13eac3261..1e082c380 100644 --- a/freqtrade/tests/strategy/test_strategy.py +++ b/freqtrade/tests/strategy/test_strategy.py @@ -50,13 +50,16 @@ def test_load_strategy(result): assert 'adx' in resolver.strategy.populate_indicators(result) -def test_load_strategy_custom_directory(result): +def test_load_strategy_invalid_directory(result, caplog): resolver = StrategyResolver() extra_dir = os.path.join('some', 'path') - with pytest.raises( - FileNotFoundError, - match=r".*No such file or directory: '{}'".format(extra_dir)): - resolver._load_strategy('TestStrategy', extra_dir) + resolver._load_strategy('TestStrategy', extra_dir) + + assert ( + 'freqtrade.strategy.resolver', + logging.WARNING, + 'Path "{}" does not exist'.format(extra_dir), + ) in caplog.record_tuples assert hasattr(resolver.strategy, 'populate_indicators') assert 'adx' in resolver.strategy.populate_indicators(result) diff --git a/freqtrade/tests/test_analyze.py b/freqtrade/tests/test_analyze.py index 89dac21c6..02225acca 100644 --- a/freqtrade/tests/test_analyze.py +++ b/freqtrade/tests/test_analyze.py @@ -42,6 +42,7 @@ def test_analyze_object() -> None: assert hasattr(Analyze, 'get_signal') assert hasattr(Analyze, 'should_sell') assert hasattr(Analyze, 'min_roi_reached') + assert hasattr(Analyze, 'stop_loss_reached') def test_dataframe_correct_length(result): diff --git a/freqtrade/tests/test_configuration.py b/freqtrade/tests/test_configuration.py index 019c0c09d..a4096cc01 100644 --- a/freqtrade/tests/test_configuration.py +++ b/freqtrade/tests/test_configuration.py @@ -55,6 +55,18 @@ def test_load_config_missing_attributes(default_conf) -> None: configuration._validate_config(conf) +def test_load_config_incorrect_stake_amount(default_conf) -> None: + """ + Test the configuration validator with a missing attribute + """ + conf = deepcopy(default_conf) + conf['stake_amount'] = 'fake' + + with pytest.raises(ValidationError, match=r'.*\'fake\' does not match \'unlimited\'.*'): + configuration = Configuration(Namespace()) + configuration._validate_config(conf) + + def test_load_config_file(default_conf, mocker, caplog) -> None: """ Test Configuration._load_config_file() method diff --git a/freqtrade/tests/test_freqtradebot.py b/freqtrade/tests/test_freqtradebot.py index 9eae2fdf5..dde2dd6d9 100644 --- a/freqtrade/tests/test_freqtradebot.py +++ b/freqtrade/tests/test_freqtradebot.py @@ -14,7 +14,7 @@ import arrow import pytest import requests -from freqtrade import DependencyException, OperationalException, TemporaryError +from freqtrade import constants, DependencyException, OperationalException, TemporaryError from freqtrade.freqtradebot import FreqtradeBot from freqtrade.persistence import Trade from freqtrade.state import State @@ -216,7 +216,238 @@ def test_refresh_whitelist() -> None: pass -def test_create_trade(default_conf, ticker, limit_buy_order, fee, mocker) -> None: +def test_get_trade_stake_amount(default_conf, ticker, limit_buy_order, fee, mocker) -> None: + """ + Test get_trade_stake_amount() method + """ + + patch_RPCManager(mocker) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_balance=MagicMock(return_value=default_conf['stake_amount'] * 2) + ) + + freqtrade = FreqtradeBot(default_conf) + + result = freqtrade._get_trade_stake_amount() + assert(result == default_conf['stake_amount']) + + +def test_get_trade_stake_amount_no_stake_amount(default_conf, + ticker, + limit_buy_order, + fee, + mocker) -> None: + """ + Test get_trade_stake_amount() method + """ + patch_RPCManager(mocker) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_balance=MagicMock(return_value=default_conf['stake_amount'] * 0.5) + ) + + # test defined stake amount + freqtrade = FreqtradeBot(default_conf) + + with pytest.raises(DependencyException, match=r'.*stake amount.*'): + freqtrade._get_trade_stake_amount() + + +def test_get_trade_stake_amount_unlimited_amount(default_conf, + ticker, + limit_buy_order, + fee, + markets, + mocker) -> None: + """ + Test get_trade_stake_amount() method + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=ticker, + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_balance=MagicMock(return_value=default_conf['stake_amount']), + get_fee=fee, + get_markets=markets + ) + + conf = deepcopy(default_conf) + conf['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT + conf['max_open_trades'] = 2 + + freqtrade = FreqtradeBot(conf) + + # no open trades, order amount should be 'balance / max_open_trades' + result = freqtrade._get_trade_stake_amount() + assert result == default_conf['stake_amount'] / conf['max_open_trades'] + + # create one trade, order amount should be 'balance / (max_open_trades - num_open_trades)' + freqtrade.create_trade() + + result = freqtrade._get_trade_stake_amount() + assert result == default_conf['stake_amount'] / (conf['max_open_trades'] - 1) + + # create 2 trades, order amount should be None + freqtrade.create_trade() + + result = freqtrade._get_trade_stake_amount() + assert result is None + + # set max_open_trades = None, so do not trade + conf['max_open_trades'] = 0 + freqtrade = FreqtradeBot(conf) + result = freqtrade._get_trade_stake_amount() + assert result is None + + +def test_get_min_pair_stake_amount(mocker, default_conf) -> None: + """ + Test get_trade_stake_amount() method + """ + + patch_RPCManager(mocker) + mocker.patch('freqtrade.exchange.Exchange.validate_pairs', MagicMock()) + mocker.patch('freqtrade.freqtradebot.Analyze.get_stoploss', MagicMock(return_value=-0.05)) + freqtrade = FreqtradeBot(default_conf) + + # no pair found + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC' + }]) + ) + with pytest.raises(ValueError, match=r'.*get market information.*'): + freqtrade._get_min_pair_stake_amount('BNB/BTC', 1) + + # no 'limits' section + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC' + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result is None + + # empty 'limits' section + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': {} + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result is None + + # no cost Min + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {"min": None}, + 'amount': {} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result is None + + # no amount Min + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {}, + 'amount': {"min": None} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result is None + + # empty 'cost'/'amount' section + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {}, + 'amount': {} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result is None + + # min cost is set + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {'min': 2}, + 'amount': {} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 1) + assert result == 2 / 0.9 + + # min amount is set + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {}, + 'amount': {'min': 2} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 2) + assert result == 2 * 2 / 0.9 + + # min amount and cost are set (cost is minimal) + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {'min': 2}, + 'amount': {'min': 2} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 2) + assert result == min(2, 2 * 2) / 0.9 + + # min amount and cost are set (amount is minial) + mocker.patch( + 'freqtrade.exchange.Exchange.get_markets', + MagicMock(return_value=[{ + 'symbol': 'ETH/BTC', + 'limits': { + 'cost': {'min': 8}, + 'amount': {'min': 2} + } + }]) + ) + result = freqtrade._get_min_pair_stake_amount('ETH/BTC', 2) + assert result == min(8, 2 * 2) / 0.9 + + +def test_create_trade(default_conf, ticker, limit_buy_order, fee, markets, mocker) -> None: """ Test create_trade() method """ @@ -229,6 +460,7 @@ def test_create_trade(default_conf, ticker, limit_buy_order, fee, mocker) -> Non get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) # Save state of current whitelist @@ -252,32 +484,8 @@ def test_create_trade(default_conf, ticker, limit_buy_order, fee, mocker) -> Non assert whitelist == default_conf['exchange']['pair_whitelist'] -def test_create_trade_minimal_amount(default_conf, ticker, limit_buy_order, fee, mocker) -> None: - """ - Test create_trade() method - """ - patch_get_signal(mocker) - patch_RPCManager(mocker) - patch_coinmarketcap(mocker) - buy_mock = MagicMock(return_value={'id': limit_buy_order['id']}) - mocker.patch.multiple( - 'freqtrade.exchange.Exchange', - validate_pairs=MagicMock(), - get_ticker=ticker, - buy=buy_mock, - get_fee=fee, - ) - - conf = deepcopy(default_conf) - conf['stake_amount'] = 0.0005 - freqtrade = FreqtradeBot(conf) - - freqtrade.create_trade() - rate, amount = buy_mock.call_args[0][1], buy_mock.call_args[0][2] - assert rate * amount >= conf['stake_amount'] - - -def test_create_trade_no_stake_amount(default_conf, ticker, limit_buy_order, fee, mocker) -> None: +def test_create_trade_no_stake_amount(default_conf, ticker, limit_buy_order, + fee, markets, mocker) -> None: """ Test create_trade() method """ @@ -291,6 +499,7 @@ def test_create_trade_no_stake_amount(default_conf, ticker, limit_buy_order, fee buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_balance=MagicMock(return_value=default_conf['stake_amount'] * 0.5), get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(default_conf) @@ -298,7 +507,87 @@ def test_create_trade_no_stake_amount(default_conf, ticker, limit_buy_order, fee freqtrade.create_trade() -def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, mocker) -> None: +def test_create_trade_minimal_amount(default_conf, ticker, limit_buy_order, + fee, markets, mocker) -> None: + """ + Test create_trade() method + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + buy_mock = MagicMock(return_value={'id': limit_buy_order['id']}) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=ticker, + buy=buy_mock, + get_fee=fee, + get_markets=markets + ) + + conf = deepcopy(default_conf) + conf['stake_amount'] = 0.0005 + freqtrade = FreqtradeBot(conf) + + freqtrade.create_trade() + rate, amount = buy_mock.call_args[0][1], buy_mock.call_args[0][2] + assert rate * amount >= conf['stake_amount'] + + +def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_order, + fee, markets, mocker) -> None: + """ + Test create_trade() method + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + buy_mock = MagicMock(return_value={'id': limit_buy_order['id']}) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=ticker, + buy=buy_mock, + get_fee=fee, + get_markets=markets + ) + + conf = deepcopy(default_conf) + conf['stake_amount'] = 0.000000005 + freqtrade = FreqtradeBot(conf) + + result = freqtrade.create_trade() + assert result is False + + +def test_create_trade_limit_reached(default_conf, ticker, limit_buy_order, + fee, markets, mocker) -> None: + """ + Test create_trade() method + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=ticker, + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_balance=MagicMock(return_value=default_conf['stake_amount']), + get_fee=fee, + get_markets=markets + ) + conf = deepcopy(default_conf) + conf['max_open_trades'] = 0 + conf['stake_amount'] = constants.UNLIMITED_STAKE_AMOUNT + + freqtrade = FreqtradeBot(conf) + + assert freqtrade.create_trade() is False + assert freqtrade._get_trade_stake_amount() is None + + +def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, markets, mocker) -> None: """ Test create_trade() method """ @@ -311,6 +600,7 @@ def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, mocke get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) @@ -325,7 +615,7 @@ def test_create_trade_no_pairs(default_conf, ticker, limit_buy_order, fee, mocke def test_create_trade_no_pairs_after_blacklist(default_conf, ticker, - limit_buy_order, fee, mocker) -> None: + limit_buy_order, fee, markets, mocker) -> None: """ Test create_trade() method """ @@ -338,6 +628,7 @@ def test_create_trade_no_pairs_after_blacklist(default_conf, ticker, get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) @@ -616,7 +907,8 @@ def test_process_maybe_execute_sell_exception(mocker, default_conf, assert log_has('Unable to sell trade: ', caplog.record_tuples) -def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, fee, mocker) -> None: +def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, + fee, markets, mocker) -> None: """ Test check_handle() method """ @@ -632,7 +924,8 @@ def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, fee, mock }), buy=MagicMock(return_value={'id': limit_buy_order['id']}), sell=MagicMock(return_value={'id': limit_sell_order['id']}), - get_fee=fee + get_fee=fee, + get_markets=markets ) patch_coinmarketcap(mocker, value={'price_usd': 15000.0}) @@ -660,7 +953,8 @@ def test_handle_trade(default_conf, limit_buy_order, limit_sell_order, fee, mock assert trade.close_date is not None -def test_handle_overlpapping_signals(default_conf, ticker, limit_buy_order, fee, mocker) -> None: +def test_handle_overlpapping_signals(default_conf, ticker, limit_buy_order, + fee, markets, mocker) -> None: """ Test check_handle() method """ @@ -677,6 +971,7 @@ def test_handle_overlpapping_signals(default_conf, ticker, limit_buy_order, fee, get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(conf) @@ -718,7 +1013,8 @@ def test_handle_overlpapping_signals(default_conf, ticker, limit_buy_order, fee, assert freqtrade.handle_trade(trades[0]) is True -def test_handle_trade_roi(default_conf, ticker, limit_buy_order, fee, mocker, caplog) -> None: +def test_handle_trade_roi(default_conf, ticker, limit_buy_order, + fee, mocker, markets, caplog) -> None: """ Test check_handle() method """ @@ -735,6 +1031,7 @@ def test_handle_trade_roi(default_conf, ticker, limit_buy_order, fee, mocker, ca get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=True) @@ -755,7 +1052,7 @@ def test_handle_trade_roi(default_conf, ticker, limit_buy_order, fee, mocker, ca def test_handle_trade_experimental( - default_conf, ticker, limit_buy_order, fee, mocker, caplog) -> None: + default_conf, ticker, limit_buy_order, fee, mocker, markets, caplog) -> None: """ Test check_handle() method """ @@ -772,6 +1069,7 @@ def test_handle_trade_experimental( get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=False) @@ -789,7 +1087,8 @@ def test_handle_trade_experimental( assert log_has('Sell signal received. Selling..', caplog.record_tuples) -def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order, fee, mocker) -> None: +def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order, + fee, markets, mocker) -> None: """ Test check_handle() method """ @@ -802,6 +1101,7 @@ def test_close_trade(default_conf, ticker, limit_buy_order, limit_sell_order, fe get_ticker=ticker, buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(default_conf) @@ -852,7 +1152,7 @@ def test_check_handle_timedout_buy(default_conf, ticker, limit_buy_order_old, fe Trade.session.add(trade_buy) # check it does cancel buy orders over the time limit - freqtrade.check_handle_timedout(600) + freqtrade.check_handle_timedout() assert cancel_order_mock.call_count == 1 assert rpc_mock.call_count == 1 trades = Trade.query.filter(Trade.open_order_id.is_(trade_buy.open_order_id)).all() @@ -893,7 +1193,7 @@ def test_check_handle_timedout_sell(default_conf, ticker, limit_sell_order_old, Trade.session.add(trade_sell) # check it does cancel sell orders over the time limit - freqtrade.check_handle_timedout(600) + freqtrade.check_handle_timedout() assert cancel_order_mock.call_count == 1 assert rpc_mock.call_count == 1 assert trade_sell.is_open is True @@ -933,7 +1233,7 @@ def test_check_handle_timedout_partial(default_conf, ticker, limit_buy_order_old # check it does cancel buy orders over the time limit # note this is for a partially-complete buy order - freqtrade.check_handle_timedout(600) + freqtrade.check_handle_timedout() assert cancel_order_mock.call_count == 1 assert rpc_mock.call_count == 1 trades = Trade.query.filter(Trade.open_order_id.is_(trade_buy.open_order_id)).all() @@ -984,7 +1284,7 @@ def test_check_handle_timedout_exception(default_conf, ticker, mocker, caplog) - 'recent call last):\n.*' ) - freqtrade.check_handle_timedout(600) + freqtrade.check_handle_timedout() assert filter(regexp.match, caplog.record_tuples) @@ -1040,7 +1340,7 @@ def test_handle_timedout_limit_sell(mocker, default_conf) -> None: assert cancel_order_mock.call_count == 1 -def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, mocker) -> None: +def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, markets, mocker) -> None: """ Test execute_sell() method with a ticker going UP """ @@ -1051,7 +1351,8 @@ def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, mocker) -> N 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) mocker.patch('freqtrade.fiat_convert.CryptoToFiatConverter._find_price', return_value=15000.0) freqtrade = FreqtradeBot(default_conf) @@ -1081,7 +1382,7 @@ def test_execute_sell_up(default_conf, ticker, fee, ticker_sell_up, mocker) -> N assert '0.919 USD' in rpc_mock.call_args_list[-1][0][0] -def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, mocker) -> None: +def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, markets, mocker) -> None: """ Test execute_sell() method with a ticker going DOWN """ @@ -1093,7 +1394,8 @@ def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, mocker) 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(default_conf) @@ -1122,7 +1424,7 @@ def test_execute_sell_down(default_conf, ticker, fee, ticker_sell_down, mocker) def test_execute_sell_without_conf_sell_up(default_conf, ticker, fee, - ticker_sell_up, mocker) -> None: + ticker_sell_up, markets, mocker) -> None: """ Test execute_sell() method with a ticker going DOWN and with a bot config empty """ @@ -1133,7 +1435,8 @@ def test_execute_sell_without_conf_sell_up(default_conf, ticker, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(default_conf) @@ -1163,7 +1466,7 @@ def test_execute_sell_without_conf_sell_up(default_conf, ticker, fee, def test_execute_sell_without_conf_sell_down(default_conf, ticker, fee, - ticker_sell_down, mocker) -> None: + ticker_sell_down, markets, mocker) -> None: """ Test execute_sell() method with a ticker going DOWN and with a bot config empty """ @@ -1174,7 +1477,8 @@ def test_execute_sell_without_conf_sell_down(default_conf, ticker, fee, 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=ticker, - get_fee=fee + get_fee=fee, + get_markets=markets ) freqtrade = FreqtradeBot(default_conf) @@ -1201,7 +1505,8 @@ def test_execute_sell_without_conf_sell_down(default_conf, ticker, fee, assert 'loss: -5.48%, -0.00005492' in rpc_mock.call_args_list[-1][0][0] -def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, fee, mocker) -> None: +def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, + fee, markets, mocker) -> None: """ Test sell_profit_only feature when enabled """ @@ -1219,6 +1524,7 @@ def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, fee, mock }), buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) conf['experimental'] = { @@ -1234,7 +1540,8 @@ def test_sell_profit_only_enable_profit(default_conf, limit_buy_order, fee, mock assert freqtrade.handle_trade(trade) is True -def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, fee, mocker) -> None: +def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, + fee, markets, mocker) -> None: """ Test sell_profit_only feature when disabled """ @@ -1252,6 +1559,7 @@ def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, fee, moc }), buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) conf['experimental'] = { @@ -1267,14 +1575,14 @@ def test_sell_profit_only_disable_profit(default_conf, limit_buy_order, fee, moc assert freqtrade.handle_trade(trade) is True -def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, fee, mocker) -> None: +def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, fee, markets, mocker) -> None: """ Test sell_profit_only feature when enabled and we have a loss """ patch_get_signal(mocker) patch_RPCManager(mocker) patch_coinmarketcap(mocker) - mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=False) + mocker.patch('freqtrade.freqtradebot.Analyze.stop_loss_reached', return_value=False) mocker.patch.multiple( 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), @@ -1285,6 +1593,7 @@ def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, fee, mocker }), buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) conf['experimental'] = { @@ -1300,7 +1609,7 @@ def test_sell_profit_only_enable_loss(default_conf, limit_buy_order, fee, mocker assert freqtrade.handle_trade(trade) is False -def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, fee, mocker) -> None: +def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, fee, markets, mocker) -> None: """ Test sell_profit_only feature when enabled and we have a loss """ @@ -1312,12 +1621,13 @@ def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, fee, mocke 'freqtrade.exchange.Exchange', validate_pairs=MagicMock(), get_ticker=MagicMock(return_value={ - 'bid': 0.00000172, - 'ask': 0.00000173, - 'last': 0.00000172 + 'bid': 0.0000172, + 'ask': 0.0000173, + 'last': 0.0000172 }), buy=MagicMock(return_value={'id': limit_buy_order['id']}), get_fee=fee, + get_markets=markets ) conf = deepcopy(default_conf) @@ -1335,6 +1645,183 @@ def test_sell_profit_only_disable_loss(default_conf, limit_buy_order, fee, mocke assert freqtrade.handle_trade(trade) is True +def test_ignore_roi_if_buy_signal(default_conf, limit_buy_order, fee, markets, mocker) -> None: + """ + Test sell_profit_only feature when enabled and we have a loss + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=True) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=MagicMock(return_value={ + 'bid': 0.0000172, + 'ask': 0.0000173, + 'last': 0.0000172 + }), + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_fee=fee, + get_markets=markets + ) + + conf = deepcopy(default_conf) + conf['experimental'] = { + 'ignore_roi_if_buy_signal': True + } + + freqtrade = FreqtradeBot(conf) + freqtrade.create_trade() + + trade = Trade.query.first() + trade.update(limit_buy_order) + patch_get_signal(mocker, value=(True, True)) + assert freqtrade.handle_trade(trade) is False + + # Test if buy-signal is absent (should sell due to roi = true) + patch_get_signal(mocker, value=(False, True)) + assert freqtrade.handle_trade(trade) is True + + +def test_trailing_stop_loss(default_conf, limit_buy_order, fee, caplog, mocker) -> None: + """ + Test sell_profit_only feature when enabled and we have a loss + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=False) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=MagicMock(return_value={ + 'bid': 0.00000102, + 'ask': 0.00000103, + 'last': 0.00000102 + }), + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_fee=fee, + ) + + conf = deepcopy(default_conf) + conf['trailing_stop'] = True + print(limit_buy_order) + freqtrade = FreqtradeBot(conf) + freqtrade.create_trade() + + trade = Trade.query.first() + trade.update(limit_buy_order) + caplog.set_level(logging.DEBUG) + # Sell as trailing-stop is reached + assert freqtrade.handle_trade(trade) is True + assert log_has( + f'HIT STOP: current price at 0.000001, stop loss is {trade.stop_loss:.6f}, ' + f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog.record_tuples) + + +def test_trailing_stop_loss_positive(default_conf, limit_buy_order, fee, caplog, mocker) -> None: + """ + Test sell_profit_only feature when enabled and we have a loss + """ + buy_price = limit_buy_order['price'] + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=False) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=MagicMock(return_value={ + 'bid': buy_price - 0.000001, + 'ask': buy_price - 0.000001, + 'last': buy_price - 0.000001 + }), + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_fee=fee, + ) + + conf = deepcopy(default_conf) + conf['trailing_stop'] = True + conf['trailing_stop_positive'] = 0.01 + freqtrade = FreqtradeBot(conf) + freqtrade.create_trade() + + trade = Trade.query.first() + trade.update(limit_buy_order) + caplog.set_level(logging.DEBUG) + # stop-loss not reached + assert freqtrade.handle_trade(trade) is False + + # Raise ticker above buy price + mocker.patch('freqtrade.exchange.Exchange.get_ticker', + MagicMock(return_value={ + 'bid': buy_price + 0.000003, + 'ask': buy_price + 0.000003, + 'last': buy_price + 0.000003 + })) + # stop-loss not reached, adjusted stoploss + assert freqtrade.handle_trade(trade) is False + assert log_has(f'using positive stop loss mode: 0.01 since we have profit 0.26662643', + caplog.record_tuples) + assert log_has(f'adjusted stop loss', caplog.record_tuples) + assert trade.stop_loss == 0.0000138501 + + mocker.patch('freqtrade.exchange.Exchange.get_ticker', + MagicMock(return_value={ + 'bid': buy_price + 0.000002, + 'ask': buy_price + 0.000002, + 'last': buy_price + 0.000002 + })) + # Lower price again (but still positive) + assert freqtrade.handle_trade(trade) is True + assert log_has( + f'HIT STOP: current price at {buy_price + 0.000002:.6f}, ' + f'stop loss is {trade.stop_loss:.6f}, ' + f'initial stop loss was at 0.000010, trade opened at 0.000011', caplog.record_tuples) + + +def test_disable_ignore_roi_if_buy_signal(default_conf, limit_buy_order, + fee, markets, mocker) -> None: + """ + Test sell_profit_only feature when enabled and we have a loss + """ + patch_get_signal(mocker) + patch_RPCManager(mocker) + patch_coinmarketcap(mocker) + mocker.patch('freqtrade.freqtradebot.Analyze.min_roi_reached', return_value=True) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + validate_pairs=MagicMock(), + get_ticker=MagicMock(return_value={ + 'bid': 0.00000172, + 'ask': 0.00000173, + 'last': 0.00000172 + }), + buy=MagicMock(return_value={'id': limit_buy_order['id']}), + get_fee=fee, + get_markets=markets + ) + + conf = deepcopy(default_conf) + conf['experimental'] = { + 'ignore_roi_if_buy_signal': False + } + + freqtrade = FreqtradeBot(conf) + freqtrade.create_trade() + + trade = Trade.query.first() + trade.update(limit_buy_order) + # Sell due to min_roi_reached + patch_get_signal(mocker, value=(True, True)) + assert freqtrade.handle_trade(trade) is True + + # Test if buy-signal is absent + patch_get_signal(mocker, value=(False, True)) + assert freqtrade.handle_trade(trade) is True + + def test_get_real_amount_quote(default_conf, trades_for_order, buy_order_fee, caplog, mocker): """ Test get_real_amount - fee in quote currency diff --git a/freqtrade/tests/test_persistence.py b/freqtrade/tests/test_persistence.py index c50ad7d2c..12ae6579f 100644 --- a/freqtrade/tests/test_persistence.py +++ b/freqtrade/tests/test_persistence.py @@ -7,6 +7,7 @@ from sqlalchemy import create_engine from freqtrade import constants, OperationalException from freqtrade.persistence import Trade, init, clean_dry_run_db +from freqtrade.tests.conftest import log_has @pytest.fixture(scope='function') @@ -14,9 +15,7 @@ def init_persistence(default_conf): init(default_conf) -def test_init_create_session(default_conf, mocker): - mocker.patch.dict('freqtrade.persistence._CONF', default_conf) - +def test_init_create_session(default_conf): # Check if init create a session init(default_conf) assert hasattr(Trade, 'session') @@ -29,20 +28,17 @@ def test_init_custom_db_url(default_conf, mocker): # Update path to a value other than default, but still in-memory conf.update({'db_url': 'sqlite:///tmp/freqtrade2_test.sqlite'}) create_engine_mock = mocker.patch('freqtrade.persistence.create_engine', MagicMock()) - mocker.patch.dict('freqtrade.persistence._CONF', conf) init(conf) assert create_engine_mock.call_count == 1 assert create_engine_mock.mock_calls[0][1][0] == 'sqlite:///tmp/freqtrade2_test.sqlite' -def test_init_invalid_db_url(default_conf, mocker): +def test_init_invalid_db_url(default_conf): conf = deepcopy(default_conf) # Update path to a value other than default, but still in-memory conf.update({'db_url': 'unknown:///some.url'}) - mocker.patch.dict('freqtrade.persistence._CONF', conf) - with pytest.raises(OperationalException, match=r'.*no valid database URL*'): init(conf) @@ -53,7 +49,6 @@ def test_init_prod_db(default_conf, mocker): conf.update({'db_url': constants.DEFAULT_DB_PROD_URL}) create_engine_mock = mocker.patch('freqtrade.persistence.create_engine', MagicMock()) - mocker.patch.dict('freqtrade.persistence._CONF', conf) init(conf) assert create_engine_mock.call_count == 1 @@ -66,7 +61,6 @@ def test_init_dryrun_db(default_conf, mocker): conf.update({'db_url': constants.DEFAULT_DB_DRYRUN_URL}) create_engine_mock = mocker.patch('freqtrade.persistence.create_engine', MagicMock()) - mocker.patch.dict('freqtrade.persistence._CONF', conf) init(conf) assert create_engine_mock.call_count == 1 @@ -407,9 +401,12 @@ def test_migrate_old(mocker, default_conf, fee): assert trade.stake_amount == default_conf.get("stake_amount") assert trade.pair == "ETC/BTC" assert trade.exchange == "bittrex" + assert trade.max_rate == 0.0 + assert trade.stop_loss == 0.0 + assert trade.initial_stop_loss == 0.0 -def test_migrate_new(mocker, default_conf, fee): +def test_migrate_new(mocker, default_conf, fee, caplog): """ Test Database migration (starting with new pairformat) """ @@ -446,6 +443,11 @@ def test_migrate_new(mocker, default_conf, fee): # Create table using the old format engine.execute(create_table_old) engine.execute(insert_table_old) + + # fake previous backup + engine.execute("create table trades_bak as select * from trades") + + engine.execute("create table trades_bak1 as select * from trades") # Run init to test migration init(default_conf) @@ -460,3 +462,54 @@ def test_migrate_new(mocker, default_conf, fee): assert trade.stake_amount == default_conf.get("stake_amount") assert trade.pair == "ETC/BTC" assert trade.exchange == "binance" + assert trade.max_rate == 0.0 + assert trade.stop_loss == 0.0 + assert trade.initial_stop_loss == 0.0 + assert log_has("trying trades_bak1", caplog.record_tuples) + assert log_has("trying trades_bak2", caplog.record_tuples) + + +def test_adjust_stop_loss(limit_buy_order, limit_sell_order, fee): + trade = Trade( + pair='ETH/BTC', + stake_amount=0.001, + fee_open=fee.return_value, + fee_close=fee.return_value, + exchange='bittrex', + open_rate=1, + ) + + trade.adjust_stop_loss(trade.open_rate, 0.05, True) + assert trade.stop_loss == 0.95 + assert trade.max_rate == 1 + assert trade.initial_stop_loss == 0.95 + + # Get percent of profit with a lowre rate + trade.adjust_stop_loss(0.96, 0.05) + assert trade.stop_loss == 0.95 + assert trade.max_rate == 1 + assert trade.initial_stop_loss == 0.95 + + # Get percent of profit with a custom rate (Higher than open rate) + trade.adjust_stop_loss(1.3, -0.1) + assert round(trade.stop_loss, 8) == 1.17 + assert trade.max_rate == 1.3 + assert trade.initial_stop_loss == 0.95 + + # current rate lower again ... should not change + trade.adjust_stop_loss(1.2, 0.1) + assert round(trade.stop_loss, 8) == 1.17 + assert trade.max_rate == 1.3 + assert trade.initial_stop_loss == 0.95 + + # current rate higher... should raise stoploss + trade.adjust_stop_loss(1.4, 0.1) + assert round(trade.stop_loss, 8) == 1.26 + assert trade.max_rate == 1.4 + assert trade.initial_stop_loss == 0.95 + + # Initial is true but stop_loss set - so doesn't do anything + trade.adjust_stop_loss(1.7, 0.1, True) + assert round(trade.stop_loss, 8) == 1.26 + assert trade.max_rate == 1.4 + assert trade.initial_stop_loss == 0.95 diff --git a/requirements.txt b/requirements.txt index c22dcbdca..34fc237da 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,5 +1,5 @@ -ccxt==1.14.253 -SQLAlchemy==1.2.8 +ccxt==1.14.301 +SQLAlchemy==1.2.9 python-telegram-bot==10.1.0 arrow==0.12.1 cachetools==2.1.0 diff --git a/scripts/download_backtest_data.py b/scripts/download_backtest_data.py index 2f76c1232..686098f94 100755 --- a/scripts/download_backtest_data.py +++ b/scripts/download_backtest_data.py @@ -3,11 +3,14 @@ """This script generate json data from bittrex""" import json import sys -import os +from pathlib import Path import arrow -from freqtrade import (arguments, misc) +from freqtrade import arguments +from freqtrade.arguments import TimeRange from freqtrade.exchange import Exchange +from freqtrade.optimize import download_backtesting_testdata + DEFAULT_DL_PATH = 'user_data/data' @@ -17,25 +20,27 @@ args = arguments.parse_args() timeframes = args.timeframes -dl_path = os.path.join(DEFAULT_DL_PATH, args.exchange) +dl_path = Path(DEFAULT_DL_PATH).joinpath(args.exchange) if args.export: - dl_path = args.export + dl_path = Path(args.export) -if not os.path.isdir(dl_path): +if not dl_path.is_dir(): sys.exit(f'Directory {dl_path} does not exist.') -pairs_file = args.pairs_file if args.pairs_file else os.path.join(dl_path, 'pairs.json') -if not os.path.isfile(pairs_file): +pairs_file = Path(args.pairs_file) if args.pairs_file else dl_path.joinpath('pairs.json') +if not pairs_file.exists(): sys.exit(f'No pairs file found with path {pairs_file}.') -with open(pairs_file) as file: +with pairs_file.open() as file: PAIRS = list(set(json.load(file))) PAIRS.sort() -since_time = None + +timerange = TimeRange() if args.days: - since_time = arrow.utcnow().shift(days=-args.days).timestamp * 1000 + time_since = arrow.utcnow().shift(days=-args.days).strftime("%Y%m%d") + timerange = arguments.parse_timerange(f'{time_since}-') print(f'About to download pairs: {PAIRS} to {dl_path}') @@ -59,21 +64,18 @@ for pair in PAIRS: print(f"skipping pair {pair}") continue for tick_interval in timeframes: - print(f'downloading pair {pair}, interval {tick_interval}') - - data = exchange.get_ticker_history(pair, tick_interval, since_ms=since_time) - if not data: - print('\tNo data was downloaded') - break - - print('\tData was downloaded for period %s - %s' % ( - arrow.get(data[0][0] / 1000).format(), - arrow.get(data[-1][0] / 1000).format())) - - # save data pair_print = pair.replace('/', '_') filename = f'{pair_print}-{tick_interval}.json' - misc.file_dump_json(os.path.join(dl_path, filename), data) + dl_file = dl_path.joinpath(filename) + if args.erase and dl_file.exists(): + print(f'Deleting existing data for pair {pair}, interval {tick_interval}') + dl_file.unlink() + + print(f'downloading pair {pair}, interval {tick_interval}') + download_backtesting_testdata(str(dl_path), exchange=exchange, + pair=pair, + tick_interval=tick_interval, + timerange=timerange) if pairs_not_available: diff --git a/scripts/plot_dataframe.py b/scripts/plot_dataframe.py index 7f9641222..1cc6b818a 100755 --- a/scripts/plot_dataframe.py +++ b/scripts/plot_dataframe.py @@ -25,11 +25,13 @@ Example of usage: --indicators2 fastk,fastd """ import logging -import os import sys +import json +from pathlib import Path from argparse import Namespace from typing import Dict, List, Any +import pandas as pd import plotly.graph_objs as go from plotly import tools from plotly.offline import plot @@ -37,7 +39,7 @@ from plotly.offline import plot import freqtrade.optimize as optimize from freqtrade import persistence from freqtrade.analyze import Analyze -from freqtrade.arguments import Arguments +from freqtrade.arguments import Arguments, TimeRange from freqtrade.exchange import Exchange from freqtrade.optimize.backtesting import setup_configuration from freqtrade.persistence import Trade @@ -46,6 +48,45 @@ logger = logging.getLogger(__name__) _CONF: Dict[str, Any] = {} +def load_trades(args: Namespace, pair: str, timerange: TimeRange) -> pd.DataFrame: + trades: pd.DataFrame = pd.DataFrame() + if args.db_url: + persistence.init(_CONF) + columns = ["pair", "profit", "opents", "closets", "open_rate", "close_rate", "duration"] + + trades = pd.DataFrame([(t.pair, t.calc_profit(), + t.open_date, t.close_date, + t.open_rate, t.close_rate, + t.close_date.timestamp() - t.open_date.timestamp()) + for t in Trade.query.filter(Trade.pair.is_(pair)).all()], + columns=columns) + + if args.exportfilename: + file = Path(args.exportfilename) + # must align with columns in backtest.py + columns = ["pair", "profit", "opents", "closets", "index", "duration", + "open_rate", "close_rate", "open_at_end"] + with file.open() as f: + data = json.load(f) + trades = pd.DataFrame(data, columns=columns) + trades = trades.loc[trades["pair"] == pair] + if timerange: + if timerange.starttype == 'date': + trades = trades.loc[trades["opents"] >= timerange.startts] + if timerange.stoptype == 'date': + trades = trades.loc[trades["opents"] <= timerange.stopts] + + trades['opents'] = pd.to_datetime(trades['opents'], + unit='s', + utc=True, + infer_datetime_format=True) + trades['closets'] = pd.to_datetime(trades['closets'], + unit='s', + utc=True, + infer_datetime_format=True) + return trades + + def plot_analyzed_dataframe(args: Namespace) -> None: """ Calls analyze() and plots the returned dataframe @@ -102,31 +143,32 @@ def plot_analyzed_dataframe(args: Namespace) -> None: if tickers == {}: exit() + if args.db_url and args.exportfilename: + logger.critical("Can only specify --db-url or --export-filename") # Get trades already made from the DB - trades: List[Trade] = [] - if args.db_url: - persistence.init(_CONF) - trades = Trade.query.filter(Trade.pair.is_(pair)).all() + trades = load_trades(args, pair, timerange) dataframes = analyze.tickerdata_to_dataframe(tickers) dataframe = dataframes[pair] dataframe = analyze.populate_buy_trend(dataframe) dataframe = analyze.populate_sell_trend(dataframe) - if len(dataframe.index) > 750: - logger.warning('Ticker contained more than 750 candles, clipping.') - + if len(dataframe.index) > args.plot_limit: + logger.warning('Ticker contained more than %s candles as defined ' + 'with --plot-limit, clipping.', args.plot_limit) + dataframe = dataframe.tail(args.plot_limit) + trades = trades.loc[trades['opents'] >= dataframe.iloc[0]['date']] fig = generate_graph( pair=pair, trades=trades, - data=dataframe.tail(750), + data=dataframe, args=args ) - plot(fig, filename=os.path.join('user_data', 'freqtrade-plot.html')) + plot(fig, filename=str(Path('user_data').joinpath('freqtrade-plot.html'))) -def generate_graph(pair, trades, data, args) -> tools.make_subplots: +def generate_graph(pair, trades: pd.DataFrame, data: pd.DataFrame, args) -> tools.make_subplots: """ Generate the graph from the data generated by Backtesting or from DB :param pair: Pair to Display on the graph @@ -187,8 +229,8 @@ def generate_graph(pair, trades, data, args) -> tools.make_subplots: ) trade_buys = go.Scattergl( - x=[t.open_date.isoformat() for t in trades], - y=[t.open_rate for t in trades], + x=trades["opents"], + y=trades["open_rate"], mode='markers', name='trade_buy', marker=dict( @@ -199,8 +241,8 @@ def generate_graph(pair, trades, data, args) -> tools.make_subplots: ) ) trade_sells = go.Scattergl( - x=[t.close_date.isoformat() for t in trades], - y=[t.close_rate for t in trades], + x=trades["closets"], + y=trades["close_rate"], mode='markers', name='trade_sell', marker=dict( @@ -299,11 +341,17 @@ def plot_parse_args(args: List[str]) -> Namespace: default='macd', dest='indicators2', ) - + arguments.parser.add_argument( + '--plot-limit', + help='Specify tick limit for plotting - too high values cause huge files - ' + 'Default: %(default)s', + dest='plot_limit', + default=750, + type=int, + ) arguments.common_args_parser() arguments.optimizer_shared_options(arguments.parser) arguments.backtesting_options(arguments.parser) - return arguments.parse_args()