Merge branch 'develop' into data_handler

docs/backtesting.md

@@ -78,9 +78,11 @@ Please also read about the [strategy startup period](strategy-customization.md#s
 #### Supplying custom fee value
 
 Sometimes your account has certain fee rebates (fee reductions starting with a certain account size or monthly volume), which are not visible to ccxt.
-To account for this in backtesting, you can use `--fee 0.001` to supply this value to backtesting.
+To account for this in backtesting, you can use the `--fee` command line option to supply this value to backtesting.
 This fee must be a ratio, and will be applied twice (once for trade entry, and once for trade exit).
 
+For example, if the buying and selling commission fee is 0.1% (i.e., 0.001 written as ratio), then you would run backtesting as the following:
+
 ```bash
 freqtrade backtesting --fee 0.001
 ```

docs/configuration.md

@@ -40,9 +40,12 @@ Mandatory parameters are marked as **Required**, which means that they are requi
 
 |  Parameter | Description |
 |------------|-------------|
-| `max_open_trades` | **Required.** Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades).<br> ***Datatype:*** *Positive integer or -1.*
+| `max_open_trades` | **Required.** Number of trades open your bot will have. If -1 then it is ignored (i.e. potentially unlimited open trades). [More information below](#configuring-amount-per-trade).<br> ***Datatype:*** *Positive integer or -1.*
 | `stake_currency` | **Required.** Crypto-currency used for trading. [Strategy Override](#parameters-in-the-strategy). <br> ***Datatype:*** *String*
-| `stake_amount` | **Required.** Amount of crypto-currency your bot will use for each trade. Set it to `"unlimited"` to allow the bot to use all available balance. [More information below](#understand-stake_amount). [Strategy Override](#parameters-in-the-strategy). <br> ***Datatype:*** *Positive float or `"unlimited"`.*
+| `stake_amount` | **Required.** Amount of crypto-currency your bot will use for each trade. Set it to `"unlimited"` to allow the bot to use all available balance. [More information below](#configuring-amount-per-trade). [Strategy Override](#parameters-in-the-strategy). <br> ***Datatype:*** *Positive float or `"unlimited"`.*
+| `tradable_balance_ratio` | Ratio of the total account balance the bot is allowed to trade. [More information below](#configuring-amount-per-trade). <br>*Defaults to `0.99` 99%).*<br> ***Datatype:*** *Positive float between `0.1` and `1.0`.*
+| `amend_last_stake_amount` | Use reduced last stake amount if necessary. [More information below](#configuring-amount-per-trade). <br>*Defaults to `false`.* <br> ***Datatype:*** *Boolean*
+| `last_stake_amount_min_ratio` | Defines minimum stake amount that has to be left and executed. Applies only to the last stake amount when it's amended to a reduced value (i.e. if `amend_last_stake_amount` is set to `true`). [More information below](#configuring-amount-per-trade). <br>*Defaults to `0.5`.* <br> ***Datatype:*** *Float (as ratio)*
 | `amount_reserve_percent` | Reserve some amount in min pair stake amount. The bot will reserve `amount_reserve_percent` + stoploss value when calculating min pair stake amount in order to avoid possible trade refusals. <br>*Defaults to `0.05` (5%).* <br> ***Datatype:*** *Positive Float as ratio.*
 | `ticker_interval` | The ticker interval to use (e.g `1m`, `5m`, `15m`, `30m`, `1h` ...). [Strategy Override](#parameters-in-the-strategy). <br> ***Datatype:*** *String*
 | `fiat_display_currency` | Fiat currency used to show your profits. [More information below](#what-values-can-be-used-for-fiat_display_currency). <br> ***Datatype:*** *String*
@@ -131,20 +134,58 @@ Values set in the configuration file always overwrite values set in the strategy
 * `sell_profit_only` (ask_strategy)
 * `ignore_roi_if_buy_signal` (ask_strategy)
 
-### Understand stake_amount
+### Configuring amount per trade
 
-The `stake_amount` configuration parameter is an amount of crypto-currency your bot will use for each trade.
+There are several methods to configure how much of the stake currency the bot will use to enter a trade. All methods respect the [available balance configuration](#available-balance) as explained below.
 
-The minimal configuration value is 0.0001. Please check your exchange's trading minimums to avoid problems.
+#### Available balance
+
+By default, the bot assumes that the `complete amount - 1%` is at it's disposal, and when using [dynamic stake amount](#dynamic-stake-amount), it will split the complete balance into `max_open_trades` buckets per trade.
+Freqtrade will reserve 1% for eventual fees when entering a trade and will therefore not touch that by default.
+
+You can configure the "untouched" amount by using the `tradable_balance_ratio` setting.
+
+For example, if you have 10 ETH available in your wallet on the exchange and `tradable_balance_ratio=0.5` (which is 50%), then the bot will use a maximum amount of 5 ETH for trading and considers this as available balance. The rest of the wallet is untouched by the trades.
+
+!!! Warning
+    The `tradable_balance_ratio` setting applies to the current balance (free balance + tied up in trades). Therefore, assuming the starting balance of 1000, a configuration with `tradable_balance_ratio=0.99` will not guarantee that 10 currency units will always remain available on the exchange. For example, the free amount may reduce to 5 units if the total balance is reduced to 500 (either by a losing streak, or by withdrawing balance).
+
+#### Amend last stake amount
+
+Assuming we have the tradable balance of 1000 USDT, `stake_amount=400`, and `max_open_trades=3`.
+The bot would open 2 trades, and will be unable to fill the last trading slot, since the requested 400 USDT are no longer available, since 800 USDT are already tied in other trades.
+
+To overcome this, the option `amend_last_stake_amount` can be set to `True`, which will enable the bot to reduce stake_amount to the available balance in order to fill the last trade slot.
+
+In the example above this would mean:
+
+- Trade1: 400 USDT
+- Trade2: 400 USDT
+- Trade3: 200 USDT
+
+!!! Note
+    This option only applies with [Static stake amount](#static-stake-amount) - since [Dynamic stake amount](#dynamic-stake-amount) divides the balances evenly.
+
+!!! Note
+    The minimum last stake amount can be configured using `amend_last_stake_amount` - which defaults to 0.5 (50%). This means that the minimum stake amount that's ever used is `stake_amount * 0.5`. This avoids very low stake amounts, that are close to the minimum tradable amount for the pair and can be refused by the exchange.
+
+#### Static stake amount
+
+The `stake_amount` configuration statically configures the amount of stake-currency your bot will use for each trade.
+
+The minimal configuration value is 0.0001, however, please check your exchange's trading minimums for the stake currency you're using to avoid problems.
 
 This setting works in combination with `max_open_trades`. The maximum capital engaged in trades is `stake_amount * max_open_trades`.
 For example, the bot will at most use (0.05 BTC x 3) = 0.15 BTC, assuming a configuration of `max_open_trades=3` and `stake_amount=0.05`.
 
-To allow the bot to trade all the available `stake_currency` in your account set
+!!! Note
+    This setting respects the [available balance configuration](#available-balance).
 
-```json
-"stake_amount" : "unlimited",
-```
+#### Dynamic stake amount
+
+Alternatively, you can use a dynamic stake amount, which will use the available balance on the exchange, and divide that equally by the amount of allowed trades (`max_open_trades`).
+
+To configure this, set `stake_amount="unlimited"`. We also recommend to set `tradable_balance_ratio=0.99` (99%) - to keep a minimum balance for eventual fees.
 
 In this case a trade amount is calculated as:
 
@@ -152,6 +193,16 @@ In this case a trade amount is calculated as:
 currency_balance / (max_open_trades - current_open_trades)
 ```
 
+To allow the bot to trade all the available `stake_currency` in your account (minus `tradable_balance_ratio`) set
+
+```json
+"stake_amount" : "unlimited",
+"tradable_balance_ratio": 0.99,
+```
+
+!!! Note
+    This configuration will allow increasing / decreasing stakes depending on the performance of the bot (lower stake if bot is loosing, higher stakes if the bot has a winning record, since higher balances are available).
+
 !!! Note "When using Dry-Run Mode"
     When using `"stake_amount" : "unlimited",` in combination with Dry-Run, the balance will be simulated starting with a stake of `dry_run_wallet` which will evolve over time. It is therefore important to set `dry_run_wallet` to a sensible value (like 0.05 or 0.01 for BTC and 1000 or 100 for USDT, for example), otherwise it may simulate trades with 100 BTC (or more) or 0.05 USDT (or less) at once - which may not correspond to your real available balance or is less than the exchange minimal limit for the order amount for the stake currency.
 

docs/edge.md

@@ -148,7 +148,7 @@ Edge module has following configuration options:
 | `enabled` | If true, then Edge will run periodically. <br>*Defaults to `false`.* <br> ***Datatype:*** *Boolean*
 | `process_throttle_secs` | How often should Edge run in seconds. <br>*Defaults to `3600` (once per hour).* <br> ***Datatype:*** *Integer*
 | `calculate_since_number_of_days` | Number of days of data against which Edge calculates Win Rate, Risk Reward and Expectancy. <br> **Note** that it downloads historical data so increasing this number would lead to slowing down the bot. <br>*Defaults to `7`.* <br> ***Datatype:*** *Integer*
-| `capital_available_percentage` | This is the percentage of the total capital on exchange in stake currency. <br>As an example if you have 10 ETH available in your wallet on the exchange and this value is 0.5 (which is 50%), then the bot will use a maximum amount of 5 ETH for trading and considers it as available capital. <br>*Defaults to `0.5`.* <br> ***Datatype:*** *Float*
+| `capital_available_percentage` | **DEPRECATED - [replaced with `tradable_balance_ratio`](configuration.md#Available balance)** This is the percentage of the total capital on exchange in stake currency. <br>As an example if you have 10 ETH available in your wallet on the exchange and this value is 0.5 (which is 50%), then the bot will use a maximum amount of 5 ETH for trading and considers it as available capital. <br>*Defaults to `0.5`.* <br> ***Datatype:*** *Float*
 | `allowed_risk` | Ratio of allowed risk per trade. <br>*Defaults to `0.01` (1%)).* <br> ***Datatype:*** *Float*
 | `stoploss_range_min` | Minimum stoploss. <br>*Defaults to `-0.01`.* <br> ***Datatype:*** *Float*
 | `stoploss_range_max` | Maximum stoploss. <br>*Defaults to `-0.10`.* <br> ***Datatype:*** *Float*

docs/index.md

@@ -11,8 +11,10 @@
 <a class="github-button" href="https://github.com/freqtrade/freqtrade/archive/master.zip" data-icon="octicon-cloud-download" data-size="large" aria-label="Download freqtrade/freqtrade on GitHub">Download</a>
 <!-- Place this tag where you want the button to render. -->
 <a class="github-button" href="https://github.com/freqtrade" data-size="large" aria-label="Follow @freqtrade on GitHub">Follow @freqtrade</a>
+
 ## Introduction
-Freqtrade is a cryptocurrency trading bot written in Python.
+
+Freqtrade is a crypto-currency algorithmic trading software developed in python (3.6+) and supported on Windows, macOS and Linux.
 
 !!! Danger "DISCLAIMER"
     This software is for educational purposes only. Do not risk money which you are afraid to lose. USE THE SOFTWARE AT YOUR OWN RISK. THE AUTHORS AND ALL AFFILIATES ASSUME NO RESPONSIBILITY FOR YOUR TRADING RESULTS.
@@ -23,18 +25,15 @@ Freqtrade is a cryptocurrency trading bot written in Python.
 
 ## Features
 
- - Based on Python 3.6+: For botting on any operating system — Windows, macOS and Linux.
- - Persistence: Persistence is achieved through sqlite database.
- - Dry-run mode: Run the bot without playing money.
- - Backtesting: Run a simulation of your buy/sell strategy with historical data.
- - Strategy Optimization by machine learning: Use machine learning to optimize your buy/sell strategy parameters with real exchange data.
- - Edge position sizing: Calculate your win rate, risk reward ratio, the best stoploss and adjust your position size before taking a position for each specific market.
- - Whitelist crypto-currencies: Select which crypto-currency you want to trade or use dynamic whitelists based on market (pair) trade volume.
- - Blacklist crypto-currencies: Select which crypto-currency you want to avoid.
- - Manageable via Telegram or REST APi: Manage the bot with Telegram or via the builtin REST API.
- - Display profit/loss in fiat: Display your profit/loss in any of 33 fiat currencies supported.
- - Daily summary of profit/loss: Receive the daily summary of your profit/loss.
- - Performance status report: Receive the performance status of your current trades.
+- Develop your Strategy: Write your strategy in python, using [pandas](https://pandas.pydata.org/). Example strategies to inspire you are available in the [strategy repository](https://github.com/freqtrade/freqtrade-strategies).
+- Download market data: Download historical data of the exchange and the markets your may want to trade with.
+- Backtest: Test your strategy on downloaded historical data.
+- Optimize: Find the best parameters for your strategy using hyperoptimization which employs machining learning methods. You can optimize buy, sell, take profit (ROI), stop-loss and trailing stop-loss parameters for your strategy.
+- Select markets: Create your static list or use an automatic one based on top traded volumes and/or prices (not available during backtesting). You can also explicitly blacklist markets you don't want to trade.
+- Run: Test your strategy with simulated money (Dry-Run mode) or deploy it with real money (Live-Trade mode).
+- Run using Edge (optional module): The concept is to find the best historical [trade expectancy](edge.md#expectancy) by markets based on variation of the stop-loss and then allow/reject markets to trade. The sizing of the trade is based on a risk of a percentage of your capital.
+- Control/Monitor: Use Telegram or a REST API (start/stop the bot, show profit/loss, daily summary, current open trades results, etc.).
+- Analyse: Further analysis can be performed on either Backtesting data or Freqtrade trading history (SQL database), including automated standard plots, and methods to load the data into [interactive environments](data-analysis.md).
 
 ## Requirements
 
@@ -61,10 +60,10 @@ To run this bot we recommend you a cloud instance with a minimum of:
 
 ## Support
 
-Help / Slack
-For any questions not covered by the documentation or for further information about the bot, we encourage you to join our Slack channel.
+### Help / Slack
+For any questions not covered by the documentation or for further information about the bot, we encourage you to join our passionate  Slack community.
 
-Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) to join Slack channel.
+Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODcwNjI1MDU3LTU1MTgxMjkzNmYxNWE1MDEzYzQ3YmU4N2MwZjUyNjJjODRkMDVkNjg4YTAyZGYzYzlhOTZiMTE4ZjQ4YzM0OGE) to join the Freqtrade Slack channel.
 
 ## Ready to try?
 

docs/plotting.md

@@ -120,16 +120,77 @@ To plot trades from a backtesting result, use `--export-filename <filename>`
 freqtrade plot-dataframe --strategy AwesomeStrategy --export-filename user_data/backtest_results/backtest-result.json -p BTC/ETH
 ```
 
+### Plot dataframe basics
+
+![plot-dataframe2](assets/plot-dataframe2.png)
+
+The `plot-dataframe` subcommand requires backtesting data, a strategy and either a backtesting-results file or a database, containing trades corresponding to the strategy.
+
+The resulting plot will have the following elements:
+
+* Green triangles: Buy signals from the strategy. (Note: not every buy signal generates a trade, compare to cyan circles.)
+* Red triangles: Sell signals from the strategy. (Also, not every sell signal terminates a trade, compare to red and green squares.)
+* Cyan circles: Trade entry points.
+* Red squares: Trade exit points for trades with loss or 0% profit.
+* Green squares: Trade exit points for profitable trades.
+* Indicators with values corresponding to the candle scale (e.g. SMA/EMA), as specified with `--indicators1`.
+* Volume (bar chart at the bottom of the main chart).
+* Indicators with values in different scales (e.g. MACD, RSI) below the volume bars, as specified with `--indicators2`.
+
+!!! Note "Bollinger Bands"
+    Bollinger bands are automatically added to the plot if the columns `bb_lowerband` and `bb_upperband` exist, and are painted as a light blue area spanning from the lower band to the upper band.
+
+#### Advanced plot configuration
+
+An advanced plot configuration can be specified in the strategy in the `plot_config` parameter.
+
+Additional features when using plot_config include:
+
+* Specify colors per indicator
+* Specify additional subplots
+
+The sample plot configuration below specifies fixed colors for the indicators. Otherwise consecutive plots may produce different colorschemes each time, making comparisons difficult.
+It also allows multiple subplots to display both MACD and RSI at the same time.
+
+Sample configuration with inline comments explaining the process:
+
+``` python
+    plot_config = {
+        'main_plot': {
+            # Configuration for main plot indicators.
+            # Specifies `ema10` to be red, and `ema50` to be a shade of gray
+            'ema10': {'color': 'red'},
+            'ema50': {'color': '#CCCCCC'},
+            # By omitting color, a random color is selected.
+            'sar': {},
+        },
+        'subplots': {
+            # Create subplot MACD
+            "MACD": {
+                'macd': {'color': 'blue'},
+                'macdsignal': {'color': 'orange'},
+            },
+            # Additional subplot RSI
+            "RSI": {
+                'rsi': {'color': 'red'},
+            }
+        }
+    }
+```
+
+!!! Note
+    The above configuration assumes that `ema10`, `ema50`, `macd`, `macdsignal` and `rsi` are columns in the DataFrame created by the strategy.
+
 ## Plot profit
 
 ![plot-profit](assets/plot-profit.png)
 
-The `freqtrade plot-profit` subcommand shows an interactive graph with three plots:
+The `plot-profit` subcommand shows an interactive graph with three plots:
 
-1) Average closing price for all pairs
-2) The summarized profit made by backtesting.
-   Note that this is not the real-world profit, but more of an estimate.
-3) Profit for each individual pair
+* Average closing price for all pairs.
+* The summarized profit made by backtesting.
+Note that this is not the real-world profit, but more of an estimate.
+* Profit for each individual pair.
 
 The first graph is good to get a grip of how the overall market progresses.