Merge branch 'develop' into add-continual-learning

2022-09-10 08:24:04 +02:00
parent a826c0eb83 d38cc06139
commit 5705b8759b
20 changed files with 563 additions and 175 deletions
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -4,7 +4,7 @@

 Freqtrade supports spot trading only.

-### Can I open short positions?
+### Can my bot open short positions?

 Freqtrade can open short positions in futures markets.
 This requires the strategy to be made for this - and `"trading_mode": "futures"` in the configuration.
@@ -12,9 +12,9 @@ Please make sure to read the [relevant documentation page](leverage.md) first.

 In spot markets, you can in some cases use leveraged spot tokens, which reflect an inverted pair (eg. BTCUP/USD, BTCDOWN/USD, ETHBULL/USD, ETHBEAR/USD,...) which can be traded with Freqtrade.

-### Can I trade options or futures?
+### Can my bot trade options or futures?

-Futures trading is supported for selected exchanges.
+Futures trading is supported for selected exchanges. Please refer to the [documentation start page](index.md#supported-futures-exchanges-experimental) for an uptodate list of supported exchanges.

 ## Beginner Tips & Tricks

@@ -22,6 +22,13 @@ Futures trading is supported for selected exchanges.

 ## Freqtrade common issues

+### Can freqtrade open multiple positions on the same pair in parallel?
+
+No. Freqtrade will only open one position per pair at a time.
+You can however use the [`adjust_trade_position()` callback](strategy-callbacks.md#adjust-trade-position) to adjust an open position.
+
+Backtesting provides an option for this in `--eps` - however this is only there to highlight "hidden" signals, and will not work in live.
+
 ### The bot does not start

 Running the bot with `freqtrade trade --config config.json` shows the output `freqtrade: command not found`.
@@ -30,7 +37,7 @@ This could be caused by the following reasons:

 * The virtual environment is not active.
  * Run `source .env/bin/activate` to activate the virtual environment.
-* The installation did not work correctly.
+* The installation did not complete successfully.
  * Please check the [Installation documentation](installation.md).

 ### I have waited 5 minutes, why hasn't the bot made any trades yet?
@@ -67,7 +74,7 @@ This is not a bot-problem, but will also happen while manual trading.
 While freqtrade can handle this (it'll sell 99 COIN), fees are often below the minimum tradable lot-size (you can only trade full COIN, not 0.9 COIN).
 Leaving the dust (0.9 COIN) on the exchange makes usually sense, as the next time freqtrade buys COIN, it'll eat into the remaining small balance, this time selling everything it bought, and therefore slowly declining the dust balance (although it most likely will never reach exactly 0).

-Where possible (e.g. on binance), the use of the exchange's dedicated fee currency will fix this.  
+Where possible (e.g. on binance), the use of the exchange's dedicated fee currency will fix this.
 On binance, it's sufficient to have BNB in your account, and have "Pay fees in BNB" enabled in your profile. Your BNB balance will slowly decline (as it's used to pay fees) - but you'll no longer encounter dust (Freqtrade will include the fees in the profit calculations).
 Other exchanges don't offer such possibilities, where it's simply something you'll have to accept or move to a different exchange.

@@ -109,7 +116,7 @@ This warning can point to one of the below problems:

 ### I'm getting the "RESTRICTED_MARKET" message in the log

-Currently known to happen for US Bittrex users.  
+Currently known to happen for US Bittrex users.

 Read [the Bittrex section about restricted markets](exchanges.md#restricted-markets) for more information.

@@ -177,8 +184,8 @@ The GPU improvements would only apply to pandas-native calculations - or ones wr
 For hyperopt, freqtrade is using scikit-optimize, which is built on top of scikit-learn.
 Their statement about GPU support is [pretty clear](https://scikit-learn.org/stable/faq.html#will-you-add-gpu-support).

-GPU's also are only good at crunching numbers (floating point operations). 
-For hyperopt, we need both number-crunching (find next parameters) and running python code (running backtesting). 
+GPU's also are only good at crunching numbers (floating point operations).
+For hyperopt, we need both number-crunching (find next parameters) and running python code (running backtesting).
 As such, GPU's are not too well suited for most parts of hyperopt.

 The benefit of using GPU would therefore be pretty slim - and will not justify the complexity introduced by trying to add GPU support.
@@ -219,9 +226,9 @@ already 8\*10^9\*10 evaluations. A roughly total of 80 billion evaluations.
 Did you run 100 000 evaluations? Congrats, you've done roughly 1 / 100 000 th
 of the search space, assuming that the bot never tests the same parameters more than once.

-* The time it takes to run 1000 hyperopt epochs depends on things like: The available cpu, hard-disk, ram, timeframe, timerange, indicator settings, indicator count, amount of coins that hyperopt test strategies on and the resulting trade count - which can be 650 trades in a year or 100000 trades depending if the strategy aims for big profits by trading rarely or for many low profit trades. 
+* The time it takes to run 1000 hyperopt epochs depends on things like: The available cpu, hard-disk, ram, timeframe, timerange, indicator settings, indicator count, amount of coins that hyperopt test strategies on and the resulting trade count - which can be 650 trades in a year or 100000 trades depending if the strategy aims for big profits by trading rarely or for many low profit trades.

-Example: 4% profit 650 times vs 0,3% profit a trade 10000 times in a year. If we assume you set the --timerange to 365 days. 
+Example: 4% profit 650 times vs 0,3% profit a trade 10000 times in a year. If we assume you set the --timerange to 365 days.

 Example:
 `freqtrade --config config.json --strategy SampleStrategy --hyperopt SampleHyperopt -e 1000 --timerange 20190601-20200601`
--- a/docs/freqai.md
+++ b/docs/freqai.md
@@ -114,6 +114,8 @@ Mandatory parameters are marked as **Required**, which means that they are requi
 | `use_SVM_to_remove_outliers` | Train a support vector machine to detect and remove outliers from the training data set, as well as from incoming data points. See details about how it works [here](#removing-outliers-using-a-support-vector-machine-svm). <br> **Datatype:** Boolean.
 | `svm_params` | All parameters available in Sklearn's `SGDOneClassSVM()`. See details about some select parameters [here](#removing-outliers-using-a-support-vector-machine-svm). <br> **Datatype:** Dictionary.
 | `use_DBSCAN_to_remove_outliers` | Cluster data using DBSCAN to identify and remove outliers from training and prediction data. See details about how it works [here](#removing-outliers-with-dbscan). <br> **Datatype:** Boolean. 
+| `inlier_metric_window` | If set, FreqAI will add the `inlier_metric` to the training feature set and set the lookback to be the `inlier_metric_window`. Details of how the `inlier_metric` is computed can be found [here](#using-the-inliermetric) <br> **Datatype:** int. Default: 0
+| `noise_standard_deviation` | If > 0, FreqAI adds noise to the training features. FreqAI generates random deviates from a gaussian distribution with a standard deviation of `noise_standard_deviation` and adds them to all data points. Value should be kept relative to the normalized space between -1 and 1). In other words, since data is always normalized between -1 and 1 in FreqAI, the user can expect a `noise_standard_deviation: 0.05` to see 32% of data randomly increased/decreased by more than 2.5% (i.e. the percent of data falling within the first standard deviation). Good for preventing overfitting. <br> **Datatype:** int. Default: 0
 | `outlier_protection_percentage` | If more than `outlier_protection_percentage` % of points are detected as outliers by the SVM or DBSCAN, FreqAI will log a warning message and ignore outlier detection while keeping the original dataset intact. If the outlier protection is triggered, no predictions will be made based on the training data. <br> **Datatype:** Float. Default: `30`
 | `reverse_train_test_order` | If true, FreqAI will train on the latest data split and test on historical split of the data. This allows the model to be trained up to the most recent data point, while avoiding overfitting. However, users should be careful to understand unorthodox nature of this parameter before employing it. <br> **Datatype:** Boolean. Default: False
 |  |  **Data split parameters**
@@ -644,6 +646,18 @@ testing; the other points are used for training.

 The test data is used to evaluate the performance of the model after training. If the test score is high, the model is able to capture the behavior of the data well. If the test score is low, either the model either does not capture the complexity of the data, the test data is significantly different from the train data, or a different model should be used.

+### Using the `inlier_metric`
+
+The `inlier_metric` is a metric aimed at quantifying how different a prediction data point is from the most recent historic data points. 
+
+User can set `inlier_metric_window` to set the look back window. FreqAI will compute the distance between the present prediction point and each of the previous data points (total of `inlier_metric_window` points). 
+
+This function goes one step further - during training, it computes the `inlier_metric` for all training data points and builds weibull distributions for each each lookback point. The cumulative distribution function for the weibull distribution is used to produce a quantile for each of the data points. The quantiles for each lookback point are averaged to create the `inlier_metric`. 
+
+FreqAI adds this `inlier_metric` score to the training features! In other words, your model is trained to recognize how this temporal inlier metric is related to the user set labels. 
+
+This function does **not** remove outliers from the data set.
+
 ### Controlling the model learning process

 Model training parameters are unique to the machine learning library selected by the user. FreqAI allows the user to set any parameter for any library using the `model_training_parameters` dictionary in the user configuration file. The example configuration file (found in `config_examples/config_freqai.example.json`) show some of the example parameters associated with `Catboost` and `LightGBM`, but the user can add any parameters available in those libraries.
--- a/docs/telegram-usage.md
+++ b/docs/telegram-usage.md
@@ -90,7 +90,8 @@ Example configuration showing the different settings:
            "trailing_stop_loss": "on",
            "stop_loss": "on",
            "stoploss_on_exchange": "on",
-            "custom_exit": "silent"
+            "custom_exit": "silent",
+            "partial_exit": "on"
        },
        "entry_cancel": "silent",
        "exit_cancel": "on",
@@ -138,7 +139,7 @@ You can create your own keyboard in `config.json`:
      "enabled": true,
      "token": "your_telegram_token",
      "chat_id": "your_telegram_chat_id",
-      "keyboard": [   
+      "keyboard": [
          ["/daily", "/stats", "/balance", "/profit"],
          ["/status table", "/performance"],
          ["/reload_config", "/count", "/logs"]
@@ -225,16 +226,16 @@ Once all positions are sold, run `/stop` to completely stop the bot.
 For each open trade, the bot will send you the following message.
 Enter Tag is configurable via Strategy.

-> **Trade ID:** `123` `(since 1 days ago)`  
-> **Current Pair:** CVC/BTC  
+> **Trade ID:** `123` `(since 1 days ago)`
+> **Current Pair:** CVC/BTC
 > **Direction:** Long
 > **Leverage:** 1.0
-> **Amount:** `26.64180098`  
+> **Amount:** `26.64180098`
 > **Enter Tag:** Awesome Long Signal
-> **Open Rate:** `0.00007489`  
-> **Current Rate:** `0.00007489`  
-> **Current Profit:** `12.95%`  
-> **Stoploss:** `0.00007389 (-0.02%)`  
+> **Open Rate:** `0.00007489`
+> **Current Rate:** `0.00007489`
+> **Current Profit:** `12.95%`
+> **Stoploss:** `0.00007389 (-0.02%)`

 ### /status table

@@ -261,26 +262,26 @@ current    max

 Return a summary of your profit/loss and performance.

-> **ROI:** Close trades  
->   ∙ `0.00485701 BTC (2.2%) (15.2 Σ%)`  
->   ∙ `62.968 USD`  
-> **ROI:** All trades  
->   ∙ `0.00255280 BTC (1.5%) (6.43 Σ%)`  
->   ∙ `33.095 EUR`  
->  
-> **Total Trade Count:** `138`  
-> **First Trade opened:** `3 days ago`  
-> **Latest Trade opened:** `2 minutes ago`  
-> **Avg. Duration:** `2:33:45`  
-> **Best Performing:** `PAY/BTC: 50.23%`  
-> **Trading volume:** `0.5 BTC`  
-> **Profit factor:** `1.04`  
-> **Max Drawdown:** `9.23% (0.01255 BTC)`  
+> **ROI:** Close trades
+>   ∙ `0.00485701 BTC (2.2%) (15.2 Σ%)`
+>   ∙ `62.968 USD`
+> **ROI:** All trades
+>   ∙ `0.00255280 BTC (1.5%) (6.43 Σ%)`
+>   ∙ `33.095 EUR`
+>
+> **Total Trade Count:** `138`
+> **First Trade opened:** `3 days ago`
+> **Latest Trade opened:** `2 minutes ago`
+> **Avg. Duration:** `2:33:45`
+> **Best Performing:** `PAY/BTC: 50.23%`
+> **Trading volume:** `0.5 BTC`
+> **Profit factor:** `1.04`
+> **Max Drawdown:** `9.23% (0.01255 BTC)`

-The relative profit of `1.2%` is the average profit per trade.  
-The relative profit of `15.2 Σ%` is be based on the starting capital - so in this case, the starting capital was `0.00485701 * 1.152 = 0.00738 BTC`.  
-Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.  
-Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.  
+The relative profit of `1.2%` is the average profit per trade.
+The relative profit of `15.2 Σ%` is be based on the starting capital - so in this case, the starting capital was `0.00485701 * 1.152 = 0.00738 BTC`.
+Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
+Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
 Max drawdown corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.

 ### /forceexit <trade_id>
@@ -309,27 +310,27 @@ Note that for this to work, `force_entry_enable` needs to be set to true.
 ### /performance

 Return the performance of each crypto-currency the bot has sold.
-> Performance:  
-> 1. `RCN/BTC 0.003 BTC (57.77%) (1)`  
-> 2. `PAY/BTC 0.0012 BTC (56.91%) (1)`  
-> 3. `VIB/BTC 0.0011 BTC (47.07%) (1)`  
-> 4. `SALT/BTC 0.0010 BTC (30.24%) (1)`  
-> 5. `STORJ/BTC 0.0009 BTC (27.24%) (1)`  
-> ...  
+> Performance:
+> 1. `RCN/BTC 0.003 BTC (57.77%) (1)`
+> 2. `PAY/BTC 0.0012 BTC (56.91%) (1)`
+> 3. `VIB/BTC 0.0011 BTC (47.07%) (1)`
+> 4. `SALT/BTC 0.0010 BTC (30.24%) (1)`
+> 5. `STORJ/BTC 0.0009 BTC (27.24%) (1)`
+> ...

 ### /balance

 Return the balance of all crypto-currency your have on the exchange.

-> **Currency:** BTC  
-> **Available:** 3.05890234  
-> **Balance:** 3.05890234  
-> **Pending:** 0.0  
+> **Currency:** BTC
+> **Available:** 3.05890234
+> **Balance:** 3.05890234
+> **Pending:** 0.0

-> **Currency:** CVC  
-> **Available:** 86.64180098  
-> **Balance:** 86.64180098  
-> **Pending:** 0.0  
+> **Currency:** CVC
+> **Available:** 86.64180098
+> **Balance:** 86.64180098
+> **Pending:** 0.0

 ### /daily <n>

@@ -376,7 +377,7 @@ Month (count)  Profit BTC      Profit USD    Profit %

 Shows the current whitelist

-> Using whitelist `StaticPairList` with 22 pairs  
+> Using whitelist `StaticPairList` with 22 pairs
 > `IOTA/BTC, NEO/BTC, TRX/BTC, VET/BTC, ADA/BTC, ETC/BTC, NCASH/BTC, DASH/BTC, XRP/BTC, XVG/BTC, EOS/BTC, LTC/BTC, OMG/BTC, BTG/BTC, LSK/BTC, ZEC/BTC, HOT/BTC, IOTX/BTC, XMR/BTC, AST/BTC, XLM/BTC, NANO/BTC`

 ### /blacklist [pair]
@@ -386,7 +387,7 @@ If Pair is set, then this pair will be added to the pairlist.
 Also supports multiple pairs, separated by a space.
 Use `/reload_config` to reset the blacklist.

-> Using blacklist `StaticPairList` with 2 pairs  
+> Using blacklist `StaticPairList` with 2 pairs
 >`DODGE/BTC`, `HOT/BTC`.

 ### /edge