adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into align_userdata

Browse Source
This commit is contained in:
Matthias
2019-08-10 20:15:07 +02:00
parent 383b24ab84 8ba7657007
commit 2c5a499a8b
33 changed files with 1007 additions and 368 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
docs/backtesting.md
View File

@@ -57,7 +57,15 @@ freqtrade backtesting --datadir freqtrade/tests/testdata-20180101
freqtrade -s TestStrategy backtesting
```
Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory
Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory.
#### Comparing multiple Strategies
```bash
freqtrade backtesting --strategy-list TestStrategy1 AwesomeStrategy --ticker-interval 5m
```
Where `TestStrategy1` and `AwesomeStrategy` refer to class names of strategies.
#### Exporting trades to file

2
docs/bot-usage.md
View File

@@ -193,7 +193,7 @@ optional arguments:
number).
-l, --live Use live data.
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
Provide a commaseparated list of strategies to
Provide a space-separated list of strategies to
backtest Please note that ticker-interval needs to be
set either in config or via command line. When using
this together with --export trades, the strategy-name

224
docs/data-analysis.md
View File

@@ -1,164 +1,114 @@
# Analyzing bot data
After performing backtests, or after running the bot for some time, it will be interesting to analyze the results your bot generated.
You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation).
A good way for this is using Jupyter (notebook or lab) - which provides an interactive environment to analyze the data.
*Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*
The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results.
## Example snippets
## Strategy development problem analysis
Debugging a strategy (are there no buy signals, ...) can be very time-consuming.
FreqTrade tries to help you by exposing a few helper-functions, which can be very handy.
It's recommended using Juptyer Notebooks for analysis, since it offers a dynamic way to rerun certain parts of the code.
The following is a full code-snippet, which will be explained by both comments, and step by step below.
### Load backtest results into a pandas dataframe
```python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = '../xmatt/strategies'
# Location of the data
data_location = '../freqtrade/user_data/data/binance/'
# Only use one pair here
pair = "XRP_ETH"
### End constants
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
### Start strategy reload
# Load strategy - best done in a new cell
# Rerun each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
### Explanation
#### Imports and constant definition
``` python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = 'user_data/strategies'
# Location of the data
data_location = 'user_data/data/binance'
# Only use one pair here
pair = "XRP_ETH"
```
This first section imports necessary modules, and defines some constants you'll probably need to adjust for your case.
#### Load candles
``` python
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
```
This second section loads the historic data and prints the amount of candles in the DataFrame.
You can also inspect this dataframe by using `bt_data.head()` or `bt_data.tail()`.
#### Run strategy and analyze results
Now, it's time to load and run your strategy.
For this, I recommend using a new cell in your notebook, since you'll want to repeat this until you're satisfied with your strategy.
``` python
# Load strategy - best done in a new cell
# Needs to be ran each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
The code snippet loads and analyzes the strategy, calculates and prints the number of buy signals.
The last 2 lines serve to analyze the dataframe in detail.
This can be important if your strategy did not generate any buy signals.
Note that using `data.head()` would also work, however this is misleading since most indicators have some "startup" time at the start of a backtested dataframe.
There can be many things wrong, some signs to look for are:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
## Backtesting
To analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis.
Freqtrade provides the `load_backtest_data()` helper function to easily load the backtest results, which takes the path to the the backtest-results file as parameter.
``` python
from freqtrade.data.btanalysis import load_backtest_data
# Load backtest results
df = load_backtest_data("user_data/backtest_results/backtest-result.json")
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload.
If you have some ideas for interesting / helpful backtest data analysis ideas, please submit a Pull Request so the community can benefit from it.
## Live data
To analyze the trades your bot generated, you can load them to a DataFrame as follows:
### Load live trading results into a pandas dataframe
``` python
from freqtrade.data.btanalysis import load_trades_from_db
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts()
```
## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Import requirements and define variables used in analyses
```python
# Imports
from pathlib import Path
import os
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# You can override strategy settings as demonstrated below.
# Customize these according to your needs.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'AwesomeStrategy'
# Path to user data
user_data_dir = 'user_data'
# Location of the strategy
strategy_location = Path(user_data_dir, 'strategies')
# Location of the data
data_location = Path(user_data_dir, 'data', 'binance')
# Pair to analyze
# Only use one pair here
pair = "BTC_USDT"
```
### Load exchange data
```python
# Load data using values set above
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print(f"Loaded {len(bt_data)} rows of data for {pair} from {data_location}")
```
### Load and run strategy
* Rerun each time the strategy file is changed
```python
# Load strategy using values set above
strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(bt_data, {'pair': pair})
```
### Display the trade details
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
#### Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
#### Comparison with full backtest
having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29).
The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

56
docs/hyperopt.md
View File

@@ -18,19 +18,24 @@ Configuring hyperopt is similar to writing your own strategy, and many tasks wil
### Checklist on all tasks / possibilities in hyperopt
Depending on the space you want to optimize, only some of the below are required.
Depending on the space you want to optimize, only some of the below are required:
* fill `populate_indicators` - probably a copy from your strategy
* fill `buy_strategy_generator` - for buy signal optimization
* fill `indicator_space` - for buy signal optimzation
* fill `sell_strategy_generator` - for sell signal optimization
* fill `sell_indicator_space` - for sell signal optimzation
* fill `roi_space` - for ROI optimization
* fill `generate_roi_table` - for ROI optimization (if you need more than 3 entries)
* fill `stoploss_space` - stoploss optimization
* Optional but recommended
* copy `populate_buy_trend` from your strategy - otherwise default-strategy will be used
* copy `populate_sell_trend` from your strategy - otherwise default-strategy will be used
Optional, but recommended:
* copy `populate_buy_trend` from your strategy - otherwise default-strategy will be used
* copy `populate_sell_trend` from your strategy - otherwise default-strategy will be used
Rarely you may also need to override:
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
* `generate_roi_table` - for custom ROI optimization (if you need more than 4 entries in the ROI table)
* `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
### 1. Install a Custom Hyperopt File
@@ -345,7 +350,7 @@ def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
### Understand Hyperopt ROI results
If you are optimizing ROI, you're result will look as follows and include a ROI table.
If you are optimizing ROI (i.e. if optimization search-space contains 'all' or 'roi'), your result will look as follows and include a ROI table:
```
Best result:
@@ -376,6 +381,41 @@ minimal_roi = {
}
```
If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps) with the values that can vary in the following ranges:
| # | minutes | ROI percentage |
|---|---|---|
| 1 | always 0 | 0.03...0.31 |
| 2 | 10...40 | 0.02...0.11 |
| 3 | 20...100 | 0.01...0.04 |
| 4 | 30...220 | always 0 |
This structure of the ROI table is sufficient in most cases. Override the `roi_space()` method defining the ranges desired if you need components of the ROI tables to vary in other ranges.
Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization in these methods if you need a different structure of the ROI table or other amount of rows (steps) in the ROI tables.
### Understand Hyperopt Stoploss results
If you are optimizing stoploss values (i.e. if optimization search-space contains 'all' or 'stoploss'), your result will look as follows and include stoploss:
```
Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params:
{ 'adx-value': 44,
'rsi-value': 29,
'adx-enabled': False,
'rsi-enabled': True,
'trigger': 'bb_lower'}
Stoploss: -0.37996664668703606
```
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.5...-0.02, which is sufficient in most cases.
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization.
### Validate backtesting results
Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.

11
docs/installation.md
View File

@@ -219,6 +219,17 @@ as the watchdog.
------
## Using Conda
Freqtrade can also be installed using Anaconda (or Miniconda).
``` bash
conda env create -f environment.yml
```
!!! Note:
This requires the [ta-lib](#1-install-ta-lib) C-library to be installed first.
## Windows
We recommend that Windows users use [Docker](docker.md) as this will work much easier and smoother (also more secure).

2
docs/requirements-docs.txt
View File

@@ -1 +1 @@
mkdocs-material==3.1.0
mkdocs-material==4.4.0