adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2108 from jraviotta/nbdocs

Browse Source 
Added jupyter notebook example and doc edits
This commit is contained in:
Matthias 2019-08-10 15:47:06 +02:00 committed by GitHub
commit 29619ccf1c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 344 additions and 140 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@ -81,7 +81,6 @@ target/
# Jupyter Notebook # Jupyter Notebook
.ipynb_checkpoints .ipynb_checkpoints
*.ipynb
# pyenv # pyenv
.python-version .python-version
@ -93,3 +92,6 @@ target/
.pytest_cache/ .pytest_cache/
.mypy_cache/ .mypy_cache/
#exceptions
!user_data/noteboks/*example.ipynb

226
docs/data-analysis.md
View File

@ -1,164 +1,114 @@
# Analyzing bot data # Analyzing bot data
After performing backtests, or after running the bot for some time, it will be interesting to analyze the results your bot generated. You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation).
A good way for this is using Jupyter (notebook or lab) - which provides an interactive environment to analyze the data. *Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*
The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results. ## Example snippets
## Strategy development problem analysis ### Load backtest results into a pandas dataframe
Debugging a strategy (are there no buy signals, ...) can be very time-consuming.
FreqTrade tries to help you by exposing a few helper-functions, which can be very handy.
It's recommended using Juptyer Notebooks for analysis, since it offers a dynamic way to rerun certain parts of the code.
The following is a full code-snippet, which will be explained by both comments, and step by step below.
```python ```python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = '../xmatt/strategies'
# Location of the data
data_location = '../freqtrade/user_data/data/binance/'
# Only use one pair here
pair = "XRP_ETH"
### End constants
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
### Start strategy reload
# Load strategy - best done in a new cell
# Rerun each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
### Explanation
#### Imports and constant definition
``` python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = 'user_data/strategies'
# Location of the data
data_location = 'user_data/data/binance'
# Only use one pair here
pair = "XRP_ETH"
```
This first section imports necessary modules, and defines some constants you'll probably need to adjust for your case.
#### Load candles
``` python
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
```
This second section loads the historic data and prints the amount of candles in the DataFrame.
You can also inspect this dataframe by using `bt_data.head()` or `bt_data.tail()`.
#### Run strategy and analyze results
Now, it's time to load and run your strategy.
For this, I recommend using a new cell in your notebook, since you'll want to repeat this until you're satisfied with your strategy.
``` python
# Load strategy - best done in a new cell
# Needs to be ran each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
The code snippet loads and analyzes the strategy, calculates and prints the number of buy signals.
The last 2 lines serve to analyze the dataframe in detail.
This can be important if your strategy did not generate any buy signals.
Note that using `data.head()` would also work, however this is misleading since most indicators have some "startup" time at the start of a backtested dataframe.
There can be many things wrong, some signs to look for are:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
## Backtesting
To analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis.
Freqtrade provides the `load_backtest_data()` helper function to easily load the backtest results, which takes the path to the the backtest-results file as parameter.
``` python
from freqtrade.data.btanalysis import load_backtest_data from freqtrade.data.btanalysis import load_backtest_data
df = load_backtest_data("user_data/backtest-result.json") # Load backtest results
df = load_backtest_data("user_data/backtest_data/backtest-result.json")
# Show value-counts per pair # Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts() df.groupby("pair")["sell_reason"].value_counts()
``` ```
This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload. This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload.
If you have some ideas for interesting / helpful backtest data analysis ideas, please submit a Pull Request so the community can benefit from it. ### Load live trading results into a pandas dataframe
## Live data
To analyze the trades your bot generated, you can load them to a DataFrame as follows:
``` python ``` python
from freqtrade.data.btanalysis import load_trades_from_db from freqtrade.data.btanalysis import load_trades_from_db
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite") df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts() df.groupby("pair")["sell_reason"].value_counts()
```
## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Import requirements and define variables used in analyses
```python
# Imports
from pathlib import Path
import os
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# You can override strategy settings as demonstrated below.
# Customize these according to your needs.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'AwesomeStrategy'
# Path to user data
user_data_dir = 'user_data'
# Location of the strategy
strategy_location = Path(user_data_dir, 'strategies')
# Location of the data
data_location = Path(user_data_dir, 'data', 'binance')
# Pair to analyze
# Only use one pair here
pair = "BTC_USDT"
```
### Load exchange data
```python
# Load data using values set above
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print(f"Loaded {len(bt_data)} rows of data for {pair} from {data_location}")
```
### Load and run strategy
* Rerun each time the strategy file is changed
```python
# Load strategy using values set above
strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(bt_data, {'pair': pair})
```
### Display the trade details
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
#### Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
#### Comparison with full backtest
having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29).
The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
``` ```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data. Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

1
environment.yml
View File

@ -37,6 +37,7 @@ dependencies:
- coveralls - coveralls
- mypy - mypy
# Useful for jupyter # Useful for jupyter
- jupyter
- ipykernel - ipykernel
- isort - isort
- yapf - yapf

10
setup.py
View File

@ -25,7 +25,13 @@ develop = [
'pytest-random-order', 'pytest-random-order',
] ]
all_extra = api + plot + develop jupyter = [
'jupyter',
'nbstripout',
'ipykernel',
]
all_extra = api + plot + develop + jupyter
setup(name='freqtrade', setup(name='freqtrade',
version=__version__, version=__version__,
@ -68,6 +74,8 @@ setup(name='freqtrade',
'dev': all_extra, 'dev': all_extra,
'plot': plot, 'plot': plot,
'all': all_extra, 'all': all_extra,
'jupyter': jupyter,
}, },
include_package_data=True, include_package_data=True,
zip_safe=False, zip_safe=False,

0
user_data/notebooks/.gitkeep Normal file
View File

243
user_data/notebooks/analysis_example.ipynb Normal file
View File

@ -0,0 +1,243 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Analyzing bot data\n",
"\n",
"You can analyze the results of backtests and trading history easily using Jupyter notebooks. \n",
"**Copy this file so your changes don't get clobbered with the next freqtrade update!** \n",
"For usage instructions, see [jupyter.org](https://jupyter.org/documentation). \n",
"*Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"from pathlib import Path\n",
"import os\n",
"from freqtrade.data.history import load_pair_history\n",
"from freqtrade.resolvers import StrategyResolver\n",
"from freqtrade.data.btanalysis import load_backtest_data\n",
"from freqtrade.data.btanalysis import load_trades_from_db"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Change directory\n",
"# Define all paths relative to the project root shown in the cell output\n",
"try:\n",
" os.chdir(Path(Path.cwd(), '../..'))\n",
" print(Path.cwd())\n",
"except:\n",
" pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example snippets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load backtest results\n",
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example\n",
"\n",
"Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in analyses"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define some constants\n",
"ticker_interval = \"5m\"\n",
"# Name of the strategy class\n",
"strategy_name = 'AwesomeStrategy'\n",
"# Path to user data\n",
"user_data_dir = 'user_data'\n",
"# Location of the strategy\n",
"strategy_location = Path(user_data_dir, 'strategies')\n",
"# Location of the data\n",
"data_location = Path(user_data_dir, 'data', 'binance')\n",
"# Pair to analyze \n",
"# Only use one pair here\n",
"pair = \"BTC_USDT\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load exchange data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load data using values set above\n",
"bt_data = load_pair_history(datadir=Path(data_location),\n",
" ticker_interval=ticker_interval,\n",
" pair=pair)\n",
"\n",
"# Confirm success\n",
"print(\"Loaded \" + str(len(bt_data)) + f\" rows of data for {pair} from {data_location}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load and run strategy\n",
"* Rerun each time the strategy file is changed"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load strategy using values set above\n",
"strategy = StrategyResolver({'strategy': strategy_name,\n",
" 'user_data_dir': user_data_dir,\n",
" 'strategy_path': strategy_location}).strategy\n",
"\n",
"# Generate buy/sell signals using strategy\n",
"df = strategy.analyze_ticker(bt_data, {'pair': pair})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Display the trade details\n",
"* Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n",
"#### Some possible problems\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units\n",
"\n",
"#### Comparison with full backtest\n",
"\n",
"having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.\n",
"\n",
"Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple \"buy\" signals for each pair in sequence (until rsi returns > 29).\n",
"The bot will only buy on the first of these signals (and also only if a trade-slot (\"max_open_trades\") is still available), or on one of the middle signals, as soon as a \"slot\" becomes available.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n",
"data.tail()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data."
]
}
],
"metadata": {
"file_extension": ".py",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
},
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3
},
"nbformat": 4,
"nbformat_minor": 2
}