adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #2282 from freqtrade/jupyter_nbconvert

Browse Source 
add plotting documentation to jupyter notebook
This commit is contained in:
Matthias 2019-10-01 19:58:20 +02:00 committed by GitHub
commit 8c5b299449
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 269 additions and 154 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

124
docs/data-analysis.md
View File

@ -61,34 +61,6 @@ except:
print(Path.cwd())
```
## Load existing objects into a Jupyter notebook
These examples assume that you have already generated data using the cli. They will allow you to drill deeper into your results, and perform analysis which otherwise would make the output very difficult to digest due to information overload.
### Load backtest results into a pandas dataframe
```python
from freqtrade.data.btanalysis import load_backtest_data
# Load backtest results
df = load_backtest_data("user_data/backtest_results/backtest-result.json")
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
### Load live trading results into a pandas dataframe
``` python
from freqtrade.data.btanalysis import load_trades_from_db
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts()
```
### Load multiple configuration files
This option can be useful to inspect the results of passing in multiple configs.
@ -114,99 +86,9 @@ Best avoid relative paths, since this starts at the storage location of the jupy
}
```
### Load exchange data to a pandas dataframe
### Further Data analysis documentation
This loads candle data to a dataframe
```python
from pathlib import Path
from freqtrade.data.history import load_pair_history
# Load data using values passed to function
ticker_interval = "5m"
data_location = Path('user_data', 'data', 'bitrex')
pair = "BTC_USDT"
candles = load_pair_history(datadir=data_location,
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print(f"Loaded len(candles) rows of data for {pair} from {data_location}")
candles.head()
```
## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Define variables used in analyses
You can override strategy settings as demonstrated below.
```python
# Customize these according to your needs.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'SampleStrategy'
# Path to user data
user_data_dir = 'user_data'
# Location of the strategy
strategy_location = Path(user_data_dir, 'strategies')
# Location of the data
data_location = Path(user_data_dir, 'data', 'binance')
# Pair to analyze - Only use one pair here
pair = "BTC_USDT"
```
### Load exchange data
```python
from pathlib import Path
from freqtrade.data.history import load_pair_history
# Load data using values set above
candles = load_pair_history(datadir=data_location,
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print(f"Loaded {len(candles)} rows of data for {pair} from {data_location}")
candles.head()
```
### Load and run strategy
* Rerun each time the strategy file is changed
```python
from freqtrade.resolvers import StrategyResolver
# Load strategy using values set above
strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(candles, {'pair': pair})
```
### Display the trade details
* Note that using `data.tail()` is preferable to `data.head()` as most indicators have some "startup" data at the top of the dataframe.
* Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
* Comparison with full backtest
* having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
* Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29). The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
```
* [Strategy debugging](strategy_analysis_example.md) - also available as Jupyter notebook (`user_data/notebooks/strategy_analysis_example.ipynb`)
* [Plotting](plotting.md)
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

9
docs/developer.md
View File

@ -149,6 +149,15 @@ print(datetime.utcnow())
The output will show the last entry from the Exchange as well as the current UTC date.
If the day shows the same day, then the last candle can be assumed as incomplete and should be dropped (leave the setting `"ohlcv_partial_candle"` from the exchange-class untouched / True). Otherwise, set `"ohlcv_partial_candle"` to `False` to not drop Candles (shown in the example above).
## Updating example notebooks
To keep the jupyter notebooks aligned with the documentation, the following should be ran after updating a example notebook.
``` bash
jupyter nbconvert --ClearOutputPreprocessor.enabled=True --inplace user_data/notebooks/strategy_analysis_example.ipynb
jupyter nbconvert --ClearOutputPreprocessor.enabled=True --to markdown user_data/notebooks/strategy_analysis_example.ipynb --stdout > docs/strategy_analysis_example.md
```
## Creating a release
This part of the documentation is aimed at maintainers, and shows how to create a release.

142
docs/strategy_analysis_example.md Normal file
View File

@ -0,0 +1,142 @@
# Strategy analysis example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
## Setup
```python
from pathlib import Path
# Customize these according to your needs.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'SampleStrategy'
# Path to user data
user_data_dir = Path('user_data')
# Location of the strategy
strategy_location = user_data_dir / 'strategies'
# Location of the data
data_location = Path(user_data_dir, 'data', 'binance')
# Pair to analyze - Only use one pair here
pair = "BTC_USDT"
```
```python
# Load data using values set above
from freqtrade.data.history import load_pair_history
candles = load_pair_history(datadir=data_location,
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print("Loaded " + str(len(candles)) + f" rows of data for {pair} from {data_location}")
candles.head()
```
## Load and run strategy
* Rerun each time the strategy file is changed
```python
# Load strategy using values set above
from freqtrade.resolvers import StrategyResolver
strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(candles, {'pair': pair})
df.tail()
```
### Display the trade details
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
* Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
* Comparison with full backtest
* having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
* Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29). The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=False)
data.tail()
```
## Load existing objects into a Jupyter notebook
The following cells assume that you have already generated data using the cli.
They will allow you to drill deeper into your results, and perform analysis which otherwise would make the output very difficult to digest due to information overload.
### Load backtest results to pandas dataframe
Analyze a trades dataframe (also used below for plotting)
```python
from freqtrade.data.btanalysis import load_backtest_data
# Load backtest results
trades = load_backtest_data(user_data_dir / "backtest_results/backtest-result.json")
# Show value-counts per pair
trades.groupby("pair")["sell_reason"].value_counts()
```
### Load live trading results into a pandas dataframe
In case you did already some trading and want to analyze your performance
```python
from freqtrade.data.btanalysis import load_trades_from_db
# Fetch trades from database
trades = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
trades.groupby("pair")["sell_reason"].value_counts()
```
## Plot results
Freqtrade offers interactive plotting capabilities based on plotly.
```python
from freqtrade.plot.plotting import generate_candlestick_graph
# Limit graph period to keep plotly quick and reactive
data_red = data['2019-06-01':'2019-06-10']
# Generate candlestick graph
graph = generate_candlestick_graph(pair=pair,
data=data_red,
trades=trades,
indicators1=['sma20', 'ema50', 'ema55'],
indicators2=['rsi', 'macd', 'macdsignal', 'macdhist']
)
```
```python
# Show graph inline
# graph.show()
# Render graph in a seperate window
graph.show(renderer="browser")
```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

4
mkdocs.yml
View File

@ -16,7 +16,9 @@ nav:
- Hyperopt: hyperopt.md
- Edge positioning: edge.md
- FAQ: faq.md
- Data Analysis: data-analysis.md
- Data Analysis:
- Jupyter Notebooks: data-analysis.md
- Strategy analysis: strategy_analysis_example.md
- Plotting: plotting.md
- SQL Cheatsheet: sql_cheatsheet.md
- Sandbox testing: sandbox-testing.md

3
requirements-dev.txt
View File

@ -13,3 +13,6 @@ pytest-asyncio==0.10.0
pytest-cov==2.7.1
pytest-mock==1.11.0
pytest-random-order==1.0.4
# Convert jupyter notebooks to markdown documents
nbconvert==5.6.0

1
setup.py
View File

@ -43,6 +43,7 @@ jupyter = [
'jupyter',
'nbstripout',
'ipykernel',
'nbconvert',
]
all_extra = api + plot + develop + jupyter + hyperopt

138
user_data/notebooks/strategy_analysis_example.ipynb
View File

@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example\n",
"# Strategy analysis example\n",
"\n",
"Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data."
]
@ -22,31 +22,7 @@
"metadata": {},
"outputs": [],
"source": [
"# Change directory\n",
"# Modify this cell to insure that the output shows the correct path.\n",
"import os\n",
"from pathlib import Path\n",
"\n",
"# Define all paths relative to the project root shown in the cell output\n",
"project_root = \"somedir/freqtrade\"\n",
"i=0\n",
"try:\n",
" os.chdirdir(project_root)\n",
" assert Path('LICENSE').is_file()\n",
"except:\n",
" while i<4 and (not Path('LICENSE').is_file()):\n",
" os.chdir(Path(Path.cwd(), '../'))\n",
" i+=1\n",
" project_root = Path.cwd()\n",
"print(Path.cwd())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Customize these according to your needs.\n",
"\n",
"# Define some constants\n",
@ -54,9 +30,9 @@
"# Name of the strategy class\n",
"strategy_name = 'SampleStrategy'\n",
"# Path to user data\n",
"user_data_dir = 'user_data'\n",
"user_data_dir = Path('user_data')\n",
"# Location of the strategy\n",
"strategy_location = Path(user_data_dir, 'strategies')\n",
"strategy_location = user_data_dir / 'strategies'\n",
"# Location of the data\n",
"data_location = Path(user_data_dir, 'data', 'binance')\n",
"# Pair to analyze - Only use one pair here\n",
@ -70,7 +46,6 @@
"outputs": [],
"source": [
"# Load data using values set above\n",
"from pathlib import Path\n",
"from freqtrade.data.history import load_pair_history\n",
"\n",
"candles = load_pair_history(datadir=data_location,\n",
@ -132,10 +107,111 @@
"source": [
"# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n",
"data = df.set_index('date', drop=False)\n",
"data.tail()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load existing objects into a Jupyter notebook\n",
"\n",
"The following cells assume that you have already generated data using the cli. \n",
"They will allow you to drill deeper into your results, and perform analysis which otherwise would make the output very difficult to digest due to information overload."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results to pandas dataframe\n",
"\n",
"Analyze a trades dataframe (also used below for plotting)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from freqtrade.data.btanalysis import load_backtest_data\n",
"\n",
"# Load backtest results\n",
"trades = load_backtest_data(user_data_dir / \"backtest_results/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"trades.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe\n",
"\n",
"In case you did already some trading and want to analyze your performance"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from freqtrade.data.btanalysis import load_trades_from_db\n",
"\n",
"# Fetch trades from database\n",
"trades = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"trades.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plot results\n",
"\n",
"Freqtrade offers interactive plotting capabilities based on plotly."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from freqtrade.plot.plotting import generate_candlestick_graph\n",
"# Limit graph period to keep plotly quick and reactive\n",
"\n",
"data_red = data['2019-06-01':'2019-06-10']\n",
"# Generate candlestick graph\n",
"graph = generate_candlestick_graph(pair=pair,\n",
" data=data_red,\n",
" trades=trades,\n",
" indicators1=['sma20', 'ema50', 'ema55'],\n",
" indicators2=['rsi', 'macd', 'macdsignal', 'macdhist']\n",
" )\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Show graph inline\n",
"# graph.show()\n",
"\n",
"# Render graph in a seperate window\n",
"graph.show(renderer=\"browser\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
@ -161,7 +237,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
"version": "3.7.4"
},
"mimetype": "text/x-python",
"name": "python",
@ -212,5 +288,5 @@
"version": 3
},
"nbformat": 4,
"nbformat_minor": 2
"nbformat_minor": 4
}