adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

edits to clarify backtesting analysis

Browse Source
This commit is contained in:
Jonathan Raviotta 2019-08-08 22:09:15 -04:00
parent 2bc67b4a96
commit ccf3c69874
3 changed files with 176 additions and 101 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

70
docs/data-analysis.md
View File

@ -2,11 +2,33 @@
You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation). You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation).
## Example snippets
### Load backtest results into a pandas dataframe
```python
# Load backtest results
df = load_backtest_data("user_data/backtest_data/backtest-result.json")
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
### Load live trading results into a pandas dataframe
``` python
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts()
```
## Strategy debugging example ## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data. Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Import requirements and define variables used in the script ### Import requirements and define variables used in analyses
```python ```python
# Imports # Imports
@ -47,12 +69,6 @@ print("Loaded " + str(len(bt_data)) + f" rows of data for {pair} from {data_loca
### Load and run strategy ### Load and run strategy
* Rerun each time the strategy file is changed * Rerun each time the strategy file is changed
* Display the trade details. Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
Some possible problems:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
```python ```python
# Load strategy using values set above # Load strategy using values set above
@ -60,33 +76,31 @@ strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir, 'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy 'strategy_path': strategy_location}).strategy
# Run strategy (just like in backtesting) # Generate buy/sell signals using strategy
df = strategy.analyze_ticker(bt_data, {'pair': pair}) df = strategy.analyze_ticker(bt_data, {'pair': pair})
```
### Display the trade details
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
#### Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
#### Comparison with full backtest
having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29).
The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results # Report results
print(f"Generated {df['buy'].sum()} buy signals") print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True) data = df.set_index('date', drop=True)
data.tail() data.tail()
``` ```
### Load backtest results into a pandas dataframe
```python
# Load backtest results
df = load_backtest_data("user_data/backtest_data/backtest-result.json")
# Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts()
```
### Load live trading results into a pandas dataframe
``` python
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts()
```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data. Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

12
setup.py
View File

@ -25,7 +25,15 @@ develop = [
'pytest-random-order', 'pytest-random-order',
] ]
all_extra = api + plot + develop jupyter = [
'jupyter',
'nbstripout',
'ipykernel',
'isort',
'yapf',
]
all_extra = api + plot + develop + jupyter
setup(name='freqtrade', setup(name='freqtrade',
version=__version__, version=__version__,
@ -68,7 +76,7 @@ setup(name='freqtrade',
'dev': all_extra, 'dev': all_extra,
'plot': plot, 'plot': plot,
'all': all_extra, 'all': all_extra,
'jupyter': [], 'jupyter': jupyter,
}, },
include_package_data=True, include_package_data=True,

195
user_data/notebooks/analysis_example.ipynb
View File

@ -4,31 +4,9 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Strategy debugging example" "# Analyzing bot data\n",
] "\n",
}, "You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation)."
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Change directory\n",
"# Define all paths relative to the project root shown in the cell output\n",
"import os\n",
"from pathlib import Path\n",
"try:\n",
"\tos.chdir(Path(os.getcwd(), '../..'))\n",
"\tprint(os.getcwd())\n",
"except:\n",
"\tpass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in the script"
] ]
}, },
{ {
@ -39,11 +17,97 @@
"source": [ "source": [
"# Imports\n", "# Imports\n",
"from pathlib import Path\n", "from pathlib import Path\n",
"import os\n",
"from freqtrade.data.history import load_pair_history\n", "from freqtrade.data.history import load_pair_history\n",
"from freqtrade.resolvers import StrategyResolver\n", "from freqtrade.resolvers import StrategyResolver\n",
"from freqtrade.data.btanalysis import load_backtest_data\n", "from freqtrade.data.btanalysis import load_backtest_data\n",
"from freqtrade.data.btanalysis import load_trades_from_db\n", "from freqtrade.data.btanalysis import load_trades_from_db"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Change directory\n",
"# Define all paths relative to the project root shown in the cell output\n",
"try:\n",
"\tos.chdir(Path(Path.cwd(), '../..'))\n",
"\tprint(Path.cwd())\n",
"except:\n",
"\tpass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example snippets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load backtest results\n",
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n", "\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example\n",
"\n",
"Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in analyses"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define some constants\n", "# Define some constants\n",
"ticker_interval = \"1m\"\n", "ticker_interval = \"1m\"\n",
"# Name of the strategy class\n", "# Name of the strategy class\n",
@ -51,9 +115,9 @@
"# Path to user data\n", "# Path to user data\n",
"user_data_dir = 'user_data'\n", "user_data_dir = 'user_data'\n",
"# Location of the strategy\n", "# Location of the strategy\n",
"strategy_location = Path(user_data_dir, 'strategies')\n", "strategy_location = os.path.join(user_data_dir, 'strategies')\n",
"# Location of the data\n", "# Location of the data\n",
"data_location = Path(user_data_dir, 'data', 'binance')\n", "data_location = os.path.join(user_data_dir, 'data', 'binance')\n",
"# Pair to analyze \n", "# Pair to analyze \n",
"# Only use one pair here\n", "# Only use one pair here\n",
"pair = \"BTC_USDT\"" "pair = \"BTC_USDT\""
@ -85,15 +149,8 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### Load and run strategy \n", "### Load and run strategy\n",
"\n", "* Rerun each time the strategy file is changed"
"* Rerun each time the strategy file is changed\n",
"* Display the trade details. Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n",
"Some possible problems:\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units"
] ]
}, },
{ {
@ -107,53 +164,49 @@
" 'user_data_dir': user_data_dir,\n", " 'user_data_dir': user_data_dir,\n",
" 'strategy_path': strategy_location}).strategy\n", " 'strategy_path': strategy_location}).strategy\n",
"\n", "\n",
"# Run strategy (just like in backtesting)\n", "# Generate buy/sell signals using strategy\n",
"df = strategy.analyze_ticker(bt_data, {'pair': pair})\n", "df = strategy.analyze_ticker(bt_data, {'pair': pair})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Display the trade details\n",
"* Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n", "\n",
"#### Some possible problems\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units\n",
"\n",
"#### Comparison with full backtest\n",
"\n",
"having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.\n",
"\n",
"Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple \"buy\" signals for each pair in sequence (until rsi returns > 29).\n",
"The bot will only buy on the first of these signals (and also only if a trade-slot (\"max_open_trades\") is still available), or on one of the middle signals, as soon as a \"slot\" becomes available.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Report results\n", "# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n", "print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n", "data = df.set_index('date', drop=True)\n",
"data.tail()" "data.tail()"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": null, "execution_count": null,
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"# Load backtest results\n", "Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data."
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
] ]
} }
], ],