diff --git a/.gitignore b/.gitignore index 9ed046c40..3a9df9852 100644 --- a/.gitignore +++ b/.gitignore @@ -81,7 +81,6 @@ target/ # Jupyter Notebook .ipynb_checkpoints -*.ipynb # pyenv .python-version @@ -93,3 +92,6 @@ target/ .pytest_cache/ .mypy_cache/ + +#exceptions +!user_data/noteboks/*example.ipynb diff --git a/docs/data-analysis.md b/docs/data-analysis.md index 2b6d6ed58..ecd94445b 100644 --- a/docs/data-analysis.md +++ b/docs/data-analysis.md @@ -1,164 +1,114 @@ # Analyzing bot data -After performing backtests, or after running the bot for some time, it will be interesting to analyze the results your bot generated. +You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation). -A good way for this is using Jupyter (notebook or lab) - which provides an interactive environment to analyze the data. +*Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)* -The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results. +## Example snippets -## Strategy development problem analysis - -Debugging a strategy (are there no buy signals, ...) can be very time-consuming. -FreqTrade tries to help you by exposing a few helper-functions, which can be very handy. - -It's recommended using Juptyer Notebooks for analysis, since it offers a dynamic way to rerun certain parts of the code. - -The following is a full code-snippet, which will be explained by both comments, and step by step below. +### Load backtest results into a pandas dataframe ```python -# Some necessary imports -from pathlib import Path - -from freqtrade.data.history import load_pair_history -from freqtrade.resolvers import StrategyResolver -# Define some constants -ticker_interval = "5m" - -# Name of the strategy class -strategyname = 'Awesomestrategy' -# Location of the strategy -strategy_location = '../xmatt/strategies' -# Location of the data -data_location = '../freqtrade/user_data/data/binance/' -# Only use one pair here -pair = "XRP_ETH" - -### End constants - -# Load data -bt_data = load_pair_history(datadir=Path(data_location), - ticker_interval = ticker_interval, - pair=pair) -print(len(bt_data)) - -### Start strategy reload -# Load strategy - best done in a new cell -# Rerun each time the strategy-file is changed. -strategy = StrategyResolver({'strategy': strategyname, - 'user_data_dir': Path.cwd(), - 'strategy_path': location}).strategy - -# Run strategy (just like in backtesting) -df = strategy.analyze_ticker(bt_data, {'pair': pair}) -print(f"Generated {df['buy'].sum()} buy signals") - -# Reindex data to be "nicer" and show data -data = df.set_index('date', drop=True) -data.tail() - -``` - -### Explanation - -#### Imports and constant definition - -``` python -# Some necessary imports -from pathlib import Path - -from freqtrade.data.history import load_pair_history -from freqtrade.resolvers import StrategyResolver -# Define some constants -ticker_interval = "5m" - -# Name of the strategy class -strategyname = 'Awesomestrategy' -# Location of the strategy -strategy_location = 'user_data/strategies' -# Location of the data -data_location = 'user_data/data/binance' -# Only use one pair here -pair = "XRP_ETH" -``` - -This first section imports necessary modules, and defines some constants you'll probably need to adjust for your case. - -#### Load candles - -``` python -# Load data -bt_data = load_pair_history(datadir=Path(data_location), - ticker_interval = ticker_interval, - pair=pair) -print(len(bt_data)) -``` - -This second section loads the historic data and prints the amount of candles in the DataFrame. -You can also inspect this dataframe by using `bt_data.head()` or `bt_data.tail()`. - -#### Run strategy and analyze results - -Now, it's time to load and run your strategy. -For this, I recommend using a new cell in your notebook, since you'll want to repeat this until you're satisfied with your strategy. - -``` python -# Load strategy - best done in a new cell -# Needs to be ran each time the strategy-file is changed. -strategy = StrategyResolver({'strategy': strategyname, - 'user_data_dir': Path.cwd(), - 'strategy_path': location}).strategy - -# Run strategy (just like in backtesting) -df = strategy.analyze_ticker(bt_data, {'pair': pair}) -print(f"Generated {df['buy'].sum()} buy signals") - -# Reindex data to be "nicer" and show data -data = df.set_index('date', drop=True) -data.tail() -``` - -The code snippet loads and analyzes the strategy, calculates and prints the number of buy signals. - -The last 2 lines serve to analyze the dataframe in detail. -This can be important if your strategy did not generate any buy signals. -Note that using `data.head()` would also work, however this is misleading since most indicators have some "startup" time at the start of a backtested dataframe. - -There can be many things wrong, some signs to look for are: - -* Columns with NaN values at the end of the dataframe -* Columns used in `crossed*()` functions with completely different units - -## Backtesting - -To analyze your backtest results, you can [export the trades](#exporting-trades-to-file). -You can then load the trades to perform further analysis. - -Freqtrade provides the `load_backtest_data()` helper function to easily load the backtest results, which takes the path to the the backtest-results file as parameter. - -``` python from freqtrade.data.btanalysis import load_backtest_data -df = load_backtest_data("user_data/backtest-result.json") +# Load backtest results +df = load_backtest_data("user_data/backtest_data/backtest-result.json") # Show value-counts per pair df.groupby("pair")["sell_reason"].value_counts() - ``` This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload. -If you have some ideas for interesting / helpful backtest data analysis ideas, please submit a Pull Request so the community can benefit from it. - -## Live data - -To analyze the trades your bot generated, you can load them to a DataFrame as follows: +### Load live trading results into a pandas dataframe ``` python from freqtrade.data.btanalysis import load_trades_from_db +# Fetch trades from database df = load_trades_from_db("sqlite:///tradesv3.sqlite") +# Display results df.groupby("pair")["sell_reason"].value_counts() +``` +## Strategy debugging example + +Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data. + +### Import requirements and define variables used in analyses + +```python +# Imports +from pathlib import Path +import os +from freqtrade.data.history import load_pair_history +from freqtrade.resolvers import StrategyResolver + +# You can override strategy settings as demonstrated below. +# Customize these according to your needs. + +# Define some constants +ticker_interval = "5m" +# Name of the strategy class +strategy_name = 'AwesomeStrategy' +# Path to user data +user_data_dir = 'user_data' +# Location of the strategy +strategy_location = Path(user_data_dir, 'strategies') +# Location of the data +data_location = Path(user_data_dir, 'data', 'binance') +# Pair to analyze +# Only use one pair here +pair = "BTC_USDT" +``` + +### Load exchange data + +```python +# Load data using values set above +bt_data = load_pair_history(datadir=Path(data_location), + ticker_interval=ticker_interval, + pair=pair) + +# Confirm success +print(f"Loaded {len(bt_data)} rows of data for {pair} from {data_location}") +``` + +### Load and run strategy + +* Rerun each time the strategy file is changed + +```python +# Load strategy using values set above +strategy = StrategyResolver({'strategy': strategy_name, + 'user_data_dir': user_data_dir, + 'strategy_path': strategy_location}).strategy + +# Generate buy/sell signals using strategy +df = strategy.analyze_ticker(bt_data, {'pair': pair}) +``` + +### Display the trade details + +* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe. + +#### Some possible problems + +* Columns with NaN values at the end of the dataframe +* Columns used in `crossed*()` functions with completely different units + +#### Comparison with full backtest + +having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting. + +Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29). +The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available. + +```python +# Report results +print(f"Generated {df['buy'].sum()} buy signals") +data = df.set_index('date', drop=True) +data.tail() ``` Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data. diff --git a/environment.yml b/environment.yml index 8f43b6991..cd3350fd5 100644 --- a/environment.yml +++ b/environment.yml @@ -37,6 +37,7 @@ dependencies: - coveralls - mypy # Useful for jupyter + - jupyter - ipykernel - isort - yapf diff --git a/setup.py b/setup.py index 202e3fd0d..41e1b8f45 100644 --- a/setup.py +++ b/setup.py @@ -25,7 +25,13 @@ develop = [ 'pytest-random-order', ] -all_extra = api + plot + develop +jupyter = [ + 'jupyter', + 'nbstripout', + 'ipykernel', + ] + +all_extra = api + plot + develop + jupyter setup(name='freqtrade', version=__version__, @@ -68,6 +74,8 @@ setup(name='freqtrade', 'dev': all_extra, 'plot': plot, 'all': all_extra, + 'jupyter': jupyter, + }, include_package_data=True, zip_safe=False, diff --git a/user_data/notebooks/.gitkeep b/user_data/notebooks/.gitkeep new file mode 100644 index 000000000..e69de29bb diff --git a/user_data/notebooks/analysis_example.ipynb b/user_data/notebooks/analysis_example.ipynb new file mode 100644 index 000000000..f5e2c12d7 --- /dev/null +++ b/user_data/notebooks/analysis_example.ipynb @@ -0,0 +1,243 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Analyzing bot data\n", + "\n", + "You can analyze the results of backtests and trading history easily using Jupyter notebooks. \n", + "**Copy this file so your changes don't get clobbered with the next freqtrade update!** \n", + "For usage instructions, see [jupyter.org](https://jupyter.org/documentation). \n", + "*Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Imports\n", + "from pathlib import Path\n", + "import os\n", + "from freqtrade.data.history import load_pair_history\n", + "from freqtrade.resolvers import StrategyResolver\n", + "from freqtrade.data.btanalysis import load_backtest_data\n", + "from freqtrade.data.btanalysis import load_trades_from_db" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Change directory\n", + "# Define all paths relative to the project root shown in the cell output\n", + "try:\n", + " os.chdir(Path(Path.cwd(), '../..'))\n", + " print(Path.cwd())\n", + "except:\n", + " pass" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example snippets" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load backtest results into a pandas dataframe" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Load backtest results\n", + "df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n", + "\n", + "# Show value-counts per pair\n", + "df.groupby(\"pair\")[\"sell_reason\"].value_counts()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load live trading results into a pandas dataframe" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Fetch trades from database\n", + "df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n", + "\n", + "# Display results\n", + "df.groupby(\"pair\")[\"sell_reason\"].value_counts()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Strategy debugging example\n", + "\n", + "Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Import requirements and define variables used in analyses" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Define some constants\n", + "ticker_interval = \"5m\"\n", + "# Name of the strategy class\n", + "strategy_name = 'AwesomeStrategy'\n", + "# Path to user data\n", + "user_data_dir = 'user_data'\n", + "# Location of the strategy\n", + "strategy_location = Path(user_data_dir, 'strategies')\n", + "# Location of the data\n", + "data_location = Path(user_data_dir, 'data', 'binance')\n", + "# Pair to analyze \n", + "# Only use one pair here\n", + "pair = \"BTC_USDT\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load exchange data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Load data using values set above\n", + "bt_data = load_pair_history(datadir=Path(data_location),\n", + " ticker_interval=ticker_interval,\n", + " pair=pair)\n", + "\n", + "# Confirm success\n", + "print(\"Loaded \" + str(len(bt_data)) + f\" rows of data for {pair} from {data_location}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Load and run strategy\n", + "* Rerun each time the strategy file is changed" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Load strategy using values set above\n", + "strategy = StrategyResolver({'strategy': strategy_name,\n", + " 'user_data_dir': user_data_dir,\n", + " 'strategy_path': strategy_location}).strategy\n", + "\n", + "# Generate buy/sell signals using strategy\n", + "df = strategy.analyze_ticker(bt_data, {'pair': pair})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Display the trade details\n", + "* Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n", + "\n", + "#### Some possible problems\n", + "\n", + "* Columns with NaN values at the end of the dataframe\n", + "* Columns used in `crossed*()` functions with completely different units\n", + "\n", + "#### Comparison with full backtest\n", + "\n", + "having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.\n", + "\n", + "Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple \"buy\" signals for each pair in sequence (until rsi returns > 29).\n", + "The bot will only buy on the first of these signals (and also only if a trade-slot (\"max_open_trades\") is still available), or on one of the middle signals, as soon as a \"slot\" becomes available.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Report results\n", + "print(f\"Generated {df['buy'].sum()} buy signals\")\n", + "data = df.set_index('date', drop=True)\n", + "data.tail()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data." + ] + } + ], + "metadata": { + "file_extension": ".py", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.3" + }, + "mimetype": "text/x-python", + "name": "python", + "npconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": 3 + }, + "nbformat": 4, + "nbformat_minor": 2 +}