adelorenzo/stable
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'develop' into align_userdata

Browse Source
This commit is contained in:
Matthias 2019-08-10 20:15:07 +02:00
commit 2c5a499a8b
33 changed files with 1007 additions and 368 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@ -81,7 +81,6 @@ target/
# Jupyter Notebook # Jupyter Notebook
.ipynb_checkpoints .ipynb_checkpoints
*.ipynb
# pyenv # pyenv
.python-version .python-version
@ -93,3 +92,6 @@ target/
.pytest_cache/ .pytest_cache/
.mypy_cache/ .mypy_cache/
#exceptions
!user_data/noteboks/*example.ipynb

14
.travis.yml
View File

@ -10,15 +10,11 @@ services:
env: env:
global: global:
- IMAGE_NAME=freqtradeorg/freqtrade - IMAGE_NAME=freqtradeorg/freqtrade
addons:
apt:
packages:
- libelf-dev
- libdw-dev
- binutils-dev
install: install:
- cd build_helpers && ./install_ta-lib.sh; cd .. - cd build_helpers && ./install_ta-lib.sh ${HOME}/dependencies/; cd ..
- export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH - export LD_LIBRARY_PATH=${HOME}/dependencies/lib:$LD_LIBRARY_PATH
- export TA_LIBRARY_PATH=${HOME}/dependencies/lib
- export TA_INCLUDE_PATH=${HOME}/dependencies/lib/include
- pip install -r requirements-dev.txt - pip install -r requirements-dev.txt
- pip install -e . - pip install -e .
jobs: jobs:
@ -55,4 +51,4 @@ notifications:
cache: cache:
pip: True pip: True
directories: directories:
- /usr/local/lib/ - $HOME/dependencies

10
build_helpers/install_ta-lib.sh
View File

@ -1,8 +1,14 @@
if [ ! -f "/usr/local/lib/libta_lib.a" ]; then if [ -z "$1" ]; then
INSTALL_LOC=/usr/local
else
INSTALL_LOC=${1}
fi
echo "Installing to ${INSTALL_LOC}"
if [ ! -f "${INSTALL_LOC}/lib/libta_lib.a" ]; then
tar zxvf ta-lib-0.4.0-src.tar.gz tar zxvf ta-lib-0.4.0-src.tar.gz
cd ta-lib \ cd ta-lib \
&& sed -i.bak "s|0.00000001|0.000000000000000001 |g" src/ta_func/ta_utility.h \ && sed -i.bak "s|0.00000001|0.000000000000000001 |g" src/ta_func/ta_utility.h \
&& ./configure \ && ./configure --prefix=${INSTALL_LOC}/ \
&& make \ && make \
&& which sudo && sudo make install || make install \ && which sudo && sudo make install || make install \
&& cd .. && cd ..

10
docs/backtesting.md
View File

@ -57,7 +57,15 @@ freqtrade backtesting --datadir freqtrade/tests/testdata-20180101
freqtrade -s TestStrategy backtesting freqtrade -s TestStrategy backtesting
``` ```
Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory Where `-s TestStrategy` refers to the class name within the strategy file `test_strategy.py` found in the `freqtrade/user_data/strategies` directory.
#### Comparing multiple Strategies
```bash
freqtrade backtesting --strategy-list TestStrategy1 AwesomeStrategy --ticker-interval 5m
```
Where `TestStrategy1` and `AwesomeStrategy` refer to class names of strategies.
#### Exporting trades to file #### Exporting trades to file

2
docs/bot-usage.md
View File

@ -193,7 +193,7 @@ optional arguments:
number). number).
-l, --live Use live data. -l, --live Use live data.
--strategy-list STRATEGY_LIST [STRATEGY_LIST ...] --strategy-list STRATEGY_LIST [STRATEGY_LIST ...]
Provide a commaseparated list of strategies to Provide a space-separated list of strategies to
backtest Please note that ticker-interval needs to be backtest Please note that ticker-interval needs to be
set either in config or via command line. When using set either in config or via command line. When using
this together with --export trades, the strategy-name this together with --export trades, the strategy-name

224
docs/data-analysis.md
View File

@ -1,164 +1,114 @@
# Analyzing bot data # Analyzing bot data
After performing backtests, or after running the bot for some time, it will be interesting to analyze the results your bot generated. You can analyze the results of backtests and trading history easily using Jupyter notebooks. A sample notebook is located at `user_data/notebooks/analysis_example.ipynb`. For usage instructions, see [jupyter.org](https://jupyter.org/documentation).
A good way for this is using Jupyter (notebook or lab) - which provides an interactive environment to analyze the data. *Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*
The following helpers will help you loading the data into Pandas DataFrames, and may also give you some starting points in analyzing the results. ## Example snippets
## Strategy development problem analysis ### Load backtest results into a pandas dataframe
Debugging a strategy (are there no buy signals, ...) can be very time-consuming.
FreqTrade tries to help you by exposing a few helper-functions, which can be very handy.
It's recommended using Juptyer Notebooks for analysis, since it offers a dynamic way to rerun certain parts of the code.
The following is a full code-snippet, which will be explained by both comments, and step by step below.
```python ```python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = '../xmatt/strategies'
# Location of the data
data_location = '../freqtrade/user_data/data/binance/'
# Only use one pair here
pair = "XRP_ETH"
### End constants
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
### Start strategy reload
# Load strategy - best done in a new cell
# Rerun each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
### Explanation
#### Imports and constant definition
``` python
# Some necessary imports
from pathlib import Path
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategyname = 'Awesomestrategy'
# Location of the strategy
strategy_location = 'user_data/strategies'
# Location of the data
data_location = 'user_data/data/binance'
# Only use one pair here
pair = "XRP_ETH"
```
This first section imports necessary modules, and defines some constants you'll probably need to adjust for your case.
#### Load candles
``` python
# Load data
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval = ticker_interval,
pair=pair)
print(len(bt_data))
```
This second section loads the historic data and prints the amount of candles in the DataFrame.
You can also inspect this dataframe by using `bt_data.head()` or `bt_data.tail()`.
#### Run strategy and analyze results
Now, it's time to load and run your strategy.
For this, I recommend using a new cell in your notebook, since you'll want to repeat this until you're satisfied with your strategy.
``` python
# Load strategy - best done in a new cell
# Needs to be ran each time the strategy-file is changed.
strategy = StrategyResolver({'strategy': strategyname,
'user_data_dir': Path.cwd(),
'strategy_path': location}).strategy
# Run strategy (just like in backtesting)
df = strategy.analyze_ticker(bt_data, {'pair': pair})
print(f"Generated {df['buy'].sum()} buy signals")
# Reindex data to be "nicer" and show data
data = df.set_index('date', drop=True)
data.tail()
```
The code snippet loads and analyzes the strategy, calculates and prints the number of buy signals.
The last 2 lines serve to analyze the dataframe in detail.
This can be important if your strategy did not generate any buy signals.
Note that using `data.head()` would also work, however this is misleading since most indicators have some "startup" time at the start of a backtested dataframe.
There can be many things wrong, some signs to look for are:
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
## Backtesting
To analyze your backtest results, you can [export the trades](#exporting-trades-to-file).
You can then load the trades to perform further analysis.
Freqtrade provides the `load_backtest_data()` helper function to easily load the backtest results, which takes the path to the the backtest-results file as parameter.
``` python
from freqtrade.data.btanalysis import load_backtest_data from freqtrade.data.btanalysis import load_backtest_data
# Load backtest results
df = load_backtest_data("user_data/backtest_results/backtest-result.json") df = load_backtest_data("user_data/backtest_results/backtest-result.json")
# Show value-counts per pair # Show value-counts per pair
df.groupby("pair")["sell_reason"].value_counts() df.groupby("pair")["sell_reason"].value_counts()
``` ```
This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload. This will allow you to drill deeper into your backtest results, and perform analysis which otherwise would make the regular backtest-output very difficult to digest due to information overload.
If you have some ideas for interesting / helpful backtest data analysis ideas, please submit a Pull Request so the community can benefit from it. ### Load live trading results into a pandas dataframe
## Live data
To analyze the trades your bot generated, you can load them to a DataFrame as follows:
``` python ``` python
from freqtrade.data.btanalysis import load_trades_from_db from freqtrade.data.btanalysis import load_trades_from_db
# Fetch trades from database
df = load_trades_from_db("sqlite:///tradesv3.sqlite") df = load_trades_from_db("sqlite:///tradesv3.sqlite")
# Display results
df.groupby("pair")["sell_reason"].value_counts() df.groupby("pair")["sell_reason"].value_counts()
```
## Strategy debugging example
Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.
### Import requirements and define variables used in analyses
```python
# Imports
from pathlib import Path
import os
from freqtrade.data.history import load_pair_history
from freqtrade.resolvers import StrategyResolver
# You can override strategy settings as demonstrated below.
# Customize these according to your needs.
# Define some constants
ticker_interval = "5m"
# Name of the strategy class
strategy_name = 'AwesomeStrategy'
# Path to user data
user_data_dir = 'user_data'
# Location of the strategy
strategy_location = Path(user_data_dir, 'strategies')
# Location of the data
data_location = Path(user_data_dir, 'data', 'binance')
# Pair to analyze
# Only use one pair here
pair = "BTC_USDT"
```
### Load exchange data
```python
# Load data using values set above
bt_data = load_pair_history(datadir=Path(data_location),
ticker_interval=ticker_interval,
pair=pair)
# Confirm success
print(f"Loaded {len(bt_data)} rows of data for {pair} from {data_location}")
```
### Load and run strategy
* Rerun each time the strategy file is changed
```python
# Load strategy using values set above
strategy = StrategyResolver({'strategy': strategy_name,
'user_data_dir': user_data_dir,
'strategy_path': strategy_location}).strategy
# Generate buy/sell signals using strategy
df = strategy.analyze_ticker(bt_data, {'pair': pair})
```
### Display the trade details
* Note that using `data.head()` would also work, however most indicators have some "startup" data at the top of the dataframe.
#### Some possible problems
* Columns with NaN values at the end of the dataframe
* Columns used in `crossed*()` functions with completely different units
#### Comparison with full backtest
having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.
Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple "buy" signals for each pair in sequence (until rsi returns > 29).
The bot will only buy on the first of these signals (and also only if a trade-slot ("max_open_trades") is still available), or on one of the middle signals, as soon as a "slot" becomes available.
```python
# Report results
print(f"Generated {df['buy'].sum()} buy signals")
data = df.set_index('date', drop=True)
data.tail()
``` ```
Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data. Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data.

56
docs/hyperopt.md
View File

@ -18,19 +18,24 @@ Configuring hyperopt is similar to writing your own strategy, and many tasks wil
### Checklist on all tasks / possibilities in hyperopt ### Checklist on all tasks / possibilities in hyperopt
Depending on the space you want to optimize, only some of the below are required. Depending on the space you want to optimize, only some of the below are required:
* fill `populate_indicators` - probably a copy from your strategy * fill `populate_indicators` - probably a copy from your strategy
* fill `buy_strategy_generator` - for buy signal optimization * fill `buy_strategy_generator` - for buy signal optimization
* fill `indicator_space` - for buy signal optimzation * fill `indicator_space` - for buy signal optimzation
* fill `sell_strategy_generator` - for sell signal optimization * fill `sell_strategy_generator` - for sell signal optimization
* fill `sell_indicator_space` - for sell signal optimzation * fill `sell_indicator_space` - for sell signal optimzation
* fill `roi_space` - for ROI optimization
* fill `generate_roi_table` - for ROI optimization (if you need more than 3 entries) Optional, but recommended:
* fill `stoploss_space` - stoploss optimization
* Optional but recommended * copy `populate_buy_trend` from your strategy - otherwise default-strategy will be used
* copy `populate_buy_trend` from your strategy - otherwise default-strategy will be used * copy `populate_sell_trend` from your strategy - otherwise default-strategy will be used
* copy `populate_sell_trend` from your strategy - otherwise default-strategy will be used
Rarely you may also need to override:
* `roi_space` - for custom ROI optimization (if you need the ranges for the ROI parameters in the optimization hyperspace that differ from default)
* `generate_roi_table` - for custom ROI optimization (if you need more than 4 entries in the ROI table)
* `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
### 1. Install a Custom Hyperopt File ### 1. Install a Custom Hyperopt File
@ -345,7 +350,7 @@ def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
### Understand Hyperopt ROI results ### Understand Hyperopt ROI results
If you are optimizing ROI, you're result will look as follows and include a ROI table. If you are optimizing ROI (i.e. if optimization search-space contains 'all' or 'roi'), your result will look as follows and include a ROI table:
``` ```
Best result: Best result:
@ -376,6 +381,41 @@ minimal_roi = {
} }
``` ```
If you are optimizing ROI, Freqtrade creates the 'roi' optimization hyperspace for you -- it's the hyperspace of components for the ROI tables. By default, each ROI table generated by the Freqtrade consists of 4 rows (steps) with the values that can vary in the following ranges:
| # | minutes | ROI percentage |
|---|---|---|
| 1 | always 0 | 0.03...0.31 |
| 2 | 10...40 | 0.02...0.11 |
| 3 | 20...100 | 0.01...0.04 |
| 4 | 30...220 | always 0 |
This structure of the ROI table is sufficient in most cases. Override the `roi_space()` method defining the ranges desired if you need components of the ROI tables to vary in other ranges.
Override the `generate_roi_table()` and `roi_space()` methods and implement your own custom approach for generation of the ROI tables during hyperoptimization in these methods if you need a different structure of the ROI table or other amount of rows (steps) in the ROI tables.
### Understand Hyperopt Stoploss results
If you are optimizing stoploss values (i.e. if optimization search-space contains 'all' or 'stoploss'), your result will look as follows and include stoploss:
```
Best result:
44/100: 135 trades. Avg profit 0.57%. Total profit 0.03871918 BTC (0.7722Σ%). Avg duration 180.4 mins. Objective: 1.94367
Buy hyperspace params:
{ 'adx-value': 44,
'rsi-value': 29,
'adx-enabled': False,
'rsi-enabled': True,
'trigger': 'bb_lower'}
Stoploss: -0.37996664668703606
```
If you are optimizing stoploss values, Freqtrade creates the 'stoploss' optimization hyperspace for you. By default, the stoploss values in that hyperspace can vary in the range -0.5...-0.02, which is sufficient in most cases.
Override the `stoploss_space()` method and define the desired range in it if you need stoploss values to vary in other range during hyperoptimization.
### Validate backtesting results ### Validate backtesting results
Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected. Once the optimized strategy has been implemented into your strategy, you should backtest this strategy to make sure everything is working as expected.

11
docs/installation.md
View File

@ -219,6 +219,17 @@ as the watchdog.
------ ------
## Using Conda
Freqtrade can also be installed using Anaconda (or Miniconda).
``` bash
conda env create -f environment.yml
```
!!! Note:
This requires the [ta-lib](#1-install-ta-lib) C-library to be installed first.
## Windows ## Windows
We recommend that Windows users use [Docker](docker.md) as this will work much easier and smoother (also more secure). We recommend that Windows users use [Docker](docker.md) as this will work much easier and smoother (also more secure).

2
docs/requirements-docs.txt
View File

@ -1 +1 @@
mkdocs-material==3.1.0 mkdocs-material==4.4.0

59
environment.yml Normal file
View File

@ -0,0 +1,59 @@
name: freqtrade
channels:
- defaults
- conda-forge
dependencies:
# Required for app
- python>=3.6
- pip
- wheel
- numpy
- pandas
- scipy
- SQLAlchemy
- scikit-learn
- arrow
- requests
- urllib3
- wrapt
- joblib
- jsonschema
- tabulate
- python-rapidjson
- filelock
- flask
- python-dotenv
- cachetools
- scikit-optimize
- python-telegram-bot
# Optional for plotting
- plotly
# Optional for development
- flake8
- pytest
- pytest-mock
- pytest-asyncio
- pytest-cov
- coveralls
- mypy
# Useful for jupyter
- jupyter
- ipykernel
- isort
- yapf
- pip:
# Required for app
- cython
- coinmarketcap
- ccxt
- TA-Lib
- py_find_1st
- sdnotify
# Optional for develpment
- flake8-tidy-imports
- flake8-type-annotations
- pytest-random-order
- -e .

2
freqtrade/configuration/cli_options.py
View File

@ -135,7 +135,7 @@ AVAILABLE_CLI_OPTIONS = {
), ),
"strategy_list": Arg( "strategy_list": Arg(
'--strategy-list', '--strategy-list',
help='Provide a comma-separated list of strategies to backtest. ' help='Provide a space-separated list of strategies to backtest. '
'Please note that ticker-interval needs to be set either in config ' 'Please note that ticker-interval needs to be set either in config '
'or via command line. When using this together with `--export trades`, ' 'or via command line. When using this together with `--export trades`, '
'the strategy-name is injected into the filename ' 'the strategy-name is injected into the filename '

22
freqtrade/configuration/configuration.py
View File

@ -1,9 +1,7 @@
""" """
This module contains the configuration class This module contains the configuration class
""" """
import json
import logging import logging
import sys
import warnings import warnings
from argparse import Namespace from argparse import Namespace
from pathlib import Path from pathlib import Path
@ -13,6 +11,7 @@ from freqtrade import OperationalException, constants
from freqtrade.configuration.check_exchange import check_exchange from freqtrade.configuration.check_exchange import check_exchange
from freqtrade.configuration.directory_operations import create_datadir, create_userdata_dir from freqtrade.configuration.directory_operations import create_datadir, create_userdata_dir
from freqtrade.configuration.json_schema import validate_config_schema from freqtrade.configuration.json_schema import validate_config_schema
from freqtrade.configuration.load_config import load_config_file
from freqtrade.loggers import setup_logging from freqtrade.loggers import setup_logging
from freqtrade.misc import deep_merge_dicts from freqtrade.misc import deep_merge_dicts
from freqtrade.state import RunMode from freqtrade.state import RunMode
@ -53,24 +52,7 @@ class Configuration(object):
logger.info('Using config: %s ...', path) logger.info('Using config: %s ...', path)
# Merge config options, overwriting old values # Merge config options, overwriting old values
config = deep_merge_dicts(self._load_config_file(path), config) config = deep_merge_dicts(load_config_file(path), config)
return config
def _load_config_file(self, path: str) -> Dict[str, Any]:
"""
Loads a config file from the given path
:param path: path as str
:return: configuration as dictionary
"""
try:
# Read config from stdin if requested in the options
with open(path) if path != '-' else sys.stdin as file:
config = json.load(file)
except FileNotFoundError:
raise OperationalException(
f'Config file "{path}" not found!'
' Please create a config file or check whether it exists.')
return config return config

30
freqtrade/configuration/load_config.py Normal file
View File

@ -0,0 +1,30 @@
"""
This module contain functions to load the configuration file
"""
import json
import logging
import sys
from typing import Any, Dict
from freqtrade import OperationalException
logger = logging.getLogger(__name__)
def load_config_file(path: str) -> Dict[str, Any]:
"""
Loads a config file from the given path
:param path: path as str
:return: configuration as dictionary
"""
try:
# Read config from stdin if requested in the options
with open(path) if path != '-' else sys.stdin as file:
config = json.load(file)
except FileNotFoundError:
raise OperationalException(
f'Config file "{path}" not found!'
' Please create a config file or check whether it exists.')
return config

3
freqtrade/exchange/exchange.py
View File

@ -725,7 +725,8 @@ class Exchange(object):
return [] return []
try: try:
# Allow 5s offset to catch slight time offsets (discovered in #1185) # Allow 5s offset to catch slight time offsets (discovered in #1185)
my_trades = self._api.fetch_my_trades(pair, since.timestamp() - 5) # since needs to be int in milliseconds
my_trades = self._api.fetch_my_trades(pair, int((since.timestamp() - 5) * 1000))
matched_trades = [trade for trade in my_trades if trade['order'] == order_id] matched_trades = [trade for trade in my_trades if trade['order'] == order_id]
return matched_trades return matched_trades

10
freqtrade/optimize/backtesting.py
View File

@ -10,8 +10,8 @@ from pathlib import Path
from typing import Any, Dict, List, NamedTuple, Optional from typing import Any, Dict, List, NamedTuple, Optional
from pandas import DataFrame from pandas import DataFrame
from tabulate import tabulate
from freqtrade import OperationalException
from freqtrade.configuration import Arguments from freqtrade.configuration import Arguments
from freqtrade.data import history from freqtrade.data import history
from freqtrade.data.dataprovider import DataProvider from freqtrade.data.dataprovider import DataProvider
@ -21,6 +21,7 @@ from freqtrade.persistence import Trade
from freqtrade.resolvers import ExchangeResolver, StrategyResolver from freqtrade.resolvers import ExchangeResolver, StrategyResolver
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.strategy.interface import IStrategy, SellType from freqtrade.strategy.interface import IStrategy, SellType
from tabulate import tabulate
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@ -88,6 +89,9 @@ class Backtesting(object):
Load strategy into backtesting Load strategy into backtesting
""" """
self.strategy = strategy self.strategy = strategy
if "ticker_interval" not in self.config:
raise OperationalException("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`")
self.ticker_interval = self.config.get('ticker_interval') self.ticker_interval = self.config.get('ticker_interval')
self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval) self.ticker_interval_mins = timeframe_to_minutes(self.ticker_interval)
@ -373,7 +377,9 @@ class Backtesting(object):
continue continue
trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1 trade_count_lock[row.date] = trade_count_lock.get(row.date, 0) + 1
trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][indexes[pair]:], # since indexes has been incremented before, we need to go one step back to
# also check the buying candle for sell conditions.
trade_entry = self._get_sell_trade_entry(pair, row, ticker[pair][indexes[pair]-1:],
trade_count_lock, stake_amount, trade_count_lock, stake_amount,
max_open_trades) max_open_trades)

41
freqtrade/optimize/default_hyperopt.py
View File

@ -5,7 +5,7 @@ from typing import Any, Callable, Dict, List
import talib.abstract as ta import talib.abstract as ta
from pandas import DataFrame from pandas import DataFrame
from skopt.space import Categorical, Dimension, Integer, Real from skopt.space import Categorical, Dimension, Integer
import freqtrade.vendor.qtpylib.indicators as qtpylib import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.optimize.hyperopt_interface import IHyperOpt from freqtrade.optimize.hyperopt_interface import IHyperOpt
@ -13,10 +13,9 @@ from freqtrade.optimize.hyperopt_interface import IHyperOpt
class DefaultHyperOpts(IHyperOpt): class DefaultHyperOpts(IHyperOpt):
""" """
Default hyperopt provided by freqtrade bot. Default hyperopt provided by the Freqtrade bot.
You can override it with your own hyperopt You can override it with your own hyperopt
""" """
@staticmethod @staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe) dataframe['adx'] = ta.ADX(dataframe)
@ -156,42 +155,6 @@ class DefaultHyperOpts(IHyperOpt):
'sell-sar_reversal'], name='sell-trigger') 'sell-sar_reversal'], name='sell-trigger')
] ]
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss Value to search
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
""" """
Based on TA indicators. Should be a copy of from strategy Based on TA indicators. Should be a copy of from strategy

52
freqtrade/optimize/hyperopt_interface.py
View File

@ -7,7 +7,7 @@ from abc import ABC, abstractmethod
from typing import Dict, Any, Callable, List from typing import Dict, Any, Callable, List
from pandas import DataFrame from pandas import DataFrame
from skopt.space import Dimension from skopt.space import Dimension, Integer, Real
class IHyperOpt(ABC): class IHyperOpt(ABC):
@ -26,56 +26,80 @@ class IHyperOpt(ABC):
@abstractmethod @abstractmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
""" """
Populate indicators that will be used in the Buy and Sell strategy Populate indicators that will be used in the Buy and Sell strategy.
:param dataframe: Raw data from the exchange and parsed by parse_ticker_dataframe() :param dataframe: Raw data from the exchange and parsed by parse_ticker_dataframe().
:return: a Dataframe with all mandatory indicators for the strategies :return: A Dataframe with all mandatory indicators for the strategies.
""" """
@staticmethod @staticmethod
@abstractmethod @abstractmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable: def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a buy strategy generator Create a buy strategy generator.
""" """
@staticmethod @staticmethod
@abstractmethod @abstractmethod
def sell_strategy_generator(params: Dict[str, Any]) -> Callable: def sell_strategy_generator(params: Dict[str, Any]) -> Callable:
""" """
Create a sell strategy generator Create a sell strategy generator.
""" """
@staticmethod @staticmethod
@abstractmethod @abstractmethod
def indicator_space() -> List[Dimension]: def indicator_space() -> List[Dimension]:
""" """
Create an indicator space Create an indicator space.
""" """
@staticmethod @staticmethod
@abstractmethod @abstractmethod
def sell_indicator_space() -> List[Dimension]: def sell_indicator_space() -> List[Dimension]:
""" """
Create a sell indicator space Create a sell indicator space.
""" """
@staticmethod @staticmethod
@abstractmethod
def generate_roi_table(params: Dict) -> Dict[int, float]: def generate_roi_table(params: Dict) -> Dict[int, float]:
""" """
Create an roi table Create a ROI table.
Generates the ROI table that will be used by Hyperopt.
You may override it in your custom Hyperopt class.
""" """
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod @staticmethod
@abstractmethod
def stoploss_space() -> List[Dimension]: def stoploss_space() -> List[Dimension]:
""" """
Create a stoploss space Create a stoploss space.
Defines range of stoploss values to search.
You may override it in your custom Hyperopt class.
""" """
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod @staticmethod
@abstractmethod
def roi_space() -> List[Dimension]: def roi_space() -> List[Dimension]:
""" """
Create a roi space Create a ROI space.
Defines values to search for each ROI steps.
You may override it in your custom Hyperopt class.
""" """
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]

2
freqtrade/optimize/hyperopt_loss_sharpe.py
View File

@ -39,7 +39,7 @@ class SharpeHyperOptLoss(IHyperOptLoss):
sharp_ratio = expected_yearly_return / np.std(total_profit) * np.sqrt(365) sharp_ratio = expected_yearly_return / np.std(total_profit) * np.sqrt(365)
else: else:
# Define high (negative) sharpe ratio to be clear that this is NOT optimal. # Define high (negative) sharpe ratio to be clear that this is NOT optimal.
sharp_ratio = 20. sharp_ratio = -20.
# print(expected_yearly_return, np.std(total_profit), sharp_ratio) # print(expected_yearly_return, np.std(total_profit), sharp_ratio)
return -sharp_ratio return -sharp_ratio

2
freqtrade/tests/conftest.py
View File

@ -45,7 +45,7 @@ def get_args(args):
def patched_configuration_load_config_file(mocker, config) -> None: def patched_configuration_load_config_file(mocker, config) -> None:
mocker.patch( mocker.patch(
'freqtrade.configuration.configuration.Configuration._load_config_file', 'freqtrade.configuration.configuration.load_config_file',
lambda *args, **kwargs: config lambda *args, **kwargs: config
) )

15
freqtrade/tests/exchange/test_exchange.py
View File

@ -2,7 +2,7 @@
# pragma pylint: disable=protected-access # pragma pylint: disable=protected-access
import copy import copy
import logging import logging
from datetime import datetime from datetime import datetime, timezone
from random import randint from random import randint
from unittest.mock import MagicMock, Mock, PropertyMock from unittest.mock import MagicMock, Mock, PropertyMock
@ -11,8 +11,8 @@ import ccxt
import pytest import pytest
from pandas import DataFrame from pandas import DataFrame
from freqtrade import (DependencyException, OperationalException, from freqtrade import (DependencyException, InvalidOrderException,
TemporaryError, InvalidOrderException) OperationalException, TemporaryError)
from freqtrade.exchange import Binance, Exchange, Kraken from freqtrade.exchange import Binance, Exchange, Kraken
from freqtrade.exchange.exchange import API_RETRY_COUNT from freqtrade.exchange.exchange import API_RETRY_COUNT
from freqtrade.resolvers.exchange_resolver import ExchangeResolver from freqtrade.resolvers.exchange_resolver import ExchangeResolver
@ -1361,7 +1361,7 @@ def test_name(default_conf, mocker, exchange_name):
@pytest.mark.parametrize("exchange_name", EXCHANGES) @pytest.mark.parametrize("exchange_name", EXCHANGES)
def test_get_trades_for_order(default_conf, mocker, exchange_name): def test_get_trades_for_order(default_conf, mocker, exchange_name):
order_id = 'ABCD-ABCD' order_id = 'ABCD-ABCD'
since = datetime(2018, 5, 5) since = datetime(2018, 5, 5, tzinfo=timezone.utc)
default_conf["dry_run"] = False default_conf["dry_run"] = False
mocker.patch('freqtrade.exchange.Exchange.exchange_has', return_value=True) mocker.patch('freqtrade.exchange.Exchange.exchange_has', return_value=True)
api_mock = MagicMock() api_mock = MagicMock()
@ -1391,6 +1391,13 @@ def test_get_trades_for_order(default_conf, mocker, exchange_name):
orders = exchange.get_trades_for_order(order_id, 'LTC/BTC', since) orders = exchange.get_trades_for_order(order_id, 'LTC/BTC', since)
assert len(orders) == 1 assert len(orders) == 1
assert orders[0]['price'] == 165 assert orders[0]['price'] == 165
assert api_mock.fetch_my_trades.call_count == 1
# since argument should be
assert isinstance(api_mock.fetch_my_trades.call_args[0][1], int)
assert api_mock.fetch_my_trades.call_args[0][0] == 'LTC/BTC'
# Same test twice, hardcoded number and doing the same calculation
assert api_mock.fetch_my_trades.call_args[0][1] == 1525478395000
assert api_mock.fetch_my_trades.call_args[0][1] == int(since.timestamp() - 5) * 1000
ccxt_exceptionhandlers(mocker, default_conf, api_mock, exchange_name, ccxt_exceptionhandlers(mocker, default_conf, api_mock, exchange_name,
'get_trades_for_order', 'fetch_my_trades', 'get_trades_for_order', 'fetch_my_trades',

110
freqtrade/tests/optimize/test_backtest_detail.py
View File

@ -14,9 +14,8 @@ from freqtrade.tests.optimize import (BTContainer, BTrade,
_get_frame_time_from_offset, _get_frame_time_from_offset,
tests_ticker_interval) tests_ticker_interval)
# Test 0 Sell signal sell # Test 0: Sell with signal sell in candle 3
# Test with Stop-loss at 1% # Test with Stop-loss at 1%
# TC0: Sell signal in candle 3
tc0 = BTContainer(data=[ tc0 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -29,9 +28,8 @@ tc0 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=4)] trades=[BTrade(sell_reason=SellType.SELL_SIGNAL, open_tick=1, close_tick=4)]
) )
# Test 1 Minus 8% Close # Test 1: Stop-Loss Triggered 1% loss
# Test with Stop-loss at 1% # Test with Stop-loss at 1%
# TC1: Stop-Loss Triggered 1% loss
tc1 = BTContainer(data=[ tc1 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -45,9 +43,8 @@ tc1 = BTContainer(data=[
) )
# Test 2 Minus 4% Low, minus 1% close # Test 2: Minus 4% Low, minus 1% close
# Test with Stop-Loss at 3% # Test with Stop-Loss at 3%
# TC2: Stop-Loss Triggered 3% Loss
tc2 = BTContainer(data=[ tc2 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -61,12 +58,12 @@ tc2 = BTContainer(data=[
) )
# Test 3 Candle drops 4%, Recovers 1%. # Test 3: Multiple trades.
# Entry Criteria Met # Candle drops 4%, Recovers 1%.
# Candle drops 20% # Entry Criteria Met
# Test with Stop-Loss at 2% # Candle drops 20%
# TC3: Trade-A: Stop-Loss Triggered 2% Loss # Trade-A: Stop-Loss Triggered 2% Loss
# Trade-B: Stop-Loss Triggered 2% Loss # Trade-B: Stop-Loss Triggered 2% Loss
tc3 = BTContainer(data=[ tc3 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -81,10 +78,10 @@ tc3 = BTContainer(data=[
BTrade(sell_reason=SellType.STOP_LOSS, open_tick=4, close_tick=5)] BTrade(sell_reason=SellType.STOP_LOSS, open_tick=4, close_tick=5)]
) )
# Test 4 Minus 3% / recovery +15% # Test 4: Minus 3% / recovery +15%
# Candle Data for test 3 – Candle drops 3% Closed 15% up # Candle Data for test 3 – Candle drops 3% Closed 15% up
# Test with Stop-loss at 2% ROI 6% # Test with Stop-loss at 2% ROI 6%
# TC4: Stop-Loss Triggered 2% Loss # Stop-Loss Triggered 2% Loss
tc4 = BTContainer(data=[ tc4 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -97,9 +94,8 @@ tc4 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)] trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)]
) )
# Test 5 / Drops 0.5% Closes +20% # Test 5: Drops 0.5% Closes +20%, ROI triggers 3% Gain
# Set stop-loss at 1% ROI 3% # stop-loss: 1%, ROI: 3%
# TC5: ROI triggers 3% Gain
tc5 = BTContainer(data=[ tc5 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4980, 4987, 6172, 1, 0], [0, 5000, 5025, 4980, 4987, 6172, 1, 0],
@ -112,9 +108,8 @@ tc5 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=3)] trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=3)]
) )
# Test 6 / Drops 3% / Recovers 6% Positive / Closes 1% positve # Test 6: Drops 3% / Recovers 6% Positive / Closes 1% positve, Stop-Loss triggers 2% Loss
# Set stop-loss at 2% ROI at 5% # stop-loss: 2% ROI: 5%
# TC6: Stop-Loss triggers 2% Loss
tc6 = BTContainer(data=[ tc6 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -127,9 +122,8 @@ tc6 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)] trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=2)]
) )
# Test 7 - 6% Positive / 1% Negative / Close 1% Positve # Test 7: 6% Positive / 1% Negative / Close 1% Positve, ROI Triggers 3% Gain
# Set stop-loss at 2% ROI at 3% # stop-loss: 2% ROI: 3%
# TC7: ROI Triggers 3% Gain
tc7 = BTContainer(data=[ tc7 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5025, 4975, 4987, 6172, 1, 0], [0, 5000, 5025, 4975, 4987, 6172, 1, 0],
@ -143,9 +137,8 @@ tc7 = BTContainer(data=[
) )
# Test 8 - trailing_stop should raise so candle 3 causes a stoploss. # Test 8: trailing_stop should raise so candle 3 causes a stoploss.
# Set stop-loss at 10%, ROI at 10% (should not apply) # stop-loss: 10%, ROI: 10% (should not apply), stoploss adjusted in candle 2
# TC8: Trailing stoploss - stoploss should be adjusted candle 2
tc8 = BTContainer(data=[ tc8 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0], [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@ -158,10 +151,8 @@ tc8 = BTContainer(data=[
) )
# Test 9 - trailing_stop should raise - high and low in same candle. # Test 9: trailing_stop should raise - high and low in same candle.
# Candle Data for test 9 # stop-loss: 10%, ROI: 10% (should not apply), stoploss adjusted in candle 3
# Set stop-loss at 10%, ROI at 10% (should not apply)
# TC9: Trailing stoploss - stoploss should be adjusted candle 3
tc9 = BTContainer(data=[ tc9 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0], [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@ -173,10 +164,9 @@ tc9 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)] trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
) )
# Test 10 - trailing_stop should raise so candle 3 causes a stoploss # Test 10: trailing_stop should raise so candle 3 causes a stoploss
# without applying trailing_stop_positive since stoploss_offset is at 10%. # without applying trailing_stop_positive since stoploss_offset is at 10%.
# Set stop-loss at 10%, ROI at 10% (should not apply) # stop-loss: 10%, ROI: 10% (should not apply), stoploss adjusted candle 2
# TC10: Trailing stoploss - stoploss should be adjusted candle 2
tc10 = BTContainer(data=[ tc10 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0], [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@ -190,10 +180,9 @@ tc10 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=4)] trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=4)]
) )
# Test 11 - trailing_stop should raise so candle 3 causes a stoploss # Test 11: trailing_stop should raise so candle 3 causes a stoploss
# applying a positive trailing stop of 3% since stop_positive_offset is reached. # applying a positive trailing stop of 3% since stop_positive_offset is reached.
# Set stop-loss at 10%, ROI at 10% (should not apply) # stop-loss: 10%, ROI: 10% (should not apply), stoploss adjusted candle 2
# TC11: Trailing stoploss - stoploss should be adjusted candle 2,
tc11 = BTContainer(data=[ tc11 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0], [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@ -207,10 +196,9 @@ tc11 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)] trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=3)]
) )
# Test 12 - trailing_stop should raise in candle 2 and cause a stoploss in the same candle # Test 12: trailing_stop should raise in candle 2 and cause a stoploss in the same candle
# applying a positive trailing stop of 3% since stop_positive_offset is reached. # applying a positive trailing stop of 3% since stop_positive_offset is reached.
# Set stop-loss at 10%, ROI at 10% (should not apply) # stop-loss: 10%, ROI: 10% (should not apply), stoploss adjusted candle 2
# TC12: Trailing stoploss - stoploss should be adjusted candle 2,
tc12 = BTContainer(data=[ tc12 = BTContainer(data=[
# D O H L C V B S # D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0], [0, 5000, 5050, 4950, 5000, 6172, 1, 0],
@ -224,6 +212,47 @@ tc12 = BTContainer(data=[
trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=2)] trades=[BTrade(sell_reason=SellType.TRAILING_STOP_LOSS, open_tick=1, close_tick=2)]
) )
# Test 13: Buy and sell ROI on same candle
# stop-loss: 10% (should not apply), ROI: 1%
tc13 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0],
[1, 5000, 5100, 4950, 5100, 6172, 0, 0],
[2, 5100, 5251, 4850, 5100, 6172, 0, 0],
[3, 4850, 5050, 4850, 4750, 6172, 0, 0],
[4, 4750, 4950, 4850, 4750, 6172, 0, 0]],
stop_loss=-0.10, roi=0.01, profit_perc=0.01,
trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=1)]
)
# Test 14 - Buy and Stoploss on same candle
# stop-loss: 5%, ROI: 10% (should not apply)
tc14 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0],
[1, 5000, 5100, 4600, 5100, 6172, 0, 0],
[2, 5100, 5251, 4850, 5100, 6172, 0, 0],
[3, 4850, 5050, 4850, 4750, 6172, 0, 0],
[4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
stop_loss=-0.05, roi=0.10, profit_perc=-0.05,
trades=[BTrade(sell_reason=SellType.STOP_LOSS, open_tick=1, close_tick=1)]
)
# Test 15 - Buy and ROI on same candle, followed by buy and Stoploss on next candle
# stop-loss: 5%, ROI: 10% (should not apply)
tc15 = BTContainer(data=[
# D O H L C V B S
[0, 5000, 5050, 4950, 5000, 6172, 1, 0],
[1, 5000, 5100, 4900, 5100, 6172, 1, 0],
[2, 5100, 5251, 4650, 5100, 6172, 0, 0],
[3, 4850, 5050, 4850, 4750, 6172, 0, 0],
[4, 4750, 4950, 4350, 4750, 6172, 0, 0]],
stop_loss=-0.05, roi=0.01, profit_perc=-0.04,
trades=[BTrade(sell_reason=SellType.ROI, open_tick=1, close_tick=1),
BTrade(sell_reason=SellType.STOP_LOSS, open_tick=2, close_tick=2)]
)
TESTS = [ TESTS = [
tc0, tc0,
tc1, tc1,
@ -238,6 +267,9 @@ TESTS = [
tc10, tc10,
tc11, tc11,
tc12, tc12,
tc13,
tc14,
tc15,
] ]

25
freqtrade/tests/optimize/test_backtesting.py
View File

@ -9,7 +9,7 @@ import pandas as pd
import pytest import pytest
from arrow import Arrow from arrow import Arrow
from freqtrade import DependencyException, constants from freqtrade import DependencyException, OperationalException, constants
from freqtrade.configuration import TimeRange from freqtrade.configuration import TimeRange
from freqtrade.data import history from freqtrade.data import history
from freqtrade.data.btanalysis import evaluate_result_multi from freqtrade.data.btanalysis import evaluate_result_multi
@ -21,7 +21,8 @@ from freqtrade.optimize.backtesting import Backtesting
from freqtrade.state import RunMode from freqtrade.state import RunMode
from freqtrade.strategy.default_strategy import DefaultStrategy from freqtrade.strategy.default_strategy import DefaultStrategy
from freqtrade.strategy.interface import SellType from freqtrade.strategy.interface import SellType
from freqtrade.tests.conftest import (get_args, log_has, log_has_re, patch_exchange, from freqtrade.tests.conftest import (get_args, log_has, log_has_re,
patch_exchange,
patched_configuration_load_config_file) patched_configuration_load_config_file)
@ -345,6 +346,23 @@ def test_backtesting_init(mocker, default_conf, order_types) -> None:
assert not backtesting.strategy.order_types["stoploss_on_exchange"] assert not backtesting.strategy.order_types["stoploss_on_exchange"]
def test_backtesting_init_no_ticker_interval(mocker, default_conf, caplog) -> None:
"""
Check that stoploss_on_exchange is set to False while backtesting
since backtesting assumes a perfect stoploss anyway.
"""
patch_exchange(mocker)
del default_conf['ticker_interval']
default_conf['strategy_list'] = ['DefaultStrategy',
'TestStrategy']
mocker.patch('freqtrade.exchange.Exchange.get_fee', MagicMock(return_value=0.5))
with pytest.raises(OperationalException):
Backtesting(default_conf)
log_has("Ticker-interval needs to be set in either configuration "
"or as cli argument `--ticker-interval 5m`", caplog.record_tuples)
def test_tickerdata_to_dataframe_bt(default_conf, mocker) -> None: def test_tickerdata_to_dataframe_bt(default_conf, mocker) -> None:
patch_exchange(mocker) patch_exchange(mocker)
timerange = TimeRange(None, 'line', 0, -100) timerange = TimeRange(None, 'line', 0, -100)
@ -618,8 +636,9 @@ def test_processed(default_conf, mocker) -> None:
def test_backtest_pricecontours(default_conf, fee, mocker) -> None: def test_backtest_pricecontours(default_conf, fee, mocker) -> None:
# TODO: Evaluate usefullness of this, the patterns and buy-signls are unrealistic
mocker.patch('freqtrade.exchange.Exchange.get_fee', fee) mocker.patch('freqtrade.exchange.Exchange.get_fee', fee)
tests = [['raise', 19], ['lower', 0], ['sine', 18]] tests = [['raise', 19], ['lower', 0], ['sine', 35]]
# We need to enable sell-signal - otherwise it sells on ROI!! # We need to enable sell-signal - otherwise it sells on ROI!!
default_conf['experimental'] = {"use_sell_signal": True} default_conf['experimental'] = {"use_sell_signal": True}

14
freqtrade/tests/test_configuration.py
View File

@ -15,6 +15,7 @@ from freqtrade.configuration import Arguments, Configuration
from freqtrade.configuration.check_exchange import check_exchange from freqtrade.configuration.check_exchange import check_exchange
from freqtrade.configuration.directory_operations import create_datadir, create_userdata_dir from freqtrade.configuration.directory_operations import create_datadir, create_userdata_dir
from freqtrade.configuration.json_schema import validate_config_schema from freqtrade.configuration.json_schema import validate_config_schema
from freqtrade.configuration.load_config import load_config_file
from freqtrade.constants import DEFAULT_DB_DRYRUN_URL, DEFAULT_DB_PROD_URL from freqtrade.constants import DEFAULT_DB_DRYRUN_URL, DEFAULT_DB_PROD_URL
from freqtrade.loggers import _set_loggers from freqtrade.loggers import _set_loggers
from freqtrade.state import RunMode from freqtrade.state import RunMode
@ -26,8 +27,7 @@ from freqtrade.tests.conftest import (log_has, log_has_re,
def all_conf(): def all_conf():
config_file = Path(__file__).parents[2] / "config_full.json.example" config_file = Path(__file__).parents[2] / "config_full.json.example"
print(config_file) print(config_file)
configuration = Configuration(Namespace()) conf = load_config_file(str(config_file))
conf = configuration._load_config_file(str(config_file))
return conf return conf
@ -54,12 +54,11 @@ def test_load_config_incorrect_stake_amount(default_conf) -> None:
def test_load_config_file(default_conf, mocker, caplog) -> None: def test_load_config_file(default_conf, mocker, caplog) -> None:
del default_conf['user_data_dir'] del default_conf['user_data_dir']
file_mock = mocker.patch('freqtrade.configuration.configuration.open', mocker.mock_open( file_mock = mocker.patch('freqtrade.configuration.load_config.open', mocker.mock_open(
read_data=json.dumps(default_conf) read_data=json.dumps(default_conf)
)) ))
configuration = Configuration(Namespace()) validated_conf = load_config_file('somefile')
validated_conf = configuration._load_config_file('somefile')
assert file_mock.call_count == 1 assert file_mock.call_count == 1
assert validated_conf.items() >= default_conf.items() assert validated_conf.items() >= default_conf.items()
@ -115,7 +114,7 @@ def test_load_config_combine_dicts(default_conf, mocker, caplog) -> None:
configsmock = MagicMock(side_effect=config_files) configsmock = MagicMock(side_effect=config_files)
mocker.patch( mocker.patch(
'freqtrade.configuration.configuration.Configuration._load_config_file', 'freqtrade.configuration.configuration.load_config_file',
configsmock configsmock
) )
@ -155,10 +154,9 @@ def test_load_config_file_exception(mocker) -> None:
'freqtrade.configuration.configuration.open', 'freqtrade.configuration.configuration.open',
MagicMock(side_effect=FileNotFoundError('File not found')) MagicMock(side_effect=FileNotFoundError('File not found'))
) )
configuration = Configuration(Namespace())
with pytest.raises(OperationalException, match=r'.*Config file "somefile" not found!*'): with pytest.raises(OperationalException, match=r'.*Config file "somefile" not found!*'):
configuration._load_config_file('somefile') load_config_file('somefile')
def test_load_config(default_conf, mocker) -> None: def test_load_config(default_conf, mocker) -> None:

12
requirements-common.txt
View File

@ -1,16 +1,16 @@
# requirements without requirements installable via conda # requirements without requirements installable via conda
# mainly used for Raspberry pi installs # mainly used for Raspberry pi installs
ccxt==1.18.992 ccxt==1.18.1021
SQLAlchemy==1.3.6 SQLAlchemy==1.3.6
python-telegram-bot==11.1.0 python-telegram-bot==11.1.0
arrow==0.14.3 arrow==0.14.4
cachetools==3.1.1 cachetools==3.1.1
requests==2.22.0 requests==2.22.0
urllib3==1.24.2 # pyup: ignore urllib3==1.25.3
wrapt==1.11.2 wrapt==1.11.2
scikit-learn==0.21.2 scikit-learn==0.21.3
joblib==0.13.2 joblib==0.13.2
jsonschema==3.0.1 jsonschema==3.0.2
TA-Lib==0.4.17 TA-Lib==0.4.17
tabulate==0.8.3 tabulate==0.8.3
coinmarketcap==5.0.3 coinmarketcap==5.0.3
@ -20,7 +20,7 @@ scikit-optimize==0.5.2
filelock==3.0.12 filelock==3.0.12
# find first, C search in arrays # find first, C search in arrays
py_find_1st==1.1.3 py_find_1st==1.1.4
#Load ticker files 30% faster #Load ticker files 30% faster
python-rapidjson==0.7.2 python-rapidjson==0.7.2

2
requirements-dev.txt
View File

@ -2,7 +2,7 @@
-r requirements.txt -r requirements.txt
-r requirements-plot.txt -r requirements-plot.txt
coveralls==1.8.1 coveralls==1.8.2
flake8==3.7.8 flake8==3.7.8
flake8-type-annotations==0.1.0 flake8-type-annotations==0.1.0
flake8-tidy-imports==2.0.0 flake8-tidy-imports==2.0.0

2
requirements-plot.txt
View File

@ -1,5 +1,5 @@
# Include all requirements to run the bot. # Include all requirements to run the bot.
-r requirements.txt -r requirements.txt
plotly==4.0.0 plotly==4.1.0

3
scripts/download_backtest_data.py
View File

@ -12,6 +12,7 @@ from freqtrade.configuration import Arguments, TimeRange
from freqtrade.configuration import Configuration from freqtrade.configuration import Configuration
from freqtrade.configuration.arguments import ARGS_DOWNLOADER from freqtrade.configuration.arguments import ARGS_DOWNLOADER
from freqtrade.configuration.check_exchange import check_exchange from freqtrade.configuration.check_exchange import check_exchange
from freqtrade.configuration.load_config import load_config_file
from freqtrade.data.history import download_pair_history from freqtrade.data.history import download_pair_history
from freqtrade.exchange import Exchange from freqtrade.exchange import Exchange
from freqtrade.misc import deep_merge_dicts from freqtrade.misc import deep_merge_dicts
@ -40,7 +41,7 @@ if args.config:
for path in args.config: for path in args.config:
logger.info(f"Using config: {path}...") logger.info(f"Using config: {path}...")
# Merge config options, overwriting old values # Merge config options, overwriting old values
config = deep_merge_dicts(configuration._load_config_file(path), config) config = deep_merge_dicts(load_config_file(path), config)
config['stake_currency'] = '' config['stake_currency'] = ''
# Ensure we do not use Exchange credentials # Ensure we do not use Exchange credentials

10
setup.py
View File

@ -25,7 +25,13 @@ develop = [
'pytest-random-order', 'pytest-random-order',
] ]
all_extra = api + plot + develop jupyter = [
'jupyter',
'nbstripout',
'ipykernel',
]
all_extra = api + plot + develop + jupyter
setup(name='freqtrade', setup(name='freqtrade',
version=__version__, version=__version__,
@ -68,6 +74,8 @@ setup(name='freqtrade',
'dev': all_extra, 'dev': all_extra,
'plot': plot, 'plot': plot,
'all': all_extra, 'all': all_extra,
'jupyter': jupyter,
}, },
include_package_data=True, include_package_data=True,
zip_safe=False, zip_safe=False,

57
setup.sh
View File

@ -11,6 +11,12 @@ function check_installed_pip() {
# Check which python version is installed # Check which python version is installed
function check_installed_python() { function check_installed_python() {
if [ -n "${VIRTUAL_ENV}" ]; then
echo "Please deactivate your virtual environment before running setup.sh."
echo "You can do this by running 'deactivate'."
exit 2
fi
which python3.7 which python3.7
if [ $? -eq 0 ]; then if [ $? -eq 0 ]; then
echo "using Python 3.7" echo "using Python 3.7"
@ -37,17 +43,19 @@ function updateenv() {
echo "-------------------------" echo "-------------------------"
echo "Updating your virtual env" echo "Updating your virtual env"
echo "-------------------------" echo "-------------------------"
if [ ! -f .env/bin/activate ]; then
echo "Something went wrong, no virtual environment found."
exit 1
fi
source .env/bin/activate source .env/bin/activate
echo "pip install in-progress. Please wait..." echo "pip install in-progress. Please wait..."
# Install numpy first to have py_find_1st install clean ${PYTHON} -m pip install --upgrade pip
${PYTHON} -m pip install --upgrade pip numpy
${PYTHON} -m pip install --upgrade -r requirements.txt
read -p "Do you want to install dependencies for dev [y/N]? " read -p "Do you want to install dependencies for dev [y/N]? "
if [[ $REPLY =~ ^[Yy]$ ]] if [[ $REPLY =~ ^[Yy]$ ]]
then then
${PYTHON} -m pip install --upgrade -r requirements-dev.txt ${PYTHON} -m pip install --upgrade -r requirements-dev.txt
else else
${PYTHON} -m pip install --upgrade -r requirements.txt
echo "Dev dependencies ignored." echo "Dev dependencies ignored."
fi fi
@ -70,6 +78,10 @@ function install_talib() {
./configure --prefix=/usr/local ./configure --prefix=/usr/local
make make
sudo make install sudo make install
if [ -x "$(command -v apt-get)" ]; then
echo "Updating library path using ldconfig"
sudo ldconfig
fi
cd .. && rm -rf ./ta-lib/ cd .. && rm -rf ./ta-lib/
cd .. cd ..
} }
@ -90,7 +102,7 @@ function install_macos() {
# Install bot Debian_ubuntu # Install bot Debian_ubuntu
function install_debian() { function install_debian() {
sudo apt-get update sudo apt-get update
sudo apt-get install build-essential autoconf libtool pkg-config make wget git sudo apt-get install -y build-essential autoconf libtool pkg-config make wget git
install_talib install_talib
} }
@ -105,30 +117,39 @@ function reset() {
echo "----------------------------" echo "----------------------------"
echo "Reseting branch and virtual env" echo "Reseting branch and virtual env"
echo "----------------------------" echo "----------------------------"
if [ "1" == $(git branch -vv |grep -cE "\* develop|\* master") ] if [ "1" == $(git branch -vv |grep -cE "\* develop|\* master") ]
then then
if [ -d ".env" ]; then
echo "- Delete your previous virtual env"
rm -rf .env
fi
git fetch -a read -p "Reset git branch? (This will remove all changes you made!) [y/N]? "
if [[ $REPLY =~ ^[Yy]$ ]]; then
if [ "1" == $(git branch -vv |grep -c "* develop") ] git fetch -a
then
echo "- Hard resetting of 'develop' branch." if [ "1" == $(git branch -vv |grep -c "* develop") ]
git reset --hard origin/develop then
elif [ "1" == $(git branch -vv |grep -c "* master") ] echo "- Hard resetting of 'develop' branch."
then git reset --hard origin/develop
echo "- Hard resetting of 'master' branch." elif [ "1" == $(git branch -vv |grep -c "* master") ]
git reset --hard origin/master then
echo "- Hard resetting of 'master' branch."
git reset --hard origin/master
fi
fi fi
else else
echo "Reset ignored because you are not on 'master' or 'develop'." echo "Reset ignored because you are not on 'master' or 'develop'."
fi fi
if [ -d ".env" ]; then
echo "- Delete your previous virtual env"
rm -rf .env
fi
echo echo
${PYTHON} -m venv .env ${PYTHON} -m venv .env
if [ $? -ne 0 ]; then
echo "Could not create virtual environment. Leaving now"
exit 1
fi
updateenv updateenv
} }

65
user_data/hyperopts/sample_hyperopt.py
View File

@ -14,20 +14,27 @@ import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.optimize.hyperopt_interface import IHyperOpt from freqtrade.optimize.hyperopt_interface import IHyperOpt
# This class is a sample. Feel free to customize it.
class SampleHyperOpts(IHyperOpt): class SampleHyperOpts(IHyperOpt):
""" """
This is a test hyperopt to inspire you. This is a sample hyperopt to inspire you.
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md Feel free to customize it.
You can:
- Rename the class name (Do not forget to update class_name)
- Add any methods you want to build your hyperopt
- Add any lib you need to build your hyperopt
You must keep:
- the prototype for the methods: populate_indicators, indicator_space, buy_strategy_generator,
roi_space, generate_roi_table, stoploss_space
"""
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md
You should:
- Rename the class name to some unique name.
- Add any methods you want to build your hyperopt.
- Add any lib you need to build your hyperopt.
You must keep:
- The prototypes for the methods: populate_indicators, indicator_space, buy_strategy_generator.
The roi_space, generate_roi_table, stoploss_space methods are no longer required to be
copied in every custom hyperopt. However, you may override them if you need the
'roi' and the 'stoploss' spaces that differ from the defaults offered by Freqtrade.
Sample implementation of these methods can be found in
https://github.com/freqtrade/freqtrade/blob/develop/user_data/hyperopts/sample_hyperopt_advanced.py
"""
@staticmethod @staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe) dataframe['adx'] = ta.ADX(dataframe)
@ -167,42 +174,6 @@ class SampleHyperOpts(IHyperOpt):
'sell-sar_reversal'], name='sell-trigger') 'sell-sar_reversal'], name='sell-trigger')
] ]
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss Value to search
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame: def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
""" """
Based on TA indicators. Should be a copy of from strategy Based on TA indicators. Should be a copy of from strategy

261
user_data/hyperopts/sample_hyperopt_advanced.py Normal file
View File

@ -0,0 +1,261 @@
# pragma pylint: disable=missing-docstring, invalid-name, pointless-string-statement
from functools import reduce
from math import exp
from typing import Any, Callable, Dict, List
from datetime import datetime
import numpy as np# noqa F401
import talib.abstract as ta
from pandas import DataFrame
from skopt.space import Categorical, Dimension, Integer, Real
import freqtrade.vendor.qtpylib.indicators as qtpylib
from freqtrade.optimize.hyperopt_interface import IHyperOpt
class AdvancedSampleHyperOpts(IHyperOpt):
"""
This is a sample hyperopt to inspire you.
Feel free to customize it.
More information in https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md
You should:
- Rename the class name to some unique name.
- Add any methods you want to build your hyperopt.
- Add any lib you need to build your hyperopt.
You must keep:
- The prototypes for the methods: populate_indicators, indicator_space, buy_strategy_generator.
The roi_space, generate_roi_table, stoploss_space methods are no longer required to be
copied in every custom hyperopt. However, you may override them if you need the
'roi' and the 'stoploss' spaces that differ from the defaults offered by Freqtrade.
This sample illustrates how to override these methods.
"""
@staticmethod
def populate_indicators(dataframe: DataFrame, metadata: dict) -> DataFrame:
dataframe['adx'] = ta.ADX(dataframe)
macd = ta.MACD(dataframe)
dataframe['macd'] = macd['macd']
dataframe['macdsignal'] = macd['macdsignal']
dataframe['mfi'] = ta.MFI(dataframe)
dataframe['rsi'] = ta.RSI(dataframe)
stoch_fast = ta.STOCHF(dataframe)
dataframe['fastd'] = stoch_fast['fastd']
dataframe['minus_di'] = ta.MINUS_DI(dataframe)
# Bollinger bands
bollinger = qtpylib.bollinger_bands(qtpylib.typical_price(dataframe), window=20, stds=2)
dataframe['bb_lowerband'] = bollinger['lower']
dataframe['bb_upperband'] = bollinger['upper']
dataframe['sar'] = ta.SAR(dataframe)
return dataframe
@staticmethod
def buy_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the buy strategy parameters to be used by hyperopt
"""
def populate_buy_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Buy strategy Hyperopt will build and use
"""
conditions = []
# GUARDS AND TRENDS
if 'mfi-enabled' in params and params['mfi-enabled']:
conditions.append(dataframe['mfi'] < params['mfi-value'])
if 'fastd-enabled' in params and params['fastd-enabled']:
conditions.append(dataframe['fastd'] < params['fastd-value'])
if 'adx-enabled' in params and params['adx-enabled']:
conditions.append(dataframe['adx'] > params['adx-value'])
if 'rsi-enabled' in params and params['rsi-enabled']:
conditions.append(dataframe['rsi'] < params['rsi-value'])
# TRIGGERS
if 'trigger' in params:
if params['trigger'] == 'bb_lower':
conditions.append(dataframe['close'] < dataframe['bb_lowerband'])
if params['trigger'] == 'macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macd'], dataframe['macdsignal']
))
if params['trigger'] == 'sar_reversal':
conditions.append(qtpylib.crossed_above(
dataframe['close'], dataframe['sar']
))
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'buy'] = 1
return dataframe
return populate_buy_trend
@staticmethod
def indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching strategy parameters
"""
return [
Integer(10, 25, name='mfi-value'),
Integer(15, 45, name='fastd-value'),
Integer(20, 50, name='adx-value'),
Integer(20, 40, name='rsi-value'),
Categorical([True, False], name='mfi-enabled'),
Categorical([True, False], name='fastd-enabled'),
Categorical([True, False], name='adx-enabled'),
Categorical([True, False], name='rsi-enabled'),
Categorical(['bb_lower', 'macd_cross_signal', 'sar_reversal'], name='trigger')
]
@staticmethod
def sell_strategy_generator(params: Dict[str, Any]) -> Callable:
"""
Define the sell strategy parameters to be used by hyperopt
"""
def populate_sell_trend(dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Sell strategy Hyperopt will build and use
"""
# print(params)
conditions = []
# GUARDS AND TRENDS
if 'sell-mfi-enabled' in params and params['sell-mfi-enabled']:
conditions.append(dataframe['mfi'] > params['sell-mfi-value'])
if 'sell-fastd-enabled' in params and params['sell-fastd-enabled']:
conditions.append(dataframe['fastd'] > params['sell-fastd-value'])
if 'sell-adx-enabled' in params and params['sell-adx-enabled']:
conditions.append(dataframe['adx'] < params['sell-adx-value'])
if 'sell-rsi-enabled' in params and params['sell-rsi-enabled']:
conditions.append(dataframe['rsi'] > params['sell-rsi-value'])
# TRIGGERS
if 'sell-trigger' in params:
if params['sell-trigger'] == 'sell-bb_upper':
conditions.append(dataframe['close'] > dataframe['bb_upperband'])
if params['sell-trigger'] == 'sell-macd_cross_signal':
conditions.append(qtpylib.crossed_above(
dataframe['macdsignal'], dataframe['macd']
))
if params['sell-trigger'] == 'sell-sar_reversal':
conditions.append(qtpylib.crossed_above(
dataframe['sar'], dataframe['close']
))
if conditions:
dataframe.loc[
reduce(lambda x, y: x & y, conditions),
'sell'] = 1
return dataframe
return populate_sell_trend
@staticmethod
def sell_indicator_space() -> List[Dimension]:
"""
Define your Hyperopt space for searching sell strategy parameters
"""
return [
Integer(75, 100, name='sell-mfi-value'),
Integer(50, 100, name='sell-fastd-value'),
Integer(50, 100, name='sell-adx-value'),
Integer(60, 100, name='sell-rsi-value'),
Categorical([True, False], name='sell-mfi-enabled'),
Categorical([True, False], name='sell-fastd-enabled'),
Categorical([True, False], name='sell-adx-enabled'),
Categorical([True, False], name='sell-rsi-enabled'),
Categorical(['sell-bb_upper',
'sell-macd_cross_signal',
'sell-sar_reversal'], name='sell-trigger')
]
@staticmethod
def generate_roi_table(params: Dict) -> Dict[int, float]:
"""
Generate the ROI table that will be used by Hyperopt
This implementation generates the default legacy Freqtrade ROI tables.
Change it if you need different number of steps in the generated
ROI tables or other structure of the ROI tables.
Please keep it aligned with parameters in the 'roi' optimization
hyperspace defined by the roi_space method.
"""
roi_table = {}
roi_table[0] = params['roi_p1'] + params['roi_p2'] + params['roi_p3']
roi_table[params['roi_t3']] = params['roi_p1'] + params['roi_p2']
roi_table[params['roi_t3'] + params['roi_t2']] = params['roi_p1']
roi_table[params['roi_t3'] + params['roi_t2'] + params['roi_t1']] = 0
return roi_table
@staticmethod
def roi_space() -> List[Dimension]:
"""
Values to search for each ROI steps
Override it if you need some different ranges for the parameters in the
'roi' optimization hyperspace.
Please keep it aligned with the implementation of the
generate_roi_table method.
"""
return [
Integer(10, 120, name='roi_t1'),
Integer(10, 60, name='roi_t2'),
Integer(10, 40, name='roi_t3'),
Real(0.01, 0.04, name='roi_p1'),
Real(0.01, 0.07, name='roi_p2'),
Real(0.01, 0.20, name='roi_p3'),
]
@staticmethod
def stoploss_space() -> List[Dimension]:
"""
Stoploss Value to search
Override it if you need some different range for the parameter in the
'stoploss' optimization hyperspace.
"""
return [
Real(-0.5, -0.02, name='stoploss'),
]
def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators. Should be a copy of from strategy
must align to populate_indicators in this file
Only used when --spaces does not include buy
"""
dataframe.loc[
(
(dataframe['close'] < dataframe['bb_lowerband']) &
(dataframe['mfi'] < 16) &
(dataframe['adx'] > 25) &
(dataframe['rsi'] < 21)
),
'buy'] = 1
return dataframe
def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
"""
Based on TA indicators. Should be a copy of from strategy
must align to populate_indicators in this file
Only used when --spaces does not include sell
"""
dataframe.loc[
(
(qtpylib.crossed_above(
dataframe['macdsignal'], dataframe['macd']
)) &
(dataframe['fastd'] > 54)
),
'sell'] = 1
return dataframe

0
user_data/notebooks/.gitkeep Normal file
View File

243
user_data/notebooks/analysis_example.ipynb Normal file
View File

@ -0,0 +1,243 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Analyzing bot data\n",
"\n",
"You can analyze the results of backtests and trading history easily using Jupyter notebooks. \n",
"**Copy this file so your changes don't get clobbered with the next freqtrade update!** \n",
"For usage instructions, see [jupyter.org](https://jupyter.org/documentation). \n",
"*Pro tip - Don't forget to start a jupyter notbook server from within your conda or venv environment or use [nb_conda_kernels](https://github.com/Anaconda-Platform/nb_conda_kernels)*\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Imports\n",
"from pathlib import Path\n",
"import os\n",
"from freqtrade.data.history import load_pair_history\n",
"from freqtrade.resolvers import StrategyResolver\n",
"from freqtrade.data.btanalysis import load_backtest_data\n",
"from freqtrade.data.btanalysis import load_trades_from_db"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Change directory\n",
"# Define all paths relative to the project root shown in the cell output\n",
"try:\n",
" os.chdir(Path(Path.cwd(), '../..'))\n",
" print(Path.cwd())\n",
"except:\n",
" pass"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example snippets"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load backtest results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load backtest results\n",
"df = load_backtest_data(\"user_data/backtest_data/backtest-result.json\")\n",
"\n",
"# Show value-counts per pair\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load live trading results into a pandas dataframe"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Fetch trades from database\n",
"df = load_trades_from_db(\"sqlite:///tradesv3.sqlite\")\n",
"\n",
"# Display results\n",
"df.groupby(\"pair\")[\"sell_reason\"].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Strategy debugging example\n",
"\n",
"Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import requirements and define variables used in analyses"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define some constants\n",
"ticker_interval = \"5m\"\n",
"# Name of the strategy class\n",
"strategy_name = 'AwesomeStrategy'\n",
"# Path to user data\n",
"user_data_dir = 'user_data'\n",
"# Location of the strategy\n",
"strategy_location = Path(user_data_dir, 'strategies')\n",
"# Location of the data\n",
"data_location = Path(user_data_dir, 'data', 'binance')\n",
"# Pair to analyze \n",
"# Only use one pair here\n",
"pair = \"BTC_USDT\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load exchange data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load data using values set above\n",
"bt_data = load_pair_history(datadir=Path(data_location),\n",
" ticker_interval=ticker_interval,\n",
" pair=pair)\n",
"\n",
"# Confirm success\n",
"print(\"Loaded \" + str(len(bt_data)) + f\" rows of data for {pair} from {data_location}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load and run strategy\n",
"* Rerun each time the strategy file is changed"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load strategy using values set above\n",
"strategy = StrategyResolver({'strategy': strategy_name,\n",
" 'user_data_dir': user_data_dir,\n",
" 'strategy_path': strategy_location}).strategy\n",
"\n",
"# Generate buy/sell signals using strategy\n",
"df = strategy.analyze_ticker(bt_data, {'pair': pair})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Display the trade details\n",
"* Note that using `data.head()` would also work, however most indicators have some \"startup\" data at the top of the dataframe.\n",
"\n",
"#### Some possible problems\n",
"\n",
"* Columns with NaN values at the end of the dataframe\n",
"* Columns used in `crossed*()` functions with completely different units\n",
"\n",
"#### Comparison with full backtest\n",
"\n",
"having 200 buy signals as output for one pair from `analyze_ticker()` does not necessarily mean that 200 trades will be made during backtesting.\n",
"\n",
"Assuming you use only one condition such as, `df['rsi'] < 30` as buy condition, this will generate multiple \"buy\" signals for each pair in sequence (until rsi returns > 29).\n",
"The bot will only buy on the first of these signals (and also only if a trade-slot (\"max_open_trades\") is still available), or on one of the middle signals, as soon as a \"slot\" becomes available.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Report results\n",
"print(f\"Generated {df['buy'].sum()} buy signals\")\n",
"data = df.set_index('date', drop=True)\n",
"data.tail()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Feel free to submit an issue or Pull Request enhancing this document if you would like to share ideas on how to best analyze the data."
]
}
],
"metadata": {
"file_extension": ".py",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
},
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3
},
"nbformat": 4,
"nbformat_minor": 2
}