Merge branch 'develop' into freqai_feature_engineering_functions

This commit is contained in:
Wagner Costa 2023-01-04 10:40:20 -03:00
commit ed2b1b1ed1
42 changed files with 846 additions and 605 deletions

View File

@ -147,15 +147,20 @@ jobs:
- name: Installation - macOS - name: Installation - macOS
if: runner.os == 'macOS' if: runner.os == 'macOS'
run: | run: |
# homebrew fails to update python 3.9.1 to 3.9.1.1 due to unlinking failure brew update
# homebrew fails to update python due to unlinking failures
# https://github.com/actions/runner-images/issues/6817
rm /usr/local/bin/2to3 || true rm /usr/local/bin/2to3 || true
# homebrew fails to update python from 3.9 to 3.10 due to another unlinking failure rm /usr/local/bin/2to3-3.11 || true
rm /usr/local/bin/idle3 || true rm /usr/local/bin/idle3 || true
rm /usr/local/bin/idle3.11 || true
rm /usr/local/bin/pydoc3 || true rm /usr/local/bin/pydoc3 || true
rm /usr/local/bin/pydoc3.11 || true
rm /usr/local/bin/python3 || true rm /usr/local/bin/python3 || true
rm /usr/local/bin/python3.11 || true
rm /usr/local/bin/python3-config || true rm /usr/local/bin/python3-config || true
# Ignore brew update failures - https://github.com/actions/runner-images/issues/6817 rm /usr/local/bin/python3.11-config || true
brew update || true
brew install hdf5 c-blosc brew install hdf5 c-blosc
python -m pip install --upgrade pip wheel python -m pip install --upgrade pip wheel
export LD_LIBRARY_PATH=${HOME}/dependencies/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=${HOME}/dependencies/lib:$LD_LIBRARY_PATH

View File

@ -15,7 +15,7 @@ repos:
additional_dependencies: additional_dependencies:
- types-cachetools==5.2.1 - types-cachetools==5.2.1
- types-filelock==3.2.7 - types-filelock==3.2.7
- types-requests==2.28.11.5 - types-requests==2.28.11.7
- types-tabulate==0.9.0.0 - types-tabulate==0.9.0.0
- types-python-dateutil==2.8.19.5 - types-python-dateutil==2.8.19.5
# stages: [push] # stages: [push]

View File

@ -1,6 +1,7 @@
# ![freqtrade](https://raw.githubusercontent.com/freqtrade/freqtrade/develop/docs/assets/freqtrade_poweredby.svg) # ![freqtrade](https://raw.githubusercontent.com/freqtrade/freqtrade/develop/docs/assets/freqtrade_poweredby.svg)
[![Freqtrade CI](https://github.com/freqtrade/freqtrade/workflows/Freqtrade%20CI/badge.svg)](https://github.com/freqtrade/freqtrade/actions/) [![Freqtrade CI](https://github.com/freqtrade/freqtrade/workflows/Freqtrade%20CI/badge.svg)](https://github.com/freqtrade/freqtrade/actions/)
[![DOI](https://joss.theoj.org/papers/10.21105/joss.04864/status.svg)](https://doi.org/10.21105/joss.04864)
[![Coverage Status](https://coveralls.io/repos/github/freqtrade/freqtrade/badge.svg?branch=develop&service=github)](https://coveralls.io/github/freqtrade/freqtrade?branch=develop) [![Coverage Status](https://coveralls.io/repos/github/freqtrade/freqtrade/badge.svg?branch=develop&service=github)](https://coveralls.io/github/freqtrade/freqtrade?branch=develop)
[![Documentation](https://readthedocs.org/projects/freqtrade/badge/)](https://www.freqtrade.io) [![Documentation](https://readthedocs.org/projects/freqtrade/badge/)](https://www.freqtrade.io)
[![Maintainability](https://api.codeclimate.com/v1/badges/5737e6d668200b7518ff/maintainability)](https://codeclimate.com/github/freqtrade/freqtrade/maintainability) [![Maintainability](https://api.codeclimate.com/v1/badges/5737e6d668200b7518ff/maintainability)](https://codeclimate.com/github/freqtrade/freqtrade/maintainability)

View File

@ -300,7 +300,11 @@ A backtesting result will look like that:
| Absolute profit | 0.00762792 BTC | | Absolute profit | 0.00762792 BTC |
| Total profit % | 76.2% | | Total profit % | 76.2% |
| CAGR % | 460.87% | | CAGR % | 460.87% |
| Sortino | 1.88 |
| Sharpe | 2.97 |
| Calmar | 6.29 |
| Profit factor | 1.11 | | Profit factor | 1.11 |
| Expectancy | -0.15 |
| Avg. stake amount | 0.001 BTC | | Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC | | Total trade volume | 0.429 BTC |
| | | | | |
@ -400,7 +404,11 @@ It contains some useful key metrics about performance of your strategy on backte
| Absolute profit | 0.00762792 BTC | | Absolute profit | 0.00762792 BTC |
| Total profit % | 76.2% | | Total profit % | 76.2% |
| CAGR % | 460.87% | | CAGR % | 460.87% |
| Sortino | 1.88 |
| Sharpe | 2.97 |
| Calmar | 6.29 |
| Profit factor | 1.11 | | Profit factor | 1.11 |
| Expectancy | -0.15 |
| Avg. stake amount | 0.001 BTC | | Avg. stake amount | 0.001 BTC |
| Total trade volume | 0.429 BTC | | Total trade volume | 0.429 BTC |
| | | | | |
@ -447,6 +455,9 @@ It contains some useful key metrics about performance of your strategy on backte
- `Absolute profit`: Profit made in stake currency. - `Absolute profit`: Profit made in stake currency.
- `Total profit %`: Total profit. Aligned to the `TOTAL` row's `Tot Profit %` from the first table. Calculated as `(End capital Starting capital) / Starting capital`. - `Total profit %`: Total profit. Aligned to the `TOTAL` row's `Tot Profit %` from the first table. Calculated as `(End capital Starting capital) / Starting capital`.
- `CAGR %`: Compound annual growth rate. - `CAGR %`: Compound annual growth rate.
- `Sortino`: Annualized Sortino ratio.
- `Sharpe`: Annualized Sharpe ratio.
- `Calmar`: Annualized Calmar ratio.
- `Profit factor`: profit / loss. - `Profit factor`: profit / loss.
- `Avg. stake amount`: Average stake amount, either `stake_amount` or the average when using dynamic stake amount. - `Avg. stake amount`: Average stake amount, either `stake_amount` or the average when using dynamic stake amount.
- `Total trade volume`: Volume generated on the exchange to reach the above profit. - `Total trade volume`: Volume generated on the exchange to reach the above profit.

View File

@ -15,7 +15,7 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `identifier` | **Required.** <br> A unique ID for the current model. If models are saved to disk, the `identifier` allows for reloading specific pre-trained models/data. <br> **Datatype:** String. | `identifier` | **Required.** <br> A unique ID for the current model. If models are saved to disk, the `identifier` allows for reloading specific pre-trained models/data. <br> **Datatype:** String.
| `live_retrain_hours` | Frequency of retraining during dry/live runs. <br> **Datatype:** Float > 0. <br> Default: `0` (models retrain as often as possible). | `live_retrain_hours` | Frequency of retraining during dry/live runs. <br> **Datatype:** Float > 0. <br> Default: `0` (models retrain as often as possible).
| `expiration_hours` | Avoid making predictions if a model is more than `expiration_hours` old. <br> **Datatype:** Positive integer. <br> Default: `0` (models never expire). | `expiration_hours` | Avoid making predictions if a model is more than `expiration_hours` old. <br> **Datatype:** Positive integer. <br> Default: `0` (models never expire).
| `purge_old_models` | Delete obsolete models. <br> **Datatype:** Boolean. <br> Default: `False` (all historic models remain on disk). | `purge_old_models` | Delete all unused models during live runs (not relevant to backtesting). If set to false (not default), dry/live runs will accumulate all unused models to disk. If <br> **Datatype:** Boolean. <br> Default: `True`.
| `save_backtest_models` | Save models to disk when running backtesting. Backtesting operates most efficiently by saving the prediction data and reusing them directly for subsequent runs (when you wish to tune entry/exit parameters). Saving backtesting models to disk also allows to use the same model files for starting a dry/live instance with the same model `identifier`. <br> **Datatype:** Boolean. <br> Default: `False` (no models are saved). | `save_backtest_models` | Save models to disk when running backtesting. Backtesting operates most efficiently by saving the prediction data and reusing them directly for subsequent runs (when you wish to tune entry/exit parameters). Saving backtesting models to disk also allows to use the same model files for starting a dry/live instance with the same model `identifier`. <br> **Datatype:** Boolean. <br> Default: `False` (no models are saved).
| `fit_live_predictions_candles` | Number of historical candles to use for computing target (label) statistics from prediction data, instead of from the training dataset (more information can be found [here](freqai-configuration.md#creating-a-dynamic-target-threshold)). <br> **Datatype:** Positive integer. | `fit_live_predictions_candles` | Number of historical candles to use for computing target (label) statistics from prediction data, instead of from the training dataset (more information can be found [here](freqai-configuration.md#creating-a-dynamic-target-threshold)). <br> **Datatype:** Positive integer.
| `follow_mode` | Use a `follower` that will look for models associated with a specific `identifier` and load those for inferencing. A `follower` will **not** train new models. <br> **Datatype:** Boolean. <br> Default: `False`. | `follow_mode` | Use a `follower` that will look for models associated with a specific `identifier` and load those for inferencing. A `follower` will **not** train new models. <br> **Datatype:** Boolean. <br> Default: `False`.

View File

@ -365,7 +365,7 @@ class MyAwesomeStrategy(IStrategy):
timeframe = '15m' timeframe = '15m'
minimal_roi = { minimal_roi = {
"0": 0.10 "0": 0.10
}, }
# Define the parameter spaces # Define the parameter spaces
buy_ema_short = IntParameter(3, 50, default=5) buy_ema_short = IntParameter(3, 50, default=5)
buy_ema_long = IntParameter(15, 200, default=50) buy_ema_long = IntParameter(15, 200, default=50)

View File

@ -1,6 +1,7 @@
![freqtrade](assets/freqtrade_poweredby.svg) ![freqtrade](assets/freqtrade_poweredby.svg)
[![Freqtrade CI](https://github.com/freqtrade/freqtrade/workflows/Freqtrade%20CI/badge.svg)](https://github.com/freqtrade/freqtrade/actions/) [![Freqtrade CI](https://github.com/freqtrade/freqtrade/workflows/Freqtrade%20CI/badge.svg)](https://github.com/freqtrade/freqtrade/actions/)
[![DOI](https://joss.theoj.org/papers/10.21105/joss.04864/status.svg)](https://doi.org/10.21105/joss.04864)
[![Coverage Status](https://coveralls.io/repos/github/freqtrade/freqtrade/badge.svg?branch=develop&service=github)](https://coveralls.io/github/freqtrade/freqtrade?branch=develop) [![Coverage Status](https://coveralls.io/repos/github/freqtrade/freqtrade/badge.svg?branch=develop&service=github)](https://coveralls.io/github/freqtrade/freqtrade?branch=develop)
[![Maintainability](https://api.codeclimate.com/v1/badges/5737e6d668200b7518ff/maintainability)](https://codeclimate.com/github/freqtrade/freqtrade/maintainability) [![Maintainability](https://api.codeclimate.com/v1/badges/5737e6d668200b7518ff/maintainability)](https://codeclimate.com/github/freqtrade/freqtrade/maintainability)

View File

@ -92,6 +92,8 @@ One account is used to share collateral between markets (trading pairs). Margin
"margin_mode": "cross" "margin_mode": "cross"
``` ```
Please read the [exchange specific notes](exchanges.md) for exchanges that support this mode and how they differ.
## Set leverage to use ## Set leverage to use
Different strategies and risk profiles will require different levels of leverage. Different strategies and risk profiles will require different levels of leverage.

View File

@ -11,9 +11,6 @@
{% endif %} {% endif %}
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" {{ hidden }}> <div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" {{ hidden }}>
<div class="md-sidebar__scrollwrap"> <div class="md-sidebar__scrollwrap">
<div id="widget-wrapper">
</div>
<div class="md-sidebar__inner"> <div class="md-sidebar__inner">
{% include "partials/nav.html" %} {% include "partials/nav.html" %}
</div> </div>
@ -44,25 +41,4 @@
<script src="https://code.jquery.com/jquery-3.4.1.min.js" <script src="https://code.jquery.com/jquery-3.4.1.min.js"
integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script> integrity="sha256-CSXorXvZcTkaix6Yvo6HppcZGetbYMGWSFlBw8HfCJo=" crossorigin="anonymous"></script>
<!-- Load binance SDK -->
<script async defer src="https://public.bnbstatic.com/static/js/broker-sdk/broker-sdk@1.0.0.min.js"></script>
<script>
window.onload = function () {
var sidebar = document.getElementById('widget-wrapper')
var newDiv = document.createElement("div");
newDiv.id = "widget";
try {
sidebar.prepend(newDiv);
window.binanceBrokerPortalSdk.initBrokerSDK('#widget', {
apiHost: 'https://www.binance.com',
brokerId: 'R4BD3S82',
slideTime: 4e4,
});
} catch(err) {
console.log(err)
}
}
</script>
{% endblock %} {% endblock %}

View File

@ -11,18 +11,3 @@
.rst-versions .rst-other-versions { .rst-versions .rst-other-versions {
color: white; color: white;
} }
#widget-wrapper {
height: calc(220px * 0.5625 + 18px);
width: 220px;
margin: 0 auto 16px auto;
border-style: solid;
border-color: var(--md-code-bg-color);
border-width: 1px;
border-radius: 5px;
}
@media screen and (max-width: calc(76.25em - 1px)) {
#widget-wrapper { display: none; }
}

View File

@ -1,5 +1,5 @@
""" Freqtrade bot """ """ Freqtrade bot """
__version__ = '2022.12.dev' __version__ = '2023.1.dev'
if 'dev' in __version__: if 'dev' in __version__:
try: try:

View File

@ -20,8 +20,8 @@ from freqtrade.persistence import LocalTrade, Trade, init_db
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# Newest format # Newest format
BT_DATA_COLUMNS = ['pair', 'stake_amount', 'amount', 'open_date', 'close_date', BT_DATA_COLUMNS = ['pair', 'stake_amount', 'max_stake_amount', 'amount',
'open_rate', 'close_rate', 'open_date', 'close_date', 'open_rate', 'close_rate',
'fee_open', 'fee_close', 'trade_duration', 'fee_open', 'fee_close', 'trade_duration',
'profit_ratio', 'profit_abs', 'exit_reason', 'profit_ratio', 'profit_abs', 'exit_reason',
'initial_stop_loss_abs', 'initial_stop_loss_ratio', 'stop_loss_abs', 'initial_stop_loss_abs', 'initial_stop_loss_ratio', 'stop_loss_abs',
@ -241,6 +241,33 @@ def find_existing_backtest_stats(dirname: Union[Path, str], run_ids: Dict[str, s
return results return results
def _load_backtest_data_df_compatibility(df: pd.DataFrame) -> pd.DataFrame:
"""
Compatibility support for older backtest data.
"""
df['open_date'] = pd.to_datetime(df['open_date'],
utc=True,
infer_datetime_format=True
)
df['close_date'] = pd.to_datetime(df['close_date'],
utc=True,
infer_datetime_format=True
)
# Compatibility support for pre short Columns
if 'is_short' not in df.columns:
df['is_short'] = False
if 'leverage' not in df.columns:
df['leverage'] = 1.0
if 'enter_tag' not in df.columns:
df['enter_tag'] = df['buy_tag']
df = df.drop(['buy_tag'], axis=1)
if 'max_stake_amount' not in df.columns:
df['max_stake_amount'] = df['stake_amount']
if 'orders' not in df.columns:
df['orders'] = None
return df
def load_backtest_data(filename: Union[Path, str], strategy: Optional[str] = None) -> pd.DataFrame: def load_backtest_data(filename: Union[Path, str], strategy: Optional[str] = None) -> pd.DataFrame:
""" """
Load backtest data file. Load backtest data file.
@ -269,24 +296,7 @@ def load_backtest_data(filename: Union[Path, str], strategy: Optional[str] = Non
data = data['strategy'][strategy]['trades'] data = data['strategy'][strategy]['trades']
df = pd.DataFrame(data) df = pd.DataFrame(data)
if not df.empty: if not df.empty:
df['open_date'] = pd.to_datetime(df['open_date'], df = _load_backtest_data_df_compatibility(df)
utc=True,
infer_datetime_format=True
)
df['close_date'] = pd.to_datetime(df['close_date'],
utc=True,
infer_datetime_format=True
)
# Compatibility support for pre short Columns
if 'is_short' not in df.columns:
df['is_short'] = 0
if 'leverage' not in df.columns:
df['leverage'] = 1.0
if 'enter_tag' not in df.columns:
df['enter_tag'] = df['buy_tag']
df = df.drop(['buy_tag'], axis=1)
if 'orders' not in df.columns:
df['orders'] = None
else: else:
# old format - only with lists. # old format - only with lists.

View File

@ -1,4 +1,6 @@
import logging import logging
import math
from datetime import datetime
from typing import Dict, Tuple from typing import Dict, Tuple
import numpy as np import numpy as np
@ -190,3 +192,119 @@ def calculate_cagr(days_passed: int, starting_balance: float, final_balance: flo
:return: CAGR :return: CAGR
""" """
return (final_balance / starting_balance) ** (1 / (days_passed / 365)) - 1 return (final_balance / starting_balance) ** (1 / (days_passed / 365)) - 1
def calculate_expectancy(trades: pd.DataFrame) -> float:
"""
Calculate expectancy
:param trades: DataFrame containing trades (requires columns close_date and profit_ratio)
:return: expectancy
"""
if len(trades) == 0:
return 0
expectancy = 1
profit_sum = trades.loc[trades['profit_abs'] > 0, 'profit_abs'].sum()
loss_sum = abs(trades.loc[trades['profit_abs'] < 0, 'profit_abs'].sum())
nb_win_trades = len(trades.loc[trades['profit_abs'] > 0])
nb_loss_trades = len(trades.loc[trades['profit_abs'] < 0])
if (nb_win_trades > 0) and (nb_loss_trades > 0):
average_win = profit_sum / nb_win_trades
average_loss = loss_sum / nb_loss_trades
risk_reward_ratio = average_win / average_loss
winrate = nb_win_trades / len(trades)
expectancy = ((1 + risk_reward_ratio) * winrate) - 1
elif nb_win_trades == 0:
expectancy = 0
return expectancy
def calculate_sortino(trades: pd.DataFrame, min_date: datetime, max_date: datetime,
starting_balance: float) -> float:
"""
Calculate sortino
:param trades: DataFrame containing trades (requires columns profit_abs)
:return: sortino
"""
if (len(trades) == 0) or (min_date is None) or (max_date is None) or (min_date == max_date):
return 0
total_profit = trades['profit_abs'] / starting_balance
days_period = max(1, (max_date - min_date).days)
expected_returns_mean = total_profit.sum() / days_period
down_stdev = np.std(trades.loc[trades['profit_abs'] < 0, 'profit_abs'] / starting_balance)
if down_stdev != 0:
sortino_ratio = expected_returns_mean / down_stdev * np.sqrt(365)
else:
# Define high (negative) sortino ratio to be clear that this is NOT optimal.
sortino_ratio = -100
# print(expected_returns_mean, down_stdev, sortino_ratio)
return sortino_ratio
def calculate_sharpe(trades: pd.DataFrame, min_date: datetime, max_date: datetime,
starting_balance: float) -> float:
"""
Calculate sharpe
:param trades: DataFrame containing trades (requires column profit_abs)
:return: sharpe
"""
if (len(trades) == 0) or (min_date is None) or (max_date is None) or (min_date == max_date):
return 0
total_profit = trades['profit_abs'] / starting_balance
days_period = max(1, (max_date - min_date).days)
expected_returns_mean = total_profit.sum() / days_period
up_stdev = np.std(total_profit)
if up_stdev != 0:
sharp_ratio = expected_returns_mean / up_stdev * np.sqrt(365)
else:
# Define high (negative) sharpe ratio to be clear that this is NOT optimal.
sharp_ratio = -100
# print(expected_returns_mean, up_stdev, sharp_ratio)
return sharp_ratio
def calculate_calmar(trades: pd.DataFrame, min_date: datetime, max_date: datetime,
starting_balance: float) -> float:
"""
Calculate calmar
:param trades: DataFrame containing trades (requires columns close_date and profit_abs)
:return: calmar
"""
if (len(trades) == 0) or (min_date is None) or (max_date is None) or (min_date == max_date):
return 0
total_profit = trades['profit_abs'].sum() / starting_balance
days_period = max(1, (max_date - min_date).days)
# adding slippage of 0.1% per trade
# total_profit = total_profit - 0.0005
expected_returns_mean = total_profit / days_period * 100
# calculate max drawdown
try:
_, _, _, _, _, max_drawdown = calculate_max_drawdown(
trades, value_col="profit_abs", starting_balance=starting_balance
)
except ValueError:
max_drawdown = 0
if max_drawdown != 0:
calmar_ratio = expected_returns_mean / max_drawdown * math.sqrt(365)
else:
# Define high (negative) calmar ratio to be clear that this is NOT optimal.
calmar_ratio = -100
# print(expected_returns_mean, max_drawdown, calmar_ratio)
return calmar_ratio

View File

@ -31,7 +31,7 @@ class Binance(Exchange):
"ccxt_futures_name": "future" "ccxt_futures_name": "future"
} }
_ft_has_futures: Dict = { _ft_has_futures: Dict = {
"stoploss_order_types": {"limit": "limit", "market": "market"}, "stoploss_order_types": {"limit": "stop", "market": "stop_market"},
"tickers_have_price": False, "tickers_have_price": False,
} }

View File

@ -2035,8 +2035,8 @@ class Exchange:
# Fetch OHLCV asynchronously # Fetch OHLCV asynchronously
s = '(' + arrow.get(since_ms // 1000).isoformat() + ') ' if since_ms is not None else '' s = '(' + arrow.get(since_ms // 1000).isoformat() + ') ' if since_ms is not None else ''
logger.debug( logger.debug(
"Fetching pair %s, interval %s, since %s %s...", "Fetching pair %s, %s, interval %s, since %s %s...",
pair, timeframe, since_ms, s pair, candle_type, timeframe, since_ms, s
) )
params = deepcopy(self._ft_has.get('ohlcv_params', {})) params = deepcopy(self._ft_has.get('ohlcv_params', {}))
candle_limit = self.ohlcv_candle_limit( candle_limit = self.ohlcv_candle_limit(
@ -2050,11 +2050,12 @@ class Exchange:
limit=candle_limit, params=params) limit=candle_limit, params=params)
else: else:
# Funding rate # Funding rate
data = await self._api_async.fetch_funding_rate_history( data = await self._fetch_funding_rate_history(
pair, since=since_ms, pair=pair,
limit=candle_limit) timeframe=timeframe,
# Convert funding rate to candle pattern limit=candle_limit,
data = [[x['timestamp'], x['fundingRate'], 0, 0, 0, 0] for x in data] since_ms=since_ms,
)
# Some exchanges sort OHLCV in ASC order and others in DESC. # Some exchanges sort OHLCV in ASC order and others in DESC.
# Ex: Bittrex returns the list of OHLCV in ASC order (oldest first, newest last) # Ex: Bittrex returns the list of OHLCV in ASC order (oldest first, newest last)
# while GDAX returns the list of OHLCV in DESC order (newest first, oldest last) # while GDAX returns the list of OHLCV in DESC order (newest first, oldest last)
@ -2082,6 +2083,24 @@ class Exchange:
raise OperationalException(f'Could not fetch historical candle (OHLCV) data ' raise OperationalException(f'Could not fetch historical candle (OHLCV) data '
f'for pair {pair}. Message: {e}') from e f'for pair {pair}. Message: {e}') from e
async def _fetch_funding_rate_history(
self,
pair: str,
timeframe: str,
limit: int,
since_ms: Optional[int] = None,
) -> List[List]:
"""
Fetch funding rate history - used to selectively override this by subclasses.
"""
# Funding rate
data = await self._api_async.fetch_funding_rate_history(
pair, since=since_ms,
limit=limit)
# Convert funding rate to candle pattern
data = [[x['timestamp'], x['fundingRate'], 0, 0, 0, 0] for x in data]
return data
# Fetch historic trades # Fetch historic trades
@retrier_async @retrier_async
@ -2745,11 +2764,16 @@ class Exchange:
""" """
Important: Must be fetching data from cached values as this is used by backtesting! Important: Must be fetching data from cached values as this is used by backtesting!
PERPETUAL: PERPETUAL:
gateio: https://www.gate.io/help/futures/perpetual/22160/calculation-of-liquidation-price gateio: https://www.gate.io/help/futures/futures/27724/liquidation-price-bankruptcy-price
> Liquidation Price = (Entry Price ± Margin / Contract Multiplier / Size) /
[ 1 ± (Maintenance Margin Ratio + Taker Rate)]
Wherein, "+" or "-" depends on whether the contract goes long or short:
"-" for long, and "+" for short.
okex: https://www.okex.com/support/hc/en-us/articles/ okex: https://www.okex.com/support/hc/en-us/articles/
360053909592-VI-Introduction-to-the-isolated-mode-of-Single-Multi-currency-Portfolio-margin 360053909592-VI-Introduction-to-the-isolated-mode-of-Single-Multi-currency-Portfolio-margin
:param exchange_name: :param pair: Pair to calculate liquidation price for
:param open_rate: Entry price of position :param open_rate: Entry price of position
:param is_short: True if the trade is a short, false otherwise :param is_short: True if the trade is a short, false otherwise
:param amount: Absolute value of position size incl. leverage (in base currency) :param amount: Absolute value of position size incl. leverage (in base currency)
@ -2789,7 +2813,7 @@ class Exchange:
def get_maintenance_ratio_and_amt( def get_maintenance_ratio_and_amt(
self, self,
pair: str, pair: str,
nominal_value: float = 0.0, nominal_value: float,
) -> Tuple[float, Optional[float]]: ) -> Tuple[float, Optional[float]]:
""" """
Important: Must be fetching data from cached values as this is used by backtesting! Important: Must be fetching data from cached values as this is used by backtesting!

View File

@ -9,8 +9,9 @@ from tabulate import tabulate
from freqtrade.constants import (DATETIME_PRINT_FORMAT, LAST_BT_RESULT_FN, UNLIMITED_STAKE_AMOUNT, from freqtrade.constants import (DATETIME_PRINT_FORMAT, LAST_BT_RESULT_FN, UNLIMITED_STAKE_AMOUNT,
Config) Config)
from freqtrade.data.metrics import (calculate_cagr, calculate_csum, calculate_market_change, from freqtrade.data.metrics import (calculate_cagr, calculate_calmar, calculate_csum,
calculate_max_drawdown) calculate_expectancy, calculate_market_change,
calculate_max_drawdown, calculate_sharpe, calculate_sortino)
from freqtrade.misc import decimals_per_coin, file_dump_joblib, file_dump_json, round_coin_value from freqtrade.misc import decimals_per_coin, file_dump_joblib, file_dump_json, round_coin_value
from freqtrade.optimize.backtest_caching import get_backtest_metadata_filename from freqtrade.optimize.backtest_caching import get_backtest_metadata_filename
@ -448,6 +449,10 @@ def generate_strategy_stats(pairlist: List[str],
'profit_total_long_abs': results.loc[~results['is_short'], 'profit_abs'].sum(), 'profit_total_long_abs': results.loc[~results['is_short'], 'profit_abs'].sum(),
'profit_total_short_abs': results.loc[results['is_short'], 'profit_abs'].sum(), 'profit_total_short_abs': results.loc[results['is_short'], 'profit_abs'].sum(),
'cagr': calculate_cagr(backtest_days, start_balance, content['final_balance']), 'cagr': calculate_cagr(backtest_days, start_balance, content['final_balance']),
'expectancy': calculate_expectancy(results),
'sortino': calculate_sortino(results, min_date, max_date, start_balance),
'sharpe': calculate_sharpe(results, min_date, max_date, start_balance),
'calmar': calculate_calmar(results, min_date, max_date, start_balance),
'profit_factor': profit_factor, 'profit_factor': profit_factor,
'backtest_start': min_date.strftime(DATETIME_PRINT_FORMAT), 'backtest_start': min_date.strftime(DATETIME_PRINT_FORMAT),
'backtest_start_ts': int(min_date.timestamp() * 1000), 'backtest_start_ts': int(min_date.timestamp() * 1000),
@ -785,8 +790,13 @@ def text_table_add_metrics(strat_results: Dict) -> str:
strat_results['stake_currency'])), strat_results['stake_currency'])),
('Total profit %', f"{strat_results['profit_total']:.2%}"), ('Total profit %', f"{strat_results['profit_total']:.2%}"),
('CAGR %', f"{strat_results['cagr']:.2%}" if 'cagr' in strat_results else 'N/A'), ('CAGR %', f"{strat_results['cagr']:.2%}" if 'cagr' in strat_results else 'N/A'),
('Sortino', f"{strat_results['sortino']:.2f}" if 'sortino' in strat_results else 'N/A'),
('Sharpe', f"{strat_results['sharpe']:.2f}" if 'sharpe' in strat_results else 'N/A'),
('Calmar', f"{strat_results['calmar']:.2f}" if 'calmar' in strat_results else 'N/A'),
('Profit factor', f'{strat_results["profit_factor"]:.2f}' if 'profit_factor' ('Profit factor', f'{strat_results["profit_factor"]:.2f}' if 'profit_factor'
in strat_results else 'N/A'), in strat_results else 'N/A'),
('Expectancy', f"{strat_results['expectancy']:.2f}" if 'expectancy'
in strat_results else 'N/A'),
('Trades per day', strat_results['trades_per_day']), ('Trades per day', strat_results['trades_per_day']),
('Avg. daily profit %', ('Avg. daily profit %',
f"{(strat_results['profit_total'] / strat_results['backtest_days']):.2%}"), f"{(strat_results['profit_total'] / strat_results['backtest_days']):.2%}"),

View File

@ -109,11 +109,10 @@ def migrate_trades_and_orders_table(
else: else:
is_short = get_column_def(cols, 'is_short', '0') is_short = get_column_def(cols, 'is_short', '0')
# Margin Properties # Futures Properties
interest_rate = get_column_def(cols, 'interest_rate', '0.0') interest_rate = get_column_def(cols, 'interest_rate', '0.0')
# Futures properties
funding_fees = get_column_def(cols, 'funding_fees', '0.0') funding_fees = get_column_def(cols, 'funding_fees', '0.0')
max_stake_amount = get_column_def(cols, 'max_stake_amount', 'stake_amount')
# If ticker-interval existed use that, else null. # If ticker-interval existed use that, else null.
if has_column(cols, 'ticker_interval'): if has_column(cols, 'ticker_interval'):
@ -162,7 +161,8 @@ def migrate_trades_and_orders_table(
timeframe, open_trade_value, close_profit_abs, timeframe, open_trade_value, close_profit_abs,
trading_mode, leverage, liquidation_price, is_short, trading_mode, leverage, liquidation_price, is_short,
interest_rate, funding_fees, realized_profit, interest_rate, funding_fees, realized_profit,
amount_precision, price_precision, precision_mode, contract_size amount_precision, price_precision, precision_mode, contract_size,
max_stake_amount
) )
select id, lower(exchange), pair, {base_currency} base_currency, select id, lower(exchange), pair, {base_currency} base_currency,
{stake_currency} stake_currency, {stake_currency} stake_currency,
@ -190,7 +190,8 @@ def migrate_trades_and_orders_table(
{is_short} is_short, {interest_rate} interest_rate, {is_short} is_short, {interest_rate} interest_rate,
{funding_fees} funding_fees, {realized_profit} realized_profit, {funding_fees} funding_fees, {realized_profit} realized_profit,
{amount_precision} amount_precision, {price_precision} price_precision, {amount_precision} amount_precision, {price_precision} price_precision,
{precision_mode} precision_mode, {contract_size} contract_size {precision_mode} precision_mode, {contract_size} contract_size,
{max_stake_amount} max_stake_amount
from {trade_back_name} from {trade_back_name}
""")) """))
@ -310,8 +311,8 @@ def check_migrate(engine, decl_base, previous_tables) -> None:
# if ('orders' not in previous_tables # if ('orders' not in previous_tables
# or not has_column(cols_orders, 'funding_fee')): # or not has_column(cols_orders, 'funding_fee')):
migrating = False migrating = False
# if not has_column(cols_trades, 'contract_size'): # if not has_column(cols_orders, 'funding_fee'):
if not has_column(cols_orders, 'funding_fee'): if not has_column(cols_trades, 'max_stake_amount'):
migrating = True migrating = True
logger.info(f"Running database migration for trades - " logger.info(f"Running database migration for trades - "
f"backup: {table_back_name}, {order_table_bak_name}") f"backup: {table_back_name}, {order_table_bak_name}")

View File

@ -293,6 +293,7 @@ class LocalTrade():
close_profit: Optional[float] = None close_profit: Optional[float] = None
close_profit_abs: Optional[float] = None close_profit_abs: Optional[float] = None
stake_amount: float = 0.0 stake_amount: float = 0.0
max_stake_amount: float = 0.0
amount: float = 0.0 amount: float = 0.0
amount_requested: Optional[float] = None amount_requested: Optional[float] = None
open_date: datetime open_date: datetime
@ -469,8 +470,8 @@ class LocalTrade():
'amount': round(self.amount, 8), 'amount': round(self.amount, 8),
'amount_requested': round(self.amount_requested, 8) if self.amount_requested else None, 'amount_requested': round(self.amount_requested, 8) if self.amount_requested else None,
'stake_amount': round(self.stake_amount, 8), 'stake_amount': round(self.stake_amount, 8),
'max_stake_amount': round(self.max_stake_amount, 8) if self.max_stake_amount else None,
'strategy': self.strategy, 'strategy': self.strategy,
'buy_tag': self.enter_tag,
'enter_tag': self.enter_tag, 'enter_tag': self.enter_tag,
'timeframe': self.timeframe, 'timeframe': self.timeframe,
@ -507,7 +508,6 @@ class LocalTrade():
'profit_pct': round(self.close_profit * 100, 2) if self.close_profit else None, 'profit_pct': round(self.close_profit * 100, 2) if self.close_profit else None,
'profit_abs': self.close_profit_abs, 'profit_abs': self.close_profit_abs,
'sell_reason': self.exit_reason, # Deprecated
'exit_reason': self.exit_reason, 'exit_reason': self.exit_reason,
'exit_order_status': self.exit_order_status, 'exit_order_status': self.exit_order_status,
'stop_loss_abs': self.stop_loss, 'stop_loss_abs': self.stop_loss,
@ -876,6 +876,7 @@ class LocalTrade():
ZERO = FtPrecise(0.0) ZERO = FtPrecise(0.0)
current_amount = FtPrecise(0.0) current_amount = FtPrecise(0.0)
current_stake = FtPrecise(0.0) current_stake = FtPrecise(0.0)
max_stake_amount = FtPrecise(0.0)
total_stake = 0.0 # Total stake after all buy orders (does not subtract!) total_stake = 0.0 # Total stake after all buy orders (does not subtract!)
avg_price = FtPrecise(0.0) avg_price = FtPrecise(0.0)
close_profit = 0.0 close_profit = 0.0
@ -917,7 +918,9 @@ class LocalTrade():
exit_rate, amount=exit_amount, open_rate=avg_price) exit_rate, amount=exit_amount, open_rate=avg_price)
else: else:
total_stake = total_stake + self._calc_open_trade_value(tmp_amount, price) total_stake = total_stake + self._calc_open_trade_value(tmp_amount, price)
max_stake_amount += (tmp_amount * price)
self.funding_fees = funding_fees self.funding_fees = funding_fees
self.max_stake_amount = float(max_stake_amount)
if close_profit: if close_profit:
self.close_profit = close_profit self.close_profit = close_profit
@ -1169,6 +1172,7 @@ class Trade(_DECL_BASE, LocalTrade):
close_profit = Column(Float) close_profit = Column(Float)
close_profit_abs = Column(Float) close_profit_abs = Column(Float)
stake_amount = Column(Float, nullable=False) stake_amount = Column(Float, nullable=False)
max_stake_amount = Column(Float)
amount = Column(Float) amount = Column(Float)
amount_requested = Column(Float) amount_requested = Column(Float)
open_date = Column(DateTime, nullable=False, default=datetime.utcnow) open_date = Column(DateTime, nullable=False, default=datetime.utcnow)

View File

@ -135,7 +135,7 @@ class VolumePairList(IPairList):
filtered_tickers = [ filtered_tickers = [
v for k, v in tickers.items() v for k, v in tickers.items()
if (self._exchange.get_pair_quote_currency(k) == self._stake_currency if (self._exchange.get_pair_quote_currency(k) == self._stake_currency
and (self._use_range or v[self._sort_key] is not None) and (self._use_range or v.get(self._sort_key) is not None)
and v['symbol'] in _pairlist)] and v['symbol'] in _pairlist)]
pairlist = [s['symbol'] for s in filtered_tickers] pairlist = [s['symbol'] for s in filtered_tickers]
else: else:

View File

@ -217,8 +217,8 @@ class TradeSchema(BaseModel):
amount: float amount: float
amount_requested: float amount_requested: float
stake_amount: float stake_amount: float
max_stake_amount: Optional[float]
strategy: str strategy: str
buy_tag: Optional[str] # Deprecated
enter_tag: Optional[str] enter_tag: Optional[str]
timeframe: int timeframe: int
fee_open: Optional[float] fee_open: Optional[float]
@ -243,7 +243,6 @@ class TradeSchema(BaseModel):
profit_pct: Optional[float] profit_pct: Optional[float]
profit_abs: Optional[float] profit_abs: Optional[float]
profit_fiat: Optional[float] profit_fiat: Optional[float]
sell_reason: Optional[str] # Deprecated
exit_reason: Optional[str] exit_reason: Optional[str]
exit_order_status: Optional[str] exit_order_status: Optional[str]
stop_loss_abs: Optional[float] stop_loss_abs: Optional[float]

View File

@ -27,7 +27,7 @@ class FreqaiExampleStrategy(IStrategy):
plot_config = { plot_config = {
"main_plot": {}, "main_plot": {},
"subplots": { "subplots": {
"prediction": {"prediction": {"color": "blue"}}, "&-s_close": {"prediction": {"color": "blue"}},
"do_predict": { "do_predict": {
"do_predict": {"color": "brown"}, "do_predict": {"color": "brown"},
}, },
@ -184,7 +184,8 @@ class FreqaiExampleStrategy(IStrategy):
# If user wishes to use multiple targets, they can add more by # If user wishes to use multiple targets, they can add more by
# appending more columns with '&'. User should keep in mind that multi targets # appending more columns with '&'. User should keep in mind that multi targets
# requires a multioutput prediction model such as # requires a multioutput prediction model such as
# templates/CatboostPredictionMultiModel.py, # freqai/prediction_models/CatboostRegressorMultiTarget.py,
# freqtrade trade --freqaimodel CatboostRegressorMultiTarget
# df["&-s_range"] = ( # df["&-s_range"] = (
# df["close"] # df["close"]

View File

@ -10,24 +10,24 @@ coveralls==3.3.1
flake8==6.0.0 flake8==6.0.0
flake8-tidy-imports==4.8.0 flake8-tidy-imports==4.8.0
mypy==0.991 mypy==0.991
pre-commit==2.20.0 pre-commit==2.21.0
pytest==7.2.0 pytest==7.2.0
pytest-asyncio==0.20.3 pytest-asyncio==0.20.3
pytest-cov==4.0.0 pytest-cov==4.0.0
pytest-mock==3.10.0 pytest-mock==3.10.0
pytest-random-order==1.1.0 pytest-random-order==1.1.0
isort==5.11.3 isort==5.11.4
# For datetime mocking # For datetime mocking
time-machine==2.8.2 time-machine==2.9.0
# fastapi testing # fastapi testing
httpx==0.23.1 httpx==0.23.1
# Convert jupyter notebooks to markdown documents # Convert jupyter notebooks to markdown documents
nbconvert==7.2.6 nbconvert==7.2.7
# mypy types # mypy types
types-cachetools==5.2.1 types-cachetools==5.2.1
types-filelock==3.2.7 types-filelock==3.2.7
types-requests==2.28.11.5 types-requests==2.28.11.7
types-tabulate==0.9.0.0 types-tabulate==0.9.0.0
types-python-dateutil==2.8.19.5 types-python-dateutil==2.8.19.5

View File

@ -5,5 +5,5 @@
scipy==1.9.3 scipy==1.9.3
scikit-learn==1.1.3 scikit-learn==1.1.3
scikit-optimize==0.9.0 scikit-optimize==0.9.0
filelock==3.8.2 filelock==3.9.0
progressbar2==4.2.0 progressbar2==4.2.0

View File

@ -1,8 +1,8 @@
numpy==1.23.5 numpy==1.24.1
pandas==1.5.2 pandas==1.5.2
pandas-ta==0.3.14b pandas-ta==0.3.14b
ccxt==2.4.27 ccxt==2.4.60
# Pin cryptography for now due to rust build errors with piwheels # Pin cryptography for now due to rust build errors with piwheels
cryptography==38.0.1; platform_machine == 'armv7l' cryptography==38.0.1; platform_machine == 'armv7l'
cryptography==38.0.4; platform_machine != 'armv7l' cryptography==38.0.4; platform_machine != 'armv7l'
@ -19,7 +19,7 @@ technical==1.3.0
tabulate==0.9.0 tabulate==0.9.0
pycoingecko==3.1.0 pycoingecko==3.1.0
jinja2==3.1.2 jinja2==3.1.2
tables==3.7.0 tables==3.8.0
blosc==1.11.1 blosc==1.11.1
joblib==1.2.0 joblib==1.2.0
pyarrow==10.0.1; platform_machine != 'armv7l' pyarrow==10.0.1; platform_machine != 'armv7l'
@ -37,7 +37,7 @@ sdnotify==0.3.2
# API Server # API Server
fastapi==0.88.0 fastapi==0.88.0
pydantic==1.10.2 pydantic==1.10.4
uvicorn==0.20.0 uvicorn==0.20.0
pyjwt==2.6.0 pyjwt==2.6.0
aiofiles==22.1.0 aiofiles==22.1.0

View File

@ -25,6 +25,11 @@ freqai_rl = [
'sb3-contrib' 'sb3-contrib'
] ]
hdf5 = [
'tables',
'blosc',
]
develop = [ develop = [
'coveralls', 'coveralls',
'flake8', 'flake8',
@ -44,7 +49,7 @@ jupyter = [
'nbconvert', 'nbconvert',
] ]
all_extra = plot + develop + jupyter + hyperopt + freqai + freqai_rl all_extra = plot + develop + jupyter + hyperopt + hdf5 + freqai + freqai_rl
setup( setup(
tests_require=[ tests_require=[
@ -78,8 +83,6 @@ setup(
'prompt-toolkit', 'prompt-toolkit',
'numpy', 'numpy',
'pandas', 'pandas',
'tables',
'blosc',
'joblib>=1.2.0', 'joblib>=1.2.0',
'pyarrow; platform_machine != "armv7l"', 'pyarrow; platform_machine != "armv7l"',
'fastapi', 'fastapi',
@ -97,6 +100,7 @@ setup(
'plot': plot, 'plot': plot,
'jupyter': jupyter, 'jupyter': jupyter,
'hyperopt': hyperopt, 'hyperopt': hyperopt,
'hdf5': hdf5,
'freqai': freqai, 'freqai': freqai,
'freqai_rl': freqai_rl, 'freqai_rl': freqai_rl,
'all': all_extra, 'all': all_extra,

View File

@ -1529,7 +1529,7 @@ def test_backtesting_show(mocker, testdatadir, capsys):
args = [ args = [
"backtesting-show", "backtesting-show",
"--export-filename", "--export-filename",
f"{testdatadir / 'backtest_results/backtest-result_new.json'}", f"{testdatadir / 'backtest_results/backtest-result.json'}",
"--show-pair-list" "--show-pair-list"
] ]
pargs = get_args(args) pargs = get_args(args)

View File

@ -12,9 +12,11 @@ from freqtrade.data.btanalysis import (BT_DATA_COLUMNS, analyze_trade_parallelis
get_latest_hyperopt_file, load_backtest_data, get_latest_hyperopt_file, load_backtest_data,
load_backtest_metadata, load_trades, load_trades_from_db) load_backtest_metadata, load_trades, load_trades_from_db)
from freqtrade.data.history import load_data, load_pair_history from freqtrade.data.history import load_data, load_pair_history
from freqtrade.data.metrics import (calculate_cagr, calculate_csum, calculate_market_change, from freqtrade.data.metrics import (calculate_cagr, calculate_calmar, calculate_csum,
calculate_max_drawdown, calculate_underwater, calculate_expectancy, calculate_market_change,
combine_dataframes_with_mean, create_cum_profit) calculate_max_drawdown, calculate_sharpe, calculate_sortino,
calculate_underwater, combine_dataframes_with_mean,
create_cum_profit)
from freqtrade.exceptions import OperationalException from freqtrade.exceptions import OperationalException
from tests.conftest import CURRENT_TEST_STRATEGY, create_mock_trades from tests.conftest import CURRENT_TEST_STRATEGY, create_mock_trades
from tests.conftest_trades import MOCK_TRADE_COUNT from tests.conftest_trades import MOCK_TRADE_COUNT
@ -30,10 +32,10 @@ def test_get_latest_backtest_filename(testdatadir, mocker):
testdir_bt = testdatadir / "backtest_results" testdir_bt = testdatadir / "backtest_results"
res = get_latest_backtest_filename(testdir_bt) res = get_latest_backtest_filename(testdir_bt)
assert res == 'backtest-result_new.json' assert res == 'backtest-result.json'
res = get_latest_backtest_filename(str(testdir_bt)) res = get_latest_backtest_filename(str(testdir_bt))
assert res == 'backtest-result_new.json' assert res == 'backtest-result.json'
mocker.patch("freqtrade.data.btanalysis.json_load", return_value={}) mocker.patch("freqtrade.data.btanalysis.json_load", return_value={})
@ -81,7 +83,7 @@ def test_load_backtest_data_old_format(testdatadir, mocker):
def test_load_backtest_data_new_format(testdatadir): def test_load_backtest_data_new_format(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
assert isinstance(bt_data, DataFrame) assert isinstance(bt_data, DataFrame)
assert set(bt_data.columns) == set(BT_DATA_COLUMNS) assert set(bt_data.columns) == set(BT_DATA_COLUMNS)
@ -182,7 +184,7 @@ def test_extract_trades_of_period(testdatadir):
def test_analyze_trade_parallelism(testdatadir): def test_analyze_trade_parallelism(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
res = analyze_trade_parallelism(bt_data, "5m") res = analyze_trade_parallelism(bt_data, "5m")
@ -256,7 +258,7 @@ def test_combine_dataframes_with_mean_no_data(testdatadir):
def test_create_cum_profit(testdatadir): def test_create_cum_profit(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
timerange = TimeRange.parse_timerange("20180110-20180112") timerange = TimeRange.parse_timerange("20180110-20180112")
@ -268,11 +270,11 @@ def test_create_cum_profit(testdatadir):
"cum_profits", timeframe="5m") "cum_profits", timeframe="5m")
assert "cum_profits" in cum_profits.columns assert "cum_profits" in cum_profits.columns
assert cum_profits.iloc[0]['cum_profits'] == 0 assert cum_profits.iloc[0]['cum_profits'] == 0
assert pytest.approx(cum_profits.iloc[-1]['cum_profits']) == 8.723007518796964e-06 assert pytest.approx(cum_profits.iloc[-1]['cum_profits']) == 9.0225563e-05
def test_create_cum_profit1(testdatadir): def test_create_cum_profit1(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
# Move close-time to "off" the candle, to make sure the logic still works # Move close-time to "off" the candle, to make sure the logic still works
bt_data['close_date'] = bt_data.loc[:, 'close_date'] + DateOffset(seconds=20) bt_data['close_date'] = bt_data.loc[:, 'close_date'] + DateOffset(seconds=20)
@ -286,7 +288,7 @@ def test_create_cum_profit1(testdatadir):
"cum_profits", timeframe="5m") "cum_profits", timeframe="5m")
assert "cum_profits" in cum_profits.columns assert "cum_profits" in cum_profits.columns
assert cum_profits.iloc[0]['cum_profits'] == 0 assert cum_profits.iloc[0]['cum_profits'] == 0
assert pytest.approx(cum_profits.iloc[-1]['cum_profits']) == 8.723007518796964e-06 assert pytest.approx(cum_profits.iloc[-1]['cum_profits']) == 9.0225563e-05
with pytest.raises(ValueError, match='Trade dataframe empty.'): with pytest.raises(ValueError, match='Trade dataframe empty.'):
create_cum_profit(df.set_index('date'), bt_data[bt_data["pair"] == 'NOTAPAIR'], create_cum_profit(df.set_index('date'), bt_data[bt_data["pair"] == 'NOTAPAIR'],
@ -294,18 +296,18 @@ def test_create_cum_profit1(testdatadir):
def test_calculate_max_drawdown(testdatadir): def test_calculate_max_drawdown(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
_, hdate, lowdate, hval, lval, drawdown = calculate_max_drawdown( _, hdate, lowdate, hval, lval, drawdown = calculate_max_drawdown(
bt_data, value_col="profit_abs") bt_data, value_col="profit_abs")
assert isinstance(drawdown, float) assert isinstance(drawdown, float)
assert pytest.approx(drawdown) == 0.12071099 assert pytest.approx(drawdown) == 0.29753914
assert isinstance(hdate, Timestamp) assert isinstance(hdate, Timestamp)
assert isinstance(lowdate, Timestamp) assert isinstance(lowdate, Timestamp)
assert isinstance(hval, float) assert isinstance(hval, float)
assert isinstance(lval, float) assert isinstance(lval, float)
assert hdate == Timestamp('2018-01-25 01:30:00', tz='UTC') assert hdate == Timestamp('2018-01-16 19:30:00', tz='UTC')
assert lowdate == Timestamp('2018-01-25 03:50:00', tz='UTC') assert lowdate == Timestamp('2018-01-16 22:25:00', tz='UTC')
underwater = calculate_underwater(bt_data) underwater = calculate_underwater(bt_data)
assert isinstance(underwater, DataFrame) assert isinstance(underwater, DataFrame)
@ -318,14 +320,15 @@ def test_calculate_max_drawdown(testdatadir):
def test_calculate_csum(testdatadir): def test_calculate_csum(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
csum_min, csum_max = calculate_csum(bt_data) csum_min, csum_max = calculate_csum(bt_data)
assert isinstance(csum_min, float) assert isinstance(csum_min, float)
assert isinstance(csum_max, float) assert isinstance(csum_max, float)
assert csum_min < 0.01 assert csum_min < csum_max
assert csum_max > 0.02 assert csum_min < 0.0001
assert csum_max > 0.0002
csum_min1, csum_max1 = calculate_csum(bt_data, 5) csum_min1, csum_max1 = calculate_csum(bt_data, 5)
assert csum_min1 == csum_min + 5 assert csum_min1 == csum_min + 5
@ -335,6 +338,69 @@ def test_calculate_csum(testdatadir):
csum_min, csum_max = calculate_csum(DataFrame()) csum_min, csum_max = calculate_csum(DataFrame())
def test_calculate_expectancy(testdatadir):
filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename)
expectancy = calculate_expectancy(DataFrame())
assert expectancy == 0.0
expectancy = calculate_expectancy(bt_data)
assert isinstance(expectancy, float)
assert pytest.approx(expectancy) == 0.07151374226574791
def test_calculate_sortino(testdatadir):
filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename)
sortino = calculate_sortino(DataFrame(), None, None, 0)
assert sortino == 0.0
sortino = calculate_sortino(
bt_data,
bt_data['open_date'].min(),
bt_data['close_date'].max(),
0.01,
)
assert isinstance(sortino, float)
assert pytest.approx(sortino) == 35.17722
def test_calculate_sharpe(testdatadir):
filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename)
sharpe = calculate_sharpe(DataFrame(), None, None, 0)
assert sharpe == 0.0
sharpe = calculate_sharpe(
bt_data,
bt_data['open_date'].min(),
bt_data['close_date'].max(),
0.01,
)
assert isinstance(sharpe, float)
assert pytest.approx(sharpe) == 44.5078669
def test_calculate_calmar(testdatadir):
filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename)
calmar = calculate_calmar(DataFrame(), None, None, 0)
assert calmar == 0.0
calmar = calculate_calmar(
bt_data,
bt_data['open_date'].min(),
bt_data['close_date'].max(),
0.01,
)
assert isinstance(calmar, float)
assert pytest.approx(calmar) == 559.040508
@pytest.mark.parametrize('start,end,days, expected', [ @pytest.mark.parametrize('start,end,days, expected', [
(64900, 176000, 3 * 365, 0.3945), (64900, 176000, 3 * 365, 0.3945),
(64900, 176000, 365, 1.7119), (64900, 176000, 365, 1.7119),

View File

@ -23,7 +23,7 @@ from tests.exchange.test_exchange import ccxt_exceptionhandlers
def test_stoploss_order_binance(default_conf, mocker, limitratio, expected, side, trademode): def test_stoploss_order_binance(default_conf, mocker, limitratio, expected, side, trademode):
api_mock = MagicMock() api_mock = MagicMock()
order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6)) order_id = 'test_prod_buy_{}'.format(randint(0, 10 ** 6))
order_type = 'stop_loss_limit' if trademode == TradingMode.SPOT else 'limit' order_type = 'stop_loss_limit' if trademode == TradingMode.SPOT else 'stop'
api_mock.create_order = MagicMock(return_value={ api_mock.create_order = MagicMock(return_value={
'id': order_id, 'id': order_id,

View File

@ -8,16 +8,19 @@ suitable to run with freqtrade.
from copy import deepcopy from copy import deepcopy
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
from typing import Tuple
import pytest import pytest
from freqtrade.enums import CandleType from freqtrade.enums import CandleType
from freqtrade.exchange import timeframe_to_minutes, timeframe_to_prev_date from freqtrade.exchange import timeframe_to_minutes, timeframe_to_prev_date
from freqtrade.exchange.exchange import timeframe_to_msecs from freqtrade.exchange.exchange import Exchange, timeframe_to_msecs
from freqtrade.resolvers.exchange_resolver import ExchangeResolver from freqtrade.resolvers.exchange_resolver import ExchangeResolver
from tests.conftest import get_default_conf_usdt from tests.conftest import get_default_conf_usdt
EXCHANGE_FIXTURE_TYPE = Tuple[Exchange, str]
# Exchanges that should be tested # Exchanges that should be tested
EXCHANGES = { EXCHANGES = {
'bittrex': { 'bittrex': {
@ -141,19 +144,19 @@ def exchange_futures(request, exchange_conf, class_mocker):
@pytest.mark.longrun @pytest.mark.longrun
class TestCCXTExchange(): class TestCCXTExchange():
def test_load_markets(self, exchange): def test_load_markets(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
markets = exchange.markets markets = exch.markets
assert pair in markets assert pair in markets
assert isinstance(markets[pair], dict) assert isinstance(markets[pair], dict)
assert exchange.market_is_spot(markets[pair]) assert exch.market_is_spot(markets[pair])
def test_has_validations(self, exchange): def test_has_validations(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
exchange.validate_ordertypes({ exch.validate_ordertypes({
'entry': 'limit', 'entry': 'limit',
'exit': 'limit', 'exit': 'limit',
'stoploss': 'limit', 'stoploss': 'limit',
@ -162,13 +165,13 @@ class TestCCXTExchange():
if exchangename == 'gateio': if exchangename == 'gateio':
# gateio doesn't have market orders on spot # gateio doesn't have market orders on spot
return return
exchange.validate_ordertypes({ exch.validate_ordertypes({
'entry': 'market', 'entry': 'market',
'exit': 'market', 'exit': 'market',
'stoploss': 'market', 'stoploss': 'market',
}) })
def test_load_markets_futures(self, exchange_futures): def test_load_markets_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures exchange, exchangename = exchange_futures
if not exchange: if not exchange:
# exchange_futures only returns values for supported exchanges # exchange_futures only returns values for supported exchanges
@ -181,11 +184,11 @@ class TestCCXTExchange():
assert exchange.market_is_future(markets[pair]) assert exchange.market_is_future(markets[pair])
def test_ccxt_fetch_tickers(self, exchange): def test_ccxt_fetch_tickers(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
tickers = exchange.get_tickers() tickers = exch.get_tickers()
assert pair in tickers assert pair in tickers
assert 'ask' in tickers[pair] assert 'ask' in tickers[pair]
assert tickers[pair]['ask'] is not None assert tickers[pair]['ask'] is not None
@ -195,11 +198,11 @@ class TestCCXTExchange():
if EXCHANGES[exchangename].get('hasQuoteVolume'): if EXCHANGES[exchangename].get('hasQuoteVolume'):
assert tickers[pair]['quoteVolume'] is not None assert tickers[pair]['quoteVolume'] is not None
def test_ccxt_fetch_ticker(self, exchange): def test_ccxt_fetch_ticker(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
ticker = exchange.fetch_ticker(pair) ticker = exch.fetch_ticker(pair)
assert 'ask' in ticker assert 'ask' in ticker
assert ticker['ask'] is not None assert ticker['ask'] is not None
assert 'bid' in ticker assert 'bid' in ticker
@ -208,21 +211,21 @@ class TestCCXTExchange():
if EXCHANGES[exchangename].get('hasQuoteVolume'): if EXCHANGES[exchangename].get('hasQuoteVolume'):
assert ticker['quoteVolume'] is not None assert ticker['quoteVolume'] is not None
def test_ccxt_fetch_l2_orderbook(self, exchange): def test_ccxt_fetch_l2_orderbook(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
l2 = exchange.fetch_l2_order_book(pair) l2 = exch.fetch_l2_order_book(pair)
assert 'asks' in l2 assert 'asks' in l2
assert 'bids' in l2 assert 'bids' in l2
assert len(l2['asks']) >= 1 assert len(l2['asks']) >= 1
assert len(l2['bids']) >= 1 assert len(l2['bids']) >= 1
l2_limit_range = exchange._ft_has['l2_limit_range'] l2_limit_range = exch._ft_has['l2_limit_range']
l2_limit_range_required = exchange._ft_has['l2_limit_range_required'] l2_limit_range_required = exch._ft_has['l2_limit_range_required']
if exchangename == 'gateio': if exchangename == 'gateio':
# TODO: Gateio is unstable here at the moment, ignoring the limit partially. # TODO: Gateio is unstable here at the moment, ignoring the limit partially.
return return
for val in [1, 2, 5, 25, 100]: for val in [1, 2, 5, 25, 100]:
l2 = exchange.fetch_l2_order_book(pair, val) l2 = exch.fetch_l2_order_book(pair, val)
if not l2_limit_range or val in l2_limit_range: if not l2_limit_range or val in l2_limit_range:
if val > 50: if val > 50:
# Orderbooks are not always this deep. # Orderbooks are not always this deep.
@ -232,7 +235,7 @@ class TestCCXTExchange():
assert len(l2['asks']) == val assert len(l2['asks']) == val
assert len(l2['bids']) == val assert len(l2['bids']) == val
else: else:
next_limit = exchange.get_next_limit_in_list( next_limit = exch.get_next_limit_in_list(
val, l2_limit_range, l2_limit_range_required) val, l2_limit_range, l2_limit_range_required)
if next_limit is None: if next_limit is None:
assert len(l2['asks']) > 100 assert len(l2['asks']) > 100
@ -245,23 +248,23 @@ class TestCCXTExchange():
assert len(l2['asks']) == next_limit assert len(l2['asks']) == next_limit
assert len(l2['asks']) == next_limit assert len(l2['asks']) == next_limit
def test_ccxt_fetch_ohlcv(self, exchange): def test_ccxt_fetch_ohlcv(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
timeframe = EXCHANGES[exchangename]['timeframe'] timeframe = EXCHANGES[exchangename]['timeframe']
pair_tf = (pair, timeframe, CandleType.SPOT) pair_tf = (pair, timeframe, CandleType.SPOT)
ohlcv = exchange.refresh_latest_ohlcv([pair_tf]) ohlcv = exch.refresh_latest_ohlcv([pair_tf])
assert isinstance(ohlcv, dict) assert isinstance(ohlcv, dict)
assert len(ohlcv[pair_tf]) == len(exchange.klines(pair_tf)) assert len(ohlcv[pair_tf]) == len(exch.klines(pair_tf))
# assert len(exchange.klines(pair_tf)) > 200 # assert len(exch.klines(pair_tf)) > 200
# Assume 90% uptime ... # Assume 90% uptime ...
assert len(exchange.klines(pair_tf)) > exchange.ohlcv_candle_limit( assert len(exch.klines(pair_tf)) > exch.ohlcv_candle_limit(
timeframe, CandleType.SPOT) * 0.90 timeframe, CandleType.SPOT) * 0.90
# Check if last-timeframe is within the last 2 intervals # Check if last-timeframe is within the last 2 intervals
now = datetime.now(timezone.utc) - timedelta(minutes=(timeframe_to_minutes(timeframe) * 2)) now = datetime.now(timezone.utc) - timedelta(minutes=(timeframe_to_minutes(timeframe) * 2))
assert exchange.klines(pair_tf).iloc[-1]['date'] >= timeframe_to_prev_date(timeframe, now) assert exch.klines(pair_tf).iloc[-1]['date'] >= timeframe_to_prev_date(timeframe, now)
def ccxt__async_get_candle_history(self, exchange, exchangename, pair, timeframe, candle_type): def ccxt__async_get_candle_history(self, exchange, exchangename, pair, timeframe, candle_type):
@ -289,17 +292,17 @@ class TestCCXTExchange():
assert len(candles) >= min(candle_count, candle_count1) assert len(candles) >= min(candle_count, candle_count1)
assert candles[0][0] == since_ms or (since_ms + timeframe_ms) assert candles[0][0] == since_ms or (since_ms + timeframe_ms)
def test_ccxt__async_get_candle_history(self, exchange): def test_ccxt__async_get_candle_history(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exc, exchangename = exchange
# For some weired reason, this test returns random lengths for bittrex. # For some weired reason, this test returns random lengths for bittrex.
if not exchange._ft_has['ohlcv_has_history'] or exchangename in ('bittrex'): if not exc._ft_has['ohlcv_has_history'] or exchangename in ('bittrex'):
return return
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
timeframe = EXCHANGES[exchangename]['timeframe'] timeframe = EXCHANGES[exchangename]['timeframe']
self.ccxt__async_get_candle_history( self.ccxt__async_get_candle_history(
exchange, exchangename, pair, timeframe, CandleType.SPOT) exc, exchangename, pair, timeframe, CandleType.SPOT)
def test_ccxt__async_get_candle_history_futures(self, exchange_futures): def test_ccxt__async_get_candle_history_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures exchange, exchangename = exchange_futures
if not exchange: if not exchange:
# exchange_futures only returns values for supported exchanges # exchange_futures only returns values for supported exchanges
@ -309,7 +312,7 @@ class TestCCXTExchange():
self.ccxt__async_get_candle_history( self.ccxt__async_get_candle_history(
exchange, exchangename, pair, timeframe, CandleType.FUTURES) exchange, exchangename, pair, timeframe, CandleType.FUTURES)
def test_ccxt_fetch_funding_rate_history(self, exchange_futures): def test_ccxt_fetch_funding_rate_history(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures exchange, exchangename = exchange_futures
if not exchange: if not exchange:
# exchange_futures only returns values for supported exchanges # exchange_futures only returns values for supported exchanges
@ -347,7 +350,7 @@ class TestCCXTExchange():
(rate['open'].min() != rate['open'].max()) (rate['open'].min() != rate['open'].max())
) )
def test_ccxt_fetch_mark_price_history(self, exchange_futures): def test_ccxt_fetch_mark_price_history(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures exchange, exchangename = exchange_futures
if not exchange: if not exchange:
# exchange_futures only returns values for supported exchanges # exchange_futures only returns values for supported exchanges
@ -371,7 +374,7 @@ class TestCCXTExchange():
assert mark_candles[mark_candles['date'] == prev_hour].iloc[0]['open'] != 0.0 assert mark_candles[mark_candles['date'] == prev_hour].iloc[0]['open'] != 0.0
assert mark_candles[mark_candles['date'] == this_hour].iloc[0]['open'] != 0.0 assert mark_candles[mark_candles['date'] == this_hour].iloc[0]['open'] != 0.0
def test_ccxt__calculate_funding_fees(self, exchange_futures): def test_ccxt__calculate_funding_fees(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange_futures exchange, exchangename = exchange_futures
if not exchange: if not exchange:
# exchange_futures only returns values for supported exchanges # exchange_futures only returns values for supported exchanges
@ -387,16 +390,16 @@ class TestCCXTExchange():
# TODO: tests fetch_trades (?) # TODO: tests fetch_trades (?)
def test_ccxt_get_fee(self, exchange): def test_ccxt_get_fee(self, exchange: EXCHANGE_FIXTURE_TYPE):
exchange, exchangename = exchange exch, exchangename = exchange
pair = EXCHANGES[exchangename]['pair'] pair = EXCHANGES[exchangename]['pair']
threshold = 0.01 threshold = 0.01
assert 0 < exchange.get_fee(pair, 'limit', 'buy') < threshold assert 0 < exch.get_fee(pair, 'limit', 'buy') < threshold
assert 0 < exchange.get_fee(pair, 'limit', 'sell') < threshold assert 0 < exch.get_fee(pair, 'limit', 'sell') < threshold
assert 0 < exchange.get_fee(pair, 'market', 'buy') < threshold assert 0 < exch.get_fee(pair, 'market', 'buy') < threshold
assert 0 < exchange.get_fee(pair, 'market', 'sell') < threshold assert 0 < exch.get_fee(pair, 'market', 'sell') < threshold
def test_ccxt_get_max_leverage_spot(self, exchange): def test_ccxt_get_max_leverage_spot(self, exchange: EXCHANGE_FIXTURE_TYPE):
spot, spot_name = exchange spot, spot_name = exchange
if spot: if spot:
leverage_in_market_spot = EXCHANGES[spot_name].get('leverage_in_spot_market') leverage_in_market_spot = EXCHANGES[spot_name].get('leverage_in_spot_market')
@ -406,7 +409,7 @@ class TestCCXTExchange():
assert (isinstance(spot_leverage, float) or isinstance(spot_leverage, int)) assert (isinstance(spot_leverage, float) or isinstance(spot_leverage, int))
assert spot_leverage >= 1.0 assert spot_leverage >= 1.0
def test_ccxt_get_max_leverage_futures(self, exchange_futures): def test_ccxt_get_max_leverage_futures(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures futures, futures_name = exchange_futures
if futures: if futures:
leverage_tiers_public = EXCHANGES[futures_name].get('leverage_tiers_public') leverage_tiers_public = EXCHANGES[futures_name].get('leverage_tiers_public')
@ -419,7 +422,7 @@ class TestCCXTExchange():
assert (isinstance(futures_leverage, float) or isinstance(futures_leverage, int)) assert (isinstance(futures_leverage, float) or isinstance(futures_leverage, int))
assert futures_leverage >= 1.0 assert futures_leverage >= 1.0
def test_ccxt_get_contract_size(self, exchange_futures): def test_ccxt_get_contract_size(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures futures, futures_name = exchange_futures
if futures: if futures:
futures_pair = EXCHANGES[futures_name].get( futures_pair = EXCHANGES[futures_name].get(
@ -430,7 +433,7 @@ class TestCCXTExchange():
assert (isinstance(contract_size, float) or isinstance(contract_size, int)) assert (isinstance(contract_size, float) or isinstance(contract_size, int))
assert contract_size >= 0.0 assert contract_size >= 0.0
def test_ccxt_load_leverage_tiers(self, exchange_futures): def test_ccxt_load_leverage_tiers(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures futures, futures_name = exchange_futures
if futures and EXCHANGES[futures_name].get('leverage_tiers_public'): if futures and EXCHANGES[futures_name].get('leverage_tiers_public'):
leverage_tiers = futures.load_leverage_tiers() leverage_tiers = futures.load_leverage_tiers()
@ -463,7 +466,7 @@ class TestCCXTExchange():
oldminNotional = tier['minNotional'] oldminNotional = tier['minNotional']
oldmaxNotional = tier['maxNotional'] oldmaxNotional = tier['maxNotional']
def test_ccxt_dry_run_liquidation_price(self, exchange_futures): def test_ccxt_dry_run_liquidation_price(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures futures, futures_name = exchange_futures
if futures and EXCHANGES[futures_name].get('leverage_tiers_public'): if futures and EXCHANGES[futures_name].get('leverage_tiers_public'):
@ -494,7 +497,7 @@ class TestCCXTExchange():
assert (isinstance(liquidation_price, float)) assert (isinstance(liquidation_price, float))
assert liquidation_price >= 0.0 assert liquidation_price >= 0.0
def test_ccxt_get_max_pair_stake_amount(self, exchange_futures): def test_ccxt_get_max_pair_stake_amount(self, exchange_futures: EXCHANGE_FIXTURE_TYPE):
futures, futures_name = exchange_futures futures, futures_name = exchange_futures
if futures: if futures:
futures_pair = EXCHANGES[futures_name].get( futures_pair = EXCHANGES[futures_name].get(

View File

@ -1,5 +1,6 @@
import pytest import pytest
from freqtrade.exceptions import OperationalException
from freqtrade.leverage import interest from freqtrade.leverage import interest
from freqtrade.util import FtPrecise from freqtrade.util import FtPrecise
@ -29,3 +30,13 @@ def test_interest(exchange, interest_rate, hours, expected):
rate=FtPrecise(interest_rate), rate=FtPrecise(interest_rate),
hours=hours hours=hours
))) == expected ))) == expected
def test_interest_exception():
with pytest.raises(OperationalException, match=r"Leverage not available on .* with freqtrade"):
interest(
exchange_name='bitmex',
borrowed=FtPrecise(60.0),
rate=FtPrecise(0.0005),
hours=ten_mins
)

View File

@ -710,6 +710,7 @@ def test_backtest_one(default_conf, fee, mocker, testdatadir) -> None:
expected = pd.DataFrame( expected = pd.DataFrame(
{'pair': [pair, pair], {'pair': [pair, pair],
'stake_amount': [0.001, 0.001], 'stake_amount': [0.001, 0.001],
'max_stake_amount': [0.001, 0.001],
'amount': [0.00957442, 0.0097064], 'amount': [0.00957442, 0.0097064],
'open_date': pd.to_datetime([Arrow(2018, 1, 29, 18, 40, 0).datetime, 'open_date': pd.to_datetime([Arrow(2018, 1, 29, 18, 40, 0).datetime,
Arrow(2018, 1, 30, 3, 30, 0).datetime], utc=True Arrow(2018, 1, 30, 3, 30, 0).datetime], utc=True

View File

@ -50,6 +50,7 @@ def test_backtest_position_adjustment(default_conf, fee, mocker, testdatadir) ->
expected = pd.DataFrame( expected = pd.DataFrame(
{'pair': [pair, pair], {'pair': [pair, pair],
'stake_amount': [500.0, 100.0], 'stake_amount': [500.0, 100.0],
'max_stake_amount': [500.0, 100],
'amount': [4806.87657523, 970.63960782], 'amount': [4806.87657523, 970.63960782],
'open_date': pd.to_datetime([Arrow(2018, 1, 29, 18, 40, 0).datetime, 'open_date': pd.to_datetime([Arrow(2018, 1, 29, 18, 40, 0).datetime,
Arrow(2018, 1, 30, 3, 30, 0).datetime], utc=True Arrow(2018, 1, 30, 3, 30, 0).datetime], utc=True

View File

@ -308,7 +308,7 @@ def test_generate_pair_metrics():
def test_generate_daily_stats(testdatadir): def test_generate_daily_stats(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
res = generate_daily_stats(bt_data) res = generate_daily_stats(bt_data)
assert isinstance(res, dict) assert isinstance(res, dict)
@ -328,7 +328,7 @@ def test_generate_daily_stats(testdatadir):
def test_generate_trading_stats(testdatadir): def test_generate_trading_stats(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
res = generate_trading_stats(bt_data) res = generate_trading_stats(bt_data)
assert isinstance(res, dict) assert isinstance(res, dict)
@ -444,7 +444,7 @@ def test_generate_edge_table():
def test_generate_periodic_breakdown_stats(testdatadir): def test_generate_periodic_breakdown_stats(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename).to_dict(orient='records') bt_data = load_backtest_data(filename).to_dict(orient='records')
res = generate_periodic_breakdown_stats(bt_data, 'day') res = generate_periodic_breakdown_stats(bt_data, 'day')
@ -472,7 +472,7 @@ def test__get_resample_from_period():
def test_show_sorted_pairlist(testdatadir, default_conf, capsys): def test_show_sorted_pairlist(testdatadir, default_conf, capsys):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_stats(filename) bt_data = load_backtest_stats(filename)
default_conf['backtest_show_pair_list'] = True default_conf['backtest_show_pair_list'] = True

View File

@ -0,0 +1,412 @@
# pragma pylint: disable=missing-docstring, C0103
import logging
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from sqlalchemy import create_engine, text
from freqtrade.constants import DEFAULT_DB_PROD_URL
from freqtrade.enums import TradingMode
from freqtrade.exceptions import OperationalException
from freqtrade.persistence import Trade, init_db
from freqtrade.persistence.migrations import get_last_sequence_ids, set_sequence_ids
from freqtrade.persistence.models import PairLock
from tests.conftest import log_has
spot, margin, futures = TradingMode.SPOT, TradingMode.MARGIN, TradingMode.FUTURES
def test_init_create_session(default_conf):
# Check if init create a session
init_db(default_conf['db_url'])
assert hasattr(Trade, '_session')
assert 'scoped_session' in type(Trade._session).__name__
def test_init_custom_db_url(default_conf, tmpdir):
# Update path to a value other than default, but still in-memory
filename = f"{tmpdir}/freqtrade2_test.sqlite"
assert not Path(filename).is_file()
default_conf.update({'db_url': f'sqlite:///{filename}'})
init_db(default_conf['db_url'])
assert Path(filename).is_file()
r = Trade._session.execute(text("PRAGMA journal_mode"))
assert r.first() == ('wal',)
def test_init_invalid_db_url():
# Update path to a value other than default, but still in-memory
with pytest.raises(OperationalException, match=r'.*no valid database URL*'):
init_db('unknown:///some.url')
with pytest.raises(OperationalException, match=r'Bad db-url.*For in-memory database, pl.*'):
init_db('sqlite:///')
def test_init_prod_db(default_conf, mocker):
default_conf.update({'dry_run': False})
default_conf.update({'db_url': DEFAULT_DB_PROD_URL})
create_engine_mock = mocker.patch('freqtrade.persistence.models.create_engine', MagicMock())
init_db(default_conf['db_url'])
assert create_engine_mock.call_count == 1
assert create_engine_mock.mock_calls[0][1][0] == 'sqlite:///tradesv3.sqlite'
def test_init_dryrun_db(default_conf, tmpdir):
filename = f"{tmpdir}/freqtrade2_prod.sqlite"
assert not Path(filename).is_file()
default_conf.update({
'dry_run': True,
'db_url': f'sqlite:///{filename}'
})
init_db(default_conf['db_url'])
assert Path(filename).is_file()
def test_migrate_new(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
amount = 103.223
# Always create all columns apart from the last!
create_table_old = """CREATE TABLE IF NOT EXISTS "trades" (
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee FLOAT NOT NULL,
open_rate FLOAT,
close_rate FLOAT,
close_profit FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
stop_loss FLOAT,
initial_stop_loss FLOAT,
max_rate FLOAT,
sell_reason VARCHAR,
strategy VARCHAR,
ticker_interval INTEGER,
stoploss_order_id VARCHAR,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
);"""
create_table_order = """CREATE TABLE orders (
id INTEGER NOT NULL,
ft_trade_id INTEGER,
ft_order_side VARCHAR(25) NOT NULL,
ft_pair VARCHAR(25) NOT NULL,
ft_is_open BOOLEAN NOT NULL,
order_id VARCHAR(255) NOT NULL,
status VARCHAR(255),
symbol VARCHAR(25),
order_type VARCHAR(50),
side VARCHAR(25),
price FLOAT,
amount FLOAT,
filled FLOAT,
remaining FLOAT,
cost FLOAT,
order_date DATETIME,
order_filled_date DATETIME,
order_update_date DATETIME,
PRIMARY KEY (id)
);"""
insert_table_old = """INSERT INTO trades (exchange, pair, is_open, fee,
open_rate, stake_amount, amount, open_date,
stop_loss, initial_stop_loss, max_rate, ticker_interval,
open_order_id, stoploss_order_id)
VALUES ('binance', 'ETC/BTC', 1, {fee},
0.00258580, {stake}, {amount},
'2019-11-28 12:44:24.000000',
0.0, 0.0, 0.0, '5m',
'buy_order', 'dry_stop_order_id222')
""".format(fee=fee.return_value,
stake=default_conf.get("stake_amount"),
amount=amount
)
insert_orders = f"""
insert into orders (
ft_trade_id,
ft_order_side,
ft_pair,
ft_is_open,
order_id,
status,
symbol,
order_type,
side,
price,
amount,
filled,
remaining,
cost)
values (
1,
'buy',
'ETC/BTC',
0,
'dry_buy_order',
'closed',
'ETC/BTC',
'limit',
'buy',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'buy',
'ETC/BTC',
1,
'dry_buy_order22',
'canceled',
'ETC/BTC',
'limit',
'buy',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'stoploss',
'ETC/BTC',
1,
'dry_stop_order_id11X',
'canceled',
'ETC/BTC',
'limit',
'sell',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'stoploss',
'ETC/BTC',
1,
'dry_stop_order_id222',
'open',
'ETC/BTC',
'limit',
'sell',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
)
"""
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(create_table_order))
connection.execute(text("create index ix_trades_is_open on trades(is_open)"))
connection.execute(text("create index ix_trades_pair on trades(pair)"))
connection.execute(text(insert_table_old))
connection.execute(text(insert_orders))
# fake previous backup
connection.execute(text("create table trades_bak as select * from trades"))
connection.execute(text("create table trades_bak1 as select * from trades"))
# Run init to test migration
init_db(default_conf['db_url'])
assert len(Trade.query.filter(Trade.id == 1).all()) == 1
trade = Trade.query.filter(Trade.id == 1).first()
assert trade.fee_open == fee.return_value
assert trade.fee_close == fee.return_value
assert trade.open_rate_requested is None
assert trade.close_rate_requested is None
assert trade.is_open == 1
assert trade.amount == amount
assert trade.amount_requested == amount
assert trade.stake_amount == default_conf.get("stake_amount")
assert trade.pair == "ETC/BTC"
assert trade.exchange == "binance"
assert trade.max_rate == 0.0
assert trade.min_rate is None
assert trade.stop_loss == 0.0
assert trade.initial_stop_loss == 0.0
assert trade.exit_reason is None
assert trade.strategy is None
assert trade.timeframe == '5m'
assert trade.stoploss_order_id == 'dry_stop_order_id222'
assert trade.stoploss_last_update is None
assert log_has("trying trades_bak1", caplog)
assert log_has("trying trades_bak2", caplog)
assert log_has("Running database migration for trades - backup: trades_bak2, orders_bak0",
caplog)
assert log_has("Database migration finished.", caplog)
assert pytest.approx(trade.open_trade_value) == trade._calc_open_trade_value(
trade.amount, trade.open_rate)
assert trade.close_profit_abs is None
assert trade.stake_amount == trade.max_stake_amount
orders = trade.orders
assert len(orders) == 4
assert orders[0].order_id == 'dry_buy_order'
assert orders[0].ft_order_side == 'buy'
assert orders[-1].order_id == 'dry_stop_order_id222'
assert orders[-1].ft_order_side == 'stoploss'
assert orders[-1].ft_is_open is True
assert orders[1].order_id == 'dry_buy_order22'
assert orders[1].ft_order_side == 'buy'
assert orders[1].ft_is_open is False
assert orders[2].order_id == 'dry_stop_order_id11X'
assert orders[2].ft_order_side == 'stoploss'
assert orders[2].ft_is_open is False
def test_migrate_too_old(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
amount = 103.223
create_table_old = """CREATE TABLE IF NOT EXISTS "trades" (
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee_open FLOAT NOT NULL,
fee_close FLOAT NOT NULL,
open_rate FLOAT,
close_rate FLOAT,
close_profit FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
);"""
insert_table_old = """INSERT INTO trades (exchange, pair, is_open, fee_open, fee_close,
open_rate, stake_amount, amount, open_date)
VALUES ('binance', 'ETC/BTC', 1, {fee}, {fee},
0.00258580, {stake}, {amount},
'2019-11-28 12:44:24.000000')
""".format(fee=fee.return_value,
stake=default_conf.get("stake_amount"),
amount=amount
)
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(insert_table_old))
# Run init to test migration
with pytest.raises(OperationalException, match=r'Your database seems to be very old'):
init_db(default_conf['db_url'])
def test_migrate_get_last_sequence_ids():
engine = MagicMock()
engine.begin = MagicMock()
engine.name = 'postgresql'
get_last_sequence_ids(engine, 'trades_bak', 'orders_bak')
assert engine.begin.call_count == 2
engine.reset_mock()
engine.begin.reset_mock()
engine.name = 'somethingelse'
get_last_sequence_ids(engine, 'trades_bak', 'orders_bak')
assert engine.begin.call_count == 0
def test_migrate_set_sequence_ids():
engine = MagicMock()
engine.begin = MagicMock()
engine.name = 'postgresql'
set_sequence_ids(engine, 22, 55, 5)
assert engine.begin.call_count == 1
engine.reset_mock()
engine.begin.reset_mock()
engine.name = 'somethingelse'
set_sequence_ids(engine, 22, 55, 6)
assert engine.begin.call_count == 0
def test_migrate_pairlocks(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
# Always create all columns apart from the last!
create_table_old = """CREATE TABLE pairlocks (
id INTEGER NOT NULL,
pair VARCHAR(25) NOT NULL,
reason VARCHAR(255),
lock_time DATETIME NOT NULL,
lock_end_time DATETIME NOT NULL,
active BOOLEAN NOT NULL,
PRIMARY KEY (id)
)
"""
create_index1 = "CREATE INDEX ix_pairlocks_pair ON pairlocks (pair)"
create_index2 = "CREATE INDEX ix_pairlocks_lock_end_time ON pairlocks (lock_end_time)"
create_index3 = "CREATE INDEX ix_pairlocks_active ON pairlocks (active)"
insert_table_old = """INSERT INTO pairlocks (
id, pair, reason, lock_time, lock_end_time, active)
VALUES (1, 'ETH/BTC', 'Auto lock', '2021-07-12 18:41:03', '2021-07-11 18:45:00', 1)
"""
insert_table_old2 = """INSERT INTO pairlocks (
id, pair, reason, lock_time, lock_end_time, active)
VALUES (2, '*', 'Lock all', '2021-07-12 18:41:03', '2021-07-12 19:00:00', 1)
"""
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(insert_table_old))
connection.execute(text(insert_table_old2))
connection.execute(text(create_index1))
connection.execute(text(create_index2))
connection.execute(text(create_index3))
init_db(default_conf['db_url'])
assert len(PairLock.query.all()) == 2
assert len(PairLock.query.filter(PairLock.pair == '*').all()) == 1
pairlocks = PairLock.query.filter(PairLock.pair == 'ETH/BTC').all()
assert len(pairlocks) == 1
pairlocks[0].pair == 'ETH/BTC'
pairlocks[0].side == '*'

View File

@ -1,78 +1,20 @@
# pragma pylint: disable=missing-docstring, C0103 # pragma pylint: disable=missing-docstring, C0103
import logging
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from pathlib import Path
from types import FunctionType from types import FunctionType
from unittest.mock import MagicMock
import arrow import arrow
import pytest import pytest
from sqlalchemy import create_engine, text
from freqtrade.constants import DATETIME_PRINT_FORMAT, DEFAULT_DB_PROD_URL from freqtrade.constants import DATETIME_PRINT_FORMAT
from freqtrade.enums import TradingMode from freqtrade.enums import TradingMode
from freqtrade.exceptions import DependencyException, OperationalException from freqtrade.exceptions import DependencyException
from freqtrade.persistence import LocalTrade, Order, Trade, init_db from freqtrade.persistence import LocalTrade, Order, Trade, init_db
from freqtrade.persistence.migrations import get_last_sequence_ids, set_sequence_ids
from freqtrade.persistence.models import PairLock
from tests.conftest import create_mock_trades, create_mock_trades_with_leverage, log_has, log_has_re from tests.conftest import create_mock_trades, create_mock_trades_with_leverage, log_has, log_has_re
spot, margin, futures = TradingMode.SPOT, TradingMode.MARGIN, TradingMode.FUTURES spot, margin, futures = TradingMode.SPOT, TradingMode.MARGIN, TradingMode.FUTURES
def test_init_create_session(default_conf):
# Check if init create a session
init_db(default_conf['db_url'])
assert hasattr(Trade, '_session')
assert 'scoped_session' in type(Trade._session).__name__
def test_init_custom_db_url(default_conf, tmpdir):
# Update path to a value other than default, but still in-memory
filename = f"{tmpdir}/freqtrade2_test.sqlite"
assert not Path(filename).is_file()
default_conf.update({'db_url': f'sqlite:///{filename}'})
init_db(default_conf['db_url'])
assert Path(filename).is_file()
r = Trade._session.execute(text("PRAGMA journal_mode"))
assert r.first() == ('wal',)
def test_init_invalid_db_url():
# Update path to a value other than default, but still in-memory
with pytest.raises(OperationalException, match=r'.*no valid database URL*'):
init_db('unknown:///some.url')
with pytest.raises(OperationalException, match=r'Bad db-url.*For in-memory database, pl.*'):
init_db('sqlite:///')
def test_init_prod_db(default_conf, mocker):
default_conf.update({'dry_run': False})
default_conf.update({'db_url': DEFAULT_DB_PROD_URL})
create_engine_mock = mocker.patch('freqtrade.persistence.models.create_engine', MagicMock())
init_db(default_conf['db_url'])
assert create_engine_mock.call_count == 1
assert create_engine_mock.mock_calls[0][1][0] == 'sqlite:///tradesv3.sqlite'
def test_init_dryrun_db(default_conf, tmpdir):
filename = f"{tmpdir}/freqtrade2_prod.sqlite"
assert not Path(filename).is_file()
default_conf.update({
'dry_run': True,
'db_url': f'sqlite:///{filename}'
})
init_db(default_conf['db_url'])
assert Path(filename).is_file()
@pytest.mark.parametrize('is_short', [False, True]) @pytest.mark.parametrize('is_short', [False, True])
@pytest.mark.usefixtures("init_persistence") @pytest.mark.usefixtures("init_persistence")
def test_enter_exit_side(fee, is_short): def test_enter_exit_side(fee, is_short):
@ -316,8 +258,7 @@ def test_interest(fee, exchange, is_short, lev, minutes, rate, interest,
(True, 3.0, 30.0, margin), (True, 3.0, 30.0, margin),
]) ])
@pytest.mark.usefixtures("init_persistence") @pytest.mark.usefixtures("init_persistence")
def test_borrowed(limit_buy_order_usdt, limit_sell_order_usdt, fee, def test_borrowed(fee, is_short, lev, borrowed, trading_mode):
caplog, is_short, lev, borrowed, trading_mode):
""" """
10 minute limit trade on Binance/Kraken at 1x, 3x leverage 10 minute limit trade on Binance/Kraken at 1x, 3x leverage
fee: 0.25% quote fee: 0.25% quote
@ -1204,347 +1145,6 @@ def test_calc_profit(
trade.open_rate)) == round(profit_ratio, 8) trade.open_rate)) == round(profit_ratio, 8)
def test_migrate_new(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
amount = 103.223
# Always create all columns apart from the last!
create_table_old = """CREATE TABLE IF NOT EXISTS "trades" (
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee FLOAT NOT NULL,
open_rate FLOAT,
close_rate FLOAT,
close_profit FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
stop_loss FLOAT,
initial_stop_loss FLOAT,
max_rate FLOAT,
sell_reason VARCHAR,
strategy VARCHAR,
ticker_interval INTEGER,
stoploss_order_id VARCHAR,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
);"""
create_table_order = """CREATE TABLE orders (
id INTEGER NOT NULL,
ft_trade_id INTEGER,
ft_order_side VARCHAR(25) NOT NULL,
ft_pair VARCHAR(25) NOT NULL,
ft_is_open BOOLEAN NOT NULL,
order_id VARCHAR(255) NOT NULL,
status VARCHAR(255),
symbol VARCHAR(25),
order_type VARCHAR(50),
side VARCHAR(25),
price FLOAT,
amount FLOAT,
filled FLOAT,
remaining FLOAT,
cost FLOAT,
order_date DATETIME,
order_filled_date DATETIME,
order_update_date DATETIME,
PRIMARY KEY (id)
);"""
insert_table_old = """INSERT INTO trades (exchange, pair, is_open, fee,
open_rate, stake_amount, amount, open_date,
stop_loss, initial_stop_loss, max_rate, ticker_interval,
open_order_id, stoploss_order_id)
VALUES ('binance', 'ETC/BTC', 1, {fee},
0.00258580, {stake}, {amount},
'2019-11-28 12:44:24.000000',
0.0, 0.0, 0.0, '5m',
'buy_order', 'dry_stop_order_id222')
""".format(fee=fee.return_value,
stake=default_conf.get("stake_amount"),
amount=amount
)
insert_orders = f"""
insert into orders (
ft_trade_id,
ft_order_side,
ft_pair,
ft_is_open,
order_id,
status,
symbol,
order_type,
side,
price,
amount,
filled,
remaining,
cost)
values (
1,
'buy',
'ETC/BTC',
0,
'dry_buy_order',
'closed',
'ETC/BTC',
'limit',
'buy',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'buy',
'ETC/BTC',
1,
'dry_buy_order22',
'canceled',
'ETC/BTC',
'limit',
'buy',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'stoploss',
'ETC/BTC',
1,
'dry_stop_order_id11X',
'canceled',
'ETC/BTC',
'limit',
'sell',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
),
(
1,
'stoploss',
'ETC/BTC',
1,
'dry_stop_order_id222',
'open',
'ETC/BTC',
'limit',
'sell',
0.00258580,
{amount},
{amount},
0,
{amount * 0.00258580}
)
"""
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(create_table_order))
connection.execute(text("create index ix_trades_is_open on trades(is_open)"))
connection.execute(text("create index ix_trades_pair on trades(pair)"))
connection.execute(text(insert_table_old))
connection.execute(text(insert_orders))
# fake previous backup
connection.execute(text("create table trades_bak as select * from trades"))
connection.execute(text("create table trades_bak1 as select * from trades"))
# Run init to test migration
init_db(default_conf['db_url'])
assert len(Trade.query.filter(Trade.id == 1).all()) == 1
trade = Trade.query.filter(Trade.id == 1).first()
assert trade.fee_open == fee.return_value
assert trade.fee_close == fee.return_value
assert trade.open_rate_requested is None
assert trade.close_rate_requested is None
assert trade.is_open == 1
assert trade.amount == amount
assert trade.amount_requested == amount
assert trade.stake_amount == default_conf.get("stake_amount")
assert trade.pair == "ETC/BTC"
assert trade.exchange == "binance"
assert trade.max_rate == 0.0
assert trade.min_rate is None
assert trade.stop_loss == 0.0
assert trade.initial_stop_loss == 0.0
assert trade.exit_reason is None
assert trade.strategy is None
assert trade.timeframe == '5m'
assert trade.stoploss_order_id == 'dry_stop_order_id222'
assert trade.stoploss_last_update is None
assert log_has("trying trades_bak1", caplog)
assert log_has("trying trades_bak2", caplog)
assert log_has("Running database migration for trades - backup: trades_bak2, orders_bak0",
caplog)
assert log_has("Database migration finished.", caplog)
assert pytest.approx(trade.open_trade_value) == trade._calc_open_trade_value(
trade.amount, trade.open_rate)
assert trade.close_profit_abs is None
orders = trade.orders
assert len(orders) == 4
assert orders[0].order_id == 'dry_buy_order'
assert orders[0].ft_order_side == 'buy'
assert orders[-1].order_id == 'dry_stop_order_id222'
assert orders[-1].ft_order_side == 'stoploss'
assert orders[-1].ft_is_open is True
assert orders[1].order_id == 'dry_buy_order22'
assert orders[1].ft_order_side == 'buy'
assert orders[1].ft_is_open is False
assert orders[2].order_id == 'dry_stop_order_id11X'
assert orders[2].ft_order_side == 'stoploss'
assert orders[2].ft_is_open is False
def test_migrate_too_old(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
amount = 103.223
create_table_old = """CREATE TABLE IF NOT EXISTS "trades" (
id INTEGER NOT NULL,
exchange VARCHAR NOT NULL,
pair VARCHAR NOT NULL,
is_open BOOLEAN NOT NULL,
fee_open FLOAT NOT NULL,
fee_close FLOAT NOT NULL,
open_rate FLOAT,
close_rate FLOAT,
close_profit FLOAT,
stake_amount FLOAT NOT NULL,
amount FLOAT,
open_date DATETIME NOT NULL,
close_date DATETIME,
open_order_id VARCHAR,
PRIMARY KEY (id),
CHECK (is_open IN (0, 1))
);"""
insert_table_old = """INSERT INTO trades (exchange, pair, is_open, fee_open, fee_close,
open_rate, stake_amount, amount, open_date)
VALUES ('binance', 'ETC/BTC', 1, {fee}, {fee},
0.00258580, {stake}, {amount},
'2019-11-28 12:44:24.000000')
""".format(fee=fee.return_value,
stake=default_conf.get("stake_amount"),
amount=amount
)
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(insert_table_old))
# Run init to test migration
with pytest.raises(OperationalException, match=r'Your database seems to be very old'):
init_db(default_conf['db_url'])
def test_migrate_get_last_sequence_ids():
engine = MagicMock()
engine.begin = MagicMock()
engine.name = 'postgresql'
get_last_sequence_ids(engine, 'trades_bak', 'orders_bak')
assert engine.begin.call_count == 2
engine.reset_mock()
engine.begin.reset_mock()
engine.name = 'somethingelse'
get_last_sequence_ids(engine, 'trades_bak', 'orders_bak')
assert engine.begin.call_count == 0
def test_migrate_set_sequence_ids():
engine = MagicMock()
engine.begin = MagicMock()
engine.name = 'postgresql'
set_sequence_ids(engine, 22, 55, 5)
assert engine.begin.call_count == 1
engine.reset_mock()
engine.begin.reset_mock()
engine.name = 'somethingelse'
set_sequence_ids(engine, 22, 55, 6)
assert engine.begin.call_count == 0
def test_migrate_pairlocks(mocker, default_conf, fee, caplog):
"""
Test Database migration (starting with new pairformat)
"""
caplog.set_level(logging.DEBUG)
# Always create all columns apart from the last!
create_table_old = """CREATE TABLE pairlocks (
id INTEGER NOT NULL,
pair VARCHAR(25) NOT NULL,
reason VARCHAR(255),
lock_time DATETIME NOT NULL,
lock_end_time DATETIME NOT NULL,
active BOOLEAN NOT NULL,
PRIMARY KEY (id)
)
"""
create_index1 = "CREATE INDEX ix_pairlocks_pair ON pairlocks (pair)"
create_index2 = "CREATE INDEX ix_pairlocks_lock_end_time ON pairlocks (lock_end_time)"
create_index3 = "CREATE INDEX ix_pairlocks_active ON pairlocks (active)"
insert_table_old = """INSERT INTO pairlocks (
id, pair, reason, lock_time, lock_end_time, active)
VALUES (1, 'ETH/BTC', 'Auto lock', '2021-07-12 18:41:03', '2021-07-11 18:45:00', 1)
"""
insert_table_old2 = """INSERT INTO pairlocks (
id, pair, reason, lock_time, lock_end_time, active)
VALUES (2, '*', 'Lock all', '2021-07-12 18:41:03', '2021-07-12 19:00:00', 1)
"""
engine = create_engine('sqlite://')
mocker.patch('freqtrade.persistence.models.create_engine', lambda *args, **kwargs: engine)
# Create table using the old format
with engine.begin() as connection:
connection.execute(text(create_table_old))
connection.execute(text(insert_table_old))
connection.execute(text(insert_table_old2))
connection.execute(text(create_index1))
connection.execute(text(create_index2))
connection.execute(text(create_index3))
init_db(default_conf['db_url'])
assert len(PairLock.query.all()) == 2
assert len(PairLock.query.filter(PairLock.pair == '*').all()) == 1
pairlocks = PairLock.query.filter(PairLock.pair == 'ETH/BTC').all()
assert len(pairlocks) == 1
pairlocks[0].pair == 'ETH/BTC'
pairlocks[0].side == '*'
def test_adjust_stop_loss(fee): def test_adjust_stop_loss(fee):
trade = Trade( trade = Trade(
pair='ADA/USDT', pair='ADA/USDT',
@ -1758,6 +1358,7 @@ def test_to_json(fee):
'amount': 123.0, 'amount': 123.0,
'amount_requested': 123.0, 'amount_requested': 123.0,
'stake_amount': 0.001, 'stake_amount': 0.001,
'max_stake_amount': None,
'trade_duration': None, 'trade_duration': None,
'trade_duration_s': None, 'trade_duration_s': None,
'realized_profit': 0.0, 'realized_profit': 0.0,
@ -1767,7 +1368,6 @@ def test_to_json(fee):
'profit_ratio': None, 'profit_ratio': None,
'profit_pct': None, 'profit_pct': None,
'profit_abs': None, 'profit_abs': None,
'sell_reason': None,
'exit_reason': None, 'exit_reason': None,
'exit_order_status': None, 'exit_order_status': None,
'stop_loss_abs': None, 'stop_loss_abs': None,
@ -1782,7 +1382,6 @@ def test_to_json(fee):
'min_rate': None, 'min_rate': None,
'max_rate': None, 'max_rate': None,
'strategy': None, 'strategy': None,
'buy_tag': None,
'enter_tag': None, 'enter_tag': None,
'timeframe': None, 'timeframe': None,
'exchange': 'binance', 'exchange': 'binance',
@ -1826,6 +1425,7 @@ def test_to_json(fee):
'amount': 100.0, 'amount': 100.0,
'amount_requested': 101.0, 'amount_requested': 101.0,
'stake_amount': 0.001, 'stake_amount': 0.001,
'max_stake_amount': None,
'trade_duration': 60, 'trade_duration': 60,
'trade_duration_s': 3600, 'trade_duration_s': 3600,
'stop_loss_abs': None, 'stop_loss_abs': None,
@ -1857,11 +1457,9 @@ def test_to_json(fee):
'open_order_id': None, 'open_order_id': None,
'open_rate_requested': None, 'open_rate_requested': None,
'open_trade_value': 12.33075, 'open_trade_value': 12.33075,
'sell_reason': None,
'exit_reason': None, 'exit_reason': None,
'exit_order_status': None, 'exit_order_status': None,
'strategy': None, 'strategy': None,
'buy_tag': 'buys_signal_001',
'enter_tag': 'buys_signal_001', 'enter_tag': 'buys_signal_001',
'timeframe': None, 'timeframe': None,
'exchange': 'binance', 'exchange': 'binance',

View File

@ -46,13 +46,11 @@ def test_rpc_trade_status(default_conf, ticker, fee, mocker) -> None:
'open_rate_requested': ANY, 'open_rate_requested': ANY,
'open_trade_value': 0.0010025, 'open_trade_value': 0.0010025,
'close_rate_requested': ANY, 'close_rate_requested': ANY,
'sell_reason': ANY,
'exit_reason': ANY, 'exit_reason': ANY,
'exit_order_status': ANY, 'exit_order_status': ANY,
'min_rate': ANY, 'min_rate': ANY,
'max_rate': ANY, 'max_rate': ANY,
'strategy': ANY, 'strategy': ANY,
'buy_tag': ANY,
'enter_tag': ANY, 'enter_tag': ANY,
'timeframe': 5, 'timeframe': 5,
'open_order_id': ANY, 'open_order_id': ANY,
@ -64,6 +62,7 @@ def test_rpc_trade_status(default_conf, ticker, fee, mocker) -> None:
'amount': 91.07468123, 'amount': 91.07468123,
'amount_requested': 91.07468124, 'amount_requested': 91.07468124,
'stake_amount': 0.001, 'stake_amount': 0.001,
'max_stake_amount': ANY,
'trade_duration': None, 'trade_duration': None,
'trade_duration_s': None, 'trade_duration_s': None,
'close_profit': None, 'close_profit': None,

View File

@ -985,6 +985,7 @@ def test_api_status(botclient, mocker, ticker, fee, markets, is_short,
'base_currency': 'ETH', 'base_currency': 'ETH',
'quote_currency': 'BTC', 'quote_currency': 'BTC',
'stake_amount': 0.001, 'stake_amount': 0.001,
'max_stake_amount': ANY,
'stop_loss_abs': ANY, 'stop_loss_abs': ANY,
'stop_loss_pct': ANY, 'stop_loss_pct': ANY,
'stop_loss_ratio': ANY, 'stop_loss_ratio': ANY,
@ -1014,11 +1015,9 @@ def test_api_status(botclient, mocker, ticker, fee, markets, is_short,
'open_order_id': open_order_id, 'open_order_id': open_order_id,
'open_rate_requested': ANY, 'open_rate_requested': ANY,
'open_trade_value': open_trade_value, 'open_trade_value': open_trade_value,
'sell_reason': None,
'exit_reason': None, 'exit_reason': None,
'exit_order_status': None, 'exit_order_status': None,
'strategy': CURRENT_TEST_STRATEGY, 'strategy': CURRENT_TEST_STRATEGY,
'buy_tag': None,
'enter_tag': None, 'enter_tag': None,
'timeframe': 5, 'timeframe': 5,
'exchange': 'binance', 'exchange': 'binance',
@ -1188,6 +1187,7 @@ def test_api_force_entry(botclient, mocker, fee, endpoint):
'base_currency': 'ETH', 'base_currency': 'ETH',
'quote_currency': 'BTC', 'quote_currency': 'BTC',
'stake_amount': 1, 'stake_amount': 1,
'max_stake_amount': ANY,
'stop_loss_abs': None, 'stop_loss_abs': None,
'stop_loss_pct': None, 'stop_loss_pct': None,
'stop_loss_ratio': None, 'stop_loss_ratio': None,
@ -1218,11 +1218,9 @@ def test_api_force_entry(botclient, mocker, fee, endpoint):
'open_order_id': '123456', 'open_order_id': '123456',
'open_rate_requested': None, 'open_rate_requested': None,
'open_trade_value': 0.24605460, 'open_trade_value': 0.24605460,
'sell_reason': None,
'exit_reason': None, 'exit_reason': None,
'exit_order_status': None, 'exit_order_status': None,
'strategy': CURRENT_TEST_STRATEGY, 'strategy': CURRENT_TEST_STRATEGY,
'buy_tag': None,
'enter_tag': None, 'enter_tag': None,
'timeframe': 5, 'timeframe': 5,
'exchange': 'binance', 'exchange': 'binance',
@ -1709,7 +1707,7 @@ def test_api_backtest_history(botclient, mocker, testdatadir):
mocker.patch('freqtrade.data.btanalysis._get_backtest_files', mocker.patch('freqtrade.data.btanalysis._get_backtest_files',
return_value=[ return_value=[
testdatadir / 'backtest_results/backtest-result_multistrat.json', testdatadir / 'backtest_results/backtest-result_multistrat.json',
testdatadir / 'backtest_results/backtest-result_new.json' testdatadir / 'backtest_results/backtest-result.json'
]) ])
rc = client_get(client, f"{BASE_URI}/backtest/history") rc = client_get(client, f"{BASE_URI}/backtest/history")

View File

@ -46,7 +46,7 @@ def test_init_plotscript(default_conf, mocker, testdatadir):
default_conf['trade_source'] = "file" default_conf['trade_source'] = "file"
default_conf['timeframe'] = "5m" default_conf['timeframe'] = "5m"
default_conf["datadir"] = testdatadir default_conf["datadir"] = testdatadir
default_conf['exportfilename'] = testdatadir / "backtest-result_new.json" default_conf['exportfilename'] = testdatadir / "backtest-result.json"
supported_markets = ["TRX/BTC", "ADA/BTC"] supported_markets = ["TRX/BTC", "ADA/BTC"]
ret = init_plotscript(default_conf, supported_markets) ret = init_plotscript(default_conf, supported_markets)
assert "ohlcv" in ret assert "ohlcv" in ret
@ -158,7 +158,7 @@ def test_plot_trades(testdatadir, caplog):
assert fig == fig1 assert fig == fig1
assert log_has("No trades found.", caplog) assert log_has("No trades found.", caplog)
pair = "ADA/BTC" pair = "ADA/BTC"
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
trades = load_backtest_data(filename) trades = load_backtest_data(filename)
trades = trades.loc[trades['pair'] == pair] trades = trades.loc[trades['pair'] == pair]
@ -299,7 +299,7 @@ def test_generate_plot_file(mocker, caplog):
def test_add_profit(testdatadir): def test_add_profit(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
bt_data = load_backtest_data(filename) bt_data = load_backtest_data(filename)
timerange = TimeRange.parse_timerange("20180110-20180112") timerange = TimeRange.parse_timerange("20180110-20180112")
@ -319,7 +319,7 @@ def test_add_profit(testdatadir):
def test_generate_profit_graph(testdatadir): def test_generate_profit_graph(testdatadir):
filename = testdatadir / "backtest_results/backtest-result_new.json" filename = testdatadir / "backtest_results/backtest-result.json"
trades = load_backtest_data(filename) trades = load_backtest_data(filename)
timerange = TimeRange.parse_timerange("20180110-20180112") timerange = TimeRange.parse_timerange("20180110-20180112")
pairs = ["TRX/BTC", "XLM/BTC"] pairs = ["TRX/BTC", "XLM/BTC"]
@ -354,7 +354,7 @@ def test_generate_profit_graph(testdatadir):
profit = find_trace_in_fig_data(figure.data, "Profit") profit = find_trace_in_fig_data(figure.data, "Profit")
assert isinstance(profit, go.Scatter) assert isinstance(profit, go.Scatter)
drawdown = find_trace_in_fig_data(figure.data, "Max drawdown 35.69%") drawdown = find_trace_in_fig_data(figure.data, "Max drawdown 73.89%")
assert isinstance(drawdown, go.Scatter) assert isinstance(drawdown, go.Scatter)
parallel = find_trace_in_fig_data(figure.data, "Parallel trades") parallel = find_trace_in_fig_data(figure.data, "Parallel trades")
assert isinstance(parallel, go.Scatter) assert isinstance(parallel, go.Scatter)
@ -395,7 +395,7 @@ def test_load_and_plot_trades(default_conf, mocker, caplog, testdatadir):
default_conf['trade_source'] = 'file' default_conf['trade_source'] = 'file'
default_conf["datadir"] = testdatadir default_conf["datadir"] = testdatadir
default_conf['exportfilename'] = testdatadir / "backtest-result_new.json" default_conf['exportfilename'] = testdatadir / "backtest-result.json"
default_conf['indicators1'] = ["sma5", "ema10"] default_conf['indicators1'] = ["sma5", "ema10"]
default_conf['indicators2'] = ["macd"] default_conf['indicators2'] = ["macd"]
default_conf['pairs'] = ["ETH/BTC", "LTC/BTC"] default_conf['pairs'] = ["ETH/BTC", "LTC/BTC"]
@ -466,7 +466,7 @@ def test_plot_profit(default_conf, mocker, testdatadir):
match=r"No trades found, cannot generate Profit-plot.*"): match=r"No trades found, cannot generate Profit-plot.*"):
plot_profit(default_conf) plot_profit(default_conf)
default_conf['exportfilename'] = testdatadir / "backtest_results/backtest-result_new.json" default_conf['exportfilename'] = testdatadir / "backtest_results/backtest-result.json"
plot_profit(default_conf) plot_profit(default_conf)

View File

@ -1 +1 @@
{"latest_backtest":"backtest-result_new.json"} {"latest_backtest":"backtest-result.json"}

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long