Merge pull request #2831 from xmatthias/feat/new_config

introduce new-config subcommand
This commit is contained in:
hroff-1902 2020-02-09 22:11:37 +03:00 committed by GitHub
commit 0ba8d13de9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
20 changed files with 603 additions and 41 deletions

View File

@ -51,12 +51,15 @@ To run this bot we recommend you a cloud instance with a minimum of:
### Software requirements
- Docker (Recommended)
Alternatively
- Python 3.6.x
- pip (pip3)
- git
- TA-Lib
- virtualenv (Recommended)
- Docker (Recommended)
## Support
@ -67,4 +70,4 @@ Click [here](https://join.slack.com/t/highfrequencybot/shared_invite/enQtNjU5ODc
## Ready to try?
Begin by reading our installation guide [here](installation).
Begin by reading our installation guide [for docker](docker.md), or for [installation without docker](installation.md).

View File

@ -31,7 +31,7 @@ Freqtrade provides the Linux/MacOS Easy Installation script to install all depen
!!! Note
Windows installation is explained [here](#windows).
The easiest way to install and run Freqtrade is to clone the bot GitHub repository and then run the Easy Installation script, if it's available for your platform.
The easiest way to install and run Freqtrade is to clone the bot Github repository and then run the Easy Installation script, if it's available for your platform.
!!! Note "Version considerations"
When cloning the repository the default working branch has the name `develop`. This branch contains all last features (can be considered as relatively stable, thanks to automated tests). The `master` branch contains the code of the last release (done usually once per month on an approximately one week old snapshot of the `develop` branch to prevent packaging bugs, so potentially it's more stable).
@ -47,6 +47,7 @@ cd freqtrade
git checkout master # Optional, see (1)
./setup.sh --install
```
(1) This command switches the cloned repository to the use of the `master` branch. It's not needed if you wish to stay on the `develop` branch. You may later switch between branches at any time with the `git checkout master`/`git checkout develop` commands.
## Easy Installation Script (Linux/MacOS)
@ -129,6 +130,17 @@ bash setup.sh -i
#### 1. Install TA-Lib
Use the provided ta-lib installation script
```bash
sudo ./build_helpers/install_ta-lib.sh
```
!!! Note
This will use the ta-lib tar.gz included in this repository.
##### TA-Lib manual installation
Official webpage: https://mrjbq7.github.io/ta-lib/install.html
```bash
@ -184,7 +196,8 @@ python3 -m pip install -e .
# Initialize the user_directory
freqtrade create-userdir --userdir user_data/
cp config.json.example config.json
# Create a new configuration file
freqtrade new-config --config config.json
```
> *To edit the config please refer to [Bot Configuration](configuration.md).*

View File

@ -36,6 +36,38 @@ optional arguments:
└── sample_strategy.py
```
## Create new config
Creates a new configuration file, asking some questions which are important selections for a configuration.
```
usage: freqtrade new-config [-h] [-c PATH]
optional arguments:
-h, --help show this help message and exit
-c PATH, --config PATH
Specify configuration file (default: `config.json`). Multiple --config options may be used. Can be set to `-`
to read config from stdin.
```
!!! Warning
Only vital questions are asked. Freqtrade offers a lot more configuration possibilities, which are listed in the [Configuration documentation](configuration.md#configuration-parameters)
### Create config examples
```
$ freqtrade new-config --config config_binance.json
? Do you want to enable Dry-run (simulated trades)? Yes
? Please insert your stake currency: BTC
? Please insert your stake amount: 0.05
? Please insert max_open_trades (Integer or 'unlimited'): 5
? Please insert your ticker interval: 15m
? Please insert your display Currency (for reporting): USD
? Select exchange binance
? Do you want to enable Telegram? No
```
## Create new strategy
Creates a new strategy from a template similar to SampleStrategy.
@ -220,20 +252,31 @@ All exchanges supported by the ccxt library: _1btcxe, acx, adara, allcoin, anxpr
Use the `list-timeframes` subcommand to see the list of ticker intervals (timeframes) available for the exchange.
```
usage: freqtrade list-timeframes [-h] [--exchange EXCHANGE] [-1]
usage: freqtrade list-timeframes [-h] [-v] [--logfile FILE] [-V] [-c PATH] [-d PATH] [--userdir PATH] [--exchange EXCHANGE] [-1]
optional arguments:
-h, --help show this help message and exit
--exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no
config is provided.
--exchange EXCHANGE Exchange name (default: `bittrex`). Only valid if no config is provided.
-1, --one-column Print output in one column.
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are: 'syslog', 'journald'. See the documentation for more details.
-V, --version show program's version number and exit
-c PATH, --config PATH
Specify configuration file (default: `config.json`). Multiple --config options may be used. Can be set to `-`
to read config from stdin.
-d PATH, --datadir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
```
* Example: see the timeframes for the 'binance' exchange, set in the configuration file:
```
$ freqtrade -c config_binance.json list-timeframes
$ freqtrade list-timeframes -c config_binance.json
...
Timeframes available for the exchange `binance`: 1m, 3m, 5m, 15m, 30m, 1h, 2h, 4h, 6h, 8h, 12h, 1d, 3d, 1w, 1M
```
@ -257,14 +300,16 @@ You can print info about any pair/market with these subcommands - and you can fi
These subcommands have same usage and same set of available options:
```
usage: freqtrade list-markets [-h] [--exchange EXCHANGE] [--print-list]
[--print-json] [-1] [--print-csv]
usage: freqtrade list-markets [-h] [-v] [--logfile FILE] [-V] [-c PATH]
[-d PATH] [--userdir PATH] [--exchange EXCHANGE]
[--print-list] [--print-json] [-1] [--print-csv]
[--base BASE_CURRENCY [BASE_CURRENCY ...]]
[--quote QUOTE_CURRENCY [QUOTE_CURRENCY ...]]
[-a]
usage: freqtrade list-pairs [-h] [--exchange EXCHANGE] [--print-list]
[--print-json] [-1] [--print-csv]
usage: freqtrade list-pairs [-h] [-v] [--logfile FILE] [-V] [-c PATH]
[-d PATH] [--userdir PATH] [--exchange EXCHANGE]
[--print-list] [--print-json] [-1] [--print-csv]
[--base BASE_CURRENCY [BASE_CURRENCY ...]]
[--quote QUOTE_CURRENCY [QUOTE_CURRENCY ...]] [-a]
@ -283,6 +328,22 @@ optional arguments:
Specify quote currency(-ies). Space-separated list.
-a, --all Print all pairs or market symbols. By default only
active ones are shown.
Common arguments:
-v, --verbose Verbose mode (-vv for more, -vvv to get all messages).
--logfile FILE Log to the file specified. Special values are:
'syslog', 'journald'. See the documentation for more
details.
-V, --version show program's version number and exit
-c PATH, --config PATH
Specify configuration file (default: `config.json`).
Multiple --config options may be used. Can be set to
`-` to read config from stdin.
-d PATH, --datadir PATH
Path to directory with historical backtesting data.
--userdir PATH, --user-data-dir PATH
Path to userdata directory.
```
By default, only active pairs/markets are shown. Active pairs/markets are those that can currently be traded
@ -304,7 +365,7 @@ $ freqtrade list-pairs --quote USD --print-json
human-readable list with summary:
```
$ freqtrade -c config_binance.json list-pairs --all --base BTC ETH --quote USDT USD --print-list
$ freqtrade list-pairs -c config_binance.json --all --base BTC ETH --quote USDT USD --print-list
```
* Print all markets on exchange "Kraken", in the tabular format:

View File

@ -7,6 +7,7 @@ Note: Be careful with file-scoped imports in these subfiles.
as they are parsed on startup, nothing containing optional modules should be loaded.
"""
from freqtrade.commands.arguments import Arguments
from freqtrade.commands.build_config_commands import start_new_config
from freqtrade.commands.data_commands import start_download_data
from freqtrade.commands.deploy_commands import (start_create_userdir,
start_new_hyperopt,

View File

@ -45,6 +45,8 @@ ARGS_TEST_PAIRLIST = ["config", "quote_currencies", "print_one_column", "list_pa
ARGS_CREATE_USERDIR = ["user_data_dir", "reset"]
ARGS_BUILD_CONFIG = ["config"]
ARGS_BUILD_STRATEGY = ["user_data_dir", "strategy", "template"]
ARGS_BUILD_HYPEROPT = ["user_data_dir", "hyperopt", "template"]
@ -136,7 +138,7 @@ class Arguments:
start_hyperopt_list, start_hyperopt_show,
start_list_exchanges, start_list_hyperopts,
start_list_markets, start_list_strategies,
start_list_timeframes,
start_list_timeframes, start_new_config,
start_new_hyperopt, start_new_strategy,
start_plot_dataframe, start_plot_profit,
start_backtesting, start_hyperopt, start_edge,
@ -180,6 +182,12 @@ class Arguments:
create_userdir_cmd.set_defaults(func=start_create_userdir)
self._build_args(optionlist=ARGS_CREATE_USERDIR, parser=create_userdir_cmd)
# add new-config subcommand
build_config_cmd = subparsers.add_parser('new-config',
help="Create new config")
build_config_cmd.set_defaults(func=start_new_config)
self._build_args(optionlist=ARGS_BUILD_CONFIG, parser=build_config_cmd)
# add new-strategy subcommand
build_strategy_cmd = subparsers.add_parser('new-strategy',
help="Create new strategy")

View File

@ -0,0 +1,193 @@
import logging
from pathlib import Path
from typing import Any, Dict
from questionary import Separator, prompt
from freqtrade.constants import UNLIMITED_STAKE_AMOUNT
from freqtrade.exchange import available_exchanges, MAP_EXCHANGE_CHILDCLASS
from freqtrade.misc import render_template
from freqtrade.exceptions import OperationalException
logger = logging.getLogger(__name__)
def validate_is_int(val):
try:
_ = int(val)
return True
except Exception:
return False
def validate_is_float(val):
try:
_ = float(val)
return True
except Exception:
return False
def ask_user_overwrite(config_path: Path) -> bool:
questions = [
{
"type": "confirm",
"name": "overwrite",
"message": f"File {config_path} already exists. Overwrite?",
"default": False,
},
]
answers = prompt(questions)
return answers['overwrite']
def ask_user_config() -> Dict[str, Any]:
"""
Ask user a few questions to build the configuration.
Interactive questions built using https://github.com/tmbo/questionary
:returns: Dict with keys to put into template
"""
questions = [
{
"type": "confirm",
"name": "dry_run",
"message": "Do you want to enable Dry-run (simulated trades)?",
"default": True,
},
{
"type": "text",
"name": "stake_currency",
"message": "Please insert your stake currency:",
"default": 'BTC',
},
{
"type": "text",
"name": "stake_amount",
"message": "Please insert your stake amount:",
"default": "0.01",
"validate": lambda val: val == UNLIMITED_STAKE_AMOUNT or validate_is_float(val),
},
{
"type": "text",
"name": "max_open_trades",
"message": f"Please insert max_open_trades (Integer or '{UNLIMITED_STAKE_AMOUNT}'):",
"default": "3",
"validate": lambda val: val == UNLIMITED_STAKE_AMOUNT or validate_is_int(val)
},
{
"type": "text",
"name": "ticker_interval",
"message": "Please insert your ticker interval:",
"default": "5m",
},
{
"type": "text",
"name": "fiat_display_currency",
"message": "Please insert your display Currency (for reporting):",
"default": 'USD',
},
{
"type": "select",
"name": "exchange_name",
"message": "Select exchange",
"choices": [
"binance",
"binanceje",
"binanceus",
"bittrex",
"kraken",
Separator(),
"other",
],
},
{
"type": "autocomplete",
"name": "exchange_name",
"message": "Type your exchange name (Must be supported by ccxt)",
"choices": available_exchanges(),
"when": lambda x: x["exchange_name"] == 'other'
},
{
"type": "password",
"name": "exchange_key",
"message": "Insert Exchange Key",
"when": lambda x: not x['dry_run']
},
{
"type": "password",
"name": "exchange_secret",
"message": "Insert Exchange Secret",
"when": lambda x: not x['dry_run']
},
{
"type": "confirm",
"name": "telegram",
"message": "Do you want to enable Telegram?",
"default": False,
},
{
"type": "password",
"name": "telegram_token",
"message": "Insert Telegram token",
"when": lambda x: x['telegram']
},
{
"type": "text",
"name": "telegram_chat_id",
"message": "Insert Telegram chat id",
"when": lambda x: x['telegram']
},
]
answers = prompt(questions)
if not answers:
# Interrupted questionary sessions return an empty dict.
raise OperationalException("User interrupted interactive questions.")
return answers
def deploy_new_config(config_path: Path, selections: Dict[str, Any]) -> None:
"""
Applies selections to the template and writes the result to config_path
:param config_path: Path object for new config file. Should not exist yet
:param selecions: Dict containing selections taken by the user.
"""
from jinja2.exceptions import TemplateNotFound
try:
exchange_template = MAP_EXCHANGE_CHILDCLASS.get(
selections['exchange_name'], selections['exchange_name'])
selections['exchange'] = render_template(
templatefile=f"subtemplates/exchange_{exchange_template}.j2",
arguments=selections
)
except TemplateNotFound:
selections['exchange'] = render_template(
templatefile=f"subtemplates/exchange_generic.j2",
arguments=selections
)
config_text = render_template(templatefile='base_config.json.j2',
arguments=selections)
logger.info(f"Writing config to `{config_path}`.")
config_path.write_text(config_text)
def start_new_config(args: Dict[str, Any]) -> None:
"""
Create a new strategy from a template
Asking the user questions to fill out the templateaccordingly.
"""
config_path = Path(args['config'][0])
if config_path.exists():
overwrite = ask_user_overwrite(config_path)
if overwrite:
config_path.unlink()
else:
raise OperationalException(
f"Configuration file `{config_path}` already exists. "
"Please delete it or use a different configuration file name.")
selections = ask_user_config()
deploy_new_config(config_path, selections)

View File

@ -139,5 +139,4 @@ def render_template(templatefile: str, arguments: dict = {}) -> str:
autoescape=select_autoescape(['html', 'xml'])
)
template = env.get_template(templatefile)
return template.render(**arguments)

View File

@ -0,0 +1,58 @@
{
"max_open_trades": {{ max_open_trades }},
"stake_currency": "{{ stake_currency }}",
"stake_amount": {{ stake_amount }},
"tradable_balance_ratio": 0.99,
"fiat_display_currency": "{{ fiat_display_currency }}",
"ticker_interval": "{{ ticker_interval }}",
"dry_run": {{ dry_run | lower }},
"unfilledtimeout": {
"buy": 10,
"sell": 30
},
"bid_strategy": {
"ask_last_balance": 0.0,
"use_order_book": false,
"order_book_top": 1,
"check_depth_of_market": {
"enabled": false,
"bids_to_ask_delta": 1
}
},
"ask_strategy": {
"use_order_book": false,
"order_book_min": 1,
"order_book_max": 9,
"use_sell_signal": true,
"sell_profit_only": false,
"ignore_roi_if_buy_signal": false
},
{{ exchange | indent(4) }},
"pairlists": [
{"method": "StaticPairList"}
],
"edge": {
"enabled": false,
"process_throttle_secs": 3600,
"calculate_since_number_of_days": 7,
"allowed_risk": 0.01,
"stoploss_range_min": -0.01,
"stoploss_range_max": -0.1,
"stoploss_range_step": -0.01,
"minimum_winrate": 0.60,
"minimum_expectancy": 0.20,
"min_trade_number": 10,
"max_trade_duration_minute": 1440,
"remove_pumps": false
},
"telegram": {
"enabled": {{ telegram | lower }},
"token": "{{ telegram_token }}",
"chat_id": "{{ telegram_chat_id }}"
},
"initial_state": "running",
"forcebuy_enable": false,
"internals": {
"process_throttle_secs": 5
}
}

View File

@ -0,0 +1,41 @@
"exchange": {
"name": "{{ exchange_name | lower }}",
"key": "{{ exchange_key }}",
"secret": "{{ exchange_secret }}",
"ccxt_config": {"enableRateLimit": true},
"ccxt_async_config": {
"enableRateLimit": true,
"rateLimit": 200
},
"pair_whitelist": [
"ALGO/BTC",
"ATOM/BTC",
"BAT/BTC",
"BCH/BTC",
"BRD/BTC",
"EOS/BTC",
"ETH/BTC",
"IOTA/BTC",
"LINK/BTC",
"LTC/BTC",
"NEO/BTC",
"NXS/BTC",
"XMR/BTC",
"XRP/BTC",
"XTZ/BTC"
],
"pair_blacklist": [
"BNB/BTC",
"BNB/BUSD",
"BNB/ETH",
"BNB/EUR",
"BNB/NGN",
"BNB/PAX",
"BNB/RUB",
"BNB/TRY",
"BNB/TUSD",
"BNB/USDC",
"BNB/USDS",
"BNB/USDT",
]
}

View File

@ -0,0 +1,24 @@
"exchange": {
"name": "{{ exchange_name | lower }}",
"key": "{{ exchange_key }}",
"secret": "{{ exchange_secret }}",
"ccxt_config": {"enableRateLimit": true},
"ccxt_async_config": {
"enableRateLimit": true,
"rateLimit": 500
},
"pair_whitelist": [
"ETH/BTC",
"LTC/BTC",
"ETC/BTC",
"DASH/BTC",
"ZEC/BTC",
"XLM/BTC",
"XRP/BTC",
"TRX/BTC",
"ADA/BTC",
"XMR/BTC"
],
"pair_blacklist": [
]
}

View File

@ -0,0 +1,15 @@
"exchange": {
"name": "{{ exchange_name | lower }}",
"key": "{{ exchange_key }}",
"secret": "{{ exchange_secret }}",
"ccxt_config": {"enableRateLimit": true},
"ccxt_async_config": {
"enableRateLimit": true
},
"pair_whitelist": [
],
"pair_blacklist": [
]
}

View File

@ -0,0 +1,36 @@
"download_trades": true,
"exchange": {
"name": "kraken",
"key": "{{ exchange_key }}",
"secret": "{{ exchange_secret }}",
"ccxt_config": {"enableRateLimit": true},
"ccxt_async_config": {
"enableRateLimit": true,
"rateLimit": 1000
},
"pair_whitelist": [
"ADA/EUR",
"ATOM/EUR",
"BAT/EUR",
"BCH/EUR",
"BTC/EUR",
"DAI/EUR",
"DASH/EUR",
"EOS/EUR",
"ETC/EUR",
"ETH/EUR",
"LINK/EUR",
"LTC/EUR",
"QTUM/EUR",
"REP/EUR",
"WAVES/EUR",
"XLM/EUR",
"XMR/EUR",
"XRP/EUR",
"XTZ/EUR",
"ZEC/EUR"
],
"pair_blacklist": [
]
}

View File

@ -28,3 +28,6 @@ flask==1.1.1
# Support for colorized terminal output
colorama==0.4.3
# Building config files interactively
questionary==1.5.1
prompt-toolkit==3.0.3

View File

@ -79,6 +79,8 @@ setup(name='freqtrade',
'sdnotify',
'colorama',
'jinja2',
'questionary',
'prompt-toolkit',
# from requirements.txt
'numpy',
'pandas',

View File

@ -17,6 +17,14 @@ function check_installed_python() {
exit 2
fi
which python3.8
if [ $? -eq 0 ]; then
echo "using Python 3.8"
PYTHON=python3.8
check_installed_pip
return
fi
which python3.7
if [ $? -eq 0 ]; then
echo "using Python 3.7"
@ -215,27 +223,8 @@ function config_generator() {
function config() {
echo "-------------------------"
echo "Generating config file"
echo "Please use 'freqtrade new-config -c config.json' to generate a new configuration file."
echo "-------------------------"
if [ -f config.json ]
then
read -p "A config file already exist, do you want to override it [y/N]? "
if [[ $REPLY =~ ^[Yy]$ ]]
then
config_generator
else
echo "Configuration of config.json ignored."
fi
else
config_generator
fi
echo
echo "-------------------------"
echo "Config file generated"
echo "-------------------------"
echo "Edit ./config.json to modify Pair and other configurations."
echo
}
function install() {

View File

@ -0,0 +1,116 @@
from pathlib import Path
from unittest.mock import MagicMock
import pytest
import rapidjson
from freqtrade.commands.build_config_commands import (ask_user_config,
ask_user_overwrite,
start_new_config,
validate_is_float,
validate_is_int)
from freqtrade.exceptions import OperationalException
from tests.conftest import get_args, log_has_re
def test_validate_is_float():
assert validate_is_float('2.0')
assert validate_is_float('2.1')
assert validate_is_float('0.1')
assert validate_is_float('-0.5')
assert not validate_is_float('-0.5e')
def test_validate_is_int():
assert validate_is_int('2')
assert validate_is_int('6')
assert validate_is_int('-1')
assert validate_is_int('500')
assert not validate_is_int('2.0')
assert not validate_is_int('2.1')
assert not validate_is_int('-2.1')
assert not validate_is_int('-ee')
@pytest.mark.parametrize('exchange', ['bittrex', 'binance', 'kraken', 'ftx'])
def test_start_new_config(mocker, caplog, exchange):
wt_mock = mocker.patch.object(Path, "write_text", MagicMock())
mocker.patch.object(Path, "exists", MagicMock(return_value=True))
unlink_mock = mocker.patch.object(Path, "unlink", MagicMock())
mocker.patch('freqtrade.commands.build_config_commands.ask_user_overwrite', return_value=True)
sample_selections = {
'max_open_trades': 3,
'stake_currency': 'USDT',
'stake_amount': 100,
'fiat_display_currency': 'EUR',
'ticker_interval': '15m',
'dry_run': True,
'exchange_name': exchange,
'exchange_key': 'sampleKey',
'exchange_secret': 'Samplesecret',
'telegram': False,
'telegram_token': 'asdf1244',
'telegram_chat_id': '1144444',
}
mocker.patch('freqtrade.commands.build_config_commands.ask_user_config',
return_value=sample_selections)
args = [
"new-config",
"--config",
"coolconfig.json"
]
start_new_config(get_args(args))
assert log_has_re("Writing config to .*", caplog)
assert wt_mock.call_count == 1
assert unlink_mock.call_count == 1
result = rapidjson.loads(wt_mock.call_args_list[0][0][0],
parse_mode=rapidjson.PM_COMMENTS | rapidjson.PM_TRAILING_COMMAS)
assert result['exchange']['name'] == exchange
assert result['ticker_interval'] == '15m'
def test_start_new_config_exists(mocker, caplog):
mocker.patch.object(Path, "exists", MagicMock(return_value=True))
mocker.patch('freqtrade.commands.build_config_commands.ask_user_overwrite', return_value=False)
args = [
"new-config",
"--config",
"coolconfig.json"
]
with pytest.raises(OperationalException, match=r"Configuration .* already exists\."):
start_new_config(get_args(args))
def test_ask_user_overwrite(mocker):
"""
Once https://github.com/tmbo/questionary/issues/35 is implemented, improve this test.
"""
prompt_mock = mocker.patch('freqtrade.commands.build_config_commands.prompt',
return_value={'overwrite': False})
assert not ask_user_overwrite(Path('test.json'))
assert prompt_mock.call_count == 1
prompt_mock.reset_mock()
prompt_mock = mocker.patch('freqtrade.commands.build_config_commands.prompt',
return_value={'overwrite': True})
assert ask_user_overwrite(Path('test.json'))
assert prompt_mock.call_count == 1
def test_ask_user_config(mocker):
"""
Once https://github.com/tmbo/questionary/issues/35 is implemented, improve this test.
"""
prompt_mock = mocker.patch('freqtrade.commands.build_config_commands.prompt',
return_value={'overwrite': False})
answers = ask_user_config()
assert isinstance(answers, dict)
assert prompt_mock.call_count == 1
prompt_mock = mocker.patch('freqtrade.commands.build_config_commands.prompt',
return_value={})
with pytest.raises(OperationalException, match=r"User interrupted interactive questions\."):
ask_user_config()