diff --git a/README.md b/README.md index b5715f743..da691230f 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,7 @@ hesitate to read the source code and understand the mechanism of this bot. - [Strategy Optimization](https://github.com/freqtrade/freqtrade/blob/develop/docs/bot-optimization.md) - [Backtesting](https://github.com/freqtrade/freqtrade/blob/develop/docs/backtesting.md) - [Hyperopt](https://github.com/freqtrade/freqtrade/blob/develop/docs/hyperopt.md) + - [Sandbox Testing](https://github.com/freqtrade/freqtrade/blob/develop/docs/sandbox-testing.md) - [Basic Usage](#basic-usage) - [Bot commands](#bot-commands) - [Telegram RPC commands](#telegram-rpc-commands) @@ -61,6 +62,7 @@ hesitate to read the source code and understand the mechanism of this bot. - [Requirements](#requirements) - [Min hardware required](#min-hardware-required) - [Software requirements](#software-requirements) + ## Quick start diff --git a/docs/index.md b/docs/index.md index f76bb125d..730f1095e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -33,3 +33,4 @@ Pull-request. Do not hesitate to reach us on - [Run tests & Check PEP8 compliance](https://github.com/freqtrade/freqtrade/blob/develop/CONTRIBUTING.md) - [FAQ](https://github.com/freqtrade/freqtrade/blob/develop/docs/faq.md) - [SQL cheatsheet](https://github.com/freqtrade/freqtrade/blob/develop/docs/sql_cheatsheet.md) +- [Sandbox Testing](https://github.com/freqtrade/freqtrade/blob/develop/docs/sandbox-testing.md)) diff --git a/docs/sandbox-testing.md b/docs/sandbox-testing.md new file mode 100644 index 000000000..572fbccef --- /dev/null +++ b/docs/sandbox-testing.md @@ -0,0 +1,151 @@ +# Sandbox API testing +Where an exchange provides a sandbox for risk-free integration, or end-to-end, testing CCXT provides access to these. + +This document is a *light overview of configuring Freqtrade and GDAX sandbox. +This can be useful to developers and trader alike as Freqtrade is quite customisable. + +When testing your API connectivity, make sure to use the following URLs. +***Website** +https://public.sandbox.gdax.com +***REST API** +https://api-public.sandbox.gdax.com + +--- +# Configure a Sandbox account on Gdax +Aim of this document section +- An sanbox account +- create 2FA (needed to create an API) +- Add test 50BTC to account +- Create : +- - API-KEY +- - API-Secret +- - API Password + +## Acccount + +This link will redirect to the sandbox main page to login / create account dialogues: +https://public.sandbox.pro.coinbase.com/orders/ + +After registration and Email confimation you wil be redirected into your sanbox account. It is easy to verify you're in sandbox by checking the URL bar. +> https://public.sandbox.pro.coinbase.com/ + +## Enable 2Fa (a prerequisite to creating sandbox API Keys) +From within sand box site select your profile, top right. +>Or as a direct link: https://public.sandbox.pro.coinbase.com/profile + +From the menu panel to the left of the screen select +> Security: "*View or Update*" + +In the new site select "enable authenticator" as typical google Authenticator. +- open Google Authenticator on your phone +- scan barcode +- enter your generated 2fa + +## Enable API Access +From within sandbox select profile>api>create api-keys +>or as a direct link: https://public.sandbox.pro.coinbase.com/profile/api + +Click on "create one" and ensure **view** and **trade** are "checked" and sumbit your 2Fa +- **Copy and paste the Passphase** into a notepade this will be needed later +- **Copy and paste the API Secret** popup into a notepad this will needed later +- **Copy and paste the API Key** into a notepad this will needed later + +## Add 50 BTC test funds +To add funds, use the web interface deposit and withdraw buttons. + + +To begin select 'Wallets' from the top menu. +> Or as a direct link: https://public.sandbox.pro.coinbase.com/wallets + +- Deposits (bottom left of screen) +- - Deposit Funds Bitcoin +- - - Coinbase BTC Wallet +- - - - Max (50 BTC) +- - - - - Deposit + +*This process may be repeated for other currencies, ETH as example* +--- +# Configure Freqtrade to use Gax Sandbox + +The aim of this document section + - Enable sandbox URLs in Freqtrade + - Configure API + - - secret + - - key + - - passphrase + +## Sandbox URLs +Freqtrade makes use of CCXT which in turn provides a list of URLs to Freqtrade. +These include `['test']` and `['api']`. +- `[Test]` if available will point to an Exchanges sandbox. +- `[Api]` normally used, and resolves to live API target on the exchange + +To make use of sandbox / test add "sandbox": true, to your config.json +``` + "exchange": { + "name": "gdax", + "sandbox": true, + "key": "5wowfxemogxeowo;heiohgmd", + "secret": "/ZMH1P62rCVmwefewrgcewX8nh4gob+lywxfwfxwwfxwfNsH1ySgvWCUR/w==", + "password": "1bkjfkhfhfu6sr", + "pair_whitelist": [ + "BTC/USD" +``` +Also insert your +- api-key (noted earlier) +- api-secret (noted earlier) +- password (the passphrase - noted earlier) + +--- +## You should now be ready to test your sandbox! +Ensure Freqtrade logs show the sandbox URL, and trades made are shown in sandbox. +** Typically the BTC/USD has the most activity in sandbox to test against. + +## GDAX - Old Candles problem +It is my experience that GDAX sandbox candles may be 20+- minutes out of date. This can cause trades to fail as one of Freqtrades safety checks + +To disable this check, edit: +>strategy/interface.py +Look for the following section: +``` + # Check if dataframe is out of date + signal_date = arrow.get(latest['date']) + interval_minutes = constants.TICKER_INTERVAL_MINUTES[interval] + if signal_date < (arrow.utcnow().shift(minutes=-(interval_minutes * 2 + 5))): + logger.warning( + 'Outdated history for pair %s. Last tick is %s minutes old', + pair, + (arrow.utcnow() - signal_date).seconds // 60 + ) + return False, False +``` + +You could Hash out the entire check as follows: +``` + # # Check if dataframe is out of date + # signal_date = arrow.get(latest['date']) + # interval_minutes = constants.TICKER_INTERVAL_MINUTES[interval] + # if signal_date < (arrow.utcnow().shift(minutes=-(interval_minutes * 2 + 5))): + # logger.warning( + # 'Outdated history for pair %s. Last tick is %s minutes old', + # pair, + # (arrow.utcnow() - signal_date).seconds // 60 + # ) + # return False, False + ``` + + Or inrease the timeout to offer a level of protection/alignment of this test to freqtrade in live. + + As example, to allow an additional 30 minutes. "(interval_minutes * 2 + 5 + 30)" + ``` + # Check if dataframe is out of date + signal_date = arrow.get(latest['date']) + interval_minutes = constants.TICKER_INTERVAL_MINUTES[interval] + if signal_date < (arrow.utcnow().shift(minutes=-(interval_minutes * 2 + 5 + 30))): + logger.warning( + 'Outdated history for pair %s. Last tick is %s minutes old', + pair, + (arrow.utcnow() - signal_date).seconds // 60 + ) + return False, False +``` \ No newline at end of file diff --git a/freqtrade/constants.py b/freqtrade/constants.py index 35238d0d1..87e354455 100644 --- a/freqtrade/constants.py +++ b/freqtrade/constants.py @@ -125,6 +125,7 @@ CONF_SCHEMA = { 'type': 'object', 'properties': { 'name': {'type': 'string'}, + 'sandbox': {'type': 'boolean'}, 'key': {'type': 'string'}, 'secret': {'type': 'string'}, 'password': {'type': 'string'}, diff --git a/freqtrade/exchange/__init__.py b/freqtrade/exchange/__init__.py index 0f89eb660..423e38246 100644 --- a/freqtrade/exchange/__init__.py +++ b/freqtrade/exchange/__init__.py @@ -96,6 +96,8 @@ class Exchange(object): except (KeyError, AttributeError): raise OperationalException(f'Exchange {name} is not supported') + self.set_sandbox(api, exchange_config, name) + return api @property @@ -108,6 +110,16 @@ class Exchange(object): """exchange ccxt id""" return self._api.id + def set_sandbox(self, api, exchange_config: dict, name: str): + if exchange_config.get('sandbox'): + if api.urls.get('test'): + api.urls['api'] = api.urls['test'] + logger.info("Enabled Sandbox API on %s", name) + else: + logger.warning(self, "No Sandbox URL in CCXT, exiting. " + "Please check your config.json") + raise OperationalException(f'Exchange {name} does not provide a sandbox api') + def validate_pairs(self, pairs: List[str]) -> None: """ Checks if all given pairs are tradable on the current exchange. diff --git a/freqtrade/tests/exchange/test_exchange.py b/freqtrade/tests/exchange/test_exchange.py index 4acd5c6b2..f149ac82f 100644 --- a/freqtrade/tests/exchange/test_exchange.py +++ b/freqtrade/tests/exchange/test_exchange.py @@ -98,6 +98,47 @@ def test_symbol_price_prec(default_conf, mocker): assert price == 2.3456 +def test_set_sandbox(default_conf, mocker): + """ + Test working scenario + """ + api_mock = MagicMock() + api_mock.load_markets = MagicMock(return_value={ + 'ETH/BTC': '', 'LTC/BTC': '', 'XRP/BTC': '', 'NEO/BTC': '' + }) + url_mock = PropertyMock(return_value={'test': "api-public.sandbox.gdax.com", + 'api': 'https://api.gdax.com'}) + type(api_mock).urls = url_mock + mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock)) + mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock()) + + exchange = Exchange(default_conf) + liveurl = exchange._api.urls['api'] + default_conf['exchange']['sandbox'] = True + exchange.set_sandbox(exchange._api, default_conf['exchange'], 'Logname') + assert exchange._api.urls['api'] != liveurl + + +def test_set_sandbox_exception(default_conf, mocker): + """ + Test Fail scenario + """ + api_mock = MagicMock() + api_mock.load_markets = MagicMock(return_value={ + 'ETH/BTC': '', 'LTC/BTC': '', 'XRP/BTC': '', 'NEO/BTC': '' + }) + url_mock = PropertyMock(return_value={'api': 'https://api.gdax.com'}) + type(api_mock).urls = url_mock + + mocker.patch('freqtrade.exchange.Exchange._init_ccxt', MagicMock(return_value=api_mock)) + mocker.patch('freqtrade.exchange.Exchange.validate_timeframes', MagicMock()) + + with pytest.raises(OperationalException, match=r'does not provide a sandbox api'): + exchange = Exchange(default_conf) + default_conf['exchange']['sandbox'] = True + exchange.set_sandbox(exchange._api, default_conf['exchange'], 'Logname') + + def test_validate_pairs(default_conf, mocker): api_mock = MagicMock() api_mock.load_markets = MagicMock(return_value={