From f658cfa3491a7050debd8f86898e3f751d92b46e Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 11 Jul 2021 11:13:27 +0200 Subject: [PATCH 01/20] Remove Slack As the community is mostly active on discord, there's little point in linking people to Slack as well --- CONTRIBUTING.md | 2 +- README.md | 10 +++------- docs/developer.md | 2 +- docs/faq.md | 2 +- docs/includes/protections.md | 2 +- docs/index.md | 8 ++------ freqtrade/rpc/rpc_manager.py | 4 ++-- 7 files changed, 11 insertions(+), 19 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 040cf3e98..c4ccc1b9f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,7 +12,7 @@ Few pointers for contributions: - New features need to contain unit tests, must conform to PEP8 (max-line-length = 100) and should be documented with the introduction PR. - PR's can be declared as `[WIP]` - which signify Work in Progress Pull Requests (which are not finished). -If you are unsure, discuss the feature on our [discord server](https://discord.gg/p7nuUNVfP7), on [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw) or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a PR. +If you are unsure, discuss the feature on our [discord server](https://discord.gg/p7nuUNVfP7) or in a [issue](https://github.com/freqtrade/freqtrade/issues) before a Pull Request. ## Getting started diff --git a/README.md b/README.md index 4082995f0..8cba78136 100644 --- a/README.md +++ b/README.md @@ -142,13 +142,9 @@ The project is currently setup in two main branches: ## Support -### Help / Discord / Slack +### Help / Discord -For any questions not covered by the documentation or for further information about the bot, or to simply engage with like-minded individuals, we encourage you to join our slack channel. - -Please check out our [discord server](https://discord.gg/p7nuUNVfP7). - -You can also join our [Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw). +For any questions not covered by the documentation or for further information about the bot, or to simply engage with like-minded individuals, we encourage you to join the Freqtrade [discord server](https://discord.gg/p7nuUNVfP7). ### [Bugs / Issues](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue) @@ -179,7 +175,7 @@ to understand the requirements before sending your pull-requests. Coding is not a necessity to contribute - maybe start with improving our documentation? Issues labeled [good first issue](https://github.com/freqtrade/freqtrade/labels/good%20first%20issue) can be good first contributions, and will help get you familiar with the codebase. -**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [discord](https://discord.gg/p7nuUNVfP7) or [Slack](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it. +**Note** before starting any major new feature work, *please open an issue describing what you are planning to do* or talk to us on [discord](https://discord.gg/p7nuUNVfP7) (please use the #dev channel for this). This will ensure that interested parties can give valuable feedback on the feature, and let others know that you are working on it. **Important:** Always create your PR against the `develop` branch, not `stable`. diff --git a/docs/developer.md b/docs/developer.md index d0731a233..dd56a367c 100644 --- a/docs/developer.md +++ b/docs/developer.md @@ -2,7 +2,7 @@ This page is intended for developers of Freqtrade, people who want to contribute to the Freqtrade codebase or documentation, or people who want to understand the source code of the application they're running. -All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel on [discord](https://discord.gg/p7nuUNVfP7) or [slack](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw) where you can ask questions. +All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome. We [track issues](https://github.com/freqtrade/freqtrade/issues) on [GitHub](https://github.com) and also have a dev channel on [discord](https://discord.gg/p7nuUNVfP7) where you can ask questions. ## Documentation diff --git a/docs/faq.md b/docs/faq.md index d015ae50e..b8a3a44d8 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -172,7 +172,7 @@ freqtrade hyperopt --hyperopt SampleHyperopt --hyperopt-loss SharpeHyperOptLossD ### Why does it take a long time to run hyperopt? -* Discovering a great strategy with Hyperopt takes time. Study www.freqtrade.io, the Freqtrade Documentation page, join the Freqtrade [Slack community](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw) - or the Freqtrade [discord community](https://discord.gg/p7nuUNVfP7). While you patiently wait for the most advanced, free crypto bot in the world, to hand you a possible golden strategy specially designed just for you. +* Discovering a great strategy with Hyperopt takes time. Study www.freqtrade.io, the Freqtrade Documentation page, join the Freqtrade [discord community](https://discord.gg/p7nuUNVfP7). While you patiently wait for the most advanced, free crypto bot in the world, to hand you a possible golden strategy specially designed just for you. * If you wonder why it can take from 20 minutes to days to do 1000 epochs here are some answers: diff --git a/docs/includes/protections.md b/docs/includes/protections.md index 3ea2dde61..5dcc83738 100644 --- a/docs/includes/protections.md +++ b/docs/includes/protections.md @@ -1,7 +1,7 @@ ## Protections !!! Warning "Beta feature" - This feature is still in it's testing phase. Should you notice something you think is wrong please let us know via Discord, Slack or via Github Issue. + This feature is still in it's testing phase. Should you notice something you think is wrong please let us know via Discord or via Github Issue. Protections will protect your strategy from unexpected events and market conditions by temporarily stop trading for either one pair, or for all pairs. All protection end times are rounded up to the next candle to avoid sudden, unexpected intra-candle buys. diff --git a/docs/index.md b/docs/index.md index 8ecb085de..8077cd303 100644 --- a/docs/index.md +++ b/docs/index.md @@ -73,13 +73,9 @@ Alternatively ## Support -### Help / Discord / Slack +### Help / Discord -For any questions not covered by the documentation or for further information about the bot, or to simply engage with like-minded individuals, we encourage you to join our slack channel. - -Please check out our [discord server](https://discord.gg/p7nuUNVfP7). - -You can also join our [Slack channel](https://join.slack.com/t/highfrequencybot/shared_invite/zt-mm786y93-Fxo37glxMY9g8OQC5AoOIw). +For any questions not covered by the documentation or for further information about the bot, or to simply engage with like-minded individuals, we encourage you to join the Freqtrade [discord server](https://discord.gg/p7nuUNVfP7). ## Ready to try? diff --git a/freqtrade/rpc/rpc_manager.py b/freqtrade/rpc/rpc_manager.py index 18ed68041..b3f45ab41 100644 --- a/freqtrade/rpc/rpc_manager.py +++ b/freqtrade/rpc/rpc_manager.py @@ -1,5 +1,5 @@ """ -This module contains class to manage RPC communications (Telegram, Slack, ...) +This module contains class to manage RPC communications (Telegram, API, ...) """ import logging from typing import Any, Dict, List @@ -13,7 +13,7 @@ logger = logging.getLogger(__name__) class RPCManager: """ - Class to manage RPC objects (Telegram, Slack, ...) + Class to manage RPC objects (Telegram, API, ...) """ def __init__(self, freqtrade) -> None: """ Initializes all enabled rpc modules """ From 0e4466ca1e46ebad19cc28a8ef9ac21123c48de4 Mon Sep 17 00:00:00 2001 From: Rokas Kupstys Date: Sun, 11 Jul 2021 12:20:31 +0300 Subject: [PATCH 02/20] Implement strategy-controlled stake sizes. Expose `self.wallet` to a strategy. --- docs/strategy-advanced.md | 31 ++++++++++++++++++++++ freqtrade/freqtradebot.py | 27 ++++++++++--------- freqtrade/optimize/backtesting.py | 15 ++++++++++- freqtrade/strategy/interface.py | 17 ++++++++++++ freqtrade/wallets.py | 43 +++++++++++++++++++++++++++---- tests/rpc/test_rpc.py | 2 +- tests/test_freqtradebot.py | 20 ++++++++++++++ 7 files changed, 136 insertions(+), 19 deletions(-) diff --git a/docs/strategy-advanced.md b/docs/strategy-advanced.md index b06cf3ecb..e2fd30575 100644 --- a/docs/strategy-advanced.md +++ b/docs/strategy-advanced.md @@ -521,6 +521,37 @@ class AwesomeStrategy(IStrategy): ``` +### Stake size management + +It is possible to manage your risk by reducing or increasing or reducing stake amount when placing a new trade. + +```python +class AwesomeStrategy(IStrategy): + def custom_stake_amount(self, pair: str, current_time: datetime, current_rate: float, + proposed_stake: float, min_stake: float, max_stake: float, + **kwargs) -> float: + + dataframe, _ = self.dp.get_analyzed_dataframe(pair=pair, timeframe=self.timeframe) + current_candle = dataframe.iloc[-1].squeeze() + + if current_candle['fastk_rsi_1h'] > current_candle['fastd_rsi_1h']: + if self.config['stake_amount'] == 'unlimited': + # Use entire available wallet during favorable conditions when in compounding mode. + return max_stake + else: + # Compound profits during favorable conditions instead of using a static stake. + return self.wallets.get_total_stake_amount() / self.config['max_open_trades'] + + # Use default stake amount. + return proposed_stake +``` + +!!! Tip + You do not _have_ to ensure that `min_stake <= returned_value <= max_stake`. Trades will succeed, as returned value will be clamped to supported range and this acton will be logged. + +!!! Tip + Returning `0` or `None` will prevent trades from being placed. + --- ## Derived strategies diff --git a/freqtrade/freqtradebot.py b/freqtrade/freqtradebot.py index 7a7371357..efbf15fca 100644 --- a/freqtrade/freqtradebot.py +++ b/freqtrade/freqtradebot.py @@ -424,16 +424,10 @@ class FreqtradeBot(LoggingMixin): if buy and not sell: stake_amount = self.wallets.get_trade_stake_amount(pair, self.edge) - if not stake_amount: - logger.debug(f"Stake amount is 0, ignoring possible trade for {pair}.") - return False - - logger.info(f"Buy signal found: about create a new trade for {pair} with stake_amount: " - f"{stake_amount} ...") bid_check_dom = self.config.get('bid_strategy', {}).get('check_depth_of_market', {}) if ((bid_check_dom.get('enabled', False)) and - (bid_check_dom.get('bids_to_ask_delta', 0) > 0)): + (bid_check_dom.get('bids_to_ask_delta', 0) > 0)): if self._check_depth_of_market_buy(pair, bid_check_dom): return self.execute_buy(pair, stake_amount) else: @@ -488,13 +482,22 @@ class FreqtradeBot(LoggingMixin): min_stake_amount = self.exchange.get_min_pair_stake_amount(pair, buy_limit_requested, self.strategy.stoploss) - if min_stake_amount is not None and min_stake_amount > stake_amount: - logger.warning( - f"Can't open a new trade for {pair}: stake amount " - f"is too small ({stake_amount} < {min_stake_amount})" - ) + + if not self.edge: + max_stake_amount = self.wallets.get_available_stake_amount() + stake_amount = strategy_safe_wrapper(self.strategy.custom_stake_amount, + default_retval=None)( + pair=pair, current_time=datetime.now(timezone.utc), + current_rate=buy_limit_requested, proposed_stake=stake_amount, + min_stake=min_stake_amount, max_stake=max_stake_amount) + stake_amount = self.wallets._validate_stake_amount(pair, stake_amount, min_stake_amount) + + if not stake_amount: return False + logger.info(f"Buy signal found: about create a new trade for {pair} with stake_amount: " + f"{stake_amount} ...") + amount = stake_amount / buy_limit_requested order_type = self.strategy.order_types['buy'] if forcebuy: diff --git a/freqtrade/optimize/backtesting.py b/freqtrade/optimize/backtesting.py index 8f818047d..885e67e0b 100644 --- a/freqtrade/optimize/backtesting.py +++ b/freqtrade/optimize/backtesting.py @@ -129,6 +129,8 @@ class Backtesting: """ self.strategy: IStrategy = strategy strategy.dp = self.dataprovider + # Attach Wallets to Strategy baseclass + IStrategy.wallets = self.wallets # Set stoploss_on_exchange to false for backtesting, # since a "perfect" stoploss-sell is assumed anyway # And the regular "stoploss" function would not apply to that case @@ -312,7 +314,18 @@ class Backtesting: stake_amount = self.wallets.get_trade_stake_amount(pair, None) except DependencyException: return None - min_stake_amount = self.exchange.get_min_pair_stake_amount(pair, row[OPEN_IDX], -0.05) + + min_stake_amount = self.exchange.get_min_pair_stake_amount(pair, row[OPEN_IDX], -0.05) or 0 + max_stake_amount = self.wallets.get_available_stake_amount() + + stake_amount = strategy_safe_wrapper(self.strategy.custom_stake_amount, + default_retval=None)( + pair=pair, current_time=row[DATE_IDX].to_pydatetime(), current_rate=row[OPEN_IDX], + proposed_stake=stake_amount, min_stake=min_stake_amount, max_stake=max_stake_amount) + stake_amount = self.wallets._validate_stake_amount(pair, stake_amount, min_stake_amount) + + if not stake_amount: + return None order_type = self.strategy.order_types['buy'] time_in_force = self.strategy.order_time_in_force['sell'] diff --git a/freqtrade/strategy/interface.py b/freqtrade/strategy/interface.py index 26bcb0369..f9772e1df 100644 --- a/freqtrade/strategy/interface.py +++ b/freqtrade/strategy/interface.py @@ -304,6 +304,23 @@ class IStrategy(ABC, HyperStrategyMixin): """ return None + def custom_stake_amount(self, pair: str, current_time: datetime, current_rate: float, + proposed_stake: float, min_stake: float, max_stake: float, + **kwargs) -> float: + """ + Customize stake size for each new trade. This method is not called when edge module is + enabled. + + :param pair: Pair that's currently analyzed + :param current_time: datetime object, containing the current datetime + :param current_rate: Rate, calculated based on pricing settings in ask_strategy. + :param proposed_stake: A stake amount proposed by the bot. + :param min_stake: Minimal stake size allowed by exchange. + :param max_stake: Balance available for trading. + :return: A stake size, which is between min_stake and max_stake. + """ + return proposed_stake + def informative_pairs(self) -> ListPairsWithTimeframes: """ Define additional, informative pair/interval combinations to be cached from the exchange. diff --git a/freqtrade/wallets.py b/freqtrade/wallets.py index 1b2ec4550..42144dad7 100644 --- a/freqtrade/wallets.py +++ b/freqtrade/wallets.py @@ -131,7 +131,22 @@ class Wallets: def get_all_balances(self) -> Dict[str, Any]: return self._wallets - def _get_available_stake_amount(self, val_tied_up: float) -> float: + def get_total_stake_amount(self): + """ + Return the total currently available balance in stake currency, including tied up stake and + respecting tradable_balance_ratio. + Calculated as + ( + free amount) * tradable_balance_ratio + """ + # Ensure % is used from the overall balance + # Otherwise we'd risk lowering stakes with each open trade. + # (tied up + current free) * ratio) - tied up + val_tied_up = Trade.total_open_trades_stakes() + available_amount = ((val_tied_up + self.get_free(self._config['stake_currency'])) * + self._config['tradable_balance_ratio']) + return available_amount + + def get_available_stake_amount(self) -> float: """ Return the total currently available balance in stake currency, respecting tradable_balance_ratio. @@ -142,9 +157,7 @@ class Wallets: # Ensure % is used from the overall balance # Otherwise we'd risk lowering stakes with each open trade. # (tied up + current free) * ratio) - tied up - available_amount = ((val_tied_up + self.get_free(self._config['stake_currency'])) * - self._config['tradable_balance_ratio']) - val_tied_up - return available_amount + return self.get_total_stake_amount() - Trade.total_open_trades_stakes() def _calculate_unlimited_stake_amount(self, available_amount: float, val_tied_up: float) -> float: @@ -193,7 +206,7 @@ class Wallets: # Ensure wallets are uptodate. self.update() val_tied_up = Trade.total_open_trades_stakes() - available_amount = self._get_available_stake_amount(val_tied_up) + available_amount = self.get_available_stake_amount() if edge: stake_amount = edge.stake_amount( @@ -209,3 +222,23 @@ class Wallets: available_amount, val_tied_up) return self._check_available_stake_amount(stake_amount, available_amount) + + def _validate_stake_amount(self, pair, stake_amount, min_stake_amount): + if not stake_amount: + logger.debug(f"Stake amount is {stake_amount}, ignoring possible trade for {pair}.") + return 0 + + max_stake_amount = self.get_available_stake_amount() + if min_stake_amount is not None and stake_amount < min_stake_amount: + stake_amount = min_stake_amount + logger.info( + f"Stake amount for pair {pair} is too small ({stake_amount} < {min_stake_amount}), " + f"adjusting to {min_stake_amount}." + ) + if stake_amount > max_stake_amount: + stake_amount = max_stake_amount + logger.info( + f"Stake amount for pair {pair} is too big ({stake_amount} > {max_stake_amount}), " + f"adjusting to {max_stake_amount}." + ) + return stake_amount diff --git a/tests/rpc/test_rpc.py b/tests/rpc/test_rpc.py index cc29dc157..0049e59bb 100644 --- a/tests/rpc/test_rpc.py +++ b/tests/rpc/test_rpc.py @@ -886,7 +886,7 @@ def test_rpcforcebuy(mocker, default_conf, ticker, fee, limit_buy_order_open) -> # Test not buying freqtradebot = get_patched_freqtradebot(mocker, default_conf) - freqtradebot.config['stake_amount'] = 0.0000001 + freqtradebot.config['stake_amount'] = 0 patch_get_signal(freqtradebot, (True, False)) rpc = RPC(freqtradebot) pair = 'TKN/BTC' diff --git a/tests/test_freqtradebot.py b/tests/test_freqtradebot.py index 99e11e893..117589b12 100644 --- a/tests/test_freqtradebot.py +++ b/tests/test_freqtradebot.py @@ -413,6 +413,26 @@ def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_ord patch_get_signal(freqtrade) + assert freqtrade.create_trade('ETH/BTC') + + +def test_create_trade_zero_stake_amount(default_conf, ticker, limit_buy_order_open, + fee, mocker) -> None: + patch_RPCManager(mocker) + patch_exchange(mocker) + buy_mock = MagicMock(return_value=limit_buy_order_open) + mocker.patch.multiple( + 'freqtrade.exchange.Exchange', + fetch_ticker=ticker, + buy=buy_mock, + get_fee=fee, + ) + + freqtrade = FreqtradeBot(default_conf) + freqtrade.config['stake_amount'] = 0 + + patch_get_signal(freqtrade) + assert not freqtrade.create_trade('ETH/BTC') From 7ea0a74c532780a89b5fd885c804e437fe0291b9 Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 11 Jul 2021 14:10:41 +0200 Subject: [PATCH 03/20] Default to proposed stake --- docs/strategy-advanced.md | 6 ++++-- freqtrade/freqtradebot.py | 2 +- freqtrade/optimize/backtesting.py | 2 +- tests/optimize/test_backtesting.py | 11 +++++++++++ tests/test_freqtradebot.py | 21 ++++++++++++++++++++- 5 files changed, 37 insertions(+), 5 deletions(-) diff --git a/docs/strategy-advanced.md b/docs/strategy-advanced.md index e2fd30575..4dba9ad4a 100644 --- a/docs/strategy-advanced.md +++ b/docs/strategy-advanced.md @@ -523,7 +523,7 @@ class AwesomeStrategy(IStrategy): ### Stake size management -It is possible to manage your risk by reducing or increasing or reducing stake amount when placing a new trade. +It is possible to manage your risk by reducing or increasing stake amount when placing a new trade. ```python class AwesomeStrategy(IStrategy): @@ -546,8 +546,10 @@ class AwesomeStrategy(IStrategy): return proposed_stake ``` +Freqtrade will fall back to the `proposed_stake` value should your code raise an exception. The exception itself will be logged. + !!! Tip - You do not _have_ to ensure that `min_stake <= returned_value <= max_stake`. Trades will succeed, as returned value will be clamped to supported range and this acton will be logged. + You do not _have_ to ensure that `min_stake <= returned_value <= max_stake`. Trades will succeed as the returned value will be clamped to supported range and this acton will be logged. !!! Tip Returning `0` or `None` will prevent trades from being placed. diff --git a/freqtrade/freqtradebot.py b/freqtrade/freqtradebot.py index efbf15fca..5ef109387 100644 --- a/freqtrade/freqtradebot.py +++ b/freqtrade/freqtradebot.py @@ -486,7 +486,7 @@ class FreqtradeBot(LoggingMixin): if not self.edge: max_stake_amount = self.wallets.get_available_stake_amount() stake_amount = strategy_safe_wrapper(self.strategy.custom_stake_amount, - default_retval=None)( + default_retval=stake_amount)( pair=pair, current_time=datetime.now(timezone.utc), current_rate=buy_limit_requested, proposed_stake=stake_amount, min_stake=min_stake_amount, max_stake=max_stake_amount) diff --git a/freqtrade/optimize/backtesting.py b/freqtrade/optimize/backtesting.py index 885e67e0b..83cd8c2bb 100644 --- a/freqtrade/optimize/backtesting.py +++ b/freqtrade/optimize/backtesting.py @@ -319,7 +319,7 @@ class Backtesting: max_stake_amount = self.wallets.get_available_stake_amount() stake_amount = strategy_safe_wrapper(self.strategy.custom_stake_amount, - default_retval=None)( + default_retval=stake_amount)( pair=pair, current_time=row[DATE_IDX].to_pydatetime(), current_rate=row[OPEN_IDX], proposed_stake=stake_amount, min_stake=min_stake_amount, max_stake=max_stake_amount) stake_amount = self.wallets._validate_stake_amount(pair, stake_amount, min_stake_amount) diff --git a/tests/optimize/test_backtesting.py b/tests/optimize/test_backtesting.py index 30d86f979..a1881c306 100644 --- a/tests/optimize/test_backtesting.py +++ b/tests/optimize/test_backtesting.py @@ -496,6 +496,17 @@ def test_backtest__enter_trade(default_conf, fee, mocker) -> None: trade = backtesting._enter_trade(pair, row=row) assert trade is not None + backtesting.strategy.custom_stake_amount = lambda **kwargs: 123.5 + trade = backtesting._enter_trade(pair, row=row) + assert trade + assert trade.stake_amount == 123.5 + + # In case of error - use proposed stake + backtesting.strategy.custom_stake_amount = lambda **kwargs: 20 / 0 + trade = backtesting._enter_trade(pair, row=row) + assert trade + assert trade.stake_amount == 495 + # Stake-amount too high! mocker.patch("freqtrade.exchange.Exchange.get_min_pair_stake_amount", return_value=600.0) diff --git a/tests/test_freqtradebot.py b/tests/test_freqtradebot.py index 117589b12..addf72bbb 100644 --- a/tests/test_freqtradebot.py +++ b/tests/test_freqtradebot.py @@ -397,7 +397,7 @@ def test_create_trade_minimal_amount(default_conf, ticker, limit_buy_order_open, def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_order_open, - fee, mocker) -> None: + fee, mocker, caplog) -> None: patch_RPCManager(mocker) patch_exchange(mocker) buy_mock = MagicMock(return_value=limit_buy_order_open) @@ -414,6 +414,7 @@ def test_create_trade_too_small_stake_amount(default_conf, ticker, limit_buy_ord patch_get_signal(freqtrade) assert freqtrade.create_trade('ETH/BTC') + assert log_has_re(r"Stake amount for pair .* is too small.*", caplog) def test_create_trade_zero_stake_amount(default_conf, ticker, limit_buy_order_open, @@ -862,6 +863,24 @@ def test_execute_buy(mocker, default_conf, fee, limit_buy_order, limit_buy_order assert trade.open_rate == 0.5 assert trade.stake_amount == 40.495905365 + # Test with custom stake + limit_buy_order['status'] = 'open' + limit_buy_order['id'] = '556' + + freqtrade.strategy.custom_stake_amount = lambda **kwargs: 150.0 + assert freqtrade.execute_buy(pair, stake_amount) + trade = Trade.query.all()[4] + assert trade + assert trade.stake_amount == 150 + + # Exception case + limit_buy_order['id'] = '557' + freqtrade.strategy.custom_stake_amount = lambda **kwargs: 20 / 0 + assert freqtrade.execute_buy(pair, stake_amount) + trade = Trade.query.all()[5] + assert trade + assert trade.stake_amount == 2.0 + # In case of the order is rejected and not filled at all limit_buy_order['status'] = 'rejected' limit_buy_order['amount'] = 90.99181073 From 8b78a3bde2b41e53061f4621982166bb5ece694e Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 11 Jul 2021 21:01:12 +0200 Subject: [PATCH 04/20] Quick fix for trades opening below min-trade amount --- freqtrade/wallets.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/freqtrade/wallets.py b/freqtrade/wallets.py index 42144dad7..ece496c90 100644 --- a/freqtrade/wallets.py +++ b/freqtrade/wallets.py @@ -229,6 +229,10 @@ class Wallets: return 0 max_stake_amount = self.get_available_stake_amount() + + if min_stake_amount > max_stake_amount: + logger.warning("Minimum stake amount > available balance.") + return 0 if min_stake_amount is not None and stake_amount < min_stake_amount: stake_amount = min_stake_amount logger.info( From f4caf9b93c9bc985530d45c8b3aee8db22d3f7f5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Jul 2021 03:01:29 +0000 Subject: [PATCH 05/20] Bump ccxt from 1.52.40 to 1.52.83 Bumps [ccxt](https://github.com/ccxt/ccxt) from 1.52.40 to 1.52.83. - [Release notes](https://github.com/ccxt/ccxt/releases) - [Changelog](https://github.com/ccxt/ccxt/blob/master/exchanges.cfg) - [Commits](https://github.com/ccxt/ccxt/compare/1.52.40...1.52.83) --- updated-dependencies: - dependency-name: ccxt dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements.txt b/requirements.txt index 528dc2ce6..08a04d5f5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ numpy==1.21.0 pandas==1.3.0 -ccxt==1.52.40 +ccxt==1.52.83 # Pin cryptography for now due to rust build errors with piwheels cryptography==3.4.7 aiohttp==3.7.4.post0 From 21ef08d9a8b4d49873f70c3ba39bf3099a39fff1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Jul 2021 03:01:34 +0000 Subject: [PATCH 06/20] Bump mkdocs-material from 7.1.9 to 7.1.10 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 7.1.9 to 7.1.10. - [Release notes](https://github.com/squidfunk/mkdocs-material/releases) - [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/docs/changelog.md) - [Commits](https://github.com/squidfunk/mkdocs-material/compare/7.1.9...7.1.10) --- updated-dependencies: - dependency-name: mkdocs-material dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- docs/requirements-docs.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/requirements-docs.txt b/docs/requirements-docs.txt index d11e5ea4e..e346fe5a5 100644 --- a/docs/requirements-docs.txt +++ b/docs/requirements-docs.txt @@ -1,4 +1,4 @@ mkdocs==1.2.1 -mkdocs-material==7.1.9 +mkdocs-material==7.1.10 mdx_truly_sane_lists==1.2 pymdown-extensions==8.2 From 81c50aca0116f9f14b1d73ed4aa7df151cb6696b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Jul 2021 03:01:38 +0000 Subject: [PATCH 07/20] Bump isort from 5.9.1 to 5.9.2 Bumps [isort](https://github.com/pycqa/isort) from 5.9.1 to 5.9.2. - [Release notes](https://github.com/pycqa/isort/releases) - [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md) - [Commits](https://github.com/pycqa/isort/compare/5.9.1...5.9.2) --- updated-dependencies: - dependency-name: isort dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- requirements-dev.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements-dev.txt b/requirements-dev.txt index c73acbe9c..e25f810cd 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -13,7 +13,7 @@ pytest-asyncio==0.15.1 pytest-cov==2.12.1 pytest-mock==3.6.1 pytest-random-order==1.0.4 -isort==5.9.1 +isort==5.9.2 # Convert jupyter notebooks to markdown documents nbconvert==6.1.0 From ed77889d6b4f51a3ee847de4888c6b4f030f35b0 Mon Sep 17 00:00:00 2001 From: Matthias Date: Mon, 12 Jul 2021 06:52:59 +0200 Subject: [PATCH 08/20] Add explicit tests for _validate_stake_amount --- tests/test_wallets.py | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tests/test_wallets.py b/tests/test_wallets.py index ff303e2ec..d083a63f6 100644 --- a/tests/test_wallets.py +++ b/tests/test_wallets.py @@ -170,3 +170,22 @@ def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_r freqtrade.config['max_open_trades'] = 0 result = freqtrade.wallets.get_trade_stake_amount('NEO/USDT') assert result == 0 + + +@pytest.mark.parametrize('stake_amount,min_stake_amount,max_stake_amount,expected', [ + (22, 11, 50, 22), + (100, 11, 500, 100), + (1000, 11, 500, 500), # Above max-stake + (20, 15, 10, 0), # Minimum stake > max-stake + (1, 11, 100, 11), # Below min stake + (1, 15, 10, 0), # Below min stake and min_stake > max_stake + +]) +def test__validate_stake_amount(mocker, default_conf, + stake_amount, min_stake_amount, max_stake_amount, expected): + freqtrade = get_patched_freqtradebot(mocker, default_conf) + + mocker.patch("freqtrade.wallets.Wallets.get_available_stake_amount", + return_value=max_stake_amount) + res = freqtrade.wallets._validate_stake_amount('XRP/USDT', stake_amount, min_stake_amount) + assert res == expected From b41c2344405c438eb3f81e60eb8eeb3cef1510ad Mon Sep 17 00:00:00 2001 From: Matthias Date: Sat, 10 Jul 2021 12:18:55 +0200 Subject: [PATCH 09/20] Extract Closed profit calculation to trade object --- freqtrade/persistence/models.py | 13 +++++++++++++ freqtrade/wallets.py | 4 +--- tests/test_persistence.py | 16 ++++++++++++++++ 3 files changed, 30 insertions(+), 3 deletions(-) diff --git a/freqtrade/persistence/models.py b/freqtrade/persistence/models.py index b4c299120..8dcfc6c94 100644 --- a/freqtrade/persistence/models.py +++ b/freqtrade/persistence/models.py @@ -801,6 +801,19 @@ class Trade(_DECL_BASE, LocalTrade): Trade.is_open.is_(False), ]).all() + @staticmethod + def get_total_closed_profit() -> float: + """ + Retrieves total realized profit + """ + if Trade.use_db: + total_profit = Trade.query.with_entities( + func.sum(Trade.close_profit_abs)).filter(Trade.is_open.is_(False)).scalar() + else: + total_profit = sum( + t.close_profit_abs for t in LocalTrade.get_trades_proxy(is_open=False)) + return total_profit or 0 + @staticmethod def total_open_trades_stakes() -> float: """ diff --git a/freqtrade/wallets.py b/freqtrade/wallets.py index ece496c90..3d80cc892 100644 --- a/freqtrade/wallets.py +++ b/freqtrade/wallets.py @@ -70,9 +70,7 @@ class Wallets: # If not backtesting... # TODO: potentially remove the ._log workaround to determine backtest mode. if self._log: - closed_trades = Trade.get_trades_proxy(is_open=False) - tot_profit = sum( - [trade.close_profit_abs for trade in closed_trades if trade.close_profit_abs]) + tot_profit = Trade.get_total_closed_profit() else: tot_profit = LocalTrade.total_profit tot_in_trades = sum([trade.stake_amount for trade in open_trades]) diff --git a/tests/test_persistence.py b/tests/test_persistence.py index 1576aaa5a..89d07ca74 100644 --- a/tests/test_persistence.py +++ b/tests/test_persistence.py @@ -1124,6 +1124,21 @@ def test_total_open_trades_stakes(fee, use_db): Trade.use_db = True +@pytest.mark.usefixtures("init_persistence") +@pytest.mark.parametrize('use_db', [True, False]) +def test_get_total_closed_profit(fee, use_db): + + Trade.use_db = use_db + Trade.reset_trades() + res = Trade.get_total_closed_profit() + assert res == 0 + create_mock_trades(fee, use_db) + res = Trade.get_total_closed_profit() + assert res == 0.000739127 + + Trade.use_db = True + + @pytest.mark.usefixtures("init_persistence") @pytest.mark.parametrize('use_db', [True, False]) def test_get_trades_proxy(fee, use_db): @@ -1298,6 +1313,7 @@ def test_Trade_object_idem(): 'open_date', 'get_best_pair', 'get_overall_performance', + 'get_total_closed_profit', 'total_open_trades_stakes', 'get_sold_trades_without_assigned_fees', 'get_open_trades_without_assigned_fees', From 786374690406d00cad290a583c87972c436dc527 Mon Sep 17 00:00:00 2001 From: Matthias Date: Sat, 10 Jul 2021 12:30:00 +0200 Subject: [PATCH 10/20] Add available_capital parameter --- freqtrade/constants.py | 4 ++++ freqtrade/wallets.py | 23 ++++++++++++++--------- 2 files changed, 18 insertions(+), 9 deletions(-) diff --git a/freqtrade/constants.py b/freqtrade/constants.py index acd143708..2f93ace1c 100644 --- a/freqtrade/constants.py +++ b/freqtrade/constants.py @@ -113,6 +113,10 @@ CONF_SCHEMA = { 'maximum': 1, 'default': 0.99 }, + 'available_capital': { + 'type': 'number', + 'minimum': 0, + }, 'amend_last_stake_amount': {'type': 'boolean', 'default': False}, 'last_stake_amount_min_ratio': { 'type': 'number', 'minimum': 0.0, 'maximum': 1.0, 'default': 0.5 diff --git a/freqtrade/wallets.py b/freqtrade/wallets.py index 3d80cc892..0ece65b2b 100644 --- a/freqtrade/wallets.py +++ b/freqtrade/wallets.py @@ -136,12 +136,18 @@ class Wallets: Calculated as ( + free amount) * tradable_balance_ratio """ - # Ensure % is used from the overall balance - # Otherwise we'd risk lowering stakes with each open trade. - # (tied up + current free) * ratio) - tied up val_tied_up = Trade.total_open_trades_stakes() - available_amount = ((val_tied_up + self.get_free(self._config['stake_currency'])) * - self._config['tradable_balance_ratio']) + if "available_capital" in self._config: + starting_balance = self._config['available_capital'] + tot_profit = Trade.get_total_closed_profit() + available_amount = starting_balance + tot_profit + + else: + # Ensure % is used from the overall balance + # Otherwise we'd risk lowering stakes with each open trade. + # (tied up + current free) * ratio) - tied up + available_amount = ((val_tied_up + self.get_free(self._config['stake_currency'])) * + self._config['tradable_balance_ratio']) return available_amount def get_available_stake_amount(self) -> float: @@ -152,10 +158,9 @@ class Wallets: ( + free amount) * tradable_balance_ratio - """ - # Ensure % is used from the overall balance - # Otherwise we'd risk lowering stakes with each open trade. - # (tied up + current free) * ratio) - tied up - return self.get_total_stake_amount() - Trade.total_open_trades_stakes() + + free = self.get_free(self._config['stake_currency']) + return min(self.get_total_stake_amount() - Trade.total_open_trades_stakes(), free) def _calculate_unlimited_stake_amount(self, available_amount: float, val_tied_up: float) -> float: From 6a8e8875a2a7893e9ad28eebce7e6cddd912e833 Mon Sep 17 00:00:00 2001 From: Matthias Date: Sat, 10 Jul 2021 12:39:02 +0200 Subject: [PATCH 11/20] Test new behaviour --- tests/test_wallets.py | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/tests/test_wallets.py b/tests/test_wallets.py index d083a63f6..a44ca243b 100644 --- a/tests/test_wallets.py +++ b/tests/test_wallets.py @@ -121,13 +121,19 @@ def test_get_trade_stake_amount_no_stake_amount(default_conf, mocker) -> None: freqtrade.wallets.get_trade_stake_amount('ETH/BTC') -@pytest.mark.parametrize("balance_ratio,result1,result2", [ - (1, 50, 66.66666), - (0.99, 49.5, 66.0), - (0.50, 25, 33.3333), +@pytest.mark.parametrize("balance_ratio,capital,result1,result2", [ + (1, None, 50, 66.66666), + (0.99, None, 49.5, 66.0), + (0.50, None, 25, 33.3333), + # Tests with capital ignore balance_ratio + (1, 100, 50, 0.0), + (0.99, 200, 50, 66.66666), + (0.99, 150, 50, 50), + (0.50, 50, 25, 0.0), + (0.50, 10, 5, 0.0), ]) -def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_ratio, result1, - result2, limit_buy_order_open, +def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_ratio, capital, + result1, result2, limit_buy_order_open, fee, mocker) -> None: mocker.patch.multiple( 'freqtrade.exchange.Exchange', @@ -141,6 +147,8 @@ def test_get_trade_stake_amount_unlimited_amount(default_conf, ticker, balance_r conf['dry_run_wallet'] = 100 conf['max_open_trades'] = 2 conf['tradable_balance_ratio'] = balance_ratio + if capital is not None: + conf['available_capital'] = capital freqtrade = get_patched_freqtradebot(mocker, conf) From 40db424363556d95cfbfd5cd71faa9a50ca86af6 Mon Sep 17 00:00:00 2001 From: Matthias Date: Sun, 11 Jul 2021 10:44:36 +0200 Subject: [PATCH 12/20] Add documentation for available capital setting --- docs/configuration.md | 17 +++++++++++++++++ freqtrade/wallets.py | 1 - 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/docs/configuration.md b/docs/configuration.md index 5c6236e58..ad7436cb2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -52,6 +52,7 @@ Mandatory parameters are marked as **Required**, which means that they are requi | `stake_currency` | **Required.** Crypto-currency used for trading.
**Datatype:** String | `stake_amount` | **Required.** Amount of crypto-currency your bot will use for each trade. Set it to `"unlimited"` to allow the bot to use all available balance. [More information below](#configuring-amount-per-trade).
**Datatype:** Positive float or `"unlimited"`. | `tradable_balance_ratio` | Ratio of the total account balance the bot is allowed to trade. [More information below](#configuring-amount-per-trade).
*Defaults to `0.99` 99%).*
**Datatype:** Positive float between `0.1` and `1.0`. +| `available_capital` | Available starting capital for the bot. Useful when running multiple bots on the same exchange account.[More information below](#configuring-amount-per-trade).
**Datatype:** Positive float. | `amend_last_stake_amount` | Use reduced last stake amount if necessary. [More information below](#configuring-amount-per-trade).
*Defaults to `false`.*
**Datatype:** Boolean | `last_stake_amount_min_ratio` | Defines minimum stake amount that has to be left and executed. Applies only to the last stake amount when it's amended to a reduced value (i.e. if `amend_last_stake_amount` is set to `true`). [More information below](#configuring-amount-per-trade).
*Defaults to `0.5`.*
**Datatype:** Float (as ratio) | `amount_reserve_percent` | Reserve some amount in min pair stake amount. The bot will reserve `amount_reserve_percent` + stoploss value when calculating min pair stake amount in order to avoid possible trade refusals.
*Defaults to `0.05` (5%).*
**Datatype:** Positive Float as ratio. @@ -192,9 +193,25 @@ You can configure the "untouched" amount by using the `tradable_balance_ratio` s For example, if you have 10 ETH available in your wallet on the exchange and `tradable_balance_ratio=0.5` (which is 50%), then the bot will use a maximum amount of 5 ETH for trading and considers this as available balance. The rest of the wallet is untouched by the trades. +!!! Danger + This setting should **not** be used when running multiple bots on the same exchange. Please look at [Available Capital to the bot](#assign-available-capital) instead. + !!! Warning The `tradable_balance_ratio` setting applies to the current balance (free balance + tied up in trades). Therefore, assuming the starting balance of 1000, a configuration with `tradable_balance_ratio=0.99` will not guarantee that 10 currency units will always remain available on the exchange. For example, the free amount may reduce to 5 units if the total balance is reduced to 500 (either by a losing streak, or by withdrawing balance). +#### Assign available Capital + +To fully utilize compounding profits when using multiple bots on the same exchange account, you'll want to limit each bot to a certain starting balance. +This can be accomplished by setting `available_capital` to the desired starting balance. + +Assuming your account has 10.000 USDT and you want to run 2 different strategies on this exchange. +You'd set `available_capital=5000` - granting each bot an initial capital of 5000 USDT. +The bot will then split this starting balance equally into `max_open_trades` buckets. +Profitable trades will result in increased stake-sizes for this bot - without affecting stake-sizes of the other bot. + +!!! Warning "Incompatible with `tradable_balance_ratio`" + Setting this option will replace any configuration of `tradable_balance_ratio`. + #### Amend last stake amount Assuming we have the tradable balance of 1000 USDT, `stake_amount=400`, and `max_open_trades=3`. diff --git a/freqtrade/wallets.py b/freqtrade/wallets.py index 0ece65b2b..0048dbf48 100644 --- a/freqtrade/wallets.py +++ b/freqtrade/wallets.py @@ -158,7 +158,6 @@ class Wallets: ( + free amount) * tradable_balance_ratio - """ - free = self.get_free(self._config['stake_currency']) return min(self.get_total_stake_amount() - Trade.total_open_trades_stakes(), free) From f94dbcd0853e87e6cb6996cbf8587ed5732b0138 Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:02:10 +0000 Subject: [PATCH 13/20] feat: censor password from logs --- .terraform/modules/vpc | 1 + freqtrade/misc.py | 14 ++++++++++++++ 2 files changed, 15 insertions(+) create mode 160000 .terraform/modules/vpc diff --git a/.terraform/modules/vpc b/.terraform/modules/vpc new file mode 160000 index 000000000..88e1c6ec7 --- /dev/null +++ b/.terraform/modules/vpc @@ -0,0 +1 @@ +Subproject commit 88e1c6ec7a8f6337b0d6bb9840bb70c8b3f14051 diff --git a/freqtrade/misc.py b/freqtrade/misc.py index 967f08299..79c6aec67 100644 --- a/freqtrade/misc.py +++ b/freqtrade/misc.py @@ -8,6 +8,7 @@ from datetime import datetime from pathlib import Path from typing import Any, Iterator, List from typing.io import IO +from urllib.parse import urlparse import rapidjson @@ -214,3 +215,16 @@ def chunks(lst: List[Any], n: int) -> Iterator[List[Any]]: """ for chunk in range(0, len(lst), n): yield (lst[chunk:chunk + n]) + + +def parse_db_uri_for_logging(uri: str): + """ + Helper method to parse the DB URI and return the same DB URI with the password censored + if it contains it. Otherwise, return the DB URI unchanged + :param uri: DB URI to parse for logging + """ + parsed_db_uri = urlparse(uri) + if not parsed_db_uri.netloc: # No need for censoring as no password was provided + return uri + pwd = parsed_db_uri.netloc.split(':')[1].split('@')[0] + return parsed_db_uri.geturl().replace(f':{pwd}@', ':****@') From 6a53e2c764ff2aaad68e83269604f364b11cdf82 Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:08:01 +0000 Subject: [PATCH 14/20] feat: apply censoring to logging --- freqtrade/commands/list_commands.py | 4 ++-- freqtrade/configuration/configuration.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/freqtrade/commands/list_commands.py b/freqtrade/commands/list_commands.py index cd26aa60e..410b9b72b 100644 --- a/freqtrade/commands/list_commands.py +++ b/freqtrade/commands/list_commands.py @@ -14,7 +14,7 @@ from freqtrade.constants import USERPATH_HYPEROPTS, USERPATH_STRATEGIES from freqtrade.enums import RunMode from freqtrade.exceptions import OperationalException from freqtrade.exchange import market_is_active, validate_exchanges -from freqtrade.misc import plural +from freqtrade.misc import parse_db_uri_for_logging, plural from freqtrade.resolvers import ExchangeResolver, StrategyResolver @@ -225,7 +225,7 @@ def start_show_trades(args: Dict[str, Any]) -> None: if 'db_url' not in config: raise OperationalException("--db-url is required for this command.") - logger.info(f'Using DB: "{config["db_url"]}"') + logger.info(f'Using DB: "{parse_db_uri_for_logging(config["db_url"])}"') init_db(config['db_url'], clean_open_orders=False) tfilter = [] diff --git a/freqtrade/configuration/configuration.py b/freqtrade/configuration/configuration.py index 1d2e3f802..ea202355a 100644 --- a/freqtrade/configuration/configuration.py +++ b/freqtrade/configuration/configuration.py @@ -15,7 +15,7 @@ from freqtrade.configuration.load_config import load_config_file, load_file from freqtrade.enums import NON_UTIL_MODES, TRADING_MODES, RunMode from freqtrade.exceptions import OperationalException from freqtrade.loggers import setup_logging -from freqtrade.misc import deep_merge_dicts +from freqtrade.misc import deep_merge_dicts, parse_db_uri_for_logging logger = logging.getLogger(__name__) @@ -144,7 +144,7 @@ class Configuration: config['db_url'] = constants.DEFAULT_DB_PROD_URL logger.info('Dry run is disabled') - logger.info(f'Using DB: "{config["db_url"]}"') + logger.info(f'Using DB: "{parse_db_uri_for_logging(config["db_url"])}"') def _process_common_options(self, config: Dict[str, Any]) -> None: From c78b2075d84bb1bcd685e141c4bcd7149fdca95c Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:27:59 +0000 Subject: [PATCH 15/20] feat: add one additional asterisk --- freqtrade/misc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/freqtrade/misc.py b/freqtrade/misc.py index 79c6aec67..6f439866b 100644 --- a/freqtrade/misc.py +++ b/freqtrade/misc.py @@ -227,4 +227,4 @@ def parse_db_uri_for_logging(uri: str): if not parsed_db_uri.netloc: # No need for censoring as no password was provided return uri pwd = parsed_db_uri.netloc.split(':')[1].split('@')[0] - return parsed_db_uri.geturl().replace(f':{pwd}@', ':****@') + return parsed_db_uri.geturl().replace(f':{pwd}@', ':*****@') From 313cf6a01307f9d7826781c13e184b31374ffee4 Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:28:34 +0000 Subject: [PATCH 16/20] test: add test for parsing db uri --- tests/test_misc.py | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/tests/test_misc.py b/tests/test_misc.py index e6ba70aee..5b5a1400f 100644 --- a/tests/test_misc.py +++ b/tests/test_misc.py @@ -9,7 +9,7 @@ import pytest from freqtrade.misc import (decimals_per_coin, file_dump_json, file_load_json, format_ms_time, pair_to_filename, plural, render_template, render_template_with_fallback, round_coin_value, safe_value_fallback, - safe_value_fallback2, shorten_date) + safe_value_fallback2, shorten_date, parse_db_uri_for_logging) def test_decimals_per_coin(): @@ -179,3 +179,18 @@ def test_render_template_fallback(mocker): ) assert isinstance(val, str) assert 'if self.dp' in val + +def test_parse_db_uri_for_logging() -> None: + postgresql_conn_uri = "postgresql+psycopg2://scott123:scott123@host/dbname" + mariadb_conn_uri = "mariadb+mariadbconnector://app_user:Password123!@127.0.0.1:3306/company" + mysql_conn_uri = "mysql+pymysql://user:pass@some_mariadb/dbname?charset=utf8mb4" + sqlite_conn_uri = "sqlite:////freqtrade/user_data/tradesv3.sqlite" + censored_pwd = "*****" + + get_pwd = lambda x: x.split(':')[2].split('@')[0] + + assert get_pwd(parse_db_uri_for_logging(postgresql_conn_uri)) == censored_pwd + assert get_pwd(parse_db_uri_for_logging(mariadb_conn_uri)) == censored_pwd + assert get_pwd(parse_db_uri_for_logging(mysql_conn_uri)) == censored_pwd + assert sqlite_conn_uri == parse_db_uri_for_logging(sqlite_conn_uri) + From 8def18b002a524a6206a4dd88519717879d41259 Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:31:13 +0000 Subject: [PATCH 17/20] style: apply flake8 formatting --- tests/test_misc.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/test_misc.py b/tests/test_misc.py index 5b5a1400f..221c7b712 100644 --- a/tests/test_misc.py +++ b/tests/test_misc.py @@ -7,9 +7,9 @@ from unittest.mock import MagicMock import pytest from freqtrade.misc import (decimals_per_coin, file_dump_json, file_load_json, format_ms_time, - pair_to_filename, plural, render_template, + pair_to_filename, parse_db_uri_for_logging, plural, render_template, render_template_with_fallback, round_coin_value, safe_value_fallback, - safe_value_fallback2, shorten_date, parse_db_uri_for_logging) + safe_value_fallback2, shorten_date) def test_decimals_per_coin(): @@ -180,6 +180,7 @@ def test_render_template_fallback(mocker): assert isinstance(val, str) assert 'if self.dp' in val + def test_parse_db_uri_for_logging() -> None: postgresql_conn_uri = "postgresql+psycopg2://scott123:scott123@host/dbname" mariadb_conn_uri = "mariadb+mariadbconnector://app_user:Password123!@127.0.0.1:3306/company" @@ -187,10 +188,9 @@ def test_parse_db_uri_for_logging() -> None: sqlite_conn_uri = "sqlite:////freqtrade/user_data/tradesv3.sqlite" censored_pwd = "*****" - get_pwd = lambda x: x.split(':')[2].split('@')[0] + def get_pwd(x): return x.split(':')[2].split('@')[0] assert get_pwd(parse_db_uri_for_logging(postgresql_conn_uri)) == censored_pwd assert get_pwd(parse_db_uri_for_logging(mariadb_conn_uri)) == censored_pwd assert get_pwd(parse_db_uri_for_logging(mysql_conn_uri)) == censored_pwd assert sqlite_conn_uri == parse_db_uri_for_logging(sqlite_conn_uri) - From 91e5562ae01e68a6f5b98f24c9926d76cc205066 Mon Sep 17 00:00:00 2001 From: anasyusef Date: Mon, 12 Jul 2021 12:31:13 +0000 Subject: [PATCH 18/20] style: apply flake8 formatting --- .terraform/modules/vpc | 1 - tests/test_misc.py | 8 ++++---- 2 files changed, 4 insertions(+), 5 deletions(-) delete mode 160000 .terraform/modules/vpc diff --git a/.terraform/modules/vpc b/.terraform/modules/vpc deleted file mode 160000 index 88e1c6ec7..000000000 --- a/.terraform/modules/vpc +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 88e1c6ec7a8f6337b0d6bb9840bb70c8b3f14051 diff --git a/tests/test_misc.py b/tests/test_misc.py index 5b5a1400f..221c7b712 100644 --- a/tests/test_misc.py +++ b/tests/test_misc.py @@ -7,9 +7,9 @@ from unittest.mock import MagicMock import pytest from freqtrade.misc import (decimals_per_coin, file_dump_json, file_load_json, format_ms_time, - pair_to_filename, plural, render_template, + pair_to_filename, parse_db_uri_for_logging, plural, render_template, render_template_with_fallback, round_coin_value, safe_value_fallback, - safe_value_fallback2, shorten_date, parse_db_uri_for_logging) + safe_value_fallback2, shorten_date) def test_decimals_per_coin(): @@ -180,6 +180,7 @@ def test_render_template_fallback(mocker): assert isinstance(val, str) assert 'if self.dp' in val + def test_parse_db_uri_for_logging() -> None: postgresql_conn_uri = "postgresql+psycopg2://scott123:scott123@host/dbname" mariadb_conn_uri = "mariadb+mariadbconnector://app_user:Password123!@127.0.0.1:3306/company" @@ -187,10 +188,9 @@ def test_parse_db_uri_for_logging() -> None: sqlite_conn_uri = "sqlite:////freqtrade/user_data/tradesv3.sqlite" censored_pwd = "*****" - get_pwd = lambda x: x.split(':')[2].split('@')[0] + def get_pwd(x): return x.split(':')[2].split('@')[0] assert get_pwd(parse_db_uri_for_logging(postgresql_conn_uri)) == censored_pwd assert get_pwd(parse_db_uri_for_logging(mariadb_conn_uri)) == censored_pwd assert get_pwd(parse_db_uri_for_logging(mysql_conn_uri)) == censored_pwd assert sqlite_conn_uri == parse_db_uri_for_logging(sqlite_conn_uri) - From 29e2b858ca1fff6e23e276b0086e545356694431 Mon Sep 17 00:00:00 2001 From: Matthias Date: Tue, 13 Jul 2021 20:40:06 +0200 Subject: [PATCH 19/20] Improve wording in docs --- docs/configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/configuration.md b/docs/configuration.md index ad7436cb2..73b0e9c9a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -194,7 +194,7 @@ You can configure the "untouched" amount by using the `tradable_balance_ratio` s For example, if you have 10 ETH available in your wallet on the exchange and `tradable_balance_ratio=0.5` (which is 50%), then the bot will use a maximum amount of 5 ETH for trading and considers this as available balance. The rest of the wallet is untouched by the trades. !!! Danger - This setting should **not** be used when running multiple bots on the same exchange. Please look at [Available Capital to the bot](#assign-available-capital) instead. + This setting should **not** be used when running multiple bots on the same account. Please look at [Available Capital to the bot](#assign-available-capital) instead. !!! Warning The `tradable_balance_ratio` setting applies to the current balance (free balance + tied up in trades). Therefore, assuming the starting balance of 1000, a configuration with `tradable_balance_ratio=0.99` will not guarantee that 10 currency units will always remain available on the exchange. For example, the free amount may reduce to 5 units if the total balance is reduced to 500 (either by a losing streak, or by withdrawing balance). From 288c92301f74abca72611c77c9ebc65e9f298a9a Mon Sep 17 00:00:00 2001 From: Matthias Date: Wed, 14 Jul 2021 06:50:14 +0200 Subject: [PATCH 20/20] Improve docs wording --- docs/configuration.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 73b0e9c9a..6207770b8 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -165,7 +165,7 @@ Values set in the configuration file always overwrite values set in the strategy ### Configuring amount per trade -There are several methods to configure how much of the stake currency the bot will use to enter a trade. All methods respect the [available balance configuration](#available-balance) as explained below. +There are several methods to configure how much of the stake currency the bot will use to enter a trade. All methods respect the [available balance configuration](#tradable-balance) as explained below. #### Minimum trade stake @@ -184,7 +184,7 @@ To limit this calculation in case of large stoploss values, the calculated minim !!! Warning Since the limits on exchanges are usually stable and are not updated often, some pairs can show pretty high minimum limits, simply because the price increased a lot since the last limit adjustment by the exchange. -#### Available balance +#### Tradable balance By default, the bot assumes that the `complete amount - 1%` is at it's disposal, and when using [dynamic stake amount](#dynamic-stake-amount), it will split the complete balance into `max_open_trades` buckets per trade. Freqtrade will reserve 1% for eventual fees when entering a trade and will therefore not touch that by default.