Commit Graph

9493 Commits

Author SHA1 Message Date
Robert Caulk 63514b0443
Merge pull request #7325 from wagnercosta/develop
Improve Freqai backtesting performance
2022-09-03 19:28:04 +02:00
robcaulk c21808ff98 remove metadata redundancy, fix pca bug 2022-09-03 16:54:30 +02:00
robcaulk c9be66b5b6 increase test coverage for dk, improve function naming, extra cleaning 2022-09-03 15:52:29 +02:00
Matthias ed4cc18cdd Migration to check order funding fee 2022-09-03 15:19:40 +02:00
Matthias 0f483ee31f Use "since last order" approach for live as well. 2022-09-03 15:01:42 +02:00
Matthias 0c6a02687a Don't calculate funding fees if we're not going to use them. 2022-09-03 15:01:42 +02:00
Matthias df50b1928d Fix funding fee calculation for backtesting 2022-09-03 15:01:42 +02:00
Matthias 80b5f035ab Remove typo in log message 2022-09-03 15:01:28 +02:00
robcaulk 599c1c79fb reorganized backtest utilities, test new functionality, improve/update doc 2022-09-03 14:00:01 +02:00
Wagner Costa Santos af5460cebf Add option to keep models only in memory for backtest 2022-09-02 22:01:53 -03:00
Timothy Pogue 05cbcf834c minor logging changes 2022-09-02 16:01:33 -06:00
Timothy Pogue cf917ad2f5 initial candle request limit, better error reporting, split up _handle_producer_connection 2022-09-02 15:05:16 -06:00
Matthias b26126cb57 Don't use ticker['symbol'] but use "pair" instead
closes #7262
2022-09-02 20:09:30 +02:00
Timothy Pogue 5b0b802f31 hybrid json ws serializer 2022-09-02 00:05:36 -06:00
Timothy Pogue eb4cd6ba82 split initial data into separate requests 2022-09-01 23:52:13 -06:00
Timothy Pogue dccde88c83 fix dataframe serializing 2022-09-01 23:15:03 -06:00
Matthias b53791fef2 Futures volumepairlist to account for contract size 2022-09-02 07:11:32 +02:00
Timothy Pogue 00f35f4870 remove old constant, add initial_data requesting, minor changes 2022-09-01 20:06:36 -06:00
th0rntwig 11b2bc269e Added missing s 2022-09-01 22:37:32 +02:00
th0rntwig 3f8400df10 Normalise PCA space 2022-09-01 21:51:33 +02:00
Matthias 11fbfd3402 Remove unnecessary assignment 2022-09-01 19:39:20 +02:00
Wagner Costa Santos d6e115178a refactoring freqai backtesting - remove duplicate code 2022-09-01 07:09:23 -03:00
Matthias f3c73189d5 Remove pointless default on wallet_balance argument 2022-09-01 06:49:51 +02:00
Matthias ba2eb7cf0f Fix BNB fee bug when selling
thanks @epigramx, for reporting and for the detailed data.
2022-09-01 06:42:51 +02:00
Wagner Costa Santos 44d3a9140d Merge branch 'develop' of https://github.com/wagnercosta/freqtrade into develop 2022-08-31 15:37:06 -03:00
Wagner Costa Santos 7bed0450d2 pr review - refactoring backtesting freqai 2022-08-31 15:36:29 -03:00
Timothy Pogue c72a2c26c7 remove external pairlist 2022-08-31 12:06:24 -06:00
Timothy Pogue 6e8abf8674 add producer name to required fields in config 2022-08-31 11:58:58 -06:00
wagnercosta 3d4497467c
Merge branch 'freqtrade:develop' into develop 2022-08-31 14:47:06 -03:00
Timothy Pogue 865b34cd6f add producer names 2022-08-31 11:43:02 -06:00
Timothy Pogue 510cf4f305 remove data waiting, remove explicit analyzing of external df 2022-08-31 10:40:26 -06:00
Wagner Costa Santos df51da22ee refactoring freqai backtesting 2022-08-31 11:23:48 -03:00
Matthias 57ff6f8ac5 Init timerange object properly 2022-08-31 10:28:31 +00:00
Matthias 13ccd940d5 Remove startup_candle_count from freqai sample config to avoid confusion 2022-08-31 10:27:08 +00:00
Matthias 7ba4fda5d7 Implement PR feedback 2022-08-31 10:26:47 +00:00
Matthias a88ffd2c9d Merge branch 'develop' into move_datadownload 2022-08-31 10:23:45 +00:00
Timothy Pogue 115a901773 minor fix for conditional in handle func 2022-08-30 19:34:43 -06:00
Timothy Pogue ddc45ce2eb message handling fix, data waiting fix 2022-08-30 19:30:14 -06:00
Timothy Pogue 346e73dd75 client implementation, minor fixes 2022-08-30 19:21:34 -06:00
Matthias c9aa09ec89 Simplify base fee handling 2022-08-30 20:46:06 +02:00
robcaulk 7e8e29e42d use continuous value for inlier_metric 2022-08-30 20:41:37 +02:00
Timothy Pogue 418bd26a80 minor fixes, rework consumer request, update requirements.txt 2022-08-30 11:04:16 -06:00
robcaulk 7f52908e87 ensure the lost points are prepended for FreqUI 2022-08-30 18:55:58 +02:00
robcaulk a58dd0bbf9 add noise feature, improve docstrings 2022-08-30 18:26:24 +02:00
robcaulk b11742a4c5 integrate inlier metric function 2022-08-30 18:26:24 +02:00
th0rntwig d3cb211283 Add inlier metric computation 2022-08-30 18:26:24 +02:00
th0rntwig 50e2808667
Fix bug in DI (#7321) 2022-08-30 12:54:39 +02:00
Timothy Pogue 47f7c384fb consumer subscriptions, fix serializer bug 2022-08-29 15:48:29 -06:00
Timothy Pogue 7952e0df25 initial rework separating server and client impl 2022-08-29 13:41:15 -06:00
robcaulk 62c0a174c8 allow users to properly reverse train-test data ordering 2022-08-29 11:04:58 +02:00
Matthias 27a9f98d5f Simplify liquidation price structure, improve test cov 2022-08-29 07:04:48 +02:00
Matthias 226fa5d93c Simplify liquidation price calling structure 2022-08-29 07:04:48 +02:00
Matthias 4def3678b7
Merge pull request #7298 from freqtrade/tif_align
align TimeInForce to ccxt usage
2022-08-29 06:36:51 +02:00
Matthias 8be8a12cc4
Merge pull request #7260 from JohanVlugt/develop
Example FreqAI hybrid strategy
2022-08-29 06:34:45 +02:00
Matthias 40c00d2d8f Version bump dev version to 2022.9 2022-08-29 06:34:20 +02:00
Timothy Pogue 8c4e68b8eb updated example configs 2022-08-28 13:00:52 -06:00
Robert Caulk 39a739eadb
Merge pull request #7296 from th0rntwig/dbscan
Improve MinPts calculation in DBSCAN, add outlier protection, and add data_kitchen tests
2022-08-28 14:37:47 +02:00
robcaulk fcb5d1cb5a remove debugging flag 2022-08-28 13:01:39 +02:00
robcaulk dd628eb525 add tests for outlier detection and removal functions 2022-08-28 12:56:39 +02:00
robcaulk 1e41c773a0 fix outlier protection 2022-08-28 12:11:29 +02:00
smarmau ff3a4995c1 remove unnecessary code 2022-08-28 11:45:20 +02:00
Matthias b9f35cadb3 add /stopentry alias for /stopbuy 2022-08-28 11:37:22 +02:00
smarmau 005594c29c simplify hybrid template 2022-08-28 11:29:48 +02:00
th0rntwig 71f7d68783 Fixed mypy error 2022-08-27 12:44:55 +02:00
Matthias c61b986c3d FTX - support time_in_force (and PO ordertype)
closes #7175
2022-08-27 10:30:38 +02:00
Matthias 104a73025d Uppercase TimeInForce (align with ccxt) 2022-08-27 10:30:06 +02:00
Matthias 6686489c06
Merge pull request #7258 from freqtrade/feat/hyp_optinal_indicator
Add flag to move hyperopt populate_indicators to epoch
2022-08-27 09:21:16 +02:00
Matthias 2b70c3d0c0 support price callback for partial exits in bt
This will align results to how live works.
closes #7292
2022-08-27 08:50:09 +02:00
Timothy Pogue 05ca673883 Catch status code errors 2022-08-27 00:06:03 -06:00
Timothy Pogue fcceb744c5 Add janus to requirements.txt 2022-08-26 23:43:05 -06:00
Timothy Pogue 2b5f067877 Refactoring, minor improvements, data provider improvements 2022-08-26 23:40:13 -06:00
Matthias 9204f01312 Don't lock pairs on partial exit 2022-08-27 07:23:02 +02:00
elintornquist 86c5ac44e4 Add outlier percentage check 2022-08-26 23:05:07 +02:00
Matthias 01126c43f7 Fix liquidation price tier calculation
closes #7294
2022-08-26 20:14:24 +02:00
Matthias 753d1b2aad Update leverage tier terminology to be clear and aligned with ccxt 2022-08-26 19:34:51 +02:00
elintornquist b2d664c63c Change MinPts calculation 2022-08-26 18:57:27 +02:00
robcaulk bb3523f383 download data homogeneously across timeframes 2022-08-26 18:51:42 +02:00
robcaulk e7261cf515 add freqai utils.py file 2022-08-26 15:30:28 +02:00
robcaulk 65b552e310 make docs reflect reality, move download_all_data to new utils.py file, automatic startup_candle detection 2022-08-26 15:30:01 +02:00
robcaulk 4b7e640f31 reduce code duplication, optimize auto data download per tf 2022-08-26 13:56:44 +02:00
Matthias 53d46a0385 align `max_entry_position_adjustment` behavior of backtesting to live
closes #7293
2022-08-25 20:36:17 +02:00
Matthias 1fd223c815
Update --prepend help string
closes #7290
2022-08-25 17:03:41 +02:00
Matthias f2a356a80c Fix some imports 2022-08-25 07:08:58 +02:00
Matthias 6636f17e0f Simplify usage of amount_to_contract precision 2022-08-25 07:08:22 +02:00
Timothy Pogue d474111a65 Renamed to external signals, controller class refactored 2022-08-24 22:42:29 -06:00
Timothy Pogue 592373f096 Remove pairlist waiting, add .db files to .gitignore 2022-08-24 18:30:30 -06:00
Matthias 32faad9333 Fix backtest calculation problem with DCA
closes #7287
2022-08-24 20:36:08 +02:00
Matthias a6d78a8615 initialize Since parameter properly
closes #7285
2022-08-23 06:43:04 +02:00
Matthias fe7108ae75 Convert amount to contracts before comparing for close
closes #7279
2022-08-23 06:37:38 +02:00
Matthias 78b161e14c add contract_size to database 2022-08-23 06:37:38 +02:00
Matthias 6036018f35 Extract contracts_to_amount and amount_to_contracts to standalone functions 2022-08-23 06:37:38 +02:00
Matthias 5f38a574ce Add okx broker id 2022-08-23 06:37:38 +02:00
th0rntwig 5ce1c69803
Improve DBSCAN epsilon identification (#7269)
* Improve DBSCAN epsilon identification
2022-08-22 19:57:20 +02:00
robcaulk ac42c0153d deprecate indicator_max_period_candles, automatically compute startup candles for FreqAI backtesting. 2022-08-22 18:19:07 +02:00
robcaulk 96d8882f1e Plug mem leak, add training timer 2022-08-22 13:30:30 +02:00
Matthias f55d5ffd8c Don't fail when `--strategy-path` is not a valid directory.
closes #7264
2022-08-22 09:20:14 +00:00
Timothy Pogue 4fa01548f6 Remove old var from strategy interface 2022-08-21 22:49:42 -06:00
Timothy Pogue 6f5478cc02 DataFrame transmission, strategy follower logic 2022-08-21 22:45:36 -06:00
Matthias 015be770c3 ccxt now defaults to base volume for all markets 2022-08-22 06:42:14 +02:00
Matthias f6d832c6d9 Add get_option to expose ft_has via method 2022-08-21 17:51:46 +02:00
Matthias 87a3115073 Add get_open_trade_count() to simplify getting open trade count. 2022-08-21 17:08:27 +02:00
Matthias 6189aa817c Fix HybridExample formatting 2022-08-20 19:50:18 +02:00
robcaulk 64b0834437 add credit in docstring 2022-08-20 17:04:38 +02:00
robcaulk 90c03178b1 provide user directions, clean up strategy, remove unnecessary code. 2022-08-20 17:02:18 +02:00
Matthias cdd4745693
Merge pull request #7263 from freqtrade/okx_cache_tiers
Okx cache tiers
2022-08-20 15:18:13 +02:00
Matthias 5b3f031590 Use hyperopt safe amount precision method 2022-08-20 14:13:15 +02:00
Matthias 738e95b875 Add tests for leverage tiers caching 2022-08-20 13:54:54 +02:00
Matthias b6e8b9df35 Use cached leverage tiers 2022-08-20 13:01:58 +02:00
Matthias 52ec0d1046 Update binance Leverage tiers 2022-08-20 11:53:15 +02:00
Matthias 0da0600836 Have backtesting respect tradable size
closes #7161
2022-08-20 11:41:11 +02:00
Matthias 54ddc1a4c2 Add --tradingmode alias 2022-08-20 11:24:20 +02:00
Matthias aa3da092a0 Dont' use classProperty - that's not supported on 3.8 2022-08-20 10:55:52 +02:00
Matthias 01d45ed12e
Merge pull request #7257 from freqtrade/feat/list-pair-time
Get min/max data in list-data command
2022-08-20 08:16:52 +02:00
Matthias 7b8b73e651
Merge pull request #7243 from lolongcovas/newbranch_test
Improve PCA and pairwise distance calcs
2022-08-20 08:13:40 +02:00
Timothy Pogue 739b68f8fd ExternalPairList plugin 2022-08-19 22:40:01 -06:00
Johan van der Vlugt b44bd0171c
Example Classifier strat 2022-08-19 19:10:37 +02:00
Matthias b9d48c3278 use numbers in HyperoptState properly ... 2022-08-19 15:40:06 +02:00
Matthias bc359675a2 Add --analyze-per-epoch - moving populate_analysis to the epoch process 2022-08-19 15:19:43 +02:00
Matthias 09f8904545 Extract analysis to separate method 2022-08-19 15:12:55 +02:00
Matthias 08ef5ad2d8 Add HyperoptState enum and container class 2022-08-19 15:11:43 +02:00
Matthias 1c6f966579 Hyperopt: simplify parameter "can_optimize" handling 2022-08-19 15:03:03 +02:00
Matthias 16af10a5bc Update notebook sample with simplified datadir configuration
closes #7252
2022-08-19 14:05:27 +02:00
Matthias b7553d20d4 Get min/max data in list-data command 2022-08-19 13:45:55 +02:00
Matthias 7d84ef2e2c Remove unused imports 2022-08-19 13:45:10 +02:00
longyu 521381ebf0 undo example strategy newline 2022-08-19 12:40:03 +02:00
Matthias b420614d65 Reduce code duplication in datahandlers 2022-08-19 09:33:07 +02:00
Matthias 975bf8fe88 Update Docstring to match actual return values 2022-08-19 09:23:53 +02:00
Timothy Pogue 6834db11f3 minor improvements and pairlist data transmission 2022-08-19 00:06:19 -06:00
Matthias 42eb508515 Attempt fix of #7184 2022-08-19 07:09:46 +02:00
Matthias 76a3e97e05 Add migrations end message
closes #7251
2022-08-19 06:39:51 +02:00
Timothy Pogue 9f6bba40af initial concept for replicate, basic leader and follower logic 2022-08-18 20:10:58 +02:00
Matthias 70a77ba3d9 Check for "last" availability in PrecisionFilter
closes #7250
2022-08-18 20:07:50 +02:00
longyu 1fada53ddd remove newline 2022-08-18 19:40:00 +02:00
Matthias 85b43ec1a1 Remove double-check for "isolated margin" 2022-08-18 15:23:58 +02:00
Matthias fde469a253 Remove unnecessary check 2022-08-18 14:53:44 +02:00
Matthias 075e9b8526 Log Exchange responses for set_leverage 2022-08-18 09:52:03 +02:00
Matthias 46e8d9a5e4 Reduce verbosity when whitelist is empty 2022-08-18 09:09:37 +02:00
Matthias 0461a89348 Fix test failures 2022-08-18 07:20:49 +02:00
longyu f70b0bab80 remove line 2022-08-17 23:49:20 +02:00
Matthias 66910bfe63 Don't fail if mark candles are missing
closes #7239
2022-08-17 20:01:57 +02:00
robcaulk 88dd9920ea sort imports for isort 2022-08-17 16:38:09 +02:00
robcaulk 5155afb4e7 clean up code remnants 2022-08-17 15:22:48 +02:00
robcaulk 0c34104e45 extract download-data from freqai to prepare for future async changes 2022-08-17 15:18:44 +02:00
longyu 9c38c27eed ignore sample itself distance for avg_mean_dist computation 2022-08-17 15:09:57 +02:00
longyu 72c34291e3 newline 2022-08-17 15:09:10 +02:00
Matthias 3983368228 gateio futures is showing base currency in volume! 2022-08-17 14:51:48 +02:00
Matthias 83ca168bb8
Merge pull request #7216 from freqtrade/precise_calcs
Precise calcs
2022-08-17 14:32:02 +02:00
Matthias c615e1bc62 Avoid loop error during ccxt tests 2022-08-17 14:31:40 +02:00
Matthias e7902bffa0 Remove checks for dataprovider existance - it's available in all modes. 2022-08-17 10:57:25 +02:00
Matthias 8d182768f9 stoploss should also use trimmed prices 2022-08-17 09:57:11 +02:00
Matthias 0b0e7eaf96 Mutex TTL Cache accesses which can be accessed by multiple threads
Apparently, cachetools is (intentionally) not threadsafe
when using the Caches directly.
It's therefore recommended to wrap these with an explicit lock to avoid
problems.

source: https://github.com/tkem/cachetools/issues/245

closes #7215
2022-08-16 19:48:21 +02:00
Matthias 24690c1918 Don't convert open_rate to precision
this may cause more problems than it solves.
2022-08-16 18:32:00 +02:00
Matthias 3b44dc52e1 Minor corrections 2022-08-16 18:10:48 +02:00
Matthias ea6bc47d7a gateio default configs should specify unknown_fee_rate 2022-08-16 17:26:53 +02:00
Matthias a1e4fbf313 Run price_to_precision for dry-run orders 2022-08-16 17:23:49 +02:00
Matthias 1ac81aa316 Show message if fee update failed due to missing
closes #7234
2022-08-16 17:09:23 +02:00
robcaulk 4c0fda400f fix input shape warning for LGBMClassifier, add sample_weights/eval_weights 2022-08-16 11:41:53 +02:00
Matthias fa89368c02 Add test for precision backpopulation 2022-08-16 11:11:52 +02:00
Matthias 96d2f61812 Properly round timestamps to avoid constant log messages 2022-08-16 10:22:59 +02:00
Matthias 15a1c59a91 Backtesting should cache precisionMode 2022-08-16 10:15:19 +02:00
Matthias a73e4f8e41 Truncate amount before comparing for closure 2022-08-16 09:49:53 +02:00
Matthias 2fb7a3091d Improve backfill of precisions 2022-08-16 09:32:31 +02:00
Matthias 711b6b1a1a Merge branch 'develop' into precise_calcs 2022-08-16 09:29:39 +02:00
Matthias a5b438e41e Run price_to_precision for dry-run orders 2022-08-16 09:28:23 +02:00
Matthias 1dd56e35d5 Ensure comparisions align when closing a trade 2022-08-16 08:21:02 +02:00
Matthias e818797427 Minor fix in amount_to_precision logic 2022-08-15 20:29:05 +02:00
Matthias c0bdb71810 Update docstring 2022-08-15 20:06:29 +02:00
Matthias f2b6ff910f Accept wrong pair in get_precision_amount 2022-08-15 20:05:22 +02:00
Matthias 09ee9089fb
Merge pull request #6832 from freqtrade/feat/freqai
Freqai: an interface for users to build/train/backtest predictive models and run them live
2022-08-15 20:03:08 +02:00
Matthias e6af9a6903 Allow empty precisionMode on conversions 2022-08-15 20:00:15 +02:00
Matthias c3f159bd57 Add precision fields to database 2022-08-15 19:58:40 +02:00
Matthias 22241c55d5 Add methods to get precision_amount from markets 2022-08-15 19:56:25 +02:00
Matthias 15e85797c2 Simplify to_precision tests and imports 2022-08-15 08:51:15 +02:00
Matthias 6c32331740 Move precision calculations to standalone functions 2022-08-15 08:43:58 +02:00
Matthias c7e1719215 Fix interface import sorting 2022-08-15 06:53:02 +02:00
robcaulk 006b11e5d5 fix leftover bug in indicator population 2022-08-14 21:42:55 +02:00
robcaulk 8961b8d560 merge in inference timer and historic predictions handling improvements. 2022-08-14 20:31:15 +02:00
robcaulk ad846cdb76 fix lock bug, update docstring 2022-08-14 20:24:29 +02:00
Matthias 464d99808f Update doc table formatting 2022-08-14 18:22:01 +02:00
Matthias a29402ddde Rename and move analysis_lock to data_kitchen 2022-08-14 17:23:14 +02:00
Matthias 3a9ec76c91 Move "freqai.lock" to backend to simplify user interface 2022-08-14 17:19:50 +02:00
Matthias 044cf8bb2e Allow new whitelist combination in "button" commands 2022-08-14 08:41:25 +02:00
Matthias 22ac291c3a
Merge pull request #7211 from ecoppen/rpc/whitelist_options
Optional /whitelist args - sorted, nobase
2022-08-14 08:26:21 +02:00
Robert Caulk c9c128f781 finalize logo, improve doc, improve algo overview, fix base tensorflowmodel for mypy 2022-08-14 02:49:01 +02:00
robcaulk 58de20af0f make BaseClassifierModel. Add predict_proba to lightgbm 2022-08-13 20:07:31 +02:00
robcaulk 31be707cc8 clean up code, add docstrings 2022-08-13 19:40:24 +02:00
robcaulk 3e38c1b0bd take dynamic sized tail off historic_predictions as return dataframe to strategy. 2022-08-13 19:40:24 +02:00
robcaulk 7d448fd4ac allow fit_live_predictions access to current pair 2022-08-13 19:40:24 +02:00
robcaulk 1f192be43b avoid denormalizing labels twice 2022-08-13 19:40:24 +02:00
robcaulk b1b76a2dbe debug classifier with predict proba 2022-08-13 19:40:24 +02:00
robcaulk 23cc21ce59 add predict_proba to base classifier, improve historic predictions handling 2022-08-13 19:40:24 +02:00
Matthias 7075b00e20 Remove odd dry run stoploss behavior
closes #7208
2022-08-13 11:37:23 +02:00
Matthias 5aaab75d1c Add test for dynamic_pairlist_expand 2022-08-13 10:18:57 +02:00
Matthias b682fc446e Graciously fail if strategy has freqAI code, but freqAI is not enabled. 2022-08-13 09:53:18 +02:00
Matthias 3918f4abbd Simplify strategy interface by removing explicit self.freqai_info assignment 2022-08-13 09:27:56 +02:00
Matthias 3b827ee60a Add "freqai.enabled" flag to disable freqAI via config flag
aligns with how other optional modules work in freqtrade.
2022-08-13 09:24:04 +02:00
Matthias f6545ebdb8 Disallow backtesting with --strategy-list for now. 2022-08-13 09:10:03 +02:00
Matthias e3a5b97b45 Update recalc_from_trades to use FtPrecise 2022-08-13 08:43:56 +02:00
Matthias 3bcb47d75d Remove usage of Decimal 2022-08-13 08:43:56 +02:00
Matthias 902afc2f02 Use FtPrecise in interest calculation 2022-08-13 08:43:56 +02:00