Commit Graph

688 Commits

Author SHA1 Message Date
Matthias c3daddc629
Merge pull request #7809 from richardjozsa/develop
Improve the RL learning process
2022-11-29 06:28:36 +01:00
Emre 9cbfa12011
Directly set model_type in base RL model 2022-11-28 16:02:17 +03:00
robcaulk 56518def42 isort 2022-11-27 21:06:01 +01:00
robcaulk 7fd6bc526e add randomize_starting_position to the rl_config 2022-11-27 21:03:13 +01:00
richardjozsa 64d4a52a56 Improve the RL learning process
Improve the RL learning process by selecting random start point for the agent, it can help to block the agent to only learn on the selected period of time, while improving the quality of the model.
2022-11-27 20:43:50 +01:00
Matthias f4025ee5de
Merge pull request #7289 from freqtrade/feat/freqai-rl-dev
Add reinforcement learning module to FreqAI
2022-11-27 17:15:21 +01:00
stm 51d21b413d
Fix 4ac update_total_profit
_update_total_profit() must be executed before "self._position = Positions.Neutral" because _update_total_profit() calls get_unrealized_profit(), which returns 0 if position is neutral and total_profit is not updated
2022-11-26 23:35:20 +03:00
robcaulk dba30393fb ensure extra_returns_per_train are set properly on first hist_preds build 2022-11-26 18:04:47 +01:00
robcaulk be890b52fd remove np import 2022-11-26 13:44:58 +01:00
robcaulk aaaa5a5f64 add documentation for net_arch, other small changes 2022-11-26 13:44:58 +01:00
Matthias 7ebc8ee169 Fix missing Optional typehint 2022-11-26 13:32:18 +01:00
Matthias bdfedb5fcb Improve typehints / reduce warnings from mypy 2022-11-26 13:03:07 +01:00
robcaulk 81fd2e588f ensure typing, remove unsued code 2022-11-26 12:11:59 +01:00
robcaulk 8dbfd2cacf improve docstring clarity about how to inherit from ReinforcementLearner, demonstrate inherittance with ReinforcementLearner_multiproc 2022-11-26 11:51:08 +01:00
robcaulk 2e82e6784a move data_provider cleanup to shutdown() 2022-11-24 19:07:38 +01:00
robcaulk 73c458d47b use importlib instead of __import___ 2022-11-24 19:04:35 +01:00
robcaulk 3a07749fcc fix docstring 2022-11-24 18:46:54 +01:00
robcaulk 44b042ba51 remove unused function 2022-11-24 17:53:26 +01:00
Matthias 8f1a8c752b Add freqairl docker build process 2022-11-24 07:00:12 +01:00
Wagner Costa d09157efb8 update code to use one prediction file / pair 2022-11-22 15:15:42 -03:00
Wagner Costa c01f25ddc9 update code to freqai_backtest_live_models only from historic predictions 2022-11-22 13:09:09 -03:00
Wagner Costa Santos fdc82af883 fix tests - update code to backtest with historic_predictions 2022-11-19 22:27:58 -03:00
Wagner Costa Santos 80d070e9ee update code to use historic_predictions for freqai_backtest_live_models 2022-11-19 14:15:58 -03:00
robcaulk 387c905a86 merge develop into RL 2022-11-17 21:59:07 +01:00
robcaulk 60fcd8dce2 fix skipped mac test, fix RL bug in add_state_info, fix use of __import__, revise doc 2022-11-17 21:50:02 +01:00
robcaulk 91df79ff44 merge dev into backtest-live-predictions 2022-11-17 21:20:47 +01:00
Robert Caulk cd6f87be17
Merge pull request #7728 from freqtrade/improve_timerange
Simplify timerange handling
2022-11-17 19:57:48 +01:00
Wagner Costa Santos 3903b04d3f save_live_data_backtest - added docs and tests 2022-11-17 15:20:07 -03:00
Wagner Costa Santos 99bff9cbfa backtesting_from_live_saved_files - code refactoring 2022-11-17 10:30:51 -03:00
Wagner Costa Santos 913749c81b backtesting_from_live_saved_files - code refactoring 2022-11-17 10:30:16 -03:00
Wagner Costa Santos b01e4e3dbf change default value - save_live_data_backtest as false 2022-11-17 10:14:30 -03:00
Wagner Costa Santos 1a19d90e2e Merge branch 'develop' into backtest_fitlivepredictions 2022-11-17 10:13:11 -03:00
robcaulk bf4d5b432a ensure model_type is defined 2022-11-13 18:50:25 +01:00
robcaulk 6394ef4558 fix docstrings 2022-11-13 17:43:52 +01:00
robcaulk 96fafb7f56 remove limit_ram_use 2022-11-13 17:14:47 +01:00
robcaulk b421521be3 help default ReinforcementLearner users by assigning the model_type automatically 2022-11-13 17:12:17 +01:00
robcaulk 90f168d1ff remove more user references. cleanup dataprovider 2022-11-13 17:06:06 +01:00
robcaulk 388ca21200 update docs, fix bug in environment 2022-11-13 16:56:31 +01:00
robcaulk af9e400562 add test coverage, fix bug in base environment. Ensure proper fee is used. 2022-11-13 15:31:37 +01:00
Matthias 942840da2d Improve setting wording to keep future possibilities open 2022-11-13 15:22:44 +01:00
robcaulk 81f800a79b switch to using FT calc_profi_pct, reverse entry/exit fees 2022-11-13 13:41:17 +01:00
robcaulk e71a8b8ac1 add ability to integrate state info or not, and prevent state info integration during backtesting 2022-11-12 18:46:48 +01:00
Wagner Costa Santos 27fa9f1f4e backtest saved dataframe from live 2022-11-12 14:37:23 -03:00
Wagner Costa Santos f9c6c538be Merge branch 'develop' into backtest_fitlivepredictions 2022-11-12 09:37:45 -03:00
robcaulk 9c6b97c678 ensure normalization acceleration methods are employed in RL 2022-11-12 12:01:59 +01:00
robcaulk 6746868ea7 store dataprovider to self instead of strategy 2022-11-12 11:33:03 +01:00
robcaulk 7a4bb040a5 merge develop into feat/freqai-rl-dev 2022-11-12 10:54:34 +01:00
robcaulk 214c622475 move dataframe converter to converter.py 2022-11-12 10:38:25 +01:00
robcaulk 9617d8143d Merge remote-tracking branch 'origin/develop' into add-single-precision-freqai 2022-11-12 10:21:38 +01:00
Matthias e6172a68d7
Merge pull request #7730 from freqtrade/fix-metric-tracker
fix loading of metric tracker from disk
2022-11-11 20:01:26 +01:00
Robert Caulk 833578716c
Merge pull request #7644 from markdregan/multi-target-classifier
Support for multi target multi-class classification (FreqaiMultiOutputRegressor for Classification)
2022-11-11 18:48:38 +01:00
robcaulk 790ff2a84b merge develop into add-single-prec 2022-11-11 18:08:51 +01:00
robcaulk e46a57bbd0 move mem logs to debug 2022-11-11 18:05:32 +01:00
robcaulk 66514e84e4 add LightGBMClassifierMultiTarget. add test 2022-11-11 17:45:53 +01:00
robcaulk 054133955b fix loading of metric tracker from disk 2022-11-11 17:24:09 +01:00
Matthias 0f9c5f8d41 Simplify timerange handling 2022-11-10 18:28:18 +01:00
Matthias 57313dd961 Update some usages of timerange to new, simplified method 2022-11-10 18:11:39 +01:00
Wagner Costa Santos 4f0f3e5b64 removed unnecessary code 2022-11-09 10:07:24 -03:00
Wagner Costa Santos 8ee95db927 refactoring backtesting_fit_live_predictions function 2022-11-09 09:51:42 -03:00
Wagner Costa Santos 3e57c18ac6 add fix_live_predictions function to backtesting 2022-11-08 18:20:39 -03:00
Wagner Costa Santos 8d9988a942 enforce date column in backtesting freqai predictions files 2022-11-08 11:06:23 -03:00
Wagner Costa Santos 9c5ba0732a save predictions with date and merge by date 2022-11-08 10:32:18 -03:00
Matthias 884014a4b9 Fix some minor typos 2022-11-07 18:35:28 +00:00
Wagner Costa Santos 6559384286 Merge branch 'develop' into backtest_live_models 2022-11-07 15:14:10 -03:00
Emre 43bdd34964
Optimize reduce_dataframe_footprint function 2022-11-05 19:13:02 +03:00
robcaulk 53df607067 avoid duplicating features with okx/gateio, ensure inference timer gets logged 2022-11-05 15:42:19 +01:00
robcaulk 257c833831 add doc for single precision, dont allow half precision, add test 2022-11-04 18:10:04 +01:00
robcaulk 3ccc120f92 add option to force single precision 2022-11-04 17:42:10 +01:00
robcaulk 8bdc99a3d6 fix self-imposed bug 2022-11-04 16:41:38 +01:00
robcaulk 19d90b813a improve readibility in start_backtesting() 2022-11-04 16:10:46 +01:00
Wagner Costa Santos 8008c63319 Merge branch 'develop' into backtest_live_models 2022-11-04 09:09:39 -03:00
Wagner Costa Santos a7acfb7ab7 fix dict key error 2022-11-04 09:02:28 -03:00
robcaulk 90c5bfb4b5 add default conv_width 2022-11-03 21:35:12 +01:00
robcaulk 05b309caf2 ensure colon replaced for cached attach 2022-11-03 21:17:48 +01:00
robcaulk 6938ed6584 change default conv_width to 1 2022-11-03 21:11:26 +01:00
robcaulk 444a068481 merge develop into track-current-candle 2022-11-03 21:09:08 +01:00
robcaulk db942321ad fix bug with lightgbm and colons 2022-11-03 21:03:48 +01:00
robcaulk d721b50230 flake8 2022-11-03 19:13:24 +01:00
robcaulk 3ba1e221eb fix typo 2022-11-03 19:08:33 +01:00
robcaulk 6c4bdb8f67 remove special characters from feature names 2022-11-03 18:49:39 +01:00
Wagner Costa Santos 17798b3397 Merge branch 'develop' into backtest_live_models 2022-11-03 13:29:25 -03:00
Wagner Costa Santos 356d79b38a verify mean and std exists in model metadata 2022-11-03 13:27:56 -03:00
robcaulk ce92731132 ensure backwards compatitibility 2022-11-02 20:20:35 +01:00
Wagner Costa Santos 23b6915dde fix issue with different backtesting prediction size 2022-11-02 15:49:51 -03:00
robcaulk 1a38c10fc6 remove old code 2022-11-02 19:37:47 +01:00
robcaulk 255eb71270 start tracking the current candle in FreqAI, add robustness to corr_df caching and inference timer, add test for cache corr_df 2022-11-02 19:32:22 +01:00
robcaulk 97df232ac6 add a warning to __init__ for get_corr_dataframes 2022-10-31 18:18:00 +01:00
robcaulk 66d8ed6c0b Merge remote-tracking branch 'origin/develop' into reduce-indicator-population 2022-10-31 09:42:01 +01:00
robcaulk d59a7fa2f9 remove analysis_lock and realign example hybrid strat 2022-10-30 17:07:33 +01:00
Matthias 391c3f56f7 Add typehint to corr_pairlist 2022-10-30 13:28:01 +01:00
Mark Regan 7053f81fa8 simplified predict and predict_proba using super(). Added duplicate class label check. 2022-10-30 09:48:30 +00:00
Matthias cf4af2175c
Merge pull request #7662 from markdregan/backtest_extra_returns_fix
Fix missing f-string from PR #7611
2022-10-30 10:47:06 +01:00
robcaulk a2843165e1 fix leftovers from merge 2022-10-30 10:31:38 +01:00
robcaulk 52e15b2070 Merge remote-tracking branch 'origin/develop' into feat/freqai-rl-dev 2022-10-30 10:16:39 +01:00
robcaulk d1a0874683 merge develop into feat/freqai-rl-dev 2022-10-30 10:13:03 +01:00
robcaulk fc53054d43 leverage list length knowledge, f-string change 2022-10-30 10:12:14 +01:00
Emre f98c7a2423 Remove loop of normalization from metadata 2022-10-30 10:12:14 +01:00
Mark Regan c26fda282f fix missing f-string from PR #7611 2022-10-30 08:19:59 +00:00
robcaulk 650bb8b7d7 ensure full pair string is used for caching dataframes. If not, revert to old behavior. Update docs. 2022-10-29 22:26:49 +02:00
Matthias 801e91c39e
Merge pull request #7618 from wizrds/fix/docker-config-record
Update function in FreqAI interface to record FreqAI config params
2022-10-29 08:56:20 +02:00
Mark Regan 6ef82dd8b6 minor change to return 2022-10-27 12:41:12 +01:00
Timothy Pogue b9bf9edb02 update rapidjson opts 2022-10-25 14:12:13 -06:00
Mark Regan 217add70bd add strat and config for testing on PR 2022-10-25 20:07:39 +01:00
Mark Regan 47056eded3 multi target classifier working but not for parallel 2022-10-25 18:24:27 +01:00
Matthias 1ef38f137d
Fix XGBoost regressor "used before assignment" 2022-10-25 13:37:04 +02:00
Timothy Pogue 51be45547f remove np object, make default str 2022-10-24 12:23:54 -06:00
Matthias 6669714a73 Update mal-formatted docstrings 2022-10-24 18:12:17 +02:00
Robert Caulk 137aa1756b
Merge pull request #7593 from th0rntwig/prediction-shape
Fix constant PCA
2022-10-24 08:33:36 +02:00
robcaulk 4d2b7a74f1 move record params to utils, use rapidjson 2022-10-23 20:51:32 +02:00
Timothy Pogue 07e813dfa0 change param record to only get certain params 2022-10-23 12:09:07 -06:00
th0rntwig 49ff51f11f Change storage loc and fix test fail 2022-10-23 16:24:02 +02:00
Timothy Pogue 4464e91256 use self.identifier in full path 2022-10-21 19:53:33 -06:00
Timothy Pogue 5ee3b8cbbb update config recording to use all configs, fix tests 2022-10-21 19:48:26 -06:00
Robert Caulk a85826bf24
Merge pull request #7611 from markdregan/backtest_extra_returns
Make extra_returns_per_train data available during backtest
2022-10-21 17:13:22 +02:00
Wagner Costa Santos 6606a0113f refactoring - remove unnecessary config file 2022-10-20 14:53:25 -03:00
Wagner Costa Santos 589944055e fix typos - live predictions candles 2022-10-20 12:15:41 -03:00
Wagner Costa Santos 52b60c5cbb Merge branch 'develop' into backtest_live_models 2022-10-20 11:59:37 -03:00
rcaulk a9db668082 avoid redundant indicator population for corr_pair list 2022-10-20 16:30:32 +02:00
Mark Regan 073ce1659e remove un-used f-string 2022-10-20 14:26:10 +01:00
Mark Regan 295ba21389 Make extra_returns_per_train values available during backtest 2022-10-20 12:05:37 +01:00
th0rntwig 033c5bd441 Make check constant pred labels agnostic 2022-10-18 12:55:47 +02:00
Matthias 880ddccaa8
Merge pull request #7590 from freqtrade/list-models
List models
2022-10-17 20:40:41 +02:00
Matthias 441032be25 Fix sys.stdout bug for CatboostRegressorMultiTarget 2022-10-17 19:48:27 +02:00
Matthias 6cb14148aa Fix random test failure due to catboost bug
https://github.com/catboost/catboost/issues/2195
2022-10-17 07:00:44 +02:00
Matthias 23a5a516f9
Merge pull request #7557 from freqtrade/add-metric-tracker
Add metric tracker to FreqAI
2022-10-16 18:20:07 +02:00
th0rntwig 20fc521771 Fix constant PCA 2022-10-15 23:30:12 +02:00
Robert Caulk 62ca822597
Merge pull request #7569 from Silur/develop
Add XGBoost random forest predictors to freqai
2022-10-15 16:09:26 +02:00
robcaulk 3b4402aaab Merge branch 'move-disk-writing-to-ram' into add-metric-tracker 2022-10-15 13:52:14 +02:00
robcaulk 99dbba6cad avoid reading from disk to instantiate large objects 2022-10-15 13:50:55 +02:00
robcaulk d81eef0b70 add timestamps to each metric, use rapidjson 2022-10-15 13:23:01 +02:00
robcaulk b236e362ba Merge remote-tracking branch 'origin/develop' into add-metric-tracker 2022-10-14 19:00:49 +02:00
Emre 7f05b44376
Add eval set to CatboostClassifier 2022-10-13 23:01:09 +03:00
Wagner Costa Santos 6919f3aa75 Backtest live models - fix utc date convert issue 2022-10-13 15:03:27 -03:00
Wagner Costa Santos 93fe2b6446 Merge branch 'develop' into backtest_live_models 2022-10-13 11:22:58 -03:00
robcaulk 1e31be562e remove whitespace 2022-10-11 21:05:42 +02:00
robcaulk dba1b573bc remove tensorboard dir from other pred models 2022-10-11 19:49:24 +02:00
robcaulk 5b5bb8aab5 catboost tensorboard bugfix 2022-10-11 19:05:46 +02:00
Robert Caulk 2e34aa9f04
Merge pull request #7544 from th0rntwig/prediction-shape
Remove constant labels from prediction
2022-10-10 21:24:25 +02:00
Robert Caulk 7bcb7d9a1a
Merge pull request #7554 from initrv/add-catboost-tensorboard
Add tensorboard for catboost
2022-10-10 21:03:45 +02:00
Wagner Costa Santos 3081e73f8a Merge branch 'develop' into backtest_live_models 2022-10-10 14:53:45 -03:00
silur 2ad086dd7a add XGBoost random forest predictors to freqai 2022-10-10 14:38:43 +02:00
Matthias eaae9c9e03 Update docstring format 2022-10-10 12:19:29 +00:00
Matthias 60de192d47 Update Classifier docstrings 2022-10-10 12:13:41 +00:00
robcaulk a4aa1b972c isolate and standardize location of tensorboard files, add doc, ensure backtesting functionality 2022-10-09 21:11:37 +02:00
robcaulk 76b33359a9 add an optional metric tracker to collect train timings, inference timings, and cpu load data 2022-10-09 20:22:42 +02:00
th0rntwig 4daf0000c7 Move check and add log warning 2022-10-08 16:15:48 +02:00
robcaulk 8d7adfabe9 clean RL tests to avoid dir pollution and increase speed 2022-10-08 12:10:38 +02:00
Emre e337d4b78a Reset dataframe index after slice 2022-10-07 20:00:05 +02:00
Matthias d42fb15608 Improve generic exception handler 2022-10-07 16:05:41 +02:00
th0rntwig a9d5e04a43 Remove constant labels from prediction 2022-10-06 19:26:33 +02:00