Commit Graph

620 Commits

Author SHA1 Message Date
Mark Regan
073ce1659e remove un-used f-string 2022-10-20 14:26:10 +01:00
Mark Regan
295ba21389 Make extra_returns_per_train values available during backtest 2022-10-20 12:05:37 +01:00
th0rntwig
033c5bd441 Make check constant pred labels agnostic 2022-10-18 12:55:47 +02:00
Matthias
880ddccaa8
Merge pull request #7590 from freqtrade/list-models
List models
2022-10-17 20:40:41 +02:00
Matthias
441032be25 Fix sys.stdout bug for CatboostRegressorMultiTarget 2022-10-17 19:48:27 +02:00
Matthias
6cb14148aa Fix random test failure due to catboost bug
https://github.com/catboost/catboost/issues/2195
2022-10-17 07:00:44 +02:00
Matthias
23a5a516f9
Merge pull request #7557 from freqtrade/add-metric-tracker
Add metric tracker to FreqAI
2022-10-16 18:20:07 +02:00
th0rntwig
20fc521771 Fix constant PCA 2022-10-15 23:30:12 +02:00
Robert Caulk
62ca822597
Merge pull request #7569 from Silur/develop
Add XGBoost random forest predictors to freqai
2022-10-15 16:09:26 +02:00
robcaulk
3b4402aaab Merge branch 'move-disk-writing-to-ram' into add-metric-tracker 2022-10-15 13:52:14 +02:00
robcaulk
99dbba6cad avoid reading from disk to instantiate large objects 2022-10-15 13:50:55 +02:00
robcaulk
d81eef0b70 add timestamps to each metric, use rapidjson 2022-10-15 13:23:01 +02:00
robcaulk
b236e362ba Merge remote-tracking branch 'origin/develop' into add-metric-tracker 2022-10-14 19:00:49 +02:00
Emre
7f05b44376
Add eval set to CatboostClassifier 2022-10-13 23:01:09 +03:00
Wagner Costa Santos
6919f3aa75 Backtest live models - fix utc date convert issue 2022-10-13 15:03:27 -03:00
Wagner Costa Santos
93fe2b6446 Merge branch 'develop' into backtest_live_models 2022-10-13 11:22:58 -03:00
robcaulk
1e31be562e remove whitespace 2022-10-11 21:05:42 +02:00
robcaulk
dba1b573bc remove tensorboard dir from other pred models 2022-10-11 19:49:24 +02:00
robcaulk
5b5bb8aab5 catboost tensorboard bugfix 2022-10-11 19:05:46 +02:00
Robert Caulk
2e34aa9f04
Merge pull request #7544 from th0rntwig/prediction-shape
Remove constant labels from prediction
2022-10-10 21:24:25 +02:00
Robert Caulk
7bcb7d9a1a
Merge pull request #7554 from initrv/add-catboost-tensorboard
Add tensorboard for catboost
2022-10-10 21:03:45 +02:00
Wagner Costa Santos
3081e73f8a Merge branch 'develop' into backtest_live_models 2022-10-10 14:53:45 -03:00
silur
2ad086dd7a add XGBoost random forest predictors to freqai 2022-10-10 14:38:43 +02:00
Matthias
eaae9c9e03 Update docstring format 2022-10-10 12:19:29 +00:00
Matthias
60de192d47 Update Classifier docstrings 2022-10-10 12:13:41 +00:00
robcaulk
a4aa1b972c isolate and standardize location of tensorboard files, add doc, ensure backtesting functionality 2022-10-09 21:11:37 +02:00
robcaulk
76b33359a9 add an optional metric tracker to collect train timings, inference timings, and cpu load data 2022-10-09 20:22:42 +02:00
th0rntwig
4daf0000c7 Move check and add log warning 2022-10-08 16:15:48 +02:00
robcaulk
8d7adfabe9 clean RL tests to avoid dir pollution and increase speed 2022-10-08 12:10:38 +02:00
Emre
e337d4b78a Reset dataframe index after slice 2022-10-07 20:00:05 +02:00
Matthias
d42fb15608 Improve generic exception handler 2022-10-07 16:05:41 +02:00
th0rntwig
a9d5e04a43 Remove constant labels from prediction 2022-10-06 19:26:33 +02:00
initrv
86c781798a Add сatboost train_dir for tensorboard 2022-10-06 19:59:35 +03:00
robcaulk
488739424d fix reward inconsistency in template 2022-10-05 20:55:50 +02:00
robcaulk
936ca24482 separate RL install from general FAI install, update docs 2022-10-05 15:58:54 +02:00
robcaulk
9c73411ac2 Merge remote-tracking branch 'origin/develop' into dev-merge-rl 2022-10-05 15:21:45 +02:00
robcaulk
292d72d593 automatically handle model_save_type for user 2022-10-03 18:42:20 +02:00
Robert Caulk
3585742b43
remove trailing whitespace 2022-10-03 17:28:45 +02:00
Robert Caulk
265795824b
make default type for close_price and date_pred np.nan 2022-10-03 11:58:22 +02:00
Robert Caulk
6ecd92de4a
Allow updating without changing identifier 2022-10-03 09:55:57 +02:00
robcaulk
b70f18f4c3 add close price and date to historic_predictions 2022-10-02 18:33:39 +02:00
robcaulk
cf882fa84e fix tests 2022-10-01 20:26:41 +02:00
robcaulk
048cb95bd6 Merge remote-tracking branch 'origin/develop' into dev-merge-rl 2022-10-01 17:48:47 +02:00
Robert Caulk
3e34f10e3d
Merge pull request #7508 from aemr3/fix-pca-errors
Fix feature list match for PCA
2022-10-01 16:50:29 +02:00
robcaulk
f4c6b99d63 remove commented lines 2022-10-01 14:23:15 +02:00
robcaulk
cd514cf15d fix inlier metric in backtesting 2022-10-01 14:18:46 +02:00
robcaulk
f2b875483f ensure raw features match when PCA is employed 2022-10-01 13:14:59 +02:00
robcaulk
51556e08c3 Merge branch 'develop' into pr/th0rntwig/7495 2022-10-01 12:45:08 +02:00
Emre
cdc01a0781
Fix feature list match for pca 2022-09-30 15:22:05 -07:00
Matthias
cc06c60fd8 Fix pandas deprecation warnings from freqAI 2022-09-30 15:43:23 +02:00
robcaulk
be48131185 make shuffle false in constants 2022-09-30 00:33:08 +02:00
Robert Caulk
6e74d46660
Ensure 1 thread available 2022-09-29 14:02:00 +02:00
Robert Caulk
7ef56e3029
Ensure at least 1 thread is available 2022-09-29 14:01:22 +02:00
Robert Caulk
555cc42630
Ensure 1 thread is available (for testing purposes) 2022-09-29 14:00:09 +02:00
Wagner Costa Santos
6845a5c6ea backtest_live_models - refactoring after PR review 2022-09-29 01:48:38 -03:00
Robert Caulk
dcf6ebe273
Update BaseReinforcementLearningModel.py 2022-09-29 00:37:03 +02:00
robcaulk
83343dc2f1 control number of threads, update doc 2022-09-29 00:10:18 +02:00
th0rntwig
683b084323 Set train-test-split shuffle=False as default and remove stratification 2022-09-28 18:23:56 +02:00
Wagner Costa Santos
55ebbeec18 backtest_live models tests refactoring 2022-09-28 08:48:32 -03:00
Timothy Pogue
099137adac remove hasattr calls 2022-09-27 22:35:15 -06:00
Timothy Pogue
9e36b0d2ea fix formatting 2022-09-27 22:02:33 -06:00
Timothy Pogue
caa47a2f47 close subproc env on shutdown 2022-09-28 03:06:05 +00:00
Wagner Costa Santos
0be115de9c backtest_live_models - added new tests and refactoring 2022-09-27 10:26:57 -03:00
Wagner Costa Santos
72aa47fc51 backtest_live_models - fix issue with timerange BT and 2 trainings within same candle (no data) 2022-09-27 00:14:12 -03:00
Wagner Costa Santos
14b96aaa38 backtesting live models - fix ci issues 2022-09-26 19:52:59 -03:00
Wagner Costa Santos
290afd9699 backtest_live_models - fix typo 2022-09-26 19:21:53 -03:00
Wagner Costa Santos
0318ca9f12 backtest_live_models - fix typo 2022-09-26 19:08:25 -03:00
Wagner Costa Santos
22bef71d5d backtest_live_models - add function comments and tests 2022-09-26 19:01:24 -03:00
Wagner Costa Santos
ec947ad65c remove commented code - backtest_live_models 2022-09-25 23:47:27 -03:00
Wagner Costa Santos
5880f7a638 backtest_live_models - params validation and get timerange from live models in BT 2022-09-25 23:14:00 -03:00
Wagner Costa Santos
f3f3917da3 Merge branch 'develop' into backtest_live_models 2022-09-25 20:05:26 -03:00
robcaulk
48e89e68b9 fix typos 2022-09-25 20:22:19 +02:00
Wagner Costa Santos
0ed7b2bfc3 change start_backtesting to handle backtest_live_models 2022-09-25 10:35:55 -03:00
robcaulk
873d2a5069 no model save backtest, plot features backtest, ensure inlier plays nice, doc 2022-09-25 11:18:10 +02:00
Wagner Costa Santos
7f116db95e added generic function to get timerange from existent models 2022-09-24 13:01:53 -03:00
Wagner Costa Santos
d9c16d4888 Merge branch 'develop' into backtest_live_models 2022-09-24 12:30:55 -03:00
Wagner Costa Santos
3ee7eb63f7 starting backtest live models 2022-09-24 12:28:52 -03:00
Matthias
585342f193
Merge pull request #7454 from freqtrade/fix_backtesting_dfsize_freqai
Ensure the DF has the same size in backtesting FreqAI
2022-09-24 16:17:39 +02:00
paranoidandy
2cc00a1a2c
Allow use of --strategy-list with freqai, with warning (#7455)
* Allow use of --strategy-list with freqai, with warning

* ensure populate_any_indicators is identical for resused identifiers

* use pair instead of metadata["pair"]

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-09-24 13:21:01 +02:00
Matthias
6643d90e64 simplify freqAI start_backtesting 2022-09-24 10:34:14 +02:00
robcaulk
647200e8a7 isort 2022-09-23 19:30:56 +02:00
robcaulk
77c360b264 improve typing, improve docstrings, ensure global tests pass 2022-09-23 19:17:27 +02:00
robcaulk
9c361f4422 increase test coverage for RL and FreqAI 2022-09-23 18:04:43 +02:00
Robert Caulk
f5cd8f62c6
Remove unused code from BaseEnv 2022-09-23 10:24:39 +02:00
Emre
a4eaff4da6
Add training elapsed time 2022-09-23 01:18:34 -07:00
robcaulk
7295ba0fb2 add test for Base4ActionEnv 2022-09-22 23:42:33 +02:00
robcaulk
eeebb78a5c skip darwin in RL tests, remove example scripts, improve doc 2022-09-22 21:16:21 +02:00
robcaulk
ea8e34e192 Merge branch 'develop' into dev-merge-rl 2022-09-22 19:46:50 +02:00
th0rntwig
71e6c54ea4
Normalise distances before Weibull fit (#7432)
* Normalise distances before Weibull

* Track inlier-metric params
2022-09-22 18:11:50 +02:00
Wagner Costa Santos
b1dbc3a65f remove function remove_training_from_backtesting and ensure BT period is correct with startup_candle_count 2022-09-22 12:13:51 -03:00
robcaulk
b5fd11f91b protect against unforeseen issues in scanning thread 2022-09-19 21:18:31 +02:00
robcaulk
42c75b4a7b combine log messages 2022-09-19 19:16:32 +02:00
Robert Caulk
9b66297cc0
Fix append 2022-09-19 12:47:20 +02:00
Robert Caulk
995396c775
Add useful log info 2022-09-19 11:42:56 +02:00
Robert Caulk
ad652817ef
Ensure train ordering after restart
Ensure lowest timestamps get trained first after restart
2022-09-19 11:11:23 +02:00
Robert Caulk
f9460c80c2
Merge pull request #7434 from freqtrade/improve-train-queue
improve train queue system in FreqAI
2022-09-19 10:55:53 +02:00
Matthias
ea58c29ded Add plot_feature_importance to schema definition 2022-09-19 08:13:54 +00:00
Matthias
225f7cd5f8
Merge pull request #7431 from initrv/add-plot-feature-importance
Add plot feature importance
2022-09-19 08:41:10 +02:00
Matthias
d930931000 Bring back sleep - it'll ensure we give up control over the thread. 2022-09-19 07:14:33 +02:00
robcaulk
470d5d8405 ensure full new pairlist is in the queue 2022-09-18 17:08:07 +02:00
robcaulk
eaa43337d2 improve train queue system, ensure crash resilience in train queue. 2022-09-18 17:00:55 +02:00
robcaulk
95457d23ca escape freqai-specific characters from file naming 2022-09-18 13:59:30 +02:00
Matthias
667853c504 Use Alias to type config objects 2022-09-18 13:20:55 +02:00
robcaulk
188f75d8ec set model in models dict 2022-09-18 12:50:44 +02:00
robcaulk
1ef875901a maintian user privacy by keeping plotly offline 2022-09-18 00:01:42 +02:00
robcaulk
68f7a31504 ensure continued operation despite not being able to plot 2022-09-18 00:00:14 +02:00
robcaulk
2c23effbf2 allow plot to plot multitargets, add test 2022-09-17 19:17:44 +02:00
initrv
1c92734f39 simplify plot_feature_importance call 2022-09-17 18:53:43 +03:00
robcaulk
7b1d409c98 fix mypy/flake8 2022-09-17 17:51:06 +02:00
initrv
86aa875bc9 plot features as html instead of png 2022-09-16 21:47:12 +03:00
initrv
b707a6da35 Add ability to plot feature importance 2022-09-16 19:17:41 +03:00
robcaulk
3b97b3d5c8 fix mypy error for strategy 2022-09-15 00:56:51 +02:00
robcaulk
8aac644009 add tests. add guardrails. 2022-09-15 00:46:35 +02:00
robcaulk
48140bff91 fix bug in 4ActRLEnv 2022-09-14 22:53:53 +02:00
robcaulk
81417cb795 Merge branch 'develop' into dev-merge-rl 2022-09-14 22:49:11 +02:00
initrv
37dd22c89e Fixed a bug that prevents clearing old models
Corrects the error of clearing old models when the model directory contains directories with names that do not match a regular expression
2022-09-14 03:40:13 +03:00
Emre
330d7068ab
Merge branch 'develop' into add-xgboostclassifier 2022-09-10 23:59:11 +03:00
Robert Caulk
075748b21a
Merge pull request #7392 from freqtrade/improve_ai_tests
Improve freqai tests by utilizing parametrization
2022-09-10 22:37:21 +02:00
robcaulk
5a0cfee27e allow user to multithread jobs (advanced users only) 2022-09-10 22:16:49 +02:00
Robert Caulk
73e122ad10
Merge pull request #7391 from wagnercosta/fixBTfreqaiV2
Fix FreaqAI backtesting - startup_candle_count bug
2022-09-10 21:53:00 +02:00
Matthias
e4caccc353
Merge pull request #7367 from freqtrade/add-continual-learning
add continual learning to catboost and friends
2022-09-10 20:17:28 +02:00
Wagner Costa Santos
311ae8bf1f freqai backtesting - add startup_candle_count at function description 2022-09-10 14:45:42 -03:00
Emre
60eb02bb62
Add XGBoostClassifier 2022-09-10 20:13:16 +03:00
Robert Caulk
2077f84f9b
Merge pull request #7363 from wagnercosta/fixHyperoptFreqai
Fix hyperopt - freqai
2022-09-10 17:02:01 +02:00
Robert Caulk
4a47c63f71
Merge pull request #7374 from th0rntwig/filter-constant-features 2022-09-10 16:58:58 +02:00
robcaulk
10b6aebc5f enable continual learning and evaluation sets on multioutput models. 2022-09-10 16:54:13 +02:00
Wagner Costa Santos
a1f88cca80 merge develop 2022-09-10 10:35:16 -03:00
Wagner Costa Santos
79985fda01 fix backtesting freqai startup candles bug 2022-09-10 10:27:17 -03:00
Matthias
5705b8759b Merge branch 'develop' into add-continual-learning 2022-09-10 08:24:04 +02:00
robcaulk
05581db4e3 ensure columns are only dropped in training 2022-09-09 20:37:21 +02:00
Matthias
97be3318f4 Use Datetime_format from constants 2022-09-09 20:31:02 +02:00
robcaulk
a826c0eb83 ensure signatures match, reduce verbosity 2022-09-09 19:30:53 +02:00
Emre
acb410a0de Remove verbosity params 2022-09-09 19:30:53 +02:00
Emre
df6e43d2c5 Add XGBoostRegressorMultiTarget class 2022-09-09 19:30:53 +02:00
Emre
1b6410d7d1 Add XGBoostRegressor for freqAI, fix mypy errors 2022-09-09 19:30:53 +02:00
Wagner Costa Santos
d38cc06139 Fix freqai backtesting time range issue 2022-09-09 19:27:05 +02:00
Wagner Costa Santos
f4f2884a66 Fix freqai backtesting time range issue 2022-09-07 18:52:58 -03:00
robcaulk
e51d352777 ensure pca is handling same DF as inlier 2022-09-07 19:11:54 +02:00
robcaulk
4c9ac6b7c0 add kwargs, reduce duplicated code 2022-09-07 18:58:55 +02:00
robcaulk
ea7bdac9ed ensure inlier metric can be combined with other cleaning methods 2022-09-07 18:45:16 +02:00
th0rntwig
cdc72bf8ca Correct indexing 2022-09-07 18:14:13 +02:00
th0rntwig
047ded1baa Check for constant columns 2022-09-07 17:47:27 +02:00
wagnercosta
6b7644029c
Merge branch 'freqtrade:develop' into fixHyperoptFreqai 2022-09-07 11:07:51 -03:00
Wagner Costa Santos
8d16dd804d hyperopt - freqai - docs and refactoring 2022-09-06 15:42:47 -03:00
Robert Caulk
d7585161b2
Merge branch 'develop' into add-inlier-metric 2022-09-06 20:40:21 +02:00
robcaulk
97077ba18a add continual learning to catboost and friends 2022-09-06 20:30:46 +02:00
robcaulk
d44296783e isort datakitchen 2022-09-06 20:10:12 +02:00
robcaulk
e83c9b276d fix whitespace 2022-09-06 19:56:52 +02:00
robcaulk
4b28d0495f fix timestamping, move imports, add words to doc 2022-09-06 19:46:58 +02:00
Wagner Costa Santos
2c8e5b191b fix hyperopt - freqai 2022-09-05 17:43:55 -03:00
Wagner Costa Santos
5b826150df fix hyperopt - freqai 2022-09-05 17:43:28 -03:00
robcaulk
78d01810ed reduce code redundancy, ensure live always gets the latest data 2022-09-05 00:12:00 +02:00
Wagner Costa Santos
d43ed186fc fix issue with freqai backtesting at slice dataframe 2022-09-05 00:12:00 +02:00
Robert Caulk
956ea43e55
Merge pull request #7340 from wizrds/sigint-freqai
Support SIGINT in FreqAI
2022-09-04 16:43:36 +02:00
robcaulk
ec76214d02 backup historical predictions pickle and load the backup in case of corruption 2022-09-04 15:56:07 +02:00
robcaulk
69b3fcfd32 Merge branch 'develop' into dev-merge-rl 2022-09-04 11:23:25 +02:00
robcaulk
27dce20b29 fix bug in Base4ActionRLEnv, improve example strats 2022-09-04 11:21:54 +02:00
Robert Caulk
8545d74378
Merge pull request #7331 from th0rntwig/pca
Normalise PCA space
2022-09-03 21:49:54 +02:00
Timothy Pogue
dae3b3d86a support shutting down freqai 2022-09-03 13:24:14 -06:00
robcaulk
5cfb4154eb revert all changes in normalize_data() 2022-09-03 19:48:30 +02:00
Robert Caulk
63514b0443
Merge pull request #7325 from wagnercosta/develop
Improve Freqai backtesting performance
2022-09-03 19:28:04 +02:00
robcaulk
c21808ff98 remove metadata redundancy, fix pca bug 2022-09-03 16:54:30 +02:00
robcaulk
c9be66b5b6 increase test coverage for dk, improve function naming, extra cleaning 2022-09-03 15:52:29 +02:00
Matthias
80b5f035ab Remove typo in log message 2022-09-03 15:01:28 +02:00
robcaulk
599c1c79fb reorganized backtest utilities, test new functionality, improve/update doc 2022-09-03 14:00:01 +02:00
Wagner Costa Santos
af5460cebf Add option to keep models only in memory for backtest 2022-09-02 22:01:53 -03:00
th0rntwig
11b2bc269e Added missing s 2022-09-01 22:37:32 +02:00
th0rntwig
3f8400df10 Normalise PCA space 2022-09-01 21:51:33 +02:00
Wagner Costa Santos
d6e115178a refactoring freqai backtesting - remove duplicate code 2022-09-01 07:09:23 -03:00
Wagner Costa Santos
44d3a9140d Merge branch 'develop' of https://github.com/wagnercosta/freqtrade into develop 2022-08-31 15:37:06 -03:00
Wagner Costa Santos
7bed0450d2 pr review - refactoring backtesting freqai 2022-08-31 15:36:29 -03:00
wagnercosta
3d4497467c
Merge branch 'freqtrade:develop' into develop 2022-08-31 14:47:06 -03:00
robcaulk
240b529533 fix tensorboard path so that users can track all historical models 2022-08-31 16:50:39 +02:00
Richard Jozsa
2493e0c8a5
Unnecessary lines in Base4, and changes for box space, to fit better for our needs (#7324) 2022-08-31 16:37:02 +02:00
Wagner Costa Santos
df51da22ee refactoring freqai backtesting 2022-08-31 11:23:48 -03:00
Matthias
57ff6f8ac5 Init timerange object properly 2022-08-31 10:28:31 +00:00
Matthias
7ba4fda5d7 Implement PR feedback 2022-08-31 10:26:47 +00:00
Matthias
a88ffd2c9d Merge branch 'develop' into move_datadownload 2022-08-31 10:23:45 +00:00
robcaulk
7e8e29e42d use continuous value for inlier_metric 2022-08-30 20:41:37 +02:00
robcaulk
7f52908e87 ensure the lost points are prepended for FreqUI 2022-08-30 18:55:58 +02:00
robcaulk
a58dd0bbf9 add noise feature, improve docstrings 2022-08-30 18:26:24 +02:00
robcaulk
b11742a4c5 integrate inlier metric function 2022-08-30 18:26:24 +02:00
th0rntwig
d3cb211283 Add inlier metric computation 2022-08-30 18:26:24 +02:00
th0rntwig
50e2808667
Fix bug in DI (#7321) 2022-08-30 12:54:39 +02:00
robcaulk
62c0a174c8 allow users to properly reverse train-test data ordering 2022-08-29 11:04:58 +02:00
robcaulk
7766350c15 refactor environment inheritence tree to accommodate flexible action types/counts. fix bug in train profit handling 2022-08-28 19:21:57 +02:00
robcaulk
fcb5d1cb5a remove debugging flag 2022-08-28 13:01:39 +02:00
robcaulk
dd628eb525 add tests for outlier detection and removal functions 2022-08-28 12:56:39 +02:00
robcaulk
1e41c773a0 fix outlier protection 2022-08-28 12:11:29 +02:00
th0rntwig
71f7d68783 Fixed mypy error 2022-08-27 12:44:55 +02:00
elintornquist
86c5ac44e4 Add outlier percentage check 2022-08-26 23:05:07 +02:00
elintornquist
b2d664c63c Change MinPts calculation 2022-08-26 18:57:27 +02:00
robcaulk
bb3523f383 download data homogeneously across timeframes 2022-08-26 18:51:42 +02:00
robcaulk
e7261cf515 add freqai utils.py file 2022-08-26 15:30:28 +02:00
robcaulk
65b552e310 make docs reflect reality, move download_all_data to new utils.py file, automatic startup_candle detection 2022-08-26 15:30:01 +02:00
robcaulk
4b7e640f31 reduce code duplication, optimize auto data download per tf 2022-08-26 13:56:44 +02:00
robcaulk
baa4f8e3d0 remove Base3ActionEnv in favor of Base4Action 2022-08-26 11:04:25 +02:00
richardjozsa
d31926efdf Added Base4Action 2022-08-26 11:04:25 +02:00
robcaulk
3199eb453b reduce code for base use-case, ensure multiproc inherits custom env, add ability to limit ram use. 2022-08-25 19:05:51 +02:00
robcaulk
05ccebf9a1 automate eval freq in multiproc 2022-08-25 12:29:48 +02:00