Commit Graph

221 Commits

Author SHA1 Message Date
robcaulk 29225e4baf add DBSCAN outlier detection feature, add supporting documentation 2022-08-04 12:15:16 +02:00
robcaulk eae82d0222 fix bug with database url during backtesting. comment out example trade db analysis. 2022-08-03 16:17:57 +02:00
robcaulk 95d3009a95 give user ability to analyze live trade dataframe inside custom prediction model. Add documentation to explain new functionality 2022-08-02 20:14:02 +02:00
robcaulk 946d4c7cfc fix trailing whitespace for flake8 2022-07-31 18:39:46 +02:00
robcaulk 4e68626bcb ensure convolutional window is prepended for frequi consistency 2022-07-31 17:51:19 +02:00
robcaulk d830105605 *BREAKING CHANGE* remove unnecessary arguments from populate_any_indicators(), accommodate tests 2022-07-31 17:05:29 +02:00
robcaulk dd8288c090 expose full parameter set for SVM outlier detection. Set default shuffle to false to improve reproducibility 2022-07-30 13:40:05 +02:00
robcaulk f22b140782 fix backtesting bug, undo move of label stat calc, fix example strat exit logic 2022-07-29 17:27:35 +02:00
robcaulk 59624181bd isort BaseRegressionModel imports 2022-07-29 08:23:44 +02:00
robcaulk c84d54b35e Fix typing issue, avoid using .get() when unnecessary, convert to fstrings 2022-07-29 08:12:50 +02:00
Matthias efbd83c56d Small type and typo fixes in freqai_interface 2022-07-28 07:24:30 +02:00
Matthias a2a0d35a24 Update missing typehints 2022-07-28 07:07:40 +02:00
Matthias f31106dc61 Minor comment fixes 2022-07-27 07:27:24 +02:00
Matthias bc760b7eb2 Simplify small segment in datadrawer 2022-07-26 19:41:49 +02:00
robcaulk 324e54c015 fix possible memory leak associated with Catboost Pool object 2022-07-26 17:29:29 +02:00
robcaulk 3f149c4067 fix return type in BaseTensorFlowModel 2022-07-26 16:01:54 +02:00
robcaulk ad25a4cb56 reduce number of pair_dict lookups, remove coin_first 2022-07-26 15:58:40 +02:00
robcaulk fb4e8430cd isort auto import sorting 2022-07-26 10:51:39 +02:00
robcaulk e213d0ad55 isolate data_drawer functions from data_kitchen, accommodate tests, add new test 2022-07-26 10:24:14 +02:00
robcaulk 56b17e6f3c allow user to pass test_size = 0 and avoid using eval sets in prediction models 2022-07-25 19:40:13 +02:00
Robert Caulk 897f18a8c8 ensure proper integer type casting for timestamps. Add check test for backtesting subdaily time periods 2022-07-25 15:07:09 +02:00
Robert Caulk 7b105532d1 fix mypy error and add test for principal component analysis 2022-07-25 11:46:59 +02:00
Robert Caulk c9d46a5237 finish bringing follow_mode up to date 2022-07-25 09:24:40 +02:00
Robert Caulk ab587747fb first fix for follower path bug 2022-07-24 23:32:24 +02:00
Matthias 520ee3f7a1 Convert freqAI into packages 2022-07-24 17:07:45 +02:00
Matthias 1885deb632 More docstring changes 2022-07-24 16:54:39 +02:00
Matthias 70b7a254af Update some areas to use default docstring formatting 2022-07-24 16:51:48 +02:00
Matthias 61c41fd919 Merge branch 'develop' into feat/freqai 2022-07-24 16:18:58 +02:00
Robert Caulk 88e10f7306 add exception for not passing timerange. Remove hard coded arguments for CatboostPredictionModels. Update docs 2022-07-24 09:01:23 +02:00
Robert Caulk fff39eff9e fix multitarget bug 2022-07-24 08:42:50 +02:00
robcaulk f3d46613ee move prediction denormalization into datakitchen. remove duplicate associated code. avoid normalization/denormalization for string dtypes. 2022-07-23 17:14:33 +02:00
Matthias 8fa6e8b4ba Remove freqAI model bridge in favor of `self.freqai` 2022-07-23 17:08:05 +02:00
robcaulk c91e23dc50 let user avoid normalizing labels 2022-07-23 16:14:13 +02:00
robcaulk 50d630a155 remove unnecessary comments from data_drawer.py 2022-07-23 13:35:44 +02:00
robcaulk a1cff377ec add record of contribution to data_kitchen.py 2022-07-23 13:32:04 +02:00
robcaulk c2d6a0e891 add record of contribution to doc and source 2022-07-23 13:04:06 +02:00
robcaulk 5559e605b8 small PR conversation resolutions 2022-07-22 17:46:14 +02:00
robcaulk 40f00196eb use cloudpickle in place of pickle. define Paths once in data_drawer. 2022-07-22 17:37:51 +02:00
robcaulk 98c8a447b2 add LightGBMPredictionMultiModel 2022-07-22 12:40:51 +02:00
robcaulk afcb0bec00 clean up obsolete comments, move remove_features_from_df to datakitchen 2022-07-22 12:29:20 +02:00
robcaulk 3205788bce extend doc to include descriptions of the return values from FreqAI to the strategy 2022-07-21 22:11:46 +02:00
robcaulk 8033e0bf23 add counter to backtesting log so users know how many more pairs and how many more models will need to be trained 2022-07-21 13:22:12 +02:00
robcaulk 183dec866a remove ability to backtest open ended timeranges (safer) 2022-07-21 13:02:52 +02:00
robcaulk e694ea1cfd make sure backtesting gets the populated indicators with slimmed down user strat 2022-07-21 12:48:09 +02:00
robcaulk ca4dd58642 remove superceded function from datakitchen 2022-07-21 12:40:54 +02:00
robcaulk 8f86b0deaa *breaking change* simplify user strat by consolidating feature loops into backend 2022-07-21 12:24:22 +02:00
robcaulk e7337728bf add separator in folder name just incase an asset ends in an integer 2022-07-21 11:25:28 +02:00
robcaulk 921a7ef216 add requirements-freqai.txt to builds 2022-07-20 15:51:25 +02:00
robcaulk d43c146676 add more tests for datakitchen functionalities, add regression tests for freqai_interface train/backtest 2022-07-20 12:56:46 +02:00
lolong 9c051958a6
Feat/freqai (#7105)
Vectorize weight setting, log training dates

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-07-19 17:49:18 +02:00
robcaulk 714d9534b6 start adding tests 2022-07-19 16:16:44 +02:00
lolong ed0f8b1189
Improve FreqAI documentation (#7072)
Improve doc + some other small fixes

Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-07-18 11:57:52 +02:00
Robert Caulk 921f3899f0 revert pickle reading for historic predictions 2022-07-17 16:06:36 +02:00
Robert Caulk 41eeb99177 load pickle file for writing 2022-07-17 10:05:21 +02:00
Robert Caulk 9d184586f1 fix bug in historic prediction saving 2022-07-16 21:16:59 +02:00
Wagner Costa Santos ca2029a46b fix issue when bot restarts with same identifier, does not load prediction history 2022-07-14 18:55:24 -03:00
robcaulk 4141d165ff add BaseTensorFlowModel class 2022-07-12 19:10:09 +02:00
robcaulk ef409dd345 Add ground work for TensorFlow models, add protections from common mistakes 2022-07-12 18:09:17 +02:00
Robert Caulk fea63fba12 Fix saving/loading historic predictions 2022-07-12 10:12:50 +02:00
Robert Caulk 8ce6b18318 start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available. 2022-07-11 22:01:48 +02:00
Matthias 3fc92b1b21 Create BaseRegression model - designed to reduce code duplication across currently available models. 2022-07-11 11:33:59 +02:00
Robert Caulk 607455919e Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!) 2022-07-10 12:35:44 +02:00
Matthias 819cc9c0e4 Fully align LightGBM with Catboost 2022-07-10 11:06:18 +02:00
Matthias 58b18770e3 Fix LightGBM missing argument in predict method 2022-07-10 11:05:35 +02:00
Matthias 2e1061af64 Fix faulty LightGBM model 2022-07-09 08:21:42 +00:00
robcaulk d9acdc9767 remove excess, increase no model warning clarity 2022-07-06 18:20:21 +02:00
robcaulk 4cac67fd66 Catch infrequent issue associated with grabbing first candle 2022-07-05 12:43:33 +02:00
robcaulk bd3a6ba2fe update backtesting to handle new output framework 2022-07-03 17:34:44 +02:00
robcaulk 8ac8d53c32 All LGBMRegressor model parameters are now set in config 2022-07-03 16:30:01 +02:00
robcaulk 4ff0ef7359 fix bug returning multiple targets for training 2022-07-03 12:15:59 +02:00
robcaulk ffb39a5029 black formatting on freqai files 2022-07-03 10:59:38 +02:00
robcaulk 106131ff0f Rehaul organization of return values 2022-07-02 18:09:38 +02:00
robcaulk 93e1410ed9 first step toward cleaning output and enabling multimodel training per pair 2022-07-01 14:00:30 +02:00
robcaulk 6c7d02cb18 expose nu in the SVM outlier detection via svm_nu in config 2022-06-28 15:12:25 +02:00
robcaulk 7dfbd432d1 fix config saving bug, assign priorities to pairs in scanning, sleep the scanning loop to reduce CPU usage 2022-06-27 11:35:33 +02:00
robcaulk 68bafa9517 archive config to the model folder, filter out features before returning dataframe to strategy (to alleviate frequi issues)` 2022-06-26 23:03:48 +02:00
robcaulk 051b99791d reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel 2022-06-26 19:04:23 +02:00
Robert Caulk 852706cd6b
Fix default behavior for expiration_hours 2022-06-21 08:12:51 +02:00
robcaulk b35c64b6c0 fix bug in backtest, typo in example strat 2022-06-19 16:41:09 +02:00
robcaulk 3599d18ff6 fix bug in follow_mode, thanks @blood4rc 2022-06-18 12:05:28 +02:00
robcaulk 6da7a98857 add docstrings to new functions, remove superceded code 2022-06-17 16:16:23 +02:00
robcaulk f631ae911b add model expiration feature, fix bug in DI return values 2022-06-17 14:55:40 +02:00
robcaulk 0b0688a91e ensure scanning purges models 2022-06-16 16:12:38 +02:00
robcaulk c5de0c49e4 first functional scanning commit 2022-06-16 00:24:18 +02:00
robcaulk 4d472a0ea1 merging datarehaul into scanning branch 2022-06-16 00:22:49 +02:00
Matthias c981ad4608 Fix missing space 2022-06-12 08:31:02 +02:00
robcaulk eb47c74096 merge datarehaul into main freqai branch 2022-06-10 20:26:19 +02:00
robcaulk d9b79d94e4 increase candle update flexibility to allow long sequential trainings that may last more than one candle 2022-06-07 20:57:10 +02:00
robcaulk 66800c7a45 ensure newest candles are always appended 2022-06-07 20:24:23 +02:00
robcaulk 15d049cffe detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check 2022-06-07 19:49:20 +02:00
robcaulk cab8f517b4 add lock to datadrawer 2022-06-07 01:07:30 +02:00
robcaulk 4b26b6aaec add lock to any historic data access 2022-06-07 00:54:18 +02:00
Robert Caulk 2451ed8c88
Quick bug fix 2022-06-06 15:11:54 -06:00
robcaulk d6b8801f41 fix follower bug 2022-06-05 04:40:58 +02:00
robcaulk e8c0dcf9f3 add debug message to timerange 2022-06-03 17:14:07 +02:00
robcaulk f2762e3b4b fix bug in return_values() 2022-06-03 16:58:51 +02:00
robcaulk 16b4a5b71f rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability. 2022-06-03 15:19:46 +02:00
robcaulk 15a971346d catch infinity values when filtering 2022-06-02 17:13:20 +02:00
robcaulk ace951bf7e another queue bug fix for fresh starts 2022-06-02 13:28:06 +02:00
robcaulk 833d25bda0 Save data after queue reorg 2022-06-01 08:05:38 +02:00