robcaulk
|
f3d46613ee
|
move prediction denormalization into datakitchen. remove duplicate associated code. avoid normalization/denormalization for string dtypes.
|
2022-07-23 17:14:33 +02:00 |
|
Matthias
|
8fa6e8b4ba
|
Remove freqAI model bridge in favor of self.freqai
|
2022-07-23 17:08:05 +02:00 |
|
robcaulk
|
c91e23dc50
|
let user avoid normalizing labels
|
2022-07-23 16:14:13 +02:00 |
|
robcaulk
|
50d630a155
|
remove unnecessary comments from data_drawer.py
|
2022-07-23 13:35:44 +02:00 |
|
robcaulk
|
a1cff377ec
|
add record of contribution to data_kitchen.py
|
2022-07-23 13:32:04 +02:00 |
|
robcaulk
|
c2d6a0e891
|
add record of contribution to doc and source
|
2022-07-23 13:04:06 +02:00 |
|
robcaulk
|
5559e605b8
|
small PR conversation resolutions
|
2022-07-22 17:46:14 +02:00 |
|
robcaulk
|
40f00196eb
|
use cloudpickle in place of pickle. define Paths once in data_drawer.
|
2022-07-22 17:37:51 +02:00 |
|
robcaulk
|
98c8a447b2
|
add LightGBMPredictionMultiModel
|
2022-07-22 12:40:51 +02:00 |
|
robcaulk
|
afcb0bec00
|
clean up obsolete comments, move remove_features_from_df to datakitchen
|
2022-07-22 12:29:20 +02:00 |
|
robcaulk
|
3205788bce
|
extend doc to include descriptions of the return values from FreqAI to the strategy
|
2022-07-21 22:11:46 +02:00 |
|
robcaulk
|
8033e0bf23
|
add counter to backtesting log so users know how many more pairs and how many more models will need to be trained
|
2022-07-21 13:22:12 +02:00 |
|
robcaulk
|
183dec866a
|
remove ability to backtest open ended timeranges (safer)
|
2022-07-21 13:02:52 +02:00 |
|
robcaulk
|
e694ea1cfd
|
make sure backtesting gets the populated indicators with slimmed down user strat
|
2022-07-21 12:48:09 +02:00 |
|
robcaulk
|
ca4dd58642
|
remove superceded function from datakitchen
|
2022-07-21 12:40:54 +02:00 |
|
robcaulk
|
8f86b0deaa
|
*breaking change* simplify user strat by consolidating feature loops into backend
|
2022-07-21 12:24:22 +02:00 |
|
robcaulk
|
e7337728bf
|
add separator in folder name just incase an asset ends in an integer
|
2022-07-21 11:25:28 +02:00 |
|
robcaulk
|
921a7ef216
|
add requirements-freqai.txt to builds
|
2022-07-20 15:51:25 +02:00 |
|
robcaulk
|
d43c146676
|
add more tests for datakitchen functionalities, add regression tests for freqai_interface train/backtest
|
2022-07-20 12:56:46 +02:00 |
|
lolong
|
9c051958a6
|
Feat/freqai (#7105)
Vectorize weight setting, log training dates
Co-authored-by: robcaulk <rob.caulk@gmail.com>
|
2022-07-19 17:49:18 +02:00 |
|
robcaulk
|
714d9534b6
|
start adding tests
|
2022-07-19 16:16:44 +02:00 |
|
lolong
|
ed0f8b1189
|
Improve FreqAI documentation (#7072)
Improve doc + some other small fixes
Co-authored-by: robcaulk <rob.caulk@gmail.com>
|
2022-07-18 11:57:52 +02:00 |
|
Robert Caulk
|
921f3899f0
|
revert pickle reading for historic predictions
|
2022-07-17 16:06:36 +02:00 |
|
Robert Caulk
|
41eeb99177
|
load pickle file for writing
|
2022-07-17 10:05:21 +02:00 |
|
Robert Caulk
|
9d184586f1
|
fix bug in historic prediction saving
|
2022-07-16 21:16:59 +02:00 |
|
Wagner Costa Santos
|
ca2029a46b
|
fix issue when bot restarts with same identifier, does not load prediction history
|
2022-07-14 18:55:24 -03:00 |
|
robcaulk
|
4141d165ff
|
add BaseTensorFlowModel class
|
2022-07-12 19:10:09 +02:00 |
|
robcaulk
|
ef409dd345
|
Add ground work for TensorFlow models, add protections from common mistakes
|
2022-07-12 18:09:17 +02:00 |
|
Robert Caulk
|
fea63fba12
|
Fix saving/loading historic predictions
|
2022-07-12 10:12:50 +02:00 |
|
Robert Caulk
|
8ce6b18318
|
start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available.
|
2022-07-11 22:01:48 +02:00 |
|
Matthias
|
3fc92b1b21
|
Create BaseRegression model - designed to reduce code duplication across currently available models.
|
2022-07-11 11:33:59 +02:00 |
|
Robert Caulk
|
607455919e
|
Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!)
|
2022-07-10 12:35:44 +02:00 |
|
Matthias
|
819cc9c0e4
|
Fully align LightGBM with Catboost
|
2022-07-10 11:06:18 +02:00 |
|
Matthias
|
58b18770e3
|
Fix LightGBM missing argument in predict method
|
2022-07-10 11:05:35 +02:00 |
|
Matthias
|
2e1061af64
|
Fix faulty LightGBM model
|
2022-07-09 08:21:42 +00:00 |
|
robcaulk
|
d9acdc9767
|
remove excess, increase no model warning clarity
|
2022-07-06 18:20:21 +02:00 |
|
robcaulk
|
4cac67fd66
|
Catch infrequent issue associated with grabbing first candle
|
2022-07-05 12:43:33 +02:00 |
|
robcaulk
|
bd3a6ba2fe
|
update backtesting to handle new output framework
|
2022-07-03 17:34:44 +02:00 |
|
robcaulk
|
8ac8d53c32
|
All LGBMRegressor model parameters are now set in config
|
2022-07-03 16:30:01 +02:00 |
|
robcaulk
|
4ff0ef7359
|
fix bug returning multiple targets for training
|
2022-07-03 12:15:59 +02:00 |
|
robcaulk
|
ffb39a5029
|
black formatting on freqai files
|
2022-07-03 10:59:38 +02:00 |
|
robcaulk
|
106131ff0f
|
Rehaul organization of return values
|
2022-07-02 18:09:38 +02:00 |
|
robcaulk
|
93e1410ed9
|
first step toward cleaning output and enabling multimodel training per pair
|
2022-07-01 14:00:30 +02:00 |
|
robcaulk
|
6c7d02cb18
|
expose nu in the SVM outlier detection via svm_nu in config
|
2022-06-28 15:12:25 +02:00 |
|
robcaulk
|
7dfbd432d1
|
fix config saving bug, assign priorities to pairs in scanning, sleep the scanning loop to reduce CPU usage
|
2022-06-27 11:35:33 +02:00 |
|
robcaulk
|
68bafa9517
|
archive config to the model folder, filter out features before returning dataframe to strategy (to alleviate frequi issues)`
|
2022-06-26 23:03:48 +02:00 |
|
robcaulk
|
051b99791d
|
reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel
|
2022-06-26 19:04:23 +02:00 |
|
Robert Caulk
|
852706cd6b
|
Fix default behavior for expiration_hours
|
2022-06-21 08:12:51 +02:00 |
|
robcaulk
|
b35c64b6c0
|
fix bug in backtest, typo in example strat
|
2022-06-19 16:41:09 +02:00 |
|
robcaulk
|
3599d18ff6
|
fix bug in follow_mode, thanks @blood4rc
|
2022-06-18 12:05:28 +02:00 |
|
robcaulk
|
6da7a98857
|
add docstrings to new functions, remove superceded code
|
2022-06-17 16:16:23 +02:00 |
|
robcaulk
|
f631ae911b
|
add model expiration feature, fix bug in DI return values
|
2022-06-17 14:55:40 +02:00 |
|
robcaulk
|
0b0688a91e
|
ensure scanning purges models
|
2022-06-16 16:12:38 +02:00 |
|
robcaulk
|
c5de0c49e4
|
first functional scanning commit
|
2022-06-16 00:24:18 +02:00 |
|
robcaulk
|
4d472a0ea1
|
merging datarehaul into scanning branch
|
2022-06-16 00:22:49 +02:00 |
|
Matthias
|
c981ad4608
|
Fix missing space
|
2022-06-12 08:31:02 +02:00 |
|
robcaulk
|
eb47c74096
|
merge datarehaul into main freqai branch
|
2022-06-10 20:26:19 +02:00 |
|
robcaulk
|
d9b79d94e4
|
increase candle update flexibility to allow long sequential trainings that may last more than one candle
|
2022-06-07 20:57:10 +02:00 |
|
robcaulk
|
66800c7a45
|
ensure newest candles are always appended
|
2022-06-07 20:24:23 +02:00 |
|
robcaulk
|
15d049cffe
|
detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check
|
2022-06-07 19:49:20 +02:00 |
|
robcaulk
|
cab8f517b4
|
add lock to datadrawer
|
2022-06-07 01:07:30 +02:00 |
|
robcaulk
|
4b26b6aaec
|
add lock to any historic data access
|
2022-06-07 00:54:18 +02:00 |
|
Robert Caulk
|
2451ed8c88
|
Quick bug fix
|
2022-06-06 15:11:54 -06:00 |
|
robcaulk
|
d6b8801f41
|
fix follower bug
|
2022-06-05 04:40:58 +02:00 |
|
robcaulk
|
e8c0dcf9f3
|
add debug message to timerange
|
2022-06-03 17:14:07 +02:00 |
|
robcaulk
|
f2762e3b4b
|
fix bug in return_values()
|
2022-06-03 16:58:51 +02:00 |
|
robcaulk
|
16b4a5b71f
|
rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.
|
2022-06-03 15:19:46 +02:00 |
|
robcaulk
|
15a971346d
|
catch infinity values when filtering
|
2022-06-02 17:13:20 +02:00 |
|
robcaulk
|
ace951bf7e
|
another queue bug fix for fresh starts
|
2022-06-02 13:28:06 +02:00 |
|
robcaulk
|
833d25bda0
|
Save data after queue reorg
|
2022-06-01 08:05:38 +02:00 |
|
robcaulk
|
0b0dd8dd80
|
Force high priority pair to be trained before anything else.
|
2022-06-01 07:55:05 +02:00 |
|
robcaulk
|
7523ed825e
|
automatically detect maximum required data based on user fed indicators (to avoid NaNs in dataset for rolling indicators), add new config parameter for backtesting to let users increase their startup_candles to accommodate high timeframe indicators, add docs to explain all. Add new feature for automatic indicator duplication according to user defined intervals (exhibited in example strat and configs now).
|
2022-05-31 18:42:27 +02:00 |
|
robcaulk
|
9b3b08a2bb
|
let follower purge old model files
|
2022-05-31 15:37:38 +02:00 |
|
robcaulk
|
bac4ced382
|
Ensure follower predictions are persistent and uniquely stored
|
2022-05-31 14:35:04 +02:00 |
|
robcaulk
|
70adf55643
|
Automatically detect and change follower data_path to accommodate remote systems
|
2022-05-31 12:35:09 +02:00 |
|
robcaulk
|
0306f5ca13
|
Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk
|
2022-05-31 11:58:21 +02:00 |
|
robcaulk
|
45f4f0f603
|
ensure follower sends back null arrays in case leader hasnt created a model file yet
|
2022-05-31 01:48:48 +02:00 |
|
robcaulk
|
29d2f59f12
|
fix PCA bug
|
2022-05-31 00:40:45 +02:00 |
|
robcaulk
|
606f18e5c1
|
Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader
|
2022-05-30 21:35:48 +02:00 |
|
robcaulk
|
5b4c649d43
|
detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly
|
2022-05-30 13:55:46 +02:00 |
|
robcaulk
|
e229902381
|
fix bug in previous commit
|
2022-05-30 12:48:22 +02:00 |
|
robcaulk
|
a20651efd8
|
Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).
|
2022-05-30 11:37:05 +02:00 |
|
robcaulk
|
2f1a2c1cd7
|
allow users to store data in custom formats, update spot config to reflect better target horizon to training period ratio
|
2022-05-30 02:12:31 +02:00 |
|
robcaulk
|
d59eac3321
|
revert a79032b
|
2022-05-29 21:33:38 +02:00 |
|
robcaulk
|
a79032bf75
|
fixing bug in training queue
|
2022-05-29 20:19:32 +02:00 |
|
robcaulk
|
3f72263278
|
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
|
2022-05-29 20:02:43 +02:00 |
|
robcaulk
|
cc6cae47ec
|
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
|
2022-05-29 19:49:43 +02:00 |
|
robcaulk
|
4eb4753e20
|
allow subdaily retraining for backtesting
|
2022-05-29 17:44:35 +02:00 |
|
robcaulk
|
ce365eb9e3
|
improve example strat so that it has dynamic buy and sell logic according to original prediction
|
2022-05-29 14:45:46 +02:00 |
|
robcaulk
|
4eb29c8810
|
Dont reset pair priority if it doesnt successfully train
|
2022-05-28 18:34:26 +02:00 |
|
robcaulk
|
83dd453723
|
catch errors occuring on background thread, and make sure to keep the ball rolling. Improve pair retraining queue.
|
2022-05-28 18:26:19 +02:00 |
|
robcaulk
|
e54614fa2f
|
remove remnants of single threaded version, ensure pair queue priority is checked before retraining
|
2022-05-28 14:55:07 +02:00 |
|
robcaulk
|
2a4d1e2d64
|
fix bug in setting new timerange for retraining
|
2022-05-28 12:23:26 +02:00 |
|
robcaulk
|
7870a86e9a
|
fix live retraining bug
|
2022-05-28 11:38:57 +02:00 |
|
robcaulk
|
0bf915054d
|
handle key check correctly
|
2022-05-28 11:22:32 +02:00 |
|
robcaulk
|
c5a16e91fb
|
throw user error if user tries to load models but feeds the wrong features (while using PCA)
|
2022-05-28 11:11:41 +02:00 |
|
robcaulk
|
b8f9c3557b
|
dirty dirty, dont look here (hacking a flag to avoid reloading leverage_tiers in dry/live)
|
2022-05-27 13:56:34 +02:00 |
|
robcaulk
|
891fb87712
|
give load_cached_data_for_updating the right flags to avoid redownloading data in dry/live
|
2022-05-27 13:38:22 +02:00 |
|
robcaulk
|
65fdebab75
|
let load_pairs_histories load futures candles in live
|
2022-05-27 13:01:33 +02:00 |
|
robcaulk
|
c080571b7a
|
help futures go dry/live with auto download feature
|
2022-05-27 12:23:32 +02:00 |
|