 Robert Caulk
|
4ef2ed2f1b
|
Merge pull request #7085 from wagnercosta/feat/freqai
freqai: fix issue when bot restarts with same identifier, does not load predi…
|
2022-07-15 20:00:53 +02:00 |
|
|
robcaulk
|
4141d165ff
|
add BaseTensorFlowModel class
|
2022-07-12 19:10:09 +02:00 |
|
|
robcaulk
|
ef409dd345
|
Add ground work for TensorFlow models, add protections from common mistakes
|
2022-07-12 18:09:17 +02:00 |
|
|
Robert Caulk
|
fea63fba12
|
Fix saving/loading historic predictions
|
2022-07-12 10:12:50 +02:00 |
|
|
Robert Caulk
|
8ce6b18318
|
start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available.
|
2022-07-11 22:01:48 +02:00 |
|
|
Robert Caulk
|
607455919e
|
Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!)
|
2022-07-10 12:35:44 +02:00 |
|
|
robcaulk
|
d9acdc9767
|
remove excess, increase no model warning clarity
|
2022-07-06 18:20:21 +02:00 |
|
|
robcaulk
|
4cac67fd66
|
Catch infrequent issue associated with grabbing first candle
|
2022-07-05 12:43:33 +02:00 |
|
|
robcaulk
|
bd3a6ba2fe
|
update backtesting to handle new output framework
|
2022-07-03 17:34:44 +02:00 |
|
|
robcaulk
|
8ac8d53c32
|
All LGBMRegressor model parameters are now set in config
|
2022-07-03 16:30:01 +02:00 |
|
|
robcaulk
|
a6077ac7f4
|
Merge feat/freqai into develop to get new features
|
2022-07-03 16:17:13 +02:00 |
|
|
robcaulk
|
4ff0ef7359
|
fix bug returning multiple targets for training
|
2022-07-03 12:15:59 +02:00 |
|
|
robcaulk
|
ffb39a5029
|
black formatting on freqai files
|
2022-07-03 10:59:38 +02:00 |
|
|
robcaulk
|
106131ff0f
|
Rehaul organization of return values
|
2022-07-02 18:09:38 +02:00 |
|
|
robcaulk
|
93e1410ed9
|
first step toward cleaning output and enabling multimodel training per pair
|
2022-07-01 14:00:30 +02:00 |
|
|
robcaulk
|
6c7d02cb18
|
expose nu in the SVM outlier detection via svm_nu in config
|
2022-06-28 15:12:25 +02:00 |
|
|
robcaulk
|
7dfbd432d1
|
fix config saving bug, assign priorities to pairs in scanning, sleep the scanning loop to reduce CPU usage
|
2022-06-27 11:35:33 +02:00 |
|
|
robcaulk
|
68bafa9517
|
archive config to the model folder, filter out features before returning dataframe to strategy (to alleviate frequi issues)`
|
2022-06-26 23:03:48 +02:00 |
|
|
robcaulk
|
051b99791d
|
reduce unnecessary verbosity, fix error on first training sweep, add LightGBMPredictionModel
|
2022-06-26 19:04:23 +02:00 |
|
|
Robert Caulk
|
852706cd6b
|
Fix default behavior for expiration_hours
|
2022-06-21 08:12:51 +02:00 |
|
|
robcaulk
|
b35c64b6c0
|
fix bug in backtest, typo in example strat
|
2022-06-19 16:41:09 +02:00 |
|
|
robcaulk
|
3599d18ff6
|
fix bug in follow_mode, thanks @blood4rc
|
2022-06-18 12:05:28 +02:00 |
|
|
robcaulk
|
6da7a98857
|
add docstrings to new functions, remove superceded code
|
2022-06-17 16:16:23 +02:00 |
|
|
robcaulk
|
5e914d5756
|
improve model youth by constantly scanning pairs in dry/live and always training new models. Fix bug in DI return values
|
2022-06-17 16:06:51 +02:00 |
|
|
robcaulk
|
f631ae911b
|
add model expiration feature, fix bug in DI return values
|
2022-06-17 14:55:40 +02:00 |
|
|
robcaulk
|
0b0688a91e
|
ensure scanning purges models
|
2022-06-16 16:12:38 +02:00 |
|
|
robcaulk
|
c5de0c49e4
|
first functional scanning commit
|
2022-06-16 00:24:18 +02:00 |
|
|
robcaulk
|
4d472a0ea1
|
merging datarehaul into scanning branch
|
2022-06-16 00:22:49 +02:00 |
|
|
robcaulk
|
eb47c74096
|
merge datarehaul into main freqai branch
|
2022-06-10 20:26:19 +02:00 |
|
|
robcaulk
|
d9b79d94e4
|
increase candle update flexibility to allow long sequential trainings that may last more than one candle
|
2022-06-07 20:57:10 +02:00 |
|
|
robcaulk
|
66800c7a45
|
ensure newest candles are always appended
|
2022-06-07 20:24:23 +02:00 |
|
|
robcaulk
|
f8f25e36ef
|
update example config/strat
|
2022-06-07 19:54:45 +02:00 |
|
|
robcaulk
|
15d049cffe
|
detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check
|
2022-06-07 19:49:20 +02:00 |
|
|
robcaulk
|
cab8f517b4
|
add lock to datadrawer
|
2022-06-07 01:07:30 +02:00 |
|
|
robcaulk
|
4b26b6aaec
|
add lock to any historic data access
|
2022-06-07 00:54:18 +02:00 |
|
|
Robert Caulk
|
3c2e314ee5
|
Fix bugs
|
2022-06-06 16:26:07 -06:00 |
|
|
Robert Caulk
|
e6c5e737a2
|
Fix other bugs
|
2022-06-06 16:24:32 -06:00 |
|
|
Robert Caulk
|
bf19055e53
|
Update function spelling
|
2022-06-06 15:56:12 -06:00 |
|
|
Robert Caulk
|
2451ed8c88
|
Quick bug fix
|
2022-06-06 15:11:54 -06:00 |
|
|
robcaulk
|
d6b8801f41
|
fix follower bug
|
2022-06-05 04:40:58 +02:00 |
|
|
robcaulk
|
e8c0dcf9f3
|
add debug message to timerange
|
2022-06-03 17:14:07 +02:00 |
|
|
robcaulk
|
f2762e3b4b
|
fix bug in return_values()
|
2022-06-03 16:58:51 +02:00 |
|
|
robcaulk
|
16b4a5b71f
|
rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.
|
2022-06-03 15:19:46 +02:00 |
|
|
robcaulk
|
15a971346d
|
catch infinity values when filtering
|
2022-06-02 17:13:20 +02:00 |
|
|
Robert Caulk
|
7d41542f93
|
process_only_new_candles in examplestrat should be True, thanks @Bloodhunter4rc
|
2022-06-02 07:24:08 -06:00 |
|
|
robcaulk
|
fea39254d9
|
fix custom_exit (thanks @paranoidandy)
|
2022-06-02 14:58:45 +02:00 |
|
|
robcaulk
|
b37c31cc21
|
fix ta-lib issue with simultaneous method access
|
2022-06-02 14:37:40 +02:00 |
|
|
robcaulk
|
4ac6ef2972
|
make defining period intervals more user friendly and flexible
|
2022-06-02 13:45:29 +02:00 |
|
|
robcaulk
|
ace951bf7e
|
another queue bug fix for fresh starts
|
2022-06-02 13:28:06 +02:00 |
|
|
robcaulk
|
833d25bda0
|
Save data after queue reorg
|
2022-06-01 08:05:38 +02:00 |
|
|
robcaulk
|
0b0dd8dd80
|
Force high priority pair to be trained before anything else.
|
2022-06-01 07:55:05 +02:00 |
|
|
robcaulk
|
7523ed825e
|
automatically detect maximum required data based on user fed indicators (to avoid NaNs in dataset for rolling indicators), add new config parameter for backtesting to let users increase their startup_candles to accommodate high timeframe indicators, add docs to explain all. Add new feature for automatic indicator duplication according to user defined intervals (exhibited in example strat and configs now).
|
2022-05-31 18:42:27 +02:00 |
|
|
robcaulk
|
9b3b08a2bb
|
let follower purge old model files
|
2022-05-31 15:37:38 +02:00 |
|
|
robcaulk
|
bac4ced382
|
Ensure follower predictions are persistent and uniquely stored
|
2022-05-31 14:35:04 +02:00 |
|
|
robcaulk
|
70adf55643
|
Automatically detect and change follower data_path to accommodate remote systems
|
2022-05-31 12:35:09 +02:00 |
|
|
robcaulk
|
0306f5ca13
|
Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk
|
2022-05-31 11:58:21 +02:00 |
|
|
robcaulk
|
45f4f0f603
|
ensure follower sends back null arrays in case leader hasnt created a model file yet
|
2022-05-31 01:48:48 +02:00 |
|
|
robcaulk
|
29d2f59f12
|
fix PCA bug
|
2022-05-31 00:40:45 +02:00 |
|
|
robcaulk
|
606f18e5c1
|
Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader
|
2022-05-30 21:35:48 +02:00 |
|
|
robcaulk
|
5b4c649d43
|
detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly
|
2022-05-30 13:55:46 +02:00 |
|
|
robcaulk
|
e229902381
|
fix bug in previous commit
|
2022-05-30 12:48:22 +02:00 |
|
|
robcaulk
|
a20651efd8
|
Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).
|
2022-05-30 11:37:05 +02:00 |
|
|
robcaulk
|
2f1a2c1cd7
|
allow users to store data in custom formats, update spot config to reflect better target horizon to training period ratio
|
2022-05-30 02:12:31 +02:00 |
|
|
robcaulk
|
d59eac3321
|
revert a79032b
|
2022-05-29 21:33:38 +02:00 |
|
|
robcaulk
|
a79032bf75
|
fixing bug in training queue
|
2022-05-29 20:19:32 +02:00 |
|
|
robcaulk
|
3f72263278
|
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
|
2022-05-29 20:02:43 +02:00 |
|
|
robcaulk
|
cc6cae47ec
|
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
|
2022-05-29 19:49:43 +02:00 |
|
|
robcaulk
|
4eb4753e20
|
allow subdaily retraining for backtesting
|
2022-05-29 17:44:35 +02:00 |
|
|
robcaulk
|
0aa7162055
|
ensure the prediction is reset in the pair_dict after any trade exit, not just custom_exit
|
2022-05-29 16:36:46 +02:00 |
|
|
robcaulk
|
fe36b08fce
|
fix key error in example strat
|
2022-05-29 16:26:34 +02:00 |
|
|
robcaulk
|
ce365eb9e3
|
improve example strat so that it has dynamic buy and sell logic according to original prediction
|
2022-05-29 14:45:46 +02:00 |
|
|
robcaulk
|
4eb29c8810
|
Dont reset pair priority if it doesnt successfully train
|
2022-05-28 18:34:26 +02:00 |
|
|
robcaulk
|
83dd453723
|
catch errors occuring on background thread, and make sure to keep the ball rolling. Improve pair retraining queue.
|
2022-05-28 18:26:19 +02:00 |
|
|
robcaulk
|
e54614fa2f
|
remove remnants of single threaded version, ensure pair queue priority is checked before retraining
|
2022-05-28 14:55:07 +02:00 |
|
|
robcaulk
|
2a4d1e2d64
|
fix bug in setting new timerange for retraining
|
2022-05-28 12:23:26 +02:00 |
|
|
robcaulk
|
7870a86e9a
|
fix live retraining bug
|
2022-05-28 11:38:57 +02:00 |
|
|
robcaulk
|
0bf915054d
|
handle key check correctly
|
2022-05-28 11:22:32 +02:00 |
|
|
robcaulk
|
c5a16e91fb
|
throw user error if user tries to load models but feeds the wrong features (while using PCA)
|
2022-05-28 11:11:41 +02:00 |
|
|
robcaulk
|
b8f9c3557b
|
dirty dirty, dont look here (hacking a flag to avoid reloading leverage_tiers in dry/live)
|
2022-05-27 13:56:34 +02:00 |
|
|
robcaulk
|
891fb87712
|
give load_cached_data_for_updating the right flags to avoid redownloading data in dry/live
|
2022-05-27 13:38:22 +02:00 |
|
|
robcaulk
|
65fdebab75
|
let load_pairs_histories load futures candles in live
|
2022-05-27 13:01:33 +02:00 |
|
|
robcaulk
|
c080571b7a
|
help futures go dry/live with auto download feature
|
2022-05-27 12:23:32 +02:00 |
|
|
robcaulk
|
8a501831d6
|
fix the error logic on previous commit
|
2022-05-27 01:15:55 +02:00 |
|
|
robcaulk
|
23c30dbc10
|
add error for user trying to backtest with backtest_period<1
|
2022-05-27 00:43:52 +02:00 |
|
|
robcaulk
|
6193205012
|
fix bug for target_mean/std array merging in backtesting
|
2022-05-26 21:07:50 +02:00 |
|
|
robcaulk
|
ff531c416f
|
reduce complexity inside start_download_data() in an effort to appease flake8
|
2022-05-25 15:31:50 +02:00 |
|
|
robcaulk
|
d79983c791
|
try to pass flake8
|
2022-05-25 14:55:19 +02:00 |
|
|
robcaulk
|
7593339c14
|
small cleanup
|
2022-05-25 14:42:46 +02:00 |
|
|
robcaulk
|
b79d4e8876
|
Allow user to go live and start from pretrained models (after a completed backtest) by simply reusing the identifier config parameter while dry/live.
|
2022-05-25 14:40:32 +02:00 |
|
|
robcaulk
|
7486d9d9e2
|
proper validation of freqai config parameters
|
2022-05-25 12:37:25 +02:00 |
|
|
robcaulk
|
7ff3258607
|
remove assertions, log error if user has not assigned freqai in config, fix stratify bug
|
2022-05-25 11:43:45 +02:00 |
|
|
robcaulk
|
35bed842cb
|
cleanup, add clarity to comments and docstrings
|
2022-05-25 11:31:03 +02:00 |
|
|
robcaulk
|
58b5abbaa6
|
improve multithreaded training queue system
|
2022-05-24 15:28:38 +02:00 |
|
|
robcaulk
|
31ae2b3060
|
alleviate FutureWarning in sklearn about ensuring svm model features are passed with identical order
|
2022-05-24 14:46:16 +02:00 |
|
|
robcaulk
|
255d35976e
|
add priority metadata to pairs to avoid a sync of train time + train period
|
2022-05-24 12:58:53 +02:00 |
|
|
robcaulk
|
059c285425
|
paying closer attention to managing live retraining on separate thread without affecting prediction of other coins on master thread
|
2022-05-24 12:01:01 +02:00 |
|
|
robcaulk
|
b0d2d13eb1
|
improve data persistence/mapping for live/dry. This accommodates quick reloads after crash and handles multi-pair cleanly
|
2022-05-23 21:05:05 +02:00 |
|
|
robcaulk
|
e1c068ca66
|
add config asserts, use .get method with default values for optional functionality, move data_cleaning_* to freqai_interface (away from user custom pred model) since it is controlled by config params.
|
2022-05-23 12:07:09 +02:00 |
|
|
robcaulk
|
5c4014ee62
|
Change default value of process_only_new_candles to True since False is an uncommon usecase for expert strategy devs
|
2022-05-23 10:24:58 +02:00 |
|
|
robcaulk
|
dede128648
|
set process_only_new_candles to true in example strat
|
2022-05-23 10:15:59 +02:00 |
|