3ee7eb63f7
starting backtest live models
2022-09-24 12:28:52 -03:00
2cc00a1a2c
Allow use of --strategy-list with freqai, with warning ( #7455 )
...
* Allow use of --strategy-list with freqai, with warning
* ensure populate_any_indicators is identical for resused identifiers
* use pair instead of metadata["pair"]
Co-authored-by: robcaulk <rob.caulk@gmail.com>
2022-09-24 13:21:01 +02:00
6643d90e64
simplify freqAI start_backtesting
2022-09-24 10:34:14 +02:00
ea8e34e192
Merge branch 'develop' into dev-merge-rl
2022-09-22 19:46:50 +02:00
b5fd11f91b
protect against unforeseen issues in scanning thread
2022-09-19 21:18:31 +02:00
42c75b4a7b
combine log messages
2022-09-19 19:16:32 +02:00
9b66297cc0
Fix append
2022-09-19 12:47:20 +02:00
995396c775
Add useful log info
2022-09-19 11:42:56 +02:00
ad652817ef
Ensure train ordering after restart
...
Ensure lowest timestamps get trained first after restart
2022-09-19 11:11:23 +02:00
f9460c80c2
Merge pull request #7434 from freqtrade/improve-train-queue
...
improve train queue system in FreqAI
2022-09-19 10:55:53 +02:00
ea58c29ded
Add plot_feature_importance to schema definition
2022-09-19 08:13:54 +00:00
225f7cd5f8
Merge pull request #7431 from initrv/add-plot-feature-importance
...
Add plot feature importance
2022-09-19 08:41:10 +02:00
d930931000
Bring back sleep - it'll ensure we give up control over the thread.
2022-09-19 07:14:33 +02:00
470d5d8405
ensure full new pairlist is in the queue
2022-09-18 17:08:07 +02:00
eaa43337d2
improve train queue system, ensure crash resilience in train queue.
2022-09-18 17:00:55 +02:00
667853c504
Use Alias to type config objects
2022-09-18 13:20:55 +02:00
2c23effbf2
allow plot to plot multitargets, add test
2022-09-17 19:17:44 +02:00
1c92734f39
simplify plot_feature_importance call
2022-09-17 18:53:43 +03:00
7b1d409c98
fix mypy/flake8
2022-09-17 17:51:06 +02:00
b707a6da35
Add ability to plot feature importance
2022-09-16 19:17:41 +03:00
3b97b3d5c8
fix mypy error for strategy
2022-09-15 00:56:51 +02:00
8aac644009
add tests. add guardrails.
2022-09-15 00:46:35 +02:00
81417cb795
Merge branch 'develop' into dev-merge-rl
2022-09-14 22:49:11 +02:00
e4caccc353
Merge pull request #7367 from freqtrade/add-continual-learning
...
add continual learning to catboost and friends
2022-09-10 20:17:28 +02:00
2077f84f9b
Merge pull request #7363 from wagnercosta/fixHyperoptFreqai
...
Fix hyperopt - freqai
2022-09-10 17:02:01 +02:00
5705b8759b
Merge branch 'develop' into add-continual-learning
2022-09-10 08:24:04 +02:00
97be3318f4
Use Datetime_format from constants
2022-09-09 20:31:02 +02:00
a826c0eb83
ensure signatures match, reduce verbosity
2022-09-09 19:30:53 +02:00
e51d352777
ensure pca is handling same DF as inlier
2022-09-07 19:11:54 +02:00
4c9ac6b7c0
add kwargs, reduce duplicated code
2022-09-07 18:58:55 +02:00
ea7bdac9ed
ensure inlier metric can be combined with other cleaning methods
2022-09-07 18:45:16 +02:00
6b7644029c
Merge branch 'freqtrade:develop' into fixHyperoptFreqai
2022-09-07 11:07:51 -03:00
8d16dd804d
hyperopt - freqai - docs and refactoring
2022-09-06 15:42:47 -03:00
d7585161b2
Merge branch 'develop' into add-inlier-metric
2022-09-06 20:40:21 +02:00
97077ba18a
add continual learning to catboost and friends
2022-09-06 20:30:46 +02:00
4b28d0495f
fix timestamping, move imports, add words to doc
2022-09-06 19:46:58 +02:00
5b826150df
fix hyperopt - freqai
2022-09-05 17:43:28 -03:00
78d01810ed
reduce code redundancy, ensure live always gets the latest data
2022-09-05 00:12:00 +02:00
d43ed186fc
fix issue with freqai backtesting at slice dataframe
2022-09-05 00:12:00 +02:00
69b3fcfd32
Merge branch 'develop' into dev-merge-rl
2022-09-04 11:23:25 +02:00
dae3b3d86a
support shutting down freqai
2022-09-03 13:24:14 -06:00
63514b0443
Merge pull request #7325 from wagnercosta/develop
...
Improve Freqai backtesting performance
2022-09-03 19:28:04 +02:00
c9be66b5b6
increase test coverage for dk, improve function naming, extra cleaning
2022-09-03 15:52:29 +02:00
80b5f035ab
Remove typo in log message
2022-09-03 15:01:28 +02:00
599c1c79fb
reorganized backtest utilities, test new functionality, improve/update doc
2022-09-03 14:00:01 +02:00
af5460cebf
Add option to keep models only in memory for backtest
2022-09-02 22:01:53 -03:00
d6e115178a
refactoring freqai backtesting - remove duplicate code
2022-09-01 07:09:23 -03:00
44d3a9140d
Merge branch 'develop' of https://github.com/wagnercosta/freqtrade into develop
2022-08-31 15:37:06 -03:00
7bed0450d2
pr review - refactoring backtesting freqai
2022-08-31 15:36:29 -03:00
3d4497467c
Merge branch 'freqtrade:develop' into develop
2022-08-31 14:47:06 -03:00
df51da22ee
refactoring freqai backtesting
2022-08-31 11:23:48 -03:00
a88ffd2c9d
Merge branch 'develop' into move_datadownload
2022-08-31 10:23:45 +00:00
7f52908e87
ensure the lost points are prepended for FreqUI
2022-08-30 18:55:58 +02:00
a58dd0bbf9
add noise feature, improve docstrings
2022-08-30 18:26:24 +02:00
b11742a4c5
integrate inlier metric function
2022-08-30 18:26:24 +02:00
7766350c15
refactor environment inheritence tree to accommodate flexible action types/counts. fix bug in train profit handling
2022-08-28 19:21:57 +02:00
f9a49744e6
add strategy to the freqai object
2022-08-24 13:00:55 +02:00
05ed1b544f
Working base for reinforcement learning model
2022-08-24 13:00:40 +02:00
96d8882f1e
Plug mem leak, add training timer
2022-08-22 13:30:30 +02:00
5155afb4e7
clean up code remnants
2022-08-17 15:22:48 +02:00
0c34104e45
extract download-data from freqai to prepare for future async changes
2022-08-17 15:18:44 +02:00
c7e1719215
Fix interface import sorting
2022-08-15 06:53:02 +02:00
8961b8d560
merge in inference timer and historic predictions handling improvements.
2022-08-14 20:31:15 +02:00
ad846cdb76
fix lock bug, update docstring
2022-08-14 20:24:29 +02:00
464d99808f
Update doc table formatting
2022-08-14 18:22:01 +02:00
a29402ddde
Rename and move analysis_lock to data_kitchen
2022-08-14 17:23:14 +02:00
31be707cc8
clean up code, add docstrings
2022-08-13 19:40:24 +02:00
3e38c1b0bd
take dynamic sized tail off historic_predictions as return dataframe to strategy.
2022-08-13 19:40:24 +02:00
7d448fd4ac
allow fit_live_predictions access to current pair
2022-08-13 19:40:24 +02:00
1f192be43b
avoid denormalizing labels twice
2022-08-13 19:40:24 +02:00
b1b76a2dbe
debug classifier with predict proba
2022-08-13 19:40:24 +02:00
2cae3c42e6
remove trade database analyzer, clean up a bit
2022-08-10 17:43:06 +02:00
77b3b8a134
Use main exchange instead of creating a separate instance.
2022-08-08 18:34:11 +00:00
eb8bde37c1
Add lightgbm classifier, add classifier check test, fix classifier bug.
2022-08-06 17:51:21 +02:00
47a30047eb
Fix typo
2022-08-06 15:03:10 +02:00
b16f57cb0d
Minor stylistic fixes
2022-08-06 14:55:46 +02:00
c172ce1011
improve flexibility of user defined prediction dataframe
2022-08-06 13:51:19 +02:00
07763d0d4f
add classifier, improve model naming scheme
2022-08-06 08:33:55 +02:00
a42a060ab5
fix DB once and for all. Make DBSCAN more efficient and robust.
2022-08-05 21:29:03 +02:00
db1d367941
fix bug associated to fit_live_predictions_candles
2022-08-05 13:46:20 +02:00
51a6b4289f
improve DBSCAN performance for subsequent trainings
2022-08-04 17:41:58 +02:00
29225e4baf
add DBSCAN outlier detection feature, add supporting documentation
2022-08-04 12:15:16 +02:00
eae82d0222
fix bug with database url during backtesting. comment out example trade db analysis.
2022-08-03 16:17:57 +02:00
95d3009a95
give user ability to analyze live trade dataframe inside custom prediction model. Add documentation to explain new functionality
2022-08-02 20:14:02 +02:00
f22b140782
fix backtesting bug, undo move of label stat calc, fix example strat exit logic
2022-07-29 17:27:35 +02:00
c84d54b35e
Fix typing issue, avoid using .get() when unnecessary, convert to fstrings
2022-07-29 08:12:50 +02:00
efbd83c56d
Small type and typo fixes in freqai_interface
2022-07-28 07:24:30 +02:00
a2a0d35a24
Update missing typehints
2022-07-28 07:07:40 +02:00
324e54c015
fix possible memory leak associated with Catboost Pool object
2022-07-26 17:29:29 +02:00
ad25a4cb56
reduce number of pair_dict lookups, remove coin_first
2022-07-26 15:58:40 +02:00
e213d0ad55
isolate data_drawer functions from data_kitchen, accommodate tests, add new test
2022-07-26 10:24:14 +02:00
897f18a8c8
ensure proper integer type casting for timestamps. Add check test for backtesting subdaily time periods
2022-07-25 15:07:09 +02:00
7b105532d1
fix mypy error and add test for principal component analysis
2022-07-25 11:46:59 +02:00
c9d46a5237
finish bringing follow_mode up to date
2022-07-25 09:24:40 +02:00
1885deb632
More docstring changes
2022-07-24 16:54:39 +02:00
70b7a254af
Update some areas to use default docstring formatting
2022-07-24 16:51:48 +02:00
f3d46613ee
move prediction denormalization into datakitchen. remove duplicate associated code. avoid normalization/denormalization for string dtypes.
2022-07-23 17:14:33 +02:00
c2d6a0e891
add record of contribution to doc and source
2022-07-23 13:04:06 +02:00
5559e605b8
small PR conversation resolutions
2022-07-22 17:46:14 +02:00
afcb0bec00
clean up obsolete comments, move remove_features_from_df to datakitchen
2022-07-22 12:29:20 +02:00
8033e0bf23
add counter to backtesting log so users know how many more pairs and how many more models will need to be trained
2022-07-21 13:22:12 +02:00
e694ea1cfd
make sure backtesting gets the populated indicators with slimmed down user strat
2022-07-21 12:48:09 +02:00
8f86b0deaa
*breaking change* simplify user strat by consolidating feature loops into backend
2022-07-21 12:24:22 +02:00
e7337728bf
add separator in folder name just incase an asset ends in an integer
2022-07-21 11:25:28 +02:00
921a7ef216
add requirements-freqai.txt to builds
2022-07-20 15:51:25 +02:00
d43c146676
add more tests for datakitchen functionalities, add regression tests for freqai_interface train/backtest
2022-07-20 12:56:46 +02:00
ef409dd345
Add ground work for TensorFlow models, add protections from common mistakes
2022-07-12 18:09:17 +02:00
8ce6b18318
start collecting indefinite history of predictions. Allow user to generate statistics on these predictions. Direct FreqAI to save these to disk and reload them if available.
2022-07-11 22:01:48 +02:00
607455919e
Change config parameter names to improve clarity and consistency throughout the code (!!breaking change, please check discord support channel for migration instructions or review templates/FreqaiExampleStrategy.py config_examples/config_freqai_futures.example.json file changes!!)
2022-07-10 12:35:44 +02:00
d9acdc9767
remove excess, increase no model warning clarity
2022-07-06 18:20:21 +02:00
4cac67fd66
Catch infrequent issue associated with grabbing first candle
2022-07-05 12:43:33 +02:00
bd3a6ba2fe
update backtesting to handle new output framework
2022-07-03 17:34:44 +02:00
ffb39a5029
black formatting on freqai files
2022-07-03 10:59:38 +02:00
106131ff0f
Rehaul organization of return values
2022-07-02 18:09:38 +02:00
93e1410ed9
first step toward cleaning output and enabling multimodel training per pair
2022-07-01 14:00:30 +02:00
6c7d02cb18
expose nu in the SVM outlier detection via svm_nu in config
2022-06-28 15:12:25 +02:00
7dfbd432d1
fix config saving bug, assign priorities to pairs in scanning, sleep the scanning loop to reduce CPU usage
2022-06-27 11:35:33 +02:00
68bafa9517
archive config to the model folder, filter out features before returning dataframe to strategy (to alleviate frequi issues)`
2022-06-26 23:03:48 +02:00
b35c64b6c0
fix bug in backtest, typo in example strat
2022-06-19 16:41:09 +02:00
3599d18ff6
fix bug in follow_mode, thanks @blood4rc
2022-06-18 12:05:28 +02:00
6da7a98857
add docstrings to new functions, remove superceded code
2022-06-17 16:16:23 +02:00
f631ae911b
add model expiration feature, fix bug in DI return values
2022-06-17 14:55:40 +02:00
0b0688a91e
ensure scanning purges models
2022-06-16 16:12:38 +02:00
c5de0c49e4
first functional scanning commit
2022-06-16 00:24:18 +02:00
4d472a0ea1
merging datarehaul into scanning branch
2022-06-16 00:22:49 +02:00
66800c7a45
ensure newest candles are always appended
2022-06-07 20:24:23 +02:00
15d049cffe
detect if upper tf candles are new or not, append if so. Correct the epoch for candle update check
2022-06-07 19:49:20 +02:00
4b26b6aaec
add lock to any historic data access
2022-06-07 00:54:18 +02:00
d6b8801f41
fix follower bug
2022-06-05 04:40:58 +02:00
16b4a5b71f
rehaul of backend data management - increasing performance by holding history in memory, reducing load on the ratelimit by only pinging exchange once per candle. Improve code readability.
2022-06-03 15:19:46 +02:00
ace951bf7e
another queue bug fix for fresh starts
2022-06-02 13:28:06 +02:00
833d25bda0
Save data after queue reorg
2022-06-01 08:05:38 +02:00
0b0dd8dd80
Force high priority pair to be trained before anything else.
2022-06-01 07:55:05 +02:00
7523ed825e
automatically detect maximum required data based on user fed indicators (to avoid NaNs in dataset for rolling indicators), add new config parameter for backtesting to let users increase their startup_candles to accommodate high timeframe indicators, add docs to explain all. Add new feature for automatic indicator duplication according to user defined intervals (exhibited in example strat and configs now).
2022-05-31 18:42:27 +02:00
9b3b08a2bb
let follower purge old model files
2022-05-31 15:37:38 +02:00
bac4ced382
Ensure follower predictions are persistent and uniquely stored
2022-05-31 14:35:04 +02:00
70adf55643
Automatically detect and change follower data_path to accommodate remote systems
2022-05-31 12:35:09 +02:00
0306f5ca13
Add autopurge feature so that FreqAI cleans up after itself when it no longer needs old models on disk
2022-05-31 11:58:21 +02:00
45f4f0f603
ensure follower sends back null arrays in case leader hasnt created a model file yet
2022-05-31 01:48:48 +02:00
606f18e5c1
Add follow_mode feature so that secondary bots can be launched with the same identifier and load models trained by the leader
2022-05-30 21:35:48 +02:00
5b4c649d43
detect variable sized dataframes coming from strat, adjust our stored/returned data accordingly
2022-05-30 13:55:46 +02:00
e229902381
fix bug in previous commit
2022-05-30 12:48:22 +02:00
a20651efd8
Increase performance by only predicting on most recent candle instead of full strat provided dataframe. Collect predictions and store them so that we can feed true predictions back to strategy (so that frequi isnt updating historic predictions based on newly trained models).
2022-05-30 11:37:05 +02:00
d59eac3321
revert a79032b
2022-05-29 21:33:38 +02:00
a79032bf75
fixing bug in training queue
2022-05-29 20:19:32 +02:00
3f72263278
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
2022-05-29 20:02:43 +02:00
cc6cae47ec
allow pairs deeper in the queue to get trained if the higher priority pairs dont need training
2022-05-29 19:49:43 +02:00
4eb4753e20
allow subdaily retraining for backtesting
2022-05-29 17:44:35 +02:00
ce365eb9e3
improve example strat so that it has dynamic buy and sell logic according to original prediction
2022-05-29 14:45:46 +02:00
4eb29c8810
Dont reset pair priority if it doesnt successfully train
2022-05-28 18:34:26 +02:00
Wagner Costa Santos
paranoidandy
Matthias
robcaulk
initrv
Timothy Pogue