Commit Graph

9536 Commits

Author SHA1 Message Date
robcaulk
78d01810ed reduce code redundancy, ensure live always gets the latest data 2022-09-05 00:12:00 +02:00
Wagner Costa Santos
d43ed186fc fix issue with freqai backtesting at slice dataframe 2022-09-05 00:12:00 +02:00
Robert Caulk
956ea43e55
Merge pull request #7340 from wizrds/sigint-freqai
Support SIGINT in FreqAI
2022-09-04 16:43:36 +02:00
robcaulk
ec76214d02 backup historical predictions pickle and load the backup in case of corruption 2022-09-04 15:56:07 +02:00
robcaulk
69b3fcfd32 Merge branch 'develop' into dev-merge-rl 2022-09-04 11:23:25 +02:00
robcaulk
27dce20b29 fix bug in Base4ActionRLEnv, improve example strats 2022-09-04 11:21:54 +02:00
Timothy Pogue
3b5e5fc57b fix method name in dummy class 2022-09-03 14:10:23 -06:00
Robert Caulk
8545d74378
Merge pull request #7331 from th0rntwig/pca
Normalise PCA space
2022-09-03 21:49:54 +02:00
Timothy Pogue
dae3b3d86a support shutting down freqai 2022-09-03 13:24:14 -06:00
Matthias
16573b19e3 Fix migration syntax error 2022-09-03 19:51:44 +02:00
robcaulk
5cfb4154eb revert all changes in normalize_data() 2022-09-03 19:48:30 +02:00
Robert Caulk
63514b0443
Merge pull request #7325 from wagnercosta/develop
Improve Freqai backtesting performance
2022-09-03 19:28:04 +02:00
robcaulk
c21808ff98 remove metadata redundancy, fix pca bug 2022-09-03 16:54:30 +02:00
robcaulk
c9be66b5b6 increase test coverage for dk, improve function naming, extra cleaning 2022-09-03 15:52:29 +02:00
Matthias
ed4cc18cdd Migration to check order funding fee 2022-09-03 15:19:40 +02:00
Matthias
0f483ee31f Use "since last order" approach for live as well. 2022-09-03 15:01:42 +02:00
Matthias
0c6a02687a Don't calculate funding fees if we're not going to use them. 2022-09-03 15:01:42 +02:00
Matthias
df50b1928d Fix funding fee calculation for backtesting 2022-09-03 15:01:42 +02:00
Matthias
80b5f035ab Remove typo in log message 2022-09-03 15:01:28 +02:00
robcaulk
599c1c79fb reorganized backtest utilities, test new functionality, improve/update doc 2022-09-03 14:00:01 +02:00
Wagner Costa Santos
af5460cebf Add option to keep models only in memory for backtest 2022-09-02 22:01:53 -03:00
Matthias
b26126cb57 Don't use ticker['symbol'] but use "pair" instead
closes #7262
2022-09-02 20:09:30 +02:00
Matthias
b53791fef2 Futures volumepairlist to account for contract size 2022-09-02 07:11:32 +02:00
th0rntwig
11b2bc269e Added missing s 2022-09-01 22:37:32 +02:00
th0rntwig
3f8400df10 Normalise PCA space 2022-09-01 21:51:33 +02:00
Matthias
11fbfd3402 Remove unnecessary assignment 2022-09-01 19:39:20 +02:00
Wagner Costa Santos
d6e115178a refactoring freqai backtesting - remove duplicate code 2022-09-01 07:09:23 -03:00
Matthias
f3c73189d5 Remove pointless default on wallet_balance argument 2022-09-01 06:49:51 +02:00
Matthias
ba2eb7cf0f Fix BNB fee bug when selling
thanks @epigramx, for reporting and for the detailed data.
2022-09-01 06:42:51 +02:00
Wagner Costa Santos
44d3a9140d Merge branch 'develop' of https://github.com/wagnercosta/freqtrade into develop 2022-08-31 15:37:06 -03:00
Wagner Costa Santos
7bed0450d2 pr review - refactoring backtesting freqai 2022-08-31 15:36:29 -03:00
wagnercosta
3d4497467c
Merge branch 'freqtrade:develop' into develop 2022-08-31 14:47:06 -03:00
robcaulk
240b529533 fix tensorboard path so that users can track all historical models 2022-08-31 16:50:39 +02:00
Richard Jozsa
2493e0c8a5
Unnecessary lines in Base4, and changes for box space, to fit better for our needs (#7324) 2022-08-31 16:37:02 +02:00
Wagner Costa Santos
df51da22ee refactoring freqai backtesting 2022-08-31 11:23:48 -03:00
Matthias
57ff6f8ac5 Init timerange object properly 2022-08-31 10:28:31 +00:00
Matthias
13ccd940d5 Remove startup_candle_count from freqai sample config to avoid confusion 2022-08-31 10:27:08 +00:00
Matthias
7ba4fda5d7 Implement PR feedback 2022-08-31 10:26:47 +00:00
Matthias
a88ffd2c9d Merge branch 'develop' into move_datadownload 2022-08-31 10:23:45 +00:00
Matthias
c9aa09ec89 Simplify base fee handling 2022-08-30 20:46:06 +02:00
robcaulk
7e8e29e42d use continuous value for inlier_metric 2022-08-30 20:41:37 +02:00
robcaulk
7f52908e87 ensure the lost points are prepended for FreqUI 2022-08-30 18:55:58 +02:00
robcaulk
a58dd0bbf9 add noise feature, improve docstrings 2022-08-30 18:26:24 +02:00
robcaulk
b11742a4c5 integrate inlier metric function 2022-08-30 18:26:24 +02:00
th0rntwig
d3cb211283 Add inlier metric computation 2022-08-30 18:26:24 +02:00
th0rntwig
50e2808667
Fix bug in DI (#7321) 2022-08-30 12:54:39 +02:00
robcaulk
62c0a174c8 allow users to properly reverse train-test data ordering 2022-08-29 11:04:58 +02:00
Matthias
27a9f98d5f Simplify liquidation price structure, improve test cov 2022-08-29 07:04:48 +02:00
Matthias
226fa5d93c Simplify liquidation price calling structure 2022-08-29 07:04:48 +02:00
Matthias
4def3678b7
Merge pull request #7298 from freqtrade/tif_align
align TimeInForce to ccxt usage
2022-08-29 06:36:51 +02:00
Matthias
8be8a12cc4
Merge pull request #7260 from JohanVlugt/develop
Example FreqAI hybrid strategy
2022-08-29 06:34:45 +02:00
Matthias
40c00d2d8f Version bump dev version to 2022.9 2022-08-29 06:34:20 +02:00
robcaulk
7766350c15 refactor environment inheritence tree to accommodate flexible action types/counts. fix bug in train profit handling 2022-08-28 19:21:57 +02:00
Robert Caulk
39a739eadb
Merge pull request #7296 from th0rntwig/dbscan
Improve MinPts calculation in DBSCAN, add outlier protection, and add data_kitchen tests
2022-08-28 14:37:47 +02:00
robcaulk
fcb5d1cb5a remove debugging flag 2022-08-28 13:01:39 +02:00
robcaulk
dd628eb525 add tests for outlier detection and removal functions 2022-08-28 12:56:39 +02:00
robcaulk
1e41c773a0 fix outlier protection 2022-08-28 12:11:29 +02:00
smarmau
ff3a4995c1 remove unnecessary code 2022-08-28 11:45:20 +02:00
Matthias
b9f35cadb3 add /stopentry alias for /stopbuy 2022-08-28 11:37:22 +02:00
smarmau
005594c29c simplify hybrid template 2022-08-28 11:29:48 +02:00
th0rntwig
71f7d68783 Fixed mypy error 2022-08-27 12:44:55 +02:00
Matthias
c61b986c3d FTX - support time_in_force (and PO ordertype)
closes #7175
2022-08-27 10:30:38 +02:00
Matthias
104a73025d Uppercase TimeInForce (align with ccxt) 2022-08-27 10:30:06 +02:00
Matthias
6686489c06
Merge pull request #7258 from freqtrade/feat/hyp_optinal_indicator
Add flag to move hyperopt populate_indicators to epoch
2022-08-27 09:21:16 +02:00
Matthias
2b70c3d0c0 support price callback for partial exits in bt
This will align results to how live works.
closes #7292
2022-08-27 08:50:09 +02:00
Matthias
9204f01312 Don't lock pairs on partial exit 2022-08-27 07:23:02 +02:00
elintornquist
86c5ac44e4 Add outlier percentage check 2022-08-26 23:05:07 +02:00
Matthias
01126c43f7 Fix liquidation price tier calculation
closes #7294
2022-08-26 20:14:24 +02:00
Matthias
753d1b2aad Update leverage tier terminology to be clear and aligned with ccxt 2022-08-26 19:34:51 +02:00
elintornquist
b2d664c63c Change MinPts calculation 2022-08-26 18:57:27 +02:00
robcaulk
bb3523f383 download data homogeneously across timeframes 2022-08-26 18:51:42 +02:00
robcaulk
e7261cf515 add freqai utils.py file 2022-08-26 15:30:28 +02:00
robcaulk
65b552e310 make docs reflect reality, move download_all_data to new utils.py file, automatic startup_candle detection 2022-08-26 15:30:01 +02:00
robcaulk
4b7e640f31 reduce code duplication, optimize auto data download per tf 2022-08-26 13:56:44 +02:00
robcaulk
baa4f8e3d0 remove Base3ActionEnv in favor of Base4Action 2022-08-26 11:04:25 +02:00
richardjozsa
d31926efdf Added Base4Action 2022-08-26 11:04:25 +02:00
Matthias
53d46a0385 align max_entry_position_adjustment behavior of backtesting to live
closes #7293
2022-08-25 20:36:17 +02:00
robcaulk
3199eb453b reduce code for base use-case, ensure multiproc inherits custom env, add ability to limit ram use. 2022-08-25 19:05:51 +02:00
Matthias
1fd223c815
Update --prepend help string
closes #7290
2022-08-25 17:03:41 +02:00
robcaulk
05ccebf9a1 automate eval freq in multiproc 2022-08-25 12:29:48 +02:00
robcaulk
94cfc8e63f fix multiproc callback, add continual learning to multiproc, fix totalprofit bug in env, set eval_freq automatically, improve default reward 2022-08-25 11:46:18 +02:00
Matthias
f2a356a80c Fix some imports 2022-08-25 07:08:58 +02:00
Matthias
6636f17e0f Simplify usage of amount_to_contract precision 2022-08-25 07:08:22 +02:00
Matthias
32faad9333 Fix backtest calculation problem with DCA
closes #7287
2022-08-24 20:36:08 +02:00
robcaulk
d1bee29b1e improve default reward, fix bugs in environment 2022-08-24 18:32:40 +02:00
robcaulk
a61821e1c6 remove monitor log 2022-08-24 16:33:13 +02:00
robcaulk
bd870e2331 fix monitor bug, set default values in case user doesnt set params 2022-08-24 16:32:14 +02:00
robcaulk
c0cee5df07 add continual retraining feature, handly mypy typing reqs, improve docstrings 2022-08-24 13:00:55 +02:00
robcaulk
b708134c1a switch multiproc thread count to rl_config definition 2022-08-24 13:00:55 +02:00
robcaulk
b26ed7dea4 fix generic reward, add time duration to reward 2022-08-24 13:00:55 +02:00
robcaulk
280a1dc3f8 add live rate, add trade duration 2022-08-24 13:00:55 +02:00
robcaulk
f9a49744e6 add strategy to the freqai object 2022-08-24 13:00:55 +02:00
richardjozsa
a2a4bc05db Fix the state profit calculation logic 2022-08-24 13:00:55 +02:00
robcaulk
29f0e01c4a expose environment reward parameters to the user config 2022-08-24 13:00:55 +02:00
robcaulk
d88a0dbf82 add sb3_contrib models to the available agents. include sb3_contrib in requirements. 2022-08-24 13:00:55 +02:00
robcaulk
8b3a8234ac fix env bug, allow example strat to short 2022-08-24 13:00:55 +02:00
mrzdev
8cd4daad0a Feat/freqai rl dev (#7)
* access trades through get_trades_proxy method to allow backtesting
2022-08-24 13:00:55 +02:00
robcaulk
3eb897c2f8 reuse callback, allow user to acces all stable_baselines3 agents via config 2022-08-24 13:00:55 +02:00
robcaulk
4b9499e321 improve nomenclature and fix short exit bug 2022-08-24 13:00:55 +02:00
sonnhfit
4baa36bdcf fix persist a single training environment for PPO 2022-08-24 13:00:55 +02:00
robcaulk
f95602f6bd persist a single training environment. 2022-08-24 13:00:55 +02:00
robcaulk
5d4e5e69fe reinforce training with state info, reinforce prediction with state info, restructure config to accommodate all parameters from any user imported model type. Set 5Act to default env on TDQN. Clean example config. 2022-08-24 13:00:55 +02:00
sonnhfit
7962a1439b remove keep low profit 2022-08-24 13:00:55 +02:00
sonnhfit
81b5aa66e8 make env keep current position when low profit 2022-08-24 13:00:55 +02:00
sonnhfit
45218faeb0 fix coding convention 2022-08-24 13:00:55 +02:00
robcaulk
b90da46b1b improve price df handling to enable backtesting 2022-08-24 13:00:55 +02:00
MukavaValkku
2080ff86ed 5ac base fixes in logic 2022-08-24 13:00:55 +02:00
robcaulk
16cec7dfbd fix save/reload functionality for stablebaselines 2022-08-24 13:00:55 +02:00
sonnhfit
0475b7cb18 remove unuse code and fix coding conventions 2022-08-24 13:00:55 +02:00
MukavaValkku
d60a166fbf multiproc TDQN with xtra callbacks 2022-08-24 13:00:55 +02:00
robcaulk
dd382dd370 add monitor to eval env so that multiproc can save best_model 2022-08-24 13:00:55 +02:00
robcaulk
69d542d3e2 match config and strats to upstream freqai 2022-08-24 13:00:55 +02:00
robcaulk
e5df39e891 ensuring best_model is placed in ram and saved to disk and loaded from disk 2022-08-24 13:00:55 +02:00
robcaulk
bf7ceba958 set cpu threads in config 2022-08-24 13:00:55 +02:00
MukavaValkku
57c488a6f1 learning_rate + multicpu changes 2022-08-24 13:00:55 +02:00
robcaulk
acf3484e88 add multiprocessing variant of ReinforcementLearningPPO 2022-08-24 13:00:55 +02:00
MukavaValkku
cf0731095f type fix 2022-08-24 13:00:55 +02:00
MukavaValkku
1c81ec6016 3ac and 5ac example strategies 2022-08-24 13:00:55 +02:00
MukavaValkku
13cd18dc9a PPO policy change + verbose=1 2022-08-24 13:00:55 +02:00
robcaulk
926023935f make base 3ac and base 5ac environments. TDQN defaults to 3AC. 2022-08-24 13:00:55 +02:00
MukavaValkku
096533bcb9 3ac to 5ac 2022-08-24 13:00:55 +02:00
MukavaValkku
718c9d0440 action fix 2022-08-24 13:00:55 +02:00
robcaulk
9c78e6c26f base PPO model only customizes reward for 3AC 2022-08-24 13:00:55 +02:00
robcaulk
6048f60f13 get TDQN working with 5 action environment 2022-08-24 13:00:55 +02:00
robcaulk
d4db5c3281 ensure TDQN class is properly named 2022-08-24 13:00:55 +02:00
robcaulk
91683e1dca restructure RL so that user can customize environment 2022-08-24 13:00:55 +02:00
sonnhfit
ecd1f55abc add rl module 2022-08-24 13:00:55 +02:00
MukavaValkku
9b895500b3 initial commit - new dev branch 2022-08-24 13:00:55 +02:00
MukavaValkku
cd3fe44424 callback function and TDQN model added 2022-08-24 13:00:55 +02:00
MukavaValkku
01232e9a1f callback function and TDQN model added 2022-08-24 13:00:55 +02:00
MukavaValkku
8eeaab2746 add reward function 2022-08-24 13:00:55 +02:00
MukavaValkku
ec813434f5 ReinforcementLearningModel 2022-08-24 13:00:55 +02:00
MukavaValkku
2f4d73eb06 Revert "ReinforcementLearningModel"
This reverts commit 4d8dfe1ff1daa47276eda77118ddf39c13512a85.
2022-08-24 13:00:55 +02:00
MukavaValkku
c1e7db3130 ReinforcementLearningModel 2022-08-24 13:00:55 +02:00
robcaulk
05ed1b544f Working base for reinforcement learning model 2022-08-24 13:00:40 +02:00
Matthias
a6d78a8615 initialize Since parameter properly
closes #7285
2022-08-23 06:43:04 +02:00
Matthias
fe7108ae75 Convert amount to contracts before comparing for close
closes #7279
2022-08-23 06:37:38 +02:00
Matthias
78b161e14c add contract_size to database 2022-08-23 06:37:38 +02:00
Matthias
6036018f35 Extract contracts_to_amount and amount_to_contracts to standalone functions 2022-08-23 06:37:38 +02:00
Matthias
5f38a574ce Add okx broker id 2022-08-23 06:37:38 +02:00
th0rntwig
5ce1c69803
Improve DBSCAN epsilon identification (#7269)
* Improve DBSCAN epsilon identification
2022-08-22 19:57:20 +02:00
robcaulk
ac42c0153d deprecate indicator_max_period_candles, automatically compute startup candles for FreqAI backtesting. 2022-08-22 18:19:07 +02:00
robcaulk
96d8882f1e Plug mem leak, add training timer 2022-08-22 13:30:30 +02:00
Matthias
f55d5ffd8c Don't fail when --strategy-path is not a valid directory.
closes #7264
2022-08-22 09:20:14 +00:00
Matthias
015be770c3 ccxt now defaults to base volume for all markets 2022-08-22 06:42:14 +02:00
Matthias
f6d832c6d9 Add get_option to expose ft_has via method 2022-08-21 17:51:46 +02:00
Matthias
87a3115073 Add get_open_trade_count() to simplify getting open trade count. 2022-08-21 17:08:27 +02:00
Matthias
6189aa817c Fix HybridExample formatting 2022-08-20 19:50:18 +02:00
robcaulk
64b0834437 add credit in docstring 2022-08-20 17:04:38 +02:00
robcaulk
90c03178b1 provide user directions, clean up strategy, remove unnecessary code. 2022-08-20 17:02:18 +02:00
Matthias
cdd4745693
Merge pull request #7263 from freqtrade/okx_cache_tiers
Okx cache tiers
2022-08-20 15:18:13 +02:00
Matthias
5b3f031590 Use hyperopt safe amount precision method 2022-08-20 14:13:15 +02:00
Matthias
738e95b875 Add tests for leverage tiers caching 2022-08-20 13:54:54 +02:00
Matthias
b6e8b9df35 Use cached leverage tiers 2022-08-20 13:01:58 +02:00
Matthias
52ec0d1046 Update binance Leverage tiers 2022-08-20 11:53:15 +02:00
Matthias
0da0600836 Have backtesting respect tradable size
closes #7161
2022-08-20 11:41:11 +02:00
Matthias
54ddc1a4c2 Add --tradingmode alias 2022-08-20 11:24:20 +02:00
Matthias
aa3da092a0 Dont' use classProperty - that's not supported on 3.8 2022-08-20 10:55:52 +02:00
Matthias
01d45ed12e
Merge pull request #7257 from freqtrade/feat/list-pair-time
Get min/max data in list-data command
2022-08-20 08:16:52 +02:00
Matthias
7b8b73e651
Merge pull request #7243 from lolongcovas/newbranch_test
Improve PCA and pairwise distance calcs
2022-08-20 08:13:40 +02:00
Johan van der Vlugt
b44bd0171c
Example Classifier strat 2022-08-19 19:10:37 +02:00
Matthias
b9d48c3278 use numbers in HyperoptState properly ... 2022-08-19 15:40:06 +02:00
Matthias
bc359675a2 Add --analyze-per-epoch - moving populate_analysis to the epoch process 2022-08-19 15:19:43 +02:00
Matthias
09f8904545 Extract analysis to separate method 2022-08-19 15:12:55 +02:00
Matthias
08ef5ad2d8 Add HyperoptState enum and container class 2022-08-19 15:11:43 +02:00
Matthias
1c6f966579 Hyperopt: simplify parameter "can_optimize" handling 2022-08-19 15:03:03 +02:00
Matthias
16af10a5bc Update notebook sample with simplified datadir configuration
closes #7252
2022-08-19 14:05:27 +02:00
Matthias
b7553d20d4 Get min/max data in list-data command 2022-08-19 13:45:55 +02:00
Matthias
7d84ef2e2c Remove unused imports 2022-08-19 13:45:10 +02:00
longyu
521381ebf0 undo example strategy newline 2022-08-19 12:40:03 +02:00
Matthias
b420614d65 Reduce code duplication in datahandlers 2022-08-19 09:33:07 +02:00
Matthias
975bf8fe88 Update Docstring to match actual return values 2022-08-19 09:23:53 +02:00
Matthias
42eb508515 Attempt fix of #7184 2022-08-19 07:09:46 +02:00
Matthias
76a3e97e05 Add migrations end message
closes #7251
2022-08-19 06:39:51 +02:00
Matthias
70a77ba3d9 Check for "last" availability in PrecisionFilter
closes #7250
2022-08-18 20:07:50 +02:00
longyu
1fada53ddd remove newline 2022-08-18 19:40:00 +02:00
Matthias
85b43ec1a1 Remove double-check for "isolated margin" 2022-08-18 15:23:58 +02:00
Matthias
fde469a253 Remove unnecessary check 2022-08-18 14:53:44 +02:00
Matthias
075e9b8526 Log Exchange responses for set_leverage 2022-08-18 09:52:03 +02:00
Matthias
46e8d9a5e4 Reduce verbosity when whitelist is empty 2022-08-18 09:09:37 +02:00
Matthias
0461a89348 Fix test failures 2022-08-18 07:20:49 +02:00
longyu
f70b0bab80 remove line 2022-08-17 23:49:20 +02:00
Matthias
66910bfe63 Don't fail if mark candles are missing
closes #7239
2022-08-17 20:01:57 +02:00
robcaulk
88dd9920ea sort imports for isort 2022-08-17 16:38:09 +02:00
robcaulk
5155afb4e7 clean up code remnants 2022-08-17 15:22:48 +02:00
robcaulk
0c34104e45 extract download-data from freqai to prepare for future async changes 2022-08-17 15:18:44 +02:00
longyu
9c38c27eed ignore sample itself distance for avg_mean_dist computation 2022-08-17 15:09:57 +02:00
longyu
72c34291e3 newline 2022-08-17 15:09:10 +02:00
Matthias
3983368228 gateio futures is showing base currency in volume! 2022-08-17 14:51:48 +02:00
Matthias
83ca168bb8
Merge pull request #7216 from freqtrade/precise_calcs
Precise calcs
2022-08-17 14:32:02 +02:00
Matthias
c615e1bc62 Avoid loop error during ccxt tests 2022-08-17 14:31:40 +02:00
Matthias
e7902bffa0 Remove checks for dataprovider existance - it's available in all modes. 2022-08-17 10:57:25 +02:00
Matthias
8d182768f9 stoploss should also use trimmed prices 2022-08-17 09:57:11 +02:00
Matthias
0b0e7eaf96 Mutex TTL Cache accesses which can be accessed by multiple threads
Apparently, cachetools is (intentionally) not threadsafe
when using the Caches directly.
It's therefore recommended to wrap these with an explicit lock to avoid
problems.

source: https://github.com/tkem/cachetools/issues/245

closes #7215
2022-08-16 19:48:21 +02:00
Matthias
24690c1918 Don't convert open_rate to precision
this may cause more problems than it solves.
2022-08-16 18:32:00 +02:00
Matthias
3b44dc52e1 Minor corrections 2022-08-16 18:10:48 +02:00
Matthias
ea6bc47d7a gateio default configs should specify unknown_fee_rate 2022-08-16 17:26:53 +02:00
Matthias
a1e4fbf313 Run price_to_precision for dry-run orders 2022-08-16 17:23:49 +02:00
Matthias
1ac81aa316 Show message if fee update failed due to missing
closes #7234
2022-08-16 17:09:23 +02:00
robcaulk
4c0fda400f fix input shape warning for LGBMClassifier, add sample_weights/eval_weights 2022-08-16 11:41:53 +02:00