Commit Graph

382 Commits

Author SHA1 Message Date
hroff-1902 e2e6b940a3 copy=False does not make the changes inline anyway, so not needed 2020-02-22 19:54:19 +03:00
hroff-1902 c651e0ac82 Fix #2948 2020-02-22 19:46:40 +03:00
Matthias 3186add87b Use explicit column list for float parsing 2020-02-22 14:46:54 +01:00
Matthias 7ecc56fa44 Load ohlcv data as float 2020-02-22 13:10:41 +01:00
Matthias d65a06947d Merge branch 'develop' into data_handler 2020-02-09 15:16:43 +01:00
hroff-1902 f3d500085c Add some type hints 2020-02-02 07:00:40 +03:00
Matthias 41945138ac Converting pairs from filename to pair corrected 2020-01-05 13:35:36 +01:00
Matthias 4eaaec9d1a Implement pair_to_filename to datahandler
includes tests - taken from #2744 and modified to adapt to new structure
2020-01-05 10:36:08 +01:00
Matthias f82c4346b6 data conversion, not data conversation
* we're not talking to the data yet ...
2020-01-05 09:55:02 +01:00
Matthias 2409261cb7 Merge branch 'develop' into data_handler 2020-01-04 11:36:27 +01:00
Matthias 2c8e8d8ef6 Align columns for btanalysis loading 2020-01-02 09:51:47 +01:00
Matthias 699c0d6bc3 Merge branch 'develop' into data_handler 2019-12-30 19:40:43 +01:00
Matthias 1ffda29fd2 Adjust improts to new exception location 2019-12-30 15:02:17 +01:00
Matthias 814cc20c6b Remove potential circular import 2019-12-28 19:58:41 +01:00
Matthias f4a532ef6d Pass format to load_data 2019-12-28 14:57:39 +01:00
Matthias ae1b28aab7 Remove get_datahandlerclass from package exposes 2019-12-28 14:32:11 +01:00
Matthias 66d18575a7 Implement abstract interface 2019-12-28 11:10:31 +01:00
Matthias 9e4fc00a0f Add test for convert_ohlcv 2019-12-28 11:03:06 +01:00
Matthias 28787a001c Move convert functions to convert module 2019-12-28 11:02:34 +01:00
Matthias 6860491189 Rename datahandler module to history module
Also move previous history.py into this module - so everything is
bundled
2019-12-28 11:01:42 +01:00
Matthias 9c5b94adf5 Pass data_format to methods 2019-12-28 11:01:42 +01:00
Matthias d65c1eea7a Add some tests for datahandler 2019-12-28 11:01:42 +01:00
Matthias 8a030e7fc0 Use exists instead of is_file 2019-12-28 11:01:42 +01:00
Matthias a3144cb2f0 remove trim_tickerlist 2019-12-28 11:01:42 +01:00
Matthias d1b52809ac Cleanup history 2019-12-28 11:01:41 +01:00
Matthias d06777b8ce Remove old "load_cached_data" method 2019-12-28 11:01:41 +01:00
Matthias 7a6476c9ba Update tests 2019-12-28 11:01:41 +01:00
Matthias df085a6f15 Fix small bug and test 2019-12-28 11:01:41 +01:00
Matthias c648d973c1 Implement new "load_data_for_updating" method based on dataframes 2019-12-28 11:01:41 +01:00
Matthias ec8fb5f308 Make no-data warning optional 2019-12-28 11:01:41 +01:00
Matthias b83487a70d Extract default dataframe columns to constant 2019-12-28 11:01:41 +01:00
Matthias 91c70a0e9c Change to use ohlcv_purge 2019-12-28 11:01:41 +01:00
Matthias 37c5b68987 Move dataframe validation to abstract class 2019-12-28 11:01:41 +01:00
Matthias e861f05b75 Move dataframe trim to within jsondatahandler 2019-12-28 11:01:41 +01:00
Matthias 552c93abf0 Improve some docstrings 2019-12-28 11:01:41 +01:00
Matthias b7c1d55491 Modify tests to point to datahandlers 2019-12-28 11:01:41 +01:00
Matthias 9876d126ca Use handler for trades 2019-12-28 11:01:41 +01:00
Matthias 9547d47ae2 Initialize datahandlers 2019-12-28 11:01:41 +01:00
Matthias 5fca17d7e1 Allow initializing handler-class just once 2019-12-28 11:01:41 +01:00
Matthias 416517b0c9 Move trim_dataframe from history to converter 2019-12-28 11:01:41 +01:00
Matthias 9d8ea2f13b Replace calls to load_tickerdata_file with DataHandler calls 2019-12-28 11:01:41 +01:00
Matthias 88fa7fc24c Simplify validate dataframe method 2019-12-28 11:01:41 +01:00
Matthias 53ee636fa0 Check if file exists before loading 2019-12-28 11:01:41 +01:00
Matthias 873f5dbe6b Revrite validate_pairdata to work with pandas 2019-12-28 11:01:41 +01:00
Matthias db520a09ee Trim dataframe, not tickerlist 2019-12-28 11:01:41 +01:00
Matthias 866908d2ca Load and save using pandas internal function 2019-12-28 11:01:41 +01:00
Matthias d9e7d64f33 Split parse_ticker_dataframe some logic to clean_ohlcv_dataframe. 2019-12-28 11:01:41 +01:00
Matthias 1b90ec58b9 Use changed pair-handling for providers 2019-12-28 11:01:41 +01:00
Matthias d923bab828 Remove abstract interface for now 2019-12-28 11:01:41 +01:00
Matthias 48728e2d66 Change DataProvider interface to accept pair per method 2019-12-28 11:01:41 +01:00
Matthias e529a4c261 Fix typehint for get_datahandlerclass 2019-12-28 11:01:41 +01:00
Matthias c6d6dbfdb1 Implement jsondatahandler file store 2019-12-28 11:01:41 +01:00
Matthias abc6b9459a Add ohlcv_store call to convert_ohlcv 2019-12-28 11:01:41 +01:00
Matthias 2a728ee68f fix bug in find-files 2019-12-28 11:01:41 +01:00
Matthias ef0fcb0e0f Make data-finding safe 2019-12-28 11:01:41 +01:00
Matthias e5a61667dd Implement first version of jsondatahandler 2019-12-28 11:00:22 +01:00
Matthias bb8acc61db Convert datadir within config to Path
(it's used as Path all the time!)
2019-12-23 15:11:29 +01:00
hroff-1902 cf4c3642ce Minor improvements in data.history 2019-12-18 01:06:03 +03:00
hroff-1902 021fa1ca1a
Merge pull request #2678 from hroff-1902/dataprovider-history-split-refresh
Dataprovider history: split refresh part
2019-12-18 00:30:47 +03:00
hroff-1902 3a542bce62
Merge pull request #2674 from freqtrade/bt_trade_open_price
Pre-calculate open_trade_price
2019-12-17 21:51:13 +03:00
hroff-1902 1537389617 Remove startup_candles argument in refresh_data 2019-12-17 18:23:31 +03:00
hroff-1902 b2796f99b6 Remove redundant refresh_pair_history 2019-12-17 14:06:21 +03:00
hroff-1902 60f89c8c01 Split refresh from load_data/load_pair_history 2019-12-17 13:43:42 +03:00
Matthias a2964afd42 Rename profit_percent to profit_ratio to be consistent 2019-12-17 08:53:30 +01:00
hroff-1902 0277cd82ea Make mypy happy 2019-12-16 23:25:57 +03:00
hroff-1902 a6fc743d85 Align code in _download_*_history() 2019-12-16 22:12:26 +03:00
hroff-1902 fa968996ed Remove useless check 2019-12-16 22:01:26 +03:00
hroff-1902 4cd45b6535 Rename download_*_history as non-public 2019-12-16 21:57:03 +03:00
hroff-1902 2af9ffa7f2 Align refresh_backtest_ to each other 2019-12-16 21:43:33 +03:00
Matthias 7c7ca1cb90 Remove min (plural) from codebase 2019-12-11 07:12:37 +01:00
Matthias 054484ad73 load_pair_history should not return None, but an empty dataframe if no
data is found
2019-12-04 06:57:44 +01:00
Matthias c8c48156dd Don't load trades twice ... 2019-11-13 20:44:55 +01:00
Matthias 6ac73f7cde Update missed strings 2019-11-13 11:28:26 +01:00
Matthias 1c57a4ac35 more replacements of ticker_interval 2019-11-12 15:13:06 +01:00
Matthias 08aedc18e1 Exchange ticker_interval with timeframe in some more places 2019-11-12 15:13:06 +01:00
Matthias e4bdb92521 Replace some occurances of ticker_interval with timeframe 2019-11-12 15:13:06 +01:00
Matthias da57396d07 Fix UTC handling of timestamp() conversation in fetch_my_trades 2019-11-08 06:55:07 +01:00
hroff-1902 f435384bf0
Merge pull request #2451 from freqtrade/bt_analysis_maxopen
Bt analysis maxopen at any time
2019-11-01 00:13:31 +03:00
hroff-1902 3149c12a14
Merge pull request #2444 from freqtrade/sql_cleanup
Fix scoped_session and add Documentation for strategy
2019-10-31 23:19:30 +03:00
Matthias 5dcf28cafb Reduce frequency of "startup-period" message 2019-10-31 06:57:37 +01:00
Matthias dd408aa5d6 Add analyze_trade_parallelism analysis function 2019-10-30 14:07:23 +01:00
Matthias dac88c6aed extract Find parallel trades per interval 2019-10-30 13:35:55 +01:00
Matthias 26a5800a7f Extract get_trades function 2019-10-30 11:12:49 +01:00
hroff-1902 5254059fe4
Merge pull request #2430 from freqtrade/startup_period_bt
Add Startup period for strategies
2019-10-28 23:33:30 +03:00
Matthias e82460bde6 Fix create_cum_profit to work with trades that don't open on candle
opens
2019-10-28 14:24:12 +01:00
Matthias 61c037f2cf Fix some typos and comment mistakes 2019-10-28 13:05:54 +01:00
Matthias 132a4da7cf Small style fixes and adjusted tests 2019-10-27 10:56:38 +01:00
Matthias 33164ac78e Refactor loading of bt data to backtesting ... 2019-10-27 09:44:56 +01:00
Matthias 704121c197 Move most logic to history 2019-10-27 09:44:56 +01:00
Matthias 9e7e051eb4 add trim-dataframe method 2019-10-27 09:44:56 +01:00
hroff-1902 14755779de
Merge pull request #2391 from freqtrade/plot_trades_in_min
[minor][plot] Plotting trades from database should show correct duration
2019-10-20 20:11:01 +03:00
hroff-1902 4f17511fdc
Merge pull request #2393 from freqtrade/remove_timeframe
Remove non-date based timeframe selection
2019-10-19 22:05:44 +03:00
Matthias 16e10d99b9 Remove timeframe logic for non-date entries 2019-10-19 15:10:48 +02:00
Matthias f41c659cb2 Plotting trades from database should show correct duration 2019-10-19 13:18:52 +02:00
Matthias 13e80e449c cleanup and better docstring 2019-10-14 06:22:10 +02:00
Matthias 3e4617be37 add pandas-based converter-functions 2019-10-13 19:25:16 +02:00
Matthias 2374cda8d0 Cleanup and tests for refresh_backtest_trades 2019-10-13 13:15:22 +02:00
Matthias 0d592f6c55 Refactor trade downloading to handle exceptions only once 2019-10-13 13:15:22 +02:00
Matthias 476adf872a Add conversion from trades to ohlcv at different intervals 2019-10-13 13:15:22 +02:00
Matthias 9584629f50 Rename argument from dl_path to datadir 2019-10-13 13:15:22 +02:00
Matthias 8069cd6689 add refresh_trades_ method 2019-10-13 13:15:22 +02:00
Matthias 19f3669fbd add docstring 2019-10-13 13:15:22 +02:00
Matthias 6e952a0aa8 Capture downloaded data 2019-10-13 13:15:22 +02:00
Matthias 2c0bb71a6e Add download_trades_history() 2019-10-13 13:15:22 +02:00
Matthias d250b67f33 Add load/store trades data 2019-10-13 13:15:22 +02:00
Matthias 42b8241541 use gz to save / load trades data 2019-10-13 13:15:22 +02:00
Matthias 63e87ef85b Add pair_trades_filename 2019-10-13 13:15:22 +02:00
Matthias 5e23cc719d Add tests for orderbook and market in dataprovider 2019-10-10 19:38:01 +02:00
Matthias bba5f54722
Merge pull request #2335 from hroff-1902/dataprovider-market
Allow to use market data in the strategies
2019-10-10 16:54:04 +02:00
hroff-1902 211b9cbe04 Cleanup in data.history 2019-10-06 18:35:09 +03:00
hroff-1902 3ac5b91899 Add market() method to dataprovider 2019-10-03 02:58:45 +03:00
hroff-1902 0268bfdbd4
Minor: fix typo in comment
Minor cosmetics. typo caught.
2019-09-26 02:04:48 +03:00
Matthias 508a35fc20 Update comment as to why certain points have not been removed 2019-09-20 20:20:16 +02:00
Matthias 9cedbc1345 Cleanup history.py and update documentation 2019-09-20 20:16:49 +02:00
Matthias 1cd8ed0c1a Remove --refresh-pairs 2019-09-20 20:02:07 +02:00
hroff-1902 dda513c923 Minor class cosmetics 2019-09-12 12:13:20 +03:00
Jonathan Raviotta a5510d14e9 de-mangling 2019-09-09 06:58:41 +02:00
Matthias 972b8a1726 Remove defaulting to test_data folder when no datadir is present 2019-09-07 21:06:20 +02:00
Matthias 816d942ded Merge branch 'develop' into plot_commands 2019-08-30 20:42:58 +02:00
Matthias 3232251fea Refactor downloading ohlcv from utils to history 2019-08-25 15:01:27 +02:00
Matthias 2ae398913d Fix bug in bt-analysis when multiple trades sell at the same time 2019-08-24 15:11:31 +02:00
Matthias 545e5c5bc6 simplify load_trades call 2019-08-24 15:11:31 +02:00
Matthias e8ee087e9d Merge branch 'develop' into download_module 2019-08-20 06:49:18 +02:00
Matthias 9e8ca8d4bf
Merge pull request #2138 from freqtrade/history_docstrings
Refactorings to history
2019-08-20 06:35:54 +02:00
hroff-1902 407a3bca62 implementation of ohlcv optimized 2019-08-18 13:00:37 +03:00
hroff-1902 310e438706 logging message improved 2019-08-18 12:55:31 +03:00
hroff-1902 8a2a8ab8b5 docstring for ohlcv improved 2019-08-18 12:47:19 +03:00
hroff-1902 84a0f9ea42 get_pair_dataframe helper method added 2019-08-17 12:57:44 +03:00
Matthias 219d0b7fb0 Adjust documentation to removed download-script 2019-08-16 15:27:48 +02:00
Matthias 12677f2d42 Adjust docstring to match functioning of load_cached_data 2019-08-15 20:13:19 +02:00
Matthias a94a89086f Don't forward timerange to load_ticker_file
when loading cached data for updating.
We always want to get all data, not just a fraction (we would end up
overwriting the non-loaded part of the data).
2019-08-15 20:09:00 +02:00
Matthias f3e6bcb20c Avoid using negative indexes 2019-08-15 06:35:50 +02:00
Matthias b2a22f1afb Fix samll errors 2019-08-14 21:39:53 +02:00
Matthias 91d1061c73 Abstract tickerdata storing 2019-08-14 20:49:06 +02:00
Matthias 0ffb184eba Change some docstrings and formatting from history 2019-08-14 20:45:24 +02:00
Matthias 84baef922c Rename get_history to get_historic_ohlcv 2019-08-14 10:14:54 +02:00
Matthias 4d1ce8178c intend if to be clearer 2019-08-04 10:38:37 +02:00
Matthias c4e30862ee load_trades_db should give as many columns as possible 2019-08-03 19:55:54 +02:00
Matthias 76d22bc743 Show correct valueerror message 2019-08-02 09:41:24 +02:00
Matthias a0cecc6c52 Fix test after pandas 0.25.0 update 2019-07-24 06:29:50 +02:00
Matthias 5144e98a82
Merge pull request #2015 from hroff-1902/refactor/config2
Make configuration a module
2019-07-15 19:41:57 +02:00
Matthias 4238ee090d Cleanup some code
after deepcode.ai suggestions
2019-07-14 20:05:28 +02:00
hroff-1902 1bdffcc73b make configuration a sep. module, including arguments 2019-07-12 00:49:23 +03:00
Matthias b3644f7fa0 Fix typo in docstring 2019-07-03 06:26:39 +02:00
Matthias 5a11ffcad8 Add test for add_profit 2019-06-30 10:24:10 +02:00
Matthias 6b387d320e extract combine_tickers to btanalysis 2019-06-30 10:04:43 +02:00
Matthias 8aa327cb8a Add load_trades abstraction (to load trades from either DB or file) 2019-06-29 20:52:23 +02:00
Matthias edd3fc8825 Add test for create_cum_profit 2019-06-29 17:22:47 +02:00
Matthias 044be3b93e Add create_cum_profit column 2019-06-29 16:57:04 +02:00
Matthias 31a2aac627
Merge pull request #1959 from freqtrade/split_btanalysis_load_trades
Split btanalysis load trades
2019-06-24 19:41:56 +02:00
Matthias eba7327058 Merge branch 'develop' into split_btanalysis_load_trades 2019-06-24 07:15:14 +02:00
Matthias a07653a6cc Merge branch 'develop' into fix/validate_dataframe 2019-06-24 06:21:08 +02:00
hroff-1902 116d8e853e typos in docstrings fixed 2019-06-23 23:10:37 +03:00
Matthias 559d5ebd1d Remove combined load-method since it's confusing 2019-06-22 16:20:41 +02:00
Matthias de38aea164 Fix sequence of loading trades 2019-06-22 15:45:20 +02:00
Matthias fc3e3c468c File existence is checked in load_backtest_data 2019-06-16 19:35:21 +02:00
Matthias 1cd8415723 Move extract_trades_of_period to btanlaysis 2019-06-16 19:33:48 +02:00
Matthias c7643e142b Move load_trades to bt_anlaysis 2019-06-16 19:33:48 +02:00
Matthias 4a916125a0 Tests need to pass pair to parse_ticker_dataframe 2019-06-15 13:48:08 +02:00
Matthias 89ff614e1d Add pair as parameter, and warn when fillup was necessary 2019-06-15 13:46:19 +02:00
Matthias cd4cf215e1 Convert validate_backtest_data to take dataframe directly 2019-06-15 13:31:27 +02:00
Matthias 01b5ece642 Log missing data filllup if necessary 2019-06-15 13:31:14 +02:00
Matthias 08105641d9
Merge pull request #1901 from yperfanov/bid_ask_strategy
Bid ask strategy
2019-06-11 11:14:39 +02:00
Matthias ce317b62f9 Add docstrings to load_pair_history 2019-06-09 14:40:45 +02:00
Matthias fdbbefdddd Make drop_incomplete optional 2019-06-09 14:35:58 +02:00
Matthias d7c63347e1 Use kwarg for parse_ticker_dataframe 2019-06-09 13:19:01 +02:00
Yuliyan Perfanov 199426460a implemented DataProvider.orderbook() 2019-06-02 13:25:09 +03:00
Matthias 15984b5c43 Adjust some tests - implement new "live" method to plot_script 2019-05-29 20:25:07 +02:00
Matthias c2f6897d8b Move download of live data to load_data
Avoids code duplication in backtesting and plot_dataframe
2019-05-29 20:20:20 +02:00
Matthias 9225cdea8a Move validate_backtest_data and get_timeframe to histoyr 2019-05-25 16:51:52 +02:00
hroff-1902 98eeec3145 renaming of make_testdata_path reverted 2019-05-22 14:04:58 +03:00
hroff-1902 11dce91281 data/history minor cleanup 2019-05-21 20:49:02 +03:00
hroff-1902 e2b83624a3 data/history cleanup 2019-05-17 19:05:36 +03:00
hroff-1902 ad85ac3dde make --refresh-pairs-cached common option for optimization; added support for it into hyperopt 2019-04-22 21:24:45 +03:00
hroff-1902 9fbe573cca limit usage of ccxt to freqtrade/exchange only 2019-04-09 12:27:35 +03:00
hroff-1902 ebf1126351 cosmetic: rename interval, tick_interval, etc --> ticker_interval 2019-04-07 16:28:53 +03:00
hroff-1902 2aa1b43f01 get rid of TICKER_INTERVAL_MINUTES dict, use ccxt's parse_timeframe() instead 2019-04-04 20:56:40 +03:00
Matthias 9f7f089d8a adjust plot_dataframe to use btanalysis 2019-03-16 17:28:28 +01:00
Matthias e1f48c2b46 Add btanalysis file 2019-03-16 17:28:28 +01:00
hroff-1902 d6c0c107ac
fixed flake
hmm, even in the comments?
2019-02-10 22:23:00 +03:00
hroff-1902 7ed15c64ba
what else? 2019-02-10 22:13:40 +03:00
hroff-1902 4dffb17dd6
fix flake 2019-02-10 22:01:46 +03:00
hroff-1902 14d6cdf9b2
OHLCV should be float for TA-LIB indicators in the strategy
Some exchanges (BitMEX) return integer values for Volume field. And sometimes even for OHLC -- same, on BitMEX, since price decrease is 0.5. TA-LIB functions assume floats and fail with exception.
Of course, this can be fixed (converted) in ccxt for particular exchange, but TA-LIB will still fail for exchanges for that such a conversion is not implemented in ccxt code. So let's make perform this conversion here in order to be sure our strategy will not crash on a new exchange.
2019-02-10 21:52:33 +03:00
Samuel Husso e3ae8d3f69 flake8 3.7.1 fixes 2019-01-31 07:51:03 +02:00
Matthias bfd8609352 Fix comment 2019-01-26 19:16:33 +01:00
Matthias e66808bb02 Add additional pairs to refresh call 2019-01-22 07:07:15 +01:00
Matthias d6cdfc58af Fix mypy hickup after changing list to tuples 2019-01-22 07:07:15 +01:00
Matthias 6525a838d1 Adjust documentation to tuple use 2019-01-22 07:07:15 +01:00
Matthias f0af4601f9 Adopt plot_dataframe to work with --live 2019-01-22 07:07:15 +01:00
Matthias 0aa0b1d4fe Store tickers by pair / ticker_interval 2019-01-22 07:07:15 +01:00
Matthias a206777fe5 Rename refresh_tickers to refresh_latest_ohlcv 2019-01-22 07:05:09 +01:00
Matthias 646e98da55 Always return dataframe 2019-01-22 07:04:19 +01:00
Matthias 2b029b2a86 Only return ohlcv if available (Live and dry modes) 2019-01-22 07:04:19 +01:00
Matthias 35c8d1dcbe Update comment 2019-01-22 07:04:19 +01:00
Matthias 8f3ea3608a some cleanup 2019-01-22 07:04:19 +01:00
Matthias d3a37db79a Provide available pairs 2019-01-22 07:04:19 +01:00
Matthias f034235af4 Tests for RunMode 2019-01-22 07:04:19 +01:00
Matthias 1340b71633 Add RunMode setting to determine bot state 2019-01-22 07:04:19 +01:00
Matthias 84cc4887ce Add copy parameter 2019-01-22 07:04:19 +01:00
Matthias e38c06afe9 Small fixes 2019-01-22 07:04:19 +01:00