Compare commits

...

256 Commits

Author SHA1 Message Date
Matthias
0afd5a7385 Improve stoploss documentation
closes #8492
2023-04-12 18:13:16 +02:00
Matthias
2131205db6 Bump tag length to 255 2023-04-12 07:19:36 +02:00
Matthias
b2b19915e6 Limit enter_tag and exit_reason to their actual field lenght
closes #8486
2023-04-12 07:19:36 +02:00
Matthias
bba6f8e133 Use length constant for tests 2023-04-12 07:19:36 +02:00
Matthias
a6d2233b95 Use constant for custom field lengths 2023-04-11 21:05:14 +02:00
Matthias
9857675a5e Update torch import 2023-04-11 19:38:24 +02:00
Robert Caulk
4ab047dfa7
Merge pull request #8297 from Yinon-Polak/feat/add-pytorch-model-support
Feat/add pytorch model support
2023-04-11 15:40:12 +02:00
Matthias
476ed938f5 Extract custom_tag limit from interface file 2023-04-11 07:26:38 +02:00
Matthias
40ffac9de0 Prevent random test failures by freezing time for certain tests 2023-04-10 19:45:24 +02:00
Matthias
b892d373cd Improve timerange parsing when accepting values from API 2023-04-10 19:45:24 +02:00
Matthias
c3647e49ad
Merge pull request #8484 from freqtrade/dependabot/pip/develop/nbconvert-7.3.1
Bump nbconvert from 7.2.10 to 7.3.1
2023-04-10 19:38:12 +02:00
Matthias
37ed37dc76
Merge pull request #8485 from freqtrade/dependabot/pip/develop/mkdocs-material-9.1.6
Bump mkdocs-material from 9.1.5 to 9.1.6
2023-04-10 19:37:54 +02:00
Matthias
5cb688c112
Merge pull request #8482 from freqtrade/dependabot/pip/develop/websockets-11.0.1
Bump websockets from 11.0 to 11.0.1
2023-04-10 19:37:37 +02:00
Matthias
3e394d0612
Merge pull request #8480 from freqtrade/dependabot/pip/develop/sqlalchemy-2.0.9
Bump sqlalchemy from 2.0.8 to 2.0.9
2023-04-10 19:37:17 +02:00
dependabot[bot]
c4c2298686
Bump mkdocs-material from 9.1.5 to 9.1.6
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.5 to 9.1.6.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.5...9.1.6)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 16:17:10 +00:00
dependabot[bot]
8564dc10b2
Bump nbconvert from 7.2.10 to 7.3.1
Bumps [nbconvert](https://github.com/jupyter/nbconvert) from 7.2.10 to 7.3.1.
- [Release notes](https://github.com/jupyter/nbconvert/releases)
- [Changelog](https://github.com/jupyter/nbconvert/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jupyter/nbconvert/compare/v7.2.10...v7.3.1)

---
updated-dependencies:
- dependency-name: nbconvert
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 16:16:42 +00:00
Matthias
3fb892fcb8
Merge pull request #8483 from freqtrade/dependabot/pip/develop/ruff-0.0.261
Bump ruff from 0.0.260 to 0.0.261
2023-04-10 18:16:24 +02:00
Matthias
9968348324
Merge pull request #8481 from freqtrade/dependabot/pip/develop/ccxt-3.0.59
Bump ccxt from 3.0.58 to 3.0.59
2023-04-10 18:15:44 +02:00
dependabot[bot]
fa293c54f8
Bump websockets from 11.0 to 11.0.1
Bumps [websockets](https://github.com/aaugustin/websockets) from 11.0 to 11.0.1.
- [Release notes](https://github.com/aaugustin/websockets/releases)
- [Commits](https://github.com/aaugustin/websockets/compare/11.0...11.0.1)

---
updated-dependencies:
- dependency-name: websockets
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 15:46:40 +00:00
Matthias
95449ca886
Merge pull request #8478 from freqtrade/dependabot/pip/develop/schedule-1.2.0
Bump schedule from 1.1.0 to 1.2.0
2023-04-10 17:45:44 +02:00
Matthias
70fa4a53cd
pre-commit - bump sqlalchemy 2023-04-10 17:45:23 +02:00
dependabot[bot]
467c63ff01
Bump ruff from 0.0.260 to 0.0.261
Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.260 to 0.0.261.
- [Release notes](https://github.com/charliermarsh/ruff/releases)
- [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md)
- [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.260...v0.0.261)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 15:25:04 +00:00
Matthias
b8a9c200fe
Merge pull request #8479 from freqtrade/dependabot/pip/develop/pre-commit-3.2.2
Bump pre-commit from 3.2.1 to 3.2.2
2023-04-10 17:24:02 +02:00
Matthias
7c10af65a1
Merge pull request #8477 from freqtrade/dependabot/pip/develop/plotly-5.14.1
Bump plotly from 5.14.0 to 5.14.1
2023-04-10 16:44:35 +02:00
Matthias
e2cd23b1d2 Remove deprecated pandas option 2023-04-10 16:33:56 +02:00
dependabot[bot]
0d408d3d43
Bump ccxt from 3.0.58 to 3.0.59
Bumps [ccxt](https://github.com/ccxt/ccxt) from 3.0.58 to 3.0.59.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ccxt/ccxt/compare/3.0.58...3.0.59)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 14:20:19 +00:00
dependabot[bot]
2309197771
Bump sqlalchemy from 2.0.8 to 2.0.9
Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.8 to 2.0.9.
- [Release notes](https://github.com/sqlalchemy/sqlalchemy/releases)
- [Changelog](https://github.com/sqlalchemy/sqlalchemy/blob/main/CHANGES.rst)
- [Commits](https://github.com/sqlalchemy/sqlalchemy/commits)

---
updated-dependencies:
- dependency-name: sqlalchemy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 14:20:14 +00:00
dependabot[bot]
66fe9abce0
Bump pre-commit from 3.2.1 to 3.2.2
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.2.1 to 3.2.2.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.2.1...v3.2.2)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 14:20:03 +00:00
dependabot[bot]
200c18f3e4
Bump schedule from 1.1.0 to 1.2.0
Bumps [schedule](https://github.com/dbader/schedule) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/dbader/schedule/releases)
- [Changelog](https://github.com/dbader/schedule/blob/master/HISTORY.rst)
- [Commits](https://github.com/dbader/schedule/compare/1.1.0...1.2.0)

---
updated-dependencies:
- dependency-name: schedule
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 14:19:59 +00:00
dependabot[bot]
351b5f6e65
Bump plotly from 5.14.0 to 5.14.1
Bumps [plotly](https://github.com/plotly/plotly.py) from 5.14.0 to 5.14.1.
- [Release notes](https://github.com/plotly/plotly.py/releases)
- [Changelog](https://github.com/plotly/plotly.py/blob/master/CHANGELOG.md)
- [Commits](https://github.com/plotly/plotly.py/compare/v5.14.0...v5.14.1)

---
updated-dependencies:
- dependency-name: plotly
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 14:19:56 +00:00
Matthias
605cc20a21
Merge pull request #8459 from freqtrade/feat/kvstore
Add initial bot start time to /profit endpoint
2023-04-10 14:49:01 +02:00
Matthias
f73d2a5371 Ensure bot_start is called when visualizing results 2023-04-10 14:48:02 +02:00
Matthias
485a074674
Merge pull request #8472 from freqtrade/dependabot/pip/develop/types-python-dateutil-2.8.19.12
Bump types-python-dateutil from 2.8.19.11 to 2.8.19.12
2023-04-10 14:42:53 +02:00
Matthias
865cf5232b
Merge pull request #8471 from freqtrade/dependabot/pip/develop/mypy-1.2.0
Bump mypy from 1.1.1 to 1.2.0
2023-04-10 14:42:35 +02:00
Matthias
95a24c3133
Merge pull request #8467 from freqtrade/dependabot/pip/develop/orjson-3.8.10
Bump orjson from 3.8.9 to 3.8.10
2023-04-10 14:41:25 +02:00
Matthias
6833059c70
Merge pull request #8474 from freqtrade/dependabot/github_actions/develop/pypa/gh-action-pypi-publish-1.8.5
Bump pypa/gh-action-pypi-publish from 1.8.4 to 1.8.5
2023-04-10 08:03:55 +02:00
Matthias
3833dc0b78
pre-commit - bump dateutil 2023-04-10 07:54:01 +02:00
Matthias
e0d3c771db
Merge pull request #8465 from freqtrade/dependabot/pip/develop/ccxt-3.0.58
Bump ccxt from 3.0.50 to 3.0.58
2023-04-10 07:53:21 +02:00
dependabot[bot]
5a18ab0784
Bump mypy from 1.1.1 to 1.2.0
Bumps [mypy](https://github.com/python/mypy) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v1.1.1...v1.2.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 05:51:33 +00:00
Matthias
1d66f82b1d
Merge pull request #8469 from freqtrade/dependabot/pip/develop/filelock-3.11.0
Bump filelock from 3.10.6 to 3.11.0
2023-04-10 07:50:48 +02:00
Matthias
2e765fe6d1
Merge pull request #8470 from freqtrade/dependabot/pip/develop/pymdown-extensions-9.11
Bump pymdown-extensions from 9.10 to 9.11
2023-04-10 07:50:25 +02:00
Matthias
21ea02bbcf
Merge pull request #8466 from freqtrade/dependabot/pip/develop/pytest-7.3.0
Bump pytest from 7.2.2 to 7.3.0
2023-04-10 07:49:57 +02:00
dependabot[bot]
2ea0157197
Bump pypa/gh-action-pypi-publish from 1.8.4 to 1.8.5
Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.8.4 to 1.8.5.
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](https://github.com/pypa/gh-action-pypi-publish/compare/v1.8.4...v1.8.5)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:57:51 +00:00
dependabot[bot]
03352f3b62
Bump types-python-dateutil from 2.8.19.11 to 2.8.19.12
Bumps [types-python-dateutil](https://github.com/python/typeshed) from 2.8.19.11 to 2.8.19.12.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-python-dateutil
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:57:04 +00:00
dependabot[bot]
26eb4f7fe6
Bump pymdown-extensions from 9.10 to 9.11
Bumps [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) from 9.10 to 9.11.
- [Release notes](https://github.com/facelessuser/pymdown-extensions/releases)
- [Commits](https://github.com/facelessuser/pymdown-extensions/compare/9.10...9.11)

---
updated-dependencies:
- dependency-name: pymdown-extensions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:56:57 +00:00
dependabot[bot]
7e1f3aa545
Bump filelock from 3.10.6 to 3.11.0
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.10.6 to 3.11.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.10.6...3.11.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:56:51 +00:00
dependabot[bot]
14532e3a56
Bump orjson from 3.8.9 to 3.8.10
Bumps [orjson](https://github.com/ijl/orjson) from 3.8.9 to 3.8.10.
- [Release notes](https://github.com/ijl/orjson/releases)
- [Changelog](https://github.com/ijl/orjson/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ijl/orjson/compare/3.8.9...3.8.10)

---
updated-dependencies:
- dependency-name: orjson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:56:42 +00:00
dependabot[bot]
a449f7c78c
Bump pytest from 7.2.2 to 7.3.0
Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.2.2 to 7.3.0.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.2.2...7.3.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:56:38 +00:00
dependabot[bot]
8854ef8cba
Bump ccxt from 3.0.50 to 3.0.58
Bumps [ccxt](https://github.com/ccxt/ccxt) from 3.0.50 to 3.0.58.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ccxt/ccxt/compare/3.0.50...3.0.58)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-10 03:56:33 +00:00
Matthias
526943f29e Remove freqUI alpha warning 2023-04-09 19:44:38 +02:00
Matthias
df51111c33 Always show strategy summary 2023-04-09 08:53:36 +02:00
Matthias
dd8900a1c6 Improve ordering of backtest output 2023-04-09 08:53:36 +02:00
Matthias
5404905d28 Fix typos in docs 2023-04-08 17:13:51 +02:00
Matthias
bed51fa790 Properly build specific Torch image 2023-04-08 17:00:25 +02:00
Matthias
f5a5c2d6b9 Improve imports 2023-04-08 16:44:33 +02:00
Matthias
a102cfdfc9 Add new /profit fields to API 2023-04-08 16:41:25 +02:00
Matthias
be72670ca2 Add documentation about /profit change 2023-04-08 16:40:14 +02:00
Matthias
cf2cb94f8d Add bot start date to /profit output 2023-04-08 16:38:44 +02:00
Matthias
fa3a81b022 convert Keys to enum 2023-04-08 16:28:50 +02:00
Matthias
7ff30c6df8 Add additional, typesafe getters 2023-04-08 16:24:38 +02:00
Matthias
7751768b2e Store initial_time value 2023-04-08 16:13:16 +02:00
Matthias
9c2cdd4fb9
Merge pull request #8388 from freqtrade/patch-pair-colon-bug
Bug fix: FreqAI backtest target setting
2023-04-08 14:16:41 +02:00
robcaulk
69b9b35a08 Merge remote-tracking branch 'origin/develop' into feat/add-pytorch-model-support 2023-04-08 13:22:25 +02:00
robcaulk
c2c97d9f78 make a fake pair_dict instead of MagicMocking it 2023-04-08 13:20:29 +02:00
robcaulk
48d3c8e62e fix model loading from disk bug, improve doc, clarify installation/docker instructions, add a torch tag to the freqairl docker image. Fix seriously outdated prediction_model docstrings 2023-04-08 12:09:53 +02:00
Matthias
ac817b7808 Improve docstrings for key-value store 2023-04-08 10:09:31 +02:00
Matthias
4d4f4bf23e Add test for key_value_store 2023-04-08 10:07:21 +02:00
Matthias
c083723698 Add initial version of key value store 2023-04-08 10:07:03 +02:00
Matthias
f8d89c46e5 Don't reset open_order_id if the order didn't cancel 2023-04-07 19:49:13 +02:00
Matthias
1952e453bb Improved formatting for fetch order_or_stop calls 2023-04-07 17:35:11 +02:00
Matthias
77985fa591 Update thread name for uvicorn worker 2023-04-07 14:49:53 +02:00
Matthias
a75d891007 Ensure minimum sqlalchemy version is respected 2023-04-07 14:45:06 +02:00
Matthias
dae3f72be7 Bump Dockerfile to latest 3.10 2023-04-07 14:11:31 +02:00
Matthias
f03a99918a Ensure hyper param file can be loaded
closes #8452
2023-04-04 20:04:28 +02:00
Yinon Polak
a655524221 pytorch mlp rename input to fix mypy error 2023-04-04 12:24:29 +03:00
Yinon Polak
26738370c7 pytorch mlp add explicit annotation to fix mypy error 2023-04-04 12:12:02 +03:00
Matthias
fe02f611fb Fix typo in reinforcement learning
closes #8431
2023-04-04 06:46:35 +02:00
Matthias
1b10a3a2bf Merge branch 'develop' of github.com:freqtrade/freqtrade into develop 2023-04-03 20:24:58 +02:00
Matthias
92a060c5b4 Make stop_price_parameter configurable by exchange 2023-04-03 20:18:57 +02:00
Matthias
096fd1916c
Merge pull request #8445 from freqtrade/dependabot/pip/develop/tensorboard-2.12.1
Bump tensorboard from 2.12.0 to 2.12.1
2023-04-03 19:14:29 +02:00
Matthias
fb09a16127
Merge pull request #8438 from freqtrade/dependabot/pip/develop/types-tabulate-0.9.0.2
Bump types-tabulate from 0.9.0.1 to 0.9.0.2
2023-04-03 18:12:30 +02:00
Yinon Polak
6b204c97ed fix pytorch data convertor type hints 2023-04-03 19:02:07 +03:00
Yinon Polak
0c4574b3b7 prevent mypy error, explicitly unpack input list of pytorch mlp model, 2023-04-03 18:10:47 +03:00
Yinon Polak
d9d9993179 add documentation 2023-04-03 17:06:39 +03:00
Yinon Polak
7b494c8333 add documentation to pytorch data convertor 2023-04-03 16:39:49 +03:00
Yinon Polak
bc9454e0f9 add device to data convertor class doc 2023-04-03 16:36:38 +03:00
Yinon Polak
36a0a14a23 clean code 2023-04-03 16:26:42 +03:00
Yinon Polak
c137666230 fix imports 2023-04-03 16:03:15 +03:00
Matthias
7fed0782d5
pre-commit types-tabulate 2023-04-03 14:19:11 +02:00
Yinon Polak
bd3b70293f add pytorch data convertor 2023-04-03 15:19:10 +03:00
dependabot[bot]
30fc24bd8c
Bump types-tabulate from 0.9.0.1 to 0.9.0.2
Bumps [types-tabulate](https://github.com/python/typeshed) from 0.9.0.1 to 0.9.0.2.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-tabulate
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 12:18:15 +00:00
Matthias
7e3de178e1
Merge pull request #8447 from freqtrade/dependabot/pip/develop/types-python-dateutil-2.8.19.11
Bump types-python-dateutil from 2.8.19.10 to 2.8.19.11
2023-04-03 14:17:24 +02:00
Matthias
0c9c9fff0e
Merge branch 'develop' into dependabot/pip/develop/types-python-dateutil-2.8.19.11 2023-04-03 13:41:10 +02:00
Matthias
b96f6670e3
pre-commit dateutil 2023-04-03 13:28:17 +02:00
Matthias
6e02743256
Merge pull request #8446 from freqtrade/dependabot/pip/develop/types-requests-2.28.11.17
Bump types-requests from 2.28.11.16 to 2.28.11.17
2023-04-03 13:27:31 +02:00
Matthias
2b4fa92d09
Merge pull request #8444 from freqtrade/dependabot/pip/develop/ruff-0.0.260
Bump ruff from 0.0.259 to 0.0.260
2023-04-03 11:40:07 +02:00
Matthias
be250230b6
Merge pull request #8443 from freqtrade/dependabot/pip/develop/plotly-5.14.0
Bump plotly from 5.13.1 to 5.14.0
2023-04-03 11:39:42 +02:00
Matthias
5d33ffc015
Merge pull request #8442 from freqtrade/dependabot/pip/develop/orjson-3.8.9
Bump orjson from 3.8.8 to 3.8.9
2023-04-03 11:04:17 +02:00
Matthias
b48498f27f
Types pre-commit 2023-04-03 10:16:56 +02:00
Matthias
e582d8bacb
Merge pull request #8434 from freqtrade/dependabot/pip/develop/sqlalchemy-2.0.8
Bump sqlalchemy from 2.0.7 to 2.0.8
2023-04-03 10:16:00 +02:00
dependabot[bot]
ff40ee655b
Bump types-python-dateutil from 2.8.19.10 to 2.8.19.11
Bumps [types-python-dateutil](https://github.com/python/typeshed) from 2.8.19.10 to 2.8.19.11.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-python-dateutil
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:49:24 +00:00
dependabot[bot]
57deaad806
Bump types-requests from 2.28.11.16 to 2.28.11.17
Bumps [types-requests](https://github.com/python/typeshed) from 2.28.11.16 to 2.28.11.17.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:49:21 +00:00
dependabot[bot]
7779b82277
Bump tensorboard from 2.12.0 to 2.12.1
Bumps [tensorboard](https://github.com/tensorflow/tensorboard) from 2.12.0 to 2.12.1.
- [Release notes](https://github.com/tensorflow/tensorboard/releases)
- [Changelog](https://github.com/tensorflow/tensorboard/blob/2.12.1/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorboard/compare/2.12.0...2.12.1)

---
updated-dependencies:
- dependency-name: tensorboard
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:49:18 +00:00
dependabot[bot]
2bd2058afa
Bump ruff from 0.0.259 to 0.0.260
Bumps [ruff](https://github.com/charliermarsh/ruff) from 0.0.259 to 0.0.260.
- [Release notes](https://github.com/charliermarsh/ruff/releases)
- [Changelog](https://github.com/charliermarsh/ruff/blob/main/BREAKING_CHANGES.md)
- [Commits](https://github.com/charliermarsh/ruff/compare/v0.0.259...v0.0.260)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:49:12 +00:00
dependabot[bot]
bf7936b0af
Bump plotly from 5.13.1 to 5.14.0
Bumps [plotly](https://github.com/plotly/plotly.py) from 5.13.1 to 5.14.0.
- [Release notes](https://github.com/plotly/plotly.py/releases)
- [Changelog](https://github.com/plotly/plotly.py/blob/master/CHANGELOG.md)
- [Commits](https://github.com/plotly/plotly.py/compare/v5.13.1...v5.14.0)

---
updated-dependencies:
- dependency-name: plotly
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:48:50 +00:00
dependabot[bot]
8236bbfd48
Bump orjson from 3.8.8 to 3.8.9
Bumps [orjson](https://github.com/ijl/orjson) from 3.8.8 to 3.8.9.
- [Release notes](https://github.com/ijl/orjson/releases)
- [Changelog](https://github.com/ijl/orjson/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ijl/orjson/compare/3.8.8...3.8.9)

---
updated-dependencies:
- dependency-name: orjson
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 07:48:43 +00:00
Matthias
4dc13ac16a
Merge pull request #8437 from freqtrade/dependabot/pip/develop/ccxt-3.0.50
Bump ccxt from 3.0.37 to 3.0.50
2023-04-03 09:47:27 +02:00
Matthias
eb5423469a
Merge pull request #8435 from freqtrade/dependabot/pip/develop/xgboost-1.7.5
Bump xgboost from 1.7.4 to 1.7.5
2023-04-03 09:47:09 +02:00
Matthias
43496d7929
bump sqlalchemy pre-commit 2023-04-03 09:46:32 +02:00
Matthias
92c70b6b90
Merge pull request #8441 from freqtrade/dependabot/github_actions/develop/pypa/gh-action-pypi-publish-1.8.4
Bump pypa/gh-action-pypi-publish from 1.8.3 to 1.8.4
2023-04-03 09:45:51 +02:00
Matthias
77897c7d6b
Merge pull request #8439 from freqtrade/dependabot/pip/develop/mkdocs-material-9.1.5
Bump mkdocs-material from 9.1.4 to 9.1.5
2023-04-03 09:45:26 +02:00
Matthias
531861573a
Merge pull request #8436 from freqtrade/dependabot/pip/develop/types-cachetools-5.3.0.5
Bump types-cachetools from 5.3.0.4 to 5.3.0.5
2023-04-03 09:45:10 +02:00
Matthias
c9b904eb0e Fix typos in documentation 2023-04-03 06:49:30 +02:00
Matthias
372f1cb37f Reduce verbosity for stop orders 2023-04-03 06:37:31 +02:00
Matthias
a3acdd5240 apply stop-reserve to minimum limits only when necessary
it's unnecessary for amount - but necessary for Cost / price limits.
2023-04-03 06:37:31 +02:00
Matthias
e6a125719e Slightly refactor _get_stake_amount_limit 2023-04-03 06:37:31 +02:00
Matthias
78a1551798 Reorder get_stake_limit 2023-04-03 06:37:31 +02:00
Matthias
6f79d14c9c
pre-commit - bump cachetools 2023-04-03 06:37:15 +02:00
Matthias
28d8722fa7
Merge pull request #8433 from freqtrade/dependabot/pip/develop/websockets-11.0
Bump websockets from 10.4 to 11.0
2023-04-03 06:36:30 +02:00
dependabot[bot]
2715b2ccf0
Bump pypa/gh-action-pypi-publish from 1.8.3 to 1.8.4
Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.8.3 to 1.8.4.
- [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases)
- [Commits](https://github.com/pypa/gh-action-pypi-publish/compare/v1.8.3...v1.8.4)

---
updated-dependencies:
- dependency-name: pypa/gh-action-pypi-publish
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:58:12 +00:00
dependabot[bot]
2ea575cb31
Bump mkdocs-material from 9.1.4 to 9.1.5
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.1.4 to 9.1.5.
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.1.4...9.1.5)

---
updated-dependencies:
- dependency-name: mkdocs-material
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:57:30 +00:00
dependabot[bot]
1b31c54162
Bump ccxt from 3.0.37 to 3.0.50
Bumps [ccxt](https://github.com/ccxt/ccxt) from 3.0.37 to 3.0.50.
- [Release notes](https://github.com/ccxt/ccxt/releases)
- [Changelog](https://github.com/ccxt/ccxt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ccxt/ccxt/compare/3.0.37...3.0.50)

---
updated-dependencies:
- dependency-name: ccxt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:57:19 +00:00
dependabot[bot]
e289c10b6c
Bump types-cachetools from 5.3.0.4 to 5.3.0.5
Bumps [types-cachetools](https://github.com/python/typeshed) from 5.3.0.4 to 5.3.0.5.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-cachetools
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:57:10 +00:00
dependabot[bot]
26ed1ca07c
Bump xgboost from 1.7.4 to 1.7.5
Bumps [xgboost](https://github.com/dmlc/xgboost) from 1.7.4 to 1.7.5.
- [Release notes](https://github.com/dmlc/xgboost/releases)
- [Changelog](https://github.com/dmlc/xgboost/blob/master/NEWS.md)
- [Commits](https://github.com/dmlc/xgboost/compare/v1.7.4...v1.7.5)

---
updated-dependencies:
- dependency-name: xgboost
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:57:05 +00:00
dependabot[bot]
b1e20bcd1e
Bump sqlalchemy from 2.0.7 to 2.0.8
Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/sqlalchemy/sqlalchemy/releases)
- [Changelog](https://github.com/sqlalchemy/sqlalchemy/blob/main/CHANGES.rst)
- [Commits](https://github.com/sqlalchemy/sqlalchemy/commits)

---
updated-dependencies:
- dependency-name: sqlalchemy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:57:00 +00:00
dependabot[bot]
12a73bc151
Bump websockets from 10.4 to 11.0
Bumps [websockets](https://github.com/aaugustin/websockets) from 10.4 to 11.0.
- [Release notes](https://github.com/aaugustin/websockets/releases)
- [Commits](https://github.com/aaugustin/websockets/compare/10.4...11.0)

---
updated-dependencies:
- dependency-name: websockets
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-03 03:56:46 +00:00
Matthias
19e112f399
Merge pull request #8427 from initrv/typo-fix-constants
Typo fix constants
2023-04-02 07:42:15 +02:00
initrv
cccf4f305b fix randomize_starting_position typo 2023-04-02 03:42:05 +03:00
Matthias
dc7e834911 Fix some type issues 2023-04-01 20:17:56 +02:00
Matthias
a630799984
Merge pull request #8423 from freqtrade/add-profit-trade-history
make trade_type value more explicit, add profit to trade_history dict
2023-04-01 15:19:54 +02:00
Matthias
916e1bbc7c
Merge pull request #8412 from freqtrade/fix/partial_stops
support partially filled stops
2023-04-01 15:18:42 +02:00
Robert Caulk
631cb44f5c ensure python code block renders 2023-04-01 15:16:48 +02:00
Robert Caulk
367186cc34 Update freqai-feature-engineering.md
The `metadata` section of `freqai-feature-engineering.md` had a misplaced whitespace in front of the title. 

This PR removes the whitespace.
2023-04-01 15:16:43 +02:00
robcaulk
92f34f262e make trade_type value more explicit, add profit to trade_history dict 2023-04-01 10:05:58 +02:00
Matthias
5e13b48648
Merge pull request #8386 from freqtrade/feature/price_to_precision_round
price to precision rounding
2023-03-31 07:20:10 +02:00
Matthias
6dfb1a1d14 Improve docker regular build caching 2023-03-31 06:49:12 +02:00
Matthias
f8330800d1 Improve docker arm builds 2023-03-31 06:49:02 +02:00
Matthias
3ec7c72da1 Bump develop version to 2023.4.dev 2023-03-30 07:06:23 +02:00
robcaulk
355fde3bca revert setting dk to live in test_plot_feature_importances 2023-03-29 22:01:54 +02:00
Matthias
861c577138 Support partially filled stop orders
closes #8374
2023-03-29 07:05:39 +02:00
Matthias
e062a74e70 Add test for partial stop order canceling
part of #8374
2023-03-29 06:57:17 +02:00
Matthias
c330c493d5 test for Handle stop on exchange partial filled
part of #8374
2023-03-29 06:57:17 +02:00
Yinon Polak
5a7ca35c6b declare class names in FreqaiExampleHybridStrategy 2023-03-28 16:24:49 +03:00
Yinon Polak
077a947972 clean code 2023-03-28 15:18:10 +03:00
Yinon Polak
8ac3a94358 add note to pytorch docs - setting class names for classifiers 2023-03-28 15:17:40 +03:00
Yinon Polak
dfbebdea9b improve comment on class_names in freqai interface 2023-03-28 14:44:44 +03:00
Yinon Polak
b795a70102 fix config example in pytorch mlp documentation 2023-03-28 14:44:43 +03:00
Yinon Polak
026b6a39a9 bugfix skip test split when empty 2023-03-28 14:40:23 +03:00
robcaulk
3cabcabcbd ensure labels are properly defined in backtesting 2023-03-27 15:23:01 +02:00
robcaulk
55781e7f10 fix tests 2023-03-26 19:22:52 +02:00
robcaulk
f1e831a7b8 fix bug in backtest target setting 2023-03-26 13:43:59 +02:00
Matthias
159090c0e7 Add explicit tests for TRUNCATE mode 2023-03-26 11:14:34 +02:00
Matthias
0cb28f3d82 Use kwarg for rounding_mode, update tests with additional parameter 2023-03-26 11:00:41 +02:00
Matthias
d0d0cbe1d1 Implement price_to_precision logic for stoploss 2023-03-26 10:37:18 +02:00
Matthias
02078456fc Merge branch 'develop' into pr/asuiu/8296 2023-03-26 10:28:02 +02:00
Matthias
01dfb1cba8 Revert having price_rounding_mode as configuration 2023-03-26 10:24:47 +02:00
Yinon Polak
8903ba5d89 fix enf of file 2023-03-24 20:35:55 +03:00
Yinon Polak
eabd321281 small docs change 2023-03-23 15:59:57 +02:00
Yinon Polak
45c6ae446f small docs change 2023-03-23 15:04:29 +02:00
Yinon Polak
952e641213 small docs change 2023-03-23 12:43:37 +02:00
Yinon Polak
c44b5b1b3a add pytorch parameters to parameter table docs 2023-03-23 12:41:20 +02:00
Yinon Polak
fc8625c5c5 add pytorch classes uml diagram 2023-03-23 12:13:27 +02:00
Yinon Polak
36a005754a add pytorch documentation 2023-03-22 18:15:57 +02:00
Yinon Polak
479aafc331 rename Torch to PyTorch 2023-03-22 17:50:00 +02:00
Yinon Polak
f81e3d8667 sort imports 2023-03-21 16:42:13 +02:00
Yinon Polak
b9c7d338b3 fix test_start_backtesting 2023-03-21 16:38:05 +02:00
Yinon Polak
4f93106755 Merge remote-tracking branch 'origin/feat/add-pytorch-model-support' into feat/add-pytorch-model-support 2023-03-21 16:26:42 +02:00
Yinon Polak
02bccd0097 add pytorch mlp models to test_start_backtesting 2023-03-21 16:20:35 +02:00
robcaulk
1ba01746a0 organize pytorch files 2023-03-21 15:09:54 +01:00
Yinon Polak
83a7d888bc type hint init in pytorch mlp classes 2023-03-21 15:19:34 +02:00
Yinon Polak
eba82360fa skip pytorch tests on python 3.11 and intel based mac os 2023-03-21 15:18:05 +02:00
Yinon Polak
3fa23860c0 skip pytorch tests on python 3.11 and intel based mac os 2023-03-21 14:34:27 +02:00
Yinon Polak
a80afc8f1b add optional target tensor squeezing to pytorch trainer 2023-03-21 13:20:54 +02:00
Yinon Polak
97339e14cf round up divisions in calc_n_epochs 2023-03-21 12:29:05 +02:00
Yinon Polak
443263803c unsqueeze target tensor when 1 dimensional 2023-03-21 11:42:05 +02:00
Yinon Polak
9906e7d646 clean code 2023-03-21 11:23:45 +02:00
Yinon Polak
e8f040bfbd add class_name attribute to freqai interface 2023-03-20 20:38:43 +02:00
Yinon Polak
a4b617e482 type hints fixes 2023-03-20 20:22:28 +02:00
Yinon Polak
c06cd38951 clean code 2023-03-20 19:55:39 +02:00
Yinon Polak
0a55753faf move default attributes of pytorch classifier to initializer,
to prevent mypy from complaining
2023-03-20 19:40:36 +02:00
Yinon Polak
6b4d9f97c1 clean code 2023-03-20 19:28:30 +02:00
Yinon Polak
bf4aa91aab Merge remote-tracking branch 'origin/feat/add-pytorch-model-support' into feat/add-pytorch-model-support
# Conflicts:
#	freqtrade/freqai/base_models/PyTorchModelTrainer.py
#	freqtrade/freqai/prediction_models/PyTorchClassifier.py
#	freqtrade/freqai/prediction_models/PyTorchMLPClassifier.py
#	freqtrade/freqai/prediction_models/PyTorchMLPModel.py
#	tests/freqai/test_freqai_interface.py
2023-03-20 18:44:24 +02:00
Yinon Polak
500c401b75 improve pytorch classifier documentation 2023-03-20 18:41:04 +02:00
Yinon Polak
81a2cbb4eb fix tests 2023-03-20 18:41:04 +02:00
Yinon Polak
0510cf4491 add config params to tests 2023-03-20 18:41:04 +02:00
Yinon Polak
68728409aa add pytorch regressor test 2023-03-20 18:41:04 +02:00
Yinon Polak
c00ffcee59 fix pytorch classifier test 2023-03-20 18:41:04 +02:00
Yinon Polak
9aec1ddb17 sort imports 2023-03-20 18:41:04 +02:00
Yinon Polak
d98890f32e sort imports 2023-03-20 18:41:04 +02:00
Yinon Polak
f659f8e309 remove unused imports 2023-03-20 18:41:04 +02:00
Yinon Polak
54db239175 add pytorch regressor example 2023-03-20 18:41:04 +02:00
Yinon Polak
601c37f862 refactor classifiers class names 2023-03-20 18:41:04 +02:00
Yinon Polak
501e746c52 improve mlp documentation 2023-03-20 18:41:04 +02:00
Yinon Polak
d04146d1b1 improve mlp documentation 2023-03-20 18:41:04 +02:00
Yinon Polak
ea08931ab3 add mlp documentation 2023-03-20 18:41:04 +02:00
Yinon Polak
ddd1b5c0ff modify feedforward net, move layer norm to start of thr block 2023-03-20 18:41:04 +02:00
Yinon Polak
e08d8190ae fix test 2023-03-20 18:41:04 +02:00
Yinon Polak
fbf7049ac5 sort imports 2023-03-20 18:41:04 +02:00
Yinon Polak
2a1a8c0e64 fix test 2023-03-20 18:41:04 +02:00
Yinon Polak
833aaf8e10 create children class to PyTorchClassifier to implement the fit method where we initialize the trainer and model objects 2023-03-20 18:41:04 +02:00
Yinon Polak
566346dd87 classifier test - set model file extension 2023-03-20 18:41:03 +02:00
Yinon Polak
d0a33d2ee7 fix tests 2023-03-20 18:41:03 +02:00
robcaulk
fab505be1b cheat flake8 for now until we can refactor save into the model class 2023-03-20 18:41:03 +02:00
Yinon Polak
2f386913ac refactor classifiers class names 2023-03-20 11:54:17 +02:00
Yinon Polak
1c11a5f048 improve mlp documentation 2023-03-19 18:10:57 +02:00
Yinon Polak
903a1dc3e5 improve mlp documentation 2023-03-19 18:04:01 +02:00
Yinon Polak
6f9a8a089c add mlp documentation 2023-03-19 17:45:30 +02:00
Yinon Polak
8bee499328 modify feedforward net, move layer norm to start of thr block 2023-03-19 17:03:36 +02:00
Yinon Polak
719faab4b8 fix test 2023-03-19 15:21:34 +02:00
Yinon Polak
9f477aa3c9 sort imports 2023-03-19 15:09:50 +02:00
Yinon Polak
61ac36c576 fix test 2023-03-19 14:49:12 +02:00
Yinon Polak
366c148c10 create children class to PyTorchClassifier to implement the fit method where we initialize the trainer and model objects 2023-03-19 14:38:49 +02:00
Yinon Polak
a49f62eecb classifier test - set model file extension 2023-03-18 20:51:30 +02:00
Yinon Polak
fab9ff1294 fix tests 2023-03-18 15:27:38 +02:00
Yinon Polak
1c91b4427b Merge remote-tracking branch 'origin/feat/add-pytorch-model-support' into feat/add-pytorch-model-support 2023-03-18 14:14:38 +02:00
Yinon Polak
244662b1a4 set class names attribute in the general classifier testing strategy 2023-03-18 14:12:31 +02:00
robcaulk
4550447409 cheat flake8 for now until we can refactor save into the model class 2023-03-14 21:13:30 +01:00
Yinon Polak
366740885a reduce mlp number of parameters for testing 2023-03-13 20:18:26 +02:00
Yinon Polak
918889a2bd reduce mlp number of parameters for testing 2023-03-13 20:09:12 +02:00
Yinon Polak
9c8c30b0e8 add test 2023-03-13 17:17:00 +02:00
Yinon Polak
d7ea750823 revert to using model_training_parameters 2023-03-13 00:35:51 +02:00
Yinon Polak
b6096efadd logging change 2023-03-13 00:35:14 +02:00
Yinon Polak
b927c9dc01 remove train loss calculation from estimate_loss 2023-03-13 00:17:34 +02:00
Yinon Polak
523a58d3d6 simplify statement for pytorch file_type extension 2023-03-13 00:16:44 +02:00
Yinon Polak
0012fe36ca sort imports 2023-03-12 16:16:04 +02:00
Yinon Polak
cb17b36981 simplify file_type check comparisons 2023-03-12 14:50:08 +02:00
Yinon Polak
f9fdf1c31b generalize mlp model 2023-03-12 14:31:08 +02:00
Yinon Polak
1cf0e7be24 use one iteration on all test and train data for evaluation 2023-03-12 12:48:15 +02:00
Yinon Polak
8a9f2aedbb improve documentation 2023-03-09 14:55:52 +02:00
Yinon Polak
e88a0d5248 convert single quotes to double quotes 2023-03-09 13:29:11 +02:00
Yinon Polak
2ef11faba7 reformat documentation 2023-03-09 13:25:20 +02:00
Yinon Polak
c9eee2944b reformat documentation 2023-03-09 13:01:04 +02:00
Yinon Polak
6f962362f2 expand pytorch trainer documentation 2023-03-09 12:45:46 +02:00
Yinon Polak
ba5de0cd00 add documentation 2023-03-09 11:21:10 +02:00
Yinon Polak
3081b9402b add documentation 2023-03-09 11:14:54 +02:00
ASU
1132fa6093 feat: Added price_rounding modes in config 2023-03-09 02:11:31 +02:00
Yinon Polak
1597c3aa89 set class names in IStrategy.set_freqai_targets method, also save class name with model meta data 2023-03-08 18:36:44 +02:00
Yinon Polak
7d26df01b8 fix tensor type hint 2023-03-08 16:17:19 +02:00
Yinon Polak
c8296ccb2d sort imports 2023-03-08 16:13:35 +02:00
Yinon Polak
8d60327d60 add missing import 2023-03-08 16:12:47 +02:00
Yinon Polak
04564dc134 add missing import 2023-03-08 16:11:51 +02:00
Yinon Polak
6161b858c4 sort imports 2023-03-08 16:10:25 +02:00
Yinon Polak
1921a07b89 sort imports 2023-03-08 16:08:04 +02:00
Yinon Polak
b65ade51be revert config_freqai_example changes 2023-03-08 16:05:02 +02:00
Yinon Polak
dfbb2e2b35 sort imports 2023-03-08 16:03:36 +02:00
Yinon Polak
1805db2b07 change documentation and small bugfix 2023-03-08 15:38:22 +02:00
Yinon Polak
76fbec0c17 ad multiclass target names encoder to ints 2023-03-08 14:29:38 +02:00
Yinon Polak
4241bff32a type hints fixes 2023-03-06 20:15:36 +02:00
Yinon Polak
5dd60eda36 type hints fixes 2023-03-06 19:37:08 +02:00
Yinon Polak
8acdd0b47c type hints fixes 2023-03-06 19:14:54 +02:00
Yinon Polak
125085fbaf add freqai.model_exists pytorch file type support 2023-03-06 18:10:49 +02:00
Yinon Polak
7eedcb9c14 reformat code 2023-03-06 17:56:07 +02:00
Yinon Polak
e6e747bcd8 reformat code 2023-03-06 17:50:02 +02:00
Yinon Polak
348a08f1c4 add todo - currently assuming class labels are strings ['0.0', '1.0' .. n_classes]. need to resolve it per ClassifierModel 2023-03-06 16:41:47 +02:00
Yinon Polak
b1ac2bf515 use data loader, add evaluation on epoch 2023-03-06 16:16:45 +02:00
Yinon Polak
751b205618 initial commit 2023-03-05 16:59:24 +02:00
97 changed files with 2058 additions and 395 deletions

View File

@ -425,7 +425,7 @@ jobs:
python setup.py sdist bdist_wheel
- name: Publish to PyPI (Test)
uses: pypa/gh-action-pypi-publish@v1.8.3
uses: pypa/gh-action-pypi-publish@v1.8.5
if: (github.event_name == 'release')
with:
user: __token__
@ -433,7 +433,7 @@ jobs:
repository_url: https://test.pypi.org/legacy/
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@v1.8.3
uses: pypa/gh-action-pypi-publish@v1.8.5
if: (github.event_name == 'release')
with:
user: __token__

View File

@ -13,12 +13,12 @@ repos:
- id: mypy
exclude: build_helpers
additional_dependencies:
- types-cachetools==5.3.0.4
- types-cachetools==5.3.0.5
- types-filelock==3.2.7
- types-requests==2.28.11.16
- types-tabulate==0.9.0.1
- types-python-dateutil==2.8.19.10
- SQLAlchemy==2.0.7
- types-requests==2.28.11.17
- types-tabulate==0.9.0.2
- types-python-dateutil==2.8.19.12
- SQLAlchemy==2.0.9
# stages: [push]
- repo: https://github.com/pycqa/isort

View File

@ -1,4 +1,4 @@
FROM python:3.10.10-slim-bullseye as base
FROM python:3.10.11-slim-bullseye as base
# Setup env
ENV LANG C.UTF-8

View File

@ -12,6 +12,7 @@ TAG=$(echo "${BRANCH_NAME}" | sed -e "s/\//_/g")
TAG_PLOT=${TAG}_plot
TAG_FREQAI=${TAG}_freqai
TAG_FREQAI_RL=${TAG_FREQAI}rl
TAG_FREQAI_TORCH=${TAG_FREQAI}torch
TAG_PI="${TAG}_pi"
TAG_ARM=${TAG}_arm
@ -42,9 +43,9 @@ if [ $? -ne 0 ]; then
return 1
fi
docker build --cache-from freqtrade:${TAG_ARM} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG_ARM} -t freqtrade:${TAG_PLOT_ARM} -f docker/Dockerfile.plot .
docker build --cache-from freqtrade:${TAG_ARM} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG_ARM} -t freqtrade:${TAG_FREQAI_ARM} -f docker/Dockerfile.freqai .
docker build --cache-from freqtrade:${TAG_ARM} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG_ARM} -t freqtrade:${TAG_FREQAI_RL_ARM} -f docker/Dockerfile.freqai_rl .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG_ARM} -t freqtrade:${TAG_PLOT_ARM} -f docker/Dockerfile.plot .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG_ARM} -t freqtrade:${TAG_FREQAI_ARM} -f docker/Dockerfile.freqai .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG_FREQAI_ARM} -t freqtrade:${TAG_FREQAI_RL_ARM} -f docker/Dockerfile.freqai_rl .
# Tag image for upload and next build step
docker tag freqtrade:$TAG_ARM ${CACHE_IMAGE}:$TAG_ARM
@ -84,6 +85,10 @@ docker manifest push -p ${IMAGE_NAME}:${TAG_FREQAI}
docker manifest create ${IMAGE_NAME}:${TAG_FREQAI_RL} ${CACHE_IMAGE}:${TAG_FREQAI_RL} ${CACHE_IMAGE}:${TAG_FREQAI_RL_ARM}
docker manifest push -p ${IMAGE_NAME}:${TAG_FREQAI_RL}
# Create special Torch tag - which is identical to the RL tag.
docker manifest create ${IMAGE_NAME}:${TAG_FREQAI_TORCH} ${CACHE_IMAGE}:${TAG_FREQAI_RL} ${CACHE_IMAGE}:${TAG_FREQAI_RL_ARM}
docker manifest push -p ${IMAGE_NAME}:${TAG_FREQAI_TORCH}
# copy images to ghcr.io
alias crane="docker run --rm -i -v $(pwd)/.crane:/home/nonroot/.docker/ gcr.io/go-containerregistry/crane"
@ -93,6 +98,7 @@ chmod a+rwx .crane
echo "${GHCR_TOKEN}" | crane auth login ghcr.io -u "${GHCR_USERNAME}" --password-stdin
crane copy ${IMAGE_NAME}:${TAG_FREQAI_RL} ${GHCR_IMAGE_NAME}:${TAG_FREQAI_RL}
crane copy ${IMAGE_NAME}:${TAG_FREQAI_RL} ${GHCR_IMAGE_NAME}:${TAG_FREQAI_TORCH}
crane copy ${IMAGE_NAME}:${TAG_FREQAI} ${GHCR_IMAGE_NAME}:${TAG_FREQAI}
crane copy ${IMAGE_NAME}:${TAG_PLOT} ${GHCR_IMAGE_NAME}:${TAG_PLOT}
crane copy ${IMAGE_NAME}:${TAG} ${GHCR_IMAGE_NAME}:${TAG}

View File

@ -58,9 +58,9 @@ fi
# Tag image for upload and next build step
docker tag freqtrade:$TAG ${CACHE_IMAGE}:$TAG
docker build --cache-from freqtrade:${TAG} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG} -t freqtrade:${TAG_PLOT} -f docker/Dockerfile.plot .
docker build --cache-from freqtrade:${TAG} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG} -t freqtrade:${TAG_FREQAI} -f docker/Dockerfile.freqai .
docker build --cache-from freqtrade:${TAG_FREQAI} --build-arg sourceimage=${CACHE_IMAGE} --build-arg sourcetag=${TAG_FREQAI} -t freqtrade:${TAG_FREQAI_RL} -f docker/Dockerfile.freqai_rl .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG} -t freqtrade:${TAG_PLOT} -f docker/Dockerfile.plot .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG} -t freqtrade:${TAG_FREQAI} -f docker/Dockerfile.freqai .
docker build --build-arg sourceimage=freqtrade --build-arg sourcetag=${TAG_FREQAI} -t freqtrade:${TAG_FREQAI_RL} -f docker/Dockerfile.freqai_rl .
docker tag freqtrade:$TAG_PLOT ${CACHE_IMAGE}:$TAG_PLOT
docker tag freqtrade:$TAG_FREQAI ${CACHE_IMAGE}:$TAG_FREQAI

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

View File

@ -274,19 +274,20 @@ A backtesting result will look like that:
| XRP/BTC | 35 | 0.66 | 22.96 | 0.00114897 | 11.48 | 3:49:00 | 12 0 23 34.3 |
| ZEC/BTC | 22 | -0.46 | -10.18 | -0.00050971 | -5.09 | 2:22:00 | 7 0 15 31.8 |
| TOTAL | 429 | 0.36 | 152.41 | 0.00762792 | 76.20 | 4:12:00 | 186 0 243 43.4 |
========================================================= EXIT REASON STATS ==========================================================
| Exit Reason | Exits | Wins | Draws | Losses |
|:-------------------|--------:|------:|-------:|--------:|
| trailing_stop_loss | 205 | 150 | 0 | 55 |
| stop_loss | 166 | 0 | 0 | 166 |
| exit_signal | 56 | 36 | 0 | 20 |
| force_exit | 2 | 0 | 0 | 2 |
====================================================== LEFT OPEN TRADES REPORT ======================================================
| Pair | Entries | Avg Profit % | Cum Profit % | Tot Profit BTC | Tot Profit % | Avg Duration | Win Draw Loss Win% |
|:---------|---------:|---------------:|---------------:|-----------------:|---------------:|:---------------|--------------------:|
| ADA/BTC | 1 | 0.89 | 0.89 | 0.00004434 | 0.44 | 6:00:00 | 1 0 0 100 |
| LTC/BTC | 1 | 0.68 | 0.68 | 0.00003421 | 0.34 | 2:00:00 | 1 0 0 100 |
| TOTAL | 2 | 0.78 | 1.57 | 0.00007855 | 0.78 | 4:00:00 | 2 0 0 100 |
==================== EXIT REASON STATS ====================
| Exit Reason | Exits | Wins | Draws | Losses |
|:-------------------|--------:|------:|-------:|--------:|
| trailing_stop_loss | 205 | 150 | 0 | 55 |
| stop_loss | 166 | 0 | 0 | 166 |
| exit_signal | 56 | 36 | 0 | 20 |
| force_exit | 2 | 0 | 0 | 2 |
================== SUMMARY METRICS ==================
| Metric | Value |
|-----------------------------+---------------------|

View File

@ -236,3 +236,161 @@ If you want to predict multiple targets you must specify all labels in the same
df['&s-up_or_down'] = np.where( df["close"].shift(-100) > df["close"], 'up', 'down')
df['&s-up_or_down'] = np.where( df["close"].shift(-100) == df["close"], 'same', df['&s-up_or_down'])
```
## PyTorch Module
### Quick start
The easiest way to quickly run a pytorch model is with the following command (for regression task):
```bash
freqtrade trade --config config_examples/config_freqai.example.json --strategy FreqaiExampleStrategy --freqaimodel PyTorchMLPRegressor --strategy-path freqtrade/templates
```
!!! note "Installation/docker"
The PyTorch module requires large packages such as `torch`, which should be explicitly requested during `./setup.sh -i` by answering "y" to the question "Do you also want dependencies for freqai-rl or PyTorch (~700mb additional space required) [y/N]?".
Users who prefer docker should ensure they use the docker image appended with `_freqaitorch`.
### Structure
#### Model
You can construct your own Neural Network architecture in PyTorch by simply defining your `nn.Module` class inside your custom [`IFreqaiModel` file](#using-different-prediction-models) and then using that class in your `def train()` function. Here is an example of logistic regression model implementation using PyTorch (should be used with nn.BCELoss criterion) for classification tasks.
```python
class LogisticRegression(nn.Module):
def __init__(self, input_size: int):
super().__init__()
# Define your layers
self.linear = nn.Linear(input_size, 1)
self.activation = nn.Sigmoid()
def forward(self, x: torch.Tensor) -> torch.Tensor:
# Define the forward pass
out = self.linear(x)
out = self.activation(out)
return out
class MyCoolPyTorchClassifier(BasePyTorchClassifier):
"""
This is a custom IFreqaiModel showing how a user might setup their own
custom Neural Network architecture for their training.
"""
@property
def data_convertor(self) -> PyTorchDataConvertor:
return DefaultPyTorchDataConvertor(target_tensor_type=torch.float)
def __init__(self, **kwargs) -> None:
super().__init__(**kwargs)
config = self.freqai_info.get("model_training_parameters", {})
self.learning_rate: float = config.get("learning_rate", 3e-4)
self.model_kwargs: Dict[str, Any] = config.get("model_kwargs", {})
self.trainer_kwargs: Dict[str, Any] = config.get("trainer_kwargs", {})
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
class_names = self.get_class_names()
self.convert_label_column_to_int(data_dictionary, dk, class_names)
n_features = data_dictionary["train_features"].shape[-1]
model = LogisticRegression(
input_dim=n_features
)
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.CrossEntropyLoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
model_meta_data={"class_names": class_names},
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer
```
#### Trainer
The `PyTorchModelTrainer` performs the idiomatic PyTorch train loop:
Define our model, loss function, and optimizer, and then move them to the appropriate device (GPU or CPU). Inside the loop, we iterate through the batches in the dataloader, move the data to the device, compute the prediction and loss, backpropagate, and update the model parameters using the optimizer.
In addition, the trainer is responsible for the following:
- saving and loading the model
- converting the data from `pandas.DataFrame` to `torch.Tensor`.
#### Integration with Freqai module
Like all freqai models, PyTorch models inherit `IFreqaiModel`. `IFreqaiModel` declares three abstract methods: `train`, `fit`, and `predict`. we implement these methods in three levels of hierarchy.
From top to bottom:
1. `BasePyTorchModel` - Implements the `train` method. all `BasePyTorch*` inherit it. responsible for general data preparation (e.g., data normalization) and calling the `fit` method. Sets `device` attribute used by children classes. Sets `model_type` attribute used by the parent class.
2. `BasePyTorch*` - Implements the `predict` method. Here, the `*` represents a group of algorithms, such as classifiers or regressors. responsible for data preprocessing, predicting, and postprocessing if needed.
3. `PyTorch*Classifier` / `PyTorch*Regressor` - implements the `fit` method. responsible for the main train flaw, where we initialize the trainer and model objects.
![image](assets/freqai_pytorch-diagram.png)
#### Full example
Building a PyTorch regressor using MLP (multilayer perceptron) model, MSELoss criterion, and AdamW optimizer.
```python
class PyTorchMLPRegressor(BasePyTorchRegressor):
def __init__(self, **kwargs) -> None:
super().__init__(**kwargs)
config = self.freqai_info.get("model_training_parameters", {})
self.learning_rate: float = config.get("learning_rate", 3e-4)
self.model_kwargs: Dict[str, Any] = config.get("model_kwargs", {})
self.trainer_kwargs: Dict[str, Any] = config.get("trainer_kwargs", {})
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
n_features = data_dictionary["train_features"].shape[-1]
model = PyTorchMLPModel(
input_dim=n_features,
output_dim=1,
**self.model_kwargs
)
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.MSELoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
init_model=init_model,
target_tensor_type=torch.float,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary)
return trainer
```
Here we create a `PyTorchMLPRegressor` class that implements the `fit` method. The `fit` method specifies the training building blocks: model, optimizer, criterion, and trainer. We inherit both `BasePyTorchRegressor` and `BasePyTorchModel`, where the former implements the `predict` method that is suitable for our regression task, and the latter implements the train method.
??? Note "Setting Class Names for Classifiers"
When using classifiers, the user must declare the class names (or targets) by overriding the `IFreqaiModel.class_names` attribute. This is achieved by setting `self.freqai.class_names` in the FreqAI strategy inside the `set_freqai_targets` method.
For example, if you are using a binary classifier to predict price movements as up or down, you can set the class names as follows:
```python
def set_freqai_targets(self, dataframe: DataFrame, metadata: Dict, **kwargs):
self.freqai.class_names = ["down", "up"]
dataframe['&s-up_or_down'] = np.where(dataframe["close"].shift(-100) >
dataframe["close"], 'up', 'down')
return dataframe
```
To see a full example, you can refer to the [classifier test strategy class](https://github.com/freqtrade/freqtrade/blob/develop/tests/strategy/strats/freqai_test_classifier.py).

View File

@ -6,8 +6,8 @@ Low level feature engineering is performed in the user strategy within a set of
| Function | Description |
|---------------|-------------|
| `feature_engineering__expand_all()` | This optional function will automatically expand the defined features on the config defined `indicator_periods_candles`, `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`.
| `feature_engineering__expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `include_periods_candles`.
| `feature_engineering_expand_all()` | This optional function will automatically expand the defined features on the config defined `indicator_periods_candles`, `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`.
| `feature_engineering_expand_basic()` | This optional function will automatically expand the defined features on the config defined `include_timeframes`, `include_shifted_candles`, and `include_corr_pairs`. Note: this function does *not* expand across `include_periods_candles`.
| `feature_engineering_standard()` | This optional function will be called once with the dataframe of the base timeframe. This is the final function to be called, which means that the dataframe entering this function will contain all the features and columns from the base asset created by the other `feature_engineering_expand` functions. This function is a good place to do custom exotic feature extractions (e.g. tsfresh). This function is also a good place for any feature that should not be auto-expanded upon (e.g., day of the week).
| `set_freqai_targets()` | Required function to set the targets for the model. All targets must be prepended with `&` to be recognized by the FreqAI internals.
@ -182,11 +182,11 @@ In total, the number of features the user of the presented example strat has cre
$= 3 * 3 * 3 * 2 * 2 = 108$.
### Gain finer control over `feature_engineering_*` functions with `metadata`
### Gain finer control over `feature_engineering_*` functions with `metadata`
All `feature_engineering_*` and `set_freqai_targets()` functions are passed a `metadata` dictionary which contains information about the `pair`, `tf` (timeframe), and `period` that FreqAI is automating for feature building. As such, a user can use `metadata` inside `feature_engineering_*` functions as criteria for blocking/reserving features for certain timeframes, periods, pairs etc.
```py
```python
def feature_engineering_expand_all(self, dataframe, period, metadata, **kwargs):
if metadata["tf"] == "1h":
dataframe["%-roc-period"] = ta.ROC(dataframe, timeperiod=period)

View File

@ -86,6 +86,27 @@ Mandatory parameters are marked as **Required** and have to be set in one of the
| `randomize_starting_position` | Randomize the starting point of each episode to avoid overfitting. <br> **Datatype:** bool. <br> Default: `False`.
| `drop_ohlc_from_features` | Do not include the normalized ohlc data in the feature set passed to the agent during training (ohlc will still be used for driving the environment in all cases) <br> **Datatype:** Boolean. <br> **Default:** `False`
### PyTorch parameters
#### general
| Parameter | Description |
|------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters` sub dictionary**
| `learning_rate` | Learning rate to be passed to the optimizer. <br> **Datatype:** float. <br> Default: `3e-4`.
| `model_kwargs` | Parameters to be passed to the model class. <br> **Datatype:** dict. <br> Default: `{}`.
| `trainer_kwargs` | Parameters to be passed to the trainer class. <br> **Datatype:** dict. <br> Default: `{}`.
#### trainer_kwargs
| Parameter | Description |
|------------|-------------|
| | **Model training parameters within the `freqai.model_training_parameters.model_kwargs` sub dictionary**
| `max_iters` | The number of training iterations to run. iteration here refers to the number of times we call self.optimizer.step(). used to calculate n_epochs. <br> **Datatype:** int. <br> Default: `100`.
| `batch_size` | The size of the batches to use during training.. <br> **Datatype:** int. <br> Default: `64`.
| `max_n_eval_batches` | The maximum number batches to use for evaluation.. <br> **Datatype:** int, optional. <br> Default: `None`.
### Additional parameters
| Parameter | Description |

View File

@ -180,7 +180,7 @@ As you begin to modify the strategy and the prediction model, you will quickly r
# you can use feature values from dataframe
# Assumes the shifted RSI indicator has been generated in the strategy.
rsi_now = self.raw_features[f"%-rsi-period-10_shift-1_{pair}_"
rsi_now = self.raw_features[f"%-rsi-period_10_shift-1_{pair}_"
f"{self.config['timeframe']}"].iloc[self._current_tick]
# reward agent for entering trades

View File

@ -1,6 +1,6 @@
markdown==3.3.7
mkdocs==1.4.2
mkdocs-material==9.1.4
mkdocs-material==9.1.6
mdx_truly_sane_lists==1.3
pymdown-extensions==9.10
pymdown-extensions==9.11
jinja2==3.1.2

View File

@ -9,9 +9,6 @@ This same command can also be used to update freqUI, should there be a new relea
Once the bot is started in trade / dry-run mode (with `freqtrade trade`) - the UI will be available under the configured port below (usually `http://127.0.0.1:8080`).
!!! info "Alpha release"
FreqUI is still considered an alpha release - if you encounter bugs or inconsistencies please open a [FreqUI issue](https://github.com/freqtrade/frequi/issues/new/choose).
!!! Note "developers"
Developers should not use this method, but instead use the method described in the [freqUI repository](https://github.com/freqtrade/frequi) to get the source-code of freqUI.

View File

@ -23,10 +23,22 @@ These modes can be configured with these values:
'stoploss_on_exchange_limit_ratio': 0.99
```
!!! Note
Stoploss on exchange is only supported for Binance (stop-loss-limit), Huobi (stop-limit), Kraken (stop-loss-market, stop-loss-limit), Gate (stop-limit), and Kucoin (stop-limit and stop-market) as of now.
<ins>Do not set too low/tight stoploss value if using stop loss on exchange!</ins>
If set to low/tight then you have greater risk of missing fill on the order and stoploss will not work.
Stoploss on exchange is only supported for the following exchanges, and not all exchanges support both stop-limit and stop-market.
The Order-type will be ignored if only one mode is available.
| Exchange | stop-loss type |
|----------|-------------|
| Binance | limit |
| Binance Futures | market, limit |
| Huobi | limit |
| kraken | market, limit |
| Gate | limit |
| Okx | limit |
| Kucoin | stop-limit, stop-market|
!!! Note "Tight stoploss"
<ins>Do not set too low/tight stoploss value when using stop loss on exchange!</ins>
If set to low/tight you will have greater risk of missing fill on the order and stoploss will not work.
### stoploss_on_exchange and stoploss_on_exchange_limit_ratio

View File

@ -279,6 +279,7 @@ Return a summary of your profit/loss and performance.
> ∙ `33.095 EUR`
>
> **Total Trade Count:** `138`
> **Bot started:** `2022-07-11 18:40:44`
> **First Trade opened:** `3 days ago`
> **Latest Trade opened:** `2 minutes ago`
> **Avg. Duration:** `2:33:45`
@ -292,6 +293,7 @@ The relative profit of `15.2 Σ%` is be based on the starting capital - so in th
Starting capital is either taken from the `available_capital` setting, or calculated by using current wallet size - profits.
Profit Factor is calculated as gross profits / gross losses - and should serve as an overall metric for the strategy.
Max drawdown corresponds to the backtesting metric `Absolute Drawdown (Account)` - calculated as `(Absolute Drawdown) / (DrawdownHigh + startingBalance)`.
Bot started date will refer to the date the bot was first started. For older bots, this will default to the first trade's open date.
### /forceexit <trade_id>

View File

@ -1,5 +1,5 @@
""" Freqtrade bot """
__version__ = '2023.3.dev'
__version__ = '2023.4.dev'
if 'dev' in __version__:
from pathlib import Path

View File

@ -116,7 +116,7 @@ class TimeRange:
:param text: value from --timerange
:return: Start and End range period
"""
if text is None:
if not text:
return TimeRange(None, None, 0, 0)
syntax = [(r'^-(\d{8})$', (None, 'date')),
(r'^(\d{8})-$', ('date', None)),

View File

@ -64,6 +64,7 @@ USERPATH_FREQAIMODELS = 'freqaimodels'
TELEGRAM_SETTING_OPTIONS = ['on', 'off', 'silent']
WEBHOOK_FORMAT_OPTIONS = ['form', 'json', 'raw']
FULL_DATAFRAME_THRESHOLD = 100
CUSTOM_TAG_MAX_LENGTH = 255
ENV_VAR_PREFIX = 'FREQTRADE__'
@ -598,7 +599,7 @@ CONF_SCHEMA = {
"model_type": {"type": "string", "default": "PPO"},
"policy_type": {"type": "string", "default": "MlpPolicy"},
"net_arch": {"type": "array", "default": [128, 128]},
"randomize_startinng_position": {"type": "boolean", "default": False},
"randomize_starting_position": {"type": "boolean", "default": False},
"model_reward_parameters": {
"type": "object",
"properties": {

View File

@ -246,14 +246,8 @@ def _load_backtest_data_df_compatibility(df: pd.DataFrame) -> pd.DataFrame:
"""
Compatibility support for older backtest data.
"""
df['open_date'] = pd.to_datetime(df['open_date'],
utc=True,
infer_datetime_format=True
)
df['close_date'] = pd.to_datetime(df['close_date'],
utc=True,
infer_datetime_format=True
)
df['open_date'] = pd.to_datetime(df['open_date'], utc=True)
df['close_date'] = pd.to_datetime(df['close_date'], utc=True)
# Compatibility support for pre short Columns
if 'is_short' not in df.columns:
df['is_short'] = False

View File

@ -34,7 +34,7 @@ def ohlcv_to_dataframe(ohlcv: list, timeframe: str, pair: str, *,
cols = DEFAULT_DATAFRAME_COLUMNS
df = DataFrame(ohlcv, columns=cols)
df['date'] = to_datetime(df['date'], unit='ms', utc=True, infer_datetime_format=True)
df['date'] = to_datetime(df['date'], unit='ms', utc=True)
# Some exchanges return int values for Volume and even for OHLC.
# Convert them since TA-LIB indicators used in the strategy assume floats

View File

@ -63,10 +63,7 @@ class FeatherDataHandler(IDataHandler):
pairdata.columns = self._columns
pairdata = pairdata.astype(dtype={'open': 'float', 'high': 'float',
'low': 'float', 'close': 'float', 'volume': 'float'})
pairdata['date'] = to_datetime(pairdata['date'],
unit='ms',
utc=True,
infer_datetime_format=True)
pairdata['date'] = to_datetime(pairdata['date'], unit='ms', utc=True)
return pairdata
def ohlcv_append(

View File

@ -75,10 +75,7 @@ class JsonDataHandler(IDataHandler):
return DataFrame(columns=self._columns)
pairdata = pairdata.astype(dtype={'open': 'float', 'high': 'float',
'low': 'float', 'close': 'float', 'volume': 'float'})
pairdata['date'] = to_datetime(pairdata['date'],
unit='ms',
utc=True,
infer_datetime_format=True)
pairdata['date'] = to_datetime(pairdata['date'], unit='ms', utc=True)
return pairdata
def ohlcv_append(

View File

@ -62,10 +62,7 @@ class ParquetDataHandler(IDataHandler):
pairdata.columns = self._columns
pairdata = pairdata.astype(dtype={'open': 'float', 'high': 'float',
'low': 'float', 'close': 'float', 'volume': 'float'})
pairdata['date'] = to_datetime(pairdata['date'],
unit='ms',
utc=True,
infer_datetime_format=True)
pairdata['date'] = to_datetime(pairdata['date'], unit='ms', utc=True)
return pairdata
def ohlcv_append(

View File

@ -8,15 +8,15 @@ from freqtrade.exchange.bitpanda import Bitpanda
from freqtrade.exchange.bittrex import Bittrex
from freqtrade.exchange.bybit import Bybit
from freqtrade.exchange.coinbasepro import Coinbasepro
from freqtrade.exchange.exchange_utils import (amount_to_contract_precision, amount_to_contracts,
amount_to_precision, available_exchanges,
ccxt_exchanges, contracts_to_amount,
date_minus_candles, is_exchange_known_ccxt,
market_is_active, price_to_precision,
timeframe_to_minutes, timeframe_to_msecs,
timeframe_to_next_date, timeframe_to_prev_date,
timeframe_to_seconds, validate_exchange,
validate_exchanges)
from freqtrade.exchange.exchange_utils import (ROUND_DOWN, ROUND_UP, amount_to_contract_precision,
amount_to_contracts, amount_to_precision,
available_exchanges, ccxt_exchanges,
contracts_to_amount, date_minus_candles,
is_exchange_known_ccxt, market_is_active,
price_to_precision, timeframe_to_minutes,
timeframe_to_msecs, timeframe_to_next_date,
timeframe_to_prev_date, timeframe_to_seconds,
validate_exchange, validate_exchanges)
from freqtrade.exchange.gate import Gate
from freqtrade.exchange.hitbtc import Hitbtc
from freqtrade.exchange.huobi import Huobi

View File

@ -30,13 +30,14 @@ from freqtrade.exceptions import (DDosProtection, ExchangeError, InsufficientFun
RetryableOrderError, TemporaryError)
from freqtrade.exchange.common import (API_FETCH_ORDER_RETRY_COUNT, remove_credentials, retrier,
retrier_async)
from freqtrade.exchange.exchange_utils import (CcxtModuleType, amount_to_contract_precision,
amount_to_contracts, amount_to_precision,
contracts_to_amount, date_minus_candles,
is_exchange_known_ccxt, market_is_active,
price_to_precision, timeframe_to_minutes,
timeframe_to_msecs, timeframe_to_next_date,
timeframe_to_prev_date, timeframe_to_seconds)
from freqtrade.exchange.exchange_utils import (ROUND, ROUND_DOWN, ROUND_UP, CcxtModuleType,
amount_to_contract_precision, amount_to_contracts,
amount_to_precision, contracts_to_amount,
date_minus_candles, is_exchange_known_ccxt,
market_is_active, price_to_precision,
timeframe_to_minutes, timeframe_to_msecs,
timeframe_to_next_date, timeframe_to_prev_date,
timeframe_to_seconds)
from freqtrade.exchange.types import OHLCVResponse, OrderBook, Ticker, Tickers
from freqtrade.misc import (chunks, deep_merge_dicts, file_dump_json, file_load_json,
safe_value_fallback2)
@ -59,6 +60,7 @@ class Exchange:
# or by specifying them in the configuration.
_ft_has_default: Dict = {
"stoploss_on_exchange": False,
"stop_price_param": "stopPrice",
"order_time_in_force": ["GTC"],
"ohlcv_params": {},
"ohlcv_candle_limit": 500,
@ -734,12 +736,14 @@ class Exchange:
"""
return amount_to_precision(amount, self.get_precision_amount(pair), self.precisionMode)
def price_to_precision(self, pair: str, price: float) -> float:
def price_to_precision(self, pair: str, price: float, *, rounding_mode: int = ROUND) -> float:
"""
Returns the price rounded up to the precision the Exchange accepts.
Rounds up
Returns the price rounded to the precision the Exchange accepts.
The default price_rounding_mode in conf is ROUND.
For stoploss calculations, must use ROUND_UP for longs, and ROUND_DOWN for shorts.
"""
return price_to_precision(price, self.get_precision_price(pair), self.precisionMode)
return price_to_precision(price, self.get_precision_price(pair),
self.precisionMode, rounding_mode=rounding_mode)
def price_get_one_pip(self, pair: str, price: float) -> float:
"""
@ -762,12 +766,12 @@ class Exchange:
return self._get_stake_amount_limit(pair, price, stoploss, 'min', leverage)
def get_max_pair_stake_amount(self, pair: str, price: float, leverage: float = 1.0) -> float:
max_stake_amount = self._get_stake_amount_limit(pair, price, 0.0, 'max')
max_stake_amount = self._get_stake_amount_limit(pair, price, 0.0, 'max', leverage)
if max_stake_amount is None:
# * Should never be executed
raise OperationalException(f'{self.name}.get_max_pair_stake_amount should'
'never set max_stake_amount to None')
return max_stake_amount / leverage
return max_stake_amount
def _get_stake_amount_limit(
self,
@ -785,43 +789,41 @@ class Exchange:
except KeyError:
raise ValueError(f"Can't get market information for symbol {pair}")
if isMin:
# reserve some percent defined in config (5% default) + stoploss
margin_reserve: float = 1.0 + self._config.get('amount_reserve_percent',
DEFAULT_AMOUNT_RESERVE_PERCENT)
stoploss_reserve = (
margin_reserve / (1 - abs(stoploss)) if abs(stoploss) != 1 else 1.5
)
# it should not be more than 50%
stoploss_reserve = max(min(stoploss_reserve, 1.5), 1)
else:
margin_reserve = 1.0
stoploss_reserve = 1.0
stake_limits = []
limits = market['limits']
if (limits['cost'][limit] is not None):
stake_limits.append(
self._contracts_to_amount(
pair,
limits['cost'][limit]
)
self._contracts_to_amount(pair, limits['cost'][limit]) * stoploss_reserve
)
if (limits['amount'][limit] is not None):
stake_limits.append(
self._contracts_to_amount(
pair,
limits['amount'][limit] * price
)
self._contracts_to_amount(pair, limits['amount'][limit]) * price * margin_reserve
)
if not stake_limits:
return None if isMin else float('inf')
# reserve some percent defined in config (5% default) + stoploss
amount_reserve_percent = 1.0 + self._config.get('amount_reserve_percent',
DEFAULT_AMOUNT_RESERVE_PERCENT)
amount_reserve_percent = (
amount_reserve_percent / (1 - abs(stoploss)) if abs(stoploss) != 1 else 1.5
)
# it should not be more than 50%
amount_reserve_percent = max(min(amount_reserve_percent, 1.5), 1)
# The value returned should satisfy both limits: for amount (base currency) and
# for cost (quote, stake currency), so max() is used here.
# See also #2575 at github.
return self._get_stake_amount_considering_leverage(
max(stake_limits) * amount_reserve_percent,
max(stake_limits) if isMin else min(stake_limits),
leverage or 1.0
) if isMin else min(stake_limits)
)
def _get_stake_amount_considering_leverage(self, stake_amount: float, leverage: float) -> float:
"""
@ -1114,11 +1116,11 @@ class Exchange:
"""
if not self._ft_has.get('stoploss_on_exchange'):
raise OperationalException(f"stoploss is not implemented for {self.name}.")
price_param = self._ft_has['stop_price_param']
return (
order.get('stopPrice', None) is None
or ((side == "sell" and stop_loss > float(order['stopPrice'])) or
(side == "buy" and stop_loss < float(order['stopPrice'])))
order.get(price_param, None) is None
or ((side == "sell" and stop_loss > float(order[price_param])) or
(side == "buy" and stop_loss < float(order[price_param])))
)
def _get_stop_order_type(self, user_order_type) -> Tuple[str, str]:
@ -1158,8 +1160,8 @@ class Exchange:
def _get_stop_params(self, side: BuySell, ordertype: str, stop_price: float) -> Dict:
params = self._params.copy()
# Verify if stopPrice works for your exchange!
params.update({'stopPrice': stop_price})
# Verify if stopPrice works for your exchange, else configure stop_price_param
params.update({self._ft_has['stop_price_param']: stop_price})
return params
@retrier(retries=0)
@ -1185,12 +1187,12 @@ class Exchange:
user_order_type = order_types.get('stoploss', 'market')
ordertype, user_order_type = self._get_stop_order_type(user_order_type)
stop_price_norm = self.price_to_precision(pair, stop_price)
round_mode = ROUND_DOWN if side == 'buy' else ROUND_UP
stop_price_norm = self.price_to_precision(pair, stop_price, rounding_mode=round_mode)
limit_rate = None
if user_order_type == 'limit':
limit_rate = self._get_stop_limit_rate(stop_price, order_types, side)
limit_rate = self.price_to_precision(pair, limit_rate)
limit_rate = self.price_to_precision(pair, limit_rate, rounding_mode=round_mode)
if self._config['dry_run']:
dry_order = self.create_dry_run_order(

View File

@ -2,11 +2,12 @@
Exchange support utils
"""
from datetime import datetime, timedelta, timezone
from math import ceil
from math import ceil, floor
from typing import Any, Dict, List, Optional, Tuple
import ccxt
from ccxt import ROUND_DOWN, ROUND_UP, TICK_SIZE, TRUNCATE, decimal_to_precision
from ccxt import (DECIMAL_PLACES, ROUND, ROUND_DOWN, ROUND_UP, SIGNIFICANT_DIGITS, TICK_SIZE,
TRUNCATE, decimal_to_precision)
from freqtrade.exchange.common import BAD_EXCHANGES, EXCHANGE_HAS_OPTIONAL, EXCHANGE_HAS_REQUIRED
from freqtrade.util import FtPrecise
@ -219,35 +220,51 @@ def amount_to_contract_precision(
return amount
def price_to_precision(price: float, price_precision: Optional[float],
precisionMode: Optional[int]) -> float:
def price_to_precision(
price: float,
price_precision: Optional[float],
precisionMode: Optional[int],
*,
rounding_mode: int = ROUND,
) -> float:
"""
Returns the price rounded up to the precision the Exchange accepts.
Returns the price rounded to the precision the Exchange accepts.
Partial Re-implementation of ccxt internal method decimal_to_precision(),
which does not support rounding up
which does not support rounding up.
For stoploss calculations, must use ROUND_UP for longs, and ROUND_DOWN for shorts.
TODO: If ccxt supports ROUND_UP for decimal_to_precision(), we could remove this and
align with amount_to_precision().
!!! Rounds up
:param price: price to convert
:param price_precision: price precision to use. Used from markets[pair]['precision']['price']
:param precisionMode: precision mode to use. Should be used from precisionMode
one of ccxt's DECIMAL_PLACES, SIGNIFICANT_DIGITS, or TICK_SIZE
:param rounding_mode: rounding mode to use. Defaults to ROUND
:return: price rounded up to the precision the Exchange accepts
"""
if price_precision is not None and precisionMode is not None:
# price = float(decimal_to_precision(price, rounding_mode=ROUND,
# precision=price_precision,
# counting_mode=self.precisionMode,
# ))
if precisionMode == TICK_SIZE:
if rounding_mode == ROUND:
ticks = price / price_precision
rounded_ticks = round(ticks)
return rounded_ticks * price_precision
precision = FtPrecise(price_precision)
price_str = FtPrecise(price)
missing = price_str % precision
if not missing == FtPrecise("0"):
price = round(float(str(price_str - missing + precision)), 14)
else:
symbol_prec = price_precision
big_price = price * pow(10, symbol_prec)
price = ceil(big_price) / pow(10, symbol_prec)
return round(float(str(price_str - missing + precision)), 14)
return price
elif precisionMode in (SIGNIFICANT_DIGITS, DECIMAL_PLACES):
ndigits = round(price_precision)
if rounding_mode == ROUND:
return round(price, ndigits)
ticks = price * (10**ndigits)
if rounding_mode == ROUND_UP:
return ceil(ticks) / (10**ndigits)
if rounding_mode == TRUNCATE:
return int(ticks) / (10**ndigits)
if rounding_mode == ROUND_DOWN:
return floor(ticks) / (10**ndigits)
raise ValueError(f"Unknown rounding_mode {rounding_mode}")
raise ValueError(f"Unknown precisionMode {precisionMode}")
return price

View File

@ -12,6 +12,7 @@ from freqtrade.exceptions import (DDosProtection, InsufficientFundsError, Invali
OperationalException, TemporaryError)
from freqtrade.exchange import Exchange
from freqtrade.exchange.common import retrier
from freqtrade.exchange.exchange_utils import ROUND_DOWN, ROUND_UP
from freqtrade.exchange.types import Tickers
@ -109,6 +110,7 @@ class Kraken(Exchange):
if self.trading_mode == TradingMode.FUTURES:
params.update({'reduceOnly': True})
round_mode = ROUND_DOWN if side == 'buy' else ROUND_UP
if order_types.get('stoploss', 'market') == 'limit':
ordertype = "stop-loss-limit"
limit_price_pct = order_types.get('stoploss_on_exchange_limit_ratio', 0.99)
@ -116,11 +118,11 @@ class Kraken(Exchange):
limit_rate = stop_price * limit_price_pct
else:
limit_rate = stop_price * (2 - limit_price_pct)
params['price2'] = self.price_to_precision(pair, limit_rate)
params['price2'] = self.price_to_precision(pair, limit_rate, rounding_mode=round_mode)
else:
ordertype = "stop-loss"
stop_price = self.price_to_precision(pair, stop_price)
stop_price = self.price_to_precision(pair, stop_price, rounding_mode=round_mode)
if self._config['dry_run']:
dry_order = self.create_dry_run_order(

View File

@ -28,6 +28,7 @@ class Okx(Exchange):
"funding_fee_timeframe": "8h",
"stoploss_order_types": {"limit": "limit"},
"stoploss_on_exchange": True,
"stop_price_param": "stopLossPrice",
}
_ft_has_futures: Dict = {
"tickers_have_quoteVolume": False,
@ -162,29 +163,12 @@ class Okx(Exchange):
return pair_tiers[-1]['maxNotional'] / leverage
def _get_stop_params(self, side: BuySell, ordertype: str, stop_price: float) -> Dict:
params = self._params.copy()
# Verify if stopPrice works for your exchange!
params.update({'stopLossPrice': stop_price})
params = super()._get_stop_params(side, ordertype, stop_price)
if self.trading_mode == TradingMode.FUTURES and self.margin_mode:
params['tdMode'] = self.margin_mode.value
params['posSide'] = self._get_posSide(side, True)
return params
def stoploss_adjust(self, stop_loss: float, order: Dict, side: str) -> bool:
"""
OKX uses non-default stoploss price naming.
"""
if not self._ft_has.get('stoploss_on_exchange'):
raise OperationalException(f"stoploss is not implemented for {self.name}.")
return (
order.get('stopLossPrice', None) is None
or ((side == "sell" and stop_loss > float(order['stopLossPrice'])) or
(side == "buy" and stop_loss < float(order['stopLossPrice'])))
)
def fetch_stoploss_order(self, order_id: str, pair: str, params: Dict = {}) -> Dict:
if self._config['dry_run']:
return self.fetch_dry_run_order(order_id)

View File

@ -66,7 +66,7 @@ class Base3ActionRLEnv(BaseEnvironment):
elif action == Actions.Sell.value and not self.can_short:
self._update_total_profit()
self._position = Positions.Neutral
trade_type = "neutral"
trade_type = "exit"
self._last_trade_tick = None
else:
print("case not defined")
@ -74,7 +74,7 @@ class Base3ActionRLEnv(BaseEnvironment):
if trade_type is not None:
self.trade_history.append(
{'price': self.current_price(), 'index': self._current_tick,
'type': trade_type})
'type': trade_type, 'profit': self.get_unrealized_profit()})
if (self._total_profit < self.max_drawdown or
self._total_unrealized_profit < self.max_drawdown):

View File

@ -52,16 +52,6 @@ class Base4ActionRLEnv(BaseEnvironment):
trade_type = None
if self.is_tradesignal(action):
"""
Action: Neutral, position: Long -> Close Long
Action: Neutral, position: Short -> Close Short
Action: Long, position: Neutral -> Open Long
Action: Long, position: Short -> Close Short and Open Long
Action: Short, position: Neutral -> Open Short
Action: Short, position: Long -> Close Long and Open Short
"""
if action == Actions.Neutral.value:
self._position = Positions.Neutral
@ -69,16 +59,16 @@ class Base4ActionRLEnv(BaseEnvironment):
self._last_trade_tick = None
elif action == Actions.Long_enter.value:
self._position = Positions.Long
trade_type = "long"
trade_type = "enter_long"
self._last_trade_tick = self._current_tick
elif action == Actions.Short_enter.value:
self._position = Positions.Short
trade_type = "short"
trade_type = "enter_short"
self._last_trade_tick = self._current_tick
elif action == Actions.Exit.value:
self._update_total_profit()
self._position = Positions.Neutral
trade_type = "neutral"
trade_type = "exit"
self._last_trade_tick = None
else:
print("case not defined")
@ -86,7 +76,7 @@ class Base4ActionRLEnv(BaseEnvironment):
if trade_type is not None:
self.trade_history.append(
{'price': self.current_price(), 'index': self._current_tick,
'type': trade_type})
'type': trade_type, 'profit': self.get_unrealized_profit()})
if (self._total_profit < self.max_drawdown or
self._total_unrealized_profit < self.max_drawdown):

View File

@ -53,16 +53,6 @@ class Base5ActionRLEnv(BaseEnvironment):
trade_type = None
if self.is_tradesignal(action):
"""
Action: Neutral, position: Long -> Close Long
Action: Neutral, position: Short -> Close Short
Action: Long, position: Neutral -> Open Long
Action: Long, position: Short -> Close Short and Open Long
Action: Short, position: Neutral -> Open Short
Action: Short, position: Long -> Close Long and Open Short
"""
if action == Actions.Neutral.value:
self._position = Positions.Neutral
@ -70,21 +60,21 @@ class Base5ActionRLEnv(BaseEnvironment):
self._last_trade_tick = None
elif action == Actions.Long_enter.value:
self._position = Positions.Long
trade_type = "long"
trade_type = "enter_long"
self._last_trade_tick = self._current_tick
elif action == Actions.Short_enter.value:
self._position = Positions.Short
trade_type = "short"
trade_type = "enter_short"
self._last_trade_tick = self._current_tick
elif action == Actions.Long_exit.value:
self._update_total_profit()
self._position = Positions.Neutral
trade_type = "neutral"
trade_type = "exit_long"
self._last_trade_tick = None
elif action == Actions.Short_exit.value:
self._update_total_profit()
self._position = Positions.Neutral
trade_type = "neutral"
trade_type = "exit_short"
self._last_trade_tick = None
else:
print("case not defined")
@ -92,7 +82,7 @@ class Base5ActionRLEnv(BaseEnvironment):
if trade_type is not None:
self.trade_history.append(
{'price': self.current_price(), 'index': self._current_tick,
'type': trade_type})
'type': trade_type, 'profit': self.get_unrealized_profit()})
if (self._total_profit < self.max_drawdown or
self._total_unrealized_profit < self.max_drawdown):

View File

@ -0,0 +1,147 @@
import logging
from typing import Dict, List, Tuple
import numpy as np
import numpy.typing as npt
import pandas as pd
import torch
from pandas import DataFrame
from torch.nn import functional as F
from freqtrade.exceptions import OperationalException
from freqtrade.freqai.base_models.BasePyTorchModel import BasePyTorchModel
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class BasePyTorchClassifier(BasePyTorchModel):
"""
A PyTorch implementation of a classifier.
User must implement fit method
Important!
- User must declare the target class names in the strategy,
under IStrategy.set_freqai_targets method.
for example, in your strategy:
```
def set_freqai_targets(self, dataframe: DataFrame, metadata: Dict, **kwargs):
self.freqai.class_names = ["down", "up"]
dataframe['&s-up_or_down'] = np.where(dataframe["close"].shift(-100) >
dataframe["close"], 'up', 'down')
return dataframe
"""
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.class_name_to_index = None
self.index_to_class_name = None
def predict(
self, unfiltered_df: DataFrame, dk: FreqaiDataKitchen, **kwargs
) -> Tuple[DataFrame, npt.NDArray[np.int_]]:
"""
Filter the prediction features data and predict with it.
:param unfiltered_df: Full dataframe for the current backtest period.
:return:
:pred_df: dataframe containing the predictions
:do_predict: np.array of 1s and 0s to indicate places where freqai needed to remove
data (NaNs) or felt uncertain about data (PCA and DI index)
:raises ValueError: if 'class_names' doesn't exist in model meta_data.
"""
class_names = self.model.model_meta_data.get("class_names", None)
if not class_names:
raise ValueError(
"Missing class names. "
"self.model.model_meta_data['class_names'] is None."
)
if not self.class_name_to_index:
self.init_class_names_to_index_mapping(class_names)
dk.find_features(unfiltered_df)
filtered_df, _ = dk.filter_features(
unfiltered_df, dk.training_features_list, training_filter=False
)
filtered_df = dk.normalize_data_from_metadata(filtered_df)
dk.data_dictionary["prediction_features"] = filtered_df
self.data_cleaning_predict(dk)
x = self.data_convertor.convert_x(
dk.data_dictionary["prediction_features"],
device=self.device
)
logits = self.model.model(x)
probs = F.softmax(logits, dim=-1)
predicted_classes = torch.argmax(probs, dim=-1)
predicted_classes_str = self.decode_class_names(predicted_classes)
pred_df_prob = DataFrame(probs.detach().numpy(), columns=class_names)
pred_df = DataFrame(predicted_classes_str, columns=[dk.label_list[0]])
pred_df = pd.concat([pred_df, pred_df_prob], axis=1)
return (pred_df, dk.do_predict)
def encode_class_names(
self,
data_dictionary: Dict[str, pd.DataFrame],
dk: FreqaiDataKitchen,
class_names: List[str],
):
"""
encode class name, str -> int
assuming first column of *_labels data frame to be the target column
containing the class names
"""
target_column_name = dk.label_list[0]
for split in self.splits:
label_df = data_dictionary[f"{split}_labels"]
self.assert_valid_class_names(label_df[target_column_name], class_names)
label_df[target_column_name] = list(
map(lambda x: self.class_name_to_index[x], label_df[target_column_name])
)
@staticmethod
def assert_valid_class_names(
target_column: pd.Series,
class_names: List[str]
):
non_defined_labels = set(target_column) - set(class_names)
if len(non_defined_labels) != 0:
raise OperationalException(
f"Found non defined labels: {non_defined_labels}, ",
f"expecting labels: {class_names}"
)
def decode_class_names(self, class_ints: torch.Tensor) -> List[str]:
"""
decode class name, int -> str
"""
return list(map(lambda x: self.index_to_class_name[x.item()], class_ints))
def init_class_names_to_index_mapping(self, class_names):
self.class_name_to_index = {s: i for i, s in enumerate(class_names)}
self.index_to_class_name = {i: s for i, s in enumerate(class_names)}
logger.info(f"encoded class name to index: {self.class_name_to_index}")
def convert_label_column_to_int(
self,
data_dictionary: Dict[str, pd.DataFrame],
dk: FreqaiDataKitchen,
class_names: List[str]
):
self.init_class_names_to_index_mapping(class_names)
self.encode_class_names(data_dictionary, dk, class_names)
def get_class_names(self) -> List[str]:
if not self.class_names:
raise ValueError(
"self.class_names is empty, "
"set self.freqai.class_names = ['class a', 'class b', 'class c'] "
"inside IStrategy.set_freqai_targets method."
)
return self.class_names

View File

@ -0,0 +1,83 @@
import logging
from abc import ABC, abstractmethod
from time import time
from typing import Any
import torch
from pandas import DataFrame
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
from freqtrade.freqai.freqai_interface import IFreqaiModel
from freqtrade.freqai.torch.PyTorchDataConvertor import PyTorchDataConvertor
logger = logging.getLogger(__name__)
class BasePyTorchModel(IFreqaiModel, ABC):
"""
Base class for PyTorch type models.
User *must* inherit from this class and set fit() and predict() and
data_convertor property.
"""
def __init__(self, **kwargs):
super().__init__(config=kwargs["config"])
self.dd.model_type = "pytorch"
self.device = "cuda" if torch.cuda.is_available() else "cpu"
test_size = self.freqai_info.get('data_split_parameters', {}).get('test_size')
self.splits = ["train", "test"] if test_size != 0 else ["train"]
def train(
self, unfiltered_df: DataFrame, pair: str, dk: FreqaiDataKitchen, **kwargs
) -> Any:
"""
Filter the training data and train a model to it. Train makes heavy use of the datakitchen
for storing, saving, loading, and analyzing the data.
:param unfiltered_df: Full dataframe for the current training period
:return:
:model: Trained model which can be used to inference (self.predict)
"""
logger.info(f"-------------------- Starting training {pair} --------------------")
start_time = time()
features_filtered, labels_filtered = dk.filter_features(
unfiltered_df,
dk.training_features_list,
dk.label_list,
training_filter=True,
)
# split data into train/test data.
data_dictionary = dk.make_train_test_datasets(features_filtered, labels_filtered)
if not self.freqai_info.get("fit_live_predictions", 0) or not self.live:
dk.fit_labels()
# normalize all data based on train_dataset only
data_dictionary = dk.normalize_data(data_dictionary)
# optional additional data cleaning/analysis
self.data_cleaning_train(dk)
logger.info(
f"Training model on {len(dk.data_dictionary['train_features'].columns)} features"
)
logger.info(f"Training model on {len(data_dictionary['train_features'])} data points")
model = self.fit(data_dictionary, dk)
end_time = time()
logger.info(f"-------------------- Done training {pair} "
f"({end_time - start_time:.2f} secs) --------------------")
return model
@property
@abstractmethod
def data_convertor(self) -> PyTorchDataConvertor:
"""
a class responsible for converting `*_features` & `*_labels` pandas dataframes
to pytorch tensors.
"""
raise NotImplementedError("Abstract property")

View File

@ -0,0 +1,49 @@
import logging
from typing import Tuple
import numpy as np
import numpy.typing as npt
from pandas import DataFrame
from freqtrade.freqai.base_models.BasePyTorchModel import BasePyTorchModel
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
logger = logging.getLogger(__name__)
class BasePyTorchRegressor(BasePyTorchModel):
"""
A PyTorch implementation of a regressor.
User must implement fit method
"""
def __init__(self, **kwargs):
super().__init__(**kwargs)
def predict(
self, unfiltered_df: DataFrame, dk: FreqaiDataKitchen, **kwargs
) -> Tuple[DataFrame, npt.NDArray[np.int_]]:
"""
Filter the prediction features data and predict with it.
:param unfiltered_df: Full dataframe for the current backtest period.
:return:
:pred_df: dataframe containing the predictions
:do_predict: np.array of 1s and 0s to indicate places where freqai needed to remove
data (NaNs) or felt uncertain about data (PCA and DI index)
"""
dk.find_features(unfiltered_df)
filtered_df, _ = dk.filter_features(
unfiltered_df, dk.training_features_list, training_filter=False
)
filtered_df = dk.normalize_data_from_metadata(filtered_df)
dk.data_dictionary["prediction_features"] = filtered_df
self.data_cleaning_predict(dk)
x = self.data_convertor.convert_x(
dk.data_dictionary["prediction_features"],
device=self.device
)
y = self.model.model(x)
pred_df = DataFrame(y.detach().numpy(), columns=[dk.label_list[0]])
return (pred_df, dk.do_predict)

View File

@ -446,7 +446,7 @@ class FreqaiDataDrawer:
dump(model, save_path / f"{dk.model_filename}_model.joblib")
elif self.model_type == 'keras':
model.save(save_path / f"{dk.model_filename}_model.h5")
elif 'stable_baselines' in self.model_type or 'sb3_contrib' == self.model_type:
elif self.model_type in ["stable_baselines3", "sb3_contrib", "pytorch"]:
model.save(save_path / f"{dk.model_filename}_model.zip")
if dk.svm_model is not None:
@ -496,7 +496,7 @@ class FreqaiDataDrawer:
dk.training_features_list = dk.data["training_features_list"]
dk.label_list = dk.data["label_list"]
def load_data(self, coin: str, dk: FreqaiDataKitchen) -> Any:
def load_data(self, coin: str, dk: FreqaiDataKitchen) -> Any: # noqa: C901
"""
loads all data required to make a prediction on a sub-train time range
:returns:
@ -537,6 +537,11 @@ class FreqaiDataDrawer:
self.model_type, self.freqai_info['rl_config']['model_type'])
MODELCLASS = getattr(mod, self.freqai_info['rl_config']['model_type'])
model = MODELCLASS.load(dk.data_path / f"{dk.model_filename}_model")
elif self.model_type == 'pytorch':
import torch
zip = torch.load(dk.data_path / f"{dk.model_filename}_model.zip")
model = zip["pytrainer"]
model = model.load_from_checkpoint(zip)
if Path(dk.data_path / f"{dk.model_filename}_svm_model.joblib").is_file():
dk.svm_model = load(dk.data_path / f"{dk.model_filename}_svm_model.joblib")

View File

@ -1291,7 +1291,7 @@ class FreqaiDataKitchen:
return dataframe
def use_strategy_to_populate_indicators(
def use_strategy_to_populate_indicators( # noqa: C901
self,
strategy: IStrategy,
corr_dataframes: dict = {},
@ -1362,12 +1362,12 @@ class FreqaiDataKitchen:
dataframe = self.populate_features(dataframe.copy(), corr_pair, strategy,
corr_dataframes, base_dataframes, True)
if self.live:
dataframe = strategy.set_freqai_targets(dataframe.copy(), metadata=metadata)
dataframe = self.remove_special_chars_from_feature_names(dataframe)
self.get_unique_classes_from_labels(dataframe)
dataframe = self.remove_special_chars_from_feature_names(dataframe)
if self.config.get('reduce_df_footprint', False):
dataframe = reduce_dataframe_footprint(dataframe)

View File

@ -83,6 +83,7 @@ class IFreqaiModel(ABC):
self.CONV_WIDTH = self.freqai_info.get('conv_width', 1)
if self.ft_params.get("inlier_metric_window", 0):
self.CONV_WIDTH = self.ft_params.get("inlier_metric_window", 0) * 2
self.class_names: List[str] = [] # used in classification subclasses
self.pair_it = 0
self.pair_it_train = 0
self.total_pairs = len(self.config.get("exchange", {}).get("pair_whitelist"))
@ -306,7 +307,7 @@ class IFreqaiModel(ABC):
if check_features:
self.dd.load_metadata(dk)
dataframe_dummy_features = self.dk.use_strategy_to_populate_indicators(
strategy, prediction_dataframe=dataframe.tail(1), pair=metadata["pair"]
strategy, prediction_dataframe=dataframe.tail(1), pair=pair
)
dk.find_features(dataframe_dummy_features)
self.check_if_feature_list_matches_strategy(dk)
@ -316,7 +317,7 @@ class IFreqaiModel(ABC):
else:
if populate_indicators:
dataframe = self.dk.use_strategy_to_populate_indicators(
strategy, prediction_dataframe=dataframe, pair=metadata["pair"]
strategy, prediction_dataframe=dataframe, pair=pair
)
populate_indicators = False
@ -332,6 +333,10 @@ class IFreqaiModel(ABC):
dataframe_train = dk.slice_dataframe(tr_train, dataframe_base_train)
dataframe_backtest = dk.slice_dataframe(tr_backtest, dataframe_base_backtest)
dataframe_train = dk.remove_special_chars_from_feature_names(dataframe_train)
dataframe_backtest = dk.remove_special_chars_from_feature_names(dataframe_backtest)
dk.get_unique_classes_from_labels(dataframe_train)
if not self.model_exists(dk):
dk.find_features(dataframe_train)
dk.find_labels(dataframe_train)
@ -567,8 +572,9 @@ class IFreqaiModel(ABC):
file_type = ".joblib"
elif self.dd.model_type == 'keras':
file_type = ".h5"
elif 'stable_baselines' in self.dd.model_type or 'sb3_contrib' == self.dd.model_type:
elif self.dd.model_type in ["stable_baselines3", "sb3_contrib", "pytorch"]:
file_type = ".zip"
path_to_modelfile = Path(dk.data_path / f"{dk.model_filename}_model{file_type}")
file_exists = path_to_modelfile.is_file()
if file_exists:

View File

@ -14,16 +14,20 @@ logger = logging.getLogger(__name__)
class CatboostClassifier(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
train_data = Pool(

View File

@ -15,16 +15,20 @@ logger = logging.getLogger(__name__)
class CatboostClassifierMultiTarget(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
cbc = CatBoostClassifier(

View File

@ -14,16 +14,20 @@ logger = logging.getLogger(__name__)
class CatboostRegressor(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
train_data = Pool(

View File

@ -15,16 +15,20 @@ logger = logging.getLogger(__name__)
class CatboostRegressorMultiTarget(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
cbr = CatBoostRegressor(

View File

@ -12,16 +12,20 @@ logger = logging.getLogger(__name__)
class LightGBMClassifier(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
if self.freqai_info.get('data_split_parameters', {}).get('test_size', 0.1) == 0:

View File

@ -13,16 +13,20 @@ logger = logging.getLogger(__name__)
class LightGBMClassifierMultiTarget(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
lgb = LGBMClassifier(**self.model_training_parameters)

View File

@ -12,18 +12,20 @@ logger = logging.getLogger(__name__)
class LightGBMRegressor(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
Most regressors use the same function names and arguments e.g. user
can drop in LGBMRegressor in place of CatBoostRegressor and all data
management will be properly handled by Freqai.
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
if self.freqai_info.get('data_split_parameters', {}).get('test_size', 0.1) == 0:

View File

@ -13,16 +13,20 @@ logger = logging.getLogger(__name__)
class LightGBMRegressorMultiTarget(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
lgb = LGBMRegressor(**self.model_training_parameters)

View File

@ -0,0 +1,89 @@
from typing import Any, Dict
import torch
from freqtrade.freqai.base_models.BasePyTorchClassifier import BasePyTorchClassifier
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
from freqtrade.freqai.torch.PyTorchDataConvertor import (DefaultPyTorchDataConvertor,
PyTorchDataConvertor)
from freqtrade.freqai.torch.PyTorchMLPModel import PyTorchMLPModel
from freqtrade.freqai.torch.PyTorchModelTrainer import PyTorchModelTrainer
class PyTorchMLPClassifier(BasePyTorchClassifier):
"""
This class implements the fit method of IFreqaiModel.
in the fit method we initialize the model and trainer objects.
the only requirement from the model is to be aligned to PyTorchClassifier
predict method that expects the model to predict a tensor of type long.
parameters are passed via `model_training_parameters` under the freqai
section in the config file. e.g:
{
...
"freqai": {
...
"model_training_parameters" : {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 5000,
"batch_size": 64,
"max_n_eval_batches": null,
},
"model_kwargs": {
"hidden_dim": 512,
"dropout_percent": 0.2,
"n_layer": 1,
},
}
}
}
"""
@property
def data_convertor(self) -> PyTorchDataConvertor:
return DefaultPyTorchDataConvertor(
target_tensor_type=torch.long,
squeeze_target_tensor=True
)
def __init__(self, **kwargs) -> None:
super().__init__(**kwargs)
config = self.freqai_info.get("model_training_parameters", {})
self.learning_rate: float = config.get("learning_rate", 3e-4)
self.model_kwargs: Dict[str, Any] = config.get("model_kwargs", {})
self.trainer_kwargs: Dict[str, Any] = config.get("trainer_kwargs", {})
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
:raises ValueError: If self.class_names is not defined in the parent class.
"""
class_names = self.get_class_names()
self.convert_label_column_to_int(data_dictionary, dk, class_names)
n_features = data_dictionary["train_features"].shape[-1]
model = PyTorchMLPModel(
input_dim=n_features,
output_dim=len(class_names),
**self.model_kwargs
)
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.CrossEntropyLoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
model_meta_data={"class_names": class_names},
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer

View File

@ -0,0 +1,83 @@
from typing import Any, Dict
import torch
from freqtrade.freqai.base_models.BasePyTorchRegressor import BasePyTorchRegressor
from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
from freqtrade.freqai.torch.PyTorchDataConvertor import (DefaultPyTorchDataConvertor,
PyTorchDataConvertor)
from freqtrade.freqai.torch.PyTorchMLPModel import PyTorchMLPModel
from freqtrade.freqai.torch.PyTorchModelTrainer import PyTorchModelTrainer
class PyTorchMLPRegressor(BasePyTorchRegressor):
"""
This class implements the fit method of IFreqaiModel.
in the fit method we initialize the model and trainer objects.
the only requirement from the model is to be aligned to PyTorchRegressor
predict method that expects the model to predict tensor of type float.
the trainer defines the training loop.
parameters are passed via `model_training_parameters` under the freqai
section in the config file. e.g:
{
...
"freqai": {
...
"model_training_parameters" : {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 5000,
"batch_size": 64,
"max_n_eval_batches": null,
},
"model_kwargs": {
"hidden_dim": 512,
"dropout_percent": 0.2,
"n_layer": 1,
},
}
}
}
"""
@property
def data_convertor(self) -> PyTorchDataConvertor:
return DefaultPyTorchDataConvertor(target_tensor_type=torch.float)
def __init__(self, **kwargs) -> None:
super().__init__(**kwargs)
config = self.freqai_info.get("model_training_parameters", {})
self.learning_rate: float = config.get("learning_rate", 3e-4)
self.model_kwargs: Dict[str, Any] = config.get("model_kwargs", {})
self.trainer_kwargs: Dict[str, Any] = config.get("trainer_kwargs", {})
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
n_features = data_dictionary["train_features"].shape[-1]
model = PyTorchMLPModel(
input_dim=n_features,
output_dim=1,
**self.model_kwargs
)
model.to(self.device)
optimizer = torch.optim.AdamW(model.parameters(), lr=self.learning_rate)
criterion = torch.nn.MSELoss()
init_model = self.get_init_model(dk.pair)
trainer = PyTorchModelTrainer(
model=model,
optimizer=optimizer,
criterion=criterion,
device=self.device,
init_model=init_model,
data_convertor=self.data_convertor,
**self.trainer_kwargs,
)
trainer.fit(data_dictionary, self.splits)
return trainer

View File

@ -18,16 +18,20 @@ logger = logging.getLogger(__name__)
class XGBoostClassifier(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
X = data_dictionary["train_features"].to_numpy()

View File

@ -18,16 +18,20 @@ logger = logging.getLogger(__name__)
class XGBoostRFClassifier(BaseClassifierModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
X = data_dictionary["train_features"].to_numpy()

View File

@ -12,16 +12,20 @@ logger = logging.getLogger(__name__)
class XGBoostRFRegressor(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
X = data_dictionary["train_features"]

View File

@ -12,16 +12,20 @@ logger = logging.getLogger(__name__)
class XGBoostRegressor(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
X = data_dictionary["train_features"]

View File

@ -13,16 +13,20 @@ logger = logging.getLogger(__name__)
class XGBoostRegressorMultiTarget(BaseRegressionModel):
"""
User created prediction model. The class needs to override three necessary
functions, predict(), train(), fit(). The class inherits ModelHandler which
has its own DataHandler where data is held, saved, loaded, and managed.
User created prediction model. The class inherits IFreqaiModel, which
means it has full access to all Frequency AI functionality. Typically,
users would use this to override the common `fit()`, `train()`, or
`predict()` methods to add their custom data handling tools or change
various aspects of the training that cannot be configured via the
top level config.json file.
"""
def fit(self, data_dictionary: Dict, dk: FreqaiDataKitchen, **kwargs) -> Any:
"""
User sets up the training and test data to fit their desired model here
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param data_dictionary: the dictionary holding all data for train, test,
labels, weights
:param dk: The datakitchen object for the current coin/model
"""
xgb = XGBRegressor(**self.model_training_parameters)

View File

@ -0,0 +1,67 @@
from abc import ABC, abstractmethod
from typing import List, Optional
import pandas as pd
import torch
class PyTorchDataConvertor(ABC):
"""
This class is responsible for converting `*_features` & `*_labels` pandas dataframes
to pytorch tensors.
"""
@abstractmethod
def convert_x(self, df: pd.DataFrame, device: Optional[str] = None) -> List[torch.Tensor]:
"""
:param df: "*_features" dataframe.
:param device: The device to use for training (e.g. 'cpu', 'cuda').
"""
@abstractmethod
def convert_y(self, df: pd.DataFrame, device: Optional[str] = None) -> List[torch.Tensor]:
"""
:param df: "*_labels" dataframe.
:param device: The device to use for training (e.g. 'cpu', 'cuda').
"""
class DefaultPyTorchDataConvertor(PyTorchDataConvertor):
"""
A default conversion that keeps features dataframe shapes.
"""
def __init__(
self,
target_tensor_type: Optional[torch.dtype] = None,
squeeze_target_tensor: bool = False
):
"""
:param target_tensor_type: type of target tensor, for classification use
torch.long, for regressor use torch.float or torch.double.
:param squeeze_target_tensor: controls the target shape, used for loss functions
that requires 0D or 1D.
"""
self._target_tensor_type = target_tensor_type
self._squeeze_target_tensor = squeeze_target_tensor
def convert_x(self, df: pd.DataFrame, device: Optional[str] = None) -> List[torch.Tensor]:
x = torch.from_numpy(df.values).float()
if device:
x = x.to(device)
return [x]
def convert_y(self, df: pd.DataFrame, device: Optional[str] = None) -> List[torch.Tensor]:
y = torch.from_numpy(df.values)
if self._target_tensor_type:
y = y.to(self._target_tensor_type)
if self._squeeze_target_tensor:
y = y.squeeze()
if device:
y = y.to(device)
return [y]

View File

@ -0,0 +1,97 @@
import logging
from typing import List
import torch
from torch import nn
logger = logging.getLogger(__name__)
class PyTorchMLPModel(nn.Module):
"""
A multi-layer perceptron (MLP) model implemented using PyTorch.
This class mainly serves as a simple example for the integration of PyTorch model's
to freqai. It is not optimized at all and should not be used for production purposes.
:param input_dim: The number of input features. This parameter specifies the number
of features in the input data that the MLP will use to make predictions.
:param output_dim: The number of output classes. This parameter specifies the number
of classes that the MLP will predict.
:param hidden_dim: The number of hidden units in each layer. This parameter controls
the complexity of the MLP and determines how many nonlinear relationships the MLP
can represent. Increasing the number of hidden units can increase the capacity of
the MLP to model complex patterns, but it also increases the risk of overfitting
the training data. Default: 256
:param dropout_percent: The dropout rate for regularization. This parameter specifies
the probability of dropping out a neuron during training to prevent overfitting.
The dropout rate should be tuned carefully to balance between underfitting and
overfitting. Default: 0.2
:param n_layer: The number of layers in the MLP. This parameter specifies the number
of layers in the MLP architecture. Adding more layers to the MLP can increase its
capacity to model complex patterns, but it also increases the risk of overfitting
the training data. Default: 1
:returns: The output of the MLP, with shape (batch_size, output_dim)
"""
def __init__(self, input_dim: int, output_dim: int, **kwargs):
super().__init__()
hidden_dim: int = kwargs.get("hidden_dim", 256)
dropout_percent: int = kwargs.get("dropout_percent", 0.2)
n_layer: int = kwargs.get("n_layer", 1)
self.input_layer = nn.Linear(input_dim, hidden_dim)
self.blocks = nn.Sequential(*[Block(hidden_dim, dropout_percent) for _ in range(n_layer)])
self.output_layer = nn.Linear(hidden_dim, output_dim)
self.relu = nn.ReLU()
self.dropout = nn.Dropout(p=dropout_percent)
def forward(self, tensors: List[torch.Tensor]) -> torch.Tensor:
x: torch.Tensor = tensors[0]
x = self.relu(self.input_layer(x))
x = self.dropout(x)
x = self.blocks(x)
x = self.output_layer(x)
return x
class Block(nn.Module):
"""
A building block for a multi-layer perceptron (MLP).
:param hidden_dim: The number of hidden units in the feedforward network.
:param dropout_percent: The dropout rate for regularization.
:returns: torch.Tensor. with shape (batch_size, hidden_dim)
"""
def __init__(self, hidden_dim: int, dropout_percent: int):
super().__init__()
self.ff = FeedForward(hidden_dim)
self.dropout = nn.Dropout(p=dropout_percent)
self.ln = nn.LayerNorm(hidden_dim)
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.ff(self.ln(x))
x = self.dropout(x)
return x
class FeedForward(nn.Module):
"""
A simple fully-connected feedforward neural network block.
:param hidden_dim: The number of hidden units in the block.
:return: torch.Tensor. with shape (batch_size, hidden_dim)
"""
def __init__(self, hidden_dim: int):
super().__init__()
self.net = nn.Sequential(
nn.Linear(hidden_dim, hidden_dim),
nn.ReLU(),
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)

View File

@ -0,0 +1,208 @@
import logging
import math
from pathlib import Path
from typing import Any, Dict, List, Optional
import pandas as pd
import torch
from torch import nn
from torch.optim import Optimizer
from torch.utils.data import DataLoader, TensorDataset
from freqtrade.freqai.torch.PyTorchDataConvertor import PyTorchDataConvertor
from freqtrade.freqai.torch.PyTorchTrainerInterface import PyTorchTrainerInterface
logger = logging.getLogger(__name__)
class PyTorchModelTrainer(PyTorchTrainerInterface):
def __init__(
self,
model: nn.Module,
optimizer: Optimizer,
criterion: nn.Module,
device: str,
init_model: Dict,
data_convertor: PyTorchDataConvertor,
model_meta_data: Dict[str, Any] = {},
**kwargs
):
"""
:param model: The PyTorch model to be trained.
:param optimizer: The optimizer to use for training.
:param criterion: The loss function to use for training.
:param device: The device to use for training (e.g. 'cpu', 'cuda').
:param init_model: A dictionary containing the initial model/optimizer
state_dict and model_meta_data saved by self.save() method.
:param model_meta_data: Additional metadata about the model (optional).
:param data_convertor: convertor from pd.DataFrame to torch.tensor.
:param max_iters: The number of training iterations to run.
iteration here refers to the number of times we call
self.optimizer.step(). used to calculate n_epochs.
:param batch_size: The size of the batches to use during training.
:param max_n_eval_batches: The maximum number batches to use for evaluation.
"""
self.model = model
self.optimizer = optimizer
self.criterion = criterion
self.model_meta_data = model_meta_data
self.device = device
self.max_iters: int = kwargs.get("max_iters", 100)
self.batch_size: int = kwargs.get("batch_size", 64)
self.max_n_eval_batches: Optional[int] = kwargs.get("max_n_eval_batches", None)
self.data_convertor = data_convertor
if init_model:
self.load_from_checkpoint(init_model)
def fit(self, data_dictionary: Dict[str, pd.DataFrame], splits: List[str]):
"""
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param splits: splits to use in training, splits must contain "train",
optional "test" could be added by setting freqai.data_split_parameters.test_size > 0
in the config file.
- Calculates the predicted output for the batch using the PyTorch model.
- Calculates the loss between the predicted and actual output using a loss function.
- Computes the gradients of the loss with respect to the model's parameters using
backpropagation.
- Updates the model's parameters using an optimizer.
"""
data_loaders_dictionary = self.create_data_loaders_dictionary(data_dictionary, splits)
epochs = self.calc_n_epochs(
n_obs=len(data_dictionary["train_features"]),
batch_size=self.batch_size,
n_iters=self.max_iters
)
for epoch in range(1, epochs + 1):
# training
losses = []
for i, batch_data in enumerate(data_loaders_dictionary["train"]):
for tensor in batch_data:
tensor.to(self.device)
xb = batch_data[:-1]
yb = batch_data[-1]
yb_pred = self.model(xb)
loss = self.criterion(yb_pred, yb)
self.optimizer.zero_grad(set_to_none=True)
loss.backward()
self.optimizer.step()
losses.append(loss.item())
train_loss = sum(losses) / len(losses)
log_message = f"epoch {epoch}/{epochs}: train loss {train_loss:.4f}"
# evaluation
if "test" in splits:
test_loss = self.estimate_loss(
data_loaders_dictionary,
self.max_n_eval_batches,
"test"
)
log_message += f" ; test loss {test_loss:.4f}"
logger.info(log_message)
@torch.no_grad()
def estimate_loss(
self,
data_loader_dictionary: Dict[str, DataLoader],
max_n_eval_batches: Optional[int],
split: str,
) -> float:
self.model.eval()
n_batches = 0
losses = []
for i, batch_data in enumerate(data_loader_dictionary[split]):
if max_n_eval_batches and i > max_n_eval_batches:
n_batches += 1
break
for tensor in batch_data:
tensor.to(self.device)
xb = batch_data[:-1]
yb = batch_data[-1]
yb_pred = self.model(xb)
loss = self.criterion(yb_pred, yb)
losses.append(loss.item())
self.model.train()
return sum(losses) / len(losses)
def create_data_loaders_dictionary(
self,
data_dictionary: Dict[str, pd.DataFrame],
splits: List[str]
) -> Dict[str, DataLoader]:
"""
Converts the input data to PyTorch tensors using a data loader.
"""
data_loader_dictionary = {}
for split in splits:
x = self.data_convertor.convert_x(data_dictionary[f"{split}_features"])
y = self.data_convertor.convert_y(data_dictionary[f"{split}_labels"])
dataset = TensorDataset(*x, *y)
data_loader = DataLoader(
dataset,
batch_size=self.batch_size,
shuffle=True,
drop_last=True,
num_workers=0,
)
data_loader_dictionary[split] = data_loader
return data_loader_dictionary
@staticmethod
def calc_n_epochs(n_obs: int, batch_size: int, n_iters: int) -> int:
"""
Calculates the number of epochs required to reach the maximum number
of iterations specified in the model training parameters.
the motivation here is that `max_iters` is easier to optimize and keep stable,
across different n_obs - the number of data points.
"""
n_batches = math.ceil(n_obs // batch_size)
epochs = math.ceil(n_iters // n_batches)
if epochs <= 10:
logger.warning("User set `max_iters` in such a way that the trainer will only perform "
f" {epochs} epochs. Please consider increasing this value accordingly")
if epochs <= 1:
logger.warning("Epochs set to 1. Please review your `max_iters` value")
epochs = 1
return epochs
def save(self, path: Path):
"""
- Saving any nn.Module state_dict
- Saving model_meta_data, this dict should contain any additional data that the
user needs to store. e.g class_names for classification models.
"""
torch.save({
"model_state_dict": self.model.state_dict(),
"optimizer_state_dict": self.optimizer.state_dict(),
"model_meta_data": self.model_meta_data,
"pytrainer": self
}, path)
def load(self, path: Path):
checkpoint = torch.load(path)
return self.load_from_checkpoint(checkpoint)
def load_from_checkpoint(self, checkpoint: Dict):
"""
when using continual_learning, DataDrawer will load the dictionary
(containing state dicts and model_meta_data) by calling torch.load(path).
you can access this dict from any class that inherits IFreqaiModel by calling
get_init_model method.
"""
self.model.load_state_dict(checkpoint["model_state_dict"])
self.optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
self.model_meta_data = checkpoint["model_meta_data"]
return self

View File

@ -0,0 +1,53 @@
from abc import ABC, abstractmethod
from pathlib import Path
from typing import Dict, List
import pandas as pd
import torch
from torch import nn
class PyTorchTrainerInterface(ABC):
@abstractmethod
def fit(self, data_dictionary: Dict[str, pd.DataFrame], splits: List[str]) -> None:
"""
:param data_dictionary: the dictionary constructed by DataHandler to hold
all the training and test data/labels.
:param splits: splits to use in training, splits must contain "train",
optional "test" could be added by setting freqai.data_split_parameters.test_size > 0
in the config file.
- Calculates the predicted output for the batch using the PyTorch model.
- Calculates the loss between the predicted and actual output using a loss function.
- Computes the gradients of the loss with respect to the model's parameters using
backpropagation.
- Updates the model's parameters using an optimizer.
"""
@abstractmethod
def save(self, path: Path) -> None:
"""
- Saving any nn.Module state_dict
- Saving model_meta_data, this dict should contain any additional data that the
user needs to store. e.g class_names for classification models.
"""
def load(self, path: Path) -> nn.Module:
"""
:param path: path to zip file.
:returns: pytorch model.
"""
checkpoint = torch.load(path)
return self.load_from_checkpoint(checkpoint)
@abstractmethod
def load_from_checkpoint(self, checkpoint: Dict) -> nn.Module:
"""
when using continual_learning, DataDrawer will load the dictionary
(containing state dicts and model_meta_data) by calling torch.load(path).
you can access this dict from any class that inherits IFreqaiModel by calling
get_init_model method.
:checkpoint checkpoint: dict containing the model & optimizer state dicts,
model_meta_data, etc..
"""

View File

View File

@ -21,10 +21,12 @@ from freqtrade.enums import (ExitCheckTuple, ExitType, RPCMessageType, RunMode,
State, TradingMode)
from freqtrade.exceptions import (DependencyException, ExchangeError, InsufficientFundsError,
InvalidOrderException, PricingError)
from freqtrade.exchange import timeframe_to_minutes, timeframe_to_next_date, timeframe_to_seconds
from freqtrade.exchange import (ROUND_DOWN, ROUND_UP, timeframe_to_minutes, timeframe_to_next_date,
timeframe_to_seconds)
from freqtrade.misc import safe_value_fallback, safe_value_fallback2
from freqtrade.mixins import LoggingMixin
from freqtrade.persistence import Order, PairLocks, Trade, init_db
from freqtrade.persistence.key_value_store import set_startup_time
from freqtrade.plugins.pairlistmanager import PairListManager
from freqtrade.plugins.protectionmanager import ProtectionManager
from freqtrade.resolvers import ExchangeResolver, StrategyResolver
@ -181,6 +183,7 @@ class FreqtradeBot(LoggingMixin):
performs startup tasks
"""
migrate_binance_futures_names(self.config)
set_startup_time()
self.rpc.startup_messages(self.config, self.pairlists, self.protections)
# Update older trades with precision and precision mode
@ -853,7 +856,8 @@ class FreqtradeBot(LoggingMixin):
logger.info(f"Canceling stoploss on exchange for {trade}")
co = self.exchange.cancel_stoploss_order_with_result(
trade.stoploss_order_id, trade.pair, trade.amount)
trade.update_order(co)
self.update_trade_state(trade, trade.stoploss_order_id, co, stoploss_order=True)
# Reset stoploss order id.
trade.stoploss_order_id = None
except InvalidOrderException:
@ -945,7 +949,7 @@ class FreqtradeBot(LoggingMixin):
return enter_limit_requested, stake_amount, leverage
def _notify_enter(self, trade: Trade, order: Order, order_type: Optional[str] = None,
def _notify_enter(self, trade: Trade, order: Order, order_type: str,
fill: bool = False, sub_trade: bool = False) -> None:
"""
Sends rpc notification when a entry order occurred.
@ -1171,7 +1175,8 @@ class FreqtradeBot(LoggingMixin):
logger.warning('Unable to fetch stoploss order: %s', exception)
if stoploss_order:
trade.update_order(stoploss_order)
self.update_trade_state(trade, trade.stoploss_order_id, stoploss_order,
stoploss_order=True)
# We check if stoploss order is fulfilled
if stoploss_order and stoploss_order['status'] in ('closed', 'triggered'):
@ -1235,7 +1240,9 @@ class FreqtradeBot(LoggingMixin):
:param order: Current on exchange stoploss order
:return: None
"""
stoploss_norm = self.exchange.price_to_precision(trade.pair, trade.stoploss_or_liquidation)
stoploss_norm = self.exchange.price_to_precision(
trade.pair, trade.stoploss_or_liquidation,
rounding_mode=ROUND_DOWN if trade.is_short else ROUND_UP)
if self.exchange.stoploss_adjust(stoploss_norm, order, side=trade.exit_side):
# we check if the update is necessary
@ -1478,8 +1485,8 @@ class FreqtradeBot(LoggingMixin):
return False
try:
order = self.exchange.cancel_order_with_result(order['id'], trade.pair,
trade.amount)
order = self.exchange.cancel_order_with_result(
order['id'], trade.pair, trade.amount)
except InvalidOrderException:
logger.exception(
f"Could not cancel {trade.exit_side} order {trade.open_order_id}")
@ -1491,17 +1498,18 @@ class FreqtradeBot(LoggingMixin):
# Order might be filled above in odd timing issues.
if order.get('status') in ('canceled', 'cancelled'):
trade.exit_reason = None
trade.open_order_id = None
else:
trade.exit_reason = exit_reason_prev
cancelled = True
else:
reason = constants.CANCEL_REASON['CANCELLED_ON_EXCHANGE']
trade.exit_reason = None
trade.open_order_id = None
self.update_trade_state(trade, trade.open_order_id, order)
logger.info(f'{trade.exit_side.capitalize()} order {reason} for {trade}.')
trade.open_order_id = None
trade.close_rate = None
trade.close_rate_requested = None
@ -1778,11 +1786,11 @@ class FreqtradeBot(LoggingMixin):
return False
# Update trade with order values
if not stoploss_order:
logger.info(f'Found open order for {trade}')
try:
order = action_order or self.exchange.fetch_order_or_stoploss_order(order_id,
trade.pair,
stoploss_order)
order = action_order or self.exchange.fetch_order_or_stoploss_order(
order_id, trade.pair, stoploss_order)
except InvalidOrderException as exception:
logger.warning('Unable to fetch order %s: %s', order_id, exception)
return False
@ -1847,7 +1855,7 @@ class FreqtradeBot(LoggingMixin):
self.handle_protections(trade.pair, trade.trade_direction)
elif send_msg and not trade.open_order_id and not stoploss_order:
# Enter fill
self._notify_enter(trade, order, fill=True, sub_trade=sub_trade)
self._notify_enter(trade, order, order.order_type, fill=True, sub_trade=sub_trade)
def handle_protections(self, pair: str, side: LongShort) -> None:
# Lock pair for one candle to prevent immediate rebuys

View File

@ -23,6 +23,8 @@ logger = logging.getLogger(__name__)
NON_OPT_PARAM_APPENDIX = " # value loaded from strategy"
HYPER_PARAMS_FILE_FORMAT = rapidjson.NM_NATIVE | rapidjson.NM_NAN
def hyperopt_serializer(x):
if isinstance(x, np.integer):
@ -76,9 +78,18 @@ class HyperoptTools():
with filename.open('w') as f:
rapidjson.dump(final_params, f, indent=2,
default=hyperopt_serializer,
number_mode=rapidjson.NM_NATIVE | rapidjson.NM_NAN
number_mode=HYPER_PARAMS_FILE_FORMAT
)
@staticmethod
def load_params(filename: Path) -> Dict:
"""
Load parameters from file
"""
with filename.open('r') as f:
params = rapidjson.load(f, number_mode=HYPER_PARAMS_FILE_FORMAT)
return params
@staticmethod
def try_export_params(config: Config, strategy_name: str, params: Dict):
if params.get(FTHYPT_FILEVERSION, 1) >= 2 and not config.get('disableparamexport', False):
@ -189,7 +200,7 @@ class HyperoptTools():
for s in ['buy', 'sell', 'protection',
'roi', 'stoploss', 'trailing', 'max_open_trades']:
HyperoptTools._params_update_for_json(result_dict, params, non_optimized, s)
print(rapidjson.dumps(result_dict, default=str, number_mode=rapidjson.NM_NATIVE))
print(rapidjson.dumps(result_dict, default=str, number_mode=HYPER_PARAMS_FILE_FORMAT))
else:
HyperoptTools._params_pretty_print(params, 'buy', "Buy hyperspace params:",

View File

@ -865,6 +865,11 @@ def show_backtest_result(strategy: str, results: Dict[str, Any], stake_currency:
print(' BACKTESTING REPORT '.center(len(table.splitlines()[0]), '='))
print(table)
table = text_table_bt_results(results['left_open_trades'], stake_currency=stake_currency)
if isinstance(table, str) and len(table) > 0:
print(' LEFT OPEN TRADES REPORT '.center(len(table.splitlines()[0]), '='))
print(table)
if (results.get('results_per_enter_tag') is not None
or results.get('results_per_buy_tag') is not None):
# results_per_buy_tag is deprecated and should be removed 2 versions after short golive.
@ -884,11 +889,6 @@ def show_backtest_result(strategy: str, results: Dict[str, Any], stake_currency:
print(' EXIT REASON STATS '.center(len(table.splitlines()[0]), '='))
print(table)
table = text_table_bt_results(results['left_open_trades'], stake_currency=stake_currency)
if isinstance(table, str) and len(table) > 0:
print(' LEFT OPEN TRADES REPORT '.center(len(table.splitlines()[0]), '='))
print(table)
for period in backtest_breakdown:
days_breakdown_stats = generate_periodic_breakdown_stats(
trade_list=results['trades'], period=period)
@ -917,11 +917,11 @@ def show_backtest_results(config: Config, backtest_stats: Dict):
strategy, results, stake_currency,
config.get('backtest_breakdown', []))
if len(backtest_stats['strategy']) > 1:
if len(backtest_stats['strategy']) > 0:
# Print Strategy summary table
table = text_table_strategy(backtest_stats['strategy_comparison'], stake_currency)
print(f"{results['backtest_start']} -> {results['backtest_end']} |"
print(f"Backtested {results['backtest_start']} -> {results['backtest_end']} |"
f" Max open trades : {results['max_open_trades']}")
print(' STRATEGY SUMMARY '.center(len(table.splitlines()[0]), '='))
print(table)

View File

@ -1,5 +1,6 @@
# flake8: noqa: F401
from freqtrade.persistence.key_value_store import KeyStoreKeys, KeyValueStore
from freqtrade.persistence.models import init_db
from freqtrade.persistence.pairlock_middleware import PairLocks
from freqtrade.persistence.trade_model import LocalTrade, Order, Trade

View File

@ -0,0 +1,179 @@
from datetime import datetime, timezone
from enum import Enum
from typing import ClassVar, Optional, Union
from sqlalchemy import String
from sqlalchemy.orm import Mapped, mapped_column
from freqtrade.persistence.base import ModelBase, SessionType
ValueTypes = Union[str, datetime, float, int]
class ValueTypesEnum(str, Enum):
STRING = 'str'
DATETIME = 'datetime'
FLOAT = 'float'
INT = 'int'
class KeyStoreKeys(str, Enum):
BOT_START_TIME = 'bot_start_time'
STARTUP_TIME = 'startup_time'
class _KeyValueStoreModel(ModelBase):
"""
Pair Locks database model.
"""
__tablename__ = 'KeyValueStore'
session: ClassVar[SessionType]
id: Mapped[int] = mapped_column(primary_key=True)
key: Mapped[KeyStoreKeys] = mapped_column(String(25), nullable=False, index=True)
value_type: Mapped[ValueTypesEnum] = mapped_column(String(20), nullable=False)
string_value: Mapped[Optional[str]]
datetime_value: Mapped[Optional[datetime]]
float_value: Mapped[Optional[float]]
int_value: Mapped[Optional[int]]
class KeyValueStore():
"""
Generic bot-wide, persistent key-value store
Can be used to store generic values, e.g. very first bot startup time.
Supports the types str, datetime, float and int.
"""
@staticmethod
def store_value(key: KeyStoreKeys, value: ValueTypes) -> None:
"""
Store the given value for the given key.
:param key: Key to store the value for - can be used in get-value to retrieve the key
:param value: Value to store - can be str, datetime, float or int
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key).first()
if kv is None:
kv = _KeyValueStoreModel(key=key)
if isinstance(value, str):
kv.value_type = ValueTypesEnum.STRING
kv.string_value = value
elif isinstance(value, datetime):
kv.value_type = ValueTypesEnum.DATETIME
kv.datetime_value = value
elif isinstance(value, float):
kv.value_type = ValueTypesEnum.FLOAT
kv.float_value = value
elif isinstance(value, int):
kv.value_type = ValueTypesEnum.INT
kv.int_value = value
else:
raise ValueError(f'Unknown value type {kv.value_type}')
_KeyValueStoreModel.session.add(kv)
_KeyValueStoreModel.session.commit()
@staticmethod
def delete_value(key: KeyStoreKeys) -> None:
"""
Delete the value for the given key.
:param key: Key to delete the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key).first()
if kv is not None:
_KeyValueStoreModel.session.delete(kv)
_KeyValueStoreModel.session.commit()
@staticmethod
def get_value(key: KeyStoreKeys) -> Optional[ValueTypes]:
"""
Get the value for the given key.
:param key: Key to get the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key).first()
if kv is None:
return None
if kv.value_type == ValueTypesEnum.STRING:
return kv.string_value
if kv.value_type == ValueTypesEnum.DATETIME and kv.datetime_value is not None:
return kv.datetime_value.replace(tzinfo=timezone.utc)
if kv.value_type == ValueTypesEnum.FLOAT:
return kv.float_value
if kv.value_type == ValueTypesEnum.INT:
return kv.int_value
# This should never happen unless someone messed with the database manually
raise ValueError(f'Unknown value type {kv.value_type}') # pragma: no cover
@staticmethod
def get_string_value(key: KeyStoreKeys) -> Optional[str]:
"""
Get the value for the given key.
:param key: Key to get the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key,
_KeyValueStoreModel.value_type == ValueTypesEnum.STRING).first()
if kv is None:
return None
return kv.string_value
@staticmethod
def get_datetime_value(key: KeyStoreKeys) -> Optional[datetime]:
"""
Get the value for the given key.
:param key: Key to get the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key,
_KeyValueStoreModel.value_type == ValueTypesEnum.DATETIME).first()
if kv is None or kv.datetime_value is None:
return None
return kv.datetime_value.replace(tzinfo=timezone.utc)
@staticmethod
def get_float_value(key: KeyStoreKeys) -> Optional[float]:
"""
Get the value for the given key.
:param key: Key to get the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key,
_KeyValueStoreModel.value_type == ValueTypesEnum.FLOAT).first()
if kv is None:
return None
return kv.float_value
@staticmethod
def get_int_value(key: KeyStoreKeys) -> Optional[int]:
"""
Get the value for the given key.
:param key: Key to get the value for
"""
kv = _KeyValueStoreModel.session.query(_KeyValueStoreModel).filter(
_KeyValueStoreModel.key == key,
_KeyValueStoreModel.value_type == ValueTypesEnum.INT).first()
if kv is None:
return None
return kv.int_value
def set_startup_time():
"""
sets bot_start_time to the first trade open date - or "now" on new databases.
sets startup_time to "now"
"""
st = KeyValueStore.get_value('bot_start_time')
if st is None:
from freqtrade.persistence import Trade
t = Trade.session.query(Trade).order_by(Trade.open_date.asc()).first()
if t is not None:
KeyValueStore.store_value('bot_start_time', t.open_date_utc)
else:
KeyValueStore.store_value('bot_start_time', datetime.now(timezone.utc))
KeyValueStore.store_value('startup_time', datetime.now(timezone.utc))

View File

@ -13,6 +13,7 @@ from sqlalchemy.pool import StaticPool
from freqtrade.exceptions import OperationalException
from freqtrade.persistence.base import ModelBase
from freqtrade.persistence.key_value_store import _KeyValueStoreModel
from freqtrade.persistence.migrations import check_migrate
from freqtrade.persistence.pairlock import PairLock
from freqtrade.persistence.trade_model import Order, Trade
@ -76,6 +77,7 @@ def init_db(db_url: str) -> None:
bind=engine, autoflush=False), scopefunc=get_request_or_thread_id)
Order.session = Trade.session
PairLock.session = Trade.session
_KeyValueStoreModel.session = Trade.session
previous_tables = inspect(engine).get_table_names()
ModelBase.metadata.create_all(engine)

View File

@ -9,13 +9,14 @@ from typing import Any, ClassVar, Dict, List, Optional, Sequence, cast
from sqlalchemy import (Enum, Float, ForeignKey, Integer, ScalarResult, Select, String,
UniqueConstraint, desc, func, select)
from sqlalchemy.orm import Mapped, lazyload, mapped_column, relationship
from sqlalchemy.orm import Mapped, lazyload, mapped_column, relationship, validates
from freqtrade.constants import (DATETIME_PRINT_FORMAT, MATH_CLOSE_PREC, NON_OPEN_EXCHANGE_STATES,
BuySell, LongShort)
from freqtrade.constants import (CUSTOM_TAG_MAX_LENGTH, DATETIME_PRINT_FORMAT, MATH_CLOSE_PREC,
NON_OPEN_EXCHANGE_STATES, BuySell, LongShort)
from freqtrade.enums import ExitType, TradingMode
from freqtrade.exceptions import DependencyException, OperationalException
from freqtrade.exchange import amount_to_contract_precision, price_to_precision
from freqtrade.exchange import (ROUND_DOWN, ROUND_UP, amount_to_contract_precision,
price_to_precision)
from freqtrade.leverage import interest
from freqtrade.persistence.base import ModelBase, SessionType
from freqtrade.util import FtPrecise
@ -597,7 +598,8 @@ class LocalTrade():
"""
Method used internally to set self.stop_loss.
"""
stop_loss_norm = price_to_precision(stop_loss, self.price_precision, self.precision_mode)
stop_loss_norm = price_to_precision(stop_loss, self.price_precision, self.precision_mode,
rounding_mode=ROUND_DOWN if self.is_short else ROUND_UP)
if not self.stop_loss:
self.initial_stop_loss = stop_loss_norm
self.stop_loss = stop_loss_norm
@ -628,7 +630,8 @@ class LocalTrade():
if self.initial_stop_loss_pct is None or refresh:
self.__set_stop_loss(new_loss, stoploss)
self.initial_stop_loss = price_to_precision(
new_loss, self.price_precision, self.precision_mode)
new_loss, self.price_precision, self.precision_mode,
rounding_mode=ROUND_DOWN if self.is_short else ROUND_UP)
self.initial_stop_loss_pct = -1 * abs(stoploss)
# evaluate if the stop loss needs to be updated
@ -692,21 +695,24 @@ class LocalTrade():
else:
logger.warning(
f'Got different open_order_id {self.open_order_id} != {order.order_id}')
elif order.ft_order_side == 'stoploss' and order.status not in ('open', ):
self.stoploss_order_id = None
self.close_rate_requested = self.stop_loss
self.exit_reason = ExitType.STOPLOSS_ON_EXCHANGE.value
if self.is_open:
logger.info(f'{order.order_type.upper()} is hit for {self}.')
else:
raise ValueError(f'Unknown order type: {order.order_type}')
if order.ft_order_side != self.entry_side:
amount_tr = amount_to_contract_precision(self.amount, self.amount_precision,
self.precision_mode, self.contract_size)
if isclose(order.safe_amount_after_fee, amount_tr, abs_tol=MATH_CLOSE_PREC):
self.close(order.safe_price)
else:
self.recalc_trade_from_orders()
elif order.ft_order_side == 'stoploss' and order.status not in ('canceled', 'open'):
self.stoploss_order_id = None
self.close_rate_requested = self.stop_loss
self.exit_reason = ExitType.STOPLOSS_ON_EXCHANGE.value
if self.is_open:
logger.info(f'{order.order_type.upper()} is hit for {self}.')
self.close(order.safe_price)
else:
raise ValueError(f'Unknown order type: {order.order_type}')
Trade.commit()
def close(self, rate: float, *, show_msg: bool = True) -> None:
@ -1253,11 +1259,13 @@ class Trade(ModelBase, LocalTrade):
Float(), nullable=True, default=0.0) # type: ignore
# Lowest price reached
min_rate: Mapped[Optional[float]] = mapped_column(Float(), nullable=True) # type: ignore
exit_reason: Mapped[Optional[str]] = mapped_column(String(100), nullable=True) # type: ignore
exit_reason: Mapped[Optional[str]] = mapped_column(
String(CUSTOM_TAG_MAX_LENGTH), nullable=True) # type: ignore
exit_order_status: Mapped[Optional[str]] = mapped_column(
String(100), nullable=True) # type: ignore
strategy: Mapped[Optional[str]] = mapped_column(String(100), nullable=True) # type: ignore
enter_tag: Mapped[Optional[str]] = mapped_column(String(100), nullable=True) # type: ignore
enter_tag: Mapped[Optional[str]] = mapped_column(
String(CUSTOM_TAG_MAX_LENGTH), nullable=True) # type: ignore
timeframe: Mapped[Optional[int]] = mapped_column(Integer, nullable=True) # type: ignore
trading_mode: Mapped[TradingMode] = mapped_column(
@ -1287,6 +1295,13 @@ class Trade(ModelBase, LocalTrade):
self.realized_profit = 0
self.recalc_open_trade_value()
@validates('enter_tag', 'exit_reason')
def validate_string_len(self, key, value):
max_len = getattr(self.__class__, key).prop.columns[0].type.length
if value and len(value) > max_len:
return value[:max_len]
return value
def delete(self) -> None:
for order in self.orders:

View File

@ -6,6 +6,7 @@ from typing import Any, Dict, Optional
from freqtrade.constants import Config
from freqtrade.exceptions import OperationalException
from freqtrade.exchange import ROUND_UP
from freqtrade.exchange.types import Ticker
from freqtrade.plugins.pairlist.IPairList import IPairList
@ -61,9 +62,10 @@ class PrecisionFilter(IPairList):
stop_price = ticker['last'] * self._stoploss
# Adjust stop-prices to precision
sp = self._exchange.price_to_precision(pair, stop_price)
sp = self._exchange.price_to_precision(pair, stop_price, rounding_mode=ROUND_UP)
stop_gap_price = self._exchange.price_to_precision(pair, stop_price * 0.99)
stop_gap_price = self._exchange.price_to_precision(pair, stop_price * 0.99,
rounding_mode=ROUND_UP)
logger.debug(f"{pair} - {sp} : {stop_gap_price}")
if sp <= stop_gap_price:

View File

@ -108,6 +108,8 @@ class Profit(BaseModel):
max_drawdown: float
max_drawdown_abs: float
trading_volume: Optional[float]
bot_start_timestamp: int
bot_start_date: str
class SellReason(BaseModel):

View File

@ -55,7 +55,7 @@ class UvicornServer(uvicorn.Server):
@contextlib.contextmanager
def run_in_thread(self):
self.thread = threading.Thread(target=self.run)
self.thread = threading.Thread(target=self.run, name='FTUvicorn')
self.thread.start()
while not self.started:
time.sleep(1e-3)

View File

@ -26,7 +26,7 @@ from freqtrade.exceptions import ExchangeError, PricingError
from freqtrade.exchange import timeframe_to_minutes, timeframe_to_msecs
from freqtrade.loggers import bufferHandler
from freqtrade.misc import decimals_per_coin, shorten_date
from freqtrade.persistence import Order, PairLocks, Trade
from freqtrade.persistence import KeyStoreKeys, KeyValueStore, Order, PairLocks, Trade
from freqtrade.persistence.models import PairLock
from freqtrade.plugins.pairlist.pairlist_helpers import expand_pairlist
from freqtrade.rpc.fiat_convert import CryptoToFiatConverter
@ -543,6 +543,7 @@ class RPC:
first_date = trades[0].open_date if trades else None
last_date = trades[-1].open_date if trades else None
num = float(len(durations) or 1)
bot_start = KeyValueStore.get_datetime_value(KeyStoreKeys.BOT_START_TIME)
return {
'profit_closed_coin': profit_closed_coin_sum,
'profit_closed_percent_mean': round(profit_closed_ratio_mean * 100, 2),
@ -576,6 +577,8 @@ class RPC:
'max_drawdown': max_drawdown,
'max_drawdown_abs': max_drawdown_abs,
'trading_volume': trading_volume,
'bot_start_timestamp': int(bot_start.timestamp() * 1000) if bot_start else 0,
'bot_start_date': bot_start.strftime(DATETIME_PRINT_FORMAT) if bot_start else '',
}
def _rpc_balance(self, stake_currency: str, fiat_display_currency: str) -> Dict:
@ -1193,6 +1196,7 @@ class RPC:
from freqtrade.resolvers.strategy_resolver import StrategyResolver
strategy = StrategyResolver.load_strategy(config)
strategy.dp = DataProvider(config, exchange=exchange, pairlists=None)
strategy.ft_bot_start()
df_analyzed = strategy.analyze_ticker(_data[pair], {'pair': pair})

View File

@ -52,7 +52,7 @@ class __RPCBuyMsgBase(RPCSendMsgBase):
direction: str
limit: float
open_rate: float
order_type: Optional[str] # TODO: why optional??
order_type: str
stake_amount: float
stake_currency: str
fiat_currency: Optional[str]

View File

@ -819,7 +819,7 @@ class Telegram(RPCHandler):
best_pair = stats['best_pair']
best_pair_profit_ratio = stats['best_pair_profit_ratio']
if stats['trade_count'] == 0:
markdown_msg = 'No trades yet.'
markdown_msg = f"No trades yet.\n*Bot started:* `{stats['bot_start_date']}`"
else:
# Message to display
if stats['closed_trade_count'] > 0:
@ -838,6 +838,7 @@ class Telegram(RPCHandler):
f"({profit_all_percent} \N{GREEK CAPITAL LETTER SIGMA}%)`\n"
f"∙ `{round_coin_value(profit_all_fiat, fiat_disp_cur)}`\n"
f"*Total Trade Count:* `{trade_count}`\n"
f"*Bot started:* `{stats['bot_start_date']}`\n"
f"*{'First Trade opened' if not timescale else 'Showing Profit since'}:* "
f"`{first_trade_date}`\n"
f"*Latest Trade opened:* `{latest_trade_date}`\n"

View File

@ -8,7 +8,7 @@ from typing import Any, Dict, Iterator, List, Optional, Tuple, Type, Union
from freqtrade.constants import Config
from freqtrade.exceptions import OperationalException
from freqtrade.misc import deep_merge_dicts, json_load
from freqtrade.misc import deep_merge_dicts
from freqtrade.optimize.hyperopt_tools import HyperoptTools
from freqtrade.strategy.parameters import BaseParameter
@ -124,8 +124,7 @@ class HyperStrategyMixin:
if filename.is_file():
logger.info(f"Loading parameters from file {filename}")
try:
with filename.open('r') as f:
params = json_load(f)
params = HyperoptTools.load_params(filename)
if params.get('strategy_name') != self.__class__.__name__:
raise OperationalException('Invalid parameter file provided.')
return params

View File

@ -10,7 +10,7 @@ from typing import Dict, List, Optional, Tuple, Union
import arrow
from pandas import DataFrame
from freqtrade.constants import Config, IntOrInf, ListPairsWithTimeframes
from freqtrade.constants import CUSTOM_TAG_MAX_LENGTH, Config, IntOrInf, ListPairsWithTimeframes
from freqtrade.data.dataprovider import DataProvider
from freqtrade.enums import (CandleType, ExitCheckTuple, ExitType, MarketDirection, RunMode,
SignalDirection, SignalTagType, SignalType, TradingMode)
@ -27,7 +27,6 @@ from freqtrade.wallets import Wallets
logger = logging.getLogger(__name__)
CUSTOM_EXIT_MAX_LENGTH = 64
class IStrategy(ABC, HyperStrategyMixin):
@ -1118,11 +1117,11 @@ class IStrategy(ABC, HyperStrategyMixin):
exit_signal = ExitType.CUSTOM_EXIT
if isinstance(reason_cust, str):
custom_reason = reason_cust
if len(reason_cust) > CUSTOM_EXIT_MAX_LENGTH:
if len(reason_cust) > CUSTOM_TAG_MAX_LENGTH:
logger.warning(f'Custom exit reason returned from '
f'custom_exit is too long and was trimmed'
f'to {CUSTOM_EXIT_MAX_LENGTH} characters.')
custom_reason = reason_cust[:CUSTOM_EXIT_MAX_LENGTH]
f'to {CUSTOM_TAG_MAX_LENGTH} characters.')
custom_reason = reason_cust[:CUSTOM_TAG_MAX_LENGTH]
else:
custom_reason = ''
if (

View File

@ -223,6 +223,7 @@ class FreqaiExampleHybridStrategy(IStrategy):
:param metadata: metadata of current pair
usage example: dataframe["&-target"] = dataframe["close"].shift(-1) / dataframe["close"]
"""
self.freqai.class_names = ["down", "up"]
dataframe['&s-up_or_down'] = np.where(dataframe["close"].shift(-50) >
dataframe["close"], 'up', 'down')

View File

@ -7,10 +7,10 @@
-r docs/requirements-docs.txt
coveralls==3.3.1
ruff==0.0.259
mypy==1.1.1
pre-commit==3.2.1
pytest==7.2.2
ruff==0.0.261
mypy==1.2.0
pre-commit==3.2.2
pytest==7.3.0
pytest-asyncio==0.21.0
pytest-cov==4.0.0
pytest-mock==3.10.0
@ -22,11 +22,11 @@ time-machine==2.9.0
httpx==0.23.3
# Convert jupyter notebooks to markdown documents
nbconvert==7.2.10
nbconvert==7.3.1
# mypy types
types-cachetools==5.3.0.4
types-cachetools==5.3.0.5
types-filelock==3.2.7
types-requests==2.28.11.16
types-tabulate==0.9.0.1
types-python-dateutil==2.8.19.10
types-requests==2.28.11.17
types-tabulate==0.9.0.2
types-python-dateutil==2.8.19.12

View File

@ -7,5 +7,5 @@ scikit-learn==1.1.3
joblib==1.2.0
catboost==1.1.1; platform_machine != 'aarch64' and 'arm' not in platform_machine and python_version < '3.11'
lightgbm==3.3.5
xgboost==1.7.4
tensorboard==2.12.0
xgboost==1.7.5
tensorboard==2.12.1

View File

@ -5,5 +5,5 @@
scipy==1.10.1
scikit-learn==1.1.3
scikit-optimize==0.9.0
filelock==3.10.6
filelock==3.11.0
progressbar2==4.2.0

View File

@ -1,4 +1,4 @@
# Include all requirements to run the bot.
-r requirements.txt
plotly==5.13.1
plotly==5.14.1

View File

@ -2,10 +2,10 @@ numpy==1.24.2
pandas==1.5.3
pandas-ta==0.3.14b
ccxt==3.0.37
ccxt==3.0.59
cryptography==40.0.1
aiohttp==3.8.4
SQLAlchemy==2.0.7
SQLAlchemy==2.0.9
python-telegram-bot==13.15
arrow==1.2.3
cachetools==4.2.2
@ -28,7 +28,7 @@ py_find_1st==1.1.5
# Load ticker files 30% faster
python-rapidjson==1.10
# Properly format api responses
orjson==3.8.8
orjson==3.8.10
# Notify systemd
sdnotify==0.3.2
@ -50,10 +50,10 @@ prompt-toolkit==3.0.38
python-dateutil==2.8.2
#Futures
schedule==1.1.0
schedule==1.2.0
#WS Messages
websockets==10.4
websockets==11.0.1
janus==1.0.0
ast-comments==1.0.1

View File

@ -59,7 +59,7 @@ setup(
install_requires=[
# from requirements.txt
'ccxt>=2.6.26',
'SQLAlchemy',
'SQLAlchemy>=2.0.6',
'python-telegram-bot>=13.4',
'arrow>=0.17.0',
'cachetools',

View File

@ -85,7 +85,7 @@ function updateenv() {
if [[ $REPLY =~ ^[Yy]$ ]]
then
REQUIREMENTS_FREQAI="-r requirements-freqai.txt --use-pep517"
read -p "Do you also want dependencies for freqai-rl (~700mb additional space required) [y/N]? "
read -p "Do you also want dependencies for freqai-rl or PyTorch (~700mb additional space required) [y/N]? "
if [[ $REPLY =~ ^[Yy]$ ]]
then
REQUIREMENTS_FREQAI="-r requirements-freqai-rl.txt"

View File

@ -48,7 +48,7 @@ def test_create_stoploss_order_binance(default_conf, mocker, limitratio, expecte
default_conf['margin_mode'] = MarginMode.ISOLATED
default_conf['trading_mode'] = trademode
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')
@ -127,7 +127,7 @@ def test_create_stoploss_order_dry_run_binance(default_conf, mocker):
order_type = 'stop_loss_limit'
default_conf['dry_run'] = True
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'binance')

View File

@ -8,6 +8,7 @@ from unittest.mock import MagicMock, Mock, PropertyMock, patch
import arrow
import ccxt
import pytest
from ccxt import DECIMAL_PLACES, ROUND, ROUND_UP, TICK_SIZE, TRUNCATE
from pandas import DataFrame
from freqtrade.enums import CandleType, MarginMode, TradingMode
@ -315,35 +316,54 @@ def test_amount_to_precision(amount, precision_mode, precision, expected,):
assert amount_to_precision(amount, precision, precision_mode) == expected
@pytest.mark.parametrize("price,precision_mode,precision,expected", [
(2.34559, 2, 4, 2.3456),
(2.34559, 2, 5, 2.34559),
(2.34559, 2, 3, 2.346),
(2.9999, 2, 3, 3.000),
(2.9909, 2, 3, 2.991),
# Tests for Tick_size
(2.34559, 4, 0.0001, 2.3456),
(2.34559, 4, 0.00001, 2.34559),
(2.34559, 4, 0.001, 2.346),
(2.9999, 4, 0.001, 3.000),
(2.9909, 4, 0.001, 2.991),
(2.9909, 4, 0.005, 2.995),
(2.9973, 4, 0.005, 3.0),
(2.9977, 4, 0.005, 3.0),
(234.43, 4, 0.5, 234.5),
(234.53, 4, 0.5, 235.0),
(0.891534, 4, 0.0001, 0.8916),
(64968.89, 4, 0.01, 64968.89),
(0.000000003483, 4, 1e-12, 0.000000003483),
@pytest.mark.parametrize("price,precision_mode,precision,expected,rounding_mode", [
# Tests for DECIMAL_PLACES, ROUND_UP
(2.34559, 2, 4, 2.3456, ROUND_UP),
(2.34559, 2, 5, 2.34559, ROUND_UP),
(2.34559, 2, 3, 2.346, ROUND_UP),
(2.9999, 2, 3, 3.000, ROUND_UP),
(2.9909, 2, 3, 2.991, ROUND_UP),
# Tests for DECIMAL_PLACES, ROUND
(2.345600000000001, DECIMAL_PLACES, 4, 2.3456, ROUND),
(2.345551, DECIMAL_PLACES, 4, 2.3456, ROUND),
(2.49, DECIMAL_PLACES, 0, 2., ROUND),
(2.51, DECIMAL_PLACES, 0, 3., ROUND),
(5.1, DECIMAL_PLACES, -1, 10., ROUND),
(4.9, DECIMAL_PLACES, -1, 0., ROUND),
# Tests for TICK_SIZE, ROUND_UP
(2.34559, TICK_SIZE, 0.0001, 2.3456, ROUND_UP),
(2.34559, TICK_SIZE, 0.00001, 2.34559, ROUND_UP),
(2.34559, TICK_SIZE, 0.001, 2.346, ROUND_UP),
(2.9999, TICK_SIZE, 0.001, 3.000, ROUND_UP),
(2.9909, TICK_SIZE, 0.001, 2.991, ROUND_UP),
(2.9909, TICK_SIZE, 0.005, 2.995, ROUND_UP),
(2.9973, TICK_SIZE, 0.005, 3.0, ROUND_UP),
(2.9977, TICK_SIZE, 0.005, 3.0, ROUND_UP),
(234.43, TICK_SIZE, 0.5, 234.5, ROUND_UP),
(234.53, TICK_SIZE, 0.5, 235.0, ROUND_UP),
(0.891534, TICK_SIZE, 0.0001, 0.8916, ROUND_UP),
(64968.89, TICK_SIZE, 0.01, 64968.89, ROUND_UP),
(0.000000003483, TICK_SIZE, 1e-12, 0.000000003483, ROUND_UP),
# Tests for TICK_SIZE, ROUND
(2.49, TICK_SIZE, 1., 2., ROUND),
(2.51, TICK_SIZE, 1., 3., ROUND),
(2.000000051, TICK_SIZE, 0.0000001, 2.0000001, ROUND),
(2.000000049, TICK_SIZE, 0.0000001, 2., ROUND),
(2.9909, TICK_SIZE, 0.005, 2.990, ROUND),
(2.9973, TICK_SIZE, 0.005, 2.995, ROUND),
(2.9977, TICK_SIZE, 0.005, 3.0, ROUND),
(234.24, TICK_SIZE, 0.5, 234., ROUND),
(234.26, TICK_SIZE, 0.5, 234.5, ROUND),
# Tests for TRUNCATTE
(2.34559, 2, 4, 2.3455, TRUNCATE),
(2.34559, 2, 5, 2.34559, TRUNCATE),
(2.34559, 2, 3, 2.345, TRUNCATE),
(2.9999, 2, 3, 2.999, TRUNCATE),
(2.9909, 2, 3, 2.990, TRUNCATE),
])
def test_price_to_precision(price, precision_mode, precision, expected):
# digits counting mode
# DECIMAL_PLACES = 2
# SIGNIFICANT_DIGITS = 3
# TICK_SIZE = 4
assert price_to_precision(price, precision, precision_mode) == expected
def test_price_to_precision(price, precision_mode, precision, expected, rounding_mode):
assert price_to_precision(
price, precision, precision_mode, rounding_mode=rounding_mode) == expected
@pytest.mark.parametrize("price,precision_mode,precision,expected", [
@ -417,7 +437,7 @@ def test__get_stake_amount_limit(mocker, default_conf) -> None:
}
mocker.patch(f'{EXMS}.markets', PropertyMock(return_value=markets))
result = exchange.get_min_pair_stake_amount('ETH/BTC', 2, stoploss)
expected_result = 2 * 2 * (1 + 0.05) / (1 - abs(stoploss))
expected_result = 2 * 2 * (1 + 0.05)
assert pytest.approx(result) == expected_result
# With Leverage
result = exchange.get_min_pair_stake_amount('ETH/BTC', 2, stoploss, 5.0)
@ -426,14 +446,14 @@ def test__get_stake_amount_limit(mocker, default_conf) -> None:
result = exchange.get_max_pair_stake_amount('ETH/BTC', 2)
assert result == 20000
# min amount and cost are set (cost is minimal)
# min amount and cost are set (cost is minimal and therefore ignored)
markets["ETH/BTC"]["limits"] = {
'cost': {'min': 2, 'max': None},
'amount': {'min': 2, 'max': None},
}
mocker.patch(f'{EXMS}.markets', PropertyMock(return_value=markets))
result = exchange.get_min_pair_stake_amount('ETH/BTC', 2, stoploss)
expected_result = max(2, 2 * 2) * (1 + 0.05) / (1 - abs(stoploss))
expected_result = max(2, 2 * 2) * (1 + 0.05)
assert pytest.approx(result) == expected_result
# With Leverage
result = exchange.get_min_pair_stake_amount('ETH/BTC', 2, stoploss, 10)
@ -476,6 +496,9 @@ def test__get_stake_amount_limit(mocker, default_conf) -> None:
result = exchange.get_max_pair_stake_amount('ETH/BTC', 2)
assert result == 1000
result = exchange.get_max_pair_stake_amount('ETH/BTC', 2, 12.0)
assert result == 1000 / 12
markets["ETH/BTC"]["contractSize"] = '0.01'
default_conf['trading_mode'] = 'futures'
default_conf['margin_mode'] = 'isolated'
@ -5281,7 +5304,7 @@ def test_stoploss_contract_size(mocker, default_conf, contract_size, order_amoun
})
default_conf['dry_run'] = False
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock)
exchange.get_contract_size = MagicMock(return_value=contract_size)
@ -5301,3 +5324,10 @@ def test_stoploss_contract_size(mocker, default_conf, contract_size, order_amoun
assert order['cost'] == 100
assert order['filled'] == 100
assert order['remaining'] == 100
def test_price_to_precision_with_default_conf(default_conf, mocker):
conf = copy.deepcopy(default_conf)
patched_ex = get_patched_exchange(mocker, conf)
prec_price = patched_ex.price_to_precision("XRP/USDT", 1.0000000101)
assert prec_price == 1.00000001

View File

@ -27,7 +27,7 @@ def test_create_stoploss_order_huobi(default_conf, mocker, limitratio, expected,
})
default_conf['dry_run'] = False
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'huobi')
@ -80,7 +80,7 @@ def test_create_stoploss_order_dry_run_huobi(default_conf, mocker):
order_type = 'stop-limit'
default_conf['dry_run'] = True
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'huobi')

View File

@ -29,7 +29,7 @@ def test_buy_kraken_trading_agreement(default_conf, mocker):
default_conf['dry_run'] = False
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, id="kraken")
order = exchange.create_order(
@ -192,7 +192,7 @@ def test_create_stoploss_order_kraken(default_conf, mocker, ordertype, side, adj
default_conf['dry_run'] = False
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'kraken')
@ -263,7 +263,7 @@ def test_create_stoploss_order_dry_run_kraken(default_conf, mocker, side):
api_mock = MagicMock()
default_conf['dry_run'] = True
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'kraken')

View File

@ -27,7 +27,7 @@ def test_create_stoploss_order_kucoin(default_conf, mocker, limitratio, expected
})
default_conf['dry_run'] = False
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'kucoin')
if order_type == 'limit':
@ -88,7 +88,7 @@ def test_stoploss_order_dry_run_kucoin(default_conf, mocker):
order_type = 'market'
default_conf['dry_run'] = True
mocker.patch(f'{EXMS}.amount_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y: y)
mocker.patch(f'{EXMS}.price_to_precision', lambda s, x, y, **kwargs: y)
exchange = get_patched_exchange(mocker, default_conf, api_mock, 'kucoin')

View File

@ -1,5 +1,6 @@
from copy import deepcopy
from pathlib import Path
from typing import Any, Dict
from unittest.mock import MagicMock
import pytest
@ -85,6 +86,22 @@ def make_rl_config(conf):
return conf
def mock_pytorch_mlp_model_training_parameters() -> Dict[str, Any]:
return {
"learning_rate": 3e-4,
"trainer_kwargs": {
"max_iters": 1,
"batch_size": 64,
"max_n_eval_batches": 1,
},
"model_kwargs": {
"hidden_dim": 32,
"dropout_percent": 0.2,
"n_layer": 1,
}
}
def get_patched_data_kitchen(mocker, freqaiconf):
dk = FreqaiDataKitchen(freqaiconf)
return dk
@ -119,6 +136,7 @@ def make_unfiltered_dataframe(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
freqai.dk.pair = "ADA/BTC"
data_load_timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(data_load_timerange, freqai.dk)
@ -152,6 +170,7 @@ def make_data_dictionary(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
freqai.dk.pair = "ADA/BTC"
data_load_timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(data_load_timerange, freqai.dk)

View File

@ -19,6 +19,7 @@ def test_update_historic_data(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180114")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -41,6 +42,7 @@ def test_load_all_pairs_histories(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180114")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -60,6 +62,7 @@ def test_get_base_and_corr_dataframes(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180114")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
sub_timerange = TimeRange.parse_timerange("20180111-20180114")
@ -87,6 +90,7 @@ def test_use_strategy_to_populate_indicators(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180114")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
sub_timerange = TimeRange.parse_timerange("20180111-20180114")
@ -103,8 +107,9 @@ def test_get_timerange_from_live_historic_predictions(mocker, freqai_conf):
exchange = get_patched_exchange(mocker, freqai_conf)
strategy.dp = DataProvider(freqai_conf, exchange)
freqai = strategy.freqai
freqai.live = True
freqai.live = False
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = False
timerange = TimeRange.parse_timerange("20180126-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
sub_timerange = TimeRange.parse_timerange("20180128-20180130")

View File

@ -180,6 +180,7 @@ def test_get_full_model_path(mocker, freqai_conf, model):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)

View File

@ -15,7 +15,8 @@ from freqtrade.optimize.backtesting import Backtesting
from freqtrade.persistence import Trade
from freqtrade.plugins.pairlistmanager import PairListManager
from tests.conftest import EXMS, create_mock_trades, get_patched_exchange, log_has_re
from tests.freqai.conftest import get_patched_freqai_strategy, make_rl_config
from tests.freqai.conftest import (get_patched_freqai_strategy, make_rl_config,
mock_pytorch_mlp_model_training_parameters)
def is_py11() -> bool:
@ -34,13 +35,14 @@ def is_mac() -> bool:
def can_run_model(model: str) -> None:
if (is_arm() or is_py11()) and "Catboost" in model:
pytest.skip("CatBoost is not supported on ARM")
pytest.skip("CatBoost is not supported on ARM.")
if is_mac() and not is_arm() and 'Reinforcement' in model:
pytest.skip("Reinforcement learning module not available on intel based Mac OS")
is_pytorch_model = 'Reinforcement' in model or 'PyTorch' in model
if is_pytorch_model and is_mac() and not is_arm():
pytest.skip("Reinforcement learning / PyTorch module not available on intel based Mac OS.")
if is_py11() and 'Reinforcement' in model:
pytest.skip("Reinforcement learning currently not available on python 3.11.")
if is_pytorch_model and is_py11():
pytest.skip("Reinforcement learning / PyTorch currently not available on python 3.11.")
@pytest.mark.parametrize('model, pca, dbscan, float32, can_short, shuffle, buffer', [
@ -48,11 +50,12 @@ def can_run_model(model: str) -> None:
('XGBoostRegressor', False, True, False, True, False, 10),
('XGBoostRFRegressor', False, False, False, True, False, 0),
('CatboostRegressor', False, False, False, True, True, 0),
('PyTorchMLPRegressor', False, False, False, True, False, 0),
('ReinforcementLearner', False, True, False, True, False, 0),
('ReinforcementLearner_multiproc', False, False, False, True, False, 0),
('ReinforcementLearner_test_3ac', False, False, False, False, False, 0),
('ReinforcementLearner_test_3ac', False, False, False, True, False, 0),
('ReinforcementLearner_test_4ac', False, False, False, True, False, 0)
('ReinforcementLearner_test_4ac', False, False, False, True, False, 0),
])
def test_extract_data_and_train_model_Standard(mocker, freqai_conf, model, pca,
dbscan, float32, can_short, shuffle, buffer):
@ -79,6 +82,11 @@ def test_extract_data_and_train_model_Standard(mocker, freqai_conf, model, pca,
freqai_conf["freqaimodel_path"] = str(Path(__file__).parents[1] / "freqai" / "test_models")
freqai_conf["freqai"]["rl_config"]["drop_ohlc_from_features"] = True
if 'PyTorchMLPRegressor' in model:
model_save_ext = 'zip'
pytorch_mlp_mtp = mock_pytorch_mlp_model_training_parameters()
freqai_conf['freqai']['model_training_parameters'].update(pytorch_mlp_mtp)
strategy = get_patched_freqai_strategy(mocker, freqai_conf)
exchange = get_patched_exchange(mocker, freqai_conf)
strategy.dp = DataProvider(freqai_conf, exchange)
@ -87,6 +95,7 @@ def test_extract_data_and_train_model_Standard(mocker, freqai_conf, model, pca,
freqai.live = True
freqai.can_short = can_short
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
freqai.dk.set_paths('ADA/BTC', 10000)
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -122,8 +131,7 @@ def test_extract_data_and_train_model_Standard(mocker, freqai_conf, model, pca,
('CatboostClassifierMultiTarget', "freqai_test_multimodel_classifier_strat")
])
def test_extract_data_and_train_model_MultiTargets(mocker, freqai_conf, model, strat):
if (is_arm() or is_py11()) and 'Catboost' in model:
pytest.skip("CatBoost is not supported on ARM")
can_run_model(model)
freqai_conf.update({"timerange": "20180110-20180130"})
freqai_conf.update({"strategy": strat})
@ -135,6 +143,7 @@ def test_extract_data_and_train_model_MultiTargets(mocker, freqai_conf, model, s
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -162,10 +171,10 @@ def test_extract_data_and_train_model_MultiTargets(mocker, freqai_conf, model, s
'CatboostClassifier',
'XGBoostClassifier',
'XGBoostRFClassifier',
'PyTorchMLPClassifier',
])
def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
if (is_arm() or is_py11()) and model == 'CatboostClassifier':
pytest.skip("CatBoost is not supported on ARM")
can_run_model(model)
freqai_conf.update({"freqaimodel": model})
freqai_conf.update({"strategy": "freqai_test_classifier"})
@ -178,6 +187,7 @@ def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -190,7 +200,20 @@ def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
freqai.extract_data_and_train_model(new_timerange, "ADA/BTC",
strategy, freqai.dk, data_load_timerange)
assert Path(freqai.dk.data_path / f"{freqai.dk.model_filename}_model.joblib").exists()
if 'PyTorchMLPClassifier':
pytorch_mlp_mtp = mock_pytorch_mlp_model_training_parameters()
freqai_conf['freqai']['model_training_parameters'].update(pytorch_mlp_mtp)
if freqai.dd.model_type == 'joblib':
model_file_extension = ".joblib"
elif freqai.dd.model_type == "pytorch":
model_file_extension = ".zip"
else:
raise Exception(f"Unsupported model type: {freqai.dd.model_type},"
f" can't assign model_file_extension")
assert Path(freqai.dk.data_path /
f"{freqai.dk.model_filename}_model{model_file_extension}").exists()
assert Path(freqai.dk.data_path / f"{freqai.dk.model_filename}_metadata.json").exists()
assert Path(freqai.dk.data_path / f"{freqai.dk.model_filename}_trained_df.pkl").exists()
assert Path(freqai.dk.data_path / f"{freqai.dk.model_filename}_svm_model.joblib").exists()
@ -204,10 +227,12 @@ def test_extract_data_and_train_model_Classifiers(mocker, freqai_conf, model):
("LightGBMRegressor", 2, "freqai_test_strat"),
("XGBoostRegressor", 2, "freqai_test_strat"),
("CatboostRegressor", 2, "freqai_test_strat"),
("PyTorchMLPRegressor", 2, "freqai_test_strat"),
("ReinforcementLearner", 3, "freqai_rl_test_strat"),
("XGBoostClassifier", 2, "freqai_test_classifier"),
("LightGBMClassifier", 2, "freqai_test_classifier"),
("CatboostClassifier", 2, "freqai_test_classifier")
("CatboostClassifier", 2, "freqai_test_classifier"),
("PyTorchMLPClassifier", 2, "freqai_test_classifier")
],
)
def test_start_backtesting(mocker, freqai_conf, model, num_files, strat, caplog):
@ -228,6 +253,10 @@ def test_start_backtesting(mocker, freqai_conf, model, num_files, strat, caplog)
if 'test_4ac' in model:
freqai_conf["freqaimodel_path"] = str(Path(__file__).parents[1] / "freqai" / "test_models")
if 'PyTorchMLP' in model:
pytorch_mlp_mtp = mock_pytorch_mlp_model_training_parameters()
freqai_conf['freqai']['model_training_parameters'].update(pytorch_mlp_mtp)
freqai_conf.get("freqai", {}).get("feature_parameters", {}).update(
{"indicator_periods_candles": [2]})
@ -371,6 +400,9 @@ def test_backtesting_fit_live_predictions(mocker, freqai_conf, caplog):
sub_timerange = TimeRange.parse_timerange("20180129-20180130")
corr_df, base_df = freqai.dd.get_base_and_corr_dataframes(sub_timerange, "LTC/BTC", freqai.dk)
df = freqai.dk.use_strategy_to_populate_indicators(strategy, corr_df, base_df, "LTC/BTC")
df = strategy.set_freqai_targets(df.copy(), metadata={"pair": "LTC/BTC"})
df = freqai.dk.remove_special_chars_from_feature_names(df)
freqai.dk.get_unique_classes_from_labels(df)
freqai.dk.pair = "ADA/BTC"
freqai.dk.full_df = df.fillna(0)
freqai.dk.full_df
@ -394,6 +426,7 @@ def test_principal_component_analysis(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
@ -425,10 +458,12 @@ def test_plot_feature_importance(mocker, freqai_conf):
freqai = strategy.freqai
freqai.live = True
freqai.dk = FreqaiDataKitchen(freqai_conf)
freqai.dk.live = True
timerange = TimeRange.parse_timerange("20180110-20180130")
freqai.dd.load_all_pair_histories(timerange, freqai.dk)
freqai.dd.pair_dict = MagicMock()
freqai.dd.pair_dict = {"ADA/BTC": {"model_filename": "fake_name",
"trained_timestamp": 1, "data_path": "", "extras": {}}}
data_load_timerange = TimeRange.parse_timerange("20180110-20180130")
new_timerange = TimeRange.parse_timerange("20180120-20180130")

View File

@ -0,0 +1,69 @@
from datetime import datetime, timedelta, timezone
import pytest
from freqtrade.persistence.key_value_store import KeyValueStore, set_startup_time
from tests.conftest import create_mock_trades_usdt
@pytest.mark.usefixtures("init_persistence")
def test_key_value_store(time_machine):
start = datetime(2023, 1, 1, 4, tzinfo=timezone.utc)
time_machine.move_to(start, tick=False)
KeyValueStore.store_value("test", "testStringValue")
KeyValueStore.store_value("test_dt", datetime.now(timezone.utc))
KeyValueStore.store_value("test_float", 22.51)
KeyValueStore.store_value("test_int", 15)
assert KeyValueStore.get_value("test") == "testStringValue"
assert KeyValueStore.get_value("test") == "testStringValue"
assert KeyValueStore.get_string_value("test") == "testStringValue"
assert KeyValueStore.get_value("test_dt") == datetime.now(timezone.utc)
assert KeyValueStore.get_datetime_value("test_dt") == datetime.now(timezone.utc)
assert KeyValueStore.get_string_value("test_dt") is None
assert KeyValueStore.get_float_value("test_dt") is None
assert KeyValueStore.get_int_value("test_dt") is None
assert KeyValueStore.get_value("test_float") == 22.51
assert KeyValueStore.get_float_value("test_float") == 22.51
assert KeyValueStore.get_value("test_int") == 15
assert KeyValueStore.get_int_value("test_int") == 15
assert KeyValueStore.get_datetime_value("test_int") is None
time_machine.move_to(start + timedelta(days=20, hours=5), tick=False)
assert KeyValueStore.get_value("test_dt") != datetime.now(timezone.utc)
assert KeyValueStore.get_value("test_dt") == start
# Test update works
KeyValueStore.store_value("test_dt", datetime.now(timezone.utc))
assert KeyValueStore.get_value("test_dt") == datetime.now(timezone.utc)
KeyValueStore.store_value("test_float", 23.51)
assert KeyValueStore.get_value("test_float") == 23.51
# test deleting
KeyValueStore.delete_value("test_float")
assert KeyValueStore.get_value("test_float") is None
# Delete same value again (should not fail)
KeyValueStore.delete_value("test_float")
with pytest.raises(ValueError, match=r"Unknown value type"):
KeyValueStore.store_value("test_float", {'some': 'dict'})
@pytest.mark.usefixtures("init_persistence")
def test_set_startup_time(fee, time_machine):
create_mock_trades_usdt(fee)
start = datetime.now(timezone.utc)
time_machine.move_to(start, tick=False)
set_startup_time()
assert KeyValueStore.get_value("startup_time") == start
initial_time = KeyValueStore.get_value("bot_start_time")
assert initial_time <= start
# Simulate bot restart
new_start = start + timedelta(days=5)
time_machine.move_to(new_start, tick=False)
set_startup_time()
assert KeyValueStore.get_value("startup_time") == new_start
assert KeyValueStore.get_value("bot_start_time") == initial_time

View File

@ -6,7 +6,7 @@ import arrow
import pytest
from sqlalchemy import select
from freqtrade.constants import DATETIME_PRINT_FORMAT
from freqtrade.constants import CUSTOM_TAG_MAX_LENGTH, DATETIME_PRINT_FORMAT
from freqtrade.enums import TradingMode
from freqtrade.exceptions import DependencyException
from freqtrade.persistence import LocalTrade, Order, Trade, init_db
@ -2037,6 +2037,7 @@ def test_Trade_object_idem():
'get_mix_tag_performance',
'get_trading_volume',
'from_json',
'validate_string_len',
)
EXCLUDES2 = ('trades', 'trades_open', 'bt_trades_open_pp', 'bt_open_open_trade_count',
'total_profit')
@ -2055,6 +2056,31 @@ def test_Trade_object_idem():
assert item in trade
@pytest.mark.usefixtures("init_persistence")
def test_trade_truncates_string_fields():
trade = Trade(
pair='ADA/USDT',
stake_amount=20.0,
amount=30.0,
open_rate=2.0,
open_date=datetime.utcnow() - timedelta(minutes=20),
fee_open=0.001,
fee_close=0.001,
exchange='binance',
leverage=1.0,
trading_mode='futures',
enter_tag='a' * CUSTOM_TAG_MAX_LENGTH * 2,
exit_reason='b' * CUSTOM_TAG_MAX_LENGTH * 2,
)
Trade.session.add(trade)
Trade.commit()
trade1 = Trade.session.scalars(select(Trade)).first()
assert trade1.enter_tag == 'a' * CUSTOM_TAG_MAX_LENGTH
assert trade1.exit_reason == 'b' * CUSTOM_TAG_MAX_LENGTH
def test_recalc_trade_from_orders(fee):
o1_amount = 100

View File

@ -883,6 +883,8 @@ def test_api_profit(botclient, mocker, ticker, fee, markets, is_short, expected)
'max_drawdown': ANY,
'max_drawdown_abs': ANY,
'trading_volume': expected['trading_volume'],
'bot_start_timestamp': 0,
'bot_start_date': '',
}
@ -1403,10 +1405,10 @@ def test_api_pair_candles(botclient, ohlcv_history):
])
def test_api_pair_history(botclient, ohlcv_history):
def test_api_pair_history(botclient, mocker):
ftbot, client = botclient
timeframe = '5m'
lfm = mocker.patch('freqtrade.strategy.interface.IStrategy.load_freqAI_model')
# No pair
rc = client_get(client,
f"{BASE_URI}/pair_history?timeframe={timeframe}"
@ -1440,6 +1442,7 @@ def test_api_pair_history(botclient, ohlcv_history):
assert len(rc.json()['data']) == rc.json()['length']
assert 'columns' in rc.json()
assert 'data' in rc.json()
assert lfm.call_count == 1
assert rc.json()['pair'] == 'UNITTEST/BTC'
assert rc.json()['strategy'] == CURRENT_TEST_STRATEGY
assert rc.json()['data_start'] == '2018-01-11 00:00:00+00:00'

View File

@ -2241,8 +2241,9 @@ def test_send_msg_buy_notification_no_fiat(
('Short', 'short_signal_01', 2.0),
])
def test_send_msg_sell_notification_no_fiat(
default_conf, mocker, direction, enter_signal, leverage) -> None:
default_conf, mocker, direction, enter_signal, leverage, time_machine) -> None:
del default_conf['fiat_display_currency']
time_machine.move_to('2022-05-02 00:00:00 +00:00', tick=False)
telegram, _, msg_mock = get_telegram_testobject(mocker, default_conf)
telegram.send_msg({

View File

@ -82,7 +82,7 @@ class freqai_test_classifier(IStrategy):
return dataframe
def set_freqai_targets(self, dataframe: DataFrame, metadata: Dict, **kwargs):
self.freqai.class_names = ["down", "up"]
dataframe['&s-up_or_down'] = np.where(dataframe["close"].shift(-100) >
dataframe["close"], 'up', 'down')

View File

@ -9,6 +9,7 @@ import pytest
from pandas import DataFrame
from freqtrade.configuration import TimeRange
from freqtrade.constants import CUSTOM_TAG_MAX_LENGTH
from freqtrade.data.dataprovider import DataProvider
from freqtrade.data.history import load_data
from freqtrade.enums import ExitCheckTuple, ExitType, HyperoptState, SignalDirection
@ -529,13 +530,13 @@ def test_custom_exit(default_conf, fee, caplog) -> None:
assert res[0].exit_reason == 'hello world'
caplog.clear()
strategy.custom_exit = MagicMock(return_value='h' * 100)
strategy.custom_exit = MagicMock(return_value='h' * CUSTOM_TAG_MAX_LENGTH * 2)
res = strategy.should_exit(trade, 1, now,
enter=False, exit_=False,
low=None, high=None)
assert res[0].exit_type == ExitType.CUSTOM_EXIT
assert res[0].exit_flag is True
assert res[0].exit_reason == 'h' * 64
assert res[0].exit_reason == 'h' * (CUSTOM_TAG_MAX_LENGTH)
assert log_has_re('Custom exit reason returned from custom_exit is too long.*', caplog)
@ -986,7 +987,8 @@ def test_auto_hyperopt_interface_loadparams(default_conf, mocker, caplog):
}
}
}
mocker.patch('freqtrade.strategy.hyper.json_load', return_value=expected_result)
mocker.patch('freqtrade.strategy.hyper.HyperoptTools.load_params',
return_value=expected_result)
PairLocks.timeframe = default_conf['timeframe']
strategy = StrategyResolver.load_strategy(default_conf)
assert strategy.stoploss == -0.05
@ -1005,11 +1007,13 @@ def test_auto_hyperopt_interface_loadparams(default_conf, mocker, caplog):
}
}
mocker.patch('freqtrade.strategy.hyper.json_load', return_value=expected_result)
mocker.patch('freqtrade.strategy.hyper.HyperoptTools.load_params',
return_value=expected_result)
with pytest.raises(OperationalException, match="Invalid parameter file provided."):
StrategyResolver.load_strategy(default_conf)
mocker.patch('freqtrade.strategy.hyper.json_load', MagicMock(side_effect=ValueError()))
mocker.patch('freqtrade.strategy.hyper.HyperoptTools.load_params',
MagicMock(side_effect=ValueError()))
StrategyResolver.load_strategy(default_conf)
assert log_has("Invalid parameter file format.", caplog)

View File

@ -356,7 +356,7 @@ def test_create_trade_no_stake_amount(default_conf_usdt, ticker_usdt, fee, mocke
@pytest.mark.parametrize("is_short", [False, True])
@pytest.mark.parametrize('stake_amount,create,amount_enough,max_open_trades', [
(5.0, True, True, 99),
(0.049, True, False, 99), # Amount will be adjusted to min - which is 0.051
(0.042, True, False, 99), # Amount will be adjusted to min - which is 0.051
(0, False, True, 99),
(UNLIMITED_STAKE_AMOUNT, False, True, 0),
])
@ -1290,6 +1290,137 @@ def test_handle_stoploss_on_exchange(mocker, default_conf_usdt, fee, caplog, is_
assert trade.exit_reason == str(ExitType.EMERGENCY_EXIT)
@pytest.mark.parametrize("is_short", [False, True])
def test_handle_stoploss_on_exchange_partial(
mocker, default_conf_usdt, fee, is_short, limit_order) -> None:
stop_order_dict = {'id': "101", "status": "open"}
stoploss = MagicMock(return_value=stop_order_dict)
enter_order = limit_order[entry_side(is_short)]
exit_order = limit_order[exit_side(is_short)]
patch_RPCManager(mocker)
patch_exchange(mocker)
mocker.patch.multiple(
EXMS,
fetch_ticker=MagicMock(return_value={
'bid': 1.9,
'ask': 2.2,
'last': 1.9
}),
create_order=MagicMock(side_effect=[
enter_order,
exit_order,
]),
get_fee=fee,
create_stoploss=stoploss
)
freqtrade = FreqtradeBot(default_conf_usdt)
patch_get_signal(freqtrade, enter_short=is_short, enter_long=not is_short)
freqtrade.enter_positions()
trade = Trade.session.scalars(select(Trade)).first()
trade.is_short = is_short
trade.is_open = True
trade.open_order_id = None
trade.stoploss_order_id = None
assert freqtrade.handle_stoploss_on_exchange(trade) is False
assert stoploss.call_count == 1
assert trade.stoploss_order_id == "101"
assert trade.amount == 30
stop_order_dict.update({'id': "102"})
# Stoploss on exchange is cancelled on exchange, but filled partially.
# Must update trade amount to guarantee successful exit.
stoploss_order_hit = MagicMock(return_value={
'id': "101",
'status': 'canceled',
'type': 'stop_loss_limit',
'price': 3,
'average': 2,
'filled': trade.amount / 2,
'remaining': trade.amount / 2,
'amount': enter_order['amount'],
})
mocker.patch(f'{EXMS}.fetch_stoploss_order', stoploss_order_hit)
assert freqtrade.handle_stoploss_on_exchange(trade) is False
# Stoploss filled partially ...
assert trade.amount == 15
assert trade.stoploss_order_id == "102"
@pytest.mark.parametrize("is_short", [False, True])
def test_handle_stoploss_on_exchange_partial_cancel_here(
mocker, default_conf_usdt, fee, is_short, limit_order, caplog) -> None:
stop_order_dict = {'id': "101", "status": "open"}
default_conf_usdt['trailing_stop'] = True
stoploss = MagicMock(return_value=stop_order_dict)
enter_order = limit_order[entry_side(is_short)]
exit_order = limit_order[exit_side(is_short)]
patch_RPCManager(mocker)
patch_exchange(mocker)
mocker.patch.multiple(
EXMS,
fetch_ticker=MagicMock(return_value={
'bid': 1.9,
'ask': 2.2,
'last': 1.9
}),
create_order=MagicMock(side_effect=[
enter_order,
exit_order,
]),
get_fee=fee,
create_stoploss=stoploss
)
freqtrade = FreqtradeBot(default_conf_usdt)
patch_get_signal(freqtrade, enter_short=is_short, enter_long=not is_short)
freqtrade.enter_positions()
trade = Trade.session.scalars(select(Trade)).first()
trade.is_short = is_short
trade.is_open = True
trade.open_order_id = None
trade.stoploss_order_id = None
assert freqtrade.handle_stoploss_on_exchange(trade) is False
assert stoploss.call_count == 1
assert trade.stoploss_order_id == "101"
assert trade.amount == 30
stop_order_dict.update({'id': "102"})
# Stoploss on exchange is open.
# Freqtrade cancels the stop - but cancel returns a partial filled order.
stoploss_order_hit = MagicMock(return_value={
'id': "101",
'status': 'open',
'type': 'stop_loss_limit',
'price': 3,
'average': 2,
'filled': 0,
'remaining': trade.amount,
'amount': enter_order['amount'],
})
stoploss_order_cancel = MagicMock(return_value={
'id': "101",
'status': 'canceled',
'type': 'stop_loss_limit',
'price': 3,
'average': 2,
'filled': trade.amount / 2,
'remaining': trade.amount / 2,
'amount': enter_order['amount'],
})
mocker.patch(f'{EXMS}.fetch_stoploss_order', stoploss_order_hit)
mocker.patch(f'{EXMS}.cancel_stoploss_order_with_result', stoploss_order_cancel)
trade.stoploss_last_update = arrow.utcnow().shift(minutes=-10).datetime
assert freqtrade.handle_stoploss_on_exchange(trade) is False
# Canceled Stoploss filled partially ...
assert log_has_re('Cancelling current stoploss on exchange.*', caplog)
assert trade.stoploss_order_id == "102"
assert trade.amount == 15
@pytest.mark.parametrize("is_short", [False, True])
def test_handle_sle_cancel_cant_recreate(mocker, default_conf_usdt, fee, caplog, is_short,
limit_order) -> None:
@ -1671,7 +1802,7 @@ def test_stoploss_on_exchange_price_rounding(
EXMS,
get_fee=fee,
)
price_mock = MagicMock(side_effect=lambda p, s: int(s))
price_mock = MagicMock(side_effect=lambda p, s, **kwargs: int(s))
stoploss_mock = MagicMock(return_value={'id': '13434334'})
adjust_mock = MagicMock(return_value=False)
mocker.patch.multiple(
@ -2824,6 +2955,9 @@ def test_manage_open_orders_exit_usercustom(
assert rpc_mock.call_count == 2
assert freqtrade.strategy.check_exit_timeout.call_count == 1
assert freqtrade.strategy.check_entry_timeout.call_count == 0
trade = Trade.session.scalars(select(Trade)).first()
# cancelling didn't succeed - order-id remains open.
assert trade.open_order_id is not None
# 2nd canceled trade - Fail execute exit
caplog.clear()
@ -3334,6 +3468,7 @@ def test_handle_cancel_exit_cancel_exception(mocker, default_conf_usdt) -> None:
# TODO: should not be magicmock
trade = MagicMock()
trade.open_order_id = '125'
reason = CANCEL_REASON['TIMEOUT']
order = {'remaining': 1,
'id': '125',
@ -3341,6 +3476,10 @@ def test_handle_cancel_exit_cancel_exception(mocker, default_conf_usdt) -> None:
'status': "open"}
assert not freqtrade.handle_cancel_exit(trade, order, reason)
# mocker.patch(f'{EXMS}.cancel_order_with_result', return_value=order)
# assert not freqtrade.handle_cancel_exit(trade, order, reason)
# assert trade.open_order_id == '125'
@pytest.mark.parametrize("is_short, open_rate, amt", [
(False, 2.0, 30.0),

View File

@ -10,6 +10,8 @@ from freqtrade.exceptions import OperationalException
def test_parse_timerange_incorrect():
timerange = TimeRange.parse_timerange('')
assert timerange == TimeRange(None, None, 0, 0)
timerange = TimeRange.parse_timerange('20100522-')
assert TimeRange('date', None, 1274486400, 0) == timerange
assert timerange.timerange_str == '20100522-'