pass metadata dictionary to feature_engineering_* and set_freqai_targets functions. Add doc

This commit is contained in:
robcaulk 2023-02-04 13:47:11 +01:00
parent 55850a5ccd
commit 5da60b718d
3 changed files with 54 additions and 5 deletions

View File

@ -28,6 +28,10 @@ It is advisable to start from the template `feature_engineering_*` functions in
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
Access metadata such as the current pair/timeframe/period with:
`metadata["pair"]` `metadata["tf"]` `metadata["period"]`
:param df: strategy dataframe which will receive the features :param df: strategy dataframe which will receive the features
:param period: period of the indicator - usage example: :param period: period of the indicator - usage example:
dataframe["%-ema-period"] = ta.EMA(dataframe, timeperiod=period) dataframe["%-ema-period"] = ta.EMA(dataframe, timeperiod=period)
@ -75,6 +79,10 @@ It is advisable to start from the template `feature_engineering_*` functions in
Features defined here will *not* be automatically duplicated on user defined Features defined here will *not* be automatically duplicated on user defined
`indicator_periods_candles` `indicator_periods_candles`
Access metadata such as the current pair/timeframe with:
`metadata["pair"]` `metadata["tf"]`
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
:param df: strategy dataframe which will receive the features :param df: strategy dataframe which will receive the features
@ -98,6 +106,10 @@ It is advisable to start from the template `feature_engineering_*` functions in
This function is a good place for any feature that should not be auto-expanded upon This function is a good place for any feature that should not be auto-expanded upon
(e.g. day of the week). (e.g. day of the week).
Access metadata such as the current pair with:
`metadata["pair"]`
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
:param df: strategy dataframe which will receive the features :param df: strategy dataframe which will receive the features
@ -113,6 +125,10 @@ It is advisable to start from the template `feature_engineering_*` functions in
Required function to set the targets for the model. Required function to set the targets for the model.
All targets must be prepended with `&` to be recognized by the FreqAI internals. All targets must be prepended with `&` to be recognized by the FreqAI internals.
Access metadata such as the current pair with:
`metadata["pair"]`
:param df: strategy dataframe which will receive the targets :param df: strategy dataframe which will receive the targets
usage example: dataframe["&-target"] = dataframe["close"].shift(-1) / dataframe["close"] usage example: dataframe["&-target"] = dataframe["close"].shift(-1) / dataframe["close"]
""" """
@ -161,6 +177,19 @@ You can ask for each of the defined features to be included also for informative
In total, the number of features the user of the presented example strat has created is: length of `include_timeframes` * no. features in `feature_engineering_expand_*()` * length of `include_corr_pairlist` * no. `include_shifted_candles` * length of `indicator_periods_candles` In total, the number of features the user of the presented example strat has created is: length of `include_timeframes` * no. features in `feature_engineering_expand_*()` * length of `include_corr_pairlist` * no. `include_shifted_candles` * length of `indicator_periods_candles`
$= 3 * 3 * 3 * 2 * 2 = 108$. $= 3 * 3 * 3 * 2 * 2 = 108$.
### Gain finer control over `feature_engineering_*` functions with `metadata`
All `feature_engineering_*` and `set_freqai_targets()` functions are passed a `metadata` dictionary which contains information about the `pair`, `tf` (timeframe), and `period` that FreqAI is automating for feature building. As such, a user can use `metadata` inside `feature_engineering_*` functions as criteria for blocking/reserving features for certain timeframes, periods, pairs etc.
```py
def feature_engineering_expand_all(self, dataframe, period, **kwargs):
if metadata["tf"] == "1h":
dataframe["%-roc-period"] = ta.ROC(dataframe, timeperiod=period)
```
This will block `ta.ROC()` from being added to any timeframes other than `"1h"`.
### Returning additional info from training ### Returning additional info from training
Important metrics can be returned to the strategy at the end of each model training by assigning them to `dk.data['extra_returns_per_train']['my_new_value'] = XYZ` inside the custom prediction model class. Important metrics can be returned to the strategy at the end of each model training by assigning them to `dk.data['extra_returns_per_train']['my_new_value'] = XYZ` inside the custom prediction model class.

View File

@ -1247,17 +1247,21 @@ class FreqaiDataKitchen:
tfs: List[str] = self.freqai_config["feature_parameters"].get("include_timeframes") tfs: List[str] = self.freqai_config["feature_parameters"].get("include_timeframes")
for tf in tfs: for tf in tfs:
metadata = {"pair": pair, "tf": tf}
informative_df = self.get_pair_data_for_features( informative_df = self.get_pair_data_for_features(
pair, tf, strategy, corr_dataframes, base_dataframes, is_corr_pairs) pair, tf, strategy, corr_dataframes, base_dataframes, is_corr_pairs)
informative_copy = informative_df.copy() informative_copy = informative_df.copy()
for t in self.freqai_config["feature_parameters"]["indicator_periods_candles"]: for t in self.freqai_config["feature_parameters"]["indicator_periods_candles"]:
metadata["period"] = t
df_features = strategy.feature_engineering_expand_all( df_features = strategy.feature_engineering_expand_all(
informative_copy.copy(), t) informative_copy.copy(), t, metadata=metadata)
suffix = f"{t}" suffix = f"{t}"
informative_df = self.merge_features(informative_df, df_features, tf, tf, suffix) informative_df = self.merge_features(informative_df, df_features, tf, tf, suffix)
generic_df = strategy.feature_engineering_expand_basic(informative_copy.copy()) metadata.pop("period")
generic_df = strategy.feature_engineering_expand_basic(
informative_copy.copy(), metadata=metadata)
suffix = "gen" suffix = "gen"
informative_df = self.merge_features(informative_df, generic_df, tf, tf, suffix) informative_df = self.merge_features(informative_df, generic_df, tf, tf, suffix)
@ -1326,8 +1330,8 @@ class FreqaiDataKitchen:
"include_corr_pairlist", []) "include_corr_pairlist", [])
dataframe = self.populate_features(dataframe.copy(), pair, strategy, dataframe = self.populate_features(dataframe.copy(), pair, strategy,
corr_dataframes, base_dataframes) corr_dataframes, base_dataframes)
metadata = {"pair": pair}
dataframe = strategy.feature_engineering_standard(dataframe.copy()) dataframe = strategy.feature_engineering_standard(dataframe.copy(), metadata=metadata)
# ensure corr pairs are always last # ensure corr pairs are always last
for corr_pair in corr_pairs: for corr_pair in corr_pairs:
if pair == corr_pair: if pair == corr_pair:
@ -1336,7 +1340,7 @@ class FreqaiDataKitchen:
dataframe = self.populate_features(dataframe.copy(), corr_pair, strategy, dataframe = self.populate_features(dataframe.copy(), corr_pair, strategy,
corr_dataframes, base_dataframes, True) corr_dataframes, base_dataframes, True)
dataframe = strategy.set_freqai_targets(dataframe.copy()) dataframe = strategy.set_freqai_targets(dataframe.copy(), metadata=metadata)
self.get_unique_classes_from_labels(dataframe) self.get_unique_classes_from_labels(dataframe)

View File

@ -58,6 +58,10 @@ class FreqaiExampleStrategy(IStrategy):
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
Access metadata such as the current pair/timeframe/period with:
`metadata["pair"]` `metadata["tf"]` `metadata["period"]`
More details on how these config defined parameters accelerate feature engineering More details on how these config defined parameters accelerate feature engineering
in the documentation at: in the documentation at:
@ -114,6 +118,10 @@ class FreqaiExampleStrategy(IStrategy):
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
Access metadata such as the current pair/timeframe with:
`metadata["pair"]` `metadata["tf"]`
More details on how these config defined parameters accelerate feature engineering More details on how these config defined parameters accelerate feature engineering
in the documentation at: in the documentation at:
@ -144,6 +152,10 @@ class FreqaiExampleStrategy(IStrategy):
All features must be prepended with `%` to be recognized by FreqAI internals. All features must be prepended with `%` to be recognized by FreqAI internals.
Access metadata such as the current pair with:
`metadata["pair"]`
More details about feature engineering available: More details about feature engineering available:
https://www.freqtrade.io/en/latest/freqai-feature-engineering https://www.freqtrade.io/en/latest/freqai-feature-engineering
@ -161,6 +173,10 @@ class FreqaiExampleStrategy(IStrategy):
Required function to set the targets for the model. Required function to set the targets for the model.
All targets must be prepended with `&` to be recognized by the FreqAI internals. All targets must be prepended with `&` to be recognized by the FreqAI internals.
Access metadata such as the current pair with:
`metadata["pair"]`
More details about feature engineering available: More details about feature engineering available:
https://www.freqtrade.io/en/latest/freqai-feature-engineering https://www.freqtrade.io/en/latest/freqai-feature-engineering