元控制器：元任务 & 元数据集 & 元模型

简介

Meta Controller 为 Forecast Model 提供指导，其目标是学习一系列预测任务中的规律模式，并利用习得的模式来指导后续预测任务。用户可基于 Meta Controller 模块实现自己的元模型实例。

元任务

Meta Task 实例是元学习框架中的基本单元。它保存可供 元模型 使用的数据。多个 Meta Task 实例可能共享由 元数据集 控制的同一 Data Handler。用户应使用 prepare_task_data() 方法获取可直接输入 Meta Model 的数据。

class qlib.model.meta.task.MetaTask(task: dict, meta_info: object, mode: str = 'full')

A single meta-task, a meta-dataset contains a list of them. It serves as a component as in MetaDatasetDS

The data processing is different

  • the processed input may be different between training and testing

    • When training, the X, y, X_test, y_test in training tasks are necessary (# PROC_MODE_FULL #) but not necessary in test tasks. (# PROC_MODE_TEST #)

    • When the meta model can be transferred into other dataset, only meta_info is necessary (# PROC_MODE_TRANSFER #)

__init__(task: dict, meta_info: object, mode: str = 'full')

The __init__ func is responsible for

  • store the task

  • store the origin input data for

  • process the input data for meta data

参数:

  • task (dict) -- the task to be enhanced by meta model

  • meta_info (object) -- the input for meta model

get_meta_input() → object

Return the processed meta_info

Meta Dataset

Meta Dataset 控制元信息生成过程，负责为训练 Meta Model 提供数据。用户应使用 prepare_tasks 方法来获取 Meta Task 实例列表。

class qlib.model.meta.dataset.MetaTaskDataset(segments: Dict[str, Tuple] | float, *args, **kwargs)

A dataset fetching the data in a meta-level.

A Meta Dataset is responsible for

  • input tasks(e.g. Qlib tasks) and prepare meta tasks

    • meta task contains more information than normal tasks (e.g. input data for meta model)

The learnt pattern could transfer to other meta dataset. The following cases should be supported

  • A meta-model trained on meta-dataset A and then applied to meta-dataset B

    • Some pattern are shared between meta-dataset A and B, so meta-input on meta-dataset A are used when meta model are applied on meta-dataset-B

__init__(segments: Dict[str, Tuple] | float, *args, **kwargs)

The meta-dataset maintains a list of meta-tasks when it is initialized.

The segments indicates the way to divide the data

The duty of the __init__ function of MetaTaskDataset - initialize the tasks

prepare_tasks(segments: List[str] | str, *args, **kwargs) → List[MetaTask]

Prepare the data in each meta-task and ready for training.

The following code example shows how to retrieve a list of meta-tasks from the meta_dataset:

# get the train segment and the test segment, both of them are lists
train_meta_tasks, test_meta_tasks = meta_dataset.prepare_tasks(["train", "test"])
参数:

segments (Union[List[Text], Tuple[Text], Text]) -- the info to select data

返回:

A list of the prepared data of each meta-task for training the meta-model. For multiple segments [seg1, seg2, ... , segN], the returned list will be [[tasks in seg1], [tasks in seg2], ... , [tasks in segN]]. Each task is a meta task

返回类型:

list

Meta Model

通用元模型

Meta Model 实例是控制工作流程的部分。Meta Model 的使用包括： 1. 用户通过 fit 函数训练他们的 Meta Model。 2. Meta Model 实例通过 inference 函数提供有用信息来指导工作流程。

class qlib.model.meta.model.MetaModel

The meta-model guiding the model learning.

The word Guiding can be categorized into two types based on the stage of model learning - The definition of learning tasks: Please refer to docs of MetaTaskModel - Controlling the learning process of models: Please refer to the docs of MetaGuideModel

abstractmethod fit(*args, **kwargs)

The training process of the meta-model.

abstractmethod inference(*args, **kwargs) → object

The inference process of the meta-model.

返回:

Some information to guide the model learning

返回类型:

object

元任务模型

这类元模型可能会直接与任务定义交互。因此，Meta Task Model 是供它们继承的基类。它们通过修改基础任务定义来指导基础任务。函数 prepare_tasks 可用于获取修改后的基础任务定义。

class qlib.model.meta.model.MetaTaskModel

This type of meta-model deals with base task definitions. The meta-model creates tasks for training new base forecasting models after it is trained. prepare_tasks directly modifies the task definitions.

fit(meta_dataset: MetaTaskDataset)

The MetaTaskModel is expected to get prepared MetaTask from meta_dataset. And then it will learn knowledge from the meta tasks

inference(meta_dataset: MetaTaskDataset) → List[dict]

MetaTaskModel will make inference on the meta_dataset The MetaTaskModel is expected to get prepared MetaTask from meta_dataset. Then it will create modified task with Qlib format which can be executed by Qlib trainer.

返回:

A list of modified task definitions.

返回类型:

List[dict]

元指导模型

这类元模型参与基础预测模型的训练过程。元模型可以在基础预测模型训练期间对其进行指导，以提高其性能。

class qlib.model.meta.model.MetaGuideModel

This type of meta-model aims to guide the training process of the base model. The meta-model interacts with the base forecasting models during their training process.

abstractmethod fit(*args, **kwargs)

The training process of the meta-model.

abstractmethod inference(*args, **kwargs)

The inference process of the meta-model.

返回:

Some information to guide the model learning

返回类型:

object

示例

Qlib 提供了 Meta Model 模块的实现 DDG-DA， 该模块能够适应市场动态。

DDG-DA 包含四个步骤：

  1. 计算元信息并将其封装为 Meta Task 实例。所有元任务构成一个 Meta Dataset 实例。

  2. 基于元数据集的训练数据训练 DDG-DA。

  3. 对 DDG-DA 进行推理以获取指导信息。

  4. 将指导信息应用于预测模型以提高其性能。

上述示例可在 examples/benchmarks_dynamic/DDG-DA/workflow.py 中找到：https://github.com/microsoft/qlib/tree/main/examples/benchmarks_dynamic/DDG-DA。