lib.sedna.core.lifelong_learning.lifelong_learning

Module Contents

Classes

LifelongLearning

Lifelong Learning (LL) is an advanced machine learning (ML) paradigm that
class lib.sedna.core.lifelong_learning.lifelong_learning.LifelongLearning(seen_estimator, unseen_estimator=None, task_definition=None, task_relationship_discovery=None, task_allocation=None, task_remodeling=None, inference_integrate=None, task_update_decision=None, unseen_task_allocation=None, unseen_sample_recognition=None, unseen_sample_re_recognition=None)[source]

Bases: sedna.core.base.JobBase

Lifelong Learning (LL) is an advanced machine learning (ML) paradigm that learns continuously, accumulates the knowledge learned in the past, and uses/adapts it to help future learning and problem solving.

Sedna provide the related interfaces for application development.

Parameters:

  • estimator (Instance) – An instance with the high-level API that greatly simplifies machine learning programming. Estimators encapsulate training, evaluation, prediction, and exporting for your model.

  • unseen_estimator (Instance) – An instance with the high-level API that greatly simplifies mechanism model learning programming. Estimators encapsulate training, evaluation, prediction, and exporting for your mechanism model.

  • task_definition (Dict) – Divide multiple tasks based on data, see task_definition.task_definition for more detail.

  • task_relationship_discovery (Dict) – Discover relationships between all tasks, see task_relationship_discovery.task_relationship_discovery for more detail.

  • task_allocation (Dict) – Mining seen tasks of inference sample, see task_allocation.task_allocation for more detail.

  • task_remodeling (Dict) – Remodeling tasks based on their relationships, see task_remodeling.task_remodeling for more detail.

  • inference_integrate (Dict) – Integrate the inference results of all related tasks, see inference_integrate.inference_integrate for more detail.

  • task_update_decision (Dict) – Task update strategy making algorithms, see ‘knowledge_management.task_update_decision.task_update_decision’ for more detail.

  • unseen_task_allocation (Dict) – Mining unseen tasks of inference sample, see unseen_task_processing.unseen_task_allocation. unseen_task_allocation for more detail.

  • unseen_sample_recognition (Dict) – Dividing inference samples into seen tasks and unseen tasks, see ‘unseen_task_processing.unseen_sample_recognition. unseen_sample_recognition’ for more detail.

  • unseen_sample_re_recognition (Dict) – Dividing unseen training samples into seen tasks and unseen tasks, see ‘unseen_task_processing.unseen_sample_re_recognition. unseen_sample_re_recognition’ for more detail.

Examples

>>> estimator = XGBClassifier(objective="binary:logistic")
>>> unseen_estimator = None
>>> task_definition = {
        "method": "TaskDefinitionByDataAttr",
        "param": {"attribute": ["season", "city"]}
    }
>>> task_relationship_discovery = {
        "method": "DefaultTaskRelationDiscover", "param": {}
    }
>>> task_mining = {
        "method": "TaskMiningByDataAttr",
        "param": {"attribute": ["season", "city"]}
    }
>>> task_remodeling = None
>>> inference_integrate = {
        "method": "DefaultInferenceIntegrate", "param": {}
    }
>>> task_update_decision = {
        "method": "UpdateStrategyDefault", "param": {}
    }
>>> unseen_task_allocation = {
        "method": "UnseenTaskAllocationDefault", "param": {}
    }
>>> unseen_sample_recognition = {
        "method": "SampleRegonitionDefault", "param": {}
    }
>>> unseen_sample_re_recognition = {
        "method": "SampleReRegonitionDefault", "param": {}
    }
>>> ll_jobs = LifelongLearning(
        estimator,
        unseen_estimator=None,
        task_definition=None,
        task_relationship_discovery=None,
        task_allocation=None,
        task_remodeling=None,
        inference_integrate=None,
        task_update_decision=None,
        unseen_task_allocation=None,
        unseen_sample_recognition=None,
        unseen_sample_re_recognition=None,
    )
train(train_data, valid_data=None, post_process=None, **kwargs)[source]

fit for update the knowledge based on training data.

Parameters:

  • train_data (BaseDataSource) – Train data, see sedna.datasources.BaseDataSource for more detail.

  • valid_data (BaseDataSource) – Valid data, BaseDataSource or None.

  • post_process (function) – function or a registered method, callback after estimator train.

  • kwargs (Dict) – parameters for estimator training, Like: early_stopping_rounds in Xgboost.XGBClassifier

Returns:

train_history

Return type:

object

update(train_data, valid_data=None, post_process=None, **kwargs)[source]

fit for update the knowledge based on incremental data.

Parameters:

  • train_data (BaseDataSource) – Train data, see sedna.datasources.BaseDataSource for more detail.

  • valid_data (BaseDataSource) – Valid data, BaseDataSource or None.

  • post_process (function) – function or a registered method, callback after estimator train.

  • kwargs (Dict) – parameters for estimator training, Like: early_stopping_rounds in Xgboost.XGBClassifier

Returns:

train_history

Return type:

object

evaluate(data, post_process=None, **kwargs)[source]

evaluated the performance of each task from training, filter tasks based on the defined rules.

Parameters:

  • data (BaseDataSource) – valid data, see sedna.datasources.BaseDataSource for more detail.

  • kwargs (Dict) – parameters for estimator evaluate, Like: ntree_limit in Xgboost.XGBClassifier

inference(data=None, post_process=None, unseen_sample_postprocess=None, **kwargs)[source]

predict the result for input data based on training knowledge.

Parameters:

  • data (BaseDataSource) – inference sample, see sedna.datasources.BaseDataSource for more detail.

  • post_process (function) – function or a registered method, effected after estimator prediction, like: label transform.

  • unseen_sample_postprocess (function) – function or a registered method, effected when unseen samples need to be saved

  • kwargs (Dict) – parameters for estimator predict, Like: ntree_limit in Xgboost.XGBClassifier