OperatorBase

class towhee.operator.Operator

Bases: ABC

Operator base class, implements __init__ and __call__,

Examples

class AddOperator(Operator):

def __init__(self, factor: int):

self._factor = factor

def __call__(self, num) -> NamedTuple(“Outputs”, [(“sum”, int)]):

Outputs = NamedTuple(“Outputs”, [(“sum”, int)]) return Outputs(self._factor + num)

abstract __init__()

Init operator, before a graph starts, the framework will call Operator __init__ function.

Args:

Raises:

An exception during __init__ can terminate the graph run. –

abstract __call__()

The framework calls __call__ function repeatedly for every input data.

Args:

Returns:

Raises:

An exception during __init__ can terminate the graph run. –

__weakref__

list of weak references to the object (if defined)

class towhee.operator.NNOperator(framework: str = 'pytorch')

Bases: Operator

Neural Network related operators that involve machine learning frameworks.

Parameters:

framework (str) – The framework to apply.

__init__(framework: str = 'pytorch')

Init operator, before a graph starts, the framework will call Operator __init__ function.

Args:

Raises:

An exception during __init__ can terminate the graph run. –

save_model()

Save model to local.

supported_model_names()

Return a list of supported model names.

train(training_config=None, train_dataset=None, eval_dataset=None, resume_checkpoint_path=None, **kwargs)

Start to train an operator.

Parameters:

• training_config (TrainingConfig) – The config of this trainer.

• train_dataset (Union[Dataset, TowheeDataSet]) – Training dataset.

• eval_dataset (Union[Dataset, TowheeDataSet]) – Evaluate dataset.

• resume_checkpoint_path (str) – If resuming training, pass into the path.

• **kwargs (Any) – Keyword Args.

setup_trainer(training_config=None, train_dataset=None, eval_dataset=None, train_dataloader=None, eval_dataloader=None, model_card=None)

Set up the trainer instance in operator before training and set trainer parameters. :param training_config: The config of this trainer. :type training_config: TrainingConfig :param train_dataset: Training dataset. :type train_dataset: Union[Dataset, TowheeDataSet] :param eval_dataset: Evaluate dataset. :type eval_dataset: Union[Dataset, TowheeDataSet] :param train_dataloader: If specified, Trainer will use it to load training data.

Otherwise, Trainer will build dataloader from train_dataset.

Parameters:

• eval_dataloader (Union[DataLoader, Iterable]) – If specified, Trainer will use it to load evaluate data. Otherwise, Trainer will build dataloader from train_dataset.

• model_card (ModelCard) – Model card contains the training informations.

Returns:

class towhee.operator.PyOperator

Bases: Operator

Python function operator, no machine learning frameworks involved.

__init__()

Init operator, before a graph starts, the framework will call Operator __init__ function.

Args:

Raises:

An exception during __init__ can terminate the graph run. –

towhee.register(name: Optional[str] = None, input_schema=None, output_schema=None, flag=None)

Register a class, function, or callable as a towhee operator.

Parameters:

• name (str, optional) – operator name, will use the class/function name if None.

• input_schema –

• output_schema –

• operators. (flag for legacy) –

Returns:

[description]

Return type:

[type]