API

class towhee.runtime.hub_ops.Ops[source]

Bases: object

Class to loading operator with _OperatorWrapper. An operator is usually referred to with its full name: namespace/name.

Examples:

>>> from towhee import ops
>>> op = ops.towhee.image_decode()
>>> img = op('./towhee_logo.png')

We can also specify the version of the operator on the hub via the revision method:

>>> op = ops.towhee.image_decode()

And the latest method is used to update the current version of the operator to the latest:

>>> op = ops.towhee.image_decode().latest()

image_embedding: ImageEmbedding = <towhee.runtime.hub_ops.image_embedding.ImageEmbedding object>

image_embedding: is a task that attempts to comprehend an entire image as a whole and encode the image’s semantics into a real vector. It is a fundamental task type that can be used in a variety of applications, including but not limited to reverse image search and image deduplication.

image_text_embedding: ImageTextEmbedding = <towhee.runtime.hub_ops.image_text_embedding.ImageTextEmbedding object>: text_image_embedding is a task that attempts to comprehend images and texts, and encode both image’s and text’s semantics into a same embedding space. It is a fundamental task type that can be used in a variety of applications, such as cross-modal retrieval.

audio_embedding: AudioEmbedding = <towhee.runtime.hub_ops.audio_embedding.AudioEmbedding object>: Audio embedding is a task that encodes audio’s semantics into a set of real vectors. It is a fundamental task type that can be used in a variety of applications, including but not limited to reverse audio search and audio deduplication.

image_decode: ImageDecode = <towhee.runtime.hub_ops.image_decode.ImageDecode object>: image_decode operators convert an encoded image back to its uncompressed format. In most cases, image decoding is the first step of an image processing pipeline.

audio_decode: AudioDecode = <towhee.runtime.hub_ops.audio_decode.AudioDecode object>: Audio Decode converts the encoded audio back to uncompressed audio frames. In most cases, audio decoding is the first step of an audio processing pipeline.

video_decode: VideoDecode = <towhee.runtime.hub_ops.video_decode.VideoDecode object>: Video deocde, in most cases, video decode is the first step of an video processing pipeline.

sentence_embedding: SentenceEmbedding = <towhee.runtime.hub_ops.sentence_embedding.SentenceEmbedding object>: sentence_embedding is the extension of word or token embedding. Instead of generating a vector for each token or word, it represents semantic information of the whole text as a vector.

data_source: DataSource = <towhee.runtime.hub_ops.data_source.DataSource object>: data_source load data from many different sources. Work with DataLoader

ann_insert: AnnInsert = <towhee.runtime.hub_ops.ann_insert.AnnInsert object>: The ANN Insert Operator is used to insert embeddings and create ANN indexes for fast similarity searches.

ann_search: AnnSearch = <towhee.runtime.hub_ops.ann_search.AnnSearch object>: The ANN search operator is used to find the closest (or most similar) point to a given point in a given set, i.e. find similar embeddings.

object_detetction: ObjectDetection = <towhee.runtime.hub_ops.object_detection.ObjectDetection object>: Object detection is a computer vision technique that locates and identifies people, items, or other objects in an image. Object detection has applications in many areas of computer vision, including image retrieval, image annotation, vehicle counting, object tracking, etc.

utils: Utils = <towhee.runtime.hub_ops.utils.Utils object>: Some utils.

rerank: ReRank = <towhee.runtime.hub_ops.rerank.ReRank object>: Re-rank the search results based on relevance

LLM: <towhee.runtime.hub_ops.llm.LLM object at 0x7f839d68b460> = <towhee.runtime.hub_ops.llm.LLM object>: The LLM ops are designed to provide a standard interface for all of them.

prompt: Prompt = <towhee.runtime.hub_ops.prompt.Prompt object>: prompt template.

data_loader: DataLoader = <towhee.runtime.hub_ops.data_loader.DataLoader object>: Load data from various sources.

towhee.register(name: str | None = None, input_schema=None, output_schema=None, flag=None)

Register a class, function, or callable as a towhee operator.

Args:

name (str, optional): operator name, will use the class/function name if None.

input_schema, output_schema, flag for legacy operators.

Returns:

[type]: [description]

class towhee.operator.Operator[source]

Bases: ABC

Operator base class, implements __init__ and __call__,

Examples:

class AddOperator(Operator):

def __init__(self, factor: int):: self._factor = factor
def __call__(self, num): return self._factor + num

flush()[source]

This method is triggered by RuntimePipeline.flush().

After the pipeline runs, some operators may need to trigger some additional operations, such as the to_faiss operator needing to trigger data storage.

class towhee.operator.NNOperator(framework: str = 'pytorch')[source]

Bases: Operator

Neural Network related operators that involve machine learning frameworks.

Args:

framework (str):: The framework to apply.

save_model()[source]: Save model to local.

supported_model_names()[source]: Return a list of supported model names.

train(training_config=None, train_dataset=None, eval_dataset=None, resume_checkpoint_path=None, **kwargs)[source]

Start to train an operator.

Args:

training_config (TrainingConfig):: The config of this trainer.
train_dataset (Union[Dataset, TowheeDataSet]):: Training dataset.
eval_dataset (Union[Dataset, TowheeDataSet]):: Evaluate dataset.
resume_checkpoint_path (str):: If resuming training, pass into the path.
**kwargs (Any):: Keyword Args.

setup_trainer(training_config=None, train_dataset=None, eval_dataset=None, train_dataloader=None, eval_dataloader=None, model_card=None)[source]

Set up the trainer instance in operator before training and set trainer parameters. Args:

training_config (TrainingConfig):
The config of this trainer.

train_dataset (Union[Dataset, TowheeDataSet]):
Training dataset.

eval_dataset (Union[Dataset, TowheeDataSet]):
Evaluate dataset.

train_dataloader (Union[DataLoader, Iterable]):
If specified, Trainer will use it to load training data. Otherwise, Trainer will build dataloader from train_dataset.

eval_dataloader (Union[DataLoader, Iterable]):
If specified, Trainer will use it to load evaluate data. Otherwise, Trainer will build dataloader from train_dataset.

model_card (ModelCard):
Model card contains the training informations.

Returns:

class towhee.operator.PyOperator[source]

Bases: Operator

Python function operator, no machine learning frameworks involved.