towhee.serve.triton.triton_config_builder.EnsembleConfigBuilder

class towhee.serve.triton.triton_config_builder.EnsembleConfigBuilder(dag, model_name, max_batch_size)[source]

Bases: object

Create ensemble config. Currently, we just support a chain struct graph.

Example of input dag: triton_dag = { 0: {

‘id’: 0, ‘mode_name’: ‘image_decode’, ‘model_version’: 1, ‘inputs’: {

‘INPUT0’: (‘TYPE_STRING’, []),

}, ‘outputs’: {

‘OUTPUT0’: (‘TYPE_INT8’, [-1, -1, 3]), ‘OUTPUT1’: (‘TYPE_STRING’, []),

}, ‘child_ids’: [1]

}, 1: {

‘id’: 1, ‘mode_name’: ‘clip_preprocess’, ‘model_version’: 1, ‘inputs’: {

‘IUTPUT0’: (‘TYPE_INT8’, [-1, -1, 3]), ‘IUTPUT1’: (‘TYPE_STRING’, []),

}, ‘outputs’: {

‘OUTPUT0’: (‘TYPE_FP32’, [-1, 3, 224, 224])

}, ‘child_ids’: [2]

}, 2: {

‘id’: 2, ‘mode_name’: ‘clip_model’, ‘model_version’: 1, ‘inputs’: {

‘IUTPUT0’: (‘TYPE_FP32’, [-1, 3, 224, 224])

}, ‘outputs’: {

‘OUTPUT0’: (‘TYPE_FP32’, [-1, 512])

}, ‘child_ids’: [3]

}, 3: {

‘id’: 3, ‘mode_name’: ‘clip_postprocess’, ‘model_version’: 1, ‘inputs’: {

‘IUTPUT0’: (‘TYPE_FP32’, [-1, 512])

}, ‘outputs’: {

‘OUTPUT0’: (‘TYPE_FP32’, [512])

}, ‘child_ids’: []

}

}

Methods

gen_config

__init__(dag, model_name, max_batch_size)[source]

Initialize self. See help(type(self)) for accurate signature.