towhee.models.clip.simple_tokenizer.SimpleTokenizer

class towhee.models.clip.simple_tokenizer.SimpleTokenizer(bpe_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/towhee/checkouts/branch0.8.1/towhee/models/clip/bpe_simple_vocab_16e6.txt.gz')[source]

Bases: object

Simple Tokenizer for text input

Methods

bpe

decode

encode

__init__(bpe_path: str = '/home/docs/checkouts/readthedocs.org/user_builds/towhee/checkouts/branch0.8.1/towhee/models/clip/bpe_simple_vocab_16e6.txt.gz')[source]