User Guide:
User API:
Developer API:
Full API:
Functions
convert_tokens_to_id
Convert tokens to token ID.
tokenize
Use SimpleTokenizer to tokenize text.