Whisper BPE Tokenizer
Description
Byte-pair encoding tokenizer for Whisper models. Create Whisper Tokenizer Load or create a Whisper tokenizer from HuggingFace vocab files.
Usage
1whisper_tokenizer(model = "tiny")
Arguments
model: Model name for vocab lookup
Value
Tokenizer object (list with encode/decode functions)