whisper_tokenizer

Whisper BPE Tokenizer

Description

Byte-pair encoding tokenizer for Whisper models. Create Whisper Tokenizer Load or create a Whisper tokenizer from HuggingFace vocab files.

Usage

1whisper_tokenizer(model = "tiny")

Arguments

  • model: Model name for vocab lookup

Value

Tokenizer object (list with encode/decode functions)