load_s3tokenizer_weights

Load S3Tokenizer weights from state dictionary

Description

Load S3Tokenizer weights from state dictionary

Usage

1load_s3tokenizer_weights(model, state_dict, prefix = "tokenizer.")

Arguments

  • model: S3Tokenizer model
  • state_dict: Named list of tensors
  • prefix: Prefix for weight keys (default “tokenizer.”)

Value

Model with loaded weights