text_encoder_native

Native CLIP Text Encoder

Description

Native R torch implementation of CLIP text encoder. Replaces TorchScript for better GPU compatibility.

Usage

text_encoder_native(vocab_size = 49408, context_length = 77, embed_dim = 768,
                    num_layers = 12, num_heads = 12, mlp_dim = 3072,
                    apply_final_ln = TRUE, gelu_type = "tanh")

Arguments

• vocab_size: Vocabulary size (default 49408)
• context_length: Maximum sequence length (default 77)
• embed_dim: Embedding dimension
• num_layers: Number of transformer layers
• num_heads: Number of attention heads
• mlp_dim: MLP hidden dimension
• apply_final_ln: Whether to apply final layer norm (default TRUE). Set to FALSE to match TorchScript exports that don’t include final LN.
• gelu_type: GELU variant: “tanh” (matches the TorchScript exports), “quick” (HF CLIP ViT-L, used by FLUX), or “exact”

Value

An nn_module representing the text encoder

• ← text_encoder2_native
• t5_encoder →