ltx23_attention

LTX-2 attention layer

Description

Attention with RMS q/k norms across heads, optional per-head output gating (LTX-2.3), separate query/key RoPE (for a2v/v2a cross attention), and optional STG perturbation (skip attention, use the value projection).

Usage

ltx23_attention(query_dim, heads = 8L, kv_heads = NULL, dim_head = 64L,
                bias = TRUE, cross_attention_dim = NULL, out_bias = TRUE,
                norm_eps = 1e-06, norm_elementwise_affine = TRUE,
                rope_type = "split", apply_gated_attention = FALSE)

Arguments

  • query_dim: Integer. Query feature dimension.
  • dim_head: Integer. Per-head dimension.
  • cross_attention_dim: Integer or NULL. Key/value input dimension (NULL for self-attention).
  • norm_eps: Numeric. RMS norm epsilon.
  • norm_elementwise_affine: Logical. RMS norms carry weights.
  • rope_type: “split” or “interleaved”.
  • apply_gated_attention: Logical. Add per-head sigmoid output gates.
  • heads,kv_heads: Integers. Attention head counts.
  • bias,out_bias: Logicals. Projection biases.