ltx23_rotary_pos_embed

LTX-2.3 audio/video rotary position embedder

Description

Computes RoPE cos/sin frequency tensors from spatiotemporal patch coordinates. Video coordinates are 3D (frames scaled to seconds via fps, height, width in pixel space); audio coordinates are 1D (seconds). Coordinates are patch boundaries [start, end); the midpoint is used as the position.

Usage

ltx23_rotary_pos_embed(dim, patch_size = 1L, patch_size_t = 1L,
                       base_num_frames = 20L, base_height = 2048L,
                       base_width = 2048L, sampling_rate = 16000L,
                       hop_length = 160L, scale_factors = c(8L, 32L, 32L),
                       theta = 10000, causal_offset = 1L, modality = "video",
                       double_precision = TRUE, rope_type = "split",
                       num_attention_heads = 32L)

Arguments

  • dim: Integer. Rotary dimension (attention head dim x heads for split type at model level; see reference).
  • scale_factors: Integer vector. VAE (time, height, width) scale factors.
  • theta: Numeric. RoPE theta.
  • causal_offset: Integer. Temporal offset for the causal VAE (first frame has stride 1).
  • modality: “video” or “audio”.
  • double_precision: Logical. Compute base frequencies in float64.
  • rope_type: “split” (LTX 2.3) or “interleaved”.
  • num_attention_heads: Integer. Needed for the split layout.
  • patch_size,patch_size_t: Integers. Spatial/temporal patch sizes.
  • base_num_frames,base_height,base_width: Integers. Base grid the coordinates are normalized against.
  • sampling_rate,hop_length: Integers. Audio spectrogram params.