txt2img_sdxl

Generate an image from a text prompt using SDXL

Description

Generate an image from a text prompt using SDXL

Usage

txt2img_sdxl(prompt, negative_prompt = NULL, img_dim = 1024, pipeline = NULL,
             devices = "auto", memory_profile = NULL, unet_dtype_str = NULL,
             download_models = FALSE, scheduler = "ddim", timesteps = NULL,
             initial_latents = NULL, num_inference_steps = 30,
             guidance_scale = 7.5, seed = NULL, save_file = TRUE,
             filename = NULL, metadata_path = NULL, use_native_decoder = FALSE,
             use_native_text_encoder = FALSE, use_native_unet = FALSE,
             verbose = TRUE, ...)

Arguments

  • prompt: A character string prompt describing the image to generate.
  • negative_prompt: Optional negative prompt to guide the generation.
  • img_dim: Dimension of the output image (e.g., 512 for 512x512).
  • pipeline: Optional A pre-loaded diffusion pipeline. If NULL, it will be loaded based on the model name and devices.
  • devices: A named list of devices for each model component (e.g., list(unet = "cuda", decoder = "cpu", text_encoder = "cpu")), or “auto” to use auto_devices(), or NULL to use memory_profile devices.
  • memory_profile: Character or list. Memory profile for GPU-poor optimization: “auto” for auto-detection, or a profile name (“full_gpu”, “balanced”, “unet_gpu”, “cpu_only”), or a list from sdxl_memory_profile(). When specified, overrides devices parameter.
  • unet_dtype_str: Optional A character for dtype of the unet component (typically “float16” for cuda and “float32” for cpu; float32 is available for cuda).
  • download_models: Logical indicating whether to download the model files if they are not found.
  • scheduler: Scheduler to use (e.g., "ddim", "euler").
  • timesteps: Optional A vector of timesteps to use.
  • initial_latents: Optional initial latents for the diffusion process.
  • num_inference_steps: Number of inference steps to run.
  • guidance_scale: Scale for classifier-free guidance (typically 7.5).
  • seed: Optional seed for reproducibility.
  • save_file: Logical indicating whether to save the generated image.
  • filename: Optional filename for saving the image. If NULL, a default name is generated.
  • metadata_path: Optional file path to save metadata.
  • use_native_decoder: Logical; if TRUE, uses native R torch decoder instead of TorchScript. Native decoder has better GPU compatibility (especially Blackwell).
  • use_native_text_encoder: Logical; if TRUE, uses native R torch text encoder instead of TorchScript. Native text encoder has better GPU compatibility (especially Blackwell).
  • use_native_unet: Logical; if TRUE, uses native R torch UNet instead of TorchScript. Native UNet has better GPU compatibility (especially Blackwell).
  • verbose: Logical. Print progress and memory status messages.
  • ...: Additional parameters passed to the diffusion process.

Value

An image array and metadata

Examples

# Basic usage with auto-detection
img <- txt2img_sdxl("a cat wearing sunglasses in space")

# GPU-poor mode (8GB VRAM)
img <- txt2img_sdxl("a sunset over mountains", memory_profile = "unet_gpu")

# Explicit memory profile
profile <- sdxl_memory_profile(vram_gb = 8)
img <- txt2img_sdxl("a forest path", memory_profile = profile)