txt2img_sd21

Generate an image from a text prompt using a diffusion pipeline

Description

This function generates an image based on a text prompt using the Stable Diffusion model. It allows for various configurations such as model name, device, scheduler, and more.

Usage

txt2img_sd21(prompt, negative_prompt = NULL, img_dim = 768, pipeline = NULL,
             devices = "auto", unet_dtype_str = NULL, download_models = FALSE,
             scheduler = "ddim", timesteps = NULL, initial_latents = NULL,
             num_inference_steps = 50, guidance_scale = 7.5, seed = NULL,
             save_file = TRUE, filename = NULL, metadata_path = NULL,
             use_native_decoder = FALSE, use_native_text_encoder = FALSE,
             use_native_unet = FALSE, ...)

Arguments

  • prompt: A character string prompt describing the image to generate.
  • negative_prompt: Optional negative prompt to guide the generation.
  • img_dim: Dimension of the output image (e.g., 512 for 512x512).
  • pipeline: Optional A pre-loaded diffusion pipeline. If NULL, it will be loaded based on the model name and devices.
  • devices: A named list of devices for each model component (e.g., list(unet = "cuda", decoder = "cpu", text_encoder = "cpu")).
  • unet_dtype_str: Optional A character for dtype of the unet component (typically “float16” for cuda and “float32” for cpu; float32 is available for cuda).
  • download_models: Logical indicating whether to download the model files if they are not found.
  • scheduler: Scheduler to use (e.g., "ddim", "euler").
  • timesteps: Optional A vector of timesteps to use.
  • initial_latents: Optional initial latents for the diffusion process.
  • num_inference_steps: Number of inference steps to run.
  • guidance_scale: Scale for classifier-free guidance (typically 7.5).
  • seed: Optional seed for reproducibility.
  • save_file: Logical indicating whether to save the generated image.
  • filename: Optional filename for saving the image. If NULL, a default name is generated.
  • metadata_path: Optional file path to save metadata.
  • use_native_decoder: Logical; if TRUE, uses native R torch decoder instead of TorchScript. Native decoder has better GPU compatibility (especially Blackwell).
  • use_native_text_encoder: Logical; if TRUE, uses native R torch text encoder instead of TorchScript. Native text encoder has better GPU compatibility (especially Blackwell).
  • use_native_unet: Logical; if TRUE, uses native R torch UNet instead of TorchScript. Native UNet has better GPU compatibility (especially Blackwell).
  • ...: Additional parameters passed to the diffusion process.

Value

An image array and metadata

Examples

img <- txt2img("a cat wearing sunglasses in space", device = "cuda")