Image Generation

ImageClient allows you to generate images using natural language descriptions, perfect for creating game art assets, concept art, or dynamic content.

Create ImageClient

import { PlayKitSDK } from 'playkit-sdk';

const sdk = new PlayKitSDK({
  gameId: 'your-game-id',
  developerToken: 'your-token'
});

await sdk.initialize();

// Use default model
const imageClient = sdk.createImageClient();

// Specify model
const imageClient = sdk.createImageClient('dall-e-3');

Text-to-Image

Generate images from text descriptions using natural language prompts.

Simple Generation

The simplest approach - just provide a prompt:

const image = await imageClient.generate('A futuristic cyberpunk city at night');

console.log('Base64:', image.base64);
console.log('Prompt:', image.originalPrompt);
console.log('Generated at:', new Date(image.generatedAt));

Display in Browser

// Method 1: Convert to HTMLImageElement
const htmlImg = await image.toHTMLImage();
document.body.appendChild(htmlImg);

// Method 2: Use Data URL
const img = document.createElement('img');
img.src = image.toDataURL();
document.body.appendChild(img);

Use in Canvas

const canvas = document.querySelector('canvas');
const ctx = canvas.getContext('2d');

const htmlImg = await image.toHTMLImage();

// Wait for image to load
htmlImg.onload = () => {
  ctx.drawImage(htmlImg, 0, 0, canvas.width, canvas.height);
};

Configuration Options

Image Size

Multiple image sizes are supported:

// Square
const square = await imageClient.generateImage({
  prompt: 'A dragon',
  size: '1024x1024'  // 256x256, 512x512, 1024x1024
});

// Landscape
const landscape = await imageClient.generateImage({
  prompt: 'A mountain range',
  size: '1792x1024'
});

// Portrait
const portrait = await imageClient.generateImage({
  prompt: 'A wizard tower',
  size: '1024x1792'
});

Size	Dimensions	Use Case
`256x256`	256 × 256	Thumbnails, icons
`512x512`	512 × 512	Small assets
`1024x1024`	1024 × 1024	Standard images (recommended)
`1792x1024`	1792 × 1024	Landscape scenes, backgrounds
`1024x1792`	1024 × 1792	Portrait scenes, characters

Different models support different image sizes. Please check the official website for model-specific supported sizes.

Random Seed

Use seed for reproducible images:

const image1 = await imageClient.generateImage({
  prompt: 'A magical forest',
  seed: 42
});

const image2 = await imageClient.generateImage({
  prompt: 'A magical forest',
  seed: 42
});

// image1 and image2 should be identical or very similar

Uses:

Generate consistent images for specific game content
Debugging and testing
Players can share seeds to reproduce images

Quality Setting (DALL-E 3)

// Standard quality (faster, cheaper)
const standard = await imageClient.generateImage({
  prompt: 'A castle',
  quality: 'standard'
});

// HD quality (slower, more expensive, more detailed)
const hd = await imageClient.generateImage({
  prompt: 'A castle',
  quality: 'hd'
});

Transparent Background

Generate images with transparent backgrounds, useful for sprites and UI elements:

const sprite = await imageClient.generateImage({
  prompt: 'A fantasy sword, game item',
  size: '512x512',
  transparent: true
});

// Check if background removal was successful
if (sprite.transparentSuccess) {
  console.log('Background removed successfully');
} else {
  console.log('Background removal may not be complete');
}

// Original image before background removal is also available
console.log('Original:', sprite.originalBase64);

Transparent background works best with images that have clear subjects and simple backgrounds.

Image-to-Image

Transform existing images using AI. Provide a reference image and optionally a text prompt to guide the transformation.

Basic Image-to-Image

// Convert a data URL to ImageInput format
const imageInput = PlayKitSDK.ImageClient.dataUrlToImageInput(dataUrl);

// Transform the image
const result = await imageClient.img2img([imageInput]);

With Text Guidance

Add a text prompt to guide the transformation:

const imageInput = PlayKitSDK.ImageClient.dataUrlToImageInput(dataUrl);

const result = await imageClient.img2img(
  [imageInput],
  'Transform into pixel art style'
);

With Options

Combine with size and other options:

const imageInput = PlayKitSDK.ImageClient.dataUrlToImageInput(dataUrl);

const result = await imageClient.img2img(
  [imageInput],
  'A futuristic version of this scene',
  {
    size: '1024x1024',
    transparent: true  // Remove background from result
  }
);

Loading Image from File Input

Handle file uploads in the browser:

const fileInput = document.querySelector('#image-input');

fileInput.addEventListener('change', async (e) => {
  const file = e.target.files[0];
  if (!file) return;

  // Validate file type
  if (!file.type.startsWith('image/')) {
    console.error('Please select a valid image file');
    return;
  }

  // Read file as data URL
  const reader = new FileReader();
  reader.onload = async (event) => {
    const dataUrl = event.target.result;

    // Convert to ImageInput
    const imageInput = PlayKitSDK.ImageClient.dataUrlToImageInput(dataUrl);

    // Transform the image
    const result = await imageClient.img2img(
      [imageInput],
      'Apply watercolor effect'
    );

    // Display result
    const img = document.createElement('img');
    img.src = result.toDataURL();
    document.body.appendChild(img);
  };

  reader.readAsDataURL(file);
});

ImageInput Format

The ImageInput interface expects base64-encoded image data:

interface ImageInput {
  data: string;        // Base64-encoded image data (without data URL prefix)
  mediaType?: string;  // e.g., 'image/png', 'image/jpeg'
}

// Helper method to convert data URL to ImageInput
const imageInput = PlayKitSDK.ImageClient.dataUrlToImageInput(
  'data:image/png;base64,iVBORw0KG...'
);
// Returns: { mediaType: 'image/png', data: 'iVBORw0KG...' }

Use Cases

Style Transfer

const result = await imageClient.img2img(
  [imageInput],
  'Convert to anime style'
);

Image Enhancement

const result = await imageClient.img2img(
  [imageInput],
  'Enhance details and improve quality'
);

Scene Transformation

const result = await imageClient.img2img(
  [imageInput],
  'Transform to night time scene with moonlight'
);

Character Variation

const result = await imageClient.img2img(
  [imageInput],
  'Same character in different pose'
);

Batch Generation

Generate multiple images at once (1-10):

const images = await imageClient.generateImages({
  prompt: 'A fantasy character',
  n: 4,
  size: '1024x1024'
});

// Display all images
images.forEach((image, index) => {
  const img = document.createElement('img');
  img.src = image.toDataURL();
  img.alt = `Generated image ${index + 1}`;
  document.body.appendChild(img);
});

Batch generation is useful for scenarios requiring multiple variations, like character design or concept art.

Complete Configuration Example

const image = await imageClient.generateImage({
  // Prompt (required)
  prompt: 'A majestic dragon perched on a mountain peak, sunset lighting, fantasy art style',

  // Image size
  size: '1024x1024',

  // Random seed (optional)
  seed: 123,

  // Model (optional, defaults to client creation model)
  model: 'dall-e-3',

  // Quality (DALL-E 3 only)
  quality: 'hd',

  // Style (DALL-E 3 only)
  style: 'vivid'
});

console.log('Original prompt:', image.originalPrompt);
console.log('Revised prompt:', image.revisedPrompt); // AI may enhance your prompt
console.log('Size:', image.size);

GeneratedImage Object

Each generated image is a GeneratedImage object:

interface GeneratedImage {
  // Raw base64 data
  base64: string;

  // Original prompt
  originalPrompt: string;

  // AI-revised prompt (may differ from original)
  revisedPrompt?: string;

  // Generation timestamp
  generatedAt: number;

  // Image size
  size?: ImageSize;

  // Original image before background removal (if transparent was used)
  originalBase64?: string;

  // Whether background removal was successful (if transparent was used)
  transparentSuccess?: boolean;

  // Convert to Data URL
  toDataURL(): string;

  // Convert to HTMLImageElement
  toHTMLImage(): Promise<HTMLImageElement>;
}

Usage Examples

const image = await imageClient.generate('A robot');

// Get base64
const base64 = image.base64;

// Data URL (for img src)
const dataUrl = image.toDataURL();
// "data:image/png;base64,iVBORw0KG..."

// HTMLImageElement (for DOM)
const htmlImg = await image.toHTMLImage();
document.body.appendChild(htmlImg);

// See how AI understood your prompt
console.log('Your prompt:', image.originalPrompt);
console.log('AI understood as:', image.revisedPrompt);

Framework Integration

P5.js

let generatedImage;

async function setup() {
  createCanvas(800, 600);

  const sdk = new PlayKitSDK({ /* ... */ });
  await sdk.initialize();

  const imageClient = sdk.createImageClient();
  const img = await imageClient.generate('A magical forest');

  // Convert to P5-compatible format
  const htmlImg = await img.toHTMLImage();
  generatedImage = loadImage(htmlImg.src);
}

function draw() {
  background(220);

  if (generatedImage) {
    image(generatedImage, 0, 0, width, height);
  }
}

Phaser

class GameScene extends Phaser.Scene {
  async create() {
    const sdk = new PlayKitSDK({ /* ... */ });
    await sdk.initialize();

    const imageClient = sdk.createImageClient();
    const img = await imageClient.generate('A game character');

    // Use Data URL
    const dataUrl = img.toDataURL();

    // Load into Phaser
    this.textures.addBase64('generated', dataUrl);
    this.add.image(400, 300, 'generated');
  }
}

React

import { useState } from 'react';
import { PlayKitSDK } from 'playkit-sdk';

function ImageGenerator() {
  const [imageSrc, setImageSrc] = useState('');
  const [loading, setLoading] = useState(false);

  const generateImage = async (prompt) => {
    setLoading(true);
    try {
      const sdk = new PlayKitSDK({ /* ... */ });
      await sdk.initialize();

      const imageClient = sdk.createImageClient();
      const image = await imageClient.generate(prompt);

      setImageSrc(image.toDataURL());
    } catch (error) {
      console.error('Generation failed:', error);
    } finally {
      setLoading(false);
    }
  };

  return (
    <div>
      <button onClick={() => generateImage('A castle')}>
        Generate Image
      </button>
      {loading && <p>Generating...</p>}
      {imageSrc && <img src={imageSrc} alt="Generated" />}
    </div>
  );
}

Error Handling

import { PlayKitError } from 'playkit-sdk';

try {
  const image = await imageClient.generate('A dragon');
} catch (error) {
  if (error instanceof PlayKitError) {
    switch (error.code) {
      case 'NOT_AUTHENTICATED':
        console.log('Need to login');
        break;
      case 'IMAGE_GENERATION_ERROR':
        console.log('Generation failed:', error.message);
        // Possible reasons:
        // - Prompt violates content policy
        // - Service temporarily unavailable
        // - Quota exhausted
        break;
      default:
        console.log('Unknown error');
    }
  }
}

Best Practices

1. Cache Generated Images

const cache = new Map();

async function getCachedImage(prompt, seed) {
  const key = `${prompt}:${seed}`;

  if (cache.has(key)) {
    return cache.get(key);
  }

  const image = await imageClient.generateImage({ prompt, seed });
  cache.set(key, image);
  return image;
}

2. Show Loading State

// Image generation may take 5-30 seconds
const loadingEl = document.querySelector('#loading');
loadingEl.style.display = 'block';

try {
  const image = await imageClient.generate('A castle');
  displayImage(image);
} finally {
  loadingEl.style.display = 'none';
}

3. Error Fallback

try {
  const image = await imageClient.generate(userPrompt);
  return image;
} catch (error) {
  // Use default placeholder image
  return fallbackImage;
}

4. Optimize Prompts

function optimizePrompt(userInput) {
  // Add style and quality descriptions
  return `${userInput}, high quality, detailed, professional`;
}

const image = await imageClient.generate(
  optimizePrompt('A forest')
);

Performance and Cost Optimization

Choose Appropriate Size

// Good: choose based on actual needs
// Thumbnail
const thumb = await imageClient.generateImage({
  prompt: 'Icon',
  size: '256x256'
});

// Background
const bg = await imageClient.generateImage({
  prompt: 'Background',
  size: '1792x1024'
});

// Bad: always use largest size
const img = await imageClient.generateImage({
  prompt: 'Icon',
  size: '1792x1024'  // Wasteful
});

Use Standard Quality

// Good: standard quality for most scenarios
const image = await imageClient.generateImage({
  prompt: 'Game asset',
  quality: 'standard'
});

// Bad: unnecessary HD
const image = await imageClient.generateImage({
  prompt: 'Simple icon',
  quality: 'hd'  // More expensive, probably not needed
});

Next Steps

Check NPC Conversations combined with image generation

Image Generation

On this page