PlayKit.ai

API Reference

Complete API documentation for the PlayKit Unity SDK

API Reference

PlayKitSDK

The main entry point for the SDK, using singleton pattern.

Static Methods

InitializeAsync

Initialize the SDK.

public static async UniTask<bool> InitializeAsync(string developerToken = null)

Parameters:

  • developerToken (string, optional) - Developer Token for development environment. Omit for production.

Returns:

  • UniTask<bool> - Whether initialization succeeded

Example:

// Wait for SDK to finish initializing (editor token or player login handled automatically)
await PlayKitSDK.InitializeAsync();

IsReady

Check if SDK is initialized and ready.

public static bool IsReady()

Returns:

  • bool - Whether SDK is ready

GetPlayerClient

Get the player client instance.

public static PlayKit_PlayerClient GetPlayerClient()

Returns:

  • PlayKit_PlayerClient - Player client instance

Factory Methods (PlayKitSDK.Factory)

CreateChatClient

Create a chat client.

public static PlayKit_AIChatClient CreateChatClient(string modelName = null)

Parameters:

  • modelName (string, optional) - Model name. Uses default model if null.

Returns:

  • PlayKit_AIChatClient - Chat client instance

CreateImageClient

Create an image generation client.

public static PlayKit_AIImageClient CreateImageClient(string modelName = null)

Parameters:

  • modelName (string, optional) - Model name. Uses default model if null.

Returns:

  • PlayKit_AIImageClient - Image generation client instance

CreateTranscriptionClient

Create an audio transcription client.

public static PlayKit_AudioTranscriptionClient CreateTranscriptionClient(string modelName)

Parameters:

  • modelName (string) - Model name (typically "whisper-1")

Returns:

  • PlayKit_AudioTranscriptionClient - Transcription client instance

CreateTextToSpeechClient

Create a text-to-speech client.

public static PlayKit_TextToSpeechClient CreateTextToSpeechClient(string modelName = null)

Parameters:

  • modelName (string, optional) - Model name. Uses the default model alias default-tts-model if null.

Returns:

  • PlayKit_TextToSpeechClient - Text-to-speech client instance

Populate Methods (PlayKitSDK.Populate)

CreateNpc

Create or configure an NPC client.

public static void CreateNpc(PlayKit_NPC recipient, string modelName = null)

Parameters:

  • recipient (PlayKit_NPC) - NPC component instance
  • modelName (string, optional) - Model name

PlayKit_AIChatClient

Handles AI text generation and conversations.

Methods

TextGenerationAsync

Generate AI text response.

public async UniTask<PlayKit_AIResult<string>> TextGenerationAsync(
    PlayKit_ChatConfig config,
    CancellationToken cancellationToken
)

Parameters:

  • config (PlayKit_ChatConfig) - Chat configuration
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<PlayKit_AIResult<string>> - Result containing AI response

Example:

var config = new PlayKit_ChatConfig
{
    Messages = messages,
    Temperature = 0.7f
};

var result = await chatClient.TextGenerationAsync(config, cancellationToken);
if (result.Success)
{
    Debug.Log(result.Response);
}

TextChatStreamAsync

Stream AI text response.

public async UniTask TextChatStreamAsync(
    PlayKit_ChatStreamConfig config,
    Action<string> onNewChunk,
    Action<string> onConcluded,
    CancellationToken cancellationToken
)

Parameters:

  • config (PlayKit_ChatStreamConfig) - Streaming chat configuration
  • onNewChunk (Action<string>) - Callback when new text chunk is received
  • onConcluded (Action<string>) - Callback when full response is complete
  • cancellationToken (CancellationToken) - Cancellation token

GenerateStructuredAsync (JObject)

Generate structured output (returns JObject).

public async UniTask<JObject> GenerateStructuredAsync(
    string schemaName,
    string prompt,
    float temperature = 0.7f,
    CancellationToken cancellationToken = default
)

Parameters:

  • schemaName (string) - Schema name
  • prompt (string) - Prompt
  • temperature (float, optional) - Temperature value (default 0.7)
  • cancellationToken (CancellationToken, optional) - Cancellation token

Returns:

  • UniTask<JObject> - JSON object

GenerateStructuredAsync<T> (Generic)

Generate structured output (returns typed object).

public async UniTask<T> GenerateStructuredAsync<T>(
    string schemaName,
    string prompt,
    float temperature = 0.7f,
    CancellationToken cancellationToken = default
)

Parameters:

  • schemaName (string) - Schema name
  • prompt (string) - Prompt
  • temperature (float, optional) - Temperature value
  • cancellationToken (CancellationToken, optional) - Cancellation token

Returns:

  • UniTask<T> - Typed object

GenerateStructuredWithSchemaAsync

Generate structured output using custom JSON Schema.

public async UniTask<JObject> GenerateStructuredWithSchemaAsync(
    string schemaJson,
    string prompt,
    float temperature = 0.7f,
    CancellationToken cancellationToken = default
)

HasSchema

Check if schema exists.

public bool HasSchema(string schemaName)

GetAvailableSchemas

Get all available schema names.

public string[] GetAvailableSchemas()

SetSchemaLibrary

Set the Schema Library.

public void SetSchemaLibrary(PlayKit_SchemaLibrary schemaLibrary)

Static Methods

PrintPrettyChatMessages

Format and print conversation messages.

public static void PrintPrettyChatMessages(
    List<PlayKit_ChatMessage> messages,
    string title = "Chat Messages"
)

PlayKit_AIImageClient

Handles AI image generation.

Methods

GenerateImageAsync

Generate a single image.

public async UniTask<PlayKit_GeneratedImage> GenerateImageAsync(
    string prompt,
    string size,
    int? seed,
    CancellationToken cancellationToken
)

Parameters:

  • prompt (string) - Image description
  • size (string) - Image size (e.g., "1024x1024")
  • seed (int?, optional) - Random seed
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<PlayKit_GeneratedImage> - Generated image

Supported sizes:

  • "256x256"
  • "512x512"
  • "1024x1024"
  • "1792x1024"
  • "1024x1792"

GenerateImagesAsync

Generate multiple images.

public async UniTask<List<PlayKit_GeneratedImage>> GenerateImagesAsync(
    string prompt,
    int count,
    string size,
    string aspectRatio,
    int? seed,
    CancellationToken cancellationToken
)

Parameters:

  • prompt (string) - Image description
  • count (int) - Number to generate
  • size (string) - Image size
  • aspectRatio (string, optional) - Aspect ratio
  • seed (int?, optional) - Random seed
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<List<PlayKit_GeneratedImage>> - List of generated images

Static Methods

Base64ToTexture2D

Convert Base64 string to Texture2D.

public static Texture2D Base64ToTexture2D(string base64Data)

Texture2DToSprite

Convert Texture2D to Sprite.

public static Sprite Texture2DToSprite(Texture2D texture)

PlayKit_AudioTranscriptionClient

Handles audio-to-text conversion.

Methods

TranscribeAsync

Transcribe raw audio data.

public async UniTask<PlayKit_TranscriptionResult> TranscribeAsync(
    byte[] audioData,
    string language,
    string prompt,
    CancellationToken cancellationToken
)

Parameters:

  • audioData (byte[]) - Audio data (WAV format)
  • language (string, optional) - Source language code (e.g., "zh", "en")
  • prompt (string, optional) - Context hint
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<PlayKit_TranscriptionResult> - Transcription result

TranscribeAudioClipAsync

Transcribe Unity AudioClip.

public async UniTask<PlayKit_TranscriptionResult> TranscribeAudioClipAsync(
    AudioClip audioClip,
    string language,
    string prompt,
    CancellationToken cancellationToken
)

Parameters:

  • audioClip (AudioClip) - Unity audio clip
  • language (string, optional) - Source language code
  • prompt (string, optional) - Context hint
  • cancellationToken (CancellationToken) - Cancellation token

PlayKit_TextToSpeechClient

Handles text-to-speech synthesis.

Methods

SynthesizeAsync

Synthesize text into audio and return the raw result.

public async UniTask<PlayKit_SpeechResult> SynthesizeAsync(
    string text,
    string voice,
    float? speed,
    CancellationToken cancellationToken
)

Parameters:

  • text (string) - Text to synthesize (maximum 10000 characters)
  • voice (string, optional) - Voice id. Uses the default voice male-qn-qingse if null.
  • speed (float?, optional) - Playback speed. Uses the default speed if null.
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<PlayKit_SpeechResult> - Synthesis result

SynthesizeToAudioClipAsync

Synthesize text and decode it into a Unity AudioClip.

public async UniTask<AudioClip> SynthesizeToAudioClipAsync(
    string text,
    string voice,
    float? speed,
    CancellationToken cancellationToken
)

Parameters:

  • text (string) - Text to synthesize (maximum 10000 characters)
  • voice (string, optional) - Voice id. Uses the default voice male-qn-qingse if null.
  • speed (float?, optional) - Playback speed. Uses the default speed if null.
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<AudioClip> - Decoded audio clip, ready to play through an AudioSource

PlayKit_NPC

Simplified NPC conversation client (MonoBehaviour).

Properties

IsTalking

public bool IsTalking { get; }

Whether NPC is processing a conversation.

IsReady

public bool IsReady { get; }

Whether NPC is initialized.

CharacterDesign

public string CharacterDesign { get; }

Get current system prompt (character design).

Methods

SetCharacterDesign

Set NPC's character and behavior.

public void SetCharacterDesign(string prompt)

Parameters:

  • prompt (string) - System prompt

Example:

npc.SetCharacterDesign("You are a friendly villager");

Talk

Converse with NPC.

public async UniTask<string> Talk(
    string message,
    CancellationToken cancellationToken
)

Parameters:

  • message (string) - Player message
  • cancellationToken (CancellationToken) - Cancellation token

Returns:

  • UniTask<string> - NPC's response

TalkStream

Streaming conversation.

public async UniTask TalkStream(
    string message,
    Action<string> onChunk,
    Action<string> onComplete,
    CancellationToken cancellationToken
)

Parameters:

  • message (string) - Player message
  • onChunk (Action<string>) - Callback when new text chunk is received
  • onComplete (Action<string>) - Callback when full response is complete
  • cancellationToken (CancellationToken) - Cancellation token

ClearHistory

Clear conversation history.

public void ClearHistory()

GetHistory

Get conversation history.

public PlayKit_ChatMessage[] GetHistory()

Returns:

  • PlayKit_ChatMessage[] - Conversation history array

GetHistoryLength

Get number of history messages.

public int GetHistoryLength()

RevertHistory

Undo last message.

public bool RevertHistory()

Returns:

  • bool - Whether undo succeeded

RevertChatMessages

Undo multiple messages.

public int RevertChatMessages(int count)

Parameters:

  • count (int) - Number of messages to undo

Returns:

  • int - Actual number of messages undone

AppendChatMessage

Manually add message to history.

public void AppendChatMessage(string role, string content)

Parameters:

  • role (string) - Message role ("user", "assistant", "system")
  • content (string) - Message content

SaveHistory

Save conversation history.

public string SaveHistory()

Returns:

  • string - Serialized history data

LoadHistory

Load conversation history.

public bool LoadHistory(string saveData)

Parameters:

  • saveData (string) - Serialized history data

Returns:

  • bool - Whether load succeeded

PrintPrettyChatMessages

Print conversation history.

public void PrintPrettyChatMessages(string title)

PlayKit_PlayerClient

Manages player information and authentication (MonoBehaviour).

Methods

InitializeAsync

Initialize with JWT and exchange for player token.

public async UniTask<(bool, string)> InitializeAsync(
    string jwt,
    CancellationToken cancellationToken
)

Returns:

  • UniTask<(bool, string)> - (success, error message)

GetPlayerInfoAsync

Get player information.

public async UniTask<ApiResult<PlayerInfo>> GetPlayerInfoAsync(
    CancellationToken cancellationToken
)

Returns:

  • UniTask<ApiResult<PlayerInfo>> - Player info result

GetCachedPlayerInfo

Get cached player information.

public PlayerInfo GetCachedPlayerInfo()

Returns:

  • PlayerInfo - Cached player info

HasValidPlayerToken

Check if valid player token exists.

public bool HasValidPlayerToken()

GetPlayerToken

Get player token.

public string GetPlayerToken()

SetPlayerToken

Set player token.

public void SetPlayerToken(string token)

Events

OnPlayerInfoUpdated

public event Action<PlayerInfo> OnPlayerInfoUpdated

Triggered when player info is updated.

OnPlayerTokenReceived

public event Action<string> OnPlayerTokenReceived

Triggered when new player token is received.

OnError

public event Action<string> OnError

Triggered when error occurs.


PlayKit_MicrophoneRecorder

Microphone recording component (MonoBehaviour).

Properties

IsRecording

public bool IsRecording { get; }

Whether currently recording.

RecordingTime

public float RecordingTime { get; }

Current recording duration (seconds).

CurrentDevice

public string CurrentDevice { get; }

Current microphone device name.

LastRecording

public AudioClip LastRecording { get; }

Last recorded audio.

Configuration Fields

maxRecordingSeconds

public float maxRecordingSeconds = 60f

Maximum recording duration (seconds).

sampleRate

public int sampleRate = 16000

Sample rate (Hz). 16000 recommended for Whisper.

useVAD

public bool useVAD = true

Whether to enable Voice Activity Detection.

silenceThreshold

public float silenceThreshold = 0.01f

Silence threshold (0.0-1.0).

maxSilenceDuration

public float maxSilenceDuration = 2.0f

Auto-stop recording after this many seconds of silence.

Methods

StartRecording

Start recording.

public bool StartRecording(string deviceName = null)

Parameters:

  • deviceName (string, optional) - Microphone device name. null uses default device.

Returns:

  • bool - Whether recording started successfully

StopRecording

Stop recording.

public AudioClip StopRecording()

Returns:

  • AudioClip - Recorded audio

CancelRecording

Cancel recording.

public void CancelRecording()

GetCurrentVolume

Get current volume level.

public float GetCurrentVolume()

Returns:

  • float - Volume level (0.0-1.0)

SetMicrophoneDevice

Set microphone device.

public void SetMicrophoneDevice(string deviceName)

Static Methods

GetAvailableDevices

Get all available microphone devices.

public static string[] GetAvailableDevices()

Returns:

  • string[] - Array of device names

Events

OnRecordingStarted

public event Action OnRecordingStarted

Triggered when recording starts.

OnRecordingStopped

public event Action<AudioClip> OnRecordingStopped

Triggered when recording stops.

OnVolumeChanged

public event Action<float> OnVolumeChanged

Triggered when volume changes.


Data Types

PlayKit_ChatMessage

Conversation message.

public class PlayKit_ChatMessage
{
    public string Role;      // "system", "user", "assistant"
    public string Content;   // Message content
}

PlayKit_ChatConfig

Chat configuration.

public class PlayKit_ChatConfig
{
    public List<PlayKit_ChatMessage> Messages { get; set; }
    public float Temperature { get; set; } = 0.7f;
}

PlayKit_ChatStreamConfig

Streaming chat configuration.

public class PlayKit_ChatStreamConfig
{
    public List<PlayKit_ChatMessage> Messages { get; set; }
    public float Temperature { get; set; } = 0.7f;
}

PlayKit_AIResult<T>

API call result wrapper.

public class PlayKit_AIResult<T>
{
    public bool Success { get; }
    public T Response { get; }
    public string ErrorMessage { get; }
}

PlayKit_GeneratedImage

Generated image.

public class PlayKit_GeneratedImage
{
    public string ImageBase64 { get; set; }
    public string OriginalPrompt { get; set; }
    public string RevisedPrompt { get; set; }
    public DateTime GeneratedAt { get; set; }

    public Texture2D ToTexture2D()
    public Sprite ToSprite()
}

PlayKit_TranscriptionResult

Transcription result.

public class PlayKit_TranscriptionResult
{
    public bool Success { get; }
    public string Text { get; }
    public string Language { get; }
    public float? DurationInSeconds { get; }
    public PlayKit_TranscriptionSegment[] Segments { get; }
    public string Error { get; }
}

PlayKit_TranscriptionSegment

Transcription segment.

public class PlayKit_TranscriptionSegment
{
    public float Start;  // Start time (seconds)
    public float End;    // End time (seconds)
    public string Text;  // Segment text
}

PlayKit_SpeechResult

Text-to-speech synthesis result.

public class PlayKit_SpeechResult
{
    public bool Success { get; }
    public byte[] AudioData { get; }        // Raw audio bytes
    public string Format { get; }           // Output format (e.g. "mp3")
    public int UsageCharacters { get; }     // Characters billed for this request
    public float? AudioLengthMs { get; }    // Audio duration in milliseconds
    public string Error { get; }
}

PlayerInfo

Player information.

public class PlayerInfo
{
    public string UserId { get; set; }
    public float Credits { get; set; }
}

Exceptions

PlayKitException

SDK base exception.

public class PlayKitException : Exception

PlayKitApiErrorException

API error exception.

public class PlayKitApiErrorException : PlayKitException

PlayKitImageSizeValidationException

Image size validation exception.

public class PlayKitImageSizeValidationException : PlayKitApiErrorException

Usage Examples

Basic Chat

var chatClient = PlayKitSDK.Factory.CreateChatClient();

var config = new PlayKit_ChatConfig
{
    Messages = new List<PlayKit_ChatMessage>
    {
        new PlayKit_ChatMessage { Role = "user", Content = "Hello" }
    }
};

var result = await chatClient.TextGenerationAsync(config, cancellationToken);
if (result.Success)
{
    Debug.Log(result.Response);
}

NPC Conversation

var npc = GetComponent<PlayKit_NPC>();
npc.SetCharacterDesign("You are a friendly merchant");

string reply = await npc.Talk("Hello", cancellationToken);
Debug.Log(reply);

Image Generation

var imageClient = PlayKitSDK.Factory.CreateImageClient();

var result = await imageClient.GenerateImageAsync(
    "a beautiful landscape",
    "1024x1024",
    null,
    cancellationToken
);

if (result != null)
{
    image.sprite = result.ToSprite();
}

Speech Recognition

var recorder = GetComponent<PlayKit_MicrophoneRecorder>();
recorder.StartRecording();

// ... wait for recording to complete ...

AudioClip clip = recorder.StopRecording();

var transcriptionClient = PlayKitSDK.Factory.CreateTranscriptionClient("whisper-1");
var result = await transcriptionClient.TranscribeAudioClipAsync(
    clip,
    "en",
    null,
    cancellationToken
);

if (result.Success)
{
    Debug.Log(result.Text);
}

Next Steps

On this page

API ReferencePlayKitSDKStatic MethodsInitializeAsyncIsReadyGetPlayerClientFactory Methods (PlayKitSDK.Factory)CreateChatClientCreateImageClientCreateTranscriptionClientCreateTextToSpeechClientPopulate Methods (PlayKitSDK.Populate)CreateNpcPlayKit_AIChatClientMethodsTextGenerationAsyncTextChatStreamAsyncGenerateStructuredAsync (JObject)GenerateStructuredAsync<T> (Generic)GenerateStructuredWithSchemaAsyncHasSchemaGetAvailableSchemasSetSchemaLibraryStatic MethodsPrintPrettyChatMessagesPlayKit_AIImageClientMethodsGenerateImageAsyncGenerateImagesAsyncStatic MethodsBase64ToTexture2DTexture2DToSpritePlayKit_AudioTranscriptionClientMethodsTranscribeAsyncTranscribeAudioClipAsyncPlayKit_TextToSpeechClientMethodsSynthesizeAsyncSynthesizeToAudioClipAsyncPlayKit_NPCPropertiesIsTalkingIsReadyCharacterDesignMethodsSetCharacterDesignTalkTalkStreamClearHistoryGetHistoryGetHistoryLengthRevertHistoryRevertChatMessagesAppendChatMessageSaveHistoryLoadHistoryPrintPrettyChatMessagesPlayKit_PlayerClientMethodsInitializeAsyncGetPlayerInfoAsyncGetCachedPlayerInfoHasValidPlayerTokenGetPlayerTokenSetPlayerTokenEventsOnPlayerInfoUpdatedOnPlayerTokenReceivedOnErrorPlayKit_MicrophoneRecorderPropertiesIsRecordingRecordingTimeCurrentDeviceLastRecordingConfiguration FieldsmaxRecordingSecondssampleRateuseVADsilenceThresholdmaxSilenceDurationMethodsStartRecordingStopRecordingCancelRecordingGetCurrentVolumeSetMicrophoneDeviceStatic MethodsGetAvailableDevicesEventsOnRecordingStartedOnRecordingStoppedOnVolumeChangedData TypesPlayKit_ChatMessagePlayKit_ChatConfigPlayKit_ChatStreamConfigPlayKit_AIResult<T>PlayKit_GeneratedImagePlayKit_TranscriptionResultPlayKit_TranscriptionSegmentPlayKit_SpeechResultPlayerInfoExceptionsPlayKitExceptionPlayKitApiErrorExceptionPlayKitImageSizeValidationExceptionUsage ExamplesBasic ChatNPC ConversationImage GenerationSpeech RecognitionNext Steps