---
title: "Speech port"
description: "The SpeechProvider, SpeechEngineInfo and SpeechRequest types, the speech routes and events, the daily budget, and the createOpenAiSpeech and createMemorySpeech adapters."
canonical_url: "https://skaile-ai.github.io/forge-editor/reference/speech-port"
---
# Speech port

> The SpeechProvider, SpeechEngineInfo and SpeechRequest types, the speech routes and events, the daily budget, and the createOpenAiSpeech and createMemorySpeech adapters.

The server types come from `packages/editor/src/server/ports.ts` (`@skaile/forge-editor/server`), the wire types from `packages/editor/src/core/backend-types.ts` (`@skaile/forge-editor/core`), the adapters from `@skaile/forge-editor/adapters/speech`. For how to provide a port and what the editor shows, see [Speech](https://skaile-ai.github.io/forge-editor/guides/speech).

## `SpeechProvider`

`EditorPorts.speech` (capability `speech`). The audio is stored through `EditorPorts.assets`; without it `POST /api/editor/speech` answers `501`.

<table>
<thead>
  <tr>
    <th>
      Member
    </th>
    
    <th>
      Type
    </th>
    
    <th>
      Meaning
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        engines
      </code>
    </td>
    
    <td>
      <code>
        (user: object) => Promise<SpeechEngineInfo[]>
      </code>
    </td>
    
    <td>
      The engines this user may use, each with its voices.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        synthesize
      </code>
    </td>
    
    <td>
      <code>
        (request: SpeechRequest, user: object, signal: AbortSignal) => Promise<{ data: Uint8Array, mime: string }>
      </code>
    </td>
    
    <td>
      Reads <code>
        request.text
      </code>
      
       with <code>
        request.voice
      </code>
      
       of <code>
        request.engine
      </code>
      
      . <code>
        signal
      </code>
      
       aborts when the client goes away. Throw to fail the synthesis: the message reaches the client. Throw <code>
        HttpError
      </code>
      
       <code>
        404
      </code>
      
       for an unknown engine or voice.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        transcribe
      </code>
    </td>
    
    <td>
      <code>
        (audio: Uint8Array, mime: string, user: object) => Promise<{ segments: SpeechSegment[] }>
      </code>
    </td>
    
    <td>
      Optional. The timed text of <code>
        audio
      </code>
      
      .
    </td>
  </tr>
</tbody>
</table>

## Types

<table>
<thead>
  <tr>
    <th>
      Type
    </th>
    
    <th>
      Fields
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        SpeechEngineInfo
      </code>
    </td>
    
    <td>
      <code>
        id
      </code>
      
      , <code>
        label
      </code>
      
       (already in the user's language), <code>
        voices: SpeechVoice[]
      </code>
      
      , optional <code>
        formats
      </code>
      
       (the formats it makes; absent: <code>
        mp3
      </code>
      
       and <code>
        wav
      </code>
      
      ; <code>
        createMemorySpeech
      </code>
      
       lists <code>
        wav
      </code>
      
      )
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SpeechVoice
      </code>
    </td>
    
    <td>
      <code>
        id
      </code>
      
      , <code>
        label
      </code>
      
      , <code>
        lang
      </code>
      
       (BCP 47, <code>
        und
      </code>
      
       for any language)
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SpeechRequest
      </code>
    </td>
    
    <td>
      <code>
        engine
      </code>
      
      , <code>
        voice
      </code>
      
      , <code>
        text
      </code>
      
      , <code>
        format: SpeechFormat
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SpeechFormat
      </code>
    </td>
    
    <td>
      <code>
        'mp3' | 'wav'
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SpeechBody
      </code>
    </td>
    
    <td>
      <code>
        SpeechRequest
      </code>
      
       plus <code>
        documentId
      </code>
      
      , the Document the audio is stored under
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        SpeechSegment
      </code>
    </td>
    
    <td>
      <code>
        start
      </code>
      
      , <code>
        end
      </code>
      
       (seconds into the audio), <code>
        text
      </code>
    </td>
  </tr>
</tbody>
</table>

## Routes

<table>
<thead>
  <tr>
    <th>
      Route (<code>
        API
      </code>
      
       key)
    </th>
    
    <th>
      Request
    </th>
    
    <th>
      Answer
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        GET /api/editor/speech/engines
      </code>
      
       (<code>
        speechEngines
      </code>
      
      )
    </td>
    
    <td>
      none
    </td>
    
    <td>
      <code>
        SpeechEngineInfo[]
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        POST /api/editor/speech
      </code>
      
       (<code>
        speechSynthesize
      </code>
      
      )
    </td>
    
    <td>
      <code>
        SpeechBody
      </code>
    </td>
    
    <td>
      SSE: <code>
        progress
      </code>
      
       <code>
        { phase: string, percent?: number }
      </code>
      
      , then <code>
        done
      </code>
      
       <code>
        { asset: string, fileName: string, mime: string }
      </code>
      
       or <code>
        error
      </code>
      
       <code>
        { message: string }
      </code>
    </td>
  </tr>
</tbody>
</table>

Both routes are served only when `ports.speech` is present. Refusals before the stream starts: `400`, `401`, `403`, `404` (unknown engine or voice), `429` (daily budget, with `Retry-After` in seconds), `501` (no `assets` port).

## Daily budget

<table>
<thead>
  <tr>
    <th>
      Export (<code>
        @skaile/forge-editor/server
      </code>
      
      )
    </th>
    
    <th>
      Meaning
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        SPEECH_CHARS_PER_DAY
      </code>
    </td>
    
    <td>
      <code>
        50_000
      </code>
      
      , the characters one user may synthesize per UTC day.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        createSpeechBudget({ limit?, now? })
      </code>
    </td>
    
    <td>
      A <code>
        SpeechBudget
      </code>
      
       held in memory. <code>
        charge(key, chars)
      </code>
      
       returns <code>
        0
      </code>
      
       when the characters fit, else the seconds until UTC midnight (at least 1) and counts nothing. <code>
        refund(key, chars)
      </code>
      
       gives characters back after a failed synthesis.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        HandlerContext.speechBudget
      </code>
    </td>
    
    <td>
      The budget the speech handler uses. Without it one budget per <code>
        ports
      </code>
      
       object applies, with the default limit.
    </td>
  </tr>
</tbody>
</table>

## `EditorBackend['speech']`

<table>
<thead>
  <tr>
    <th>
      Member
    </th>
    
    <th>
      Type
    </th>
    
    <th>
      Meaning
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        engines
      </code>
    </td>
    
    <td>
      <code>
        (signal?) => Promise<SpeechEngineInfo[]>
      </code>
    </td>
    
    <td>
      The engines and voices of the user.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        synthesize
      </code>
    </td>
    
    <td>
      <code>
        (request: SpeechBody, { onProgress?, signal? }?) => Promise<RenderResult>
      </code>
    </td>
    
    <td>
      Streams one synthesis; <code>
        RenderResult
      </code>
      
       is <code>
        RenderDone
      </code>
      
       (<code>
        asset
      </code>
      
      , <code>
        fileName
      </code>
      
      , <code>
        mime
      </code>
      
      ) plus the <code>
        url
      </code>
      
       of the stored asset. Rejects with the server's message when it fails.
    </td>
  </tr>
</tbody>
</table>

## `@skaile/forge-editor/adapters/speech`

<table>
<thead>
  <tr>
    <th>
      Export
    </th>
    
    <th>
      Meaning
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        createOpenAiSpeech(options)
      </code>
    </td>
    
    <td>
      A <code>
        SpeechProvider
      </code>
      
       over <code>
        POST <endpoint>/audio/speech
      </code>
      
      . Options <code>
        endpoint
      </code>
      
      , <code>
        model
      </code>
      
      , <code>
        apiKey?
      </code>
      
      , <code>
        id?
      </code>
      
       (default <code>
        openai
      </code>
      
      ), <code>
        label?
      </code>
      
      , <code>
        voices?
      </code>
      
      , <code>
        fetch?
      </code>
      
      . An upstream error rejects with its status and the first 300 characters of its body.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        OPENAI_VOICES
      </code>
    </td>
    
    <td>
      The voices of OpenAI's speech models.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        createMemorySpeech(options?)
      </code>
    </td>
    
    <td>
      A <code>
        SpeechProvider
      </code>
      
       that answers one second of silence as WAV; option <code>
        engines
      </code>
      
      . Default engine <code>
        memory
      </code>
      
      , voice <code>
        silent
      </code>
      
      . <code>
        mp3
      </code>
      
       is refused with <code>
        422
      </code>
      
      .
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        silentWav(seconds?)
      </code>
    </td>
    
    <td>
      The silent mono 16-bit WAV (8 kHz) as <code>
        Uint8Array
      </code>
      
      .
    </td>
  </tr>
</tbody>
</table>

## Subtitle export (`@skaile/forge-editor/transcript`)

<table>
<thead>
  <tr>
    <th>
      Export
    </th>
    
    <th>
      Meaning
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        vttOf(cues)
      </code>
    </td>
    
    <td>
      WebVTT text of <code>
        TranscriptCue[]
      </code>
      
      .
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        srtOf(cues)
      </code>
    </td>
    
    <td>
      SubRip text of <code>
        TranscriptCue[]
      </code>
      
      .
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        transcriptCuesOf(node)
      </code>
    </td>
    
    <td>
      The <code>
        TranscriptCue[]
      </code>
      
       of the timed <code>
        transcriptSegment
      </code>
      
      s in a node.
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        TranscriptCue
      </code>
    </td>
    
    <td>
      <code>
        { start, end, text, speaker? }
      </code>
      
      , seconds. A <code>
        SpeechSegment
      </code>
      
       fits.
    </td>
  </tr>
</tbody>
</table>


## Sitemap

See the full [sitemap](https://skaile-ai.github.io/forge-editor/sitemap.md) for all pages.