---
title: "Media, mentions and transcript features"
description: "Add image, audio, video and file blocks, @ mentions and # tags, and speaker transcripts as independent features, with uploads, metadata lists and server schemas."
canonical_url: "https://skaile-ai.github.io/forge-editor/guides/media-mentions-transcript"
---
# Media, mentions and transcript features

> Add image, audio, video and file blocks, @ mentions and # tags, and speaker transcripts as independent features, with uploads, metadata lists and server schemas.

Three independent features extend the core editor. Each is one factory you add to `defineEditor({ features })` next to `coreContentFeature()` and `coreUiFeature()`; none needs Typed blocks, AI or Yjs, and none imports the others (`bun run check:editor-entries` proves it per feature).

```vue
<script setup lang="ts">
import { coreContentFeature, coreUiFeature, defineEditor } from '@skaile/forge-editor/core'
import { EditorSurface, EditorToolbar } from '@skaile/forge-editor/vue'
import { mediaFeature } from '@skaile/forge-editor/media/vue'
import { mentionsFeature } from '@skaile/forge-editor/mentions'
import { transcriptFeature } from '@skaile/forge-editor/transcript/vue'
import type { Editor } from '@tiptap/vue-3'
import { shallowRef } from 'vue'

const preset = defineEditor({ features: [coreContentFeature(), coreUiFeature(), mediaFeature(), mentionsFeature(), transcriptFeature()] })
const editor = shallowRef<Editor>()
</script>

<template>
  <EditorToolbar :editor="editor" />
  <EditorSurface v-model="content" :preset="preset" @ready="editor = $event" />
</template>
```

Import the feature CSS after `tailwindcss`, `@nuxt/ui` and `core.css`:

```css
@import "@skaile/forge-editor/styles/media.css";
@import "@skaile/forge-editor/styles/mentions.css";
@import "@skaile/forge-editor/styles/transcript.css";
```

## Dependency setup

Installing the package brings the feature dependencies; nothing else is required in the consumer beyond the peers listed in [Installation](https://skaile-ai.github.io/forge-editor/getting-started/installation#prerequisites).

<table>
<thead>
  <tr>
    <th>
      Feature
    </th>
    
    <th>
      Entry
    </th>
    
    <th>
      Vue-free schema entry
    </th>
    
    <th>
      Runtime dependencies it adds
    </th>
    
    <th>
      Needs a backend
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Media
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/media
      </code>
      
       (framework-free), <code>
        /media/vue
      </code>
      
      , <code>
        /media/react
      </code>
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/media/schema
      </code>
    </td>
    
    <td>
      <code>
        @tiptap/extension-image
      </code>
      
      , <code>
        @tiptap/extension-file-handler
      </code>
    </td>
    
    <td>
      only for uploads (see below); a backend with <code>
        assets
      </code>
      
       stores them
    </td>
  </tr>
  
  <tr>
    <td>
      Mentions
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/mentions
      </code>
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/mentions/schema
      </code>
    </td>
    
    <td>
      <code>
        @tiptap/extension-mention
      </code>
      
      , <code>
        @tiptap/suggestion
      </code>
    </td>
    
    <td>
      only for the built-in lists (see below); host lists replace them
    </td>
  </tr>
  
  <tr>
    <td>
      Transcript
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/transcript
      </code>
      
       (framework-free), <code>
        /transcript/vue
      </code>
      
      , <code>
        /transcript/react
      </code>
    </td>
    
    <td>
      <code>
        @skaile/forge-editor/transcript/schema
      </code>
    </td>
    
    <td>
      none beyond core
    </td>
    
    <td>
      no
    </td>
  </tr>
</tbody>
</table>

The `/schema` entries import no Vue, no node view and no feature factory. A server that validates or seeds documents builds the matching schema from them:

```ts
import { getSchema } from '@tiptap/core'
import { coreSchemaExtensions } from '@skaile/forge-editor/core/schema'
import { mediaSchemaExtensions } from '@skaile/forge-editor/media/schema'
import { mentionsSchemaExtensions } from '@skaile/forge-editor/mentions/schema'
import { transcriptSchemaExtensions } from '@skaile/forge-editor/transcript/schema'

const schema = getSchema([...coreSchemaExtensions(), ...mediaSchemaExtensions(), ...mentionsSchemaExtensions(), ...transcriptSchemaExtensions()])
```

The schema entries also export the helpers servers need: `MEDIA_TYPES`, `MAX_MEDIA_BYTES`, `mediaKindOf`, `assetUrl`, `assetRefOf`, `ASSET_NAME`, `ASSET_DOCUMENT_ID`, `SCRATCH_DOCUMENT_ID` (media), `mentionsFromText`, `mentionUsersIn`, `tagsFromText`, `collectTagNames`, `normalizeTagName` (mentions), `speakerParagraphsToSegments`, `lastSpeaker`, `speakersIn` (transcript).

## Upload

`mediaFeature()` adds image, audio, video and file blocks, drop/paste of files, toolbar buttons (Bild, Audio-Datei, Video, Aufnahme), the matching `/` commands and the audio recorder modal. Every path ends in one function, `editor.storage.media.upload(file): Promise<{ url: string, sha256?: string }>`, which calls the backend's `assets` capability: `backend.assets.upload(documentId, file)`; `documentId` is the surface's prop, default `scratch`.

- The HTTP backend (`createHttpBackend()`, the default) sends `POST /api/editor/assets/<documentId>` with a `multipart/form-data` body (field `file`); the response must be `{ url }` and may add `sha256` (hex SHA-256 of the bytes; the bundled server always sends it, and stores it in the asset record). The media blocks keep it as their `hash` attribute; backends that send none (the memory backend) leave `hash` empty.
- To store files elsewhere (S3, a data URL, …), implement `assets.upload` in your own `EditorBackend` (`createEditor({ backend })`); the returned `url` becomes the block's `src`. `createMemoryBackend()` answers with a `blob:` URL.

```ts
const backend: EditorBackend = { name: 'my-store', assets: { upload: async (_documentId, file) => ({ url: await storeSomewhere(file) }) } }
createEditor({ backend })
```

Accepted types and size limits are `MEDIA_TYPES` and `MAX_MEDIA_BYTES` (Vue-free, from `@skaile/forge-editor/media/schema`): images up to 5 MB (png, jpeg, gif, webp), audio 50 MB, video 200 MB, and any other type up to 50 MB as a downloadable file block. A server endpoint should validate against the same constants; the Nuxt module's `/api/editor/assets/*` routes do. A failed upload shows a toast (`Import fehlgeschlagen`) and inserts nothing.

**File import.** Drop, paste, the toolbar and `/` commands all call `editor.storage.media.importFiles(editor, files, pos?)`; `useFileImport()` (from `@skaile/forge-editor/media/vue` or `/media/react`; call it in `setup` / while rendering: it binds the notifier of `editor`'s surface when you pass the editor, else the app's: a Nuxt UI toast on Vue and a sonner toast on React unless the host's `notify` says otherwise) returns the same `importFiles` for your own buttons; the framework-free model is `fileImport(notify, t)` from `@skaile/forge-editor/media` (`t` is the editor context's translator). Every file is planned by the [importer registry](./34.importers): a Markdown, subtitle (`.vtt`, `.srt`) or BibTeX (`.bib`) file by its own importer; any other text file (`.txt`, at most 2 MB) is parsed through the editor's block pipeline (Markdown parse, then `normalize` — mention/tag chips —, `segment` — speaker lines —, `structure`); every other file is uploaded and appears as its media or file block. A wrapping importer (`composite-media`) may then wrap the plan; the resulting blocks are inserted as they are at the drop position (anchored while uploads run, so editing meanwhile does not shift it) or at the selection. Files are imported sequentially; a failed file is reported and skipped.

The recorder (`AudioRecorderModal`) is mounted by `EditorSurface` as a feature overlay. In a custom layout without `EditorSurface`, render `<AudioRecorderModal :editor="editor" />` yourself (exported from `@skaile/forge-editor/media/vue`, or `/media/react`) and open it with `openAudioRecorder(editor)`. It records through `MediaRecorder` (needs HTTPS or localhost and microphone permission) and imports the result like any file. The recording model itself is framework-free: `createAudioRecorder()` from `@skaile/forge-editor/media` (a store with `start`, `stop`, `cancel`, `dispose`).

## Metadata

The mentions feature reads two named lists, the transcript feature one; hosts replace any of them through the `metadata` prop (`EditorMetadata`: a static array or a search function `(q: MetadataQuery) => items | Promise<items>`). Lists the host does not supply fall back to the feature's built-in source; the fallback is only searched when a suggestion popup is actually used.

<table>
<thead>
  <tr>
    <th>
      List
    </th>
    
    <th>
      Used by
    </th>
    
    <th>
      Built-in source when the host supplies none
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        mentions
      </code>
    </td>
    
    <td>
      <code>
        @
      </code>
      
       popup, <code>
        normalize
      </code>
      
       stage
    </td>
    
    <td>
      <code>
        GET /api/editor/users?q=<query>
      </code>
      
       → <code>
        { id, username, color }[]
      </code>
      
       (through the context's HTTP client)
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        hashtags
      </code>
    </td>
    
    <td>
      <code>
        #
      </code>
      
       popup
    </td>
    
    <td>
      <code>
        GET /api/editor/tags?q=<normalized query>
      </code>
      
       → <code>
        string[]
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        speakers
      </code>
    </td>
    
    <td>
      speaker field of a segment
    </td>
    
    <td>
      speakers already in the document (no backend)
    </td>
  </tr>
</tbody>
</table>

A backend-free app passes its own lists and never calls those endpoints:

```vue
<EditorSurface
  :preset="preset"
  :metadata="{
    mentions: [{ id: 'u1', label: 'anna', color: '#e11d48' }],
    hashtags: q => searchTags(q.query),
  }"
  @metadata-create="(list, item) => persist(list, item)"
/>
```

Mentions are `@username` chips tinted with the item's `color` (a `#rgb`/`#rrggbb` value; anything else is dropped). Tags are `#tag` chips; the typed name is normalized (`normalizeTagName`: lowercase, spaces → `-`, letters/digits/`-`/`_` only plus `/` as the separator of a nested name like `inbox/to-read`, at most 64 characters), and the popup offers „#name erstellen“ when it is not in the list — picking it emits `metadataCreate('hashtags', { id, label })`.

**Pipeline stages.** The mentions feature contributes the `normalize` stage: `@name` / `#tag` in Markdown that is imported from a file or written by AI turns into chips when the name matches a user already in the document or in the `mentions` list, or a tag already in the document. Without the feature installed, those stay plain text. The transcript feature contributes the `segment` stage: a paragraph that starts with a bold `**Name:**` followed by text becomes a speaker segment (a bare `**Aufgaben:**` label stays a paragraph). Both stages run for imported files and for AI results.

## Transcript

`transcriptFeature()` adds the speaker segment block: a „Sprecherblock“ toolbar button (Mod-Shift-S), Enter inside a segment splits it keeping the speaker, a speaker field with suggestions from the `speakers` list, the „Sprecherzeile“ entry in the „Umwandeln in …“ menus (paragraphs, headings, lists, quotes and code blocks become segments carrying the last speaker; segments convert back into any core target), and Markdown `**Name:** text` export. A segment holds only inline text: headings, lists, quotes, code blocks, media and Typed blocks are revoked inside it.


## Sitemap

See the full [sitemap](https://skaile-ai.github.io/forge-editor/sitemap.md) for all pages.