Sentence Diagrams karaoke service

This directory is the one editable implementation of Sentence Diagrams karaoke. Its behavioral authority is the current ALT Dictionary renderer, captured as the alt-dictionary-v1 profile.

The service is a deterministic package, not a per-frame network dependency. Browser consumers use the ESM entry point. Server renderers use the generated CommonJS bundle and the FFmpeg adapter. Both are built from the same source.

Contract

A consumer supplies:

diagram mode (condensed or expanded);
ordered word/contact timings in milliseconds;
contact placements and their declared coordinate space;
an explicit clock origin when timings come from source media; and
an explicit visual mode.

The service owns:

contact-at-word-start semantics;
trajectory profiles, arcs, entry, wrap, hold, and fade;
a 260ms wrapped-line entry from a fixed 60px-left/20px-up launch, clamped only when adjacent contacts leave less time;
source-time linePosition interpolation over that exact entry interval so scrolling consumers remain synchronized during playback, seeking, and rewind;
the fixed 12 CSS/logical-pixel ball;
impact squash/stretch, trail, contact overlap, and ball paint;
which visual modes are offered by condensed and expanded diagrams; and
translation of a completed canonical trajectory into FFmpeg expressions.

On legacy input, startMs remains the compatibility contact-time fallback. Canonical output mirrors the speech bounds in startMs/endMs and carries the independent bounce contact in contactMs/contactBasis; the kernel uses contactMs when it is present. This is the intentional extension point for later phoneme/syllable work. Human and source-media provenance is protected by the canonical program contract: automatic audio generation must fail closed for those inputs.

Audio generation and timing boundary

Audio producers must authorize synthesis through the standalone immutable contract before calling a TTS or audio-writing adapter:

const job = karaokeService.createKaraokeAudioGenerationJob({
  provenance: 'generated',
  operation: 'manager-sentence-tts'
});
karaokeService.assertKaraokeAudioGenerationAllowed(job);

Only generated provenance can create a generation job. human and source-media fail closed even if a caller supplies forged protected or generationAllowed flags. The service derives all policy fields from provenance and locks the job to the package identity and behavior profile.

Timing producers and importers normalize word records through normalizeKaraokeWordTiming or normalizeKaraokeWordTimings. The immutable result always emits startMs, endMs, speechStartMs, speechEndMs, contactMs, and contactBasis, while preserving unrelated metadata. Missing contact metadata defaults to the current ALT Dictionary behavior: contactMs = speechStartMs and contactBasis = "word-start". Explicit future phoneme or syllable contact metadata is preserved.

Shared timing adjuster

Browser consumers open the one framework-free timing editor with createTimingAdjuster. The service owns its waveform, zoom, playback speed, contact-marker drag rules, and serialization. The consumer remains the sole authority for audio retrieval and persistence:

const editor = karaokeService.createTimingAdjuster({
  document,
  onSave: async (change) => occurrenceApi.createTimingRevision(change)
});

await editor.open({
  titleText: 'Adjust Timing',
  audioUrl: pinnedAudioExcerptUrl,
  audioProvenance: 'source-media',
  durationMs,
  audioWindow: {
    schemaVersion: 'sentence-diagrams-karaoke-timing-audio-window-v1',
    timebase: 'provided-audio-ms',
    durationMs,
    contentStartMs,
    contentEndMs,
    sourceMediaOriginMs
  },
  context: { occurrenceId, timingRevisionId },
  words
});

The onSave adapter receives every identity/evidence field supplied by the consumer plus canonical speechStartMs, speechEndMs, contactMs, and contactBasis values. Compatibility startMs/endMs mirrors are emitted but never replace or erase the explicit fields. The editor does not generate audio, choose a database, or write to a sentence archive; Manager, source media, and future human-audio consumers therefore share one interaction model without sharing or confusing their persistence authorities.

The provided audio is the timing editor's immutable clock. Every speech and contact time is measured from time zero of those supplied bytes. A bookended source-media consumer declares the editable occurrence inside those bytes with contentStartMs/contentEndMs and declares the corresponding absolute source origin with sourceMediaOriginMs. The editor preserves this versioned audioWindow through save and refresh, clamps drags to its content range, and never trims, pads, synthesizes, or remaps the audio. The consumer must return the same authoritative window with a refreshed track after persistence.

The initial horizontal viewport is exactly 2750ms. Longer audio scrolls and shorter audio shows a blank tail; viewport width does not change that time span. Waveform height uses deterministic, file-global absolute-peak normalization for appearance only. The decoded samples and every timing value remain unchanged. A decoded duration more than 25ms from the declared window fails closed instead of silently stretching the timing coordinate system.

Playback speed is applied to both the media element's defaultPlaybackRate and live playbackRate. This is required because the browser media-load algorithm restores the live rate from the default whenever a resource is loaded.

Consumers that need product-specific controls may mount them in the returned editor.extensions element and may provide onInteraction, renderOverlay, isWordContactMarkerVisible, and onClose hooks. The optional marker-visibility hook receives the live { model, word, index } and may return false when a consumer overlay owns that word's marker; hidden markers are neither drawn nor eligible for the service's contact-marker hit testing. These hooks may add persistence, lesson, or prosody capabilities, but they do not replace the service-owned audio element, waveform, zoom, playback rate, marker drag rules, or serialization. A consumer that only needs waveform inspection opens the same widget with readOnly: true; it must not maintain a second private waveform viewer.

decodeKaraokeTimingAudio(blob) is the shared preflight decoder for consumers that need the exact decoded duration or samples for an extension such as prosody analysis. Passing its result back as decodedAudio transfers that decode to the widget and avoids a second decode.

Save operations are generation-isolated. Overlapping Save and Save & Close actions share one persistence request, a completed clean editor closes without creating a no-op revision, and a result from an older closed or reopened editor cannot replace the active model or close the new modal. Consumers must return the newly authoritative revision context from onSave so a later, genuinely dirty save uses the new compare-and-swap token.

Canonical ball sprite

The browser and FFmpeg do not paint separate approximations. The build renders every supported ball theme from the ALT Dictionary Canvas reference painter into one padded, 6× RGBA PNG atlas. It commits that PNG, decoded-pixel and PNG SHA-256 hashes, per-theme alpha/bounds metrics, and an embedded copy used by the browser bundle. Unknown theme values normalize to amber-sun.

The browser preloads and crops the embedded atlas. A server renderer materializes the same bytes and crops the selected row:

const atlas = await karaokeService.materializeKaraokeBallSpriteAtlas({
  filePath: path.join(renderCache, 'canonical-karaoke-ball-atlas.png')
});
const sprite = karaokeService.karaokeBallSpriteAtlasTheme(ballColor, {
  radius: 12,
  supersample: 4
});

ffmpegArgs.push(
  '-loop', '1',
  '-framerate', String(fps),
  '-i', atlas.filePath
);
filterChains.push(
  `[${inputIndex}:v]${sprite.cropFilter},format=rgba[karaokeBallSprite]`
);

Composition must scale the entire padded cell from baseSpriteWidthPx/baseSpriteHeightPx, and use the adapter's contactYExpr. Scaling both axes from the ball diameter clips or distorts the shadow and glow. The retired buildKaraokeBallSpriteFilter GEQ approximation throws intentionally.

Distribution and upgrades

src/ is the only hand-edited implementation. dist/, identity.generated.js, sprite.generated.js, and integrity.json are generated release artifacts:

node src/shared/karaoke-service/scripts/build.mjs
node src/shared/karaoke-service/scripts/verify-build.mjs

The build uses the repository's pinned Playwright/Chromium installation to render the reference painter. Verification does not rerender: it decodes the committed PNG, verifies pixel and file hashes, verifies every theme row, and rebuilds both distribution bundles byte-for-byte.

Consumers pin an exact package version and source hash. Release automation may propose and test upgrades, but consumers never load an unpinned remote "latest". This preserves automatic propagation without sacrificing deterministic, reproducible video output.

Adapter rule

Coordinate and clock translation happen before trajectory construction:

Convert source-media time to sentence/audio-local time.
Convert source/viewBox coordinates to CSS or logical output pixels.
Build one canonical program.
Paint it with the browser adapter or compile its completed segments with the FFmpeg adapter.

FFmpeg supersampling happens only after step 3. Running physics on 4× coordinates changes the effective arc cap, upward clearance, and drop threshold and is forbidden.

Change process

A behavior change requires:

edit only files in this package's src/;
update the profile/version when the contract changes;
regenerate immutable bundles and integrity metadata;
run package golden/parity tests;
run every registered consumer's compatibility suite; and
publish exact-version update proposals only when all consumers pass.