Skip to content

docs: audio playback position reports - #208

Open
LautaroPetaccio wants to merge 1 commit into
mainfrom
docs/audio-playback-position-reports
Open

LautaroPetaccio wants to merge 1 commit into
mainfrom
docs/audio-playback-position-reports

Conversation

@LautaroPetaccio

Copy link
Copy Markdown

Documents the new audio playback position reports on the SDK7 Sounds page (creator/sdk7/3d-essentials/sounds.md).

What's new on the page

  • Playback position reports section: the tickNumber, currentOffset and clipLength fields the renderer writes into AudioEvent while a clip plays, and why they're the only reliable way to know what the player is actually hearing (currentTime is a write-only seek, and a clip starts some milliseconds after the scene asks for it).
  • Opt in to position reports: reportPlaybackPosition on AudioSource is required, otherwise no position is ever written and the playback callbacks never run. Explains why it's opt-in: a position report is written every time the playhead moves, far more often than a state change, and a scene can have many audio sources, so scenes only pay for it where they need it. Every code example creates its AudioSource with the flag set.
  • Read the playback position: registerAudioPlaybackEntity(entity, callback) with its { report, sceneTime, offset } payload, plus getAudioPlayback() and removeAudioPlaybackEntity().
  • Sync gameplay to the sound: why a report must be compared against the scene clock at the tick it was sampled, that sceneTime - offset is the moment the audible clip started rather than a lag, and getSceneTimeAtTick() for scenes that handle raw reports (it works for VideoEvent too).
  • Accuracy caveat: currentOffset is the decoder's read position, not the moment sound leaves the speaker. Output latency adds tens of milliseconds that no field carries, roughly constant per device, so calibrate once if you need better than tick accuracy.
  • Marked as requiring an SDK release that includes the feature and an explorer that implements it, so the fields stay undefined elsewhere.

Corrections to the existing audio events content

  • An AudioSource clip only ever reaches MS_NONE, MS_ERROR, MS_LOADING, MS_READY and MS_PLAYING. MS_PAUSED, MS_BUFFERING and MS_SEEKING only occur on an AudioStream, and there's no pause for a clip: setting playing to false stops and rewinds it.
  • timestamp is a monotonic counter the renderer increments per report, not a time value. The page previously described it as the time of the state change.
  • The currentTime property bullet now says it's a seek command that the renderer never writes back.

Related work

Supersedes decentraland/documentation#609, which targeted the now-archived decentraland/documentation repo. The content has been converted from Hugo shortcodes to GitBook syntax and reconciled with the audio events content that already exists on this page.

Add a "Playback position reports" section to the SDK7 sounds page,
covering the opt-in reportPlaybackPosition flag on AudioSource and the
audioEventsSystem APIs that read the reports: registerAudioPlaybackEntity,
getAudioPlayback, removeAudioPlaybackEntity and getSceneTimeAtTick.

Explain why the reports are the only reliable way to know what the player
is hearing, how to resolve sceneTime and offset against the scene clock to
get the moment the audible clip started, and the output latency caveat that
no field reports.

Also correct the audio events section: an AudioSource clip only reaches
MS_NONE, MS_ERROR, MS_LOADING, MS_READY and MS_PLAYING, the paused,
buffering and seeking states only occur on an AudioStream, and timestamp is
a monotonic counter rather than a time value.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant