feat: report playback position in AudioEvent - #488
Open
LautaroPetaccio wants to merge 2 commits into
Open
LautaroPetaccio wants to merge 2 commits into
LautaroPetaccio wants to merge 2 commits into
Conversation
This was referenced Sep 15, 2026
Open
Draft
Test this pull request
|
Renderers start an AudioSource some milliseconds after being asked to and PBAudioSource.current_time is a write-only seek, so a scene has no way to tell where the audio it hears actually is. Add optional fields to PBAudioEvent so renderers can report the clip position on every state change and periodically while playing: - tick_number: the scene tick of the report (EngineInfo.tick_number) - current_offset: playback position in seconds at that tick - clip_length: total clip length in seconds, when known All fields are optional, so existing renderers and scenes are unaffected.
LautaroPetaccio
force-pushed
the
feat/audio-event-playback-position
branch
from
September 16, 2026 13:01
b29c1cb to
7a449b7
Compare
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 16, 2026
CI regenerates the ecs components from the installed @dcl/protocol, which did not yet carry tick_number, current_offset and clip_length, so the build failed on audioEvents.ts. Pin both protocol dependencies to the package published from decentraland/protocol#488 until it is released, and regenerate the API report. The serialization specs deserialize into objects that carry every proto3 optional field explicitly, so AudioEvent.spec.ts now lists them and also covers a full position report and a position without a clip length. ExplorerUiEventsResult gained a required request_id upstream in the same protocol window; its spec expects the zero value.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 16, 2026
CI regenerates the ecs components from the installed @dcl/protocol, which did not yet carry tick_number, current_offset and clip_length, so the build failed on audioEvents.ts. Pin both protocol dependencies to the package published from decentraland/protocol#488 until it is released, and regenerate the API report. The serialization specs deserialize into objects that carry every proto3 optional field explicitly, so AudioEvent.spec.ts now lists them and also covers a full position report and a position without a clip length. ExplorerUiEventsResult gained a required request_id upstream in the same protocol window; its spec expects the zero value.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 16, 2026
CI regenerates the ecs components from the installed @dcl/protocol, which did not yet carry tick_number, current_offset and clip_length, so the build failed on audioEvents.ts. Pin both protocol dependencies to the package published from decentraland/protocol#488 until it is released, and regenerate the API report. The serialization specs deserialize into objects that carry every proto3 optional field explicitly, so AudioEvent.spec.ts now lists them and also covers a full position report and a position without a clip length. ExplorerUiEventsResult gained a required request_id upstream in the same protocol window; its spec expects the zero value.
pravusjif
previously approved these changes
Sep 17, 2026
pravusjif
left a comment
pravusjif
left a comment
Member
There was a problem hiding this comment.
good catch the current_offset is a real need, like the one we have in video_event 👍
This was referenced Sep 17, 2026
LautaroPetaccio
marked this pull request as ready for review
September 17, 2026 14:16
LautaroPetaccio
added a commit
to decentraland/unity-explorer
that referenced
this pull request
Sep 17, 2026
…ition Adds TickNumber, CurrentOffset and ClipLength to PBAudioEvent, with the Has/Clear accessors proto3 optional fields generate. Without them the system in this branch does not compile. Pins @dcl/protocol to the build of decentraland/protocol#488. The pin goes back to a released version once that merges.
A position report is written whenever the playhead moves, which is far more often than a media state changes, and a scene can hold many more audio sources than videos: video playback is capped by a prioritisation system, audio is not. Reporting unconditionally would make every fire-and-forget sound effect pay for a signal only rhythm and sync scenes read. report_playback_position gates only the position fields. Media state changes are reported either way, so nothing an existing scene observes today changes. Also correct the AudioEvent comment, which still described the reports as periodic. There is no interval; a report goes out when the playhead moves.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 21, 2026
Repins @dcl/protocol to the build carrying PBAudioSource.report_playback_position (decentraland/protocol#488) and regenerates. The flag is a plain generated field, so a scene sets it like any other property; the SDK does not set it on the scene's behalf. Registration is allowed before the AudioSource exists, and a later scene-written createOrReplace would clear an SDK-written flag silently, so it stays explicit and owned by the scene. Documents on registerAudioPlaybackEntity that the source has to opt in, since the failure mode otherwise is a callback that simply never runs. Covers the field in the AudioSource serialization spec.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 21, 2026
CI regenerates the ecs components from the installed @dcl/protocol, which did not yet carry tick_number, current_offset and clip_length, so the build failed on audioEvents.ts. Pin both protocol dependencies to the package published from decentraland/protocol#488 until it is released, and regenerate the API report. The serialization specs deserialize into objects that carry every proto3 optional field explicitly, so AudioEvent.spec.ts now lists them and also covers a full position report and a position without a clip length. ExplorerUiEventsResult gained a required request_id upstream in the same protocol window; its spec expects the zero value.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 21, 2026
Repins @dcl/protocol to the build carrying PBAudioSource.report_playback_position (decentraland/protocol#488) and regenerates. The flag is a plain generated field, so a scene sets it like any other property; the SDK does not set it on the scene's behalf. Registration is allowed before the AudioSource exists, and a later scene-written createOrReplace would clear an SDK-written flag silently, so it stays explicit and owned by the scene. Documents on registerAudioPlaybackEntity that the source has to opt in, since the failure mode otherwise is a callback that simply never runs. Covers the field in the AudioSource serialization spec.
LautaroPetaccio
added a commit
to decentraland/unity-explorer
that referenced
this pull request
Sep 21, 2026
Adds PBAudioSource.report_playback_position (field 8, optional bool), the opt-in that gates the playback position reports, and picks up the PBAudioEvent doc change that describes when a position is written. Moves the @dcl/protocol pin to the newer build of decentraland/protocol#488. The pin goes back to a released version once that merges.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 28, 2026
CI regenerates the ecs components from the installed @dcl/protocol, which did not yet carry tick_number, current_offset and clip_length, so the build failed on audioEvents.ts. Pin both protocol dependencies to the package published from decentraland/protocol#488 until it is released, and regenerate the API report. The serialization specs deserialize into objects that carry every proto3 optional field explicitly, so AudioEvent.spec.ts now lists them and also covers a full position report and a position without a clip length. ExplorerUiEventsResult gained a required request_id upstream in the same protocol window; its spec expects the zero value.
LautaroPetaccio
added a commit
to decentraland/js-sdk-toolchain
that referenced
this pull request
Sep 28, 2026
Repins @dcl/protocol to the build carrying PBAudioSource.report_playback_position (decentraland/protocol#488) and regenerates. The flag is a plain generated field, so a scene sets it like any other property; the SDK does not set it on the scene's behalf. Registration is allowed before the AudioSource exists, and a later scene-written createOrReplace would clear an SDK-written flag silently, so it stays explicit and owned by the scene. Documents on registerAudioPlaybackEntity that the source has to opt in, since the failure mode otherwise is a callback that simply never runs. Covers the field in the AudioSource serialization spec.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three optional fields to
PBAudioEventso renderers can report where a clip's playhead actually is, whenever the playhead moves while a clip plays:tick_number: scene tick of the report, equalsEngineInfo.tick_numbercurrent_offset: playback position of the clip in seconds at that tickclip_length: total clip length in seconds, when knownAnd one optional field to
PBAudioSource, by which a scene asks for them:report_playback_position: default falseWhy the reports are opt-in
A position report is written whenever the playhead moves, far more often than a media state changes, and a scene can hold many more audio sources than video players: video playback is capped by a prioritisation mechanism in the renderer, audio is not. Reporting unconditionally would make every fire-and-forget sound effect pay for a signal only rhythm and synchronisation scenes read.
The cost is not only traffic.
AudioEventis a grow-only value set bounded per entity, so a single source reporting on every playhead movement fills its own window within seconds, evicting state-change history a scene reading the component directly might want.Media state changes are reported whatever the flag says, so a scene that never sets it observes exactly what it observes today.
Why
PBAudioSource.current_timeis a write-only seek andPBAudioEventcarries only a media state and a counter, so a scene cannot tell when the audio it hears started or where it is. Renderers start a clip some milliseconds after being asked to (100 to 250 ms measured on the Unity explorer, varying per start), which rhythm and video-synchronized scenes cannot correct without this signal.PBVideoEventalready reportscurrent_offsetandtick_numberfor video; this brings audio to the same shape.Why optional
The fields are absent rather than zero when a renderer has no position to give, which is the case for streams whose player does not expose one, and on any renderer that has not implemented the reports. A zero offset is a valid playback position, so presence has to be distinguishable from it.
PBPointerEventsResult.analogsets the precedent in a renderer-written grow-only event set, and the rest of the audio family is optional throughout.PBVideoEventis the outlier here, not this message.What it does not carry
current_offsetis the decoder's read position, not the moment a sample leaves the speaker. The output path adds tens of milliseconds more, consistently signed and roughly constant per device, and no field here reports it. A renderer able to estimate its own output latency should expose it, and a later revision may add a field; ADR-318 records the limit meanwhile.Compatibility
All fields are optional. Existing renderers and scenes are unaffected. The component id does not change.
audio_event.protocompiles with protoc, and the compatibility check againstmainpasses.Related