Media3 1.5.0 — what’s new?

Posted by Kristina Simakova – Engineering Supervisor

This text is cross-published on Medium

Transformer now helps movement pictures and quicker picture encoding. We’ve additionally simplified the setup for DefaultPreloadManager and ExoPlayer, making it simpler to make use of. However that’s not all! We’ve included a brand new IAMF decoder, a Kotlin listener extension, and simpler Participant optimization by delegation.

To be taught extra about all new APIs and bug fixes, try the full launch notes.

Transformer enhancements

Movement photograph assist

Transformer now helps exporting movement pictures. The movement photograph’s picture is exported if the corresponding MediaItem’s picture period is about (see MediaItem.Builder().setImageDurationMs()) In any other case, the movement photograph’s video is exported. Word that the EditedMediaItem’s period shouldn’t be set in both case as it’s going to mechanically be set to the corresponding MediaItem’s picture period.

Sooner picture encoding

This launch accelerates image-to-video encoding, due to optimizations in DefaultVideoFrameProcessor.queueInputBitmap(). DefaultVideoFrameProcessor now treats the Bitmap given to queueInputBitmap() as immutable. The GL pipeline will resample and color-convert the enter Bitmap solely as soon as. In consequence, Transformer operations that take massive (e.g. 12 megapixels) photographs as enter execute quicker.

AudioEncoderSettings

Much like VideoEncoderSettings, Transformer now helps AudioEncoderSettings which can be utilized to set the specified encoding profile and bitrate.

Edit checklist assist

Transformer now shifts the primary video body to start out from 0. This fixes A/V sync points in some recordsdata the place an edit checklist is current.

Unsupported observe sort logging

This launch contains improved logging for unsupported observe varieties, offering extra detailed data for troubleshooting and debugging.

Media3 muxer

In one of many earlier releases we added a brand new muxer library which can be utilized to create MP4 container recordsdata. The media3 muxer gives assist for a variety of audio and video codecs, enabling seamless dealing with of various media codecs. This new library additionally brings superior options together with:

The muxer library will be included as a gradle dependency:

implementation ("androidx.media3:media3-muxer:1.5.0")

Media3 muxer with Transformer

To make use of the media3 muxer with Transformer, set an InAppMuxer.Manufacturing facility (which internally wraps media3 muxer) because the muxer manufacturing unit when making a Transformer:

val transformer = Transformer.Builder(context)
    .setMuxerFactory(InAppMuxer.Manufacturing facility.Builder().construct())
    .construct()

Easier setup for DefaultPreloadManager and ExoPlayer

With Media3 1.5.0, we added DefaultPreloadManager.Builder, which makes it a lot simpler to construct the preload parts and the participant. Beforehand we requested you to instantiate a number of required parts (RenderersFactory, TrackSelectorFactory, LoadControl, BandwidthMeter and preload / playback Looper) first, and be tremendous cautious on appropriately sharing these parts when injecting them into the DefaultPreloadManager constructor and the ExoPlayer.Builder. With the brand new DefaultPreloadManager.Builder this turns into rather a lot easier:

• Construct a DefaultPreloadManager and ExoPlayer situations with all default parts.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer situations with customized sharing parts.

val preloadManagerBuilder = DefaultPreloadManager.Builder().setRenderersFactory(customRenderersFactory)
// The ensuing preloadManager makes use of customRenderersFactory
val preloadManager = preloadManagerBuilder.construct()
// The ensuing participant makes use of customRenderersFactory
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer situations, whereas setting the customized playback-only configurations on the ExoPlayers.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
// Tune the playback-only configurations
val playerBuilder = ExoPlayer.Builder().setFooEnabled()
// The ensuing participant can have playback function "Foo" enabled
val participant = preloadManagerBuilder.buildExoPlayer(playerBuilder)

Preloading the following playlist merchandise

We’ve added the power to preload the following merchandise within the playlist of ExoPlayer. By default, playlist preloading is disabled however will be enabled by setting the period which needs to be preloaded to reminiscence:

participant.preloadConfiguration =
    PreloadConfiguration(/* targetPreloadDurationUs= */ 5_000_000L)

With the PreloadConfiguration above, the participant tries to preload 5 seconds of media for the following merchandise within the playlist. Preloading is barely began when no media is being loaded that’s required for the continuing playback. This fashion preloading doesn’t compete for bandwidth with the first playback.

When enabled, preloading may help reduce be a part of latency when a person skips to the following merchandise earlier than the playback buffer reaches the following merchandise. The primary interval of the following window is ready and video, audio and textual content samples are preloaded into its pattern queues. The preloaded interval is later queued into the participant with preloaded samples instantly accessible and able to be fed to the codec for rendering.

As soon as opted-in, playlist preloading will be turned off once more by utilizing PreloadConfiguration.DEFAULT to disable playlist preloading:

participant.preloadConfiguration = PreloadConfiguration.DEFAULT

New IAMF decoder and Kotlin listener extension

The 1.5.0 launch features a new media3-decoder-iamf module, which permits playback of IAMF immersive audio tracks in MP4 recordsdata. Apps wanting to do that out might want to construct the libiamf decoder regionally. See the media3 README for full directions.

implementation ("androidx.media3:media3-decoder-iamf:1.5.0")

This launch additionally features a new media3-common-ktx module, a house for Kotlin-specific performance. The primary model of this module incorporates a droop perform that lets the caller take heed to Participant.Listener.onEvents. It is a constructing block that’s utilized by the upcoming media3-ui-compose module (launching with media3 1.6.0) to energy a Jetpack Compose playback UI.

implementation ("androidx.media3:media3-common-ktx:1.5.0")

Simpler Participant customization through delegation

Media3 has offered a ForwardingPlayer implementation since model 1.0.0, and now we have beforehand urged that apps ought to use it after they need to customise the way in which sure Participant operations work, by utilizing the decorator sample. One quite common use-case is to permit or disallow sure participant instructions (to be able to present/disguise sure buttons in a UI). Sadly, doing this appropriately with ForwardingPlayer is surprisingly arduous and error-prone, as a result of you must persistently override a number of strategies, and deal with the listener as properly. The instance code to exhibit how fiddly that is too lengthy for this weblog, so we’ve put it in a gist as an alternative.

As a way to make these kinds of customizations simpler, 1.5.0 features a new ForwardingSimpleBasePlayer, which builds on the consistency ensures offered by SimpleBasePlayer to make it simpler to create constant Participant implementations following the decorator sample. The identical command-modifying Participant is now a lot easier to implement:

class PlayerWithoutSeekToNext(participant: Participant) : ForwardingSimpleBasePlayer(participant) {
  override enjoyable getState(): State {
    val state = tremendous.getState()
    return state
      .buildUpon()
      .setAvailableCommands(
        state.availableCommands.buildUpon().take away(COMMAND_SEEK_TO_NEXT).construct()
      )
      .construct()
  }

  // We needn't override handleSeek, as a result of it's assured to not be known as for
  // COMMAND_SEEK_TO_NEXT since we have marked that command unavailable.
}

MediaSession: Command button for media gadgets

Command buttons for media gadgets permit a session app to declare instructions supported by sure media gadgets that then will be conveniently displayed and executed by a MediaController or MediaBrowser:

You will discover the detailed documentation on android.developer.com.

That is the Media3 equal of the legacy “customized browse actions” API, with which Media3 is absolutely interoperable. In contrast to the legacy API, command buttons for media gadgets don’t require a MediaLibraryService however are a function of the Media3 MediaSession as an alternative. Therefore they’re accessible for MediaController and MediaBrowser in the identical method.

In case you encounter any points, have function requests, or need to share suggestions, please tell us utilizing the Media3 problem tracker on GitHub. We look ahead to listening to from you!