TPESystem/Media/DevicePortingIssues

TPESystem/Media/DevicePortingIssues

Device porting issues of media playback on Firefox OS

Problems And Solutions

We've encountered some similar media playback issues while porting different devices. The following list contains some common problems we've found and our current solutions as well:

1. Problem: I cannot playback video files with large resolution.
   • Solution: Make sure ro.moz.omx.hw.max_width & ro.moz.omx.hw.max_height values are suitable for video decoding on the target device.
   • Reference: bug 819609 and bug 911113
     • Each time one video clip is going to be played, OMXCodecProxy is used to handle resource management of hardware video decoders.
     • After one video OMXCodec has been created successfully, OMXCodecProxy will double check the product of these two values with the actual video resolution.
     • If the video resolution is bigger than the product of these two values, OMXCodecProxy will destroy this newly created OMXCodec. As a result, the playback of this video clip will be abandoned in this case.
2. Problem: I cannot generate thumbnails of my video files in the video application.
   • Solution: Check the color format enum definition, HAL_PIXEL_FORMAT_XXX, to see if GrallocImage::GetAsSurface() can handle color format conversion correctly.
   • Reference: bug 891274 and bug 931733
     • Currently HAL_PIXEL_FORMAT_XXX is hardcoded in GrallocImages.h.
     • The way how HAL_PIXEL_FORMAT_XXX should be ported in Firefox OS will be improved in the near future.
3. Problem: I always encounter OMXCodec::waitForBufferFilled_l() timeout with "Timed out waiting for output buffers: 0/xxx" message while playing some specific video files (usually mp4 files.)
   • Solution: Make sure all the buffers allocated from ANativeWindows can be used for video decoding.
   • Reference: attachment 744257 [edit] on bug 864230
     • In AOSP, ANativeWindow is used as a buffer source and also as a render target. ANativeWindow needs to keep at least minUndequeuedBufs while rendering video. Although some number of buffers have to be kept in ANativeWindow, every buffer in the buffer queue can takes turns to be filled, be rendered, and be emptied. So utilization of buffers in the queue is ok.
     • In Firefox OS, ANativeWindow is used as a buffer source, but not as a render target. If we keep minUndequeuedBufs buffers in ANativeWindow, these buffers will always be kept in ANativeWindow and never have chances to be filled. As a result, it's very easy to encounter buffer starvation during video decoding.
4. Problem: I cannot resume playback by clicking the play button after the video/audio has been paused.
   • Solution: Make sure to return ERROR_UNSUPPORTED in OMXCodec::pause() if underlying OMX IL component doesn't support pause and resume function, or OMXCodec itself cannot be resumed by OMXCodec::start() after OMXCodec::pause() has been called.
   • Reference: attachment 8340486 [edit] on bug 919590, or this pull request.
5. Problem: Keyboard or dial sound is missing in first touch
   • Solution: Make sure that PCM data decoded by AudioFlinger is right.
     • Find AudioFlinger source code, Each project has different location. According Tarako, source code are in frameworks/base/services/audioflnger
     • Add some code bug 977481 to dump audio pcm data at AudioFlinger::MixerThread::threadLoop()
   • Reference: bug 967581
6. Problem: Publicnotification audio channel type is not supported
   • Solution: add SL_ANDROID_STREAM_SYSTEM_ENFORCED & CUBEB_STREAM_TYPE_ENFORCED_AUDIBLE in FxOS
     • add SYSTEM_ENFORCED (ENFORCED_AUDIBLE) in sles_definitions.h
     • support CUBEB_STREAM_TYPE_SYSTEM_ENFORCED in cubeb_opensl.c & AudioStream.cpp
   • Reference: bug 1007552
7. Problem: Playing music is choopy when the A2DP is connected
   • Solution: restrict the framecount value
   • Reference: bug 1045489
8. Problem: I encounter a weird bug only on JellyBean from CodeAuroraForum with Gecko v2.0 (or above).
   • Solution: Sorry. There are no plans for CAF to support v2.0 with JB.
   • Reference: Comment 16 on bug 1045351