TPESystem/Media/DevicePortingIssues

TPESystem/Media/DevicePortingIssues

Device porting issues of media playback on Firefox OS

Problems And Solutions

We've encountered some similar media playback issues while porting different devices. The following list contains some common problems we've found and our current solutions as well:

1. Problem: I cannot playback video files with large resolution.
   • Solution: Make sure ro.moz.omx.hw.max_width & ro.moz.omx.hw.max_height values are suitable for video decoding on the target device.
   • Reference: bug 819609 and bug 911113
     • Each time one video clip is going to be played, OMXCodecProxy is used to handle resource management of hardware video decoders.
     • After one video OMXCodec has been created successfully, OMXCodecProxy will double check the product of these two values with the actual video resolution.
     • If the video resolution is bigger than the product of these two values, OMXCodecProxy will destroy this newly created OMXCodec. As a result, the playback of this video clip will be abandoned in this case.
2. Problem: I cannot generate thumbnails of my video files in the video application.
   • Solution: Check the color format enum definition, HAL_PIXEL_FORMAT_XXX, to see if GrallocImage::GetAsSurface() can handle color format conversion correctly.
   • Reference: bug 891274 and bug 931733
     • Currently HAL_PIXEL_FORMAT_XXX is hardcoded in GrallocImages.h.
     • The way how HAL_PIXEL_FORMAT_XXX should be ported in Firefox OS will be improved in the near future.
3. Problem: I always encounter OMXCodec::waitForBufferFilled_l() timeout with "Timed out waiting for output buffers: 0/xxx" message while playing some specific video files (usually mp4 files.)
   • Solution: Make sure all the buffers allocated from ANativeWindows can be used for video decoding.
   • Reference: attachment 744257 [edit] on bug 864230
     • In AOSP, ANativeWindow is used as a buffer source and also as a render target. ANativeWindow needs to keep at least minUndequeuedBufs while rendering video. Although some number of buffers have to be kept in ANativeWindow, every buffer in the buffer queue can takes turns to be filled, be rendered, and be emptied. So utilization of buffers in the queue is ok.
     • In Firefox OS, ANativeWindow is used as a buffer source, but not as a render target. If we keep minUndequeuedBufs buffers in ANativeWindow, these buffers will always be kept in ANativeWindow and never have chances to be filled. As a result, it's very easy to encounter buffer starvation during video decoding.
4. Problem: I cannot resume playback by clicking the play button after the video/audio has been paused.
   • Solution: Make sure to return ERROR_UNSUPPORTED in OMXCodec::pause() if underlying OMX IL component doesn't support pause and resume function.
   • Reference: attachment 8340486 [edit] on bug 919590
     • This solution is not yet review+, but this is the suggested solution based on the discussion on bugzilla.
5. Problem: Keyboard or dial sound is missing in first touch
   • Solution: Make sure that PCM data decoded by AudioFlinger is right.
     • Find AudioFlinger source code, Each project has different location. According Tarako, source code are in frameworks/base/services/audioflnger
     • Add some code bug 977481 to dump audio pcm data at AudioFlinger::MixerThread::threadLoop()
   • Reference: bug 967581
6. Problem: Publicnotification audio channel type is not supported
   • Solution:
     • (1)add SYSTEM_ENFORCED (ENFORCED_AUDIBLE) in sles_definitions.h
     • (2)support CUBEB_STREAM_TYPE_SYSTEM_ENFORCED in cubeb_opensl.c & AudioStream.cpp
   • Reference: bug 1007552