Platform/Features/MPAPI
Status
| Media Plugin API (MPAPI) | |
| Stage | Planning |
| Status | In progress |
| Release target | Firefox 15 or 16 (TBD) - Mobile only, by the end of Q2 |
| Health | OK |
| Status note | Most of the design has been completed by Andreas. Media is finishing off the design and planning the implementation. |
Team
| Product manager | Maire Reavy |
| Directly Responsible Individual | Maire Reavy |
| Lead engineer | Rob O'Callahan (formerly Andreas Gal) |
| Security lead | TBD |
| Privacy lead | ` |
| Localization lead | ` |
| Accessibility lead | ` |
| QA lead | TBD |
| UX lead | ` |
| Product marketing lead | ` |
| Operations lead | ` |
| Additional members | Chris Double, Chris Pearce, Edwin Flores |
Open issues/risks
TBD
Stage 1: Definition
1. Feature overview
This feature gives gecko the ability to use a device's hardware decoder to play back encoded video or audio (in particular H.264, AAC, and MP3) on that device. We are targeting Mobile for the first release.
We may use this same approach to support certain codecs (like MP3) on desktop; desktop support is still TBD.
2. Users & use cases
- App wishes to playback MP3 file on user's mobile device
- App wishes to playback H.264 on user's mobile device (note: the H.264 video may use AAC to encode the audio track of the video)
3. Dependencies
- The user's device must have the codec needed for playback already installed, or else this feature will appear not to work.
4. Requirements
- Playback performance on a device that has the necessary codecs installed must be snappy and appear seamless. (If anyone needs more detail like how quickly playback should begin or what qualifies as seamless, please contact Maire.)
- Audio/Video synchronization must be maintained
Non-goals
- Playback will not work if the user's device does not have the codec already installed.
- The user will not be prompted to download a codec in order to play a video/audio clip, if the codec is not available for playback.
Stage 2: Design
5. Functional specification
6. User experience design
`
Stage 3: Planning
7. Implementation plan
Phase 1 : Mobile
1) Review, polish and land existing patches for using system codecs on Android/B2G (bug 714408). Those patches create an "MPAPI" interface into which various codec libraries can be plugged. These libraries will be wrappers around libstagefright or other Android system codec interfaces --- different libraries customized for different Android versions and/or devices as required. Currently these patches only work on devices where we can get the decoded YUV data in main memory (MPAPI currently depends on this). Chris Double is currently working on this. Some contributors are also involved.
2) Following on from #1, extend MPAPI to handle cases where we don't get YUV data. The frames might be made available only as some kind of overlay. To integrate this into Gecko as best we can, we'll need to modify the layers system to work with those overlays. The exact details of this depend on exactly what the hardware and APIs can do; needs investigation. Not sure who'll work on this yet, but Chris D will be involved and I'll probably help.
3 (below) is not part of the product, but will land to help contributors:
3) Review and land latest contributed GStreamer patch (bug 422540), which is quite good. We currently have no plans to ship this in a Mozilla product, but contributors want it so it might as well be in the tree as they're using it. This patch was contributed by Alessandro Decina. Review and landing will be done by Chris Double. UPDATE: Patch landed on 4/19/2012.
Phase 2 : Desktop (still TBD if we do this)
If we decide to support MP3 on desktop via MPAPI, #4 (below) explains the plan we would follow:
4) Investigate MP3 support using system codecs on desktop platforms.
- On Windows (XP and up) we can use DirectShow. Chris Pearce is going to look into this.
- On Mac we can use some Mac APIs. Edwin Flores is going to look into this.
- On Linux we could use libav (when MP3 codec is available) directly, or possibly via GStreamer (see #3). Not sure who's going to work on that. The currently plan is to use these system APIs directly in dedicated codec backends rather than wrap them in some other abstraction such as MPAPI, but this may be revisited.
8. Reviews
Security review
`
Privacy review
`
Localization review
`
Accessibility
`
Quality Assurance review
`
Operations review
`
Stage 4: Development
9. Implementation
`
Stage 5: Release
10. Landing criteria
`
Feature details
| Priority | P1 |
| Rank | 999 |
| Theme / Goal | Experience |
| Roadmap | Firefox Mobile |
| Secondary roadmap | Platform |
| Feature list | Mobile |
| Project | Apps |
| Engineering team | Media |
Team status notes
| status | notes | |
| Products | ` | ` |
| Engineering | ` | ` |
| Security | sec-review-needed | bug 749221 |
| Privacy | ` | ` |
| Localization | ` | ` |
| Accessibility | ` | ` |
| Quality assurance | ` | ` |
| User experience | ` | ` |
| Product marketing | ` | ` |
| Operations | ` | ` |
