MozillaWiki

WebAPI/AudioChannels: Difference between revisions

Page Discussion

WebAPI/AudioChannels (view source)

Revision as of 04:00, 20 October 2015

3,754 bytes added ,  20 October 2015
→‎Audio Competing Policy
No edit summary
 
(21 intermediate revisions by 7 users not shown)
Line 20: Line 20:
* ''publicnotification'': Forced camera shutter sounds. '''This will not be in V1'''.
* ''publicnotification'': Forced camera shutter sounds. '''This will not be in V1'''.


Whenever an audio channel is used, or lower priority channels are automatically paused. The only exception to this is that "normal" and "content" has the same priority which means that if the "content" channel is used, it's simply mixed with audio from the "normal" channel.
Whenever an audio channel is used, lower priority channels are automatically paused. The only exception to this is that "normal" and "content" has the same priority which means that if the "content" channel is used, it's simply mixed with audio from the "normal" channel.


If two apps try to use the "content" channel at the same time, the foreground app wins. If both of the apps are background apps, then the last app to try to use the channel wins.
If two apps try to use the "content" channel at the same time, the foreground app wins. If both of the apps are background apps, then the last app to try to use the channel wins.
Line 41: Line 41:
   attribute boolean telephonySpeaker; // '''Not in V1''' Makes the "telephony" channel go through the speaker.
   attribute boolean telephonySpeaker; // '''Not in V1''' Makes the "telephony" channel go through the speaker.
   
   
   readonly attribute DOMString[] occupiedChannels;
   attribute DOMString volumeControlChannel; // The channel to adjust the volume for when volume keys are pressed
  void setOccupyChannel(DOMString channel, boolean occupy);
  }
  }


We'll additionally introduce a new attribute "mozchannel" on <audio> and <video> which selects which audio channel will be used for any played audio.
We'll additionally introduce a new attribute "mozAudioChannelType" on <audio> and <video> which selects which audio channel will be used for any played audio.


  partial interface HTMLMediaElement {
  partial interface HTMLMediaElement {
   attribute DOMString mozchannel; // Or are we using some other name?
   attribute DOMString mozAudioChannelType;
   
   
   readonly attribute boolean mozchannelpaused; // Returns true if this element is currently paused due to the channel being paused.
   readonly attribute boolean mozchannelpaused; // Returns true if this element is currently paused due to the channel being paused. Not implemented yet.
   // Should these events only fire if the element is currently played?
   // These events only fire if the element is currently played.
   attribute EventTarget onmozchannelpaused;
   attribute EventTarget onmozinterruptbegin;
   attribute EventTarget onmozchannelunpaused;
   attribute EventTarget onmozinterruptend;
  }
  }


Line 69: Line 68:
   // range: 0.0 to 1.0 - a custom volume
   // range: 0.0 to 1.0 - a custom volume
   attribute double audioVolume;
   attribute double audioVolume;
  // list of active audio channels
  readonly attribute DOMString[] activeAudioChannels;
   
   
   // list of active and non muted audio channels
   // list of active and non muted audio channels
   readonly attribute DOMString[] soundingAudioChannels;
   readonly attribute boolean hasSoundingChannels;
   
   
   // when this attribute is set to true, any normal channel will be 'converted'
   // when this attribute is set to true, any normal channel will be 'converted'
Line 82: Line 78:
   // visibility and normal channels...
   // visibility and normal channels...
   attribute boolean treatNormalAsContent;
   attribute boolean treatNormalAsContent;
  // When called, if this app is currently attempting to play audio on the "content"
  // channel, but is non-visible and the "content" channel is muted for this app, calling
  // this function will make the system pick this app or tab as the one to provide content audio.
  // If this function is called and there is another visible app or tab which uses the "content"
  // channel, then this function is a no-op.
  void startBackgroundContentAudio();
  }
  }


Line 132: Line 121:


I'm not sure if we need to have prompts if non-privileged apps try to use channels beyond the "normal" or "content" channels.
I'm not sure if we need to have prompts if non-privileged apps try to use channels beyond the "normal" or "content" channels.
== Audio Competing Policy ==
<table border="1" cellspacing="0" cellpadding="3" align="center">
<tr>
<th>AudioChannelType
<th>Priority
<th>Foreground
<th>Background
<th>Timing of Resuming
<th>Faded Effect
<tr>
<td>normal
<td>0
<td>Play
<td>[1]
<td>Transit to foreground.
<td>If notification is fired, volume will be reduced to 20% of current level.
<tr>
<td>content
<td>1
<td>Play
<td>[1], [2]
<td valign="middle" align="left">
If the muted/paused is caused by
  1. [1] then will be resumed when [3].
  2. [2] then will be resumed to play when transit to foreground.
<td>As above.
<tr>
<td>notification
<td>2
<td>Play
<td>[1]
<td>[3]
<td>N/A
<tr>
<td>alarm
<td>3
<td>Play
<td>[1]
<td>[3]
<td>N/A
<tr>
<td>telephony
<td>4
<td>Play
<td>[1]
<td>[3]
<td>N/A
<tr>
<td>ringer
<td>5
<td>Play
<td>[1]
<td>[3]
<td>N/A
<tr>
<td>public notification
<td>6
<td>Play
<td>Play
<td>N/A
<td>N/A
</table>
[1] If there is any higher priority channel in playing then muted or paused.
[2] If there is any other content channel starting to play in foreground or background then current content channel in playing will be muted or paused.
[3] No higher priority channels are in playing.


== Use cases and requirements ==
== Use cases and requirements ==
Line 144: Line 215:


Ability for the system app to:
Ability for the system app to:
* Turn down the volume of background audio rather than completely silence it when the notification channel is used
* Turn down the volume of background audio rather than completely silence it when the notification channel is used
* Figure out which volume to modify when the user is pressing the volume buttons.
* Figure out which volume to modify when the user is pressing the volume buttons.


Ability for apps to:
Ability for apps to:
* Act as if it's occupying a channel without actually using a media element playing on that channel. This is useful for example to prepare for audio which is about to start playing, or to prepare for turning on the camera.
* Act as if it's occupying a channel without actually using a media element playing on that channel. This is useful for example to prepare for audio which is about to start playing, or to prepare for turning on the camera.
=== Volume control ===


We also have a set of usage scenarios around volume control. We currently have 4 different volumes which makes it tricky to decide which one (or ones) to modify when the user presses the volume buttons. Here's a set of situations where we need to figure out which volume to control:
We also have a set of usage scenarios around volume control. We currently have 4 different volumes which makes it tricky to decide which one (or ones) to modify when the user presses the volume buttons. Here's a set of situations where we need to figure out which volume to control:
Line 159: Line 234:
* When using the built-in fmradio or music app and audio is playing.
* When using the built-in fmradio or music app and audio is playing.
* When using the built-in fmradio or music app and audio is not playing.
* When using the built-in fmradio or music app and audio is not playing.
* User lowers volume to a very low volume, this likely either affects content or ringer volume. Suddenly there's an incoming notification. However since this is a separate volume this can be very loud which is surprising to the user.
* User enters dialer and lowers volume. Then presses DTMF tone buttons and expects this to play at a low volume.
=== Refactor Audio Channel Service ===
(04/30/2015 updated)
* Gecko: The concept of refactoring the audio channel service is to deal with those issues that we couldn't solve by the existing mechanism in gecko, and we will need the changes(in below) in both gecko and gaia to have a new management.
** '''New browser api''': {{Bug|1113086}} - this is the major part for the new browser api so that the gaia system app is able to control each iframe/app's audio channel, such as allowing or denying those types we have in b2g.
*** It's almost completed, just lacking of the volume control of the FM Radio.
** '''Telephony''': {{Bug|1129882}} - The new audio channel browser api will be implemented base on the foundation of media element, but the telephony api does not use media element to produce sound, so we need the telephony api to bind on a specific window(callscreen), just like the other apps did.
*** Review finish, wait for landing.
** '''VOIP''': {{Bug|1126224}} - There will be 3rd party apps that use webrtc api to implement voip services, and to fit the ux spec, these apps might behave different from the gaia callscreen app, so probably we will need a new type called "voip" to distinguish from "telephony" and "voip" audio channels.
*** This issue is paused now, since we have no the consensus about whether we really need the "voip" type.
** '''System''': {{Bug|1142933}} - The ux spec has specified a new audio channel type called "system" to represent those sounds that system use, such as the keyboard, screen lock or screenshot...
*** Review finish, wait for landing.
* Gaia: The system app will use the browser api({{Bug|1113086}}) to control all the audio channels in gaia, it means we will move the audio channel service from gecko to gaia, since gaia has more window related information then gecko and can do a better management.
** '''Audio Channel Manager''': {{Bug|1100822}} - we will introduce this new module in system app, to manage the audio channels.
*** Landed.
** '''System''': {{Bug|1157140}} - Manage System app's audio channels in the AudioChannelManager.
*** Figuring out a good way to manage the audio channels in System app.
** '''Callscreen''': {{Bug|1159610}} - The Callscreen app needs to call the telephony's audio channel api.
** '''Cleanup''': {{Bug|1139838}} - Remove all the hacks for audio channel.
* UX: Before the refactoring started, developers and ux had meetings to discuss about the current audio issues and future needs, then came out a sound spec to define the audio channel behaviours.
** '''Spec''': {{Bug|1068219}} - as the developers are implementing the new modules, meanwhile ux is also updating the spec if we encounter some problems on the behaviours or coding limitation.
[[Category:Web APIs]]
Evanxd
Confirmed users
381

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/639850...1101414"