OpenSL Audio Fast Path and Bluetooth output issues

[Yury Habets]

Hi!

Some background: we are using FMOD as the audio middleware, and have
changed the default output to OpenSL (for devices that report
FEATURE_AUDIO_LOW_LATENCY).
(I'm skipping the sad part about the devices which report the feature, but
actually do not support fast mixer :( ).
For capable devices, we are detecting the native params (sampling frequency
and buffer size), and applying them accordingly. We are "feeding" the
simple buffer queue with buffers of native size on each callback. Default
total ring buffer size is 512 samples x 4, it is being filled in a separate
mixer thread to prevent underruns. The things are working pretty fine so
far.

However, attaching a Bluetooth speaker/headset breaks the sound.
I’m testing on a Nexus 6 (48kHz/192 samples native device params) and a
small BT speaker. A simplest project that plays an audio clip on button
click.

I've used a logging version of FMOD to understand what’s going on, and here
is what I see:

Before I connect the Bluetooth device, updateOutputCallback that I
registered for the SL_IID_ANDROIDSIMPLEBUFFERQUEUE is being called every
~4ms which perfectly matches the native params. I’m “feeding” the buffer
queue with 192 samples of audio. The sound goes perfectly well.

After I connect the Bluetooth device, the updateOutputCallback timing is
becoming strange. It is being called every 50 to 150 ms, consuming
enourmous amount of data (around 5000 samples), and then sleeping again for
50-150 ms. Since the default total audio ring buffer size is 512x4 = 2048
samples, the sound gets played “looped” and slowly progressing through the
audio clip.

I can fix it by forcing the buffer size to 2048x4 (I am still submitting
192 samples to the buffer queue every callback). This is not a nice
solution since changing buffer configuration in runtime means FMOD reset
and reloading sounds, but it’s better than nothing. Also it means *HUGE*
latency :(

Here is what I see in logcat when connecting the speaker:

01-21 12:58:50.838: I/AudioFlinger(373): AudioStreamOut::open(),
mHalFormatIsLinearPcm = 1
01-21 12:58:50.838: I/AudioFlinger(373): HAL output buffer size 2560
frames, normal sink buffer size 2560 frames
01-21 12:58:50.839: I/AudioFlinger(373): AudioFlinger's thread 0xafb40000
ready to run

Do my observations look correct? Is the strange callback timing related to
power saving needed for Bluetooth?
Does it make sense to change the ring buffer size in runtime to 4x2048, and
then reset it back to initial value when BT output is disconnected?

Or is it really easier to switch the audio backend to AudioTrack for
Bluetooth output?

Another question is - how do I receive the notifications about BT audio
output connected/disconnected?
I checked this
http://developer.android.com/training/managing-audio/audio-output.html,
tried registering a receiver for the intent
http://developer.android.com/reference/android/media/AudioManager.html#ACTION_SCO_AUDIO_STATE_UPDATED,
but it does not seem to work well for me (no disconnect notification). Is
this the correct direction?

Thanks!
Yury

[a***@soundbrenner.com]

Hi Yury! Did you come to any conclusions with this subject? It would really
help me out. We're using the optimal sample rate with Bluetooth speakers
and seeing a lot of instability in the audio callback. Also the AudioTrack
denies it and defaults to 44100 and logs a warning about that.

Post by Yury Habets
Hi!
Some background: we are using FMOD as the audio middleware, and have
changed the default output to OpenSL (for devices that report
FEATURE_AUDIO_LOW_LATENCY).
(I'm skipping the sad part about the devices which report the feature, but
actually do not support fast mixer :( ).
For capable devices, we are detecting the native params (sampling
frequency and buffer size), and applying them accordingly. We are "feeding"
the simple buffer queue with buffers of native size on each callback.
Default total ring buffer size is 512 samples x 4, it is being filled in a
separate mixer thread to prevent underruns. The things are working pretty
fine so far.
However, attaching a Bluetooth speaker/headset breaks the sound.
I’m testing on a Nexus 6 (48kHz/192 samples native device params) and a
small BT speaker. A simplest project that plays an audio clip on button
click.
I've used a logging version of FMOD to understand what’s going on, and
Before I connect the Bluetooth device, updateOutputCallback that I
registered for the SL_IID_ANDROIDSIMPLEBUFFERQUEUE is being called every
~4ms which perfectly matches the native params. I’m “feeding” the buffer
queue with 192 samples of audio. The sound goes perfectly well.
After I connect the Bluetooth device, the updateOutputCallback timing is
becoming strange. It is being called every 50 to 150 ms, consuming
enourmous amount of data (around 5000 samples), and then sleeping again for
50-150 ms. Since the default total audio ring buffer size is 512x4 = 2048
samples, the sound gets played “looped” and slowly progressing through the
audio clip.
I can fix it by forcing the buffer size to 2048x4 (I am still submitting
192 samples to the buffer queue every callback). This is not a nice
solution since changing buffer configuration in runtime means FMOD reset
and reloading sounds, but it’s better than nothing. Also it means *HUGE*
latency :(
01-21 12:58:50.838: I/AudioFlinger(373): AudioStreamOut::open(),
mHalFormatIsLinearPcm = 1
01-21 12:58:50.838: I/AudioFlinger(373): HAL output buffer size 2560
frames, normal sink buffer size 2560 frames
01-21 12:58:50.839: I/AudioFlinger(373): AudioFlinger's thread 0xafb40000
ready to run
Do my observations look correct? Is the strange callback timing related to
power saving needed for Bluetooth?
Does it make sense to change the ring buffer size in runtime to 4x2048,
and then reset it back to initial value when BT output is disconnected?
Or is it really easier to switch the audio backend to AudioTrack for
Bluetooth output?
Another question is - how do I receive the notifications about BT audio
output connected/disconnected?
I checked this
http://developer.android.com/training/managing-audio/audio-output.html,
tried registering a receiver for the intent
http://developer.android.com/reference/android/media/AudioManager.html#ACTION_SCO_AUDIO_STATE_UPDATED,
but it does not seem to work well for me (no disconnect notification). Is
this the correct direction?
Thanks!
Yury

[Philippe Simons]

low latency (Fast Path) on Bluetooth is just not possible.

The main reason for this is because Bluetooth audio stream is compressed
before being sent in the air.
That compression needs a big buffer to be filled, and that buffer is the
culprit.

There is no nothing you can do about it.

Post by a***@soundbrenner.com
Hi Yury! Did you come to any conclusions with this subject? It would
really help me out. We're using the optimal sample rate with Bluetooth
speakers and seeing a lot of instability in the audio callback. Also the
AudioTrack denies it and defaults to 44100 and logs a warning about that.

Post by Yury Habets
Hi!
Some background: we are using FMOD as the audio middleware, and have
changed the default output to OpenSL (for devices that report
FEATURE_AUDIO_LOW_LATENCY).
(I'm skipping the sad part about the devices which report the feature,
but actually do not support fast mixer :( ).
For capable devices, we are detecting the native params (sampling
frequency and buffer size), and applying them accordingly. We are "feeding"
the simple buffer queue with buffers of native size on each callback.
Default total ring buffer size is 512 samples x 4, it is being filled in a
separate mixer thread to prevent underruns. The things are working pretty
fine so far.
However, attaching a Bluetooth speaker/headset breaks the sound.
I’m testing on a Nexus 6 (48kHz/192 samples native device params) and a
small BT speaker. A simplest project that plays an audio clip on button
click.
I've used a logging version of FMOD to understand what’s going on, and
Before I connect the Bluetooth device, updateOutputCallback that I
registered for the SL_IID_ANDROIDSIMPLEBUFFERQUEUE is being called every
~4ms which perfectly matches the native params. I’m “feeding” the buffer
queue with 192 samples of audio. The sound goes perfectly well.
After I connect the Bluetooth device, the updateOutputCallback timing is
becoming strange. It is being called every 50 to 150 ms, consuming
enourmous amount of data (around 5000 samples), and then sleeping again for
50-150 ms. Since the default total audio ring buffer size is 512x4 = 2048
samples, the sound gets played “looped” and slowly progressing through the
audio clip.
I can fix it by forcing the buffer size to 2048x4 (I am still submitting
192 samples to the buffer queue every callback). This is not a nice
solution since changing buffer configuration in runtime means FMOD reset
and reloading sounds, but it’s better than nothing. Also it means *HUGE*
latency :(
01-21 12:58:50.838: I/AudioFlinger(373): AudioStreamOut::open(),
mHalFormatIsLinearPcm = 1
01-21 12:58:50.838: I/AudioFlinger(373): HAL output buffer size 2560
frames, normal sink buffer size 2560 frames
01-21 12:58:50.839: I/AudioFlinger(373): AudioFlinger's thread 0xafb40000
ready to run
Do my observations look correct? Is the strange callback timing related
to power saving needed for Bluetooth?
Does it make sense to change the ring buffer size in runtime to 4x2048,
and then reset it back to initial value when BT output is disconnected?
Or is it really easier to switch the audio backend to AudioTrack for
Bluetooth output?
Another question is - how do I receive the notifications about BT audio
output connected/disconnected?
I checked this
http://developer.android.com/training/managing-audio/audio-output.html,
tried registering a receiver for the intent
http://developer.android.com/reference/android/media/AudioManager.html#ACTION_SCO_AUDIO_STATE_UPDATED,
but it does not seem to work well for me (no disconnect notification). Is
this the correct direction?
Thanks!
Yury
--

You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/15a95005-1294-4cb0-8d0e-33c143210347%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/15a95005-1294-4cb0-8d0e-33c143210347%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.