How to use OpenSL ES with USB Digital Audio devices (Headphones)

Thank you for your post.
Android "M" includes new SDK APIs for explicit routing; see
http://developer.android.com/reference/android/media/AudioTrack.html#setPreferredDevice(android.media.AudioDeviceInfo)
A similar explicit routing API is not available in the "M" NDK (native
audio based on OpenSL ES).
Thank you also for your experience with IN_COMMUNICATION on some devices.
I will investigate that.

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in c/c++ to
be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or Wired
Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration
(sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html we
can read that system automatically routes audio playback and capture
appropriately, based on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed through
it?
I know that the system route audio automatically to USB but It would be
good to know about that and open the device in correct configuration e.g.
stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Rich Eakin

2015-10-02 21:43:59 UTC

Post by Glenn Kasten
Thank you for your post.
Android "M" includes new SDK APIs for explicit routing; see
http://developer.android.com/reference/android/media/AudioTrack.html#setPreferredDevice(android.media.AudioDeviceInfo)
A similar explicit routing API is not available in the "M" NDK (native
audio based on OpenSL ES).

I've also been wondering if there are any ways to get more audio device
details. Is it possible to utilize AudioTrack.setPreferredDevice() while
still running OpenSL / native audio code? Similar to how we are using
AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE, for example, to correctly set the
samplerate on SLDataFormat_PCM in C / C++.

cheers,
Rich

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in c/c++
to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or Wired
Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration
(sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html
we can read that system automatically routes audio playback and capture
appropriately, based on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed
through it?
I know that the system route audio automatically to USB but It would be
good to know about that and open the device in correct configuration e.g.
stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Glenn Kasten

2015-10-02 21:49:52 UTC

The difference is that AudioManager returns static information, and so
information returned using the SDK API is
also useful with the NDK API.

However, AudioTrack is a per-stream object, and so the SDK and NDK APIs are
not inter-operable on the same stream.
("stream" here meaning a sequence of audio data, not "stream type" as used
for the volume control APIs).

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in c/c++
to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or
Wired Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration
(sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html
we can read that system automatically routes audio playback and capture
appropriately, based on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed
through it?
I know that the system route audio automatically to USB but It would be
good to know about that and open the device in correct configuration e.g.
stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Rich Eakin

2015-10-03 00:20:54 UTC

(Created new thread so as not to hijack the original)

Thanks for the information. Are there any plans to also provide device
information to OpenSL users? Currently, I'm wondering how to query whether
the device is mono or stereo, but I could also see targeting specific
hardware a nice feature as well, such as what you can do in Android M with
AudioTrack.

Thanks,
Rich

Post by Glenn Kasten
The difference is that AudioManager returns static information, and so
information returned using the SDK API is
also useful with the NDK API.
However, AudioTrack is a per-stream object, and so the SDK and NDK APIs
are not inter-operable on the same stream.
("stream" here meaning a sequence of audio data, not "stream type" as used
for the volume control APIs).

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in c/c++
to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or
Wired Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration
(sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html
we can read that system automatically routes audio playback and capture
appropriately, based on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed
through it?
I know that the system route audio automatically to USB but It would be
good to know about that and open the device in correct configuration e.g.
stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Glenn Kasten

2015-10-03 15:34:56 UTC

You can use JNI to call SDK APIs from native code.
Note that the OpenSL ES buffer completion callback runs on a
non-JNI-capable thread,
so you cannot call SDK APIs from that callback handler and will need to use
a JNI-capable thread instead.
If you are requesting a pure native API, please file a feature request here:
https://code.google.com/p/android/issues/entry
I cannot give forward-looking statements, such as about plans.

Post by Rich Eakin
(Created new thread so as not to hijack the original)
Thanks for the information. Are there any plans to also provide device
information to OpenSL users? Currently, I'm wondering how to query whether
the device is mono or stereo, but I could also see targeting specific
hardware a nice feature as well, such as what you can do in Android M with
AudioTrack.
Thanks,
Rich

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in
c/c++ to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or
Wired Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate configuration
(sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation https://source.android.com/devices/audio/usb.html
we can read that system automatically routes audio playback and capture
appropriately, based on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed
through it?
I know that the system route audio automatically to USB but It would
be good to know about that and open the device in correct configuration
e.g. stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Rich Eakin

2015-10-04 06:14:38 UTC

Post by Glenn Kasten
You can use JNI to call SDK APIs from native code.
Note that the OpenSL ES buffer completion callback runs on a
non-JNI-capable thread,
so you cannot call SDK APIs from that callback handler and will need to
use a JNI-capable thread instead.
https://code.google.com/p/android/issues/entry
I cannot give forward-looking statements, such as about plans.

Not sure if I'm understanding your answer, so I'll be more specific - can I
make use of something in java, perhaps android.media.AudioDeviceInfo
<http://developer.android.com/reference/android/media/AudioDeviceInfo.html>,
to correctly configure the channels of my audio I/O setup in OpenSL? Right
now, in very early stages of my implementation, I'm just using stereo for
output and mono for input, but I've also read comments in others' code that
even stereo output is not always reliable. Basically if I could get that
info the AudioDeviceInfo and _be sure_ that the device it represents is the
same as the one I am configuring with OpenSL, then I'd feel much better
about it. Perhaps I can assume that AudioTrack.getRoutedDevice() represents
the same output device (SL_DATALOCATOR_OUTPUTMIX) that I'll be configuring
in OpenSL? Can I be sure?

Though I don't see why any OpenSL user wouldn't want to have a native API
to give them the information they needed, rather then having to jump
through an entirely different language to get basic device information, so
I've filed a feature request (188502
<https://code.google.com/p/android/issues/detail?id=188502>) for that.

Thanks again for the help,
Rich

Rafał Kanus

2015-10-05 14:45:03 UTC

OK. So is it true that if we read from Java
AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE we will get sample rate of current
primary audio output device?
If I switch to Speaker, I will get sample rate to built-in speaker.
If I plug in USB Headphones, I will get sample rate for USB Headphone
device (BTW what if device supports multiple sample rates).
Is this true?

What about Input device (e.g. USB Microphone). The is no equivalent
property for microphone sample rate.

Maybe the solution for USB Audio Devices will be to read USB Audio Device
Descriptor and get correct sample rate and channels from there.
Or parse /proc/asound/card1/stream0.
Then we can just pass that information to native C/C++ and open Input and
Output in correct configuration via OpenSL ES.
What do you think about that?

BTW: Since in Android 5 we don't have possibility to change manually route
from/to USB Aidio, I assume that USB Audio Device become primery right
after we plug it in.

W dniu sobota, 3 paÅºdziernika 2015 17:34:56 UTC+2 uÅŒytkownik Glenn Kasten

Post by RafaÅ Kanus
Welcome everybody,
I am new on this group and this is my first topic.
We are working on Voip application which is mostly implemented in
c/c++ to be multi-platform.
On Android we are using OpenSL ES API to access audio devices.
We don't have problems with Earpiece, Speaker, Bluetooth Headset or
Wired Headset devices.
We are able to detect which device is connected and in use using
AudioManager from Java.
We are also able to switch devices.
Based on that information we open devices in appropriate
configuration (sample rate, channels etc).
But how to deal with USB Audio devices?
Android 5 have official support for them.
In the documentation
https://source.android.com/devices/audio/usb.html we can read that
system automatically routes audio playback and capture appropriately, based
on the audio policy rules.
How to detect if USB Audio device is connected and audio is routed
through it?
I know that the system route audio automatically to USB but It would
be good to know about that and open the device in correct configuration
e.g. stereo instead mono.
Did someone had the same problems?
BTW: Nexus devices are working correctly with USB Audio devices but
Samsung do not route audio via USB when mode is set to IN_COMMUNICATION.
That seems to be broken.
Best regards,
Rafal Kanus

Rich Eakin

2015-10-05 15:02:23 UTC

Post by RafaÅ Kanus
OK. So is it true that if we read from Java
AudioManager.PROPERTY_OUTPUT_SAMPLE_RATE we will get sample rate of current
primary audio output device?
If I switch to Speaker, I will get sample rate to built-in speaker.
If I plug in USB Headphones, I will get sample rate for USB Headphone
device (BTW what if device supports multiple sample rates).
Is this true?

What about Input device (e.g. USB Microphone). The is no equivalent

Post by RafaÅ Kanus
property for microphone sample rate.

Since my original reply, I noticed AudioManager.getDevices()
<http://developer.android.com/reference/android/media/AudioManager.html#getDevices(int)>,
that returns an AudioDeviceInfo
<http://developer.android.com/reference/android/media/AudioDeviceInfo.html>
object.
Maybe this will help setup usb and input devices? I don't have anything
running Android M so can't test myself.

Glenn Kasten

2015-10-05 18:29:34 UTC

Rafal, thanks for the device names. We're investigating the discrepancy.

Post by RafaÅ Kanus

What about Input device (e.g. USB Microphone). The is no equivalent

Post by RafaÅ Kanus
property for microphone sample rate.

Since my original reply, I noticed AudioManager.getDevices()
<http://developer.android.com/reference/android/media/AudioManager.html#getDevices(int)>,
that returns an AudioDeviceInfo
<http://developer.android.com/reference/android/media/AudioDeviceInfo.html> object.
Maybe this will help setup usb and input devices? I don't have anything
running Android M so can't test myself.

Rafał Kanus

2015-10-08 10:44:05 UTC

W dniu poniedziaÅek, 5 paÅºdziernika 2015 17:18:32 UTC+2 uÅŒytkownik Rich

Post by RafaÅ Kanus

What about Input device (e.g. USB Microphone). The is no equivalent

Post by RafaÅ Kanus
property for microphone sample rate.

Since my original reply, I noticed AudioManager.getDevices()
<http://developer.android.com/reference/android/media/AudioManager.html#getDevices(int)>,
that returns an AudioDeviceInfo
<http://developer.android.com/reference/android/media/AudioDeviceInfo.html> object.
Maybe this will help setup usb and input devices? I don't have anything
running Android M so can't test myself.

Hi,

Yes, I will use in on Android M, but this API is not available on Android 5
so I have to get that information from USB Descriptor or directly from Alsa
sound card (/proc/asound/card1 )

mike digioia

2015-10-08 20:47:48 UTC

Submit me

Post by RafaÅ Kanus
W dniu poniedziaÅek, 5 paÅºdziernika 2015 17:18:32 UTC+2 uÅŒytkownik Rich

Post by RafaÅ Kanus

What about Input device (e.g. USB Microphone). The is no equivalent

Post by RafaÅ Kanus
property for microphone sample rate.

Since my original reply, I noticed AudioManager.getDevices()
<http://developer.android.com/reference/android/media/AudioManager.html#getDevices(int)>,
that returns an AudioDeviceInfo
<http://developer.android.com/reference/android/media/AudioDeviceInfo.html> object.
Maybe this will help setup usb and input devices? I don't have anything
running Android M so can't test myself.

Hi,
Yes, I will use in on Android M, but this API is not available on Android
5 so I have to get that information from USB Descriptor or directly from
Alsa sound card (/proc/asound/card1 )
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/android-ndk.
For more options, visit https://groups.google.com/d/optout.

Paul Mclean

2015-10-02 14:30:05 UTC

I've been informed that there is a problem with consistently connecting USB
audio to a Samsung device. Can someone tell me which devices are involved?

- Paul

Rafał Kanus

2015-10-05 14:22:45 UTC

Hi,

I've observed problems on Samsung S5, S6 and S6 edge with Android 5+.
We are working on Voip client and use OpenSL ES library. When I set mode to
IN_COMMUNICATION and then open Input and Output device, the sound is not
routed to USB Headphones and USB microphone is alos not use.
Internal Earpiece and Microphone is used. It is like there was no USB
headphones connected. I've checked in a shell what is in /proc/asound/cards
and device in correctly detected as soundcard ( also data in
/proc/asound/card1 are correct).

I've tried to change to mode to NORMAL or IN_CALL. In this configuration
the output is working but input is not recording.
When I use any Music Player app which use only Java API USB Headphones are
working correctly but recording does not.

Rafał Kanus

2015-10-05 14:47:48 UTC