OpenSL: Position of playback head relative to beginning of content is reset during playback

[wb]

Issue: Position of playback head relative to beginning of content is
reset during playback.

- OpenSL ES for Android.
- Android API level 9, for 2.3 devices.
- Multiple Devices: Nexus One, Nexus S, Galaxy S

We are trying to aggressively keep a constant low latency playback of
our audio packets onto an android device. To accomplish this, our
heuristics require that we know how many bytes of data are queued for
playback at any given time inside the audio engine.

We calculate the bytes queued by counting the bytes sent to the audio
engine and subtracting that number from buffer position value returned
by the audio engine. Most audio engine paradigms have a method of
determining how many bytes have been played relative to some reference
point, usually the beginning of the content.

The OpenSL api provides a GetPosition() method inside the player
interface (SLPlayItf) that appears to do what we need it to do.

"Returns the current position of the playback head relative to
the beginning of the content.
The returned value is bounded between 0 and the duration of the
content. In the case where the data source
is a buffer queue, the current position is the cumulative
duration of all buffers since the last buffer queue Clear() method. "

This is great, and it works with our established algorithms until
there is a delay in delivery of the audio data. Any number of events
can cause the Android device to delay the delivery of audio to the
audio engine. When this happens, the buffers associated with the
OpenSL player can become empty. At that point, we have observed that
the position of the cumulative duration of the content is reset to
zero. This effectively destroys all our calculations and makes it
impossible for us to accurately recalibrate so that we can continue.

We believe that this is a bug with the OpenSL implementation for the
Android platform. Our experiences with other platforms and audio
engine APIs tell us that one should not reset the cumulative content
duration position because all the currently queued buffers have been
played.

If the observed behaviour is the correct behaviour, then we need to
figure out a way to accurately determine the number of bytes queued
inside the audio engine. We have tried numerous approaches to figure
out a solution but have failed to come up with a workable solution.

To demonstrate the undesired effect, I hacked a thread into the native-
audio sample program provided by the NDK. The thread loop constantly
tries to keep 4000 to 6000 bytes queued in the audio engine's internal
buffer. On the thread loops 100th iteration, it intentionally stalls
for an extra long period, and then resumes. The log messages sent to
the console shows that when then internal buffer becomes empty, the
content position becomes zero (GetPosition() result).

The following is code snippets taken from the native-audio application
that demonstrates the issue ….

==========================================================================

static unsigned bufferOffset;
static unsigned bytesEnqueued = 0;
static unsigned bytesPlayed = 0;


static pthread_t m_thread;
static pthread_mutex_t m_thread_mutex;
static pthread_cond_t m_thread_condition_var;

static void* ThreadCallback( void* arg )
{
__android_log_print(ANDROID_LOG_DEBUG, "native-audio",
"threadcallback - start" );

int loopCounter = 0;

for( ;; ) {
if( NULL != nextBuffer )
{
SLmillisecond nPositionMs;
SLresult result = (*bqPlayerPlay)->GetPosition( bqPlayerPlay,
&nPositionMs );

bytesPlayed = nPositionMs * ((8000 * 2) / 1000);

unsigned int bytesInQueue = bytesEnqueued - bytesPlayed;

__android_log_print(ANDROID_LOG_DEBUG, "native-audio", "result=
%d bytesPlayed=%d bytesInQueue=%d", result, bytesPlayed,
bytesInQueue );
if( bytesInQueue < 6000 )
{
unsigned int bytesToQueue = 6000 - bytesInQueue;

if( bufferOffset+bytesToQueue > nextSize ) {
bufferOffset = 0;
}
else {
bufferOffset += bytesToQueue;
}

result = (*bqPlayerBufferQueue)->Enqueue(bqPlayerBufferQueue,
nextBuffer, bytesToQueue);
//__android_log_print(ANDROID_LOG_DEBUG, "native-audio", "ennqueue -
result=%d", result );

bytesEnqueued += bytesToQueue;
}
else {
if( bytesPlayed == 0 ) {
nextBuffer = NULL;
}
}
}

loopCounter++;
if( loopCounter > 100 ) {
loopCounter = 0;
__android_log_print(ANDROID_LOG_DEBUG, "native-audio",
"threadcallback - extra long sleep" );
usleep( 5 * 200 * 1000 );
}
else {
usleep( 200 * 1000 );
}
}
__android_log_print(ANDROID_LOG_DEBUG, "native-audio",
"threadcallback - done" );
}

static void CreateThread()
{
pthread_mutex_init(&m_thread_mutex,NULL);
pthread_cond_init(&m_thread_condition_var,NULL);
pthread_create( &m_thread, NULL, &ThreadCallback, NULL );
}


===============================================================================================

Any insight into what we may be doing wrong would helpful, or possibly
an alternate method of accomplishing the task.

wb

['Phil Burk' via android-ndk]

Post by wb
Issue: Position of playback head relative to beginning of content is
reset during playback. [if an underrun occurs]

This is being tracked internally at b/22628197

It seems that AudioTrack::stop() is being called when an underflow occurs.
That causes the position to be reset.