What is the correct way to use dlopen()?

[Andreas Falkenhahn]

I have a shared object bundled with my Android app. It is in the "lib" folder of the
APK. On Android 7 I can just do the following to load it:

lib = dlopen("libfoo.so", RTLD_LAZY);

However, this doesn't work on Android 4. On Android 4 I have to do the following instead:

lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);

But opening from /data/data/.../ doesn't work on Android 7. So the solution I've come
up with is something like this:

if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}

This will work on both, Android 4 and Android 7, but of course it feels hackish.
So what is the official way to do it please?

NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.
--
Best regards,
Andreas Falkenhahn mailto:***@falkenhahn.com

[Andreas Falkenhahn]

Helloo-oo? How come nobody knows this? Isn't this one of the most common things you do with the Android NDK? There surely must be someone here who can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib" folder of the
lib = dlopen("libfoo.so", RTLD_LAZY);
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the solution I've come
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels hackish.
So what is the official way to do it please?
NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.
--
Best regards,

--
Best regards,
Andreas Falkenhahn mailto:***@falkenhahn.com

['Dan Albert' via android-ndk]

Post by Andreas Falkenhahn
NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.

You don't really want call dlclose anyway. There are bugs with thread
locals that will cause crashes until P if you dlclose a library:
https://github.com/android-ndk/ndk/issues/360

Or leaks prior to r17: https://github.com/android-ndk/ndk/issues/620

Post by Andreas Falkenhahn
Helloo-oo? How come nobody knows this? Isn't this one of the most common
things you do with the Android NDK? There surely must be someone here who
can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib"

folder of the

Post by Andreas Falkenhahn
lib = dlopen("libfoo.so", RTLD_LAZY);
However, this doesn't work on Android 4. On Android 4 I have to do the
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the

solution I've come

Post by Andreas Falkenhahn
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels

hackish.

Post by Andreas Falkenhahn
So what is the official way to do it please?
NB: I know that it is easier from the Java side using

System.loadLibrary() but I

Post by Andreas Falkenhahn
don't want to use that either because it doesn't allow me to unload

libraries.

Post by Andreas Falkenhahn
--
Best regards,

--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com
.
For more options, visit https://groups.google.com/d/optout.

[J Decker]

https://github.com/d3x0r/SACK/blob/master/src/deadstart/android/client_android_main.c#L462
When I start, I scan /proc/self/maps and find the map which covers the
address of the routine I'm in; this gives me the full path of the current
code. I can then strip off the last part of the name and keep the path
part to build dlopen paths.

You previously had to manually load shared object libraries that others
also depend on as it wouldn't auto-resolve the paths... (libC depends on
libB which depends on libA; you have to load A, B and then C manually;
loading C won't also bring in A and B... )


On Fri, Apr 27, 2018 at 10:28 AM, 'Dan Albert' via android-ndk <

Post by Andreas Falkenhahn
NB: I know that it is easier from the Java side using System.loadLibrary()

Post by Andreas Falkenhahn
but I
don't want to use that either because it doesn't allow me to unload libraries.

You don't really want call dlclose anyway. There are bugs with thread
https://github.com/android-ndk/ndk/issues/360
Or leaks prior to r17: https://github.com/android-ndk/ndk/issues/620

Post by Andreas Falkenhahn
Helloo-oo? How come nobody knows this? Isn't this one of the most common
things you do with the Android NDK? There surely must be someone here who
can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib"

folder of the

Post by Andreas Falkenhahn
lib = dlopen("libfoo.so", RTLD_LAZY);
However, this doesn't work on Android 4. On Android 4 I have to do the
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the

solution I've come

Post by Andreas Falkenhahn
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels

hackish.

Post by Andreas Falkenhahn
So what is the official way to do it please?
NB: I know that it is easier from the Java side using

System.loadLibrary() but I

Post by Andreas Falkenhahn
don't want to use that either because it doesn't allow me to unload

libraries.

Post by Andreas Falkenhahn
--
Best regards,

--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/CAFVaGhvMD3M%3Dw2BKORqVx%2Bkhs0%2BfWFVTAaij-
6qmBAYy9gk4EQ%40mail.gmail.com
<https://groups.google.com/d/msgid/android-ndk/CAFVaGhvMD3M%3Dw2BKORqVx%2Bkhs0%2BfWFVTAaij-6qmBAYy9gk4EQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Andreas Falkenhahn]

I'm not using thread locals or C++ so AFAICS I'm not affected by these issues or is there anything else I should be aware of? My code is really just basic C with pthreads but no thread locals.

Post by 'Dan Albert' via android-ndk
You don't really want call dlclose anyway. There are bugs with
thread locals that will cause crashes until P if you dlclose a
library: https://github.com/android-ndk/ndk/issues/360
Or leaks prior to r17: https://github.com/android-ndk/ndk/issues/620
Helloo-oo? How come nobody knows this? Isn't this one of the most
common things you do with the Android NDK? There surely must be
someone here who can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib" folder of the
lib = dlopen("libfoo.so", RTLD_LAZY);
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the solution I've come
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels hackish.
So what is the official way to do it please?
NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.
--
Best regards,

--
Best regards,
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--
Best regards,
Andreas Falkenhahn mailto:***@falkenhahn.com

[Kenneth Geisshirt]

As I recall, there are some changes in what you can do with dlopen:

https://developer.android.com/about/versions/nougat/android-7.0-changes#ndk

Post by Andreas Falkenhahn
Helloo-oo? How come nobody knows this? Isn't this one of the most common
things you do with the Android NDK? There surely must be someone here who
can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib"

folder of the

Post by Andreas Falkenhahn
lib = dlopen("libfoo.so", RTLD_LAZY);
However, this doesn't work on Android 4. On Android 4 I have to do the
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the

solution I've come

Post by Andreas Falkenhahn
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels

hackish.

Post by Andreas Falkenhahn
So what is the official way to do it please?
NB: I know that it is easier from the Java side using

System.loadLibrary() but I

Post by Andreas Falkenhahn
don't want to use that either because it doesn't allow me to unload

libraries.

Post by Andreas Falkenhahn
--
Best regards,

--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com
.
For more options, visit https://groups.google.com/d/optout.

--
Kenneth Geisshirt, M.Sc., Ph.D.
MajbÞl Allé 18, DK-2770 Kastrup, [phone]

[J Decker]

Post by Kenneth Geisshirt
https://developer.android.com/about/versions/nougat/android-
7.0-changes#ndk

the parts of that - you can't load the system versions; only the local
version; but the OP seems to want to load their own version; so none of
that really matters.

Post by Kenneth Geisshirt

Post by Andreas Falkenhahn
Helloo-oo? How come nobody knows this? Isn't this one of the most common
things you do with the Android NDK? There surely must be someone here who
can answer this!?

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib"

folder of the

Post by Andreas Falkenhahn
lib = dlopen("libfoo.so", RTLD_LAZY);
However, this doesn't work on Android 4. On Android 4 I have to do the
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the

solution I've come

Post by Andreas Falkenhahn
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib =
dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels

hackish.

Post by Andreas Falkenhahn
So what is the official way to do it please?
NB: I know that it is easier from the Java side using

System.loadLibrary() but I

Post by Andreas Falkenhahn
don't want to use that either because it doesn't allow me to unload

libraries.

Post by Andreas Falkenhahn
--
Best regards,

--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/6110171509.20180427150518%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

--
Kenneth Geisshirt, M.Sc., Ph.D.
<https://maps.google.com/?q=Ph.D.+%0D%0AMajb%C3%B8l+All%C3%A9+18,+DK-2770+Kastrup&entry=gmail&source=g>
MajbÞl Allé 18, DK-2770 Kastrup, [phone]
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/CAFLzvupxrR2p9V%3DiLqK3zqGzFNq6bPaGp--R1%
3DBdAsb2AZq38w%40mail.gmail.com
<https://groups.google.com/d/msgid/android-ndk/CAFLzvupxrR2p9V%3DiLqK3zqGzFNq6bPaGp--R1%3DBdAsb2AZq38w%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.

[Andreas Falkenhahn]

Yes. I'm still completely puzzled by the fact that it is apparently so very difficult to dlopen() a shared object stored inside the APK. This is such a basic functionality and am I really expected to scan /proc/self/maps and then apply lots of kludges just to be able to dlopen() a shared object stored inside my APK? Seriously Google?

Post by J Decker
the parts of that - you can't load the system versions; only the
local version; but the OP seems to want to load their own version; so none of that really matters.

--
Best regards,
Andreas Falkenhahn mailto:***@falkenhahn.com

[J Decker]

Post by Andreas Falkenhahn
Yes. I'm still completely puzzled by the fact that it is apparently so
very difficult to dlopen() a shared object stored inside the APK. This is
such a basic functionality and am I really expected to scan /proc/self/maps
and then apply lots of kludges just to be able to dlopen() a shared object
stored inside my APK? Seriously Google?

I entirely concur; The LD_LIBRARY_PATH is read at program startup so if you
modify that, it doesn't apply. If you do a dlopen() and there's a
dependancy missing, you can't then parse the error and load the missing
dependancy and then re-call dlopen() on the first one, because the library
caches that it was an error and returns with the same error, even though
the symbol/library can now be resolved.

Post by Andreas Falkenhahn

Post by J Decker
the parts of that - you can't load the system versions; only the
local version; but the OP seems to want to load their own version; so

none of that really matters.
--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/1310529704.20180427233858%40falkenhahn.com.
For more options, visit https://groups.google.com/d/optout.

[Andreas Falkenhahn]

But maybe the issue has been resolved already because, as I said, starting with Android 7 it is possible to just pass the name of the shared object and it will work, i.e.

lib = dlopen("libfoo.so", RTLD_LAZY);

So maybe the kludge is only needed for older Android versions. Unfortunately, I have no Android 5 and Android 6 test devices here so I don't know about Android 5 and 6. All I can say is that on Android 4 just passing "libfoo.so" definitely doesn't work. It needs a full path on Android 4.

It would really be nice if there was an official word on this. As I said, it looks like it has been resolved already because on Android 7 and up just passing "libfoo.so" works fine.

Maybe I should file a ticket so that this is forwarded to the people who know more about it?

Post by J Decker
I entirely concur; The LD_LIBRARY_PATH is read at program startup
so if you modify that, it doesn't apply. If you do a dlopen() and
there's a dependancy missing, you can't then parse the error and
load the missing dependancy and then re-call dlopen() on the first
one, because the library caches that it was an error and returns
with the same error, even though the symbol/library can now be resolved.

--
Best regards,
Andreas Falkenhahn mailto:***@falkenhahn.com

['Dan Albert' via android-ndk]

+dimitry and enh, who hopefully remember the changes and when they happened
better than I do...

Post by Andreas Falkenhahn
But maybe the issue has been resolved already because, as I said, starting
with Android 7 it is possible to just pass the name of the shared object
and it will work, i.e.
lib = dlopen("libfoo.so", RTLD_LAZY);
So maybe the kludge is only needed for older Android versions.
Unfortunately, I have no Android 5 and Android 6 test devices here so I
don't know about Android 5 and 6. All I can say is that on Android 4 just
passing "libfoo.so" definitely doesn't work. It needs a full path on
Android 4.
It would really be nice if there was an official word on this. As I said,
it looks like it has been resolved already because on Android 7 and up just
passing "libfoo.so" works fine.
Maybe I should file a ticket so that this is forwarded to the people who
know more about it?

Post by J Decker
I entirely concur; The LD_LIBRARY_PATH is read at program startup
so if you modify that, it doesn't apply. If you do a dlopen() and
there's a dependancy missing, you can't then parse the error and
load the missing dependancy and then re-call dlopen() on the first
one, because the library caches that it was an error and returns
with the same error, even though the symbol/library can now be resolved.

--
Best regards,
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/675660551.20180428122739%40falkenhahn.com
.
For more options, visit https://groups.google.com/d/optout.

[rprichard via android-ndk]

Post by Andreas Falkenhahn
But maybe the issue has been resolved already because, as I said, starting
with Android 7 it is possible to just pass the name of the shared object
and it will work, i.e.
lib = dlopen("libfoo.so", RTLD_LAZY);
So maybe the kludge is only needed for older Android versions.
Unfortunately, I have no Android 5 and Android 6 test devices here so I
don't know about Android 5 and 6. All I can say is that on Android 4 just
passing "libfoo.so" definitely doesn't work. It needs a full path on
Android 4.
It would really be nice if there was an official word on this. As I said,
it looks like it has been resolved already because on Android 7 and up just
passing "libfoo.so" works fine.
Maybe I should file a ticket so that this is forwarded to the people who
know more about it?

I suspect this is the same issue as https://issuetracker.google.com/36950617,
which was fixed for JB-MR2 / API 18 / Android 4.3. I think the fix for that
issue added the app's shared library directory to the search path the
dynamic loader uses both for dlopen and for finding DT_NEEDED dependencies.

Fixes:
- https://android-review.googlesource.com/c/platform/libcore/+/48407
- https://android-review.googlesource.com/c/platform/dalvik/+/48409
- https://android-review.googlesource.com/c/platform/bionic/+/48534

FWIW: The last time I was interested in testing a bunch of Android
versions, I used Android Studio to download and run the x86 AVD emulator
images.

Post by Andreas Falkenhahn

Post by J Decker
I entirely concur; The LD_LIBRARY_PATH is read at program startup
so if you modify that, it doesn't apply. If you do a dlopen() and
there's a dependancy missing, you can't then parse the error and
load the missing dependancy and then re-call dlopen() on the first
one, because the library caches that it was an error and returns
with the same error, even though the symbol/library can now be resolved.

I think caching the error might have also been fixed, maybe by the Bionic
change I linked above. Also see https://issuetracker.google.com/36935779.

-Ryan

[b***@reification.io]

If you are packaging the .so directly in libs/ that may be your problem.
I'm doing exactly this loading pattern with a shared library and it works
without issue if you put it in the correct abi sub-directory
e.g. libs/arm64-v8a/libYouLibrary.so

Post by Andreas Falkenhahn
I have a shared object bundled with my Android app. It is in the "lib" folder of the
lib = dlopen("libfoo.so", RTLD_LAZY);
lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
But opening from /data/data/.../ doesn't work on Android 7. So the solution I've come
if(!(lib = dlopen("libfoo.so", RTLD_LAZY))) {
lib = dlopen("/data/data/<my_package_identifier>/lib/libfoo.so", RTLD_LAZY);
}
This will work on both, Android 4 and Android 7, but of course it feels hackish.
So what is the official way to do it please?
NB: I know that it is easier from the Java side using System.loadLibrary() but I
don't want to use that either because it doesn't allow me to unload libraries.
--
Best regards,
<javascript:>