ndkBuild and CMake in one project but dependencies don't fire, libraries not found

[Andi McClure]

Short version: I have constructed a gradle NDK project where the main
project uses ndkBuild and a subproject uses CMake. I want the CMake project
to produce a library the ndkBuild project links, but the dependency to the
CMake project is not being applied, the library is not found by the main
project, and I do not know how to specify to the main project the filename
it’s supposed to be extracting from the CMake project.

Context: I have an existing video game which I am porting to the Oculus
Mobile SDK 1.16.0
https://developer.oculus.com/downloads/package/oculus-mobile-sdk/ . This
SDK contains libraries and sample code, both of which are built using
Gradle and ndkBuild. The SDK’s ndkBuild scripts for the sample projects are
pretty complicated and I do not know enough about how Oculus works to port
them to CMake myself. Meanwhile the project I am porting is based on CMake
and would be *very* difficult or impossible to port to ndkBuild, since it
includes CMake builds of very complex projects such as Assimp and LuaJIT.
For this reason, it is necessary I use both CMake and ndkBuild in this
single project.

Mixing CMake and ndkBuild in one build.gradle turns out to be not allowed (
it produces an error
https://android.googlesource.com/platform/tools/base/+/studio-2.2-preview3/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/ExternalNativeBuildTaskUtils.java#179
). But what I *can* do is create a project and a subproject, and have one
use ndkBuild and the other CMake.

Currently my project is based on the VrCubeWorld_Framework example from the
Oculus Mobile SDK. Its dependencies {} section depends on five projects,
four Oculus libraries (:VrAppFramework:Projects:Android, etc) and one mine
(“implementation project(':cmakelib’)”). :cmakelib is a project at the top
of my repository. Its build.gradle is simple (attached). It includes a
CMakeLists.exe which builds a library named “lovr”, created with CMake
add_library(lovr SHARED ${SOURCES}). I am using command line gradlew to
compile because I could not get Android Studio to create a project without
errors.

Some of what is happening makes sense to me:
1. I can build cmakelib by itself with "gradlew installDebug” and it builds
and uploads an apk that is (unsurprisingly) not a valid Android program,
but if I unzip it, I can see it contains a file in lib/[ARCH] named
liblovrd.so.
2. I can build VrCubeWorld_Framework with “gradlew installDebug” and it
builds and uploads an apk which is a valid Android program I can execute on
my Oculus Go, *and* if I unzip the APK I can see it contains that same
lib/[ARCH]/liblovrd.so.

Some of what is happening does *not* make sense to me, and is very bad:

1. Although builds of VrCubeWorld_Framework are triggering builds of
cmakelib, as if it were a dependency, the builds of cmakelib are not
occurring “before” the build of VrCubeWorld_Framework, as you would expect
for a dependency.

What I mean by this: Say that I introduce a failure into cmakelib, for
example I add “message(FATAL_ERROR test)” to the CMakeLists.txt. When I
installDebug the VRCubeWorld_Framework project, it will fail with the
expected message. But let’s say that I introduce failures to cmakelib and
VRCubeWorld_Framework at once (I put an error in the source file listed in
VRCubeWorld_Framework’s jni/Android.mk). VRCubeWorld_Framework will be
built first, and will print its error, and the build will halt, and
cmakelib will not be attempted. It appears VRCubeWorld_Framework compiles
first and cmakelib after.

This is especially a problem because:

2. VRCubeWorld_Framework, when its jni/Android.mk is being built, cannot
find liblovrd.so, and the symbols in liblovrd.so are not visible to its
source files. There is a symbol bridgeLovrInit(); in liblovrd.so. If I add
“lovr” to LOCAL_SHARED_LIBRARIES in VRCubeWorld_Framework’s jni/Android.mk,
and add a call to bridgeLovrInit() in the VRCubeWorld_Framework’s main cpp
file, then build, I get these errors at the end of the gradlew output:

/Users/mcc/Library/Android/sdk/ndk-bundle/build/core/build-binary.mk:688:
Android NDK: Module vrcubeworld depends on undefined modules: lovr
[arm64-v8a] Compile++ : vrcubeworld <= VrCubeWorld_Framework.cpp
[arm64-v8a] Prebuilt : libvrapi.so <=
/Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/jni/../../../../../VrApi/Projects/AndroidPrebuilt/jni/../../../Libs/Android/arm64-v8a/Debug/
[arm64-v8a] SharedLibrary : libvrcubeworld.so

/Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/build/intermediates/ndkBuild/debug/obj/local/arm64-v8a/objs-debug/vrcubeworld/__/__/__/Src/VrCubeWorld_Framework.o:
In function `OVR::VrCubeWorld::EnteredVrMode(OVR::ovrIntentType, char
const*, char const*, char const*)':

/Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/jni/../../../Src/VrCubeWorld_Framework.cpp:116:
undefined reference to `bridgeLovrInit()'
clang++: error: linker command failed with exit code 1 (use -v to see
invocation)

Notice the warning on the first line and the error on the final line.

Other things which do not work: If I name lovr in the
LOCAL_SHARED_LIBRARIES line to “liblovr”, it fails the same way, with the
“undefined module lovr” error; if I say “liblovrd” it gives me an error of
“undefined module lovrd”. If I construct “lovr” as a static library (I
would actually rather it be static) it can’t find it like that either.

What am I supposed to do here? The other shared libraries included by
Android.mk are also coming out of subprojects, but those are all ndkBuild
based subprojects. It feels like some kind of magic is allowing the
ndkBuild projects to see each other which is not working when the
subproject is cmake. How are projects and subprojects in gradle supposed to
communicate outputs/inputs to each other?

[Andi McClure]

As a minor update, I have found I am able to include my CMake-built code
into the ndkBuild project if I do two things: First, I add this to my
Android.mk ...


include $(CLEAR_VARS)

ifdef NDK_DEBUG

LOVR_LIB_SUFFIX=d

else

LOVR_LIB_SUFFIX=

endif

LOCAL_MODULE := lovr

LOCAL_SRC_FILES :=
../../../../../cmakelib/build/intermediates/cmake/debug/obj/$(TARGET_ARCH_ABI)/liblovr$(LOVR_LIB_SUFFIX).so

include $(PREBUILT_SHARED_LIBRARY)


... and two, I always run `gradlew build` on the cmakelib subproject before
I run `gradlew installDebug` on the main project. The Android.mk change
means that when the code of the main project is built it can link the
symbols in liblovrd.so, because it views the liblovrd.so in the CMake
intermediate build products directory as a prebuilt library; then running
the CMake build first means that the .so is there and has the correct
symbols even though gradle triggers its own dependency of the CMake build
after the ndkBuild build. (There are several .so files that cmakelib
generates that liblovrd.so depends on, but which are not directly
referenced by the main project; for whatever reason, the project manages to
correctly copy these additional .sos into the APK even though I don't cite
them in the Android.mk).


This is all *really scary*. This approach is brittle, sketchy, requires
this two-step build that if I forget to do it in the right order can lead
to strange errors, and depends on the paths and names of temporary files
created by CMake remaining stable in the future. I have doubts I will be
able to make a static library, or have liblovrd.so link back into the main
ndkBuild project, with this solution. I am at least temporarily unblocked
such I can start writing code instead of beating my head against CMake, but
I really hope I won't need to ship a product with this approach. It seems
like all the information to make this work more properly is available to
gradle, it's just somehow not being communicated to ndkBuild that the CMake
subproject is a "module". Is there really no way to do this with the
current system?

Post by Andi McClure
Short version: I have constructed a gradle NDK project where the main
project uses ndkBuild and a subproject uses CMake. I want the CMake project
to produce a library the ndkBuild project links, but the dependency to the
CMake project is not being applied, the library is not found by the main
project, and I do not know how to specify to the main project the filename
it’s supposed to be extracting from the CMake project.
Context: I have an existing video game which I am porting to the Oculus
Mobile SDK 1.16.0
https://developer.oculus.com/downloads/package/oculus-mobile-sdk/ . This
SDK contains libraries and sample code, both of which are built using
Gradle and ndkBuild. The SDK’s ndkBuild scripts for the sample projects are
pretty complicated and I do not know enough about how Oculus works to port
them to CMake myself. Meanwhile the project I am porting is based on CMake
and would be *very* difficult or impossible to port to ndkBuild, since it
includes CMake builds of very complex projects such as Assimp and LuaJIT.
For this reason, it is necessary I use both CMake and ndkBuild in this
single project.
Mixing CMake and ndkBuild in one build.gradle turns out to be not allowed
( it produces an error
https://android.googlesource.com/platform/tools/base/+/studio-2.2-preview3/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/ExternalNativeBuildTaskUtils.java#179
). But what I *can* do is create a project and a subproject, and have one
use ndkBuild and the other CMake.
Currently my project is based on the VrCubeWorld_Framework example from
the Oculus Mobile SDK. Its dependencies {} section depends on five
projects, four Oculus libraries (:VrAppFramework:Projects:Android, etc) and
one mine (“implementation project(':cmakelib’)”). :cmakelib is a project at
the top of my repository. Its build.gradle is simple (attached). It
includes a CMakeLists.exe which builds a library named “lovr”, created with
CMake add_library(lovr SHARED ${SOURCES}). I am using command line gradlew
to compile because I could not get Android Studio to create a project
without errors.
1. I can build cmakelib by itself with "gradlew installDebug” and it
builds and uploads an apk that is (unsurprisingly) not a valid Android
program, but if I unzip it, I can see it contains a file in lib/[ARCH]
named liblovrd.so.
2. I can build VrCubeWorld_Framework with “gradlew installDebug” and it
builds and uploads an apk which is a valid Android program I can execute on
my Oculus Go, *and* if I unzip the APK I can see it contains that same
lib/[ARCH]/liblovrd.so.
1. Although builds of VrCubeWorld_Framework are triggering builds of
cmakelib, as if it were a dependency, the builds of cmakelib are not
occurring “before” the build of VrCubeWorld_Framework, as you would expect
for a dependency.
What I mean by this: Say that I introduce a failure into cmakelib, for
example I add “message(FATAL_ERROR test)” to the CMakeLists.txt. When I
installDebug the VRCubeWorld_Framework project, it will fail with the
expected message. But let’s say that I introduce failures to cmakelib and
VRCubeWorld_Framework at once (I put an error in the source file listed in
VRCubeWorld_Framework’s jni/Android.mk). VRCubeWorld_Framework will be
built first, and will print its error, and the build will halt, and
cmakelib will not be attempted. It appears VRCubeWorld_Framework compiles
first and cmakelib after.
2. VRCubeWorld_Framework, when its jni/Android.mk is being built, cannot
find liblovrd.so, and the symbols in liblovrd.so are not visible to its
source files. There is a symbol bridgeLovrInit(); in liblovrd.so. If I add
“lovr” to LOCAL_SHARED_LIBRARIES in VRCubeWorld_Framework’s jni/Android.mk,
and add a call to bridgeLovrInit() in the VRCubeWorld_Framework’s main cpp
Android NDK: Module vrcubeworld depends on undefined modules: lovr
[arm64-v8a] Compile++ : vrcubeworld <= VrCubeWorld_Framework.cpp
[arm64-v8a] Prebuilt : libvrapi.so <=
/Users/mcc/work/h/ovr_sdk_mobile_1.16.0/VrSamples/VrCubeWorld_Framework/Projects/Android/jni/../../../../../VrApi/Projects/AndroidPrebuilt/jni/../../../Libs/Android/arm64-v8a/Debug/
[arm64-v8a] SharedLibrary : libvrcubeworld.so
In function `OVR::VrCubeWorld::EnteredVrMode(OVR::ovrIntentType, char
undefined reference to `bridgeLovrInit()'
clang++: error: linker command failed with exit code 1 (use -v to see
invocation)
Notice the warning on the first line and the error on the final line.
Other things which do not work: If I name lovr in the
LOCAL_SHARED_LIBRARIES line to “liblovr”, it fails the same way, with the
“undefined module lovr” error; if I say “liblovrd” it gives me an error of
“undefined module lovrd”. If I construct “lovr” as a static library (I
would actually rather it be static) it can’t find it like that either.
What am I supposed to do here? The other shared libraries included by
Android.mk are also coming out of subprojects, but those are all ndkBuild
based subprojects. It feels like some kind of magic is allowing the
ndkBuild projects to see each other which is not working when the
subproject is cmake. How are projects and subprojects in gradle supposed to
communicate outputs/inputs to each other?

[Steven Winston]

This is admittedly an area that Android studio could use a bit of work in. There's two suggestions I have:
Use cmake and create a parent project in cmake to build everything. This would give you externalproject* family of cmake functions that correctly handle things in a sane way.
This has the advantage of having a single point where your build goes between Gradle and the native build system.
Or you could go down the route of using Gradle to invoke an "install" step for your module and place the Libs in the right place. You can have the install step get invoked on build automatically.
The advantage here is the ide feels a bit better and you can take advantage of some module logic that's a bit more difficult to get access to.
I personally recommend the first path as the preview AS has some features that makes this easier.

[Andi McClure]

One update to my previous post— I have figured out my "problem 1", where
the dependencies were not running in the correct order— I reordered the
lines in my build.gradle and this problem fixed itself completely. That was
actually the larger of the two problems, so that's good.

Steven, thank you for the advice-- there's a couple issues.

As for using externalproject in cmake— in this *specific* case I need to
invoke cmake from ndkbuild rather than the other way around. But if I did
use externalproject, how would cmake invoke ndkbuild? Is the idea
externalproject would invoke the gradlew executable?

For the other suggestion— You're right, I *could* get either cmake or
gradle to install the libs, or "install" them to a known local directory,
and that would definitely be less scary than linking into the build
intermediates directory. A weird thing about that solution though is that
the installed-intermediates directory would only be used so the ndkBuild
step could accurately check symbols against the cmake outputs! Gradle is
actually copying the cmake outputs into the apk, it just doesn't seem to be
making them visible to the ndkbuild step. I suppose at least it would work,
though.

Post by Steven Winston
This is admittedly an area that Android studio could use a bit of work in.
Use cmake and create a parent project in cmake to build everything. This
would give you externalproject* family of cmake functions that correctly
handle things in a sane way.
This has the advantage of having a single point where your build goes
between Gradle and the native build system.
Or you could go down the route of using Gradle to invoke an "install" step
for your module and place the Libs in the right place. You can have the
install step get invoked on build automatically.
The advantage here is the ide feels a bit better and you can take
advantage of some module logic that's a bit more difficult to get access to.
I personally recommend the first path as the preview AS has some features
that makes this easier.

[Alex Cohn]

The main drawback of the externalproject approach is that this way Android
Studio will not sniff correctly the ndk-build files for code navigation,
completion, etc. If some library is stable and you don't need to open its
sources in AS, it's actually recomended to move it out of code inspection:
AS does not yet handle big C++ projects yet. We keep our CMake builds for
stable libraries 'hidden' from Gradle, otherwise the IDE becomes unuseably
slow.

If libraries that are in active development (and may be edited and debugged
in AS) require both CMake and ndk-build, there is no alternative to keep
them is separate sub-projects (a.k.a. Modules), but there is no 'build tool
agnostic' way to get the two understand one another, e.g. let ndk-build
pick the location of the built dependency from CMake.

BR,
Alex

Post by Steven Winston
This is admittedly an area that Android studio could use a bit of work in.
Use cmake and create a parent project in cmake to build everything. This
would give you externalproject* family of cmake functions that correctly
handle things in a sane way.
This has the advantage of having a single point where your build goes
between Gradle and the native build system.
Or you could go down the route of using Gradle to invoke an "install" step
for your module and place the Libs in the right place. You can have the
install step get invoked on build automatically.
The advantage here is the ide feels a bit better and you can take
advantage of some module logic that's a bit more difficult to get access to.
I personally recommend the first path as the preview AS has some features
that makes this easier.

[Alex Cohn]

This post might be inappropriate. Click to display it.