Building Native Android Libraries with the Latest Experimental Android Plugin

GradleAndroidStudio

A major challenge we have faced with NDK development on our Android Video SDK is integrating with the prescribed Android Studio Gradle-based workflow. Our primary issue is debugging native components inside of an Android library. The latest updates to Android Studio and the Android Gradle Plugin indicate that Google is investing in improving the developer experience for native development on Android and ensuring that it has first class functionality in their flagship IDE. We have been closely following the progress of this new plugin. The recent addition of static library support enticed us to deep dive into migrating from the traditional Application.mk, Android.mk, and ndk-build work flow. We took the plunge and decided to share some of our findings migrating to this new build workflow.

Migration Discoveries

The new experimental plugin should look familiar for Gradle users and those familiar with the NDK build process. Most of the usual flags are available within the android.ndk model. Google’s full documentation to the experimental plugin is available here.

We put together a sample migration process that represents some of the challenges we faced that are not mentioned in the current documentation. The sections below provide micro conversion examples to ensure you are able to recreate your .mk files in your build.gradle.

Starting Out

The NDK Samples provide a good introduction into the experimental plugin and how it can be leveraged with native development. The new plugin leverages Gradle’s new model approach. The following snippet shows the base model configuration that will be added to in the ensuing conversions.

DK_TOOLCHAIN_VERSION := 4.9
APP_PLATFORM := android-21
APP_STL := c++_static
APP_ABI := armeabi-v7a arm64-v8a x86 x86_64

Here are the equivalent parameters inserted into the android model. Note that the plugin will not complain if you specify 64 bit architectures and a platform version less than 21 See #201561.

android.ndk {
    // DK_TOOLCHAIN_VERSION
    toolchainVersion "4.9"

    // APP_PLATFORM
    // Note this must be >=21 for 64 bit architectures
    platformVersion 21

    // APP_STL
    stl "c++_static"

    // APP_ABI
    abiFilters.addAll([
        "armeabi-v7a",
        "arm64-v8a",
        "x86",
        "x86_64"
    ])
}

Converting Android.mk

As mentioned, we designed this sample migration process to highlight the following challenges we faced:

1. Using debug builds of our native dependencies
2. Including static libraries
3. Handling dependencies on dependencies (foo.a depends on bar.a)
4. Declaring a whole static library (–whole-archive)

Here is an Android.mk file that presents each of these challenges:

model {
    android.ndk {
         // LOCAL_MODULE
        moduleName "my_module_so"
        toolchainVersion "4.9"
        platformVersion 21
        stl "c++_static"

        // LOCAL_CPPFLAGS
        cppFlags.addAll([
                "-std=gnu++11",
                "-fexceptions"
                "-Wunused-function"
        ])

        // LOCAL_LDLIBS
        ldLibs.addAll([
                "log",
               "GLESv2",
                "android"
        ])
        abiFilters.addAll([
                "armeabi-v7a",
                "arm64-v8a",
                "x86",
                "x86_64"
        ])
    }
}

Working with Static Libraries

Referencing static libraries requires adding libs entries to our repositories model. The snippet below highlights solutions for challenges one and two. Note the absence of my-whole-static-library.

def prebuiltHome = "${buildDir}/prebuilt"

def myStaticLibHome = "${prebuiltHome}/my-static-library"
def myStaticLibHeaders = "${myStaticLibHome}/include"
def myStaticLib = "libmy-static-library.a"

def myOtherStaticLibHome = "${prebuiltHome}/my-other-static-library"
def myOtherStaticLibHeaders = "${myOtherStaticLibHome}/include"
def myOtherStaticLib = "libmy-other-static-library.a"

model {
    repositories {
        libs(PrebuiltLibraries) {
            myStaticLibrary {
                headers.srcDir "${myStaticLibHeaders}"
                binaries.withType(StaticLibraryBinary) {
                    // Solution for Challenge 1
                    // We can reference buildType within StaticLibraryBinary
                    // to determine whether we are building a debug or release
                    // artifact and then pull in the appropriate dependency
                    def myStaticLibPath = "${myStaticLibHome}/lib/${buildType.getName()}/" +
                            "${targetPlatform.getName()}/${myStaticLib}"
                    staticLibraryFile = file("${myStaticLibPath}")
                }
            }
        }
        libs(PrebuiltLibraries) {
            myOtherStaticLibrary {
                headers.srcDir "${myOtherStaticLibHeaders}"
                binaries.withType(StaticLibraryBinary) {
                    def myOtherStaticLibPath = "${myOtherStaticLibHome}/lib/${buildType.getName()}/" +
                            "${targetPlatform.getName()}/${myOtherStaticLib}"
                    staticLibraryFile = file("${myOtherStaticLibPath}")
                }
            }
        }
    }

    android.sources {
        main {
            jni {
                dependencies {
                    // Solution for Challenge 2
                    // We pull in our static library dependencies
                    // declared in our repository model
                    library "myStaticLibrary" linkage "static"
                    library "myOtherStaticLibrary" linkage "static"
                }
            }
        }
    }
}

Uncharted Waters

Up to this point the migration process could probably be performed just by following the NDK samples and sifting through the experimental plugin guide. Let’s revisit the remaining challenges.

Handling dependencies on dependencies (foo.a depends on bar.a)
Declaring a whole static library (–whole-archive)

The solutions required for these challenges are slightly more involved and here is why:

First Assumption
Let’s assume we do not need the whole-static-library and we can just move forward with our other two static libraries, but as mentioned my-other-static-library depends on my-static-library. This means that if we were to try to compile, Gradle would complain about not being able to resolve symbols at link time. We could add whole-static-library to our ldLibs array, but there is no mechanism to specify which architecture.

Second Assumption
Let’s assume challenge three did not exist and we add whole-static-library in the same way we have added the others. We would be able to compile, but would hit method not found exceptions if we were to call native methods from the Java layer because the compiler will strip away functions it deems not being used. We could manually add whole-static-library in the ldFlags wrapped in --whole-archive, but again we have no way to specify which architecture.

The solutions required for challenges three and four required some research into Gradle’s incubating Rule based model configuration and an understanding of the tasks generated by the experimental plugin. If you were to run ./gradlew tasks --all in our sample you would find the following:

linkMy_module_soArmeabi-v7aDebugSharedLibrary - Links shared library 'my_module_so:armeabi-v7a:debug:sharedLibrary'
linkMy_module_soArm64-v8aDebugSharedLibrary - Links shared library 'my_module_so:arm64-v8a:debug:sharedLibrary'
linkMy_module_soX86DebugSharedLibrary - Links shared library 'my_module_so:x86:debug:sharedLibrary'
linkMy_module_soX86_64DebugSharedLibrary - Links shared library 'my_module_so:x86_64:debug:sharedLibrary'

linkMy_module_soArmeabi-v7aReleaseSharedLibrary - Links shared library 'my_module_so:armeabi-v7a:release:sharedLibrary'
linkMy_module_soArm64-v8aReleaseSharedLibrary - Links shared library 'my_module_so:arm64-v8a:release:sharedLibrary'
linkMy_module_soX86ReleaseSharedLibrary - Links shared library 'my_module_so:x86:release:sharedLibrary'
linkMy_module_soX86_64ReleaseSharedLibrary - Links shared library 'my_module_so:x86_64:release:sharedLibrary'

These tasks are dedicated to linking, and if we modify the arguments passed to the linker we can solve challenges three and four: handling dependencies on dependencies and declaring a whole static library. By creating a RuleSource, we can apply mutations to models within the Gradle configuration. Here is the snippet that presents how to modify each of these link tasks. This snippet can be added to the bottom of build.gradle:

class SampleMigrationRuleSource extends RuleSource {
    static final def projectDir = new File("your-gradle-module").absolutePath
    static final def prebuiltHome = "${projectDir}/build/prebuilt"
    static final def myStaticLib = "libmy-static-library.a"
    static final def myWholeStaticLib = "libmy-whole-static-library.a"

    @Mutate
    void injectArmeabiV7aDebugLinkerFlags(
            @Path('tasks.linkMy_module_soArmeabi-v7aDebugSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'armeabi-v7a', 'debug')
    }

    @Mutate
    void injectArmeabiV7aReleaseLinkerFlags(
            @Path('tasks.linkMy_module_soArmeabi-v7aReleaseSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'armeabi-v7a', 'release')
    }

    @Mutate
    void injectArm64v8aDebugLinkerFlags(
            @Path('tasks.linkMy_module_soArm64-v8aDebugSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'arm64-v8a', 'debug')
    }

    @Mutate
    void injectArm64v8aReleaseLinkerFlags(
            @Path('tasks.linkMy_module_soArm64-v8aReleaseSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'arm64-v8a', 'release')
    }

    @Mutate
    void injectX86DebugLinkerFlags(
            @Path('tasks.linkMy_module_soX86DebugSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'x86', 'debug')
    }

    @Mutate
    void injectX86ReleaseLinkerFlags(
            @Path('tasks.linkMy_module_soX86ReleaseSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'x86', 'release')
    }

    @Mutate
    void injectX86_64DebugLinkerFlags(
            @Path('tasks.linkMy_module_soX86_64DebugSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'x86_64', 'debug')
    }

    @Mutate
    void injectX86_64ReleaseLinkerFlags(
            @Path('tasks.linkMy_module_soX86_64ReleaseSharedLibrary')
                    Task linkTask) {
        injectLinkerFlags(linkTask, 'x86_64', 'release')
    }

    private void injectLinkerFlags(linkTask, arch, buildType) {
        def myStaticLibHome = "${prebuiltHome}/my-static-library/lib/${buildType}"
        def myWholeStaticLibHome = "${prebuiltHome}/my-whole-static-library/lib/${buildType}"

        // Before we actually perform the link task let us add our
        // solutions for challenge three and four
        linkTask.doFirst {
            // We are pretty clueless on this one but it is needed
            if (arch.equals('arm64-v8a')) {
                properties["linkerArgs"].add("-fuse-ld=gold")
            }

            properties["linkerArgs"].addAll([
                    // Solution for Challenge 3
                    "-l${myStaticLibHome}/${arch}/${myStaticLib}".toString(),

                    // Solution for Challenge 4
                    "-Wl,--whole-archive,-l${myWholeStaticLibHome}/${arch}/${myWholeStaticLib},"
                            .toString() + "--no-whole-archive"
            ])
        }
    }
}
apply plugin: SampleMigrationRuleSource

Bonus – Creating Javadoc Task

There are a handful of posts about creating Javadocs for Android library projects, but they are not applicable to this new experimental plugin. Here is a simple snippet to add a Javadoc creation task to your project:

model {
    tasks {
        createJavadocs(Javadoc) {
            def androidConfig = $.android
            def androidJar = "${getSdkDir()}/platforms/${androidConfig.compileSdkVersion}/android.jar"

            source = androidConfig.sources.main.java.source
            classpath += project.files(androidJar);
            options.links("http://docs.oracle.com/javase/7/docs/api/");
            options.linksOffline("http://d.android.com/reference", "${getSdkDir()}/docs/reference");
            exclude '**/BuildConfig.java'
            exclude '**/R.java'
            failOnError false
        }
    }
}

Retrospective

Although slightly more involved than we thought, we have found the migration to the experimental plugin much more conducive to native development on Android. The added benefits of code completion, hybrid debug mode, and Gradle’s flexibility have brought about much needed efficiency improvements to our Android SDK development. Although the plugin is marked as experimental, the stability is impressive and the integration with Android Studio 2.0 is seamless. Unfortunately, the plugin documentation is a little sparse and the ramp up can be daunting, but overall the benefits are well worth the effort.

Follow Aaron Alaniz on Twitter here @webheadz3.

Authors

Aaron Alaniz