Discussion:
How to read file from res/raw folder
idea
2018-03-14 02:21:29 UTC
Permalink
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/00f1f5c2-3675-4bb9-8671-2ed541005f80%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-15 19:24:07 UTC
Permalink
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a ZIP
file under the hood.

BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Dan Albert' via android-ndk
2018-03-15 20:50:34 UTC
Permalink
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
it appropriately:
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a ZIP
file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhvjzjbhepagdmrjFVXrq9PVdgr-Ct8-yws-p1ZuMRXQjQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-16 16:19:07 UTC
Permalink
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.

There is very little added value in having native access to resources, and
many tricky questions to resolve while implementing it.

BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a ZIP
file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
<javascript:>.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/b32ab556-c1ad-4dda-8b79-f3d020808e3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Dan Albert' via android-ndk
2018-03-16 17:44:07 UTC
Permalink
Fair enough. It's certainly easier to not add the API :)
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources, and
many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/b32ab556-c1ad-4dda-8b79-f3d020808e3c%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b32ab556-c1ad-4dda-8b79-f3d020808e3c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhttb8_iZOXTT8dd14%3DrwrWi5Xt8vj5_H7kpJ4OnWps%3DdQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-16 17:53:39 UTC
Permalink
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.

The reasoning I'd give is two fold:
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in their
arsenal of libraries a tool that handles APK file access. It's common to
everyone and a concern of everyone. Thus it stands to reason that new
comers are likely to not have these tools/libraries and will wind up
reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly varying
levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to get
the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.

A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.

Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources, and
many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/196877bb-05fe-4c8f-b757-9b1d2f33c179%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
'Dan Albert' via android-ndk
2018-03-17 19:02:58 UTC
Permalink
+some folks that have more useful opinions on this than I do
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly varying
levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to get
the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources,
and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/196877bb-05fe-4c8f-b757-9b1d2f33c179%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/196877bb-05fe-4c8f-b757-9b1d2f33c179%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFVaGhv5%2Be1HTamL18pt%2BiqTWASS_%2BLJ3%2B22y9FZ3HW4owBfgA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-21 10:37:01 UTC
Permalink
@Steven,

Could you point to some sample code that works decently? Or maybe Google
can help you opensource your toolset?

OTOH, I would be glad to learn in what use cases your native code needs
access specifically to /res and not to /assets.

Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly varying
levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to get
the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources,
and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/c60af6df-7b44-4e45-bd69-d2a1e70f36d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-21 18:13:10 UTC
Permalink
@Alex,

While I can't offer up my companies' code (not a particular hurdle I'd want
to jump through for this), it's also not exactly earth shattering. Zlib as
provided by cdep and caching of files as they are extracted. If you need
such a tool open sourced, I can certainly write a new tool and provide
that. There's a ton of samples that aren't always the highest quality of
ideas. https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
results in many SO questions that point people to using some form of:
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.

For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity, Unreal
or other common game engine here would select the first option, unzip the
apk. These are quite competent devs that understand caching so there's no
real problem there that a new API would help. That said, they are the
people I'd most expect to have a valid reason to select the first path and
already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are where you
want to place NDK resources and use AAssetManager. However, I'd propose a
relatively new use case where if my understanding is correct, the only
available answer facing a new dev would be treat the apk as a zip.

Imagine, having a project that is targeting a VR/AR app that you wish to
have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling 2d
user interaction (system messages, help screens, or options menu) that must
display classic Android GUI elements for. Now imagine those GUI elements
include something that would be useful to both GUI resource sets (logos
come to mind). The natural solution for this use case would be to inform
the dev to duplicate the resource in both Assets and Res and stay with
AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.

Again, this isn't a major requirement as using zlib or other zip library is
quite easy, most devs are used to it from other platforms, also even if I
weren't in a similar use-case requirement for the above scenario I'd still
opt for using zlib to better support handling resources in a common method
between Android/iOS/PS4 etc. Hopefully this provides an idea for when it
could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe Google
can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code needs
access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly varying
levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to get
the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources,
and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/c1eee821-dd1f-4a7a-b203-3fe43766cfec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-22 07:55:24 UTC
Permalink
Thank you for this reply.

This is not very convincing, I must confess.

When we speak about cross platform, no new API to access resources will
really help (unless Android finds a way to bring iOS and Unity to the same
ground). An independent 3rd party library could be better positioned in
this case.

As for sharing some resources with the "classic Android GUI", we must keep
in mind that the API will need all the logic of screen size / orientation /
locale / whatnot that is transparent when using Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.

I agree that the existing Assets API which requires a Java object for Asset
Manager is not very convenient for native developers.

BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle I'd
want to jump through for this), it's also not exactly earth shattering.
Zlib as provided by cdep and caching of files as they are extracted. If
you need such a tool open sourced, I can certainly write a new tool and
provide that. There's a ton of samples that aren't always the highest
quality of ideas.
https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity, Unreal
or other common game engine here would select the first option, unzip the
apk. These are quite competent devs that understand caching so there's no
real problem there that a new API would help. That said, they are the
people I'd most expect to have a valid reason to select the first path and
already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are where
you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish to
have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling 2d
user interaction (system messages, help screens, or options menu) that must
display classic Android GUI elements for. Now imagine those GUI elements
include something that would be useful to both GUI resource sets (logos
come to mind). The natural solution for this use case would be to inform
the dev to duplicate the resource in both Assets and Res and stay with
AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip library
is quite easy, most devs are used to it from other platforms, also even if
I weren't in a similar use-case requirement for the above scenario I'd
still opt for using zlib to better support handling resources in a common
method between Android/iOS/PS4 etc. Hopefully this provides an idea for
when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe Google
can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code needs
access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to get
the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets and
resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources,
and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is a
ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-22 17:53:01 UTC
Permalink
@Alex,

Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.

That said, the points you make a valid. No Google provided API would be
considered cross platform, and naturally there's an xkcd to cover this
situation: Loading Image...

However, I would mention that perhaps we're a bit quick to say there isn't
a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk space on
app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.

Moving on to the Resources access for shared GUI concerns. I'd wager that
when sharing resources one wouldn't need to have concern about which screen
size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.

Assuming my understanding is correct, the time an end user would get into a
use case requirement of needing their own GUI is when the frustum are not
the same as the full screen one provided by Android's GUI. This would be
the case of VR or AR and some games. Thus, if you have a custom projection
matrix, you likely have need of one version of a resource and wouldn't care
what the Java mapping is for a resource that wouldn't match your viewport
in the first place; after all, if you did care, you likely would be in a
scenario where Android's GUI is a better path forward.

So to clarify the use case here is Android GUI can't fit the needs due to
not using the custom viewport (which by the way you guys provide a means to
render the Android GUI to a GLTexture and I'll sing the highest of praises
for Android's GUI). However, there are some resources that can be used
sans changes between GUIs (logos and other sharable disk guzzling assets
that are high res etc). So my point in all this is it wouldn't likely
matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.

What would ultimately matter is generic access to the file in the first
place as the reasoning behind the correlation is lost. It would have to be
incumbent upon the developer to determine the correct mapping within the
resources directory of reusable resources as that developer is the only one
capable of knowing the frustum they will be using.

If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.

In this case what I'd propose is packaging zlib along with an easy to use
interface (API) to return a file for a given path within the APK. I'm not
sure there's a need to be more clever than that for the shared GUI problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources will
really help (unless Android finds a way to bring iOS and Unity to the same
ground). An independent 3rd party library could be better positioned in
this case.
As for sharing some resources with the "classic Android GUI", we must keep
in mind that the API will need all the logic of screen size / orientation /
locale / whatnot that is transparent when using Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle I'd
want to jump through for this), it's also not exactly earth shattering.
Zlib as provided by cdep and caching of files as they are extracted. If
you need such a tool open sourced, I can certainly write a new tool and
provide that. There's a ton of samples that aren't always the highest
quality of ideas. https://www.google.com/search?ei=9piyWoyAA4fUjwP57Kj
oCA&q=ndk+read+files+in+resources+of+apk results in many SO questions
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity, Unreal
or other common game engine here would select the first option, unzip the
apk. These are quite competent devs that understand caching so there's no
real problem there that a new API would help. That said, they are the
people I'd most expect to have a valid reason to select the first path and
already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are where
you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish to
have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling 2d
user interaction (system messages, help screens, or options menu) that must
display classic Android GUI elements for. Now imagine those GUI elements
include something that would be useful to both GUI resource sets (logos
come to mind). The natural solution for this use case would be to inform
the dev to duplicate the resource in both Assets and Res and stay with
AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip library
is quite easy, most devs are used to it from other platforms, also even if
I weren't in a similar use-case requirement for the above scenario I'd
still opt for using zlib to better support handling resources in a common
method between Android/iOS/PS4 etc. Hopefully this provides an idea for
when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe Google
can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code needs
access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to
get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets
and resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to resources,
and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is
a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-
438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFcKJqjhCrLv-HJQX7AFO4SeahYn4iPfy28AMGzMQdtiGfVutw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-25 10:29:08 UTC
Permalink
@Steven,

My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to *assets/background.png*)
to reduce duplication for images and sounds that must be used both through
the Resources API (easier in Java) and through Assets API (easier in C)?
This could be an extension of the existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in Java, *getDrawable(int)
*is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.

As for the zipped nature of APK, it's under your control. You can force
your asset to be stored uncompressed in the APK, e.g. by faking its
filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will provide the
unpacked (cached) copy when you ask for it – both in C and in Java.

For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.

Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would be
considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there isn't
a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk space
on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager that
when sharing resources one wouldn't need to have concern about which screen
size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get into
a use case requirement of needing their own GUI is when the frustum are not
the same as the full screen one provided by Android's GUI. This would be
the case of VR or AR and some games. Thus, if you have a custom projection
matrix, you likely have need of one version of a resource and wouldn't care
what the Java mapping is for a resource that wouldn't match your viewport
in the first place; after all, if you did care, you likely would be in a
scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs due to
not using the custom viewport (which by the way you guys provide a means to
render the Android GUI to a GLTexture and I'll sing the highest of praises
for Android's GUI). However, there are some resources that can be used
sans changes between GUIs (logos and other sharable disk guzzling assets
that are high res etc). So my point in all this is it wouldn't likely
matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the first
place as the reasoning behind the correlation is lost. It would have to be
incumbent upon the developer to determine the correct mapping within the
resources directory of reusable resources as that developer is the only one
capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to use
interface (API) to return a file for a given path within the APK. I'm not
sure there's a need to be more clever than that for the shared GUI problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources will
really help (unless Android finds a way to bring iOS and Unity to the same
ground). An independent 3rd party library could be better positioned in
this case.
As for sharing some resources with the "classic Android GUI", we must
keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle I'd
want to jump through for this), it's also not exactly earth shattering.
Zlib as provided by cdep and caching of files as they are extracted. If
you need such a tool open sourced, I can certainly write a new tool and
provide that. There's a ton of samples that aren't always the highest
quality of ideas.
https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are where
you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish to
have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling 2d
user interaction (system messages, help screens, or options menu) that must
display classic Android GUI elements for. Now imagine those GUI elements
include something that would be useful to both GUI resource sets (logos
come to mind). The natural solution for this use case would be to inform
the dev to duplicate the resource in both Assets and Res and stay with
AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip library
is quite easy, most devs are used to it from other platforms, also even if
I weren't in a similar use-case requirement for the above scenario I'd
still opt for using zlib to better support handling resources in a common
method between Android/iOS/PS4 etc. Hopefully this provides an idea for
when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code needs
access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to
get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets
and resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a bug
component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it is
a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
<javascript:>.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-26 07:10:32 UTC
Permalink
@alex

Thanks for this; I learned quite a bit of android lore and it made me think
about a system I've long since taken for granted. If nothing else comes of
our discussion I view it as useful to me for no other reason than to have
thought through it.

I like your thought of extending alias resources. Admittedly, I didn't
know that resources could be aliased to be reproduced in that manner. Very
nifty. I did know that GetDrawable was deprecated. Lint has been very
helpful in pointing deprecation out but glad to know the why; makes sense.
It would seem that you've found a way to tackle this from the Java side
instead of the NDK side. I think it's pretty cool to be able to utilize
assets files as though they were a res however, I worry that it might blur
the designed lines of what should be in assets and what should be in res.
I guess shared UIs could be created in Java so would make sense to solve
for both places and this is the path of least resistance.

For the compressed nature. I've seen that solution before. I was under
the impression that this is a "packaging option." Meaning that it is never
compressed and thus the apk is larger. If our goal is to have the smallest
sized apk (instant apps), then compression is good; and likely what we want
to keep. Also when thinking of networking; I might be one of the few devs
at the office that worries about end user's data, but why make them
download a larger apk when they don't need to? What would be really cool
is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt, gradle
has a very robust dsl available that is easier than the eclipse method Alex
was kind enough to link to:
https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds that
must be used both through the Resources API (easier in Java) and through
Assets API (easier in C)? This could be an extension of the existing alias
resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in Java, *getDrawable(int)
*is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can force
your asset to be stored uncompressed in the APK, e.g. by faking its
filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will provide
the unpacked (cached) copy when you ask for it – both in C and in Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would be
considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk space
on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager
that when sharing resources one wouldn't need to have concern about which
screen size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get into
a use case requirement of needing their own GUI is when the frustum are not
the same as the full screen one provided by Android's GUI. This would be
the case of VR or AR and some games. Thus, if you have a custom projection
matrix, you likely have need of one version of a resource and wouldn't care
what the Java mapping is for a resource that wouldn't match your viewport
in the first place; after all, if you did care, you likely would be in a
scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs due to
not using the custom viewport (which by the way you guys provide a means to
render the Android GUI to a GLTexture and I'll sing the highest of praises
for Android's GUI). However, there are some resources that can be used
sans changes between GUIs (logos and other sharable disk guzzling assets
that are high res etc). So my point in all this is it wouldn't likely
matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the first
place as the reasoning behind the correlation is lost. It would have to be
incumbent upon the developer to determine the correct mapping within the
resources directory of reusable resources as that developer is the only one
capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to use
interface (API) to return a file for a given path within the APK. I'm not
sure there's a need to be more clever than that for the shared GUI problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources will
really help (unless Android finds a way to bring iOS and Unity to the same
ground). An independent 3rd party library could be better positioned in
this case.
As for sharing some resources with the "classic Android GUI", we must
keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle I'd
want to jump through for this), it's also not exactly earth shattering.
Zlib as provided by cdep and caching of files as they are extracted. If
you need such a tool open sourced, I can certainly write a new tool and
provide that. There's a ton of samples that aren't always the highest
quality of ideas. https://www.google.com/search?ei=9piyWoyAA4fUjwP57Kj
oCA&q=ndk+read+files+in+resources+of+apk results in many SO questions
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are where
you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish
to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling
2d user interaction (system messages, help screens, or options menu) that
must display classic Android GUI elements for. Now imagine those GUI
elements include something that would be useful to both GUI resource sets
(logos come to mind). The natural solution for this use case would be to
inform the dev to duplicate the resource in both Assets and Res and stay
with AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has in
their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to
get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into the
category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets
and resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support for
providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a
bug component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it
is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the Google
Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-
438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFcKJqhRCYy%2B6XxRwNKaZHjcc4ZJErgb%3Dn7fudoQkS62fvxbPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-26 17:28:05 UTC
Permalink
There is a cute fact about the APK size is that Google Play will perform
another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.

Cheers,
Alex Cohn

PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I didn't
know that resources could be aliased to be reproduced in that manner. Very
nifty. I did know that GetDrawable was deprecated. Lint has been very
helpful in pointing deprecation out but glad to know the why; makes sense.
It would seem that you've found a way to tackle this from the Java side
instead of the NDK side. I think it's pretty cool to be able to utilize
assets files as though they were a res however, I worry that it might blur
the designed lines of what should be in assets and what should be in res.
I guess shared UIs could be created in Java so would make sense to solve
for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was under
the impression that this is a "packaging option." Meaning that it is never
compressed and thus the apk is larger. If our goal is to have the smallest
sized apk (instant apps), then compression is good; and likely what we want
to keep. Also when thinking of networking; I might be one of the few devs
at the office that worries about end user's data, but why make them
download a larger apk when they don't need to? What would be really cool
is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt, gradle
has a very robust dsl available that is easier than the eclipse method Alex
https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds
that must be used both through the Resources API (easier in Java) and
through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in Java, *getDrawable(int)
*is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can force
your asset to be stored uncompressed in the APK, e.g. by faking its
filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will provide
the unpacked (cached) copy when you ask for it – both in C and in Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would be
considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk space
on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager
that when sharing resources one wouldn't need to have concern about which
screen size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get
into a use case requirement of needing their own GUI is when the frustum
are not the same as the full screen one provided by Android's GUI. This
would be the case of VR or AR and some games. Thus, if you have a custom
projection matrix, you likely have need of one version of a resource and
wouldn't care what the Java mapping is for a resource that wouldn't match
your viewport in the first place; after all, if you did care, you likely
would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs due
to not using the custom viewport (which by the way you guys provide a means
to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the first
place as the reasoning behind the correlation is lost. It would have to be
incumbent upon the developer to determine the correct mapping within the
resources directory of reusable resources as that developer is the only one
capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to
use interface (API) to return a file for a given path within the APK. I'm
not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources will
really help (unless Android finds a way to bring iOS and Unity to the same
ground). An independent 3rd party library could be better positioned in
this case.
As for sharing some resources with the "classic Android GUI", we must
keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle I'd
want to jump through for this), it's also not exactly earth shattering.
Zlib as provided by cdep and caching of files as they are extracted. If
you need such a tool open sourced, I can certainly write a new tool and
provide that. There's a ton of samples that aren't always the highest
quality of ideas.
https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish
to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling
2d user interaction (system messages, help screens, or options menu) that
must display classic Android GUI elements for. Now imagine those GUI
elements include something that would be useful to both GUI resource sets
(logos come to mind). The natural solution for this use case would be to
inform the dev to duplicate the resource in both Assets and Res and stay
with AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has
in their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard to
get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is quite
helpful; but underused by several companies / developers. If anything I'd
love to get a folder to place resources that gets auto expanded to a common
location on app install; for convenience if nothing else.
Please note, this is not a concern for me directly as I fall into
the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between assets
and resources exists because the former can be accessed as read-only files,
while the latter are used via Android SDK with built-in support
for providing alternatives for different languages, OS versions, screen
orientations, etc <https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a
bug component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it
is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it,
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
<javascript:>.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-26 17:56:59 UTC
Permalink
I'm speechless. If my understanding of what you just said is correct; why
is the apk a zip file at all? The only place there seems to be a benefit,
is in disk space. Disk space is much cheaper than cell phone data. Even
with that consideration; given the fact that data in the apk should be
viewed as "read only" why not simply use a partition that's compressed
(i.e. brtfs or similar?). I'm not sure I see a benefit for apk being a zip
file at all now.

By the way, that's a really awesome feature; I'm going to have to start
using it in projects. You just made the aapt tool much more intriguing.
Post by Alex Cohn
There is a cute fact about the APK size is that Google Play will perform
another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.
Cheers,
Alex Cohn
PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I didn't
know that resources could be aliased to be reproduced in that manner. Very
nifty. I did know that GetDrawable was deprecated. Lint has been very
helpful in pointing deprecation out but glad to know the why; makes sense.
It would seem that you've found a way to tackle this from the Java side
instead of the NDK side. I think it's pretty cool to be able to utilize
assets files as though they were a res however, I worry that it might blur
the designed lines of what should be in assets and what should be in res.
I guess shared UIs could be created in Java so would make sense to solve
for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was under
the impression that this is a "packaging option." Meaning that it is never
compressed and thus the apk is larger. If our goal is to have the smallest
sized apk (instant apps), then compression is good; and likely what we want
to keep. Also when thinking of networking; I might be one of the few devs
at the office that worries about end user's data, but why make them
download a larger apk when they don't need to? What would be really cool
is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt,
gradle has a very robust dsl available that is easier than the eclipse
https://google.github.io/android-gradle-dsl/current/com.
android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds
that must be used both through the Resources API (easier in Java) and
through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in Java, *getDrawable(int)
*is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can force
your asset to be stored uncompressed in the APK, e.g. by faking its
filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will provide
the unpacked (cached) copy when you ask for it – both in C and in Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would
be considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk
space on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager
that when sharing resources one wouldn't need to have concern about which
screen size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get
into a use case requirement of needing their own GUI is when the frustum
are not the same as the full screen one provided by Android's GUI. This
would be the case of VR or AR and some games. Thus, if you have a custom
projection matrix, you likely have need of one version of a resource and
wouldn't care what the Java mapping is for a resource that wouldn't match
your viewport in the first place; after all, if you did care, you likely
would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs due
to not using the custom viewport (which by the way you guys provide a means
to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the first
place as the reasoning behind the correlation is lost. It would have to be
incumbent upon the developer to determine the correct mapping within the
resources directory of reusable resources as that developer is the only one
capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to
use interface (API) to return a file for a given path within the APK. I'm
not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources
will really help (unless Android finds a way to bring iOS and Unity to the
same ground). An independent 3rd party library could be better positioned
in this case.
As for sharing some resources with the "classic Android GUI", we must
keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle
I'd want to jump through for this), it's also not exactly earth
shattering. Zlib as provided by cdep and caching of files as they are
extracted. If you need such a tool open sourced, I can certainly write a
new tool and provide that. There's a ton of samples that aren't always the
highest quality of ideas. https://www.google.com
/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+
resources+of+apk results in many SO questions that point people to
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe the
understanding is incorrect and the developer should place the files within
Assets successfully ignoring the res directory when in the ndk. For these
use cases selecting AAssetManager is a quite competent solution; however,
it fails on one key area, cross platform and requires maintaining the java
object in code that doesn't really need it (I've seen some less than
optimal solutions caused by accessing the java object from other threads
etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you wish
to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of handling
2d user interaction (system messages, help screens, or options menu) that
must display classic Android GUI elements for. Now imagine those GUI
elements include something that would be useful to both GUI resource sets
(logos come to mind). The natural solution for this use case would be to
inform the dev to duplicate the resource in both Assets and Res and stay
with AAssetManager. That works until someone cares about apk size, someone
might say "hey how about we do an instant app." Now we have to care about
the size of the apk a whole lot more. If my understanding is correct, this
set of circumstances will transition a dev from being fully able to
successfully only use AAssetManager. They'd now be in the corner case of
using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has
in their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard
to get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is
quite helpful; but underused by several companies / developers. If
anything I'd love to get a folder to place resources that gets auto
expanded to a common location on app install; for convenience if nothing
else.
Please note, this is not a concern for me directly as I fall into
the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between
assets and resources exists because the former can be accessed as read-only
files, while the latter are used via Android SDK with built-in
support for providing alternatives for different languages, OS versions,
screen orientations, etc
<https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a
bug component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that it
is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-
438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-
4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFcKJqh_di_%2BZD0cfjG27acRedP5ub293ELxBi-QsALGRJ_Q8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-27 16:14:30 UTC
Permalink
There are three reasons why APK is and will be a ZIP. First, backwards
compatibility and tools that depend on this implementation detail. This
format is so widespread in the Java world: JAR, AAR. Second, there still
exist scenarios where the APK is served not through Play Store, e.g. during
debugging. ZIP compression may be handy in these cases. Third, there are no
dramatic disadvantages.

Cheers,
Alex
Post by Steven Winston
I'm speechless. If my understanding of what you just said is correct; why
is the apk a zip file at all? The only place there seems to be a benefit,
is in disk space. Disk space is much cheaper than cell phone data. Even
with that consideration; given the fact that data in the apk should be
viewed as "read only" why not simply use a partition that's compressed
(i.e. brtfs or similar?). I'm not sure I see a benefit for apk being a zip
file at all now.
By the way, that's a really awesome feature; I'm going to have to start
using it in projects. You just made the aapt tool much more intriguing.
Post by Alex Cohn
There is a cute fact about the APK size is that Google Play will perform
another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.
Cheers,
Alex Cohn
PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I didn't
know that resources could be aliased to be reproduced in that manner. Very
nifty. I did know that GetDrawable was deprecated. Lint has been very
helpful in pointing deprecation out but glad to know the why; makes sense.
It would seem that you've found a way to tackle this from the Java side
instead of the NDK side. I think it's pretty cool to be able to utilize
assets files as though they were a res however, I worry that it might blur
the designed lines of what should be in assets and what should be in res.
I guess shared UIs could be created in Java so would make sense to solve
for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was under
the impression that this is a "packaging option." Meaning that it is never
compressed and thus the apk is larger. If our goal is to have the smallest
sized apk (instant apps), then compression is good; and likely what we want
to keep. Also when thinking of networking; I might be one of the few devs
at the office that worries about end user's data, but why make them
download a larger apk when they don't need to? What would be really cool
is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt,
gradle has a very robust dsl available that is easier than the eclipse
https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds
that must be used both through the Resources API (easier in Java) and
through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in Java, *getDrawable(int)
*is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can force
your asset to be stored uncompressed in the APK, e.g. by faking its
filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will provide
the unpacked (cached) copy when you ask for it – both in C and in Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would
be considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk
space on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager
that when sharing resources one wouldn't need to have concern about which
screen size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get
into a use case requirement of needing their own GUI is when the frustum
are not the same as the full screen one provided by Android's GUI. This
would be the case of VR or AR and some games. Thus, if you have a custom
projection matrix, you likely have need of one version of a resource and
wouldn't care what the Java mapping is for a resource that wouldn't match
your viewport in the first place; after all, if you did care, you likely
would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs due
to not using the custom viewport (which by the way you guys provide a means
to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the
first place as the reasoning behind the correlation is lost. It would have
to be incumbent upon the developer to determine the correct mapping within
the resources directory of reusable resources as that developer is the only
one capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to
use interface (API) to return a file for a given path within the APK. I'm
not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources
will really help (unless Android finds a way to bring iOS and Unity to the
same ground). An independent 3rd party library could be better positioned
in this case.
As for sharing some resources with the "classic Android GUI", we must
keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object for
Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle
I'd want to jump through for this), it's also not exactly earth
shattering. Zlib as provided by cdep and caching of files as they are
extracted. If you need such a tool open sourced, I can certainly write a
new tool and provide that. There's a ton of samples that aren't always the
highest quality of ideas.
https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe
the understanding is incorrect and the developer should place the files
within Assets successfully ignoring the res directory when in the ndk. For
these use cases selecting AAssetManager is a quite competent solution;
however, it fails on one key area, cross platform and requires maintaining
the java object in code that doesn't really need it (I've seen some less
than optimal solutions caused by accessing the java object from other
threads etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you
wish to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of
handling 2d user interaction (system messages, help screens, or options
menu) that must display classic Android GUI elements for. Now imagine
those GUI elements include something that would be useful to both GUI
resource sets (logos come to mind). The natural solution for this use case
would be to inform the dev to duplicate the resource in both Assets and Res
and stay with AAssetManager. That works until someone cares about apk
size, someone might say "hey how about we do an instant app." Now we have
to care about the size of the apk a whole lot more. If my understanding is
correct, this set of circumstances will transition a dev from being fully
able to successfully only use AAssetManager. They'd now be in the corner
case of using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time has
in their arsenal of libraries a tool that handles APK file access. It's
common to everyone and a concern of everyone. Thus it stands to reason
that new comers are likely to not have these tools/libraries and will wind
up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard
to get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is
quite helpful; but underused by several companies / developers. If
anything I'd love to get a folder to place resources that gets auto
expanded to a common location on app install; for convenience if nothing
else.
Please note, this is not a concern for me directly as I fall into
the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between
assets and resources exists because the former can be accessed as read-only
files, while the latter are used via Android SDK with built-in
support for providing alternatives for different languages, OS versions,
screen orientations, etc
<https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing a
bug component for NDK API requests. I've asked for that to be fixed, but I
suppose for now you can file it in the Java category and we'll try to route
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that
it is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from
Visit this group at https://groups.google.com/group/android-ndk
.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
<javascript:>.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-27 18:39:37 UTC
Permalink
Cool, that makes sense.

I question the utility of "during debugging" being a reasoning due to
simple availability of rsync over usb or wifi. The major disadvantage to
zip is the entire file is a monolithic item to be sent rather than just a
patch delta; and other stores could reasonably be expected to support both
a zipped package and an unzipped package with enough time for them to
catch if enough apps take advantage of this functionality. I also have an
expectation that such patch systems would save end user's data on their
cell phones, plus battery life given less requires transfer anytime any app
updates.

That said, I think this is getting into a rabbit hole and far away from the
primary concern of our topic. What you propose, using aapt to leave files
uncompressed is a definite possibility for helping out cross platform
concerns. In practice, there's a few things I'd request to make this
easier:
1.) native method of getting file handle to uncompressed file. <-- ideally
just make some soft links that maintain in zip directory hierarchy in a
reasonable location that's easily referenced.
2.) update the play console to measure the apk after it has been compressed
by play store for instant apps (i.e. uncompressed apk is counted if using
this route and eliminates this option rapidly).

I think those two things would make this an approachable solution; what are
your thoughts? I do worry about other stores not enjoying the same
patching mechanism so Oculus Mobile would still need the same already
existing zlib style solutions. However, this at least makes sense to
experiment with.

I also think it's a good idea to extend the resource map alias generation
to handle files in the assets directory so Java side can benefit from a
shared GUI.
Post by Alex Cohn
There are three reasons why APK is and will be a ZIP. First, backwards
compatibility and tools that depend on this implementation detail. This
format is so widespread in the Java world: JAR, AAR. Second, there still
exist scenarios where the APK is served not through Play Store, e.g. during
debugging. ZIP compression may be handy in these cases. Third, there are no
dramatic disadvantages.
Cheers,
Alex
Post by Steven Winston
I'm speechless. If my understanding of what you just said is correct;
why is the apk a zip file at all? The only place there seems to be a
benefit, is in disk space. Disk space is much cheaper than cell phone
data. Even with that consideration; given the fact that data in the apk
should be viewed as "read only" why not simply use a partition that's
compressed (i.e. brtfs or similar?). I'm not sure I see a benefit for apk
being a zip file at all now.
By the way, that's a really awesome feature; I'm going to have to start
using it in projects. You just made the aapt tool much more intriguing.
Post by Alex Cohn
There is a cute fact about the APK size is that Google Play will perform
another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.
Cheers,
Alex Cohn
PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I didn't
know that resources could be aliased to be reproduced in that manner. Very
nifty. I did know that GetDrawable was deprecated. Lint has been very
helpful in pointing deprecation out but glad to know the why; makes sense.
It would seem that you've found a way to tackle this from the Java side
instead of the NDK side. I think it's pretty cool to be able to utilize
assets files as though they were a res however, I worry that it might blur
the designed lines of what should be in assets and what should be in res.
I guess shared UIs could be created in Java so would make sense to solve
for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was
under the impression that this is a "packaging option." Meaning that it is
never compressed and thus the apk is larger. If our goal is to have the
smallest sized apk (instant apps), then compression is good; and likely
what we want to keep. Also when thinking of networking; I might be one of
the few devs at the office that worries about end user's data, but why make
them download a larger apk when they don't need to? What would be really
cool is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt,
gradle has a very robust dsl available that is easier than the eclipse
https://google.github.io/android-gradle-dsl/current/com.
android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds
that must be used both through the Resources API (easier in Java) and
through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in
Java, *getDrawable(int) *is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can
force your asset to be stored uncompressed in the APK, e.g. by faking
its filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will
provide the unpacked (cached) copy when you ask for it – both in C and in
Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API would
be considered cross platform, and naturally there's an xkcd to cover this
situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk
space on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd wager
that when sharing resources one wouldn't need to have concern about which
screen size / orientation etc are used. Allow me to elaborate this strange
assertion with pointing out that if one were to go down the path of having
their own GUI and requiring shared resources, then likely it's shared
because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get
into a use case requirement of needing their own GUI is when the frustum
are not the same as the full screen one provided by Android's GUI. This
would be the case of VR or AR and some games. Thus, if you have a custom
projection matrix, you likely have need of one version of a resource and
wouldn't care what the Java mapping is for a resource that wouldn't match
your viewport in the first place; after all, if you did care, you likely
would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs
due to not using the custom viewport (which by the way you guys provide a
means to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the
first place as the reasoning behind the correlation is lost. It would have
to be incumbent upon the developer to determine the correct mapping within
the resources directory of reusable resources as that developer is the only
one capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy to
use interface (API) to return a file for a given path within the APK. I'm
not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources
will really help (unless Android finds a way to bring iOS and Unity to the
same ground). An independent 3rd party library could be better positioned
in this case.
As for sharing some resources with the "classic Android GUI", we
must keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the best
strategy in the rare cases that this is justified, is to call the native
Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object
for Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle
I'd want to jump through for this), it's also not exactly earth
shattering. Zlib as provided by cdep and caching of files as they are
extracted. If you need such a tool open sourced, I can certainly write a
new tool and provide that. There's a ton of samples that aren't always the
highest quality of ideas. https://www.google.com
/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+
resources+of+apk results in many SO questions that point people to
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe
the understanding is incorrect and the developer should place the files
within Assets successfully ignoring the res directory when in the ndk. For
these use cases selecting AAssetManager is a quite competent solution;
however, it fails on one key area, cross platform and requires maintaining
the java object in code that doesn't really need it (I've seen some less
than optimal solutions caused by accessing the java object from other
threads etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you
wish to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of
handling 2d user interaction (system messages, help screens, or options
menu) that must display classic Android GUI elements for. Now imagine
those GUI elements include something that would be useful to both GUI
resource sets (logos come to mind). The natural solution for this use case
would be to inform the dev to duplicate the resource in both Assets and Res
and stay with AAssetManager. That works until someone cares about apk
size, someone might say "hey how about we do an instant app." Now we have
to care about the size of the apk a whole lot more. If my understanding is
correct, this set of circumstances will transition a dev from being fully
able to successfully only use AAssetManager. They'd now be in the corner
case of using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy of
consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time
has in their arsenal of libraries a tool that handles APK file access.
It's common to everyone and a concern of everyone. Thus it stands to
reason that new comers are likely to not have these tools/libraries and
will wind up reinventing the wheel.
2.) There's tons of implementations for reading the APK of vastly
varying levels of quality.
That point speaks for itself, but suffice it to say, if it's hard
to get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is
quite helpful; but underused by several companies / developers. If
anything I'd love to get a folder to place resources that gets auto
expanded to a common location on app install; for convenience if nothing
else.
Please note, this is not a concern for me directly as I fall into
the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between
assets and resources exists because the former can be accessed as read-only
files, while the latter are used via Android SDK with built-in
support for providing alternatives for different languages, OS versions,
screen orientations, etc
<https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing
a bug component for NDK API requests. I've asked for that to be fixed, but
I suppose for now you can file it in the Java category and we'll try to
route it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that
it is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from
.
Visit this group at https://groups.google.com/grou
p/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-
438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-
4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-
4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFcKJqgZLyDTd4JLZ6jaSY_KcEsKH43S6Y97POtEh8bTfTomtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-27 19:55:40 UTC
Permalink
your 1) is in android/asset_manager.h

Best of luck
Alex
Post by Steven Winston
Cool, that makes sense.
I question the utility of "during debugging" being a reasoning due to
simple availability of rsync over usb or wifi. The major disadvantage to
zip is the entire file is a monolithic item to be sent rather than just a
patch delta; and other stores could reasonably be expected to support both
a zipped package and an unzipped package with enough time for them to
catch if enough apps take advantage of this functionality. I also have an
expectation that such patch systems would save end user's data on their
cell phones, plus battery life given less requires transfer anytime any app
updates.
That said, I think this is getting into a rabbit hole and far away from
the primary concern of our topic. What you propose, using aapt to leave
files uncompressed is a definite possibility for helping out cross platform
concerns. In practice, there's a few things I'd request to make this
1.) native method of getting file handle to uncompressed file. <-- ideally
just make some soft links that maintain in zip directory hierarchy in a
reasonable location that's easily referenced.
2.) update the play console to measure the apk after it has been
compressed by play store for instant apps (i.e. uncompressed apk is counted
if using this route and eliminates this option rapidly).
I think those two things would make this an approachable solution; what
are your thoughts? I do worry about other stores not enjoying the same
patching mechanism so Oculus Mobile would still need the same already
existing zlib style solutions. However, this at least makes sense to
experiment with.
I also think it's a good idea to extend the resource map alias generation
to handle files in the assets directory so Java side can benefit from a
shared GUI.
Post by Alex Cohn
There are three reasons why APK is and will be a ZIP. First, backwards
compatibility and tools that depend on this implementation detail. This
format is so widespread in the Java world: JAR, AAR. Second, there still
exist scenarios where the APK is served not through Play Store, e.g. during
debugging. ZIP compression may be handy in these cases. Third, there are no
dramatic disadvantages.
Cheers,
Alex
Post by Steven Winston
I'm speechless. If my understanding of what you just said is correct;
why is the apk a zip file at all? The only place there seems to be a
benefit, is in disk space. Disk space is much cheaper than cell phone
data. Even with that consideration; given the fact that data in the apk
should be viewed as "read only" why not simply use a partition that's
compressed (i.e. brtfs or similar?). I'm not sure I see a benefit for apk
being a zip file at all now.
By the way, that's a really awesome feature; I'm going to have to start
using it in projects. You just made the aapt tool much more intriguing.
Post by Alex Cohn
There is a cute fact about the APK size is that Google Play will
perform another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.
Cheers,
Alex Cohn
PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I
didn't know that resources could be aliased to be reproduced in that
manner. Very nifty. I did know that GetDrawable was deprecated. Lint has
been very helpful in pointing deprecation out but glad to know the why;
makes sense. It would seem that you've found a way to tackle this from the
Java side instead of the NDK side. I think it's pretty cool to be able to
utilize assets files as though they were a res however, I worry that it
might blur the designed lines of what should be in assets and what should
be in res. I guess shared UIs could be created in Java so would make sense
to solve for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was
under the impression that this is a "packaging option." Meaning that it is
never compressed and thus the apk is larger. If our goal is to have the
smallest sized apk (instant apps), then compression is good; and likely
what we want to keep. Also when thinking of networking; I might be one of
the few devs at the office that worries about end user's data, but why make
them download a larger apk when they don't need to? What would be really
cool is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt,
gradle has a very robust dsl available that is easier than the eclipse
https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and sounds
that must be used both through the Resources API (easier in Java) and
through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in
Java, *getDrawable(int) *is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can
force your asset to be stored uncompressed in the APK, e.g. by
faking its filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will
provide the unpacked (cached) copy when you ask for it – both in C and in
Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API
would be considered cross platform, and naturally there's an xkcd to cover
this situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say there
isn't a solution that we can craft to address the needs of a cross platform
developer having access to apk resources. There's tradeoffs to other
solutions, however, if we first accept that all languages the NDK might
support now and in the future can natively read files off the disk with
basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk
space on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd
wager that when sharing resources one wouldn't need to have concern about
which screen size / orientation etc are used. Allow me to elaborate this
strange assertion with pointing out that if one were to go down the path of
having their own GUI and requiring shared resources, then likely it's
shared because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would get
into a use case requirement of needing their own GUI is when the frustum
are not the same as the full screen one provided by Android's GUI. This
would be the case of VR or AR and some games. Thus, if you have a custom
projection matrix, you likely have need of one version of a resource and
wouldn't care what the Java mapping is for a resource that wouldn't match
your viewport in the first place; after all, if you did care, you likely
would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs
due to not using the custom viewport (which by the way you guys provide a
means to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the
first place as the reasoning behind the correlation is lost. It would have
to be incumbent upon the developer to determine the correct mapping within
the resources directory of reusable resources as that developer is the only
one capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy
to use interface (API) to return a file for a given path within the APK.
I'm not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources
will really help (unless Android finds a way to bring iOS and Unity to the
same ground). An independent 3rd party library could be better positioned
in this case.
As for sharing some resources with the "classic Android GUI", we
must keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the
best strategy in the rare cases that this is justified, is to call the
native Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object
for Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular hurdle
I'd want to jump through for this), it's also not exactly earth
shattering. Zlib as provided by cdep and caching of files as they are
extracted. If you need such a tool open sourced, I can certainly write a
new tool and provide that. There's a ton of samples that aren't always the
highest quality of ideas.
https://www.google.com/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+resources+of+apk
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe
the understanding is incorrect and the developer should place the files
within Assets successfully ignoring the res directory when in the ndk. For
these use cases selecting AAssetManager is a quite competent solution;
however, it fails on one key area, cross platform and requires maintaining
the java object in code that doesn't really need it (I've seen some less
than optimal solutions caused by accessing the java object from other
threads etc).
So most of the time legacy game devs that aren't already on Unity,
Unreal or other common game engine here would select the first option,
unzip the apk. These are quite competent devs that understand caching so
there's no real problem there that a new API would help. That said, they
are the people I'd most expect to have a valid reason to select the first
path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you
wish to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of
handling 2d user interaction (system messages, help screens, or options
menu) that must display classic Android GUI elements for. Now imagine
those GUI elements include something that would be useful to both GUI
resource sets (logos come to mind). The natural solution for this use case
would be to inform the dev to duplicate the resource in both Assets and Res
and stay with AAssetManager. That works until someone cares about apk
size, someone might say "hey how about we do an instant app." Now we have
to care about the size of the apk a whole lot more. If my understanding is
correct, this set of circumstances will transition a dev from being fully
able to successfully only use AAssetManager. They'd now be in the corner
case of using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or maybe
Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native code
needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy
of consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time
has in their arsenal of libraries a tool that handles APK file access.
It's common to everyone and a concern of everyone. Thus it stands to
reason that new comers are likely to not have these tools/libraries and
will wind up reinventing the wheel.
2.) There's tons of implementations for reading the APK of
vastly varying levels of quality.
That point speaks for itself, but suffice it to say, if it's
hard to get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is
quite helpful; but underused by several companies / developers. If
anything I'd love to get a folder to place resources that gets auto
expanded to a common location on app install; for convenience if nothing
else.
Please note, this is not a concern for me directly as I fall
into the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between
assets and resources exists because the former can be accessed as read-only
files, while the latter are used via Android SDK with built-in
support for providing alternatives for different languages, OS versions,
screen orientations, etc
<https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're missing
a bug component for NDK API requests. I've asked for that to be fixed, but
I suppose for now you can file it in the Java category and we'll try to
https://issuetracker.google.com/issues/new?component=328403
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust that
it is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets
folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from
To post to this group, send email to
Visit this group at
https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe
.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
<javascript:>.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/512c026f-2326-4c2b-b50a-d803e2d1deb0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Steven Winston
2018-03-27 22:04:52 UTC
Permalink
:) Ideally it'd be something that doesn't require a custom Android only
API. In lieu of that, something that doesn't require a java object and is
multi-thread ready. However, I'll admit AAssetManager is much more
powerful than I original thought it was.
Post by Alex Cohn
your 1) is in android/asset_manager.h
Best of luck
Alex
Post by Steven Winston
Cool, that makes sense.
I question the utility of "during debugging" being a reasoning due to
simple availability of rsync over usb or wifi. The major disadvantage to
zip is the entire file is a monolithic item to be sent rather than just a
patch delta; and other stores could reasonably be expected to support both
a zipped package and an unzipped package with enough time for them to
catch if enough apps take advantage of this functionality. I also have an
expectation that such patch systems would save end user's data on their
cell phones, plus battery life given less requires transfer anytime any app
updates.
That said, I think this is getting into a rabbit hole and far away from
the primary concern of our topic. What you propose, using aapt to leave
files uncompressed is a definite possibility for helping out cross platform
concerns. In practice, there's a few things I'd request to make this
1.) native method of getting file handle to uncompressed file. <--
ideally just make some soft links that maintain in zip directory hierarchy
in a reasonable location that's easily referenced.
2.) update the play console to measure the apk after it has been
compressed by play store for instant apps (i.e. uncompressed apk is counted
if using this route and eliminates this option rapidly).
I think those two things would make this an approachable solution; what
are your thoughts? I do worry about other stores not enjoying the same
patching mechanism so Oculus Mobile would still need the same already
existing zlib style solutions. However, this at least makes sense to
experiment with.
I also think it's a good idea to extend the resource map alias generation
to handle files in the assets directory so Java side can benefit from a
shared GUI.
Post by Alex Cohn
There are three reasons why APK is and will be a ZIP. First, backwards
compatibility and tools that depend on this implementation detail. This
format is so widespread in the Java world: JAR, AAR. Second, there still
exist scenarios where the APK is served not through Play Store, e.g. during
debugging. ZIP compression may be handy in these cases. Third, there are no
dramatic disadvantages.
Cheers,
Alex
Post by Steven Winston
I'm speechless. If my understanding of what you just said is correct;
why is the apk a zip file at all? The only place there seems to be a
benefit, is in disk space. Disk space is much cheaper than cell phone
data. Even with that consideration; given the fact that data in the apk
should be viewed as "read only" why not simply use a partition that's
compressed (i.e. brtfs or similar?). I'm not sure I see a benefit for apk
being a zip file at all now.
By the way, that's a really awesome feature; I'm going to have to start
using it in projects. You just made the aapt tool much more intriguing.
Post by Alex Cohn
There is a cute fact about the APK size is that Google Play will
perform another round of compression for the full file. This means that any
unpacked resource will be compressed as tightly as possible while it's
downloaded; the APK file in /data/apps will be uncompressed, to let you
access the unpacked resources 'in place'. This is not necessarily a waste
of user's disk space: you don't need to keep a cache of unpacked assets.
From 'M' and up, same logic applies to the native libs: instead of
unpacking them to /data/app/package/lib/arm, you simply use them 'in
place'. Also, Play Store supports delta updates: it will download only the
patch, and produce the new APK from the old one. Such deltas may become
smaller when native libraries and other assets are not compressed in the
APK. Finally, for asset types that are not compressed by default (e.g. jpg
or mp3), zip compression will hardly provide any gain.
Cheers,
Alex Cohn
PS Thanks for the Aapt options link
<https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.AaptOptions.html>,
it's very welcome in the context of this discussion.
Post by Steven Winston
@alex
Thanks for this; I learned quite a bit of android lore and it made me
think about a system I've long since taken for granted. If nothing else
comes of our discussion I view it as useful to me for no other reason than
to have thought through it.
I like your thought of extending alias resources. Admittedly, I
didn't know that resources could be aliased to be reproduced in that
manner. Very nifty. I did know that GetDrawable was deprecated. Lint has
been very helpful in pointing deprecation out but glad to know the why;
makes sense. It would seem that you've found a way to tackle this from the
Java side instead of the NDK side. I think it's pretty cool to be able to
utilize assets files as though they were a res however, I worry that it
might blur the designed lines of what should be in assets and what should
be in res. I guess shared UIs could be created in Java so would make sense
to solve for both places and this is the path of least resistance.
For the compressed nature. I've seen that solution before. I was
under the impression that this is a "packaging option." Meaning that it is
never compressed and thus the apk is larger. If our goal is to have the
smallest sized apk (instant apps), then compression is good; and likely
what we want to keep. Also when thinking of networking; I might be one of
the few devs at the office that worries about end user's data, but why make
them download a larger apk when they don't need to? What would be really
cool is an ability to inform that the apk should expand certain files or a
directory on install, or some other caching mechanism. Ideally, the final
solution should be better than the current zlib everyone has a different
yet common solution scenario.
btw, for all that see this conversation and wish to play with aapt,
gradle has a very robust dsl available that is easier than the eclipse
https://google.github.io/android-gradle-dsl/current/com.
android.build.gradle.internal.dsl.AaptOptions.html
<https://www.google.com/url?q=https%3A%2F%2Fgoogle.github.io%2Fandroid-gradle-dsl%2Fcurrent%2Fcom.android.build.gradle.internal.dsl.AaptOptions.html&sa=D&sntz=1&usg=AFQjCNGhoen5nuzRQ7DWDfM_QxAaIWlBaw>
Post by Alex Cohn
@Steven,
My takeaway from your description is that there may be another piece
missing in the Android resources API. What if we could easily map some
resources (e.g. *R.drawable.my_background_image* to
*assets/background.png*) to reduce duplication for images and
sounds that must be used both through the Resources API (easier in Java)
and through Assets API (easier in C)? This could be an extension of the
existing alias resources
<https://developer.android.com/guide/topics/resources/providing-resources.html#AliasResources> feature,
and avoid the custom workarounds
<https://stackoverflow.com/a/23706774/192373>. Note that even in
Java, *getDrawable(int) *is now deprecated, for good
<https://www.androiddesignpatterns.com/2016/08/contextcompat-getcolor-getdrawable.html>
.
As for the zipped nature of APK, it's under your control. You can
force your asset to be stored uncompressed in the APK, e.g. by
faking its filename extension
<https://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps>,
and this means that the asset manager will give you handle to the piece of
your APK with no need to cache the unpacked asset on disk (make sure that
you don't drop the offset for the asset while working with it). On the
other hand, if you can force your asset to be compressed
<http://banachowski.com/deprogramming/2012/06/android-assets-where-did-you-go/>,
if you prepare it as *whatever.ext*.gz. The Asset Manager will
provide the unpacked (cached) copy when you ask for it – both in C and in
Java.
For resources, the Java API
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResourceFd(int)>
only gives you a file descriptor if the raw resource is stored as
uncompressed data. The nice shortcut that Java has for us, though, is to provide
an InputStream
<https://developer.android.com/reference/android/content/res/Resources.html#openRawResource(int)> with
no extra boilerplate code.
Sincerely,
Alex
Post by Steven Winston
@Alex,
Happy to follow up, and please remember these are just thoughts for
improvement as I have no problems with the current system. I wasn't
intending to present an argument or present even an advocacy for any
particular solution; my apologies as my meaning was muddled with the
intention of presenting the use case of a new user using Google only
provided technologies and winding up in a situation that warranted a
requirement of accessing the /res directory from the NDK. The goal was
merely to show it is a non-zero chance (and I'd wager an increasing chance)
that only going down the path of Google best practices results in
developers having an unaddressed need.
That said, the points you make a valid. No Google provided API
would be considered cross platform, and naturally there's an xkcd to cover
this situation: https://imgs.xkcd.com/comics/standards.png
However, I would mention that perhaps we're a bit quick to say
there isn't a solution that we can craft to address the needs of a cross
platform developer having access to apk resources. There's tradeoffs to
other solutions, however, if we first accept that all languages the NDK
might support now and in the future can natively read files off the disk
with basic file I/O and all other platforms share similar basic file I/O
capabilities. A solution presents itself if we eliminate the zipped nature
of the apk.
Obvious trade off with that solution is it would take up more disk
space on app install. This could be mitigated by expanding only portions a
developer would care about; which providing an api would allow a developer
to say what files should be available or label them in the manifest for
easy NDK access then explode to temp directory on either app install, run,
or when requested via API. There's also a chance if one were to be a bit
clever of crafting a ramdisk solution in Java that C sees as a file system.
Moving on to the Resources access for shared GUI concerns. I'd
wager that when sharing resources one wouldn't need to have concern about
which screen size / orientation etc are used. Allow me to elaborate this
strange assertion with pointing out that if one were to go down the path of
having their own GUI and requiring shared resources, then likely it's
shared because they're still using the very excellent Android GUI for some
interactions. This would indicate with my wild assumption that the reason
they might have a second GUI requirement is because the Android GUI doesn't
completely fit their needs. Now the reason why there's different resources
for different orientations, screen densities etc is because the GUI is
operating on a different viewport dictated by being different size,
oriented or on different screen densities etc.
Assuming my understanding is correct, the time an end user would
get into a use case requirement of needing their own GUI is when the
frustum are not the same as the full screen one provided by Android's GUI.
This would be the case of VR or AR and some games. Thus, if you have a
custom projection matrix, you likely have need of one version of a resource
and wouldn't care what the Java mapping is for a resource that wouldn't
match your viewport in the first place; after all, if you did care, you
likely would be in a scenario where Android's GUI is a better path forward.
So to clarify the use case here is Android GUI can't fit the needs
due to not using the custom viewport (which by the way you guys provide a
means to render the Android GUI to a GLTexture and I'll sing the highest of
praises for Android's GUI). However, there are some resources that can be
used sans changes between GUIs (logos and other sharable disk guzzling
assets that are high res etc). So my point in all this is it wouldn't
likely matter that R.drawable.my_background_image maps to
/res/hdpi/xxxx/background.png instead of /res/xxx/yyy/background.jpg.
What would ultimately matter is generic access to the file in the
first place as the reasoning behind the correlation is lost. It would have
to be incumbent upon the developer to determine the correct mapping within
the resources directory of reusable resources as that developer is the only
one capable of knowing the frustum they will be using.
If the dev wants to use a completely custom GUI, then they have all
resources in Assets and they have an existing solution. If the dev wants
to use completely Android GUI, then this isn't an issue as they can use
Java. If the dev has need for keeping the apk small (instant apps) and
they're looking for cutting file size costs, then providing a means to
access resources that might be shared between the two is attractive.
In this case what I'd propose is packaging zlib along with an easy
to use interface (API) to return a file for a given path within the APK.
I'm not sure there's a need to be more clever than that for the shared GUI
problem.
Post by Alex Cohn
Thank you for this reply.
This is not very convincing, I must confess.
When we speak about cross platform, no new API to access resources
will really help (unless Android finds a way to bring iOS and Unity to the
same ground). An independent 3rd party library could be better positioned
in this case.
As for sharing some resources with the "classic Android GUI", we
must keep in mind that the API will need all the logic of screen size /
orientation / locale / whatnot that is transparent when using
Context.getResources()
<https://developer.android.com/reference/android/content/Context.html#getResources()>.
Not only that, resources use generated id's (e.g.
*R.drawable.my_background_image*), not names. As a result, the
best strategy in the rare cases that this is justified, is to call the
native Resources
<https://developer.android.com/reference/android/content/res/Resources.html> API
via JNI. All you need a valid Context
<https://developer.android.com/reference/android/content/Context.html#getResources()>
.
I agree that the existing Assets API which requires a Java object
for Asset Manager is not very convenient for native developers.
BR,
Alex
Post by Steven Winston
@Alex,
While I can't offer up my companies' code (not a particular
hurdle I'd want to jump through for this), it's also not exactly earth
shattering. Zlib as provided by cdep and caching of files as they are
extracted. If you need such a tool open sourced, I can certainly write a
new tool and provide that. There's a ton of samples that aren't always the
highest quality of ideas. https://www.google.com
/search?ei=9piyWoyAA4fUjwP57KjoCA&q=ndk+read+files+in+
resources+of+apk results in many SO questions that point people
unzip the apk
AAssetManager
Read in Java pass through JNI
etc.
For almost every use case I've seen in those questions, I believe
the understanding is incorrect and the developer should place the files
within Assets successfully ignoring the res directory when in the ndk. For
these use cases selecting AAssetManager is a quite competent solution;
however, it fails on one key area, cross platform and requires maintaining
the java object in code that doesn't really need it (I've seen some less
than optimal solutions caused by accessing the java object from other
threads etc).
So most of the time legacy game devs that aren't already on
Unity, Unreal or other common game engine here would select the first
option, unzip the apk. These are quite competent devs that understand
caching so there's no real problem there that a new API would help. That
said, they are the people I'd most expect to have a valid reason to select
the first path and already have toolsets in house that handle it.
For new devs, the challenge should be to instruct that Assets are
where you want to place NDK resources and use AAssetManager. However, I'd
propose a relatively new use case where if my understanding is correct, the
only available answer facing a new dev would be treat the apk as a zip.
Imagine, having a project that is targeting a VR/AR app that you
wish to have an in Vr/Ar GUI. Now that VR/AR app has a requirement of
handling 2d user interaction (system messages, help screens, or options
menu) that must display classic Android GUI elements for. Now imagine
those GUI elements include something that would be useful to both GUI
resource sets (logos come to mind). The natural solution for this use case
would be to inform the dev to duplicate the resource in both Assets and Res
and stay with AAssetManager. That works until someone cares about apk
size, someone might say "hey how about we do an instant app." Now we have
to care about the size of the apk a whole lot more. If my understanding is
correct, this set of circumstances will transition a dev from being fully
able to successfully only use AAssetManager. They'd now be in the corner
case of using the apk as a zip.
Again, this isn't a major requirement as using zlib or other zip
library is quite easy, most devs are used to it from other platforms, also
even if I weren't in a similar use-case requirement for the above scenario
I'd still opt for using zlib to better support handling resources in a
common method between Android/iOS/PS4 etc. Hopefully this provides an idea
for when it could prove useful to have such an API for others.
Post by Alex Cohn
@Steven,
Could you point to some sample code that works decently? Or
maybe Google can help you opensource your toolset?
OTOH, I would be glad to learn in what use cases your native
code needs access specifically to /res and not to /assets.
Sincerely,
Alex
Post by Steven Winston
I have to admit to agreeing with Dan that this might be worthy
of consideration. I think you're right about the underlining intention for
Assets to hold assets and resources to hold android sdk stuff that changes
per several different factors. However, a standardized method of accessing
it from native would actually prove quite beneficial.
1.) It lowers the barrier for entry into using the NDK.
Everyone that works with the NDK for any longer length of time
has in their arsenal of libraries a tool that handles APK file access.
It's common to everyone and a concern of everyone. Thus it stands to
reason that new comers are likely to not have these tools/libraries and
will wind up reinventing the wheel.
2.) There's tons of implementations for reading the APK of
vastly varying levels of quality.
That point speaks for itself, but suffice it to say, if it's
hard to get the API right, then it's likely hard for new comers to get the
implementation right. Lots of bad code out there, can point to some sample
code that does incredibly bad things in this area too.
A bonus reason is this is a good common place where caching is
quite helpful; but underused by several companies / developers. If
anything I'd love to get a folder to place resources that gets auto
expanded to a common location on app install; for convenience if nothing
else.
Please note, this is not a concern for me directly as I fall
into the category of having an already existing competent toolset.
Post by Alex Cohn
I beg to differ. I believe that the whole separation between
assets and resources exists because the former can be accessed as read-only
files, while the latter are used via Android SDK with built-in
support for providing alternatives for different languages, OS versions,
screen orientations, etc
<https://stackoverflow.com/a/5583782/192373>.
There is very little added value in having native access to
resources, and many tricky questions to resolve while implementing it.
BR,
Alex
Post by 'Dan Albert' via android-ndk
Sounds like a reasonable API request. Looks like we're
missing a bug component for NDK API requests. I've asked for that to be
fixed, but I suppose for now you can file it in the Java category and we'll
try to route it appropriately: https://issuetracker.google.co
m/issues/new?component=328403
On Thu, Mar 15, 2018 at 12:24 PM Alex Cohn <
Post by Alex Cohn
There is no API, but you can find your APK file
<https://stackoverflow.com/q/7701801/192373>, and trust
that it is a ZIP file under the hood.
BR,
Alex
Post by idea
We know there is Android NDK API to read file from assets
folder.
Is there any way to read file from res/raw folder?
--
You received this message because you are subscribed to the
Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails
To post to this group, send email to
Visit this group at https://groups.google.com/grou
p/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-
438d-8f0f-d38f9ca55d96%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/a76b0327-7ccf-438d-8f0f-d38f9ca55d96%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/android-ndk/A9jdEgAqkNw/un
subscribe.
To unsubscribe from this group and all its topics, send an email
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-
4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/852c4dfe-a28a-4b1e-a6db-1e2dc0dd4dfa%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in
the Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-
4ed5-acae-904c3f1a96b6%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/7513c7e6-b569-4ed5-acae-904c3f1a96b6%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit
https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-
4f0f-9e93-51a3bca9e426%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/b451cc2f-2fe4-4f0f-9e93-51a3bca9e426%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/to
pic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/ms
gid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/79f6d983-e8d4-41c3-8c95-2e19fcab331c%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the
Google Groups "android-ndk" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/android-ndk/A9jdEgAqkNw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/
msgid/android-ndk/512c026f-2326-4c2b-b50a-d803e2d1deb0%40googlegroups.com
<https://groups.google.com/d/msgid/android-ndk/512c026f-2326-4c2b-b50a-d803e2d1deb0%40googlegroups.com?utm_medium=email&utm_source=footer>
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAFcKJqjE4%2B4FH8sg5wDMx2%2B9XYoLyQj6KBSG57njH7yhpOGmjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Chris Browet
2018-03-22 11:57:05 UTC
Permalink
In XBMC derivatives, we always leave the "res" folder alone in native (only
for JAVA), and we plainly access the "assets" folder as a zip file from the
apk.

Just my $.02
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/41bfdbb7-1594-4165-b18d-bb5db079c111%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-25 11:16:23 UTC
Permalink
Why don't you use the AAsset API? Just curious.

Alex
Post by Chris Browet
In XBMC derivatives, we always leave the "res" folder alone in native
(only for JAVA), and we plainly access the "assets" folder as a zip file
from the apk.
Just my $.02
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/c5262d3b-40b6-452a-b206-f2e4918d3e64%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Chris Browet
2018-03-25 11:51:01 UTC
Permalink
Good Point.

I guess the API was overlooked.
Is there a performance gain to be expected vs "plain" zip handling?
Post by Alex Cohn
Why don't you use the AAsset API? Just curious.
Alex
Post by Chris Browet
In XBMC derivatives, we always leave the "res" folder alone in native
(only for JAVA), and we plainly access the "assets" folder as a zip file
from the apk.
Just my $.02
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/2766e2a0-5c90-483a-9dcd-a0cc6a3359d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alex Cohn
2018-03-26 17:08:27 UTC
Permalink
Post by Chris Browet
Good Point.
I guess the API was overlooked.
Is there a performance gain to be expected vs "plain" zip handling?
If the asset is stored 'as is' (e.g. an mp3 file), then the Asset Manager
will let you use the APK file with offset to the beginning of the asset. If
the asset is compressed, the Asset Manager will seemlessly unpack it for
you and cache the result. Both tricks can be reproduced with "plain zip".
Actually, your custom unzip may use a smarter caching strategy for
compressed resources. But "dumb zip" will definitely be less efficient in
both cases.

BR,
Alex
--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk+***@googlegroups.com.
To post to this group, send email to android-***@googlegroups.com.
Visit this group at https://groups.google.com/group/android-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/eb0a408b-871f-4204-ac44-0583272d0420%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Continue reading on narkive:
Loading...