Better file handling on Android, multi-platform C++ project for FlashDevelop

[Joshua Granick]

Hi everyone!

I have some news, for anyone who is interested.


I created a "Multi-Platform C++ Project" template for FlashDevelop. It
currently supports webOS, Android and Flash, but I'll probably add more
targets in the future. You can check it out here:

http://www.eclecticdesignstudio.com/code/flashdevelop/Multi-Platform%20C++%20Project.fdz


As a part of this process, I'm working on improving the Android target --
specifically, right now, the file handling. On webOS (and many other
targets), supporting assets are extracted and are readily available to the
application. Unfortunately, Android keeps applications inside of an APK
package ... forever. This means that haXe has worked on Android up till
now (thanks, of course, to Hugh Sanderson), but you couldn't access any
supporting files unless they were downloaded or they were accessed from
external storage using absolute paths.

The current release of the template also includes an initial system for
fixing this problem. Supporting assets from your project are copied to an
internal storage directory on the phone the first time you run the
application. From that point on, haXe will redirect paths which don't have
a starting slash to look in this directory for files. That means that if
you reference "MyImage.png", it will work in Flash, webOS, and now
Android, thanks to some "magic" behind the scenes. It doesn't support
directories yet (as the internal storage directory Android provides does
not allow directories as far as I can tell), but I'm going to try and work
out a convention soon so that you can "directories" will still work.


My updates to NME and HXCPP are available from
http://code.google.com/p/hxcpp-webos, which is what I'm building against
with this template. To compile for webOS you will need to install the Palm
SDK and CodeSourcery GNU toolchain for ARM. To compile for Android, you
will need to extract the Android SDK, Android NDK r5, Apache Ant and
install the Java JDK, set up a bunch of environment variables and place
your project in a directory without spaces, as Ant is very finicky and is
required to build the application. To compile for Flash, of course, you
just need FlashDevelop.

I can go into greater detail if anyone is interested. You'll also need a
webOS or Android phone, as the applications need to be run on a device to
work.

[Hugh Sanderson]

Hi Joshua,
I have also been revisiting android - and just started looking at
resources.
My current approach is to call back into the Java code to extract
the resources and then pass them to nme.

I also notice that the NDK r5 has an issue with libstdc++.a, which
shows up in android versions <= 2.1 - something to watch out for because
I wasted a day on that one.

Something that may also be of interest to you is that I've added a
templating system to NME to allow the construction of android projects
(and eventually others) with the "InstallTool.hx". This is still WIP,
but I hope to make it the easy way to get started with NME.

I will probably do an NME release in a few weeks - I would like to
get the webOS stuff merged in officially by then.

Hugh

Post by Joshua Granick
Hi everyone!
I have some news, for anyone who is interested.
I created a "Multi-Platform C++ Project" template for FlashDevelop. It
currently supports webOS, Android and Flash, but I'll probably add more
http://www.eclecticdesignstudio.com/code/flashdevelop/Multi-Platform%20C++%20Project.fdz
As a part of this process, I'm working on improving the Android target
-- specifically, right now, the file handling. On webOS (and many other
targets), supporting assets are extracted and are readily available to
the application. Unfortunately, Android keeps applications inside of an
APK package ... forever. This means that haXe has worked on Android up
till now (thanks, of course, to Hugh Sanderson), but you couldn't access
any supporting files unless they were downloaded or they were accessed
from external storage using absolute paths.
The current release of the template also includes an initial system for
fixing this problem. Supporting assets from your project are copied to
an internal storage directory on the phone the first time you run the
application. From that point on, haXe will redirect paths which don't
have a starting slash to look in this directory for files. That means
that if you reference "MyImage.png", it will work in Flash, webOS, and
now Android, thanks to some "magic" behind the scenes. It doesn't
support directories yet (as the internal storage directory Android
provides does not allow directories as far as I can tell), but I'm going
to try and work out a convention soon so that you can "directories" will
still work.
My updates to NME and HXCPP are available from
http://code.google.com/p/hxcpp-webos, which is what I'm building against
with this template. To compile for webOS you will need to install the
Palm SDK and CodeSourcery GNU toolchain for ARM. To compile for Android,
you will need to extract the Android SDK, Android NDK r5, Apache Ant and
install the Java JDK, set up a bunch of environment variables and place
your project in a directory without spaces, as Ant is very finicky and
is required to build the application. To compile for Flash, of course,
you just need FlashDevelop.
I can go into greater detail if anyone is interested. You'll also need a
webOS or Android phone, as the applications need to be run on a device
to work.

[Joshua Granick]

Hey Hugh,

I didn't realize that OpenRead was only used twice inside of NME. My
original perspective had been that I didn't want to modify the entire
library to understand an offset and length for reading files, but instead
wanted to provide a solution which would be compatible with normal fopen
call, where you would read until you run out of bytes.

My only concern with accessing files directly is that you wouldn't have
write access, but I suppose that is a small inconvenience for being able
to access your data. I had also been concerned about compatibility with
other libraries, such as zlib or sqlite, but I suppose that they just
won't be able to work with files which are packaged with the application.
That seems like a fair trade-off as well.

If you haven't written the code for accessing them directly, yet, this was
the most useful thread I found yesterday, which may be helpful to you:

http://groups.google.com/group/android-ndk/browse_thread/thread/a69084018e87a5a8/9c513f8eb2956b87

I made some changes to android-toolchain.xml in order to compile NME and
all of the extra libraries. The only library I can't compile right now is
std, which gets upset about not being able to find some string functions.
I haven't revisited it lately, but perhaps it just needs the std::
namespace in front of some of its calls, or perhaps the libstdc++.a file
you have fixes this problem?

At runtime, I can't use sqlite for some reason. It gets stuck asking for a
connection, and eventually locks the phone if I don't force quit the
application. I've been able to successfully see the menu screens for some
of my games, but they each close when I try to start a new level.

I haven't had the time to dig in and see what is causing the problems.
Before I commented out my audio code, I received a runtime exception from
haXe in the device logs, and then the application would close. Whatever is
causing the current crash seems to be a hiccup in the internal code, as it
goes through a full crash rather than throwing a runtime error.

I have been maintaining my changes to hxcpp, nekonme and sdl-static using
directories that are still connected to your repository. I have been
updating whenever you make a new commit, and have done my best to combine
your changes with my own. When I are ready to make a commit so others can
compile using my version of the libraries, I export them into a combined
folder and commit it to my own repository. As a result, I can get the diff
file for any of these repositories if that is something you are interested
in.

Post by Hugh Sanderson
Hi Joshua,
I have also been revisiting android - and just started looking at
resources.
My current approach is to call back into the Java code to extract
the resources and then pass them to nme.
I also notice that the NDK r5 has an issue with libstdc++.a, which
shows up in android versions <= 2.1 - something to watch out for because
I wasted a day on that one.
Something that may also be of interest to you is that I've added a
templating system to NME to allow the construction of android projects
(and eventually others) with the "InstallTool.hx". This is still WIP,
but I hope to make it the easy way to get started with NME.
I will probably do an NME release in a few weeks - I would like to
get the webOS stuff merged in officially by then.
Hugh

Post by Joshua Granick
Hi everyone!
I have some news, for anyone who is interested.
I created a "Multi-Platform C++ Project" template for FlashDevelop. It
currently supports webOS, Android and Flash, but I'll probably add more
http://www.eclecticdesignstudio.com/code/flashdevelop/Multi-Platform%20C++%20Project.fdz
As a part of this process, I'm working on improving the Android target
-- specifically, right now, the file handling. On webOS (and many other
targets), supporting assets are extracted and are readily available to
the application. Unfortunately, Android keeps applications inside of an
APK package ... forever. This means that haXe has worked on Android up
till now (thanks, of course, to Hugh Sanderson), but you couldn't
access any supporting files unless they were downloaded or they were
accessed from external storage using absolute paths.
The current release of the template also includes an initial system for
fixing this problem. Supporting assets from your project are copied to
an internal storage directory on the phone the first time you run the
application. From that point on, haXe will redirect paths which don't
have a starting slash to look in this directory for files. That means
that if you reference "MyImage.png", it will work in Flash, webOS, and
now Android, thanks to some "magic" behind the scenes. It doesn't
support directories yet (as the internal storage directory Android
provides does not allow directories as far as I can tell), but I'm
going to try and work out a convention soon so that you can
"directories" will still work.
My updates to NME and HXCPP are available from
http://code.google.com/p/hxcpp-webos, which is what I'm building
against with this template. To compile for webOS you will need to
install the Palm SDK and CodeSourcery GNU toolchain for ARM. To compile
for Android, you will need to extract the Android SDK, Android NDK r5,
Apache Ant and install the Java JDK, set up a bunch of environment
variables and place your project in a directory without spaces, as Ant
is very finicky and is required to build the application. To compile
for Flash, of course, you just need FlashDevelop.
I can go into greater detail if anyone is interested. You'll also need
a webOS or Android phone, as the applications need to be run on a
device to work.

[Joshua Granick]

Hey Hugh,

Thanks for getting things started with the direct file access. I realized
that my first problem is that Android wasn't refreshing my application
when I hadn't touched one of the Java files. I know that you mentioned
this in your tutorial and said that it was because of Eclipse, but I
suppose it is the Java compiler, or package installer.

Once I realized that I couldn't actually see any of my changes (and that's
why none of my debug messages appeared), I also realized that you'd gotten
started on the resource code but was returning 0 for the time being. Of
course that would explain why I couldn't see images yet.

I invite you to refactor this code ... I might not be freeing resources or
otherwise doing things in the best performing way, but it works! It's
really nice to see it working. I thought that you would be able to
recognize inefficiencies in the code, but might like to see the working
example.



This is in GameActivity.java:


static public short [] getResource(String inResource) {
Log.e("GameActivity","getResource " + inResource);

try {
byte [] bytes = new byte[0];

java.io.InputStream inputStream =
mAssets.open(inResource,AssetManager.ACCESS_BUFFER);
byte [] buffer = new byte[1024];
while(true)
{
int read = inputStream.read(buffer,0,1024);
if (read<=0)
break;

byte[] total = new byte[bytes.length + read];
System.arraycopy (bytes, 0, total, 0, bytes.length);
System.arraycopy(buffer, 0, total, bytes.length, read);
bytes = total;

if (read<1024)
break;
}
inputStream.close();

short[] result = new short[bytes.length];
for (int i = 0; i < bytes.length; i++) {
result[i] = (short) ((short) bytes[i] & 0xFF);
}

return result;

} catch (java.io.IOException e) {
Log.e("GameActivity",e.toString());
}

Log.e("GameActivity","No resource");

return null;
}




This is in AndroidFrame.cpp:



ByteArray *AndroidGetAssetBytes(const char *inResource)
{
jclass cls = gEnv->FindClass("org/haxe/nme/GameActivity");
jmethodID mid = gEnv->GetStaticMethodID(cls, "getResource",
"(Ljava/lang/String;)[S");
if (mid == 0)
return 0;

jstring str = gEnv->NewStringUTF( inResource );
jshortArray value = (jshortArray)gEnv->CallStaticObjectMethod(cls, mid,
str);

if (value==0)
return 0;

jint len = gEnv->GetArrayLength(value);
jshort *data = gEnv->GetShortArrayElements(value, 0);

ByteArray *result = new ByteArray;
result->mBytes.resize(len);

int i;
for(i = 0; i < len ; i++){
result->mBytes[i] = (unsigned char)data[i];
}

gEnv->ReleaseShortArrayElements(value, data, 0);

return result;
}



I found a recommendation to use a short array instead of a byte array,
since it converts straight to unsigned char, which ByteArray was already
using. It might be more efficient to read values into a short[] array
directly, though, rather than a byte[] array than converting to a short[]
array. I'm not sure? Either way its nice to see it working. Also, I guess
that Android doesn't support the Java 6 features of Arrays (or Java 6 at
all?) in older builds. Their issue tracker seemed to suggest that it was
fixed in Gingerbread, though I'm not sure if that's compiler related, or
runtime related, though I would guess the latter. That might explain why I
couldn't compile Arrays.copyOf originally.

Post by Hugh Sanderson
Hi Joshua,
I have also been revisiting android - and just started looking at
resources.
My current approach is to call back into the Java code to extract
the resources and then pass them to nme.
I also notice that the NDK r5 has an issue with libstdc++.a, which
shows up in android versions <= 2.1 - something to watch out for because
I wasted a day on that one.
Something that may also be of interest to you is that I've added a
templating system to NME to allow the construction of android projects
(and eventually others) with the "InstallTool.hx". This is still WIP,
but I hope to make it the easy way to get started with NME.
I will probably do an NME release in a few weeks - I would like to
get the webOS stuff merged in officially by then.
Hugh

Post by Joshua Granick
Hi everyone!
I have some news, for anyone who is interested.
I created a "Multi-Platform C++ Project" template for FlashDevelop. It
currently supports webOS, Android and Flash, but I'll probably add more
http://www.eclecticdesignstudio.com/code/flashdevelop/Multi-Platform%20C++%20Project.fdz
As a part of this process, I'm working on improving the Android target
-- specifically, right now, the file handling. On webOS (and many other
targets), supporting assets are extracted and are readily available to
the application. Unfortunately, Android keeps applications inside of an
APK package ... forever. This means that haXe has worked on Android up
till now (thanks, of course, to Hugh Sanderson), but you couldn't
access any supporting files unless they were downloaded or they were
accessed from external storage using absolute paths.
The current release of the template also includes an initial system for
fixing this problem. Supporting assets from your project are copied to
an internal storage directory on the phone the first time you run the
application. From that point on, haXe will redirect paths which don't
have a starting slash to look in this directory for files. That means
that if you reference "MyImage.png", it will work in Flash, webOS, and
now Android, thanks to some "magic" behind the scenes. It doesn't
support directories yet (as the internal storage directory Android
provides does not allow directories as far as I can tell), but I'm
going to try and work out a convention soon so that you can
"directories" will still work.
My updates to NME and HXCPP are available from
http://code.google.com/p/hxcpp-webos, which is what I'm building
against with this template. To compile for webOS you will need to
install the Palm SDK and CodeSourcery GNU toolchain for ARM. To compile
for Android, you will need to extract the Android SDK, Android NDK r5,
Apache Ant and install the Java JDK, set up a bunch of environment
variables and place your project in a directory without spaces, as Ant
is very finicky and is required to build the application. To compile
for Flash, of course, you just need FlashDevelop.
I can go into greater detail if anyone is interested. You'll also need
a webOS or Android phone, as the applications need to be run on a
device to work.