Getting the output from a sub-process

A few times recently i’ve needed to capture the output from an OS specific command in Java.   It’s not terribly difficult, but this little example may save you some time.

package test;

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class Main {
    private final static String COMMAND = "/usr/bin/uptime";

    public static void main(String[] args) throws Exception {

       Process process = Runtime.getRuntime().exec(COMMAND);
       BufferedReader stdout =
            new BufferedReader(new InputStreamReader(process.getInputStream()));
       String line = "";
       while ((line=stdout.readLine())!=null) {
          System.out.println(line);
       }
       process.waitFor();  // unlikely we'd ever wait as we've got EOF on stdout
       System.out.println("Exit code = "+process.exitValue());
    }
}

And here is the output as you would expect :

 08:30:47 up 12 days, 23:53,  1 user,  load average: 1.87, 1.99, 2.07
Exit code = 0

The empire strikes back…. or does it?

I still chuckle from the completely non-scientific way of benchmarking Java versus Python in a previous posting. Anyway, to further rub salt into the wounds we can have a look at the results from the Debian language shootout .   Maybe I should just present the results without commentary …. nah where would be the fun in that 🙂

The graph is indicating that yes, java outperforms python in just about all of the benchmarks.  Sure the memory usage of the java application is higher and I concede that that is one of the major drawbacks of the java platform.  If the bug discussed in this prior post was actually fixed then there would be little to whinge about – but i’m sure people would find something, because people love whinging, me included  🙂

Interestingly here are the results of a Java versus Perl comparison as well.

What! Java is faster than perl, how can this be ! /me removes tongue from cheek.

And for those who still tell me that php is faster than Java…. well let’s see….

Oh dear, poor php.

This sort of data all ways makes me chuckle.  The only bit that makes me uncomfortable is that I didn’t run the tests myself and I haven’t looked at the code contained in the benchmarks.   This is something that I want to do agains the openjdk code base as i’m doing research on the hotspot compiler and this code provides for an interesting test bed.

The 20 cent summary is if you think Python is faster than Java then you really need to have a look at the data.  If you want Python to be faster than Java then analyze the code, explore the Python code base and see if you can make it better – take that as a challenge as everyone benefits from the results.

Yes, I use Python as well as Perl and Java, make them all quicker – see if I care 😛

System.exit(0) Universe Terminated

I’m always interested in cool things running Java, mostly because i’m a nerd, but also because it annoys people when I tell them how cool Java is and let’s face it – that’s a fair bit of fun in itself 🙂

Anyway,  the Large Hadron Collider uses Linux, Java and other platforms with both Linux and Java playing a major role in the control systems.   So, how does Java lead to the end of the universe. Well, of course that is a ‘tongue-in-cheek’ view of the legal challenge raised against the LHC where it is speculated that the LHC could create a variety of conditions leading to the end of us all (read more over at wikipedia).   Let’s hope no one leaves a stray System.exit() in the code and accidentally destroy the universe – that really would be a poor programming practice  🙂

The list of really cool practical applications of Java and Linux technology continues to grow.  It seems to me that the LHC is possibly the biggest machine ever created – are there others bigger?

building netbeans from source on ubuntu

Everyone needs a hobby.   Clearly I don’t have one.

I’ve decided that I want the latest and greatest set of features (and bugs) for netbeans so i’m installing netbeans from the code repository and building it on my currently preferred platform (ubuntu 8.04 – yes I know it’s a beta which is only more proof that I need a hobby).

I prefer to do all my bleeding edge test builds on the ‘server’ version of ubuntu – why?  I like small footprint build systems that only have the required packages to satisfy my build.  I guess I just don’t like packages sitting there looking at me and this way I know exactly what’s required to make it work on my primary desktop system later.  ie. I can do everything in an experimental manner in the VM and develop detailed documentation of what is required to implement it on other systems.

So, going from the top, I’ve created a brand new hardy server as a VM so that I have a clear view of everything that is required to get the job done.   If your system already has some of the pre-req software loaded then great.

netbeans uses mercurial so to get the source code you need that installed

~$ sudo apt-get install mercurial
~$ mkdir netbeans && cd netbeans
~/netbeans$ hg clone http://hg.netbeans.org/main/
destination directory: main
requesting all changes
adding changesets
adding manifests
adding file changes
added 78686 changesets with 365473 changes to 80849 files
77968 files updated, 0 files merged, 0 files removed, 0 files unresolved

note: to build netbeans we need a few tools, first of them is ant which will drag on a number of dependencies.

sudo apt-get install ant ant-optional
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following extra packages will be installed:
.
.
.

and given that there are issues using gcj  to build netbeans (and I really can’t be bothered looking closer to tell you the truth) I suggest not bothering and just install the sun jdk (note: java5 is required and anything higher wont work at the time of writing this).  Also, don’t forget the ant-optional otherwise you will get complaints about regexp during compilation.

~netbeans$ sudo apt-get install sun-java5-jdk

Add the following to your ~/.nbbuild.properties file

nbjdk.home=/usr/lib/jvm/java-1.5.0-sun
build.compiler=extJavac
javac.compilerargs=-J-Xmx512m

and the following to your ~/.antrc file

ANT_OPTS=-Xmx384m

~/netbeans$ ant -f main/build.xml

and hey presto netbeans should be built (it will take some time).   Now as I’m ssh -X into this server I also need to ensure the xauth package is installed

~/netbeans$ sudo apt-get install xauth

and then I can try out the newly compiled netbeans using

~/netbeans$ ant -f main/build.xml tryme

which should see the netbeans splash screen appear and I can start using the latest development version of netbeans.   Of course, this means you have to deal with any bugs, performance problems and mysterious quirks yourself.   Join the mailing lists and enjoy.

Now that I have a very good understanding of what is required to get netbeans built from source I am confident that doing so will not interfere with anything else running on my primary development system.  Using the results of the above exploration it should be a simple matter of matching the package requirements, rsync-ing over the downloaded repository and you’re good to go.

watching junk with java

Ok,  I had to admit this press release did catch my eye.  To quote :

“The Eglin Space Surveillance Radar (FPS-85), which schedules and tracks catalogued space objects, is currently undergoing a complex modernization process to replace legacy mission-critical components built from one-of-a-kind equipment and custom software. The Sun Java Real-Time System, a high-level development platform for creating applications that require unprecedented execution predictability, will enable ITT to run the new solution using Java technology on Solaris 10 OS and standard hardware.”

As a Linux nerd it’s a shame that it’s Solaris 10 and not one of the RT variants of Linux but any technology that controls a 1250 ton radar array that can track basketball sized objects 22000 nautical miles away is pretty cool in my book 🙂

It is disappointing that the Sun Java Real-Time system isn’t opensource, but it least it’s good to note that it is available for real time linux variants such as SLERT.

Oh java…. what are you doing?

Lots of people complain about java performance (and you know who you are 😉 )

But if you ask them if they’ve actually looked to see what their application is doing many say ‘nope’.  This is of course frustrating for everyone.

Mostly this happens because people just don’t know how to investigate how their application is performing and in the past it’s been difficult to do.  It’s also possible that they’d rather just complain rather than fixing things – much like me previously whinging about the shortcomings of HTML5 😉

However, if you’re running a  JDK (1.5.0+) which implements the Java Management Extensions then you can use tools such as jconsole or jstat to gather information about the execution profile of your application with a goal to correcting any anti-social behaviours.

Let me give you an example of using these tools on a well known Java application – Tomcat.To utilize these tools you have to turn on the management interface which allows the application VM to be monitored from the local network or across the network ( yes, you can secure the management connection )

So, modifying the Tomcat startup to include -Dcom.sun.management.jmxremote – start up Tomcat and then you can use jconsole <pid>  to get a look at what the JVM is doing for your application.

Attached are some pictures. Of course now that you have some better diagnostic data you can try to understand what may be causing it, possibly tune it better, produce better bug reports for the developers or even use them to assist you in fixing the application yourself – it’s ok, you can still say Java sucks but at least now you’ll have some evidence to support you.

snapshot1.png

jconsole window showing some performance statistics.

snapshot2.png

Some basic performance statistics.

snapshot3.png

snapshot4.png

Of course, if you want to collect trending information over a longer period then make sure you have a look at jstat.

Just the JVM m’aam

Previously I was looking at Java versus Python performance (here) and I was quite surprised (well not surprised, but more intruiged) by the JVM startup speed so I thought i’d have a bit more of a look to satisfy my curiosity. I don’t have all the answers so don’t hassle me, but what I started doing was using the JNI to create a JVM from a simple C command as opposed to using the java command to do the work and invoke the classloader etc. Anyway, this is how you do it in case you’re suitably bored and want to have a go 🙂

Here is the ‘c’ code which is a slightly modified version from the JNI reference manual:


#include <jni.h>
#include <stdlib.h>

#define USER_CLASSPATH “.” /* where to find Prog.class */

int main() {

JNIEnv *env;
JavaVM *jvm;
jint res;
jclass cls;
jmethodID mid;
jclass stringClass;
jobjectArray args;

JavaVMInitArgs vm_args;
JavaVMOption options[1];
options[0].optionString =
“-Djava.class.path=” USER_CLASSPATH;
vm_args.version = JNI_VERSION_1_6;
vm_args.options = options;
vm_args.nOptions = 1;
vm_args.ignoreUnrecognized = JNI_TRUE;
/* Create the Java VM */
res = JNI_CreateJavaVM(&jvm, (void**)&env, &vm_args);

if (res < 0) {
fprintf(stderr, “Can’t create Java VM\n”);
exit(1);
}
exit(0); /* here */
cls = (*env)->FindClass(env, “Prog”);
if (cls == NULL) {
goto destroy;
}

mid = (*env)->GetStaticMethodID(env, cls, “main”,
“([Ljava/lang/String;)V”);
if (mid == NULL) {
goto destroy;
}
args = NULL;
(*env)->CallStaticVoidMethod(env, cls, mid, args);

destroy:
if ((*env)->ExceptionOccurred(env)) {
(*env)->ExceptionDescribe(env);
}
(*jvm)->DestroyJavaVM(jvm);
}
Additionally here is the Java code for the class executed by the JVM when instantiated.

public class Prog {
public static void main(String[] args) {
System.out.println("Hello World");
}
}

We wont talk about how wordpress has formatted the above – oh well you get the idea 😉

Get things compiled and linked

javac Prog.java
gcc -I /usr/local/jdk1.7.0/include/ -I /usr/local/jdk1.7.0/include/linux/ -L /usr/local/jdk1.7.0/jre/lib/i386 -L /usr/local/jdk1.7.0/jre/lib/i386/client -ljava -ljvm -lverify -o invoke invoke.c

noting that i’m using an openjdk code drop

export LD_LIBRARY_PATH=/usr/local/jdk1.7.0/jre/lib/i386:/usr/local/jdk1.7.0/jre/lib/i386/client

and you should be able to ./invoke to check it out. If you review the invoke.c code above you’ll see an exit(0) statement with comment ‘here’. If you leave that in and time the execution you will see that the JVM instantiation isn’t that bad which is more in line with the empty python example mentioned in the previous blog entry. Removing the exit should see the JVM instantiated and the class file executed.

Interestingly just such a test reveals a far better comparion for VM instantiation with nothing to execute than the previous test. In this case the ratio now becomes :

Test Ratio (Java/Python)
Interpreter Init 1.6