Given the vast amount of crap that we as programmers need to know these days (which is growing exponentially) I typically wrap unknowns into a black box and add them to my list to check out at a later time. java.lang.reflect.Proxy fell onto this list.

I typically associated a "magic" factor to anything that's in the core Java classes. Take for example how NIO's InterruptibleChannel interacts with Thread. I still want to know how Sun expects third parties to use the SPI to create other NIO implementations but that's another battle for another day.

I assumed incorrectly that Proxy had some magic tie-ins to the JVM that allowed it to masquerade as another class. Instead, Proxy actually goes the sane route. It generates a class (as a byte array) using reflection to inspect the specified interfaces. The class is then loaded using something very similar to ClassLoader.defineClass() (why it does not just use ClassLoader is not known but if there is one thing that I've learned over there years is that programmers love to keep secrets). By default, these generated files are not persisted to disk. You can set the system property sun.misc.ProxyGenerator.saveGeneratedFiles to true to save the files for examination (I do not know where they are saved to).

It's actually unfortunate that Proxy does not use JVM magic since then it might be possible to create a proxy to a class (rather than just interfaces) which would provide a truely useful generic proxy mechanism (facilitating AOP, for example).

I have voiced my concerns in the past about maintaining and growing code that uses AOP. I am not one to quickly shun a technology and never look back. I constantly review technologies.

I was reading this paper on recovery oriented computing (ROC) which is in line with my autonomic persuits and they mentioned FIG. FIG is fault injection in glibc. If you've ever tried to set up a networking test where write() would block long enough to see the impact without massively mutating your code, FIG (or its concept) would be something to look into.

When I read about FIG, the little 10W light blub in my head flicked to a dim glow. Use AOP to do fault injection!

Unit testing, even if the developer of the unit test is not the developer of the code being tested, is of limited use. No matter how hard we might try, unit tests tend to be parallel to the concerns of the code rather than orthogonal. In other words, unit tests tend to miss many of the bugs that aren't directly in line with the use / purpose of the code. For example, if you have a class that reads data from the network and writes it back then the unit tests are typically going to align themselves with that -- they will ensure that stuff in matches stuff out. But what about the orthogonal concerns, most of which involve difficult to trigger network concerns? (Yes, I'm horrible at giving motivational examples.) Attempting to write code that will cause exceptions to occur is likely to require a harness that's bigger than the original code and test combined. And how do you guarantee the validity of the harness? In other words, who tests the tests? The coast guard? I don't think so!

Using AOP (or just java.lang.reflect.Proxy) to inject faults will allow you to test the corner cases without a massive harness and without changes to your existing code or unit tests.

On a parallel note, this article brings up some interesting uses of AOP for exception handling.

For reasons still left unstated I have been doing research in autonomic computing. For those unfamiliar, here are some interesting links:

• The Vision of Autonomic Computing
• Autonomic Computing
• The role of ontologies in autonomic computing systems
• Bootstrapping Semantics in an Autonomic Computing System

I'm a constructor-based dependency injection kinda guy but with everyone always talking about setter-based dependency injection I started to question my approach. When Dave Thomas reminded me about class invariants I knew that my constructor-based approach was the right one.

From Wikipedia:

...a class does not allow use of all possible values for the state of the object, only those that are well-defined by the semantics of the intended use...

and

The main purpose of a constructor is to establish the invariant of the class, failing if the invariant isn't valid.

You know that you should use constructor-based dependency injection when the dependency answers the question is this a class invariant.

For those that know me, seeing a title like Things that really tick me off elicits a large groan and probably some wonder as to if there is enough storage available on the planet to house all of them. But I digress.

I have always hated the term "cookie" with regard to the state mechanism used with HTTP. It's a token people! Call it what it is and there's less opportunity for confusion. Well, it seems that Sun has one-upped that with muffin (no, really).

• "What's a muffin?"
• "It's a cookie."
• "What's a cookie?"
• "It's a token."
• "Why didn't they just call it a token then?"
• [sigh]

Whee!

I received a number of equiries to get performance numbers with larger numbers of clients. Unfortunately, I am limited to five client machines and one server machine. To increase the number of clients communicating with the server, I had to have multiple clients per machine. From the previous tests, a hypothesis can be made that the clients are either CPU or I/O bound. Adding more clients to each machine is not going to produce interesting results. The is essentially what was seen.

There are a total of eight configurations (three with SSL and four without). To simplify analysis, each graph contains the results either from the three SSL servers or the four non-SSL. Three cases were chosen:

• 1 client per machine (so that comparisons could be made back to previous results)
• 5 clients per machine
• 10 clients per machine

The same environment was used as in the previous tests.

The choices (besides one client per machine) was completely arbitrary. Numbers were chosen such that the tests would complete in a reasonsable amount of time.

Without SSL

With SSL

Analysis (see the other previous tests for more analysis):

• As expected, being bound (either CPU or I/O) has not yielded interesting results.
• For the non-SSL case with more than 1 client per machine, the performance for NIO, IO, Converted IO and Converted IO with Selector have effectively merged. I attribute this to "fill in the blanks". By increasing the number of clients per machine, any nearly-bounded resources were maxed out.
• The SSL case is known to be CPU bound (as the encryption and decryption are being done in software). It too had a "fill in the blanks"-style result (i.e. it is asymptotically reaching it's maximum value per machine).

A few tests were made to determine if the clients were CPU bound or IO bound. It could be guessed from previous results that they were IO bound (given the signature of the SSL results). Futher testing has shown this to be the case (e.g. all client echo validation was removed). Since the clients are IO bound, adding more clients to each machine would show no greater throughput to the server which is precisely what was observed in these tests.

Link-back to main entry: NIO and SSL.

In following with the previous tests, I performed a performance test of IO and Converted IO with SSL.

The testing environment is the same as the previous tests except that anonymous software SSL was enabled.

There are a total of three cases:

• IO Server, IO Client
• Converted IO Server, Converted IO Client
• Converted IO w/ Selector Server, Converted IO Client

"IO" uses the standard Java IO (from the java.net). "Converted IO" is an NIO wrapper to InputStream and OutputStream. The server with "Converted IO" uses a separate thread per client. The server with "Converted IO w/ Selector" uses a single thread for all clients and switches between them using an NIO Selector.

Analysis (see the previous tests for more analysis):

• Comparing with the non-SSL tests you can see that there is a significant (~50%) but expected loss in throughput.
• As hoped, the difference between Converted IO and Converted IO using a Selector decreased dramatically (from ~33% to ~7%) due to the overhead of SSL.
• Unfortunately, the difference between IO and Converted IO became more pronounced (from ~11% to ~25%). I do not have an explanation at this point and more investigation is needed.

A special thanks goes out to Carlo Segre for use of the cluster.

Link-back to main entry: NIO and SSL.

I took the opportunity to create a standard IO client and server and performed some changes / optimizations on the Converted IO. The source is available at the usual place.

The testing environment is the same as the previous tests.

There are a total of four cases:

• NIO Server, NIO Client
• IO Server, IO Client
• Converted IO Server, Converted IO Client
• Converted IO w/ Selector Server, Converted IO Client

"NIO" means that the component was created using only NIO. "IO" uses the standard Java IO (from the java.net). "Converted IO" is an NIO wrapper to InputStream and OutputStream. The server with "Converted IO" uses a separate thread per client. The server with "Converted IO w/ Selector" uses a single thread for all clients and switches between them using an NIO Selector.

Analysis (see the previous tests for more analysis):

• Standard IO performs slightly better (and with less variance) than NIO. This follows the standard claim that the use of a selector adds a bit of overhead (even more than that seen by using multiple threads). A future test should use many more clients to see if the overhead of a selector overcomes the overhead of context switching many threads.
• The clean up of the Converted IO appears to have created a positive result. The difference between ~10.3 MB/s (NIO), ~11.1 MB/s (IO) and ~9.8 MB/s (Converted IO) (~4% and ~11%, Converted IO to NIO and Converted IO to IO, respectively) is much better than the previous difference of ~20%.
• The Converted IO using a Selector has a similar trend as before: large variance in throughput and much lower throughput. More investigation is needed.

A special thanks goes out to Carlo Segre for use of the cluster.

Link-back to main entry: NIO and SSL.

I finally had an opportunity to perform some performance testing on the source I made available.

A few notes about the testing environment:

• It was not a closed environment. It is a cluster of machines running Linux (2.4.21) with MOSIX with a number of NFS mounted drives. There were a few other processes bouncing around the cluster at the time. This caused a number of dips in the throughput. All processes for this test were pinned to their respective machines.
• All machines are AMD Athlon(tm) XP 2500+ with 512MB RAM
• Sun JDK 1.4.2_05
• The network is 100Mb but it is not known if it is switched (highly doubtful)
• The echo server was started. The first connected to the server, followed by the second approximately seven second (arbitrary) later, followed by the third approximately seven seconds later, etc for all five clients.
• Each client transferred 500MB to the server before disconnecting.
• The data sent from the clients is random and in random sized batches of less than 4096 bytes.
• All data received by the server is immediately written back to the client.
• The clients wait for the server to echo back their batch of data and validate it before sending another.

As with most performance tests, the results must be interpreted correctly and cannot be taken at face value. You should not look at absolute values but instead you should look at relative values and trends. For "pure test" results the environment was not ideal but for a more "real world" feel for how applications behave, the environment was adequate.

There are a total of six cases:

• NIO Server, NIO Client
• NIO Server, Converted IO Client
• Converted IO Server, NIO Client
• Converted IO Server, Converted IO Client
• Converted IO w/ Selector Server, NIO Client
• Converted IO w/ Selector Server, Converted IO Client

"NIO" means that the component was created using only NIO. "Converted IO" is an NIO wrapper to InputStream and OutputStream. The server with "Converted IO" uses a separate thread per client. The server with "Converted IO w/ Selector" uses a single thread for all clients and switches between them using an NIO Selector.

All of the source for the clients and servers is available but the test harness is not available. It should be a trivial matter to create you own testing mechanism ideal for your environment.

Ideally, there should be a standard Java IO implementation as a control but unfortunately time is not on my side.

Analysis:

• The large dips in the graphs were caused by MOSIX and the processes that were running on the clusters.
• The NIO server performed slightly better than the Converted IO server as expected since the Converted IO server has an additional delay associated with the pipe that converts from standard IO to non-blocking IO. The difference between ~7.5 MB/s (Converted IO) and ~9.5 MB/s (NIO) (~20% difference) is significant and further work needs to be performed to tune Converted IO.
• The NIO client performed slightly better than the Converted IO client (for the same reasons as the server).
• The selector-based server to multiplex multiple clients performed worse than using a separate thread per client. Based on other results found on the internet, this is a typical result. The overhead of the selector is not mitigated with only a few clients.
• The fluctuation of the selector-based server was not expected. Further investigation is warranted.
• It appears that client 3 was not behaving consistently as is seen by its lower throughput and longer times. It is understandable that client 3's trend is not seen in the "Converted IO Server, NIO Client" case considering the large number of times that MOSIX interfered.

A special thanks goes out to Carlo Segre for use of the cluster.

Link-back to main entry: NIO and SSL.

Kris mentioned something that I hear all the time in regards to SOA's:

it's probably something you could implement using reliable JMS topics

(I'm pulling this slightly out of context but it's relevant regardless.) To developers that have kept their eyes open for at least part of the past few years, SOA's will be "yeah, so what?" or "I can just do / I am already doing that with [blah]". For example, an enterprise service bus (ESB) can be considered to be MOM (message oriented middleware).

Is "ESB" just another TLA (three letter acronym) that business people use to make themselves appear to be more intelligent? Maybe just a little. But really it's wrangling in a whole bunch of existing ideas (and some new ones like WS-*) and putting it under one umbrella. So if you get that deja-vu feeling you shouldn't feel uneasy.

Rather than spouting out more goop, just the links below to get you a feel for what's going on:

• The enterprise service bus--a ride you shouldn't miss
• SOA
• Understand Enterprise Service Bus scenarios and solutions in Service-Oriented Architecture

I recently attended a CJUG talk given by Dave Thomas of The Pragmatic Programmers regarding decoupling code. This was a very well put together talk that was able to reach both novice and advanced developers. Based on this talk I am seriously considering purchasing some of the books that they publish.

Thank you Dave for an excellent talk.

There are a number of cases where something needs to be done only in the case where an exception is thrown (checked or not). A first pass on this would look like:

    ...
    // allow the user to do something.  If it fails for any reason 
    // the error flag must be set so that further operations are not
    // attempted.
    try
    {
        doSomething();
    } catch(final Throwable t)
    {
        // some exception has been thrown; set the error flag.
        error = true;

        // continue the exception
        throw t;
    }
    ...

The problem with this is that unless the method signature includes throws Throwable you're out of luck. To circumvent this, I do the following:

    ...
    // allow the user to do something.  If it fails for any reason 
    // the error flag must be set so that further operations are not
    // attempted.
    boolean exceptionThrown = true; // set to false -only- if successful
    try
    {
        doSomething();

        // no exceptions were thrown
        exceptionThrown = false;
    } finally
    {
        // if there was an exception thrown (exceptionThrown will have
        // been set to false if an exception was -not- thrown) then set
        // the error flag.
        if(exceptionThrown)
            error = true;
        /* else -- there was no exception thrown */
    }
    ...

Are there any better techniques out there or is this acceptable?

Some quick notes on JMX and XML descriptors.

• MDF from BEA
• XMBean from JBoss
• Modeler from Jakarta

I don't see anything about standardizing the XML format which is very surprising. Personally, I think the XMBean looks the most palatable.

Currently there is only XDoclet support for XMBean. Modeler 1.1 mentions future XDoclet support. JMX is one of the few "Rob approved" XDoclet uses since it is not a "let's use a new technology everywhere it could possibly be applicable and more often than not, not applicable" case (we'll save that rant for another day).

Update (August 13^th)

My XDoclet statements above may be a bit misleading. The JMX XDoclet task will write out standard JMX interfaces (which is very convenient). It will also write out XMBean and JBoss <servicefile>-service.xml files (along with a few other things). I don't want to give the impression that there is no XDoclet support for standard MBeans. And yes, I'm confusing XDoclet tag support with XDoclet Ant task support, but to me and the way I use them, they're completely coupled and without one, the other is uninteresting.

As was alluded to in the main NIO and SSL entry, I have made convenience code available at:

    http://www.realityinteractive.com/software/oss/index.html

Refer to the release notes for information about what is been made available. If you have any comments or questions, just post a comment and I will respond as soon as possible.

It is not mentioned in the source or readme (I will rectify this shortly) that the intention for the conversion is specifically for long running clients. No thought has been given to "fast attack" clients or server (e.g. HTTP).

Performance results are available through the following links:

• NIO and Converted IO performance results
• More NIO and IO performance results
• SSL performance results
• More clients performance results

Link-back to main entry: NIO and SSL.

I am using a NIO CharsetDecoder to covert from bytes to chars in a UTF-8 environment. I received the following CoderResult error:

MALFORMED[1]

OK, that's helpful. After a little code splunking I determined that this means that the error is "malformed" (pretty obvious) and the length is "1" (not so obvious).

What's interesting about the CoderResult is that not only does it not fit any other paradigm used in the SDK but telling me the length of the erroneous input is, for all intents and purposes, useless. What would have been more helpful is to have included the position in the input buffer at which the malformed result occurred. Luckily CharsetDecoder.decode() advances the buffers as it reads so that you can use its current position as a guide (I should point out that this is mentioned in decode()'s javadoc).

Now I just need to determine why bytes that are supposedly UTF-8 have a value of -82. Uuugh!

Since my problem is clearly not on my end, I have added:

decoder.onMalformedInput(CodingErrorAction.REPLACE);

to circumvent the problem. This will use the CharsetDecoder's replacement value to replace any malformed characters.

There has been a lot of press around Aspect-Oriented Programming (AOP) and Software Development (AOSD). Every time I read an article such as this one the QA guy in me shudders uncontrollably. How can I possibly resolve the risk associated with AOP with the benefits that it is purported to provide? Also, given the inherent decoupled nature of AOP from the actual code (using, for example, deployment time AOP or byte-code based AOP), how can one effectively perform change managment?

Recently, I attended a JBoss discussion in hopes that it would quell some of my AOP concerns. Instead, the exact opposite occurred. Scott Stark managed to scare the bejesus out of me with transactions and protocol concerns being injected at deployment.

• How in the world can I test and certify a single deployment of my application if significant and complex components are deployment specific?
• Can I repackage this deployed application after testing and certification so that I'm guaranteed my clients will receive the same application?
• How can debug a stack trace that I get back from a client?
• How can I reproduce the client's environment in my test lab?

I know that these "advances" provided by AOP sound great to the trench developer (to which Mr. Stark was directing his discussion) that would normally have to struggle to create this functionality but there are clearly maintenance concerns with these approaches that have yet to be addressed.

Rickard Öberg voices some of my current concerns but unfortunately, like most developers, he limits it to "testing". Testing isn't the only concern; it's the full product life-cycle. I typically associate a 5 to 1 ratio of maintenance and debugging time to initial development time on any piece of complex code (where I will leave complex undefined here) throughout its lifecycle. If AOP is only addressing the "1" part of that ratio while increasing the "5" part then that's pretty crappy!

This thread (based on Rickard Öberg's blog entry) has some interesting insights. Do check other months for follow ups to the thread or related threads. [The AOSD links go down from time to time.]

People have spent a good deal of time claiming the programmatic benefits of AOP, but now it is time to start looking forward at debugging, maintaining, changing and growing AOP based code.

I'm interested in an SOP (service oriented platform) for some of the work that I'm currently doing. It would make my life much easier if there was a container with which I could register my services that would take care of lifecycle concerns.

After doing a little research to see what's going on out there I started looking at JBoss's org.jboss.system.Service interface as well as their org.jboss.deployment.Deployer. This is the Service interface:

/**
 * The Service interface.
 */
public interface Service
{
   /**
    * create the service, do expensive operations etc 
    */
   void create() throws Exception;
   
   /**
    * start the service, create is already called
    */
   void start() throws Exception;
   
   /**
    * stop the service
    */
   void stop();
   
   /**
    * destroy the service, tear down 
    */
   void destroy();
}

(The above code is available under the LGPL.)

Do you notice anything missing from the above interface? What's the threading contract?!? Should start() start its own thread if necessary? Does the container provide a thread to start() so that it can manage the lifecycle better and ensure that a faulty start() would not block the entire infrastructure? Etc. After a few minutes of code splunking I discovered that it's just simply undefined (the assumption is that it's the first case).

I'll spare everyone the rant and I will just say: Please document the complete contract on important interfaces. When you write javadocs, ask yourself what would someone need to know that has never seen the code. Attempt to place yourself into their shoes and you will likely end up with more useful javadocs.

In my persuit of a 1.4 NIO + SSL solution I had a momentary glimmer of hope in SSLServerSocket.getChannel(). This would allow me to registed an accept Selector to watch for connections and then I could use the SSL server socket to accept them. Unfortunately, the javadocs for getChannel() read:

A server socket will have a channel if, and only if, the channel 
itself was created via the ServerSocketChannel.open() method. 

This was confirmed with a trivial test. At first I thought that I was cut off at the knees. I now believe that I have been cut off at the torso.

I should mention that because of java.net.ServerSocket.accept():

IllegalBlockingModeException - if this socket has an associated 
channel, and the channel is in non-blocking mode.

I would have been screwed in any case but at least getting at the channel would have made me feel better.

Link-back to main entry: NIO and SSL.

This blog entry mentions serious performance concerns regarding try - finally blocks. I have yet to do any experimenting on my own but if this is true, then that sucks!

It boils the blood that the pair javax.net.ssl.SSLSocket and javax.net.ssl.SSLServerSocket as well as the pair javax.net.ssl.SSLSocketFactory and javax.net.ssl.SSLServerSocketFactory do not have common interfaces. You have to have separate and completely identical code to configure each socket type as well as set the enabled cipher suites.

These interface-type defects and inconsistencies are common throughout the package hierarchy. I was hoping that the next major release of Java would make a concerted effort to clean these up but it looks like that's not going to happen. Phooey!

Note to self: file RFE on Java bug parade for these interfaces.

Link-back to main entry: NIO and SSL.

I was attempting to use a vanilla SSL server and client socket (such as outlined in this article) but kept getting the dreaded:

javax.net.ssl.SSLException: No available certificate corresponds 
to the SSL cipher suites which are enabled.

The usual searches turned up a million posts about junk I already knew. The JSSE ref guide is great for people that already know what they're doing an is therefore self deprecating.

The long and short of it is that if you use a default SSLServerSocketFactory and create a socket then you must have an anonymous cipher suite installed. For example:

final SSLServerSocketFactory sslSocketFactory = 
    (SSLServerSocketFactory)SSLServerSocketFactory.getDefault();
final SSLServerSocket sslServerSocket = 
    (SSLServerSocket)sslSocketFactory.createServerSocket(port);

// use an anonymous cipher suite so that a KeyManager or TrustManager
// is not needed
// NOTE:  this assumes that the cipher suite is known.  A check -should-
//        be done first.
final String[] enabledCipherSuites = { "SSL_DH_anon_WITH_RC4_128_MD5" };
sslServerSocket.setEnabledCipherSuites(enabledCipherSuites);

A unless you do the same on the client side, you will receive the following:

javax.net.ssl.SSLHandshakeException: no cipher suites in common
javax.net.ssl.SSLHandshakeException: 
    Received fatal alert: handshake_failure

Link-back to main entry: NIO and SSL.

To aid in debugging JSSE (J2SDK 1.4 and greater) use:

-Djavax.net.debug=all

The usefulness of this cannot be expressed in mere words.

While looking for the current paradigms on storing passwords in Java I stumbled on this Security IQ Test. It's a bit thin but at least you can get a feel for if you know what's going on at a fundamental level. Perhaps the best part is the answers provided after you get your score.

This is also an interesting thread.

The question that I currently have is: what is the correct techique for obtaining passwords from a configuration file? Currently I store system passwords in an encrypted properties file. Do I have to read and decrypt the properties file each time I need the passwords? I don't think that just reading the passwords once on start makes sense (for the same reason that you use char[] over String for storing the password).

I was doing some work this morning with passwords stored as char arrays when I reverted to my C upbringing and wrote the following:

final char[] passwordCopy = new char[password.length];
System.arraycopy(password, 0, passwordCopy, 0, password.length);

I stopped myself and said: Hey! Why am I doing that when arrays have a convenient .clone() method on them?!? I rewrote the code to be the following:

final char[] passwordCopy = (char[])password.clone();

The QA side of me really likes the latter approach as it has a much lower risk associated with it (i.e. there are fewer ways to make a mistake), but the performance side said Whoa! Let's take a look at performance first!

I was going to write up a quick test but the lazy side of me went to Google first. This page has a nice test and performance numbers. The shocking result is System.arraycopy() vs. a for loop. Based on a few JVM's I tried (all on win32) I get the following normalized results:

          .clone():  2.26
System.arraycopy():  1.27
        for-loop():  1.00