Woozle Wuzzle
More Pipe inconsistencies

The number of bytes written to a java.nio.channels.Pipe's sink at a time will determine the apparent size of the pipe (i.e. the number of bytes that can be written before the pipe blocks). If one byte is written at a time, the size of the pipe is 33012 for Windows or 1 (yes, 1) for Linux. If two bytes are written at a time the sizes are 33012 and 2. Four bytes gives 33012 and 4. Seeing a trend?

Windows will eventually give you the known value of 32768. With Linux, you need to specify more bytes than the known buffer size (4096) to get the buffer size.

The code used is as follows:

// create a Pipe and retrieve its sink and source
// NOTE:  the sink and source are SelectableChannels
final Pipe pipe = Pipe.open();
final WritableByteChannel sink = pipe.sink();
final ReadableByteChannel source = pipe.source();

// set the sink to non-blocking and create and register a write 
// Selector on it.  The Selector is used to determine when the sink 
// is "full".
// NOTE:  the cast is required since there is no common super-type
//        for selectable + readable / writable
((SelectableChannel)sink).configureBlocking(false/*non-blocking*/);
final Selector writeSelector = Selector.open();
((SelectableChannel)sink).register(writeSelector, SelectionKey.OP_WRITE);

// continue to write to the sink until it is "full"
// NOTE:  the sanity upper-bound is used to ensure that, in a remote
//        case, a sink is not infinite
final ByteBuffer writeBuffer = ByteBuffer.allocate(BUFFER_SIZE);
for(int i=0; i<BUFFER_SIZE; i++)
    writeBuffer.put((byte)(i & 0xFF)); // arbitrary
writeBuffer.flip();
boolean isInfinite = true;  // set to false if limit found on write
int numberOfBytesWritten = 0;
for(int i=0; i<UPPER_BOUND/*sanity*/; i++)
{
    // ensure that data can be written
    // NOTE:  selectNow() is used so that it does not block
    if(writeSelector.selectNow() > 0)
    {
        // clear the selected keys (required)
        writeSelector.selectedKeys().clear();

        // write the data 
        // NOTE:  the actual data written is arbitrary
        numberOfBytesWritten += sink.write(writeBuffer);
        writeBuffer.rewind();
    } else
    {
        // the sink is full.  Flag that a limit was found and break 
        // out of loop.
        isInfinite = false;
        break;
    }
}

UPPER_BOUND is some large number (say 10000) and BUFFER_SIZE has values as described above.

And, no, changing selectNow() to something like select(1000L/*1s*/) doesn't matter (for those worried about concurrency problems).

I should mention that Pipes javadoc does state platform inconsistencies and that this case falls under that unbrella. It's one thing to read a javadoc. It's another to actually see those inconsistencies first hand.

An interesting Linux tidbit: if the following code is added after the code listed above with a BUFFER_SIZE less than 4096 then numberOfBytesWritten will be non-zero.

// attempt to write more to the sink even though we shouldn't be 
// able to
writeBuffer.limit(1/*writes one byte*/);
numberOfBytesWritten = sink.write(writeBuffer);
writeBuffer.rewind();

It seems that the Linux write selector is lying to us. This is not the case on Windows.

Link-back to main entry: NIO and SSL.

Comments
Comment by Scott Lamb at July 7, 2005 03:07 PM

This might be an edge-triggered vs. level-triggered selection thing. (Does java.nio.channels.spi.SelectorProvider.getSelector() yield an epoll-based selector under Linux?) That's the behavior I would expect if so - write availability will not be reported again until a write actually fails with EWOULDBLOCK.

You could alter the test to ignore this difference by removing the selector - you don't need it. Just continue writing until a write fails; the pipe is then full.

In fact, you don't even need to loop. Just write something really big. If less than that is actually written (and the return value of write will tell you), that's the buffer size.

 

Comment by rgrzywinski at July 8, 2005 06:47 AM

I certainly appreciate your insights. When I get the opportunity I'll remove the selector and just watch the number of bytes written.

I should point out though that the point of this exercise was less about what each platform does and why but the fact that WORA (write once run anywhere) is a fantasy when it comes to pipes.

 

Post a comment













Remember personal info?






Creative Commons License Unless otherwise expressly stated, all original material of whatever nature created by Rob Grzywinski and included in this weblog and any related pages, including the weblog's archives, is licensed under a Creative Commons License.