Some things I have discovered while working at Delver (formerly Semingo) that have surprised me at the time.
At least in Sun’s JDK 1.6u4 (latest production quality release at the time of writing), the implementation of java.io.RandomAccessFile.writeInt(int) (and similar multi-byte read and write methods in the same class) works on a per byte basis. This is not specifically mentioned in the documentation, and is probably completely ok as per the specs and a majority of users, but I have found it not to be the behavior I wanted.
The implications of implementing multi-byte reads and writes as a sequence of single byte operations are plenty. I will demonstrate the two major ones: atomicity and performance.
Most file systems on most storage devices guarantee that operations on single bytes are atomic. Moreover, most (AFAIK, all) file systems on most devices I have worked on, guarantee that operations on 512 byte long sectors are atomic. File systems can also guarantee atomicity on operations on logical blocks (which are frequently 4Kb). Of course, unless some kind of transactional extension is available, the granularity is a single I/O operation. So if you write four bytes in a single write, you get an atomic four byte write operation. If you write four bytes as four separate single byte operations – some other file handle on the same file could have gotten off an I/O between your bytes. The demonstration code*:
public class tester {
public static void main(String[] args)
throws IOException, InterruptedException {
final boolean ATOMIC = false;
final int NUM_WRITES = 1000;
final String path = "foo.dat";
final int pattern1 = 0x55555555;
final int pattern2 = 0xAAAAAAAA;
RandomAccessFile raf = new RandomAccessFile(path, "rw");
raf.seek(0);
raf.writeInt(pattern1);
raf.close();
Runnable writer1 = new Runnable() {
public void run() {
System.out.println("writer1 start!");
byte[] bytes;
if (ATOMIC) {
bytes = new byte[4];
ByteBuffer buffer = ByteBuffer.wrap(bytes);
buffer.putInt(pattern1);
}
try {
RandomAccessFile raf = new RandomAccessFile(path, "rw");
for (int i = 0; i < NUM_WRITES; i++) {
raf.seek(0);
if (ATOMIC) {
raf.write(bytes);
}
else {
raf.writeInt(pattern1);
}
}
raf.close();
}
catch (Exception e) {
System.out.println(e.getMessage());
}
System.out.println("writer1 done!");
}
};
Runnable writer2 = new Runnable() {
public void run() {
System.out.println("writer2 start!");
byte[] bytes;
if (ATOMIC) {
bytes = new byte[4];
ByteBuffer buffer = ByteBuffer.wrap(bytes);
buffer.putInt(pattern2);
}
try {
RandomAccessFile raf = new RandomAccessFile(path, "rw");
for (int i = 0; i < NUM_WRITES; i++) {
raf.seek(0);
if (ATOMIC) {
raf.write(bytes);
}
else {
raf.writeInt(pattern2);
}
}
raf.close();
}
catch (Exception e) {
System.out.println(e.getMessage());
}
System.out.println("writer2 done!");
}
};
Runnable reader = new Runnable() {
public void run() {
System.out.println("reader start!");
try {
RandomAccessFile raf = new RandomAccessFile(path, "r");
byte[] bytes;
ByteBuffer buffer;
if (ATOMIC) {
bytes = new byte[4];
buffer = ByteBuffer.wrap(bytes);
}
int ans;
for (;;) {
raf.seek(0);
if (ATOMIC) {
raf.read(bytes, 0, 4);
buffer.position(0);
ans = buffer.getInt();
}
else {
ans = raf.readInt();
}
if (ans != pattern1 && ans != pattern2)
System.out.println("read " + Integer.toHexString(ans).toUpperCase());
}
}
catch (Exception e) {
System.out.println(e.getMessage());
}
}
};
Thread t1 = new Thread(null, reader, "reader");
Thread t2 = new Thread(null, writer1, "writer1");
Thread t3 = new Thread(null, writer2, "writer2");
t1.start();
t2.start();
t3.start();
t2.join();
t3.join();
t1.stop();
}
}
Two writer threads write either 0x55555555 or 0xAAAAAAAA to the first four bytes of a file. A reader thread constantly reads the first 4 bytes of same file. If the operations are atomic, the reader will always see either of the values, but never a scrambled value. Running the code (a few runs may be required) we see that we do get scrambled values. After we change the value of ATOMIC to true, we never get scrambled values.
As far as performance goes, we can use strace –f –v java tester on Linux and SysInternals' Process Monitor on Windows to see that with atomic==false, we get four syscalls for each operation, while with atomic==true we get only one. So you might say “my CPU can do many thousands of context switches per second, the number of syscalls will never be a bottleneck!” and let it go, but there is a case where this is important – namely, network file systems. Over an NFS mount, each syscall is eventually translated into a network packet. The operations block, so you get the full network latency for each byte instead of each int. a quick check:
public class tester {
public static void main(String[] args) throws IOException {
RandomAccessFile raf = new RandomAccessFile("foo.dat", "r");
byte[] bytes = new byte[4];
long t0, t1;
raf.seek(0);
t0 = java.lang.System.currentTimeMillis();
for (int i = 0; i < 100000; i++) {
raf.readInt();
}
t1 = java.lang.System.currentTimeMillis() - t0;
System.out.println("Reading 100,000 ints using readInt takes: " +
t1 + " milliseconds.");
raf.seek(0);
t0 = java.lang.System.currentTimeMillis();
for (int i = 0; i < 100000; i++) {
raf.read(bytes, 0, 4);
}
t1 = java.lang.System.currentTimeMillis() - t0;
System.out.println("Reading 100,000 ints using read(new byte[4],0,4) takes: " +
t1 + " milliseconds.");
raf.close();
}
}
And we will see that the time it takes four times as much time to write N ints using writeInt() rather than write(byte[]), and eight times as much for longs. Basically, you pay per I/O operation regardless of data size, as long as the data fits in a single packet, so you better "batch" your writes.
* - I know the code is ugly and one should never stop() a thread, but let's concentrate on the demonstration. Obviously no production code will ever look like this.
