Java Daemonization with posix_spawn(2) 24 Jan 2012

The traditional way of daemonizing a process involves forking the process and daemonizing it from the current running state. This doesn’t work so well in Java because the JVM relies on several additional worker threads, and fork only keeps the thread calling fork. So, basically, you need to start a new JVM from scratch to create a child process.

The traditional way of launching a new program is to fork and exec the program you wish to start. Sadly, this also fails on Java because the calls to fork and exec are seperate, non-atomic (in the platonic sense, not the JMM sense) operations. There is no guarantee that the exec will be reached, or that the memory state of the JVM will even be sound when the exec is reached, because you could be in the middle of a garbage collection and pointers could be all over. In practice, this would happen exceptionally rarely, at least. Charles Nutter has talked about this problem in JRuby as well.

The best method I know of to daemonize a Java process, then, is to use posix_spawn. This function launches a new process based on the program image passed to it – it is like fork and exec in one system call. Over the weekend I took some time to put together a small library to do this, Gressil using jnr-ffi. It turned out that the hardest part, by far, was reconstructing the full ARGV array.

On linux, with ProcFS it is pretty straightforward. On OS X it is supposed to be possible but all the code samples I have found seem to be based on that snippet, and they look forward into the “String area” which usually has ARGV laid out in the correct order, but sometimes doesn’t. Trying to look backward from the pointer returned by the sysctl would be straightforward in C (decrement the pointer) but jnr-ffi seems to copy the memory space into a Java byte array, so there is no going backward. Attempting to guess how far back to jump the pointer to get a new block of memory usually segfaults for me :-) Anyone with thoughts, a version based on the above reference is in Gressil, MacARGVFinder.

Anyway, it turns out you can get most of this information from the Sun JVM. The only dodgy part of that is the program arguments (the stuff passed into Java’s public static void main(String[] args) method. Luckily, those are passed to main, so we can just ask for them in the library.

Given that, you can use Gressil to daemonize Java processes via re-launching like so:

package org.skife.gressil.examples;

import org.skife.gressil.Daemon;

import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.Date;

import static org.skife.gressil.Daemon.remoteDebugOnPort;

public class ChattyDaemon
{
    public static void main(String[] args) throws IOException
    {
        new Daemon().withMainArgs(args)
                    .withPidFile(new File("/tmp/chatty.pid"))
                    .withStdout(new File("/tmp/chatty.out"))
                    .withExtraMainArgs("hello", "world,", "how are you?")
                    .withExtraJvmArgs(remoteDebugOnPort(5005))
                    .daemonize();

        while (!Thread.currentThread().isInterrupted()) {
            System.out.println(new Date() + " " + Arrays.toString(args));
            try {
                Thread.sleep(1000);
            }
            catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
    }
}

In the parent process the call to Daemon#daemonize() will call System.exit(), in the child process it will return normally.

The child process, in this case, will also listen for a Java debugger to attach on port 5005. It will attach stdout to /tmp/chatty.out, and stdin and stderr will default to /dev/null (which stdout would also attach to by default if it were not specified).

I cut a 0.0.1 release which should be synced in maven central by now as org.skife.gressil:gressil:0.0.1 by now. It should work pretty much anywhere with a posix-compliant libc, but I have only extensively tested on OS X.