runtime byte code update in java and performance

Question

I have some very high performance critical code, which at a high level takes a bunch of decision by doing some floating point comparisons or simple predicates. The code can be written in the form of a decision tree. Basically each nodes does a predicate check and then decides which path to take until we reach some leaf node.

To improve performance instead of having this decision tree , i generate some if-else code blocks that gets compiled when programs run (basically i generate code for the entire decision tree since i know the predicates beforehand). This does improve performance quite a bit. Now the next optimization i want to do is modify the code at runtime i.e in the old world of no code generation and having a decision trees with nodes , i could have copied the tree and short circuited/jumped over some nodes , hence compressing the tree and made overall computation faster. But in the generated code world , is there any tool that can achieve the same by modifying my generated if-else code based on some partial runtime data available during computation. Also what are the performance implication of modifying run time code.

Answer 1

First of all, the JVM doesn't allow you to modify running code. What it does allow you to do is to generate new code at runtime and load it.

• You can do this by generating source code and compiling it, as you have been doing.
• You can do this at the bytecode level; e.g. by using something like BCEL; see https://commons.apache.org/proper/commons-bcel/manual/bcel-api.html.

In addition, there is an Oracle sponsored project (GraalVM) where (if I understand things correctly) you can produce (Truffle) ASTs and have the framework take care of the code generation. (It seems to be experimental at the moment, and it may not yet be ready for production use.).

---

Also what are the performance implication of modifying run time code.

That is the $65,536 question!

One implication is that each time you modify code (by regenerating it), the method must be reloaded, and (for a conventional JVM) go through the interpretation and JIT compilation phases again. And potentially de-optimization / re-optimization of other code that depends on the modified code.

That won't be cheap. I'm guessing in the order of tens or hundreds of thousands of native instructions per method, each time you do this.

---

Now the next optimization i want to do is modify the code at runtime ...

OK. So I think you would actually be better off doing more sophisticated code generation.

But also, bear in mind that the JIT compiler will be optimizing your generated bytecodes, doing things like branch-prediction based on stats gathered while the method was being interpreted (prior to JIT compilation).

I would also advise trying out some of your optimizations by hand and benchmarking them before you go to the effort of implementing an optimizer of your own.

Answer 2

This is what I use my https://github.com/OpenHFT/Java-Runtime-Compiler for

e.g.

// dynamically you can call
String className = "mypackage.MyClass";
String javaCode = "package mypackage;\n" +
                 "public class MyClass implements Runnable {\n" +
                 "    public void run() {\n" +
                 "        System.out.println(\"Hello World\");\n" +
                 "    }\n" +
                 "}\n";
Class aClass = CompilerUtils.CACHED_COMPILER.loadFromJava(className, javaCode);
Runnable runner = (Runnable) aClass.newInstance();
runner.run();

I suggest using a known interface which doesn't change you can call and have your generated code implement the interface.

This doesn't support reloading a class but you can generate a new class name each time or use a different class loader each time.