Java Python Integration

Frankly most ways to somehow run Python directly from within JVM don't work. They are either not-quite-compatible (new release of your third party library can use python 2.6 features and will not work with Jython 2.5) or hacky (it will break with cryptic JVM stacktrace not really leading to solution).

My preferred way to integrate the two would use RPC. XML RPC is not a bad choice here, if you have moderate amounts of data. It is pretty well supported — Python has it in its standard library. Java libraries are also easy to find. Now depending on your setup either Java or Python part would be a server accepting connection from other language.

A less popular but worth considering alternative way to do RPCs is Google protobuffers, which have 2/3 of support for nice rpc. You just need to provide your transport layer. Not that much work and the convenience of writing is reasonable.

Another option is to write a C wrapper around that pieces of Python functionality that you need to expose to Java and use it via JVM native plugins. You can ease the pain by going with SWIG SWIG.

Essentially in your case it works like that:

1. Create a SWIG interface for all method calls from Java to C++.
2. Create C/C++ code that will receive your calls and internally call python interpreter with right params.
3. Convert response you get from python and send it via swig back to your Java code.

This solution is fairly complex, a bit of an overkill in most cases. Still it is worth doing if you (for some reason) cannot afford RPCs. RPC still would be my preferred choice, though.

Many years later, just to add an option which is more popular these days...

If you need CPython functionality, py4j is a good option. py4j has seen seen frequent updates in 2016 2017 2018 2019 2020 and has gained some popularity, because it is used e.g. by Apache Spark to achieve CPython interoperability.

The best solutions, is to use Python programs throw REST API. You define your services and call them. You perhaps need to learn some new modules. But you will be more flexible for futures changes.

Here a small list of use full modules for this purpose: Python modules

• Flask
• Flask-SQLAlchemy
• Flask-Restful
• SQlite3
• Jsonify

Java modules (for calling rest api) Jersey or Apache CXF

You will need a small Learning curve, but later you will get more productivity and modularity and even elasticity...

My other thought is to use JNI to communicate with the C bindings for Python.

I like very much JNA:

JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. This functionality is comparable to Windows' Platform/Invoke and Python's ctypes. Access is dynamic at runtime without code generation.

My 0.02$ :)

You could use a messaging service like ActiveMQ. It has both Python and Java support. This way, you can leave the complicated JNI or C bindings as they are and deal solely with what I consider a simple interface. Moreover, when the library gets updated, you don't need to change much, if anything.

Have you considered running Jython on the Java VM?

I've investigated a similar setup with JNI. Maybe this will help if haven't seen it yet:

http://wiki.cacr.caltech.edu/danse/index.php/Communication_between_Java_and_Python

http://jpe.sourceforge.net/

These are some of the tools which make it easier to bridge the gap between Python and Java:

1.Jython Python implemented in Java

2.JPype Allows Python to run java commands

3.Jepp Java embedded Python

4.JCC a C++ code generator for calling Java from C++/Python

5.Javabridge a package for running and interacting with the JVM from CPython

6.py4j Allows Python to run java commands.

7.voc Part of BeeWare suite. Converts python code to Java bytecode.

8.p2j Converts Python code to Java. No longer developed.

If you can get your Python code to work in Jython, then you should be able to use that to call it from Java:

• http://jython.sourceforge.net/cgi-bin/faqw.py?req=show&file=faq06.001.htp

I also think that run command line in the Java wouldn't by bad practice (stackoverflow question here). Potentially share the data through some database.

I like the way to connect two apps via the bash pipe, but I have not practice in this, so I'm wondering how difficult is to write logic to handle this on both python/java sides.

Or other productive way could be to use the Remote Procedure Call (RPC) which supports procedural programming. Using RPC you can invokes methods in shared environments. As an example you can call a function in a remote machine from the local computer using RPC. We can define RPC as a communication type in distributed systems. (mentioned above by Marcin)

Or, very naive way would to be to communicate by the common file. But for sake of simplicity and speed, my vote is to use the shared database x rest API x socket communication.

Also I like the XML RPC as Marcin wrote.

I would like to recommend to avoid any complication to run Python under JVM or C++ binding. Better to use today trends which are obviously web technologies.

As a shared database the MongoDB may be good solution or even better the Redis as in memory database.

Use pipes to communicate with python(a python script calls python library) via subprocesses, the whole process is like java<-> Pipes <-> py.

If JNI, you should be familiar with python's bindings(not recommended) and compiled files like *.so to work with java. The whole process is like py -> c -> .so/.pyd -> JNI -> jar.

Here is a good practice of stdio. https://github.com/JULIELab/java-stdio-ipc