Sending data over TCP between Java and C++

Question

I have an application that constantly fetches input in Java from a frame buffer and sends it over the wire to a C++ application.

I am having issues on the receiver side. I am trying to send one packet (or at least let TCP reconstruct it as a single packet) for each frame buffer. The frame buffer is getting splitted into multiple small packets on the receiving end.

The Java code is the following:

OutputStream os = clientSocket.getOutputStream();
FileInputStream fos = new FileInputStream("fb");
while (true) {
    int nb = fos.read(buff);
    if (nb <= 0)
        break;
    os.write(buff, 0, nb);
}
fos.close();
os.close();

On the client side, I am trying to read the same amount. nb and size are the same value here:

n = read(sockfd, buffer, size);
while (n > 0) {
    // We have a new frame
    fprintf(stderr, "New frame: %d\n", n);

    n = read(sockfd, buffer, size);
}

The value of n that is printed is much smaller than size. It receives a lot of packets when I would hope the return from read() would be a packet of the size 'size' (or nb).

Does anyone please know why is that please?

Thank you very much for your help!

Michael Dillon · Accepted Answer · 2011-10-14 03:56:21Z

2

You don't have control over TCP frame sizes. Network devices can and do fragment packets so that you will receive more packets than you sent.

The word "stream" is used for a very good reason. You are getting a stream of bytes but there is no framing unless the sender and receiver application impose framing on the stream by including frame markers in the data. One common way used by a lot of TCP protocols is to use \r\n as a frame marker. This means that the receiver collects incoming data but does not process it until it encounters a frame marker. At that point, the receiver processes one frame and then goes on looking for the next frame marker.

One thing that you should consider is using a cross-platform cross-language library like ZeroMQ to handle framing for you.

answered Oct 14, 2011 at 3:56

Michael Dillon

32.5k7 gold badges76 silver badges107 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jary Over a year ago

Oh I see. I thought that fragmentation happened at the lower level, and not the application (ie: write() may fragment the data at the lower level, but read() would return data only when defragmented by the lower layer. I don't think \r\n will work in my case, as I am using raw data. Would it? I can just keep a counter, and when I reach 'size' then I know it is a frame. I'll look into ZeroMQ. Thanks a lot for your help.

Michael Dillon Over a year ago

In fact, ZeroMQ does use a byte count and that would work fine. Have a look at the memcache protocol for one that uses both \r\n and byte count depending on what mode it is in. code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

Collectives™ on Stack Overflow

Sending data over TCP between Java and C++

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related