The Wayback Machine - https://web.archive.org/web/20201102190244/https://github.com/rsocket/rsocket-java/issues/758
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rsocket - Reactive Cassandra less throughput compared to blocking http #758

Open
pckeyan opened this issue Mar 19, 2020 · 22 comments
Open

Rsocket - Reactive Cassandra less throughput compared to blocking http #758

pckeyan opened this issue Mar 19, 2020 · 22 comments
Assignees
Labels

Comments

@pckeyan
Copy link

@pckeyan pckeyan commented Mar 19, 2020

My use case revolves around retrieving the data from Cassandra using RSocket-Java in a non-blocking way. I use reactive cassandra template built on top of reactor-core. Throughput I see is very low in the range of 300 RPS compared to blocking http calls(800 RPS). If I remove Cassandra and use stub, I am getting close to 5000 rps. Can you please let me know whether RSocket-Java is not a perfect fit for above use cases? Sample non-blocking code is written similar to
https://github.com/OlegDokuka/rsocket-issue-717/tree/master/src/main/java/com/rsocket/examples 1” and Cassandra Connections are configured as mentioned here - “https://dzone.com/articles/reactive-streams-with-spring-data-cassandra”. Please help and advise.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Mar 19, 2020

Hey @pckeyan.

Can you please share your benchmark setup or a simple project which let me reproduce your issue | understand the setup behavior?

Cheers,
Oleh

@OlegDokuka OlegDokuka added the question label Mar 19, 2020
@pckeyan
Copy link
Author

@pckeyan pckeyan commented Mar 19, 2020

Hi @OlegDokuka,

I used directly into my business service, Let me try to peel it out. But from the environment standpoint, I used one Cassandra Docker container running on AWS EC2 - c5n.18xlarge. Same Backend Codebase to connect to Cassandra.

http Rest Call (API) - 800 TPS
Non Blocking Http (API) - 1600 TPS
RSocket(One Connection/Server) Connection Cache at Client - 300 TPS

I used the same code we tested on Issue 177 as mentioned above and I am using the Reactive Spring Cassandra with Datastax driver.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Mar 19, 2020

I'm asking because of different prefetch setup which may vary between reactive and blocking implementations in Spring.

Also, you point to a sample project, but it would be much better to have a simple client which can show RPS per a client

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Mar 19, 2020

Definitely let me put together one for you to check. But if I remove the Cassandra and mock the data, I am getting ~5000 TPS. Something is blocking to Cassandra.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Mar 19, 2020

So, if you look at my slides on reactive database access, and in particular on the slide 33, which shows how reactive Cassandra works underneath, you may get an idea, why it may be slow, especially if backpressure is applied

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Mar 19, 2020

Thank You, Let me go through the slide and will get back to you.

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Mar 20, 2020

@OlegDokuka I tried to modify the fetchSize it has no effect. Let me try to get a sample for you to check.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Mar 24, 2020

@pckeyan any updates?

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Mar 24, 2020

@OlegDokuka I tested with different parameters and I do not see any improvement. If you wish we can close this issue for now. I will reopen once I have a sample because I was testing this in my business application.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Mar 24, 2020

@pckeyan marking as need information. Feel free to provide a sample when it is ready

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Apr 7, 2020

@OlegDokuka I added you as.a collaborator to my private projects. Please check this readMe for instructions. https://github.com/pckeyan/rsocket-load-balancer/blob/master/README.md. Please let me know your feedback.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Apr 7, 2020

@pckeyan looking

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Apr 13, 2020

Hi @OlegDokuka, Any findings so far?

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Apr 23, 2020

@OlegDokuka I removed my load balancer component and used LoadBalancedRSocketMono in my Client and directly called Server has increased the throughput to 980 transactions per second. So 3x increase if I remove the component. Any insights or findings? But still this is less than non-blocking http rate.

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented Apr 23, 2020

Hey. I still was not able to look due to hard work on the upcoming release, however, I'm not recommending to use a load-balanced RSocket for now.

I hope I will give you some updates shortly

@OlegDokuka OlegDokuka self-assigned this Apr 23, 2020
@pckeyan
Copy link
Author

@pckeyan pckeyan commented Apr 23, 2020

Sure @OlegDokuka I Understand and will wait for your comments.

@pckeyan
Copy link
Author

@pckeyan pckeyan commented May 14, 2020

Just a gentle reminder @OlegDokuka

@pckeyan
Copy link
Author

@pckeyan pckeyan commented May 21, 2020

@OlegDokuka In the mean time, I tried with Postgresql. The RPS is at the same level of around 200-300. I tried with R2DBC but it is behaving weird. So could not complete the test with r2dbc. Can you please explain why the throughput is very low?

@OlegDokuka
Copy link
Member

@OlegDokuka OlegDokuka commented May 21, 2020

@pckeyan

do you mean HTTP vs RSocket on the pure JDBC is on the same level? If yes, I guess this can be simply DB limitation to have that number of requests per sec, I'm wondering what is the performance of the same without a network at all.

And is the second question related to jdbc vs r2dbc?

@pckeyan
Copy link
Author

@pckeyan pckeyan commented May 21, 2020

@OlegDokuka Either Cassandra or Postgresql, the throughput is same around 200 - 300 RPS. This is way less compared to HTTP (800 RPS), These testing is on same local device. The test uses JDBC. Cassandra states that it can support upto 80K RPS with One connection. For Postgresql, I increased the connections, but no success. Can you please test against a database and provide some metrics to compare?

R2dbc - When I tried with it, it is not even fetching data after a loop. I need to dig into it.

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Jun 1, 2020

@OlegDokuka Any benchmark available against a database backend? Can you please share a sample? Please let me know your recommendations.

@pckeyan
Copy link
Author

@pckeyan pckeyan commented Jun 22, 2020

@OlegDokuka Any help here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.