doc: benchmark results is not so good,what is the problem,help

[xianshun163]

Issue description

I have see the doc about the benchmark，the result is good： https://apisix.apache.org/docs/apisix/benchmark/
I try it myself， but it is not so good，any one who can tell me how to optimize it.
Here is my env:
I have three host, and they are all 4c8g
host13: I deploy apisix2.11.0 use the source code
host14: I use it to run wrk, the HTTP benchmarking tool
host15: I deploy a nginx, as a upstream, and k8s, I also deploy a apisix as a pod in this host

I have not use any plugins, and only config the nginx as a upstream of the apisix. And the nginx return a string directory

the test result like this:
wrk directory call the nginx, qps is 30217
wrk call the host13 apisix, qps is 11003.17
wrk call the host15 apisix, qps is 8002.80
here is the wrk detail:
[root@k8s_test1 data]# ./wrk -t200 -c5000 -d30s http://172.16.11.15:1980/hello
Running 30s test @ http://172.16.11.15:1980/hello
200 threads and 5000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 215.86ms 235.20ms 2.00s 88.07%
Req/Sec 153.61 98.81 1.50k 71.78%
912979 requests in 30.21s, 302.13MB read
Socket errors: connect 0, read 4, write 163, timeout 1100
Requests/sec: 30217.23
Transfer/sec: 10.00MB

[root@k8s_test1 data]# ./wrk -t200 -c5000 -d30s http://172.16.11.13:9080/hello
Running 30s test @ http://172.16.11.13:9080/hello
200 threads and 5000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 449.90ms 298.54ms 2.00s 81.53%
Req/Sec 55.02 32.43 420.00 67.66%
332622 requests in 30.23s, 110.40MB read
Socket errors: connect 0, read 18, write 263, timeout 3732
Non-2xx or 3xx responses: 548
Requests/sec: 11003.17
Transfer/sec: 3.65MB

[root@k8s_test1 data]# ./wrk -t200 -c5000 -d30s http://172.16.11.15:30980/hello
Running 30s test @ http://172.16.11.15:30980/hello
200 threads and 5000 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 109.96ms 89.03ms 2.00s 93.34%
Req/Sec 100.90 92.24 717.00 76.64%
242268 requests in 30.27s, 80.39MB read
Socket errors: connect 0, read 2712, write 14878, timeout 2282
Requests/sec: 8002.80
Transfer/sec: 2.66MB

Environment

• apisix version (cmd: apisix version):2.11.0
• OS (cmd: uname -a):centos 7
• OpenResty / Nginx version (cmd: nginx -V or openresty -V):
• etcd version, if have (cmd: run curl http://127.0.0.1:9090/v1/server_info to get the info from server-info API):
• apisix-dashboard version, if have:
• the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
• luarocks version, if the issue is about installation (cmd: luarocks --version):

[juzhiyuan]

Hi @xianshun163, here has one benchmark case from @membphis https://gist.github.com/membphis/137db97a4bf64d3653aa42f3e016bd01

For your question, let's wait for more replies.

[tzssangglass]

ref: #5950

hi, @laughingyear It seems to be the same confusion and we can merge the discussions here.

[membphis]

@xianshun163 ./wrk -t200 -c5000 -d30s http://172.16.11.13:9080/hello

if you want to run the benchmark with 5000 connections, you need to update the https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L238 .

Connections exceeding this number will become short connections.

more doc: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive

[xianshun163]

@xianshun163 ./wrk -t200 -c5000 -d30s http://172.16.11.13:9080/hello

if you want to run the benchmark with 5000 connections, you need to update the https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L238 .

Connections exceeding this number will become short connections.

more doc: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive

ok, I get it, thanks

[xianshun163]

I close the issue now， thanks every one。

[juzhiyuan]

Hi @xianshun163, may I know if you like to add @membphis's comment to the Benchmark page? then others could also see it :)

[xianshun163]

@juzhiyuan ok, I can do it.

[juzhiyuan]

Assign to you :)

[xianshun163]

@xianshun163 ./wrk -t200 -c5000 -d30s http://172.16.11.13:9080/hello

if you want to run the benchmark with 5000 connections, you need to update the https://github.com/apache/apisix/blob/master/conf/config-default.yaml#L238 .

Connections exceeding this number will become short connections.

more doc: http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive

@membphis , do you know how to get the "Flame diagram",?
I use openresty-systemtap-toolkit to do it , but I can not get the right lua parameter, the err like this:

[root@k8s_test1 xs]# ./openresty-systemtap-toolkit/ngx-sample-lua-bt -p 44121 --luajit51 -t 20 > tmp.bt
Unknown option: luajit51
Usage:
ngx-sample-lua-bt [optoins]

Options:
-a Pass extra arguments to the stap utility.
-d Dump out the systemtap script source.
-h Print this usage.
-l Only output most frenquent backtrace samples.
(Default to 1024)
--lua51 Specify that the Nginx is using the standard Lua 5.1.
interpreter.
--luajit20 Specify that the Nginx is using LuaJIT 2.0.
-p Specify the nginx process pid.
-t Specify the number of seconds for sampling.

Examples:
ngx-sample-lua-bt --lua51 -p 12345 -t 10
ngx-sample-lua-bt --luajit20 -p 12345 -t 5 -a '-DMAXACTION=100000'
[root@k8s_test1 xs]# ./openresty-systemtap-toolkit/ngx-sample-lua-bt -p 44121 --lua51 -t 20 > tmp.bt
The --lua51 option is specified but seen the LuaJIT library: /usr/local/openresty/luajit/lib/libluajit-5.1.so.2.1.0

[tzssangglass]

You can refer to this PR: #6136

[HelloBug0]

Does this PR mean apisix can generate flame diagrm audomatically? How can I trigger it? @tzssangglass

[tzssangglass]

Does this PR mean apisix can generate flame diagrm audomatically? How can I trigger it? @tzssangglass

this PR is in progress

[HelloBug0]

Does this PR mean apisix can generate flame diagrm audomatically? How can I trigger it? @tzssangglass

this PR is in progress

I see. I will follow it.

[yunwei37]

Hi there! Can I take this issue to improve the Benchmark page?

[laughingyear]

Alright 发自我的iPhone

…

------------------ Original ------------------ From: 云微 ***@***.***> Date: Wed,May 11,2022 6:24 PM To: apache/apisix ***@***.***> Cc: laughingyear ***@***.***>, Mention ***@***.***> Subject: Re: [apache/apisix] doc: benchmark results is not so good,what is the problem,help (Issue #5966) Hi there! Can I take this issue to improve the Benchmark page? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[tzssangglass]

Hi there! Can I take this issue to improve the Benchmark page?

Of course you can, but it's best to say what you intend to do first.

[yunwei37]

@tzssangglass Thank you! I'd like to do following things:

1. As described above: doc: benchmark results is not so good,what is the problem,help #5966 (comment), I need to add @membphis's comment to the Benchmark page;

2. It seems currently we have both English version and Chinese version doc about benchmark:

   • https://github.com/apache/apisix/blob/master/docs/en/latest/benchmark.md
   • https://github.com/apache/apisix/blob/master/docs/zh/latest/benchmark.md
     but the Chinese doc is missing some parts, so maybe I can translate them from the English version;

3. Maybe I can add a little more guides on how to run the benchmark and get the Flame diagram?

   I can refer to this PR ci: support for generating flame graph #6136 and the benchmark cases here https://gist.github.com/membphis/137db97a4bf64d3653aa42f3e016bd01 The guide may not be as important as the benchmark results, so I can add it after showing all the benchmark result images.

[tzssangglass]

Maybe I can add a little more guides on how to run the benchmark and get the Flame diagram?

I don't think it's necessary, it's not about benchmark. If you wish, you can use a separate PR to interpret the flamegraph and analyse the benchmark results through the flamegraph.