这篇文章上次修改于 902 天前,可能其部分内容已经发生变化,如有疑问可询问作者。

1. 压测目标

  • 了解框架性能上限,即获得 HelloWorld 接口容量。在响应时间不超过 1s,CPU 峰值利用率不高于 70% 时,获得该接口的容量。
  • 发现性能瓶颈。例如,当吞吐量不再上升时,查找原因。

2. 指标

  • 响应时间
  • 吞吐量
  • 错误率

3. 压测执行

使用 wrk2:https://github.com/giltene/wrk2
因为要发起 post 请求,所以需要准备对应的 lua 脚本:

wrk.method = "POST"
wrk.body   = '{"Version":"1.0.0","AppId":123,"Uin":"123","SubAccountUin":"123","Action":"HelloWorld"}'
wrk.headers["Content-Type"] = "application/json"

压测执行示例:

# -t 线程数
# -c 总 http 并发数
# -d 持续压测时间
# -R 每秒请求数
# -L 打印延迟详细信息
# --script 脚本路径

wrk -t4 -c200 -d600s -R800 -L --script=hello.lua http://test.server.com

4. 压测记录

这里请注意被压测的集群有 2 台机器,所以最终计算 QPS 时需要除以 2。

结果说明:

  • latency:响应时长分布。
  • req/sec:每秒处理请求数。
  • timeout:连接超时数目。
  • Socket errors:错误数。

4.1. 200QPS

$ wrk -t3 -c100 -d600s -R200 -L --script=hello.lua http://test.server.com

Running 10m test @ http://test.server.com
  3 threads and 100 connections
  Thread calibration: mean lat.: 84.350ms, rate sampling interval: 173ms
  Thread calibration: mean lat.: 84.086ms, rate sampling interval: 173ms
  Thread calibration: mean lat.: 84.479ms, rate sampling interval: 173ms
 
 
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    73.28ms   20.65ms 545.79ms   92.64%
    Req/Sec    66.48     65.00   191.00     47.06%
  Latency Distribution (HdrHistogram - Recorded Latency)
 50.000%   72.00ms
 75.000%   80.51ms
 90.000%   84.80ms
 99.000%  161.41ms
 99.900%  276.48ms
 99.990%  441.86ms
 99.999%  528.90ms
100.000%  546.30ms
 
  Detailed Percentile spectrum:
       Value   Percentile   TotalCount 1/(1-Percentile)
 
      38.367     0.000000            1         1.00
      58.175     0.100000        11897         1.11
      60.383     0.200000        23600         1.25
      63.071     0.300000        35410         1.43
      65.439     0.400000        47259         1.67
      71.999     0.500000        58963         2.00
      75.135     0.550000        64924         2.22
      76.671     0.600000        70901         2.50
      77.951     0.650000        76734         2.86
      79.231     0.700000        82662         3.33
      80.511     0.750000        88485         4.00
      81.151     0.775000        91532         4.44
      81.791     0.800000        94595         5.00
      82.431     0.825000        97423         5.71
      83.135     0.850000       100480         6.67
      83.903     0.875000       103362         8.00
      84.351     0.887500       104858         8.89
      84.799     0.900000       106215        10.00
      85.375     0.912500       107695        11.43
      86.015     0.925000       109131        13.33
      86.847     0.937500       110547        16.00
      87.487     0.943750       111344        17.78
      88.255     0.950000       112041        20.00
      89.663     0.956250       112775        22.86
      93.567     0.962500       113491        26.67
     109.951     0.968750       114228        32.00
     133.503     0.971875       114595        35.56
     139.519     0.975000       114979        40.00
     143.615     0.978125       115343        45.71
     148.095     0.981250       115704        53.33
     153.471     0.984375       116074        64.00
     155.775     0.985938       116253        71.11
     158.079     0.987500       116444        80.00
     160.255     0.989062       116626        91.43
     162.175     0.990625       116809       106.67
     164.735     0.992188       116992       128.00
     166.399     0.992969       117082       142.22
     168.063     0.993750       117175       160.00
     170.367     0.994531       117268       182.86
     173.055     0.995313       117358       213.33
     177.279     0.996094       117450       256.00
     180.863     0.996484       117496       284.44
     184.959     0.996875       117542       320.00
     192.767     0.997266       117588       365.71
     202.111     0.997656       117634       426.67
     218.623     0.998047       117680       512.00
     224.383     0.998242       117703       568.89
     236.287     0.998437       117726       640.00
     251.263     0.998633       117749       731.43
     267.775     0.998828       117772       853.33
     277.503     0.999023       117795      1024.00
     282.879     0.999121       117807      1137.78
     290.559     0.999219       117820      1280.00
     300.031     0.999316       117832      1462.86
     304.895     0.999414       117841      1706.67
     335.615     0.999512       117853      2048.00
     350.463     0.999561       117859      2275.56
     357.631     0.999609       117864      2560.00
     373.247     0.999658       117870      2925.71
     396.031     0.999707       117876      3413.33
     402.943     0.999756       117882      4096.00
     405.759     0.999780       117885      4551.11
     410.367     0.999805       117888      5120.00
     411.903     0.999829       117890      5851.43
     416.511     0.999854       117893      6826.67
     439.295     0.999878       117896      8192.00
     441.855     0.999890       117898      9102.22
     454.399     0.999902       117899     10240.00
     456.447     0.999915       117900     11702.86
     465.151     0.999927       117902     13653.33
     469.247     0.999939       117903     16384.00
     484.607     0.999945       117904     18204.44
     503.039     0.999951       117905     20480.00
     503.039     0.999957       117905     23405.71
     509.183     0.999963       117906     27306.67
     526.847     0.999969       117907     32768.00
     526.847     0.999973       117907     36408.89
     528.383     0.999976       117908     40960.00
     528.383     0.999979       117908     46811.43
     528.383     0.999982       117908     54613.33
     528.895     0.999985       117909     65536.00
     528.895     0.999986       117909     72817.78
     528.895     0.999988       117909     81920.00
     528.895     0.999989       117909     93622.86
     528.895     0.999991       117909    109226.67
     546.303     0.999992       117910    131072.00
     546.303     1.000000       117910          inf
#[Mean    =       73.281, StdDeviation   =       20.646]
#[Max     =      545.792, Total count    =       117910]
#[Buckets =           27, SubBuckets     =         2048]
----------------------------------------------------------
  119991 requests in 10.00m, 28.08MB read
Requests/sec:    199.98
Transfer/sec:     47.91KB

CPU 使用率峰值:20%

4.2. 500QPS

$ wrk -t3 -c100 -d600s -R500 -L --script=hello.lua http://test.server.com

Running 10m test @ http://test.server.com
  3 threads and 100 connections
  Thread calibration: mean lat.: 698.326ms, rate sampling interval: 3860ms
  Thread calibration: mean lat.: 664.252ms, rate sampling interval: 3745ms
  Thread calibration: mean lat.: 685.529ms, rate sampling interval: 3821ms
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   736.99ms  812.45ms   5.30s    83.13%
    Req/Sec   165.59     45.77   304.00     67.60%
  Latency Distribution (HdrHistogram - Recorded Latency)
 50.000%  385.79ms
 75.000%    1.24s
 90.000%    1.91s
 99.000%    3.23s
 99.900%    3.95s
 99.990%    4.42s
 99.999%    5.02s
100.000%    5.31s
 
  Detailed Percentile spectrum:
       Value   Percentile   TotalCount 1/(1-Percentile)
 
      38.495     0.000000            1         1.00
      62.559     0.100000        29549         1.11
      74.111     0.200000        58942         1.25
      81.087     0.300000        88530         1.43
     106.815     0.400000       117836         1.67
     385.791     0.500000       147285         2.00
     559.103     0.550000       162017         2.22
     730.111     0.600000       176767         2.50
     899.583     0.650000       191474         2.86
    1070.079     0.700000       206238         3.33
    1242.111     0.750000       220921         4.00
    1330.175     0.775000       228340         4.44
    1423.359     0.800000       235674         5.00
    1523.711     0.825000       243024         5.71
    1633.279     0.850000       250408         6.67
    1759.231     0.875000       257741         8.00
    1829.887     0.887500       261442         8.89
    1907.711     0.900000       265114        10.00
    1999.871     0.912500       268811        11.43
    2105.343     0.925000       272532        13.33
    2232.319     0.937500       276162        16.00
    2303.999     0.943750       278025        17.78
    2383.871     0.950000       279865        20.00
    2471.935     0.956250       281682        22.86
    2570.239     0.962500       283552        26.67
    2678.783     0.968750       285377        32.00
    2740.223     0.971875       286292        35.56
    2805.759     0.975000       287209        40.00
    2873.343     0.978125       288131        45.71
    2949.119     0.981250       289051        53.33
    3033.087     0.984375       289965        64.00
    3080.191     0.985938       290436        71.11
    3131.391     0.987500       290884        80.00
    3188.735     0.989062       291344        91.43
    3254.271     0.990625       291808       106.67
    3327.999     0.992188       292264       128.00
    3366.911     0.992969       292500       142.22
    3407.871     0.993750       292721       160.00
    3452.927     0.994531       292953       182.86
    3506.175     0.995313       293186       213.33
    3563.519     0.996094       293414       256.00
    3600.383     0.996484       293533       284.44
    3637.247     0.996875       293647       320.00
    3674.111     0.997266       293756       365.71
    3723.263     0.997656       293875       426.67
    3774.463     0.998047       293990       512.00
    3805.183     0.998242       294044       568.89
    3835.903     0.998437       294103       640.00
    3868.671     0.998633       294161       731.43
    3909.631     0.998828       294216       853.33
    3958.783     0.999023       294276      1024.00
    3985.407     0.999121       294303      1137.78
    4012.031     0.999219       294331      1280.00
    4044.799     0.999316       294361      1462.86
    4075.519     0.999414       294390      1706.67
    4118.527     0.999512       294421      2048.00
    4136.959     0.999561       294432      2275.56
    4159.487     0.999609       294447      2560.00
    4182.015     0.999658       294461      2925.71
    4218.879     0.999707       294475      3413.33
    4251.647     0.999756       294491      4096.00
    4263.935     0.999780       294497      4551.11
    4292.607     0.999805       294504      5120.00
    4333.567     0.999829       294514      5851.43
    4345.855     0.999854       294518      6826.67
    4374.527     0.999878       294526      8192.00
    4415.487     0.999890       294530      9102.22
    4427.775     0.999902       294533     10240.00
    4440.063     0.999915       294536     11702.86
    4476.927     0.999927       294540     13653.33
    4534.271     0.999939       294544     16384.00
    4542.463     0.999945       294545     18204.44
    4607.999     0.999951       294547     20480.00
    4616.191     0.999957       294549     23405.71
    4669.439     0.999963       294551     27306.67
    4706.303     0.999969       294553     32768.00
    4706.303     0.999973       294553     36408.89
    4755.455     0.999976       294554     40960.00
    4775.935     0.999979       294555     46811.43
    4837.375     0.999982       294556     54613.33
    4923.391     0.999985       294557     65536.00
    4923.391     0.999986       294557     72817.78
    5021.695     0.999988       294558     81920.00
    5021.695     0.999989       294558     93622.86
    5124.095     0.999991       294559    109226.67
    5124.095     0.999992       294559    131072.00
    5124.095     0.999993       294559    145635.56
    5226.495     0.999994       294560    163840.00
    5226.495     0.999995       294560    187245.71
    5226.495     0.999995       294560    218453.33
    5226.495     0.999996       294560    262144.00
    5226.495     0.999997       294560    291271.11
    5308.415     0.999997       294561    327680.00
    5308.415     1.000000       294561          inf
#[Mean    =      736.988, StdDeviation   =      812.454]
#[Max     =     5304.320, Total count    =       294561]
#[Buckets =           27, SubBuckets     =         2048]
----------------------------------------------------------
  299615 requests in 10.00m, 69.79MB read
  Socket errors: connect 0, read 0, write 0, timeout 83
Requests/sec:    499.35
Transfer/sec:    119.10KB

CPU 使用率峰值:57%

4.3. 800 QPS

wrk -t4 -c200 -d600s -R800 -L --script=hello.lua http://test.server.com

Running 10m test @ http://test.server.com
  4 threads and 200 connections
  Thread calibration: mean lat.: 312.210ms, rate sampling interval: 2279ms
  Thread calibration: mean lat.: 362.269ms, rate sampling interval: 2920ms
  Thread calibration: mean lat.: 430.283ms, rate sampling interval: 3317ms
  Thread calibration: mean lat.: 332.868ms, rate sampling interval: 2615ms
 
 
 
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   598.80ms  791.08ms   5.17s    83.19%
    Req/Sec   199.47     54.34   371.00     66.67%
  Latency Distribution (HdrHistogram - Recorded Latency)
 50.000%   94.40ms
 75.000%  951.81ms
 90.000%    1.84s
 99.000%    3.23s
 99.900%    4.14s
 99.990%    4.65s
 99.999%    4.89s
100.000%    5.17s
 
  Detailed Percentile spectrum:
       Value   Percentile   TotalCount 1/(1-Percentile)
 
      37.343     0.000000            1         1.00
      60.671     0.100000        47349         1.11
      66.047     0.200000        94529         1.25
      76.863     0.300000       141715         1.43
      82.431     0.400000       189246         1.67
      94.399     0.500000       235944         2.00
     192.767     0.550000       259495         2.22
     352.255     0.600000       283078         2.50
     524.287     0.650000       306682         2.86
     721.407     0.700000       330270         3.33
     951.807     0.750000       353884         4.00
    1077.247     0.775000       365646         4.44
    1210.367     0.800000       377511         5.00
    1349.631     0.825000       389243         5.71
    1499.135     0.850000       401091         6.67
    1657.855     0.875000       412836         8.00
    1744.895     0.887500       418785         8.89
    1835.007     0.900000       424667        10.00
    1932.287     0.912500       430575        11.43
    2035.711     0.925000       436418        13.33
    2154.495     0.937500       442325        16.00
    2220.031     0.943750       445298        17.78
    2291.711     0.950000       448291        20.00
    2371.583     0.956250       451216        22.86
    2457.599     0.962500       454125        26.67
    2562.047     0.968750       457059        32.00
    2625.535     0.971875       458527        35.56
    2693.119     0.975000       460007        40.00
    2768.895     0.978125       461477        45.71
    2863.103     0.981250       462971        53.33
    2971.647     0.984375       464433        64.00
    3035.135     0.985938       465177        71.11
    3100.671     0.987500       465910        80.00
    3178.495     0.989062       466637        91.43
    3260.415     0.990625       467382       106.67
    3354.623     0.992188       468117       128.00
    3403.775     0.992969       468487       142.22
    3461.119     0.993750       468847       160.00
    3518.463     0.994531       469218       182.86
    3579.903     0.995313       469589       213.33
    3649.535     0.996094       469955       256.00
    3690.495     0.996484       470142       284.44
    3739.647     0.996875       470321       320.00
    3790.847     0.997266       470505       365.71
    3848.191     0.997656       470690       426.67
    3915.775     0.998047       470876       512.00
    3948.543     0.998242       470969       568.89
    3991.551     0.998437       471058       640.00
    4036.607     0.998633       471152       731.43
    4089.855     0.998828       471243       853.33
    4149.247     0.999023       471336      1024.00
    4182.015     0.999121       471382      1137.78
    4222.975     0.999219       471429      1280.00
    4255.743     0.999316       471473      1462.86
    4304.895     0.999414       471528      1706.67
    4337.663     0.999512       471567      2048.00
    4358.143     0.999561       471588      2275.56
    4386.815     0.999609       471611      2560.00
    4419.583     0.999658       471636      2925.71
    4448.255     0.999707       471657      3413.33
    4485.119     0.999756       471683      4096.00
    4505.599     0.999780       471694      4551.11
    4517.887     0.999805       471703      5120.00
    4542.463     0.999829       471716      5851.43
    4567.039     0.999854       471727      6826.67
    4612.095     0.999878       471738      8192.00
    4628.479     0.999890       471744      9102.22
    4648.959     0.999902       471749     10240.00
    4665.343     0.999915       471755     11702.86
    4694.015     0.999927       471761     13653.33
    4718.591     0.999939       471767     16384.00
    4739.071     0.999945       471770     18204.44
    4743.167     0.999951       471772     20480.00
    4771.839     0.999957       471777     23405.71
    4780.031     0.999963       471779     27306.67
    4800.511     0.999969       471782     32768.00
    4804.607     0.999973       471784     36408.89
    4804.607     0.999976       471784     40960.00
    4808.703     0.999979       471785     46811.43
    4833.279     0.999982       471787     54613.33
    4845.567     0.999985       471788     65536.00
    4853.759     0.999986       471789     72817.78
    4890.623     0.999988       471790     81920.00
    4890.623     0.999989       471790     93622.86
    4894.719     0.999991       471791    109226.67
    4915.199     0.999992       471792    131072.00
    4915.199     0.999993       471792    145635.56
    5001.215     0.999994       471793    163840.00
    5001.215     0.999995       471793    187245.71
    5001.215     0.999995       471793    218453.33
    5042.175     0.999996       471794    262144.00
    5042.175     0.999997       471794    291271.11
    5042.175     0.999997       471794    327680.00
    5042.175     0.999997       471794    374491.43
    5042.175     0.999998       471794    436906.67
    5169.151     0.999998       471795    524288.00
    5169.151     1.000000       471795          inf
#[Mean    =      598.804, StdDeviation   =      791.076]
#[Max     =     5165.056, Total count    =       471795]
#[Buckets =           27, SubBuckets     =         2048]
----------------------------------------------------------
  479946 requests in 10.00m, 117.25MB read
  Socket errors: connect 0, read 0, write 0, timeout 964
Requests/sec:    799.90
Transfer/sec:    200.10KB

CPU 使用率峰值:74%

5. 结论

  • 框架 QPS 约为 400(一共有两台机器提供服务,800/2)。单台 QPS 在 250(500/2)时,25% 响应时间超过了 1s,没有出现错误,在 QPS400 时维持不变,但 CPU 使用率升高。
  • 250QPS 和 400QPS 在响应时间上没有差别,说明 250QPS 时 uwsgi 的 worker 已经到达了瓶颈。