QPS 性能提高方案 #348

使用 pip install onnxruntime-gpu
开启服务 cnocr serve -p 8501

测试结果

wrk -t4 -c10 -d10s -s upload.lua http://0.0.0.0:8501/ocr 24-12-29 - 1:19:07
Running 10s test @ http://0.0.0.0:8501/ocr
4 threads and 10 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 1.30s 457.68ms 1.89s 50.00%
Req/Sec 0.22 0.55 2.00 83.33%
25 requests in 10.01s, 39.45KB read
Socket errors: connect 0, read 0, write 0, timeout 19
Requests/sec: 2.50
Transfer/sec: 3.94KB

显卡资源利用率5%一下，
请问有什么提高qps的方法吗