三黍zip流下载配置更新记录

zhangfanglin

问题排查初期（第一天）：

1、报错
docker内部ng

2025/03/12 18:43:34 [warn] 29#29: *218120 an upstream response is buffered to a temporary file /tmp/fastcgi_temp/0000006707 while reading upstream, client: 10.233.90.0, server: _, request: "GET /api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw HTTP/1.0", upstream: "fastcgi://127.0.0.1:8991", host: "omic.sanshugroup.com"
10.233.90.0 - - [12/Mar/2025:18:47:51 +0800] "GET /api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw HTTP/1.0" 200 52940341857 "-" "Wget/1.19.4 (linux-gnu)" "103.114.101.5" 265.672 265.672 . -

服务器NG

2025/03/12 18:43:20 [error] 2052141#0: *166617 readv() failed (104: Connection reset by peer) while reading upstream, client: 103.114.101.5, server: omic.sanshugroup.com, request: "GET /api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw HTTP/1.1", upstream: "http://192.168.30.202:30000/api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw", host: "omic.sanshugroup.com"
2025/03/12 18:53:32 [error] 2052141#0: *166073 readv() failed (104: Connection reset by peer) while reading upstream, client: 113.132.179.7, server: omic.sanshugroup.com, request: "GET /api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw HTTP/1.1", upstream: "http://192.168.30.202:30000/api/dow_report/c656a052-9ea1-8de2-efd0-11c7a051eb6c?type=raw", host: "omic.sanshugroup.com", referrer: "https://omic.sanshugroup.com/"

2、问题排查
根据上放问题是临时文件的缓冲区不够
修改NGINX配置

	proxy_buffer_size   32m;
	proxy_buffers       12 16m;
	proxy_busy_buffers_size   64m;
        fastcgi_connect_timeout 24h;
        fastcgi_send_timeout 24h;
        fastcgi_read_timeout 24h;

问题排查初期（第二天）：

1、每次下载执行到49.30G就无故断开，连续测试多次

secure_download?key=54d4e2deca0598059daa524255a9c685     [                                                                                                                  <=> ]  49.30G  4.57MB/s    in 3h 21m

2、修改php配置
但是实际观察占用内存并不是很高

ini_set('memory_limit', '100G');

测试还是和第二天一样

问题排查初期（第三天）：

找到外部nginx配置，修改如下：

        client_header_buffer_size 20m;
	large_client_header_buffers 4 18m;
	client_max_body_size 100g;
	proxy_max_temp_file_size 1001024m;

晚上同时执行两个下载

浏览器和命令行，两个下载均已成功

总结：

问题报错与断开原因有两个
1、缓冲区的配置不够，配置如下

	proxy_buffer_size   32m;
	proxy_buffers       12 16m;
	proxy_busy_buffers_size   64m;

2、固定下载到49.30G断开，配置如下

	client_max_body_size 100g;
	proxy_max_temp_file_size 1001024m;