从上游读取响应报头时,上游发送了太大的报头

我得到这样的错误:

2014/05/24 11:49:06 [error] 8376#0: *54031 upstream在读取来自upstream的响应头时发送了太大的头,客户端:107.21.193.210,服务器:aamjanata.com,请求:“把% 20 https / the-brainwash-chronicles-sponsored-by-gujarat-government /: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/, % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government / % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/, % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government / % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/, % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government / % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/ % 20 https: / aamjanata.com/the-brainwash-chronicles-sponsored-by-gujarat-government/, % 20 ht

总是一样的。一个url重复使用逗号分隔。不知道是什么引起的。有人知道吗?

更新:另一个错误:

http request count is zero while sending response to client

这是配置。还有其他不相关的东西,但这部分是添加/编辑的

fastcgi_cache_path /var/nginx-cache levels=1:2 keys_zone=WORDPRESS:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_use_stale error timeout invalid_header http_500;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;
# Upstream to abstract backend connection(s) for PHP.
upstream php {
#this should match value of "listen" directive in php-fpm pool
server unix:/var/run/php5-fpm.sock;
}

然后在服务器块: $skip_cache 0;

    # POST requests and urls with a query string should always go to PHP
if ($request_method = POST) {
set $skip_cache 1;
}
if ($query_string != "") {
set $skip_cache 1;
}


# Don't cache uris containing the following segments
if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
set $skip_cache 1;
}


# Don't use the cache for logged in users or recent commenters
if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
set $skip_cache 1;
}


location / {
# This is cool because no php is touched for static content.
# include the "?$args" part so non-default permalinks doesn't break when using query string
try_files $uri $uri/ /index.php?$args;
}




location ~ \.php$ {
try_files $uri /index.php;
include fastcgi_params;
fastcgi_pass php;
fastcgi_read_timeout 3000;


fastcgi_cache_bypass $skip_cache;
fastcgi_no_cache $skip_cache;


fastcgi_cache WORDPRESS;
fastcgi_cache_valid  60m;
}


location ~ /purge(/.*) {
fastcgi_cache_purge WORDPRESS "$scheme$request_method$host$1";
}`
319975 次浏览

将以下内容添加到conf文件中

fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;

如果nginx作为代理/反向代理运行

也就是说,对于ngx_http_proxy_module的用户

除了fastcgiproxy模块还将请求头保存在临时缓冲区中。

所以你可能还需要增加proxy_buffer_sizeproxy_buffers,或者完全禁用它(请阅读nginx的文档)。

代理缓存配置示例

http {
proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;
}

禁用代理缓冲区的示例(推荐用于长轮询服务器)

http {
proxy_buffering off;
}

有关更多信息:Nginx代理模块文档

upstream sent too big header while reading response header from upstream是nginx表示“我不喜欢我所看到的”的通用方式。

  1. 上游服务器线程崩溃
  2. 上游服务器发回无效报头
  3. 从STDERR发送回来的通知/警告溢出了它们的缓冲区,它和STDOUT都被关闭

3:查看消息上面的错误日志,它是否与消息前面的日志行一起流?PHP message: PHP Notice: Undefined index: 循环日志文件的示例片段:

2015/11/23 10:30:02 [error] 32451#0: *580927 FastCGI sent in stderr: "PHP message: PHP Notice:  Undefined index: Firstname in /srv/www/classes/data_convert.php on line 1090
PHP message: PHP Notice:  Undefined index: Lastname in /srv/www/classes/data_convert.php on line 1090
... // 20 lines of same
PHP message: PHP Notice:  Undefined index: Firstname in /srv/www/classes/data_convert.php on line 1090
PHP message: PHP Notice:  Undefined index: Lastname in /srv/www/classes/data_convert.php on line 1090
PHP message: PHP Notice:  Undef
2015/11/23 10:30:02 [error] 32451#0: *580927 FastCGI sent in stderr: "ta_convert.php on line 1090
PHP message: PHP Notice:  Undefined index: Firstname

在从底部开始的第3行中,您可以看到缓冲区限制被击中,破坏了,下一个线程在它上面写入。然后Nginx关闭连接并返回502给客户端。

2:记录每个请求发送的所有报头,检查它们并确保它们符合标准(nginx不允许任何超过24小时的内容删除/过期cookie,发送无效的内容长度,因为错误消息在内容计数之前被缓冲了…)getallheaders函数调用通常可以在抽象代码情况下帮助解决PHP获取所有头文件

例子包括:

<?php
//expire cookie
setcookie ( 'bookmark', '', strtotime('2012-01-01 00:00:00') );
// nginx will refuse this header response, too far past to accept
....
?>

这:

<?php
header('Content-type: image/jpg');
?>


<?php   //a space was injected into the output above this line
header('Content-length: ' . filesize('image.jpg') );
echo file_get_contents('image.jpg');
// error! the response is now 1-byte longer than header!!
?>

1:验证,或做一个脚本日志,以确保您的线程到达正确的终点,并在完成之前不退出。

Plesk指令

我把上面两个答案结合起来

在Plesk 12中,我让nginx作为反向代理运行(我认为这是默认的)。所以当前的上面的回答不工作,因为nginx也作为代理运行。

我去了Subscriptions | [subscription domain] | Websites & Domains (tab) | [Virtual Host domain] | Web Server Settings

然后在该页的底部,你可以设置额外的nginx指令,我设置为上面两个答案的组合:

fastcgi_buffers         16  16k;
fastcgi_buffer_size         32k;
proxy_buffer_size          128k;
proxy_buffers            4 256k;
proxy_busy_buffers_size    256k;

我们最终意识到,我们的一台服务器遇到了这种情况,导致了fpm配置崩溃,导致php错误/警告/通知通过FCGI套接字发送,而这些错误/警告/通知通常会记录到磁盘上。当头的一部分被分割到缓冲区块时,看起来有一个解析错误。

因此,将php_admin_value[error_log]设置为实际可写的内容并重新启动php-fpm就足以解决这个问题。

我们可以用一个更小的脚本重现这个问题:

<?php
for ($i = 0; $i<$_GET['iterations']; $i++)
error_log(str_pad("a", $_GET['size'], "a"));
echo "got here\n";

提高缓冲使502更难命中,但也不是不可能。

bash-4.1# for it in {30..200..3}; do for size in {100..250..3}; do echo "size=$size iterations=$it $(curl -sv "http://localhost/debug.php?size=$size&iterations=$it" 2>&1 | egrep '^< HTTP')"; done; done | grep 502 | head
size=121 iterations=30 < HTTP/1.1 502 Bad Gateway
size=109 iterations=33 < HTTP/1.1 502 Bad Gateway
size=232 iterations=33 < HTTP/1.1 502 Bad Gateway
size=241 iterations=48 < HTTP/1.1 502 Bad Gateway
size=145 iterations=51 < HTTP/1.1 502 Bad Gateway
size=226 iterations=51 < HTTP/1.1 502 Bad Gateway
size=190 iterations=60 < HTTP/1.1 502 Bad Gateway
size=115 iterations=63 < HTTP/1.1 502 Bad Gateway
size=109 iterations=66 < HTTP/1.1 502 Bad Gateway
size=163 iterations=69 < HTTP/1.1 502 Bad Gateway
[... there would be more here, but I piped through head ...]

fastcgi_buffers 16 16k; fastcgi_buffer_size 32k;:

bash-4.1# for it in {30..200..3}; do for size in {100..250..3}; do echo "size=$size iterations=$it $(curl -sv "http://localhost/debug.php?size=$size&iterations=$it" 2>&1 | egrep '^< HTTP')"; done; done | grep 502 | head
size=223 iterations=69 < HTTP/1.1 502 Bad Gateway
size=184 iterations=165 < HTTP/1.1 502 Bad Gateway
size=151 iterations=198 < HTTP/1.1 502 Bad Gateway

所以我相信正确的答案是:修改你的fpm配置,这样它就会把错误记录到磁盘上。

如果你使用Symfony框架: 在搞砸Nginx配置之前,请先禁用ChromePHP

1 -打开app/config/config_dev.yml

注释这些行:

#chromephp:
#type:   chromephp
#level:  info

ChromePHP将调试信息json编码在X-ChromePhp-Data头文件中,这对于使用fastcgi的nginx的默认配置来说太大了。

来源:https://github.com/symfony/symfony/issues/8413#issuecomment-20412848

当搜索这个错误时,这仍然是谷歌上最高的so问题,所以让我们把它撞掉。

当得到这个错误并且不想立即深入NGINX设置时,你可能想检查你在调试控制台的输出。 在我的情况下,我正在输出文本负载到FirePHP / Chromelogger控制台,由于这都是作为头发送的,这导致了溢出

如果这个错误是由于发送大量日志消息引起的,可能不需要更改web服务器设置。

我不确定这个问题与php发送的头文件有关。 确保缓冲已启用。简单的方法是创建一个proxy.conf文件:

proxy_redirect          off;
proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size    100m;
client_body_buffer_size 128k;
proxy_connect_timeout   90;
proxy_send_timeout      90;
proxy_read_timeout      90;
proxy_buffering         on;
proxy_buffer_size       128k;
proxy_buffers           4 256k;
proxy_busy_buffers_size 256k;

和一个fasci .conf文件:

fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;
fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;
fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;
fastcgi_buffers 128 4096k;
fastcgi_buffer_size 4096k;
fastcgi_index  index.php;
fastcgi_param  REDIRECT_STATUS    200;

接下来你需要在默认配置服务器中调用它们,如下所示:

http {
include    /etc/nginx/mime.types;
include    /etc/nginx/proxy.conf;
include    /etc/nginx/fastcgi.conf;
index    index.html index.htm index.php;
log_format   main '$remote_addr - $remote_user [$time_local]  $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
#access_log   /logs/access.log  main;
sendfile     on;
tcp_nopush   on;
# ........
}

添加:

fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

服务器{}在nginx.conf

对我有用。

在我们的例子中,我们得到了这个nginx错误,因为我们的后端生成了一个非常长的URL重定向响应:

HTTP/1.1 301 Moved Permanently
Location: https://www.example.com/?manyParamsHere...

出于好奇,我们将这个大URL保存到一个文件中,它的大小为4.4 Kb。

在配置文件/etc/nginx/conf.d/some_site.conf中添加两行帮助我们修复了这个错误:

server {
# ...
location ~ ^/index\.php(/|$) {
fastcgi_pass php-upstream;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;


# Add these lines:
fastcgi_buffer_size 32k;
fastcgi_buffers 4 32k;
}
}

在官方nginx的文档中阅读更多关于这些参数的信息。

我有一个django应用程序部署到EBS,我使用Python 3.8运行在64bit亚马逊Linux 2上。下面的方法适用于我(注意,如果您使用的是以前的Linux版本,文件夹结构可能会有所不同。更多信息,请参见官方文件在这里

创建.platform文件夹及其子目录,如下所示:

|-- .ebextensions          # Don't put nginx config here
|   |-- django.config
|-- .platform              # Make ".platform" folder and its subfolders
|-- nginx
|   -- conf.d
|       -- proxy.conf


注意,proxy.conf文件应该放在.platform文件夹中,而不是.ebextensions文件夹或.elasticbeanstalk文件夹中。扩展应该以. conf NOT config结尾。

proxy.conf文件中,复制&直接粘贴这些行:

client_max_body_size 50M;
large_client_header_buffers 4 32k;
fastcgi_buffers 16 32k;
fastcgi_buffer_size 32k;
proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

不需要发出命令重新启动nginx(适用于Amazon Linux 2)

再次将源代码部署到elastic beanstalk。

在php-fpm和nginx的docker容器中运行Symfony应用程序时也遇到了同样的问题。

经过一些研究发现,这是由php-fpm的stderr写入nginx日志引起的。例如,php警告(在Symfony调试模式中集中生成)在docker logs php-fpm中重复:

[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\DisallowRobotsIndexingListener::onResponse"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\StreamedResponseListener::onKernelResponse"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\LocaleListener::onKernelFinishRequest"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\RouterListener::onKernelFinishRequest"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\LocaleAwareListener::onKernelFinishRequest"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: "NOTICE: PHP message: [debug] Notified event "kernel.terminate" to listener "Symfony\Component\HttpKernel\EventListener\ProfilerListener::onKernelTerminate"."
[09-Jul-2021 12:25:46] WARNING: [pool www] child 38 said into stderr: ""


docker logs nginx:

2021/07/09 12:25:46 [error] 30#30: *2 FastCGI sent in stderr: "ller" to listener "OblgazAPI\API\Common\Infrastructure\EventSubscriber\LegalAuthenticationChecker::checkAuthentication".


PHP message: [debug] Notified event "kernel.controller_arguments" to listener "Symfony\Component\HttpKernel\EventListener\ErrorListener::onControllerArguments".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\ResponseListener::onKernelResponse".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\DataCollector\RequestDataCollector::onKernelResponse".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\ProfilerListener::onKernelResponse".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Bundle\WebProfilerBundle\EventListener\WebDebugToolbarListener::onKernelResponse".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\DisallowRobotsIndexingListener::onResponse".


PHP message: [debug] Notified event "kernel.response" to listener "Symfony\Component\HttpKernel\EventListener\StreamedResponseListener::onKernelResponse".


PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\LocaleListener::onKernelFinishRequest".


PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\RouterListener::onKernelFinishRequest".


PHP message: [debug] Notified event "kernel.finish_request" to listener "Symfony\Component\HttpKernel\EventListener\LocaleAwareListener::onKernelFinishRequest".


PHP message: [debug] Notified event "kernel.exception" to listener "Symfony\Component\HttpKernel\EventListener\ErrorListener::logKernelException".


PHP message: [debug] Notified event "kernel.exception" to listener "Symfony\Component\HttpKernel\EventListener\ProfilerListener::onKernelException".

然后nginx log以

2021/07/09 12:25:46 [error] 30#30: *2 upstream sent too big header while reading response header from upstream ...

我得到了502个错误。

在nginx配置中增加fastcgi_buffer_size是有帮助的,但这似乎更像是抑制问题,而不是治疗。

一个更好的解决方案是禁用php-fpm通过FastCGI发送日志。可以通过在php.ini中设置fastcgi.logging=0来实现(默认值为1)。

在将其更改为0之后,问题就消失了,nginx日志看起来更干净了。

172.18.0.1 - - [09/Jul/2021:12:36:02 +0300] "GET /my/symfony/app HTTP/1.1" 401 73 "-" "PostmanRuntime/7.26.8"
172.18.0.1 - - [09/Jul/2021:12:36:04 +0300] "GET /my/symfony/app HTTP/1.1" 401 73 "-" "PostmanRuntime/7.26.8"


并且所有的php-fpm日志仍然在它们在php-fpm日志中的位置。

对于Symfony项目,尝试将这一行添加到你的.env文件中:

SHELL_VERBOSITY=0

https://symfony.com/doc/current/console/verbosity.html

我将把它留在这里,因为我花了大量的时间在我的项目中调试这个,只有这个特定的解决方案100%适合我(出于正确的原因),我从未找到与这个主题相关的答案。也许有人会觉得有用。

fastcgi_busy_buffers_size 512k;
fastcgi_buffer_size 512k;
fastcgi_buffers 16 512k;

当我增加数字时,它对我有用

我有这个错误,我找到了3种方法来修复它:

  • 设置SHELL_VERBOSITY=0或另一个值<在这种情况下,您禁用了PHP日志,但它们对开发和调试很有用。
  • php.ini中设置fastcgi.logging=0。与上述结果相同。
  • 将Symfony从5.2更新到5.3。我认为旧版本在这方面有问题。

所有的PHP Symfony日志都被认为是Nginx的错误,但PHP工作正常。

我有Nginx 1.17, PHP 8.0.2, PHP- fpm, Symfony 5.2, Xdebug, Docker。

我尝试了新版本Nginx 1.21, PHP 8.0.14,但没有任何结果。这个问题不在于Apache。

我改变了Nginx的配置,但也没有任何结果。

我正在使用Symfony,它有一个很好的异常页面(当使用ErrorHandler时)。这将把异常的消息作为创建的响应的头。

供应商/ symfony /欢心/ ErrorRenderer / SerializerErrorRenderer.php

头部被称为:X-Debug-Exception

所以要小心,如果你构造了一个非常大的异常消息,无论是nginx还是chrome (limit 256k)还是curl (~128kb)都不能显示你的页面,并且很难调试,输出这些大标题是什么。

我的建议是不要盲目地复制粘贴增加的缓冲区大小到你的nginx配置中,他们治标不治本。