全局 maxconi 与服务器 maxconi haxy 的区别

关于我的 haxy 配置，我有一个问题:

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log         127.0.0.1 syslog emerg
maxconn     4000
quiet
user        haproxy
group       haproxy
daemon
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode        http
log         global
option      abortonclose
option      dontlognull
option      httpclose
option      httplog
option      forwardfor
option      redispatch
timeout connect 10000 # default 10 second time out if a backend is not found
timeout client 300000 # 5 min timeout for client
timeout server 300000 # 5 min timeout for server
stats       enable


listen  http_proxy  localhost:81


balance     roundrobin
option      httpchk GET /empty.html
server      server1 myip:80 maxconn 15 check inter 10000
server      server2 myip:80 maxconn 15 check inter 10000

正如您所看到的，它是直截了当的，但是我对 maxconi 属性是如何工作的感到有点困惑。

在 listen 块中，服务器上有一个全局的数据库和 maxconi。我的想法是这样的: 全局连接管理作为服务的 haxy 将在同一时间排队或处理的连接总数。如果数字超过这个数字，它要么终止连接，要么在某个 linux 套接字池？我不知道如果数字超过4000会发生什么。

然后，将服务器 maxconi 属性设置为15。首先，我将它设置为15，因为我的 php-fpm 是在一个单独的服务器上转发的，它只有这么多子进程可以使用，所以我确保将请求集中在这里，而不是在 php-fpm 中。我觉得这样更快。

但回到主题上，我的理论是这个区域里的每个服务器一次只能发送15个连接。然后连接将等待一个打开的服务器。如果我打开了 cookie，连接将等待 CORRECT 打开的服务器。但我没有。

所以问题是:

  1. 如果全局连接超过4000会发生什么? 它们会死吗? 或者在 Linux 中以某种方式池？
  2. 除了服务器连接的总数不能大于全局连接之外，全局连接是否与服务器连接有关？
  3. 在计算全局连接时，难道不应该是服务器部分中累计的连接数量，再加上一定比例的连接池吗？显然，您对连接还有其他限制，但实际上您想要发送给代理的数量是多少呢？

先谢谢你。

76047 次浏览
小开
最佳答案

Willy got me an answer by email. I thought I would share it. His answers are in bold.

I have a question about my haproxy config:

   #---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
log         127.0.0.1 syslog emerg
maxconn     4000
quiet
user        haproxy
group       haproxy
daemon
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode        http
log         global
option      abortonclose
option      dontlognull
option      httpclose
option      httplog
option      forwardfor
option      redispatch
timeout connect 10000 # default 10 second time out if a backend is not found
timeout client 300000 # 5 min timeout for client
timeout server 300000 # 5 min timeout for server
stats       enable


listen  http_proxy  localhost:81


balance     roundrobin
option      httpchk GET /empty.html
server      server1 myip:80 maxconn 15 check inter 10000
server      server2 myip:80 maxconn 15 check inter 10000

As you can see it is straight forward, but I am a bit confused about how the maxconn properties work.

There is the global one and the maxconn on the server, in the listen block.

And there is also another one in the listen block which defaults to something like 2000.

My thinking is this: the global one manages the total number of connections that haproxy, as a service, will que or process at one time.

Correct. It's the per-process max number of concurrent connections.

If the number gets above that, it either kills the connection, or pools in some linux socket?

The later, it simply stops accepting new connections and they remain in the socket queue in the kernel. The number of queuable sockets is determined by the min of (net.core.somaxconn, net.ipv4.tcp_max_syn_backlog, and the listen block's maxconn).

I have no idea what happens if the number gets higher than 4000.

The excess connections wait for another one to complete before being accepted. However, as long as the kernel's queue is not saturated, the client does not even notice this, as the connection is accepted at the TCP level but is not processed. So the client only notices some delay to process the request. But in practice, the listen block's maxconn is much more important, since by default it's smaller than the global one. The listen's maxconn limits the number of connections per listener. In general it's wise to configure it for the number of connections you want for the service, and to configure the global maxconn to the max number of connections you let the haproxy process handle. When you have only one service, both can be set to the same value. But when you have many services, you can easily understand it makes a huge difference, as you don't want a single service to take all the connections and prevent the other ones from working.

Then you have the server maxconn property set at 15. First off, I set that at 15 because my php-fpm, this is forwarding to on a separate server, only has so many child processes it can use, so I make sure I am pooling the requests here, instead of in php-fpm. Which I think is faster.

Yes, not only it should be faster, but it allows haproxy to find another available server whenever possible, and also it allows it to kill the request in the queue if the client hits "stop" before the connection is forwarded to the server.

But back on the subject, my theory about this number is each server in this block will only be sent 15 connections at a time. And then the connections will wait for an open server. If I had cookies on, the connections would wait for the CORRECT open server. But I don't.

That's exactly the principle. There is a per-proxy queue and a per-server queue. Connections with a persistence cookie go to the server queue and other connections go to the proxy queue. However since in your case no cookie is configured, all connections go to the proxy queue. You can look at the diagram doc/queuing.fig in haproxy sources if you want, it explains how/where decisions are taken.

So questions are:

  1. What happens if the global connections get above 4000? Do they die? Or pool in Linux somehow?

    They're queued in linux. Once you overwhelm the kernel's queue, then they're dropped in the kernel.

  2. Are the global connection related to the server connections, other than the fact you can't have a total number of server connections greater than global?

    No, global and server connection settings are independant.

  3. When figuring out the global connections, shouldn't it be the amount of connections added up in the server section, plus a certain percentage for pooling? And obviously you have other restrains on the connections, but really it is how many you want to send to the proxies?

    You got it right. If your server's response time is short, there is nothing wrong with queueing thousands of connections to serve only a few at a time, because it substantially reduces the request processing time. Practically, establishing a connection nowadays takes about 5 microseconds on a gigabit LAN. So it makes a lot of sense to let haproxy distribute the connections as fast as possible from its queue to a server with a very small maxconn. I remember one gaming site queuing more than 30000 concurrent connections and running with a queue of 30 per server ! It was an apache server, and apache is much faster with small numbers of connections than with large numbers. But for this you really need a fast server, because you don't want to have all your clients queued waiting for a connection slot because the server is waiting for a database for instance. Also something which works very well is to dedicate servers. If your site has many statics, you can direct the static requests to a pool of servers (or caches) so that you don't queue static requests on them and that the static requests don't eat expensive connection slots. Hoping this helps, Willy


提交答案