在多线程环境中使用 HttpClient 的最佳实践

有一段时间，我一直在多线程环境中使用 HttpClient。对于每个线程，当它启动连接时，它将创建一个全新的 HttpClient 实例。

最近，我发现，通过使用这种方法，可能会导致用户打开太多端口，并且大多数连接处于 TIME _ WAIT 状态。

Http://www.opensubscriber.com/message/commons-httpclient-dev@jakarta.apache.org/86045.html

因此，不是每个线程做:

HttpClient c = new HttpClient();
try {
c.executeMethod(method);
}
catch(...) {
}
finally {
method.releaseConnection();
}

我们计划:

[方法 A ]

// global_c is initialized once through
// HttpClient global_c = new HttpClient(new MultiThreadedHttpConnectionManager());


try {
global_c.executeMethod(method);
}
catch(...) {
}
finally {
method.releaseConnection();
}

在正常情况下，global _ c 将由50 + + 个线程并发访问。我想知道，这会不会造成性能问题？MultiThreadedHttpConnectionManager 是否使用无锁机制来实现其线程安全策略？

如果有10个线程使用 global _ c，其他40个线程是否会被锁定？

或者，如果我在每个线程中创建一个 HttpClient 的实例，但是显式地释放连接管理器，这样会更好吗？

[方法 B ]

MultiThreadedHttpConnectionManager connman = new MultiThreadedHttpConnectionManager();
HttpClient c = new HttpClient(connman);
try {
c.executeMethod(method);
}
catch(...) {
}
finally {
method.releaseConnection();
connman.shutdown();
}

关闭()会出现性能问题吗？

我可以知道哪种方法(A 或 B)对于使用50 + + 线程的应用程序来说更好吗？

131768 次浏览
小开

My reading of the docs is that HttpConnection itself is not treated as thread safe, and hence MultiThreadedHttpConnectionManager provides a reusable pool of HttpConnections, you have a single MultiThreadedHttpConnectionManager shared by all threads and initialised exactly once. So you need a couple of small refinements to option A.

MultiThreadedHttpConnectionManager connman = new MultiThreadedHttpConnectionManag

Then each thread should be using the sequence for every request, getting a conection from the pool and putting it back on completion of its work - using a finally block may be good. You should also code for the possibility that the pool has no available connections and process the timeout exception.

HttpConnection connection = null
try {
connection = connman.getConnectionWithTimeout(
HostConfiguration hostConfiguration, long timeout)
// work
} catch (/*etc*/) {/*etc*/} finally{
if ( connection != null )
connman.releaseConnection(connection);
}

As you are using a pool of connections you won't actually be closing the connections and so this should not hit the TIME_WAIT problem. This approach does assuume that each thread doesn't hang on to the connection for long. Note that conman itself is left open.

小开
最佳答案

Method A is recommended by httpclient developer community.

Please refer http://www.mail-archive.com/httpclient-users@hc.apache.org/msg02455.html for more details.

小开

I think you will want to use ThreadSafeClientConnManager.

You can see how it works here: http://foo.jasonhudgins.com/2009/08/http-connection-reuse-in-android.html

Or in the AndroidHttpClient which uses it internally.

小开

Definitely Method A because its pooled and thread safe.

If you are using httpclient 4.x, the connection manager is called ThreadSafeClientConnManager. See this link for further details (scroll down to "Pooling connection manager"). For example:

    HttpParams params = new BasicHttpParams();
SchemeRegistry registry = new SchemeRegistry();
registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
ClientConnectionManager cm = new ThreadSafeClientConnManager(params, registry);
HttpClient client = new DefaultHttpClient(cm, params);
小开

With HttpClient 4.5 you can do this:

CloseableHttpClient httpClient = HttpClients.custom().setConnectionManager(new PoolingHttpClientConnectionManager()).build();

Note that this one implements Closeable (for shutting down of the connection manager).


提交答案