Connection Timeout with Elasticsearch

from datetime import datetime
from elasticsearch import Elasticsearch
es = Elasticsearch()


doc = {
'author': 'kimchy',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime(2010, 10, 10, 10, 10, 10)
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)
print(res['created'])

This simples code is returning the following error:

elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10))

Very strange, because the server is ready and set (http://localhost:9200/ is returning some json).

155309 次浏览

elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)) mean the request didn't end in the specified time (by default, timeout=10).

This will work with 30 seconds :

res = es.index(index="test-index", doc_type='tweet', id=1, body=doc, timeout=30)

By default, the timeout value is set to 10 secs. If one wants to change the global timeout value, this can be achieved by setting the flag timeout=your-time while creating the object.

If you have already created the object without specifying the timeout value, then you can set the timeout value for particular request by using request_timeout=your-time flag in the query.

es.search(index="my_index",
doc_type="document",
body=get_req_body(),
request_timeout=30)

The connection timed out problem could occur if you are using Amazon Elastic Search service.

es = Elasticsearch([{'host': 'xxxxxx.us-east-1.es.amazonaws.com', 'port': 443,  'use_ssl': True}])

The above python code where you override the default port from 9200 to 443 and setting the SSL to true will resolve the issue.

If no port is specified, it is trying to connect to the port 9200 in the specified host and fails after time out

This is nothing to do with increasing your timeout to 30 seconds. Do people actually think that elastic search should need up to 30 seconds to return one tiny hit?

The way I fixed this problem was go to config/elasticsearch.yml uncomment the following

http.port: 9200
network.host: 'localhost'

Network.host might be set to 192.168.0.1 which might work But I just changed it to 'localhost'

my personal problem was solved with (timeout = 10000) which was practically never reached because the entries on server were only 7.000 but it had heavy traffic and its resources were being hogged and that was why the connection was dropping

Note that one of the common reasons for timeouts when doing es.search (or es.index) is large query size. For example, in my case of a pretty large ES index size (> 3M documents), doing a search for a query with 30 words took around 2 seconds, while doing a search for a query with 400 words took over 18 seconds. So for a sufficiently large query even timeout=30 won't save you. An easy solution is to crop the query to the size that can be answered below the timeout.

Increasing timeout or doing retries on timeout will help you if the cause was in traffic, otherwise this might be your culprit.

Two options that help:

1: increase the timeout

Setting a timeout solved this problem for me. Note that newer versions need a unit, e.g. timeout="60s":

es.index(index=index_name, doc_type="domains", id=domain.id, body=body, timeout="60s")

Without a unit, for example by setting timeout=60, you'll get

elasticsearch.exceptions.RequestError: RequestError(400, 'illegal_argument_exception', 'failed to parse setting [timeout] with value [60] as a time value: unit is missing or unrecognized')

2: reduce text length

It also helps to reduce the text length, e.g. by cutting of long texts, so elastic can store the text faster which will avoid timeouts, too:

es.index(index=index_name, doc_type="domains", id=domain.id, body=text[:5000], timeout="60s")

Try setting timeout in Elasticsearch initialization:

es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30)

You can even set retry_on_timeout to True and give the max_retries an optional number:

es = Elasticsearch([{'host': HOST_ADDRESS, 'port': THE_PORT}], timeout=30, max_retries=10, retry_on_timeout=True)

The reasons for the timeout could be many and it seems worth checking the logs on elasticsearch side (logs/elasticsearch.log) to see the detailed error. In our case, the error on ES was:

primary shard is not active Timeout: [1m]

As described in this post, this was because our disk was full. We had resized it (and the partition) a day ago to take care of that but ES needs to be restarted if the high/low watermark has been hit once (we are on 5.5.x) which we had not done.

Simply restarting the ES on production resolved the issue for us.

I got this timeout problem when trying out elasticsearch 7.11.1 (installed using apt install) at port 9202 (9200 is in use by an elasticsearch 6) in ubuntu.

I googled around for nearly two hours before finally fixing it. It turns out I nneed to set (comment out) one line in /etc/elasticsearch/elasticserch.yml

cluster.initial_master_nodes: ["node-1"]  # earlier in the same file: node.name: node-1

I hope this may help someone.

extend the requestTimeout

client = new elasticsearch.Client({
host          : 'http://localhost:9200',
requestTimeout: 60000
});

I got this error when specifying the fields option in my query for some reason. Removing the option made the query work (in OpenSearch at least):

Before:

elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}, "fields": ["zip_code", "polygon"]}, size=5000, scroll="15m")

After:

elasticsearch.helpers.scan(es, index="my_index", query={"query": {"match_all": {}}}, size=5000, scroll="15m")

I had a similar problem.

> client.ping()
True
> client.info()
ObjectApiResponse({'name': .... })

Searching also worked, but indexing failed with an exception: ConnectionTimeout: Connection timed out.

The reason was that my disk was almost full. I freed up some space and then I could index without errors...

I faced the same error while running python elasticsearch client's es.index() method in a docker container.

The reason was that the disk space was >90% full. Elasticsearch cluster detects the disk usage and if its >90% full, it's unable to start the shard.

Once I freed up some space, connection timeout error disappeared.