丢弃弹性搜索的所有文件

有没有办法创建一个转储文件，其中包含索引的所有数据及其设置和映射？

与 mongoDB 使用 混蛋的方法类似
或者像在太阳 其数据文件夹被复制到一个备份位置。

干杯！

164414 次浏览
小开

The data itself is one or more lucene indices, since you can have multiple shards. What you also need to backup is the cluster state, which contains all sorts of information regarding the cluster, the available indices, their mappings, the shards they are composed of etc.

It's all within the data directory though, you can just copy it. Its structure is pretty intuitive. Right before copying it's better to disable automatic flush (in order to backup a consistent view of the index and avoiding writes on it while copying files), issue a manual flush, disable allocation as well. Remember to copy the directory from all nodes.

Also, next major version of elasticsearch is going to provide a new snapshot/restore api that will allow you to perform incremental snapshots and restore them too via api. Here is the related github issue: https://github.com/elasticsearch/elasticsearch/issues/3826.

小开

Here's a new tool we've been working on for exactly this purpose https://github.com/taskrabbit/elasticsearch-dump. You can export indices into/out of JSON files, or from one cluster to another.

小开

Elasticsearch supports a snapshot function out of the box:

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

小开

ElasticSearch itself provides a way to create data backup and restoration. The simple command to do it is:

CURL -XPUT 'localhost:9200/_snapshot/<backup_folder name>/<backupname>' -d '{
"indices": "<index_name>",
"ignore_unavailable": true,
"include_global_state": false
}'

Now, how to create, this folder, how to include this folder path in ElasticSearch configuration, so that it will be available for ElasticSearch, restoration method, is well explained here. To see its practical demo surf here.

小开

You can also dump elasticsearch data in JSON format by http request: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
CURL -XPOST 'https://ES/INDEX/_search?scroll=10m'
CURL -XPOST 'https://ES/_search/scroll' -d '{"scroll": "10m", "scroll_id": "ID"}'

小开

To export all documents from ElasticSearch into JSON, you can use the esbackupexporter tool. It works with index snapshots. It takes the container with snapshots (S3, Azure blob or file directory) as the input and outputs one or several zipped JSON files per index per day. It is quite handy when exporting your historical snapshots. To export your hot index data, you may need to make the snapshot first (see the answers above).

小开

We can use elasticdump to take the backup and restore it, We can move data from one server/cluster to another server/cluster.

1. Commands to move one index data from one server/cluster to another using elasticdump.

# Copy an index from production to staging with analyzer and mapping:
elasticdump \
--input=http://production.es.com:9200/my_index \
--output=http://staging.es.com:9200/my_index \
--type=analyzer
elasticdump \
--input=http://production.es.com:9200/my_index \
--output=http://staging.es.com:9200/my_index \
--type=mapping
elasticdump \
--input=http://production.es.com:9200/my_index \
--output=http://staging.es.com:9200/my_index \
--type=data

2. Commands to move all indices data from one server/cluster to another using multielasticdump.

Backup

multielasticdump \
--direction=dump \
--match='^.*$' \
--limit=10000 \
--input=http://production.es.com:9200 \
--output=/tmp

Restore

multielasticdump \
--direction=load \
--match='^.*$' \
--limit=10000 \
--input=/tmp \
--output=http://staging.es.com:9200

Note:

  • If the --direction is dump, which is the default, --input MUST be a URL for the base location of an ElasticSearch server (i.e. http://localhost:9200) and --output MUST be a directory. Each index that does match will have a data, mapping, and analyzer file created.

  • For loading files that you have dumped from multi-elasticsearch, --direction should be set to load, --input MUST be a directory of a multielasticsearch dump and --output MUST be a Elasticsearch server URL.

  • The 2nd command will take a backup of settings, mappings, template and data itself as JSON files.

  • The --limit should not be more than 10000 otherwise, it will give an exception.

  • Get more details here.
小开

For your case Elasticdump is the perfect answer.
First, you need to download the mapping and then the index

# Install the elasticdump
npm install elasticdump -g


# Dump the mapping
elasticdump --input=http://<your_es_server_ip>:9200/index --output=es_mapping.json --type=mapping


# Dump the data
elasticdump --input=http://<your_es_server_ip>:9200/index --output=es_index.json --type=data

If you want to dump the data on any server I advise you to install esdump through docker. You can get more info from this website Blog Link

小开

If you want to massage the data on its way out of Elasticsearch, you might want to use Logstash. It has a handy Elasticsearch Input Plugin.

And then you can export to anything, from a CSV file to reindexing the data on another Elasticsearch cluster. Though for the latter you also have the Elasticsearch's own Reindex.

小开
最佳答案

At the time of writing this answer(2021), the official way of backing up an ElasticSearch cluster is to snapshot it. Refer to: https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html


提交答案