简策小栈

Appearance

数据迁移与跨集群复制 ​

数据迁移方案 ​

1. Reindex ​

json
// 基础迁移
POST /_reindex
{
  "source": {"index": "old_index"},
  "dest": {"index": "new_index"}
}
json
// 带条件迁移
POST /_reindex
{
  "source": {
    "index": "old_index",
    "type": "products",
    "query": {"term": {"status": "active"}}
  },
  "dest": {"index": "new_index"}
}

2. 远程迁移 ​

json
// 跨集群迁移
POST /_reindex
{
  "source": {
    "remote": {
      "host": "http://192.168.1.100:9200",
      "username": "elastic",
      "password": "password"
    },
    "index": "old_index"
  },
  "dest": {"index": "new_index"}
}

3. Snapshot备份 ​

json
// 创建快照
PUT /_snapshot/my_backup
{
  "type": "fs",
  "settings": {
    "location": "/backup"
  }
}

PUT /_snapshot/my_backup/snapshot_1?wait_for_completion=true
{
  "indices": "my_index"
}
json
// 恢复快照
POST /_snapshot/my_backup/snapshot_1/_restore
{
  "indices": "my_index",
  "rename_pattern": "my_index",
  "rename_replacement": "my_index_new"
}

4. 跨集群恢复 ​

json
// 修改仓库配置
PUT /_snapshot/remote_backup
{
  "type": "url",
  "settings": {
    "url": "http://192.168.1.100:9200/_snapshot/my_backup"
  }
}

CCR（跨集群复制） ​

1. 部署CCR ​

bash
# 7.x后需要在每个节点安装
elasticsearch-plugin install x-pack-ccr

2. 配置Leader索引 ​

json
PUT /leader-index
{
  "settings": {
    "index.ccr.following": false
  }
}

3. 启动Follow ​

json
POST /leader-index/_ccr/follow
{
  "follower_index": "follower-index",
  "remote_cluster": "leader_cluster"
}

4. 管理Follow ​

json
// 查看follow状态
GET /follower-index/_ccr/stats

// 暂停follow
POST /follower-index/_ccr/pause

// 恢复follow
POST /follower-index/_ccr/resume

// 删除follow
POST /follower-index/_ccr/unfollow

数据同步方案 ​

1. Logstash同步 ​

conf
input {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "source_index"
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "target_index"
  }
}

2. Beats同步 ​

yaml
# filebeat配置
filebeat.inputs:
- type: log
  paths:
    - /var/log/*.log

output.elasticsearch:
  hosts: ["localhost:9200"]

3. 工具同步 ​

工具优点
elasticsearch-dump简单，支持导出导入
esmElasticsearch分片迁移工具
python脚本可定制
bash
# elasticsearch-dump
elasticdump \
  --input=http://localhost:9200/source \
  --output=http://localhost:9200/target \
  --type=mapping

elasticdump \
  --input=http://localhost:9200/source \
  --output=http://localhost:9200/target \
  --type=data

零停机迁移 ​

方案流程 ​

┌─────────────────────────────────────────┐
│          零停机��移步骤                  │
├─────────────────────────────────────────┤
│                                          │
│  1. 创建新索引                           │
│     PUT /new_index                       │
│                                          │
│  2. 同步别名指向新索引                   │
│     POST /_aliases                       │
│     {"actions": [{"add": {...}}]}       │
│                                          │
│  3. Reindex数据                          │
│     POST /_reindex                      │
│     {"source": {"index": "old_index"},  │
│      "dest": {"index": "new_index"}}     │
│                                          │
│  4. 切换读写                             │
│     POST /_aliases                       │
│                                          │
│  5. 删除旧索引                           │
│     DELETE /old_index                     │
│                                          │
└─────────────────────────────────────────┘

别名切换 ​

json
POST /_aliases
{
  "actions": [
    {
      "add": {
        "index": "new_index",
        "alias": "products",
        "write_index": true
      }
    },
    {
      "remove": {
        "index": "old_index",
        "alias": "products"
      }
    }
  ]
}

面试考点 ​

Q: Reindex vs Snapshot恢复？ ​

方案适用场景
Reindex索引结构变化
Snapshot全量恢复
CCR实时同步

Q: 数据迁移注意什么？ ​

  • 版本兼容性
  • 分片配置
  • 别名切换
  • 验证数据完整性

Q: CCR延迟多少？ ​

通常秒级，受网络影响

Q: 为什么用别名迁移？ ​

  • 不影响读写
  • 可回滚

最后更新: