chore(cluster): stop key scanning when master-slave switch.#3270
chore(cluster): stop key scanning when master-slave switch.#3270hll1213181368 wants to merge 5 commits intoapache:unstablefrom
Conversation
|
@hll1213181368 What do we need to stop the key scanning when the master becomes the slave role? I feel it's a bit strange to do that. |
@git-hulk Because key scanning is performed slot-by-slot, it is time-consuming. For example, one four shard cluster, with each shard containing approximately 1TB of data, key scanning would take about 6 hours. We implemented parallelization for key scanning, achieving a parallelism equal to the number of CPU cores, which improved performance by 6 times, but increased CPU utilization to 90%. A parallelism of the half number of CPU cores improved performance by 3 times, but increased CPU utilization to 50%. Under normal scene, We execute key scans on slave nodes. To prevent excessive CPU load during master-slave failover if a switchover occurs while the new master node is still performing key scanning tasks. I will commit a pr to parallel execute key sanning. |
No description provided.