-
Notifications
You must be signed in to change notification settings - Fork 710
statistics: deprecate stats v1 in 8.5 docs #22624
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
0xPoe
wants to merge
13
commits into
pingcap:release-8.5
Choose a base branch
from
0xPoe:poe-patch-stats-v1-8.5
base: release-8.5
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+16
−15
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
c21e4fd
docs: deprecate stats v1 in 8.5 docs
0xPoe 693391b
docs: remove extra stats v1 warnings
0xPoe 1df6143
docs: clarify stats v2 summary
0xPoe e493ced
docs: trim stats v2 summary
0xPoe 530c96e
docs: shorten stats v2 wording
0xPoe 735772a
docs: remove wrong migration note
0xPoe 7795935
docs: refine stats v1 migration notes
0xPoe 45c1966
docs: clarify stats v1 wording
0xPoe f383bf1
docs: align stats migration terms
0xPoe b985712
minor wording updates
qiancai 7c7bd67
add v8.5.6
qiancai f976e48
refine descriptions
qiancai 24cee72
Update statistics.md
qiancai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -355,13 +355,17 @@ | |||||
|
|
||||||
| ## Versions of statistics | ||||||
|
|
||||||
| The [`tidb_analyze_version`](/system-variables.md#tidb_analyze_version-new-in-v510) variable controls the statistics collected by TiDB. Currently, two versions of statistics are supported: `tidb_analyze_version = 1` and `tidb_analyze_version = 2`. | ||||||
| > **Warning:** | ||||||
| > | ||||||
| > Starting from v8.5.6, statistics Version 1 (`tidb_analyze_version = 1`) is deprecated and will be removed in a future release. It is recommended that you use Statistics Version 2 (`tidb_analyze_version = 2`) and [migrate existing objects that use Statistics Version 1 to Version 2](#switch-between-statistics-versions). | ||||||
|
|
||||||
| The [`tidb_analyze_version`](/system-variables.md#tidb_analyze_version-new-in-v510) variable controls the statistics collected by TiDB. Currently, TiDB supports two statistics versions: `tidb_analyze_version = 1` and `tidb_analyze_version = 2`. | ||||||
|
|
||||||
| - For TiDB Self-Managed, the default value of this variable changes from `1` to `2` starting from v5.3.0. | ||||||
| - For TiDB Cloud, the default value of this variable changes from `1` to `2` starting from v6.5.0. | ||||||
| - If your cluster is upgraded from an earlier version, the default value of `tidb_analyze_version` does not change after the upgrade. | ||||||
|
|
||||||
| Version 2 is preferred, and will continue to be enhanced to ultimately replace Version 1 completely. Compared to Version 1, Version 2 improves the accuracy of many of the statistics collected for larger data volumes. Version 2 also improves collection performance by removing the need to collect Count-Min sketch statistics for predicate selectivity estimation, and also supporting automated collection only on selected columns (see [Collecting statistics on some columns](#collect-statistics-on-some-columns)). | ||||||
| Version 2 is the recommended statistics version. Compared to Version 1, Version 2 improves the accuracy of many statistics for larger data volumes. Version 2 also improves collection performance by removing the need to collect Count-Min sketch statistics. | ||||||
|
Check warning on line 368 in statistics.md
|
||||||
|
|
||||||
| The following table lists the information collected by each version for usage in the optimizer estimates: | ||||||
|
|
||||||
|
|
@@ -376,29 +380,22 @@ | |||||
|
|
||||||
| ### Switch between statistics versions | ||||||
|
|
||||||
| It is recommended to ensure that all tables/indexes (and partitions) utilize statistics collection from the same version. Version 2 is recommended, however, it is not recommended to switch from one version to another without a justifiable reason such as an issue experienced with the version in use. A switch between versions might take a period of time when no statistics are available until all tables have been analyzed with the new version, which might negatively affect the optimizer plan choices if statistics are not available. | ||||||
| It is recommended that all tables, indexes, and partitions use the same statistics version. If your cluster still uses Statistics Version 1, migrate to Statistics Version 2 as soon as possible. Until Version 2 statistics are collected for an object (such as a table, an index, or a partition), TiDB continues to use the existing Version 1 statistics for that object. | ||||||
|
|
||||||
| Examples of justifications to switch might include - with Version 1, there could be inaccuracies in equal/IN predicate estimation due to hash collisions when collecting Count-Min sketch statistics. Solutions are listed in the [Count-Min Sketch](#count-min-sketch) section. Alternatively, setting `tidb_analyze_version = 2` and rerunning `ANALYZE` on all objects is also a solution. In the early release of Version 2, there was a risk of memory overflow after `ANALYZE`. This issue is resolved, but initially, one solution was to set `tidb_analyze_version = 1` and rerun `ANALYZE` on all objects. | ||||||
| One major reason to migrate is that Version 1 might produce inaccurate estimates for equal/IN predicates because the Count-Min sketch can have hash collisions. For more information, see [Count-Min Sketch](#count-min-sketch). To avoid this issue, set `tidb_analyze_version = 2` and rerun `ANALYZE` on all objects. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| To prepare `ANALYZE` for switching between versions: | ||||||
| To prepare `ANALYZE` for migrating from Statistics Version 1 to Statistics Version 2: | ||||||
|
|
||||||
| - If the `ANALYZE` statement is executed manually, manually analyze every table to be analyzed. | ||||||
|
|
||||||
| ```sql | ||||||
| SELECT DISTINCT(CONCAT('ANALYZE TABLE ', table_schema, '.', table_name, ';')) | ||||||
| FROM information_schema.tables JOIN mysql.stats_histograms | ||||||
| ON table_id = tidb_table_id | ||||||
| WHERE stats_ver = 2; | ||||||
| WHERE stats_ver = 1; | ||||||
| ``` | ||||||
|
|
||||||
| - If TiDB automatically executes the `ANALYZE` statement because the auto-analysis has been enabled, execute the following statement that generates the [`DROP STATS`](/sql-statements/sql-statement-drop-stats.md) statement: | ||||||
|
|
||||||
| ```sql | ||||||
| SELECT DISTINCT(CONCAT('DROP STATS ', table_schema, '.', table_name, ';')) | ||||||
| FROM information_schema.tables JOIN mysql.stats_histograms | ||||||
| ON table_id = tidb_table_id | ||||||
| WHERE stats_ver = 2; | ||||||
| ``` | ||||||
| - If TiDB automatically executes the `ANALYZE` statement because auto-analysis is enabled, after you set `tidb_analyze_version = 2`, TiDB gradually refreshes statistics to Version 2 through subsequent auto-analysis. Before Version 2 statistics are collected for an object, TiDB can continue to use its existing Version 1 statistics. To speed up the migration for important objects, run `ANALYZE` on them manually. | ||||||
|
|
||||||
| - If the result of the preceding statement is too long to copy and paste, you can export the result to a temporary text file and then perform execution from the file like this: | ||||||
|
|
||||||
|
|
||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -1129,6 +1129,10 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a | |||||
|
|
||||||
| ### tidb_analyze_version <span class="version-mark">New in v5.1.0</span> | ||||||
|
|
||||||
| > **Warning:** | ||||||
| > | ||||||
| > Starting from v8.5.6, statistics Version 1 (`tidb_analyze_version = 1`) is deprecated and will be removed in a future release. It is recommended that you use `tidb_analyze_version = 2`. | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| - Scope: SESSION | GLOBAL | ||||||
| - Persists to cluster: Yes | ||||||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||||||
|
|
||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.