gh-135883: Fix sqlite3 CLI keeps part of previous commands when scrolling history

[tanloong]

Non-printing characters in the prompt need to be surrounded with \001 and \002 so readline knows those characters don't occupy screen space on redisplay.

Reference: https://lists.gnu.org/archive/html/bug-readline/2017-04/msg00018.html

• Issue: sqlite3 CLI keeps part of previous commands when scrolling history #135883

[StanFromIreland]

@ambv Maybe it would be better to implement this in _colorize.py instead, it could be an option for themes with_prompt_ignore=True?

[tanloong]

Hi @erlend-aasland, could you take a look at this when you have some time? Thank you very much!

[tanloong]

Hi, the issue this PR addresses is still present in main branch. Would it be possible to get a review from core developers? Thank you very much!

[tanloong]

Thank you for the review! I've committed the change.

It confused me why test_cli.Completion should use a plain "sqlite> " while test_cli.InteractiveSession should use "\001\002sqlite> \001\002" to make tests pass. After some digging: The Completion class uses run_pty() which creates real terminal environment, where SqliteInteractiveConsole invokes GNU Readline during running input(prompt). Readline goes through the call chain (readline(), readline_internal(), readline_internal_setup()) and finally calls _rl_strip_prompt() stripping the \001 and \002 from the output.

I added a comment in Completion about this in case any future developers are confused as well.

[erlend-aasland]

Thank you so much for the fix! (Sorry 'bout the late review!)

[miss-islington-app]

Sorry, @tanloong and @erlend-aasland, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 745947cda0377bb44d791b7d8e58b327a9d84844 3.13

[erlend-aasland]

@tanloong, do you want to take a stab at the backports?