Add `__length_hint__` methods to `itertools` helpers

[KowalskiThomas]

Following-up on #145333, I would like to add more __length_hint__ methods to itertools helper iterators. Not all of them can have such a method (those that are not finite, like count or unknown like takewhile) but I think we could at least add them for starmap, batched, and possibly others like product, pairwise etc that will require some more attention.

I plan to work on this in the next few days.

[picnixz]

Please, wait before working on it. We need @rhettinger or a maintainer's consensus on that one. I think it's fine but would it be really useful in general cases?

[cmaloney]

Some background bits: __length_hint__ was added in PEP-424, discussed at https://mail.python.org/pipermail/python-dev/2012-July/120920.html, and implemented in gh-60352.

I think it's goood reading overall, some points that seem relevant to this:

• Enables a performance optimization: Replace continual re-allocation with an up front (over-)allocation reducing or eliminating copies and reallocations resulting in a significant speedup.
• There are interesting cases to consider from test_iterlen: https://mail.python.org/pipermail/python-dev/2012-July/120966.html
• How well passing through / calling __length_hint__ on underlying may work in various cases.

With that in mind I suspect for these would be useful to measure the performance tradeoff between having another function vs. how many elements are in the iterable. My intuition says small iterables (ex. 1-4 items) will be slightly slower while mid to long will see significant speedups. Being explicit about that cost/benefit tradeoff I think will be very useful. Most useful is if there are cases in pyperformance or well used public code bases which will speedup; pyperf microbenchmarks also potentially useful.

That definitely takes work to measure for a proposal and doesn't guarantee acceptance but will likely help get consensus. Significant performance gains in real-world code bases are very motivating :)

[KowalskiThomas]

Thanks so much for that comment, it's extremely helpful. I'll take some time to unravel that, read the things you recommended and eventually have a go at measuring the impact when I get a chance.

Anecdotally, I was wondering before making the PR whether this would be useful at all so I made some very basic benchmarks on just building lists where I saw noticeable improvement (order of 10-15%), but that was just one specific example.

I'll update this ticket with my progress !