Add --runs flag to typecheck benchmark daily runner

[yangdanny97]

A single run per day seems like it could be noisy, maybe we can do 3 or 5 runs for more stable results. The job already takes 2 hours so we probably shouldn't do more than that

Example run

  python3 -m typecheck_benchmark \
    --packages 5 \
    --checkers pyright pyrefly ty mypy zuban \
    --runs 3 \
    --timeout 300 \
    --output typecheck_benchmark/results \
    --os-name macos

[lolpack]

I worry about the length of these runs already. Is there another approach that doesn't make this take X times longer?

[yangdanny97]

You could shard the runs so that we check 20-25 packages per shard something, and merge the results later.

That's what we do for mypy primer

[yangdanny97]

We should probably also do a warmup run where we throw away the results, to reduce variance further