feat: framework for parallelizable statistic computation#77
feat: framework for parallelizable statistic computation#77laggycomputer wants to merge 35 commits intomainfrom
Conversation
dfc44cb to
921bc53
Compare
# Conflicts: # examples/from_vcf_indexed.rs # src/stats.rs
15aae01 to
63569fc
Compare
# Conflicts: # examples/from_vcf_indexed.rs
|
It seems that we need some tests? A computation done in parallel should be "close enough" to the serial version? |
laggycomputer
left a comment
There was a problem hiding this comment.
We definitely need tests, and we should at least prepare, if not also provide, a sketch demonstrating the use of indexed queries to do parallelism without the bottleneck of writing and cloning a records on a single thread
# Conflicts: # src/stats.rs
|
As discussed today:
|
|
@laggycomputer -- is this ready for more look/review? |
|
We're still missing one other stat, but feel free to look at the test for pi to see if it makes sense |
|
I think that the strategy looks sound. Boy, it would be handy to abstract this out for use for other stats. (Can definitely not do that for this PR though!) |
molpopgen
left a comment
There was a problem hiding this comment.
Approve. Only issue (which we can address later) is if we want anything "parallel" to be feature-gated? I'm agnostic on that at the moment.
SiteComposabletrait, guaranteeing that aGlobalStatisticcan be computed from its parts indepedently, concurrently, and potentially out-of-order.Use this trait to create generic code capable of parallelizing any such statistic.There is an example instead.