distributed downloading feature by synchronizing multiple computers and running a large distributed fork-join operation #40
Description
have two modes:
- many-to-many
or:
- many-to-one
this downloads to either one central computer or to the computers they are running on, but there would be 1 central computer that both initiates the main process of linking up with the other active nodes and then sending off sharded chunks of data to have processed.