Re: Parallel Collections fail
Aleksandar Prokopec <aleksandar.prokopec <at> gmail.com>
2011-07-04 15:25:06 GMT
I believe this is due to the fact that the `map` in `ParIterableLike`
checks whether the builder factory is parallel, instead of
instantiating the builder and checking if the builder is parallel.
I'll fix this.
Cheers,
Alex
On Jul 4, 1:18 pm, Johannes Rudolph <johannes.rudo... <at> googlemail.com>
wrote:
> Hi all,
>
> this weekend, I was stumped that abstracting over parallel and
> sequential collections with the generic GenSeq etc types is completely
> broken. See this example:
>
> val items = (0 to 10000)
> val parItems: collection.GenSeq[Int] = if (true) items.par else items.seq
> parItems.map {
> _ => Thread.currentThread.getName
>
> }
>
> ParVector(Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Thread-107, Thread-107, Thread-107, Thread-107,
> Thread-107, Threa...
>
> In summary, an operation is never executed in parallel if the static
> type of the collection is not a parallel one which is defeating the
> purpose of the generic collections construction completely. This is,
> because dynamic dispatch - as I would have expected it - is
> circumvented by deciding if something should be run in parallel by the
> instance of the CanBuildFrom which is chosen statically instead of at
> runtime.
>
> I find this a bit upsetting since we had quite a discussion months ago
> about the design of the parallel collections and it seems no one
> (including me) did actually test that the resulting implementation did
> what was promised.
>
> Another problem here is that you can't write
>
> val parItems = if (true) items.par else items.seq
>
> without the type annotation, because type inference fails badly here
> (though that seems to be fixed in trunk already).
>
> --Johannes
>
> -----------------------------------------------JohannesRudolphhttp://virtual-void.net