There are several articles and Python modules talking about how to do concurrent/distributed/parallel computing in Python. There is an article in Python wik addressing “Parallel Processing and Multiprocessing in Python” which is a good start point for that. But the most interests me is how to make map parallel. One reason is that I started functional programming for a while and I started to switch my mind from imperative for-loop + if-else into map + filter + reduce + comprehension list; the other reason is that parallel map is just so good to boost up your program without chaning your code a lot.

Here is a snippet from Wai Yip Tung on ActiveState Code.

The first hack for this is to make it capable of accepting iterable argument, like an iterator by itertools, or something from your generator function. My idea is just to remove the code about initializing result with pre-determined size, but instead, making use of dictionary for keeping returns from each thread.

And then, it would be great to have this concurrent map supporting multiple arguments, just like a normal map function. I make use of *args (the anonymous, variant-length, positional arguments with zip to do that.