python multiprocessing Pool with map_async -
i trying use multiprocessing package in python pool.
i have function f called map_async function:
from multiprocessing import pool def f(host, x): print host print x hosts = ['1.1.1.1', '2.2.2.2'] pool = pool(processes=5) pool.map_async(f,hosts,"test") pool.close() pool.join()
this code has next error:
traceback (most recent call last): file "pool-test.py", line 9, in <module> pool.map_async(f,hosts,"test") file "/usr/lib/python2.7/multiprocessing/pool.py", line 290, in map_async result = mapresult(self._cache, chunksize, len(iterable), callback) file "/usr/lib/python2.7/multiprocessing/pool.py", line 557, in __init__ self._number_left = length//chunksize + bool(length % chunksize) typeerror: unsupported operand type(s) //: 'int' , 'str'
i don't know how pass more 1 argument f function. there way?
"test"
interpreted map_async
's chunksize
keyword argument (see the docs).
your code should (here copy-pasted ipython session) :
from multiprocessing import pool def f(arg): host, x = arg print host print x hosts = ['1.1.1.1', '2.2.2.2'] args = ((host, "test") host in hosts) pool = pool(processes=5) pool.map_async(f, args) pool.close() pool.join() ## -- end pasted text -- 1.1.1.1 test 2.2.2.2 test
note: in python 3 can use starmap
, unpack arguments tuples. you'll able avoid doing host, x = arg
explicitely.
Comments
Post a Comment