fiber.Pool runs tasks in serial in local mode
Hi,
I am trying fiber v0.2.1 on local machine. the experiment runs the same task on 4 processes. the task takes approx. 68 secs. while multiprocessing lib works as expected, fiber looks like to be running in serial.
.fiberconfig [default] log_level=info log_file=stdout backend=local
def process(mplib, problem, num_tasks): pool = mplib.Pool(processes=4) futs = [ pool.apply_async(simple_solve, kwds={"problem": problem}) for _ in range(num_tasks) ] return [fut.get() for fut in futs]
res, elapsed = timeit(process, mp, problem, num_tasks) res2, elapsed2 = timeit(process, fiber, problem, num_tasks)
print("multiprocessing takes {} secs, results = {}".format(elapsed, res)) print("fiber takes {} secs, results = {}".format(elapsed2, res2))
multiprocessing takes 92.11180663108826 secs, results = [{'pid': 27242, }, {'pid': 27240, }, {'pid': 27241, }, {'pid': 27243, }] fiber takes 255.9156596660614 secs, results = [{'pid': 27265, }, {'pid': 27265, }, {'pid': 27265, }, {'pid': 27265, }]
fiber is always using the same process pid 27265 either push or pull queue, while the multiprocessing lib is distributing the works to 4 worker processes. but fiber's parzen_estimation example is working as expected on local machine.