[ACCEPTED]-Using defaultdict with multiprocessing?-defaultdict

Accepted answer
Score: 18

You can subclass BaseManager and register additional 5 types for sharing. You need to provide a 4 suitable proxy type in cases where the default 3 AutoProxy-generated type does not work. For defaultdict, if 2 you only need to access the attributes that 1 are already present in dict, you can use DictProxy.

from multiprocessing import Pool
from multiprocessing.managers import BaseManager, DictProxy
from collections import defaultdict

class MyManager(BaseManager):
    pass

MyManager.register('defaultdict', defaultdict, DictProxy)

def test(k, multi_dict):
    multi_dict[k] += 1

if __name__ == '__main__':
    pool = Pool(processes=4)
    mgr = MyManager()
    mgr.start()
    multi_d = mgr.defaultdict(int)
    for k in 'mississippi':
        pool.apply_async(test, (k, multi_d))
    pool.close()
    pool.join()
    print multi_d.items()
Score: 3

Well, the Manager class seems to supply only a 6 fixed number of predefined data structures 5 which can be shared among processes, and 4 defaultdict is not among them. If you really just need 3 that one defaultdict, the easiest solution would be 2 to implement the defaulting behavior on 1 your own:

def test(k, multi_dict):
    if k not in multi_dict:
        multi_dict[k] = 0
    multi_dict[k] += 1

More Related questions