Python Multiprocessing, Can't Pickle Thread.lock (pymongo)

I have a class with the following method: def get_add_new_links(self, max_num_links): self.get_links_m2(max_num_links) processes = mp.cpu_count() pool = mp.Pool(process

Solution 1:

As stated in the docs:

Never do this:

client = pymongo.MongoClient()

# Each child process attempts to copy a global MongoClient# created in the parent process. Never do this.deffunc():
  db = client.mydb
  # Do something with db.proc = multiprocessing.Process(target=func)
proc.start()

Instead, inside the worker function a client must be initialized.