Linux

Python multiprocessing

Python Multiprocessing is similar to multithreading, instead of creating threads multiprocessing create sub-processes. By using subprocess multiprocessing evade the GIL. ( Global Interpreter Lock) It runs on both Unix and Windows.

The communication between the processes using manager

The Python processes does not share the normal lists and dictionary. By creating the list through the manager, it is shared and updates are seen in all processes. Dictionaries are also supported. Let's understand the working of manager class by examples.
Python multiprocessing communication
Python process manager
Click to view code

In above example, the list is created by manager class, can be shared between the processes.

Python Multiprocessing Queue

A simple way to pass the message between processes is to use a Queue. Unlike Multithreading the Multiprocessing use own Queue.
Let us discuss with the example of producer and consumer problem.
Python multiprocessing
Python multiple process Queue
Click to view code

You can see, Queue has been created by using multiprocessing module. You can not create a global queue. So we have passed queue to processes in arguments.
One process is enqueue and another process is dequeue.
The multiprocessing Queue implements all the methods of Queue.Queue except for task_done() and join().






admin