multi-threaded & multi-process in Python – Penetration Testing

Use Python multiprocessing (multiprocessing) enables multiple CPU can work at the same time, greatly enhance the utilization efficiency.

IO-intensive and CPU intensive

CPU-intensive: linear program execution, CPU intensive, always close to 100% (eg: replacing a large number of regular text matching)
IO-intensive: a lot of time spent waiting for program I/O operation, CPU always idle, at about 10% (such as network request)

multi-threaded and multi-process

Python multi-threaded and other programming languages are very different from the multi-threaded, Python multi-threaded even in the multi-core CPU host can only use a CPU, not the true sense of the concurrent implementation.

For IO-intensive programs, the use of multi-threaded technology can wait at intervals of CPU blocking the implementation of other operations, you can improve CPU efficiency.

For CPU-intensive programs, the original CPU is a busy state, forced the use of multi-threaded technology, will not increase the efficiency of CPU use, but increase the cost of thread switching, may lead to multi-threaded implementation efficiency is not as single-threaded Implementation efficiency. In the multi-core CPU, a host can use python multi-process (multiprocessing) makes the program using multiple CPU, you can achieve the program in the real sense of the parallel (the number of processes does not exceed the CPU core).



#!/usr/bin/env python
# -*- coding: utf-8 -*-
import Queue
import threading
queueLock = threading.Lock()
workQueue = Queue.Queue()
thread_num = 11
threads = []
class MyThread (threading.Thread):
def __init__(self, Queue,id):
self.q = Queue = id
def run(self):
while not workQueue.empty():
def get_info(data,threading_id):
for i in range(1000000):
def main():
for i in range(1000):
for i in range(thread_num):
thread = MyThread(workQueue, i)
for t in threads:
if __name__ == ‘__main__’:

Run the Python program to monitor CPU usage, Only one CPU is in use


#!/usr/bin/env python
# -*- coding: utf-8 -*-
import Queue
from multiprocessing import Process
processors = []
processor_num = 11
workQueue = Queue.Queue()
def single_processor(q,id):
while not workQueue.empty():
def get_info(data,processor_id):
for i in range(1000000):
def main():
for i in range(1000):
for i in range(processor_num):
p = Process(target=single_processor, args=(workQueue,i))
for i in processors:
if __name__ == ‘__main__’:

Run the Python program, monitor the use of the CPU can be seen that 11 CPU are in 100% of the use of state

The post multi-threaded & multi-process in Python appeared first on Penetration Testing.


Deixe um comentário

Preencha os seus dados abaixo ou clique em um ícone para log in:

Logotipo do

Você está comentando utilizando sua conta Sair / Alterar )

Imagem do Twitter

Você está comentando utilizando sua conta Twitter. Sair / Alterar )

Foto do Facebook

Você está comentando utilizando sua conta Facebook. Sair / Alterar )

Foto do Google+

Você está comentando utilizando sua conta Google+. Sair / Alterar )

Conectando a %s