11.1 爬虫的多线程与多进程

#### 11.1.1 多线程爬虫

多线程可以同时发起多个请求，提高爬取效率。Python的`threading`模块可以方便地实现多线程爬虫。

Python复制

```python
import threading
import requests

def fetch_url(url):
    response = requests.get(url)
    print(f"Fetched {url}, Status Code: {response.status_code}")

urls = ["https://jsonplaceholder.typicode.com/posts/1", "https://jsonplaceholder.typicode.com/posts/2"]

threads = []
for url in urls:
    thread = threading.Thread(target=fetch_url, args=(url,))
    threads.append(thread)
    thread.start()

for thread in threads:
    thread.join()
```

#### 11.1.2 多进程爬虫

多进程可以利用多核CPU的优势，进一步提升爬取效率。Python的`multiprocessing`模块可以实现多进程爬虫。

Python复制

```python
from multiprocessing import Process
import requests

def fetch_url(url):
    response = requests.get(url)
    print(f"Fetched {url}, Status Code: {response.status_code}")

urls = ["https://jsonplaceholder.typicode.com/posts/1", "https://jsonplaceholder.typicode.com/posts/2"]

processes = []
for url in urls:
    process = Process(target=fetch_url, args=(url,))
    processes.append(process)
    process.start()

for process in processes:
    process.join()
```

###