5.2 解析HTML页面（BeautifulSoup、lxml）

#### 5.2.1 安装BeautifulSoup和lxml

在开始解析HTML页面之前，需要安装`beautifulsoup4`和`lxml`库。如果尚未安装，可以通过以下命令安装：

bash复制

```bash
pip install beautifulsoup4 lxml
```

#### 5.2.2 使用BeautifulSoup解析HTML

`BeautifulSoup`是一个用于解析HTML和XML文档的库，非常方便提取数据。

Python复制

```python
from bs4 import BeautifulSoup

html_doc = """
<html>
<head>
 <title>The Dormouse's story</title>
</head>
<body>
 The Dormouse's story
 <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>
 <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>
 <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>
</body>
</html>
"""

soup = BeautifulSoup(html_doc, 'lxml')

# 提取标题
print(soup.title.string)  # 输出: The Dormouse's story

# 提取所有链接
for link in soup.find_all('a'):
    print(link.get('href'))  # 输出链接的href属性
```

#### 5.2.3 使用lxml解析HTML

`lxml`是一个高效的HTML和XML解析库，支持XPath语法，适合复杂的数据提取。

Python复制

```python
from lxml import etree

html_doc = """
<html>
<body>
 The Dormouse's story
 <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>
 <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>
 <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>
</body>
</html>
"""

# 将HTML文档解析为lxml.etree
tree = etree.HTML(html_doc)

# 使用XPath提取数据
title = tree.xpath('//p[@class="title"]/b/text()')
print(title)  # 输出: ['The Dormouse's story']

links = tree.xpath('//a/@href')
print(links)  # 输出: ['http://example.com/elsie', 'http://example.com/lacie', 'http://example.com/tillie']
```

###