农业新闻内容获取 python

最新推荐文章于 2025-03-28 12:28:27 发布

原创最新推荐文章于 2025-03-28 12:28:27 发布 · 609 阅读

2 ·

本内容遵循CC 4.0 BY-SA版权协议

收录于

Python

当前文章被收录于：

Python

20 篇文章 0 人学习

订阅专栏查看详情

当前文章被以下社区和专栏收录：

代码：

#!-*- coding: utf-8 -*-
# @Time : 2020/4/20 22:19
# @Author : Oneqq
# @File : 27农业新闻内容获取.py
# @Software: PyCharm

import requests
from fake_useragent import UserAgent
from lxml import etree

url = "http://www.farmer.com.cn/2020/02/01/99847779.html"
headers = {
    "User-Agent": UserAgent().chrome
}
response = requests.get(url, headers=headers)
response.encoding = "utf-8"
e = etree.HTML(response.text)
title = e.xpath('string(//h1/text())')
content = e.xpath('//div[@class="article-main"]/p[@style="text-indent: 2em;"]')
all_p = []
for c in content:
    info = c.xpath('string(.)')
    all_p.append(info)
contents = ''.join(all_p)
print("标题：" + title)
print("内容：" + contents)
img_urls = e.xpath('//div[@class="article-main"]//img/@src')
img_names = e.xpath('//div[@class="article-main"]/p/span')
print("图片：")
i = 0
for img_name in img_names:
    img_name = img_name.xpath('string(.)')
    print("第" + str(i+1) + "张：" + img_name + ".jpg")
    url = img_urls[i]
    response = requests.get(url, headers=headers)
    with open('imgs/'+img_name + '.jpg', 'wb') as f:
        f.write(response.content)
    i += 1

结果：