建造您自己的 SERP(搜索结果页) API

通常在进行搜索引擎优化(SEO)的时候,我们第一步要做的就是分析关键词的搜索结果页(SERP) 在以前,我们通常都是人工搜索相应的关键词,然后获取搜索结果,这种方法费时费力, 而且对我们真正需要解决的问题并没有多少帮助。

注解

一般情况我们只对搜索结果页的内容感兴趣,对获取结果的过程并不感兴趣

注解

SERP详细的解释: https://ahrefs.com/blog/zh/serps/

因此,如果有 工具/API 能帮助我们自动获取关键词的搜索结果就好了。

不幸的是,大多数搜索引擎并不提供对外的API

注解

Google 提供搜索API: Google Custom Search

价格约为: 5美金/千次搜索

现有的解决方案

虽然各个搜索引擎并不对外开放 API,但是有不少第三方提供相应的服务。

警告

以下的第三方跟我们没有任何关系,我们也并没有使用过他们提供的服务。

https://serpapi.com/

这是一家专门提供搜索引擎查询接口的公司, 他们提供了简单易用的 API

最大的缺点是他们的价格昂贵, 约为(量大优惠): 1$/100次请求

../../_images/price.png

亮数据

这是一家专门为爬虫提供代理服务的公司 1

他们也提供了针对搜索引擎抓取的服务, 即用即付价格为: 5$/1000次请求

亮数据搜索引擎抓取价格

注解

看起来他们的搜索引擎服务仅仅支持 Google,并没有说支持其他家。

自建搜索引擎 API

虽然各个搜索引擎对外并不提供接口,但是我们可以借助浏览器自动化 2 来完成搜索引擎结果的提取。

以下是演示的示例:

../../_images/search_ui.png

当我们输入 python 搜索:

百度的搜索结果

../../_images/baidu_result.png

Google 的搜索结果

../../_images/google_result.png

数据提取的源代码 0

API 接口

API 请求示例:

#!/usr/bin/env bash

curl --request POST \
    --url http://127.0.0.1:18082/v1/search \
    --header 'Authorization: Bearer xxx' \
    --header 'Content-Type: application/json' \
    --data '{
	"keyword": "python",
	"se_list": ["百度搜索", "Google"]
}'

API 返回结果:

[
    {
        "se_name": "Google",
        "keyword": "python",
        "search_result": {
            "items": [
                {
                    "summary": "The official home of the Python Programming Language.",
                    "url": "https://www.python.org/",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Welcome to Python.org</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb tjvcx\">https://www.python.org</cite></div>",
                    "innerText": "\nWelcome to Python.org\nhttps://www.python.org",
                    "isAd": false
                },
                {
                    "summary": "The official home of the Python Programming Language.",
                    "url": "https://www.python.org/",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Welcome to Python.org</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb tjvcx\">https://www.python.org</cite></div>",
                    "innerText": "\nWelcome to Python.org\nhttps://www.python.org",
                    "isAd": false
                },
                {
                    "summary": "Python 基础教程Python 是一种解释型、面向对象、动态数据类型的高级程序设计语言。 Python 由Guido van Rossum 于1989 年底发明,第一个公开发行版发行于1991 年。",
                    "url": "https://www.runoob.com/python/python-tutorial.html",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Python 基础教程| 菜鸟教程</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb qLRx3b tjvcx\">https://www.runoob.com<span class=\"dyjrff qzEoUe\"> › python › python-tutorial</span></cite></div>",
                    "innerText": "\nPython 基础教程| 菜鸟教程\nhttps://www.runoob.com › python › python-tutorial",
                    "isAd": false
                },
                {
                    "summary": "Python(英國發音:/ˈpaɪθən/ 美國發音:/ˈpaɪθɑːn/)是一种广泛使用的解释型、高级和通用的编程语言。Python支持多种编程范型,包括函数式、指令式、结构化、面向对象 ...\n設計者: 吉多·范罗苏姆\n\t\n預覽版本: 3.10.0b4 (2021年7月10日,8天前 )\n\n\n穩定版本: 3.9.6 (2021年6月28日,20天前 )\n\t\n實作者: Python软件基金会\n‎歷史 · ‎語法 · ‎开发环境 · ‎應用",
                    "url": "https://zh.wikipedia.org/zh-hans/Python",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Python - 维基百科,自由的百科全书</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb qLRx3b tjvcx\">https://zh.wikipedia.org<span class=\"dyjrff qzEoUe\"> › zh-hans › Python</span></cite></div>",
                    "innerText": "\nPython - 维基百科,自由的百科全书\nhttps://zh.wikipedia.org › zh-hans › Python",
                    "isAd": false
                },
                {
                    "summary": "中文,免费,零起点,完整示例,基于最新的Python 3版本。 Python是一种计算机程序设计语言。你可能已经听说过很多种流行的编程语言,比如非常难学的C语言,非常流行 ...",
                    "url": "https://www.liaoxuefeng.com/wiki/1016959663602400",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Python教程- 廖雪峰的官方网站</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb qLRx3b tjvcx\">https://www.liaoxuefeng.com<span class=\"dyjrff qzEoUe\"> › wiki</span></cite></div>",
                    "innerText": "\nPython教程- 廖雪峰的官方网站\nhttps://www.liaoxuefeng.com › wiki",
                    "isAd": false
                },
                {
                    "summary": "プログラミング言語 Pythonの紹介・ダウンロードとインストール・Python入門講座など.",
                    "url": "https://www.python.jp/",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">python.jp: プログラミング言語 Python 総合情報サイト</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb tjvcx\">https://www.python.jp</cite></div>",
                    "innerText": "\npython.jp: プログラミング言語 Python 総合情報サイト\nhttps://www.python.jp",
                    "isAd": false
                },
                {
                    "summary": "PythonはGuido van Rossumが設計した動的型付け言語です。Pythonは多くのデベロッパーによってライブラリの開発が行われており、Webページから科学研究まで幅広く利用 ...",
                    "url": "https://qiita.com/tags/python",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Python - Qiita</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb qLRx3b tjvcx\">https://qiita.com<span class=\"dyjrff qzEoUe\"> › tags › python</span></cite></div>",
                    "innerText": "\nPython - Qiita\nhttps://qiita.com › tags › python",
                    "isAd": false
                },
                {
                    "summary": "「Python」はWebアプリケーション開発や機械学習、統計処理など、様々な分野で使われています。 このレッスンではPythonの基本的な使い方について学びます。",
                    "url": "https://prog-8.com/courses/python",
                    "title": "",
                    "innerHtml": "<br><h3 class=\"LC20lb DKV0Md\">Python | プログラミングの入門なら基礎から学べるProgate ...</h3><div class=\"TbwUpd NJjxre\"><cite class=\"iUh30 Zu0yb qLRx3b tjvcx\">https://prog-8.com<span class=\"dyjrff qzEoUe\"> › courses › python</span></cite></div>",
                    "innerText": "\nPython | プログラミングの入門なら基礎から学べるProgate ...\nhttps://prog-8.com › courses › python",
                    "isAd": false
                }
            ],
            "related": [
                "python教程",
                "python下载",
                "python廖雪峰",
                "python学习",
                "python是什么",
                "python @用法",
                "python中文",
                "python菜鸟"
            ],
            "pages": [
                {
                    "page": 2,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=10&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARA8"
                },
                {
                    "page": 3,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=20&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARA-"
                },
                {
                    "page": 4,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=30&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBA"
                },
                {
                    "page": 5,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=40&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBC"
                },
                {
                    "page": 6,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=50&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBE"
                },
                {
                    "page": 7,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=60&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBG"
                },
                {
                    "page": 8,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=70&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBI"
                },
                {
                    "page": 9,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=80&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBK"
                },
                {
                    "page": 10,
                    "url": "https://www.google.com/search?q=python&ei=NHz2YOu3BdSU-AbfvKDQBQ&start=90&sa=N&ved=2ahUKEwjroceFkfHxAhVUCt4KHV8eCFoQ8tMDegQIARBM"
                }
            ],
            "google": {
                "news": []
            }
        },
        "func_log": []
    },
    {
        "se_name": "百度搜索",
        "keyword": "python",
        "search_result": {
            "items": [
                {
                    "summary": "达内python编程培训课程,python+人工智能+数据分析,全领域教学,培养IT多面手!达内助你成为被追捧的AI工程师7天0元试听,试听后再付款!",
                    "url": "http://www.baidu.com/baidu.php?url=000000j0xnZgmfstWuBIBfW7trVLYOfVdLVjCIHsNMdDVuOyCVsOLS4tozRwPuqaX08r3TmW8FOWHK0StdLe_cXw1cMAO9B3phtBRPhnqVTBmwkAWwUT4MOgtO3wsaYgV2sxHdgHSGRbsr9aCVvu88avC65Al6EwldVNK1J4PpW45gwvWmnRMP8UBOXYJ0b_wPEYvXZ92mFUX5dbutLsQGUEj1G6.7b_ig4xQZCCNrBtJaU_ZFKsYcn--XSjES9exqxMSgvt_IL_1L4TOot_O4E91OWSrxZCCNrB1q_rxFvNeVSHkzuenh15gEo_3dvyu5A1xzOCXLOg4O-qEgCC_n4rzEIEzXrHzepS1WgEoLtpSM_LSR5ZF8txZG3ePdMH4p81F3eQ5ZGLqM-Wooer5HYtU5ZzY452lEzCy2N9h9mLIMW3tN0.U1Yk0ZDqTZ-YpAq80ZKGm1Ys0ZK1pyI85yD3mhwhPHFbrjP9m1bvuj9-mHbLnjnsuhndn19hmhfL0ZfqTZ-YpAq8_Xa11ng6YUR0pyYqnWcd0ATqUvNsT100Iybqmh7GuZR0TA-b5HD0mv-b5Hn3n6KVIjYknjD4g1DsnHIxnW0dnNtznjmzg1nkn1uxn1msnfKopHYs0ZFY5HTYPfKBpHYkPHNxnHR3g1csP7tznHIxPH010AdW5HD3PWnvnWf3PWPxnHbsrHnYnj0LPdtznjRkg1D1PjDvnjRkPHR3g1DdP-ts0Z7spyfqn0Kkmv-b5H00ThIYmyTqn0K9mWYsg100ugFM5H00TZ0qn0K8IM0qna3snj0snj0sn0KVIZ0qn0KbuAqs5H00ThCqn0KbugmqTAn0uMfqn0KspjYs0Aq15H00mMTqnH00UMfqn0K1XWY0mgPxpywW5gK1QyPV0A-bm1dcfbR0TA9YXHY0IA7zuvNY5Hnkg1nkP7tkrfKYIgnqnHmvP1fdPj6kP1RdPHTLPWD4n1n0ThNkIjYkPWcvP1mvn16LrHck0ZPGujdbmhcYPjK9Pj0snjnkuAn10AP1UHdawWf3fHTYnjfkfbDsrDnv0A7W5HD0TA3qn0KkUgfqn0KkUgnqn0KbugwxmLK95H00XMfqn0KVmdqhThqV5HKxn7tsg1Kxn7ts0Aw9UMNBuNqsUA78pyw15HKxn7tsg1Kxn7ts0ZK9I7qhUA7M5H00uAPGujYknjT1P1fkrjcY0ANYpyfqQHD0mgPsmvnqn0KdTA-8mvnqn0KkUymqn0KhmLNY5H00pgPWUjYs0A7buhk9u1Yk0Akhm1Ys0AwWmvfq0Zwzmyw-5HmLnjRsnfKBuA-b5HDLfRwDPjT3rRPDPHDLP1ndrHnknDnznYn1fWPaPbP70AqW5HD0mMfqnsKEmgwL5H00ULfqnfKETMKY5HcWnankrin1c1TsPW0snHfWPjbsnj0WPjbsnj08nj0snj0sc1DWnznsc1DLc108njckP1Tvc108P1bsnj0s0Z91IZRqnWfkPHbzP1c0TNqv5H08rHuxna3sn7tsQW0sg108njNxna3sP7tkQW0sg108nHuxnHc30AF1gLKzUvwGujYs0ZFEpyu_myTqnsKWIWY0pgPxmLK95H00mL0qn0K-TLfqn0KWThnqnHRdPjf&us=newvui&xst=mWYkPY7DwjfLrj-jwjRkP1T1PHb1nHKjnWPjnYc1fWujwf715HDdrjnkn1f4PWm1nWn1nHRYPjb4g1czPNts0gTqTZ-YpAq8_Xa11ng6YURKTHdsXgwCUv3KIHYzPjDdrHcLn67Y5HDvnWmLPWm1rj6KUgDqn0cs0BYKmv6quhPxTAnKn1nvnHczrj6Yrf&cegduid=nWfkPHbzP1c&solutionId=7177041&word=&ck=0.0.0.0.0.0.0.0&shh=www.baidu.com",
                    "title": "",
                    "innerHtml": "<font color=\"#CC0000\">python</font>编程培训_可低基础入门_开启IT新生涯",
                    "innerText": "python编程培训_可低基础入门_开启IT新生涯",
                    "isAd": true
                },
                {
                    "summary": "The official home of the Python Programming Language",
                    "url": "http://www.baidu.com/link?url=s8wmGIpoe3JYtjsqMGte03ikomzjd7T4YjisrNWQfmf62alfkf_w5fdD2uMC1ofL",
                    "title": "",
                    "innerHtml": "Welcome to <em>Python</em>.org",
                    "innerText": "Welcome to Python.org",
                    "isAd": false
                },
                {
                    "summary": "Python由荷兰数学和计算机科学研究学会的Guido van Rossum 于1990 年代初设计,作为一门叫做ABC语言的替代品。Python提供了高效的高级数据结构,还能简单有效地面向对象编程。Python语法和动态类型,以及解释型语言的本质,使它成为多数平台上写脚本和快速...\n\n发展历程 语言特点 基本语法 帮助 接口 应用领域 更多 >\n\nbaike.baidu.com/",
                    "url": "http://www.baidu.com/link?url=Xv5QNNI_5QYWmZg6iVtXS_2vmGoJI6WycZOK9dSrMS_5J_5PaD81bmB65hDVuUuah7Jm-E9zD17sGuiUsNoox_",
                    "title": "",
                    "innerHtml": "\n                                        <em>Python</em>(计算机编程语言) - 百度百科\n                    ",
                    "innerText": "Python(计算机编程语言) - 百度百科",
                    "isAd": false
                },
                {
                    "summary": "",
                    "url": "http://www.baidu.com/aladdin.php?url=000000awulRvdnrhKNZ5APiWbyHg2isF_6YpvaG8PR09kRfyN82a0w_zOsQIU1pwrOCfQKkNl7lWEkRL8BcryTzHpgwDlqRiX4fnQGFUtCYDmxWlimhXzQRrMSRPht7IaYaSbSC7VR-z6HA2NxNM3p9HNs00DHupj6n1no0tNiOk2ZJbQSpzN7KINsgt-nOJ3kjM3U35dK_NDzv5GmzzzRphCpkH.7Y_a6heKzfupZwGCfprAj9CfRgkmrBjZCCNrB6eOCXLOg4O-qEQ-Cf1X15-CuOg4uO1xjMOZtTePAe5fxfCG14P-MZx_o34vpPvxFuu_zIP----hH-OfqESE1OQeXJ-muCy2qSek36.THdsXgwCUv30uhN-uA-b5HDkn1bsr0KGTdqLpgF-UAN1T1Ys0ANzTAPEuARqRjcLnHD0mLFW5HT4rHf",
                    "title": "",
                    "innerHtml": "达内-<em>Python</em>培训 血拼120天=2年工作经验+高薪就业 夏季名额...",
                    "innerText": "达内-Python培训 血拼120天=2年工作经验+高薪就业 夏季名额...",
                    "isAd": false
                },
                {
                    "summary": "",
                    "url": "https://www.baidu.com/sf/vsearch?pd=video&tn=vsearch&lid=dbb440a400031dc3&ie=utf-8&rsv_pq=dbb440a400031dc3&wd=python&rsv_spt=5&rsv_t=dcc5il457t2aDVqCkhZCWup4lFHP%2BmOuEPDUc3T0v3ztfbKdZbLRhkznN9A&rsv_bp=1&f=8",
                    "title": "",
                    "innerHtml": "\n                                        <em>python</em> - 视频大全 - 高清在线观看\n                    ",
                    "innerText": "python - 视频大全 - 高清在线观看",
                    "isAd": false
                },
                {
                    "summary": "3小时前 Python是一门强类型的动态类型语言,开发者可以给对象动态指定类型(动态),但类型不匹配的操作是不被允许的(强类型,如str和int两个变量无法相加)。动态类型帮助开发者写代码轻松愉快...",
                    "url": "http://www.baidu.com/link?url=Q_Inke4aJQs1ufObwOIDilA6mrzzEe_AwDLXYwkEB3k5xYZnIwlu3d8EZiQ0AUIsN7M00fBSdrq-bmoZoQ8HEuIeSp0QD0R6E0XYd741IKm",
                    "title": "",
                    "innerHtml": "<em>Python</em>静态类型解析工具简介和实践",
                    "innerText": "Python静态类型解析工具简介和实践",
                    "isAd": false
                },
                {
                    "summary": "The official home of the Python Programming Language",
                    "url": "http://www.baidu.com/link?url=kzOHi6AXj-9IWVTp8wrE61CIxt1dqHVLbAX1Cxej-2IQsMFbv6w6texteCj0zusP",
                    "title": "",
                    "innerHtml": "Download <em>Python</em> | <em>Python</em>.org",
                    "innerText": "Download Python | Python.org",
                    "isAd": false
                },
                {
                    "summary": "",
                    "url": "http://www.baidu.com/link?url=y4WsUBQrtcCOOxsBx5EmHo6N9Uq0NYXqgV7pupGCTTBaQ6JlGj57hEqvkHQy0WUFIErtpWBlOK9dA29wO3da0a",
                    "title": "",
                    "innerHtml": "\n                                        <em>python</em>吧 - 百度贴吧\n                    ",
                    "innerText": "python吧 - 百度贴吧",
                    "isAd": false
                },
                {
                    "summary": "2020年1月1日 本教程适合想从零开始学习 Python 编程语言的开发人员。当然本教程也会对一些模块进行深入,让你更好的了解 Python 的应用。 本教程主要针对 Python 2.x 版本的学习,如果你使用的是 Py...",
                    "url": "http://www.baidu.com/link?url=QdcBJHdWPDVlfmP694qai1tN2PKKtXlTdHpV_Mwo9YVALmDU91TDebgnzlNwOoIYUJCXzp4jMT-acY_ByZ1hcK",
                    "title": "",
                    "innerHtml": "<em>Python</em> 基础教程 | 菜鸟教程",
                    "innerText": "Python 基础教程 | 菜鸟教程",
                    "isAd": false
                }
            ],
            "related": [
                "python培训",
                "python编程",
                "python入门",
                "免费python全套教程",
                "python教程",
                "python基础知识",
                "python怎么学",
                "python下载",
                "python编程例子"
            ],
            "pages": [
                {
                    "page": 2,
                    "url": "https://www.baidu.com/s?wd=python&pn=10&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 3,
                    "url": "https://www.baidu.com/s?wd=python&pn=20&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 4,
                    "url": "https://www.baidu.com/s?wd=python&pn=30&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 5,
                    "url": "https://www.baidu.com/s?wd=python&pn=40&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 6,
                    "url": "https://www.baidu.com/s?wd=python&pn=50&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 7,
                    "url": "https://www.baidu.com/s?wd=python&pn=60&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 8,
                    "url": "https://www.baidu.com/s?wd=python&pn=70&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 9,
                    "url": "https://www.baidu.com/s?wd=python&pn=80&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                },
                {
                    "page": 10,
                    "url": "https://www.baidu.com/s?wd=python&pn=90&oq=python&ie=utf-8&usm=5&rsv_pq=dbb440a400031dc3&rsv_t=2e42A1FSWmef%2BXnBC2LT28%2Bats4YHGj9I78P%2B%2FRNgMsCE5FBxH3ZVUuG%2B5I&topic_pn="
                }
            ],
            "baidu": {
                "hots": [
                    {
                        "position": 1,
                        "title": "卫健委:5月31日后均可生三孩",
                        "url": "https://www.baidu.com/s?wd=%E5%8D%AB%E5%81%A5%E5%A7%94%3A5%E6%9C%8831%E6%97%A5%E5%90%8E%E5%9D%87%E5%8F%AF%E7%94%9F%E4%B8%89%E5%AD%A9&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=b7d4UE2vk%2FEF5T1ikrrixsR%2BoJEc7DxbX4ocjjOmgDX9MlVf0YgvkmS%2Bw8s&rsf=b721dda9eea90f19667ebe756116e182_1_10_1&rqid=dbb440a400031dc3",
                        "inner_text": "卫健委:5月31日后均可生三孩",
                        "hot_value": "495万"
                    },
                    {
                        "position": 2,
                        "title": "台风查帕卡正面袭击广东",
                        "url": "https://www.baidu.com/s?wd=%E5%8F%B0%E9%A3%8E%E6%9F%A5%E5%B8%95%E5%8D%A1%E6%AD%A3%E9%9D%A2%E8%A2%AD%E5%87%BB%E5%B9%BF%E4%B8%9C&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=23d7rNan1RQTNe7Ygrm5%2Bmt1xwjCjMDPK0d6Co2GUJ2VzuspIr4kn1Q7c1k&rsf=b721dda9eea90f19667ebe756116e182_1_10_2&rqid=dbb440a400031dc3",
                        "inner_text": "台风查帕卡正面袭击广东",
                        "hot_value": "488万"
                    },
                    {
                        "position": 3,
                        "title": "阿富汗总统府疑似遭火箭弹袭击",
                        "url": "https://www.baidu.com/s?wd=%E9%98%BF%E5%AF%8C%E6%B1%97%E6%80%BB%E7%BB%9F%E5%BA%9C%E7%96%91%E4%BC%BC%E9%81%AD%E7%81%AB%E7%AE%AD%E5%BC%B9%E8%A2%AD%E5%87%BB&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=87efmB63Bb5KrK43QtFf8PWSVhammKWVFLcn%2BU3tsrKS5h2dv%2B731WbbbOg&rsf=b721dda9eea90f19667ebe756116e182_1_10_3&rqid=dbb440a400031dc3",
                        "inner_text": "阿富汗总统府疑似遭火箭弹袭击",
                        "hot_value": "473万"
                    },
                    {
                        "position": 4,
                        "title": "都美竹疑被带到医院做新冠筛查",
                        "url": "https://www.baidu.com/s?wd=%E9%83%BD%E7%BE%8E%E7%AB%B9%E7%96%91%E8%A2%AB%E5%B8%A6%E5%88%B0%E5%8C%BB%E9%99%A2%E5%81%9A%E6%96%B0%E5%86%A0%E7%AD%9B%E6%9F%A5&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=78b8xypsezNZBdukGCyxgKogkjc%2BOna6R7aG6r1Zk7%2FNyNfqUr%2B9dtpQlG4&rsf=b721dda9eea90f19667ebe756116e182_1_10_4&rqid=dbb440a400031dc3",
                        "inner_text": "都美竹疑被带到医院做新冠筛查",
                        "hot_value": "460万"
                    },
                    {
                        "position": 5,
                        "title": "河南遭遇极端强降雨",
                        "url": "https://www.baidu.com/s?wd=%E6%B2%B3%E5%8D%97%E9%81%AD%E9%81%87%E6%9E%81%E7%AB%AF%E5%BC%BA%E9%99%8D%E9%9B%A8&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=102bGjYnM01tV2Hhqvb9YZLS3HceOKZOMqSmx4ha%2FMXiv%2BQ%2BW%2BS7t82LUak&rsf=b721dda9eea90f19667ebe756116e182_1_10_5&rqid=dbb440a400031dc3",
                        "inner_text": "河南遭遇极端强降雨",
                        "hot_value": "450万"
                    },
                    {
                        "position": 6,
                        "title": "夫妻举报盗砂被碾亡 女儿不满1岁",
                        "url": "https://www.baidu.com/s?wd=%E5%A4%AB%E5%A6%BB%E4%B8%BE%E6%8A%A5%E7%9B%97%E7%A0%82%E8%A2%AB%E7%A2%BE%E4%BA%A1+%E5%A5%B3%E5%84%BF%E4%B8%8D%E6%BB%A11%E5%B2%81&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=59c4B9xpYT0CtlwX3kUAJjS7VPJurowYtOUFvAbyeLS7LCKkSEoUxMmTku8&rsf=b721dda9eea90f19667ebe756116e182_1_10_6&rqid=dbb440a400031dc3",
                        "inner_text": "夫妻举报盗砂被碾亡 女儿不满1岁",
                        "hot_value": "442万"
                    },
                    {
                        "position": 7,
                        "title": "兰蔻官宣与吴亦凡解约",
                        "url": "https://www.baidu.com/s?wd=%E5%85%B0%E8%94%BB%E5%AE%98%E5%AE%A3%E4%B8%8E%E5%90%B4%E4%BA%A6%E5%87%A1%E8%A7%A3%E7%BA%A6&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=cc70kosU9MVei5RjHfiECDZA3IsSllvFLhSjbqe2vmwKX1H9NObB%2ByqrDVk&rsf=b721dda9eea90f19667ebe756116e182_1_10_7&rqid=dbb440a400031dc3",
                        "inner_text": "兰蔻官宣与吴亦凡解约",
                        "hot_value": "438万"
                    },
                    {
                        "position": 8,
                        "title": "杭州涉事电瓶车商家首发声",
                        "url": "https://www.baidu.com/s?wd=%E6%9D%AD%E5%B7%9E%E6%B6%89%E4%BA%8B%E7%94%B5%E7%93%B6%E8%BD%A6%E5%95%86%E5%AE%B6%E9%A6%96%E5%8F%91%E5%A3%B0&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=0539%2BA%2F%2BhNfFPLFfqVF1nmNABgktEbP904xUemGdgTyextZgBEgV4LzJqqo&rsf=b721dda9eea90f19667ebe756116e182_1_10_8&rqid=dbb440a400031dc3",
                        "inner_text": "杭州涉事电瓶车商家首发声",
                        "hot_value": "421万"
                    },
                    {
                        "position": 9,
                        "title": "喜茶回应收购乐乐茶传言",
                        "url": "https://www.baidu.com/s?wd=%E5%96%9C%E8%8C%B6%E5%9B%9E%E5%BA%94%E6%94%B6%E8%B4%AD%E4%B9%90%E4%B9%90%E8%8C%B6%E4%BC%A0%E8%A8%80&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=6b96Njmb5MarD9QAsYtg6yUESNGGmsMBTbA3z63vlWXb6CwZUmLMZ%2FU%2FAjo&rsf=b721dda9eea90f19667ebe756116e182_1_10_9&rqid=dbb440a400031dc3",
                        "inner_text": "喜茶回应收购乐乐茶传言",
                        "hot_value": "411万"
                    },
                    {
                        "position": 10,
                        "title": "上半年人均消费榜:京沪超2万",
                        "url": "https://www.baidu.com/s?wd=%E4%B8%8A%E5%8D%8A%E5%B9%B4%E4%BA%BA%E5%9D%87%E6%B6%88%E8%B4%B9%E6%A6%9C%3A%E4%BA%AC%E6%B2%AA%E8%B6%852%E4%B8%87&usm=5&ie=utf-8&rsv_cq=python&rsv_dl=0_right_fyb_pchot_20811_01&rsv_pq=dbb440a400031dc3&oq=python&rsv_t=68093Li5nNtrWBjNxVX%2FJcanF8MQKx6Yttd1vOUjNyozPGfsI8KB6kE68EI&rsf=b721dda9eea90f19667ebe756116e182_1_10_10&rqid=dbb440a400031dc3",
                        "inner_text": "上半年人均消费榜:京沪超2万",
                        "hot_value": "409万"
                    }
                ],
                "news": []
            }
        },
        "func_log": []
    }
]

使用我们的工具, 您可以自建各个搜索引擎的 API, 更快的完成搜索引擎结果页面内容的获取 (当然还有其他更多的功能)。

其他使用场景

  1. 数据提取

  2. 网页监控

注解

我们会在未来的博客分享更多的信息,敬请关注。

参考资料

0

extract

1

亮数据

2

puppeteer