Key takeaways
DuckDuckBot is DuckDuckGo's own web crawler, used mainly to verify URL availability, check link health, and gather data for DuckDuckGo's Instant Answers. It is far smaller in scale than Bingbot and is not a primary ranking input. DuckDuckGo sources most of its traditional links from Bing's index plus hundreds of other sources, so ranking on DuckDuckGo is largely downstream of ranking on Bing.
- DuckDuckBot uses the user agent
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)and crawls from a published list of over 200 IP addresses. - DuckDuckGo runs DuckDuckBot to support results and Instant Answers, but draws most traditional link and image results from Bing.
- Optimizing specifically for DuckDuckBot does not lift DuckDuckGo rankings; ranking well on Bing is the real lever.
- DuckDuckBot respects WWW::RobotRules, so you can allow or disallow it by name in robots.txt.
- DuckDuckBot is the traditional search crawler; DuckAssistBot is DuckDuckGo's separate AI-assist crawler.
What is DuckDuckBot?
Definition
DuckDuckBot is the web crawler operated by DuckDuckGo to verify URL availability, check link health, and assemble the data behind its Instant Answers. It is one of many inputs DuckDuckGo blends to build a results page, not the system that decides which links rank.
That distinction matters. DuckDuckGo states that it sources its traditional links and images largely from Bing, while also running its own crawler and many indexes alongside hundreds of other sources. So DuckDuckBot is a supporting crawler, useful for freshness and DuckDuckGo's own features, rather than the engine that ranks the web. For the system that does most of that ranking work, see our guide to Bing ranking algorithms.
DuckDuckBot at a glance
- Operator
- DuckDuckGo
- User agent
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)- Primary purpose
- URL verification, link health, Instant Answers data
- robots.txt
- Respects WWW::RobotRules
- IP addresses
- 200+, published at duckduckgo.com/duckduckbot.json
- Scale vs Bingbot
- Far smaller; a fraction of crawl volume
- Ranking input
- No or minor; rankings ride on Bing
- Related crawler
- DuckAssistBot (separate AI-assist bot)
What DuckDuckBot is and does
DuckDuckBot is DuckDuckGo's first-party crawler. In DuckDuckGo's own words, the bot exists to "constantly improve our search results and offer users the best and most secure search experience possible." In practice that breaks down into a few concrete jobs: confirming that URLs are still reachable, checking that links are healthy and not broken, and collecting the structured data DuckDuckGo needs to build its Instant Answers.
What DuckDuckBot is not is a full-scale indexing crawler trying to map the entire web on its own. DuckDuckGo describes its results as a blend of its own crawler, many indexes, Bing, and hundreds of additional sources. DuckDuckBot is the smallest and most targeted of those inputs. It fills gaps, verifies freshness, and feeds DuckDuckGo's own features, while the heavy lifting of indexing the open web happens elsewhere.
This is why DuckDuckBot shows up only lightly in server logs. Independent crawler-traffic data from Cloudflare places DuckDuckBot at roughly a tenth of a percent of crawl requests across its network, an order of magnitude below the major search crawlers. Low volume is expected behavior here, not a sign that DuckDuckGo is ignoring your site.
What DuckDuckBot crawls and why
Because DuckDuckGo leans on Bing for most of its link and image results, DuckDuckBot does not need to recrawl and reindex the whole web. Its crawling is purposeful and narrow, concentrated on the tasks DuckDuckGo cannot simply borrow from a partner index.
URL availability and link health
A search engine that shows a stale or dead link erodes trust fast. DuckDuckBot revisits URLs to confirm they still resolve, that they have not started returning errors, and that redirects behave as expected. This verification keeps DuckDuckGo's results clean even when the underlying link data came from another source.
Instant Answers data
DuckDuckGo's Instant Answers, the direct boxes that show definitions, conversions, or quick facts above the links, need their own freshly gathered and verified data. DuckDuckBot helps assemble and check that data so the answer shown is accurate and current.
Filling gaps and freshness checks
Where a partner index is thin, slow to update, or missing a page, DuckDuckBot can crawl directly to fill the gap. This makes it a complement to Bing's coverage rather than a replacement for it, which is the recurring theme of how DuckDuckGo's results are built.
DuckDuckBot vs Bingbot
The single most useful way to understand DuckDuckBot is to contrast it with Bingbot, because the two crawlers play very different roles even though DuckDuckGo depends on both.
Bingbot is Microsoft's large-scale production crawler. It discovers, fetches, and indexes the open web at enormous volume, and that index is what most of DuckDuckGo's traditional link results are drawn from. If you want your pages to appear in DuckDuckGo's links, Bingbot is the crawler that has to find and index them. The mechanics of how Bing then orders those results are covered in our explainers on Bing's Prometheus system and the SPTAG vector index.
DuckDuckBot is DuckDuckGo's small, first-party crawler. It does not try to index the web at Bingbot scale. Instead it verifies URLs, checks link health, and gathers Instant Answers data for DuckDuckGo specifically. It runs from a few hundred IP addresses rather than the vast infrastructure Microsoft devotes to Bingbot.
The practical implication is direct: DuckDuckBot crawling your site does not by itself determine your DuckDuckGo ranking. The page first has to be indexed by Bingbot and ranked by Bing's systems. DuckDuckBot's visit is more about verification and DuckDuckGo's own features than about deciding your position.
How DuckDuckGo actually ranks results: the Bing relationship
DuckDuckGo sources most of its traditional link results from Bing's index, so ranking on DuckDuckGo is largely downstream of ranking on Bing rather than a product of DuckDuckBot.
DuckDuckGo has a formal search syndication agreement with Microsoft, and it states that it draws traditional links and images in its results largely from Bing. The degree of that dependence became visible in 2024, when a Bing API outage caused DuckDuckGo to stop returning results, a strong signal that Bing supplies a substantial portion of DuckDuckGo's link output.
DuckDuckGo is careful to note it is more than a Bing wrapper. It combines Bing with its own crawler, its own indexes, and what it has described as hundreds of sources, layering privacy protection on top by proxying partner requests through its own servers so they stay anonymous. The 400-plus sources figure is DuckDuckGo's own characterization and has varied across its statements, so treat it as company framing rather than an audited count.
For the working SEO, the conclusion is clear and low-effort. You do not optimize for DuckDuckBot to climb DuckDuckGo's links. You optimize for Bing, because that is the index DuckDuckGo borrows from. The same technical and content work that earns Bing visibility, covered in our Bing ranking algorithms pillar, is what moves your DuckDuckGo rankings too.
Instant Answers: where DuckDuckBot adds direct value
Instant Answers are the direct result boxes DuckDuckGo displays above or beside its blue links, covering things like definitions, calculations, weather, and sports scores. They are the part of the DuckDuckGo experience the company builds most directly itself, and they are where DuckDuckBot's crawling has the clearest payoff.
To populate these answers, DuckDuckGo pulls from many sources, including specialized providers such as Sportradar and crowd-sourced sites such as Wikipedia. DuckDuckBot helps gather and verify the data that feeds them so that the box shown to a user is accurate and current. Historically, the community could also extend these answers through the DuckDuckHack project, which DuckDuckGo placed in maintenance mode in 2017.
Practically, this means clean, well-structured, factual pages, especially those with clear structured data, are the ones most likely to feed an Instant Answer. The lever is the same as for any modern engine: publish accurate content and mark it up clearly, and let the crawler do the rest.
robots.txt handling and verification
DuckDuckBot is a well-behaved crawler. DuckDuckGo states that it respects WWW::RobotRules, the standard implementation of the robots exclusion protocol. That means you can manage it by name in your robots.txt file exactly as you would any other compliant search crawler.
To control DuckDuckBot, add a named directive block. For example, to allow everything you would write a User-agent: DuckDuckBot group with an empty Disallow: line, and to block a path you would list it under Disallow: in that same group. Because the bot honors these rules, the directives take effect without any extra registration step.
Verification works through two signals together. First, check the request's user agent against the published string, DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html). Second, confirm the source IP against DuckDuckGo's official list of more than 200 addresses, published in JSON at duckduckgo.com/duckduckbot.json. A request that matches the user agent but comes from an IP outside that list is an impostor and should not be trusted. Checking both is the reliable way to tell a genuine DuckDuckBot visit from a spoofed one.
A short timeline
DuckDuckGo has always been a hybrid, pairing its own crawler with partner indexes, and that shape has held steady from launch to today.
-
2008
DuckDuckGo launches
Gabriel Weinberg launches DuckDuckGo as a privacy-focused search engine that blends its own crawler and indexes with third-party sources rather than building a single proprietary index.
-
2012
DuckDuckHack opens Instant Answers
DuckDuckGo launches DuckDuckHack, letting the community build and contribute Instant Answers, the direct result boxes DuckDuckBot helps populate.
-
2017
DuckDuckHack enters maintenance mode
DuckDuckGo places the DuckDuckHack community platform into maintenance mode, shifting Instant Answer development back in-house.
-
2022
Microsoft syndication terms revisited
DuckDuckGo updates the limits of its Microsoft search syndication agreement around tracker handling, underlining how closely its results are tied to the Bing partnership.
-
2024
Bing outage stops DuckDuckGo results
A Bing API outage causes DuckDuckGo to stop returning results, publicly demonstrating how much of its link output depends on Bing rather than on DuckDuckBot.
DuckDuckBot versus Bingbot at a glance
The two crawlers are easy to confuse because both matter to DuckDuckGo visibility. This table separates what each one actually does.
| Attribute | DuckDuckBot | Bingbot |
|---|---|---|
| Operator | DuckDuckGo | Microsoft Bing |
| Main job | Verify URLs, check link health, feed Instant Answers | Discover and index the open web at scale |
| Crawl scale | Small, narrow, low volume | Very large, broad, high volume |
| Drives link rankings? | No or minor | Yes, indirectly powers most DuckDuckGo links |
| Where to optimize | Nothing specific needed | Standard technical and content SEO for Bing |
The takeaway is that DuckDuckGo visibility is a Bing problem dressed up as a DuckDuckGo question. Get the page indexed and ranked by Bing, and DuckDuckBot's verification visits will follow naturally.
What site owners should actually do
There is very little DuckDuckBot-specific work to do. The short checklist below covers everything that genuinely affects your DuckDuckGo presence.
-
Let DuckDuckBot crawl
It is a legitimate, low-volume search crawler that keeps your DuckDuckGo results and Instant Answers accurate. There is rarely a good reason to block it.
-
Optimize for Bing, not for DuckDuckBot
DuckDuckGo draws most of its links from Bing's index, so standard technical and content SEO that earns Bing visibility is what moves your DuckDuckGo rankings.
-
Submit and verify your site in Bing Webmaster Tools
Getting Bingbot to discover and index your pages cleanly is the upstream step that makes your content eligible to appear on DuckDuckGo at all.
-
Keep links and redirects healthy
DuckDuckBot checks URL availability and link health, so fixing broken links and clean redirects keeps your pages presentable in DuckDuckGo's results.
-
Mark up factual content with structured data
Clear, accurate, well-structured pages are the ones most likely to feed DuckDuckGo's Instant Answers, the feature DuckDuckBot supports most directly.
-
Verify suspicious crawler hits before blocking
Match both the user agent and the source IP against DuckDuckGo's published list so you block spoofed traffic without accidentally shutting out the real bot.
DuckDuckBot myths vs. reality
DuckDuckBot attracts a surprising amount of misunderstanding, mostly because people assume it works like Googlebot. Here are the common myths and what is actually true.
Myth Optimizing for DuckDuckBot boosts your DuckDuckGo rankings.
Reality DuckDuckGo draws most of its traditional links from Bing's index, so rankings are largely downstream of Bing. Optimizing for Bing, not DuckDuckBot, is what moves your DuckDuckGo position.
Myth DuckDuckBot indexes the whole web like Googlebot or Bingbot.
Reality DuckDuckBot is a small, targeted crawler that verifies URLs and feeds Instant Answers. The large-scale indexing that powers DuckDuckGo's links is done by Bingbot.
Myth Low DuckDuckBot crawl volume means DuckDuckGo is ignoring your site.
Reality Light DuckDuckBot traffic is normal and expected, because DuckDuckGo relies on partner indexes for most coverage. Your visibility depends on Bing indexing, not on heavy DuckDuckBot activity.
Myth DuckDuckBot and DuckAssistBot are the same crawler.
Reality They are distinct user agents with different jobs. DuckDuckBot is the traditional search crawler; DuckAssistBot is a separate bot tied to DuckDuckGo's AI-assisted answers, and each can be controlled independently.
Myth You cannot tell a real DuckDuckBot from a fake one.
Reality You can. Match the request against the published user agent string and confirm the source IP against DuckDuckGo's list of over 200 addresses at duckduckgo.com/duckduckbot.json.
Frequently asked questions
DuckDuckBot is DuckDuckGo's own web crawler. DuckDuckGo uses it mainly to verify that URLs are reachable, check the health of links, and gather data for its Instant Answers. It runs alongside hundreds of other sources DuckDuckGo blends, and is far smaller in scale than Bingbot.
DuckDuckGo lists the user agent as DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html). The bot crawls from a published list of more than 200 IP addresses, available in JSON form at duckduckgo.com/duckduckbot.json, which you can use to verify that a request is genuinely from DuckDuckGo.
Not directly. DuckDuckGo sources most of its traditional link results from Bing's index, so ranking well on DuckDuckGo is mostly downstream of ranking well on Bing. DuckDuckBot is not a primary ranking input, so the practical lever is strong technical and content SEO for Bingbot.
Bingbot is Microsoft's large-scale crawler that builds the Bing index, which powers most of DuckDuckGo's link results. DuckDuckBot is a much smaller crawler that verifies URLs and supports Instant Answers. Bingbot does the heavy indexing; DuckDuckBot fills gaps and checks freshness.
Yes. DuckDuckGo states that DuckDuckBot respects WWW::RobotRules, the standard robots exclusion protocol. You can allow or disallow it by name using a User-agent: DuckDuckBot block in your robots.txt file, the same way you would manage any other compliant search crawler.
Usually no. DuckDuckBot is a legitimate, low-volume search crawler that helps keep DuckDuckGo's results and Instant Answers accurate for your pages. Blocking it offers little benefit and can remove your site from features DuckDuckGo builds itself. Most owners should simply let it crawl.
No. DuckDuckBot is DuckDuckGo's traditional search crawler. DuckAssistBot is a separate, newer crawler tied to DuckDuckGo's AI-assisted answer features. They are different user agents with different purposes, so you can allow or disallow each one independently in robots.txt.
Instant Answers are the direct results DuckDuckGo shows above or beside its links, such as definitions, weather, and sports scores. They draw on specialized sources like Sportradar and crowd-sourced sites like Wikipedia. DuckDuckBot helps assemble and verify the underlying data for these answers.
The bottom line
Bottom line
DuckDuckBot is a small, well-behaved crawler that verifies URLs, checks link health, and feeds DuckDuckGo's Instant Answers. It is not the engine that ranks the web. Because DuckDuckGo draws most of its links from Bing's index, your DuckDuckGo visibility is mostly downstream of your Bing performance. Let DuckDuckBot crawl, keep your links healthy, and put your real SEO effort into ranking well on Bing.