Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
The surface web is everything a search engine has indexed. The deep web is everything it has not, which is most of the internet and mostly mundane. Neither term means the dark web.

What Is the Difference Between the Deep Web and Surface Web?
The difference is one of indexing, not of legality or secrecy. The surface web is the set of pages a search engine crawler can reach and has chosen to index. The deep web is everything else served over HTTP: your webmail inbox, an internal wiki, a banking dashboard, an API response, a database-driven report behind a search form. Most of the deep web is dull, lawful, and belongs to somebody who simply does not want it indexed.
A crawler needs a path to a page and permission to store it. Several ordinary conditions remove one or both.
The widely repeated figure that the deep web is 96 or 99 per cent of the internet traces back to a single study by BrightPlanet in 2001, which estimated the size of database-backed content relative to the then-tiny indexed web. The ratio is unmeasurable in any rigorous sense—how do you count the pages a dynamic database could generate?—and the picture it creates is misleading, because it implies the hidden portion is sinister. The overwhelming majority of unindexed content is private email, corporate applications, and machine-generated API output.
The dark web is a small subset of the deep web that requires specific software to reach—most commonly Tor, which routes traffic through volunteer relays and resolves .onion addresses not served by public DNS. Being unindexed is incidental; the defining feature is the anonymity layer. Tor hosts criminal markets, but also mirrors run by the BBC, ProPublica, and secure drop services used by journalists and whistleblowers. Conflating the three layers is the most common mistake in security awareness material, and it causes real confusion when an executive is told their data is “on the deep web”, which could mean anything.
For reconnaissance, the surface web is where open source intelligence starts and stops being easy. An attacker preparing a campaign reads indexed job adverts to learn your technology stack, pulls employee names from LinkedIn, and searches for exposed documents using advanced search operators—the technique commonly called Google dorking. What they find there costs them nothing and is entirely passive, which is why an attack surface review should begin with what a search engine already knows about you.
For exposure management, the risk runs the other way. Deep web content becomes surface web content the moment indexing protection fails: a misconfigured storage bucket, an internal application published without authentication, a staging site that was never excluded. Shodan and Censys index services rather than pages, so an unauthenticated database listening on a public address is discoverable even though no search engine ever crawled a page from it. Relying on robots.txt to keep something private is worse than useless, because the file itself advertises the paths you consider sensitive.
Run the searches yourself, regularly. Enumerate your own domains and subdomains, check what is indexed, and look for document metadata, exposed directory listings, and development environments. Use authentication rather than obscurity for anything that matters, and set noindex headers only as a tidying measure on top of access control. Where you need to monitor the layers below, use a service that covers criminal forums and markets properly—see digital risk protection—rather than asking staff to browse Tor themselves, which creates legal and malware exposure for no analytic gain.
Our penetration test services begin with exactly this external reconnaissance, so you see your organisation the way an attacker first does.
No, and you use it constantly. Checking your email, logging into online banking, or opening a document in a corporate SaaS application are all deep web activity, because none of those pages is indexed by a search engine. Illegality attaches to specific content and conduct, not to whether a page appears in search results.
No. The dark web is a small part of the deep web that requires anonymity software such as Tor and uses addresses not resolvable through public DNS. The deep web is simply everything unindexed, the vast majority of which is ordinary private content on normal websites reached with a normal browser.
Start with a site-restricted search across each domain you own, including forgotten subdomains, and review what comes back for staging environments, exposed documents, and directory listings. Add a check of Shodan and Censys for services published on your IP ranges, and set up alerts for your brand in certificate transparency logs to catch new hosts as they appear.
Not meaningfully. Robots.txt and noindex tags are honoured only by cooperative crawlers, and the robots file is publicly readable, so it effectively publishes a list of the paths you want hidden. Anything that must not be seen needs authentication and authorisation controls; indexing directives are housekeeping, not security.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.