How Do Search Engines Find My Website?

How do we keep this site running? This post may contain affiliate links — the cost is the same to you, but we get a referral fee. Compensation does not affect rankings. Thanks!

Search engines scour the website for new content all the time. The three largest search engines are Google, Baidu (China’s most popular search engine), Yahoo, and Bing – listed in order of market share with Google owning the vast majority (over 2/3) of the global search engine market.

How Do Search Engines Find My Website Pages?

Search engines use robots (aka Spiders) to crawl the Internet for new and updated content. They follow hyperlinks to navigate around your website, as well as to jump to external sites that you link out to. When a search engine finds a page, it analyzes the page to find its identity and its prominence on the Internet. The prominence, or, as Google calls it, PageRank of a page, is decided by internal as well as external factors. Long story short, if a search engine likes your page, and you have not requested search engines not to index your site, it will index it in its massive search database (the database that is queried when people conduct searches via Getting your pages indexed is the first successful step in your search campaign.

How do I get my pages indexed by search engines?

Nowadays, search engines will index just about every page very shortly after it goes live on the Internet, as long as:

  1. The content on the page is unique
  2. The content on the page is readable by the search engine spiders. In the past, this meant text, but these days just about any form of content (text, flash, images, video, etc.) can be indexed.

You can speed up the indexing process by opening up Webmaster Accounts with the search engines. These will let you specify a site map (in XML format) that will communicate to the search engine which pages exist on your website, and how often they’re updated. There are several other benefits to having a Webmaster account, which include being able to see the rankings of your pages, how many impressions and clicks they get, and more. Here are the sign-up links to Webmaster accounts for the top three search engines. (Note: Yahoo has merged its webmaster tool service with Bing):

Next: OPtimizing Your Website Page for SEO

Next up in our SEO 101 series: how about we give you some actual practical steps you can implement to make your website pages more attractive to the search engines as well as your visitors? Browse on over to our website page optimization article to get started.

Alex bring a series of in-depth articles on search marketing and content management systems as well as troubleshooting tips to We Rock Your Web's collection. He is an avid tennis player, nature enthusiast, and hiker, and enjoys spending time with his wife, friends, and dogs, Bella and Lily.

Leave a Reply

1 Comment on "How Do Search Engines Find My Website?"


Use DMOZ for the giant Engine of Google to notice a listing in it’s directory. This is more an SEO task. The result is more on having your own Title with Google. The Organic and paid search optimization could come as a next factor to consider. This relates to the ranking of your web site. At this stage it’s very much contact-key word and title specific to have a better ranking.


Send this to a friend