Search: How to get your site indexed faster
18 July 2006 in Google, Intergen, Search Technology | Comments enabled

Search Series

Over the next few weeks I’ll be posting about common challenges that people face with website search (both with public search engines like Google, and website search such as SharePoint search). If you have any questions about a topic or would like to see an entry about a specific challenge you have then please leave a comment :)

The challenge
It’s the usual story, somebody has spent a lot of money on a website and they want to start seeing a return on that investment as soon as possible. There is nothing wrong with that, I’d be concerned if I was building a website for somebody who doesn’t want to see a return. The challenges arise when a site goes live and it is expected that the site is indexed by the major search engines in a matter of minutes of that go-live. So how do we get a website indexed by search engines as quickly as possible?

A solution?
Back in the 1990′s it was common practice to need to submit your website to search engine for it to index you. Some started following links and automatically found sites however this was usually seen as not a priority compared to manually submitted sites. The view that this practice is still the best is strong with some and they see site submission as the best way of getting their site indexed fast.

These days it’s not uncommon to see sites offering to submit your site to more than 1000 search engines for a small fee of $500 or more. Some people would see that as value for money without realising it’s almost a complete waste of time.

The real solution
Modern search engines don’t prioritise on submitted sites. They don’t see them as valuable and for good reason – would you trust a website that was brand new or one that has been around for a significant period of time with many links to it? I’ve read many accounts (and see this in practice) where a site can be manually submitted to a search engine and not appear in search results for several weeks. Yet, when another website that is mildly popular links to that same site they’re in search results within days.

One way to get links to your site early is to allow a case study of the site development to be posted on the vendors website. I’ll go into more ways to help build your link collection legitimately in future search posts.

When considering the value of paying somebody to submit your website to hundreds of search engines, remember that 95% of your search result referrals will be coming from the big 3: Google, Yahoo, MSN Search. I would never advise on the use of such submission tools because, frankly, they’re a waste of time in terms of return on investment.

Always an exception to the rule
It is important to note that I’m not advocating never submitting yourself to an index manually. There are specific industry sites, directories and community sites that are related to your business and submitting to these sites is important for visibility within your industry sector. Automated tools won’t even know that these sites exist or that they’re more important to you than a mass spamming to various indexes around the world.

– JD


6 comments. Add your own comment.

Geek says 19 July 2006 @ 13:44

Yep I’ve found the best way to get indexed is to write good content and participate in the online community you are trying to communicate that content to. In the end it’s all about the quality of you content and the quality of the links linking to your content.

shane says 19 July 2006 @ 23:28

I have found searchnz to be very responsive when indexing a site for the first time. And the quickest way to get indexed by the big 3 is to pay for a link on an already popular or well visited site – the bots will follow the link.
(Sorry if that steals any of your thunder :)

traskjd says 20 July 2006 @ 23:42

Hi Shane,

That doesn’t steal my thunder – more reinforces my point :) It’s unlikely that bulk submission tools will even know New Zealand search sites exist (I would class NZ search as more “local” and what I talk about in “always an exception to the rule”).

Sadly you’re right that paying for a link can work. The downside to this is that it would signify to me that if you’re already paying for links then perhaps people don’t want to link in the first place. Along with this companies like Google crack down on this behavior by penalising sites for listing on websites that are known to take links for money (they see it as trying to manipulate their indexes). There is usually better ways to spend your money.

– JD

traskjd says 20 July 2006 @ 23:43

Hi Geek,

You’re exactly right. No matter how great you technology is or how much you spent, the content is what matters. On the net, Content is King.

– JD

Chan says 21 July 2006 @ 00:22

Excellent post JD. Some very valid points. Looking forward to the next article.

Digitalmelon : Chan's Blog says 21 July 2006 @ 00:28

Blog Post on Search from JD…

Leave a Comment

Name (required)

E-mail (required - not published)

Website

Your comment: