What this means is that cutting-edge designs are generally not the best way to go. Interestingly enough, your site visitors may agree. Even though we SEO geeks have cable modems and DSL, our site visitors probably don't. Slow-loading Flash sites, for example, may stop the search engine spiders right in their tracks. There's nothing of interest on the average Flash site to a search engine spider anyway, so they're certainly not going to wait for it to download!

Another source of grief towards getting your pages thoroughly crawled is the use of the exact same Title tags on every page of your site. This sometimes happens because of Webmaster laziness, but often it's done because a default Title tag is automatically pulled up through a content management system (CMS). If you have this problem it's well worth taking the time to fix it.

What about the use of JavaScript? We've often heard that JavaScript is unfriendly to the crawlers. This is partly true, and partly false. Nearly every site I look at these days uses some sort of JavaScript within the code. It's certainly not bad in and of itself. As a rule of thumb, if you're using JavaScript for mouseover effects and that sort of thing, just check to make sure that the HTML code for the links also uses the traditional "a href" tag. As long as that's there, you'll most likely be fine. For extra insurance, you can place any JavaScript links into the "noscript" tag, put text links at the bottom of your pages, and create a visible link to a sitemap page which contains links to all your other important pages. It's definitely not overkill to do *all* of those things!

There are plenty more things you can worry about where your site's crawlability is concerned, but those are the main ones I've been seeing lately. One day, I'm sure that any type of page under the sun will be crawler-friendly, but for now, we've still gotta give our little arachnid friends some help.