Official Google Webmaster Central Blog: June 2010

Sitemaps: One file, many content types

Tuesday, June 29, 2010

Sitemapspecialized Sitemap formats<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:image="http://www.google.com/schemas/sitemap-image/1.1" xmlns:video="http://www.google.com/schemas/sitemap-video/1.1"> <url> <loc>http://example.com/foo.html</loc> <image:image> <image:loc>http://example.com/image.jpg</image:loc> </image:image> <video:video> <video:content_loc>http://example.com/videoABC.flv</video:content_loc> <video:title>Grilling tofu for summer</video:title> </video:video> </url> </urlset>Webmaster Tools

Sitemap rulesWebmaster Help ForumWritten by Jonathan Simon, Webmaster Trends Analyst

Quality links to your site

Monday, June 21, 2010

Webmaster Help Forumour viewpointGoogle BuzzBuying PageRank-passing linksexchanging links

Social bookmarks on YouTube enable users to share content easilyWebmaster Help ForumWritten by Kaspar Szymanski, Search Quality Strategist, Dublin

Google Videos best practices

Friday, June 11, 2010

best practices

Best Practice 1: Verify your video URLs are crawlable: check your robots.txt

Sometimes publishers unknowingly include video URLs in their Sitemap that are robots.txt disallowed. Please make sure your robots.txt file isn't blocking any of the URLs specified in your Sitemap. This includes URLs for the:
- Playpage
- Content and player
- Thumbnail
More information about robots.txt.

Best Practice 2: Tell us what countries the video may be played in

Is your video only available in some locales? The optional attribute “restriction” has recently been added (documentation at http://www.google.com/support/webmasters/bin/answer.py?answer=80472), which you can use to tell us whether the video can only be played in certain territories. Using this tag, you have the option of either including a list of all countries where it can be played, or just telling us the countries where it can't be played. If your videos can be played everywhere, then you don't need to include this.

Best Practice 3: Indicate clearly when videos are removed -- protect the user experience

Sometimes publishers take videos down but don't signal to search engines that they've done so. This can result in the search engine's index not accurately reflecting content of the web. Then when users click on a search result, they're taken to a page either indicating that the video doesn't exist, or to a different video. Users find this experience dissatisfying. Although we have mechanisms to detect when search results are no longer available, we strongly encourage following community standards.

To signal that a video has been removed,
1. Return a 404 (Not found) HTTP response code, you can still return a helpful page to be displayed to your users. Check out these guidelines for creating useful 404 pages.
2. Indicate expiration dates for each video listed in a Video Sitemap (use the <video:expiration_date> element) or mRSS feed (<dcterms:valid> tag) submitted to Google.

Help CenterHelp ForumPosted by Nelson Lee, Product Manager, Video Search

Our new search index: Caffeine

Tuesday, June 08, 2010

(Cross-posted on the Official Google Blog)
Today, we're announcing the completion of a new web indexing system called Caffeine. Caffeine provides 50 percent fresher results for web searches than our last index, and it's the largest collection of web content we've offered. Whether it's a news story, a blog or a forum post, you can now find links to relevant content much sooner after it is published than was possible ever before.
Some background for those of you who don't build search engines for a living like us: when you search Google, you're not searching the live web. Instead you're searching Google's index of the web which, like the list in the back of a book, helps you pinpoint exactly the information you need. (Here's a good explanation of how it all works.)
So why did we build a new search indexing system? Content on the web is blossoming. It's growing not just in size and numbers but with the advent of video, images, news and real-time updates, the average webpage is richer and more complex. In addition, people's expectations for search are higher than they used to be. Searchers want to find the latest relevant content and publishers expect to be found the instant they publish.
To keep up with the evolution of the web and to meet rising user expectations, we've built Caffeine. The image below illustrates how our old indexing system worked compared to Caffeine:

Our old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, we would analyze the entire web, which meant there was a significant delay between when we found a page and made it available to you.

With Caffeine, we analyze the web in small portions and update our search index on a continuous basis, globally. As we find new pages, or new information on existing pages, we can add these straight to the index. That means you can find fresher information than ever before — no matter when or where it was published.

Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.

We've built Caffeine with the future in mind. Not only is it fresher, it's a robust foundation that makes it possible for us to build an even faster and comprehensive search engine that scales with the growth of information online, and delivers even more relevant search results to you. So stay tuned, and look for more improvements in the months to come.

Posted by Carrie Grimes, Software Engineer

Crawl Errors now reports soft 404s

Monday, June 07, 2010

HTTP response codeFarewell to Soft 404sWebmaster Tools

Check whether you have soft 404s listed in Webmaster Tools

For the soft 404s, determine whether the URL:

Contains the correct content and properly returns a 200 response (not actually a soft 404)
Should 301 redirect to a more accurate URL
Doesn’t exist and should return a 404 or 410 response

Confirm that you’ve configured the proper HTTP Response by using Fetch as Googlebot in Webmaster Tools

If you now return 404s, you may want to customize your 404 page to aid your users. Our custom 404 widget can help.

Webmaster Help ForumWritten by Jonathan Simon, Webmaster Trends Analyst

Grab bag videos are back!

Tuesday, June 01, 2010

We’re kicking off June with the start of a new round of webmaster Q&A on the Webmaster Central YouTube channel. You submitted and voted on questions for Matt Cutts to answer, and Matt sat in the studio for a full day sharing advice for webmasters.
For those of you who watch each video (and who doesn’t?), we’ve worked hard to keep things interesting. Not only did Matt wear different colored shirts, we changed the backgrounds as well! Just don’t submit any screen grabs to We Have Lasers, okay?
To get you started, here’s the first video, which addresses a question about geographic targeting in Webmaster Tools:

We’ll be posting links to new videos as they’re posted on our Twitter account, so follow us there or subscribe to our YouTube channel to be notified of new answers.
Posted by Michael Wyszomierski, Search Quality Team

Webmaster Central Blog

Sitemaps: One file, many content types

Quality links to your site

Google Videos best practices

Our new search index: Caffeine

Crawl Errors now reports soft 404s

Grab bag videos are back!

Labels

Archive

Feed

Subscribe via email