Real-time Search – A Definitive Rundown

Posted by | Sajit N | August 14, 2009 | No Comments

Real-time search is a topic beaten to death by numerous SEOs and internet thought leaders. And yet, all the hair-splitting and blasé discussions on this issue have left so many loose ends that a proverbial cloud seems to have descended on the concept as a whole. What exactly should be termed as “real-time search”? How is it different from a “non-real-time search”? Why should website owners/bloggers take notice? This post makes an attempt to address these issues.

The Real McCoy

Real-time search is that which seeks to elicit a slice of the global ‘thoughtstream’ (more on that, later) over a particular topic. In other words, the searcher wishes to follow a developing story as well as read related content/opinions that are bandied around in the webosphere. Therefore, a real-time search engine should deliver a set of results tailored to meet the demands of “content freshness” and “crowdsourced popularity” in the form of latest updates, topmost links and most popular comments for the searched queries.

In a way, the results thrown up by a real-time search engine are similar to what a news engine delivers. However, unlike a news engine, the real-time search engine will act largely like a Mechanical Turk – it will employ a human-controlled system to keep the visitor in loop with the on-goings, rather than an algorithmic solution. For this, a real-time search engine gleans social media data (primarily from Twitter APIs), which will be processed & sorted into result types and then displayed accordingly.

Riding the wave

Sensing an opportunity, all mainstream search engines have already started incorporating real-time results for queries keyed in. A query for any contemporary topic will produce news results which partly realize real-time search’s requirement. However, what mainstream search lacks is social media driven “thoughtstream” data. Thoughtstream can be defined as the sum total of opinions, information, ideas, rants, comments, tidbits, debates or any form of thought that gets projected on to the social media landscape.

Take a look at the top results for keyword “recession” in Google. The first three results are purely informational in nature, without any element of real-time contextuality. The news results do some justice to temporal relevance. However, what’s lacking is masses driven grassroot-level feedback for “recession”, as opposed to a reporter’s account derived from research, not often emanating from first-hand experience.

On the other hand, Twitter search returns a panoply of result sets in the form of comments, news items, links and the occasional spam. While it’s hard to make real sense out the results returned, this bunch of incoherent and raw results is exactly what defines the “thoughtstream” for “recession” today. If one needs to get a primer on “recession”, then search engines are the best place to look for such information. However, if one wishes to get a peek into the street buzz about “recession”, then real-time search engines are better equipped to deliver the goods.

And this “thoughtstream” data is what is coveted by all mainstream search engines. While, the crawlable World Wide Web can be the indexed and processed to provide searchers with a semblance of real-time search results, nothing can beat the “tweetfall” of data that Twitter generates, in capturing the present state of meme metamorphosis or news story development. News items about Google’s attempts to acquire Twitter, Yahoo’s interest in Twitter-powered real-time search & Bing embracing Twitter results show the kind of influence Twitter wields presently, not just on search realm but on the whole of Internet, all on account of its “thoughtstream” database.

Show me the money

Real-time search engine optimization has to follow where rankings determine visitor footfall for a website. For enterprises, merely having and updating a Twitter channel would be more fruitful than ranking high on real-time searches. However, blogs and news websites have a serious opportunity in boosting their site traffic via real-time search optimization. For one, consider the search volumes for “Michael Jackson” related terms in the hours following his death: 30% of tweets on Twitter related to “Michael Jackson”, Yahoo News recording its highest single-day visits(at 16.4 million unique visitors) and a search spike big enough for Google News to mistake it for a ghoulish cyber-attack. If your website or blog wishes to capitalize on its freshness quotient, then real-time search is the place to score big in.

First of all, one needs to know what goes into ranking of real-time search results.  There are myriad real-time search services, most of which use Twitter’s data to rank results. These search engines display results based on either freshness or popularity. Twitter Search itself, along with Almost.at ranks results solely on the basis of freshness of results. Others like Collecta, Tweetmeme, OneRiot etc. show results that are popular. Popular results are those which are retweeted more frequently, shared more frequently or those coming from twitterers with authority(as measured by ‘follower’ to ‘followed” ratio, follower authority, PR of twitter profile etc.)

Here are the basic rules of optimizing a page for real-time search engine ranking:
•    Find suitable keywords for your tweets & insert these keywords in the beginning of your tweets to let these be the initial part of tweet page’s title tag
•    Insert keywords in hash tags for individual tweets
•    Put retweet references in the end as these should not form part of your title tag,
•    Make your tweets worthy of being retweeted(make them funny or informative, insert hyperlinks etc.).
•    Ensure that your profile moves up the authority ladder, by keeping a healthy follower-to-followed ratio and following those who are authoritative