Over the past couple of weeks the chatter about search functionality with regards to Twitter has come up again so I thought it might make sense to take a look back at how search has evolved with this exploding worldwide platform.

The Early Days

While most of Twitter’s users today weren’t around in the early days, many of you were. These were the days of the whale of fail, the rat in your pc and the general fun. Back then, there was no search function. If you wanted to know what your friend had for lunch or if your friend’s cat made a #2 on your friend’s carpet, you had to go to the account for your friend or his/her cat and just page back and forth. It was a tough time.

Summize

Then, like cream cheese on a fresh bagel, here comes NY-based Summize. Now we are talking! (well searching). Summize allowed us to search in near real-time for our friend’s messages. Want to know if that really was an earthquake? Pop “earthquake” into Summize. It was a real pleasure. Soon after the search engine was acquired by Twitter and renamed to Twitter Search and it still lives today. Twit messages are ranked by last posted in the results and there is no weighting to the results. The search functionality is also available within the Twitter site itself on the right side of a profile page.

Authority Search

Late last year during the “made up fights” between several valley bloggers and entrepreneurs, a discussion arose about creating an awesome Twitter search engine that would rank search results based on who has more followers. This was awesome…users who have lots of followers would appear higher in search. Bloody brilliant concept! Surely there would be no way to game the system to get more followers, right? Well, six months later and the rank by followers chatter has died out.

Note: somewhere in here Twitter introduced the Default List – this is the list that pumps certain celebs, FOT, fluffers, people who deliver cupcakes to twitter hq, etc.

Retweet Search

I thought I was excited to learn that they are remaking The Taking of Pelham 1-2-3. But now I know what real excitement is when I saw the launch of Topsy. Topsy is the engine that has got Twitter Default User Mike Arrington really excited. Here’s how Topsy works…rather than displaying results by authority or post date/time, Topsy ranks the results by how many times a link or query has been retweeted. If you need a moment, take one please, I know I did. So now let’s get serious for a moment. It’s interesting that in Mike’s gushingly positive post about Topsy, he failed to mention that those on the Default List will have an overwhelmingly higher chance of getting better rankings than us mere-non-cupcake-mortals.

Louis Gray did a Columbo-style investigation and found that, indeed, the default list members do control many of the Topsy topics. Louis looks at search queries including the iPhone, Tesla Motors and more. Duncan Riley took a look at Topsy from the “short url” standpoint.

Could a retweet search engine work? Perhaps but certainly not the way Twitter is setup today.

What’s Next?

Robert Scoble and others have said that Friendfeed does a better job at searching Twitter. Will Friendfeed become the search of choice for Twitter users? Maybe…if Friendfeed can win over the mainstream crowd.

Will Google begin to index twit messages? My magic 8 ball says “all signs point to yes”. Google wants to get some of the “realtime” loving that’s going around now and once they can index Twitter messages in some fashion, users will have even more reason to remain on Google. Google has already added a time panel on search results pages. Although I am not even sure that realtime matters for Google. I could see the search engine starting with just relevancy first – I do hope they separate Twitter or any of the other services from traditional online content.

My only hope is that all of the search engines will allow me to filter “-user: -cat -dog -sheep”.

Find more stories about: FriendFeed, Michael Arrington, social search, Summize, TechCrunch, Topsy, Twitter

Former Techcrunch editor Ouriel Ohayon is reporting via The Globes that social search Delver has been acquired by Sears. Yes, the powertools and "sponsored conversations" company Sears. The acquisition price for the Israeli-based startup was not disclosed. Ouriel notes that the price is probably not that high as the company was running out of cash.

Delver raised $4 million in a Series A round. About a month ago the company announced that they would either sell or close the company as they were unable to raise another round of funding the tune of $6-8 million.

The Globes notes, "Delver co-founder and CEO Liad Agmon will move to the US and become a VP at Sears. Delver’s employees will continue to work in Israel and the company will operate as a Sears’ development center."

Find more stories about: acquisition, acquisitions, Israel, Quick News, search, social search

The recent skittles twitter campaign used a feature in limited testing at twitter. It’s called integrated search, or real-time search. You could see it at work Sunday & Monday on the skittles.com homepage, or in the picture here taken from a deck by Fred Wilson and covered recently on Cnet. New search results are posted to the top of a search results page in real-time, effectively transforming search into conversation.

It has the effect of aggregating conversation within twitter, by threading posts around the search phrase or keyword. This strikes me as a potential game-changer for twitter, for several reasons.

We currently hold "conversations" on twitter with followers. We have to search to find non-followers around topics. But there are barriers to bringing them into the conversation. Results are past results, and we have to follow/be followed back before conversation becomes possible.

So conversations tend to happen between people who follow each other. If they are topical, they tend not to mention the topic. And this makes them less easy to find in search. Twitter addressed this recently. If there has been conversation between users (using @replies), it is now visible with the "show conversation" link.

But there are limitations to the usefulness of the "show conversation" implementation:

• to engage in that conversation would require that we follow and are followed back
• "conversations" are often off topic, or get off topic quickly
• the focus is on the people @replying to each other, not on keywords

"Show conversations" doesn’t really capture conversations, but captures an exchange between users who have @replied each other. Only the first tweet in the exchange has to contain the search keyword.

Twitter certainly realizes that it needs to searchable. But it also realizes that search results are limited to our use of search words and phrases. And limited by the fact that we have only 140 characters at our disposal. If twitter went after conversationality, it could do so only by aggregating the conversation around an exchange between users who follow one another — not around topics.

The following-follower model that has made twitter so incredibly viral has been a constraint on conversations. Each of us has only a small window through with to see what a small number of people are talking about. And only a limited means of capturing and sustaining conversation with people around a topic.

The theoretical description of this problem is this: tweets are only loosely coupled. They are loosely coupled between users, and loosely coupled by topic:

• Tweets are not coupled to each other unless they include an @reply or D message. The latter doesn’t count for public conversations. @replies only count if our account settings are to set generously (there are three settings).
• Tweets tend not to sustain topics because they must be so short, because we tend to initiate and then drop and change what we tweet about, and because the twitterverse serves the purpose of talking about and creating news. In news, we are more likely to pass something along than to engage in discussion.

Twitter was designed in such a way to prohibit conversations. Not intentionally, of course, but symptomatically. Conversations require a kind of coupling between statements and responses, and people in conversation, that twitter makes incredibly hard to achieve.

Find more stories about: Adrian Chan, chat, Quick News, search, social search, Twitter

NY-based Dorthy (pronounced Dorothy) has announced the hire of About.com co-founder Jim Anderson as CTO. The company noted that Anderson has experience with artificial intelligence, natural language processing, and machine learning technologies.

Earlier this month, Dorthy founder and COO Jordan English Gross took me through the site. He asked me not to write about it yet and, as always, I’ve honored his request. The company describes the service as, "a new destination site that delivers the best content, communications, and connections around whatever you dream. The site reverses the search by filtering and focusing all the stuff available around your interests providing targeted information based on your distinct ideals and philosophies." I would say it’s an interesting take on search. I will have a full review once permitted.

Gillian Reagan at the NY Observer posted a lengthy interview with Gross late in 2008 and has more information about the company including their $3 million venture capital funding. Reagan notes that Anderson was formerly on the Board of Directors of Dorthy.

Find more stories about: Dorthy, nextNY, NYC, Quick News, search, social search

While in San Francisco earlier this month, I met with Surf Canyon CEO Mark Cramer. Surf Canyon began operations in 2006 and launched their search application in February 2008. The Surf Canyon search application sits on top of search engines Google and Yahoo. Compared to some of the other search engine addons I’ve reviewed in the past, Surf Canyon offers an interesting and innovative model. Surf Canyon is trying to solve the problem of too many search results for your search queries.

Cramer explained how the application works — basically you enter a search query on Google or Yahoo as you always do. You select a search result and go to that page. If that page isn’t what you were looking for, when you come back to the search result, Surf Canyon looks at results 11-1000 and then recommends other results similar to the one you clicked on. It ignores results you skipped over because it recognizes you weren’t interested in those results. Cramer noted that the idea is to make search results dynamic instead of the static results that Google and Yahoo provide by default.

Surf Canyon has raised $600k in angel funding, has 3 full time employees and several consultants.

The Surf Canyon application is installed via a Firefox/Internet Explorer addon. Cramer tells me that there’s been 250,000 downloads of the addon to-date. I asked Cramer about how he is marketing the search application because clearly mainstream Internet users just go to Google or Yahoo for search. He said that a large percentage of their marketing has come via word of mouth and that once users use the application they find it beneficial and share it with others. They have also been listed as a featured download in the Firefox addon catalog. The key to success for Surf Canyon is to continue to raise awareness of their application across the mainstream market.

Find more stories about: Interviews, Quick News, San Francisco, search, social search, Surf Canyon

At the Web 2.0 Expo, I met with Sightix VP Ari Gottesmann. Ari took me through a demo of their social search application which integrates into blogs and social networks. Currently Sightix finds connections within a network but they are working towards exposing connections across multiple networks.

Check out other Sightix reviews on Alt Search Engines and Webware. Here’s Ari explaining how Sightix works when trying to find models like Bar Rafaeli:

Find more stories about: Conferences, Interviews, Quick News, Sightix, social search, Video, Web 2.0, Web 2.0 Expo

Wikia Search has launched a new Firefox toolbar which they call Wikia Evolution. Wikia Evolution is available for download and offers an easy to way to add pages to the Wikia Search project.

Here’s how the Wikia Evolution toolbar works once installed. When you search on Google, Yahoo, etc. or view any Web page, a new set of options will be displayed (see the sample below). The options allow you to add the page to the Wikia Search index instantly and tag the page for the appropriate keywords. There’s also a rating option available for each page that is submitted into the search database.

Wikia founder Jimmy Wales said on the Evolution toolbar launch, “This toolbar, like everything we are doing at Wikia Search, is open source. We hope that if you are a toolbar fan and programmer, you will let us know what features need to be added and/or take this and do something surprising and cool with it.” 

Find more stories about: Quick News, search, social search, wiki, Wikia

Earlier this week a new search engine launched to the world. The name of the engine is Cuil (pronounced Cool) and there are great reviews on Search Engine Land and GigaOm. Danny from Search Engine Land notes that Cuil is pushing four main features of the search engine: Big web index, Unique relevance algorithm, Unique results display and Privacy.

I’ve created a short video below to discuss why Cuil is like toilet paper. The bottom line is that when you come out of the gate and immediately challenge the absolute leader, it’s a mistake. In addition, Google has absolute loyalty and to get average consumers to switch when they are loyal to a product or brand is very, very difficult. Making it even more difficult is that Google is free, so a lower-cost product pitch won’t work either. What Cuil should have done was only speak about their features and let the users and the search industry decide if it’s better than Google. So far it seems Cuil is the un-cool.

Find more stories about: Cuil, Insights, search, social search, Video

Expedia is announcing this morning that they have partnered with Baynote to include Baynote’ social search in the Expedia booking engine. Baynote’s social search takes search queries and then can present options to future searchers for refining their query to help find exactly what they are seeking. Financial terms of the deal were not disclosed.

While the information provided talks about increasing the customer experience on Expedia, this type of search might lead to increased transaction rates. Rather than a customer potentially leaving Expedia when he or she doesn’t find what they are seeking, now Baynote can help get the customer to what they are seeking and then book the travel item.

Baynote also powers other corporate customer’s social search including NASA and Motorola.

Earlier this month Expedia lost their deal with MSN. Also check out our Google Trends showdown between Expedia and TripAdvisor.

Find more stories about: Expedia, Quick News, search, social search, Travel

Found via Mr. Lowensohn, Scour is a new social search engine. The concept behind Scour is to combine results from Google, Yahoo and MSN and present the best results first. It’s a pretty interesting model and reminds me a bit of TWERQ which shows results from the major engines in tabs inside the application.

At first I was thinking this idea was awesome — combining the search engine algorithms together to create the Voltron of search results (where’s the princess?). Not to pop the bubble, but does the average person care about the ultimate right result or just the good enough results from the engine they use currently?

The second piece of Scour is commenting and rating on results. This is similar to the new engine from Wikia. You can leave comments inline on sites listed in the search results and also vote results up or down. I hope they will have a comment feed and a subscribe option so I can subscribe to the sites I am interested in. Scour says the votes will eventually make their way into the search engine. This part differs from Wikia which allows you to move whatever you want wherever you want.

The last piece of Scour is cash payment for searching, commenting and rating. You can earn points and then those points can turn into Visa gift cars. I have to be honest here – I wish more sites would drop the point reward systems. It’s a great idea but it just leads to more issues than it’s worth. If Scour was going after the mainstream market, it might make more sense. But as I’ve noted above Scour is a tech-focused engine and I doubt the incentive will get techies to switch. I know on the surface the idea seems to be a great idea to create loyal users.

Overall I think the concept has a lot of potential and is good for moving the social search engine conversation forward. Scour needs some refinement but could make an impression in the techie search engine arena.

Here’s the Scour product demo:

Find more stories about: Reviews, Scour, search, social search, startups

Sematic search engine Quintura has announced a new distribution partnership in which they will power the search for Maxim Digital’s Web properties which include Maxim.com and Blender.com.  Yakov Sadchikov, President & CEO of Quintura said, “Our strategic partnership with the online leader in men’s lifestyle, Maxim Digital, demonstrates Quintura’s ability to provide interactive site search solutions to consumer web publishers with millions of monthly users."

The search functionality is now live on Maxim’s site. Google Trends shows about 30k users a day to Maxim and about 15k to Blender.

Find more stories about: Quick News, Quintura, search, Semantic Web, social search

Mahalo CEO Jason Calacanis had a busy week last week. We learned of three major Mahalo updates: they are now a research engine, if you want to get paid you must write more and link less, SEO is critical for the success of the research engine.

Mahalo is no longer a search engine as it was classified upon the initial launch nearly a year ago. Calacanis is now calling his company a "research engine". While I don’t think it’s a search engine, it’s not a research engine either. The closest thing I could compare it to is a combo of delicious plus a couple paragraphs of juicy content.

When Mahalo launched, I asked Calacanis whether the site was created for the purpose of driving traffic through search engines. He noted, "Are we an SEO master? If Google wants to index us well that’s fine with us, but our model is not based on being well indexed in Google or Yahoo. Our model is in creating human curated search results that are very helpful to users."

In an email to his greenhouse workers last week, he changed his tune. "No guide notes, no google/yahoo rankings, and no traffic. No traffic, no money. No money, we all go home." With nearly 75% of Mahalo traffic coming from Google, this guide note change is important. It would be interesting to learn what percentage of traffic must leave via advert or affiliate link for Mahalo to be profitable.

Last week, Calacanis changed the policy for his large workforce by mandating a minimum of 300-400 word guide notes. These are typically the only content on a (non how-to) page. My guess is that this lengthening of the guide notes is a direct response to Aaron Wall’s post that Mahalo is Spam.

In case you are interested in where the search engine traffic is coming from, I ran a paid report on Compete for the 50 most popular terms sending traffic from Google to Mahalo:

mahalo, how to play guitar, big brother 9 spoilers, halo 3 skulls, 2 girls 1 cup, best computer speakers, halo 3 armor, ashley dupree, gene simmons sex tape, bridget mccain, how to write a resume, guitar hero 3 cream, christopher allport, how to make mash potatoes, abby, mcgrew, tax rebate 2008, how to make sushi, vicki iseman, guitar hero cheats, lindsay lohan new york magazine, guitar hero 3 cheats, guitar hero 3 song list, super smash bros brawl secret characters, brawl secret stages, guitar hero iii cheats, pokemon crater, amy fisher video, sarah larson, big brother 9 spoiler, is new york pregnant, cloverfield monster pictures, how to speak french, flat belly diet, wikipedia ashlee simpson, guitar hero 3 wii cheats, parkville maryland bethany, bush tax rebate, wwe smackdown v.s. raw 2008 online guide, lil romeo 50 million, daisy de la hoya, flavorofloveworld, youtube heavy metal don felder, dorothy hamill skating timeline, net gun, dmc4 secret achievements, lost odyssey game guide, nfl tour review, blazing angels 2 cheats, silver skulls in halo 3, halo 3 armor unlock

I believe we will continue to see a shift with Mahalo which will push unpaid people to create, edit and maintain the content on Mahalo. Guide notes have already been opened for editing if you have more than one published item and links are already provided by the community. Though as Dunan Riley noted yesterday about another site similar to Mahalo which also works perfectly here, "With no revenue sharing model there’s no obvious reason why someone would contribute to the Wiki (after all there’s no for the good of humanity angle like Wikipedia)".

Later this week we will look at why Calacanis is pretty smart.

Find more stories about: human search, Insights, Mahalo, search, social search

I wanted to love the Wikia Search from the minute I joined the mailing list months ago. I was very excited to see the photos of the servers being loaded into the racks and was looking forward to using the alpha. I was one of the first to get access and so far, sadly, I have to say that it doesn’t "wow" me. When I spoke with the PR person, she said that they let Wikipedia out early and look how well it worked. I noted that Wikipedia basically had no competitors and that helped it get a 1st mover advantage. I am sure that any review you read of Wikia Search today, you will see comparisons to Google, Yahoo, Mahalo and Hakia.

I spoke with Jimmy Wales for a few minutes on Friday. He said they are looking for (eventually) a 5% market share and will be showing ads later on. He said Wikia Search is a "general Web search" and competes with Google, Yahoo, and Ask. The biggest positive I see so far is that the Wikia Search team is very open to feedback and appears to be building this search engine with the community in mind.

Update: Check out a couple of other reviews as well from Mike Arrington and Paris Lemon. Of course read mine below first :)

Search Engine

The search engine works just like any other search engine. They use "Nutch" for the algorithm and display the numerical result for each result next to the text. I sure hope that this scoring is only shown for the Alpha/Beta as no mainstream user cares about what the score for a result is, they just want the best result. The indexed sites list is small and is growing so finding accurate results is a bit tricky. A search for Web 2.0 didn’t return anything that seemed correct, though searching for TechCrunch did provide a more accurate set of results. Rex suggested that I do a vanity search — the results were the NYU Stern School of Business along with about 8 other .de German "Stern" web sites. Do a search for sex and you can see why they need to index more results asap.

There is one function on the search that is new and pretty groovy. Instead of reloading the page to see the next batch of results, the page just expands on the click to show the next batch. This is very cool and has high usability but it might be limiting for cpm-based advertising.

Social Aspects

One of the major pieces of the Wikia Search are the social aspects. When you setup your profile, the keywords you attach allow you to show up in searches for those keywords. I like this because it can help you to find others that are interested in the same topics you are and build new relationships.

You can also create a profile page which is similar to a profile page on any social network. Even though I am not a fan of all of these profile pages, I have to give the nod here to Mahalo’s social page only because Mahalo lets you associate other network accounts. I can show my delicious, digg, facebook, etc. You can also friend people just like with any other social network.

Mini Articles

"Mini Articles" are basically stubs with links to full wiki pages. When you search for any term, above the results is a Mini Article which anyone can edit. The articles link goes to a full wiki page about the subject which again anyone can edit. I believe these Mini Articles will have tons of abuse. Jimmy said that there will be community admins who will be selected by Jimmy to keep the spam and blackhat activities out. He said payment will be unlikely which leads me to believe that it will be easy to game the system, even though Jimmy said on our call that "gaming is impossible".

Conclusion

Overall I would say that Wikia Search has a long way to go in terms of interface, usability and features to get me to want to change (or add) it to the search engines. I understand that it’s in Alpha and things will improve and change as the plan moves forward, however I would have liked it to be a bit more polished before hitting the public eye.

Find more stories about: Reviews, search, social search, Wikia

Today we learned that Mahalo has added a social profile component to their system. Search Engine Land has a very lengthy overview of the features, but let me try to explain it in twitter-like style. You can add a profile with your social networks, and see your submitted links. And you can invite friends to Mahalo. That’s it.

Jason explains the new features as, "layering the social graph on top of search". Mike Arrington noted, "Today, they are adding user profiles and other social networking features to further incentivize users to submit quality content." I asked in the comments how this "incentivizes" anyone to leave more links – wouldn’t a forced login process cause less content to be submitted? The link I submitted to Mahalo is displayed on the page, about 4-5 full scrolls down. But it hasn’t been approved by Mahalo. What incentive does that give me to submit more links? I’d suggest moving those to the top to, at a minimum, say thanks for the submission. And would the average Internet user have a profile on Pownce or Twitter? I am not sold on what the benefit is to invite my friends to another profile generator. Can anyone help me understand some benefits to forwarding the profile creator to my friends?

Mike also notes that Mahalo traffic is up big which is great news for such a new startup. I’d just put an * there that shows the Alexa chart and who’s actually using the service. 60% of traffic appears to be employee traffic. Not taking anything away from the great growth curve displayed by comScore. (and yes, I know Alexa is the suck, but it shows a breakdown of where the traffic is coming from). Interesting note, "vanessa hudgens nude" is the most popular search term driver to Mahalo from Google reports Compete. I’d wonder what percentage of traffic comes from blogger mentions and Google vs. home-grown organic traffic.

I’ve written before about why the Community CEO doesn’t work and posted Jason’s response of why it does work. Today, it’s important to note that it’s not only community ceo, it’s also now unpaid community site growth with the belief that people will submit links to better the search engine to then help me back on future searches. I am not sure I’d take that bet but with their goal of hitting the mainstream, it may work.

Jason is a very smart marketer. I’ve said this several times before. We know this by the way he releases items in timing/depth on Mahalo and to whom he hangs with to drive traffic. I certainly can’t fault him for this.

In the video presentation below from LeWeb, Jason attacks Seth Godin for not policing Squidoo and Dave Sifty for the spam on Technorati. He then calls Ted Murphy an "idiot" and notes he is "not being so bright" with his PayPerPost product. Jason also notes in his discussion that we should use RSS subscribers as a metric for who’s an authority. I sure hope he is kidding on this last claim.

Here’s the video from his speech at LeWeb:

Find more stories about: Calacanis, Mahalo, Reviews, social search

So it's a rainy night here in NYC and Carvel is out of my favorite flavor. As I walked back to my apartment, I thought to myself, let's see what the domain history is for Mahalo. Surely, Mr. Calacanis couldn't have been the first registrant in 1995, right? Alright, so let's pop over to the Internet Archive Way Back Machine, spin the wheel and see what comes up!

Actually, there isn't much which was disappointing. Only nine links over four years. So I started clicking, and six of the pages are your standard "your webserver is ready" crap. So I click "December 3, 1998" and what do I find? A PORN SITE. That's right folks, the new "Super great human awesome going to beat Google search engine" used to be a porn site. :)

Here is a screengrab (images didn't load):

Anyway, just a bit of fun on a hot, hazy and humid night. And I guess this post is better than eating a 2,000 calorie Flying Saucer.

Find more stories about: human search, Mahalo, social search