What this means is when you complete a search on Hakia, you can select to see only sites that Hakia deems "credible". The credible sites list will be vetted by librarians and informational professionals.

Hakia has also launched "hakia Galleries" which bring together all types of search results (image, text, credible, etc.) on a topic basis. It’s an interesting way to provide the data rather than for an absolute specific query.

I like the Hakia team and love that they are in NYC. But they have to start marketing the search engine. Get up to Times Square and setup a corner where you can show tourists why Hakia is better than Google. Let those tourists go back to their countries and spread the word. Sure I am simplifying things, but they have to raise the level of awareness to drive search volume. They don’t have a CEO who can push links to tens of thousands of Twitter users for quick traffic boosting. Booths at tradeshows won’t cut it alone.

The business model is welcoming; offer 30,000 searches per day free of charge and free of advertising then they will discuss a relationship with you past the 30,000. What this means is that small social networks may never pay anything for using the Hakia service but provide a great benefit for their users.

The Syndication Web Services include Web search, News search, Vertical search, Summarizer, Categorizer, Characterizer and Text Meaning Representation. The services provide an XML feed, and options to customize the feed. The first company using the new service is Berggi. Berggi has created a worldwide mobile search application.

This month Hakia added PubMed to their search index and also checkout our comparison of Hakia and Powerset.

Hakia announced

The Hakia blog post on the PubMed announcement is well worth a read if you are into semantic search. Apparently the PubMed search has holes that may return incorrect or no results. Hakia uses their intelligence to return relevant results.

Check out our comparison of Valley darling Powerset vs. NY-based Hakia.

Today’s addition brings social networking into play with, "Meet Others." The Meet Others option allows searchers to find other hakia users who have searched for the same terms. From there, each person can contribute to the conversation around the search terms.

Here is how Hakia positions the new Meet Others service, " Meet Others is initiated by entering a query at hakia.com search engine, then clicking on the Meet Others button which opens a room of postings by others who asked the same or similar query. The user can also go directly to meet.hakia.com to search for query rooms. Contact links are available for each posting, but user information is hidden from view. Registration is not required to view rooms and contact people, but posting requires user authentication via email confirmation. The entire operation is optional to participate."

From skeptic to user

I became familiar with Powerset’s only competitor, Hakia initially because they are a New York company. I became intrigued with Hakia because several months ago I tried their search engine, and it worked – really well. This was a surprising result for me since I have always been a skeptic regarding all things relating to artificial intelligence, speech recognition, natural language processing, and other such fuzzy technologies.

At least in the area of natural language processing Hakia that has changed my mind. In fact, it has become common for me to use the Hakia search engine when Google does not deliver sufficient results.

Hakia and Powerset are part of the same general area of natural language search. The idea with both services is that you can actually ask specific questions and get answers. But there are critical differences between Hakia and Powerset. And those differences bring me back to my incredulity at the idea that Microsoft is taking a serious look at Powerset.

Powerset indexes 750 times slower than Hakia!

I have no expertise in natural language processing or semantic search, or any type of full text search for that matter. But as far as I can tell, Hakia’s technology is *far* superior to that of Powerset’s. Why would I say that?

Well first, as I have already said, it works. It is a real live search engine. I use it. I can’t say the same for Powerset. Powerset has yet to show anything but a search engine for Wikipedia. A big part of the reason Powerset doesn’t seem able to offer a real search engine is the fact that according to their own reports, it takes them about 25 seconds to index a page, based on an average of 25 sentences per page. According to Hakia it takes them 1/30th of a second to index a page. Essentially this means that Powerset cannot scale. It is seven hundred fifty times slower than Hakia!

Now you might assume that Powerset is slower because it’s applying some serious, and superior indexing mojo, and therefore what it is doing is much more valuable than what Hakia is doing. But alas that is also not true.

Hakia really knows how to read

Hakia is doing something called “ontological semantics”. What this means is that over the last four years, Hakia has developed an “ontology” for human expression. In layman terms, what this means is that what Hakia does when it indexes a page is to look at each sentence and figure out what the *questions* are that each sentence answers. Any given sentence usually answers 3 or 4 questions. These questions are coded and go into what Hakia calls their Qdex, or question index.

In order to be able to figure out what the relevant questions are for a given sentence, Hakia’s indexer has to literally read the sentence. By “read” I mean it has to understand the actual meaning of the sentence semantically. This is a big deal.

Powerset uses statistics + syntax but can’t actually read

So, while Hakia is actually reading, Powerset, does not actually attempt to understand what sentences mean. It uses a system that parses the syntax of the sentence and guesses matches based on statistics. But this approach means that for questions that do not match previously encountered syntactical patterns, the system will not be able to find answers, even if there are in fact answers in the database.

Powerset benefits from the Silicon Valley echo chamber

Now, if, for a moment, you presume that it is true, or even *possibly* true that Hakia is the superior service and technology, or if you even assume that Hakia is just equivalent to Powerset, why would Powerset be so continuously celebrated while Hakia is overshadowed?

The only answer I can come up with is that the west coast is such an echo chamber that very little sound gets in or out. And so it must be shocking when a New York company develops a technology that seems to beat the pants off something that should be pure Silicon Valley. Just a thought.

In any case, it seems, for the record, worth noting that we have the clear leader in natural language processing and search technology right here. And, as an admitted New York partisan, after a while it does get a little annoying to hear such continued fawning over a west coast company that is very likely, at the end of the day, just another Silicon Valley also-ran.

This article was authored by Hank Williams who is a New York-based entrepreneur who recently launched a new blog: Why Does Everything Suck? exploring the tech marketplace from 10,000 feet.

Hakia describes the updates as, "To ensure credibility, hakia is providing results from sources that have met the quality criteria set forth by the Medical Library Association. hakia’s users can now trust what they find, because experts have vetted the sources."

I’ve said before that I think Hakia does a better job than Mahalo as it brings together human and machine search into a smarter page. Hakia needs to spend more time and effort on marketing now. They are competing with Google on the machine side, Mahalo on the human side, and now iMedix on the health side. All three competitors have serious money and some level of famous executives.

We’ve written about Hakia before including their demo at the NY Tech Meetup which I have embedded below.

I stand firm that hakia is a better solution than Mahalo and could actually be stronger than Wikia Search hopes to be. It’s unfortunate that they haven’t found the right avenues for promotion and buzz generation so far. I also think they need to figure out marketing messages for the technical people and for the mainstream users.

1. Is Mahalo a clone of Hakia Gallery?
2. Corporate social tool usage
3. Where does Mahalo get their high Alexa ranking from?
4. Positive employee feedback and encouragement from management

Mahalo: A Hakia Clone?

Last week I met with semantic search engine Hakia.  One of the features they demo’ed to me is the "Hakia Gallery". The Gallery is a partially human-edited results for over several thousand results. The immediate reaction I had was "Is Mahalo a Hakia clone?" – do a search for cancer on both Hakia and Mahalo to see an example. Mahalo probably wins a point on design but gives up multiple points as the Hakia Gallery uses some human and some computer to create a much more robust page. So is Mahalo a clone? We already know that Jason knows how to clone… see Netscape.

Update: Based on my offline emails with Jason regarding this section, here is my clarification. I am in no way suggesting that Mahalo lifted any design elements from Hakia. I am merely noting that the pages are similar. I think Mahalo’s design is easier on the eyes and Hakia’s technology appears better by combining human-edited results plus their search technology. Also, please note that the link to Netscape being a Digg clone goes to TechCrunch.

Mahalo’s social media usage

One of the ways Mahalo is getting traffic is by leveraging their employees and contractors to submit and vote up stories on the social media sites. Here is an example post on the news group:

When I first came across this, I was completely outraged. How could a CEO ask his or her employees to digg, reddit, stumble, scape a story to get it on the front page. It’s one thing to ask some buddies to help out with a digg, but for a corporation to ask their employees to do this, just seemed wrong, very wrong.

I decided to hold off posting about this as I wanted to really think about it. And here is the conclusion I came to. It’s ok in moderation. I guess there is no difference to this type of cheerleading as if Ford asked their employees to pass out Ford info as they walk in a park. Jason isn’t forcing the employees to vote, nor is he asking for positive votes. As Mahalo grows in number of employees/contractors, we will see more Mahalo stories hit the front page. I do wonder, if the social sites were all "corporate run" would the largest corporations always make the front page of x site?

Investigating Mahalo’s Alexa ranking

Jason has touted his Alexa ranking (for whatever its worth, which is basically nil) all over the Internet, mostly at Valleywag. And currently Mahalo sits at a rank of 5,267 for the 3-month average and an awesome 3,256 for last week. I decided to put on my Columbo analytics hat and have a look deeper (ok, only as deep as Alexa goes!). Here is the chart of number of pageviews and what parts of the Mahalo site are generating the traffic:

Ten pageviews per visitor – who wouldn’t love that! And then we see where the traffic is coming from. Almost half of the traffic is to internal, non-public sections of Mahalo. Greenhouse is the section of the site where the "guides" work on creating the pages that appear on Mahalo. And this means that at least a portion of the guides are using the Alexa toolbar. When you enter the Greenhouse, you search for the terms that you might want to write a results-page for. But it might take you 20 searches to find a page that isn’t created already. Hence the 9.8 pageviews per visitor!

Now back to that 5,267 ranking. Since we know that ~50% of the traffic is to internal sections and we know that the majority of views are from guides, we can apply this to the other ~50 of the traffic that is going to the main Mahalo.com site. In essence, the majority of the traffic to Mahalo is actually internal, employee traffic! I wonder what their ranking would be if this was removed. My HP10B++ estimates it to be in the 60-80k range.

Jason is smart, very smart. Get your employees to use a publicly available tool and over time, as you increase employees, you increase monetizable page views. Even if it’s not on his radar today, it should be. Imagine if you had 1 million page views, and your employees added another 3 million creating pages to support the other 1 million. To the average Internet marketer (i.e. most of the ones I have dealt with), that’s 4 million page views to buy/sell! Genius!

Employee Encouragement

The last part of this conversation is around employee encouragement and feedback. Jason has done a great job here. Since Jason is so public online, it’s easy to tell when he is having a bad day. I have watched him throw a fit on x person or y site and just a few minutes later congratulate the guides on Mahalo. It’s so critical to give your employees the encouragement they need to be satisfied. Writing search pages is probably about as boring (most times) as watching paint dry.

But if you can get people excited about the work they are doing, they will feel better about it. I hope this post encourages Jason to talk about employee encouragement in the Web worker field. I have managed teams for years now and it’s easy to walk over to someone’s desk and tell them how great a job they did. But online where your workers are spread out, it’s not as easy. Might be a great niche there for someone to blog about!

Here are my notes from the demo:

• The ScoopBar is great for anyone doing research – after installation, searching in the Hakia toolbar (i.e. Scoopbar) allows you to click a link and be taken to the actual Web site with the search highlighted (no need to find it) and then you can save the text (plus a buffer you select) to a local page on your hard drive. I could see this as a great tool for students and educators.
• Downloads are available for IE currently, Firefox version due out very soon
• A Web API is coming soon
• Downloads available through Download.com

Have you tried Hakia or the ScoopBar? If so, please report in!

Check out the video of the demo below (you will note the person who let their phone ring!):