Wikia

While on the surface this might look like a replacement for Google’s site search or other local search tools, Wikia’s WISE is more of a developer complement. WISE could be used to create some VERY large and powerful mashups. You can combine the search with other APIs including Yahoo’s BOSS search API. There’s a developer’s guide to help you get startup working with WISE.

Here’s a sample application from the presidential debates.

Wikia’s Search remains a community-edited search index. The basic idea is if you know a better result for a search query, you can add it instantly instead of recommending it like you have to on other mainstream search engines.

Launch partners include NY-based Snooth, Digg, Last.fm, AccuWeather and the Washington Post. The Vaynerchuk brothers’ startup PleaseDress.Me is also using this new WISE search application framework for their tshirt search engine. I was able to grab some screenshots of some of the current apps and they are displayed below.

Check out Wikia’s launch in August of their "evolution toolbar" and all of our Wikia coverage.

Here’s how the Wikia Evolution toolbar works once installed. When you search on Google, Yahoo, etc. or view any Web page, a new set of options will be displayed (see the sample below). The options allow you to add the page to the Wikia Search index instantly and tag the page for the appropriate keywords. There’s also a rating option available for each page that is submitted into the search database.

Wikia founder Jimmy Wales said on the Evolution toolbar launch, “This toolbar, like everything we are doing at Wikia Search, is open source. We hope that if you are a toolbar fan and programmer, you will let us know what features need to be added and/or take this and do something surprising and cool with it.” 

There are also a few site tools launching today:

• Most Queries
• Most Contributions
• Recent Top Queries
• Recent Top Contributions
• Most Wanted – the difference between top queries and top contributions
• Keyword Trends

Most recently we reported on Wikia Search adding Google AdSense advertising to the search results.

Update 11:45am – Looks like they removed the box around the ads.

It immediately occurred to me that “Dave” would be a great contributor to ArmchairGM. After all, he had written excellent historical sports content to the point where I was able to base my writing almost exclusively off his. Scrolling down on his page, I found his email address, and sent him an email. It bounced.

Dave’s page admits that it was last updated on May 15, 1999. A quick look at his yearly summaries buttresses this fact, as the last summary written is 1998. And there’s one other interesting thing on Dave’s page: a copyright notice.

So here I have:

• Content I want to use
• An author who I can’t (easily) locate
• A clear indicator that the content is unavailable without permission.

That last bullet is, legally, meaningless, but anecdotally important. Web culture wrongly tends to assume that, in absence of a copyright notice or the equivalent, all content is available so long as due attribution is given. In this case, even that assumption is clearly false. Unless I have the author’s permission, I cannot use the content. And getting permission is, at best, difficult, as the author’s email address is inactive. (As it turns out, I probably could track the guy down. But let’s assume he had a more common last name; say, like, “Lewis.”)

The side effect of all this? Even if I were willing to pay for rights, I cannot, because there is no one to which I can write the check. In a very real sense, the content is held captive by an absent rights-owner, and the cost of me either (a) tracking down the rights-owner or (b) ignoring his rights and re-publishing in violation thereof probably exceed the value of the content altogether. Per a press release from Senator Pat Leahy, "Potential users of orphan works often fail to display or use such works out of concern that they may be found liable for statutory damages, amounting to as much as $150,000."

If you find this silly, rest assured you are not alone. In 2005-06, the U.S. Copyright Office studied the issue — called “orphan works” — and released its report in early 2006. Since then? A whole lot of nothing. A few bills and hearings in Congress, but no final action. Senator Leahy introduced his bill only last months, so there is some hope there, but call me skeptical regarding its odds of passage.

Even if it does pass, it only solves half the problem — that is, it removes liability from those who use “orphan” works and the rights-holder appears, but it does not remove the burdensome cost of seeking out the rights-holder. Indeed, the bill would require the republisher to “perform and document a good faith – but ultimately unsuccessful – search for the owner of the copyright in the work being used prior to such use.” And even then, “[i]f the owner later emerges and provides notice of infringement to the user, the user must negotiate reasonable compensation in good faith and render any such compensation agreed upon in a timely fashion.” Basically:

• I spend time finding some content I want to use
• I can’t locate the author easily, so
• I have to spend time and/or money to locate the author or risk significant statutory damages, and
• If the author emerges, the author has significant negotiating leverage that he’d not have held if he were easily locatable in the first place.

Wow! The words “perverse incentives” pop into mind. So do the words “not really a solution, Senator Leahy.”

A Better Idea: The Duty to Maintain

In the digital age, with content available over the Web, why put all the burden on the subsequent user of the content? Instead, let’s shift the burden on the rights-holder to, in the very least, maintain his content and/or contact information. Here are my givens and, therefore, the rubric for the idea:

• It is unreasonable to require that the author divulge his or her identity or contact information in order to receive copyright in the work

I would be surprised if there is anything controversial about that. Necessarily, the author would need to reveal his or her identity in order to enforce the rights associated with the work, but that’s a distinct question.

The corollary to that given is that a work is not considered an orphan merely because it, in the words of Leahy’s bill, “lacks identifying information pertaining to its owner[.]” That seems self-evident; otherwise, anything anonymous or even accidentally unsigned would be immediately available for republication. In short, without this tenet, copyright is eviscerated to a meaningful degree in many cases where it should not.

• If the author is unknown or unreachable, a certain amount of time must pass before the work is to be considered an orphan.

To a large degree, that is a restatement of the bullet item preceding. Using my Mets history example, though, here’s a distinction — let’s say that today were May 30, 1999, and I sent the same email to the same juno.com email address, and, again, the email bounced. The author is unreachable, sure, but he worked on the content in question (or some related content thereto) just 15 days prior. Would it be acceptable for me to declare the work and orphan and republish it? Hardly.

In the context of blogging, this point is even more meaningful. Imagine an anonymous blogger who posts once every three days. He goes on vacation for two weeks. His last post should, by no means, become succeptible to re-use as an orphan. On the other hand, say he stops blogging for two years. Different result? I think so. Similarly, apply the same logic to a known-but-unreachable blogger — in the case of a two week vacation, it’s unfair to claim that the work is an orphan, but in the case of a two year hiatus, it’s fair game.

That seems right to me. The conclusion, then: If an author is unknown and/or unreachable, and he fails to maintain his work, his works are considered orphans, and thereby are available for use by third parties without prior permission.

And, per the problem that Senator Leahy’s bill aims to address, those works should be available for use by third parties. Maintenance can be something as simple as writing an occasional blog post or, in the case of Dave’s Mets Page, changing the date to say “Last updated January 1, 2008?, or something like that. (In this case, pleas bear in mind that I am really only discussing compilations of Web-based content, such as blogs and other continually growing websites. It’d be more difficult, by leaps and bounds, to continually “maintain” a photograph, for example.) Or even more simply? Make sure you, the author, provide an easy and reliable way for others to contact you. You know, like keeping your email address current.

The question then becomes: How do we effectively put the burden on the rights-holder? My solution: a time-lapse Creative Commons license. For the first n months of unmaintained, anonymous work, the copyright holder would retain all rights. Another n months after that, the content becomes available via a non-commercial, no derivative works (NC-ND) license. And another n months after that, the content becomes available under a pure attribution license — basically, a link back. Perfectly tailored for the Web, as the default way to give attribution is via a hyperlink back to the source of the content. And for our purposes, it meets the initial goal, by shifting the burden onto the original author and not onto the subsequent user.

Could this be done legislatively? Probably, although the details would be murky and, again, I am skeptical that Congress will ever act on Leahy’s proposal, let alone my admittedly more controversial one. I would not consider Congressional action to be a realistic goal here, or, for that matter, in regard to copyright reform in any meaningful sense.

However, this rubric could be achieved socially. Already, many authors choose, of their own accord, to use “copyleft” licenses such as the Creative Commons menu or the GNU Free Documentation License (used by Wikipedia, for famous example). I believe that many authors who do not subscribe to the copyleft dogma would find the time-lapse Creative Commons licensing scheme less controversial, as so long as they maintain either (a) the content itself or (b) their contact information, the copyleft scheme will never come into play. As the risk of losing rights to one’s work only comes from sloth, it would be very hard for an author to socially defend their refusal to adopt this time-lapse license. In short, what appears controversial legislatively should be less so socially — and an effective solution to Web-based orphaned works.

Sure, there are details that need resolution, such as how the hypothetical license would define “maintain,” but those definitions are more for edge cases than the core problem set caused by orphan works. One hopes that the eventual solution for orphan works will include a duty on behalf of the rights-holder to maintain their work — or allow for licensing which permits re-use.

Dan Lewis is a co-founder of ArmchairGM.com and now works at Wikia. He just launched a new blog at DLewis.net. You can also subscribe to his RSS feed.

The biggest feature is the ability to add and delete search results at will. So basically if you aren’t happy with the results for a term, you can add whatever you’d like. And if something is not correct, just delete it. For example, a search for CenterNetworks produced the main URL as the top result then a link on a story on Techcrunch about Yahoo – seemed out of place so I deleted it. It’s easy to say this will be spammed to heck but could actually prove itself similar to Wikipedia with limited spam. Not sure it’s a competitor to SEO Mahalo but it’s certainly more "human" than Google. If the spam was limited, this could be pretty hot over time.

You can also edit the header text and the description to make them more meaningful to future searchers. And you can comment on a search engine result (you know how I feel about this!), again to provide insight for the future.

The other big update is preview and annotations. Click the link and get a preview before the jump. Annotations allow you to pick up parts of the Web page and bring it back to the search engine result. I am not super crazy about this because it may remove the need for a person to visit the actual content creator’s Web page. The key for the search engine should be as a guide not a replacement.

Bottom line for this major update? Everything is editable.

Wikia is building tools which allow the community to actually participate in the search engine creation, modification and fosters engagement. I’ve seen good discussion on the Wikia Search mailing lists since January.

The idea of integration between a social network and the very popular Mediawiki software is very strong. Marshall explains what’s available in this open source release, ‘The source code released offers Wikia’s features for profile creation, avatar upload, friending (and "foeing" – the making of enemies in a network!), and a Facebook-"Wall" style messaging system for individual and group friend messaging called the Board. These features are now available for Mediawiki-based sites anywhere on the web, including for commercial use.’

It’s unfortunate however that the editor within the social networking section is still so gosh darn ugly. Here is a snippit of what the code looks like:

====Play-in (March 13th, 2007):====

”’Niagara over Florida A&M 85-83”’

The 22-11 Niagara Purple Eagles took the final spot in the 2007 NCAA March Madness tournament with a 2-point win over the Florida A&M Rattlers, despite being down by 5 at the half. A&M goes home, while the Purple Eagles move onto a first round matchup against number 1 Kansas. This is where the fun begins.

[[Image:73616024.jpg|right|200px|thumb|Charron Fisher led Niagara to the Tourney with a win over Florida A&M, but wasn't able to lead the Purple Eagles to a big upset win against Kansas, as the Eagles lost by 33.]]

While there are some WYSIWYG buttons, the editor is still lacking (as it is in Mediawiki as well). Companies like Wetpaint (and maybe the new Google Sites) allow for a much friendlier experience.

In the end, this move could help to extend Wikis into the space currently owned by Ning, KickApps and Magnify as together "Wiki + Digg + Social Network" is a very strong combination.

I wanted to love the Wikia Search from the minute I joined the mailing list months ago. I was very excited to see the photos of the servers being loaded into the racks and was looking forward to using the alpha. I was one of the first to get access and so far, sadly, I have to say that it doesn’t "wow" me. When I spoke with the PR person, she said that they let Wikipedia out early and look how well it worked. I noted that Wikipedia basically had no competitors and that helped it get a 1st mover advantage. I am sure that any review you read of Wikia Search today, you will see comparisons to Google, Yahoo, Mahalo and Hakia.

I spoke with Jimmy Wales for a few minutes on Friday. He said they are looking for (eventually) a 5% market share and will be showing ads later on. He said Wikia Search is a "general Web search" and competes with Google, Yahoo, and Ask. The biggest positive I see so far is that the Wikia Search team is very open to feedback and appears to be building this search engine with the community in mind.

Update: Check out a couple of other reviews as well from Mike Arrington and Paris Lemon. Of course read mine below first :)

Search Engine

The search engine works just like any other search engine. They use "Nutch" for the algorithm and display the numerical result for each result next to the text. I sure hope that this scoring is only shown for the Alpha/Beta as no mainstream user cares about what the score for a result is, they just want the best result. The indexed sites list is small and is growing so finding accurate results is a bit tricky. A search for Web 2.0 didn’t return anything that seemed correct, though searching for TechCrunch did provide a more accurate set of results. Rex suggested that I do a vanity search — the results were the NYU Stern School of Business along with about 8 other .de German "Stern" web sites. Do a search for sex and you can see why they need to index more results asap.

There is one function on the search that is new and pretty groovy. Instead of reloading the page to see the next batch of results, the page just expands on the click to show the next batch. This is very cool and has high usability but it might be limiting for cpm-based advertising.

Social Aspects

One of the major pieces of the Wikia Search are the social aspects. When you setup your profile, the keywords you attach allow you to show up in searches for those keywords. I like this because it can help you to find others that are interested in the same topics you are and build new relationships.

You can also create a profile page which is similar to a profile page on any social network. Even though I am not a fan of all of these profile pages, I have to give the nod here to Mahalo’s social page only because Mahalo lets you associate other network accounts. I can show my delicious, digg, facebook, etc. You can also friend people just like with any other social network.

Mini Articles

"Mini Articles" are basically stubs with links to full wiki pages. When you search for any term, above the results is a Mini Article which anyone can edit. The articles link goes to a full wiki page about the subject which again anyone can edit. I believe these Mini Articles will have tons of abuse. Jimmy said that there will be community admins who will be selected by Jimmy to keep the spam and blackhat activities out. He said payment will be unlikely which leads me to believe that it will be easy to game the system, even though Jimmy said on our call that "gaming is impossible".

Conclusion

Overall I would say that Wikia Search has a long way to go in terms of interface, usability and features to get me to want to change (or add) it to the search engines. I understand that it’s in Alpha and things will improve and change as the plan moves forward, however I would have liked it to be a bit more polished before hitting the public eye.

Here’s the email:

Jimmy Wales wrote:

Re: [Search-l] private pre-alpha invites available

Ping me if you want one…. we’re launched. :-) I’m going to be letting people in slowly over the next few days and we are aiming for a January 7th public launch. We want to run over the system with help from people to complain about what is broken… Best way to ask is by email, but please don’t be offended if I don’t answer right away. I am expecting a bit of a flood here. –Jimbo

I am looking forward to reviewing the product!

[15:14] <jwales> the search engine *will* launch before the end of the year, probably in private beta first, and then open to the public in early january. No specific dates are certain yet. But sooon.

It was less than a week ago that we saw the 1,000 servers being delivered. They got those power cords in quick!

Many people have speculated that this is for a crawl and/or Hadoop cluster and this was confirmed by Jimmy Wales on IRC when I asked if the servers were for a web crawl he replied “they are for the wikia search web crawl, yes”.

Here is the image that Jimmy pointed the list to:

Of course this news might be moot after Scoble declared that Mahalo will be the search leader in four years.

John McCormac asked, "What exactly is Wikia? Is it a search engine or just a form of hybrid super directory based on Wikipedia and similar sites?"

Jeremie Berrebi provided the following response, "We are not building yet-another-search-engine, we are putting our efforts into making building ANY search engine easier, better tools, better methods, more shared systems, etc.  This isn't one project, it's tens or even hundreds of them, and likely to take years."

Check out our previous Wikia coverage.

From the release:  “The Internet and Web are founded on completely open principles, I’ve championed this philosophy for instant messaging and believe that the awesome power of search should be based on the same fundamental rules,” said Jeremie Miller. “The power of a simple protocol is that it enables networks of resources to collaborate openly, to be constructive instead of competitive. I'm eager to work with Jimmy and empower everyone in the search industry with a transparent collaborative open protocol, from researchers, to developers, vertical search startups, and most importantly, end users.”

“Jeremie is a brilliant thinker and a natural fit to help revolutionize the world of search,” said Jimmy Wales, founder and chairman of Wikia. “I believe Internet search is currently broken and the way to fix it is to build a community whose mission is to develop a search platform that is open and totally transparent. This is exactly what we’ve set out to do at search.wikia.com and we’re thrilled to have Jeremie helping evolve this vision.”

I am on the search mailing list and there really are some passionate people around this search engine. Whether it can make it to the big time is still to be seen, but Wikipedia has some mainstream appeal, appeal which is critical for making a search engine work. I have heard the term Wikipedia referenced several times on TV news and report shows. Of course Google is still the most popular Internet term, it has reached Kleenex in branding.

Lately there has been some Google bashing so this might be a perfect time for a launch. I think they could use some design work as well to really get this to take off – way too much content on the home page currently.