Then, this morning I read an article By Nick Carr where he discussed the significance of open source to buyers. His thesis is that what is most important are the meat and potatoes issues around reliability security, etc. Specifically, Carr says:

We can (and will) have debates about the relative openness of Azure and AWS and Force.com and all the other "cloud platforms" that are available or will be available. And those will be important debates. But in this early stage of the cloud’s development, openness means little to the buyer (or user). The buyers, particularly those in big companies, are nervous about the cloud even as they are becoming increasingly eager to reap the benefits the cloud can provide. What they care about right now is security, reliability, features, compatibility with their existing systems and applications, ease of adoption, stability of the vendor, and other practical concerns. In the long run, they may come to regret their lack of stress on openness, but in the here-and-now it’s just not a major consideration. They want stuff that works and won’t blow up in their faces.

This is very much in line with my thinking from yesterday. Azure is a big deal. No one is going to care about the fact that it is not open source. Basic hosting is going to become a commodity business very quickly, with Microsoft, Amazon, and Google (MAG) competing in the game of creating highly scalable services that use traditional development methodologies. Microsoft is now ahead in that game from a technology perspective. Amazon is ahead in customers, and Google, for now, is left in the dust but can obviously catch up. But I don’t see any of these guys making any of their cloud technology open source, and I don’t think it matters.

I liken open source in this space to DRM in the music business. Its one of those things that a small number of people complain about but will later be proven totally irrelevant to the rank and file buyer. We now have statistics to prove that DRM was irrelevant in terms of sales, and we are beginning to see the outlines of the irrelevancy of openness in the cloud.

The real issue here is that small companies are not going to be able to compete selling basic “get your applications into the cloud” type services. MAG is going to own that business. I think that 10Gen and other companies providing baseline services are going to have a rough time playing that game.

Startups who wish to compete in the cloud business will have to provide great value added services that facilitate unique new application categories sitting on top of one or more of the MAG clouds. The services will have to be hard to copy and/or narrow enough to not attract the attention of MAG.

Given the need to innovate in some unique way, and the need to be interoperable with MAG clouds, I am not at all clear how you can create innovate cloud platform services using an open source business model in a money making way. Being open source in this space is akin to what it might be like if Apple made OSX open source and optimized it to run on standard Intel PCs. Good karma perhaps. Good profits, not so much.

Of course many open source businesses hang their hat on services, consulting and support. I personally hate time and materials type businesses masquerading as scalable software businesses, but my opinions aside, these are by and large tough businesses to succeed at.

In short, while being open source may be politically correct, I fear it may be a grave hindrance towards providing a defensible, unique, money-making offering in the cloud.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

I take issue with the concept.

Here’s the problem with the formulation. It belies a misunderstanding of what an actionable “idea” really is. A good idea is almost never some light bulb moment that occurs where you realize some insight that no one else has seen. In truth there are few of those. Very, very few people are that smart or that lucky. Great actionable ideas are really a collection of much smaller ideas, weaved together in such a way as to create something useful unique and compelling. There are few actionable “aha” moments.

In other words, to me, coming up with great actionable ideas requires lots of perspiration, iteration, and ideation. However, once you have an actionable idea that has been achieved through this process it is worth *way* more than 1%. I would say getting to this point is worth easily 50% and perhaps well more than 50% of the value of your enterprise. Actionable and truly compelling business ideas are incredibly valuable. And most people that say otherwise probably don’t have them. For example if you open up a shoe store on Amazon, there is likely no “idea” there. But if you have developed a set of insights which allows you to develop a cost effective and safe hovercraft, that is certainly a valuable idea.

The problem is that people confuse the idea creation process with the execution process. They are different. I think one can, at times, blend the idea creation process with the *development* process, but there are important distinctions. When development is just execution of some defined idea, that is not idea creation. That is part of execution.

On the other hand, when the development process is part of the ideation process, you have set the stage for an environment where real creativity is possible. But in order for this to work, the development process must be more interactive and less goal-oriented. Great ideas come from having a bit of a “lab-like” environment in the early stages of your process. This is because exploration is almost always required to achieve a great compelling concept. Few of us has the ability to see with clarity a really useful idea from the beginning of the process, which is why iteration and stepwise refinement is so important.

This leads to what I think is a very important issue in the idea development process. There are lots of people who strongly suggest that you should do your development in public. It is part of the “release early and often” concept. But I also believe that this concept is not effective in developing great ideas because it is limiting. The minute that you get real customers involved, their needs become much more pedestrian. They will yell loudly about things that may be important to their use of the product, but they will rarely yell about some new game-changing concept. In fact they will resist radical change and rethinking because it messes with their now committed workflow. And now you are comitted to supporting them.

To be clear, I am not saying that a mediocre idea can’t be a good business. But good businesses are not all great ideas.

So as I see it, if you want to do something great, you should strongly consider whether you have enough meat on your conceptual bone before you decide to release publicly. Because when you get users involved, it is the equivalent of putting the saw and the screwdriver down and grabbing the sand paper. There will likely be few additional big ideas after that point.

And so the point of all of this is that I feel few people really respect the process of creating big ideas. Compelling idea creation is hard and it is incredibly valuable. And as I see it, the “release early and often” meme is a reflection of a broad-based acceptance of incrementalism, in lieu of real creativity. In truth some of these incredibly popular concepts may be behind the incredible slow down we have seen in real innovation in the tech economy in the last decade.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

And while many of the speakers had lots of really interesting stuff to say, one statement by Nancy Schulman, Director, 92nd Street Y Nursery School really got my attention. She said “kindergarten is becoming more like regular school, but I think regular school and life should become more like kindergarten.”

For those that may miss the nuance, what she was suggesting is that joyful playful exploration is critical to learning. Rote learning and memorization is less effective. As I sat, I realized how much the ideas I was hearing relate to how we make products. Much of this way of thinking I believe is already embedded in my thought processes, but there is something different and crucially important about codifying it, and expressing it concisely. What Schulman was saying got me immediately thinking about Apple.

I believe that a big part of the reason that Apple has been successful is that they figured out long ago that their products had to have the elements of joyful exploration that are the hallmarks of great toys. The concept of play is generally something associated with children, but I believe that that desire and that need never die. It is just muted by the expectations of adulthood.

The best example of this is my mother’s excitement about her new iPhone. My mother loves her iPhone because it is the best toy she has had since childhood. No, she has not said this to me, but I can see it clear as day. When she played her first YouTube video, she could not wait to tell me. For her, the iPhone is hard enough to still present challenges, and yet easy enough that she can overcome them. And the payoffs are joyous. The sound, the animation, the smooth virtual physics are incredibly compelling and toy-like. But of course it is not a “toy” it is a phone. It has a real function so she could never be accused of “playing.”

And so, the real question here is what does Apple, and what does the Sandbox Summit teach us about our way forward. First, I think that Apple sets a great example of what is possible, and it should inspire us to make our products more playful, accessible and exploratory.

But the second and more important lesson is that adulthood, and teen-hood are not demarcation points for a reduced interest in play – in fact the converse may be the case, meaning play may in fact become *more* important as we get older. And with our high school graduation rates at 50% in our 20 largest cities, something is obviously and seriously wrong with our current process. With that said I suggest that perhaps our educational system could use a serious injection of not so serious exploration and play. It seems to work for Apple.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

I have discussed this issue in the past. I have also discussed Apple’s response to the issue, which is an improved way of handling notification of inbound events like instant messages, which would normally be implemented through a background task.

I understand Apple’s concern about background processing, but I personally think its pretty lame. As I have written, there are ways to handle Apple’s concern. But the interesting thing is that the first Android phone, The T-Mobile G1 has been announced and will be out in the wild available on October 22nd. Android has no restrictions at all on background processing, and the first Android handset, the G1 is pretty similar to the iPhone hardware.

The question is, will Android demonstrate that Steve was somehow disingenuous about this whole background processing thing, or will the Android team be shown to be foolhardy about this critical performance issue? Nokia’s Symbian-based phones seem to handle background tasks just fine, but admittedly I don’t think many people are motivated to load up a Nokia smartphone with software in the same way that people are doing with the iPhone. But Android is really a true test. There will be lots of software, an easy to use app exchange, and a sense of openness that will encourage experimentation.

I am personally very curious to see what happens in the Petri dish.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

Much of this exploration has lead me to believe that the most powerful “pregnant” web concept is the simple idea that the web should be a web of objects, and should become less a web of text or pages. Indeed the web has been moving in that direction, but the road map has not been entirely clear.

Web inventor Sir Tim Berners-Lee and the W3C have pioneered the broad outlines of the concept of objectifying the web with the ideas embodied in the W3C semantic web specifications for RDF, OWL, and SPARQL technologies. But in truth, most developers have no idea what the term the “semantic web” means and are totally unfamiliar with RDF, OWL and SPARQL.

Despite the fact that the officially proposed terminology and methodologies have not quite taken hold, the idea of “objects not pages” most definitely has. Application developers are creating APIs to allow people to access their data objects, and other application developers are using those APIs to consume data objects. And because the need is so great, when developers do not make their data objects easily accessible, other applications are going as far as scraping web pages, in effect manually objectifying source sites.

And so, while the most common term for the idea of “objects not pages” has been the “semantic web”, I would really like to get everyone around the lesser known but more encompassing term, Web 3.0.

I know the idea of glomming onto the Web 2.0 bandwagon rubs some people the wrong way, but we need a “big tent” term to describe stuff that is so important, and the truth is the word “semantic web” just doesn’t cut it. In fact, in my informal surveys, it almost universally turns people off.

But terminology aside, the concepts here are really important and are building momentum. We must, as a developer/entrepreneur community begin to focus on best practices for this object-oriented web, and to discuss its broader implications. The emerging mashups and semantic applications are compelling, but they are just the beginning. Facebook and its social graph is really the first major Web 3.0 application, so make no mistake, these ideas are powerful.

Because I believe this is such an important mission, and because I strongly believe it needs more shepherding, I have committed to doing my part to move these ideas forward. I am co-chairing the Jupiter Web 3.0 Conference Series, which launches in Santa Clara next month. My co-chair is Dan Grigorovici who writes lots of interesting stuff on this space at web3beat.

The Web 3.0 Conference is the first in what will be a regular series that we hope will become *the* gathering ground for talking about how we can, should, and will approach these next generation issues. And indeed since I have been thinking a lot about these issues I will be writing a lot about them in the next few weeks.

Particularly if you are in the Bay Area, but really no matter where you are, if you want to get a view into where the next generation of the web is going and how you can leverage it, this will be the place to be. But whether you come to the conference or not, I am hoping to spark a discussion about moving the ball forward. Needless to say I have my own ideas, which I will be sharing, both in person at the conference and on these pages in the next few weeks, but this should be a multi-way discussion. If you blog about this issue and let me know I will link to you in upcoming posts, and I will try to respond as well.

Let the Web 3.0 Era begin!

Web 3.0 Conference, October 16th & 17th, Santa Clara, CA

The Wall Street sky is falling. but what does that mean to tech companies, and particularly to startups?

The last five or six years have been all about community, "social media" and other related types of communications. That era has ended and the next phase of the Web will be about *real* productivity. That means products that make you more efficient, and more effective. It means software that saves you money or makes you money. And yes, we are really going to have to start paying for the good stuff.

One theme that has been emerging is being referred to as "web meets world". It’s an idea that has been discussed by Brad Burnham from Union Square Ventures, and also the folks at the Web 2.0 Summit. The concept is that the web needs to actually help you do things in the real world, and not just meet other folks on the web. I think this is all true but it is really just a fancy abstraction for helping people do things that matter, and things that they will pay for. As an example, Union Square just invested in Meetup — a terrific investment. Meetup makes real money charging people for helping connect them to other people. They are providing real value and so people pay real money.

I find this "web meets world" concept particularly interesting because of a controversial piece I wrote back in April called "Free Is Killing Us, Blame The VCs." The core of my thesis in that piece is not that free is inherently bad, but that too much free was distorting the value of the market because the free is only supported by VC money and not real value being delivered to users.

As a result, I opined, it was way too hard to start a small business and to grow it because you need to "get to scale" since everything is expected to be free and monetized by advertising, which requires lots of users. Perhaps the idea people found most objectionable was when I said the following:

In today’s “free” world, in most online business categories, it is inherently impossible to start a small self-sustaining business and to grow it. This is because in the digital world, advertising, the only real revenue stream, cannot support a small digital business. If businesses were based on the idea that people paid for services then small companies could succeed at a small scale and grow. But it is very hard to charge when your competition is free.

People really objected to the idea that "in most online business categories, it is inherently impossible to start a small self-sustaining business and to grow it." And of course there is room for debate here. But what is not debatable is that by and large, tech startups engaged in offering totally free services ( I am not talking about freemium here) are not making money, and they are not getting acquired. Its fine not to get acquired, but you can’t do that very long if you’re not making money. And now that "free" VC capital is drying up, sustaining such businesses will be really tough.

Interestingly, at the time, Brad, among many others, took me to task for having a dated view of the online world, and for not understanding how it really works.

But in my view, Brad’s stated new thesis is exactly in line with my writing at the time. "web meets world" really might be better phrased "web meets money." There will be fewer and fewer companies getting funded by offering services that help online folks interact with other online folks, because cool as it is, people won’t pay for it, and the bottom is going to fall out. Brad and Union Square’s new investment thesis is the canary in the coalmine for that strategy.

Brad’s rebuttal to my April piece talks a lot about new business models that are going to emerge that I am just missing. But five months later, I see no evidence of it, and "web meets world" to me, suggests that in their heart of hearts, they don’t either.

In fact, I think companies like 37 Signals have had it right all along. They preach charging people for services, and staying small, and adding real productive value. Scale is irrelevant in this model because the software ads value to the individual without the network effect. In this model, scale is a benefit, not a requirement. I am not saying there will not be successful advertising based companies, but I am saying they will have to solve really serious issues like improving the value equation of online banner ads, in order to be successful.

As I see it, this is a fantastic shift in the marketplace, because it means if you have a company that adds real value, you are less likely to get thrown off course by a flood of capital creating unsustainable competition. I am very happy the venture markets are making this shift.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

At my company, Kloudshare, a big part of what we are developing involves pushing boundaries of what browsers are expected to do. Generally speaking this is the case industry wide as the web browser is becoming more and more a real application delivery system.

Google understands this issue and has apparently been focused on some of the more glaring weaknesses of the current crop of browsers. As such, they have decided to launch a new browser called Chrome, to try to bring browsers into the 21st century.

This has the blogosphere all excited. Everyone is writing about the features of the new browser, and its strategic significance. The product sounds great, but I can only get but so excited.

Why?

Because as a developer, Chrome seems to me to be little more than pissing in the wind.

Microsoft’s Internet Explorer controls around 75% of the browser market, and that’s not the bad news. The bad news is that Internet Explorer version 6 has 25% of the market.

IE 6 launched in August of 2001.When IE 6 Launched the attacks of 9/11 hadn’t happened yet. We were in the middle of the 1.0 tech bubble. In fact, if I had had kids when IE 6 was introduced they would be in second grade this year.

And yet 25% of the market is still using it. I’m not sure, but I believe it still comes on XP installation disks. In any case the fact that Microsoft has nothing in place to induce a higher upgrade rate is damn near criminal.

And so I must contrast all of the breathless excitement over chrome with the fact that the browser with #2 market share is so bad in 2008 terms it is just barely capable of delivering modern experiences. And even to do that, *lots* of engineering goes into supporting this trailing edge of the browser market.

I’d love to see a study of how much time is wasted developing special case crap for IE 6. I suspect if you added it all up we could solve world hunger or something.

All I know is that for me, as a writer, Chrome is a fun story — as a developer, not so much. As a developer, Chrome is very much a story for the next decade and has nothing to do with my 2008 or even 2009 challenges. In fact it will be a cause for celebration if I care at all even in 2010.

The bottom line is Microsoft has been fighting the browser wars with spitballs and plastic knives and they are still beating Firefox handily. So Chrome, from a business perspective, for the forseeable future, is totally irrelevant.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

The gist of Mike’s argument is that Amazon should offer Kindle up as an operating system and reference design. This would allow third parties to create Kindle compatible devices in the same way that Dell, for example, makes PCs compatible with Windows. This would create an ecosystem around the product which would be incredibly powerful. And at the same time, Amazon would still be doing what it really wants to do, which is to sell books. By offering their own product which they should continue to sell, they get to work out all the kinks without any meddling third party companies telling it what to do. But by opening up the platform, they really get to have their control cake and to eat their large marketplace cake too.

Interestingly, this is really what Google should be doing with Android. Google is indeed licensing the Android OS to third party phone manufacturers, but by not creating and controling an initial reference design they are leaving important pieces of the design to third parties, in a field (mobile phones) where important design elements can be critical.

Anyway, getting back to Kindle, I have been a fan of the product concept but I do believe it will be very hard for Amazon to build up the kind of market that they really need and should have with such a device without getting some help. I hope they take Mike’s advice.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

But what triggered me wanting to write about this was not the rumor, but the reaction to the rumor by certain quarters. Specifically, whenever you read about subscription services, you always read vehement and angry comments from people that don’t like the idea of subscriptions. It is certainly fine to not want it for yourself. But what is odd to me is the anger that said people have at the idea that I might want something different.

It’s fascinating that this response always comes from the "free" music crowd, and yet I am confident, if they had a magic wand, they would make such services illegal, or so socially or politically unacceptable that they would not be offered.

This vehemence strikes me as strange because clearly I should have the right to buy something the way I want and a vendor should have the right to sell it to me in the way that s/he wants.

In short, it strikes me that the free music crowd is really more a religious movement than one based in logic and reason. It is very similar to the way that certain fundamentalist religious groups demonize people for different beliefs. Here the free music community demonizes subscription services because, by definition, subscriptions must use DRM, which is "evil". And the irrational zealous passion brought to bear is exactly analogous to the behavior of every out of control religious group in human history. Ok, they haven’t gotten to burning people at the stake, but you know what I mean.

The point is, even if you have the wacky view that all music or intellectual property should be free, the idea that you should consider business and interaction models, and technologies like DRM that don’t match your world view to be "evil", is, to me, bizarre. This is particularly true when the DRM *enables* a, compelling, at least for some, business model such as subscriptions.

As I see it, this movement would be more appropriately lead by a religious figure like Pat Robertson, or John Hagee, or Richard Stallman, or… oh wait, it is!

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

On the day of the iPhone launch, she waited in line at an AT&T store (and I waited with her) to get her new phone.

My mom loves her phone, but one issue came up which triggered an interesting thought.  She had a hard time using the phone initially because of her fingernails.

My mom does not have big garish fingernails, but they were longer than any man’s fingernails, extending perhaps a quarter or a half inch beyond her fingertips. The problem is that the iPhone screen requires touch by skin. the tip of a fingernail will not work. This is a problem because it means that the angle that your finger touches the screen at is such that the you end up making contact with the screen with a very large imprecise area of your finger. In short, my mom kept missing the intended screen  buttons.

Now in reality, my mom clipped her nails and everything was fine. But I guess my question is whether that is a reasonable expectation in the product design. Were there any women on the product team? We’re there any *girly* women on the product team — women that like the idea of painted fingernails that extend a bit beyond the fingertip?

I am sure many of you will say that is the price of technology. But I myself wonder if there were some equivalent male focused impediment if it would have been considered acceptable.

I am not sure that this is the perfect example, but this whole episode just got me thinking about design issues for men vs women. How many other, perhaps more subtle issues like this are there that I and other product designers/developers don’t think about? It is indeed striking that such a basic issue for the iPhone, as far as I can tell, really has not been discussed at all. Will most of my male readers, or readers in general, argue that such issues are irrelevant?

Editor’s note: In addition to Hank’s findings, last winter we wrote about the inability to use the iPhone with gloves on.

This article was authored by Hank Williams who is a New York-based entrepreneur who recently launched a new blog: Why Does Everything Suck? exploring the tech marketplace from 10,000 feet.

I think I have the solution. But to be honest, though I did think of it independently, with a little bit of research I was able to establish that none other than Bill Gates himself proposed a similar concept in his 1995 book, The Road Ahead.

That said, this is 2008, and based on my discussions with folks in the record business, the idea is really not one being looked at, but there don’t appear to be any good reasons why. And so I am proposing it here to try to get the conversation going.

The idea is fairly straightforward. We should not be buying bits when we buy songs, or for that matter, digital content in general. We should be buying lifetime rights to access. So when you buy a song, or album, you can freely re-download it from the cloud, you may stream it to yourself, you can load it on any device you have with full privileges, for the rest of your life.

In this scenario, the cloud manages your ownership. So if you lose your iPod, the cloud knows what songs you own and is ready to re-download them to you. If you don’t have your iPod with you, you can still go to any PC and stream music from your library to yourself.

The point of this is to give long-term value to the purchase of a song. Right now, if you steal a song, once you have stolen it there is not much value difference between having stolen it and having purchased the real thing, other than, perhaps, a clear conscience, which it appears is not enough.

As I see it, a set of managed relationships between artists and labels on the one side, and fans on the other, has great value, to both sides of the equation. As a consumer, this would allow you to opt-in to relationships with artists, fan clubs, discount tickets, notification of concerts you might like in your area etc. It is a way for fans, without going out of their way, to establish a deeper relationship with their favorite artists.

The interesting aspect of this is the politics of it. Who controls what? And there, I think I have a solution that works both technologically and politically.

There would need to be one, or several rights societies, kind of like ASCAP and BMI. They would be non-profit organizations that do one thing. They manage the database that stores the relationships between users and songs. But they would not be responsible for selling anything.

The idea is that all the types of businesses that exist today would service customers including traditional digital music stores, as well as streaming on demand vendors, and online music lockers. So, for example, Rhapsody would still offer an on-demand streaming service as well as the right to play purchased music with the additional privileges that would afford. When you log in, it would ask the central database what songs you have played recently. Essentially, your streamed play history is in the cloud, just the way it is  with Rhapsody today.

But what is interesting with this new model is that you can switch from Rhapsody to Napster, and retain all of your purchases and play history. Or you could go to iTunes and re-download a purchased song you lost. In each case there is probably a service fee of some sort for allowing you to re-download music, perhaps on an annual basis, but those fees would be up to the service provider, and would presumably be driven by competition.

To touch on the details for a bit, the database would be exceedingly simple. It would store two tables. I have outlined them below, and I am sure there are other fields that I have missed. But it does, at least, suggest the framework.

User Table
user ID
last access date/time

Transaction Table
The Song ID or Album ID
user ID
Password or OpenID
the seller ID
transaction date/Time
Unit Type (e.g. single stream or full purchase)

Both the service providers and the record labels would have access to the database. The service providers would have write access, and would only have read access to a given customers history if the customer gave permission for that, which might allow the service provider to use collaborative filtering to make listening suggestions, as well as allowing the user to see what they have played or purchased and when.

The record labels would have access to the database for tallying purchases in order to bill service providers. The labels would not have permission to access individual purchase records, but could pay service providers to send messages to fans of a given artist, presuming the fans have opted in to receiving such messages.

With regard to identity, the central database would have no knowledge of who anyone actually is. Credit card information is held by the service provider, but that information, and actual identity information such as name or address is only kept as needed by the service provider for facilitating transactions, and not in the central database.

Finally, the idea of this is that accounts are not transferable and become inactive after death. But since we don’t know who people really are, we have to guess, and to make it unattractive to use someone else’s account. For this, we employ several tactics.

First, we set time limits. An account may not exist longer than the average human life. This would discourage someone from using an account since it will cut off after a certain time and then all of your personal songs go away.

Second, we say if an account is not used for several years – I am not yet suggesting a particular time frame – it becomes invalid. But most importantly, service providers create such personalized experiences and suggestions that you don’t want to use someone else’s account any more than anyone wants to share a Last.fm account.

From an economic perspective, the rights society would operate based on membership fees from labels and service providers, and it probably would make sense to have several competing rights societies for the consumer to choose from. It probably also makes sense for there to be some nominal annual fee to the consumer, but since I haven’t done any modeling around this I am not sure. The main point is that the organization needs to make enough money to operate successfully given demand.

And so, the reason I am writing this is because I want to see it happen. I have no economic incentive to do so, and am pretty busy doing my own business. But I have begun exploring this as an idea with interested parties to see what people thought.

From what I hear on the label side, they would likely be receptive to such an idea, given the state of things, but the impetus from this must come from the tech side first. This is because the labels are not capable of creating the infrastructure, and without the service providers it won’t work.

For this reason, the labels really need buy-in from Microsoft, Amazon, or some other big player or players. I don’t even suggest Apple here, though they would be the obvious choice, since the concept is a direct threat to iTunes hegemony. This threat, however, is a great reason for every other tech business in consumer facing content delivery to love this idea. It radically changes the status quo.

And so really, at the end of the day, this is a request to the big tech guys to reach out to talk about this idea. Though it may be a bit presumptuous, if I can serve as a midwife in this situation, it would be my pleasure. Because of the competitive interests involved here I do think some kind of neutral third party will be required to grease the wheels.

This article was authored by Hank Williams who is a New York-based entrepreneur who explores the tech marketplace from 10,000 feet at Why Does Everything Suck?.

From skeptic to user

I became familiar with Powerset’s only competitor, Hakia initially because they are a New York company. I became intrigued with Hakia because several months ago I tried their search engine, and it worked – really well. This was a surprising result for me since I have always been a skeptic regarding all things relating to artificial intelligence, speech recognition, natural language processing, and other such fuzzy technologies.

At least in the area of natural language processing Hakia that has changed my mind. In fact, it has become common for me to use the Hakia search engine when Google does not deliver sufficient results.

Hakia and Powerset are part of the same general area of natural language search. The idea with both services is that you can actually ask specific questions and get answers. But there are critical differences between Hakia and Powerset. And those differences bring me back to my incredulity at the idea that Microsoft is taking a serious look at Powerset.

Powerset indexes 750 times slower than Hakia!

I have no expertise in natural language processing or semantic search, or any type of full text search for that matter. But as far as I can tell, Hakia’s technology is *far* superior to that of Powerset’s. Why would I say that?

Well first, as I have already said, it works. It is a real live search engine. I use it. I can’t say the same for Powerset. Powerset has yet to show anything but a search engine for Wikipedia. A big part of the reason Powerset doesn’t seem able to offer a real search engine is the fact that according to their own reports, it takes them about 25 seconds to index a page, based on an average of 25 sentences per page. According to Hakia it takes them 1/30th of a second to index a page. Essentially this means that Powerset cannot scale. It is seven hundred fifty times slower than Hakia!

Now you might assume that Powerset is slower because it’s applying some serious, and superior indexing mojo, and therefore what it is doing is much more valuable than what Hakia is doing. But alas that is also not true.

Hakia really knows how to read

Hakia is doing something called “ontological semantics”. What this means is that over the last four years, Hakia has developed an “ontology” for human expression. In layman terms, what this means is that what Hakia does when it indexes a page is to look at each sentence and figure out what the *questions* are that each sentence answers. Any given sentence usually answers 3 or 4 questions. These questions are coded and go into what Hakia calls their Qdex, or question index.

In order to be able to figure out what the relevant questions are for a given sentence, Hakia’s indexer has to literally read the sentence. By “read” I mean it has to understand the actual meaning of the sentence semantically. This is a big deal.

Powerset uses statistics + syntax but can’t actually read

So, while Hakia is actually reading, Powerset, does not actually attempt to understand what sentences mean. It uses a system that parses the syntax of the sentence and guesses matches based on statistics. But this approach means that for questions that do not match previously encountered syntactical patterns, the system will not be able to find answers, even if there are in fact answers in the database.

Powerset benefits from the Silicon Valley echo chamber

Now, if, for a moment, you presume that it is true, or even *possibly* true that Hakia is the superior service and technology, or if you even assume that Hakia is just equivalent to Powerset, why would Powerset be so continuously celebrated while Hakia is overshadowed?

The only answer I can come up with is that the west coast is such an echo chamber that very little sound gets in or out. And so it must be shocking when a New York company develops a technology that seems to beat the pants off something that should be pure Silicon Valley. Just a thought.

In any case, it seems, for the record, worth noting that we have the clear leader in natural language processing and search technology right here. And, as an admitted New York partisan, after a while it does get a little annoying to hear such continued fawning over a west coast company that is very likely, at the end of the day, just another Silicon Valley also-ran.

This article was authored by Hank Williams who is a New York-based entrepreneur who recently launched a new blog: Why Does Everything Suck? exploring the tech marketplace from 10,000 feet.

First, Jakob says we read a very small amount of the text on most pages. Apparently we only, on average, read at best 28% of the words on the page and most likely around 20% of the words.

Second, in an older article from late last year, Jakob also says users avoid anything that looks like a banner. This may seem obvious, but the interesting thing to consider is that all web pages on a given site (and even across the web on different sites) are likely to have the same pattern for what looks like an ad vs what looks like content. Your eyes quickly learn that anything that is on the sides, or anything with a border around it, or anything that looks photographic is an ad.

Newspapers and magazines generally do not have any such easily to discern pattern. Pages are generally laid out manually by a designer on an issue by issue basis. Smaller ads are blended in with the content, breaking columns and such so that advertising will have unavoidable and unpredictable proximity to content. Moreover, many ads take up the full page which requires at least some minimal scanning before turning the page. And because the ads are often so big, you almost can’t avoid at least seeing a bit of it before turning the page.

The bottom line is that users see too much stuff on the web and so they focus on very little. On top of that they have learned effective patterns for avoiding ads. And while we try to avoid ads in print, the physical size of the advertising, the unpredictable patterns, and the high degree of mixing advertising and content make pattern based ad avoidance much harder in print.

In my view, the model for how we organize information on the web is fundamentally broken as it relates to siphoning off attention for advertisers. We need to inject more randomness into our designs. Indeed, the kinds of thing that Jakob Nielsen promotes as improving usability do in fact promote usability while *reducing* monetizability.

Magazines and newspapers are *not at all useable*. I am often frustrated that it is very difficult to find a table of contents in a magazine, or that some pages are missing page numbers that would help me find an article. This is all on purpose!

Print publications are very effective advertising platforms in large part specifically because they place "attention speedbumps" in your way. If print publications were as streamlined as websites, with ads neatly presented off to the side, ads wouldn’t work there either.

And so we have a decade of smart folks like Jakob teaching us exactly the right stuff to make our sites easy, but exactly the wrong strategies for making money. Amusingly, at least a part of the message is that a little disorganization is absolutely critical.

This article was authored by Hank Williams who is a New York-based entrepreneur who recently launched a new blog: Why Does Everything Suck? exploring the tech marketplace from 10,000 feet.

It does appear that most of us have, at least for a time, checked our ethics at the door. But despite the tongue-in-cheek name of my blog (Why Does Everything Suck?), I am an optimist. I do believe it is possible that the justifications for the theft we are engaged in is nothing more than temporary cognitive dissonance, and that we will indeed come to our senses.

I believe this because the consequences of deciding as a society that teaching our young children that stealing is OK is ultimately too dire. We will ultimately come to understand it is impossible to create a viable or credible ethical framework that separates stealing a song from stealing a loaf of bread. In fact, stealing the bread may be more justifiable if you are hungry and yet intuitively most of us know, no matter how hungry you are, that stealing the bread is still wrong.

On the other hand, we steal entertainment content with guiltless reckless abandon, and you certainly can’t eat a Foo Fighters song. There are statistics that suggest that plagiarism in college is up substantially, and I would not be at all shocked to find that there is a correlation between the mindset of stealing a song vs. stealing a paper.

The permissive and even supportive attitude of thought leaders in our industry on this subject is poisoning the minds of our children and we need to stop it. Ethics matter, and there is no way to explain to a child that theft is OK when you are “mad at the man”, and “big corporations are evil.” Our brains don’t work that way.

The psychology of our youth has already been irreparably damaged, but I do think we can turn things around, at least for the next generation. And we must. There are those that will suggest that because we can steal, that we will. They believe that we are no more ethical or honest than our circumstances will allow. I do not subscribe to this theory of human nature. I think we are all villagers, and we will for the most part adhere to the mores of the village. If it was considered wrong to steal music or movies or software, and there was actual shame associated with it, people wouldn’t do it.

And so my prescription: Think about how your parents might react if you told them you’d just stolen 1000 songs from Wal-Mart every time you play your ill gotten MP3 collection. If you are a parent, and you see your kid has a 1000 song MP3 collection *you* should be reacting like they stole it from Wal-Mart.

All it will take is just a little bit of shame.

This article was authored by Hank Williams who is a New York-based entrepreneur who recently launched a new blog: Why Does Everything Suck? exploring the tech marketplace from 10,000 feet.