Showing posts with label info ecosystem. Show all posts
Showing posts with label info ecosystem. Show all posts

The future of publishing: Why ebooks failed in 2000, and what that means for 2010

This post is adapted from a speech I gave at the O'Reilly Tools of Change publishing industry conference in February.

It's a great time for ebooks. There are at least six ebook reader devices on the market or in preparation. A major business magazine predicts that up to seven million of these devices will be sold next year. A major consulting firm says ebook sales will account for ten percent of the publishing market in five years. And an executive at the leading computing firm predicts that 90 percent of all publishing will switch to electronic form in just 20 years.

But the year isn't 2010 -- it's 2000, and the ebook market is about to go into hibernation for a decade. What went wrong, and what can the failure tell us about the prospects for ebooks in 2010?

I had a front row seat for the last generation of ebooks: In 1999 I was at Softbook (one of the early ebook reader companies), and later I interacted with the folks at Peanut Press (an ebook publisher) after they were bought by Palm. My short summary of the lessons I learned: Although some of the barriers that stopped ebooks in 2000 have been reduced, most of them are still in place. So I think the market isn't likely to grow as quickly as many optimists are predicting. However, the economics of traditional publishing are very vulnerable to a paradigm change. That change is likely to happen later than most people expect, but once it happens it'll probably move very quickly indeed. So stay out of the avalanche zone.

Here are the details on why, and how to avoid the avalanche when it does happen.


Why ebooks failed in 2000

I know I'm going to get some comments reminding me that ebooks didn't ever completely fail. They've been around for a long time, and some people read books on their computers every day. Granted. But the market for ebooks and ebook reader devices utterly failed to take off the way that most observers expected in 2000. It's important to understand why, or we may be at risk of repeating history.

I think the failure of ebooks ten years ago was due to five problems:

1. Not enough ebooks. The core customers for an ebook reader are reading enthusiasts, meaning they like to read a lot of books. If you ask them how many books they'd like to have available for their reader, they'll look at you funny and say, "All of them, of course. What's the point in paying for an ebook reader device that doesn't let you any book you want to read?"

In 2000, we had a huge problem with ebook availability. They were expensive to convert to ebook format (hundreds of dollars per title), and publishers were reluctant to make that sort of investment. I don't have any statistics on the number of ebooks available back then, but I remember that it was an ongoing, major problem for the company.

Today, the situation is better but not ideal. Looking at the New York Times bestseller list for February 28, all but one of the top 10 books in hardcover fiction and nonfiction were available in ebook format. However, there is still a problem with the timing of availability. Barnes & Noble had 15 books on its "Coming Soon" list for March 10, but only six of them were to be released as ebooks at the same time as they came out in print. That's a poor ratio, and would be a significant annoyance to an ebook user.

Looking at older books, availability seems to be hit or miss. Many more books are available in ebook format today than in 2000, but there are weird gaps. For example, many of the most popular works of Robert Heinlein (one of the leading science fiction authors of all time) are not currently available in the Kindle store, but are available for Barnes & Noble's Nook device. For Isaac Asimov (another all-time great), only a small subset of his work is available electronically from either Amazon or Barnes & Noble.

This sort of confusion frustrates many ebook users.

2. Ebooks were too expensive. Many book buyers feel they get extra value when they buy a hardcover book. It's more substantial than a paperback, and has a nice slipcover. The pages don't turn yellow, and the printing is generally very clear. If they like the book, they can put it on a bookcase somewhere to show their friends how tasteful they are. An ebook has none of these benefits. To many users, it feels more like a paperback -- disposable, intangible, slightly cheap. But in 2000, many ebooks were priced the same as hardcover books.

Combine high book pricing with limited availability, and most people didn't feel ebook readers were a reasonable value. The market stalled right there.

The problem with ebook vs. hardcover pricing is that publishers bundle two sorts of value when they create a hardcover book: The physical product is more impressive, and you get earlier availability of the book, often a year or more before the paperback version comes out. Unfortunately, book buyers think most of the extra value they're paying for from a hardcover is the physical book. Meanwhile, publishers (and authors) often think the main value of a hardcover is early availability. Many authors and publishers don't want to say this to the public, but hardcover books are a tax on the most enthusiastic fans of an author.

E-publishing breaks that cozy little arrangement, by separating the early availability value from the better production value. Publishers couldn't figure out what to do about that in 2000. So they often did the conservative thing, pricing ebooks the same as hardcovers. To ebook customers, that felt like exploitation, if not outright fraud.

It still feels that way today.

The situation now is somewhat improved, in that ebook prices are often somewhat lower than hardcover prices. But it has not been resolved. For example, Amazon lists Payback Time by Phil Town as a hot new release. Its list price is $26.99 and the ebook price is $13.36, so that looks like a huge discount. But the hardcover version is already being discounted to $14.57. So the ebook price is about the same as the hardcover's street price. That's not acceptable to a lot of ebook customers.

Until very recently, Amazon had been subsidizing down the price of most ebooks to $9.99 in an effort to deal with conflicts just like this, but that arrangement broke down when challenged by Macmillan. The result was a very nasty public spat in which Amazon briefly pulled all Macmillan books (paper and electronic) from its online store. That drove many book authors into a frenzy, with most of them siding with Macmillan (examples here and here and here).

Hey, you want to know how to piss off an author? It’s easy: Keep people from buying their books. You want to know how to really piss them off? Keep people from buying their books for reasons that have nothing to do with them. And you know how to make them absolutely incandescent with rage? Keep people from buying their books for reasons that have nothing to do with them, and keep it a surprise until it happens. Which, as it happens, is exactly what Amazon did. As a result: Angry, angry authors. Oh so very angry.
Amazon apparently forgot that when it moved against Macmillan, it also moved against Macmillan’s authors. Macmillan may be a faceless, soulless baby-consuming corporate entity with no feelings or emotions, but authors have both of those, and are also twitchy neurotic messes who obsess about their sales, a fact which Amazon should be well aware of because we check our Amazon numbers four hundred times a day, and a one-star Amazon review causes us to crush up six Zoloft and snort them into our nasal cavities, because waiting for the pills to digest would just take too long.
These are the people Amazon pissed off. Which was not a smart thing, because as we all know, the salient feature of writers is that they write. And they did, about this, all weekend long. And not just Macmillan’s authors, but other authors as well, who reasonably feared that their corporate parent might be the next victim of Amazon’s foot-stompery.
--Science fiction writer John Scalzi

Hey, Amazon. When cutting off publishers, don’t start with the one that has the most science fiction writers. We will blog you dead!
--Science fiction author Scott Westerfield

What is it about the tech industry and authors? Both Amazon and Google have shown a unique ability to make authors bond with publishers, people they otherwise tend to view as parasitic scum.

The relationship between Amazon and Macmillan is very complicated, and I don't want to get into the details of their contracts here. There's ample evidence for labeling either one of them a villain and/or idiot if you want to. But my point is that ebook pricing remains screwed up today. Maybe not as uniformly screwed up as it was in 2000, but it's still a mess.

3. The hardware form factor was wrong. When ebook readers failed to sell well, ebook producers tried to focus on other electronic devices -- PCs, PDAs, and smartphones.

The trouble is that for most people, the ergonomics and psychology of reading are wrong on computers and smartphones. A laptop is the wrong size and weight to create an immersive reading experience, and the backlit displays on most laptops create eyestrain compared to reading ink on paper.

PDAs and smartphones are too small for immersive reading for most people, and besides people are usually in a different mindset when they use a pocket device. They use it briefly, in short spurts throughout the day, when they are bored or need to find a bit of information. It's like the information equivalent of snacking. A reference book might be useful in this context, and holy books like the Bible sell well in electronic form because some people take comfort in reading a bit of them every day. But for most people, a pocket device isn't something that you'd curl up with for a couple of hours, the way you would with a book.

This is an area where we're obviously making a lot of progress. Amazon and Sony have both been willing to subsidize their tablet devices for years while the ebook market develops, and Apple and other big computer companies are now entering the tablet market, not to mention a host of smaller startups.

Just remember that most electronics companies are sheep. If tablets don't sell well, they will exit the market as quickly as they entered it.

4. Periodicals weren't ready. Although we call these devices "ebook readers," if you look at user attitudes and usage patterns, in many ways they are a better fit for reading periodicals (newspapers and magazines) than they are for books. Most printed magazines and newspapers are viewed as disposable, so many people don't object to paying the same price for an e-version as they do for the printed version. And most periodicals can be read in short bursts, which fits the usage pattern for mobile devices.

Even better, an e-magazine can get to the reader faster than a printed version, because it doesn't have to be printed and mailed.

When I was at Softbook, there was a lot of user interest in getting magazines on our devices. Unfortunately, very few were available, and the effort to get them converted started too late to save the company.

Today, there are electronic versions of a number of publications targeted at the ebook readers. But a couple of additional problems have surfaced. One is that often the e-versions are inferior to the printed versions. On the Kindle store, 64% of the reviews for the e-version of The Economist magazine are a single star (the lowest possible rating). Here are some sample comments:

"I was very happy and interested in the Economist on Kindle despite the cost until I learned that the subscriber content on the Economist web site is not included....For the cost involved the Kindle subscription should at least equal the print subscription benefits."
"Why does it take a week to make the Kindle version available. I find it very convenient to read and search but do not want to be a week behind in reading."
"I only receive part of the magazine. Overseas users don't get images -- including the cover image and graphs/charts."
"Many of the charts and graphs are so small the legend is unreadable which in turn renders the displays meaningless."

Time Magazine, 46% of reviews are one star:

"This is a rather embarrassing electronic version of Time Magazine. There are NO pictures, no charts, no illustrations. Instead whenever you run into an article that has these in any decent amount, they've inserted an entry telling you to go get a PDF or print version....It looks and feels like some cheap RSS reader collected this rather than being an electronic version of the magazine."
"I'd like to read some parts of Time but not others, so I very much miss having a convenient table of contents. As it is, we have to (slowly) leaf thru all articles to find out what's of interest."

Wall Street Journal, 50% of reviews are one star:

"The pricing makes absolutely no sense: $99/year for the WSJ print edition with the Online Web edition included. $119.88/year ($9.99/month) for the Kindle edition.... That makes no sense because I could buy the Web edition and read it through the Kindle Browser for no additional charge."
"I also subscribe to the print and wsj.com which shows how the Kindle edition is very limited in his layout and pizzaz....Compared to the NYTimes Reader or even the wsj.com it is a sad commentary on their apparent lack of effort. There should be a more detailed table of contents instead of just very general catagories of articles. To find a specific article is sort of a blind proposition...being forced to go through the all article until you find what you're looking for."

In fairness, some other publications are better-reviewed. The Kindle version of the International Herald Tribune has an average rating of four stars, as does the New England Journal of Medicine. But overall, there are a lot of teething problems as the publishers figure out how to produce their e-versions and how to price them. This is likely to hinder customer adoption until the problems get sorted out.

5. Poor marketing. In my opinion, the right way to create a technology product is to identify a group of customers who have a major problem, and to solve that problem decisively. It's not clear that ebooks, especially as they are constituted today, do that. Paper books simply aren't broken, from the perspective of most users. S. David Mash had a good quote on this (link):

The reading device for the paperback is widely available for free (sunlight). This device can be used for other tasks as well.

A lot of the investment in ebook devices today seems to be driven more by strategy than by user needs. E-books are believed to be an important future business opportunity, and companies are maneuvering to be in position when that opportunity takes off.

The problem is, unless they solve a user problem, and communicate it to the users, the market won't take off in the first place. This tripped up the ebook companies in 2000, and I think it is still true today. Check out Amazon's pitch for why you should buy Kindle:



Can you spot the problem? It's a list of features, not a list of benefits. Now let's look at Sony:



They're doing a tiny bit better, in that they do list a user benefit. Unfortunately, how many people do you know who want to carry 350 books at one time? I call this situation "phantom value," and it's something that happens a lot to tech companies. They've made a product without really thinking through the value proposition. When it comes time to market it, they pick one feature of the product and try through brute force to persuade customers that they should care about it. Usually the only people they convince are themselves.

This same thing happened when the music industry was first trying to defend itself from MP3 players. There was a huge fuss over the superior audio quality of CDs, and a lot of people in the music industry put a lot of effort into talking up the quality aspect of CDs. The only problem was that the average music listener couldn't hear the difference and didn't care about it.


What it means for ebooks in 2010

Although ebooks are doing much better than they were in 2000, there are still very significant structural barriers to the broad adoption of ebooks. We're in a chicken and egg situation where the content isn't fully ready for use because there aren't enough device users to force investment, but people won't buy more devices until the content gets better. As long as Amazon and Sony continue to subsidize the market, I think it will continue to grow moderately. And I think the iPad and related tablet products may help. But overall, the prospects for near-term explosive growth don't look good.


What happens next, and what can we do about it?

First, let's talk about a couple of opportunities. Paper books published today are not broken, but there are a couple of notable places where the publishing industry as it works today really is breaking down, and ebooks could help.

Save the short story. The first problem is the market for short stories. I wrote about this several years ago at length (link), so I won't repeat the whole situation here. But a quick summary is that the magazines that used to produce a lucrative market for short stories have mostly gone out of business or moved on to other sorts of content. As a result, authors have relatively little incentive to write short fiction these days.

Speaking as someone who grew up reading and appreciating short fiction, this is a loss for readers and an opportunity for e-reader devices. Short fiction is a great fit for e-readers because it can be consumed in small bites, and if authors could sell directly to their readers, the revenue could eventually be good enough that people would go back to writing short fiction. Plus it would give e-reader devices a real benefit -- content that you can't get anywhere else.

What's missing is the marketplace to make that happen. We need the equivalent of an iTunes store for short stories, tied to a mass market tablet device.

Free the backlist. At the O'Reilly conference I heard a fascinating statistic from Brewster Kahle of the Internet Archive: 70% of all the books ever written in English are out of print but still under copyright. In other words, you can't legally make copies of them, but there's not enough demand for them that the publishers can afford to reprint them. They are orphans.

These aren't just obscure books. In science fiction, my favorite category, award-winning books from the 1950s and 1960s are frequently out of print, and forget about finding less-known books even from major authors. The best you can do is a used book search, which if you're lucky will get you a smelly and dog-eared paperback in the mail. And those are the famous authors! Books from many others are unavailable in any form.

In my opinion, this is appalling. And it's also an opportunity.

Kahle is working on a project to let universities lend out electronic copies of the books in their stacks, which include many of these orphan books. As I understand it, the idea is that the library owns the right to lend out one copy of the book. If a central server keeps track of that single electronic copy, it's possible to legally read e-versions of orphaned books. It sounds like an incredibly cumbersome approach -- and it is. But it's better than nothing, and once again it's producing content for e-readers that can't be obtained any other way.

The project is called BookServer, and 1,000 more books are being digitized every day (link). It's the most hopeful thing I've heard about the future of libraries in years.


Rethink the periodical

The Internet is flooded with videos of prototype electronic magazines that publishers have been working on. Most of them look pretty similar -- there's an electronic image that looks just like a printed magazine page. The user moves from page to page by swiping a finger back and forth on the device's screen. You can zoom in to look at a graphic more closely, and zoom out to a thumbnail view that shows several pages side by side. The pages include both ads and stories, just like the magazine. In some prototypes, static pictures are replaced by videos and animations. Most of the demo is made up of page swiping and zooming, and you're left thinking, "hey, that looks just like a print magazine on the screen."

I am reminded of this:



It's called the Horsey Horseless Carriage. Time Magazine wrote about it in 2007 (link). It was supposedly an early automobile design in which a horse's head (thankfully a carved wooden one rather than stuffed) was mounted on the front of an automobile. The idea was apparently to make the car look more like a horse-drawn carriage, so the real horses would not be frightened by it. Just as striking as the horse's head is the rest of the car's design. From the wheels to the body design to the weird tiller the driver uses to steer, it is a basically a horse-drawn carriage that has a motor affixed to it.

We laugh now because we know the carriage needed a total rethink to translate it into a car -- everything from the wheels to the controls to the seat designs had to change radically. And yet when it's our turn to create something new, we create electronic magazines that look just like printed magazines.

It's a failure of the imagination, in my opinion. Most of the design of a magazine was driven by the economics of printing and mailing a paper publication. Why are the ads and text arranged the way they are? Because in a paper magazine, you can force people to skim past the ads while they look for the articles. Why is a magazine that particular size? Because that's what the post office will deliver, it fits easily in mailboxes, and it's a paper size we're used to handling. Why does it come out once a month or once a week? Because you have to bundle up a critical mass of content and ads before it makes financial sense to mail it. And on and on and on.

None of those assumptions apply to an electronic publication. They are all rules that we've absorbed from the print world, so deeply that we don't even think to question them. Some of those assumptions may still make sense in the electronic world, but many of them won't. One area where I feel strongly that our assumptions are faulty is advertising.

People reading paper magazines are used to fumbling past ads while they read. It's a standard part of the experience. But people using an electronic device have been conditioned by the web to expect to click and jump directly to the content they want. Making them flip through simulated electronic pages full of ads simply won't work. That means the ads in an electronic publication probably can't be as numerous as they are in a print publication. What's worse, the ads that pay the most money -- the inside front cover and the back cover -- don't even necessarily exist in an electronic publication.

I think some magazines believe they can force the current ad experience on users. Some of them even have persuaded themselves that readers see the ads as part of the value of the magazine (see my discussion of phantom value, above). But publications need to understand that they'll be competing with a new crop of publishers who grew up online and are not hamstrung by the same thinking.

The best example of this new thinking is Yahoo. It's very trendy to dismiss Yahoo these days because it's not Google, but in reality the company is a very different beast. Google is all about search and direct-response advertising associated with it. Yahoo is basically an electronic publisher supported by "display" ads -- brand-building ads created by large national advertisers, targeted at the specific demographic groups Yahoo delivers.

Yahoo today runs a hugely successful electronic newspaper. It has a news section:



A finance section:



And a sports section:



All of them are totally supported by ads, with no subscription fee.

If you're a magazine or newspaper publisher, you may think that e-publishing finally gives you a path out of the free-web-content trap. But ask yourself what happens when companies like Yahoo realize they too can create electronic publications for ebook readers. Will they charge for subscriptions, or will they create completely ad-supported publications? What does that do to your business model?

I think the periodical has to be rethought much more thoroughly than it has been to date. At its core, the thing that makes a magazine or newspaper valuable to readers is its editorial staff -- a group of writers, editors, and artists who work in synergy to produce a unified product. Rather than asking how to make a magazine electronic, we need to ask what must be built around an editorial staff to make it viable in the electronic world. I don't know what the result will be, but I'm pretty sure it won't look like a print magazine scanned and transferred to a screen.


Publishers: Rethink your value

Although publishers today are focusing on what ebooks do to their distribution channels, the real threat to them, in my opinion, is the likelihood that in the future authors will publish their books directly to the public, bypassing the entire publishing value chain. To understand this challenge, it's necessary to look at the current value chain for books...

An author typically gets about 12% of the list price of a book. The rest of the revenue is consumed by the distribution channel -- the publisher's overhead, the cost of printing and shipping books, the expenses of the bookseller, etc. This is not to say that publishers and booksellers are getting rich. Typically a small number of bestselling books generate the revenue that covers the losses a publisher takes on everything else it publishes. Something similar happens to bookstores. The reality is that the whole publishing value chain is grossly inefficient -- it absorbs a lot of cash, and almost no one gets rich from it.

This distribution chain was stable only when it was the sole way to get a book to a customer. It's already under attack by Amazon, which avoids the overhead of a physical bookstore; and by discount retailers who skim off the best-selling books, absorbing the revenue that formerly supported local bookstores. But that's only a prelude to what's coming.

Because authors get such a small percentage of the sales price of a book today, any system that let them capture more of the revenue from a book sale will be very attractive to them, even if it sells a lot fewer books.

The chart below illustrates my point. For simplicity, I've assumed a best-selling author who gets 15% of the book's revenue, a bit more than usual. The author's new book is going to sell 100 printed copies through the traditional retail channel at $20 each. That means the total revenue for the book will be $2,000, of which the author gets $300.

But if the author sells the book direct to the public as an ebook, he or she will be able to keep 70% to 80% of the revenue (because that's what the online content stores are typically returning). If the store's cut is 25%, the author will make $300 after he or she sells only 20 books.



The red and blue bars show the author's revenue as ebook readers reach various levels of penetration in the book-buying population. The chart's kind of complex, but its main message is that once e-readers are in the hands of about 20% of the book-buying public, an author has a financial incentive to sell direct rather than selling through a publisher.

Fortunately for publishers, e-readers are far below 20% penetration today. They're probably at about 2%. So the business is stable for the moment. In fact, it's probably a little more stable than a lot of publishers believe. We're likely to have a latency period of at least several years while the e-reader installed base gradually grows. During this time nothing terribly dramatic will happen to publishers, and they may think they have the situation under control. But then we'll reach a tipping point, and suddenly established authors will have a financial incentive to go direct rather than bothering with paper publication of their books. Once that happens, all book buyers will have a very strong incentive to get e-readers -- some books by bestselling authors simply won't be available in paper form, or will be available first electronically. This will drive more rapid sales of e-readers, which will give authors even more incentive to bypass the publishers.

Once the dam cracks, the water will move very quickly.

Some notes on this scenario:

--I simplified the pricing story by assuming that ebooks are priced the same as hardcovers. They aren't, so the tipping point is probably a bit higher than 20%.

--On the other hand, Macmillan's move to raise the price of ebooks actually brings the tipping point closer. Every time ebook prices go up, that creates more incentive for an author to go electronic.

--The authors most likely to switch to electronic publication are the established names who don't need a publisher's help in marketing. Those authors are also often the most profitable for a publisher. That means the impact of the switch may be even greater than what I laid out here.

--Products like the iPad bring the tipping point closer, because they are tablets that do other things than just reading books. This bypasses the chicken and egg situation that killed e-readers in 2000. Every time Apple convinces someone to buy an iPad to do browsing or watch videos, that's another potential book-buyer who's ready for ebooks.

--The competition between Apple and Amazon will also probably bring the tipping point closer, because it holds down the cut charged by the online ebook stores. In January, anticipating the iPad announcement, Amazon cut its charge on self-published ebooks to 30%, matching Apple's terms on the iPhone app store (link).


Six critical questions for book publishers

Are publishers doomed? Not necessarily. I think we're going to end up with a range of situations in which some authors sell direct on their own, some use selected services to help them self-publish, and some partner with publishers for services similar to the things they do today. But the publishers will be dealing with new competitors and new economics, and they'll need to rethink who their customers are, and what unique value they can add from the perspective of those people. The time to do that thinking is now, before e-readers reach the tipping point. Here are the questions to ask:

1. Who is my customer, the author or the book-buyer? Most publishers today would say "both," and might add the bookstore to that list as well. But that reflects the print publishing channel structure. In the electronic world, those audiences do not have to be bundled together. There may be some publishers who partner primarily with authors, and are more or less invisible to readers. There may be other publishers that play a very prominent role in the eyes of readers (examples below). The point is to understand which type of publisher you are, and adjust your business accordingly.

2. How much value do my editing services add from the reader's point of view? I've seen quotes from publishers saying that ebook consumers will want to pay more for ebooks that are properly edited. If you believe this, I invite you to re-read the discussion of CDs vs. MP3s above. If a book is poorly edited, people will just blame the author. That means editing is actually a service for authors, not readers. Which brings us to the next question...

3. How much value do my editing services add from the author's point of view? Many authors acknowledge that their editors add tremendous value to their books; others hate their editors. But the key question is, could they hire a freelancer to do the same thing? Question for a publisher: What if some of the people you just laid off form an editing cooperative and then contact your authors with a cut-rate offer?

4. How much demand generation do we really do? This is a place where the perspectives of authors and publishers often differ. Publishers tell me that they do a lot to create demand for books. Authors typically say the publishers just shovel books onto the market and wait to see which ones sell themselves. If the publisher doesn't generate demand, then an author might as well self-publish electronically as soon as it pays more money.

5. Which brand are the readers buying? This varies tremendously from publisher to publisher. In fiction, the author's name is generally the brand that readers respond to. No reader cares who published Steven King's latest book; they just buy Steven King. But in other fields, especially nonfiction, it's more common for a publisher to control the brand. Think of the For Dummies franchise, or Sunset's How-to books, or the role that O'Reilly plays in technical books. I think e-publishing may make those brands even more powerful. A traditional publisher can help a paper book sell well by working behind the scenes to get bookstores to promote it -- put it on the table out front, place it on an endcap, and so on. Most of that promotional opportunity doesn't exist in an online store. Instead, your product is just tossed out there in a sea of other products, and it has to succeed or fail on its own. In that world, a recognized brand naturally floats to the top. That's why the Madden football game on iPhone costs $7 while many other iPhone games sell for 99 cents.

6. What sort of book am I selling? Writer/publisher Craig Mod wrote a splendid essay discussing the difference between books that have form and books that do not have form (link). Books that have form get some value from the physical book itself -- maybe it's the arrangement of text and images that creates a certain impression, or maybe it's the need for something physical (think of a coffee table book or a gift book). Those books are not going to be cannibalized easily by electronic publishing.

On the other hand, formless books (those that don't get any special value from the physical form of the book) are ripe for the picking. Think paperbacks and general-consumption fiction and nonfiction.

I'll leave you with Craig's hopeful picture of what this all means for the future of books:

You already know the potential gains: edgier, riskier books in digital form, born from a lower barrier-to-entry to publish. New modes of storytelling. Less environmental impact. A rise in importance of editors. And, yes — paradoxically — a marked increase in the quality of things that do get printed.

When we're confronted by all the downsides of change, it's important to remember that change also brings progress. If publishing gets a lot more efficient, we should see greater diversity of new sorts of publications, as well as the rebirth of a lot of old books and stories that we can't get to today. That's a future to look forward to -- as long as you can figure out how to keep your job during the transition.

Amazon Kindle: Not a home run, but an interesting start

By now I assume you've read about Amazon's Kindle e-book device. I think it's interesting and important, but more for its business infrastructure than for the device itself. And I'm not at all sure that it'll be a commercial success, unless it gets a lot more content quickly.


What they announced

Kindle's hardware is a lot like that of the Sony and Iliad e-book readers. I won't bother repeating all of the specs; you can find a good summary on Engadget here and here and in a lengthy Newsweek essay here.

The industrial design of the device looks uninspiring to me. It's made of white plastic, a color scheme that most people associate with ease of use, low price, and limited features. Considering Amazon's strong emphasis on ease of use in its announcement today, I guess the color makes sense, but it's at odds with the $399 price.

I haven't touched a Kindle yet, so maybe it looks nicer in person. But in the photographs its sloping edges and slant-key keyboard do nothing for me. It looks a bit like a badly-carved wedge of Parmesan cheese. There are a total of 54 buttons, controls, and keys on the face of the device, so naturally it looks cluttered. There's virtually none of the lust-inducing elegance of the iPhone; the design screams "utilitarian."

"Is it just me or is that thing one hell of an ugly thing to walk around with?" --Comment posted to Newsweek's article on the Kindle

The design is not necessarily a bad thing; the device is going to live or die based on its usefulness, not its looks. But the lack of a lust factor makes people much more willing to nit-pick its features and price. So far Kindle is rated 2.5 out of 5 stars on Amazon's own website, with most of the negative ratings coming from people who have never even touched the device.


Clever wireless, vulnerable business model

Interesting use of the network. Things get a lot more elegant when you look at the services attached to Kindle. Amazon has built in a radio that talks to Sprint's EVDO data network. Wireless is used to deliver almost all content to the device (except for MP3 files, which sync via a USB cable). This is both attractive and disturbing.

The attractive part is that Amazon can pre-test each Kindle device to make sure they connect to the Sprint network before they get shipped to the customer. This is a huge advantage over WiFi. One of the dirty little secrets of WiFi is that non-PCs often have a lot of trouble connecting to WiFi routers in homes and offices. I don't know why this happens, but I suspect it's because the router vendors test their hardware mostly against PCs, and never find the bugs in connecting to other devices. Trouble-shooting a Kindle that couldn't find the network would be a nightmare, and Amazon has bypassed the whole issue by leaving WiFi out of the device.

I also like Amazon's decision not to hit its users with a monthly fee to access the network. Instead, the charges are embedded in the cost of downloaded content. This means that users who buy a lot of content will be subsidizing the ones who read only a little, but Amazon has hidden the charges so well that I don't think anyone will notice. Kindle makes the wireless network do what it should do: Disappear.

I have two worries about the use of EVDO. The first is that if someone lives outside of network coverage (like at my house) their Kindle won't work properly. I would have preferred to see WiFi included as a backup. The second problem is that because Amazon has to pay for that wireless connection, it has to tax virtually any information transmitted to the device. You can load documents onto the device by sending it an e-mail, but you'll pay 10 cents for every message. That doesn't sound like much, but it's annoying to have to pay anything at all for something that's normally free.

Likewise weblogs: You have to pay $1-2 per month for every weblog that you want delivered to your device. That's understandable if you look at Amazon's expenses, but it's astonishing for something that's free on a PC. What's worse, the most enthusiastic readers -- the people most likely to buy Kindle -- are the people likely to be scanning 20 blogs a day. They won't pay $20-$40 a month just to read blogs.

One workaround would be to subscribe to an e-mail blog aggregator like Feed Blitz and have it send a daily digest to your Kindle. That'll presumably cost 10 cents a day -- $3 a month, for unlimited blogs. That is, assuming Amazon doesn't put a size limit on the messages sent to Kindle.

The relatively closed nature of Kindle has led to some angry commentary on ebook enthusiast sites that you'd expect to cheer the product. For an example, there's an essay on Mobile Read here.


Self-publishing: Nice idea, but...

I was delighted to see that Amazon is allowing authors to self-publish e-books for the Kindle. You just submit them to the Amazon Digital Text Platform, set the suggested price, and Amazon adds them to its catalog (link).

The catch is that Amazon pays you only 35% of the suggested price of the book (link). They keep 65% -- for the amazing service of adding your book to their catalog (basically, they shift some bits around on a server). And by the way, if there is any bad debt, Amazon doesn't pay you any royalties at all on that sale, even though they're the ones who failed to collect.

By comparison, Apple takes 30% of iTunes revenue, and NTT DoCoMo takes about 11% of revenue from content and apps sold over its network.



I'd love to hear from the folks at Amazon if there's a reasonable business justification for keeping such a huge cut of self-publishing revenue, but I think it's probably for two reasons:

--Amazon is greedy, and/or

--They don't want to completely undercut the royalty structure of print publishers (who typically pay up to 15% royalties on a printed book)

Either way, Amazon's royalty structure is outrageous. And it won't last. One of the most important aspects of electronic publishing is its ability to change the wretched economic structure of the industry so authors get the majority of the revenue for their work (I've written about the economics of it here). The change is inevitable, and if Amazon tries to hold its current royalty structure it'll eventually just drive people to other e-book platforms that don't rip off authors.


Will it succeed? It's the content, dummy.

All of the issues covered above will affect the success of Kindle, but ultimately the sales of an ebook reader depend on having a huge library of reasonably priced content -- books and periodicals. Lack of sufficient books is what killed the last generation of ebook readers, Rocket eBook and Softbook (I worked at Softbook for a short while, so I saw the situation there first hand).

Judged by that standard, Kindle is off to a surprisingly mediocre start. There are some promising signs. For example, people don't like paying hardcover prices for intangible ebooks, so Amazon is pricing current best-sellers at $9.99, compared to about $15-$16 for hardcover. There are hints in some articles that Amazon is even subsidizing some books to hit this price. The price difference isn't big enough to make people buy Kindle, but it helps to overcome resistance. Good move.

The problem is in the library of other books. Or I should say the non-library. There are supposedly about 90,000 books available for Kindle currently, which sounds like a lot but actually makes for a poor selection. To get an idea of what was available, I took a quick look at the titles available from several prominent science fiction authors -- Niven, Brin, Asimov, Simak, Vinge, etc. (hey, I work in the tech industry, that's what I read). The selection is quite bad -- for many authors, the only Kindle editions are their second-rate books. Or there are a bunch of individual short stories available for 99 cents each, but not most of the novels. I strongly suspect that Amazon is counting each of those short stories as one of the 90,000 "books," because they are all labeled as books in the website. If true, that means the actual number of real books for the device has been heavily exaggerated.

Try the test yourself -- go to the search page here and type in your favorite author's name. Let me know what you find. Maybe fields other than science fiction have a better selection. I hope so.

There's nothing that makes an ebook customer angrier than paying $400 for a device and then finding that most of the things they want to read on it are not available. The iPod succeeded even though a lot of songs were missing from iTunes at first -- but remember that people could rip their own CD collections, and install MP3s for free. Amazon doesn't have a base of content that its users can shift to the reader, and it charges money for any document transferred to the device. So it has to fill the library on its own, quickly.

I think Amazon has a lot of work to do here.

I'm intrigued that about 16 newspapers and magazines are available for Kindle. Unlike books, newspapers and magazines are viewed as disposable, so people are less resistant to buying them electronically. And getting instant delivery of a weekly magazine is a significant advantage over waiting a few days for it to appear in the mail.

Judging by Amazon's price to receive the San Jose Mercury News (Silicon Valley's Incredible Shrinking Newspaper) on Kindle, prices are about 40% less than print subscription. That's not bad. What I don't know is whether the Kindle editions of the papers and magazines will be the full text of the print version, or just excerpts. If anything's left out, people will be turned off.

Amazon must get a critical mass of content -- meaning a lot more magazines and newspapers, and rapid growth in books. If it can do that, Kindle may finally jump-start the ebook industry. It won't explode overnight, but Amazon has a long history of forcing its investors to wait years for the full payoff on investments. If Amazon can maintain that patience, I think it Kindle has a chance.

But I sure hope they make the next version of it look nicer.

Impact of Amazon Flexible Payments Service: Computing as a utility

The announcement of Amazon FPS made my whole week, on a lot of different levels. I'm excited about the service itself, I'm excited about what it means for the development of web applications, and I'm excited about what it'll eventually do for the mobile data world.

Okay, I'm just excited.

About FPS. Before I talk about what it means, I should give a quick overview of what it is. FPS is a web service, meaning it's a set of online APIs that the creator of a website or web application can use to perform tasks. What FPS does for you is billing -- you can use it to accept payments for something you sell online. Basically, you transmit the customer's info to Amazon, and they take care of the credit check, credit card processing, billing, and so on. They send you the money, less a percentage cut that they take.

That's not at all revolutionary. PayPal and Google Checkout offer the same thing already. Amazon's cut is about the same as PayPal -- about 2% to 3% of your revenue, depending on the amount of business you do, plus 30 cents per transaction. Google is a tad cheaper, plus you get AdSense credits for using it.

(For more information on FPS, there are good articles here and here).

What impressed me about FPS is its flexibility. Amazon says you can set different payment terms for every customer, set up subscriptions and multiple payment schedules, manage a store in which you pass payments from a customer to your suppliers, set up either pre- or post-payment systems, and most importantly you can manage micropayments down to a couple of pennies per transactions (link).

The competing systems either don't offer this at all, or do it badly. I think FPS is a really important change to the competitive situation in payment services. And, because the payment services are all available to any website, that means it's an important change to the whole web platform.

New forms of online business. So far, e-commerce online has been limited mostly to selling things that we could already get through regular stores -- books, clothing, software, etc. One of the main culprits for this was payments. The current credit card system, with its strong discouragement of small transactions, makes it very hard to sell anything priced below a few dollars online. I think the most interesting use of online commerce will be the creation of markets for things that we can't buy through stores today. Most of those things are intellectual property of various sorts, and the natural market for them is a buck or less a copy. So the payment system is a big barrier.

I won't recap my whole argument for minipayments; I wrote about it recently, and you can read it here. Minipayments have already changed the world in music, where Apple's proprietary minipayment system in iTunes has revived the market for music singles, something that was virtually dead in stores. Another example: iStockPhoto has created a market for low-cost stock photography. By creating an easy system of practical minipayments, Amazon FPS will help to enable the creation of lots of iTunes and iStockPhoto equivalents for other products and forms of intellectual property. Think short stories, art, games, and probably a lot of other things we haven't even thought of yet.

I know FPS isn't perfect -- for example, small payments have to be aggregated and then billed in a single larger transaction. But it advances the state of the art dramatically, and more importantly it challenges Google and PayPal to improve their own minipayment handling. That competitive dynamic should eventually result in a truly great minipayment mechanism online, no matter who makes it.

Amazon vs. Google: A contrast in strategies. I think Amazon's approach to web services makes Google look bad. Both companies are taking on PayPal, but Google's approach so far has been pure blunt force -- duplicate PayPal's features, underprice them a bit, and tie it to another Google product (you get AdSense credits for using Google Checkout). Let's see...you compete by duplicating someone else's features, underpricing, and tying back to your dominant product. Does that remind you of a certain company in Redmond?

In contrast, Amazon has been trying to find holes in the infrastructure that nobody has filled yet. Its storage and compute services provided very important infrastructure that helped accelerate the growth of Web 2.0 companies. Although its payment system is not as unique, the emphasis on minipayments is, and I think it too will play an important part in the online ecosystem.

Bottom line: Google is often copying, Amazon innovating. I'd say that I'm disappointed in Google, but actually given their size they would crush everyone else if they were also innovative. So maybe we should be grateful.

What will Amazon do next? Their pattern is clear -- they're picking out things that they know how to do well (because of their retail operation) and turning them into services for other developers. A logical next step would be if they offered developers the infrastructure needed to set up an online store -- order tracking, support request tracking, inventory, displaying merchandise, etc. That would work with their other services, and would put them in a position to start draining business from eBay.

I'd also love to see them offer some sort of unified product and content discovery system. One of the things missing from the online ecosystem is an easy way to find goods and services that are for sale online, and comparison shop between them. You can use search for it, but it's not very well organized, and comparisons are difficult. eBay kind of does that, but you have to be registered as one of their sellers, and eBay does the billing. I'd love to see a looser directory than eBay that doesn't take the payments directly, but just points you to things you can buy.

That's what I thought Google Base would evolve into, but Google hasn't made the move yet, so there's still time for Amazon to seize that territory.

What it means for mobile. You can probably guess what I'm going to say here. The operators consistently charge up to about 50% of revenue for any songs, games, or other content sold through their networks. The mobile software stores like Motricity and Handango charge about the same. Amazon, Google, and PayPal each take about 2-3% of revenue, and that cost is likely to decline due to competition. As the wireless Internet takes hold, how many users will be willing to pay 50% extra just for the pleasure of having a game appear on their Sprint or Verizon bill rather than their Amazon bill?

If an operator bit the bullet now and priced competitively, they might be able to hold onto about 10% of revenue in exchange for the greater convenience of running content purchases through the mobile bill. But a 50% cut is like waving a red flag in front of a bull. There's no way Amazon and friends will be able to resist the temptation to target the mobile web. The question is not if, it's when.

The name of the game is infrastructure. In an open, decentralized computing environment like the Web, the best way for a software company to succeed is to create a control point -- to offer a piece of critical infrastructure that others need, and build a franchise around it.

Google understood that concept with search + advertising, and did well with maps, but has been remarkably inept at creating other strong points. I think that's because, to be blunt, engineering PhDs don't necessarily make the best business strategists. Google, if you want to go to the next level, ya got to hire business people who are as smart as your technical people. And you have to give them some authority.

Microsoft seems to get it, but is still trying to retrofit its applications into services rather than really thinking through what's needed in an online ecosystem. Apple seems to understand, but so far hasn't been interested in opening up its services to others (it could easily have turned iTunes into a content discovery and billing service, long before either Google or Amazon hit the market). Some other big Internet companies, like Yahoo, don't seem to really understand yet that this is the competitive battleground of their future.

Amazon is the one major web company that seems to both understand the situation, and be able to consistently come up with good new services. They already have two strong points (computing services and storage), and payments looks to be the third. If some of the other players don't wake up soon, Amazon's going to end up in an extremely powerful position online.

How to really piss off a college basketball fan

One of the stranger rituals of American sports is the country's affection for the annual college basketball championship tournament. Why this country is so obsessive about college basketball and football, when every other college sport is completely ignored, is a mystery that the nation's greatest sociologists and standup comedians have never been able to explain.

But there it is. If your alma mater's basketball team is participating in "March Madness," as we call it, it's mandatory to watch the games on TV. Unless, of course, you have to take your son to a futsal game.

Futsal is a separate story in itself, basically the bastard spawn of soccer (what the rest of the world calls football) and basketball. In the US it's about a million times more obscure than college basketball, but my son likes it and his team had a game scheduled for the exact same time as my college was playing in the tournament. So I did the dad thing and drove him to the game.

But I also did the Silicon Valley dad thing and took my notebook computer with me. The high school where the game was played has a fully open WiFi network, and I had registered to receive a live streaming video feed of the basketball tournament from the CBS television network. Finally, a practical use for Web video!

So I sat down in the gymnasium, started up my notebook, logged into the network, went to the CBS website, and selected my team's game. Excitement building, I clicked on "Watch now," waited a few seconds for the feed to buffer, and...



In case you can't read it, the message said I was "prevented from accessing this game due to local blackout restrictions."

Bastards!

Here is the deal, CBS. If I had access to a TV, do you think I'd be trying to watch your crummy, pixelated, low-res, business card sized video feed? The only people interested in watching online video of a sports event are those who have no access to a television. There is absolutely zero chance of cannibalization of the TV station's audience.

Besides, think about it for a minute. Who are the people most likely to want to watch that feed? People in the school's home town who can't get to a TV. But that's exactly where the game is certain to be blacked out. So CBS has created an incredibly elaborate system to systematically tease and frustrate its most enthusiastic customers. You can't see the game you really care about, but you're welcome to watch the games that are meaningless to you.

The word for this, folks, is "perverse."

I know why CBS did the blackout. Its contracts with local broadcast stations prohibit it from streaming games they're airing. CBS had the same problem with last year's tournament -- meaning they have had more than a year to fix this thing, and failed to do so. Instead, CBS and its local stations are once again missing a great chance to build customer loyalty and develop a nice online business.

I did try out an interesting feature that CBS allowed me to access, called a "glog" (I guess that stands for game blog). It's basically written commentary on the game, streamed live. Unfortunately, if you look closely at the first and last comments below, you'll see that the commentary got caught in a loop and repeated endlessly. I was confused when the teams started running the same plays over and over.



This mess is typical of the foolishness that often happens when old line companies try to deal with the Internet. They spend millions setting up an elaborate technological tour de force but neglect to take care of the basics, like letting fans actually watch the games they want to see, and making sure all the features work.

The lesson: The product you deliver through a website isn't a bunch of HTML and Java code, it's a solution to the problem of a user. Unless all of the elements of that solution line up properly, your product is a failure.

I know CBS's online coverage isn't a total waste; some people do get to some games they want. But if you'd like a taste of what CBS is doing to a lot of fans, check out this message board where fans of Georgetown University tried to figure out how to access the online feed. It's pitiful.

As for me, I was reduced to watching the scoreboard thing you see below, and waiting for it to refresh every 15 seconds or so.



A hundred and sixty years of telecommunication progress, and I'm reduced to watching a basketball game by telegraph.

PS: My son's team lost, although he did score a booming goal from beyond halfcourt. He's always wanted to do that.

Can we please stop talking about convergence?

I know, I'm having a fantasy. The term "convergence" and the idea behind it -- that various industries in technology and entertainment are gradually merging -- is pervasive online and in the popular press. Nothing I write here is going to change that.

But let me say this for the record anyway: The problem with convergence is that when you look closely, it's not happening.

Markets aren't converging, they're diverging. The web deconstructs mass markets, by making it economically attractive for a company to address narrower market segments. Online marketing can be targeted at much more specific demographic groups than mass media could reach, and online communities help companies to talk directly with their most important customers. I've already written about this happening in mobile devices, but if you want another example, look at television: the mass markets are slipping away from the big networks, eaten by a gazillion cable channels. Or look at newspapers, chewed down by a blizzard of websites.

"Convergence" is definitely not the right word for what's happening to markets.

Are the industries converging? If the markets aren't converging, then maybe it's the industries that are doing it. The computer industry gradually merges with telecommunications, which seamlesly blends with entertainment. I can kind of get into that, except for phrases like "gradually merge" and "seamlessly blend." They sound far too gentle. What's actually happening is more like they way they make steel: coal, lime, iron ore, and oxygen get fed into a blast furnace and utterly consumed by unearthly fire. If that's what you mean when you say "convergence," then yeah we're converging.

OK, so what should we call it? I kind of like "spontaneous combustion;" it captures the spirit of what's happening. But I can't picture the Economist running a 16-page special section on the spontaneous combustion of the mobile phone industry, so we need to come up with something snappier. Maybe "deconstruction" or "re-forging." "Collision" might do the trick, but it doesn't sound quite violent enough.

I don't know, what do you think? If you have a better term for it, please post a comment.

What we're learning from Web apps, part two: Community = shared obsession

I recently wrote that the argument over the viability of Web 2.0 applications misses the point -- most of the applications on any new computing platform die. What matters are the innovations and new business models that we learn from them.

One of the things we're clearly learning about from Web 2.0 is how to organize an online community.

There have been obsessive communities in society for thousands of years: dog breeders, fraternal societies, Amiga users, and so on. What the Web has done is make it much easier for those people to find each other and hang out together. Although most people will tell you that a good online community is motivated by passion or enthusiasm, I think it runs a little deeper than that. The best Web communities are about shared obsession. They're run by people who share the community's obsessions and celebrate them together.

Ted Rheingold, founder of dogster.com and catster.com, is one of the best explainers of the whole online community phenomenon. If you get a chance to see him talk, don't miss it (you can listen to an MP3 of one of his speeches here). He gives a lot of sensible rules on how to manage an online community, the most important of which (in my opinion) is to let the users lead, but always moderate the community to weed out antisocial people. If you want to see a great example of his philosophy at work, look at the company weblog to see how the obsession with pets permeates everything the company does.

Because community members are self-motivated, it's possible for groups to exist online solely through volunteer effort. The work they can do together is often very impressive. One example I've mentioned before is that a volunteer group called the Pacific Bulb Society is gradually using a wiki to create a very comprehensive reference work on species flower bulbs. (Yes, it's esoteric, but remember what I said about shared obsessions.)

Although volunteers are great, a community can do a lot more if there are full-time people working on it, and that means forming a company and bringing in revenue. Many online communities charge money for special services, but advertising can also play an important role because for almost every community there's usually a company that wants to market to them. This is another area where the Web 2.0 companies are learning a lot. The original assumption was that it'd be easy to advertise on a community site -- just put up a banner offering a product that's relevant to the community. As Rheingold points out, that doesn't work well. People reading a community site aren't there to buy something, they're there to hang out with people who share their interests. The advertising that works best on a community site is subtle brand building tailored to the community's interests, rather than traditional offers. Basically, the advertiser needs to join the community. Because of this, Rheingold strongly urges community sites to develop their own ad sales teams, rather than working through an ad sales syndicate, because a syndicate can't fully represent the community to the advertiser.

Communities don't have to be built around a particular subject, like cars or computers. Any shared enthusiasm, emotion, or identity can be the basis of a community if it bonds people together and gives them something to talk about. For example, one frequently cited community is the photo site Cute Overload (#114 on Technorati). The obsession that ties those people together is their shared love of fuzzy animals that have pudgy faces.

Another popular site that's intended to be a community is Digg. That came as a surprise to me, because I thought of it as a collaborative news filter, a kind of group effort to replace the New York Times. (At least, that's what some folks said online, producing quite an argument at the time.)

But despite the posturing, the New York Times and Digg are doing entirely different things. The Times is hoping to be the dominant online news source in the US, funded by advertising and supplemented by fees for access to the archives, columnists, and whatever other souvenirs they can sell you. The Times says its online revenues have been growing, and publisher Arthur Sulzberger has taken to saying grandiose things about the website totally replacing the print edition: "I really don't know whether we'll be printing the Times in five years, and you know what? I don't care either."

Digg, meanwhile, is a leader in a category called "social media," with an emphasis on the word social. It's really a community site, explains founder Kevin Rose. He says his long-term intent is to use Digg's user ratings of news stories and other web content to assemble communities of people who have similar interests. Digg's system for commenting is also intended to help build the community feel. He says one of his favorite features is the ability for users to rate each-others' comments. That reinforces the community dynamic; and besides, Rose says, it's fun to use. He likes to peek at comments that have been buried by the community, just to see if he agrees.

So the news is just a vehicle for Digg to assemble communities.

True community is almost impossible to fake, because obsessive people can instantly spot anyone who's not also obsessed. If your company is setting up a community website, make sure the people running it are as obsessive as the visitors you're recruiting, or the site won't work. In fact, all it will do is make you look bad.


Mobile communities

One area where we still have a lot of learning to do is the role of communities in the mobile world. The mobile industry is very enthusiastic (you might say obsessive) about moving online communities onto mobile devices. I think that's going to be a lot trickier than it sounds. The assumption made by the mobile industry is that since community members are obsessed, they will want to carry their obsession with them at all times. But check out one of the community sites, and look at all the features that make the community work. Let's jump back to Dogster for an example. Here's their home page:



Ladies and gentlemen, that is one of the ugliest home pages on the web, and it's packed with about ten times more links than Web design principles say you should ever put on a single page. That's entirely deliberate; the site is laid out like a giant box of Valentine candies, so you can't resist trying at least one link.

How in the world are you going to deploy that on a mobile device?

You'll redesign the page. You'll remove some features, and most of the pictures. You'll make the type bigger and you'll put the most popular features up front. In other words, you'll display Dogster Lite on the phone. And that changes the user dynamic. Now instead of visiting Dogster, they're visiting a limited subset of the site. The opportunities to disappoint people are enormous.

I'm not saying a mobile version of a web community can't succeed; I'm sure a lot of them will. But it won't be easy, and it won't be straightforward.

I've been at some telecom conferences where speakers said the operators had an opportunity to create and lead new communities on the mobile Web. That's twisted thinking in two ways. The first is that communities are led by people who share obsessions with their visitors. Unless the operators are forming communities on cell tower placement and the details of FCC regulatory compliance, I don't think they are well suited to lead user communities.

Second, there is already a huge supply of vibrant communities online. It's far too late to displace them. I'm sure there will be some new communities created that take special advantage of mobility, but for the most part the challenge for the operators is not creating new communities, it's inviting the current communities in. That means giving them the opportunity to run experiments in how to format their sites for mobile, and letting them lead the resulting communities. In other words, the operators need to establish an open garden for communities, and we all know what an uncomfortable issue that is.

Finally, there's the question of revenue. The whole reason the operators want mobile data is to increase their billings. But most online communities are very low-revenue, if they bring in any money at all. There isn't a big revenue stream for the operators to tap into. They can certainly use online communities to increase the amount of traffic on their networks, but with more and more operators moving to flat pricing for mobile data, increasing traffic isn't necessarily a goal.

Unless the industry is careful, the operators could end up with a situation similar to what happened with cameraphones -- the users like the feature, but it doesn't actually generate a lot of revenue.

Next time: How the Web spawns new forms of media.

The rise of the information ecosystem: How mobile devices, personal computing, media, and the Internet all fit together

Fair warning: This is going to be one of those philosophical posts on strategy. If you're looking for quick gratification, I recommend browsing the archives here.

Anyone still reading?

Okay. The other day I got a bit of flak for posting a note about Hollywood's view of the Web. "Your weblog's about mobility," the comments said. "Stay on topic."

I sincerely appreciate the feedback, but it was a surprise. As far as I was concerned, I was staying on topic. But then I realized that I've never actually explained what I'm trying to accomplish in this weblog, and so of course people might confused. I'd like to fix that right now.

I started this weblog to comment on the mobile industry, but as time went on and my work at Rubicon exposed me to a wider range of tech companies, I found that the boundaries of mobile were getting harder and harder to define. I'm now convinced that you can't understand the mobile world as a separate industry, because it's deeply interconnected with three other industries that deal with information: the Internet, personal computing, and the media (including video, print publications, games, and so on).

All four industries like to think of themselves as separate. But in reality, they depend on each other heavily, and the connections are deepening all the time. In each industry, it's commonplace for people to be blind-sided by unexpected changes, or for major initiatives to fail dismally. I think that's a symptom of the growing connections. Because we can't yet see all the connections, success and failure become more a matter of luck than skill.

The idea that those industries are merging has been around for years -- I remember a colleague making that argument at Apple back in the early 1990s. But I think "merging" or "convergence" isn't the right metaphor. What's emerging is more like a tropical jungle where a rare tree is the favorite roost of a bat that's fed on by a mosquito whose larvae are eaten by a fish that secretes the cure to cancer in its skin. Everything's connected in subtle ways that we don't understand.

Call it the information ecosystem.

Some very bright people have used the term "information ecosystem" in the past to refer to the Internet or Web 2.0, but I think it encompasses all four industries.

That ecosystem is what I'm trying to map in this weblog, because that's where the opportunity is. I don't pretend to have all the connections mapped yet; nobody does. But what we can see so far suggests that we're still in the early stages of the new ecosystem. I think the big changes are still to come.


The new information ecosystem

Back in ancient times (around 1975), the old information ecosystem looked like the diagram below. Most information (and I'm using that word very broadly to include everything from written words to movies to photographs) was passed through a distribution hierarchy that filtered and distilled it down to the most marketable items. Delivery of information was generally through mass media -- bookstores, magazines, newspapers, television stations, etc. The prevailing metaphor was one-to-many, with information flowing from a relatively small elite of creators to the population as a whole. People also communicated directly between one-another, of course, but most of that communication was one-to-one or one-to-few via letters, meetings, and phone calls.



Even before the Internet, this old ecosystem had already started to erode. For example, computer-based desktop publishing in the 1980s made it much easier for small groups and individuals to create newsletters and magazines, giving them some of the power of mass media (although their creations still had to be printed and distributed through traditional mechanisms).

"Freedom of the Press is guaranteed only to those who own one." --AJ Liebling
"Let's give everybody a press." --Simultaneous thought of several million Internet users, sometime in the 1990s

The new information ecosystem. It was the rise of the Web that really challenged the old structure. Although we're still in a transitional period, I think it's clear that the new information ecosystem will look something like the diagram below. In the new system, the filtering role of the publishers and commentators is radically eroded. Any information that anyone wants to share can be fed directly into the Internet. Tools like the personal computer make it much easier for people to create information, and mobile devices are also starting to play a minor but important role in info creation as well (for example, at the end of 2006 a cellphone video of the execution of Saddam Hussein created worldwide news and intense political debate). The net effect should be that information flows faster, and between more sources, than ever before (by the way, that's an assumption I want to test in future posts; I'm not sure it's correct).



The diagram shows why mobile devices can no longer stand alone as a separate industry. As soon as they get any data capabilities, they're embedded in the larger ecosystem. Want to add apps to a mobile device? You need to understand the trends driving PC and Internet app development. Want to tie your customers to you more closely? Make sure you know how online communities form (and why most of them fail). Want to play content on a mobile? Don't link yourself too closely to a content company that was part of the old ecosystem -- you might be pulled down by the suction when it sinks.


What's the most important part of the ecosystem?

A lot of people would tell you that the center of the ecosystem is the Internet; that the other industries are just appendages. On the other hand, many mobile enthusiasts would tell you the dominant part will soon be the mobile phone, and I'm sure Microsoft and Apple would tell you that it's the personal computer. But I think they're all wrong. The most important part of the ecosystem isn't any technology, it's the ideas themselves: the articles and music and essays and videos and memes that we use to make decisions and entertain ourselves. The Internet and the servers that hang on it like Christmas ornaments are the storage and transport mechanism for those ideas. PCs and mobile devices are capture and playback systems, and the software programs we run on those devices are the tools that we use to create and work with the ideas.

Meanwhile, the publishers, producers, editors, and critics who used to control the idea factory are struggling to find relevant roles in the new world. I think some will succeed, and many will fail.


The real mobile opportunity

So I know it'll feel irrelevant to some people, but I'm going to be writing more about subjects like web apps and communities and Hollywood, because they're all part of the same system. I'll try to label the posts that focus on the broader ecosystem, so you can skip them if you want to. But if you're working in the mobile world I think you should tune in. You need to understand the whole ecosystem or chances are you'll be left twisting slowly in the breeze by a competitor who does get it.

The real mobile opportunity of the 21st century isn't mobilizing technology, it's mobilizing ideas.

That's what this weblog is all about.

We need a new mobile platform. Sort of.

[I'm reposting this "classic" post to fix a large number of broken links. My apologies if you receive an extra copy in your feed.]

Two weeks ago, I asked why mobile application sales are dropping. A great discussion followed, with many different perspectives (if you haven't read the comments, I encourage you to check them out).


To me, one of the most striking comments was the one no one made – nobody came back and said, "you're wrong, sales are actually going well." I think we have a consensus that there's something wrong with sales of sophisticated mobile data apps – native apps that are more than just games or ringtones.

There are two schools of thought on why app sales are down. One perspective is that the market was a mirage in the first place – most people don't care that much about mobile apps, and to the extent that they do, mobile browsing is a good substitute. I think this perspective is growing quickly among industry insiders, and even someone from Microsoft largely seconded it.

Another interesting example of this view is the weblog i-Mode Business Strategy, which tracks adoption of i-Mode data services. It's carrying a lot of quotes from operators and others saying that a mobile device should be focused on a single purpose, and the whole idea of deploying a lot of different apps or functions is low-value.

"All i-mode applications are in fact secondary, the primary being the voice. The secondary applications typically struggle through out their life cycles for the lack of focus and synergy."

The other perspective is that a diverse market for mobile data exists, but we just haven't tackled it correctly yet. For example, Bob Russell at Mobile Read wrote a very kind commentary on my post, but also strongly encouraged people not to panic and give up on the future of mobile data. Several of the other replies to my post echoed that theme.

I'm somewhere in the middle. As I've written before, I believe that a mobile device has to solve a compelling problem before people will buy it. Solving fifteen problems is too hard to market, so there needs to be one flagship function or usage that gets a device sold in the first place. In this respect, I think the "mobile device = appliance" folks are right.

But once the user gets a mobile device, I think many customers will allow it to blossom into more usages – if that blossoming process is handled properly. That's where I think the mobile data world fails today, and that's what I think we need to fix.


The role of mobile data

When I started work at Palm in 1999, I was already using a handheld to track my calendar and contacts, but not much else. I was lucky enough to get a cubicle in the middle of the developer relations team, who taught me all about the third party apps that were just then starting to take off for the platform.

The variety was so huge that I'd spend hours browsing around on PalmGear, picking out new things to download. Many of the applications turned out to be things I tried once and never touched again, but a few filled meaningful roles in my life and I came to depend on them.


There was my buddy Convert-It, which converts between English and Metric measurements (a must for any competitive analyst based in the US).


Tide Tool tells me when it's a good time to go tide pooling at the beach.


Mars Clock lets me track the time of day on Mars for the Mars Rovers. (Why does that one matter? Because if you're browsing the Mars Rover pictures that NASA posts on the Web every day, Mars Clock lets you convert between the Mars days that NASA lists and actual Earth days – so you can figure out when a picture was taken.)



This is a treasure chest

To me, the Palm Launcher became a treasure chest of useful little software gems, things that enhanced my life and made me happy. But I discovered that my gem collection was different from everyone else's. Tide Tool didn't matter to most people, and Mars Clock was interesting only to the fanatics who followed the photo stream from the Mars Rovers. Other people had their own favorite apps that I didn't care about.

Even among the Palm enthusiasts, there was never a single "killer" application everybody used. Instead, everyone had a different set of personal killer applications that met their own individual needs. Each person was a vertical market of one, the exact opposite of a mass market.

I discovered that when I was talking with potential customers or the press, I could hook almost anyone on Palm if I had fifteen minutes to show them the range of software available. But you can't spend a quarter hour per customer on a product that sells for only a couple of hundred dollars – that's a great way to go broke slowly. So we started to look for mass market ways to have that conversation.

At one point Palm planned an extensive TV advertising campaign for the range of applications, called "The Perfect Day." The first commercial was already finished when the tech bubble burst and Palm's stock value collapsed. The company decided to cut expenses, and the commercials were the first thing to go. The video was shown at a few trade shows, and that was all. (I tried to find a copy of the commercial online, but couldn't. If you know where to see it, please post a link.)

Later, when I was at PalmSource, we didn't have nearly enough money to do TV ads, so we tried to use the Web to spread the story. The result was something called the Expert Guides, a set of about 50 volunteer-written guides to applications for Palm OS. I'm more satisfied by these guides than I am by just about anything else I was involved in at Palm.

If you want to understand the power of mobile data software, and what it can mean to a person's life, browse some of the Guides. The ones on personal health, aviation, and professional medicine are especially impressive, but you can also find quirky things like cigar and liquor software, scuba diving software, and software for various religions.

In addition to creating the Guides, we ran a couple of contests soliciting first person usage stories from users. They were big successes, generating several thousand stories. They ranged from comical to heart-wrenching – people using handhelds to overcome disabilities or start new careers. Almost every story featured different apps. One story I remember in particular was reprinted in the Aviation expert guide:

"I'm a Mission Pilot with the Civil Air Patrol, the auxiliary of the U.S. Air Force. Installed on my Palm Powered device (a Tungsten T now, but over the past six years I've owned an i705, Palm Vx, Palm IIIx and Pilot Professional) are several aviation application including the Airplane Owners and Pilots Association (AOPA) airport directory. It contains virtually all information about thousands of airports across the US. On a January 11 night flight over northeastern Oklahoma, we heard a pilot call over the radio that he was having difficulty finding his destination airport in the darkness. He was alone and his charts were inadequate, out of date or missing. While my crewmember dug through his flight bag of charts, maps, directories and other guides, I pulled my Palm i705 out of my flight suit, turned on its internal light, opened the AOPA eDirectory and found information on the airport that this pilot was flying to. Within 60 seconds, my radio call advised the lost pilot of the airport's location, frequencies, hours of operation and instructions for turning on runway lights and rotating beacon from his aircraft. And my hands never left the flight controls. Is the Palm a great device? You bet. Is there a wide variety of software for use on it? An unbelievable amount. Is it convenient to use, even while flying an airplane? Absolutely. A lifesaver? The other pilot would certainly say so. And it's never farther away than the front pocket of my flight suit."

We had a database with several thousand of those stories. We never figured out what to do with them, other than list a few of the best ones in the Guides.

Based on all this interaction with customers, I an utterly convinced that mobile data, and all the myriad applications associated with it, will eventually be a very common, well-loved part of the lives of huge numbers of people around the world. This stuff is just too useful once you really dig into it.

I'm also convinced that conventional marketing won't make mobile data happen, because the compelling thing about mobile data isn't the total number of applications – it's the individual discovery of an application that does something critical just for you. We have to find a way to explain mobile data differently to every individual person.

After you think about it for a while, the right place – maybe the only place – to explain this is on the device itself. I think the right way to do it would be something like this:

When the user first gets started, the device asks the user a few questions about their work and interests. Based on the user's responses, the device suggests the installation of appropriate software. So, for example, if the user is a doctor and a boy scout troop leader, the device suggests a selection of medical software, and a boy scout troop management program (yes, there's an Expert Guide on that, listing about 70 relevant applications, including knot-tying instructional software -- something critical for a Scout). If the user agrees, these are downloaded to the device, and the user's wireless account is automatically charged for them. If the user doesn't agree, the device makes it easy to come back and buy the software later.

By customizing and automating the whole application shopping process, we make it easy for people to discover what they can do and get used to the idea of using their device for more than one purpose.

I think this integrated discovery and install process is the key to making mobile data take off.


Why hasn't it happened already?

If the right thing to do is so obvious, why hasn't some mobile OS company done it already? I spent the last seven years wondering about that question, and never got a satisfactory answer.

But I get it now.

Making real, personalized mobile data succeed isn't in the critical interest of any of the powers in the mobile design chain.

Hardware vendors focus on the lead solution that's built into their device, because that's what drives the hardware sale. So, for example, Palm has a strong incentive to spend all its time making e-mail on the Treo work great.

The OS vendors focus on the needs of the people who control their sales. That means supporting the feature requirements of the mobile operators, because they're the ones who decide if devices with a particular OS will be offered to users.

To an OS company, the operators are a feature requirements black hole, an "endless aching need" in the words of Bette Midler and Amanda McBroom. There's always one more feature they want implemented, or one more deal you can close if you just respond to a request. These opportunities expand to consume all available engineering resources, no matter how many engineers your company has.

I finally understood this a couple of months ago when I was talking with a former Palm employee who had moved on to one of the biggest mobile-related companies – a firm so huge that you'd assume they have enough engineers to do anything. We were talking about flaws in the user interface of a new product the company had released.

"Hey," I said to my friend, "You're on the inside now. Why don't you just call the product manager and explain to him what's wrong?"

"I did," my friend replied. "But the product manager said all the engineers are busy answering operator requests, and there's no one left to work on user features."

"There's no one left to work on user features." If mobile data fails, you can carve that on the tombstone.


We need a new platform – sort of

If we can't count on the OS companies to make mobile data work, then the obvious answer is to get away from the OS companies. Separate applications discovery, purchase, and compatibility from the underlying OS. Let the OS company serve the needs of the hardware companies and operators, while this new software layer serves developers and users.

What we need isn't a new OS, we need a new software layer on top of the OS.

I think that software layer should include:

--The APIs needed to run the applications, consistent across all devices so developers can write once and run anywhere (think of this as Java done right). That creates the largest possible market, encouraging the creation of the focused vertical applications that drive mobile data adoption.

--The software discovery and sales experience. The whole chain from learning to purchase to billing to installation should be built in, to make it effortlessly easy for people to try and install new mobile apps.

--The billing system needs to be managed carefully. The right thing is take a sustainable, restrained cut of developers' revenue and grow along with them. Many online mobile software stores take an enormous cut of the developer's revenue. That doesn't cultivate a developer community, it's more like running a McCormick harvester across it – you bring in an impressive harvest for a little while, but in your wake you leave an empty field of stubble.

--An open garden. Any developer should be free to add an application to the store. Real mobile data is so diverse that no entity on this earth is capable of determining in advance what people will want. Rather than trying to pick winners, the platform vendor should take a uniform cut from everyone and let Charles Darwin choose the winners. Better yet, put in a user rating system to help the best apps rise to the top.

--Sandboxing. Because anyone can publish an app, the software layer should be thoroughly sandboxed so that a rogue app can't mess with the phone network. This is much easier to do when the layer is built on top of the OS rather than within it.

--The layer should include enough of the user interface so that developers don't have to rewrite their apps for every different device.


Which companies could make it happen?

No one's putting together the whole offering yet, but there are some promising possibilities...

Adobe Apollo. Adobe says it's creating a software layer that will run on top of both PCs and mobile devices. Adobe has enough financial resources to make the necessary investments, and operators are anxious enough to get Flash content that they might agree to bundle Adobe's software.

But to succeed, Adobe would need to give away Apollo. Today the company is charging for Flash in the mobile world, which will limit its deployment and prevent the creation of a standard. Also, Adobe hasn't shown any signs of including application discovery or purchase in Apollo.

Microsoft WPF. Microsoft is working on a software layer that's conceptually similar to Apollo, called Windows Presentation Foundation. Like Apollo, it's not clear if WPF will include the software discovery and purchase experience. Also, Microsoft is holding some features out of the cross-platform version of WPF in order to prevent it from cannibalizing native Windows sales. That's a very difficult line to walk.

Nokia S60. Nokia's S60 software is closely tied to Symbian, but the company could theoretically separate it and offer it as a layer. I have no idea if that has been considered, or how hard it would be to implement. Nokia has been making some efforts at improving the app discovery experience, including the recent announcement of the "Nokia Content Discoverer," one of the most awkwardly-named software products I've heard of in the last five years. I like the idea, though – it's supposed to help people find content and apps. I haven't been able to find any pictures or video of the product in action. If you're aware of any, please post a link in the comments below.

Nokia's other handicap is that it's very closely tied to the operators. It would have to hive off resources to support the apps platform separately from operator influence.

StyleTap. This small Canadian company has created a Palm OS emulator and is selling it for use on Windows Mobile devices. So you can run most Palm OS apps on Windows Mobile. There's no software discovery element, but it is a nice software layer, and could be evolved into a layer for all mobile devices. Unfortunately, StyleTap is charging $30 a pop for the emulator. If they wanted to become the mobile software layer of the future, they'd need to give it away and make money through app sales. I doubt a company of their size can afford to do that.

Brew. Qualcomm's Brew has one of the nicest software downloading and billing infrastructures I've seen, and includes a set of APIs. So it's a full software layer. Its two problems are first that it's made by Qualcomm, a company that already holds – and charges for – many fundamental mobile patents. The last thing mobile companies want to do is give Qualcomm more control over their lives. The second problem is that Brew is set up as a series of closed gardens. The operators choose which apps to offer, and there's no discovery experience that I'm aware of. So I'd classify Brew as great technology hamstrung by a completely wrongheaded business model.

Savaje. This company has gone through a full cycle of being unknown, hyped into extreme prominence, and then dropped back into obscurity. What they're trying to do is fix Java, by making it consistent across devices, efficient, and wrapping a better business and technical infrastructure around it. The question about Savaje has always been whether or not they can deliver. I'm not close enough to them to judge that, but if they get their act together they could be a promising option.

I know I've skipped a lot of other candidates, but hopefully you get the idea (if I missed your favorite, feel free to post a comment about them). There are a lot of companies trying to make various sorts of software layers, but most of them are focusing on the APIs and technologies, the traditional control point in the PC world. That's nice, but what's really needed is a new business layer and infrastructure, not just another set of APIs. I think the first company that realizes this will be the one that drives the blossoming of mobile data.

I hope it'll happen soon.