Google's Continued Growing Pains

eldavojohn writes "The Mercury News is reporting that Google's 500 percent growth since its IPO hasn't come without a cost. With the purchase of DoubleClick, Google is facing antitrust charges in both the United States and the European Union. And with their rising success, there are open source alternatives springing up."

Comments

misleading "OpenSource" use?

[El_Muerte_TDS]

OpenAds (formerly known as phpAdsNew) may be open source software, but it mostly relies on 3rd party ad sellers. It will allow you to sell ads on your own site, but as long as no advertisers are buying your ad space you have to rely on things like AdSense.
Also, open source software for rotating banners with click/impression counting has been around for ages, it's not new. phpAds was created in 1999.

Open source search engines. Well, the source might be open (just like htdig has been open source for ages). But it's not like any end user of search engines is going to run their own search engine, it's simply impossible for consumers to run their own search engine. Website operators may run their own search engine, but usually limited to their own site.

So the whole reference to open source "competitors" to google products is complete bullshit.

Re:misleading "OpenSource" use?

[vidarh]

The point is that if the source to a competitive search engine was freely available, the cost of actually competing with Google is pretty low. Google has tremendous hardware costs, but mostly due to scale. The minimum cost to start indexing the web is pretty modest - you can crawl at least tens of millions of pages a day on a single Xeon box (been there, done that), and you can create a search index from tens to hundreds of millions of documents in a day on a single box depending on how good your indexer is (again, been there, done that) and how much metadata etc. you want to index. As such, you could get going with a dozen EC2 licenses or a few thousand a month in hosting fees.

The sticking point is search quality. That's a hard problem. And that is what is limiting most of the companies trying to compete with Google: Finding ways of getting results that are as good as or better than Google's without having Google's resources available.

The crawling, the indexing and returning basic results is easy. Back in '97 I shortly did a Linux specific search engine. It took only about a week to write the basic elements, and I indexed about a million pages to test it. Today it would've taken more care, as the number of documents you'd need even for a narrow niche vertical search engine is far higher, but there are also more off the shelf full text engines available that can easily handle at least up to the tens of millions, some into the hundreds or more (though _all_ of them are slow) if you don't want to do the work yourself. But that part is not hard work.

Most of the rest of what Google does well is scale the system, and that contributes to their margins, but it's something any competitor would have a lot of time to figure out too. Besides, over the next few years we're going to see a huge number of ex-Google employees who have learned a lot about scaling from Google's system, and who are back in the job market for various reasons, or looking to start their own companies.

I used to share your misconception ...

[AHumbleOpinion]

I'm sure that the reason companies have been pouring billions and billions of dollars into advertising for decades isn't that it works, but that nobody even though to check.

I used to share your misconception. My undergraduate was Computer Science, however now I have had some graduate level marketing classes and I was surprised to find out how quantitative professional marketing is. There is massive experimentation to determine what works and what does not.

Re:I used to share your misconception ...

[TheRaven64]

Usually big companies do this by using slightly different combinations of advertising material in different geographical regions and then comparing the changes in purchasing patterns with the regions before and after the advertising campaigns. If one does well, then it is promoted to national and international levels (international is harder, since different cultures respond to different forms of advertising). A company like Coca-Cola has sales figures for individual stores and vending machines, and so can accurately correlate seemingly trivial things, like proximity of vending machines to billboards. With online advertising, people like Amazon are fairly good at tracking the people who click on a link, then return later and make a purchase.

Re:I used to share your misconception ...

[jollyreaper]

I used to share your misconception. My undergraduate was Computer Science, however now I have had some graduate level marketing classes and I was surprised to find out how quantitative professional marketing is. There is massive experimentation to determine what works and what does not. Really? Wow. My undergrad marketing class confirmed everything I'd suspected. We had several case studies to present as group projects. One in particular was presenting a new frozen food product in supermarkets. I asked him how we were supposed to go about surveying the market to figure out if there was even a demand for what we were selling before we pitched the board, i.e. him and the other groups. "Just make up the figures," he said. "The important thing here is the exercise in making the presentation." Doing my damnedest to keep my smart mouth shut, I went along with it. But seriously, what the fucking fuck? Sure, we did a bang-up job of selling the idea to the class, we got an A on the project, but the facts we presented were all bullshit. I had no more idea of whether or not the product would succeed than before I started the project.

I'd hoped that this was just a poor class and that marketing professionals would actually know what the hell they're doing. Sadly, it hasn't been the case. I worked my way through college and I've seen advertising and marketing done by mail-order catalog companies, dot.coms, homebuilders, etc. I could gather all those marketing people together in one room and you wouldn't find a single clue in the entire lot. When I was still in school, I was working in one marketing department as a jr. report cruncher. Now the way I learn things, I cannot go by rote. I need to know why we're doing things, how the process works. If I just get a series of steps handed to me like a magical incantation, devoid of rational explanation, I'll never remember it. So I ask tons of questions, ask the why's, and end up pissing people off because they think I'm using socratic questioning to point out how big of dumbasses they are. That isn't even the case! I'm just sitting there wondering why nothing makes sense. By the time I realize it's because nobody knows what the hell they're doing and I should just shut up, they're already put me on their shitlist.

What the other poster said about marketing types being good at selling their ideas, that's exactly right. The marketers I've worked with have all been con men, good at backslapping and getting an idea sold but unqualified to even understand if the idea is a good one.

I'm sure there are marketing professionals out there, somewhere, who are actual professionals. But in my experience, sales and marketing is where football jocks go when they get fat.

Competitors

[Arabani]

Apparently Yahoo! is catching up to Google, at least in terms of customer satisfaction, so I really don't think Google's dominance in search is that big of a deal. In advertising, maybe, but that's why the FTC and EU are looking into possible antitrust violations ... nothing particularly special there. Now, if they actually stopped the merger because of antitrust violations, THEN that's news. Until then, it's just hypothetical bullshit and dreams. The article does manage to make on good point, though, which is that sooner or later the market would manage to break Google's (hypothetical) monopoly. Heck, there's already countless startups all hoping to displace Google. Google does not come close to enjoying the dominance that Microsoft once did, which is why all this concern about Google shutting out the competition seems premature, at best.

Re:Big is evil?

[Epsillon]

Thanks for the correction, Phil, although I notice that they do say "in the coming months" and, since the blog post was made in July, there's no indication of when the policy kicks in. They also fail to mention what happens to users who already have the long expiry cookie on their system.

Don't get me wrong, Google is the best engine out there, period. Tinfoil aside, I simply think that people may have misconstrued the motto; "Do no evil" and the unwritten subtext "because we are watching you" extrapolated from their actions seems to be an admonition rather than a corporate statement of ethics.

Re:Google's master plan.

[somersault]

Yep, I don't get the 'googleisthenewmicrosoft' tag in this article. People hate Microsoft because they make shoddy products, and still dominate the market. If they made decent products, people really wouldn't care so much. I wouldn't anyway. A good product deserves to dominate. Poor products dominating purely through marketing and underhanded tactics is what is disgusting, but google's products are actually pretty useful, stable and have decent interfaces.

Using google...

[Ortega-Starfire]

... Has yet to bite me in the ass. Until then, I will continue to use them, regardless of alternatives, unless I find something that works 100% better.

Other companies, such as Symantec, Microsoft, Apple (read iTunes crapola), and other various smaller groups will have to work very hard to regain my business.

Personal search

[Colin Smith]

Bayesian searching.

Put in some terms, it comes back with some preliminary results, rate them as what you're after/not and it then starts rating sites by the match closeness.

Spamming becomes very difficult... Unless that's what you're searching for and ads on the search site could use the same corpus to determine which ads to display to the searcher.

people who call Google evil

[someone1234]

What they say about Microsoft?

Google didn't do ANY evil to me, it is a free service for me.
Google won the search engine market by simply serving the best search results.
If i don't like Google, i could try other search engines.
Why do I still use Google? Because it is the best.

M$ on the other hand provides its mediocre software force-fed to me.
I cannot avoid it even when I want to.
Most software types already exist on Linux, but games are still scarce.

Re:Unregulated capitalism == evil

[Catil]

Google knows that new searchengines or advertising companies are barely a competition to them because those can usually just be bought - most of them were even started with the intention to hopefully sell out to one of the big players. Like Youtube and Doubleclick, they will probably continue to buy everything that has a strong userbase or many customers, even if the used technology is not interesting to them.
I think what they really fear are sites that make more sense to use as a startpage than a searchengine. Those are usually trying to combine everything people use on a daily basis on the net, like email, news, social networks, and especially RSS. Just like Microsoft, Google tries to have every new technology available on their own, so that new features that get very popular on other sites can be added to their own tools in no time. I am quite sure that if Google wouldn't have implemented Gagdets so fast, some other site offering something like that (and there are a lot by now) would have taken some serious share from the... umm, "startpage-market."
However, as long as those other sites still use Adsense to monetize their work and Google custom search, there is not much to be worried about. Sites that make money through subscriptions on the other hand are more of a threat to them, especially if those subscriptions will continue to be paid with Paypal and not with Google Checkout - I think they were kind of late seeing that coming.

For me, websites that could compete with Google would have to offer whole new ways to surf on the web, with very strong but easy to use personalization and custumization features, some kind of "See the sites like YOU want them to see" or "Stop using a searchengine, we already found what you are looking for." It's a bit hard to get into details, but imagine some kind of webbased RSS reader that not only shares every feed a user added with others but also crawls the web for feeds on it's own. You can then select what you are interested in and it would feature some kind of very fancy recommendations engine based on an algorithm yet to be invented - not this "users that like this also like that" stuff but perhaps something semantic. Then you surf the web just like you watch TV, switching channels from politics to sport. It would probably have Firefox toolbar where you can see the feed headline of the next site to visit with skip, proceed and "try something else" buttons.
Another possible competitor would be said Gadgets-like startpage that implements even more stuff - in fact, everything; where you could post a comment on Slashdot right from your startpage or see if someone answered your somethingawful forum thread. There are already some tools and websites that can or at least claim to convert every website and forum to RSS feeds. Some already do a pretty good job at it but they all have some way to go. I guess Microsoft could even be the one doing something like this. It could be the reason they are supporting OpenID because one problem would be logging in "cross-over" to the various sites.

Re:How long can it last?

[dk.r*nger]

The thing that marketers are best at selling is...advertising.

This is probably true for small advertising budgets - everywhere, all the time, good salesmen are selling bad businessmen stuff they don't need.

But for just medium-sized ad budgets and up, there is some serious metering going on (as with any other non-trivial investment). And as someone occasionally working in the metering end of a pretty big advertising budget, I can guarantee you that either (a) all 10.000 persons that respond positively to advertising are currently customers of the company I work for, or (b) advertising works.

Personally, I'd be happy to just pay a couple bucks per show, or a penny per search, or whatever. I'd have cancelled my cable TV long ago if it weren't for my PVR.

You can't (unfortunately) expect big companies to respond so quickly to so radical markettrends. But look at iTunes. No-one believed that digital-only distrubution of music over the internet would ever work just a few years ago, and now we've even got non-DRM files. A similar model will emerge for TV shows any time now.
Oh, and I'm more than happy to look at google's sponsored links. Even if there was a paid model (which there is, look at the APIs), I wouldn't take it. I often search for products and services, and I often find what I'm looking for in the sponsored links section.

So, compete with them

[Colin Smith]

Join Majestic 12 and contribute to an alternative search engine. You can have your machines index a certain amount per day and contribute the result to the index.

Having alternatives is what keeps companies honest. Government regulation just makes the regulators a target to be corrupted.

Re:How long can it last?

[jamiethehutt]

Personally, I'd be happy to just pay a couple bucks per show, or a penny per search, or whatever.

In the last 14 months, according to Google Web History, I've done 6707 searches. If that was at a penny a search I'd not be pleased...

Double click was a bad move

[Anonymous Coward]

All the people I know that work at google are smart folks. All the people I know that work at doubleclick are complete dumbasses, I'm dead serious.

If only they had gone after Microsoft this fast

[VGfort]

Interesting the speed of which they claim Google to be a monopoly.