Slashdot Mirror
Craigslist Blocks Yahoo Pipes
Romy Maxwell posted a blog piece on Craigslist apparently shutting off access to Yahoo Pipes. Maxwell was working on a project, one of 2,111 using Craigslist as a data source, for a (non-commercial) Pipes-based mashup. He sent Craig Newmark an invitation to the alpha test, after a few rounds of friendly communication — "...as a rule of thumb, okay to use RSS feeds for noncommercial purposes." The apparent response, 4 days later, was for Craigslist to redirect any request with an HTTP referrer of pipes.yahoo.com to the Craigslist home page. Maxwell writes: "It's a sad day for me. I'm not too upset about my own project, as Flippity was already removing Craigslist as a data source. With the likes of eBay and Oodle not only providing open APIs but encouraging and rewarding developers, spending my time wrestling with Craigslist is just plain stupid and exhausting. I'm sure I'm not the only person to have come to that conclusion, and I wish it were different. ... If Craigslist wants to keep its doors shut to the world, so be it."
here
I understand that Craigslist doesn't want to go out of it's way to make it's website more elaborate, (In fact, I appreciate it) but I don't understand what purpose it serves to prevent others from adding their own features to the site. (In the same way greasemonkey is so great) I wonder what they are trying to do with this move.
Craigslist want to make it moderately difficult to quickly access its listings for more than one location at a time. As soon as it becomes super easy to access listings and perform more powerful searches, then the spammers and corporations will move in and make craigslist into what ebay has become in recent years. I personally want craigslist to stay just how it is, and so I support any attempt to block access for silly things like Yahoo Pipes.
Old wine in new bottles seems to be the constant theme of the computer business. We are always redefining old ideas with new monikers and names as if something drastic has changed. It's a sucker's game.
For example, the so-called Web 2.0 revolution is essentially a rewording of things that were going on in 1998, an era now called Web 1.0. I'm reminded of this only because I attended a social networking meetup (also called a meeting or gathering) and realized that all the buzz over social networking is really nothing new. You can read book after book about the social networking revolution and soon realize that these books are not much different than generalized "how to do marketing" books that floated around in the 1960s. The rules, the philosophies, the ideas are all old but re-jiggered to fit into the social networking meme.
This is the way the computer scene operates. Everything is gussied up to look hip and new when it's really putting lipstick on a pig. When all is said and done, the computer is good for a limited number of uses. These include calculations, entertainment, information retrieval, image manipulation, and word processing. That's it. Everything is a subset of those Big Five.
But when you boil computing down to five basic mechanisms, you have to constantly jazz up the categories with new terms. Word processing evolves into desktop publishing or blogging or content management, for example. It's all variations on the theme.
In the early days I would generalize about these same Big Five using early terminology. Back then, before it was actually boiled down, only "word processing" remained as a constant insofar as a naming convention is concerned. "Entertainment" was always referred to as "gaming." "Information retrieval" was "database management." "Calculations" were always "spreadsheets." There was no image manipulation in any serious way until the invention of Photoshop, and that was the last brick in the wall.
So if we are going to really boil down computers and try and project the future, it turns out to be rather simple. They get faster and faster and faster but not really any more useful (except for the fact that they are faster). This basic idea has been lost in the "there's an app for that" world of confused Web 2.0 jargon and the Intel Atom chip. The industry as a whole is losing its way. Each new development fails to increase performance Performance is the only thing important to the basic computer. All improvements such as newer and slicker versions of Photoshop, for example, require higher and higher performance machines. This holds true for networks and everything else. As performance increases things become more practical and easier to use. So where is the performance?
Part of the problem stems from the emergence of cheapskate computing. Getting the cheapest machine you can find that will manage to do the job--meaning it will boot an OS and actually run some sluggish apps.
When desktop computing got its start a good machine cost about $3,500, and to keep up with the technology you generally bought a machine every year or two and typically spent between $2,500 to $3,500 until the prices started to erode. By the time of the dot-com crash in 2000 a typical rig was selling for $1,500. Now its' gotten to the point where the median price is hovering around $800 and usable machines can be had for $400.
Instead of using Moore's Law to make machines more powerful, the "make them cheap" switch has been thrown and now everyone has a cheap machine in one form or another. The problem with cheap computing is that it's really not exciting. Moore's Law can affect performance, price and size. Size is the other direction the industry is going with the iPhone computing platform. This is another move away from the performance direction.
The trend, unfortunately, is not going to change. Once people get into cheap and small they seldom return to extravagance. So what do they do? They turn to old wine in new bottles. We'll just keep changing the name for everyth
scraping other websites' content over http is generally a huge waste of resources (and money) for that websites' operator, so unless you can give him something of considerable value in return (like Google does - I'll gladly serve 4 million pages/day to their bots if I get 200k visitors through Google in the same time, visiting my website and not just looking at my content somewhere else), be prepared to get locked out. Naturally, something you consider "a cool feature" isn't necessarily the sites' owner's idea of sufficient compensation. Perhaps some day ISPs will pay websites for the traffic and bill their clients for it, then websites might react differently.
"I love my job, but I hate talking to people like you" (Freddie Mercury)
i hope all these link agregators die. good stuff.
If you mod me down, I will become more powerful than you can imagine....
Craigslist is basically run as a public service. They are well within their rights to block something that increases their bandwidth costs and has no benefit for them. Heck, the way the project was described, I'm not sure it had benefits for anyone!
(-1, Raw and Uncut is the only way to read)
Craigslist are doing fine without you, me and yahoo.
http://michaelsmith.id.au
In breaking news!! A new search website has been released called "Google"
The owner doesn't give a shit, it works.
In my opinion, the executive summary is that Craig Newmark values his notion of small, local communities more highly than he values money. I mean it in as cool and non-bleeding-heart a manner as possible.
He has the ability to direct the flow of visitors to his site to make money, or he has the ability to encourage what he sees as small, local communities basically unconnected to one another. He uses his site for the latter, and consequently forgoes substantial amounts of income. Sites that aggregate content or otherwise amalgamate the disconnected communities run afoul of his personal and, perhaps, business preferences.
Blocking some irresponsible Yahoo's pipes is the only way to stop it from reproducing.
Every mans' island needs an ocean; choose your ocean carefully.
So this means that Craigslist has plugged up the portion of the Intertubes belonging to Yahoo? Sounds like lawsuit material!
Warning: this article may contain humor, sarcasm, parody, and perhaps even irony. Read at your own risk.
I would make the case that Craigslist makes money, rather than foregos it, because it does that.
Indeed, FTA:
In all the complaints and requests we get from users, this is never one of them. Time spent on the site, the number of people who post--we're the leader. It could be we're doing one or two things right.
From their CEO.
They have 30 employees. 30.
Whomever has dicked up Slashdot's UI could learn a thing or two by browsing Craigslist after reading the above quote.
"Sacrifice for the good of The State" - The State
While I can't comment on the logic behind the actions documented here, I can definitely say a word or two on what I believe to be the end of Craigslist's usefulness (at least for me).
About two years ago, I used Craigslist for everything. From iPhone purchases to small free stuff in my neighborhood (and others), Craigslist did it all. I even used its Personals section, which I actually had some success with (NO, not the NSA area...get your head out of there!).
Nowadays, every time I try to use Craigslist for those same purposes, I leave utterly disappointed. Almost every search I've run on the site has returned 95% SPAM. It's ridiculous that I can't trust a single entry because spam on there has gotten clever enough to resemble real listings. If you're even thinking of finding a mate on there, don't; it's a cesspool of fakes and cheap prostitutes. If I've left Craigslist for that reason, so has many other people, which means that it gets more noise, less hits.
I understand that the service is free, but let's put things in perspective. This very site sees ridiculously high traffic on a daily basis, yet does a very good job at moderating spam postings on EVERY discussion. We get dupes and stupidity, sure, but not (that much) spam.
Kind of sad, really. I shouldn't have to use eBay to buy something from a seller 5 miles away and hope that he's cool with local pickup...
(BTW: That project is awesome.)
Comment removed based on user account deletion
I've moved a lot the last few years. I find personally that craigslist was mostly genuine in areas with a lot of smaller towns, whereas in larger metro areas, it's generally loaded with spam and scams.
That being said, it's definitely easier to find yourself a happy ending massage parlor in the bigger areas.
Google and MS are businesses that do far more than what Craig's list does. Even just with the software development, it takes far more than 30 people to do that at either company. Let alone the other work of promoting the products and doing the accounting. And even with the staff that MS has the company is probably, if anything, understaffed for the real needs. Had they had far more people working on developing software they'd probably have released there first 32bit only OS a couple years earlier.
But Craig is actually leveraging tech in the appropriate way if the goal is to do as much as possible with as little as possible.
That's the thing so many companies just don't get. They feel they need big teams. They think they need to spend this or that and have Flash and Flex and AJAX and all sorts of stuff. But sometimes simplicity, lightweight, and a small team can do amazing things.
Mashups are mostly cool because they look hard to implement. You look at them and think oh wow that's cool it looks like a desktop. Maybe you play with one for a few hours before forgetting about it. They're one of many things that seem like impressive technical achievements- but that no one really asked for- and that people tire of quickly.
The best example that comes to mind is that voice recognition feature that works over the phone. At first, you hear, "Press 1, or say... yes!" and you decide to say "yes" and it understands you. So you think oh wow that's cool... it can recognize my voice, how impressive. But really, you've seen enough, and afterward you always use touch tones as before. Until... voice recognition infects all the phone tree systems you use like a plague, and the touch tone option disappears! "The serial number you entered is XXXXX. Is that the correct number? Please say 'yes' for yes, 'no' for no", or... 'I'm not sure' if you're not sure!" Now since I don't want the guy taking a dump in the next stall to hear me say "yes" and find out that I'm in there on a cellphone (ew) I try to get away with what worked in the good old days... pressing 1. "Sorry, I did not understand your answer. Please say 'yes' for yes, 'no' for no, or... 'I'm not sure' if you're not sure!" ARRGH... People who are forced to undergo credit counseling under the terms of the recent bankruptcy reform law are put on the phone with these things, and have to sit there like idiots all day talking to them. It sounds like hell. Many cellphones also have voice recognition- you set it up so you can say "Mary" and it calls Mary and you think oh wow that's cool... but have you ever seen anyone use it? Me either. Mary has a bad reputation- I don't want you to know I'm calling her.
Drag and drop is another one. I can see drag and drop is useful in some situations. You drag a file to a folder, an icon to another window to open it there, etc. So everybody has to implement drag and drop everywhere, whether it makes sense or not, even though nobody outside a feature design meeting has ever asked for it. I have never wanted to drag and drop anything on any "web 2.0" site. But a lot of times my finger clicks the mouse by accident, and I find myself dragging a mouse pointer around that's pregnant with some strange little icon dragged from who knows where. I usually keep dragging it across the screen until I see it turn into that little "no not here" thingy and then I let go. Unless I'm too slow to catch it, and I get stuck trying to figure out WTF I just did and how I can undo it to get things back to the way they were.
Touchscreens- that's another example. When you first use one, you think, oh wow that's cool. Then you put your oily fingers all over it until all you see is a mash of filthy fingerprints dimly lit from underneath... and suddenly you realize those horrible little thumb keyboards weren't so bad after all.
Google and MS are businesses that do far more than what Craig's list does.
Well, ok, but you have to admit that single-handedly destroying the newspaper business model is pretty impressive. Not too many people can say they've collapsed an industry.
Literalism isn't a form of humor, it's you being irritating.
They're not making nearly the same revenue per employee, though, so there seem to be some diminishing returns. Craigslist brings in somewhere around $6 million per employee, while Microsoft brings in about $600,000, and Google about $1.1 million.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
The guy just has to do a scrape of the cache from the results of a google query of "site:craigslist.org" - it returns results from iowa, hawaii, san francisco, manila, singapore ...
Or scrape each cached result of a query based on each geographical area: "site:kansascity.craigslist.org", "site:losangeles.craigslist.org", etc.
Never need to hit the actual craigslist domain at all.
Craigslist is doing fine. There's only one!
:)
(/grammar police)
Last time I tried to use Yahoo Pipes (on InstantWatcher.com), I couldn't build a pipe because Yahoo obeyed the robots.txt file. Redirecting based on referer seems like overkill when they can just change their robots.txt.
http://en.wikipedia.org/wiki/English_plural#Discretionary_plurals
(/grammar police police)
How can I believe you when you tell me what I don't want to hear?
30 sounds about right. I'm always flabbergasted to hear how many employees Google, or Microsoft have.
Estimated revenue of Craigslist in 2007 was $150 million, and mere $25 million in 2006. Net income of Microsoft in 2009 - the worst year ever, IIRC - was $14 billion. Net income of Google in 2008 was $4.2 billion.
(All figures are taken from Wikipedia)
Ultimately I just wrote my own setup that worked very much like Yahoo Pipes, but without the GUI to configure things (I just wrote perl code to do what I wanted) and it also did caching of the RSS feeds for a while and if there was an error it would simply work with the cached data rather than failing. Took a while to get right, but now that I have it working properly, I love it.
Craigslist leverages the Internet to provide a hell of a lot of service to a hell of a lot of people without doing much work at all. They skim a little money from some of those people and say that's enough.
Microsoft creates a lot of work for themselves by making lots of new features and then convincing people that they need them. It's how they leverage their advantage as the world's largest software company, and the rest of the industry (and lots of people doing OSS) fall for it.
Google is probably pretty much the same these days. The point is that these companies are worried about shareholder value first, they're worried about winning. That's why they make all this work for themselves. Craigslist just provides the service. Take it or leave it.
I use cellphone voice recognition all the time with my Droid. "call " "navigate to " or just "website name" when I don't want to be bothered to type in an address. real nice stuff.
-The world would be a better place if everyone had a hoverboard
Not a mess. Just happens to be a little idiosyncratic - two years ago I tried to offer a complete web2.0y rebuild of their site for free and they refused! http://zwadia.com/
normally i hate when people say this but as an avid user of craigslist to buy/sell things without fees I couldn't agree with you, and the overall ethos of craigslist, more and wish I had mod points....
But Microsoft has 93,000 employees and $58 billion in revenue, and Google has 20,000 employees and $22 billion in revenue (I'm quoting revenue, seeing as wages come out of revenue, not out of profit).
So Craigslist pulls in $4,687,500 per employee, Microsoft $623,655 per employee and Google $1,100,000 per employee.
Don't forget that Craigslist likely has the lowest R&D costs and investment costs out of any of the three.
Maybe they went to your site without javascript enabled and weren't impressed with your "web2.0" skills?
Hint: keyword spamming is pathetic. Totally failing your site layout because of keyword spamming is just hilarious.
small, local communities basically unconnected to one another
You make it sound so wholesome. But it's really small, local communities of hookers, real-estate agents, and those looking for unusual, risky sex.
For Microsoft, their income mostly unrelated to what their tech employees are doing. Just look at what's been happening from Windows NT to Windows 7--mostly theming. Microsoft's success comes mostly from marketing and business "strategy" (aka questionable practices).
I see it's time to add a new moderation category: SPAM!
I see even classic Slashdot is now pretty much unusable on dial up anymore.
it's a wordpress site with nothing special done to it... quit whining I didn't even have this blog when we were exploring this.
I laughed the other day when a (much younger) workmate complained about automatically generated mails he had to handle: Thunderbird was very unresponsive when he opened those 100KB (~1700 lines) pure ASCII e-mails. The reason was apparently some badly installed/configured "Internet security" app, but it was hilarious to see him not find it unusual that his modern PC could not handle such text files and asking not have to work with them, when our 50-100 times slower PCs were handling them fine ~15 years ago. Perhaps the sluggishness of the Web in general has lowered people's expectations regarding the performance of PCs (esp. with web-based AJAX apps that try to provide faster deskop apps' functionality).
"I love my job, but I hate talking to people like you" (Freddie Mercury)
It's still early in the day for me, but so far your post is the most stupid and idiotic piece of shit that I've read.
Congratulations, or something.
Craigslist wants to disable mashups? Their prerogative.
Not really. They can try by banning user agents, but you can get around that easily.
As long as they give you their data they can't control how you use it.
Time spent on the site, the number of people who post--we're the leader.
You know who else has a ton of comments? Youtube.
Craigslist is full of inarticulate twats and the flat message mode makes any serious discussion nearly impossible.
Also annoying is the amount of censorship (This comment has been removed by moderators...) At least Slashdot (mostly) just hides stuff from casual view.
we're the leader. It could be we're doing one or two things right.
It's like they stopped webdev in '96 or so. If they're successful, it's inertia.
I set up my e-mails to my boss and boss's boss in 2 sections, executive summary then detailed information. I pretty much know the things that they want but give the additional detail in case the need more info and I'm not present to answer questions.
Have you ever read something and thought "get to the point already"? When directors and above is wading through tons of e-mails executive summaries help move the process along faster.
open source sub sim. I might start coding again for this. http://dangerdeep.sourceforge.net/contribute/
That's why they don't allow browsing by images: bandwidth. They're providing the best service possible given the constraints they've adopted, and I happen to hope they don't change.
Are you trolling? I use all those features and I know many people who use at least one or two of them, and like them.
New low for anonymous cowards!
Best Slashdot Co
Insightful my ass, if you believe the most differences between windows NT and 7 come from the themes its time to pull your fucking head out of the sand.
Is that so shocking?
From TFA:
Craigslist's popularity and sheer functionality makes it an Internet and American national treasure and it will be a sad day when the company finally folds due to lacking interest from within.
Support my political activism on Patreon.
Microsoft has collapsed a hell of a lot. They've hidden other stuff though (i.e. the Mainframe and Unix industry, beating strong but so many professionals clamor to scream how both are long dead).
Support my political activism on Patreon.
You mean suck dick without fees?
Support my political activism on Patreon.
You're probably one of the few people that actually understand what executives do and why they have multi-million-dollar salaries.
Support my political activism on Patreon.
Definitely a very shrewd business decision on Craigslist's part. If you don't take risk (i.e. build new things, hire more people) you can keeping making money as long as your goods/services are deemed worthwhile. The great thing is with just 30 employees, they can adapt pretty quickly to changing market conditions that affect there very focused product, whereas Microsoft, while not just an Operating System maker, has a much tougher time moving from one direction to another. It's just a fact of having that big a company, you end up having to cut thousands of jobs because you can't wait for the behemoth to adapt and keep the stockholders happy. However, it does come down to the motivating factors for those at the helm and in control of the money. If craigslist had investors that were demanding billions in growth, chances are they'd have to expand into the hundreds and eventually thousands of employees to succeed. It sounds like, though, that they don't have that pressure, and if an average of 4.6million per employee per year in revenue makes everyone enough money along with keeping the company running well, that is a pretty sweet deal. Their biggest risk is the lack of diversity in offerings. Basically all of that revenue comes from a single source. While no one is currently predicting the demand for this type of service dropping significantly, competition in that space could creep in quickly and cause them some major trouble.
Yeah, we'll post it as soon as an "executive" shows up
I don't care why you're posting AC
Comment removed based on user account deletion
Honestly? This is ridiculous. No, the MAIN changes are not themes. That's just apparently all you've noticed. I'm sure you're just being coy, but give me a break. Microsoft is imperfect enough on its own without people spreading complete nonsense. Give them credit when they earn it, for crying out loud.
"Those who would sacrifice essential liberties for a little temporary safety deserve neither liberty nor safety." - BenF
Guess I'll spending my day re-implementing Yahoo Pipes on my own server.
$ echo "ceci n'est pas une pipe" | sed -Ee 's/(eci n|pas )//g'
In any event, Offtopic does cover it.
Once you start adding fancy features, the site starts becoming responsible for the content. The atmosphere on craigslist is one that, if you get screwed over, it's entirely your own fault. On eBay, people can run to customer support if there's a scam. On craigslist it's such a basic site that it gives a real atmosphere of all responsibility being placed on the users.
It makes sense to me.
They'll probably be reunited once Yahoo posts something in Missed Connections.
Thing I hate about certain web pages. When I read long paragraphs of text (as I was doing before), I highlight the text that I'm reading. I also periodically reset the highlighting. It's a way to keep track of where I am in the text.
Well, some web sites have put it upon themselves to make the text clickable. So suddenly, when I get to a word that's clickable and I reset my highlighting, instead, I'm thrown into another web page, or some crap pops up asking me if I want to buy a product related to the word I just tried to highlight.
And even worse is the NY Times page, where it'll look up your highlighted text string in the dictionary if you click on it. I highlight some text, click on it to reset, and suddenly, I'm diverted to a page that says "y th" can't be found in their dictionary.
Now, I would like being able to look up a word I don't know with a right-click, and then selecting "look up" in the pop-up menu. And I'm sure when somebody was designing the offending feature, this was what they had in mind. But the way it is now is a hindrance more than help, and more often than now, I just stop reading, sometimes because I'm frustrated over the behavior, and sometimes because I'm now distracted and am suddenly interested by something other than the previous text.
"If a nation expects to be ignorant and free in a state of civilization, it expects what never was and never will be."
I'm the same way. More often than not, when websites do that, I stop reading and try never to come back to their site again. Other times I'll add what ever bit of javascript or whatever it is from the site. It's one reason why I hate snopes.com. You can't highlight their text. Who goes to the trouble of making text unhighlightable? I'll often highlight words or phrases right click and select 'Search Google for "$highlighted_words"'. Snopes makes that difficult. But the other day I was on a site (I can't remember which one right now) and tried highlighting some text to search more about it in Google and they actually had a pop up dialog box that said their content was copyrighted and they didn't allow copying. Out of spite I hit Ctrl-U, got the source for the page and the copied what I wanted while yelling out "Stop me from copying now!" My wife looked at me weird.
Stop Global Warming!
Just say no to irreversible processes!
Of course not. I wouldn't dream of violating the Agreement between Verizon and myself as defined in their brand new AUP.
A few years back, I was trying to use Yahoo Pipes on CL feeds, since CL's search sucks pretty badly. I quickly found that it was blocked.
So, I run a cron job that wgets the RSS feeds I want, twice a day, caching them on my server. Then I refer Yahoo Pipes to that cached version.
Works well for my needs, though obviously not for something as general as this.
they aren't in the same business though, so it's a flawed comparison. compare ebay and craigs and you might be more on the money
If you mod me down, I will become more powerful than you can imagine....
Go look up the word "hyperbole" in a dictionary. And while you're at it, look up the word "coy" as well, since you seem to have trouble with the English language.
Nevertheless, how much effort Microsoft has put in under the covers really doesn't matter, what matters is how much Windows has improved for users. And for most users, Windows really is still just a Start menu, Explorer, IE, and Office, plus the occasional crash, virus, and incomprehensible dialog box.