Slashdot Mirror
Twitter Not Rocket Science, but Still a Work in Progress
While it may not be rocket science, the Twitter team has been making a concerted effort to effect better communication with their community at large. Recently they were set-upon by a barrage of technical and related questions and the resulting answers are actually somewhat interesting. "Before we share our answers, it's important to note one very big piece of information: We are currently taking a new approach to the way Twitter functions technically with the help of a recently enhanced staff of amazing systems engineers formerly of Google, IBM, and other high-profile technology companies added to our core team. Our answers below refer to how Twitter has worked historically--we know it is not correct and we're changing that."
Man, who *IS* this guy that his trolling has gotten its own front page article?
Answers on a postcard please.
Deleted
Making 10 slashdot accounts is not rocket science.
Scaling. *snicker*
...social networking sites affect any hope of better communication.
Oh well, as long as people will doggy-paddle the slough of mediocrity, less competition for me.
This is one of those cases where "effect" is a verb.
please tag article "whogivesashit"
uh... "kind of scary" (to finish my sentence)
Kinda surprised. I use neither Twitter nor Ruby, but seems like the talk has been than Rails wasn't scaling well. Thought the Q&A would confirm or deny this.
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
I wonder what they mean by "elegant filesystem-based approach"? Maybe their going to treat tweets like an email and store it all in the filesystem rather than a database? There's certainly some proven extremely high volume email servers so you know that method scales.
I wonder what the disadvantages of setting up a front end to an email system and covert incoming tweets to actual an actual email is. On the retrieval side you just read the mailbox and convert back to the tweet format then send them on to the destination.
I came to the datacenter drunk with a fake ID, don't you want to be just like me?
You know, I wonder how hard it would be to do a Twitter clone on Google's App Engine. It seems like it would be the perfect fit: relatively simple application that needs to be massively scalable.
Grammar can be fun!
Hiring folks who used to work at IBM or Google is not the same thing as "large companies control[ling] how Twitter works." Some day, you'll have a job and you'll understand that. [Sorry to be an asshole about this, but your comment just shouts "teenage kid who's never had a serious job."] People with experience with large-scale applications may already know solutions to some of the problems Twitter is seeing. Those solutions aren't always in the text books; and if they were trivial and obvious, then such applications would be much more common.
...twitter blog is hosted on blogger (Google), and this morning it was out of service.
how long until
and I fail to see what's so special about it. I have plenty of other methods to keep in touch with those I care to keep in the know.
...the Twitter team has been making a concerted effort to affect better communication...From what I've heard the only affect twitter's had on communication isn't one worth bragging about.
We just hired a bunch of guys who couldn't cut it at IBM/Google/other large successful companies. We're trusting them to completely overhaul our system because they must have been so smart to have worked there.
"I would much rather have some no-name 'genius' from a basement come and work for them, than people who could (possibly) be 'tainted' by how a large coproration works and handles things, and what friends, they may have (who married into, etc)"
Would you choose Anthony Anderson or Kevin Smith?
Well, I didn't say it was the same, just that it is a possibility.
And am aware that people who formerly worked at places such as Google (especially) and IBM would most likely have a better understanding of how a large-scale system like this works.
But, Google itself (along with a lot of other major internet/software/hardware companies) were started in a 'grass roots'/garage/basement method.
It seems to me that more often than not, bringing in the help of (formerly employees of) large corporations tends to conform them to that previous corporation, even if its not intentional.
Not that it doesnt happen in smaller ones, "back when I used to work for [...], we did it this way..."
To argue against myself on your behalf though, having someone from a larger corporation, can be benificial aswell, because they may have been fired, or quit because the larger corporation wasn't listening to them, or was doing something in a way that went against their ideology, thereby possibly preventing said smaller company from becomming similar to the larger one.
My main concern is that it will lean towards the former though, rather than the latter.
It seems to be meant to suggest that the article's use of "affect" is incorrect. Surely this is mistaken. If suggesting that twitter has anything to do with better communication isn't an affectation, I don't know what is.
"No offence to the people at Twitter, and maybe not the people who recently migrated there. But am I alone in being hesitant to welcome people who formerly worked at IBM, et al, and even Google?
Granted, they seem to be more directed at the physical performance of the system, but I see this as somewhat of an inroad to Twitter actually working, thereby requiring me to acknowledge its existence. And I would much rather have some no-name website, where people living in their mother's basements come to tell a world that would rather ignore them that they just had a shit, go under, than sap the life-blood of a decent technology company.
I have no idea what Twitter could be useful for, but I just find having big boys come in and having control over something that's (potentially, and already has been) so useless, uh...kind of boring."
Honestly though, what? They hired some people from real technology companies who might have the first clue of how to do what they should already have been doing. I have only the vaguest idea of what Twitter is, based on this and having checked it out, have had my suspicions confirmed. Call me a curmudgeon, but they just sound like they're trying desperately to survive in a marketplace already brimming with (better) competition by hiring experienced technologists. Seriously, how long before 'phones that can handle Facebook are the norm?
To marry? probably Kevin Smith. :P he's kinda sexy when he's quiet.
If you mean, someone who has experience directing movies, as apposed to someone who (seems) to only have experience being directed, and probably has "lots of ideas" on how to direct.
I'd be willing to take advice from either with an equal amount of salt.
Unless of course I was planning on directing, or writing a movie similar to one of Kevin Smiths, enwich case, the answer is obvious.
However, as for "working with", I would probably be more comfortable working with Anthony, given that we would both be "sort new" to the idea (directing, making our own movie, etc)
If you meant something else, well, enlighten me.
Fargin' hilarious. Twitter seems to me to be absolutely February 2001, just milliseconds before the crash stupid.
Dog is my co-pilot.
Talk about sites the US Gov should shut down. Facebook and MySpace are up there too.
No sig for you!!
Like the AC said, I think you're wildly exaggerating how ideological workplaces are, particularly from the point of view of a server monkey.
What I'm listening to now on Pandora...
Plurk has been gaining popularity in the past 24 hours, and it's handling scalability rather well so far (after having been mentioned by Leo Laporte, Robert Scoble, TechCrunch, and others). I'm very curious to see how well it would hold up if it had the same number of users as Twitter, though.
the JoshMeister on Security
because your twitter conversations are so secure right now
"Thanks for all the money you paid to us. We've used it to buy off ISO among other things" -Microsoft
They both played basement dwelling 'geniuses' in two abysmal movies last summer, Die Hard 4 and Transformers.
And the correct answer however is actually Laslo Holyfeld
This confused me too, but I check dictionary.com.
Effect is used as a verb when there's no direct object. When used as a verb you can replace it with "bring about" and see if it makes sense. So in this case, "The Twitter team has been making a concerted effort to bring about better communication with their community at large." So, effect is the right spelling.
Using affect as a verb here, would suggest that the better communication is being acted on, which it isn't. Well... that's not the intention of the author's words anyway.
but who would quit Google to work for Twitter???
You may find my appearance and demeanor foolish, but it is you who plays the fool.
I'm a twitter-shitter!
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
But is the twit named after the service, or is the service named after the twit?
Why would they try to "affect" better communication when they could actually put it into effect instead?
WHOOSH
Ruby is not Twitter's problem. The algorithm Twitter uses is the problem.
When I started irately writing this post, I wrote it in a tone that would have gotten me modded into oblivion. But then I realized that ignorance, not idiocy, drives the particular myth I'm debunking. let me educate, not flame, those of you who haven't formally studied computer science.
It's become fashionable to blame Ruby for Twitter's problems, but that's wrong. The particular choice of language doesn't matter a bit when you talk about scalability, no matter what the language or the problem.
First, sending a twitter message is an algorithm. An algorithm is just a recipe for doing something to some data. Although most computer science literature deals with more abstract and general algorithms, like those for sorting and searching, the same principles applies to even the most mundane processes, like what rm foo does to a file system, or how a database engine runs an INSERT.
One way we can talk about algorithms is to use something called Big-O Notion, which describes the relationship between how much stuff an algorithm processes and how long it takes to run.*
It's easier to see things with examples. Say we have an algorithm and we give it three sets of data, D1 and D2, and D3, each twice as large as the last, so that D2 is twice as large as D1, and D3 is four times as large as D1.
If we call the algorithm O(1), it will take the same amount of time to process D1, D2, and D3. If we instead say it's O(N), D2 will take twice as long to run as D1, and D3 will take four times as long.
If N represents the number of users for a web application, and we want to double N, twice as many users, we'd need twice as many web servers if the bottleneck algorithms are O(N). If the database is the bottleneck, we'd need a twice-beefier database server, or some partitioning.
Things start to get interesting with O(N^2). In that case, D2 takes four times longer to run than D1, and D3 takes four times longer than D2, which sixteen times longer than D1.
That means that if we want to support twice as many users, we need four times as many web servers, or more likely, a four-times beefier database server.
It can get a lot worse than O(N^2) too, especially if you're not paying attention to complexity. For example, many graph (think social networking) algorithms can easily become O(2^N), which is a lot worse than N to a constant power.
When you try to scale a poorly-designed algorithm (pretty much anything worse than O(N)), you start running out of cores, rack space, electricity, and atoms in the universe.
One useful bit about big-O notation is that it lets us ignore piddly details that don't matter. Say we had an O(2N) version of the O(N) algorithm. Sure, the O(2) algorithm might take twice as long to run, but it can still handle double the data with double the capacity or double the time. Even if it's O(10N), you don't start boiling the oceans to cool your data center when you want to increase your visit capacity a thousandfold.
This observation is why the choice of language doesn't matter. If a language implementation is slow, all it does is add a constant factor to any algorithms written in that language. A Python application might be ten times slower than one written in C, but its big-O complexity will be the same.
At the worst, that means you'll need ten times as many servers as with the C web application. The increase in development efficiency writing in Python (or Ruby on Rails, or Lisp, or anything else) might make the trade-off worth it. You can deal with a constant factor slowdown.
If on the other hand, you code a wicked fast implementation of an O(N^3) algorithm in C, no amount of hardware will save you. You'll hit a number of users beyond which your servers slow to a crawl and you lose blagosphereic karma. Even if you double your capacity, or buy a four-times-beefier database server, that
.... oh, the website.
Oh, ideology affects cage monkeys. The use of open source versus closed source, burchasing enough licenses for the software you use, and making '99.999%' uptime actually mean that instead of simply hiding downtime, and forcing people to spend more time documenting how much time they spent on a task than actually doing the task are all policies I've seen affect server work. Those may not be idologies per se, but they certainly arise from philosophies about how things should be done.
You mean with a direct object, right? When saying "Bob effects X", X is the direct object.
Instead of dismissing the OP with "WHOOSH", why don't you take the time to educate yourself about what he means? You can even take the radical step of asking a question! All your "WHOOSH" tells me is that you believe willful ignorance is acceptable. That attitude harms not only you, but society.
You, also, failed to get the joke the original poster was making. Allow me to explain: It seems to be meant to suggest that the article's use of "affect" is incorrect. Surely this is mistaken. If suggesting that twitter has anything to do with better communication isn't an affectation, I don't know what is. This is a joke. The person who wrote it is well aware of the difference between "affect" and "effect". He deliberately used the word "affectation" (which looks related to "affect" but really means conspicuously artificial or unnatural speech or conduct). The WHOOSH was not a dismissal of the response; the WHOOSH was intended to be the sound of the joke passing over the respondent's head.
Whether the joke is funny or not is another matter entirely.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
...was using Ruby on Rails"
I doubt they'll come out and say that. But look at how flaky twitter is (try using it for a while). The two biggest sites on the internet that are built on RoR - Penny Arcade and twitter - are flaky as hell. I've met the PA coder, and I'm not willing to believe the twitter guys are incompetent, so it's obviously not their fault.
RoR is known for obscenely high resource usage. I can't believe it'd be a good choice for large sites like twitter, long term.
There's an old saying that says pretty much whatever you want it to.
bottom line... move from Rails to Java and continue to work on real-time social networking... all will be well.
Psst, there's actually no "Twitter team." It's just one guy with like ten accounts.
DRM: Terminator crops for your mind!
For Pete's sake, learn how to use commas. Reading that penultimate paragraph makes me think of William Shatner.
At the bottom of the
Who says Twitter isn't rocket science?
(T>t && O(n)--) == sqrt(666)
No in that case you would say, "Bob affects X". Affect takes a direct object. Effect does not.
All transitive verbs take direct objects. If "effect" were intransitive, the sentence "Bob effects" would be correct, but it is clearly ungrammatical. "Run" is an intransitive verb. "Bob runs" is a correct sentence. Since "effect" is a transitive verb, it must therefore take a direct object.
Also, try replacing "effect" with "create". "Bob creates X" is correct as well.
You're right. I stand corrected.
How much of a tard do you have to be to build a very-short-message spamming system that won't scale?
The mind boggles, clearly this is what happens when things like RoR make things 'easy' and lead to dumb-assed trendy kids with "Extensive HTML Experience!" on their resumes writing garbage that accidentally gets popular.