Just to clarify: all four examples use a fixed buffer that's constantly recycled to "go back in time". Yes, it's kind of cool. No, it's not worth wasting a judge's time over.
It IS obvious and trivial. Quick quiz, how many people tried to do the following before TiVo came along:
o You have 50 years of stock data on disk. You want to calculate a 5-year moving average as efficiently as possible (particularly, minimizing disk IO).
o You have a security camera. You have a fixed number of video tapes. If you discover at some point that someone has stolen expensive electronics from your store, you want to go back as far as possible to try see who stole it.
o You want to write/usr/bin/tail, and you want to pipe things to it.
Yeah. It's a basic computer science concept. Security guards and programmers have implemented this thousands of times before.
Yeah, and the very large majority of today's TV sets still don't use much of the resolution that DVD's have, but you don't see anybody bitching about that. If HDFM catches on, it'll be a nice feature eventually, and good future-proofing now.
For a real example of this, see the XP SP2 discussion, and witness all the posts complaining that 95% of the world is stuck on a browser that has piss-poor standards support, and there's very little we can do about it. Yes, we have a better alternative. No, sheep^H^H^H^H^Hpeople aren't going to switch to it in droves, for whatever reason. It's partially consumers' fault, but microsoft is also complicit in holding everyone back.
I'm betting the accountants at google wouldn't be able to justify that a scant few users are taking up 10,000 times as much CPU/IO as normal users because they really need to, and a multitude of other users using 10,000 times as much just because they think it's cool but they don't really understand or need its power. Not to mention that google would have to spend a ton of development costs just to get to that point, to where regexps are optimized as much as possible and aren't starving resources from normal users and are working to the uptime and quality standards of google. R&D costs which are being used by <1% of users.
Granted, end-user searching may not be where google is making its money. If google is making a lot of its money from corporate search-appliances, and business users would pay for optimized regexp functionality (and the extra CPU and hard drive space to support it), then maybe there's a business justification there. I don't know. But all the intranet search engines I've used were unbearably slow even when they used dictionary-lookup methods.
From what I understand, the reason that google can do many many searches at once and still complete each in 0.5 seconds (besides having a huge linux farm) is that they make a lot of algorithmic shortcuts and precompute datastructures as much as possible. There really aren't any such precomputed algorithmic shortcuts to take with regular expressions, so searches would either be much much slower, or google would need to buy a vastly larger linux farm, for a feature that's used by less than 1% of the population.
20kb as an average size for email?? No way, most of mine are 0.3 - 1.0 kB. Currently a lot of spam (at least the stuff I'm seeing) is HTML only (no plaintext multipart alternative), presumably to save the spammers money, since bandwidth is the only thing they DO pay for (and lately with hacked cable modem machines, they don't even have to pay for that, the pissers...).
Does anyone know if Yahoo is doing something similar? My mom can't email my sister's yahoo account unless she sends the email as plain text, no HTML multipart alternative. There's no bounce, no spambox that my sister can check, just poof gone.
As far as Acrobat/Eudora/GSView/MikTex goes, where I work, 99% of people use Outlook for messaging, and far far too many finalized documents are emailed around as Word/Excel/Powerpoint files.
Microsoft doesn't have to or care to get into the text-terminal emulation business, they have NetMeeting and XP's RemoteDesktop.
Windows Media is used by a fair number of people, but yeah, a lot of normal people still use Winamp. Though microsoft always needs a couple tries before are able to dominate a market.
Microsoft doesn't have an answer to Photoshop, but that could easily change at any point.
And the mathematical stuff isn't used by a ton of people, so you could similarly ask why Microsoft doesn't have great MIDI sequencing or circuit layout tools, but microsoft is more interested in software that further their goal of world domination. Or they don't want to get into niche tools, or, I dunno.:)
But yes, it's less of a one-player-dominates-all game... On the alien side (where resources are evenly distributed across all players and each must decide which structures they think would best help either them personally or the team as a whole), since resources are spread evenly, and resources are key to getting good upgrades and thereby winning the game, it's not enough that one player spend his resources wisely... the team only wins if the team's average skill is higher than the other team's. (this seems to be true for the marines as well, but it's not obvious to me why, as only one player on the marine team makes decisions regarding resources)
On the other hand, this shifts the the game quite a bit. Instead of becoming the best individual player you can, the key to ensuring victory is to remember which players on a particular server are better, and to join the team at the begining that contains the higher average skill.
One very good feature of web forums is that they're all implemented a little differently. If not different code packages, at least different HTML templates.
Why is this good? Because there's not a single interface that scummy spammers can code to. If they want to spam one forum, fine, but there's 1800 other ones out there that are relatively spam free. Please leave web forums alone as they are.
The most important thing about debian isn't necessarily that apt is cool... it's that the package managers put a lot of work into producing and making sure only good packages get accepted. Like you said, the packages aren't hostile to people making changes to conf files and they try to provide as much documentation as possible about how they've set things up. Apt-on-redhat won't fix that unless you pull in all of the packages from debian package managers, at which point you've got debian anyway.
As for installing a good system with reasonable defaults... this might not be quite as well known, but try this 1) burn knoppix, 2) boot up into knoppix (notice that your hardware is autodetected just like redhat/suse/whatever), 3) run knx-hdinstall, 4) click through 19 simple dialogs, and voila, you have debian installed on your hard drive with hardware detected and tons of reasonable defaults picked. The only place it's semi-lacking i that it doesn't have a TON of things installed (but it does have quite a lot as anyone who's played with knoppix can tell you) since it's just once CD for god's sake (eg. it's missing tcsh), but as stated, it's trivial to do "apt-get install tcsh" to get whatever else you want.
Definitely, I can't wait. I just discovered the joy of knoppix's knx-hdinstall... it plops down debian-testing on your hard drive, with all your hardware autodetected. It was the easiest debian install I've ever done, and I've got apt-get, I couldn't be happier.
Re:Alternative ways of searching and spidering
on
Spidering Hacks
·
· Score: 1
Slashdot's policy on RSS is sort of a response to this problem (that it's far too easy for one person to eat up far too much bandwidth, even if they no longer pay attention to the site very much). But yeah, it's not terribly difficult for a website operator to look in their logs and see one dude snarfing way too much stuff way too regularly.
And like spam, the perp's natural first step in the battle is to start using anonymous proxies to help avoid detection / retribution.
LWP runs as part of perl, so it gives you a little easier control over the variety of options (eg. user agent and such). And it's easier to get working cross-platform (it's a bitch that you have to do extra work to get around the shell parsing of arguments to subprocesses on Win32). Also, you can do fancy asynchronous stuff with LWP, so you can have interactive programs, or stuff going in parallel, etc...
In general, most people use LWP, and if you write very many programs that use the web, you're going to want to go to LWP eventually, so you might as well start learning now (and there are easier interfaces to facilitate that too).
Re:Be sure to get LWP & Perl
on
Spidering Hacks
·
· Score: 1
Or, if you have minimal working knowledge of objects and modules, you can just read the lwpcook manpage. Yeah the O'Reilly LWP book goes into a little more, but look those modules up on search.cpan.org too, and buy the spidering book instead because it goes so much further.
Re:Spidering and exceeding ISP bandwidth limits
on
Spidering Hacks
·
· Score: 4, Insightful
If it's a full spider where you're considering competing with google or reimplementing google with extra features, then yes, you'd obviously need an industrial-strength account.
More likely though, you leave the big jobs to the big boys, and you want to do very specific things, maybe even building on top of google... eg. find porn movies, copying edmunds' database so you can sort cars by their power/weight ratio (or list all RWD cars, or find the lightest RWD car, or...), or make your own third-party feed of slashdot from their homepage since they watch you like a hawk when you download their.rss too often, but not when you download their homepage too often.
Little custom jobs like that can take a minimal amount of code (especially if you're a regex wizard), take minimal bandwidth, and take enough skill that target sites aren't likely to track you down because there's only three of you doing it.
Re:Ok I admit it
on
Spidering Hacks
·
· Score: 5, Informative
Is it open source? I wish there was more adult open-sores software. UBH running from cron is what I use currently to automate porn consumtion, but I'm sure there are tons of other opportunities....
Read the article, the badges are "passive" in that they only reflect radio waves sent to it. Also, the RF transmitters/sensors are placed only at entrances and pop machines, so attendees weren't tracked really closely, and apparently they can't sense much more than 20 feet away, making RF interference much less of a problem.
Then government representatives will be able to make political hay by showing up for photo-ops when the toolbooths are converted back for real-live-people mode. Illinois' governor recently did his toll-booth photo-op for a two-lane toll booth an hour outside Chicago.
(or they'd make political hay from mandating a no-evil-uses-with-EZPass policy, but this is Slashdot, so we all just assume a police state is inevitable, right?)
Big areas of the SVG specification where we're still lacking include clipping, filters and declarative animations.
You can see screenshots of what the patched Mozilla is capable of here. It can do basic drawing of shapes. However, without filters (eg. embossing, shadows, etc) or animation (eg. smoothly interpolate a color or shape from one state to another), much of the really sexy parts of SVG aren't available. And if you have a stock browser, none of SVG will be available until the code's good enough to bring in.
If splitting your resources is an obvious tactical mistake, then capitalism in general is doomed. We have, nay encourage, multiple companies who work independantly on the same problem (the more the duplication of effort, the better, so they tell me!). Not only that, but they're practically and legally encouraged and helped with government police/judiciary salaries to reduce the amount of cooperation beween them. Pure madness, I say.
Have open-source cores been available to implement those specs been available for a LONG time? Sounds like you're saying that just because we understand most of the MSWord file format, that means that we don't need StarOffice...
Slashdot Mirror
Just to clarify: all four examples use a fixed buffer that's constantly recycled to "go back in time". Yes, it's kind of cool. No, it's not worth wasting a judge's time over.
o You have 50 years of stock data on disk. You want to calculate a 5-year moving average as efficiently as possible (particularly, minimizing disk IO).
o You have a security camera. You have a fixed number of video tapes. If you discover at some point that someone has stolen expensive electronics from your store, you want to go back as far as possible to try see who stole it.
o You want to write /usr/bin/tail, and you want to pipe things to it.
Yeah. It's a basic computer science concept. Security guards and programmers have implemented this thousands of times before.
Yeah, and the very large majority of today's TV sets still don't use much of the resolution that DVD's have, but you don't see anybody bitching about that. If HDFM catches on, it'll be a nice feature eventually, and good future-proofing now.
For a real example of this, see the XP SP2 discussion, and witness all the posts complaining that 95% of the world is stuck on a browser that has piss-poor standards support, and there's very little we can do about it. Yes, we have a better alternative. No, sheep^H^H^H^H^Hpeople aren't going to switch to it in droves, for whatever reason. It's partially consumers' fault, but microsoft is also complicit in holding everyone back.
Granted, end-user searching may not be where google is making its money. If google is making a lot of its money from corporate search-appliances, and business users would pay for optimized regexp functionality (and the extra CPU and hard drive space to support it), then maybe there's a business justification there. I don't know. But all the intranet search engines I've used were unbearably slow even when they used dictionary-lookup methods.
From what I understand, the reason that google can do many many searches at once and still complete each in 0.5 seconds (besides having a huge linux farm) is that they make a lot of algorithmic shortcuts and precompute datastructures as much as possible. There really aren't any such precomputed algorithmic shortcuts to take with regular expressions, so searches would either be much much slower, or google would need to buy a vastly larger linux farm, for a feature that's used by less than 1% of the population.
20kb as an average size for email?? No way, most of mine are 0.3 - 1.0 kB. Currently a lot of spam (at least the stuff I'm seeing) is HTML only (no plaintext multipart alternative), presumably to save the spammers money, since bandwidth is the only thing they DO pay for (and lately with hacked cable modem machines, they don't even have to pay for that, the pissers...).
Does anyone know if Yahoo is doing something similar? My mom can't email my sister's yahoo account unless she sends the email as plain text, no HTML multipart alternative. There's no bounce, no spambox that my sister can check, just poof gone.
Microsoft doesn't have to or care to get into the text-terminal emulation business, they have NetMeeting and XP's RemoteDesktop.
Windows Media is used by a fair number of people, but yeah, a lot of normal people still use Winamp. Though microsoft always needs a couple tries before are able to dominate a market.
Microsoft doesn't have an answer to Photoshop, but that could easily change at any point.
And the mathematical stuff isn't used by a ton of people, so you could similarly ask why Microsoft doesn't have great MIDI sequencing or circuit layout tools, but microsoft is more interested in software that further their goal of world domination. Or they don't want to get into niche tools, or, I dunno. :)
But yes, it's less of a one-player-dominates-all game... On the alien side (where resources are evenly distributed across all players and each must decide which structures they think would best help either them personally or the team as a whole), since resources are spread evenly, and resources are key to getting good upgrades and thereby winning the game, it's not enough that one player spend his resources wisely... the team only wins if the team's average skill is higher than the other team's. (this seems to be true for the marines as well, but it's not obvious to me why, as only one player on the marine team makes decisions regarding resources)
On the other hand, this shifts the the game quite a bit. Instead of becoming the best individual player you can, the key to ensuring victory is to remember which players on a particular server are better, and to join the team at the begining that contains the higher average skill.
Why is this good? Because there's not a single interface that scummy spammers can code to. If they want to spam one forum, fine, but there's 1800 other ones out there that are relatively spam free. Please leave web forums alone as they are.
The most important thing about debian isn't necessarily that apt is cool... it's that the package managers put a lot of work into producing and making sure only good packages get accepted. Like you said, the packages aren't hostile to people making changes to conf files and they try to provide as much documentation as possible about how they've set things up. Apt-on-redhat won't fix that unless you pull in all of the packages from debian package managers, at which point you've got debian anyway.
As for installing a good system with reasonable defaults... this might not be quite as well known, but try this 1) burn knoppix, 2) boot up into knoppix (notice that your hardware is autodetected just like redhat/suse/whatever), 3) run knx-hdinstall, 4) click through 19 simple dialogs, and voila, you have debian installed on your hard drive with hardware detected and tons of reasonable defaults picked. The only place it's semi-lacking i that it doesn't have a TON of things installed (but it does have quite a lot as anyone who's played with knoppix can tell you) since it's just once CD for god's sake (eg. it's missing tcsh), but as stated, it's trivial to do "apt-get install tcsh" to get whatever else you want.
Definitely, I can't wait. I just discovered the joy of knoppix's knx-hdinstall... it plops down debian-testing on your hard drive, with all your hardware autodetected. It was the easiest debian install I've ever done, and I've got apt-get, I couldn't be happier.
Mod up! Fravia is most famous for his previous work, tons of hard-core documents about reverse-engineering. Anyway, anything he writes is likely Really Deep Stuff, and deserves to be pored over.
And like spam, the perp's natural first step in the battle is to start using anonymous proxies to help avoid detection / retribution.
In general, most people use LWP, and if you write very many programs that use the web, you're going to want to go to LWP eventually, so you might as well start learning now (and there are easier interfaces to facilitate that too).
Or, if you have minimal working knowledge of objects and modules, you can just read the lwpcook manpage. Yeah the O'Reilly LWP book goes into a little more, but look those modules up on search.cpan.org too, and buy the spidering book instead because it goes so much further.
More likely though, you leave the big jobs to the big boys, and you want to do very specific things, maybe even building on top of google... eg. find porn movies, copying edmunds' database so you can sort cars by their power/weight ratio (or list all RWD cars, or find the lightest RWD car, or...), or make your own third-party feed of slashdot from their homepage since they watch you like a hawk when you download their .rss too often, but not when you download their homepage too often.
Little custom jobs like that can take a minimal amount of code (especially if you're a regex wizard), take minimal bandwidth, and take enough skill that target sites aren't likely to track you down because there's only three of you doing it.
Is it open source? I wish there was more adult open-sores software. UBH running from cron is what I use currently to automate porn consumtion, but I'm sure there are tons of other opportunities....
Read the article, the badges are "passive" in that they only reflect radio waves sent to it. Also, the RF transmitters/sensors are placed only at entrances and pop machines, so attendees weren't tracked really closely, and apparently they can't sense much more than 20 feet away, making RF interference much less of a problem.
(or they'd make political hay from mandating a no-evil-uses-with-EZPass policy, but this is Slashdot, so we all just assume a police state is inevitable, right?)
You can see screenshots of what the patched Mozilla is capable of here. It can do basic drawing of shapes. However, without filters (eg. embossing, shadows, etc) or animation (eg. smoothly interpolate a color or shape from one state to another), much of the really sexy parts of SVG aren't available. And if you have a stock browser, none of SVG will be available until the code's good enough to bring in.
If splitting your resources is an obvious tactical mistake, then capitalism in general is doomed. We have, nay encourage, multiple companies who work independantly on the same problem (the more the duplication of effort, the better, so they tell me!). Not only that, but they're practically and legally encouraged and helped with government police/judiciary salaries to reduce the amount of cooperation beween them. Pure madness, I say.
Looks like you're right, it's GPL'd and such. Wish I had points to mod you up....
Have open-source cores been available to implement those specs been available for a LONG time? Sounds like you're saying that just because we understand most of the MSWord file format, that means that we don't need StarOffice...