The problem is that while BBC got their own pipe in full from hosting provider, the consumer side of things (ISPs) was knowingly undersold and ISPs build flawed models that assumed bandwidth will not be used much, hence high contention ratios and they don't even have bandwidth for those. Initially they "solved" problem by kicking off heavy users, but with the advent of youtube and others it all went mainstream so they can't kick off their whole customer base. They might try though.
Disclaimer: I am a founder of a competing search engine concept that is based on volunteers running distributed software that crawls pages, conducts partial analysis and indexing before passing over results to central server that will index data into main central index: human aspect here is the people who take part in the project, and participants can actually change ranking formulaes, and shortly will be able to assist in human detection of spam etc.
Searching the Web is a very challenging problem (that's why few companies do it): volume of data is huge and one only appreciates value of good algorithms when faced with situation when poor algorithms make stuff run for weeks failing near the end and you have to restart the run to wait another week. You can either try to handle this very big problem, which is very hard even if you have the money (look at Amazon's A9 funded with millions, yet they licensed Google's code and database), or you can try to reduce the problem: only focus on a handful of "important" pages - Yahoo did that when they were human edited directory/search engine hybrid.
It seems to me that Mr Wales entertains the illusion that a very small number of manually checked pages in the Web space will be sufficient to satisfy vast majority (and it has got to be 98%+ as I won't be hopping from one search engine to another) of search queries. If this was the case then we would still be using Yahoo that did pretty much just that, yet almost everyone (including Yahoo) moved to algorithmic search engines because it is the only way to handle billions of pages, and billions of pages you will have to handle: even if you just index homepages of all registered domain names you will be dealing with 100 mln+ pages, that's good 20 times more than articles in Wikipedia and checking pages can be far more duller than reading nice article you have some personal interest in.
What I find ironic that our own concept of the search engine was removed from Wikipedia because we were supposedly "not noteable enough", that's the sign how they handle problem of "too much data" in Wikipedia - they just reduce the problem by reducing datasets greatly, sometimes this is done wrongly, sometimes rightly and it might well work for Wikipedia, but it sure as hell won't work for Web scale searches. Oh, and by the way who said Google and others don't use human reviewers? They sure do, just check TrustRank, this link is ranked as #1 match on Google for search TrustRank! Notice what Wikipedia tells us: "While human experts can easily identify spam, it is too expensive to evaluate manually a large number of pages."
Human input plays an important (although fairly unknown as they prefer to keep it secret) role in the state of the art search engines, however suggestion that humans can handle billions of pages and/or that a handful of pages will be sufficient for a general purpose search engine is wrong and a very backwards move that will result in exactly the kind of wrong attitude present in Wikipedia now.
Most of very few people who would use OSS will use it for free without paying a dime, a small minority of them will actually need proper business support - but businesses have fixed costs and if sales are low this means unit price will be high. Naturally this is a catch 22 as the high price puts people off and they either use same software for free or buy into non-OSS.
This would effectively cancel out the UK's recent successes in reducing greenhouse gas emissions, and would have wider global implications as well."
Cancel out? Its not like by trying to reduce CO2 in area X, another area Y produced more CO2 in response to reduction in X -- this is not the case, and while knowledge of what produces CO2 is not complete, it is just plain silly to imply that there was no point to even try reducing it!
If it had not been reduced then there would have been MORE of it, not less.
It was noted above that while there are plenty of CPU sucking projects they don't seem to have end results that can actually be used in daily life.
OK, d.net proved the point by breaking crypto that was thought to be too strong. Fine, done that, why waste CPU cycles further?
SETI@Home -- okay, its cool to search for aliens, but lets be realistic here -- its cool, but not exactly useable.
Lots of effort, heck, lots is too small of a word to describe amount of CPU that went into these projects! Cool scoreboards, teams etc, but what are the end results for millions of users after good 10 years of d.net's existance!?!?! Not much.
This is why I created my own project to build something that I use every day -- search engine. I can live without aliens or crypto, but I sure as hell can't live without a good WWW search engine. Can you?
Distributed projects have been around for a long time. Some of them are (SETI@Home) are cool, some were even useful to prove the point (Distributed.net), but what are the real-life uses of calculating Pi to the gazzillionth number? Come on - where is the benefit that you can take advantage today, tomorrow and every day after that?
There are none - in many respects those well known projects is waste of CPU time and electricity. But there are projects that aim to create something that will be of daily use to everyone... like a search engine. How many times you use WWW search engine every day? I use loads, so much that I could live with Internet Explorer 1.0, but I could not live without a search engine.
One of those projects is the Majestic-12 Distributed Search Engine project (http://www.majestic12.co.uk/) that aims to build a search engine contributed to and controlled by the community. We need help - if just a fraction of people who run other distributed projects joined us we could have build a major search engine in no time! You can run it with CPU intensive projects!
Microdrive won't work because they don't support Compact Flash for whatever their excuse is. I am personally waiting for device with at least 10Gb of memory and at least 10 hour _VIDEO_ playtime
I beg to differ! Since dark ages of MSDOS games were parallel, and I mean games like Commander Keen, Wolf 3D, DOOM, etc!
There would be an Int08 handler (possibly reprogrammed to have more than meagre 18.2Hz freq like in DOOM) which would simply DRAW the CURRENT view scene, that's it nothing else, except for safe check to avoid re-entering same handler if rendering was slow for any reason.
Int09 (keyboard) would handle movements, which would only be someting like, change player actor X to X-10, and Int08 handler would redraw it next time. Same would go for mouse interrupt Int33, if my memory serves me well (callback function).
So, there was no such thing as the loop you described in all more or less well built games (id's). I actually did this thing like 6-7 years ago, no one it his sane mind would do differently these days, it's no longer state of the art approach.
Opt-in is when you or someone else can give your email addressed and you will be spammed from now on. Since someone else can "kindly" "opt you in" it can easily be equiv of not opt-in.
Double opt-in is like opt-in but it at first send confirmation email, and if you confirm that you really want this stuff, then you will receive it.
You are absolutely right in my opinion asserting that Microsoft paid them money (and they have all right to do so, nothing bad here). However, since EPIC always targeted DOS/Windows as their primary gaming platform, it would never have become a problem to run it on Xbox. The actual answer as I think is that if Microsoft were to compete with Sony and SEGA in the console arena (and that's what they presumably trying to do with the XBox), then Microsoft would have to have some EXCLUSIVE games to lure players to buy Xboxes in the first place. Usage of DirectX ensures that whoever crazy enough to port this game will have hell of a time.
What's really interesting is that EPIC seems to be desperate enough to bank on their relationship with Microsoft, and they do all that in a very foggy situation when Microsoft can be split up and God know what else can happen to them. EPIC is really taking a major bet here, and this probably is because they are not really doing too well in financial terms, hence comes cash infusion from Microsoft. Games development is an expensive business you know... and hey, EPIC guys deserve to drive Ferraries, too, right?
Sure thing, but G3 doesn't run x86 software as efficiently as Crusoe does, and that's was the main idea to have LOW-power chip that runs more or less ok x86 stuff. G3 doesnt' qualify.
Re:Shouldn't we consider this a "good" thing?
on
Athlons Sold Out
·
· Score: 1
Perhaps AMD needs to look into more modern production processes, like JIT manufacturing.
You don't know what you are talking about. JIT (JustInTime) is not possible when it takes $1bln and at least a year or about that, to build a new fab to product chips. There is no way around it.
Re:Shouldn't we consider this a "good" thing?
on
Athlons Sold Out
·
· Score: 1
Not that fast. You can't increase capacity overnight. There is no way around -- building state of the art $1bln+ fab takes more than just few months. AMD is in trouble with capacity partly because of the very same troubles with Intel, as consumer demand shifted to AMD since Intel can't cope. Thus, if AMD was to predict increase in demand, they would have to predict capacity problems on Intel's side. Correct me if I am wrong, but this used to be top-classed info at Intel, they denied it for a long time, and hence AMD was not able to reliable predict that AND build a fab overnight. Hope that explains.
Re:How does this all affect the X-Box?
on
Athlons Sold Out
·
· Score: 1
X-Box is irrelavant, it will only be introduced sometimes in (late) 2001, ie next year. By that time quite a few things will be different as we should be way above 1GHz.
His algo was written in asm of course. His target machines were 8086/8088, and only later 80286. C would be too slow, especialy for such a CPU intensive process with very small CPU caches.
A quick prove is that he had about 1Kb PKUNZIPJR.COM program in his distribution which could UnZIp, and was darn small. Obviously COM file was written in ASM.
There are some cases where double compression helped, most notable when you had a lot of small files. Since PKZip didn't compress filenamesthemselves (it kept them at the end of file), further compression was compressing this uncompressed data storing it as just one filename inside. You got the idea.
I fully agree with you. However, there is a strong driving force behind these proposals. Remember, the Net is not what it used to be 10 years ago. The best (short, memorable words) domain names are taken. Some of these are for sale for obscene amounts of money. Obviously a company that wishes to capitalize on the Inet gold rush needs a good domain name (quite a few CEOs read that in their first Internet book on that the domain name is 95% of things done (not)). Hence, they either got an option to get stuck with hardtotypeandremember.com OR pay hefty price for a easy.com domain name OR get their chance with new TLDs.
Althought most of them will fail anyway, many wish to find yet another new way to fail their business on the Net, without considering fundamentals at all.
Cybersquatters can't wait for new TLDs too, as they want to resell them yet another time to existing companies.
Not necessarily. That long delay implies there were serious problems with architecture/design. Given that, they probably had to rebuild/redegin the whole thing more than once. Many bugs happen when you connect alien (they never developed their engine, and althought they know it well, they lack Carmack to help, and he is a real asset) pieces of the puzzle. If I remember correctly they started with Quake engine, then switched to Quake2 and only God know what outdates stuff they used.
Tell this to DukeNuken4Ever team, I lost count how many game engines they wanted to use, Prey, Quake2, Unreal? I would not be surprised to see loads of bugs, but then again, who cares about bugs it's not RPG, it's FPS (as long as there are no bugs in "blood making" code).
Hey Romero, go back to Commander Keen series or Shadow Knights or something you do best!
Re:Great! I get to re-install NeXT Step...
on
Carmack Speaks
·
· Score: 2
I was told Doom was made originally on the NeXT and ported to Dos (the Doom Operating System) to reach the widest market posable. </i>
Almost correct. DOOM was developed on NeXT and consisted of 98% of portable C code and 2% assembler, which was the main reason for high portability. Now, the term 'DOOM Operating System' was used because JohnC had to do a weird hack to ensure DOOM runs fine on 4Mb RAM PCs (few people could afford to buy RAM for $40-50 per 1 Mb back in 1993). Since he used DOS4/GW DOS Extender which only could use extended memory (ie not the first 640k, thereby limiting memory to 3Mb on a 4Mb machine), he wrote his own memory manager to use base memory (what was left out of 640kb). Remember line saying XXX Mb allocated to zone during the start-up? Since memory manager is one of few things every operating system provides, he called it DOOM Operating System.
No so fast. First of all have a look at C/C++ Portability Guidelines coustesy of mozilla.org. By observing this document you will find that lots of yummy features of C++ are hardly portable. By suggesting to use some kind of standard memory allocator/g-collector, you will then rely on this thing to get decent performance, and probably seriously screw your chances to port this thing anywhere. If you are so desperate for this thing, then you can code it yourself and use it.
I have not tried yet, but I think you will have MAJOR problems compiling Mozilla using BC++5.5. There are no MFC libs, besides, who cares about compiling, it's the development that matters, and the development will be tough without nice IDE, maybe not for Mozilla where most of UI is defined in itself, but if you plan on developing a GUI app for Windows, then you will struggle.
There is no way one is going to develop a real Win app w/o decent GUI tools, are you going to make up your RES files where all widgets are defined manually?
Command line compiler is cool, but it is useless for RAD development without a RAD tool (C++Builder), of course you can write Win drivers, but then again you will need some decent Win32 SDK, which can usually be obtained by paying some good bucks, where you will getting a compiler anyway.
Now, I highly doubt the statement that students need optimizing versions of compiler. Learn how to optimize your own algos, don't rely on compiler!!!
The fact is -- you can not develop any serious commercial _GUI_ Win32 applications with command line tools and be as effective as those who use proper RAD tools.
No. They can always find out the number of downloads from their FTP, and this is the only thing that matters (ie real interest). Put your hand on your heart, and tell the whole world, did you feel in their questionare properly? Well, I highly suspect that quite a few people probably not.
Slashdot Mirror
> So what's the problem? Why are ISPs bitching?
The problem is that while BBC got their own pipe in full from hosting provider, the consumer side of things (ISPs) was knowingly undersold and ISPs build flawed models that assumed bandwidth will not be used much, hence high contention ratios and they don't even have bandwidth for those. Initially they "solved" problem by kicking off heavy users, but with the advent of youtube and others it all went mainstream so they can't kick off their whole customer base. They might try though.
Disclaimer: I am a founder of a competing search engine concept that is based on volunteers running distributed software that crawls pages, conducts partial analysis and indexing before passing over results to central server that will index data into main central index: human aspect here is the people who take part in the project, and participants can actually change ranking formulaes, and shortly will be able to assist in human detection of spam etc.
Searching the Web is a very challenging problem (that's why few companies do it): volume of data is huge and one only appreciates value of good algorithms when faced with situation when poor algorithms make stuff run for weeks failing near the end and you have to restart the run to wait another week. You can either try to handle this very big problem, which is very hard even if you have the money (look at Amazon's A9 funded with millions, yet they licensed Google's code and database), or you can try to reduce the problem: only focus on a handful of "important" pages - Yahoo did that when they were human edited directory/search engine hybrid.
It seems to me that Mr Wales entertains the illusion that a very small number of manually checked pages in the Web space will be sufficient to satisfy vast majority (and it has got to be 98%+ as I won't be hopping from one search engine to another) of search queries. If this was the case then we would still be using Yahoo that did pretty much just that, yet almost everyone (including Yahoo) moved to algorithmic search engines because it is the only way to handle billions of pages, and billions of pages you will have to handle: even if you just index homepages of all registered domain names you will be dealing with 100 mln+ pages, that's good 20 times more than articles in Wikipedia and checking pages can be far more duller than reading nice article you have some personal interest in.
What I find ironic that our own concept of the search engine was removed from Wikipedia because we were supposedly "not noteable enough", that's the sign how they handle problem of "too much data" in Wikipedia - they just reduce the problem by reducing datasets greatly, sometimes this is done wrongly, sometimes rightly and it might well work for Wikipedia, but it sure as hell won't work for Web scale searches. Oh, and by the way who said Google and others don't use human reviewers? They sure do, just check TrustRank, this link is ranked as #1 match on Google for search TrustRank! Notice what Wikipedia tells us: "While human experts can easily identify spam, it is too expensive to evaluate manually a large number of pages."
Human input plays an important (although fairly unknown as they prefer to keep it secret) role in the state of the art search engines, however suggestion that humans can handle billions of pages and/or that a handful of pages will be sufficient for a general purpose search engine is wrong and a very backwards move that will result in exactly the kind of wrong attitude present in Wikipedia now.
Most of very few people who would use OSS will use it for free without paying a dime, a small minority of them will actually need proper business support - but businesses have fixed costs and if sales are low this means unit price will be high. Naturally this is a catch 22 as the high price puts people off and they either use same software for free or buy into non-OSS.
> Does anyone know whether Majestic Reasearch has any
> connections to Majestic 12 (http://www.majestic12.co.uk/ [majestic12.co.uk])?
As the founder of the Majestic-12 project I can assure you that we are not related in any way, shape or form.
Cancel out? Its not like by trying to reduce CO2 in area X, another area Y produced more CO2 in response to reduction in X -- this is not the case, and while knowledge of what produces CO2 is not complete, it is just plain silly to imply that there was no point to even try reducing it!
If it had not been reduced then there would have been MORE of it, not less.
Disclaimer -- I run a distributed search engine project so my opinion is biased.
It was noted above that while there are plenty of CPU sucking projects they don't seem to have end results that can actually be used in daily life.
OK, d.net proved the point by breaking crypto that was thought to be too strong. Fine, done that, why waste CPU cycles further?
SETI@Home -- okay, its cool to search for aliens, but lets be realistic here -- its cool, but not exactly useable.
Lots of effort, heck, lots is too small of a word to describe amount of CPU that went into these projects! Cool scoreboards, teams etc, but what are the end results for millions of users after good 10 years of d.net's existance!?!?! Not much.
This is why I created my own project to build something that I use every day -- search engine. I can live without aliens or crypto, but I sure as hell can't live without a good WWW search engine. Can you?
Distributed projects have been around for a long time. Some of them are (SETI@Home) are cool, some were even useful to prove the point (Distributed.net), but what are the real-life uses of calculating Pi to the gazzillionth number? Come on - where is the benefit that you can take advantage today, tomorrow and every day after that?
There are none - in many respects those well known projects is waste of CPU time and electricity. But there are projects that aim to create something that will be of daily use to everyone... like a search engine. How many times you use WWW search engine every day? I use loads, so much that I could live with Internet Explorer 1.0, but I could not live without a search engine.
One of those projects is the Majestic-12 Distributed Search Engine project (http://www.majestic12.co.uk/) that aims to build a search engine contributed to and controlled by the community. We need help - if just a fraction of people who run other distributed projects joined us we could have build a major search engine in no time! You can run it with CPU intensive projects!
Disclaimer: I am the founder of this project.
Microdrive won't work because they don't support Compact Flash for whatever their excuse is. I am personally waiting for device with at least 10Gb of memory and at least 10 hour _VIDEO_ playtime
> and that's new purchases, not just the installed base, which is even more favorable to Palm
Your information is not correct. The balance of NEW sales shifted drastically in favour of WinCE (due to iPAQ)
I beg to differ! Since dark ages of MSDOS games were parallel, and I mean games like Commander Keen, Wolf 3D, DOOM, etc!
There would be an Int08 handler (possibly reprogrammed to have more than meagre 18.2Hz freq like in DOOM) which would simply DRAW the CURRENT view scene, that's it nothing else, except for safe check to avoid re-entering same handler if rendering was slow for any reason.
Int09 (keyboard) would handle movements, which would only be someting like, change player actor X to X-10, and Int08 handler would redraw it next time. Same would go for mouse interrupt Int33, if my memory serves me well (callback function).
So, there was no such thing as the loop you described in all more or less well built games (id's). I actually did this thing like 6-7 years ago, no one it his sane mind would do differently these days, it's no longer state of the art approach.
Not opt-in is when you simply get spammed.
Opt-in is when you or someone else can give your email addressed and you will be spammed from now on. Since someone else can "kindly" "opt you in" it can easily be equiv of not opt-in.
Double opt-in is like opt-in but it at first send confirmation email, and if you confirm that you really want this stuff, then you will receive it.
What's really interesting is that EPIC seems to be desperate enough to bank on their relationship with Microsoft, and they do all that in a very foggy situation when Microsoft can be split up and God know what else can happen to them. EPIC is really taking a major bet here, and this probably is because they are not really doing too well in financial terms, hence comes cash infusion from Microsoft. Games development is an expensive business you know... and hey, EPIC guys deserve to drive Ferraries, too, right?
Sure thing, but G3 doesn't run x86 software as efficiently as Crusoe does, and that's was the main idea to have LOW-power chip that runs more or less ok x86 stuff. G3 doesnt' qualify.
You don't know what you are talking about. JIT (JustInTime) is not possible when it takes $1bln and at least a year or about that, to build a new fab to product chips. There is no way around it.
Not that fast. You can't increase capacity overnight. There is no way around -- building state of the art $1bln+ fab takes more than just few months. AMD is in trouble with capacity partly because of the very same troubles with Intel, as consumer demand shifted to AMD since Intel can't cope. Thus, if AMD was to predict increase in demand, they would have to predict capacity problems on Intel's side. Correct me if I am wrong, but this used to be top-classed info at Intel, they denied it for a long time, and hence AMD was not able to reliable predict that AND build a fab overnight. Hope that explains.
X-Box is irrelavant, it will only be introduced sometimes in (late) 2001, ie next year. By that time quite a few things will be different as we should be way above 1GHz.
His algo was written in asm of course. His target machines were 8086/8088, and only later 80286.
C would be too slow, especialy for such a CPU intensive process with very small CPU caches.
A quick prove is that he had about 1Kb PKUNZIPJR.COM program in his distribution which could UnZIp, and was darn small. Obviously COM file was written in ASM.
There are some cases where double compression helped, most notable when you had a lot of small files. Since PKZip didn't compress filenamesthemselves (it kept them at the end of file), further compression was compressing this uncompressed data storing it as just one filename inside. You got the idea.
I fully agree with you. However, there is a strong driving force behind these proposals. Remember, the Net is not what it used to be 10 years ago. The best (short, memorable words) domain names are taken. Some of these are for sale for obscene amounts of money. Obviously a company that wishes to capitalize on the Inet gold rush needs a good domain name (quite a few CEOs read that in their first Internet book on that the domain name is 95% of things done (not)). Hence, they either got an option to get stuck with hardtotypeandremember.com OR pay hefty price for a easy.com domain name OR get their chance with new TLDs.
Althought most of them will fail anyway, many wish to find yet another new way to fail their business on the Net, without considering fundamentals at all.
Cybersquatters can't wait for new TLDs too, as they want to resell them yet another time to existing companies.
Not necessarily. That long delay implies there were serious problems with architecture/design. Given that, they probably had to rebuild/redegin the whole thing more than once. Many bugs happen when you connect alien (they never developed their engine, and althought they know it well, they lack Carmack to help, and he is a real asset) pieces of the puzzle. If I remember correctly they started with Quake engine, then switched to Quake2 and only God know what outdates stuff they used.
Tell this to DukeNuken4Ever team, I lost count how many game engines they wanted to use, Prey, Quake2, Unreal? I would not be surprised to see loads of bugs, but then again, who cares about bugs it's not RPG, it's FPS (as long as there are no bugs in "blood making" code).
Hey Romero, go back to Commander Keen series or Shadow Knights or something you do best!
I was told Doom was made originally on the NeXT and ported to Dos (the Doom Operating System) to reach the widest market posable.
</i>
Almost correct. DOOM was developed on NeXT and consisted of 98% of portable C code and 2% assembler, which was the main reason for high portability. Now, the term 'DOOM Operating System' was used because JohnC had to do a weird hack to ensure DOOM runs fine on 4Mb RAM PCs (few people could afford to buy RAM for $40-50 per 1 Mb back in 1993). Since he used DOS4/GW DOS Extender which only could use extended memory (ie not the first 640k, thereby limiting memory to 3Mb on a 4Mb machine), he wrote his own memory manager to use base memory (what was left out of 640kb). Remember line saying XXX Mb allocated to zone during the start-up? Since memory manager is one of few things every operating system provides, he called it DOOM Operating System.
IMHO.
No so fast. First of all have a look at C/C++ Portability Guidelines coustesy of mozilla.org. By observing this document you will find that lots of yummy features of C++ are hardly portable. By suggesting to use some kind of standard memory allocator/g-collector, you will then rely on this thing to get decent performance, and probably seriously screw your chances to port this thing anywhere. If you are so desperate for this thing, then you can code it yourself and use it.
I have not tried yet, but I think you will have MAJOR problems compiling Mozilla using BC++5.5.
There are no MFC libs, besides, who cares about compiling, it's the development that matters, and the development will be tough without nice IDE, maybe not for Mozilla where most of UI is defined in itself, but if you plan on developing a GUI app for Windows, then you will struggle.
There is no way one is going to develop a real Win app w/o decent GUI tools, are you going to make up your RES files where all widgets are defined manually?
Command line compiler is cool, but it is useless for RAD development without a RAD tool (C++Builder), of course you can write Win drivers, but then again you will need some decent Win32 SDK, which can usually be obtained by paying some good bucks, where you will getting a compiler anyway.
Now, I highly doubt the statement that students need optimizing versions of compiler. Learn how to optimize your own algos, don't rely on compiler!!!
The fact is -- you can not develop any serious commercial _GUI_ Win32 applications with command line tools and be as effective as those who use proper RAD tools.
No. They can always find out the number of downloads from their FTP, and this is the only thing that matters (ie real interest).
Put your hand on your heart, and tell the whole world, did you feel in their questionare properly? Well, I highly suspect that quite a few people probably not.