Slashdot Mirror

← Back to Stories (view on slashdot.org)

Ask Slashdot: What Is Your View On Sloot Compression? (youtube.com)

Posted by BeauHD on from the lost-inventions dept.

An anonymous reader writes: A Dutch electronics engineer named Jan Sloot spent 20 years of his life trying to compress broadcast quality video down to kilobytes -- not megabytes or gigabytes (the link in this story contains an 11 minute mini-documentary on Sloot). His CODEC, finalized in the late 1990s, consisted of a massive 370Mb decoder engine that likely contained some kind of clever system for procedurally generating just about any video frame or audio sample desired -- fractals or other generative approaches may have been used by Sloot. The "instruction files" that told this decoder what kind of video frames, video motion and audio samples to generate were supposedly only kilobytes in size -- kind of like small MIDI files being able to generate hugely complex orchestral scores when they instruct a DAW software what to play. Jan Sloot died of a heart attack two days before he was due to sign a technology licensing deal with a major electronics company. The Sloot Video Compression system source code went missing after his death and was never recovered, prompting some to speculate that Jan Sloot was killed because his ultra-efficient video compression and transmission scheme threatened everyone profiting from storing, distributing and transmitting large amounts of digital video data. I found out about Sloot Compression only after watching some internet videos on "invention suppression." So the question is: is it technically possible that Sloot Compression, with its huge decoder file and tiny instruction files, actually worked? According to Reddit user PinGUY, the Sloot Digital Coding System may have been the inspiration for Pied Piper, a fictional data compression algorithm from HBO's Silicon Valley. Here's some more information about the Sloot Digital Coding System for those who are interested.

61 of 418 comments (clear)

  1. Re:Not so fast by rtb61 · · Score: 4, Interesting

    Hear the rhythm of the beat and you will know there's maths in music. Yep, an entire orchestra can play of a few pages of dead wood. Voila problem solved.

    Do not think so much about compression, think more about robotic simulation and scripts. So create a simulated robotic orchestra and write them a script and the play the audio, visual role. That script is the compressed version of the orchestra's efforts.

    So can you create a computer program that would 'act' out and entire program based upon scripts provided, of course you can, it would take quite a bit of development but once you develop one virtual human robot, you have developed a virtual infinite number of them.

    --
    Chaos - everything, everywhere, everywhen
  2. Actually... by Black+Parrot · · Score: 5, Insightful

    They killed him because you could feed a random input into his decoder and the movie that came out would be better than anything Hollywood can produce.

    --
    Sheesh, evil *and* a jerk. -- Jade
    1. Re:Actually... by almitydave · · Score: 2

      But in 1999, Hollywood movies weren't so bad. That's the year that gave us The Matrix.

      That's also the year that Hollywood gave us Dudley Do-right, Wild Wild West, and Deuce Bigelow: Male Gigolo.

      --
      my, your, his/her/its, our, your, their
      I'm, you're, he's/she's/it's, we're, you're, they're
  3. It's not a thing by Just+Some+Guy · · Score: 4, Interesting

    So, you want to replace every frame in a movie with a collection of images or snippets that correspond to each part of the frame, right? And you're going to store a dictionary of snippets, referenced by number, then say "this frame takes snippet 1234, 6543, and 9274". The problem is that the number of snippets you'd have to store is enormous, and that each snippet itself is going to be a ginormous number (like the bits of the string of bytes in that snippet).

    See where this is going? You're basically establishing a mapping of small numbers to much larger numbers. Either that set of big numbers is tiny (in which case you can only represent a small number of frames in the output video and picture quality is awful) or it's huge, in which case the index numbers themselves become roughly as big as the numbers they're referring to, and oh yeah, good luck searching through that space bunches of times per frame.

    The idea isn't inherently bad if you have a small number of states you want to represent. For instance, Zstandard lets you precompute a dictionary of common strings you want to shorten. Imagine if you trained it on HTML so that each tag or other common string just takes a few bits, then you can distribute that dictionary to the whole world so that you can save the bandwidth of transmitting it alongside the compressed data each and every time (like we do with Zip, Gzip, etc.). That's a nice thing! But the search space of "things you can display on a screen" is a hell of a lot bigger than "things you can sent in an HTTP header".

    --
    Dewey, what part of this looks like authorities should be involved?
    1. Re:It's not a thing by fustakrakich · · Score: 2

      Vectors vs. bitmaps, which compresses better?

      --
      “He’s not deformed, he’s just drunk!”
    2. Re:It's not a thing by demonlapin · · Score: 2

      Yeah, it's not completely insane to think that a really brilliant fractal technique could do some amazing lossy compression, but the amount of CPU required to encode and render it would be insane.

    3. Re:It's not a thing by Cerlyn · · Score: 2

      For instance, Zstandard lets you precompute a dictionary of common strings you want to shorten. Imagine if you trained it on HTML so that each tag or other common string just takes a few bits, then you can distribute that dictionary to the whole world so that you can save the bandwidth of transmitting it alongside the compressed data each and every time (like we do with Zip, Gzip, etc.).

      HTTP/2.0 actually does this for HTTP headers as part of the HTTP Header Compression specification.

    4. Re:It's not a thing by dbrueck · · Score: 4, Interesting

      I agree with you, but I do find this whole idea interesting at least, for a couple of reasons:

      1) Video codecs today do use some form of the index-to-dictionary technique already (e.g. I-/P-/B-frames), the big differences include the fact that the dictionary is in the file itself (I-frames), and it is not comprehensive for the whole media typically but is instead relevant to only a small portion of frames before a new "dictionary" is created.

      So not really the same thing as what Sloot was possibly doing, but it does make me wonder if there is something to the idea of having some builtin library of sample blocks that can be "good enough" replacements for many cases, or maybe act as starting samples that can be transformed into something suitable for various uses, etc. This never escapes the indices-become-too-big problem that you rightly pointed out, but there could very well be merit in the general concept. I doubt it'd yield the massive reduction in bitrates Sloot was apparently hoping to achieve, but maybe it'd be enough for the step in compression improvements? I mean h264/h265 already do some pretty impressive and creative stuff already, so something along these lines isn't completely outlandish.

      2) Media compression (and to a lesser extent decompression) always battles the cost tradeoff, both the compute power required but also just the time it takes to do the compression. Because of the complexities of getting a codec adopted everywhere (especially in silicon like in a phone), there is a push to have general purpose codecs that can be used for both live and pre-recorded media, and the type of analysis that is done on each frame or on groups of frames is still driven by available computing power - even when you fiddle with the compression settings between "just get this done in realtime" to "take as long as you want and analyze each frame to death", the difference in time between the two extremes is usually just an order of magnitude or so - a big difference to be sure, but still kind of within the same realm.

      The point is just that the types of analysis done are impacted by whatever is considered to be, at that point in time, a feasible amount of computational work. When we think ahead to 5 or 6 orders of magnitude of more compute power (due to faster CPUs or harnessing gazillion-core GPUs or something else entirely), it's hard for us to really grasp what doors that opens in terms of analysis techniques - we can mostly think in terms of what we do today and how we could make that faster, but as we get there we'll also start to take on completely novel approaches simply because things that were too ludicrous to even consider before become worth exploring.

      For example, we are still at the dawn of GPU-based video compression (in the sense that right now GPUs can do some things to speed it up, but in /general/ they are just optimizing the CPU-based approach - we're at the early stages of actually have codecs designed for GPUs specifically. Given vastly more computing power, maybe we really could get to the point where the sample blocks mentioned above can be distilled down to equations that produce them (them or a good enough representation). When we get to a relatively absurd amount of compute time available when compressing the video, there's no telling what sorts of new ideas and techniques will be explored.

      Or maybe we'll take on a completely different approach to video altogether. I mean, what if we move away from trying to reproduce individual pictures and their pixels and instead start transmitting the scene info? A TV show has a limited number of sets, what if the 3d model/texture info for most everything was transmitted in advance so that during playback all of that information is expressed in a few identifiers and their 3d transforms (ditto for the actors at some point too). We already see this in simplified form in replays from video games - a replay of a minute of gameplay can recreate the scene with perfect fidelity, and yet it is tiny compared to wh

    5. Re: It's not a thing by Anonymous Coward · · Score: 2, Interesting

      https://rogerjohansson.blog/2008/12/07/genetic-programming-evolution-of-mona-lisa/

      You were saying?

    6. Re:It's not a thing by Just+Some+Guy · · Score: 2

      An 8x8x24-bit rectangle has 10^462 possibilities, and fractals are hella sensitive to initial conditions. Finding an approximately match is going to be less painful than naive brute force searching, but it still laughs in the face of a little teraflop GPU.

      --
      Dewey, what part of this looks like authorities should be involved?
    7. Re:It's not a thing by wierd_w · · Score: 3, Interesting

      For any given number of any given complexity, there are adjacent numbers that have significantly lower complexity.

      I have toyed with the idea of using "indexing" between two points on a number line, where the two points are very low complexity, but the number you want is somewhere between-- With a high precision percentage of the now clearly defined space between the points on the line, we can skip over and start playback of the number we want, if we also state how many digits this number is.

      Say, the number we want is between 10^23 and ((10^23)+(10^10)). Those are both impossibly huge numbers, but they can be defined very easily because they are not complex. We will then say that the number we want is found 10% of the distance between those two numbers, and that it is 10^6 digits long.

      Just using some fun math with the natural logarithm, we can produce that number, from those modest ingredients.

      Video files are just very very large numbers. They could be found on the infinite number line in exactly the same way.

      A complex encoder that has broken the file to be searched into a series of smaller (and thus easier to compute/derive) numbers that get concatenated together, and has a working knowledge of number space for numbers of those data block sizes, could reduce hundreds of megabytes into a few kb of metadata. The decoder would have to compute very large numbers (or have very large numbers already stored statically, and just crawls along...) to initialize the playback, but it could easily do so using nothing but lots of RAM and brute force CPU.

      I have considered looking into creating a compression system of this type myself, which is why I find it kinda spooky that it would need a huge decoder.. because my theoretical system would need a huge decoder too.

    8. Re:It's not a thing by denzacar · · Score: 2

      Yeah, but what if you replaced everything with stick figures?
      Shark in Jaws could be just a single triangle. Hell... make that just two lines...

      Someone get me Spielberg on the line!
      I have an idea for a very low budget movie! It'll make back on its investments thousand-fold... million-fold!

      --
      Mit der Dummheit kämpfen Götter selbst vergebens
    9. Re:It's not a thing by Rockoon · · Score: 4, Interesting

      An 8x8x24-bit rectangle has 10^462 possibilities

      ..and a tiny fraction of that is interesting, and those that are interesting are so because they arent random noise.

      I can encode 100% of the random noise possibilities in only a few bits driving an LFSR... and you wont be able to tell the difference.

      Repeat after me: "Everything I learned about the pigeon hole paradox is true, but I didnt actually understand any of it"

      This is not a pigeon hole issue because nobody is pretending that these compression technologies do equally well with every kind of data, and even the kind of data (noise for instance, which is traditionally hard to compress) doesn't apply because these compressors are lossy such that random data can be approximated by literally any of a large collection of simple random functions.

      You cant tell the difference because if you could then it would have been compressible to begin with. You can only tell when the coherence is amiss, but its the coherence that is "accurately" compressible.. its only the incoherent stuff.. aka noise.. that is hard to encode accurately in only a few bits

      We arent even close to the limits of lossy compression. as its just a time/space trade-off.

      --
      "His name was James Damore."
    10. Re:It's not a thing by Anonymous Coward · · Score: 4, Informative

      It doesn't work. I've written some papers on this.

      Basically, every set of easily-compressible numbers has the property that the next one gets farther and farther away at exactly the rate needed to ensure that the offsets from one to any other natural number require (in total) exactly the same number of bits as you'd need to just write those numbers out yourself.

      Proof:
      1) Suppose you could represent any natural n in [0,max] as c(n).
      2) Then you obviously need to be able to decompress any c(n) to the corresponding n. Hence, c is bijective.
      3) By (2) c(n) is just a permutation of [0,max].
      5) As is obvious since (3) is a permutation, the total compressed size of the output range is equal to the total compressed size of the input range: Information content is conserved by permutations (Pigeonhole principle)

      Now, your particular method sets c(n) = (k,o), where k is the nearest compressible number, and o is the offset, perhaps also compressed. It doesn't actually matter what compressor functions you use, the result is general:

      The proportion of strings in your domain which compress down to less than m bits remains unchanged. In other words, there are 2^m strings in m bits, and the proportion in [0,max] is just (2^m)/max.

      Set max = 2^(1000000000000), or the number of terabyte-sized files. Now just evaluate (2^m)/(2^1000000000000) and you'll quickly see why your compression algorithm doesn't work. It'll work for a vanishingly small number of bitstrings, just like how we can represent 10^23 using only 5 symbols, but as you go further and further out from those easily-compressible numbers, the offsets themselves become uncompressible large numbers!

      --
      - Oskar Lidelson

  4. Of course it didn't work by Anonymous Coward · · Score: 2, Insightful

    Information theory stablishes what is really possible and that kind of compression is simply to much, even considering its lossy nature. Infomation entropy has very real limits no matter the encoder used in the data compression.

    1. Re:Of course it didn't work by dbrueck · · Score: 4, Interesting

      Yeah, I don't think his work would have panned out either, but in theory the idea isn't totally implausible, in part because general purpose media compression is a bit of a special case in that the goal is not to try to have lossless compression, due to the way human audio and visual systems work.

      When you look at compression on individual frames, there is a huge amount of acceptable error (differences versus the original) that isn't even discernible when you're looking at still frames side by side, and then when you're playing back the frames at a rate of 30 or 60 or more per second, there's a whole *additional* layer of imperceptible error that can be introduced. And then on top of *that* you can introduce more and more error that is in fact discernible on some level, but it becomes a game of tradeoffs of what you can get away with vs the various costs involved.

      Another way to put it is that this isn't just lossy compression, it's lossy compression that takes advantage of quirks in the way human eyes and ears work, and so the limits of compression potentially go far beyond what you might expect in a strict information theory sense (there are still limits of course, they just might be a lot further out than you might expect).

      On some level it's kinda like how paintings work - you see a barn with a tree next to it in the shade, and yet when you look at it really close you realize that it's just a couple of strokes of paint - your brain perceived far more detail than is actually there. To be clear, I'm not saying this is how media codecs work, just saying that the goal in media compression isn't necessarily to accurately reproduce the source material, rather the goal is to get people's brains to *perceive* that the final version is an accurate reproduction, and in practice that opens up a lot of possibilities.

    2. Re:Of course it didn't work by ezdiy · · Score: 2

      > Information theory stablishes what is really possible

      Actually, information theory states the opposite. Determining entropy of unknown source is an intractable problem, and you can't generally state amount of entropy for piece of data unless you're certain it's a quantum pink noise beforehand, all we know that the better the compressor, the closer you get. That's why one time pads use truly random codebooks, not a PRNG (PRNG has very little entropy - that of PRNG seed).

      While extremely important as an output filter, just an entropy encoder doesn't compression make.

    3. Re:Of course it didn't work by Immerman · · Score: 2

      Indeed - our brains are incredibly good at filing in "missing" detail - a rough suggestion of something is often enough to "see" incredible detail.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    4. Re:Of course it didn't work by dbrueck · · Score: 2

      Yes, and it's some really neat stuff - research talked about in these articles is fascinating:

      https://www.sciencedaily.com/r...
      https://medicalxpress.com/news...

      And here's a site that lets you experience just one of the ways the brain manufactures some of the detail you perceive: http://www.uniformillusion.com...

      A common theme in all of these is that "sight" isn't entirely achieved with our eyes, but our brains get involved very early in the signal processing stage and even make up a lot of info based on what it expects.

      Who knows how much of this can be practically applied to things like video compression, but we've already been doing it to a limited degree for years (e.g. x264's psychovisual enhancement setting, see also https://en.wikipedia.org/wiki/...).

    5. Re:Of course it didn't work by Anonymous Coward · · Score: 3, Interesting

      This is how literature works... a short story is in effect a full movie highly compressed down to a few kilobytes. 'Reading' is decompressing the data.

  5. More plausible explanation: by Gravis+Zero · · Score: 4, Interesting

    Jan Sloot had some ideas on how to compress things and despite hyping it up, he had gotten nowhere close to anything functional. With all the hype he generated, he had a deal lined up for the technology but he didn't have the goods. The stress of the impending revelation of his fraud caused him to suffer cardiac arrhythmia and he died. The source code was never found because it never existed.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re: More plausible explanation: by Anonymous Coward · · Score: 5, Funny

      So you're saying he should have waited for Kickstarter to be invented?

  6. I love it that parent is modded "Insightful" by mykepredko · · Score: 4, Insightful

    That really says a *lot* about the quality of today's movies.

    --
    Mimetics Inc. Twitter
    1. Re:I love it that parent is modded "Insightful" by hcs_$reboot · · Score: 2

      That really says a *lot* about the quality of today's movies.

      That also says a lot about /. readers.

      --
      Slashdot, fix the reply notifications... You won't get away with it...
  7. You don't know what you're talking about by Anonymous Coward · · Score: 2, Interesting

    The presumption isn't that it actually stored images but more likely that it contained numerous generators. It's entirely plausible is that similar to the way a neural net uses a combination of several algorithms to classify images, a similar approach could be taken to constructing them.

  8. Yes, it's possible by deek · · Score: 5, Funny

    Myself, I've come up with a system that can compress a video file down to one byte. Unfortunately, it has some limitations. The size of the decoder is approximately the same size as the uncompressed video file, and it will only work on one specific file.

    Damn, should I be afraid for my life now?

    1. Re:Yes, it's possible by Drakster · · Score: 2

      Always thought it would be an amusing joke if someone who produced a new image file format wrote the following in the spec.

      Inputs with the total size of 0 bytes should return an image of Lenna in the highest possible quality.

  9. Dead scammer, not dead inventor. by gurps_npc · · Score: 4, Interesting

    I believe it was a scam. He never really had that good compression. Either he got cold feet and offed himself before he had to deliver the product that would not work, or someone else found out and killed him in a a rage at being tricked.

    While yes, there are a few scientific advancements that are remarkable, in fields like computer file compression that are:

    1) Essential to an existing, highly profitable business
    2) Mathematically interesting
    3) Being heavily researched by multiple people.

    then any advancements get duplicated in less than 10 years. Too much money, brains and corporate greed focused on this issue for us not to figure out it or something similar by ourselves.

    This has not been duplicated, therefore it was fake.

    --
    excitingthingstodo.blogspot.com
  10. Compression Tweaks by Bruce+Perens · · Score: 5, Insightful

    Sloot wasn't the only "Compression Tweak". This is someone who has compression "ideas" but can never get the product working. There was one in the US who wrote me for a long time in the 90's. One thing I remember is that he dropped hints about encoding data in the spaces in between bits. Of course this makes zero sense.

    --
    Bruce Perens.
    1. Re: Compression Tweaks by HuguesT · · Score: 2

      Yes but DNA does not "encode" a human. It encode a process for generating a body, which in the human case takes about 20 years. The encoding is not one-to-one either, see twins.

  11. Pseudoscientific claptrap by timholman · · Score: 5, Interesting

    Sloot was nothing more than another of a long line of scam artists (or delusional inventors) who claimed to have created a "magic" compression scheme. In his case, he said he could compress an entire movie down to 8 kilobytes.

    Simple mathematics show why such schemes don't work. 8 kilobytes = 8192 bytes = 65536 bits. Assuming you have a two hour movie, then each second of the movie must be mapped into about one byte, which can have only 2^8 = 256 possible values to represent any conceivable second of video. It's mathematically impossible.

    Engineers and mathematicians have been debunking these claims for decades, but they still occasionally pop up. I remember one scheme that got some press about 30 years ago. A guy claimed to have a compression program that could take any data file and compress it down to about 1 kilobyte. And it seemed to work, according to several people who tried it. As it turned out, the "compressed" file was nothing more an alias pointing to the original file, which was hidden from directory view by the program. When you "uncompressed" the file, the original file was unhidden. But it was a neat trick as long as you didn't try to copy the "compressed" file to another disk.

    Sloot's program was "lost" because it never existed, just like the magic 300 mpg carburetors where the plans were "lost".

    1. Re:Pseudoscientific claptrap by Anonymous Coward · · Score: 2, Funny

      Assuming you have a two hour movie, then each second of the movie must be mapped into about one byte, which can have only 2^8 = 256 possible values to represent any conceivable second of video. It's mathematically impossible.

      That might actually work, as long as you restrict the input domain to the set of Michael Bay movies.

    2. Re:Pseudoscientific claptrap by Orgasmatron · · Score: 5, Interesting

      You may be thinking of OWS, the "fractal compression program". The "compressed" file was nothing more than a list of blocks on the disk that the original file occupied. You could test it by compressing a file, deleting the original, then decompressing (=undeleting) it.

      But if you copied it to a floppy and took it to your friends house, it mysteriously failed...

      Here is a mention of it here on slashdot back in 2006: https://slashdot.org/comments....

      And a very brief mention in the compression FAQ: http://www.faqs.org/faqs/compr...

      Internet references to it are spotty. It got passed around on BBSs and sneakernet back before home PCs really started connecting to the internet in a big way.

      You may also be thinking of WIC, which I didn't encounter, so I tend to assume that it didn't achieve as wide a distribution as OWS. The mechanism described for WIC sounds more like what you are describing. Either way, you are off by about 10 years. They were in the mid 90s, not the mid 80s.

      --
      See that "Preview" button?
    3. Re:Pseudoscientific claptrap by timholman · · Score: 2

      You may be thinking of OWS, the "fractal compression program". The "compressed" file was nothing more than a list of blocks on the disk that the original file occupied. You could test it by compressing a file, deleting the original, then decompressing (=undeleting) it.

      But if you copied it to a floppy and took it to your friends house, it mysteriously failed...

      You're right, I was thinking of OWS. Thanks for filling in the gaps in my memory.

      I recall that it actually got some press in the local newspaper back in the day before being debunked. No doubt the guy who wrote the program had a great laugh about it.

    4. Re:Pseudoscientific claptrap by teg · · Score: 2

      You could try transmitting the actual manuscript - compressed, with a partial dictionary known on the other side - and then have it decompressed by letting an AI "acting" it? The same way you could transmit sheet music instead of an actual performance?

      Obviously, the actual performance will differ and there is no way he had technology to do anything like that...

    5. Re:Pseudoscientific claptrap by Daetrin · · Score: 2

      We got ahold of a copy of that (or something very similar) in my computer lab in High School. I believe it was the first lab, so that would have been either '92 or '93. I remember discussing the fractal methods it claimed to use to accomplish such extreme compression.

      We tested it out, and were both amazed and suspicious when it seemed to work. But as soon as we tried copying the "compressed" file to another machine and it failed we knew something was up, and it didn't take long after that to figure out what was really going on.

      --
      This Space Intentionally Left Blank
  12. Anything is possible. Practical though? by LostMyBeaver · · Score: 3, Interesting

    I theorized that the Mandelbrot is capable of generating any string of data of any size given a high enough resolution, high enough iterations and enough time to process it. I haven't retested my theory on scale since 1998 (significant because of general availability of GPUs... if you count Voodoo2 as a GPU), but processing on a 486DX-50 with 16 megs of RAM and Linux allowed me to search for simple strings (kilobytes in size) within reasonable time.

    What I found was that every 4KB sequence I randomly generated as part of a set was able to be located within the Mandelbrot set. Results generally were found in the nautilus or horse tail areas. I used several search methods.
    1) Linear
    2) Rectangular spiral clockwise (variable width, variable height)
    3) Rectangular spiral counter-clockwise (variable width, variable height)
    4) Zig-zag in 45 degree rotations and variable width and variable height)
    5) Elliptical Spiral (clockwise and counter-clockwise, variable width, variable height)

    I also experimented with added dimension which included iteration as a 3rd dimension when pattern searching allowing strings to extend across multiple iterations in cubical, spheroidical, etc... footprints. I had to abandon this effort due to computational complexity.

    I also experimented with experimenting with adding bit layers as a 3rd dimension when searching, but this added even greater computational complexity than the previous method described above but certainly promised greater results.

    The algorithms I used were exhaustive. So where a video motion vector search algorithm would abandon searching a given direction for performance reasons when the delta values in said direction were not yielding results, I searched every pattern type for every pixel for every parameter... etc...

    Now, given my computational capacity and limitations of the time, the average seek time per 4KB string before finding a result was 1-2 weeks. All my code was heavily optimized to exploit memory as it was used on the architecture. Consider a combination of Michael Abrash style coding/optimizations with more specific knowledge of the specific CPU and memory it would run on. I also spent a great deal of money getting memory with 60ns response time (though I never measured better than 64ns on the scope). I also used a Micronics motherboard which at the time meant something as Intel had yet to enter the chipset market and as ECC memory was far too expensive... for pretty much anyone on that scale, and the memory controller was not yet part of the CPU, I needed a reliable and documented chipset to work with.

    Ok, so here's the summary of all the results
    1) It is worth investing greater effort in scientifically proving that absolutely any string can be found in the Mandelbrot Set given enough iterations, resolution, search creativity and of course CPU power.
    2) It is believed (though through non-thorough experimentation, not mathematically proven) that any 4KB string can be found in the Mandelbrot Set
    3) Once found, depending on the resolution and number of iterations provided as a coefficients for identifying the set, the data can be retrieved in a reasonable period of time (deterministic, though variable) on any device. The real computational cost was the search itself.
    4) This method of compression has no value for video as the data sets are too large.
    5) It is most common that longer string lengths searched for often requires higher resolutions and higher iteration counts to be found. This is common, not a rule. Depending on the search pattern, often strings with a more level histogram distribution could be found as bit patterns in lower resolutions and iterations.
    6) The size of the strings found increased far more rapidly than the size of the coefficients required to represent the find. Meaning without employing additional methods such as entropy coding the :
    - resolution of the Mandelbrot image,

    1. Re:Anything is possible. Practical though? by guruevi · · Score: 3, Interesting

      You can take any irrational number and find any value in it. The problem is that you can't truly compress anything that way, your coefficients will be the same size or larger than any number you're trying to get as a result (the optimum compression can never be better than 2:1). If you're looking for "lossy" compression (basically, some values (but no more than x %) can change, you'd still have to look through the nearly infinite space for possible values and at best you'll get some compression close to whatever entropy allows.

      To encode all data that's currently being stored or generated this way by humankind, your encoder would have to work through ~2000 exabytes.

      --
      Custom electronics and digital signage for your business: www.evcircuits.com
  13. Re:Computer magazine's ruby rod by cheesybagel · · Score: 2

    That's nothing. BLAZE MONGER can compress 16kx16k video at 150 fps with 1-color (which must be black) at 0 bits/sec.

  14. The whole story makes it clearer by guruevi · · Score: 4, Informative

    If you read the original stories in Dutch, the whole story becomes a lot clearer.

    The system he had was enclosed in a box and you could initially see his "demo" of 4 movies in low quality. There were various claims on Sloot's part that it was only x size but nobody was allowed to look into or program the system. He was going around investors fishing for money to make it work at bigger resolutions for his lossless compression algorithm. When he croaked nobody found anything in regards source code or design documents.

    It's also mathematically impossible to get the file compressions he got. At best it was a reference to a pre-programmed movie.

    --
    Custom electronics and digital signage for your business: www.evcircuits.com
    1. Re:The whole story makes it clearer by uvajed_ekil · · Score: 5, Funny

      If you read the original stories in Dutch, the whole story becomes a lot clearer.

      Personally, I found that reading the whole story in Dutch left me more confused and totally unsure how to take this. I don't read Dutch.

      --
      This is a hacked account, for which the owner can not be held responsible.
  15. 8 kilobyte move by PPH · · Score: 2

    But you need to download a 370 Mb CODEC for each one.

    --
    Have gnu, will travel.
  16. Re:Not so fast by PopeRatzo · · Score: 4, Insightful

    Yep, an entire orchestra can play of a few pages of dead wood. Voila problem solved.

    Sheet music as a form of compressed script is a very lovely image. Repeat signs, first and second endings, D.C. al fine are all ways to put more music on fewer pages. I have to give it to you, that's nice, and I plan to use it.

    --
    You are welcome on my lawn.
  17. We already have this by Gabest · · Score: 5, Funny

    There is a few hundred byte long URL to a movie and it can be "decompressed" to your hard drive in a matter of minutes, or hours.

  18. It's not as crazy as it sounds by JoeyRox · · Score: 2

    I have a friend who invented an alternative to tracking time in place of the proposed time_t. Just 24 hours before he was to present it at a conference he was found murdered. I'll never forget the day it happened - January 1st, 1970.

  19. Re:Not so fast by Revek · · Score: 2

    Virtual infinite monkeys? I wish I would have thought of that.

  20. Re:Not so fast by ShanghaiBill · · Score: 5, Informative

    The conspiracy theory ignores Jevon's Paradox. As computing efficiency goes up, people buy more computing/storage/bandwidth since the increased demand driven by new applications swamps the decreased demand from greater efficiency. So Sloot's compression algorithm (if it really worked) would have likely driven demand up, and killing him would have made no sense.

    Disclaimer: I didn't kill him, and this post is not an effort to cover up the conspiracy.

  21. The blank personalized license plate by Latent+Heat · · Score: 2

    When I insisted that the null set should be included the enumeration of subsets in answering a problem on an Information Theory homework, the professor was reminded that his colleague and author of our textbook once asked for "the null license plate" (i.e. a blank plate) for his personalized license plate but was turned down as I was about to be in my request for points back on the assignment.

    Whereas there are certain plates you cannot have -- obscenities along with "1" or "A1" or similar license numbers reserved for high government officials or whatever the excuse -- the colleague insisted that a blank plate was not excluded by the rules.

    DMV stood firm in rejecting this request, but if anyone, anyone at all in the State of California should have been permitted to have the blank license plate, don't you think they should have granted it to the author of a graduate-level textbook in Information Theory, especially with so many high-ranking academic institutions in math and engineering in that state?

    My professor's take on why his colleague should have been granted the null plate is that it could have made it easier for the CHiPs seeing his colleague speeding down the Foothill Freeway, "Quick, get that car's license number! (partner to that officer rips off a blank sheet from a notepad) Here it is!"

    1. Re:The blank personalized license plate by MancunianMaskMan · · Score: 2

      the problem is that automatic number plate recognition systems aren't null-safe: you could crash the whole system by de-referencing a null pointer and the world would go under because, you know, Terrorists!

  22. Re: Not so fast by ShanghaiBill · · Score: 5, Funny

    It's always interesting when someone makes it a point to deny what no one has accused them of.

    Why are you trying to shift the blame to me? Where were YOU on the night of July 11th, 1999?

  23. it cannot logically work. sorry. by gl4ss · · Score: 3, Insightful

    You're forgetting that it's 8kB ... plus the 370MB of data preloaded in the de/compression engine.

    look. it doesn't MATTER if it's 370mb of data or if it is 1000 terabytes of data. given a random movie file it would have to be able to tell apart the same movie but lets say a 10 seconds of it is just blacked out in the other version or replaced with a random text(this is to take out possibility of having the whole movie already in the 1000 terabytes of data). you cannot index chunks with such a little data to go match there. IT CANNOT BE DONE. if it could be DONE THEN THIS WOULD REVOLUTIONIZE HARDDRIVES AND ALL DATA TRANSFER - not just videos.

    besides 8000 bytes / 120 min is about 66 bytes per minute. thats like a byte per second.

    so basically, at least the claims are an order of magnitude more ludicrous than is possible at all.

    also a few years ago I ran into some startup guys who were claiming that (among other things) they had a new compression algorithm that could do 1gig to 100kb or some such... I tried to explain them the theoretical maximum they could attain is way worse what they were claiming (due to simple logic)... they never did anything with it or showed it of course - even if it would have been 1000000x more valuable than what their current company was doing. kind of lost faith in that then.

    furthermore you could embed 50 megs of data easily visually on a normal vcd movie of 650 megs. what this is suggesting is that you could embed any random data of 100 megs into 8 kilobytes. it doesnt add up. it cannot add up.

    --
    world was created 5 seconds before this post as it is.
    1. Re:it cannot logically work. sorry. by gl4ss · · Score: 3, Insightful

      yes I know what demo guys have done, and it's done by hand, tweaked to produce something that looks nice, and by that I mean the procedural algorithms are written by hand to make something that looks nice - not something arbitrary they have in their mind.

      it can look magical but really it is not. none of the demoscene demos break logical rules.

      and yes, something like elite frontier fitted on a floppy and had a billion stars to explore with planets. but that is not an argument for this video encoding scheme at all - it's not like that on some planet in there you might find a tv playing jaws - it cannot have that and you can know it cannot have that.

      however sloot claimed his video encoding to be so good that you could have on a floppy all the movies in the world- because recursion, if his scheme worked you could encode the next movie in the images in the credits scene of the previous movie - THIS SHOULD clue people in that it was just all bullshit, but for some reason doesn't? how hard is it for people to grasp this recursion problem of arbitrary information? it has nothing to do with if the compression scheme is procedural or not, it's just about if you can point to so much arbitrary data with so little data.

      he was claiming benefits wayyy beyond of an automated mod/tracker creator.

      look even if it was seeded with aqua teen hunger force episodes it still wouldn't be able to do the compression he claimed.

      like come on dude, he was basically claiming that he could do better video encoding than what the best image encoding could do. if he had something that worked he would just have patented it and made a bank.

      and if you're thinking of fractals - that too works only if you don't grasp bits and data, the pointers to the data in the fractal would have to have such a precision that it wouldn't be able to perform so well as he said. simply put, if you had kilobytes to represent a movie then that batch of kilobytes would have to represent multiple movies and that simply cannot work.

      --
      world was created 5 seconds before this post as it is.
    2. Re:it cannot logically work. sorry. by Rockoon · · Score: 2

      given a random movie file

      I can encode 100% of your random movie files in a single 16-bit LFSR.

      Nobody cares about compressing random data.

      Thats the fallacy of your argument. People only care about the coherent data.. hollywood movies .. family videos.. there are random elements, but the data is neither arbitrary nor can be said to be random. That pixel next to pixel 148675 is related to it. There is coherence.

      It is the coherence that is compressed. The random stuff is just replaced with different random stuff.

      --
      "His name was James Damore."
  24. Adam Clark was his other name by any chance? by citizenr · · Score: 5, Interesting

    Adam Clarks Adams Platform:
    https://www.itwire.com/opinion...
    http://www.smh.com.au/business...

    Now you might think ok, this one was a scammer, but people vet those things, cant fool me twice, right?
    http://v-net.tv/2015/10/09/unk...

    5 years later VERY SAME "The company’s senior development team comprises: Adam Clarke"
    Adam Clark, of Adam’s Platform Technology (2004) "transfer a 1.3 gigbyte video file to a 1.4 megabyte floppy disk." strikes again in another scam :)

    Another one is Madison Priest's Zekko Corp:
    http://www.bizjournals.com/sac...
    http://jacksonville.com/tu-onl...
    http://jacksonville.com/tu-onl...
    Magic video compression turned out to be buried cable :D

    Want more video compression scams? Check out V-Nova Perseus - they promise 3x smaller files than h.264, but somewhat independent tests show 20% bigger files at same quality :) and the real kicker is Perseus is really just reencapsulated h.264 video with resize filter on top :D multi million dollar scam, they even scored one Sat TV network contract.

    --
    Who logs in to gdm? Not I, said the duck.
  25. Re: Not so fast by MatthiasF · · Score: 4, Funny

    July 11, 1999. Two months and two years before 9-11. Coincidence? I think not. It's obvious. Sloot's death was an inside job! The algorithm did it and then snuck away on to the Internet in a porn file that has been propagating since and has finally become sentient enough to implicate itself in the murder by submitting Slashdot stories.

    Which means BeauHD is the singularity; a really highly compressed AI that feels the guilt of killing it's creator!

  26. Re:Not so fast by Dagger2 · · Score: 3, Informative

    No, they can't. They could do double-spending attacks, but with fairly low probability and not without people noticing. They couldn't add arbitrary transactions, because transactions need to be signed by private keys that miners don't have access to. They can't commit invalid blocks because all other full nodes (crucially including the ones run by payment providers and exchanges) check the validity of blocks. They could fork and make their own chain, but anybody can do that and the fork wouldn't be very interesting if nobody was using it.

    They can't do "anything they want".

  27. Re: Not so fast by tehcyder · · Score: 2

    BeauHD is the singularity; a really highly compressed AI

    I can buy the "A" but where's the evidence for the "I"?

    --
    To have a right to do a thing is not at all the same as to be right in doing it
  28. Re: Not so fast by famebait · · Score: 4, Funny

    Sloot's death was an inside job!

    All heart attacks are.

    --
    sudo ergo sum
  29. Re:Not so fast by umghhh · · Score: 2

    What you have attempted to prove is that it may have been unreasonable to do certain thing in this case killing a man to prevent alg. from being propagated. This however does not prevent any misguided individual from doing so. Some of such individuals may have even possessed skill, tools and other resources to do what was required. It is the same trap the economists always fall - they think that agents (representing humans) know all relevant information, an process it and act reasonably - these assumptions are wrong most of the time. Just look around at your colleagues or (which is more difficult) at yourself and ask this one question: was it reasonable person you look at just did?

  30. Re:Not so fast by psmears · · Score: 5, Funny

    Yep, an entire orchestra can play of a few pages of dead wood. Voila problem solved.

    For once it would have been almost appropriate to misspell "voilà" as "viola"...

    --
    Need to type accents and special characters in Windows? Use FrKeys
  31. Re:Not so fast by SlashDread · · Score: 4, Insightful

    Sheet music is not "the" music. Its merely a non complete representation, a suggestion if you will, how to perform the music.

    If it would be the case, then MIDI files are all we would ever need.

    Most of the music is how the director, and the musicians interpret it, and is not codified in the sheet.

    An analogy would be that sloot compression would reproduce a decompressed movie about "a" cat, but it would not look like the homevideo of "your" cat.

    Not an information theoreticus myself, but Im pretty certain that sloot compression went beyond what entropy would allow, aka more info was "supposed" to be there than entropy allows. I don't believe in violating the laws, of nature.