Slashdot Mirror
A Distributed DivX Ripper?
RJ asks: "I know much about Java/C++ and
sockets programming and I'd like to use this knowledge to build a distributed program to rip a DVD into DivX. It will work by breaking the job into chunks, sending chunks to other computers to encode, then patching it back together.
Despite my searching efforts, I have been unable to find a decent resource to teach me how to program the DivX core to encode mpeg2 and re-join parts. I'm hoping that some readers of slashdot can point me in the right direction?"
Oh yeah... Uh uh Ughhhh...
You know you want it baby, come and get it.
- Reading around 4 gigs of data in a fairly fast time: not much of an issue on a normal computer, but distributed computing might introduce issues with speeds of the network.
- Deencoding the mpeg2: this seems to me to be the biggest slow down. The problem seems to one of not having a fast enough decoder that is capable of good image output. Distributed computing would help, but it seems to me to be somewhat of a macgyvered solution to the real problem.
- Resizing, cropping etc: not much of an issue, but of course distributed computing would help.
- Mpeg4 encoding: Again, one of the major problems seems to be with fairly slow encoding alogarithims. Therefore, helped by distributed computing.
I'm sure setting up a beowulf cluster of computers or some other setup seems to be nice and all that, but unless you're reencoding several dvds a day, it seems like a normal athlon would probably do the trick, and working on the code for better decoding and coding of the various codec would seem to me to be a more efficient way of improving speed. I know that I can reencode a dvd at half real time with 1-pass divx 5 encoding and flaskmpeg, and a 1/4 real time with 2-pass encoding, and I've only got a Athlon 1.4. So unless you are reencoding Das Boot quite a bit, it seems to me like one normal high powered consumer computer would be plenty sufficient.Do not go gentle into that good night. Rage, rage against the dying of the light.
Taco, I want you to fuck me in the ass please. I am dying to be anally accosted. I want to be ravaged like hog. I want you to dress like a farmer and make me oink like a pig. I want an ass reaming like no other. Taco, I haven't had this kind of lust for you since the crazy college days. We used to butt fuck each other in the stalls. You always told me not to flush and preferred using my feces as apposed to real lubricant. I remember your chocolaty member, your manhood, draped in my feces. Man, Robbie, I remember. I was day dreaming, escaping into a nether world where we used to fornicate, and live in fornicatory bliss. You used to like to keep your tubes socks on to enhance they gay look. We were so flitty and light on out feet. I am so very confused these days. I have difficulty conceptualizing the time that was then in contrast to now. I mean, first you were a raging homosexual, now you deprecate me in favor of this "woman." I know that bitch is a transvestite. You are closeting your homosexuality and denying your roots in my ass!
I am destabilizing. The world is going dark to me. I have scintillating threads of motley thoughts, my ability to control my self evanesces away! I have only an adamantine desire to see your balloon knot once again, and to have you ravage mine! I see a world of GOATS. A goat fucking extravaganza. I invoke the ANUS of DOOM! I hate Taco.
SON of the GOAT, HUGE ASS WIDENER, This is a massive, massive ass attack from the Minister of Goat, Ayatollah man-meat.
Dilated Meat Pie. Most suppressed people really like seeing this. It gives them new masturbatory fodder.
Two cucumbers, better than one. This is to show that the giver is really smaller than what is needed to fill GOATSE man. He east Cheerioatse brand O's
A Disney product right where it belongs. Up a goat's ass. Death to Mike Eisner, the butt buddy of Commander Tak0.
Raw and dilated man-pussy. Put back the trouser snake, Tako. You dick is way too small for this man's ass.
A Prolapsed rectum is sure to whet even the most jaded flaming fuck's appetite. Tak0, your penis is regrettably way too small, even for your "Fiancée's" unfettered anus. She doesn't want to dirty her ass with the likes of your pathetic member.
GOAT KORAN
Classic HIT ME IN THE SHITTER BABY, UNGH HUH
Classic Oh yeah, in the shitter some more, in the shitter.
Classic More ass stretching goodness.
Female Goater My pussy is too small for this APPLE.
Goatse Grandpas - GRANPA GOAT S3X0R5
Son of a Goat - Holy fucking son of a goat. Kind of looks like Tako from behind, but to be sure I'd have to ask CowGryl Kneel
1 Oh, pardon me sir, would you happen to have any ANAL LUBE?
2 UNGH FART, pssssbt, ungh, tweeep, squeaaaaaak ungh
3 PFFFFFFFFFFT AHH pffft
4 FOOOOOOOOOOOOF blud dribble dribble
Prime Number Shitting Goatse Man See The Prime numbers flow like the river SHIT
Goatse Returns! Fuck yeah, the goat man is a coming back to Trollaxor
I summon the powers of HUGE GAPING ASS!
Our Goatse Prayer
Llegaré a ser petador, el mejor que habrá jamás.
Mi causa es ser reventador, tras mi gran petada anal.
Petaré cualquier animal llegaré a partirlo en dos.
Al fin podré agujerear el culo de Goatsemon. (Es Goatsemon)
Petalos todos. Es mi destino, mi misión. (Es Goatsemon)
Tú eres mi amigo fiel, pero deja ya mi esfinter. (Es Goatsemon)
Petalos todos. Seremos mejor al fin.
Te peto yo, y tú a mí Goatsemon.
Hazte con todos. Goatsemon..
Please, it's "decode," not "deencode."
Like my stupid high cshool programming teacher who yankee-ily kept saying "disenable," when it's really "disable," you don't have to include BOTH positive and negative prefixes -- only one.
Anyway, you're pretty much right. Though I think the original intent was a bit more distributed than a cluster (wan), since such a thing (with oh, just a couple athlons) is beyond most would-be rippers, I'd assume.
However, if you had a good sized pool of cohorts, and they all had cable modems that can sustain 100k+ between them boh ways, then the networking part seems more feasible than one might at first think. But it's not something I'd release and let and old Joe 56k try, of course, but I wouldn't (personally) consider it McGyver-ish.
I've tried to do something similar before -- namely, breaking up .wav files so that I could distribute the pieces to other machines to encode.
.avi / .vob in that sense.
I ran into a snag.
It seems the encoder I was using at the time (bladeenc) was inserting silence at the end of each mp3, to keep it to spec. What I can imagine is that even with DVD encoding, you'd need a "master" that would give out file chunks to the worker bees. But -- it would have to be intelligent enough to know when you wanted a new keyframe, and split up the
In other words, you may as well just build one heck of a fast machine, and try to get 30-40fps encoding out of it, rather than try to put together something to distribute it and encode it. That's my 2 cents, and I may be wrong....
Karnal
Watch this post get modded up, and not my qualified response to the From Coder to Game Designer question. Humbug!
Anyway, as brought up in the last Ask Slashdot remotely similar to this one (Archiving DVD's with Linux), dvd::rip, which is a Perl+GTK front-end to transcode, has a fairly insecure cluster mode, whereby it will split up the video transcoding task among however many machines you can coerce into doing it, and rip and mux the audio with the video on the host machine.
Sounds like just what the doctor ordered. Now someone go mod up that other answer of mine. Please?
> I know much about Java/C++ and sockets
That's impressive, but did you know that BSD is dying ?
You want us to help you break the law?!? What do you think we are?!
"Remember when the U.S. had a drug problem, and then we declared a War On Drugs, and now you can't buy drugs anymore?"
Yeah, i've been thinking of something like that for bit now. On a local network, you wouldn't have to worry much about speeds (some guy above said that). One of the things i was thinking of was more of a distributed encoding system, that on the servers (that the chunks are being sent to) could have sorta like plugins for different media types...so you could encode many types at once.
cat divx2.avi >> divx1.avi
when developers have put years of work /etc/passwd and SSL
in PVM or MPI. I do not know if
"mpi_allgather" and "mpi_allscatter"
would stand an 2gb array like found on DVDs,
but at least this would put several 1M$
beowulves I know of to a somewhat useful
purpose (besides cracking
sniffs, of course), instead of boring
quantum chemical computations or climate simulations.
Google passes Turing test : see my journal
The Ask /. section seems like it's generally moderation-neglected, particularly when the article doesn't make the front page (like the Programmer To Designer article).
Vidomi is a badass little program to turn mpg, vob, ... into DivX. One of the recently added features is "Distributed Encoding" (read: Scalability via network slaves).
This answer your question?
Why don't you pay for the DVD, instead of trying to copy it from somebody else?
If it's worth enough for you to expend the effort to make a copy, it's obviously worth enough for you to spend your allowance on it. Head over to Half.com or another retailer of used DVDs if you don't think it's worth the cover price, but by all means don't steal it. You're giving open source users a bad name.
Well, I'm not sure exactly how DivX compression works, but in general, the more data the compression algorithm can see at once, the more redundancy it can find, and therefore the more it can compress. Chopping a piece of data up into bits will at some point start reducing the seen compression ratio.
It's 10 PM. Do you know if you're un-American?
Obviously, the ripping part can't be broken up - But if you get a good encoder it should be SMP capable. Then all you need is to use MOSIX.
It seems the encoder I was using at the time (bladeenc) was inserting silence at the end of each mp3, to keep it to spec.
Dedicate one machine to running LAME on the audio, and you won't have this problem.
Will I retire or break 10K?
The hardest part is load balancing. How do you make sure the slowest computer doesn't get the last job and force the faster computers to wait forever for the last job?
There's a set of utilities called transcode, that already does something similar, perhaps you should take a look at that. Just do a google search for transcode.