Chinese Lab Speeds Through Genome Processing With GPUs
Eric Smalley writes "The world's largest genome sequencing center once needed four days to analyze data describing a human genome. Now it needs just six hours. The trick is servers built with graphics chips — the sort of processors that were originally designed to draw images on your personal computer. They're called graphics processing units, or GPUs — a term coined by chip giant Nvidia. This fall, BGI — a mega lab headquartered in Shenzhen, China — switched to servers that use GPUs built by Nvidia, and this slashed its genome analysis time by more than an order of magnitude."
Sounds like these newfangled "GPUs" are gonna change the world.
I always wondered what GPUs are. Thanks Slashdot!
Explaining what a GPU is in a slashdot summary? Come on.
This is similar to someone telling you a story about something funny happening to them while shopping at the store, pausing mid-story to inform you that a 'store' is a business where goods are displayed and exchanged for a papery substance called 'money'.
Submitter couldn't find a more technically-oriented one?
It reads like some Reader's Digest piece. I can't believe timothy published it like that. :)
It's hardly news that GPUs can be used to speed up parallel tasks/computations, but even so this article is a useful reminder of two things; 1) there are still many important processes that can be sped up by using GPUs, and 2) this can be achieved pretty much anywhere in the world.
I wonder if the AMD use of more cores, whereas Nvidia uses faster cores, would change the time. I have no idea how genetic algorithms work. I do know simple hashes like bitcoins are best on AMD.
Excerpt:
At BGI, he says, they are currently able to sequence 6 trillion base pairs per day and have a stored database totaling 20 PB.
The data deluge problem stems from an imbalance between the DNA sequencing technology and computer technology. According to Dr. Wang, using second-generation sequencing machines, genomes can now be mapped 50,000 times faster than just a decade ago. The technology on track to increase approximately 10-fold every 18 months. That is 5 times the rate of Moore's Law, and therein lies the problem.
Obviously it would be impractical to upgrade one's computational infrastructure at that rate, so BGI has turned to NVIDIA GPUs to accelerate the analytics end of the workflow. The architecture of the GPU is particularly suitable for DNA data crunching, thanks to its many simple cores and its high memory bandwidth.
This post contains no rudeness or derision of any kind. All arguments are friendly. Terms and exclusions may apply.
I get that programmers are offloading certain tasks to the GPU because they are able to perform specific tasks faster, but why is this even necessary. If the GPUs are so good at it then why can't there be a dedicated part of the CPU to perform these same computations in parallel streams the same way the GPU does?
SIMD chips will always show computational gains to any class of problem that makes significant use of matrix multiplication or linear algebra. So graphics, crypto, etc..
What word rhymes with buried alive?
So, a site dedicated to nerds needs to explain what a GPU is? Are we not nerds anymore?
Write boring code, not shiny code!
Yeah, Wired, what gives?
(Hint: the summary is a direct quote from the enterprise-y TFA.)
Bio questions? Ask me to start a Q&A journal. Computer analogies available for most topics!
And other assorted distro-computing tasks. Hell, my old x1800's stopped being supported for the current Folding software years ago.
A nice list of distro computing projects.
Another nice list of such projects.
Slashdot "libertarians": Small government for me, big government for those I disagree with. -1, I disagree with you
Genomic analysis involves extensive use of recursive techniques, which are well suited to parallel processing and combinatoric problems. GPU's are small independent components originally designed to handle large matrices of pixel elements for video programming very quickly for video display and refresh. Thus, they can when suitably programmed, for example using CUDA, in parallel to compute solutions required to map problems of high combinatoric dimensionality onto a one dimensional space (sequence) very quickly compared to a CPU that would require serial computation on an extremely large combinatoric space. Effectively, it puts massive supercomputers in the petabyte and exabye processing speeds to be built with standard components at modest prices.
The amazing thing about this technology and the responses of the supposedly technologically sophisticated responses on a site such as slashdot, is that the Chinese are picking up on the technology and on genomic data mining far faster and with more intensity than is the broader US tech community. Given the size of their brainpower base and the rate at which they are adapting the technology the Chinese are well on their way to dominating the drug development and physiological/functional genomic sciences in the next 10 years. The race will largely be over before most American tech types even know it happened.
The even more amazing thing is the potential of unlocking the genetics which control human intelligence, memory and learning capacity. Once these are patented and developed for a host of applications all other forms technology will become increasingly inconsequential. While the US is putting its most powerful computers to use cracking into and reading people's email, the Chinese have a more ambitious agenda.
Actually, this is a good thing, since the Chinese are far more cognizant of the dangers posed by imminent global warming due to carbon dioxide pollution to their economy and the the stability of their political system. If you have any doubts about the Chinese propensity to use their wits, I suggest you see the movie Red Cliff, which very dramatically displays the remarkable triumph of wits over shear military superiority. Its based on a true story.
"GPUs . . . ? . . . I was informed that this project was powered by GNUs . . . ?"
". . . now where is that Apple MAC chip that generates the GPL number that allows the PC to connect to the Internet . . . ?"
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Only if you have a Beowulf cluster of them!
Turns out I've been wrong about you all this time. It's the goats having sex with you isn't it?! You just love the burn of a hot goat cock in your tender brown bud don't you. HA, no lube for you!
Although at least in my field the problem is that no one ever thought to set lower limits on the quality of what you can call a genome. So now we get "genomes" made up of 100,000 contigs (many only a couple of hundred base pairs long) and even counting all of those, the total sequence might account for only 70% of the total size of the genome. But it's still a "genome" paper, which is still an instant ticket to Nature Genetics (or Nature Biotechnology if the assembly is REALLY bad).
BGI is certainly one of the biggest offenders (Cucumber and Pigeonpea are both examples of the sort of terrible genomes-in-name-only BGI puts out) but I think the real problem is that Illumina sequence data is so cheap people keep trying to use it to sequence genomes, thinking if they throw enough raw data and enough mate-pair libraries at the problem it'll eventually make up for the fact that Illumina reads are so short. Illumina data is great for a lot of things. Calling SNPs, measuring gene expression, studying methylation patterns.
But, at least for any genome significant transposon content, it simply does not work.
Just thing how cool it will be to have cards that can do this on the CPU BUS.
... this is what a Chinese lab looks like.
Ah, arrogance and stupidity, all in the same package. How efficient of you. -- Londo Mollari
According to Jackson Lab’s TeHennepe, the feat BGI and NVIDIA pulled off was porting key genome analysis tools to NVIDIA’s GPU architecture, a nontrivial accomplishment that the open source community and others have been working toward.
Can anyone familiar with current efforts shed more light on this? Who is working on open source bioinformatics and how much work has been done?
========== "Hello World" in my programming language of choice: ATG - LET THERE BE LIFE - TAG ==========
Commodore didn't. The C64 had the "VIC-II" (video interface chip II). The Amiga had Agnus and Denise and later the AGA.
I'm don't think Atari did either. Sure they had a Blitter chips and there was graphics accelerators and so on. But I don't think the term "GPU" was used.
I don't recall anyone arguing with it at the time either: http://www.tgdaily.com/hardware-brief/18947-nvidia-launches-worlds-first-gpu.
Still PU was common enough already.
"The world's largest genome sequencing center once needed four days to analyze data describing a human genome. Now it needs just six hours. The trick is servers built with...GPUs — a term coined by chip giant Nvidia. This fall, BGI — a mega lab headquartered in Shenzhen, China — switched to servers that use GPUs built by Nvidia, and this slashed its genome analysis time by more than an order of magnitude."
A search of Usenet reveals the Atari Jaguar had a unit called a "GPU" in 1993, considerably before NVIDIA's "first GPU" in 1999. The Amiga unit was also called a GPU.
The term's generic, and NVIDIA knows it... they don't have it registered as a trademark.
Except that genomics has as of yet proven minimally useful for drug development. Until they actually develop significant amounts of homegrown technology (which, to be fair, they are actually doing in the bioinformatics arena, as opposed to sequencing), I'm not convinced that they're that much of a threat.
What if they simply avoid competing by patenting the sequence for Caucasians and then pulling an Apple and suing us out of existence? ;^)
Orwell: "In a Time of Universal Deceit, telling the Truth is a Revolutionary Act"
That seems to be the general attitude across the board in the US, but it seems unlikely to be warranted any more. They keep growing their economy at between 8-10% per year. Estimates are their GDP will overtake that of the US in about 2025, if not sooner. The days of resting on laurels will have been gone by then. In any event if we are that far ahead, it seems hard to get a sense of that on slashdot judging from the sophistication of most comments.
Besides, I be curious to know what specific research has been falsified? Its not as if the US fossil fuels industry hasn't been doing the same here with respect to climate science.
First, I didn't read the AC's comment. Then I thought the FP in question was pretty relevant to the quality of the summary. Not a troll in my view and I explained why.
Of course, it depends your point of view I guess. My post was probably badly worded.
Write boring code, not shiny code!
Imagine if the calculus were processed in a FPGA, it would be another magnitude faster :P
i should really get a life.
clone53421 = stephen alongi... i am not stephen alongi... i am michael kristopeit. you don't understand the difference, because you're an idiot.
cower in my shadow some more, feeb.
you're completely pathetic.
Besides, I be curious to know what specific research has been falsified?
Here's a decent summary of the problem:
http://www.nature.com/news/2010/100112/full/463142a.html
Its not as if the US fossil fuels industry hasn't been doing the same here with respect to climate science.
"Microsoft has put out some faulty software, so I'm gonna buy my next operating system from VaporWare Inc"
They keep growing their economy at between 8-10% per year
That's part of the problem:
A new study from Wuhan University, for instance, estimates that the market for dubious science-publishing activities, such as ghostwriting papers on nonexistent research, was of the order of 1 billion renminbi (US$150 million) in 2009 - five times the amount in 2007.
clone53421 is stephen alongi. i am michael kristopeit. stephen alongi =/= michael kristopeit. you can't comprehend that simple application of the transitive property because you're an idiot.
cower in my shadow some more, feeb.
you're completely pathetic.
clone53421 is stephen alongi. i am michael kristopeit. stephen alongi =/= michael kristopeit. you can't comprehend that simple application of the transitive property because you're an idiot.
you're also an ignorant hypocrite.
cower in my shadow some more, feeb.
you're completely pathetic.
if you believe you're being attacked, you're even dumber than that... but considering i merely responded to your idiocy with a statement of fact concerning it's idiotic content, it's already clear you're an idiot and an ignorant hypocrite.
you are not defending yourself. you are denying the existence of yourself. you don't understand the difference because you're a moron.
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
you're completely pathetic.
The explanation isn't even correct. When I wrote a Super Nintendo emulator in the 90s, various documents referred to its GPU as a, er, GPU. The SNES predates NVidia itself, so let's not call it a "term coined by chip giant NVidia".
GPUs are nowhere near new. What's relatively new is GPGPUs (General-Purpose computation on Graphics Processing Units).
you're an ignorant hypocrite.
clone53421 = stephen alongi. i am michael kristopeit. stephen alongi is no-one's ma. michael kristopeit =/= stephen alongi... you don't understand the difference because you're an idiot.
cower in my shadow some more, feeb.
you're completely pathetic.
you're worthless.
you're worthless.
you're worthless.
alex, p, k?... is that what your mom tells you to when you've been alone in the basement too long without a potty break?
still living in the bushes in the middle of shantytown, alex?
you are NOTHING
you are NOTHING
cower in my shadow in your paid in full shanty some more, feeb.
you're completely pathetic.
cower in my shadow in your paid in full shanty some more, feeb.
you're completely pathetic.
LMAO -> http://www.youtube.com/watch?v=JrUGYT_gl9I = MikeK, lol...
APK
P.S.=> How many accounts is that you're up to now? Well, lol, let's see: I said 500 on a guess, & MichaelKristopeit412 looks like your 412th, lmao... I was wrong, but (rotflmao)... who cares?? Why??
Well - You're the "Man..." (See song above, lol, same quality as you, lol).. /quote)... apk
Troll fight! Humongous host file vs. pig fucker.
The only possible winner is - everybody else.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
cower in my shadow some more, feeb.
you're completely pathetic.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
cower in my shadow some more, feeb.
you're completely pathetic.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
you're an ignorant hypocrite.
having sex without consent is being a party to rape. you claimed, in quotes, that it was your "job"... who said it was your job? are you a registered sex worker? did you have a pimp? either you consented, and stated your desire to engage in wanted sexual contact, and later lied about it; or you were raped. are you a liar?
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
cower in my shadow some more, feeb.
did jay little regret you claiming that he libeled you? you said that he would too, but it seems his website is still up after many years, and no action was ever taken..., as well as many other individuals who you've threatened with libel suits that resulted in no action. you're a demonstrated liar. you're a moronic buffoon. you are completely worthless.
searching for "Alexander Peter Kowalski" on google responds with the top result "How to Respond When People Threaten to Sue You on the Web"... full of people mocking your pathetic attempts to invoke legal action where none is warranted. why do you think that is?
cower in my shadow some more, feeb.
you're completely pathetic.
Oh geez. For some reason that was posted anonymously. Old habits die hard...
By the way, sedans aren't sports cars.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
trolllolllollloll.
Wanna see the .40? Okay sure.
Distributed Denial of APK: It takes 15 seconds to reply to him anonymously, but wastes tons of his time if we all do it.
No, it can't. No mainstream operating system has ever shipped with a version of ping that outputs malformed ping packets, let alone ping packets malformed in a way that would cause a PoD. Ever. Exploiting the PoD requires a specially written tool to output a malformed ping packet. You cannot use the ping command to do that.
If I'm wrong, it's fairly easy to prove. Show me the bug report that reports to Microsoft, Canonical, or Apple, that the ping command shipped with their system outputs malformed ping packets. And in the mean time, demonstrate for me how a Windows 7 machine can crash an unpatched Windows 95 machine using the ping command. Don't just handwave and claim it's "fact", put up or shut up. Tell me the exact command line to issue from a Windows 7 Administrator cmd.exe window to crash a Windows 95 machine whose IP address is 10.0.0.1.
You won't because you can't. You can't because you don't know what you're talking about. You've never known what you're talking about. The same thing goes for the "Spybot SAD" hosts file modification thing. You don't understand why CA was asking about modifying HOSTS, which interestingly enough probably means you unintentionally fed false information to them about the operation of your application.
You claim your app isn't scriptable, yet also claim it's "two lines of code" which means it cannot possibly verify that the application its hiding is what you intended it to be. So it is scriptable, and it is usable by a malware writer. Which is probably why most malware organizations still report the software as malware.
Meanwhile you post here over and over again with increasingly ridiculous assertions about your work and knowledge, and cannot understand it when a bunch of us find it hilarious and needle you over it.
If you know of a way to use the ping command to exploit the PoD please post it here. More importantly, make sure Microsoft, Canonical/GNU, the various *BSD groups, and Apple are made aware of it immediately. If their ping utilities, as shipped, are outputting malformed packets, then they have major bugs in them. You would be utterly irresponsible not to let them know.
You are not alone. This is not normal. None of this is normal.