LHC Data Generation Expected To Scale Up To 400PB a Year

DW100 writes: Cern has said it expects its experiments with the Large Hadron Collider to generate as much as 400PB of information per year by 2023 as the scope of its work continues to expand. Currently LHC experiments have generated an archive of 100PB and this is growing by 27PB per year. Cern infrastructure manager Tim Bell, speaking at the OpenStack Summit in Paris, said the organization is using OpenStack to underpin this huge data growth, hoping it can handle such vast reams of potentially universe-altering information.

Comments

universe-altering information?

[Racemaniac]

you mean how we see the universe? because i doubt the universe cares much about the data we generate....

Re:universe-altering information?

[Anonymous Coward]

I question this stuff, only because scientists are expecting Einstein and Higgs to be correct. And that is the very problem with science anymore, you shouldn't be expecting anything, but looking for something that has nothing to do with previous human theories. On top of that what in the hell does any of this have to do with the current downfall of society?

OFF TOPIC
I'm not sorry for the way I feel, and /. is only proving the downfall of society by reporting on stupid shit, and fucking NON-free beer. Where in the do you get free beer, or believe you have freedom in the US? (home brewery, before people say it, and F**k beer /. needs to grow a set, drink some real alcohol, never drank beer ANY of it, it smells like urine)

Re:universe-altering information?

[Anonymous Coward]

They mean like tearing space-time and destroying it incidentally. Also, you may like the live CERN webcam. http://www.cyriak.co.uk/lhc/lhc-webcams.html

Re:universe-altering information?

[Anonymous Coward]

Unless the data is so heavy, that they warp space.

Re:universe-altering information?

[Urkki]

Well, universe damn well should care! Once we trigger the next phase change in the currently metastable quantum space, there'll be no going back. There won't even be time so even talking about going back makes no sense.

It's a bit like kids, once you get one started, you won't have much time (or any, depending on your moral values and local laws) to get rid of it, before it will trigger irreversible phase change in your life.

If only someone had explained these things to our universe, before it started to experiment with rocky planets. Once you go that far, accidents start happening.

Re:universe-altering information?

[Thanshin]

Technically speaking, everything we generate, including data, alters the universe.

Re:universe-altering information?

[Anonymous Coward]

You can't always redo every study that the current experiment is based on, but if you find that something does not match, then you have to backtrack your assumptions.

Re:universe-altering information?

[Anonymous Coward]

You can't always redo every study that the current experiment is based on, but if you find that something does not match, then you have to backtrack your assumptions.

Or blame it on measurement errors.

Re:universe-altering information?

[Anonymous Coward]

I question this stuff, only because scientists are expecting Einstein and Higgs to be correct.

Err... what?

And that is the very problem with science anymore [...]

You mean that problem you just made up and isn't true?

[...] you shouldn't be expecting anything, but looking for something that has nothing to do with previous human theories.

Like testing unexplored realms where the theory might fail? Such as higher energies? Like the realm the LHC was built to explore? And upgraded to explore more?

Re:universe-altering information?

[Anonymous Coward]

Technically speaking, everything we generate, including data, alters the universe.

He stop altering the universe with your posts and thoughts....

Re:universe-altering information?

[necro81]

because i doubt the universe cares much about the data we generate....

Oh, I don't know. Eventually we'll have so many hard drives dedicated to it that it'll collapse into a black hole.

Or - wait for it - the computing power requirements scale so large that the only way to keep the whole enterprise going is to build a Dyson sphere.

Maybe the universe won't care even then, but we'll at least come closer to leaving our mark!

Re:universe-altering information?

[rvw]

Unless the data is so heavy, that they warp space.

HEAVY! Space warps!

Re:universe-altering information?

[Anonymous Coward]

Well that rather depends on whether you buy the whole "information has mass" argument (google it). Still, even if it does, I suspect you'd need a smidge more than a measly few hundred petabytes to make space curve noticeably.

Re:universe-altering information?

[Anonymous Coward]

Oh, I don't know. Eventually we'll have so many hard drives dedicated to it that it'll collapse into a black hole.

It will burn out from all the human generated heat first.
Universal warming! Do something NOW before it is too late!

Ohh and ThinkOfTheChildren!

Re:universe-altering information?

[Anonymous Coward]

Sure it does. *we* are the universe in the process becoming self aware.

Re:universe-altering information?

[Anonymous Coward]

I question this stuff, only because scientists are expecting Einstein and Higgs to be correct.

Which is why there are studies checking for alternative models without a Higgs mechanism? You might have at least noticed in the news that a lot of effort was spent to see if they could even find a Higgs particle with a certain set of properties, and tests were made that the particle they found actually matched those properties. There are even studies to find violation of Lorentz invariance, which would be looking to see if Einstein was wrong, along with other fundamental tests of basic ideas in QM and QFT.

Re:universe-altering information?

[Anonymous Coward]

To be fair, some people think the universe cares about whether they think good or evil so...

Re:universe-altering information?

[lymond01]

I sat through a lecture on the Higgs Boson. It explained why they were expecting it -- basically the final jigsaw puzzle piece to a long-time theory. If the theory was correct, they would be able to find the Higgs Boson at certain energy levels. If they didn't find it, then it's back to the drawing board to figure out what they missed. So no, they weren't necessarily doing basic "Let's ram particles together and see what we get" science -- we've been doing that for decades. This was more of a "If we ram these particles together at this velocity, this is what we should get". And we got it.

Re:universe-altering information?

[Krymzn]

Changing our understanding changes our brains, which are part of the universe. Then the knowledge in those brains can be used to change the universe.

Re:universe-altering information?

[MightyYar]

you shouldn't be expecting anything, but looking for something that has nothing to do with previous human theories.

I can't agree with you. Science is completely dependent on falsifying theories. If Einstein made a theory, then it is highly scientific to try and falsify that theory - especially at it's margins and where results have never been confirmed. If it holds up, great! He's right again. If it fails, great! Time for an improved theory. That's the fun thing about science, is that the learning about the natural world never ends. Any result of an experiment always leads to more work, more learning. And no one says what you have to work on... you can work your ass off trying to falsify Einstein, or you can try to come up with a falsifiable string theory and test that instead.

Also, beer is delicious.

Compared to Facebook

[pmontra]

To put this in perspective, Facebook states to be generating 4 PB per day, so 3.6 times more than the LHC. Does anybody know about anything generating more data than that?

Re:Compared to Facebook

[Bob_Who]

Reality.

Re:Compared to Facebook

[Anonymous Coward]

NSA, they generate all the data of everyone combined.

Re:Compared to Facebook

[Guspaz]

Facebook is generating 4 PB per day *now*, while the LHC will be generating 400PB per year by *2023*. 27PB to 400PB in 9 years is MUCH slower than Moore's Law, so their annual storage costs/space requirements will decrease each year.

With the highest density servers I know of (1U 136TB SSD servers), LHC generates around five racks of data per year today. By 2023, they will only be generating around one rack of data per year, based on an 18-month Moore's Law.

Re:Compared to Facebook

[Anonymous Coward]

To put this in perspective, Facebook states to be generating 4 PB per day, so 3.6 times more than the LHC. Does anybody know about anything generating more data than that?

Google as an umbrella corporation?
Youtube claim to get 100s of hours uploaded every minute.
Not to mention that google indexs most of the public parts of facebook at least.

Re:Compared to Facebook

[CrankyFool]

I'm not doubting or challenging you, but I'm interested in knowing about your 1U 136TB SSD servers. Can you suggest some specs?

The highest-density boxes I get to have some familiarity with are Netflix's OpenConnect caches, described at https://openconnect.itp.netfli... -- where it's mentioned that they fit 36 6TB drives in a 2U chassis, for a total of 216TB, or 108TB/U. You're beating that, and with SSDs, which is ... impressive.

Re: Compared to Facebook

[Anonymous Coward]

Skyera SkyHawk.

http://www.anandtech.com/show/8658/skyera-releases-skyhawk-fs-allflash-array-up-to-136tb-of-nand-in-1u

Re:Compared to Facebook

[elbonia]

I believe these are the boxes that are being used.

Re:Compared to Facebook

[elbonia]

The 4PB they generate is actually highly compressible and they are most likely referring to raw logs. According to the page which quotes the 4 PB figure; FB states they have 300 petabytes of data in 800,000 tables in total. Since that's about 75 days worth of data and FB has been around far longer than that they are most likely referring to logs. So the actual disk space they need per day is far less than 4PB.

Re:Compared to Facebook

[behrooz0az]

Just about $400K for their 136TB 1U server. I don't think anyone needs any more detailed specs than that.

Re:Compared to Facebook

[Zocalo]

The porn industry would be a good bet, but on a more serious note the SKA will easily exceed that, once it becomes operational. Lots of radio telescopes, listening on lots of frequencies, and able to run 24/7... I found a few articles on likely data sizes and some of the figures are insanely large - the test size, a mere 1% of the size that final project project will reach, spits out raw data at 60Tbit/s and even after compression that's 1GByte/s. Figures for the completed array are in the range of 1EByte of compressed data (1EByte/day uncompressed) every two weeks or so, or about 70PBytes/day, so even allowing for further data aggregation they are going to be *well* beyond Facebook and the LHC in terms of daily data production.

What's missing from the stuff I looked at was any indication of long term storage capacities for retained data. I get the impression that 70PBytes is then processed and a smaller data set is actually retained, but couldn't find any hard figures on how much volume that the processed data might have, which is the value I think should be used for comparison purposes with Facebook. Maybe someone familiar with the project is reading and can fill in the blanks?

Re:Compared to Facebook

[Thanshin]

So, the LHC should just create a Facebook profile and store all the data on steganographied selfies and baby pictures.

Re:Compared to Facebook

[Anonymous Coward]

I thought Moores law was becoming obselete, at some point it will get to the stage when you physically cannot squeeze anything more on to a chip, no matter how small you make it.

Re:Compared to Facebook

[Anonymous Coward]

Good point, but I wonder whether the NSA is backing-up the LHC too...

Re:Compared to Facebook

[Anonymous Coward]

The LHC has to make sure that all the collected data stays available for decades, does this also apply to facebook states?

Re:Compared to Facebook

[Danathar]

Very true, but consider the sources and what is generating it.

Facebook is a large percentage of the Internet.

Cern is ONE project (with multiple experiments).

Also, this data has to be ARCHIVED and ACCESSIBLE for all time so that scientists can go back and compare/research past experiments.

Although I'm sure facebook is archiving a large portion of data, I doubt they archive ALL of it for all time.

Re:Compared to Facebook

[tibit]

For that kind of money, you could hire an equivalent of two full-time engineers for a year to design that thing from the scratch for you, and you'd probably get a couple of production units out of that deal, too. You'd need an ME to do thermal and case design, shouldn't take longer than a couple months for that. An EE to do any custom mezzanine boards that one might need + wiring and overall electrical design. Finally, a software guy to make a config console etc. I assume that project management is not included in this. Overall, a startup with people who know what the heck they're doing would need $500k to pull it off, if they're any good at it.

Re:Compared to Facebook

[Anonymous Coward]

I think SKA will have similar setup as LHC. I.e - there will be heavy filtering on several levels to get only "interesting" data in the end. See LHC example

Disclaimer: Posting AC since affiliated with CERN storage.

Re:Compared to Facebook

[mlts]

On the cheap, there are always Backblaze's storage pods. They take up more than 1 RU, but for something that is about $10,000 in price, the price is right.

This is tier 3 storage, though. If you want actual enterprise-grade stuff, it costs a lot more, but it will come with enterprise-grade performance and enterprise-grade warranties.

Of course, for long term storage for a lot of data, it is hard to beat LTO-6 for I/O speed and cheap capacity. After the drives and silos are in, if another PB is needed, that is only $20,000 (assuming $50 per tape, which will be a lot less for large orders,) perhaps far fewer tapes required, if the data is at all compressible.

By the time the LHC is generating data, IBM and Quantum should have LTO9 (25 TB native) or LTO10 (48 TB native) ready to go. Yes, it is still a lot of tapes, but it can be done.

Re:Compared to Facebook

[Guspaz]

Eventually, sure, but NAND hasn't reached that point yet, and we're just starting to see 3D flash memory hit the market, offering dramatic increases in density.

Oracle

[DogDude]

I doubt that OpenStack can handle it, but if they have the $$ for it, I'm sure that it's no big deal for Oracle.

Re:Oracle

[Ritz_Just_Ritz]

OpenStack is simply a cloud framework. What does any of that have to do with Oracle? In any case, this would be a great test case for a ginormous ceph cluster. I use ceph in conjunction with approximately 10PB of storage and am looking to increase that by at least an order of magnitude over the next year or two.

More info on ceph: http://en.wikipedia.org/wiki/C...

Re:Oracle

[Anonymous Coward]

Not sure if you're serious.... but really... they should go for MongoDB... It's Web Scale...!

Compress it

[Anonymous Coward]

Unless they are capturing pure noise, it should be compressable.

Re:Compress it

[Urkki]

Unless they are capturing pure noise, it should be compressable.

Well, it's raw data from physical instruments. It probably is quite noisy, so it will not compress much, except with jpeg-like lossy compression, but that would kinda defeat the purpose I think.

Re:Compress it

[Anonymous Coward]

The raw data rate that's generated by the particle detectors themselves is unreal. Based on some poorly remembered numbers of the this-many-of-that variety I think it's in the region of 10TB/second: 144 2.5Gsps 8 bit channels per card, a few dozen cards per cartridge, and some dozens of cartridges that run like a ring around the ATLAS detector's front.

The first level trigger/filter rejects the 99.5% of events that are boring and dumb (two protons strike a glancing blow and emit photons; Two protons' quarks exchange a gluon and fire out jets, ...). The second level trigger does more detailed analysis of a torrent of data somewhere in the GB/second. The final stage records the best-available-reconstruction data on several hundred events per second and I think it ends up around a few 100MB/s or less.

The recorded data will not be appreciably further compressible; Events are just a list of the origin, 4-momenta and tagged types of particles associated with each event.

It's already undergone its first refit/upgrade coincident with the magnet re-engineering shutdown: Increased luminosity, increased energy mean more events per crossing (from ~6 to ~20), requiring faster particle trackers and faster event readout (to increase from 400 to 1000Hz at final to-disk stage).

The second shutdown/refit around 2020 is planned to increase the luminosity even further, resulting in many dozens of interactions per crossing and requiring even faster electronics. I believe they're going to completely replace the ATLAS inner tracking detector because by that point it'll have absorbed over one million rads of radiation...

piece of glass

[Anonymous Coward]

so they used to take photos of these (forced) collisions.
now they're converting the data to zeros and Ones and "printing" them on magnetic platters.
now the question, when do the two curves, that is photo printing and platter printing converge?
when might it be feasible again to store it on media you can examine under a microscope?
pretty sure you can forget all about difficult computer algorithms to pull and compare data, if you
can just overlay two, three layers of glass under the microscope and instantly SEE!

Just to be pendantic...

[Anonymous Coward]

(Disclaimer: Grad student)

ATLAS generates O(PB) of raw data per second, but we only trigger on events that look interesting (e.g. have an isolated lepton, a sign that something more than QCD background happened in the event), and save those for offline analysis. That works out to something on the order of 100s of MBps being saved during run time. I assume the other experiments have similar data rates.

Solved problem

[Anonymous Coward]

A few hundred of these should do: https://www.backblaze.com/blog/backblaze-storage-pod-4/

  (3387 + 180000*0.0517)/0.18 = $70000/PB , For the already existing data it's 7 million, easily within the reach of CERN.

Of course that means very low reliability, but adding redundancy would still keep the cost around 10 megabucks.

Re:Solved problem

[DaCo]

The Backblaze Pods are probably way too slow for this sort of stuff, there's also no support. At least they can get that from RH.

Re: Solved problem

[Anonymous Coward]

I think it gets "close". Every pod has a gigabit connection to the outside world. If you write in parallel to hundreds pods, and every pod can shovel the full gigabit (minus overheads) into the platters, we're looking at ~100 Gb/s. That's still shy of the TB/s that popped up elsewhere in the submission, but suppose we bump the interconnect to 10Gb Ethernet, and make it a few thousand pods, then it gets in the same ballpark.

CERN probably isn't so cost-sensitive to go with this kind of barebones, DIY solution, but if some other business or research needs to store PBs of data, writing at ~Tb/s, and do it on the cheap, the pods, or some beefed up version of them, seem fairly attractive.

Re: Solved problem

[Anonymous Coward]

The new version of the Backblaze Pod (v4) is faster and it ha two Gbs ports.
https://www.backblaze.com/blog/backblaze-storage-pod-4/

Compared to Facebook

[Anonymous Coward]

But the LHC will be generating original data...

What do they store it all on? Or rather archive

[Anonymous Coward]

How do they archive all that data?

Seriously, all that data is being generated, how are they storing it? Standard magnetic tape backup?

A new theory

[justthinkit]

The LHC is one approach. The "make it bigger and then it might get better" approach.

Another approach is to conceive of a completely different model. I have come up with a different model from the Standard Model, and Quantum Mechanics, and String Theory.

Spring-And-Loop Theory resolves issues the other three theories are stuck on. It is also simpler. Unifies the four forces. And works from the very small to the very large.

But it is a different approach. Most are not ready for this.

Re:A new theory

[Anonymous Coward]

Coward downmods. Why don't you try to say something about my theory? Pro or con, I don't care. But no one does this. I think it is because my theory doesn't have the major flaws other theories have.

Jealousy is perhaps the ugliest human emotion.

Re:A new theory

[Roger W Moore]

A word to the wise: you might have better luck as a theoretical physicist if you actually knew some physics. For example the Standard Model is based on quantum mechanics: QM is not a separate theory. Also most of the questions you ask in chapter 1 of you linked site have already been answered by physics e.g. we know exactly why light slows down in a medium.

If you want people to take you seriously then you need to show that you understand the current knowledge of physics. If you cannot do that then how can you possibly really understand any of the problems with it? Popular science articles do not contain anything close to the detailed understanding you need.

Re:A new theory

[Anonymous Coward]

But it is a different approach. Most are not ready for this.

Even if right, you're not ready for it, unless I missed pages with precise calculations of various values current theories can predict to high accuracy. Random ideas of how to do things differently are kind of a dime a dozen, to the point most physicists in such fields have more ideas than they have grad students to do the grunt work of getting quantitative predictions out of it. And some numeric predictions are easy to make, considering how simple the Rydberg formula for hydrogen-like spectral lines is, it isn't too hard to stumble into alternative ideas that will produce similar formulas. I get about two new ones a week in my inbox that get at least that far. But finding ones that can work out the hyperfine splitting or make other detailed predictions of magnetic dipole moment for other particles is much, much rarer, and tend to be tailored to a single prediction. Too many people don't realize the vast number of predictions made by current theories that have been tested by measurement, and then think it is physicists' closed mindedness that is their problem when their theory can only tackle a very small subset of those.

Re:A new theory

[justthinkit]

Roger,

Thanks for your reply.

Regarding my mention of the Standard Model, you are quite right. Please replace that with "classic physics" or "old physics" or "non-relativistic non-quantum-mechanical" physics, or "the physics of Newton". It doesn't change my point, or my theory one iota.

As to my comments about light, in my first COASALT talk, Mr. David Thornley made this very same observation. It is rather like complaining that the numbers I chose for the street address of my building are a little too big, or too small. As I explained to Mr. Thornley, that was part of my preamble. Why don't you try commenting on my theory itself?

And as to my points about what we don't know about the SoL, it is like my points regarding gravity. When we don't know how gravity works...at the lowest level...then we have no guarantee that our theories of gravity (non-relativistic, or relativistic) are the complete answer. Similary, with light I maintain that physicists do not understand what exactly light is. We are aware it slows down in a medium, but we don't understand WHAT exactly it is. A working theory is not the same as a deeper understanding.

Re:A new theory

[justthinkit]

(1) You want specific predictions? Hard numbers? Here you go.

(2) As to "new theories are a dime a dozen, I get two new ones a week." Do you ever ask yourself why this is so? Do chemists get two new theories of chemistry a week? No. Because they have a good base model. I maintain that physics lacks a good base model.

(3) Too many people don't realize the vast number of predictions made by current theories that have been tested by measurement

The most obviously broken parts of physics, like the inflationary miracle after the Big Bang, are based on what measurements, exactly? The CMB? The same CMB that BICEP 2 based its nonsense on?

And Black Hole information retention is based on...?

And our completely broken notion of how stars should be orbiting the Black Hole at the center of our galaxy is a confirmation of our theory? Surely you gest.

Wikipedia's list of Unsolved Problems in Physics has, by my count, 148 questions (and another 74 things that need to be discussed). Wiki's corresponding U.P. in Chemistry looks to have 25 or 30.

There are at least a few modern physicists trying to deal with the horrendous state of physics today -- Lee Smolin, Frank Close, Peter Woit, and Amit Goswani come immediately to mind. Others like Anton Z Capri (& Feynman & Einstein) at least kept their sense of humor throughout their career.

Far too many are followers, and the system encourages this, big time. Lee Smolin talked about this, and how he tried to go against his gut at first, before ultimately coming out with Loop Quantum Gravity.

So is it all a bed of roses to you, "Roger"? Or have you a better theory? Or are you just interested in nitpicking?

Re:A new theory

[justthinkit]

Sorry. Please replace "Roger" with "AC".

Re:A new theory

[Anonymous Coward]

(1) You want specific predictions?....

That was a single calculation, yet you are claiming vast implications. You briefly discuss implications on the impact on the structure of an atom, then you should be able to discuss calculations on the atomic spectrum. No need then to ramble on about more complex issues or problems associated with measurements in deep space etc., but instead would work with data that at the simplest level can be collected in a high school physics course lab, although has been done in detail to very high precision in better equipped labs.

like the inflationary miracle after the Big Bang, are based on what measurements, exactly? The CMB? The same CMB that BICEP 2 based its nonsense on?

Entire books have been written on the backgrounds of such experiments, and there is already a lot written on them freely available online, repeating any of that ad nauseam here wouldn't be worth anyone's time. Yes, they are built upon several layers of theories, but if your new theory is so fundamental, you're wasting time with the more abstract stuff when you could give concrete examples and results for much more straightforward things. At some point, people start to wonder if someone proposing a new theory is just more interested in rambling and ranting than actually proposing a new theory and taking the basic steps to checking their own theory.

Do chemists get two new theories of chemistry a week?

By the way, yes, some do, as I know quite a few people in the chemistry department, and there are plenty of cold-emails about new theories of the atom or ways of bypassing conservation of energy just using chemistry, or that chemistry is fundamentally wrong and that there is some other energy source that can be tapped via basic chemical reactions. The number of emails any particular person gets from cold contacts in a department seems random though, with some getting none and others getting lots, as the senders seem to randomly copy paste sections of the department directory (you can sometimes see where they chose just a random section of the alphabet and then stopped). At least these days it is easier for people to put together an electrical circuit, misunderstand it, and think they have an overunity device than to put together a chemistry experiment, which does bias the numbers for some topics regardless of the state of modern research.

Re:A new theory

[justthinkit]

(1) the most obvious thing you are missing is that my calculation and prediction is something that can be measured. Period. "Vast implications"? Who cares. Is it right or wrong is all that matters?

(2) Books and books have indeed been written about the Big Bang, etc. And I imagine in Ptolemy's time the same was true. It was certainly true with Newton. And of course Einstein. While String Theory probably caused bookstores to open up whole new wings. Is "number of books" your metric?

if your new theory is so fundamental, you're wasting time with the more abstract stuff when you could give concrete examples

How many concrete things did GR have, at the start, that physicists could rush out and test? Eclipse data is all I am aware of, at least for the first x years.

I think your comment mostly reflects how you find my theory "shocking"...to you. It is different, very different. I'll give you that. But does it ring?

By the way, I bet these two sentences are ones you wish you could have back:
You briefly discuss implications on the impact on the structure of an atom, then you should be able to discuss calculations on the atomic spectrum. No need then to ramble on about more complex issues or problems associated with measurements in deep space etc., but instead would work with data that at the simplest level can be collected in a high school physics course lab, although has been done in detail to very high precision in better equipped labs.

The first sentence, about how I should be immediately working on "atomic spectrum" stuff, directly contradicts your second sentence. Which assumes I have a well stocked research lab that I should be busy working in...but that would be pointless because others have "done in detail"...my theory?!

Spring-And-Loop Theory came about by my trying to understand how gravity, an "attractive" force, worked across empty space. It is not my fault that it has had other repercussions. I am not deliberately trying to jump all over the place with it. But it is a reimagining of physics, no two ways about it. Again, some will have a harder time with this than others.

Re:A new theory

[Anonymous Coward]

"Do chemists get two new theories of chemistry a week?"

With the quantum mechanics and atomic structure as the foundations of modern chemistry, I'm not sure why you would think chemists don't deal with that. I could have a special category just for those proposing new electron shapes, including rings, figure-of-eight, helices, etc. A student made a joke version of these, and has a theory on their office door about pony shaped electrons. Another popular category is new layouts for the periodic table, with claims that it would unlock new reactions or free energy. A colleague gets emails about chemist colluding with petrol companies and hiding that water powered cars work, while another who does public lectures, spoken against stuff like water powered cars, and has much more of a presence on the university webpage gets almost none of these.

Re:A new theory

[Anonymous Coward]

" Which assumes I have a well stocked research lab that I should be busy working in...but that would be pointless because others have "done in detail"...my theory?!"

Not the same AC, but it looks pretty obvious that you misread what was said. Spectroscopic data already exists, in great detail, and is freely available on the web. That is work already done, and the measurements are very simple in principle, so there are no arguments about bogging up assumptions. If you are proposing a new theory that haves implications there, any predictions you can make about spectra would be instantly testable, without any lab usage, because such great data already exist.

Re:A new theory

[justthinkit]

It is not that the basic tenants of chemistry are not tested, challenged, strengthened, or weakened.

It is that the periodic changes and refinements are not Earth shattering.

When you get things right with the foundation of your house, you can build a house that will last. You will still have issues along the way, and there will be maintenance. But you won't have to build 10^^500 houses, or anti-houses, or a house that becomes a cloud that becomes a house again.

Re:A new theory

[Anonymous Coward]

The most obviously broken parts of physics, like the inflationary miracle after the Big Bang, are based on what measurements, exactly?

Ask yourself how a galaxy can have a red shift that indicates that it's moving away from us almost 3x the speed of light. That is right now and can be measured. The gravity-well of the galaxy can only account for a small portion of the red shift. The mast majority is coming from something else. They assume it's the expansion of space.

Re:A new theory

[justthinkit]

Thanks for your comment.

I am not sure why the original AC mentioned "spectroscopic data", nor am I sure how it is to be used to test my theory.

Spring-And-Loop Theory has a Planck-scale basis. Hence my repeated point that it will need to be simulated to allow bigger things (like atoms) and much bigger things (like Earth) and still bigger things (like Solar Systems) and the biggest things (like galaxies) to be modeled.

Spring-And-Loop Theory is like LEGO. It is a building system, not a smashing-things-against-other-things system.

I think the future of physics is simulation, not atom-smashing (or low hanging fruit like measuring the most distant star or galaxy or looking for Earth-like planets). We are stuck in the middle of the order-of-magnitude spectrum. We will never get within a factor of a billion of the Planck-scale. We can just "keep trying" to get there, or we can move to Plan B. I've chosen to move.

Note: This doesn't mean I reject all past and present physics. It does mean that, when an existing theory reaches its "divide by zero" moment, I have no choice but to discard that theory...at least locally...and proceed on foot.

Re:A new theory

[justthinkit]

We know that space is expanding. We don't exactly know why -- i.e. the original "explosion" idea makes no sense (horizon problem and space being altogether too uniform), and the fact that it is (relatively recently) seen to be accelerating in its expansion throws off the whole Big Bang theory itself. So we kludge with "dark energy".

I've answered this, in the original paper, when I explained what Spring-And-Loop Theory thinks "dark energy" is.

By the way, my theory's explanation of dark energy also happens to resolve a rather substantial problem with the Standard Model. The SM calculates a background energy of space that is 10^^120 times larger than what we actually measure. My theory says that energy is in fact there. It is not measurable due to the lack of matter in space. When you stick a thermometer in water, you are measuring how many water molecules slam into the glass walls of your thermometer. In space there are no molecules (approximately), so how do you measure the temperature of space?

It is a short step from that to questioning the CMB values themselves, since these are 10^^120 times too small anyway.

Re:A new theory

[Anonymous Coward]

You're a crackpot lunatic that has no understanding of physics whatsoever.

Re:A new theory

[Anonymous Coward]

You have an entire page dedicated to talking about the atom and reinterpreting it. Spectroscopic data is fundamentally a very basic and straightforward test of atomic structure theories. That is about as straightforward of a test of a large class of new theories as it gets.

Re:A new theory

[Anonymous Coward]

Who cares. Is it right or wrong is all that matters?

In response to a post that asks exactly where are the numbers, trying to get to that very point and gives a suggestion on how to do so in as straightforward way as possible, you instead go off on a tangent waffling about people being scared. Whether intentional or not, you're being very obtuse in both your comments and on your website. If all that matters is whether it is right or wrong, you're impeding any communication efforts by burying actual claims among a lot of fluff. The testable implications is exactly what is needed, not quotes and opinions.

Re:A new theory

[justthinkit]

This wiki page says the finest resolution achievable today is 0.2nm.

The Planck scale is 25 orders of magnitude smaller.

10 million million million million times smaller.

If present spectroscopy's best resolution were the 15 billion light years we can see back in time, Planck scale resolution would be seeing things the size of a tree.

Re:A new theory

[Anonymous Coward]

You're completely misreading that wikipedia page, which has to do with analytic chemistry techniques, and mentions use of a 0.2 nm bandpass, not a 0.2 nm resolution limit. It even refers to that as a medium resolution spectrometer. Using just standard printed spectral list, atomic line spectra data can be found for some lines down to 0.000 000 1 nm accuracy, and more specific sources than comprehensive tables will get more detailed than that. And it is irrelevant how much smaller the Planck scale is, because if your theory is in agreement with measurements, you would have to show it reproduces measurements we already have, just as relativity can reproduce Newtonian measurements at scales far short of where relativistic effects seperate.

Re:A new theory

[Anonymous Coward]

It is pretty common to look take theories that apply at extreme scales and look for subtle effects on scales we can observe. Things like the structure of the atom and of the proton are really sensitive to some of these effects, and theories that would only jump out at extreme scales still can have measurable effects on things far from those scales. At the very least, if advocating a theory that has things like the speed of light changing at amounts much larger than just a Planck length, depending on relative to what changes and what unitless values change as a result, some of the most detailed measurements made that would see such effects would be spectroscopy or particle physics, with the latter having a lot more baggage.

Re:A new theory

[Anonymous Coward]

Not sure what your point is, since you are saying what you just said before, except now someone pointed out it is wrong to assume that chemists don't get spammed with new proposed theories. It seems to have to do in the end more with how easy it is to propose new ideas while not understanding what is needed to actually back up those new ideas. By far, the most crackpot spam I've seen, more than chemistry and physics faculty get combined, is for people I know in the math department. It seems way too easy for someone to come up with their own math idea, e.g. yet another way to trisect an angle with a compass and straight edge, and not see the mistake they made. It is likewise easy to come up with a new physics theory if one doesn't know what has already been measured by experiments and why theories evolved to the point they are now. The basic aspects of chemistry are at least taught in school, but plenty of people get the quantum aspects way off and come up with their own idea without knowing how to tell they are wrong. Any of the above can just take the easy way out too, and brush off disagreement as a sign that mainstream researchers are trying to bury them instead of taking the time to see actual problems. It doesn't matter how solid of a foundation you have if people coming up with ideas have no idea what is involved in actually validating and confirming or disproving a theory.

Re:A new theory

[Anonymous Coward]

Ask yourself how a galaxy can have a red shift that indicates that it's moving away from us almost 3x the speed of light.

Err... what? You do realize that red-shift would disappear at 1c for obvious reasons.

That is right now and can be measured.

It can? Right now you say? Wow. Although I have to say: I'm the tiniest bit skeptical.

The gravity-well of the galaxy can only account for a small portion of the red shift.

Phew. Glad to hear it.

The mast majority is coming from something else.

Yeah, I don't know why they feel the need to share it with slashdot though.

They assume it's the expansion of space.

Oh, right. It'll be that theory they have with all the accurate predictions. Idiots!

Re:A new theory

[Roger W Moore]

Sorry but your theory is nonsense. Every measurement ever made is consistent with a constant speed of light and special relativity is the most accurate tested scientific theory ever. It is clear from your page that you fail to understand science at even the most basic possible level: "One of the absurdly limiting notions in physics these days is that if something is not testable, it is not science.". The only limitation of this is that it limits us to reality which is a good thing since we are doing science not writing fairy tales.

Frankly you need to see a psychiatrist because it is very clear from your page that you hold on reality is not what it should be. I realize that your condition means that you will almost certainly see this post as coming from just another physicist who doesn't understand how incredible your work is but sadly this really is just not the case. I'm sorry if this post has upset you but you need to hear the truth and I can only hope that it might persuade you to seek some professional medical help.

Re:A new theory

[justthinkit]

(1) SoL - I realize that my prediction is of a very slight change in the SoL -- 50% in 100B years, so it will be a tall order to measure that.

(2) I think you failed to grasp what I meant by "absurdly limiting notions". Say you have an idea, that does not appear to be testable. The consensus today would be your idea is worthless because it is untestable. I think that is absurdly limiting because (a) it could still allow you to see how something works more clearly, (b) it might become testable in the future. In short, I think the whole "must be testable" concept is an unnecessary limitation.

(3) I didn't see anything else in your comment related to my theory, so the way I score your review of it is "One thing you think is nonsense, just because, and one thing that we can agree to disagree on but that otherwise has nothing whatsoever to do with my theory, or physics in general."

Re:A new theory

[Anonymous Coward]

. The consensus today would be your idea is worthless because it is untestable. I think that is absurdly limiting because (a) it could still allow you to see how something works more clearly

I think you don't even have much of an idea what "testable" means. If you come up with an alternative theory that explains what we've measured now more clearly, but doesn't make any different prediction than current theories, that is still testable: it is still making predictions that match currently accessible predictions. It is untestable in the sense that it can't be distinguished from current theories, but if it makes all of the same predictions that have already been tested by current theories, then it is not considered worthless. Several theories have advanced because they started from this point, or close to it. String theory and many extensions to the Standard Model are this way, that they make new predictions that are not currently testable, but they also can be shown to reproduce current theories' predictions in situations we've already measured. On the smaller scale, especially in condensed matter physics and plasma physics where it comes down to finding better ways of explaining complex many-bodied problems, theories get replaced by ones that are simply just better explanations (look at the original papers proposing some ideas and compare them to modern refinements, sometimes original papers do a really bad job of explaining their position).

The key point is that it still must be shown consistent with what we know of the world today, which is actually a very big task because of the immense amount of data we already have. New versions of gravity to explain the dark matter issue have popped up a lot, get treated seriously, but then found in some situation to contradict observations. The key step to getting the theory noticed is doing some of the grunt work of showing it is consistent.

But if you're talking about something that is truly untestable, that makes no predictions at all, then you're not doing science. You're going back to natural philosophy of yore, trying to deduce how the universe works with your eyes closed, assuming whatever feels right must be how the world works. The world doesn't care about how simple or complex you think something should be...

Re:A new theory

[justthinkit]

Care to give an example of how String Theory has advanced anything?

The key point is that it still must be shown consistent with what we know of the world today.

Completely agree.

The key step to getting the theory noticed is doing some of the grunt work of showing it is consistent.

Agreed also. I wouldn't want you to think I am just dumping my theory on the world and walking away. For anything to have a value, it must grow, and grow healthily.

But if you're talking about something that is truly untestable, that makes no predictions at all, then you're not doing science.

Well, luckily for both of us I'm not. But are you?

Re:A new theory

[Anonymous Coward]

You previously said:

The consensus today would be your idea is worthless because it is untestable

Either you're talking about something that is completely untestable, in which case this statement is an irrelevant distraction, or you're talking about the situation where something is testable, but not distinguishable from another theory. In the latter case, there is no such consensus and you're grossly misrepresenting things. Either way, it amounts to a strawman, and arguing against it or wasting time on it doesn't do anything to advance your theory, and in fact just creates another red-flag that makes you look exactly the same as every other crackpot on the internet that is more concerned with playing games than actually trying to develop new ideas.

Re:A new theory

[Roger W Moore]

Well your theory relates the speed of light to the temperature of the universe. This is not uniform but fluctuates at the one in 10^-6 level (look up the cosmic microwave background). However we have seen zero variation in the speed of light down to the level of ~10^-20 so that's your theory eliminated by 14 orders of magnitude of measurement. I'd say this was pretty convincing evidence that it's nonsense.

Now, rather than waste my time giving a blow by blow account of all your other nonsense claims, how about you read some physics papers like this one and explain how your theory is completely consistent with all these measurements and those contained in other papers? Like most of the people with a loose grip on reality you seem to think that it is physics' job to explain to you why you are wrong. That is not how it works - if you want people to take you seriously you have to explain how your theory is consistent with current data. Afterall it is hardly reasonable to expect people to spend time reading and understanding your theories if you are completely unprepared to read and understand others is it?

Re:A new theory

[justthinkit]

No, not to the "temperature of the universe" as you call it (i.e. the CMB temp). To the calculated temperature. The one that is 10^^120 times bigger. The one that physicists can not explain why it isn't what is measured.

The observed CMB "temperature" is indeed uniform, but this destroys the Big Bang theory, not my theory.

Thanks for the link to Gamow paper. I'll have a look at it but really, if scientists are out by 10^^120 in their measure of the background energy of space, how likely are they to detect motion relative to it?

As to you and I, it is high time we stop chatting. Your statements like:

like most of the people with a loose grip on reality

reveal you to be a positively uncivilized person to interact with.

Re:A new theory

[Roger W Moore]

You can't detect motion relative to space and if you could relativity would be wrong. This would mean the Lorentz symmetry is broken is which one of the most fundamental symmetries in physics and yet there is absolutely no evidence of that whatsoever. The only logical an rational explanation is that your theory is nonsense and inconsistent with reality.

Yes you can argue that the effect is too small to measure but at this point you might as well be claiming that the universe is full of flying pigs that just happen to be too small to see. Can you disprove this? No, but that does not mean that anyone is going to waste time looking for them. This is effectively what your argument boils down to.

Can you not see how illogical and irrational you are being? It's great that you are interested in physics but it is really sad that you fail to understand so much of it. Your work is not that of some undiscovered genius who nobody understands: it is far more consistent with someone who does not understand current physics and just made up a story to explain some problem they had heard about with physics.

I suppose that you can't see this at all and remained convinced that you are correct and I'm sure you also probably want to call me worse than uncivilized now. However if there is even a tiny doubt in your mind that something is not quite right then please act on it and seek professional mental help.

Re:A new theory

[justthinkit]

Roger,

I'm going to look at your Lorentz paper.

And you are going to relax. Don't worry about me. Stick to physics, it is clearly what you do best.

Best,

Floyd

Re:A new theory

[Anonymous Coward]

I think your comment mostly reflects how you find my theory "shocking"...

Setting aside the issues caused by your lack of physics background, you seem to have a serious communication issue. You managed to catch the attention of actual physicists who could be tapped as a great resource, and then squander that by spending more time stroking your ego than trying to actual move things forward. Your site and posts here hit more red-flags that make you look like any other crackpot than attempts I've seen at parody of such people. This isn't flags in terms of physics content, but more lack of physics content and the incoherence and diversions you spend more time on than actual physics. Keeping up on physics work across various fields takes a lot of work and time, and coming across someone who is actively trying to make material even more difficult is just going to get passed over.

So get over yourself, and don't go around acting like your theory is shocking, because it isn't. I've come across many papers in serious journals proposing spacetime structures that involve changes in fundamental constants. The difference is in 3-5 pages they convey an immense amount of work, with dozen or more quantitative predictions and comparisons to observations. The only thing shocking is how special you seem to think your work is considering how little you've done with it and how poorly you try to fluff it up like someone trying to turn a single paragraph into a three page book report.

Re:A new theory

[justthinkit]

I am aware of a single replier who wasn't an AC. The last thing I think I did was squander my encounter with "Roger W. Moore".

Out of curiousity, why is it you post as an AC? In fact, most of the nastiest posts in this sub-thread are from ACs. What do you hope to accomplish by this? With me, your attacks are, if anything, a proof of the value of my theory.

If what I was proposing was truly nonsense, the proper response would be to ignore it or offer a kindly word of condolence.

Your level of anger and caustic criticism is out of all proportion.

Get a grip, man.

399.999999 PB of Noise . . .

[Anonymous Coward]

and MAYBE 0.000001 PB of useful data, massaged down to a 10kB pdf file for publication for some hand-waving TOE argument. Welfare for engineers. physicists, and arithmeticians. ROI - zip.

As for the 4TB/day of "data" generated on facebook: washerwomen gossiping over the back fence. content - zero.

Actually the LHC is bigger!

[Roger W Moore]

Actually the LHC generates more data than this. The talk is only talking about the data at CERN. The last count of all the files in the ATLAS experiment's DQ2 store (a distributed dataset access system with storage around the globe) was 161PB. This value includes all the simulated data, analysis data etc. I'm certain CMS has a comparable amount and then there are Alice and LHCb as well so the total will be well over the 300PB which Facebook stores.

While Facebook generates 4 PB of new data per day they only store 300 PB according to that page so most of this is either discarded or overwrites existing data. If we look at the LHC then the raw data rate is probably about 1 PB/min but we throw away most of this (using computers on the surface, not 100m underground as the original talk says) because it's physics we already know about and we can't afford the storage for it. Then there is the generation of new data by analysis and simulation to include.

So if you actually look at the whole system, not just what is at CERN, we have a larger total storage capacity and generate more data than Facebook...and we plan to scale up.

Re:Actually the LHC is bigger!

[Anonymous Coward]

...more data than Facebook...and we plan to scale up.

Please let us know when we reach the point that we can determine which will be the more egregious waste of resources.

Thank God for the dead

[Lawrence_Bird]

hmm... lets see

COBOL.. dead
BSD.. dead
TAPE .. dead

And yet there would be no LHC datacenter without tape.

The CERN Tier-0 in Meyrin currently provides around 45 PetaBytes of data storage on disk and 90 PetaBytes on tape for physics, and includes the majority of the 100,000 processing cores in the CERN Data Centre.

ref:

http://information-technology....
http://www.economist.com/blogs...
http://storageservers.wordpres...
http://scribol.com/science/hal...

This sounds like ...

[PPH]

... PornHub for physicists.

Huh

[koan]

That amount of data is something only an AI could get through.

Re:Huh

[iggymanz]

there are no AI and none are planned to exist in high energy physics, the usual methods of filtering and statistics will instead be used and will suffice

Re:Huh

[Anonymous Coward]

Processing alot of data does not require artificial intelligence... It merely requires a so called "Automatic Computer" to minimize the labor of doing it manually.

Doing statistically analysis on time series sample data, even when trillions or more lines long, is actually very "simple" algorithmically and therefore AI is not "required." It does, however, require lots of I/O and CPU resources.

Using other people's money

[AndyKron]

Sounds like they really don't know what they want, except they want it all. Using other people's money, of course!

An archive of 100PB

[CopaceticOpus]

LHC experiments have generated an archive of 100PB...

Torrent link?