Hadoop is open source. The companies building it are LinkedIn, Yahoo, Facebook and then the Hadoop vendors: Hortonworks (tightly tied to Microsoft), IBM, Cloudera (enterprise support vendor)...
You are overestimating the difficulty at this point. This not compsci anymore and hasn't been for many many years. It isn't even hard administration. It is probably easier to get a big data system running in 2015 than it was to use Oracle in 1995.
As far as your examples you went way too big. GE is a huge DevOps shop, they know what Big Data is. Exxon has massive supercomputing datasets. I would bet they were doing big data long before it got cool. Pfizer has an IT department that is some of everything but they have many many data warehouses so I can't imagine they aren't playing with data lakes.
I don't see any evidence that Oracle is heavily invested in either Solaris or Sparc. They were dying technologies when Oracle bought them and they have done little to revive them. Oracle has the cash for a massive push for Solaris or Sparc innovation and they haven't done it. They've let them fade.
Anyway I've met the thought leaders for Oracle Linux. Their work right now is focused on really large customer support. Oracle is using Oracle Linus to get further into the hosting / cloud / IaaS reoccurring revenue business (think classic Terremark, Sungard...) The people are strong. Areas like application centric virtualization, and virtual compute appliance (Oracle's cheap version of Netezza / Exadata) are where the focus is. Just not the sort of stuff the/. crowd cares about.
XFS has changed the way it repairs. That has nothing to do with systemd.
Systemd does, as you say, old things in new ways. Which means config files are probably worse than they used to be. This is a reasonable concern. There is a one time training hurdle, just like when you changing end user's applications. There does need to be some improvement in best practices as we move from a more mature set of configurations to a much less mature set of configurations.
My answer was clear. The original poster and yourself are both wrong. The disk is not in the middle. In journald.conf set Storage=none and log remotely. Then the journal won't touch the local disk at all yet the export to other systems still works fine. There is an application running and it is sending data from memory to multiple streams. It is simply not true that a disk is required to send data.
systemd allows you to configure logging however you want and is more flexible than the init based systems of years ago. Which was my answer. You are gaining not losing flexibility. The system has more easily (to a developer) hooks. And as the external tooling develops there will be far more hooks for system admins.
First off grep is generally bad with minor variations. It would work if there was a good uniform identifier system. But because there isn't using grep to search a large log you can get lots of false positives and false negatives.
For example if the error says/dev/sda1 or it could say/dev/sda or it could say something about the mount point or the file it errored on. Also grep is not going to organize the material by anything other than sequence in original file and source file.
Why bother with that rubbish when proper text logs are so superior and easy to handle?
They aren't superior they are simpler. A list of random sentences of news articles is less valuable than a hierarchical index of the news stories. Similarly for computer events. Organization and hierarchy helps? How do I ask a text log to give me all events related to a particular hard drive hardware errors in the last 2 weeks easily?
a) Oracle has to deliver stability. They have stability guarantees in their contracts b) He specifically cited Oracle, so that's why I brought them up.
I do think highly of Oracle. There are certainly problems. But partially those are caused by over promising and partially those are caused by being held to a much higher standard than other vendors where. Oracle can be jerks but there is a good reason they have grown the way they have. So FWIW I would consider their judgement good.
I thought it was a close call on init systems (and to be fair, systemd isn't exactly the mature, rock-solid solution a replacement init should be!)
It depends when you mean. The first vote upstart and OpenRC were falling behind. By the second vote they were far behind. There was no viable replacement. At that point the question was a) Whether to support multiple init systems b) Whether to unify
There weren't many upstream dependencies so this was close. Then the upstream dependencies started to spread like wildfire through Debian and the choice became:
a) Spend a ton of resources fighting the tide and offering multiple systems. Unclear how to do this. b) Switch at Jessie c) Hold out for one more version
There were some subtle aspects but mainly it was never really close. Systemd pulled way ahead and created dependencies.
The votes for a replacement on the Debian list should have gone with Upstart IMHO as it was the most popular option
Again when? By the time of the final vote, by now upstart isn't viable.
Still, it doesn't really matter now - what does matter is that the init system is rock-solid, has buy-in from the customer base (ie the community who use Linux, including server admins) and doesn't require too much re-training to understand and administer it. I'm not sure it has any of those 3 currently.
That's not going to be the criteria. I don't think init based systems are all that stable. Closer using your list is going to be:
a) Has buy in from the developer base b1) offer a total environment that is more stable than init's b2) Is designed to be a standard API replaceable with an IaaS for better stability without applications needing to be changed c) Be easy to administer.
The 1990s concept of stability where one is worried about the stability of individual instances is being replaced by the virtualization concept of cloud stability.
Remember the OP's claim. Oracle had to agree to the switch. You don't get more enterprise than Oracle. So their failure to pull away means they didn't think systemd was a terrible idea.
Oracle Linux is an easy location for systemd. It is usually installed in a minimally modified form in order to maintain support for Oracle applications
Yes. That's the bulk of enterprise deployment.
The biggest problems with systemd is when you try to do something that the systemd developers didn't expect, or didn't think was worth the effort.
Excluding conversion (I'll agree conversions can be complex) like what?
The claim of the other poster is that systemd is not enterprise ready. I'd say Oracle's endorsement is a fairly clear refutation of that or at least strong counter evidence.
I'd agree that Oracle's Linux team didn't care much. They were following RedHat's lead but the fact that they followed RedHat's lead still means a lot. If they hated it, they would have picked a non-systemd choice for Oracle Linux instead of RHEL.
You are right. I had your question backwards. But that makes the answer even easier: journalctl -o short -f | ncat remote-destination.com 12345
As for the JSON on the receiver end, the receiver just links to the library and puts it into whatever logging format it wants. That's unavoidable unless you just want to dump the data structure as text for every entry.
As for converting to text, you can't advocate for syslogd and then object to text. You have 3 options:
a) Leave it in native binary format and let the receiving system parse b) Convert to some independent format on sending system c) Convert to receiver's binary format on the sending system
Systemd supports all 3. So I'm not sure what the issue is. It allows you to do what you want.
It is also why Apple is basically non-existent in the business world, and is not seen as a suitable platform for anything important.
I'm not sure about non-existent Apple has, and has had for a long time, a massive chunk of the over $1k laptop market which is developers and executives. I'm fairly sure at their price point they are quite common. Even people like Forrestor many years ago said that keeping Apple out was no longer a viable option for business.
Third party applications are not always tested against new releases... at most software companies I have worked with this usually happens well after the release of the operating system.
In the Apple world they are tested. That's part of the Apple culture. As for testing after the release again not part of the Apple culture. Because a substantial chunk of the customer base is going to expect to be able to upgrade the day of the release the software companies have got to be resolving bugs against the betas and release candidates. Not being ready day-of is considered a black mark and hurts sales.
At the same token developers can freely require end users to be upgraded fairly quickly. So for example OSX 10.7.3 had some new cloud features, release was Feb 2012. By Oct 2012 you had many 3rd party applications that made use of those features and wouldn't run on anything older than 10.7.3.
It is a tradeoff for the software company they have to Johnny on the spot with their upgrades but they don't have to support a lot of old versions of their products or support a lot of old operating systems versions.
When they are only the latest version will be tested...
That is true. The Apple culture is a culture of constant upgrades. Very much like the 1990s.
That does not even consider companies which no longer exits, or company specific code where the programmer has long since left.
Sure it does. If you use custom code in your custom applications you as a company are responsible for testing against betas. That means for a complex application one at least minor release per year needs to be scheduled and possibly more.
If they foolishly try to change culture to follow apple's disregard for backwards compatibility, it really will be the year of Linux on the desktop.
Consider the problem of migrating Windows -> Windows version. Then compare Windows -> Linux. The costs are at least a full order of magnitude higher. I don't see companies switching to Linux to avoid the complexity of migrations.
SysD's binary logs have another, serious flaw: they are not designed to be sent over a network.
No actually they are. Systemd has an export format and a JSON library you can attach to that produces a version of the log designed for combining. Systemd works far better with messaging clients than syslogd because it was written after messaging and networks.
Let me just point out, Oracle Linux 7 is systemd only. Oracle switched Jul 23, 2014. So yet another enterprise vendor that doesn't buy into the whole "systemd isn't mission critical ready" meme you all are pushing.
I don't think that's terribly useful information regarding users. The question has never been which do end users support, end users mostly don't care. Among those who do care, you have a lot of traditionalist system admin types and those are probably substantially biased in the anti-systemd direction. Systemd breaks stuff they care about. The benefits won't be seen till years later and mostly will be experienced by people other than them.
What's important is not that. But rather as upstream packages drop support what happens to Mint. Say in mid-2017 if Mint is still offering the option and non-systemd has substantial costs, how many choose it. That's the relevant number for the Debian systemd debate. What percentage of users are willing to write off huge numbers of applications in exchange for a more traditionalist process management system, as more packages become dependent on advanced process management?
I'd guess about 5 years to you'll need to be on anti-systemd distributions that have hundreds of upstream packages they don't support or only support with lesser known (less supported) compile options. Upstream is aggressively putting systemd dependencies in, Linux software has chains of dependencies. So distributions like Crux and Alpine will have to become very aggressive about "no".
Yes and no. Certainly what you say is generally true. But there have been some components that were sticky. For example GCC was sticky and distributions had to pick a fork. Certainly EGCS vs. GCC2 was sticky and you couldn't swap without changing components. In the early days KDE and Gnome apps didn't run or didn't run well in each other's environments. There wasn't an easy way to swap on an individual level. Picking your example of Emacs one of the classic choices: X-Emacs vs. Emacs were not easy swaps and generally you couldn't have both. TeX is famous for incompatibilities.
The reason Systemd won so decisively is the other projects for an advanced init replacement failed. Had OpenRC been earlier, had upstart not floundered we are looking at a situation much closer to Gnome vs. KDE. Today we have a situation much closer to what the world would have looked like if there had been no Gnome project: old fashioned window managers and you have one GUI, KDE. In such a situation almost all of the distributions would have been KDE distributions...
For Microsoft that's about $30. If they are buying a computer every 7.5 years or so that's $4/year. They would have to lose 250m of those customers to even amount to $1b/yr in revenue. Customers that cheap do nothing for Microsoft but hold them back.
Slashdot Mirror
Hadoop is open source. The companies building it are LinkedIn, Yahoo, Facebook and then the Hadoop vendors: Hortonworks (tightly tied to Microsoft), IBM, Cloudera (enterprise support vendor)...
You are overestimating the difficulty at this point. This not compsci anymore and hasn't been for many many years. It isn't even hard administration. It is probably easier to get a big data system running in 2015 than it was to use Oracle in 1995.
As far as your examples you went way too big. GE is a huge DevOps shop, they know what Big Data is. Exxon has massive supercomputing datasets. I would bet they were doing big data long before it got cool. Pfizer has an IT department that is some of everything but they have many many data warehouses so I can't imagine they aren't playing with data lakes.
I don't see any evidence that Oracle is heavily invested in either Solaris or Sparc. They were dying technologies when Oracle bought them and they have done little to revive them. Oracle has the cash for a massive push for Solaris or Sparc innovation and they haven't done it. They've let them fade.
Anyway I've met the thought leaders for Oracle Linux. Their work right now is focused on really large customer support. Oracle is using Oracle Linus to get further into the hosting / cloud / IaaS reoccurring revenue business (think classic Terremark, Sungard...) The people are strong. Areas like application centric virtualization, and virtual compute appliance (Oracle's cheap version of Netezza / Exadata) are where the focus is. Just not the sort of stuff the /. crowd cares about.
There are two issues here
XFS has changed the way it repairs. That has nothing to do with systemd.
Systemd does, as you say, old things in new ways. Which means config files are probably worse than they used to be. This is a reasonable concern. There is a one time training hurdle, just like when you changing end user's applications. There does need to be some improvement in best practices as we move from a more mature set of configurations to a much less mature set of configurations.
OK show me the grep command to pull all variants that every application on a Linux system uses in syslog to refer to a drive failure?
My answer was clear. The original poster and yourself are both wrong. The disk is not in the middle. In journald.conf set Storage=none and log remotely. Then the journal won't touch the local disk at all yet the export to other systems still works fine. There is an application running and it is sending data from memory to multiple streams. It is simply not true that a disk is required to send data.
systemd allows you to configure logging however you want and is more flexible than the init based systems of years ago. Which was my answer. You are gaining not losing flexibility. The system has more easily (to a developer) hooks. And as the external tooling develops there will be far more hooks for system admins.
Solaris had a process manager years before Linux did. The ideas for systemd came from big box Unixes.
First off grep is generally bad with minor variations. It would work if there was a good uniform identifier system. But because there isn't using grep to search a large log you can get lots of false positives and false negatives.
For example if the error says /dev/sda1 /dev/sda or it could say something about the mount point or the file it errored on. Also grep is not going to organize the material by anything other than sequence in original file and source file.
or it could say
OK what enterprise applications have indicated they have problems with systemd?
They aren't superior they are simpler. A list of random sentences of news articles is less valuable than a hierarchical index of the news stories. Similarly for computer events. Organization and hierarchy helps? How do I ask a text log to give me all events related to a particular hard drive hardware errors in the last 2 weeks easily?
Two things:
a) Oracle has to deliver stability. They have stability guarantees in their contracts
b) He specifically cited Oracle, so that's why I brought them up.
I do think highly of Oracle. There are certainly problems. But partially those are caused by over promising and partially those are caused by being held to a much higher standard than other vendors where. Oracle can be jerks but there is a good reason they have grown the way they have. So FWIW I would consider their judgement good.
It depends when you mean. The first vote upstart and OpenRC were falling behind. By the second vote they were far behind. There was no viable replacement. At that point the question was
a) Whether to support multiple init systems
b) Whether to unify
There weren't many upstream dependencies so this was close. Then the upstream dependencies started to spread like wildfire through Debian and the choice became:
a) Spend a ton of resources fighting the tide and offering multiple systems. Unclear how to do this.
b) Switch at Jessie
c) Hold out for one more version
There were some subtle aspects but mainly it was never really close. Systemd pulled way ahead and created dependencies.
Again when? By the time of the final vote, by now upstart isn't viable.
That's not going to be the criteria. I don't think init based systems are all that stable. Closer using your list is going to be:
a) Has buy in from the developer base
b1) offer a total environment that is more stable than init's
b2) Is designed to be a standard API replaceable with an IaaS for better stability without applications needing to be changed
c) Be easy to administer.
The 1990s concept of stability where one is worried about the stability of individual instances is being replaced by the virtualization concept of cloud stability.
Remember the OP's claim. Oracle had to agree to the switch. You don't get more enterprise than Oracle. So their failure to pull away means they didn't think systemd was a terrible idea.
Yes. That's the bulk of enterprise deployment.
Excluding conversion (I'll agree conversions can be complex) like what?
The claim of the other poster is that systemd is not enterprise ready. I'd say Oracle's endorsement is a fairly clear refutation of that or at least strong counter evidence.
I'd agree that Oracle's Linux team didn't care much. They were following RedHat's lead but the fact that they followed RedHat's lead still means a lot. If they hated it, they would have picked a non-systemd choice for Oracle Linux instead of RHEL.
You are right. I had your question backwards. But that makes the answer even easier:
journalctl -o short -f | ncat remote-destination.com 12345
As for the JSON on the receiver end, the receiver just links to the library and puts it into whatever logging format it wants. That's unavoidable unless you just want to dump the data structure as text for every entry.
As for converting to text, you can't advocate for syslogd and then object to text. You have 3 options:
a) Leave it in native binary format and let the receiving system parse
b) Convert to some independent format on sending system
c) Convert to receiver's binary format on the sending system
Systemd supports all 3. So I'm not sure what the issue is. It allows you to do what you want.
I'm not sure about non-existent Apple has, and has had for a long time, a massive chunk of the over $1k laptop market which is developers and executives. I'm fairly sure at their price point they are quite common. Even people like Forrestor many years ago said that keeping Apple out was no longer a viable option for business.
In the Apple world they are tested. That's part of the Apple culture. As for testing after the release again not part of the Apple culture. Because a substantial chunk of the customer base is going to expect to be able to upgrade the day of the release the software companies have got to be resolving bugs against the betas and release candidates. Not being ready day-of is considered a black mark and hurts sales.
At the same token developers can freely require end users to be upgraded fairly quickly. So for example OSX 10.7.3 had some new cloud features, release was Feb 2012. By Oct 2012 you had many 3rd party applications that made use of those features and wouldn't run on anything older than 10.7.3.
It is a tradeoff for the software company they have to Johnny on the spot with their upgrades but they don't have to support a lot of old versions of their products or support a lot of old operating systems versions.
That is true. The Apple culture is a culture of constant upgrades. Very much like the 1990s.
Sure it does. If you use custom code in your custom applications you as a company are responsible for testing against betas. That means for a complex application one at least minor release per year needs to be scheduled and possibly more.
Consider the problem of migrating Windows -> Windows version. Then compare Windows -> Linux. The costs are at least a full order of magnitude higher. I don't see companies switching to Linux to avoid the complexity of migrations.
No actually they are. Systemd has an export format and a JSON library you can attach to that produces a version of the log designed for combining. Systemd works far better with messaging clients than syslogd because it was written after messaging and networks.
Your specific example is also wrong. You just setup a Splunk forwarder: http://answers.splunk.com/answ...
Oh that makes sense.
That's not systemd. You should read the man page for fsck.xfs ( http://man7.org/linux/man-page...) which recommends you use xfs_repair.
And you can easily boot if a single non-boot partition is corrupted.
Let me just point out, Oracle Linux 7 is systemd only. Oracle switched Jul 23, 2014. So yet another enterprise vendor that doesn't buy into the whole "systemd isn't mission critical ready" meme you all are pushing.
I don't think that's terribly useful information regarding users. The question has never been which do end users support, end users mostly don't care. Among those who do care, you have a lot of traditionalist system admin types and those are probably substantially biased in the anti-systemd direction. Systemd breaks stuff they care about. The benefits won't be seen till years later and mostly will be experienced by people other than them.
What's important is not that. But rather as upstream packages drop support what happens to Mint. Say in mid-2017 if Mint is still offering the option and non-systemd has substantial costs, how many choose it. That's the relevant number for the Debian systemd debate. What percentage of users are willing to write off huge numbers of applications in exchange for a more traditionalist process management system, as more packages become dependent on advanced process management?
I'd guess about 5 years to you'll need to be on anti-systemd distributions that have hundreds of upstream packages they don't support or only support with lesser known (less supported) compile options. Upstream is aggressively putting systemd dependencies in, Linux software has chains of dependencies. So distributions like Crux and Alpine will have to become very aggressive about "no".
Short but actual useful advice. This should be modded up.
Yes and no. Certainly what you say is generally true. But there have been some components that were sticky. For example GCC was sticky and distributions had to pick a fork. Certainly EGCS vs. GCC2 was sticky and you couldn't swap without changing components. In the early days KDE and Gnome apps didn't run or didn't run well in each other's environments. There wasn't an easy way to swap on an individual level. Picking your example of Emacs one of the classic choices: X-Emacs vs. Emacs were not easy swaps and generally you couldn't have both. TeX is famous for incompatibilities.
The reason Systemd won so decisively is the other projects for an advanced init replacement failed. Had OpenRC been earlier, had upstart not floundered we are looking at a situation much closer to Gnome vs. KDE. Today we have a situation much closer to what the world would have looked like if there had been no Gnome project: old fashioned window managers and you have one GUI, KDE. In such a situation almost all of the distributions would have been KDE distributions...
For Microsoft that's about $30. If they are buying a computer every 7.5 years or so that's $4/year. They would have to lose 250m of those customers to even amount to $1b/yr in revenue. Customers that cheap do nothing for Microsoft but hold them back.