RHEL 6 broke that. They stopped supporting syslog and went to rsyslog which is incompatible and is breaking things left and right. Yes, we entered a show-stopper bug on certification with RH.
No, those of you who are rsyslog fans, it is not syslog. It is an abomination wearing a poorly fitting skin of syslog, but the rotting flesh is visible underneath, the miasma of decay of what was once a useful tool has left a shambling terror that cannot be abided.
For a moment, I thought you were one of the people on our log analysis team that I work with.
There are places where a database is good. I recommend them heavily, but the original log output shouldn't be database. If you want a database of logs (appropriate in many cases), send a copy of the logs off host (which you have to do anyway for PCI and many other audits) and store that copy in a database. That makes it easier to comply with other requirements.
Best of both worlds. Plain text local to box in case Things Go Wrong, and database for cross-system forensic analysis.
Actually, if you are at that point, you need to centralize your logs or use a proper alerting system beyond traditional logging because you are at the point you need to correlate events across systems. "Alert if this event occurs on system A unless you see this other event on system B or C"
Putting a database to manage logs on each system causes grief. We tried that once with wtmp. It hasn't worked out well, and we still don't have a clean replacement for wtmp that records logout times. So instead, I'm faced with custom tools to parse the wtmp file and eliminate events where the user has logged out, leaving events in the file where the user has logged in but has not logged out. (it's painful)
All the commercial folks have gone away from standard relational databases for the log storage because it is suboptimal. Instead, a proprietary database optimized just for log analysis and retention is used. It's a standard sedan being used as a commercial truck. You can make it sort of work, but you'll feel the pain.
My experience is based on an order of magnitude higher than the threshold I indicated, but I can tell we exceeded traditional databases around the 5k hosts level, and we aren't even doing complex log analysis. A few filters to identify specific log event types (e.g. sudo traffic is reformatted so that key items of the sudo become specific columns for easier searching.
What made your needs more complex was the request for contextual filtering. I also assumed you were needing contextual filtering across multiple hosts. That's when it gets hairy and you really notice the difference with a proper tool.
For low numbers of systems, splunk is great. When you have to go geographically diverse, my advice is to avoid splunk. They don't concentrate their log events into a single datacenter for cross-system reports and filters. (They tried to sell to us, and that was a deal breaker here.)
As I said, for smaller numbers of systems, hardware can overcome less efficient code. It's when you try to scale up that you experience the pain, or you try and bolt on things like centralized log protection, long term retention of your alerts because the auditors want proof six months later you sent out an alert, etc. It turns out you hit that level surprisingly quick in our experience.
And for the record, I'm quickly coming to the conclusion that rsyslogd is evil. We've blocked RHEL 6 certification on the use of rsyslogd alone because it messes with log output format and is so devilishly hard to auto-configure and validate that configuration is correct.
Depends on your scale. If you are talking 5k+ systems, then yes, you need something more serious. You could try and grow it at home, but expect problems when it grows or you add too many alerts. If you are talking about trivial usage (at most a few hundred systems, and none of them are badly misconfigured so as to be spamming the log daemons with dozens if not hundreds of messages per second), then you almost don't care what you use. Hardware can then overcome inefficient code.
Real time analysis isn't my primary specialty (I do long term storage and forensic analysis mostly), but usually, one needs to concentrate the logs into an analysis engine that can do comparisons of log events from different systems (if condition A on system primary, and not condition B on system secondary, alert). Once that is done, store the event for long enough to understand any failures where alerts failed to go out so you can build better alerts for next time. Typically, that's order of magnitude one to two weeks, though I've heard of shorter. You run into the risk of not finding the key event before it expires if you go too short. If you have large numbers of clients or geographic distribution, then a series of log concentrators is important, especially if you want to put your main log storage systems in an isolated network for additional safety.
If you just have a small number of systems, then it's relatively easy. If you have large numbers of systems, then that central analysis engine becomes a major bottleneck and that's where the commercial products shine. Obviously, if you have no cases where the logs of system 1 alter the interpretation of the logs of system 2, that makes it much simpler, and you can then parallelize your log analysis much easier. I rarely see that as a need.
The act of invoking some action is the easiest portion of all. I've seen almost nothing (including vanilla syslog) that can't feed a line into a program that executes an alert action. If nothing else, use named pipes to a program that calls snmptrap or wget or curl or whatever your alert program requires for input.
That's called context filtering, and is a standard feature of any serious log analysis tool. Okay, so Loglogic is not a serious log analysis tool. If you find yourself needing context filtering, odds are, you have sufficient needs to consider genuine log analysis tools (e.g. splunk, sensage, etc.). I've worked heavily with such tools and the real power is when you have a full year of forensic log storage to analyze and can fairly readily run very unusual reports on that data that you never thought was significant when you captured it nine months ago. Yes, contextual comprehension is normal.
To answer the question you didn't ask, real time analysis and forensic after-action analysis are different enough areas as to warrant different tools. The best approach I've seen done thus far is to collect them at the same point and then split the log feed, sending one to a long term storage, after-action analysis tool, and a copy of such to a real time analysis tool. It may seem on first glance like they should be easily combined, but there are some incompatible optimizations for the two uses that have come out not only in my experience, but the experience of a number of other people I've worked with who do log storage/analysis for a living.
Execution of arbitrary action to trigger alerts is very normal.
If you really want my wish list for syslog, it is to reexamine the facilities. We don't need uucp or news any longer, but it would be nice to offer support for extended facilities.
Beyond that, everything I've ever found myself wanting is already in syslog-ng but one. I want a program, distributed with any log daemon, that will parse the configuration file and if given a message string, program or facility.priority pair, will tell me all destinations it would go to. Some of us have to automatically ensure that this or that custom log file is configured correctly.
There already is a few syslog compatible daemons for Windows. Syslog-ng has one such port. Many of the large scale commercial syslog aggregators (spunk, sensage,...) have tools/products to extract the native Windows logs. They don't necessarily work the same way as syslog does, but there are still options there.
I'm more looking for a usable oracle log extractor since Oracle is ripping out the syslog support.
Expected rate of log messages from systems I monitor is low enough that ten seconds downtime of syslog wouldn't be noticed except for a few misbehaved systems. Temporary network disconnects, nothing interesting going on, etc. all makes logging faster than that cause an unacceptable volume of traffic. Yes, I could log all those hashes to a table with a mere two week retention instead of the full year required by PCI, but I don't think it would fly the way you intend.
In a well designed network, a compromise of a target system does not give one increased ability to compromise the log system, because there is no trust relationship, and the central log host does not even have the same user accounts. No user who has an account on the production system is permitted to have an account on the log system.
This topic is basic PCI stuff, and common also for SOX compliance. The problems are far from complex.
The so called solution does not provide sufficient security based on description to eliminate the requirement for central log storage, especially since that is an explicit requirement of PCI. Some may have that as an explicit SOX control as well. The obvious problem with the tool is it is only tamper detection, not tamper protection. Any fool with root can erase the evidence that they were the one logged in. It may be a tad harder to hide the fact that the logs were modified, but even that could be bypassed with the above description by simple virtue of rotating the log post-compromise and "losing" the entries in question.
PCI not only accepts, but mandates central log storage. Recommendation is to do it using real time log transmission. Log tampering detection is expected on even the central system, where you use syslog to receive, and then store the central copy in something a bit fancier than a text file, often a database of some kind.
Local to a system, there is no such thing as acceptable tamper resistant unless it is a write once/read many drive that is physically that way.
Centralize the logs, and stop messing with my syslog. (This means you Red Hat)
No law degree required, just fairly straightforward computer security for the most part. There are subtleties, but if people actually read the entire thing, it is amazing how clear much of it is.
PCI DSS 3.2 refers to the types of data that are not permitted to be stored. This does not necessarily include the actual credit card number. The original statement of payment account numbers not being permitted to be stored digitally is false. That does not refer to the magnetic track data or the card verification code (the number on the back of the card).
PCI DSS 3.4 discusses the requirements if PAN data is stored. One option, and a frequent subject of discussion in PCI certifications in my experience, is the encryption of such data and protections around ensuring all such data is encrypted and that the encrypting key is itself encrypted with a separate key that is independently protected.
The common failure is to accidentally store the magnetic stripe data or the CVV code. Even in encrypted form, that is prohibited.
PCI DSS does not prohibit storing the full payment account number (PAN) electronically, as long as it is encrypted. The note on PCI DSS 3.2.1 specifically talks about retaining the PAN in the normal course of business. PCI DSS 3.2.2 does prohibit storing the security code printed on the back, or the full magnetic track data. PCI DSS 3.4's requirement to render the PAN unreadable when stored makes it clear that storing that credit card number is permitted, if it is properly protected. The definition of properly protected is given.
I read the announcement as saying that the same database that housed some of the forum data also housed PAN data, but that they were claiming that table of the database was encrypted and thus don't believe it compromised.
One could argue that PCI DSS 2.2.1 (implement only one primary function per system) as violated, but that is debatable based on the few details publicly available.
There is too little available to gauge the incident at this time and guess specific PCI compliance failures.
Knowing one password does not materially help attacks on other passwords. However, depending on the algorithm used, it may be possible to brute force the password. For example, if the old Unix crypt(3c) algorithm is used, then most passwords can be brute forced in reasonable time now. Recent advances have led to use of the graphics card on your system to perform those attacks.
Longer hashes like MD-5 are significantly harder as they support a much longer search space, but few people use a password over twelve characters. Certainly, any password under seven characters should be considered vulnerable, regardless of algorithm used to salt/hash them.
Assuming (big if) they are using standard password hashing algorithms, long (at least 15 characters long) passwords that are pasted, not typed because they are completely randomly generated is your best protection in such cases.
Uh.. what expects/usr to be a separate filesystem? Where? Which decade?
Unfortunately, current Red Hat installers, including RHEL. I don't think they've changed it in RHEL 6 (we're certifying that now). Yes, it's bad, and it has the added problem of making cross platform script very annoying (Wait, RHEL puts that in/bin, not/usr/bin, but this other is/usr/bin, but Solaris, HP-UX and AIX it's all/usr/bin or/sbin for a few critical items).
Actually, we use it successfully in the on a very large number of systems across multiple operating systems.. Common problems we used to encounter which drove us to this is one vendor insists/usr/local is owned by them, another says they own/opt, a third claimed they should drop binaries in/usr/bin, named the same as OS binaries of course (yes, deliberately replacing some key binaries). By using a neutral directory on all systems, you protect yourself from badly behaved vendors overwriting or blocking your company local utilities and eliminate unnecessary arguments about/opt vs/usr/local. You once make one change to your standard PATH and you are done.
It helps that we recompile all code ourselves, we don't use pre-built packages. We know better than to trust that code that isn't compiled by us to cooperate nicely with others.
Applications using the OS packaging is just evil. In the real world, every time any application runs as root or installs as root, you've lost a key battle in your effort to secure your system and keep it maintainable. Those applications that follow our standard are a breeze to keep running. They don't touch any other applications on the system, and you never worry about conflicting library requirements because you end up stacking 30-60 applications on a single logical system. We even can patch with fewer problems because the application doesn't rely on crazy libraries. They want a library version specifically? They provide their own. (Used to bite us a lot on Red Hat).
The problem with packaging is that it runs as root. The SA either has to spend a lot of time validating the package to ensure it won't break the OS or violate security standards, or just give up and assume that the application just installed a back door on the box in the package scripts. Keep the apps away from root forever, and you've just closed one of the most significant sources of errors. I used to review application packages, and it was common to see established vendors pull stunts like create UID 0 accounts, open up network listeners as root, add root.rhost entries and turn on r-services, or otherwise create numerous back doors. Some even did chmods on key binaries to make them world writable.
Save yourself pain. Make a new filesystem naming point, say/usr/companyname. Put your company specific code into that directory. Isolate all applications into some application naming structure, say/appl/foo for application foo. Don't allow applications to put so much as an init script elsewhere, because you were smart and told them they aren't allowed to run any code as root, not even startup scripts.
/bin separate from/usr/bin has been a major annoyance to me in trying to administrate RHEL boxes alongside HP-UX, AIX and Solaris. "Oh, yeah, RHEL puts that one in/bin, but this other one in/usr/bin, gotta remember which is where."
40MB hard drives have been gone for a very long time./bin should only be a symlink to/usr/bin as they are on say HP-UX and Solaris and AIX. The only things that should go into/sbin are those things absolutely required to boot or single-user repair (fsck, mount, etc.) a filesystem so it can be booted in case you have an idiot who still thinks/usr is too big to fit on the same spindle as/. (That said, combining/sbin and/usr/sbin is a good idea to me).
Now, this does not cover some of the other filesystem stunts pulled by RHEL and Fedora, but binary locations is something I've never seen Red Hat do intelligently.
Give me a fresh from college student any day over some of the people I have to work with who have 20+ years (and in a few cases, 40+ years) experience in computers. There is a lot less to unlearn, many fewer problems to train out of them. I've seen MIS degrees (far from very technical) turned into useful to help out the senior SA within a month. They weren't doing the heavy lifting, but they took off some of the simpler stuff to give more time for heavy lifting.
A coder should also be a damn good system admin and be capable of database admin duties as well. Being able to do tech writing as well won't hurt. You don't know what's going to be in demand tomorrow, you only know what was in demand when you last applied for work.
As a security and system administrator, my nightmare are coders who think they have a clue how to do system administration. Developers are constantly whining that they should have full root because they "know more than the system administrators" (often in so many words). Then we go in and look at what the developers did before we took root away and the system is in the act of falling over because so much was kludged it'd be simpler to simply reload the OS from scratch and reload the application, except the developers coded the application to require the broken OS configuration because it was easier for them.
I've met system admins who stopped being coders and became dedicated sysadmins. I've met a few generalists who know a little coding and a little sysadmin and a little security. Coders should realize they are not system administrators and that it is not just a subset of system administration but a completely different area that they should not pretend to know unless they really are doing it with just as much vigor as they are expected to approach programming.
During World War I and II, individuals in some cases joined the armies of those nations fighting against the US. That made them legitimate targets for military action. The most significant precedent however, is the US Civil War.
It would be hard to argue that a leader in a group that the US has effectively declared war on (including resolutions of Congress that authorize military force) is not a legitimate military target.
Depends on the model. At least some models (Saturn before they were killed for example), there is an external antenna. While it may not be obvious, without significant tools, the antenna can be readily removed. The portion in the dash I was told was the dedicated fuse/circuit breaker. The reason to go to the dealer was to get the cosmetic cover to go over where the antenna was.
Last time I shopped for a car, I told the dealer that disabling the interface so it couldn't be activated remotely was a deal breaker. Manager came over and on a demo car showed how in two minutes they could remove the antenna and attach a cosmetic cover where the antenna used to be. That was about four years ago. Even then, it was known that the service was being activated to monitor position without permission of the owner.
RHEL 6 broke that. They stopped supporting syslog and went to rsyslog which is incompatible and is breaking things left and right. Yes, we entered a show-stopper bug on certification with RH.
No, those of you who are rsyslog fans, it is not syslog. It is an abomination wearing a poorly fitting skin of syslog, but the rotting flesh is visible underneath, the miasma of decay of what was once a useful tool has left a shambling terror that cannot be abided.
For a moment, I thought you were one of the people on our log analysis team that I work with.
There are places where a database is good. I recommend them heavily, but the original log output shouldn't be database. If you want a database of logs (appropriate in many cases), send a copy of the logs off host (which you have to do anyway for PCI and many other audits) and store that copy in a database. That makes it easier to comply with other requirements.
Best of both worlds. Plain text local to box in case Things Go Wrong, and database for cross-system forensic analysis.
Actually, if you are at that point, you need to centralize your logs or use a proper alerting system beyond traditional logging because you are at the point you need to correlate events across systems. "Alert if this event occurs on system A unless you see this other event on system B or C"
Putting a database to manage logs on each system causes grief. We tried that once with wtmp. It hasn't worked out well, and we still don't have a clean replacement for wtmp that records logout times. So instead, I'm faced with custom tools to parse the wtmp file and eliminate events where the user has logged out, leaving events in the file where the user has logged in but has not logged out. (it's painful)
All the commercial folks have gone away from standard relational databases for the log storage because it is suboptimal. Instead, a proprietary database optimized just for log analysis and retention is used. It's a standard sedan being used as a commercial truck. You can make it sort of work, but you'll feel the pain.
My experience is based on an order of magnitude higher than the threshold I indicated, but I can tell we exceeded traditional databases around the 5k hosts level, and we aren't even doing complex log analysis. A few filters to identify specific log event types (e.g. sudo traffic is reformatted so that key items of the sudo become specific columns for easier searching.
What made your needs more complex was the request for contextual filtering. I also assumed you were needing contextual filtering across multiple hosts. That's when it gets hairy and you really notice the difference with a proper tool.
For low numbers of systems, splunk is great. When you have to go geographically diverse, my advice is to avoid splunk. They don't concentrate their log events into a single datacenter for cross-system reports and filters. (They tried to sell to us, and that was a deal breaker here.)
As I said, for smaller numbers of systems, hardware can overcome less efficient code. It's when you try to scale up that you experience the pain, or you try and bolt on things like centralized log protection, long term retention of your alerts because the auditors want proof six months later you sent out an alert, etc. It turns out you hit that level surprisingly quick in our experience.
And for the record, I'm quickly coming to the conclusion that rsyslogd is evil. We've blocked RHEL 6 certification on the use of rsyslogd alone because it messes with log output format and is so devilishly hard to auto-configure and validate that configuration is correct.
Depends on your scale. If you are talking 5k+ systems, then yes, you need something more serious. You could try and grow it at home, but expect problems when it grows or you add too many alerts. If you are talking about trivial usage (at most a few hundred systems, and none of them are badly misconfigured so as to be spamming the log daemons with dozens if not hundreds of messages per second), then you almost don't care what you use. Hardware can then overcome inefficient code.
Real time analysis isn't my primary specialty (I do long term storage and forensic analysis mostly), but usually, one needs to concentrate the logs into an analysis engine that can do comparisons of log events from different systems (if condition A on system primary, and not condition B on system secondary, alert). Once that is done, store the event for long enough to understand any failures where alerts failed to go out so you can build better alerts for next time. Typically, that's order of magnitude one to two weeks, though I've heard of shorter. You run into the risk of not finding the key event before it expires if you go too short. If you have large numbers of clients or geographic distribution, then a series of log concentrators is important, especially if you want to put your main log storage systems in an isolated network for additional safety.
If you just have a small number of systems, then it's relatively easy. If you have large numbers of systems, then that central analysis engine becomes a major bottleneck and that's where the commercial products shine. Obviously, if you have no cases where the logs of system 1 alter the interpretation of the logs of system 2, that makes it much simpler, and you can then parallelize your log analysis much easier. I rarely see that as a need.
The act of invoking some action is the easiest portion of all. I've seen almost nothing (including vanilla syslog) that can't feed a line into a program that executes an alert action. If nothing else, use named pipes to a program that calls snmptrap or wget or curl or whatever your alert program requires for input.
That's called context filtering, and is a standard feature of any serious log analysis tool. Okay, so Loglogic is not a serious log analysis tool. If you find yourself needing context filtering, odds are, you have sufficient needs to consider genuine log analysis tools (e.g. splunk, sensage, etc.). I've worked heavily with such tools and the real power is when you have a full year of forensic log storage to analyze and can fairly readily run very unusual reports on that data that you never thought was significant when you captured it nine months ago. Yes, contextual comprehension is normal.
To answer the question you didn't ask, real time analysis and forensic after-action analysis are different enough areas as to warrant different tools. The best approach I've seen done thus far is to collect them at the same point and then split the log feed, sending one to a long term storage, after-action analysis tool, and a copy of such to a real time analysis tool. It may seem on first glance like they should be easily combined, but there are some incompatible optimizations for the two uses that have come out not only in my experience, but the experience of a number of other people I've worked with who do log storage/analysis for a living.
Execution of arbitrary action to trigger alerts is very normal.
Replication is in syslog if you want.
If you really want my wish list for syslog, it is to reexamine the facilities. We don't need uucp or news any longer, but it would be nice to offer support for extended facilities.
Beyond that, everything I've ever found myself wanting is already in syslog-ng but one. I want a program, distributed with any log daemon, that will parse the configuration file and if given a message string, program or facility.priority pair, will tell me all destinations it would go to. Some of us have to automatically ensure that this or that custom log file is configured correctly.
There already is a few syslog compatible daemons for Windows. Syslog-ng has one such port. Many of the large scale commercial syslog aggregators (spunk, sensage, ...) have tools/products to extract the native Windows logs. They don't necessarily work the same way as syslog does, but there are still options there.
I'm more looking for a usable oracle log extractor since Oracle is ripping out the syslog support.
Expected rate of log messages from systems I monitor is low enough that ten seconds downtime of syslog wouldn't be noticed except for a few misbehaved systems. Temporary network disconnects, nothing interesting going on, etc. all makes logging faster than that cause an unacceptable volume of traffic. Yes, I could log all those hashes to a table with a mere two week retention instead of the full year required by PCI, but I don't think it would fly the way you intend.
In a well designed network, a compromise of a target system does not give one increased ability to compromise the log system, because there is no trust relationship, and the central log host does not even have the same user accounts. No user who has an account on the production system is permitted to have an account on the log system.
This topic is basic PCI stuff, and common also for SOX compliance. The problems are far from complex.
The so called solution does not provide sufficient security based on description to eliminate the requirement for central log storage, especially since that is an explicit requirement of PCI. Some may have that as an explicit SOX control as well. The obvious problem with the tool is it is only tamper detection, not tamper protection. Any fool with root can erase the evidence that they were the one logged in. It may be a tad harder to hide the fact that the logs were modified, but even that could be bypassed with the above description by simple virtue of rotating the log post-compromise and "losing" the entries in question.
PCI not only accepts, but mandates central log storage. Recommendation is to do it using real time log transmission. Log tampering detection is expected on even the central system, where you use syslog to receive, and then store the central copy in something a bit fancier than a text file, often a database of some kind.
Local to a system, there is no such thing as acceptable tamper resistant unless it is a write once/read many drive that is physically that way.
Centralize the logs, and stop messing with my syslog. (This means you Red Hat)
No law degree required, just fairly straightforward computer security for the most part. There are subtleties, but if people actually read the entire thing, it is amazing how clear much of it is.
PCI DSS 3.2 refers to the types of data that are not permitted to be stored. This does not necessarily include the actual credit card number. The original statement of payment account numbers not being permitted to be stored digitally is false. That does not refer to the magnetic track data or the card verification code (the number on the back of the card).
PCI DSS 3.4 discusses the requirements if PAN data is stored. One option, and a frequent subject of discussion in PCI certifications in my experience, is the encryption of such data and protections around ensuring all such data is encrypted and that the encrypting key is itself encrypted with a separate key that is independently protected.
The common failure is to accidentally store the magnetic stripe data or the CVV code. Even in encrypted form, that is prohibited.
PCI DSS does not prohibit storing the full payment account number (PAN) electronically, as long as it is encrypted. The note on PCI DSS 3.2.1 specifically talks about retaining the PAN in the normal course of business. PCI DSS 3.2.2 does prohibit storing the security code printed on the back, or the full magnetic track data. PCI DSS 3.4's requirement to render the PAN unreadable when stored makes it clear that storing that credit card number is permitted, if it is properly protected. The definition of properly protected is given.
I read the announcement as saying that the same database that housed some of the forum data also housed PAN data, but that they were claiming that table of the database was encrypted and thus don't believe it compromised.
One could argue that PCI DSS 2.2.1 (implement only one primary function per system) as violated, but that is debatable based on the few details publicly available.
There is too little available to gauge the incident at this time and guess specific PCI compliance failures.
Knowing one password does not materially help attacks on other passwords. However, depending on the algorithm used, it may be possible to brute force the password. For example, if the old Unix crypt(3c) algorithm is used, then most passwords can be brute forced in reasonable time now. Recent advances have led to use of the graphics card on your system to perform those attacks.
Longer hashes like MD-5 are significantly harder as they support a much longer search space, but few people use a password over twelve characters. Certainly, any password under seven characters should be considered vulnerable, regardless of algorithm used to salt/hash them.
Assuming (big if) they are using standard password hashing algorithms, long (at least 15 characters long) passwords that are pasted, not typed because they are completely randomly generated is your best protection in such cases.
Passwords are just evil though.
Uh.. what expects /usr to be a separate filesystem? Where? Which decade?
Unfortunately, current Red Hat installers, including RHEL. I don't think they've changed it in RHEL 6 (we're certifying that now). Yes, it's bad, and it has the added problem of making cross platform script very annoying (Wait, RHEL puts that in /bin, not /usr/bin, but this other is /usr/bin, but Solaris, HP-UX and AIX it's all /usr/bin or /sbin for a few critical items).
Actually, we use it successfully in the on a very large number of systems across multiple operating systems.. Common problems we used to encounter which drove us to this is one vendor insists /usr/local is owned by them, another says they own /opt, a third claimed they should drop binaries in /usr/bin, named the same as OS binaries of course (yes, deliberately replacing some key binaries). By using a neutral directory on all systems, you protect yourself from badly behaved vendors overwriting or blocking your company local utilities and eliminate unnecessary arguments about /opt vs /usr/local. You once make one change to your standard PATH and you are done.
It helps that we recompile all code ourselves, we don't use pre-built packages. We know better than to trust that code that isn't compiled by us to cooperate nicely with others.
Applications using the OS packaging is just evil. In the real world, every time any application runs as root or installs as root, you've lost a key battle in your effort to secure your system and keep it maintainable. Those applications that follow our standard are a breeze to keep running. They don't touch any other applications on the system, and you never worry about conflicting library requirements because you end up stacking 30-60 applications on a single logical system. We even can patch with fewer problems because the application doesn't rely on crazy libraries. They want a library version specifically? They provide their own. (Used to bite us a lot on Red Hat).
The problem with packaging is that it runs as root. The SA either has to spend a lot of time validating the package to ensure it won't break the OS or violate security standards, or just give up and assume that the application just installed a back door on the box in the package scripts. Keep the apps away from root forever, and you've just closed one of the most significant sources of errors. I used to review application packages, and it was common to see established vendors pull stunts like create UID 0 accounts, open up network listeners as root, add root .rhost entries and turn on r-services, or otherwise create numerous back doors. Some even did chmods on key binaries to make them world writable.
Save yourself pain. Make a new filesystem naming point, say /usr/companyname. Put your company specific code into that directory. Isolate all applications into some application naming structure, say /appl/foo for application foo. Don't allow applications to put so much as an init script elsewhere, because you were smart and told them they aren't allowed to run any code as root, not even startup scripts.
/bin separate from /usr/bin has been a major annoyance to me in trying to administrate RHEL boxes alongside HP-UX, AIX and Solaris. "Oh, yeah, RHEL puts that one in /bin, but this other one in /usr/bin, gotta remember which is where."
40MB hard drives have been gone for a very long time. /bin should only be a symlink to /usr/bin as they are on say HP-UX and Solaris and AIX. The only things that should go into /sbin are those things absolutely required to boot or single-user repair (fsck, mount, etc.) a filesystem so it can be booted in case you have an idiot who still thinks /usr is too big to fit on the same spindle as /. (That said, combining /sbin and /usr/sbin is a good idea to me).
Now, this does not cover some of the other filesystem stunts pulled by RHEL and Fedora, but binary locations is something I've never seen Red Hat do intelligently.
Give me a fresh from college student any day over some of the people I have to work with who have 20+ years (and in a few cases, 40+ years) experience in computers. There is a lot less to unlearn, many fewer problems to train out of them. I've seen MIS degrees (far from very technical) turned into useful to help out the senior SA within a month. They weren't doing the heavy lifting, but they took off some of the simpler stuff to give more time for heavy lifting.
A coder should also be a damn good system admin and be capable of database admin duties as well. Being able to do tech writing as well won't hurt. You don't know what's going to be in demand tomorrow, you only know what was in demand when you last applied for work.
As a security and system administrator, my nightmare are coders who think they have a clue how to do system administration. Developers are constantly whining that they should have full root because they "know more than the system administrators" (often in so many words). Then we go in and look at what the developers did before we took root away and the system is in the act of falling over because so much was kludged it'd be simpler to simply reload the OS from scratch and reload the application, except the developers coded the application to require the broken OS configuration because it was easier for them.
I've met system admins who stopped being coders and became dedicated sysadmins. I've met a few generalists who know a little coding and a little sysadmin and a little security. Coders should realize they are not system administrators and that it is not just a subset of system administration but a completely different area that they should not pretend to know unless they really are doing it with just as much vigor as they are expected to approach programming.
If a US born person happened to serve with the Barbary Pirates, you think that would have made them not a legitimate target for the military reprisal?
Authorized military action isn't only a congressional resolution that explicitly calls it war.
During World War I and II, individuals in some cases joined the armies of those nations fighting against the US. That made them legitimate targets for military action. The most significant precedent however, is the US Civil War.
It would be hard to argue that a leader in a group that the US has effectively declared war on (including resolutions of Congress that authorize military force) is not a legitimate military target.
Depends on the model. At least some models (Saturn before they were killed for example), there is an external antenna. While it may not be obvious, without significant tools, the antenna can be readily removed. The portion in the dash I was told was the dedicated fuse/circuit breaker. The reason to go to the dealer was to get the cosmetic cover to go over where the antenna was.
Last time I shopped for a car, I told the dealer that disabling the interface so it couldn't be activated remotely was a deal breaker. Manager came over and on a demo car showed how in two minutes they could remove the antenna and attach a cosmetic cover where the antenna used to be. That was about four years ago. Even then, it was known that the service was being activated to monitor position without permission of the owner.