It's actually even worse. The system in question (SIP) is an aggregation system from Nasdaq which aggregates orders and trades into stock quotes for people and companies that don't want to subscribe to the whole data flow just to get a nice stock quote once in a while. Which is why this to begin with only affected web sites and never any trades or HFT firms.
This system usually sends out test data after market close for various reasons. On the 3d of July the Nasdaq had an early close due to the 4th of July holiday so the test data sent this day where sent earlier than it usually is.
Now this should of course not be a problem because the system have so called market states, i.e you know when the market opens, when it closes. So if you are an average skilled developer you know that if you receive data after CLOSE then you discard it as test data until you get the OPEN the next trading day.
Now enter the not so average developers who instead rely on the timetstamp of the data in order to determine if the market is open or closed and you "accidentally" send out the test data as real data to various web sites.
This is why only some 3d party firms where affected and not everyone, only the ones with retard developers.
These prices where never distributed on systems where HFT source their data (the latency is to high) and it also happened after market close so there where no where to perform the trades anyway.
No it never happened that way, the problem happened after market close due to retard developers at some of the market data firms. Also the HFT firms would never source data from the system that "caused" this since the latency if far to high. The problem is that one of the aggregation systems from Nasdaq (SIP) sends out test data after market close and since the 3d of July was early close on Nasdaq this meant that the test data was sent earlier than before so devs who only looked at the timestamp instead of the market state displayed these values.
I agree that it's crappy (just not as crappy as the millions of slashdot comments have made it out to be) and I do hope and think that this will be fixed. I.e I noticed that the bug on github was not closed but closed from public access which means that people within the systemd project can still discuss it internally and LP is not the systemd dictator since they have several maintainers now a days.
I see that you have not written sysv init scripts before.
The sysv init scripts from Linux does not work on FreeBSD i.e because they don't use sysv init to begin with. And there are big differences between Debian and Red Hat systems and then you have all the other distros with their little changes here and there. For the most part the init scripts are written by the distro maintainers and not by upstream due to these differences. And this is one of the things that made systemd be so quickly adopted by the distributions because they no longer had to maintain their old specific scripts.
Of course, but then we also have to dream the pipe dream that running a exploited daemon with a non privileged user will still protect the system. So far this have not worked out so well unfortunately. And there is a warning logged when this particular bug is encountered, that is how it was found in the first place.
And no I don't think that this behaviour of systemd is good or OK but I also don't see the sky falling.
The goal really is to be distribution agnostic, that is why the master list of unit files are managed in a central location from which the various distributions fetches the majority of their unit files (and contribute changes). Things like exec of course does exist but if you use a distro specific script there then your unit file will not be accepted upstream, but writing your own personal unit files are of course ok.
This is hardly a security hole. All sysv scripts have run as root for ages without people crying wolf, and for this to impact your security you have to first come up with a way where I can attack you with this bug with no other possibility for me to run things as root on your system first.
All that said, I believe that this behaviour is going to change, LP is not the sole dictator of systemd and note that the bug on GitHub was not closed but closed for public discussion so to me this looks like they discuss it internally.
You make it out to be far more complicated that it really is. Things only get close to complicated if the unit adds dependencies to other events and if that happens then whoever wrote the unit file did so on purpose, and boy o boy would a corresponding sysv script be a mess.
And the code comments I cannot take seriously since the "everyone and their mom" seams to be people who actually never looked at the code nor know how code looks like. I've cloned the git repository and looked at the code when I was wondering a specific thing and had no problem following the logic, but if that makes me some wizard coder then I'm all for it:)
But it will only happen on home written unit files. If you where a maintainer and wrote a unit file with the intention of running under user X and #1 ignored the warning in the log and #2 didn't test if it really started as user X then you should think over if you should be a maintainer in the fist place. And with that said how many sysv scripts runs as non root? I would say none does, yes some daemons changes user once spawned if configure to do so but then they would do the very same under systemd regardless of this bug or not.
But this is not how it works, in your scenario the unit would not start.
Note that if you specify a valid user name but where the user doesn't exist, then we'll instead fail the service on start, because in that case there's not just something wrong with the syntax the service author used but actually something inconsistent on the system, and that should be considered fatal.
That is not how this bug works though. If the user in the unit file is valid from the viewpoint of systemd but no longer found on the system then the unit will not start:
Note that if you specify a valid user name but where the user doesn't exist, then we'll instead fail the service on start, because in that case there's not just something wrong with the syntax the service author used but actually something inconsistent on the system, and that should be considered fatal.
The problem is not having to wait a few ms for syslog to startup, the problem is that syslog have dependencies (like i.e access to a file system) that needs to be up before it can be started and thus everything that logs before that is missed unless something else does it.
And journald is not something that "everyone hates", some people here on Slashdot does yes, but there are lot's of people out in the real world who uses it and prefers it to the old syslog. I was myself quite sceptical when journald was announced with binary files but now that I've used it for some time I do realise that there are things it does and that it does well that would be just impossible or extremely complicated to do with the old syslog text format (i.e that the journal catches all output from stdout and stderr and joins it with the syslog output by adding the unit name in the meta-data, or that you can search for all logs produced by a special unit without accidentally get logs from other applications mixed in due to them matching the same grep pattern).
Since when has the service control manager in Windows "done things well"? It's a complicated mess that you need 3d party tools just in order to add services, dependencies are hell to setup and you need special code in your application in order to run as a system service; why only have main() when you can have ServiceMain()!
Where are not talking about moving files, we are talking about executing unit files on different systems. Like i.e trying to execute the sysv script from RHat on a FreeBSD system.
They are not only in a low UID range, they are specifically lower than SYS_UID_MAX which in a well setup system means that you can separate them from the other users, i.e if you make sure that UID_MIN is > SYS_UID_MAX. But what matters more in this particular context is what they are intended to be used for; to be used as users for system daemons. The quote from Lennart that makes you so upset is that he thinks that the naming rules for such users should adhere to a higher/stricter standard which for one would rule out "0day" as a valid name.
Neither UNIX nor Unix nor Linux even notices if you violate the conventions around "system" users.
That's the price you have to pay when the goal is to have distro agnostic unit files. And apparently there are Linux distros which forbids usernames with prefixing numbers, it does work on my Ubuntu so I guess that it's blocked on Rhat or something.
And your sysv scripts runs as what now? And if someone could change your unit file to contain a "User=0xxx" then they already own you since it requires root to edit the file and to launch it.
Slashdot Mirror
It's actually even worse. The system in question (SIP) is an aggregation system from Nasdaq which aggregates orders and trades into stock quotes for people and companies that don't want to subscribe to the whole data flow just to get a nice stock quote once in a while. Which is why this to begin with only affected web sites and never any trades or HFT firms.
This system usually sends out test data after market close for various reasons. On the 3d of July the Nasdaq had an early close due to the 4th of July holiday so the test data sent this day where sent earlier than it usually is.
Now this should of course not be a problem because the system have so called market states, i.e you know when the market opens, when it closes. So if you are an average skilled developer you know that if you receive data after CLOSE then you discard it as test data until you get the OPEN the next trading day.
Now enter the not so average developers who instead rely on the timetstamp of the data in order to determine if the market is open or closed and you "accidentally" send out the test data as real data to various web sites.
This is why only some 3d party firms where affected and not everyone, only the ones with retard developers.
No they are not. Stocks priced at 1 USD and above are traded at two decimals an stocks with a price below 1 USD are traded at four decimals.
These prices where never distributed on systems where HFT source their data (the latency is to high) and it also happened after market close so there where no where to perform the trades anyway.
No it never happened that way, the problem happened after market close due to retard developers at some of the market data firms. Also the HFT firms would never source data from the system that "caused" this since the latency if far to high. The problem is that one of the aggregation systems from Nasdaq (SIP) sends out test data after market close and since the 3d of July was early close on Nasdaq this meant that the test data was sent earlier than before so devs who only looked at the timestamp instead of the market state displayed these values.
I agree that it's crappy (just not as crappy as the millions of slashdot comments have made it out to be) and I do hope and think that this will be fixed. I.e I noticed that the bug on github was not closed but closed from public access which means that people within the systemd project can still discuss it internally and LP is not the systemd dictator since they have several maintainers now a days.
I see that you have not written sysv init scripts before.
The sysv init scripts from Linux does not work on FreeBSD i.e because they don't use sysv init to begin with. And there are big differences between Debian and Red Hat systems and then you have all the other distros with their little changes here and there. For the most part the init scripts are written by the distro maintainers and not by upstream due to these differences. And this is one of the things that made systemd be so quickly adopted by the distributions because they no longer had to maintain their old specific scripts.
Of course, but then we also have to dream the pipe dream that running a exploited daemon with a non privileged user will still protect the system. So far this have not worked out so well unfortunately. And there is a warning logged when this particular bug is encountered, that is how it was found in the first place.
And no I don't think that this behaviour of systemd is good or OK but I also don't see the sky falling.
Oh a funny AC who sees no merit in man pages. Junior College? Now I'm not American, but that would have been over 20 years too late if I where.
The goal really is to be distribution agnostic, that is why the master list of unit files are managed in a central location from which the various distributions fetches the majority of their unit files (and contribute changes). Things like exec of course does exist but if you use a distro specific script there then your unit file will not be accepted upstream, but writing your own personal unit files are of course ok.
This is hardly a security hole. All sysv scripts have run as root for ages without people crying wolf, and for this to impact your security you have to first come up with a way where I can attack you with this bug with no other possibility for me to run things as root on your system first.
All that said, I believe that this behaviour is going to change, LP is not the sole dictator of systemd and note that the bug on GitHub was not closed but closed for public discussion so to me this looks like they discuss it internally.
You make it out to be far more complicated that it really is. Things only get close to complicated if the unit adds dependencies to other events and if that happens then whoever wrote the unit file did so on purpose, and boy o boy would a corresponding sysv script be a mess.
And the code comments I cannot take seriously since the "everyone and their mom" seams to be people who actually never looked at the code nor know how code looks like. I've cloned the git repository and looked at the code when I was wondering a specific thing and had no problem following the logic, but if that makes me some wizard coder then I'm all for it :)
Sorry, forgot that what you need should dictate the need of any other admin out there. So easy to forget such little details.
But it will only happen on home written unit files. If you where a maintainer and wrote a unit file with the intention of running under user X and #1 ignored the warning in the log and #2 didn't test if it really started as user X then you should think over if you should be a maintainer in the fist place. And with that said how many sysv scripts runs as non root? I would say none does, yes some daemons changes user once spawned if configure to do so but then they would do the very same under systemd regardless of this bug or not.
But this is not how it works, in your scenario the unit would not start.
Note that if you specify a valid user name but where the user doesn't exist, then we'll instead fail the service on start, because in that case there's not just something wrong with the syntax the service author used but actually something inconsistent on the system, and that should be considered fatal.
That is not how this bug works though. If the user in the unit file is valid from the viewpoint of systemd but no longer found on the system then the unit will not start:
Note that if you specify a valid user name but where the user doesn't exist, then we'll instead fail the service on start, because in that case there's not just something wrong with the syntax the service author used but actually something inconsistent on the system, and that should be considered fatal.
How are the unit files not "human readable configurations", especially when compared with the old sysv init scripts?
The problem is not having to wait a few ms for syslog to startup, the problem is that syslog have dependencies (like i.e access to a file system) that needs to be up before it can be started and thus everything that logs before that is missed unless something else does it.
And journald is not something that "everyone hates", some people here on Slashdot does yes, but there are lot's of people out in the real world who uses it and prefers it to the old syslog. I was myself quite sceptical when journald was announced with binary files but now that I've used it for some time I do realise that there are things it does and that it does well that would be just impossible or extremely complicated to do with the old syslog text format (i.e that the journal catches all output from stdout and stderr and joins it with the syslog output by adding the unit name in the meta-data, or that you can search for all logs produced by a special unit without accidentally get logs from other applications mixed in due to them matching the same grep pattern).
Since when has the service control manager in Windows "done things well"? It's a complicated mess that you need 3d party tools just in order to add services, dependencies are hell to setup and you need special code in your application in order to run as a system service; why only have main() when you can have ServiceMain()!
Yeah systemd is such a hard dependency on Gnome that you cannot run Gnome3 on say FreeBSD, oh wait, it does!
So that is why I can run Gnome3 on FreeBSD, because FreeBSD adopted systemd? Are you sure?
He asked, and he asked in a way that suggested that he didn't believe that system users existed so I quoted a man page as a form of QFT.
Where are not talking about moving files, we are talking about executing unit files on different systems. Like i.e trying to execute the sysv script from RHat on a FreeBSD system.
They are not only in a low UID range, they are specifically lower than SYS_UID_MAX which in a well setup system means that you can separate them from the other users, i.e if you make sure that UID_MIN is > SYS_UID_MAX. But what matters more in this particular context is what they are intended to be used for; to be used as users for system daemons. The quote from Lennart that makes you so upset is that he thinks that the naming rules for such users should adhere to a higher/stricter standard which for one would rule out "0day" as a valid name.
Neither UNIX nor Unix nor Linux even notices if you violate the conventions around "system" users.
Which of course no one claimed, nice strawman.
That's the price you have to pay when the goal is to have distro agnostic unit files. And apparently there are Linux distros which forbids usernames with prefixing numbers, it does work on my Ubuntu so I guess that it's blocked on Rhat or something.
And your sysv scripts runs as what now? And if someone could change your unit file to contain a "User=0xxx" then they already own you since it requires root to edit the file and to launch it.
And logs that it did happen. Compare that with what would happen if say /var/log/syslog would be corrupted by inserting say a 0x00 byte somewhere ;)