Capsicum, POSIX and NFS4 ACLs are all about adding complexity to allow for greater administrative policy enforcement
This is almost true for ACLs. ACLs are no more expressive than standard UNIX permissions, but they are significantly simpler for implementing the same thing - you no longer need to create a group for every set of people who want to share things. This lets you leave your default at share-nothing, and explicitly share the things that you need to share with the people that you need to share it with. The code for implementing them is significantly less complex than the work arounds that you need for their absence if you want the same level of access control, and if you don't want the same level of access control it's because you're fine with leaving things more widely readable than they need to be. Neither of these attitudes is good for security.
Capsicum is definitely not about adding complexity. The implementation adds an extra bitmask check on file accesses and restricts system calls to a whitelisted set. The total code changes in the kernel are very small and easy to audit (and have been audited by several groups). The code changes in userspace code are far more significant. The sandboxing in Chromium, for example, is six times more lines of code on OpenBSD using chroot() than it is on FreeBSD using Capsicum, and offers less isolation (for example, the renderer processes on OpenBSD can create network sockets, so an image in an email that exploits libpng or libjpeg vulnerabilities can phone home and send copies of all of your emails if you use webmail from OpenBSD, with Capsicum is can't). The privilege separation code in OpenSSH is also cleaner and easier to audit when it uses Capsicum.
In OpenBSD, security means that you eliminate bugs so that the most basic promise is held true.
In FreeBSD, we care about mitigation. Useful software is never bug free, no matter how simple you make it. The goal is to ensure that once an attacker finds a bug, they can't use it to exploit the system. That doesn't mean 'they can't get root', because on a huge number of modern systems, from single-user laptops to single-service VMs, getting ambient authority for a single user can mean the same as getting root, when it comes to having access to the data that the user cares about. Jails, Capsicum, and so on are all about enforcing the principle of least privilege, so when a bug is discovered the attacker only gets control of a sandbox with no access to the rest of the user's data. This used to be something that OpenBSD people cared about.
This phrasing implies that FreeBSD doesn't care about security, which is somewhat misleading. Take a look at the number of papers at Oakland that use FreeBSD as a base and the number of those that we've accepted into the base system sometime.
For example, FreeBSD has a modular mandatory access control framework with pluggable policies. This is used by some downstream projects for things like the OS X / iOS sandboxing subsystem and the JunOS code signing infrastructure. It's also used for the type enforcement mechanism in FreeBSD and a few other things.
FreeBSD has had jails for a long time, which are an easy way of creating a completely isolated environment (distinct root user, filesystem hierarchy, and so on), sharing the same kernel (so much cheaper than a VM). With ZFS and cheap clones, it's easy to keep a stock FreeBSD install around, clone it, and have a new jail up and running in a few seconds. Poudriere (French for Powderkeg), the package building program uses this mechanism to create a pristine environment for building each package, allowing untrusted intermediate binaries to be run on the build server without giving them any access to the host environment.
In FreeBSD 10, Capsicum is enabled by default in the kernel, and a number of base system utilities are sandboxed out of the box. Capsicum sandboxing means that things run without the ability to create file descriptors and have to request a more-privileged daemon pass them the ones that they need. For example, tcpdump (which doesn't have the best security record in the world and is often used to inspect malicious network traffic) runs in compartmentalised mode and just delegates DNS lookups to a daemon that runs outside of the sandbox. The daemon only accepts records in fixed formats (IP addresses) and returns a string, so it has a very small attack surface.
FreeBSD supports both POSIX and NFSv4 ACLs, for people who want finer-grained filesystem security. With 10 (and, I think, 9.2?), the audit log daemon is complemented by auditdistd, which allows you to distribute cryptographically signed audit logs across machines, allowing you to detect suspicious activity.
The binary packages that we're distributing are signed, by a key that is distributed via a different mechanism to the packages, and the pkg audit command allows you to display known vulnerabilities in any of your installed third-party packages.
ZFS now is pretty solid, but there were some interesting teething problems with ZFS. The rule of thumb that filesystems people tell me is that it takes 10 years for a new FS to be solid. ZFS is now 9 years old, so is very close...
In early versions, there was a problem that a write, including a delete, would cause the filesystem to grow (if it's a delete, it would then shrink) and you could get into a state if you let your filesystem (almost) completely fill up where you didn't have enough space to be able to do the copy-on-write deletion and the only way to recover was to copy all of the data off, recreate the pool, and copy it back on. You didn't lose data, but you did lose the ability to write to the pool. There have also been some cases where data loss (including the entire pool) could happen, although those are far rarer.
The down side is that 10 uses a newer version of the ZFS on-disk format that 9 can't load. I managed to hit this, accidentally doing an installkernel with a custom kernel config from a 9-STABLE tree just after doing a binary update to 10-RC3. The 9 kernel couldn't mount the 10 ZFS root, and I then had to find a bootable 10 CD (it turns out the machine I did this on can't boot from USB) to reinstall the 10 kernel. Worst of all, I discovered that the kernel option I wanted actually was in the default config in 10, I just didn't think it was because I'd told freebsd-update not to update my src tree to speed up the updates. A whole sequence of operator errors, and fortunately a recoverable one (once I'd replaced the kernel with the one from the CD, it worked perfectly).
With 10.0, you most likely want to be using binary packages, either from FreeBSD.org, or by rolling your own with poudriere. If you're used to using ports with custom options, the best thing to do is install poudriere and put all of the configuration options in a make.conf, dump the installed package list to a file, and then use poudriere bulk to build that set. You can then point pkg at the local repository and install things. Ideally, make a cron job that updates the ports tree and reruns the build overnight, so you can update whenever you want.
For the P3, I'd recommend using freebsd-upgrade and pkg, unless you really need a custom kernel. You can also do make toolchain on a faster machine and then copy your obj tree across and use the XDEV stuff if you really need to be building kernel and world on it.
The en0 becoming xn0 thing surprised me too, when I switched from a GENERIC kernel to a XENHVM one on 9.0. With 10.0, I think we're compiling the Xen HVM drivers into the GENERIC kernel, so you'll get the new devices. In the Xen block device drivers, I think there's some extra magic so that they'll appear with a different device node name if the device was previously used with the emulated devices, but that isn't present in the network drivers, which I think is a shame.
For the Geode, it shouldn't be an issue since September. Prior to that, clang would emit long nops for some things that would break the Geode.
Re:I wish FreeBSD had a decent VM server/hyperviso
on
FreeBSD 10.0 Released
·
· Score: 2
I think your numbers for KVM are for the entire kernel, not just the VM support, but your bhyve numbers are for the bhyve kernel module, which depends on a lot of other stuff in the kernel (the VM subsystem, device drivers, at least one out of the network and storage stacks). The Xen number includes, I think, includes just the hypervisor, not the domain 0 guest that is responsible for running the control plain, providing all of the emulated devices, and so on.
Re:I wish FreeBSD had a decent VM server/hyperviso
on
FreeBSD 10.0 Released
·
· Score: 2
Some guys at Spectra Logic and Citrix have been pushing Xen PVH support into FreeBSD recently. It didn't make it for 10.0, but hopefully should be in 10.1. In PVH mode, a guest boots as if it is a PV guest, with Xen's entry point and event channels instead of interrupts, but then uses the hardware page tables and either PCI pass-through devices or PV devices. This is important, because PVH guests can also run as dom0, if they implement the management interfaces (which are mostly userspace and shared across platforms). Getting PVH working in domU is much more effort than going from a working domU PVH to a working dom0 PVH.
That said, with bhyve and the new vps work (shared kernel like jails, but with some nifty features like live migration between hosts), there's a lot less of a reason for me to care about Xen, in relation to FreeBSD.
Re:I wish FreeBSD had a decent VM server/hyperviso
on
FreeBSD 10.0 Released
·
· Score: 1
The distinction between Type 1 and Type 2 hypervisors doesn't really make sense anymore, and is largely marketing. For example, Xen running on a system with no IOMMU is still a Type 1 hypervisor, but the whole of the dom0 kernel and a chunk of its userspace are in the trusted computing base (TCB), where a compromise can extend to every other domain. In the case of Linux's KVM and bhyve, the host kernel (and anything that can poke kernel memory, or exploit kernel vulnerabilities) are in the TCB.
The distinction makes more sense for certain ARM, SPARC, and PowerPC hypervisors, where the hardware handles most of the virtualisation (devices all support multiple isolated command queues and run behind an IOMMU), so a small Type 1 hypervisor can be a few tens of thousands of lines of code and that's the entire shared TCB. Guests run their own device drivers, the hypervisor just handles the control plane.
The important distinction is between hypervisors that do software partitioning of devices (either via emulation or paravirtualised devices), and ones that don't. Xen, KVM, and byhve all fall into the former category and so have a TCB that's several MBs of object code.
The major release numbers in FreeBSD mean breaks in binary compatibility. Any binary, including kernel modules, that runs on FreeBSD x.y is expected to run on FreeBSD x.(y+1). Userspace binaries should also work on (x+1).z, but may require compatibility packages to be installed (for userspace libraries that had ABI changes or were removed) and may require the kernel to be built with compatibility options enabled. This is the default for the GENERIC kernel, but some embedded builds disable them, and for some architectures there's no point in enabling them if FreeBSD didn't run on that architecture at the time.
Add to that, when we test compiler optimisations we do it on some body of code, on some set of microarchitectures, and enable it if it is a net benefit over our sample set. We don't (and can't) test every possible combination of code and microarchitectures. One of my favourite stories in this regard is actually a CPU optimisation rather than a compiler one. A newer generation of ARM core improved the branch predictor, and most things got faster. One core library used by a certain mobile phone OS got noticeably slower. It turned out that in the old CPU, the wrong branch was being predicted at a specific point that caused a load instruction to be speculatively executed and then discarded. When they improved the prediction, the correct path was taken. The value of the load was required some time later in this case. The bad behaviour was costing them a pipeline flush, but pulling the data into the cache. The good behaviour was causing them to block for a memory read. A stall of a dozen or so cycles became a stall of a hundred or so cycles, even though the new behaviour was effectively better.
For compilers, you'll see similar problems. A few years ago, I found that my Smalltalk compiler was generating faster code than gcc for a fibonacci microbenchmark. It turned out that gcc just happened to hit a pathological case for cache layout in their code, which was triggering a load of instruction cache misses. Trivial tweaks to the C code made it an order of magnitude faster.
If you really care about performance for your code, you should be regularly building it with the trunk revision of your favourite compiler and reporting regressions early. Once there's a test case, we have something to work with.
It's much easier if your country doesn't follow the insane zoning policies of most of the US, which seem designed to make sure that places people live, places people go to for work, and places people go to for shopping or fun are all as far away from each other as possible, unless you're right in the middle of a big city (and often even then, because most companies can't afford big offices in the middle of cities).
I experience the same kind of surprise at people who want to live in London. I've been offered a few jobs there, and the salary increases don't cover the hit you take to cost and quality of living, unless you work in the financial sector, and then they don't adequately balance the associated feeling of self-loathing.
Anonymised data rarely is. It's very easy to remove the relationship between John and Decaf, and as long as that dataset is completely isolated from every other dataset, then it's fine. If, however, it's combined with the things-John-bought-on-his-credit-card dataset, or the people-working-near-this-Starbucks-who-have-breaks-at-this-time dataset, then you can infer that this decaf drinker is John. Often you'll need two or three independent datasets to be able to reconstruct the original anonymised version, but data storage is cheap, and the processing power required to do the cross referencing is getting cheaper...
The problem was highlighted by the likes of Google in the recent NSA revelations. It doesn't matter why you collect the data, once it's collected and indexed conveniently it can be used for other purposes. Google builds a system that collects a large body of emails sent and received by millions of people. They may only intend to do anonymous, aggregate, data analysis on it to determine broad patterns, but once the data is collected and centralised it's easy to index it and search for specific emails. Even if you completely trust Google, Starbucks, or whoever is collecting the data now, do you trust every company that might buy them in the future, do you trust that their management in ten years will have the same views, and do you trust that no one malicious can compromise their systems? Snowden showed that even the NSA can't manage to keep systems that are completely secure, so what chance does anyone else have? Today, you are likely safe from invasive monitoring because no one has the processing capacity to handle tracking people as uninteresting as you, but we're still seeing processing power per dollar doubling roughly every 18 months, so how long will that be the case for?
Prices in Europe have also gone up a lot, so while you might expect to see different absolute values, you'd expect to see the same decline. I live in the UK and don't drive. Owning a car is a huge expense (insurance and maintenance, even before you add in the fuel) for little benefit. I've always taken jobs where I could either work remotely or walk / cycle less than 10 minutes each way. I wouldn't consider working somewhere where I'd need to drive to work, or where I'd spend more than 10 minutes commuting, and I'm always amazed at people in the US who are happy to spend more than the equivalent of one working day a week just getting to and from work. At least cycling in, I get some exercise at the same time.
None of these devices have floppy disks though. Compact Flash cards are a more common requirement, as they're basically IDE devices. I had one a few years ago with a custom firewall distribution that fitted onto a 32MB CF card, but a year later it was hard to buy a CF card smaller than 4GB and so I switched to a full OS install.
FreeBSD systems traditionally built their own packages from the ports tree. The shipped packages are really just the ones that go on the install media, which are typically out of date by the time you get around to installing it. With the new pkg(7) infrastructure, we are properly supporting binary packages and part of the requirement for this was that we'd sign them and distribute the keys out of band. The signing keys are currently distributed using freebsd-update, which was designed to do binary updates of the base system and does use signed deltas. These are then used by the pkg tool to verify packages.
Prior to this, we were distributing the ports tree using the portsnap utility, which also does cryptographic verification of the downloads using the same mechanism as freebsd-update.
What you do is, check out the source repository, which does make sure the data you get hasn't been tampered with, then build it from source.
Actually, it doesn't. OpenBSD is still using CVS for revision control. After the FreeBSD cluster compromise a couple of years ago, we found that the CVS repository was the one thing whose integrity we could not verify. The current FreeBSD CVS repository was created by exporting from subversion (which could be verified) and validating it against git (which also can be verified).
Oh, and OpenBSD does recommend getting the binary packages over using ports (but they don't release security fixes for binary packages, and don't support using a ports tree revision that doesn't match your base system revision), and they recommend getting binary packages with unencrypted, unauthenticated, FTP.
The Sale of Goods Act in the UK requires that goods are suitable for the purpose for which sold and must function in this manner, in normal use, for a period which is expected for the class of product. This electronic devices, unless explicitly stated before the time of sale (i.e. placing text clearly in the advert saying 'this device has a shorter useable life than others of its class'), this is six years. For other devices, such as kitchen appliances, it can be 10-20 years, during which the manufacturer is obliged to provide service for all faults that are not directly caused by the owner. The EU warranty law actually gives you weaker protection than the UK law, but feel free to keep boycotting UK companies because of that if it makes you feel better...
I've seen the opposite. In the UK, if you quote the Sale of Goods Act when you call them up, they'll quickly replace faulty parts 4 years after the original purchase. In the US, as soon as the warranty expires, you're fucked if anything goes wrong.
If Darwin's theory had come from a computer model and it turned out his computer model didn't match real world observations, nobody would believe it.
People argued that Darwin's theory was wrong because people's grandfathers obviously (or, in some cases, less obviously) weren't monkeys. It was a nonsense argument, because they weren't actually comparing Darwin's theory with reality, they were comparing the media soundbite version of his theory with reality. AGW also fails to stand up to this kind of attack, but that doesn't mean that the theories are wrong.
Add to that, unless you're in the running for a Nobel, your academic career isn't likely to be tied to a specific theory. No one is discredited for saying 'we did this work, and based on the evidence that we had we advanced this theory, but now we have more data we've disproven it and now advance this theory'. In fact, that's what a lot of papers do say. It's much easier to get that kind of paper published than one that says 'we thought this, and now we have more evidence, and we think it's still true.'
Slashdot Mirror
Capsicum, POSIX and NFS4 ACLs are all about adding complexity to allow for greater administrative policy enforcement
This is almost true for ACLs. ACLs are no more expressive than standard UNIX permissions, but they are significantly simpler for implementing the same thing - you no longer need to create a group for every set of people who want to share things. This lets you leave your default at share-nothing, and explicitly share the things that you need to share with the people that you need to share it with. The code for implementing them is significantly less complex than the work arounds that you need for their absence if you want the same level of access control, and if you don't want the same level of access control it's because you're fine with leaving things more widely readable than they need to be. Neither of these attitudes is good for security.
Capsicum is definitely not about adding complexity. The implementation adds an extra bitmask check on file accesses and restricts system calls to a whitelisted set. The total code changes in the kernel are very small and easy to audit (and have been audited by several groups). The code changes in userspace code are far more significant. The sandboxing in Chromium, for example, is six times more lines of code on OpenBSD using chroot() than it is on FreeBSD using Capsicum, and offers less isolation (for example, the renderer processes on OpenBSD can create network sockets, so an image in an email that exploits libpng or libjpeg vulnerabilities can phone home and send copies of all of your emails if you use webmail from OpenBSD, with Capsicum is can't). The privilege separation code in OpenSSH is also cleaner and easier to audit when it uses Capsicum.
In OpenBSD, security means that you eliminate bugs so that the most basic promise is held true.
In FreeBSD, we care about mitigation. Useful software is never bug free, no matter how simple you make it. The goal is to ensure that once an attacker finds a bug, they can't use it to exploit the system. That doesn't mean 'they can't get root', because on a huge number of modern systems, from single-user laptops to single-service VMs, getting ambient authority for a single user can mean the same as getting root, when it comes to having access to the data that the user cares about. Jails, Capsicum, and so on are all about enforcing the principle of least privilege, so when a bug is discovered the attacker only gets control of a sandbox with no access to the rest of the user's data. This used to be something that OpenBSD people cared about.
OpenBSD's goal is security above all else.
This phrasing implies that FreeBSD doesn't care about security, which is somewhat misleading. Take a look at the number of papers at Oakland that use FreeBSD as a base and the number of those that we've accepted into the base system sometime.
For example, FreeBSD has a modular mandatory access control framework with pluggable policies. This is used by some downstream projects for things like the OS X / iOS sandboxing subsystem and the JunOS code signing infrastructure. It's also used for the type enforcement mechanism in FreeBSD and a few other things.
FreeBSD has had jails for a long time, which are an easy way of creating a completely isolated environment (distinct root user, filesystem hierarchy, and so on), sharing the same kernel (so much cheaper than a VM). With ZFS and cheap clones, it's easy to keep a stock FreeBSD install around, clone it, and have a new jail up and running in a few seconds. Poudriere (French for Powderkeg), the package building program uses this mechanism to create a pristine environment for building each package, allowing untrusted intermediate binaries to be run on the build server without giving them any access to the host environment.
In FreeBSD 10, Capsicum is enabled by default in the kernel, and a number of base system utilities are sandboxed out of the box. Capsicum sandboxing means that things run without the ability to create file descriptors and have to request a more-privileged daemon pass them the ones that they need. For example, tcpdump (which doesn't have the best security record in the world and is often used to inspect malicious network traffic) runs in compartmentalised mode and just delegates DNS lookups to a daemon that runs outside of the sandbox. The daemon only accepts records in fixed formats (IP addresses) and returns a string, so it has a very small attack surface.
FreeBSD supports both POSIX and NFSv4 ACLs, for people who want finer-grained filesystem security. With 10 (and, I think, 9.2?), the audit log daemon is complemented by auditdistd, which allows you to distribute cryptographically signed audit logs across machines, allowing you to detect suspicious activity.
The binary packages that we're distributing are signed, by a key that is distributed via a different mechanism to the packages, and the pkg audit command allows you to display known vulnerabilities in any of your installed third-party packages.
ZFS now is pretty solid, but there were some interesting teething problems with ZFS. The rule of thumb that filesystems people tell me is that it takes 10 years for a new FS to be solid. ZFS is now 9 years old, so is very close...
In early versions, there was a problem that a write, including a delete, would cause the filesystem to grow (if it's a delete, it would then shrink) and you could get into a state if you let your filesystem (almost) completely fill up where you didn't have enough space to be able to do the copy-on-write deletion and the only way to recover was to copy all of the data off, recreate the pool, and copy it back on. You didn't lose data, but you did lose the ability to write to the pool. There have also been some cases where data loss (including the entire pool) could happen, although those are far rarer.
The down side is that 10 uses a newer version of the ZFS on-disk format that 9 can't load. I managed to hit this, accidentally doing an installkernel with a custom kernel config from a 9-STABLE tree just after doing a binary update to 10-RC3. The 9 kernel couldn't mount the 10 ZFS root, and I then had to find a bootable 10 CD (it turns out the machine I did this on can't boot from USB) to reinstall the 10 kernel. Worst of all, I discovered that the kernel option I wanted actually was in the default config in 10, I just didn't think it was because I'd told freebsd-update not to update my src tree to speed up the updates. A whole sequence of operator errors, and fortunately a recoverable one (once I'd replaced the kernel with the one from the CD, it worked perfectly).
With 10.0, you most likely want to be using binary packages, either from FreeBSD.org, or by rolling your own with poudriere. If you're used to using ports with custom options, the best thing to do is install poudriere and put all of the configuration options in a make.conf, dump the installed package list to a file, and then use poudriere bulk to build that set. You can then point pkg at the local repository and install things. Ideally, make a cron job that updates the ports tree and reruns the build overnight, so you can update whenever you want.
For the P3, I'd recommend using freebsd-upgrade and pkg, unless you really need a custom kernel. You can also do make toolchain on a faster machine and then copy your obj tree across and use the XDEV stuff if you really need to be building kernel and world on it.
The en0 becoming xn0 thing surprised me too, when I switched from a GENERIC kernel to a XENHVM one on 9.0. With 10.0, I think we're compiling the Xen HVM drivers into the GENERIC kernel, so you'll get the new devices. In the Xen block device drivers, I think there's some extra magic so that they'll appear with a different device node name if the device was previously used with the emulated devices, but that isn't present in the network drivers, which I think is a shame.
For the Geode, it shouldn't be an issue since September. Prior to that, clang would emit long nops for some things that would break the Geode.
I think your numbers for KVM are for the entire kernel, not just the VM support, but your bhyve numbers are for the bhyve kernel module, which depends on a lot of other stuff in the kernel (the VM subsystem, device drivers, at least one out of the network and storage stacks). The Xen number includes, I think, includes just the hypervisor, not the domain 0 guest that is responsible for running the control plain, providing all of the emulated devices, and so on.
Some guys at Spectra Logic and Citrix have been pushing Xen PVH support into FreeBSD recently. It didn't make it for 10.0, but hopefully should be in 10.1. In PVH mode, a guest boots as if it is a PV guest, with Xen's entry point and event channels instead of interrupts, but then uses the hardware page tables and either PCI pass-through devices or PV devices. This is important, because PVH guests can also run as dom0, if they implement the management interfaces (which are mostly userspace and shared across platforms). Getting PVH working in domU is much more effort than going from a working domU PVH to a working dom0 PVH.
That said, with bhyve and the new vps work (shared kernel like jails, but with some nifty features like live migration between hosts), there's a lot less of a reason for me to care about Xen, in relation to FreeBSD.
The distinction between Type 1 and Type 2 hypervisors doesn't really make sense anymore, and is largely marketing. For example, Xen running on a system with no IOMMU is still a Type 1 hypervisor, but the whole of the dom0 kernel and a chunk of its userspace are in the trusted computing base (TCB), where a compromise can extend to every other domain. In the case of Linux's KVM and bhyve, the host kernel (and anything that can poke kernel memory, or exploit kernel vulnerabilities) are in the TCB.
The distinction makes more sense for certain ARM, SPARC, and PowerPC hypervisors, where the hardware handles most of the virtualisation (devices all support multiple isolated command queues and run behind an IOMMU), so a small Type 1 hypervisor can be a few tens of thousands of lines of code and that's the entire shared TCB. Guests run their own device drivers, the hypervisor just handles the control plane.
The important distinction is between hypervisors that do software partitioning of devices (either via emulation or paravirtualised devices), and ones that don't. Xen, KVM, and byhve all fall into the former category and so have a TCB that's several MBs of object code.
The major release numbers in FreeBSD mean breaks in binary compatibility. Any binary, including kernel modules, that runs on FreeBSD x.y is expected to run on FreeBSD x.(y+1). Userspace binaries should also work on (x+1).z, but may require compatibility packages to be installed (for userspace libraries that had ABI changes or were removed) and may require the kernel to be built with compatibility options enabled. This is the default for the GENERIC kernel, but some embedded builds disable them, and for some architectures there's no point in enabling them if FreeBSD didn't run on that architecture at the time.
Add to that, when we test compiler optimisations we do it on some body of code, on some set of microarchitectures, and enable it if it is a net benefit over our sample set. We don't (and can't) test every possible combination of code and microarchitectures. One of my favourite stories in this regard is actually a CPU optimisation rather than a compiler one. A newer generation of ARM core improved the branch predictor, and most things got faster. One core library used by a certain mobile phone OS got noticeably slower. It turned out that in the old CPU, the wrong branch was being predicted at a specific point that caused a load instruction to be speculatively executed and then discarded. When they improved the prediction, the correct path was taken. The value of the load was required some time later in this case. The bad behaviour was costing them a pipeline flush, but pulling the data into the cache. The good behaviour was causing them to block for a memory read. A stall of a dozen or so cycles became a stall of a hundred or so cycles, even though the new behaviour was effectively better.
For compilers, you'll see similar problems. A few years ago, I found that my Smalltalk compiler was generating faster code than gcc for a fibonacci microbenchmark. It turned out that gcc just happened to hit a pathological case for cache layout in their code, which was triggering a load of instruction cache misses. Trivial tweaks to the C code made it an order of magnitude faster.
If you really care about performance for your code, you should be regularly building it with the trunk revision of your favourite compiler and reporting regressions early. Once there's a test case, we have something to work with.
It's much easier if your country doesn't follow the insane zoning policies of most of the US, which seem designed to make sure that places people live, places people go to for work, and places people go to for shopping or fun are all as far away from each other as possible, unless you're right in the middle of a big city (and often even then, because most companies can't afford big offices in the middle of cities).
I experience the same kind of surprise at people who want to live in London. I've been offered a few jobs there, and the salary increases don't cover the hit you take to cost and quality of living, unless you work in the financial sector, and then they don't adequately balance the associated feeling of self-loathing.
Anonymised data rarely is. It's very easy to remove the relationship between John and Decaf, and as long as that dataset is completely isolated from every other dataset, then it's fine. If, however, it's combined with the things-John-bought-on-his-credit-card dataset, or the people-working-near-this-Starbucks-who-have-breaks-at-this-time dataset, then you can infer that this decaf drinker is John. Often you'll need two or three independent datasets to be able to reconstruct the original anonymised version, but data storage is cheap, and the processing power required to do the cross referencing is getting cheaper...
The problem was highlighted by the likes of Google in the recent NSA revelations. It doesn't matter why you collect the data, once it's collected and indexed conveniently it can be used for other purposes. Google builds a system that collects a large body of emails sent and received by millions of people. They may only intend to do anonymous, aggregate, data analysis on it to determine broad patterns, but once the data is collected and centralised it's easy to index it and search for specific emails. Even if you completely trust Google, Starbucks, or whoever is collecting the data now, do you trust every company that might buy them in the future, do you trust that their management in ten years will have the same views, and do you trust that no one malicious can compromise their systems? Snowden showed that even the NSA can't manage to keep systems that are completely secure, so what chance does anyone else have? Today, you are likely safe from invasive monitoring because no one has the processing capacity to handle tracking people as uninteresting as you, but we're still seeing processing power per dollar doubling roughly every 18 months, so how long will that be the case for?
Now try saying that and running for elected office.
If you're driving at one mile an hour, you're not likely to get in many accidents...
Prices in Europe have also gone up a lot, so while you might expect to see different absolute values, you'd expect to see the same decline. I live in the UK and don't drive. Owning a car is a huge expense (insurance and maintenance, even before you add in the fuel) for little benefit. I've always taken jobs where I could either work remotely or walk / cycle less than 10 minutes each way. I wouldn't consider working somewhere where I'd need to drive to work, or where I'd spend more than 10 minutes commuting, and I'm always amazed at people in the US who are happy to spend more than the equivalent of one working day a week just getting to and from work. At least cycling in, I get some exercise at the same time.
None of these devices have floppy disks though. Compact Flash cards are a more common requirement, as they're basically IDE devices. I had one a few years ago with a custom firewall distribution that fitted onto a 32MB CF card, but a year later it was hard to buy a CF card smaller than 4GB and so I switched to a full OS install.
FreeBSD systems traditionally built their own packages from the ports tree. The shipped packages are really just the ones that go on the install media, which are typically out of date by the time you get around to installing it. With the new pkg(7) infrastructure, we are properly supporting binary packages and part of the requirement for this was that we'd sign them and distribute the keys out of band. The signing keys are currently distributed using freebsd-update, which was designed to do binary updates of the base system and does use signed deltas. These are then used by the pkg tool to verify packages.
Prior to this, we were distributing the ports tree using the portsnap utility, which also does cryptographic verification of the downloads using the same mechanism as freebsd-update.
What you do is, check out the source repository, which does make sure the data you get hasn't been tampered with, then build it from source.
Actually, it doesn't. OpenBSD is still using CVS for revision control. After the FreeBSD cluster compromise a couple of years ago, we found that the CVS repository was the one thing whose integrity we could not verify. The current FreeBSD CVS repository was created by exporting from subversion (which could be verified) and validating it against git (which also can be verified).
Oh, and OpenBSD does recommend getting the binary packages over using ports (but they don't release security fixes for binary packages, and don't support using a ports tree revision that doesn't match your base system revision), and they recommend getting binary packages with unencrypted, unauthenticated, FTP.
The Sale of Goods Act in the UK requires that goods are suitable for the purpose for which sold and must function in this manner, in normal use, for a period which is expected for the class of product. This electronic devices, unless explicitly stated before the time of sale (i.e. placing text clearly in the advert saying 'this device has a shorter useable life than others of its class'), this is six years. For other devices, such as kitchen appliances, it can be 10-20 years, during which the manufacturer is obliged to provide service for all faults that are not directly caused by the owner. The EU warranty law actually gives you weaker protection than the UK law, but feel free to keep boycotting UK companies because of that if it makes you feel better...
I've seen the opposite. In the UK, if you quote the Sale of Goods Act when you call them up, they'll quickly replace faulty parts 4 years after the original purchase. In the US, as soon as the warranty expires, you're fucked if anything goes wrong.
If Darwin's theory had come from a computer model and it turned out his computer model didn't match real world observations, nobody would believe it.
People argued that Darwin's theory was wrong because people's grandfathers obviously (or, in some cases, less obviously) weren't monkeys. It was a nonsense argument, because they weren't actually comparing Darwin's theory with reality, they were comparing the media soundbite version of his theory with reality. AGW also fails to stand up to this kind of attack, but that doesn't mean that the theories are wrong.
Add to that, unless you're in the running for a Nobel, your academic career isn't likely to be tied to a specific theory. No one is discredited for saying 'we did this work, and based on the evidence that we had we advanced this theory, but now we have more data we've disproven it and now advance this theory'. In fact, that's what a lot of papers do say. It's much easier to get that kind of paper published than one that says 'we thought this, and now we have more evidence, and we think it's still true.'