Sounds like you're logged into gmail when you go to youtube.
Logout [of gmail] first [possibly clearing some cookies] and you'll have no problem. I have a gmail account [but I only access it through POP3/IMAP from thunderbird--thus, it's never logged in] and I don't have the same problem. I did have the same problem one time when I was logged into gmail.
If you'd rather not logout/login on gmail repeatedly, you can create a separate browser profile [Firefox, at least] for youtube, etc.
The article mentions that one can change the bandgap of a material with the laser. Isn't this what has been holding back graphene semiconductors--that they have a zero bandgap? Could this technique be used to produce practical graphene semiconductors?
Yes. Just artificially dropping some packets (either deliberately or just to implement some notion of quality-of-service) can be problematic. While an established TCP socket can deal with this, doing a DNS lookup [which is datagram based] can be severely affected. My ISP implements QoS and most of the delay I experience is a failed DNS query that must timeout and be retried (e.g. I'll wait a minute to get a page load but 55 seconds of that is waiting for the DNS request to succeed).
It's a way to censor things without doing outright censorship (e.g. blocking Google 100%). It's my belief that when Google was negotiating with the Chinese government about access, they argued strenuously, but in the end, they took the best deal they could get [were offered]. I mean, if Google had taken a stronger stance (e.g. "we won't limit access"), what would the government's response have been (e.g. 100% blockage) and what would the Chinese people's response have been?
Rest assured that your government's artificial push for Baidu was known here in the US, not that we've been able to do much to help. Our government people do talk with Chinese officials about such things [and many more], but your government is usually quite stoic in its responses;-).
Economic freedom and political freedom are two sides of the same coin. You can't really have one without the other. Note that I'm not talking about capitalism per se. Many European countries have a modified form of socialism, but still have a government that fosters political freedom.
Hopefully, Google's initiative will provide some improvement/relief. Time will tell.
Seems to me the limiting factor will be ISP datacaps.
The ISPs that tend to have them are the ones that also want to send content (e.g. U-Verse, Comcast, to name a few). Datacaps limit peer-to-peer networks.
A more sinister interpretation is that datacaps limit the amount of traffic that the NSA has to sift through. The ISPs that seem to have the greatest track record of caving to NSLs, etc. are also the ones with datacaps. Coincidence?
Thus, datacaps also apply when one's "friend" routes traffic through one's connection to support a distributed VPN scheme.
One of the comments in the original article ["Julian"] claims to have the router and have verified it.
It's not hard to verify it. Create a perl/python/whatever program [on a PC] that mimics the "User-Agent:" string and tries to do something that would be password challenged otherwise. If it succeeds, the access method exists [and is exploitable--from the local LAN if nowhere else].
What isn't clear is whether the User-Agent: hack can be used from the router's public IP address vs. just 192.168.x.y [local LAN] or even if 192.168.x.y can be used.
Someone else posted that the hack is used by firmware programs running inside the router to change configuration [which is legitimate for them to do] by sending the browser inside the router the request. So, it seems the intent of this was less of a backdoor (e.g. where D-Link et. al. could remotely take control of the router) and more of a way for the router to do its job.
The fact that the User-Agent: string is a funky, password-like string vs "internal_legitimate_request" indicates that the code author knew it could be exposed to the public/LAN IP addresses and tried to [weakly] mitigate this. The weakness is akin to disassembling code for a shared secret key.
A better way might be to add an additional restriction that such internal requests must also be from a known internal source (e.g. an AF_UNIX [vs. AF_INET] socket that presumeably could not be faked from an external source). But, this would take more time-to-code, sophistication, code space. Perhaps deemed not worth it for a $100 router.
Lament the hobbling of innovation by an incompetent patent office.
Agreed.
But, in fairness to them, it isn't just competence, but also they're [very] understaffed. They have a [huge] backlog which they've been tasked [by Congress?] with reducing. Unfortunately, if they deny a patent, the inventor may refile. Deny again and refile again,... The only way for them to truly clear the backlog is to approve the patent. This kicks frivolous patents into the court system--which may have less expertise but more resources [jury pools, appellate courts, etc.]. Not the right thing to do, but possibly understandable.
I have some hope for the "America Invents Act". This makes it easier for interested third parties to dig up prior art and submit it to the USPTO and trigger a reexamination. Crowdsourcing by experts in a given field may help stem the tide. But, further legislation is probably required (e.g. outlawing software patents). And, I say this as a programmer.
With some poetic justice/irony, Apple has lost its iOS "rubber banding" patent in the EU due to a recent German court decision [was reported on Slashdot, I believe]. This is because Steve Jobs demonstrated the rubber banding feature at at a conference and said "Boy, have we got it patented". The actual patent application was dated a few months later. Unfortunately, while this is okay in the US, in the EU such a [public] disclosure before the patent is filed nullifies the patent [Steve's talk is considered prior art].
IIRC, the Apple magnetic connectors are under patent.
Ironically, someone at my local Starbuck's had a charger that had hinged power prongs. One of them broke off.
I hope the EU is allowing for the new USB 3.0 "high power" [100 watts vs 5 watts] spec because we'll need that to charge the higher capacity batteries that are about 3 years away. Some of these have 10x the capacity of present batteries and who wants to wait 80 hours to charge them?
Myst-like games still live on with some changes [I've played Myst/Riven and all of the following]: - 7th Guest/11th Hour (historical) -- lots of puzzles - Lara Croft series -- shoot em up, but that wasn't the whole game - Nancy Drew series (20+ games) -- solve a murder mystery - Art of Murder series -- be a female FBI agent and stop a serial killer - Yesterday -- play various characters (including one with no memory) with a progressive mystery storyline
All of these have tough problems/puzzles but do this in a more conventional setting. It's tough enough to solve the puzzles without having to do this in an unfamiliar world.
Myst had mythos [of Atrus, etc.], but it was a bit sterile. There wasn't as much of a storyline [where characters can actually converse]. The newer games are more like interactive fiction. Each of the characters has a distinct personality and you actually start to care about them, just like you would in any good story.
As to game mechanics, some of the modern games have a button to highlight the sensitized areas in a scene. Otherwise, it's genuine work to poke every pixel looking for something. Newer games also have a "give me a clue" button. This helps if you're engrossed in the storyline and reach a point where you admit you're stuck but still want to proceed through the game--for the story.
Myst had no such "I admit I'm flummoxed" help. After a while, it just became work to drudge around the same area looking for the way out. Of course, one could always consult a walkthrough, but that highlights the need in the game itself for something to make it enjoyable for all.
Also, Myst may be having problems because of Ubisoft's stance on DRM. I'll never buy another Ubisoft game, ever, because of it.
If anyone was thinking of breaking up with the NSA family, the letter states, “We want to put the information you are reading and hearing about in the press into context and reassure you that this Agency and its workforce are deserving and appreciative of your support.”
Family == Mafia [*]
[*] or used to be until the National Stasi Agency sullied the term...
Yes. From the blockquote: "constitute a powerful endorsement of the carful work". Guess somebody wasn't carful "E"-nough:-)
The self congratulation aside, they had different goals at that time. They were looking for a unified/legal definition of time that could be universal.
I don't think even many programmers realized the implications at the time either. I mean, Intel was rolling out 5Mhz chips then. The focus was rolling out PC's as a credible alternative to the mainframe world. As the CPU's sped up, the problems became more apparent.
Nice website--thanks.
I found another page on it that has an interesting idea: Set up an ntp stratum 1 server driven by a GPS clock. Don't transmit leap seconds. Adjust the time offset a bit [to compensate for the initial syncup of GPS time with UTC], modify NTP client code, then handle all leap seconds via the zoneinfo data files.
During the one second interval when they are applied, what happens to [UTC] timestamps on files that are modified?
Whoosh.
Whoosh yourself...
You're not describing a problem, you are the problem. If an event occurs during a leap second, you simply timestamp the time, 23:59:60.x.
Yes, that's what the spec talks about. But, it isn't how times are represented internally in systems. They use binary [64 bit] numbers that are [usually] offsets from Jan 1, 1970 [the Unix epoch]--except for MS, of course.
It's what happens to a [normally] monotonic sequence: n,n+1,n+2 when you present n,n,n+1 or n,n+2,n+3 instead at the binary level and how you handle the discontinuities [if you're even in a position to know the discontinuity has occurred]. Most software just sees any and all of the above as k,k+1,k+2 so it has no way to compensate. In most cases, it can't because the leap second insertion is done in such a way (at the NTP server) that it is invisible even to the OS, let along a userspace program. The user program would have to know the internal OS policy [which it usually doesn't].
Some programs don't care what n/k is [it could be 31579 or 0] because they're more interested in a time delta across a [very] short interval. Leap seconds wreak havoc with that. That's why Google came up with its "leap smear" solution.
Really, your description of how you timestamp a file shows complete ignorance of how OS's, filesystems, processes actually handle these problems.
And that you're not a programmer. I am. In the kernel/driver/realtime arena for forty years. I deal with keeping systems synchronized at the nanosecond level all the time. So, please stop preaching to the choir.
What you describe simply demonstrates the problem with incorrect assumptions and sloppy coding: that a minute can't have more than 60 seconds. That hasn't been the case for over 40 years.
The assumptions weren't incorrect. They were just ones you weren't aware of [see above]. Your [repeated] use of "sloppy coding" sounds like armchair quarterbacking. Have you analyzed the cost/benefit across a wide range of applications and the risk associated with trying to apply a fix that may have little to no benefit? For example, which programs and what remedy should be applied?
based on POSIX-compliant UTC
'taint no such thing. There's UTC, and there's POSIX, which is in no way compatible with UTC, which existed long before POSIX.
If I had said "POSIX-compliant [use of] UTC" instead, you'd have nothing to argue about.
And leap seconds were not part of the original UTC spec. The folks that added the leap seconds [however well intentioned] didn't anticipate the complications to existing systems already using UTC (e.g. Unix which predated leap seconds by a few years). Now, many are questioning their [marginal] utility versus the complexity they [still] engender [particularly for realtime systems, such as video broadcast], which is why the discussions are going on about removing them from UTC.
Just another example of brain-dead, incorrect assumptions about timescales.
I've also worked on commercial grade realtime video broadcast systems, where I've had to coordinate 5-6 timescales at one time. So, my friend, what's your specific expertise that justifies the invectives?
Since July 2012, IIRC, all corrections are now formally set at six month intervals.
YRI. You should at least try to understand the subject you're discussing.
"A positive or negative leap-second should be the last second of a UTC month, but first preference should be given to the end of December and June, and second preference to the end of March and September. -
Leap seconds are problematic not because of sloppy coding but because they are a messy problem.
During the one second interval when they are applied, what happens to [UTC] timestamps on files that are modified? If you're compiling, an object file may be updated or not. Or, with rsync, a file may be transferred or not. Or, what about medical equipment [based on POSIX-compliant UTC (e.g. Linux, etc.)] that gives a patient an extra one second of radiation because the master clock is based on UTC?
Also, applications [which generally don't account for leap seconds] would get lurched, sometimes with disastrous results.
The solution we came up with came to be known as the 'leap smear'. We modified our internal Network Time Protocol [NTP] servers to gradually add a couple of milliseconds to every update, varying over a time window before the moment when the leap second actually happens. This meant that when it became time to add an extra second at midnight, our clocks had already taken this into account, by skewing the time over the course of the day. All of our servers were then able to continue as normal with the new year, blissfully unaware that a leap second had just occurred.
I confused nothing [re. leap days]. That was just a loose analogy. Nobody would notice if we applied leap seconds or not. Not even a leap minute every fifteen years would be noticed. And that's the maximum error you can have. Nobody will care if the sun sets in the west a minute earlier than you expect it to [based on TAI]. Better yet, since we tolerate daylight savings time, wait until the leap second error hits an hour. That will take 900 years minimum.
Since July 2012, IIRC, all corrections are now formally set at six month intervals. While most corrections have been in one direction, if we just kept track of the error, it might self correct over a long enough period.
Converting all systems (computers, cell phones, NTP, etc.) to use TAI internally would be a good thing. Convert to UTC when displaying the time [could be a user preference]. Everything becomes simpler and more accurate.
Those that truly need to track solar time are a minority (e.g. astronomers, etc.). Let them do the extra conversion rather than burdening the rest of the world [and the systems we use in our daily hitech/21st century lives].
Perhaps it's time to go the other way and use GPS/TAI for computer clocks instead of UTC. In other words, perhaps civil time should be changed. Just because it used to be based on something doesn't mean it needs to be in the future.
Leap second adjustments at the time that they occur are problematic for OS scheduling during that period. Also, you have to maintain a table of whether the twice a year leap second has been applied [and in which direction]. The table must be updated every six months because a table entry can't be predicted beforehand [because it's decided upon by IERS]. The application of leap seconds is actually quite irregular.
All this complexity for a [possible] adjustment of one second every six months?
Alternative schemes have been proposed that would accumulate leap second error until it truly becomes significant (e.g. do a leap minute adjustment every thirty years).
If the timebase that we use in the modern world (e.g. computers, cell phones, banking transactions, etc.) needs to track the sun that accurately, why do we tolerate error accumulation of up to a day every four years [leap days]?
A person with a security clearance revealing classified information [to another person] is treason. But, the receipt of such information is not [although a stretch, some cases have tried to charge the receiver under the Espionage Act, even though they are U.S. citizens].
Side note to U.K. citizens (the ones that [still] insist they don't need a formal constitution), receipt of classified information is treated exactly as disclosing it under "The Official Secrets Act" [IIRC].
However, "aid and comfort" might cover Meyer's assertion (e.g. failure to comply gives "aid and comfort" to an enemy) ala harboring a fugitive [electronically speaking].
In any case, it really is time to say: Enough is enough...
A long time ago I benchmarked perl's regex engine against about 5 others. At the time, it was 10x faster than the nearest competitor for the same regex/data.
Also, you can use perl's "study". Or, split the regexes across threads.
Also, with perl you can do some hierarchical saviings. For example: /Ffoo/... /Fbar/... /Fbaz/...
Could be redone as:
if (/F/) { ... if (/Ffoo/) ... if (/Fbar/ ... if (/Fbaz/)
}
The above is trivial example, but you get the idea.
Also, how much time is spent compiling (vs. executing) the regexes in egrep? I imagine a lot and you have to do this for each incoming message.
Note that spamassassin (and hence perl) can be set up as a daemon where the regexes are compiled once. The messages are passed through a socket to the daemon. This means that the only CPU time spent is on executing the regexes--a considerable savings.
Additionally, perl regexes have [considerably] more functionality/utility than egrep ones. You might be able to recode/consolidate yours and get the same [or better] bang for less buck.
The ABI has been changed upon occasion. If a struct passed to a syscall (or ioctl) has some spare option bits (that were reserved [and therefore zero], that can be the way to go (e.g. turning on the bit indicates that the program is aware of the new semantics).
Otherwise, a new syscall (or ioctl) number is assigned. For example, the stat syscall originally had a syscall number of 18. When the "struct stat" was modified [added some new fields and/or expanded the size of others], the syscall number was bumped to 106. Old programs that were not recompiled issued stat with 18 and worked unchanged. If you recompiled, you got syscall 106 and the new semantics.
vfork was definitely useful in its day [before COW]. I was curious as to whether vfork still had any advantages over fork in a modern system, so I did a little search. vfork can still be faster than fork [even with COW]. It's particularly useful on non-MMU systems (e.g. embedded).
But, it's considered a [small] security hazard due to race conditions with the child: privileged process does vfork, child lowers its privilege level [in prep for execve], but then child gets a signal. The signal can corrupt the parent and/or pin it (you have a non-privileged [child] process suspending a privileged [parent] one). vfork is used by some libc's to implement posix_spawn, but it's use is controversial unless some precautions are taken.
BSD's most lasting/widespread contribution may be BSD sockets. Prior to 1989 we had the [ghastly] OSI/ATT Streams/TLI. In 1989, BSD became unencumbered by licensing issues [from AT&T] and could release the socket code/specs. Thus, the internet in the [technical] form that we know today comes from BSD.
There were the unix wars awhile back, and I knew a few people at AT&T related companies who would frown whenever BSD was mentioned. The definitions of what is true Unix has changed over the years.
I was in two different mc68000 based startups. The first used Unix v7 and a C compiler that had int's at 16 bits. The second company used Sys III [and later Sys V] and a C compiler that had int's at 32 bits. The latter was much easier to program.
This second company had its CPU card and disk controller designed by a guy who had his own startup. Virtually identical hardware, but he used BSD. His company survived/thrived but the one I was in didn't. He attributed it to the selection of BSD: Many people had used BSD on a Vax and preferred that.
There was also a technical performance difference. Unix/v7/Sys3/Sys5 filesystems used a 512 byte block size. BSD used a 4KB block. I benchmarked this and the BSD system got 4x the performance [remember disk controller was the same]. I was able to compensate somewhat by putting full track read caching into the disk device driver and got a 3x bump. But, this was a "best effort" tweak against a better architectural choice [of BSD]
Basically since V6/V7 things have branched often but subsequent merging was less common; sometimes only the ideas merged rather than the actual code. There may be a common ancestor but the amount of that common ancestor that is still present is extremely small.
At that time, AT&T was extremely aggressive in enforcing the licensing, so not much code could be shared. With POSIX standards (and others like IETF RFC's), code sharing isn't as necessary/desirable as it once was. Compatible implementations are more important.
For example, Linux uses glibc. glibc has seamless multithread support using NPTL. FreeBSD's libc needs a special call to be issued at the outset to say "enable threading". After that, there are a bunch of:
if (threading_enabled) {...
}
else {...
} This was FreeBSD's architectural choice (e.g. they felt the overhead of the thread lock primitives warranted this approach). My personal experience has been that the thread lock overhead is so low for the uncontended case that glibc's way is better. OTOH, glibc's passion for complex linking (e.g. weak symbols, weak aliases, symbol hiding, etc.) makes me crazy.
I was doing Unix device driver work in startups since 1981 [v7, system III, system V] and we always referred to them with "Unix" attached (e.g. "Unix System III" or "Unix v7" or "v7 Unix"). Likewise, BSD was "BSD Unix"
POSIX was born out of a desire to produce standards that allowed Unix-compatible systems to be created that were not AT&T derived [i.e. eliminate the AT&T licensing stranglehold]. AT&T could copyright Unix source/binary distros. But, it couldn't copyright the API's [Note to Oracle;-)]
The non-Unix derived systems that truly aspired to look, act, feel like Unix [and be free] were: Minix, Linux, Mach. No doubt there are others that didn't quite make it to completion. Obviously, Linux took the crown [I loaded my first distro in 1993]. Though not an OS, the other piece of the puzzle is GNU/FSF providing "clean room" reimplementations of the standard POSIX (nee Unix) utilities.
I stopped using the term "Unix" a long time ago. Now, I usually say Linux, *BSD, Solaris, or "POSIX-compliant OS", etc.
Agreed. However, Wind River Systems used to do software consulting [VxWorks was a by product of that]. They were doing an automated storyboarding project for Francis Ford Coppola back in the 1980's.
This is theory, reality is you have > half a second of buffer in the cable modem, this manifests in broken congestion control and high latency when buffer is saturated.
Yes, I believe I already covered the effects of overly large buffers [within a network switch].
However, the cable modem [CM] buffer has far less to do with this. DOCSIS cable modems are a TDM [time division multiplexed] medium. A cable modem may not just transmit upstream data whenever it feels like it. Even with multiple upstream subchannels, a cable system "neighborhood" usually has more subscribers (~500) than it has subchannels.
Thus, the CM must make a request to the CMTS for an upstream channel and be given a grant. It may then transmit upstream [on that subchannel] for its grant period. The CM needs sufficient data to make the most use of the bandwidth available in its grant TDM slot. A host CPU's NIC just can't respond that fast--hence the pre-buffering. This is physical and/or link layer stuff. That's what the primary use of the CM's [large] buffer is for. To smooth out the inherently bursty nature of the "last mile" of a TDM medium. [*]
It has almost nothing to do with congestion control mechanisms in the upper protocol layers or queue management in a network switch/router.
Only end nodes can know the true flow as it depends on the weakest link and the particular routing at the time. A given intermediate node may route one packet, but the next one may be routed by another node. Thus, it has no visibility on the flow [other than its own] in a multihop path. For example, the weakest [slowest] link might be a 9600 baud modem on one end, on a noisy line, that is losing 50% of the data being sent/received. TCP accommodates this well, as I've described previously.
Network queue management can have some influence on the flow calculations done by end nodes. Before AQM, intermediate nodes would simply fill their [fixed size] buffers, and drop a packet if there was no space. Now that memory's cheap, buffer size isn't a problem, but latency is [particularly, if you're trying to implement QoS]. Also, if there's a "bad actor" (e.g. an end node where the congestion logic is broken due to a bug, or deliberate due to a DDoS attack), AQM can help [proactively] mitigate such behaviors.
Large buffers [queues] aren't inherently bad. They help in the situation where you get a sudden, brief burst of incoming data. If that's what it is, a short burst, the larger buffer can make things more robust. Just let the data drain normally--no AQM. However, if you get a lot of data continuously that repeatedly exceeds the average drain rate, then you've got a bad actor, and some active queue management is required.
Pick your mitigation algorithm: The MIT one [yet another instance of MIT's AI lab trying for relevance], the CoDel one, or just a good old "drop every other packet". YMMV...
[*] If you're truly P/O'ed by your [own] cable modem latency, try its config page. Or go fiber or DSL, which have no latency at the first hop and can send data at full line rate w/o delay.
Active queue management [dropping packets] is already being done by most intermediate routers already. It's been done somewhere along the line in getting this reply to you.
TCP is a "virtual circuit" [stream] protocol. In every TCP packet, there are upstream/downstream byte offsets (e.g. "I've sent you X bytes", "I've received from you Y bytes"). These are examined by the end-to-end nodes only, not any intermediate node or its associated buffers. At this level, packets [per se], dropped or not, don't matter--only byte offsets do.
If end node A is sending to end node B, and has transmitted X bytes, but has only received an ack from B for X-n, after a time, A will resend X-n to X again. This happens bidirectionally with B to A as well.
With overly large buffers, B will [eventually] get two [or more] copies of these retransmitted bytes. This just exacerbates the problem, particularly if there are a number of hops between A and B because most of the intermediate buffers become filled with needless retransmit copies [from disparate virtual circuits (e.g. end nodes C and D that are communicating) that have nothing to do with one another other than they must route through the same intermediate node on their way to their distinct destinations].
The TCP virtual circuit retransmit algorithm assumes that some packets will be lost [due to transmission CRC errors] and/or dropped [or duplicated]. The TCP end-to-end retransmit algorithm [and backoff] is orthogonal to the AQM strategies the articles are talking about. But, large intermediate buffers actually confuse/defeat the TCP backoff strategy and make the problem worse--not better.
When the AQM algorithms work correctly, most packets aren't dropped, because each transmitting end node [eventually] syncs itself to the maximum rate it can send through the [entire] circuit without having data dropped/replicated/delayed unnecessarily.
People don't care about "dropped packets". They care about throughput and low latency. If a packet were never dropped, you'd have neither.
Slashdot Mirror
Researchers at the University of Maryland have been using the tobacco mosaic virus for similar purposes: http://phys.org/news/2010-12-virally-nano-electrodes-boost-energy-capacity.html
Sounds like you're logged into gmail when you go to youtube.
Logout [of gmail] first [possibly clearing some cookies] and you'll have no problem. I have a gmail account [but I only access it through POP3/IMAP from thunderbird--thus, it's never logged in] and I don't have the same problem. I did have the same problem one time when I was logged into gmail.
If you'd rather not logout/login on gmail repeatedly, you can create a separate browser profile [Firefox, at least] for youtube, etc.
See Jane Mayer's New Yorker piece http://www.newyorker.com/reporting/2010/08/30/100830fa_fact_mayer to get a truer sense of the depth and breadth of the machinations.
The article mentions that one can change the bandgap of a material with the laser. Isn't this what has been holding back graphene semiconductors--that they have a zero bandgap? Could this technique be used to produce practical graphene semiconductors?
Yes. Just artificially dropping some packets (either deliberately or just to implement some notion of quality-of-service) can be problematic. While an established TCP socket can deal with this, doing a DNS lookup [which is datagram based] can be severely affected. My ISP implements QoS and most of the delay I experience is a failed DNS query that must timeout and be retried (e.g. I'll wait a minute to get a page load but 55 seconds of that is waiting for the DNS request to succeed).
It's a way to censor things without doing outright censorship (e.g. blocking Google 100%). It's my belief that when Google was negotiating with the Chinese government about access, they argued strenuously, but in the end, they took the best deal they could get [were offered]. I mean, if Google had taken a stronger stance (e.g. "we won't limit access"), what would the government's response have been (e.g. 100% blockage) and what would the Chinese people's response have been?
Rest assured that your government's artificial push for Baidu was known here in the US, not that we've been able to do much to help. Our government people do talk with Chinese officials about such things [and many more], but your government is usually quite stoic in its responses ;-).
Economic freedom and political freedom are two sides of the same coin. You can't really have one without the other. Note that I'm not talking about capitalism per se. Many European countries have a modified form of socialism, but still have a government that fosters political freedom.
Hopefully, Google's initiative will provide some improvement/relief. Time will tell.
Seems to me the limiting factor will be ISP datacaps.
The ISPs that tend to have them are the ones that also want to send content (e.g. U-Verse, Comcast, to name a few). Datacaps limit peer-to-peer networks.
A more sinister interpretation is that datacaps limit the amount of traffic that the NSA has to sift through. The ISPs that seem to have the greatest track record of caving to NSLs, etc. are also the ones with datacaps. Coincidence?
Thus, datacaps also apply when one's "friend" routes traffic through one's connection to support a distributed VPN scheme.
One of the comments in the original article ["Julian"] claims to have the router and have verified it.
It's not hard to verify it. Create a perl/python/whatever program [on a PC] that mimics the "User-Agent:" string and tries to do something that would be password challenged otherwise. If it succeeds, the access method exists [and is exploitable--from the local LAN if nowhere else].
What isn't clear is whether the User-Agent: hack can be used from the router's public IP address vs. just 192.168.x.y [local LAN] or even if 192.168.x.y can be used.
Someone else posted that the hack is used by firmware programs running inside the router to change configuration [which is legitimate for them to do] by sending the browser inside the router the request. So, it seems the intent of this was less of a backdoor (e.g. where D-Link et. al. could remotely take control of the router) and more of a way for the router to do its job.
The fact that the User-Agent: string is a funky, password-like string vs "internal_legitimate_request" indicates that the code author knew it could be exposed to the public/LAN IP addresses and tried to [weakly] mitigate this. The weakness is akin to disassembling code for a shared secret key.
A better way might be to add an additional restriction that such internal requests must also be from a known internal source (e.g. an AF_UNIX [vs. AF_INET] socket that presumeably could not be faked from an external source). But, this would take more time-to-code, sophistication, code space. Perhaps deemed not worth it for a $100 router.
Lament the hobbling of innovation by an incompetent patent office.
Agreed.
But, in fairness to them, it isn't just competence, but also they're [very] understaffed. They have a [huge] backlog which they've been tasked [by Congress?] with reducing. Unfortunately, if they deny a patent, the inventor may refile. Deny again and refile again, ... The only way for them to truly clear the backlog is to approve the patent. This kicks frivolous patents into the court system--which may have less expertise but more resources [jury pools, appellate courts, etc.]. Not the right thing to do, but possibly understandable.
I have some hope for the "America Invents Act". This makes it easier for interested third parties to dig up prior art and submit it to the USPTO and trigger a reexamination. Crowdsourcing by experts in a given field may help stem the tide. But, further legislation is probably required (e.g. outlawing software patents). And, I say this as a programmer.
With some poetic justice/irony, Apple has lost its iOS "rubber banding" patent in the EU due to a recent German court decision [was reported on Slashdot, I believe]. This is because Steve Jobs demonstrated the rubber banding feature at at a conference and said "Boy, have we got it patented". The actual patent application was dated a few months later. Unfortunately, while this is okay in the US, in the EU such a [public] disclosure before the patent is filed nullifies the patent [Steve's talk is considered prior art].
IIRC, the Apple magnetic connectors are under patent.
Ironically, someone at my local Starbuck's had a charger that had hinged power prongs. One of them broke off.
I hope the EU is allowing for the new USB 3.0 "high power" [100 watts vs 5 watts] spec because we'll need that to charge the higher capacity batteries that are about 3 years away. Some of these have 10x the capacity of present batteries and who wants to wait 80 hours to charge them?
Offtopic, but I like your sig ...
Agreed about the 3D shoot-em-ups.
Myst-like games still live on with some changes [I've played Myst/Riven and all of the following]:
- 7th Guest/11th Hour (historical) -- lots of puzzles
- Lara Croft series -- shoot em up, but that wasn't the whole game
- Nancy Drew series (20+ games) -- solve a murder mystery
- Art of Murder series -- be a female FBI agent and stop a serial killer
- Yesterday -- play various characters (including one with no memory) with a progressive mystery storyline
All of these have tough problems/puzzles but do this in a more conventional setting. It's tough enough to solve the puzzles without having to do this in an unfamiliar world.
Myst had mythos [of Atrus, etc.], but it was a bit sterile. There wasn't as much of a storyline [where characters can actually converse]. The newer games are more like interactive fiction. Each of the characters has a distinct personality and you actually start to care about them, just like you would in any good story.
As to game mechanics, some of the modern games have a button to highlight the sensitized areas in a scene. Otherwise, it's genuine work to poke every pixel looking for something. Newer games also have a "give me a clue" button. This helps if you're engrossed in the storyline and reach a point where you admit you're stuck but still want to proceed through the game--for the story.
Myst had no such "I admit I'm flummoxed" help. After a while, it just became work to drudge around the same area looking for the way out. Of course, one could always consult a walkthrough, but that highlights the need in the game itself for something to make it enjoyable for all.
Also, Myst may be having problems because of Ubisoft's stance on DRM. I'll never buy another Ubisoft game, ever, because of it.
If anyone was thinking of breaking up with the NSA family, the letter states, “We want to put the information you are reading and hearing about in the press into context and reassure you that this Agency and its workforce are deserving and appreciative of your support.”
Family == Mafia [*]
[*] or used to be until the National Stasi Agency sullied the term ...
Yes. From the blockquote: "constitute a powerful endorsement of the carful work". Guess somebody wasn't carful "E"-nough :-)
The self congratulation aside, they had different goals at that time. They were looking for a unified/legal definition of time that could be universal.
I don't think even many programmers realized the implications at the time either. I mean, Intel was rolling out 5Mhz chips then. The focus was rolling out PC's as a credible alternative to the mainframe world. As the CPU's sped up, the problems became more apparent.
Nice website--thanks.
I found another page on it that has an interesting idea: Set up an ntp stratum 1 server driven by a GPS clock. Don't transmit leap seconds. Adjust the time offset a bit [to compensate for the initial syncup of GPS time with UTC], modify NTP client code, then handle all leap seconds via the zoneinfo data files.
During the one second interval when they are applied, what happens to [UTC] timestamps on files that are modified?
Whoosh.
Whoosh yourself ...
You're not describing a problem, you are the problem. If an event occurs during a leap second, you simply timestamp the time, 23:59:60.x.
Yes, that's what the spec talks about. But, it isn't how times are represented internally in systems. They use binary [64 bit] numbers that are [usually] offsets from Jan 1, 1970 [the Unix epoch]--except for MS, of course.
It's what happens to a [normally] monotonic sequence: n,n+1,n+2 when you present n,n,n+1 or n,n+2,n+3 instead at the binary level and how you handle the discontinuities [if you're even in a position to know the discontinuity has occurred]. Most software just sees any and all of the above as k,k+1,k+2 so it has no way to compensate. In most cases, it can't because the leap second insertion is done in such a way (at the NTP server) that it is invisible even to the OS, let along a userspace program. The user program would have to know the internal OS policy [which it usually doesn't].
Some programs don't care what n/k is [it could be 31579 or 0] because they're more interested in a time delta across a [very] short interval. Leap seconds wreak havoc with that. That's why Google came up with its "leap smear" solution.
Really, your description of how you timestamp a file shows complete ignorance of how OS's, filesystems, processes actually handle these problems.
And that you're not a programmer. I am. In the kernel/driver/realtime arena for forty years. I deal with keeping systems synchronized at the nanosecond level all the time. So, please stop preaching to the choir.
What you describe simply demonstrates the problem with incorrect assumptions and sloppy coding: that a minute can't have more than 60 seconds. That hasn't been the case for over 40 years.
The assumptions weren't incorrect. They were just ones you weren't aware of [see above]. Your [repeated] use of "sloppy coding" sounds like armchair quarterbacking. Have you analyzed the cost/benefit across a wide range of applications and the risk associated with trying to apply a fix that may have little to no benefit? For example, which programs and what remedy should be applied?
based on POSIX-compliant UTC
'taint no such thing. There's UTC, and there's POSIX, which is in no way compatible with UTC, which existed long before POSIX.
If I had said "POSIX-compliant [use of] UTC" instead, you'd have nothing to argue about.
And leap seconds were not part of the original UTC spec. The folks that added the leap seconds [however well intentioned] didn't anticipate the complications to existing systems already using UTC (e.g. Unix which predated leap seconds by a few years). Now, many are questioning their [marginal] utility versus the complexity they [still] engender [particularly for realtime systems, such as video broadcast], which is why the discussions are going on about removing them from UTC.
Just another example of brain-dead, incorrect assumptions about timescales.
I've also worked on commercial grade realtime video broadcast systems, where I've had to coordinate 5-6 timescales at one time. So, my friend, what's your specific expertise that justifies the invectives?
Since July 2012, IIRC, all corrections are now formally set at six month intervals.
YRI. You should at least try to understand the subject you're discussing.
"A positive or negative leap-second should be the last second of a UTC month, but first
preference should be given to the end of December and June, and second preference to the end of
March and September.
-
Leap seconds are problematic not because of sloppy coding but because they are a messy problem.
During the one second interval when they are applied, what happens to [UTC] timestamps on files that are modified? If you're compiling, an object file may be updated or not. Or, with rsync, a file may be transferred or not. Or, what about medical equipment [based on POSIX-compliant UTC (e.g. Linux, etc.)] that gives a patient an extra one second of radiation because the master clock is based on UTC?
Also, applications [which generally don't account for leap seconds] would get lurched, sometimes with disastrous results.
That's why Google came up with a "leap smear": http://www.theregister.co.uk/2011/09/19/google_has_to_lie_to_computers_about_time/
The solution we came up with came to be known as the 'leap smear'. We modified our internal Network Time Protocol [NTP] servers to gradually add a couple of milliseconds to every update, varying over a time window before the moment when the leap second actually happens. This meant that when it became time to add an extra second at midnight, our clocks had already taken this into account, by skewing the time over the course of the day. All of our servers were then able to continue as normal with the new year, blissfully unaware that a leap second had just occurred.
I confused nothing [re. leap days]. That was just a loose analogy. Nobody would notice if we applied leap seconds or not. Not even a leap minute every fifteen years would be noticed. And that's the maximum error you can have. Nobody will care if the sun sets in the west a minute earlier than you expect it to [based on TAI]. Better yet, since we tolerate daylight savings time, wait until the leap second error hits an hour. That will take 900 years minimum.
Since July 2012, IIRC, all corrections are now formally set at six month intervals. While most corrections have been in one direction, if we just kept track of the error, it might self correct over a long enough period.
Converting all systems (computers, cell phones, NTP, etc.) to use TAI internally would be a good thing. Convert to UTC when displaying the time [could be a user preference]. Everything becomes simpler and more accurate.
Those that truly need to track solar time are a minority (e.g. astronomers, etc.). Let them do the extra conversion rather than burdening the rest of the world [and the systems we use in our daily hitech/21st century lives].
Perhaps it's time to go the other way and use GPS/TAI for computer clocks instead of UTC. In other words, perhaps civil time should be changed. Just because it used to be based on something doesn't mean it needs to be in the future.
Leap second adjustments at the time that they occur are problematic for OS scheduling during that period. Also, you have to maintain a table of whether the twice a year leap second has been applied [and in which direction]. The table must be updated every six months because a table entry can't be predicted beforehand [because it's decided upon by IERS]. The application of leap seconds is actually quite irregular.
All this complexity for a [possible] adjustment of one second every six months?
Alternative schemes have been proposed that would accumulate leap second error until it truly becomes significant (e.g. do a leap minute adjustment every thirty years).
If the timebase that we use in the modern world (e.g. computers, cell phones, banking transactions, etc.) needs to track the sun that accurately, why do we tolerate error accumulation of up to a day every four years [leap days]?
More likely obstruction of justice or some such.
A person with a security clearance revealing classified information [to another person] is treason. But, the receipt of such information is not [although a stretch, some cases have tried to charge the receiver under the Espionage Act, even though they are U.S. citizens].
Side note to U.K. citizens (the ones that [still] insist they don't need a formal constitution), receipt of classified information is treated exactly as disclosing it under "The Official Secrets Act" [IIRC].
However, "aid and comfort" might cover Meyer's assertion (e.g. failure to comply gives "aid and comfort" to an enemy) ala harboring a fugitive [electronically speaking].
In any case, it really is time to say: Enough is enough ...
A long time ago I benchmarked perl's regex engine against about 5 others. At the time, it was 10x faster than the nearest competitor for the same regex/data.
Also, you can use perl's "study". Or, split the regexes across threads.
Also, with perl you can do some hierarchical saviings. For example:
/Ffoo/ ...
/Fbar/ ...
/Fbaz/ ...
Could be redone as:
... if (/Ffoo/)
... if (/Fbar/
... if (/Fbaz/)
if (/F/) {
}
The above is trivial example, but you get the idea.
Also, how much time is spent compiling (vs. executing) the regexes in egrep? I imagine a lot and you have to do this for each incoming message.
Note that spamassassin (and hence perl) can be set up as a daemon where the regexes are compiled once. The messages are passed through a socket to the daemon. This means that the only CPU time spent is on executing the regexes--a considerable savings.
Additionally, perl regexes have [considerably] more functionality/utility than egrep ones. You might be able to recode/consolidate yours and get the same [or better] bang for less buck.
The ABI has been changed upon occasion. If a struct passed to a syscall (or ioctl) has some spare option bits (that were reserved [and therefore zero], that can be the way to go (e.g. turning on the bit indicates that the program is aware of the new semantics).
Otherwise, a new syscall (or ioctl) number is assigned. For example, the stat syscall originally had a syscall number of 18. When the "struct stat" was modified [added some new fields and/or expanded the size of others], the syscall number was bumped to 106. Old programs that were not recompiled issued stat with 18 and worked unchanged. If you recompiled, you got syscall 106 and the new semantics.
vfork was definitely useful in its day [before COW]. I was curious as to whether vfork still had any advantages over fork in a modern system, so I did a little search. vfork can still be faster than fork [even with COW]. It's particularly useful on non-MMU systems (e.g. embedded).
But, it's considered a [small] security hazard due to race conditions with the child: privileged process does vfork, child lowers its privilege level [in prep for execve], but then child gets a signal. The signal can corrupt the parent and/or pin it (you have a non-privileged [child] process suspending a privileged [parent] one). vfork is used by some libc's to implement posix_spawn, but it's use is controversial unless some precautions are taken.
BSD's most lasting/widespread contribution may be BSD sockets. Prior to 1989 we had the [ghastly] OSI/ATT Streams/TLI. In 1989, BSD became unencumbered by licensing issues [from AT&T] and could release the socket code/specs. Thus, the internet in the [technical] form that we know today comes from BSD.
There were the unix wars awhile back, and I knew a few people at AT&T related companies who would frown whenever BSD was mentioned. The definitions of what is true Unix has changed over the years.
I was in two different mc68000 based startups. The first used Unix v7 and a C compiler that had int's at 16 bits. The second company used Sys III [and later Sys V] and a C compiler that had int's at 32 bits. The latter was much easier to program.
This second company had its CPU card and disk controller designed by a guy who had his own startup. Virtually identical hardware, but he used BSD. His company survived/thrived but the one I was in didn't. He attributed it to the selection of BSD: Many people had used BSD on a Vax and preferred that.
There was also a technical performance difference. Unix/v7/Sys3/Sys5 filesystems used a 512 byte block size. BSD used a 4KB block. I benchmarked this and the BSD system got 4x the performance [remember disk controller was the same]. I was able to compensate somewhat by putting full track read caching into the disk device driver and got a 3x bump. But, this was a "best effort" tweak against a better architectural choice [of BSD]
Basically since V6/V7 things have branched often but subsequent merging was less common; sometimes only the ideas merged rather than the actual code. There may be a common ancestor but the amount of that common ancestor that is still present is extremely small.
At that time, AT&T was extremely aggressive in enforcing the licensing, so not much code could be shared. With POSIX standards (and others like IETF RFC's), code sharing isn't as necessary/desirable as it once was. Compatible implementations are more important.
For example, Linux uses glibc. glibc has seamless multithread support using NPTL. FreeBSD's libc needs a special call to be issued at the outset to say "enable threading". After that, there are a bunch of: ... ...
if (threading_enabled) {
}
else {
}
This was FreeBSD's architectural choice (e.g. they felt the overhead of the thread lock primitives warranted this approach). My personal experience has been that the thread lock overhead is so low for the uncontended case that glibc's way is better. OTOH, glibc's passion for complex linking (e.g. weak symbols, weak aliases, symbol hiding, etc.) makes me crazy.
YMMV
BSD is AT&T [pre v7] Unix derived. http://en.wikipedia.org/wiki/Berkeley_Software_Distribution
I was doing Unix device driver work in startups since 1981 [v7, system III, system V] and we always referred to them with "Unix" attached (e.g. "Unix System III" or "Unix v7" or "v7 Unix"). Likewise, BSD was "BSD Unix"
POSIX was born out of a desire to produce standards that allowed Unix-compatible systems to be created that were not AT&T derived [i.e. eliminate the AT&T licensing stranglehold]. AT&T could copyright Unix source/binary distros. But, it couldn't copyright the API's [Note to Oracle ;-)]
The non-Unix derived systems that truly aspired to look, act, feel like Unix [and be free] were: Minix, Linux, Mach. No doubt there are others that didn't quite make it to completion. Obviously, Linux took the crown [I loaded my first distro in 1993]. Though not an OS, the other piece of the puzzle is GNU/FSF providing "clean room" reimplementations of the standard POSIX (nee Unix) utilities.
I stopped using the term "Unix" a long time ago. Now, I usually say Linux, *BSD, Solaris, or "POSIX-compliant OS", etc.
Agreed. However, Wind River Systems used to do software consulting [VxWorks was a by product of that]. They were doing an automated storyboarding project for Francis Ford Coppola back in the 1980's.
This is theory, reality is you have > half a second of buffer in the cable modem, this manifests in broken congestion control and high latency when buffer is saturated.
Yes, I believe I already covered the effects of overly large buffers [within a network switch].
However, the cable modem [CM] buffer has far less to do with this. DOCSIS cable modems are a TDM [time division multiplexed] medium. A cable modem may not just transmit upstream data whenever it feels like it. Even with multiple upstream subchannels, a cable system "neighborhood" usually has more subscribers (~500) than it has subchannels.
Thus, the CM must make a request to the CMTS for an upstream channel and be given a grant. It may then transmit upstream [on that subchannel] for its grant period. The CM needs sufficient data to make the most use of the bandwidth available in its grant TDM slot. A host CPU's NIC just can't respond that fast--hence the pre-buffering. This is physical and/or link layer stuff. That's what the primary use of the CM's [large] buffer is for. To smooth out the inherently bursty nature of the "last mile" of a TDM medium. [*]
It has almost nothing to do with congestion control mechanisms in the upper protocol layers or queue management in a network switch/router.
Only end nodes can know the true flow as it depends on the weakest link and the particular routing at the time. A given intermediate node may route one packet, but the next one may be routed by another node. Thus, it has no visibility on the flow [other than its own] in a multihop path. For example, the weakest [slowest] link might be a 9600 baud modem on one end, on a noisy line, that is losing 50% of the data being sent/received. TCP accommodates this well, as I've described previously.
Network queue management can have some influence on the flow calculations done by end nodes. Before AQM, intermediate nodes would simply fill their [fixed size] buffers, and drop a packet if there was no space. Now that memory's cheap, buffer size isn't a problem, but latency is [particularly, if you're trying to implement QoS]. Also, if there's a "bad actor" (e.g. an end node where the congestion logic is broken due to a bug, or deliberate due to a DDoS attack), AQM can help [proactively] mitigate such behaviors.
Large buffers [queues] aren't inherently bad. They help in the situation where you get a sudden, brief burst of incoming data. If that's what it is, a short burst, the larger buffer can make things more robust. Just let the data drain normally--no AQM. However, if you get a lot of data continuously that repeatedly exceeds the average drain rate, then you've got a bad actor, and some active queue management is required.
Pick your mitigation algorithm: The MIT one [yet another instance of MIT's AI lab trying for relevance], the CoDel one, or just a good old "drop every other packet". YMMV ...
[*] If you're truly P/O'ed by your [own] cable modem latency, try its config page. Or go fiber or DSL, which have no latency at the first hop and can send data at full line rate w/o delay.
Active queue management [dropping packets] is already being done by most intermediate routers already. It's been done somewhere along the line in getting this reply to you.
TCP is a "virtual circuit" [stream] protocol. In every TCP packet, there are upstream/downstream byte offsets (e.g. "I've sent you X bytes", "I've received from you Y bytes"). These are examined by the end-to-end nodes only, not any intermediate node or its associated buffers. At this level, packets [per se], dropped or not, don't matter--only byte offsets do.
If end node A is sending to end node B, and has transmitted X bytes, but has only received an ack from B for X-n, after a time, A will resend X-n to X again. This happens bidirectionally with B to A as well.
With overly large buffers, B will [eventually] get two [or more] copies of these retransmitted bytes. This just exacerbates the problem, particularly if there are a number of hops between A and B because most of the intermediate buffers become filled with needless retransmit copies [from disparate virtual circuits (e.g. end nodes C and D that are communicating) that have nothing to do with one another other than they must route through the same intermediate node on their way to their distinct destinations].
The TCP virtual circuit retransmit algorithm assumes that some packets will be lost [due to transmission CRC errors] and/or dropped [or duplicated]. The TCP end-to-end retransmit algorithm [and backoff] is orthogonal to the AQM strategies the articles are talking about. But, large intermediate buffers actually confuse/defeat the TCP backoff strategy and make the problem worse--not better.
When the AQM algorithms work correctly, most packets aren't dropped, because each transmitting end node [eventually] syncs itself to the maximum rate it can send through the [entire] circuit without having data dropped/replicated/delayed unnecessarily.
People don't care about "dropped packets". They care about throughput and low latency. If a packet were never dropped, you'd have neither.