Slashdot Mirror

← Back to Stories (view on slashdot.org)

HTTP GZIP Compression Leaks Data On the Location of Tor Web Servers

Posted by timothy on from the they're-around-here-somewhere dept.

An anonymous reader writes: The GZIP compression format includes a field in its header that shows the Web server's local date, at which the data was gzipped. Almost all Web servers use "zeros" to pad this field by default, citing performance issues. Around 10% of Tor site operators have removed this feature and are printing the packet's compression date. Unknown to them, this "server local date" leaks the Tor site's timezone which law enforcement can then narrow down to a specific geographical area. Coupled with other Tor protocol leaks, this could help deanonymize .onion sites.

24 of 79 comments (clear)

  1. Re:Use a single timezone by The-Ixian · · Score: 5, Insightful

    Or just pad it with zero's like everything else does, apparently.

    Better to go with the flow in this case instead of trying to be clever.

    --
    My eyes reflect the stars and a smile lights up my face.
  2. It leaks the server's location by AmiMoJo · · Score: 5, Funny

    For very large values of location.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  3. Re:And there you are... by bugs2squash · · Score: 2

    You thought it was the onion ring because of the layers, no it's because of the hole in the middle.

    --
    Nullius in verba
  4. What the gzip spec says about MTIME by Anonymous Coward · · Score: 5, Informative

    Relevant parts of the Gzip specification, RFC-1952:

    2.3.1
                      MTIME (Modification TIME)
                            This gives the most recent modification time of the original
                            file being compressed. The time is in Unix format, i.e.,
                            seconds since 00:00:00 GMT, Jan. 1, 1970. (Note that this
                            may cause problems for MS-DOS and other systems that use
                            local rather than Universal time.) If the compressed data
                            did not come from a file, MTIME is set to the time at which
                            compression started. MTIME = 0 means no time stamp is
                            available.

    7.
    When compressing or decompressing a file, gzip preserves the
          protection, ownership, and modification time attributes on the local
          file system, since there is no provision for representing protection
          attributes in the gzip file format itself. Since the file format
          includes a modification time, the gzip decompressor provides a
          command line switch that assigns the modification time from the file,
          rather than the local modification time of the compressed input, to
          the decompressed output.

    1. Re:What the gzip spec says about MTIME by unrtst · · Score: 5, Informative

      Vote parent up.

      The article the summary references is just a summary of this: http://jcarlosnorte.com/securi...

      In which, he notes:
      Offset Size Value Description
          0 2 0x1f 0x8b Magic number to idenitfy gzip streams
          2 1 Compression method
          3 1 Flags
          4 4 Compression Date
          8 1 Compression flags
          9 1 Operating system

      He references that as coming from: http://www.forensicswiki.org/w...
      But that document does not say "Compression Date". It actually says:

      4 4 Last modification time. Contains a POSIX timestamp.

      Even his proof of concept shows that he's parsing that field as a POSIX timestamp: https://github.com/jcarlosn/gz...

      echo date('l jS \of F Y h:i:s A', $rdate);

      It appears that either:

      a) Something else in his php script is setting the TZ before doing that parse
      b) The server is calculating the POSIX timestamp incorrectly, which is a similar issue but quite a different root cause.

    2. Re:What the gzip spec says about MTIME by unrtst · · Score: 5, Informative

      ... just to confirm, the answer is "b": The server is calculating the POSIX timestamp incorrectly, which is a similar issue but quite a different root cause.

      I updated his script to print the difference between the current POSIX timestamp and the value returned by the server.
      bing.com: current - server_value = 28800
      reddit.com: 0
      instragram.com: 0

      Those were his three tests. I'm not surprised the Microsoft server is the one calculating a POSIX timestamp incorrectly. MS folks tend to do timestamp math very poorly. I suspect this only affects Microsoft servers, or horribly misconfigured $anything_else.

    3. Re:What the gzip spec says about MTIME by Anonymous Coward · · Score: 2, Interesting

      Why should it change?

      It's bad enough that any encrypted zip files get bullshit corruption error messages on windows XP and AES encrypted zip files get bullshit corruption error messages on newer versions of windows (see also the bullshit error messages you get when you try to use XP/IE on an HTTPS server configured with modern TLS ciphers, jesus fucking christ Microsoft, can't you just write "this file/website uses a newer, more secure protocol than this version of windows supports upgrade now to windows 10 WINDOWS 10 WINDOWS 10!!!1! I mean, fuck, you might actually convince someone to switch that way...)

      *cough* sorry about that. Anyway, we've got bzip2 and now xz (same algorithm as 7zip) if you want your incompatible next version of gzip.

  5. Not if they follow the spec by marcansoft · · Score: 5, Interesting

    RFC1952 clearly states that the mtime header is a POSIX timestamp, i.e., it is in universal time and not local time. The author of TFA somehow either completely missed or neglected to mention the fact that, per spec, there is no leakage of the timezone, and in fact two of his examples demonstrate exactly that.

    Of the three examples cited in TFA, two of them - reddit.com and instagram.com - follow the spec and use POSIX time. Just run the php tool from TFA and you'll see that the time returned matches the current UTC time. Those servers aren't leaking their location because they follow the spec.

    Only one example - bing.com - uses something other than POSIX time. Surprise surprise, some Windows-based server - presumably IIS? - ignores the standard and leaks the timezone in the process.

    Now the question is, are people seriously running TOR hidden services on Windows machines? That just seems like asking for trouble. The operational security requirements of TOR hidden services are significantly higher than your average server, and I bet the chances of screwing that up with a Windows server are much higher. Leaking the timezone is probably the least of your worries in that case.

    TL;DR Some Windows web server mis-implements the gzip standard and leaks the local timezone in the process. Spec-compliant web servers are not affected. TFA mis-identified two compliant servers as being affected. TFA did not list any Tor hidden services that are affected to allow for confirmation. This is mostly a non-issue.

    1. Re:Not if they follow the spec by jcarlosn · · Score: 2

      Hi marcan, nice to have you included in the discussion. I'm the author of the article. To be fair, the gzip specification was clear from the beginning. In fact I was reading this exact specification when I thought about the impact it could have on tor hidden services. The specification itself clearly states that there are potential problems with universal times under certain systems (i.e. MS-DOS at the time of the specification writing). I thought that maybe current implementations could be flawed, developed the little PoC and started doing tests in the wild. I found that a lot of servers were sending local times instead of unviersal times. The problem with this for me, is that even the universal time leak could be dangerous for hidden services. A lot of hidden services are filtering out 'Date' headers in their http responses in order to prevent getting information from their clock skew. So, yesterday I was with a PoC that was able to extract local time under certain conditions (conditions that I didn't understand clearly) and universal time (thus, clock skew) in other conditions. After discussing it with some friends related to the tor project, I decided that the best was to share it 'as-is'. Maybe further investigation could have been a good idea before sharing, but the possibility of this affecting hidden service privacy forced me to publish it 'as-is', even if further investigation was needed. Today, after a lot of research and help and comments from a lot of people I have better understood the problem and I still consider it a problem. We have found broken implementations of the gzip library under other operating systems than windows and of course, in the end this is a source of potential clock skew leak. So, I consider dangerous and irresponsible considering this a non-issue. I consider important for people managing hidden services to know that gzip is potentially leaking its clock skew, and potentially leaking its local time. I consider important that anyone that is filtering out, or disabling the 'date' header, also takes gzip into consideration. I think that this is just another example of how complicated is to setup a really anonymous hidden service and how even in the gzip stream there could leaks about your local time, or your clock skew. Openly discussing it is very interesting. What worries me is that I received a lot of feedback from people that is running hidden services that didn't knew that gzip was leaking any time information at all. You can read at reddit, or just here how people is surprised to read that in their gzip compressed data there is information about the time, operating system, etc. About your comments regarding why I didn't provide examples of affected hidden services, for me, it is not responsible to openly discussing a privacy issue like that while putting real people privacy at stakes. I consider that putting examples of hidden services compromised by this issue is not going to help in any way to better understand the potential impact of the information contained in a gzip header.

    2. Re:Not if they follow the spec by DarkOx · · Score: 2

      Still leaks information. Even if the time stamp is always in UTC, it remains possible to confirm the server is not traveling at a high enough relative relative to the requester to cause a difference in observed time.

      This is serious folks, it likely means you can in fact determine the server to be on the same planetary body or even in the same orbit!

      --
      Repeal the 17th Amendment TODAY! Also Please Read http://www.gnu.org/philosophy/right-to-read.html
  6. Re:Use a single timezone by Gr8Apes · · Score: 3, Informative

    All my servers are set to GMT. Why? Because when you're running across multiple TZs, it's a hell of a lot easier to trace logs when they use a single common global time. My activities don't care if they're in Asia/Tokyo, Europe/Berlin, Australia/Melbourne, or America/New_York, especially when services cross those regions.

    --
    The cesspool just got a check and balance.
  7. Re:And there you are... by The+MAZZTer · · Score: 2

    This is not a problem with Tor. This is the server operator failing to properly anonymize their server.

    It's like if I go and download and use the Tor Browser, but then fall victim to a phishing scam and give out personal information while using it. Tor will anonymise your connection to websites perfectly fine, but you the user are leaking information about yourself and Tor can't do anything about that. This is the same kind of issue.

  8. Re:Use a single timezone by ShanghaiBill · · Score: 2

    Or just pad it with zero's like everything else does, apparently.

    Even better would be to fill it with a value for a randomly selected TZ. That way you are poisoning the data, so "they" cannot be sure if any TZ fields are valid.

  9. undocumented gzip by TopSpin · · Score: 5, Informative

    There are undocumented gzip command line switches (-m, -M) that control embedding timestamps in gzip archives. They're not mentioned in the man page or --help output, but you can see them in the source here (line 344): http://git.savannah.gnu.org/cgit/gzip.git/tree/gzip.c

    #ifdef UNDOCUMENTED
    " -m, --no-time do not save or restore the original modification time",
    " -M, --time save or restore the original modification time",
    #endif

    I learned about this because I had to ensure consistent hash values of build artifacts for regulatory reasons and I believe it is a misfeature. For me the Principle of Least Surprise would have gzip produce this exact same output given the same input, by default. As it is you get a slightly different output each time you compress the same set of bits, and that is entirely down to this timestamp. I think the fact that switches to achieve that behavior exist yet are undocumented belies some conflict about this.

    --
    Lurking at the bottom of the gravity well, getting old
  10. Re:And there you are... by omnichad · · Score: 4, Insightful

    find a way to slap a VPN after TOR

    You don't have any control over the "after TOR" side of the connection. You could slap a VPN before TOR, or operate an exit node that uses a VPN, but there's no way you'd want to be using your own exit node if you wanted the protection of TOR.

  11. Not nothing-- everythings by XXongo · · Score: 2

    This has nothing to do with Tor and has everything to do with incompetent sysadmins.

    And if the sysadmins of Tor nodes are incompetent, it has everything to do with Tor.

  12. Re:And there you are... by pr0nbot · · Score: 2

    I see TOR kind of like HTTPS: it won't necessarily keep your transmission from being decrypted and deanonymized, but it probably makes it much harder to do so. As such it just sort of raises your default level of privacy (from plain HTTP).

  13. Re:And there you are... by Anonymous Coward · · Score: 2, Interesting

    but there's no way you'd want to be using your own exit node if you wanted the protection of TOR.

    And using someone else's exit node to access your own VPN is also a bad idea.

  14. Re:And there you are... by arth1 · · Score: 2

    Yea because knowing the servers location at the resolution of a timezone will help a lot...

    If the time zone is the Winamac time zone in Indiana, or some of the other very regional time zones, it may.

    It's just another datum in fingerprinting, but in some cases, it may be the crucial one.

  15. Re:And there you are... by Motherfucking+Shit · · Score: 4, Insightful

    It could be more helpful than you think. If the server says its timezone is in the US, for example, that may be enough for a judge to grant the FBI a warrant authorizing god-knows-what attacks against it.

    --
    "BSD: Free as in speech. Linux: Free as in beer. Windows 10: Free as in herpes." --Man On Pink Corner in #52607549.
  16. Re:Use a single timezone by Bert64 · · Score: 2

    You're thinking of BST, GMT is constant and the uk switches to BST during the summer.

    --
    http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  17. Re:Use a single timezone by Actually,+I+do+RTFA · · Score: 3, Insightful

    Almost every attempt to poison data turns into another datapoint. That datapoint is likely more valuable than a NULL value.

    For instance, that leaks data about your pseudo-random number generator, opens up timing based identification, etc.

    --
    Your ad here. Ask me how!
  18. Re:And there you are... by mitcheli · · Score: 2

    Let's see... FBI takes over Onion servers, supposedly paid universities for "research", managed to find the operator to silk road 2.0, and managed a rather large bust of criminals. Then you've got the issue of "fake tor clients"... Seems to me that considering tor (be it with a VPN before or after) is really irrelevant and that the underlying technology of tor has just been under attack for two long and needs to be replaced.

    --
    Select from tblFriends where interesting >= 4;
  19. Re:Default Timezone by TheCarp · · Score: 2

    Exactly...

    You would think nobody who decided to rob a bank would write the note on the back of mail addressed to themselves.

    Hell you might think that nobody would think "hey, in this live production stock exchange trading system, lets try entering a value of -6"

    Then you might think "Surely nobody developing a live production stock exchange trading system would ever simply cast a signed integer into an unsigned integer and allow a user to accidentally post a 69 trilliong dollar trade as a result?"

    You would be wrong on all three, but, the first happened so many times you can find those for yourself:
    http://news.slashdot.org/story...

    --
    "I opened my eyes, and everything went dark again"