Slashdot Mirror
Deserialization Issues Also Affect .NET, Not Just Java (bleepingcomputer.com)
"The .NET ecosystem is affected by a similar flaw that has wreaked havoc among Java apps and developers in 2016," reports BleepingComputer. An anonymous reader writes:
The issue at hand is in how some .NET libraries deserialize JSON or XML data, doing it in a total unsecured way, but also how developers handle deserialization operations when working with libraries that offer optional secure systems to prevent deserialized data from accessing and running certain methods automatically. The issue is similar to a flaw known as Mad Gadget (or Java Apocalypse) that came to light in 2015 and 2016. The flaw rocked the Java ecosystem in 2016, as it affected the Java Commons Collection and 70 other Java libraries, and was even used to compromise PayPal's servers.
Organizations such as Apache, Oracle, Cisco, Red Hat, Jenkins, VMWare, IBM, Intel, Adobe, HP, and SolarWinds , all issued security patches to fix their products. The Java deserialization flaw was so dangerous that Google engineers banded together in their free time to repair open-source Java libraries and limit the flaw's reach, patching over 2,600 projects. Now a similar issue was discovered in .NET. This research has been presented at the Black Hat and DEF CON security conferences. On page 5 [of this PDF], researchers included reviews for all the .NET and Java apps they analyzed, pointing out which ones are safe and how developers should use them to avoid deserialization attacks when working with JSON data.
Organizations such as Apache, Oracle, Cisco, Red Hat, Jenkins, VMWare, IBM, Intel, Adobe, HP, and SolarWinds , all issued security patches to fix their products. The Java deserialization flaw was so dangerous that Google engineers banded together in their free time to repair open-source Java libraries and limit the flaw's reach, patching over 2,600 projects. Now a similar issue was discovered in .NET. This research has been presented at the Black Hat and DEF CON security conferences. On page 5 [of this PDF], researchers included reviews for all the .NET and Java apps they analyzed, pointing out which ones are safe and how developers should use them to avoid deserialization attacks when working with JSON data.
Just don't use JSON or XML. You can thank me later.
"Transparent" is a shit show that trades on every stereotype going. A man in drag is NOT a transsexual.
Real developers use an XML or JSON reader instead of using direct deserialization. Trust me I've built systems both ways and deserialization directly into objects is no bueno. You end up with more problems with version compatability alone to negate the benefits. There are also performance issues as well.
We'll make great pets
This is a programming problem that can happen anywhere. No language is immune. No project is automatically secure from exploits, or able to patch framework universally for all deployments.
Java and .NET will always have security issues, along with literally every other programming language. Anyone shocked, surprised, upset, or hostile to that concept is in the wrong profession.
Assume everything is compromised. Assume nothing is secure. Design around that assumption and you will survive.
Completely agree. We used .net binary serialization/deserialization because it was such a quick way to get things up and running...with like two lines of code. The fact that the serialized objects were about 10x bigger than they needed to be was not a problem.
It turns out the namespaces are included in the serialized data, so the moment we did an ounce of lightweight refactoring we broke it. It took us less than a day to write our own serializer, but an extra three days of combined manpower to get a format converter built and deployed.
We would have saved all that if we had just started with our own serializer.
Prefab solutions to simple problems are a deal with the devil. Beware!
I'm kind of surprised this hasn't already built into a more prominent issue over time.
Performance issues I can stomach - there's going to be some unavoidable parsing logic no matter how you go at translating from runtime to storage or network logic - but instead, large swaths of objects just get ignored in major libraries. When using unity, for instance, can't serialize dictionaries, and many other objects in the default serializer - which is a major oversight.
Google actually has provided some rather nice tools to help with this - I tend to use their 'Protocol buffer' libraries for their rather nice serialization options. This doesn't address security on its own - nothing does completely, but designing careful locked signal processing and independent cross-checking steps can help a lot. Well-salted encryption alone won't really save you.
My pet peeve with protocol buffers the need to give everything an index number, with no real auto-numbering for rapid design - I can see the logical need, to be able to rely on that order for processing - it's just an extra babysitting step that gets me sometimes. For what it does, it's still the best I've found to be consistent between diverse projects and still leaving room for decent security.
Ryan Fenton
Google engineers banded together in their 20% time.
Peter predicted that you would "deliberately forget" creation 2000 years ago...
I presume that the code performing the marshaling or unmarshaling of objects tries to be overly generic and so treats it's input as if it were a mini scripting language of it's own. ie crawls through the input and handles what it finds by having the input dictate what methods to call to get the job done. I'm sure it can be done safely, but it's probably easy to err on the side of "clever generic" code that is exploitable.
Nullius in verba
Walking a data tree is 1st year CS level work. If you're spending half your efforts on it then you're either vastly short on resources or your coders suck donkey balls.
Data by itself doesn't do much. The way to think about data is that it is being fed into a machine that is doing stuff. That means I can program the internal state of that machine using data. Normally we just call this "processing" but bugs like this illustrate that you have to be very careful with how you handle state. Even for "simple" formats that are just "text" like JSON, XML, YAML, and everything else. Image (binary) formats are also not immune as there have been browser attacks using bugs in common image libraries. If your data is used as input, it is executing passively.
Personally I think "exposing" objects is the problem. Your border should be a mailbox that exchanges messages and those messages should be inspected carefully before internal delivery. I have no idea why people want to dump a class they wrote onto a live internet service and just hope that it dumps data into the correct table somewhere. They dragged the "security api" icon onto the project space so it's secure.
It appears that the market is flooded with developers who can write scripts but not algorithms. They believe that something like parsing JSON is really hard and complicated, that any home-grown solution to doing that will be extremely buggy and slow, all because they themselves haven't taken the mental step-up.
Of course, this mental step-up used to be a standard part of a CS degree. College students would be writing code that does this sort of thing as homework. This has changed, and I have seen the change in the candidates we interview. I ask them questions about their courses in algorithms and what they did, and they say things like "we learned what the foundational algorithms are and how to compare their performance." Did you actually write a merge sort? "No, there's no need because every major language has that sort of thing built in."
So, there's the rub. They paid good money for a degree that glanced over the most important bits. Naturally, they feel completely justified in their beliefs that stringing third-party solutions together is the best way to write code.
And a whole new crop of these scripters hits the job market every year, more than we have seen in a decade. Colleges have been lowering the bar due to higher interest among students that aren't really cut out for it, that in turn due to successful social engineering on the part of the tech giants.
At least, that's my hypothesis.
JSON only defines a bunch of basic data types. It defines no ability to run anything. These bugs are in (de)serialization layer above it, which uses JSON as a transport and extend the meaning of the data stored to be able to deserialize higher-level objects.
JSON or XML are not the problem here. The same problem could happen if you serialized to CSV or TXT or anything else for that matter.
Can someone explain what the problem is here? Serialized objects are just code, and if you're running untrusted code you've got bigger problems than bugs in your serialization libraries.
These frameworks will de-serialize any object. Send them a Process object in .NET, and the framework will deserialize it into something that can fork a new process. The APIs in Java and .NET are so huge, that it is extremely difficult to filter out every kind of object that might cause problems (some frameworks try......and fail).
There is no 'pure' data here, the purpose of these frameworks is to deserialize into objects, and objects by definition are functions combined with data.
"First they came for the slanderers and i said nothing."
In this case it happens when "Object Oriented" is taken too literally. People think of data as inert. People think of "Objects" as inert. So they figure translating between data and objects is just transforming one inert thing into another.
But "objects" are not inert in almost any dynamic language. They are quite active, with instantiation methods, etc., and some are quite dangerous. One has to adjust one's paradigm when learning OO programming from a procedural background.
Someone had to do it.
They aren't part of. Net itself, just third party libraries.
Tired of my customary (Score:1)
Please clarify instead of posting cryptic pointless posts.
"First they came for the slanderers and i said nothing."
As stated int he linked document, for JSON.NET to be vulnerable, you have to explicitly set an option making it less secure.
As with encryption and security libraries, you are better off using well-established libraries like JSON.NET than rolling your own. A solo developer, or corporate team, just doesn't have the resources or time to work out all the security vulnerabilities, as can be done with a dedicated library.
He'll be telling us Rust is webscale next
This is not surprising ! We discovered recently some "billion-laughs"-style DOS attacks that exploit vulnerabilities in Java, and ported some of them to .NET and Ruby. Details here:
http://drops.dagstuhl.de/opus/... (paper, there is also an artefact to run attacks in a VM), and the source code is here: https://bitbucket.org/jensdiet... .
We did have some problems porting this from Java to .NET but managed eventually. Interestingly, some of these problems were caused by a bug in .NET: a broken contract between equals and hashcode (see https://github.com/dotnet/core...) .
Walking a data tree is 1st year CS level work. If you're spending half your efforts on it then you're either vastly short on resources or your coders suck donkey balls.
Correct and then 2nd year CS work is red/black trees and other advanced algorithms. The whole time you're learning this you're taking increasingly higher levels of mathematics. You see the problem I run into is that many people couldn't make it to the 2nd year in Computer Science and went into Information Technology or some other "Computer Science Lite" field of study. These are the majority of people in the field now that really lack to ability to understand the difference between different implementations logically and mathematically. Computer Science is a hard degree. It's called Computer _SCIENCE_ for a reason.
We'll make great pets
> will deserialize it into something that can fork a new process
Only if you tell it "hey please put this into this insanely insecure class that will fork a process". Serialize your shit into stupid DTOs and you are dandy. That has nothing to do with the API surface.