Slashdot Mirror
Microsoft To Banish Memcpy()
kyriacos notes that Microsoft will be adding memcpy() to its list of function calls banned under its secure development lifecycle. This reader asks, "I was wondering how advanced C/C++ programmers view this move. Do you find this having a negative impact on the flexibility of the language, and do you think it will restrict the creativity of the programmer?"
Just like removing printf, scanf, and most other copy/string functions. There are safe versions of memcpy that work just fine and are just as easy to use...
Lame story (Trying for flamebait here?)
I have mod points and I am not afraid to use them
This should have been done thirty years ago.
Those are also dangerous functions. And also array indexing! That should also be eliminated.
Figures, Microsoft had to go kill of python and do it all in the name of security. No more accessing MEMory in C structures from our .PY files, damn it this really pisses me off.
AFAIK memcpy() is only one of the many ways in which you can hang any half-awake coder himself in C (some manage it when awake, but let's say charitable - I'm ignoring the chorus asking "why?").
Well done for thinking about security, boohoo for going straight to the press and trying to milk it instead of bloody DOING something for a change. Trust comes from casual "oh that? Yeah, we thought that was unsafe so we improved it" instead of trumpeting high and wide "look everyone, we bought a new padlock. Look how shiny it is, and how big" whilst still having a wooden backdoor with a simple latch only.
Do, don't talk. Show me. We've 20 years of marketing so pardon me for being cynical.
=Smidge=
Is it just my observation, or is eldavojohn an idiot?
good
Do you find this having a negative impact on the flexibility of the language, and do you think it will restrict the creativity of the programmer?"
You can replace memcpy entirely with memmove (the latter is slightly slower and handles overlaps), and nothing in the article suggests that memmove is banned.
But, no, it shouldn't hurt creativity--they're introducing a memcpy_s, which is the same aside from taking a size parameter for the destination. That's something that is generally easy to track in new code (obviously this secure developement lifecycle is not backwards compatible).
rage, rage against the dying of the light
A corporate-level "improvement" officer at Microsoft asked: What part of our operating system is the source of slow downs in our applications and partner applications?
A profile of code showed that memcpy() was taking the most CPU time. Instead of asking why memcpy was being called so much and suggesting removal of unneeded memcpy calls, they decided that nobody should use memcpy because it has the possibility of being used incorrectly when their own programmers don't know how to determine the proper size of data structures.
...and pop up a message box asking the user to confirm they want to copy the memory, and if they press OK then they should have to enter a captcha.
Seriously though, how is it supposed to make your code safer if you pass the size you think your destination buffer is? With memcpy, that size is implicitly greater or equal to the copy size and it's the caller's responsibility to make sure this is the case. Putting bounds checking into the copy function is ridiculous if you're responsible for passing the bounds yourself, and it goes against basic good design. I'm surprised they aren't passing the source buffer size too, just to be extra safe. Also, what happened to the __restrict keyword? It's strangely absent from the memcpy_s function declaration.
=Smidge=
Is it just my observation, or is eldavojohn an idiot?
I can understand the warning "Warning: you are using an Evil Command(R) "memcopy" and your program will be a piece of crap as a result", but there's no need to break already compiling code.
(FP btw)
First they came for gets, then they took scanf and strcpy, now they want memcpy? Outrageous! How are virus writers going to be able to take advantage of buffer overflows if I'm continuously keeping track of how big my buffers are? I may have to start lying about their size just to give hackers a chance.
Someone already explained this better than I could.
Lots of hand-waving marketing bullshit. I'm sure they're going to keep using it internally, and the exploits will still happen with Microsoft code. Just like Microsoft applications will be ignored by UAC in Windows 7.
If they wanted to do something useful, they should have removed createRemoteThread instead.
www.lucernesys.comHorizon: Calendar-based personal finance
I have often used memcpy instead of strcpy when I have known the length of the strings, and also known the destination to be large enough.
I'm guessing many developers will just #define memcpy to something else and continue as nothing happened.
/ The Arrow
"How lovely you are. So lovely in my straightjacket..." - Nny
If you consider memcpy too dangerous then you should be using something besides C. If you're using C++ and memcpy then you really do need to know what both you and the compiler are doing.
The problem with strcpy() and sprintf() and like functions is that you don't know when calling them the length of the source to be copied into the supplied buffer. But with memcpy() you specify this length.
Frequently, the size of the target is calculated at run time, so bugs in memcpy() tend to be in the area of this calculation, rather than in not checking if the source fits the target.
Any lack of memcpy() would be easy to overcome, just use
memcpy_s (dst, len, src, len)
which is functionally identical to
memcpy (dst, src, len)
I don't trust outright removals without a decent period of deprecation. Microsoft has a bad history of deciding some API or function is dirty and obsolete, only to find that they've broken some of their own code or made some functions impossible to implement in some environments.
For example, they deprecated a Windows Mobile database API without having support for the new version in ActiveSync. Lots of suckers upgraded only to find out that the Grand Pubas down at Seattle Central pulled the rug out from under them.
I worry very much about problems in eccentric environments like mobile or embedded development.
I swear to God...I swear to God! That is NOT how you treat your human!
Java in inefficient clumsy and bloated. For doing any serious computationally intensive apps C/C++ is still far more effective. The work I do couldn't even be managed in java and memory usage would go through the roof (we tried it 8gb++).
Wow, all this new lingo is confusing me. I bet everyone on Twitter knows it, though. Well...I memcpy you too!
Most any security problem can be traced back to this function.
As Windows products are now (and have been) mainstream products used extensively in banks and other financial institutions, reliability and security (RS) have prime importance. The speed that "memcpy()" gets you is not worth the price of reduced RS.
Modern complex C++ class objects and structures, always seem to have a 'duplicate', 'transfer' or 'swap' function rather than a simple 'memcpy'.
With a complex data structure with dynamically allocated linked lists, trees or chains of data, the assignment over '=' is rather ambiguous. Does it mean duplicate the contents, transfer the contents and clear the original copy, or just swap the contents of the items, which might be quicker.
Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
Java, and other managed languages are the way to go.
Until you're trying to do something for which Java has no standard API. Last time I checked, USB joysticks were like this. There is JInput, but if you include JInput in your distribution, your project is no longer 100% Pure Java and will not run as an applet.
Or unless your target platform is incapable of running managed code. Some handheld platforms have only 4 MB of RAM, not enough for a JIT compiler and the bytecode in addition to the translated code except in trivial examples that act a generation old. And per Apple's developer agreement, the only managed language that can run on an iPod Touch is JavaScript in Safari.
Or unless your existing program's model is written in an unmanaged language, and you want to reuse the old code so that you can be sure that the model is bit-for-bit accurate. How hard is it to automatically translate C code to, say, C++/CLI?
This move my M$ is intended to remove creative solutions from the table. Granted, many creative solutions are used to undermined widely used software security, but this isn't the right solution. Unfortunately, I don't have any better ideas.
Oh that's so cute. Another emoticon! I'd like to say--main() you too!
This is nothing like sprintf. In sprintf there is no way to know how much data will be created ahead of time, so limit on buffer size is useful to make sure there is no buffer overrun.
With memcpy it is *precisely* known how much data will be copied. It is right there, 3rd parameter. If a developer can't do "if (sizetocopy = sizeofdstbuffer)", it is just as unlikely that he will be able to properly state that additional parameter that specifies the destination buffer size.
Of course if Microsoft is so concerned with security, why the heck did it take them years to add snptinf()? All this is is another attempt to make crossplatform development that much harder (much like all those "obsolete" POSIX functions that will barf warnings unless you use a cryptic define).
That said, if this silliness ever becomes a rule, I have an easy solution:
#define memcpy(dst, src, size) memcpy_s((dst), (src), (size), (size))
Problemo solved, now let's go actually write some real code.
This is not the first time MS has done this. They have plenty of other standard functions that they have deprecated.
Yes, you read that right. Microsoft is deprecating parts of an ISO Standard all by themselves. Not that this should surprise anyone. I would have absolutely no objection to them proposing to WG14 to deprecate those functions; heck, I'd encourage it! But besides going out and deciding to 'deprecate' parts of the standards, the replacement functions actually violate those same standards.
And the warnings are irritating. You can't write a nice cross-platform library without either spewing tons of warnings or having to put in a bunch of #defines to shut the compiler up. And if you do that, your users get irritated if they depend on these warnings because you just turned them off (and of course, if you don't, they'll complain that your library is unsafe).
Screw Microsoft.
The safe alternative is called '='. In standard C, you can just use the assignment operator between any data type (including struct and everything). The compiler will even type-check that assignment for you. That's probably one of the most underused features in C. I think it was introduced with C89, but the use of memcpy() was perpetuated by programmers raised on K&R.
Yes, '=' doesn't do deep-copy, but neither does memcpy(). If that is an issue, use C++ and write an assignment operator.
Why? I can see some justification on the strXXX functions where you don't know how many bytes are going to be copied unless you call strlen first, but in memcpy you pass how many bytes to copy in as a parameter. So this is to protect programmers who can't do math?
Yes. The article describes a replacement function memcpy_s that compares the copy size to the destination buffer size and throws an exception if the copy size is larger. It's still unsafe if the program lies to memcpy_s about the destination buffer size, and now it appears to needs exception support in the runtime (which based on my tests can add an extra 64 KB to your elevator controller's firmware).
Now ban the rest of the C library, and then ban the C language itself.
Oh lol, captcha for this post is Pervert.
I noticed it worked one time, but thought one of our targets would whine about it.
Some of these reactions are quite funny.
The goal of asking you to specify the length of the destination buffer is to force you to think about the data you're working with *while* you're writing the code and not afterwards in an unconnected security audit. Furthermore, it provides documentation to other people reading the code who may not have the same mental model of what's going on as you do. And as usual "other people" includes you, six months after you wrote the code.
The difference between theory and practice is that, in theory, there is no difference between theory and practice.
Hey Microsoft, instead of banishing a function that is perfectly safe and standardized for decades why not hire competent programmers to begin with? I mean, the function copies a fixed number of bytes from an object to a pointer. How hard is it to understand that you need to allocate memory and that writing to a memory location that isn't yours is a bad thing?
Seriously, what idiots are doing the hiring and training in that company?
#define memcpy(s1, s2, n) memcpy_s((s1), (n), (s2), (n))
Another bad Slashdot summary. Safer alternatives (e.g. memcpy_s) are already in place, but somehow that wasn't mentioned up where it should have been.
Bad Slashdot editors -- no soup for you!
"It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."
. . . so ban it. If I really need it, I'll write my own. What happened when the US banned alcohol? Bootlegged Moonshine.
" . . .do you think it will restrict the creativity of the programmer?"
Quite the opposite, it will inspire them to find other creative ways around the restrictions.
Oh, and you can grep your code and say, "Look! No memcpy()! I'm secure!" But what about self-written functions that does the same thing as memcpy(), with 1,000 different names?
Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
Indeed, throwing an exception is the worst thing you can do to such a low-level function (and yes, memcpy is about as low level as you can get).
High-level C++ code shouldn't use memcpy, simply because it's too low-level (it won't even work correctly if there's a non-trivial copy constructor). Use std::copy instead. For low level code, memcpy makes sense, but exceptions don't.
Why not just doing the same as the strnXXX functions, i.e. just stop copying at the end of the destination buffer? That is, just implement it as
The Tao of math: The numbers you can count are not the real numbers.
How hard is it to automatically translate C code to, say, C++/CLI?
My guess would be less difficult than first suspected, but fairly laborious.
feldicus
Or you can use a general purpose language and be done with it.
Python has its uses, but it also has its overhead. Good luck getting any kind of performance with pure Python code on a handheld device with 4 MB of RAM and a 67 MHz CPU. Or good luck getting a good frame rate with a graphics engine that uses PyOpenGL. (And why is the snake wrapping itself around EA Games' old box-ball-cone logo?)
In soviet russia...
NO SIG
As a competent developer, I get extremely annoyed by this sort of shit.
Removing/banning memcpy doesn't change a damn thing cause the first thing I do with things that have to compile in VisualStudio now is add the following defines which turn this shit off:
_CRT_SECURE_NO_WARNINGS
_CRT_NONSTDC_NO_DEPRECATE
If the remove that option I'll simply add memcpy to my standard MS compatibility library that deals with all the other bullshit MS decides to do.
You can't fix stupid. Stop trying. People fuck up VB and C# apps just as much as the fuck up C and C++ apps. So they don't do it with a buffer overflow, they do it by shear stupidity. You'll be more secure by taking away languages that allow non-programmers to pretend to be programmers than making it harder on those of us that are just going to work around what you do anyway.
You're not going to fix broken shitty apps with exploits by removing functions, the functions aren't the problem they do exactly as they are told (or atleast they are supposed to :). You need to fix the programmers who can't clarify what they want done.
http://www.xkcd.com/568/
Second pane:
You'll never find a programming language that frees you from the burden of clarifying your ideas.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
It should be noted that it does make it harder to do the wrong thing, when memory leak tools test the code.
Moderators: Before moderating a comment Insightful/Informative, check to see if a child post has already refuted it.
...I'll just have to assume this is a good thing and take it from you C/C++ guys.
How long until someone circumvents that with a while() with strncpy() and strlen() inside?
As long as the new version is just as efficient then there's not really an issue. Not everyone likes to play the MHZ card to justify bloated, inefficient code.
Removing the functionality or crippiling it is as stupid as requiring training wheels on all bikes because some people fall down and hurt themselves.
Work Safe Porn
Just write a one-liner that replaces all calls to memcpy with a call to memcpy_s, duplicating the size parameter.
I'm only half-joking. This is exactly how people will (mis)use memcpy_s. If you want safe memory access, you need to ban the entire C language. For those cases where you need C, you'll just have to make sure your programmers know what they're doing.
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
How to easily make your code compliant with the new safety requirements:
#define memcpy(dest,src,len) memcpy_s(dest,len,src,len)
The only use there is in C++ for usage of memcpy is when you want to run afoul the strict aliasing rules.
Otherwise, you use std::copy.
Firstly, the specification of C anf C++ standard library is governed by the corresponding standard commitee. Microsoft has absolutely no authority to "banish" anything from neither C nor C++. They can deprecate it in their .NET code, C# etc., but it has absolutely no relevance to C and C++ languages. So, why would the author of the original question direct it to "advanced C and C++" programmers is beyond me. In general, C and C++ programmers will never know about this "interesting" development.
Secondly, the tryly unsafe and useless functions in the C standard library are the functions like "gets", which offer absolutely no protection agains buffer overflow, regardless of how careful the develoiper is. Functions like 'memcpy', on the other hand, offer sufficient protection to a qualified developer. There's absolutely no sentiment against these functions in C/C++ community and there is absolutely no possiblity of these functions to get deprecated as long as C language exists.
Fail.
You clearly don't know what you're doing. Anything a non-managed language can do, a managed language can do. The only things that should be coded in C are (parts of) OS and embedded code. CPU is cheap, tracing down buffer overflows and segfaults is expensive. And for your dataset...did you make every piece of information its own class? I bet you did, didn't you. Incompetent failure...go back to IT Tech.
Why not just doing the same as the strnXXX functions, i.e. just stop copying at the end of the destination buffer?
Because the copy functions are often used to copy array-lists whose end is defined by a sentinel value. You might recognize C strings, which are array-lists terminated with a '\0' character. Using min() doesn't guarantee that the sentinel value gets written. It'd be safer than plain memcpy() in that it can't overwrite the data after the destination buffer (if the size is accurate), but reading from that unterminated buffer probably still leaves a hole for some attacks.
Oh, stars above, what have we done...
If you mod me down, I shall become more powerful than you could possibly imagine.
but mine already has an argument for the size of the destination buffer, it's the third parameter. Why not just update the documentation? Are they really just adding a second size parameter, just to compare them? or will they use min(size1, size2) on the actual memcpy call, so that nobody knows exactly how much data is copied anymore? you don't want to read past your src buffer after all, right?
--
Stay tuned for some shock and awe coming right up after this messages!
... if I'm still using it as long as I'm not using Microsoft tools to write/compile my source?
Have gnu, will travel.
WHOOOOOOOOOOOOOSH.
Now, all that's going to happen is that programmers are going to write their own memcpy-like routines using a quicky for-loop or something. It'll be just as bug prone, and harder to detect via automated source code analysis.
...but it's being eaten...by some...Linux or something...
Microsoft yet again demonstrating they believe everyone in the world except them only deserves plastic scissors.
I hope that in the name of safety, Microsoft continue to remove all remaining potentially useful functionality so that eventually even developers wont be able to do anything productive with their products at all.
Then finally, even consumers and upper managers will have to understand how shitty Microsoft products are, so will stop buying and enforcing the use of Microsoft products on the rest of us.
Java? Unlambda!
<marketingspeak>
</marketingspeak>
The Tao of math: The numbers you can count are not the real numbers.
Silly humans. Use Ada if you want to build something that works.
If you wrote a program that used 8+ gigs of memory that means you're an incompetent code monkey.
Until you're trying to do something for which Java has no standard API. Last time I checked, USB joysticks were like this. There is JInput, but if you include JInput in your distribution, your project is no longer 100% Pure Java and will not run as an applet.
But your C/C++ code is never able to be run as an applet, while Java code might. So, on these grounds Java is better.
Or unless your target platform is incapable of running managed code. Some handheld platforms have only 4 MB of RAM, not enough for a JIT compiler and the bytecode in addition to the translated code except in trivial examples that act a generation old.
And for that we have Java ME edition.
Okay, there are devices that are even more tiny in which C or assembly is the only option. I've also dealt with a processor that couldn't be programmed in C without extensive non-standard C extensions. Either way, this is a small minority of situations -- there is no one solution that suits all purposes.
And per Apple's developer agreement, the only managed language that can run on an iPod Touch is JavaScript in Safari.
And they allow just any old C code to run?!!
Or unless your existing program's model is written in an unmanaged language, and you want to reuse the old code so that you can be sure that the model is bit-for-bit accurate. How hard is it to automatically translate C code to, say, C++/CLI?
Eeek! Don't do this!
If you want bit-by-bit accuracy, you absolutely don't want to try translating C code to C++! You even have to worry about changing C compilers if you are doing that. (And yes, I have worked with C code which would break if we used a different compiler. It was not fun.)
What you should be thinking of doing is to write an interface between the existing model code and your new one. This can be done in both C++ and Java. When the new code has to manipulate the model, it goes through this interface and calls the old code to do the necessary operation.
And for [platforms with small CPU and small RAM] we have Java ME edition.
That is, if your platform vendor offers a port of Java ME.
If you want bit-by-bit accuracy, you absolutely don't want to try translating C code to C++! You even have to worry about changing C compilers if you are doing that.
I already test my code on GCC targeting x86 and ARM. Or did you mean compilers that a hobbyist looking to start a business can't easily afford?
What you should be thinking of doing is to write an interface between the existing model code and your new one. This can be done in both C++ and Java. When the new code has to manipulate the model, it goes through this interface and calls the old code to do the necessary operation.
Mostly I was thinking about trying to port an existing video game to a phone that runs Java ME MIDP or a game console that runs XNA, while preserving frame-for-frame accuracy of the physics. You can't run the old model on such hardware because you don't have the certificate to digitally sign native code. Or did your "interface" include communication over a network to a server running the model?
I've just finished evaluating approximatley 500kloc code using Coverity Prevent and Klocwork Insight (Static analysis tools). Together they found around 250 issues with the code. Null pointer dereferences, array boundary violations and a bunch of other nasty stuff.
How many of these do you think involved memcpy? - Nil, Zip, Nada.
Break the sound barrier - bring the noise.
According to MSDN docs, memcpy_s takes the size of the destination buffer as the second parameter. Gee, that's easy! memcpy_s(&dest[offset], sizeof dest, src, count);
Except, oh wait, if you write "sizeof dest" bytes starting from the middle of dest, that can still overflow.
The fact is, this doesn't affect security AT ALL. If someone's enough of an idiot that they can't even use a function as simple as memcpy safely, they'll find a way to screw up memcpy_s too. You want secure code, Microsoft? Stop letting morons write code!
On Windows, it throws if you access unmapped memory, even in your own code.
http://xkcd.com/327/
Unwise move by Micro$lop, or perhaps another 'subvert Linux' attempt.
Firstly, in the grand scheme of things, memcpy() is NOT a source of very many security problems. Maybe it is for the Microsoft-way-of-coding it is, but that's a self-inflicted bit of stupidity. Speaking from many years of developing an enterprise-grade HIPS (Host Intrusion Prevention System).
Secondly, memcpy() is one of the key performance routines for any computer system. This is one of the handful of routines which is perpetually written in assembly, hand unrolled, does prefetching, and is highly aware of cache lines/sizes.
Hmmm - wait a minute, this is great for me - I currently work at a chip company, and this will require everyone to buy much faster CPUs!
If memcpy() is hazardous, that means they also need to ban any operations that can mimic its effects, such as for loops, array indexing, ++ and --. And any call to read() or fread(). That sure would tighten things up!
What C really needs is variants of malloc and mem* that actually explicitly and officially keep track of the sizes of the allocated buffers.
Right now, you can get that during debugging with valgrind, but you don't get those checks at runtime.
Besides:
#pragma warning( disable: 4996 )
Thanks
There have been several suggestions to replace memcpy with memcpy_s as the safer alternative. That's fine, I guess, if memcpy_s is part of the ANSI/ISO standard for C, which as far as I know, it is not; just like all the *_s functions.
Microsoft says your code is safer when using the *_s, but it will no longer be portable, it'll be Microsoft-only. They put in a warning in the compiler from VS2005 onwards about using "unsafe" functions, and that you should use *_s, which is a pain because you have to disable it as the project level, there doesn't seem to be anywhere that I've found that can just turn it off permanently. Even using the STL that comes with VS2008 will generate these warnings, even if you never do any explicit memory stuff yourself.
Microsoft did the same thing with the _* functions; a lot of them are just wrappers around their ANSI-compliant versions (_sprintf -> sprintf), but are also not portable; I worked with a guy who wrote/tested all his C code in VS6 then gave it to me to port to Unix and VMS, and the compilers would choke on not having these particular functions.
Microsoft is trying to get lock-in at the language level instead of providing a good set of Win32 API-based functions that make using memcpy() unnecessary.
msmemcpy (TM), the proprietary alternative that is completely incompatible with any c/c++ standards, will only run on the MS platform, will have plenty of undocumented "features", and therefore at least as insecure as the original. Heck, we might even get another ISO norm out of it.
Given that you do not know the size of the problem, claiming that any program that uses 8+ gigs of memory means that its programmers are incompetent shows a severe lack of foresight on your part. Unless of course you think 8GB should be enough for everyone.
No need for everyone to freak out because functionality is going to disappear. From TFA:
So you're not really losing functionality, but the new API tries to cajole the developer into considering the size of the destination. And the old memcpy() will still be available, just considered unsafe.
Foolish mammal, they cannot be defeated so easily. http://xkcd.com/292/
#define memcpy(a,b,c) memcpy_s(a,b,c,c)
Chuckle....
Let's also ban assembler code, fine, great!
To ban a function is a symptom of idiocy, plain and simple.
What's in a sig?
Comment removed based on user account deletion
If you wrote a program that used 8+ gigs of memory that means you're an incompetent code monkey.
I have an hourly job that processes about 8GB of input data files. We found that copying the data to a tmpfs filesystem instead of leaving it on a HDD cut the work time from 20 minutes to about 30 seconds because the job necessarily requires an enormous number of random seek()s. Since mmap()ing a file on tmpfs is roughly identical to read()ing the whole file into RAM, I guess that makes me an incompetent code monkey.
Of course, my boss who got a 40x speedup in exchange for $250 worth of RAM might see things differently from your inexperienced little self. Sometimes the Real World throws pretty big datasets at you.
Dewey, what part of this looks like authorities should be involved?
What does this have to do with the creativity of the programmer? I can tell you right now that my creativity hits a brick wall when the code crashes down the line because I corrupted the stack after assuming my destination buffer is large enough to hold the specified number of bytes. If one doesn't know the size of the dest buffer before copying into it, maybe one should put creativity on hold for 2 minutes while one finds out.
Moves like this, plus the whole notion of managed code, automatic garbage collection, and other crap is to allow incompetent programmers to be able to write code. Most code additions like this that have been added in the last 20 years have created bloated, slow, and overly complex runtime systems that aren't really needed if you know what you're doing.
"Real" software engineers don't need all this hand-holding. They don't need "training wheels" on their code.
all i can think is this is whats best for...
developers developers developers developers......
Good people go to bed earlier.
What, too soon?
I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered. My life is my own.
C/C++ are fundamentally unsafe and should only be used by someone who knows what they are doing.
Writing > 1 Million LOC in C++ you are bound to have serious memory problems.
Passing size parameters only helps slightly.
Runtime size checking is what is needed and is a much smarter solution.
Of course it might be said that forcing the programmer to state it twice will (for low error incidence rates) half amount of bugs, this only applies when the probabilities are uncorrelated and I can see a problem looming right there. What they should have done is use C++'s type checking features to make sure that invalid copy operations simply don't compile. But that would have been way too practical. Also, what happened to "never postpone to runtime what you can do during compilation"? Catching bugs is nice and all, but wouldn't it be better to catch them before you ship? Nonetheless, banishing Memcpy is in itself a good idea and I think Microsoft does deserve some points for that.
We use Ada for embedded code. It combines the power of C with the safety of ... Ada.
I'm no fan of MS for a variety of reasons, but there are a couple things about these secure_s functions that are worthy of consideration.
First of all, there's the explicit max-bytes-to-copy that's declared in memcpy_s(). Many people have pointed out that this is only as safe as you're trusting the programmer to be anyway, which is true to a certain extent, but it also calls on the coder to be acutely aware of the maximum buffer size.
In this way, it makes the call safer against accidental overrun, and also against accidental overrun 5 years from now when an intern modifies the code as part of his or her code-familiarization project.
This being said, I can also imagine people doing things like this:
void dostuff(char *src, char *dest)
{
memcpy_s(dest, sizeof dest, src, strlen(src));
}
Yay! (That example could be worse, but could overrun dest for short buffers.)
Secondly, the memcpy_s() function, like the rest of the _s functions, can invoke the "invalid parameter handler". This is a way to channel all your security violations into the same handler, which is rather nice in some ways. Instead of doing error checking all over (or failing to do error checking all over), you can have these things caught for you. IMHO, this feature is much more important than what you get from passing in the destSize.
Here's my untested guess at the guts of the function (sans invalid parameter handler support):
errno_t memcpy_s(void *dest, size_t numberOfElements, const void *src, size_t count)
{
if (count > numberOfElements) return errno = EINVAL;
memcpy(dest, src, count);
return 0;
}
I'm a little annoyed that "dest" is not returned like it is with memcpy(), but they probably don't want you to use it without testing it.
One final thing that is irksome is that if this is supposed to be some kind of robust call, why does it still invoke undefined behavior for overlapping regions? Why not detect the overlap and return an error--or automagically switch to memmove_s()? Or just make memcpy_s() an alias for memmove_s()?
I think I would assert that memcpy_s() will reduce the number of security defects. But I also think some percentage of the time it will be used incorrectly, and I don't know what that percentage is.
Stop using Visual Studio and go use a real toolchain.
From the MSDN blog:
If nothing else, memcpy_s makes you think about the size of the target buffer. . . Of course, you can easily make a call to memcpy_s() insecure by getting the buffer sizes wrong.
I don't know a lot about C, but this seems a bit absurd to me. If a C programmer gets the target buffer size wrong with memcpy(), they're going to start being less sloppy with memcpy_s()?
Geeks like to think that they can ignore politics, you can leave politics alone, but politics won't leave you alone.-rms
Good software doesn't force arbitrary limitations on the user. The user should only be limited by their hardware. So, if the algorithm requires reading the entire data set in at once, and the user has 8 GB of data, that program will use that much memory. In 10 years, 8 GB of memory will probably be considered small. If the program mistakenly enforced a memory limitation, it would have to be updated as hardware gets updated.
The change is no big deal, but I have to admit that as a C coder, I find the whole "need" to remove my ability to screw up my own programs... well... I dunno... It sorta robs the danger from the whole experience... How dare they civilize my C? :)
http://www.beanleafpress.com
see memcpy used incorrectly in Windows by typing
calc /inp1111111111111111 /outp1111111111111111
or
calc
from 'Run' box or cmd prompt
Fault found via a brute force prg I wrote. FWIW Brute force hacking trumps hacking by knowledge in that it is both thorough and complete.
Neil
... dont use microsoft products!
As Bruce Schneier would say, this is simply Security Theater. As others have pointed out, making ones code "compliant" is trivial, and worthless! As a professional, and published, software engineer with almost 30 years experience in the development of large-scale, high-availability distributed systems, I think that this is absurd in the extreme, and it takes focus away from techniques that can truly improve system reliability and security.
Sometimes, real fast is almost as good as real-time.
How hard is it to automatically translate C code to, say, C++/CLI?
It's certainly doable for C and I can't see any reason why it wouldn't be for C++ too (though writing a correct C++ compiler targetting anything is a PITA afaict)
http://cluecc.sourceforge.net/
One thing I do find interesting is their benchmark results. translating C to java using thier tool (which is probablly not optimal) brings a pretty substantial performance penalty but the penalty for converting to java is far lower than the penalty for converting to any of the other managed languages they support.
note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
That's a lower level function which is not normally used. Since std::vector is widely available, the need to use memcpy in C++ is minimal.
*((char *) mempcpy (dst, src, n)) = '\0';
Trying to fool-proof the C makes us neglect the more important task of C-proofing the fools. (Thanks jcr)
As others have pointed out, this is isn't about buffer overruns, as memcpy() takes a len argument. It's also not about safety. Type-safety makes sense, since memcpy() takes void pointers. But that makes much more sense in C# and (at a huge stretch) C++, than in C.
I'm wondering if this has more to do with limiting programmers from doing interesting things like accessing raw memory addresses, bypassing DRM, etc.
If a developer can't do "if (sizetocopy = sizeofdstbuffer)"
Uh oh, we'd better ban the = operator too, so no one can mistake it for == in an if statement ever again.
Everyone is claiming that developers will just turn this:
memcpy(dst, src, size)
into this:
memcpy_s(dst, src, size, size);
but for a large amount of MS code, this usage of memcpy_s is not advised and not common, and will be flagged in code review. So where does the size of the destination buffer come from if it is not simply a copy of the size of the src buffer? --> Often times the destination buffer, and indeed also the size of the destination buffer is passed in to your code by other code. In this case you simply and blindly pass the destination buffer and the size of the destination buffer into memcpy_s. In this case, allocation and size of the src buffer is something that your code determines, but the allocation and size of the dest buffer is not. You could say: well, should take into account the size of the destination buffer while allocating the source buffer -- to make sure the src buffer's size does not exceed that of the dst buffer you have been given. However, this is harder to audit. What memcpy_s does is, it joins more of the critical information about src and dst buffer sizes into a single line of auditable code.
IMO this change doesn't necessarily it harder to write bad code, it just makes it easier to audit for and find bad code.
GLib has the functions g_sprintf, g_strdup, etc. which take care of allocating their own memory and are UTF-8 compliant. (You can replace the allocator if you wish.) The whole "size of buffer" problem does not exist when using GLib.
Those who would give up liberty to obtain working drivers, deserve neither liberty nor working drivers.
#define memcpy( dst, src, len ) memcpy_s( dst, len, src, len ) /* Fix't */
You know, when you have a logically contiguous sequence of data items of a given type that are physically discontinuous and represented in their discontinuous form by an array of pointers to there representative parts and lengths?
At some point you need a physically contiguous representation of them. Short of magical virtual memory hackery (which might work if the objects are of just the right size, and all aligned on just the right boundaries), you need a method to copy an specified block of contiguous objects from one location to some offset from another location.
Not as type- and memory-unsafe as memcpy, but similar enough to get one into trouble, especially if the output size is not known at compile-time (which it most certainly won't be).
A bounded memcpy, where the presumed extents of the sources and targets helps (and, IIRC is what MSFT now demands), but ultimately within it, a bald memcpy will be necessary for efficiency's sake.
But, that can't be written.
This is similar to the problem with "smart" pointers to reference-counted objects, that are destroyed and their memory reclaimed when the last reference to them disappears. At some implementation level, real, bald, pointers will have to be dealt with.
This right solution here is to constrain where the tricky code lives and verify it more carefully than the "protected" code.
In Liberty, Rene
Their security assessments are troubling. memcpy() is generally pretty safe; you specify the number of bytes you are writing very explicitly.
This is completely unlike strcat() or strcpy(). Memcpy is worlds apart from functions that blindly copy to a NUL, without requiring a length to be specified, I would agree that is indeed dangerous.
Also, for inexplicable reasons, Strncpy(), Strncat(), and Snprintf() are on the list of banned functions. Although using these as safer alternatives to strcpy and friends IS a security best practice.
Instead they recommend proprietary functions they have created, which are not part of the C standard.
In other words, this is about forcing developers to use THEIR compiler and C libraries that provide these PROPRIETARY NON-STANDARD, NON-INTEROPERABLE functions... in the name of security.
I applaud their efforts to improve security, but they as a compiler vendor should propose and STANDARDIZE the new functions, before introducing them to the C library and producing them as part of a "standard" C compiler.
Until they do so, this is just another effort to lock developers in and expand and perpetuate their platform monopoly.
You want to implement "Microsoft's best security practices?" Sure... and Develop multi-platform software utilizing the GCC C compiler or G++?
Oh... SORRY, you can't do that.
Nobody seems to have mentioned the other problem with memcpy(), which is that it only works on untyped data. This is always dangerous and rarely required.
> "That's definitely one of those notoriously dangerous C commands," said Johannes Ullrich, CTO of the SANS Institute, who teaches secure coding classes to developers. He likened memcpy() to other risky functions such as strcpy() and strcat(), which Microsoft has already banned after exacting untold misery over the years.
These sound like the comments of a self-declared teacher rather than an actual programmer. I'm a C++ programmer and use memcpy() when I need to. If you're a decent programmer, it's not a problem. Bad programmers can write bad code regardless of what you give them. They can also write code that doesn't work, regardless of what tools they have. What's he going to do about that? What does he propose as an alternative? He sound re-read "Why Pascal isn't my favorite language", which addresses exactly this point. What else is Ullrich & co. going to "ban"? Pointers? C++ is used for game/system/real-time programming. You can't "ban" memcpy. Tell this guy to get a life and stop shilling his "institute", and someone send him "C++ for Dummies" too.
If this is done like their other *_s functions, then for arrays, you don't have to add an extra parameter. They provide templated versions that automatically know the size of the array, since it's known as compile time.
Passing the extra parameter is only required when the size isn't known at compile time - in which cases, your laughable "MIN(sizeof(dst), bytes_to_copy)" would fail anyway.
I hate to be the bearer of bad news to Microsoft (they're a great company after all...and helped computers move forward a bit), but the microsoft standard library is not the only way to copy memory. They can deprecate or disable calls to memcpy, but there are plenty of libraries out there that implement memcpy in ASM. There is no mandate in C programming that you use what comes with the standard c library.
This is what managed languages are here to solve, so we can all write safer code without having to think too much about it.
www.rexguo.com - Technologist + Designer
Any decent C programmer knows how to implement standard library functions. It's a pity that memcpy is at most a 3-liner.
void *
memcpy(void *s1, const void *s2, size_t n)
{
int i;
for (i = 0; i n; ++i)
s1[i] = s2[i];
}
it would have been much cooler to think about limiting where memcpy can copy to
This is an evidence that Microsoft lacks the most basic understanding what secure programming practices are.
If you don't know why, please avoid touching a compiler for the rest of your life.
Contrary to the popular belief, there indeed is no God.
I guess Microsoft is trying to get people that don't understand C++ to program better in the language. IMHO, this will not solve fundamental problems with people programming C++, just cause them to learn the language slower by not having an experience working through a bastard of a bug. I think the world (and not just MS) needs to realize that writing a piece of complex software is difficult. Bugs like an errant memcpy of a subclass into a baseclass instance are *very* easy bugs to solve if you're looking for them. Memory overwrites are easy to detect if you are looking for them. Bad pointers are easy to detect if you're looking for them.
Such problems will always exist in software. I've found a fair number in my own code. What we *really* need to do is train a better caliber of programmer. OTOH, Microsoft seems hell-bent on trying to make writing software easy to do
It's normal for a compiler to substitute 1 to 5 assembly instructions in place of a library call, especially if the number of bytes is known to be small. It runs faster.
I'm thinking memcpy_s is a library call though. That makes it slower.
Safeguarding potentially dangerous functions is what makes incompetent programmers, and limits the language beacuse of people who cant' use it well.
(I have 20+ years proffesional programming experience, a good chunk of it in C/C++, and in recent years have spent a lot of time debugging other peoples code, especially ported, re-ported and generally hacked around code !)
IMHO a lot of the posts here are missing the point when they say, "these functions are safe, you just need to check the size before hand" etc. The fact is, in the Real World, not everyone is an ace-programmer, deadlines loom, mistakes get made. Problems do not appear in testing, but can further down the line. IMHO a language simply should *not allow* the programmer to do crazy things. Potential buffer overflows, de-referencing null pointers etc should not get past the *compiler*.
To be honest I do not have a prefered language to suggest, but I do not think that tweaking C is the answer, but better, high level, languages are needed.
I've seen much discussion that with enough RAM and a smart kernel/fs, file access will be cached in RAM. Any idea why the fs you were using did not, but tmpfs did? Could it have been due to flush()? You managed a big win, so I am wondering is tmpfs just a bigger risk if ones system should go down?
I note they've also banned such wildly dangerous functions as strlen. What's even more interesting to me is that _INLINE ASM_ is perfectly acceptable under their guidelines, but not memcpy.
You have just made a macro which evaluates one of its arguments twice. Very bad idea...
Imagine someone writes:
memcpy (ptr1, ptr2, size++);
Demons will fly out of the user's nose.
The AACS key is NOT 0xF606EEFD628B1CA427BEA93A9CA9773F
Slashdot entertains. Windows pays the mortgage.
memcpy(flamewar, slashdot, all_your_base);
It is just like them to take something that is a standard and modify it in such a way that transportable software will not compile. They will not be happy until they get us all using their single source language technology and software will run only on Windows and impossible/impractical to port. They have been moving this way for years, what's new.
Hey, you do realize that these days there's never an excuse to waste precious developer time on something as mundane as performance improvements, don't you? The TCO of the $250 worth of RAM is just a fraction of the gazzilion dollars it must have cost to have you automate the copying of the file to tmpfs. This is simply unacceptable: your boss should have to learn to live with dog-slow software just like everyone else!
I'm amazed at how persistently Microsoft doesn't get it.
Look, here's how it works. The operating system protects processes from interfering with each other. This has been industry practice since the 1960's. The same concept extends to protecting users from each other and system resources from unprivileged access. The principle is called privilege containment.
I appreciate that Microsoft has not, historically, seen fit to consider privilege containment when designing its operating systems. After all, it takes more programming effort, and a longer development cycle might cut into market share. Plus, as Bill Gates once explained to me, users didn't specifically ask for it. But on the other hand, it's very hard to build security into an insecure system, especially one whose functioning still relies on crazy stuff like executable content.
So blame the victim. Encourage vendors to develop executable content, and encourage users to click on it. Make it everyone else's fault that the system doesn't itself provide the necessary protections. And just to underscore the abdication of responsibility, pop up lots and lots of confirmation windows so that there is no (legal) question that it was the user who made the choice. Remember, guns don't kill people, which is why it's okay to leave them firearms around ready to be discharged.
In a saner, more conventional world, what processes do within their own address space is their own business. To think otherwise is to misunderstand the Halting Problem. We can certainly develop programming environments which do their own memory management, and that's great. Before Java there was Lisp. It's not a new idea. And it's a very useful idea, as far as it goes.
Maybe we should ban C, is that the new idea? Who needs memcpy except someone who needs to do their own memory management? Indeed, someone who needs the freedom to program at a lower level than Java or Lisp, perhaps someone who is implementing an environment like Java or Lisp, extending PHP or Tcl/Tk, writing a protocol layer or a device driver or an embedded system, or doing any kind of fundamental operating system development.
So rather than providing reasonable privilege containment so that those people can get on with their work, we'll just forbid the practice entirely. Or maybe we should just have lots and lots more popups? "Are you SURE that you want to use memcpy here?"
Parity: What to do when the weekend comes.
Why do we care what Microsoft dictates to it's followers? As programmers we have the choice to program what and how we want.