Microsoft To Banish Memcpy()

kyriacos notes that Microsoft will be adding memcpy() to its list of function calls banned under its secure development lifecycle. This reader asks, "I was wondering how advanced C/C++ programmers view this move. Do you find this having a negative impact on the flexibility of the language, and do you think it will restrict the creativity of the programmer?"

Comments

No - there are plenty of safer alternatives

[MerlynEmrys67]

Just like removing printf, scanf, and most other copy/string functions. There are safe versions of memcpy that work just fine and are just as easy to use...
Lame story (Trying for flamebait here?)

Re:No - there are plenty of safer alternatives

[Anonymous Coward]

And they aren't even removed, but (by default) a warning is issued when using them. I'd say it's a good move - passing the size of the destination buffer is usually not that complicated.

Re:No - there are plenty of safer alternatives

[Anonymous Coward]

safe versions - if you prefer to blindly program away, not worrying about where your objects end up in memory. But - what is "safe"? Is there any replacement for properly testing all I/O from all possible sources?

Re:No - there are plenty of safer alternatives

[Anonymous Coward]

Internally to Microsoft, "banned" means that no products can be shipped using these functions. Externally, this is just a recommendation.

Re:No - there are plenty of safer alternatives

[Anonymous Coward]

I'd say it's a good move - passing the size of the destination buffer is usually not that complicated.

Are you high? It already takes a size argument. If this were about strcpy(3), then you'd have a point, but I do not think memcpy(3) means what you think it means.

I'm not saying you can't get yourself into trouble with inappropriate use of memcpy(3), but buffer overruns aren't the go-to threat every time.

NAME
memcpy - copy memory area

SYNOPSIS
#include <string.h>

void *memcpy(void *dest, const void *src, size_t n);

DESCRIPTION
The memcpy() function copies n bytes from memory area src to memory area dest. The memory
areas should not overlap. Use memmove(3) if the memory areas do overlap.

Re:No - there are plenty of safer alternatives

[Chris Burke]

Just like removing printf, scanf, and most other copy/string functions. There are safe versions of memcpy that work just fine and are just as easy to use...

There's nothing unsafe about printf (since compilers started doing format type checking), as long as you don't use user input as the format string. To print user input, you use printf("%s", user_input).

strcpy() is unsafe because you don't know how many bytes you are going to be copying. strncpy() is completely safe as long as you aren't brain dead and set the 'n' to the size of the destination buffer (as opposed to strlen(src) which would be brain dead) and then slap an '\0' into the last index of the dest. sprintf, same deal, just use snprintf and tell it the max bytes it can print.

So what's unsafe about memcpy()? You explicitly specify the number of bytes to copy. If that number of bytes is greater than the known size of the destination buffer, then you've got a problem that simply adding a second 'size of dest' paramater to the copy won't fix because you already screwed the pooch on figuring that out now didn't you?

Yes memcpy() doesn't work if src and dest overlap. When that's happening, you typically know about it (you've got some clever in-situ array modification going on) and can use memmove(). memmove(), on the other hand, is equally unsafe if you can't properly specify the number of bytes to copy.

Bottom line: There's no such thing as a "safe" copy in C when we're assuming the programmer can't figure out the destination buffer size.

Re:No - there are plenty of safer alternatives

[Anonymous Coward]

It's a psychological thing. Having a separate parameter for the size of the destination buffer forces the programmer to think about what that size is. Too often programmers call memcpy passing the size of the data that needs copying and forget to check that the destination is big enough. And that's why we see so many buffer overflows.

If you never make this mistake continue to use memcpy. I don't care and neither does Microsoft.

Re:No - there are plenty of safer alternatives

[FlyingBishop]

That's physically impossible, even given infinite time. Read up on the halting problem.

However, programming a framework in which we may rule out certain things, for example a process jumping over and altering the OS, is perfectly possible. It just has to be verified through reasoning, rather than testing. The unit testing methodology is really the problem here. You cannot unit test everything.

Don't get me wrong, testing is a good start, but it's no proof of security, and a proof of security, while very hard, is possible. Kudos to Microsoft.

And to expand on the GP for those that didn't RTFA, they replaced Memcpy with a memcpy that forced you to state the size of the destination buffer, which is a constant time operation, and a much needed one. So this only forces C coders to make their code a little more clear.

And when you're being intentionally unclear to the computer in addition to the reader, your code has no place in a secure production setting.

Re:No - there are plenty of safer alternatives

[uniqueUser]

In the same way that having a seat belt and an air bag in my car, doesn't change my driving habits

I consider myself a safe driver. Seat belts and air bags don't really affect my driving habits either (as far as I know). But what If there was a 7 inch (~18cm) spike sticking out of your steering wheel and 100lbs (~45kg) of high explosives attached to both bumpers, would your driving habits change? Me? Maybe. Sometimes a little danger can make things a little safer. I once heard of a study that showed some poorly painted and poorly lit roads had an increased accident rate when lighting was added and the yellow lines were re-painted. Apparently people began speeding b/c of the perceived increase in safety. I don't remember the source. Maybe Uncle Google can help me later, but I have to get back to work now.

Re:No - there are plenty of safer alternatives

[e9th]

strncpy() always bothered me. If len is too short, dst winds up unterminated, and if too long, as you say, you have a bunch of extra nulls at the end. What was the point of that? "Oh, we'll null fill it in case strlen() et al. miss the first one?"
It would have been nicer if it returned the number of non-null bytes copied, so you could do a quick compare with len to check for the unterminated case.

Re:No - there are plenty of safer alternatives

[Duhavid]

It still will not help.

If they are a sloppy enough programmer not to look at what is going on, and to ensure the size of the destination, they will be sloppy enough to use the same dratted variable in both spots, drool all over the keyboard and move on to the next sloppy bit of code.

Re:No - there are plenty of safer alternatives

[dhavleak]

Right -- but a static analysis tool can catch precisely that behavior at compile time.

Re:No - there are plenty of safer alternatives

[buchner.johannes]

I'm not saying you can't get yourself into trouble with inappropriate use of memcpy(3), but buffer overruns aren't the go-to threat every time.

Didn't we already defeat the goto threat?

More to the point, if the developer doesn't know what memcpy does and how to use it correctly ... I mean ...
You might aswell write the 3 lines of code behind memcpy yourself.

Re:No - there are plenty of safer alternatives

[makomk]

If you're a competent programmer then nothing is unsafe, but obviously there are a lot of stupid programmers out there who make fundamental mistakes fucking with memory when they don't understand what they're doing. What Microsoft is trying to do here is to eliminate a low hanging fruit of software security that has led to hundreds if not thousands of buffer overflow conditions and associated vulnerabilities/exploits.

The trouble is, it doesn't. Banning functions like strcpy made sense, because they were nearly always unsafe to use. On the other hand, if you're memcpying too much data for the destination, there's probably something more fundamentally wrong with your code. This, at best, conceals the bug by truncating the copy - leading to unpredictable issues later in execution instead.

Re:No - there are plenty of safer alternatives

[dannannan]

Technically one size argument is enough, but in a large enough software project the code that allocates the destination buffer is maintained separately from the code that copies into it. Any failure in communication (e.g. building against an outdated library) will lead to someone's linker writing a binary with code that will overrun a buffer.

With an explicit destination size parameter, the buffer copy code is no longer as sensitive to changes at the allocation site. A breakdown in communication will lead to a binary that produces a controlled runtime error instead of a buffer overrun.

Re:No - there are plenty of safer alternatives

[George Reilly]

There's nothing unsafe about printf (since compilers started doing format type checking), as long as you don't use user input as the format string. To print user input, you use printf("%s", user_input).

%n writes to the stack. It's disabled by default in VS2005 onwards. More at http://weblogs.asp.net/george_v_reilly/archive/2007/02/06/printf-n.aspx and http://julianor.tripod.com/bc/formatstring-1.2.pdf

Re:No - there are plenty of safer alternatives

[Ciaran Power]

That's physically impossible, even given infinite time. Read up on the halting problem.

No it's not. A computer is not a Turing machine - it has finite memory (=> finite # of states), an algorithm has to halt or visit a state it's already been in.

And to expand on the GP for those that didn't RTFA, they replaced Memcpy with a memcpy that forced you to state the size of the destination buffer, which is a constant time operation, and a much needed one. So this only forces C coders to make their code a little more clear.

Fair enough, well done MS. But their new memcpy can be lied to (memcpy_s(dst, 9999, src, 40)) and guys who aren't keeping track of (and checking) their remaining destination size are the guys likely to lie to memcpy_s

Re:No - there are plenty of safer alternatives

[Chris Burke]

What Microsoft is trying to do here is to eliminate a low hanging fruit of software security that has led to hundreds if not thousands of buffer overflow conditions and associated vulnerabilities/exploits.

They might be trying, but they are failing, because the mistake that leads to the error in the first place (miscalculating destination buffer size) has the same effect (buffer overrun) whether you use memcpy() or memcpy_s().

Re:No - there are plenty of safer alternatives

[Opportunist]

So copying with the destination buffer size specified makes it safe? How so?

If I knew that the value I want to copy won't fit into the destination buffer, I wouldn't copy it there. Simple as that. Oh, because the size of the source might be variable? Then maybe the coder should do a size_ssize_d?size_s:size_d as the size argument. Taking a variable that isn't under the coder's full control as a size in any memory manipulation operation is asking for trouble anyway.

Do you think it will be safer now that you "force" the programmer to specify the maximum destination size? If people are sloppy enough to not check sizes, they'll simply pass the same parameter twice and the security you add is zero.

Re:No - there are plenty of safer alternatives

[ZerothAngel]

Have a look at strlcpy. It's non-standard, sure, having originated in OpenBSD. But it can now be found in the libc of all the *BSDs, Mac OS X, and even Solaris.

It guarantees the destination is always nul-terminated and it makes it easy to check if your destination buffer was short.

Re:No - there are plenty of safer alternatives

[DamnStupidElf]

So why is strncpy in the banned function list?

I think this is just Microsoft trying to embrace and extend. There's no better way to do that then making most existing C and C++ code invalid. The quickest alternative, of course, is to write it in C# or some other embraced language.

Hypocritically, Microsoft did NOT add memset to the banned list despite it having almost exactly the same problems as memcpy. Why? Almost every MSDN example begins with "memset(somestruct,0,sizeof(somestruct))" and invalidating every MSDN example would probably look bad.

As you pointed out, the size of the destination buffer makes no sense when dealing with pure pointers. Often memcpy is used to move memory around inside larger buffers, which completely invalidates memcpy_s as a safe replacement. memcpy is also often used to copy smaller buffers into larger ones, and accidentally copying the uninitialized (or carefully crafted by some exploit) data that comes after the source object can be just as dangerous. The correct replacement, memcpy_overkill(void *source_object, size_t source_size, size_t source_offet, void *dest_object, size_t dest_size, size_t dest_offset, size_t count) is what they're REALLY looking for, but this is impractical primarily because of the heavy use of context-less pointers (to objects within arrays, or within some other structure; the void * in memcpy's prototype hints at further possibilities) in C and C++.

Re:No - there are plenty of safer alternatives

[TemporalBeing]

memcpy_s's buffer size argument should be used for the actual size of the destination buffer. It shouldn't be variable based on user input / program flow, and can likely be determined at compile time.

Obviously you have never made a parser of any kind. Any time you read a file in, or use a data stream (cin, cout, cerr, etc.), and many more situations (printf, aprintf, aprintf, etc. not to mention document editors, web browsers, etc) you need to be able to have a dynamically sized buffer to at least manage the data between states.

Now, if you can always guarantee that you'll read a file with the same exact format every single time you run the application, then yes, you can determine it at compile time.

However, most applications are more like web browsers in that respect - their input changes every time they run; the format changes. (Think of typing in your post - is there any way the computer could have predicted how much text you'd type? Or the server that it would receive that amount of text from you? Or...the list goes on.)

Please turn in your nerd badge as the XmlHttpRequest hits you while you close the page...

Re:No - there are plenty of safer alternatives

[TheLink]

> Drivers are going to try to drive safer with a steel spike in their face instead of a friendly pillow

When it comes to programming languages, that approach just means either lots more dead or broken code, or a lot less code AND a lot less good code.

There's a higher percentage of C programs where "an attacker can execute arbitrary code of the attacker's choice", compared to say Java or Python programs. Just a look at Bugtraq over the years.

I'm half joking but there might be fewer than 10 people in the world who can write secure, reliable AND useful C programs (useful to more than just one person ;) ). I'm not one of them.

Re:No - there are plenty of safer alternatives

[James Skarzinskas]

In an effort to "one-up" Microsoft, Apple promises to replace their own memcpy() with one that not only does not require a size for the destination buffer, but does not require a destination buffer at all. While Apple programmers call the move "totally pointless" and "absolute proof of functional retardation", Steve Jobs has simply responded, sagely, that the future of Apple development is through so-called "intuitive APIs". It just works.

Re:No - there are plenty of safer alternatives

[strangepics]

Also take a look at the excellent stralloc which is also part of libowfat.

Re:No - there are plenty of safer alternatives

[parlancex]

I know you were kidding, but I'd like to point out that the internal implementation of memcpy on many platforms will be much faster than the equivilent C using a loop for large copies, including x86/64 due to the use of architecture specific instructions designed to facilitate the operation that most compilers probably don't use even on the highest optimization levels.

Re:No - there are plenty of safer alternatives

[jd]

Whilst you are correct, if Microsoft is going to essentially replace the standard C library with one that has an incompatible API, why not just call it a new library and have done with it?

Or, better yet, if security really was the goal, develop a C-like language that was secure by design?

By simply making things awkward for people to write portable code, all they do is ensure that there are multiple code bases for projects (which increases the opportunity for error) or ensures that people won't write portably. Which is a more likely goal, given who we are talking about.

Re:No - there are plenty of safer alternatives

[drew]

I understand the problem you are describing, but I fail to see how this solution addresses it. If there is already a disconnect between the programmer doing the copying and the programmer doing the allocating, then making the programmer doing the copying repeat himself is not going to fix the problem.

The only problem this function solves is buffer over flows caused by a programmer calculating a number of bytes to copy at runtime (e.g. by reading it from a Content-Length header) and failing to check the calculated value against what he believes is the actual size of the buffer. If the value that he believes to be the size of the buffer is wrong, changing from memcpy to memcpy_s will not catch the mistake. In other words, changing from memcpy to memcpy_s will only protect against sloppy programmers, and if they don't understand what the function is supposed to be protecting them from (which is likely) they'll probably just use the same value for copy_size and dst_size anyway (or switch to memmove), which will completely defeat the purpose of blacklisting memcpy in the first place.

Not to mention, if you're doing any pointer arithmetic and writing to an offset some number of bytes past *buffer, then passing the size of *buffer doesn't really help, unless the function is smart enough to know that (I don't see how it could be unless we pass that as a parameter as well), or the user is smart enough to calculate the remaining size of *buffer. If the user is one of the sloppy programmers that this function is meant to protect against in the first place, I think that is highly unlikely, don't you?

Re:No - there are plenty of safer alternatives

[niteice]

Or, better yet, if security really was the goal, develop a C-like language that was secure by design?

And then why don't you make it compile to non-native code, so you can do code analysis at runtime? Might as well give it a good standard library that uses all the features so people would try writing stuff. Of course, you can't name it C then, maybe you should give it a catchier name with some punctuation or other pun on the language.

Hey, wait a minute...

Re:No - there are plenty of safer alternatives

[mustafap]

>Or, better yet, if security really was the goal, develop a C-like language that was secure by design?

Or, better yet, if security really was the goal, use Ada.

There, fixed that for you :o)

Re:No - there are plenty of safer alternatives

[bored]

So why is strncpy in the banned function list?

Because strncpy() is as bad as strcpy(). The problem lies in the fact that if the source string is longer than the destination len, then strncpy simply stops the copy without writing a NULL. The next str* function used on the string is likely to crash.

Re:No - there are plenty of safer alternatives

[chkn0]

Or the server that it would receive that amount of text from you?

Actually, yes.

Re:No - there are plenty of safer alternatives

[setagllib]

Who do you think has to write those high level wrappers? memcpy is one of the most ridiculously popular functions in systems level C/C++ code, especially for copying arrays or sub-arrays, where it can be much faster than a hand-written loop. You can wrap it in a function for every type you need, but that's still a lot of memcpy you have to write properly. Fortunately it's easy and this whole argument is moot.

Re:No - there are plenty of safer alternatives

[TapeCutter]

"In the few cases where someone really needs to use memcpy instead of relying on a library"

First of all, memcpy IS a libary call.

"but they should have to explain their need and the benefit over using a higher level wrapper to lots and lots of people."

One source tree, many O/S's. Memcpy is a ANSI C library call, I have been using it for more than 20yrs without a problem. IF MS want to pop up a warning that tells me my source will compile on gcc I can't stop them from doing so.

Re:No - there are plenty of safer alternatives

[jd]

Perhaps that was a little bit ambiguous of me. What I was referring to were programming languages which reduce the possibility of error (eg: ADA) and/or which are designed to enforce good programming practice and rigorous standards (eg: Occam).

I consider these to be "secure by design" because they were designed to make the more common security flaws impossible and were also designed to make it possible to validate the software. (Both, if I understand the histories correctly, were linked to military efforts to produce highly robust, highly secure code.)

Should have been done 30 years ago.

[Animats]

This should have been done thirty years ago.

Re:Should have been done 30 years ago.

[james_shoemaker]

Why? I can see some justification on the strXXX functions where you don't know how many bytes are going to be copied unless you call strlen first, but in memcpy you pass how many bytes to copy in as a parameter. So this is to protect programmers who can't do math?

Re:Should have been done 30 years ago.

[darkwing_bmf]

If you haven't tried Ada yet, I highly suggest looking into it. It keeps track of data sizes, types, etc... for the programmer but it will also let you get close to the hardware like C does. It's often used for safety critical software such as that used in aviation.

Unfortunately I can't recommend using Ada to develop windows apps. It's technically possible but you end up importing C library functions to do it. And if you're going to do that, you might as well just use a native development environment that is better suited to the task.

malloc() and free()

[Anonymous Coward]

Those are also dangerous functions. And also array indexing! That should also be eliminated.

Re:malloc() and free()

[plague3106]

Well there's a reason Java and C# have GCs to free you from having to worry about calling malloc and free..

Python is done

[sunami]

Figures, Microsoft had to go kill of python and do it all in the name of security. No more accessing MEMory in C structures from our .PY files, damn it this really pisses me off.

Re:Python is done

[Rycross]

No its not. This is only banned under Microsoft's Security Development Lifecycle, which means you only care about this if you're following those set of development guidelines. Its still in the language. And you can always use memcopy_s:

Developers who want to be SDL compliant will instead have to replace memcpy() functions with memcpy_s, a newer command that takes an additional parameter delineating the size of the destination buffer.

No mention of memmove...

[pthisis]

Do you find this having a negative impact on the flexibility of the language, and do you think it will restrict the creativity of the programmer?"

You can replace memcpy entirely with memmove (the latter is slightly slower and handles overlaps), and nothing in the article suggests that memmove is banned.

But, no, it shouldn't hurt creativity--they're introducing a memcpy_s, which is the same aside from taking a size parameter for the destination. That's something that is generally easy to track in new code (obviously this secure developement lifecycle is not backwards compatible).

Re:No mention of memmove...

[ifdef]

Okay, I'm obviously missing something here. How is having an extra parameter for the destination size any safer? I always thought the third parameter to memcpy was the amount of data to copy, and since obviously it should never be set to anything larger than the size of the destination, how will having the destination size explicitly passed in help any?

Or are we just talking about a convenience feature that will make it easier for lazy programmers?

Re:No mention of memmove...

[ksheff]

big deal. Now developers will write

memcpy_s(dst, sizeof(dst), src, sizeof(dst));

instead of

memcpy(dst, src, sizeof(dst));

If they've been screwing up and using the wrong size for the number of bytes to copy, what's going to stop them from screwing up and putting the wrong size for the size of the destination buffer? Nothing! Now the coders that have been using something like

MIN(sizeof(dst), bytes_to_copy)

for the last parameter for years will have to change their code. Oh well...it's job security for someone.

Re:No mention of memmove...

[pthisis]

Okay, I'm obviously missing something here. How is having an extra parameter for the destination size any safer? I always thought the third parameter to memcpy was the amount of data to copy, and since obviously it should never be set to anything larger than the size of the destination, how will having the destination size explicitly passed in help any?

That's the error that this is trying to fix. I'm skeptical as to how much this will help; if you're that lazy, you can just set the destination size parameter to the same value as the amount to copy.

But it might be easier to enforce at a code-review level in the organization: destination size always has to be a size tracked based on memory allocation.

Re:No mention of memmove...

[pthisis]

Now developers will write

        memcpy_s(dst, sizeof(dst), src, sizeof(dst));

I get the feeling that this is mainly for Microsoft internally developed code which conforms to their security guidelines. As such, it's probably mainly intended to help in code reviews. Still pretty dubious.

Now the coders that have been using something like

        MIN(sizeof(dst), bytes_to_copy)

for the last parameter for years will have to change their code.

That fails in the common case of dst being a real pointer (whether it's indexing into a static array or dynamically allocated memory or whatever).

Re:No mention of memmove...

[Tony Hoyle]

Given that dst is a pointer, sizeof(dst) is generally going to be 4 or 8, and not do what you want.

It's more likely that programmers will just pass len to both parameters, defeating the point. Unless you define a pointer type that contains a length attribute (which wouldn't be a bad idea, but MS haven't done that) you're just relying on lengths being passed around the code being accurate, which isn't any safer.

A bad programmer will always be a bad programmer. Someone who would use strcpy on user data or memcpy not knowing the destination length, is the same kind of person that's going to work around this. For everyone else it's just a pain.

Re:No mention of memmove...

[Tony Hoyle]

memcpy_s(dst, sizeof(dst), src, sizeof(dst));

Whilst that will work, it probably doesn't do what you think.

Hint: dst and src are pointers.

They should go one better...

[Smidge207]

...and pop up a message box asking the user to confirm they want to copy the memory, and if they press OK then they should have to enter a captcha.

Seriously though, how is it supposed to make your code safer if you pass the size you think your destination buffer is? With memcpy, that size is implicitly greater or equal to the copy size and it's the caller's responsibility to make sure this is the case. Putting bounds checking into the copy function is ridiculous if you're responsible for passing the bounds yourself, and it goes against basic good design. I'm surprised they aren't passing the source buffer size too, just to be extra safe. Also, what happened to the __restrict keyword? It's strangely absent from the memcpy_s function declaration.

=Smidge=

Re:They should go one better...

[Creepy]

The problem is memcpy returns a void *. If this is dynamically cast, it needs to be checked at runtime and may even be set to a value the programmer never intended (say unsigned 16 bit values instead of unsigned 8 bit characters). It may be an issue with updating the code - say the code was originally written for 8 bit ASCII and got updated to, say UTF-16 (16 bit). A dynamically cast void* doesn't care what the size is, it just shoves the values in the buffer. This may work fine in basic testing even, because you never overflow the buffer with 1-2 characters, and maybe even gets past a QA team, but once you go past 1/2, you've got a buffer overrun.

As I understand it, __restrict wouldn't work in a C++ program using dynamic_cast because it doesn't know the size at compile time (sorry, I'm not sure what is done in C as I haven't kept up with the language, so I have to use a C++ example). My guess is memcpy_s does runtime bounds checking (it isn't specified on the memcpy_s page, maybe the security ref - too busy to read it though).

Re:They should go one better...

[Prof.Phreak]

Eh? the 'n' in memcpy call is number of -bytes-. not "things" you're trying to copy. it doesn't matter if you give it an array of signed 8 bit characters and copy it over to 32bit unsigned longs... you just specify n to be number of -bytes- to copy.

How can this be confusing?

First they take my gets..

[adonoman]

First they came for gets, then they took scanf and strcpy, now they want memcpy? Outrageous! How are virus writers going to be able to take advantage of buffer overflows if I'm continuously keeping track of how big my buffers are? I may have to start lying about their size just to give hackers a chance.

Re:First they take my gets..

[Anonymous Coward]

First they came for gets, And I didn't speak up because I didn't use gets
Then they came for scanf, And I didn't speak up because I didn't use scanf
Then they came for strcpy, And I didn't speak up because I didn't use strcpy
And then... they came for memcpy... And by that time there was no one left to speak up.

What an idiotic idea.

[Anonymous Coward]

Someone already explained this better than I could.

Re:What an idiotic idea.

[residieu]

taking the min is probably not what you want to do. If you don't have room in the destination for everything you want to copy, you need to find a new destination. Copying just part of the source memory is going to lead to strange behavior down the road (when you start interpretting the copied memory and it cuts off suddenly). So this just changes checking the sizes before the memcpy call and handling the error there into check the error code from memcpy_s, and handling the error afterwards.

Re:What an idiotic idea.

[iluvcapra]

The more important step is that it will encourage programmers to actually attempt to track both size1 and size2.

I think you misunderstand his point... the 'size' parameter isn't the number of bytes in either buffer, it's the number of bytes you want to move. Obviously this has a lot to do with the size of either allocated buffer, but it's not the same thing.

memcpy doesn't know what a buffer is-- no, it really doesn't. At it's heart, all it does is copy a byte from one pointer to another and increment both, until it's done 'size' of them. There's no requirement that the pointers passed to memcpy be pointers to arrays. They just have to be pointers.

The idea that 'size' actually corresponds to the number of allocated elements in an array is a leaky abstraction. Obviously you never want to copy more bytes than either buffer has the capacity for, but that's really not the language's responsibility to enforce, because the language has no first-class concept of a "buffer" of known length. By making the function 'safer,' all they're doing is encouraging the leaky abstraction and confusing people who think memcpy's job is to copy "buffers" when all it really does is copy n bytes. These developers who have written memcpy_s may have made it "safer," but at the expense of giving it another error condition that must be tested for, and making the actual workings of the function more obscure.

Let's say, If I only want to copy the first 3 bytes of src to dest, do I put in "3" for the source length or the destination length? And I'd better test that against the length of both first, because if the function fails, was it because there's only 2 bytes available in dest, or because there's only 2 available in src? OR, if I'm starting at byte 2 of src to dest, do I put "1" in for srcLength, because that's alll that's left in the array, or "3," because that's how big the array is? I know it probably should be the first one, but the people who don't know better, the people this function is theoretically trying to engineer against, probably will make the wrong choice, because they have it in their head the array is length "3" and this function wants to know the length of the array, right?

#define memcpy memmove

[the_arrow]

I have often used memcpy instead of strcpy when I have known the length of the strings, and also known the destination to be large enough.
I'm guessing many developers will just #define memcpy to something else and continue as nothing happened.

then why are you using C?

[Anonymous Coward]

If you consider memcpy too dangerous then you should be using something besides C. If you're using C++ and memcpy then you really do need to know what both you and the compiler are doing.

The difference bewteen memcpy() and strcpy()

[Anonymous Coward]

The problem with strcpy() and sprintf() and like functions is that you don't know when calling them the length of the source to be copied into the supplied buffer. But with memcpy() you specify this length.

Frequently, the size of the target is calculated at run time, so bugs in memcpy() tend to be in the area of this calculation, rather than in not checking if the source fits the target.

Any lack of memcpy() would be easy to overcome, just use

          memcpy_s (dst, len, src, len)

which is functionally identical to

          memcpy (dst, src, len)

Re:Isn't security the programmer's responsibility?

[Anonymous Coward]

you didnt read.

MSFT is banning it from their development process, not the language, use it as much as you like.

the worst offender is main()

[JeanBaptiste]

Most any security problem can be traced back to this function.

Re:the worst offender is main()

[Khashishi]

you mean WinMain()

"memmove()" is safer than "memcpy()".

[reporter]

Though "memcpy()" is slightly faster than "memmove()", the latter is safer.

As Windows products are now (and have been) mainstream products used extensively in banks and other financial institutions, reliability and security (RS) have prime importance. The speed that "memcpy()" gets you is not worth the price of reduced RS.

Re:A half-measure, at best...

[wampus]

So, Ben... or is it Peter? Do you always copy your comments verbatim from the linked article, or only when you agree with them?

Silly and useless

[ugen]

This is nothing like sprintf. In sprintf there is no way to know how much data will be created ahead of time, so limit on buffer size is useful to make sure there is no buffer overrun.

With memcpy it is *precisely* known how much data will be copied. It is right there, 3rd parameter. If a developer can't do "if (sizetocopy = sizeofdstbuffer)", it is just as unlikely that he will be able to properly state that additional parameter that specifies the destination buffer size.

Of course if Microsoft is so concerned with security, why the heck did it take them years to add snptinf()? All this is is another attempt to make crossplatform development that much harder (much like all those "obsolete" POSIX functions that will barf warnings unless you use a cryptic define).

That said, if this silliness ever becomes a rule, I have an easy solution:
#define memcpy(dst, src, size) memcpy_s((dst), (src), (size), (size))

Problemo solved, now let's go actually write some real code.

Re:Silly and useless

[harry666t]

> if (sizetocopy = sizeofdstbuffer)

ouch.

When will MS learn?

[coppro]

This is not the first time MS has done this. They have plenty of other standard functions that they have deprecated.

Yes, you read that right. Microsoft is deprecating parts of an ISO Standard all by themselves. Not that this should surprise anyone. I would have absolutely no objection to them proposing to WG14 to deprecate those functions; heck, I'd encourage it! But besides going out and deciding to 'deprecate' parts of the standards, the replacement functions actually violate those same standards.

And the warnings are irritating. You can't write a nice cross-platform library without either spewing tons of warnings or having to put in a bunch of #defines to shut the compiler up. And if you do that, your users get irritated if they depend on these warnings because you just turned them off (and of course, if you don't, they'll complain that your library is unsafe).

Screw Microsoft.

Re:When will MS learn?

[theCoder]

In case anyone is curious, this is the type of thing that coppro is talking about:

    c:\Program Files\Microsoft Visual Studio 8\VC\include\io.h(318) : see declaration of 'close'
    Message: 'The POSIX name for this item is deprecated. Instead, use the ISO C++ conformant name: _close. See online help for details.'

Now, as far as I know, no ISO body has deprecated functions like close(2), open(2), read(2), and write(2). And I've always heard that methods that start with an underscore are internal compiler functions and shouldn't be called directly. I don't know why the MS compiler writers think they can do this, but it is really annoying to get hundreds of warnings like this when compiling. In addition, it hides legitimate warnings that could indicate real problems.

As to the article in question, I can't think of any good reason why memcpy(3C) would be considered unsafe, since it specifies the amount of memory to copy. Sure, you could use it to copy outside the bounds of dst, but that's just calling it incorrectly. It's not like sprintf(3C) where you could easily accidentally write outside the bounds of the string.

Re:When will MS learn?

[hackerjoe]

no ISO body has deprecated functions like close(2), open(2), read(2), and write(2)

That's correct, because ISO C++ never included those functions in the first place. POSIX != ISO C. (Not that MSVC is on any kind of reasonable schedule for keeping up with ISO standards, but that's a whole different issue...)

Basically MS is deprecating their own terrible implementation of some POSIX compatibility. This is actually required for ISO C compliance: the compiler is not supposed to define a bunch of extraneous functions in the global namespace, because they might conflict with your names. Once those functions are removed entirely (and I believe you can #define them away right now) you can implement your own compatibility functions for software you're porting to Windows.

Now, this is all entirely separate from the SDL warnings GP is complaining about, which show up when you use standard ISO C functions like strcpy, sprintf, and apparently now memcpy. Which, honestly, I wish weren't quite so irritatingly implemented, although I'm torn because using those functions really is terrible.

It's not really that worth getting up in arms about, though, because JESUS CHRIST there's a compiler flag to disable the warnings, just put it in your makefile and quit bitching already!

Re:When will MS learn?

[RightSaidFred99]

Wrong, wrong, wrong.

open, close, etc.. are not in ISO C++. Functions not in the spec (vendor-specific) are supposed to have an _ in front.

Any function you can think of (including strcpy) is "safe" if the developer specifies correct parameters. Whoopdy Do. That doesn't mean it's something that's easily verified by runtime checkers, static analysis, etc... That's why it's deprecated - it's easy to perform additional safety checks if you include both the source and destination sizes.

Re:When will MS learn?

[shutdown -p now]

This is not the first time MS has done this. They have plenty of other standard functions that they have deprecated.

Yes, you read that right. Microsoft is deprecating parts of an ISO Standard all by themselves.

No, Microsoft isn't deprecating "parts of an ISO Standard" - only the standard committee can do that, by marking those parts as deprecated in the next version of the standard. Microsoft has enabled warnings on use of those "unsafe" functions by default, yes, but it is very much not the same thing.

Regarding "all by themselves" part - do you realize that all those "safe" *_s functions are actually covered by an ISO C99 TR?. There's also a FOSS implementation available under the MIT license.

And the warnings are irritating. You can't write a nice cross-platform library without either spewing tons of warnings or having to put in a bunch of #defines to shut the compiler up.

You don't have to use #defines for that purpose, you use compiler flags in your makefile. You'll have to write one specifically for MSVC anyway (since it uses its own Make), so it's not a big deal.

And if you do that, your users get irritated if they depend on these warnings because you just turned them off

That doesn't make sense. If you turn them off for the code of your library, they're obviously not turned off for code of your users - unless you put #defines in your headers, which is obviously a dumb thing to do for many reasons (and I've already explained the proper way to do this above).

Re:Not for C++

[JSBiff]

If = is ambiguous, then you must have a habit of abusing it. While you can overload = to mean anything you want, I suppose, it would seem like you should try to preserve the general notion of the assignment operator in C and C++, which is that = never modifies the right-hand side of the equal sign, only the left hand side.

"Does it mean duplicate the contents, transfer the contents and clear the original copy, or just swap the contents of the items, which might be quicker."

I would say if you are trying to stay true to the way the operator behaves in the language when you overload it, the last two operations would NOT be overloaded to the equal sign, because they modify the right-hand argument. Give them a meaningful method name instead.

Workaround

[pensivepuppy]

#define memcpy(s1, s2, n) memcpy_s((s1), (n), (s2), (n))

Stop protecting me from me!

[BitZtream]

As a competent developer, I get extremely annoyed by this sort of shit.

Removing/banning memcpy doesn't change a damn thing cause the first thing I do with things that have to compile in VisualStudio now is add the following defines which turn this shit off:

_CRT_SECURE_NO_WARNINGS
_CRT_NONSTDC_NO_DEPRECATE

If the remove that option I'll simply add memcpy to my standard MS compatibility library that deals with all the other bullshit MS decides to do.

You can't fix stupid. Stop trying. People fuck up VB and C# apps just as much as the fuck up C and C++ apps. So they don't do it with a buffer overflow, they do it by shear stupidity. You'll be more secure by taking away languages that allow non-programmers to pretend to be programmers than making it harder on those of us that are just going to work around what you do anyway.

You're not going to fix broken shitty apps with exploits by removing functions, the functions aren't the problem they do exactly as they are told (or atleast they are supposed to :). You need to fix the programmers who can't clarify what they want done.

http://www.xkcd.com/568/
Second pane:
You'll never find a programming language that frees you from the burden of clarifying your ideas.

Re:Not for C++

[sqlrob]

= never modifies the right-hand side of the equal sign, only the left hand side.

std::auto_ptr would like a word with you. This was one of the dumbest decisions the committee made.

easy fix

[Chris Snook]

Just write a one-liner that replaces all calls to memcpy with a call to memcpy_s, duplicating the size parameter.

I'm only half-joking. This is exactly how people will (mis)use memcpy_s. If you want safe memory access, you need to ban the entire C language. For those cases where you need C, you'll just have to make sure your programmers know what they're doing.

How to easily ...

[DemoLiter3]

How to easily make your code compliant with the new safety requirements:

#define memcpy(dest,src,len) memcpy_s(dest,len,src,len)

Re:How to easily ...

[jdoverholt]

Not quite "just as bad," in my opinion. Writing outside of your boundary is much more likely to cause problems (overwriting other things) than reading out of bounds. A 40-byte null-terminated string, for instance, wouldn't be hurt by another 40 bytes of heap data, so long as the null terminator was intact. You may still throw an error if it's not your memory that's being read. Just saying... writing (approx. equals) 100% trouble, reading < writing.

Re:How to easily ...

[cool_story_bro]

He didn't say "how to make your code safe." He said "how to make your code comply with the safety standards." Rarely are the two the same. It's perfectly possible to safely use memcpy(), just like it's perfectly possible to abuse about a billion other system calls.

Re:Think about it

[TrekkieGod]

Some of these reactions are quite funny.

The goal of asking you to specify the length of the destination buffer is to force you to think about the data you're working with *while* you're writing the code and not afterwards in an unconnected security audit. Furthermore, it provides documentation to other people reading the code who may not have the same mental model of what's going on as you do. And as usual "other people" includes you, six months after you wrote the code.

Uh-huh. Because everyone is not just going to add a line of code to one of the base headers:

#define memcpy(dst, len, src) memcpy_s((dst),(len),(src),(len))

C is not a safe language, and stupid programmers will always find ways to mess it up. There are safer languages where you can't hang yourself as easily, and if you don't understand C, you should use them. Microsoft can't fix this problem.

The wole thing is just a bunch of nonsense

[LanceUppercut]

Firstly, the specification of C anf C++ standard library is governed by the corresponding standard commitee. Microsoft has absolutely no authority to "banish" anything from neither C nor C++. They can deprecate it in their .NET code, C# etc., but it has absolutely no relevance to C and C++ languages. So, why would the author of the original question direct it to "advanced C and C++" programmers is beyond me. In general, C and C++ programmers will never know about this "interesting" development.

Secondly, the tryly unsafe and useless functions in the C standard library are the functions like "gets", which offer absolutely no protection agains buffer overflow, regardless of how careful the develoiper is. Functions like 'memcpy', on the other hand, offer sufficient protection to a qualified developer. There's absolutely no sentiment against these functions in C/C++ community and there is absolutely no possiblity of these functions to get deprecated as long as C language exists.

Shooting themselves in the foot.

[Dimwit]

Now, all that's going to happen is that programmers are going to write their own memcpy-like routines using a quicky for-loop or something. It'll be just as bug prone, and harder to detect via automated source code analysis.

Re:Shooting themselves in the foot.

[GleeBot]

Now, all that's going to happen is that programmers are going to write their own memcpy-like routines using a quicky for-loop or something. It'll be just as bug prone, and harder to detect via automated source code analysis.

Not to mention it'll be slower. memcpy is one of the most optimized functions in any C library. It's frequently handled as a compiler intrinsic, that can do stuff like unroll short copies, generate optimal machine code, etc.

Re:Shooting themselves in the foot.

[Just Some Guy]

Actually, memcpy in and of itself is slow. Hand writing your own asm version of memcpy using extended cpu functions is a lot faster as memcpy itself is usually kept basic enough to work on any cpu, including the older cpu's without MMX, SSE, etc.

glibc contains specific implementations for sparc32, powerpc32, powerpc64, i386, i586, cris, i860, rs6000, and m68k. I don't know where you got your idea.

Re:Think about it

[TrekkieGod]

The idea that smart people don't make mistakes is thoroughly ridiculous.

They make mistakes, they don't make that mistake.

Smart people recognize that they make mistakes, so they create systems that help them catch and prevent their own mistakes. If you're foolish enough to believe that you can't make mistakes, then you should just turn off all the warnings on the compiler and not bother with lesser workarounds like redefining a single symbol.

That's not the issue. The new function is still relying on the programmer not making a mistake. What makes you think that anyone who would make a mistake on the number of bytes to copy wouldn't make a mistake on the size of the buffer? Alternatively, what happens if the src buffer size isn't large enough? Do we need to add another length parameter here?

I have nothing against making a language safer, but if the language can't ensure the size of the buffers itself (and it can't in this case), relying on the programmer to pass in the correct number is exactly as safe as relying on the programmer to do the bounds checking. Exactly as safe, no safer.

Re:Let Java do it for you.

[samweber]

Until you're trying to do something for which Java has no standard API. Last time I checked, USB joysticks were like this. There is JInput, but if you include JInput in your distribution, your project is no longer 100% Pure Java and will not run as an applet.

But your C/C++ code is never able to be run as an applet, while Java code might. So, on these grounds Java is better.

Or unless your target platform is incapable of running managed code. Some handheld platforms have only 4 MB of RAM, not enough for a JIT compiler and the bytecode in addition to the translated code except in trivial examples that act a generation old.

And for that we have Java ME edition.

Okay, there are devices that are even more tiny in which C or assembly is the only option. I've also dealt with a processor that couldn't be programmed in C without extensive non-standard C extensions. Either way, this is a small minority of situations -- there is no one solution that suits all purposes.

And per Apple's developer agreement, the only managed language that can run on an iPod Touch is JavaScript in Safari.

And they allow just any old C code to run?!!

Or unless your existing program's model is written in an unmanaged language, and you want to reuse the old code so that you can be sure that the model is bit-for-bit accurate. How hard is it to automatically translate C code to, say, C++/CLI?

Eeek! Don't do this!

If you want bit-by-bit accuracy, you absolutely don't want to try translating C code to C++! You even have to worry about changing C compilers if you are doing that. (And yes, I have worked with C code which would break if we used a different compiler. It was not fun.)

What you should be thinking of doing is to write an interface between the existing model code and your new one. This can be done in both C++ and Java. When the new code has to manipulate the model, it goes through this interface and calls the old code to do the necessary operation.

Re:Let Java do it for you.

[tepples]

And for [platforms with small CPU and small RAM] we have Java ME edition.

That is, if your platform vendor offers a port of Java ME.

If you want bit-by-bit accuracy, you absolutely don't want to try translating C code to C++! You even have to worry about changing C compilers if you are doing that.

I already test my code on GCC targeting x86 and ARM. Or did you mean compilers that a hobbyist looking to start a business can't easily afford?

What you should be thinking of doing is to write an interface between the existing model code and your new one. This can be done in both C++ and Java. When the new code has to manipulate the model, it goes through this interface and calls the old code to do the necessary operation.

Mostly I was thinking about trying to port an existing video game to a phone that runs Java ME MIDP or a game console that runs XNA, while preserving frame-for-frame accuracy of the physics. You can't run the old model on such hardware because you don't have the certificate to digitally sign native code. Or did your "interface" include communication over a network to a server running the model?

Lock in from Microsoft

[wandazulu]

There have been several suggestions to replace memcpy with memcpy_s as the safer alternative. That's fine, I guess, if memcpy_s is part of the ANSI/ISO standard for C, which as far as I know, it is not; just like all the *_s functions.

Microsoft says your code is safer when using the *_s, but it will no longer be portable, it'll be Microsoft-only. They put in a warning in the compiler from VS2005 onwards about using "unsafe" functions, and that you should use *_s, which is a pain because you have to disable it as the project level, there doesn't seem to be anywhere that I've found that can just turn it off permanently. Even using the STL that comes with VS2008 will generate these warnings, even if you never do any explicit memory stuff yourself.

Microsoft did the same thing with the _* functions; a lot of them are just wrappers around their ANSI-compliant versions (_sprintf -> sprintf), but are also not portable; I worked with a guy who wrote/tested all his C code in VS6 then gave it to me to port to Unix and VMS, and the compilers would choke on not having these particular functions.

Microsoft is trying to get lock-in at the language level instead of providing a good set of Win32 API-based functions that make using memcpy() unnecessary.

The goto threat == Raptors

[beathach]

Foolish mammal, they cannot be defeated so easily. http://xkcd.com/292/

Re:Let Java do it for you.

[Just Some Guy]

If you wrote a program that used 8+ gigs of memory that means you're an incompetent code monkey.

I have an hourly job that processes about 8GB of input data files. We found that copying the data to a tmpfs filesystem instead of leaving it on a HDD cut the work time from 20 minutes to about 30 seconds because the job necessarily requires an enormous number of random seek()s. Since mmap()ing a file on tmpfs is roughly identical to read()ing the whole file into RAM, I guess that makes me an incompetent code monkey.

Of course, my boss who got a 40x speedup in exchange for $250 worth of RAM might see things differently from your inexperienced little self. Sometimes the Real World throws pretty big datasets at you.

Ban = operator also

[Safiire Arrowny]

If a developer can't do "if (sizetocopy = sizeofdstbuffer)"

Uh oh, we'd better ban the = operator too, so no one can mistake it for == in an if statement ever again.

Re:Good.

[JustNiz]

Yes and yes. I've been a developer for over 30 years now(nearly all C or C++). How about you?

Mixed views...

[jschmerge]

I'm a C++ developer, and I'm mixed on this decision... On one hand, memcpy is a function that you can really hurt yourself with. On the other, it maps to extremely fast assembly that most processors can perform very quickly. There is a time & a place for everything. I think that a memcpy inside a class' constructor or assignment functions is perfectly acceptable, yet doing a memcpy(&destclass, &fromclass, sizeof(destclass)) is fundamentally dumb for more reasons than I care to illucidate (read one of Stroudstrup's or Meyer's books on C++ if you want to know). Doing something like that *really* demonstrates that you don't know the language.

I guess Microsoft is trying to get people that don't understand C++ to program better in the language. IMHO, this will not solve fundamental problems with people programming C++, just cause them to learn the language slower by not having an experience working through a bastard of a bug. I think the world (and not just MS) needs to realize that writing a piece of complex software is difficult. Bugs like an errant memcpy of a subclass into a baseclass instance are *very* easy bugs to solve if you're looking for them. Memory overwrites are easy to detect if you are looking for them. Bad pointers are easy to detect if you're looking for them.

Such problems will always exist in software. I've found a fair number in my own code. What we *really* need to do is train a better caliber of programmer. OTOH, Microsoft seems hell-bent on trying to make writing software easy to do

Please ban them

[Zoxed]

(I have 20+ years proffesional programming experience, a good chunk of it in C/C++, and in recent years have spent a lot of time debugging other peoples code, especially ported, re-ported and generally hacked around code !)

IMHO a lot of the posts here are missing the point when they say, "these functions are safe, you just need to check the size before hand" etc. The fact is, in the Real World, not everyone is an ace-programmer, deadlines loom, mistakes get made. Problems do not appear in testing, but can further down the line. IMHO a language simply should *not allow* the programmer to do crazy things. Potential buffer overflows, de-referencing null pointers etc should not get past the *compiler*.

To be honest I do not have a prefered language to suggest, but I do not think that tweaking C is the answer, but better, high level, languages are needed.

Re:Please ban them

[khuber]

Thank you for a sensible post. I agree that the only real solution has to be some abstraction or alternate mode for memory access that prevents the problem outright. C and C++ not only make it possible, but extremely easy to trash memory.

The people claiming that the problem is programmers who are incompetent or stupid and believing that they are superhuman genius programmers who are somehow immune from errors are living under a delusion. You are not a beautiful and unique snowflake. We are all human and imperfect. I have seen many intelligent and competent programmers write incorrect code. I have personally written code with errors. We need to move beyond unproductive blame assigning mentalities to achieve real solutions or we will never be able to advance the state of software. Realistically, the prima donnas will continue on and other people will actually do something.

memcpy_s is not a step forward for numerous reasons already mentioned. strncpy provides functionality that strcpy does not, namely a bound on its region of effect. memcpy was already bounded and neither memcpy_s or strncpy can prevent bad parameters from causing memory corruption. Hardware support like the NX/ED bits to mark pages as not executable are a good advancement on the security front, but still do not prevent data corruption. Complete removal of memory access like many languages do solves the problem with a high performance cost. Emerging techniques like STM may be able to incorporate data safety while maintaining hardware efficiency.