Old-School Slashdotter Discovers and Solves Longstanding Flaw In Basic Calculus (mindmatters.ai)

Longtime Slashdot reader johnnyb (Jonathan Bartlett) shares the findings of a new study he, along with co-author Asatur Zh. Khurshudyan, published this week in the journal DCDIS-A: Recently a longstanding flaw in elementary calculus was found and corrected. The "second derivative" has a notation that has confused many students. It turns out that part of the confusion is because the notation is wrong. Note -- I am the subject of the article. Mind Matters provides the technical details: "[T]he second derivative of y with respect to x has traditionally had the notation 'd2 y/dx 2.' While this notation is expressed as a fraction, the problem is that it doesn't actually work as a fraction. The problem is well-known but it has been generally assumed that there is no way to express the second derivative in fraction form. It has been thought that differentials (the fundamental 'dy' and 'dx' that calculus works with) were not actual values and therefore they aren't actually in ratio with each other. Because of these underlying assumptions, the fact that you could not treat the second derivative as a fraction was not thought to be an anomaly. However, it turns out that, with minor modifications to the notation, the terms of the second derivative (and higher derivatives) can indeed be manipulated as an algebraic fraction. The revised notation for the second derivative is '(d 2 y/dx 2) - (dy/dx)(d 2 x/dx 2).'"

The report adds that while mathematicians haven't been getting wrong answers, "correcting the notation enables mathematicians to work with fewer special-case formulas and also to develop a more intuitive understanding of the nature of differentials."

Comments

Summary's accuracy seems questionable

[JoshuaZ]

There's no "flaw" in calculus. They've proposed a notation which if one used it would allow a broader range of formal manipulations to be valid. This is interesting but it isn't groundbreaking.

Re:Summary's accuracy seems questionable

[sexconker]

It's well known to most students who took a calculus course and then took the next level calculus course.
Suddenly your d and dx start doing weird shit, and they can't do other shit you've been told to do with them, then you find yourself questioning WTF the d or the x actually mean, and then you wonder why you never thought d/dx was just 1/x, and you realize your teacher / professor can't explain it either.

If you don't grok the fundamental theorem of calculus, you'll never grok d/dx and the bullshit that has been done with them. If you do grok it, you learn to ignore the bullshit and separate those symbols from the rest of the algebraic manipulations they're mixed up in.

Re:Summary's accuracy seems questionable

[johnnyb]

It's a bit of both. Some of the facts of the matter were known, but it was assumed that this was just "the way it was". That is, no one considered it an open problem. For instance, we view the inability to divide by zero just a fact of mathematics, not a flaw. Likewise, this was not known to be a flaw, it was just assumed that this was the way things worked.

If you need to point to a definitive flaw, it was in our understanding of how it was supposed to work - the relationship between our understanding and the notation. Once *that* flaw was discovered, the actual notation just spilled right out. That is, the flaw was that people were *not* treating dy/dx *sufficiently* as a fraction, due to 19th century preferences against infinitesimals. Once you realize that dy/dx really is a fraction, and has to be treated accordingly, everything automatically works.

It's almost humorous because there was no real advanced work to do. Literally everything needed is available in intro calculus. The problem was (a) the mathematics community had a habit of *not* treating dy/dx as a fraction, and (b) new students who didn't know better were simply taught *what* to do, not *why* to do it, and continued to repeat the mistake for over a century.

Congrats

[yodleboy]

Figured I'd better say congratulations before the inevitable flood of people shitting on your contribution to math gets up to speed.

Re:Congrats

[johnnyb]

Thanks! I appreciate it. Given that this was my first peer-reviewed mathematics paper, I had no idea how long the process was. I submitted the paper over a *year* ago. The necessary changes were minor. But the actual time it took to go through the process was excruciating. I'm happy to finally be on the other side :)

Re:Congrats

[johnnyb]

My coauthor has been doing this to good effect. His book "Controllability of Dynamic Systems: The Green's Function Approach" utilizes it. My role in mathematics is primarily in teaching high schoolers, so I don't spend a lot of time with differential equations. That's also the reason I *have* a co-author. I needed someone to tell me I wasn't crazy :)

Ugh

[sexconker]

The d, dx, dy, etc. are not things to be generally operated on.
Writing the second derivative as d^2 / dx^2, or worse, d^2y / dx^2 is doubly absurd. (I'm using the ^ to denote supersripting, not exponentiation.)

d represents the instantaneous rate of change (which itself is a flawed concept - a rate of change cannot be instantaneous as a rate depends on the passage of time), dx represents that instantaneous rate of change of x. d/dx represents the instantaneous rate of some value, possibly some value dependent on x, with respect to the instantaneous rate of change of x. dy/dx, dv/dt, etc. are all the same deal. That rate of change of some variables with respect to other variables.

What is that instantaneous rate of change? The slope of a line (plane, or whatever if you've got more free variables) tangent to your function at a given point, presuming such a thing exists.

How do you determine that tangent line? You take the target point and some point h past it ((f(x) vs f(x+h)) (or before it!) and determine what the line does when you consider h approaching 0. You make sure you can define that shit from both ends and both ends agree. If that works out, have a limit, you've got a derivative, and baby, you've got the fundamental theorem of calculus goin'.

Whoever tried to slap that shit together as a fraction or take shortcuts and try to manipulate those symbols in a way that looks sort of like algebraic manipulation is a clown. Trying to fix that is going to be an uphill battle, but using more of the busted notation isn't really the solution.

Re:Ugh

[BKX]

dy/dx doesn't represent instantaneous rate of change. That would be nonsense. The d in dx and dy means "small difference that will eventually go to zero". This is why dy/dx is a fraction. It represents the limit of a small change in y divided by small change in x, as the changes go to zero. This is why we teach students about the limit definition of the derivative as being what the derivative really is. As far dy and dx being tricks of notation, they're really not. They really are small changes. There's no instantaneous rate of change. dy and dx are always finite real numbers. They never actually become zero. dy/dx is the ratio that is approached as they get smaller and smaller.

As far as this guy's new version of the second derivative, I call bullshit. I seriously doubt that this is correct. And the notation d^2y/dx^2 actually makes sense when you think about. It's really just d(dy/dx)/dx, that is, a small change in dy/dx divided by a small change in x, where dy/dx is a small change in y divided by a small change in x. Writing it in the other way is just a good way of doing it. If you draw out what this means graphically, is becomes clear that it's really a small change between two consecutive small changes in y divided by two small changes in x, that is d(dy)/dx^2, hence d^2y/dx^2.

This guy's new version, on the other hand, doesn't make sense at all. I mean, how do you get that from taking the derivative of the first derivative. Let's take a pretty standard function: x=1/2*t^2+2*t+12. x'=t+2; x''=1, whereas his version would be x''=1-t, which doesn't make any sense, unless he has completely redefined everything. I mean, d^2y/dx^2 would have to be something like 2t+5 and d^2x/dx^2 would have to be something like 2, and then we get x''=2t+5-(t+2)*2. I didn't read the paper so I don't know what it would actually be, but there's no doubt that x''=1, so if his method is to make any sense at all it would have to give the same results in the end. I just don't see how it could.

Re:And in a sane curriculum

[TeknoHog]

The messed up notation by Newton is not used and instead the much saner stuff from Riemann is used. Newton was smart, but a hack and a crank. And he tried to suppress Riemann notation. Mathematics would probably have done better without Newton.

Surely you mean Leibniz (1646-1716), not Riemann (1826-1866).

this is actually useful

[epine]

It took me a few minutes to get to the nub of the matter.

If you're mentally reading the notation d^2 y / dx^2 as the second derivative of y divided by dx squared, you're doing it wrong.

Because what this notion really intends to mean is d(d(y)/dx)/dx, which as the paper points out is a different order of operation.

A more compact notation less misleading than the traditional d^2 y / dx^2 might be (d/dx)^2 dy, which expands via two repeated function applications to d(d(y)/dx)/dx, with the underlying operations now in the right order.

Calculus was never my best thing, so I might be all wet, but it seems to make sense.

I never liked the dx/dy notation much, regarding it more as a cryptic code than anything conceptually helpful (when its not cryptic, it's not helpful, because that's the common case you already know).

With the right lambda notation (riffing on what I proposed above) the fundamental operator nature of d() could be correctly expressed, even if you don't want into these algebraic manipulations, which mostly strike me as far too detailed and tedious.

Well thank

[ArchieBunker]

god that's settled. Now we can figure out that P=NP problem that nobody can give a coherent answer on why its even a thing.

Re:Seems quite a lot larger...

[johnnyb]

This is my thought as well. Interestingly, I developed this while writing a book (Calculus from the Ground Up) to use for my homeschool co-op calculus classes. I was trying to find a good way to explain the notation, and I literally had 20 calculus books that I read through trying to find a good explanation for the standard notation in any of them. None of them even attempted an explanation, just "this is the way it is, but don't treat it as a fraction." So, I tried to deduce the notation myself. That's when I realized that it was not just limited, it was actually wrong. So I wrote the paper and finished the book (it's Appendix B in the book).

Linear regression stumper

[Tablizer]

I have a "math issue" that has stumped most of my professors and online math forums. Linear regression typically uses the "least squares" algorithm. However, the power of 2 seems arbitrary to me, and possibly over-emphasizes outliers.

One professor at first said that the power of 2 makes the "best fit" in an objective sense, but later admitted that he doesn't really know, and couldn't find an answer before the end of the semester.

While it is true that the power of 2 may simplify the computation process*, that doesn't necessarily means it produces a better result in terms of line or curve fitting. Now that we have computers to do the number crunching, perhaps it's time to embrace arbitrary or different powers (superscripts).

(Disclaimer, I'm not a math expert.)

* In other words, power-of-2 produces the simplest known algorithm. But my question revolves around best data fit, not computational resources nor algorithm or formula brevity. Note that when using other powers, one may have to add an absolute value function because power-of-2 automatically provides the equivalent. I actually did a simulation that tested different powers; "blurring" known datasets and seeing which power best matched the original. I couldn't find any significant difference, but probably didn't try enough samples. I tested with fractional powers also, such as 1.5, 2.5, etc.

Re:Linear regression stumper

[Mendenhall]

It's not arbitrary. There's actually a good reason for minimizing (y-yobs)^2, assuming that your observations have a Gaussian distribution. The resulting estimators provide a maximum likelihood estimator of the parameters of the distribution, if and only if it really was Gaussian. Thus, of course, if it isn't Gaussian (outliers of various sorts, et.c), the x^2 may not be the best bet. There is an entire field of 'robust estimators' of quantities, which are more resistant to outliers than least squares. There are also cases in which the underlying distribution is pathologically different from Gaussian; it could be Lorentzian (Cauchy), in which case it is so completely unlike a Gaussian, it doesn't even have a defined standard deviation (it is infinite). There are weighted methods which can fix this too.

So, in short, least squares is the right answer (in the sense that it yields results which provable have the maximum likelihood describing the data at hand) if you have a perfect Gaussian variate; otherwise, it may well not be.

It's a complete nonsense.

[porky_pig_jr]

Leibniz' notation is normally treated as a "suggestive kind", never to be understood literally. The origin of notation d^2/dx^2 goes from applying d/dx to d/dx, but d/dx only means "a derivative w.r.t. x" and nothing else. Sometime taking this notation literally and doing manipulations as if it were the regular fractions work (and that's b.t.w. is attributed to the early discoveries of many differentiation and integration rules), but it doesn't work most of the time. Any decent book on Calculus should point out that fact. Working with fractions helps to discover some rules, yes, but it's never rigorous, it's more like discovering something in a heuristic way, but then you still need a rigorous proof and that involves going back to basic definition of limits, not arguing in terms of "infinitesimals" (yes, I'm aware of Robinson's "non-standard calculus", but IMHO it's not a mainstream approach. Cheers.

Newton and _Leibnitz_ both useful

[Roger W Moore]

The messed up notation by Newton is not used and instead the much saner stuff from Riemann is used.

The advantage of the Newtonian notation is that it is a lot faster and easier for, unsurprisingly, basic Newtonian mechanics where you only really differentiate with respect to time. This is why it is used extensively in this area of physics. Leibnitz's (not Riemann's!) notation is a lot more versatile which is not surprising: Leibnitz was a mathematician who was interested in the abstract concept whereas Newton was a physicist who only developed calculus so he could describe mechanics and so did not really need a broader, more flexible notation.

It is actually quite a common that fundamental physics can find itself ahead of maths. For example String theory today is really a joint venture between maths and physics since they are having to develop the maths needed to describe the physical models they work on.

Finally, Newton was neither a hack or a crank but he was a somewhat evil genius. He could be quite nasty and viscous, sometimes in extremely petty ways. For example he discredited Leibnitz and he fell out with Robert Hooke and had all contemporary portraits of him destroyed which so angered a modern artist that she spent the time an effort painting multiple portraits of Hooke from contemporary descriptions so that, today, there are more portraits of Hooke than Newton!

Old School

[fahrbot-bot]

Old-School Slashdotter Discovers and Solves Longstanding Flaw In Basic Calculus

Can occasionally be heard yelling at younger mathematicians: "Get off my lambda"

Re:Old School

[johnnyb]

You never did a second derivative test to determine whether you are at a local minima or maxima?

Most intro calculus books at least show the notation for the second derivative. However, it is true that they rarely take it far enough to hit any problems with the notation.

I actually figured this out while trying to find a good way to explain the notation to my students, which is a homeschool co-op class (I have a range of 9-12 graders - the 9th grader is an exception, but she is ridiculously smart). I read through numerous calculus textbooks trying to find the justification for the notation, and none of them even attempted it. So, I decided to try it out myself, and found out that the standard notation was wrong.

Standard form just as accurate

[Roger W Moore]

I appreciate the new form is technically more accurate

It's only technically more accurate if you read the standard form as a fraction. If you actually read the standard form as intended - a notation indicating the second derivative of y with respect to x - the standard notational form is just as accurate as is the Newtonian notation of dots to denote derivatives with respect to time.

Re:And in a sane curriculum

[sfcat]

The messed up notation by Newton is not used and instead the much saner stuff from Riemann is used. Newton was smart, but a hack and a crank. And he tried to suppress Riemann notation. Mathematics would probably have done better without Newton.

Riemann lived 2 centuries after Newton. And your conclusions aren't correct, they aren't even wrong!!!

Re:Seems quite a lot larger...

[Obfuscant]

because the old way ways "you can treat the derivative as a fraction,

Except the second derivative notion isn't a fraction. It's a way of writing "the second derivative of Y with respect to X" in a short form. Not all '/' create "fractions". Unless, of course, you want to argue that I'm putting a lot of "< divided by quote>" fractions in my /. postings.

The error is not in the notation, it's in the inability to overload the / operator when dealing with more complex and abstract mathematical concepts. It's like not being able to differentiate between "e as a variable" and "e as a constant". Do you ever think the the mass of an object times the speed of light squared is equal to ~2.718? Einstein says so, it must be true.

Re:Seems quite a lot larger...

[johnnyb]

Except that, in the first derivative, it *is* used as a fraction. Otherwise you couldn't reformulate your equation for integration (i.e., you have to multiply both sides by dx, which is treating it as a fraction). So, to say that in one case, it is a fraction, but this next case it isn't, but still written as a fraction, even though it *could* be written as a fraction, but we just decided not to, seems strange, at least to me.

Re:What about partial derivatives?

[johnnyb]

I've actually got a second paper on partial derivatives just about ready to go. It was originally part of this paper, but it got a little long, and I wanted to rethink and clarify a few concepts. Anyway, partial differentials have the same notational problem *plus* one more. The problem is that there are several partial differentials which all go by the same name. Once you name them properly (i.e., give them each a distinct name) the problems go away.

Re: Seems quite a lot larger...

[Anonymous Coward]

Yes, you do, to integrate dy/dx = x, you would multiply both sides by dx, cancelling out the denominator in dy/dx to form, dy = x dx. Throw both sides under an integral sign and go! You just memorized the rules with no understanding of why it worked...

Sigh.

[ledow]

Often in maths, a mere change of notation, analogous equation in another field, or just looking at things in a slightly different way will open up whole new areas of maths.

Fermat's Last Theorem took forever to prove and the proof relies on translating the problem to a completely unrelated area of maths, solving it there, and then translating the results back.

And if you do things like use polar coordinates, etc. some areas of maths burst open with good sense and nice equations.

Something as simple as a notation change can work wonders. But this is just for convenience of amateurs who don't understand what a derivative actually is and does. It's like saying "Don't use the word multiplication for vectors, because it's not the same as for scalars". We know. Anyone handling it knows. Anyone dumb enough to confuse the notations is going to find out very quickly that nothing works. Sure, it might help if you've literally never done those kinds of equations before, but likely then you'll not be making any ground-breaking mathematical discoveries any time soon.

Things don't tend to survive hundreds of years for no reason, especially when they are one pen-stroke away from being changed, and have themselves gone through several notational iterations in their time.

I got through a degree in maths without thinking "Well, this notation is stupid", including three years of advanced calculus.

If you don't understand the notation, that's the very least of your worries as regards actually doing any calculus.

Re: Seems quite a lot larger...

[johnnyb]

The problem with e-book math books is trying to make it look right on a small screen. If you just want a PDF of it, send me an email and I'll send you one, especially if you consider telling other people how great it is. Unfortunately, you can't just tell Amazon to take your PDF and make it an e-book :(