Approximation by linear functions, differentiability and derivative
Recall: linear spaces, linear functions…
A vector (or linear) space (מרחב וקטורי, sometimes we add explicitly, space over ) is a set equipped with two operations: addition/subtraction and multiplication by (real) numbers, . These operations obey all the natural rules. The simplest example is the real line itself: to “distinguish” it from the “usual” real numbers, we denote it by . The plane is the next simplest case.
A function defined on a vector space, is called linear, if it respects both operations, . The set of all linear functions on the given space is itself a linear space (called dual space, מרחב הדואלי, with the natural operations of addition and rescaling on the functions).
Linear functions on can be easily described.
Example. Let be a linear function. Denote by its value at 1: . Then for any other point , we have (meaning: vector = number vector in ), so by linearity .
Question. Prove that any linear function of two variables has the form , where and . Prove that the dual space to the plane is again the plane of vectors as above
Warning!! In the elementary geometry and algebra, a linear function is a function whose graph is a real line. Such functions have the general form , and are linear in the above sense only when . We will call such functions affine (פונקציות אפיניות). The coefficient will be called the slope (שיפוע) of the affine function.
We first look at the linear functions of one variable only and identify each function with its coefficient , called multiplicator (מכפיל): it acts on the real line by multiplication by . The product of two linear functions is non-linear, yet their composition is linear, does not depend on the order and the multiplicator can be easily computed as the product of the individual multiplicators:
Problem. Compute the composition of two affine functions and . Prove that the slope of the composition is the product of the slopes. Is it true that they also always commute? Find an affine function that commutes (in the sense of the composition) with any affine function .
Obviously, linear functions are continuous, bounded on any compact set. To know a linear function, it is enough to know its value at only one point (for affine functions, two points are sufficient).
Approximation by a linear function near the origin
Let be a (nonlinear) function. In some (“good”) cases, the graph of such function looks almost like a straight line under sufficiently large magnification.
Example. Consider : this function is obviously nonlinear, and , so that its graph passes through the point . Let be a small positive number. The transformation (change of variables) magnifies the small square to the square . After this magnification we see that the equation of the curve becomes . Clearly, as , the magnified curve converges (uniformly on ) to the graph of the linear function .
In other words, we see that
as , where is the linear approximation to the function . In particular, we can set and see that the limit exists and is equal to 2, the multiplicator of the linear function .
Example. Consider the function and treat it by the same magnification procedure. Will there be any limit behavior? Will the limit function be linear?
Approximation near an arbitrary point. Derivative
What if we want to find a linear approximation to the function at a point different from the origin, and without the assumption that ? One has to change first the coordinates to . In the new “hat” coordinates we can perform the same calculation and see that existence of a linear approximation for is equivalent to the existence of the limit as .
Definition. If is an interior point of the domain of a function and the limit
exists, then the function is called differentiable at and the value of the limit (denoted by ) is called the derivative of at this point. The function , defined where it is defined, is called the derivative (function) of .
Warning! Despite everything, the derivative is not a linear function (and even not affine!) The value is just the multiplicator (the principal coefficient) of the affine function which approximates near the point and depends on .
Notation. There are several notations for the derivative, used in different sources and on different occasions some are more convenient than others. They include (but not reduced to):
We will explain the origin (and convenience) of some of these notations in due time.
First rules of derivation. The “derivation map” (or the “differential operator” ) is linear: , and , assuming that are differentiable on the common interval. Indeed, the sum and the multiple of affine functions approximating the initial ones, are again affine.
The derivative of an affine function is constant and equal to , since this function ideally approximates itself at each point. In particular, derivative of a constant is identical zero.
Leibniz rule. The product of two affine functions is not affine anymore, yet admits easy approximation. At the origin the product of two affine functions and , is the quadratic function which is approximated by the affine function . Note that the four constants are the values of the initial functions and their derivatives at the origin, we can write the last formula as
This is called Leibnitz formula. To show that it is true for the product of any two differentiable functions, note that any such function can be written under the form , where is an affine function and is a small function such that . If is any bounded function, and is such small function, then the linear approximation to the product is identically zero (prove it!). Use the linearity of the approximation to complete the proof of the Leibniz formula for arbitrary differentiable .
Chain rule of differentiation. Let be two differentiable functions and their composition. Let be an arbitrary point and its -image. To compute the derivative of at , we replace both by their affine approximation at the points and respectively. The composition of the affine approximations is again an affine map (see above) and its slope is equal to the product of the slopes. Thus we obtain the result
An easy computation shows that adding the small nonlinear terms does not change the computation: the derivative of a composition is the product of the derivatives at the appropriate points.
Problem. Consider differentiable functions and their composition . Prove that , where .
In particular, for the pair of mutually inverse functions such that , the derivatives at the points and are reciprocal.
Example. Let , . Then by induction one can prove that . The inverse function has the derivative at the point . Substituting , we see that
This allows to prove that for all rational powers .