Differentiability and derivative
Continuity of functions (and maps) means that they can be nicely approximated by constant functions (maps) in a sufficiently small neighborhood of each point. Yet the constant maps (easy to understand as they are) are not the only “simple” maps.
Linear maps naturally live on vector spaces, sets equipped with a special structure. Recall that is algebraically a field: real numbers cane be added, subtracted between themselves and the ratio is well defined for .
Definition. A set is said to be a vector space (over ), if the operations of addition/subtraction and multiplication by constant are defined on it and obey the obvious rules of commutativity, associativity and distributivity. Some people prefer to call vector spaces linear spaces: the two terms are identical.
Warning. There is no “natural” multiplication !
- The field itself. If we want to stress that it is considered as a vector space, we write .
- The set of tuples is the Euclidean -space. For it can be identified with the “geometric” plane and space, using coordinates.
- The set of all polynomials of bounded degree with real coefficients.
- The set of all polynomials without any control over the degree.
- The set of all continuous functions on the segment .
Warning. The two last examples are special: the corresponding spaces are not finite-dimensional (we did not have time to discuss what is the dimension of a linear space in general…)
Let be two (different or identical) vector spaces and is a function (map) between them.
Definition. The map $f$ is linear, if it preserves the operations on vectors, i.e., .
Sometimes we will use the notation .
Obvious properties of linearity.
- (Note: the two zeros may lie in different spaces!)
- For any two given spaces the linear maps between them can be added and multiplied by constants in a natural way! If , then we define for any (define yourselves). The result will be again a linear map between the same spaces.
- If and , then the composition is well defined and again linear.
- Any linear map has the form (do you understand why the notations are used?)
- Any linear map has the form for some numbers . Argue that all such maps form a linear space isomorphic to back again.
- Explain how linear maps from to can be recorded using -matrices. How the composition of linear maps is related to the multiplication of matrices?
The first example shows that linear maps of to itself are “labeled” by real numbers (“multiplicators“). Composition of linear maps corresponds to multiplication of the corresponding multiplicators (whence the name). A linear 1-dim map is invertible if and only if the multiplicator is nonzero.
Corollary. Invertible linear maps constitute a commutative group (by composition) isomorphic to the multiplicative group .
Maps of the form for a fixed vector (the domain and source coincide!) are called shifts (a.k.a. translations). Warning: The shifts are not linear unless ! Composition of two shifts is again a shift.
Prove that all translations form a commutative group (by composition) isomorphic to the space itself. (Hint: this is a tautological statement).
A map between two vector spaces is called affine, if it is a composition of a linear map and translations.
Any affine map has the form for some . Sometimes it is more convenient to write the map under the form : this is possible for any point . Note that the composition of affine maps in dimension 1 is not commutative anymore.
Key computation. Assume you are given a map in the sense that you can evaluate it at any point . Suppose an oracle tells you that this map is affine. How can you restore the explicit formula for ?
Obviously, . To find , we have to plug into it any point and the corresponding value . Given that , we have for any choice of .
The expression for a non-affine function is in general not-constant and depends on the choice of the point .
Definition. A function is called differentiable at the point , if the above expression for , albeit non-constant, has a limit as , where is a function which tends to zero. The number is called the derivative of at the point and denoted by (and also by half a dozen of other symbols: , …).
Existence of the limit means that near the point the function admits a reasonable approximation by an affine function : , i.e., the “non-affine part” is small not just by itself, but also relative to small difference .
Differentiability and algebraic operations
The only non-obvious moment is differentiability of the product: the product (unlike the composition) of affine functions is not affine anymore, but is immediately differentiable:
, but the quadratic term is vanishing relative to , so the entire sum is differentiable.
Exercise. Derive the Leibniz rule for the derivative of the product.
Derivative and the local study of functions
Affine functions have no (strong) maxima or minima, unless restricted on finite segments. Yet absence of the extremum is a strong property which descends from the affine approximation to the original function. Details here and here.