[ TOC ]

Mathematics 309

Linear approximations

In 1D

If f(x) is a reasonable function of one variable, then its graph looks linear when you look only at small parts of it.

Of course, the line it looks like is the tangent line.

In other words, for small values of and an arbitrary value of x0 we have an approximate equality

f(x0 + ) ~ f(x0) + f'(x0)

We can understand this a bit better if we look at an example, say f(x) = x2. Then we have exactly

f(x0 + ) = f(x0) + 2x0 + 2

and since f'(x) = x2 x, this means that the approximation by the tangent line is one which ignores the term 2. In general the linear approximation means ignoring powers k where k is 2 or more. Of course if is small, then these will be an order of magnitude smaller.

For 2D to 1D

Suppose now that f(x, y) is a function of two variables. An example would be f(x, y) = x2 + y2. It turns out that near any particular 2D point (x0, y0) the function is approximately an affine function. In other words, there exist constants A and B such that for small values of and

f(x0 + , y0 + ) ~ f(x0, y0) + A + B

For example

(x0 + )2 + (y0 + )2 = x02 + y02 + 2 x0 + 2 y0 + 2 + 2
~ x02 + y02 + 2 x0 + 2 y0

so that here A = 2 x0 and B = 2 y0. This example suggests how to compute the coefficients A and B. To see what A is, fix y temporarily to be constant, and view F(x, y) as a function of x alone. The coefficient A is then the derivative with respect to x of this function. It is called the partial derivative of f(x, y) with respect to x Similarly for B: fix x and take the derivative with respect to y. There are symbols for the partial derivatives:

fx = and fy =

For example, if f(x, y) = (x2+y)2 then the partial derivatives are 4x(x2 + y) and 2(x2+ y).

These linear approximations can be seen in a plot of level lines f(x, y) = const. If we look only near a fixed point (x0, y0) then the level lines look very much like the level lines of an affine function - they are close to being straight, and they are evenly spaced. Here is what happens for the function f(x, y) = x4 + y4 as we zoom in.


If f(x, y) is a function of two variables, in the neightbourhood of any point (x0, y0) it is approximated by an affine function

f(x0, y0) + A (x - x0) + B (y - y0)

where A and B are the partial derivatives of f. As a consequence, in small regions its level lines are very nearly the same as the level lines of this affine function. Now the level lines of the affine function AX+By+C are straight lines, and they are perpendicular to the vector [A, B]. Therefore:

  • The level curve of f(x, y) through the point (x0, y0) is perpendicular to the vector

    [ , ]

    which is called the gradient of f at that point.

For 2D to 2D

A 2D-to-2D transformation is a transformation from 2D points (x, y) to other 2D points (u(x, y), v(x, y)). We shall see examples later on, in the analysis of lens systems. But here I just wwant to derive a linear approximation formula. Here, too, there is a linear approximation formula, and it follows from the previous one. Here's how it goes:

f(x0 + , y0 + ) = ( u(x0 + , y0 + ), v(x0 + , y0 + ) )
~ ( u(x0, y0) + ( ) + ( ) , v(x0, y0) + ( ) + ( ) )
= f(x0, y0) +

In other words, in small regions of the plane such a transformation is approximated by an affine transformation, one obtained from a linear transformation by a translation.