We wish to consider a special type of optimization problem:
Find the maximum (or minimum) of a function subject to the condition
If it is possible to solve for so that it is expressed explicitly as , by substituting in (1), it becomes
Find the maximum (or minimum) of a single variable function .
In the case that can not be obtained from solving , we re-state the problem as:
Find the maximum (or minimum) of a single variable function where is a function of , implicitly defined by
Following the traditional procedure of finding the maximum (or minimum) of a single variable function, we differentiate with respect to :
Similarly,
By grouping and (3), we have
The fact that at any stationary point means for all where ,
If then from (4),
Substitute it into (6),
Let , we have
Combining (7) and (8) gives
It follows that to find the stionary points of , we solve
for and .
This is known as the method of Lagrange’s multiplier.
Let .
Since
,
,
,
(9) is equivalent to
Let’s look at some examples.
Example-1 Find the minimum of subject to the condition that
Let .
Solving
for gives .
When .
, we have
.
That is,
.
Hence,
.
The target function with constraint indeed attains its global minimum at .
I first encountered this problem during junior high school and solved it:
.
I solved it again in high school when quadratic equation is discussed:
In my freshman calculus class, I solved it yet again:
Example-2 Find the shortest distance from the point to the parabola .
We minimize where .
If we eliminate in , then . Solving gives , Clearly, this is not valid for it would suggest that from , an absurdity.
By Lagrange’s method, we solve
Fig. 1
The only valid solution is . At . It is the global minimum:
.
Example-3 Find the shortest distance from the point to the line .
We want find a point on the line so that the distance between and is minimal.
To this end, we minimize where (see Fig. 2)
Fig. 2
We found that
and the distance between and is
To show that (11) is the minimal distance, .
Let , we have
.
Since ,
That is
.
By the fact that , we have
Compute (see Fig. 3)
Fig. 3
yields
After some rearrangement and factoring, it becomes
By (12), it reduces to
.
This is clearly a positive quantity. Therefore,