Introduction to Optimization
Finding the best result within the given conditions
How do you find the best result within the given conditions? In decision analysis (14), we judged which of several alternatives to choose. Optimization goes one step further: it expresses the values you can choose and the conditions you must meet in mathematical form, and finds the best result among them.
Suppose a company makes products A and B. Production time and raw materials are limited, so making more of A leaves fewer resources for B. You can’t solve this problem just by comparing which product is better. You have to find how to allocate the limited resources so that total profit is as large as possible, and that is the basic purpose of optimization.
What is optimization?
Optimization is the process of finding, among the many options that satisfy the given conditions, the solution that best fits your goal.
Maximize profit on a limited budget
Minimize delivery cost and delivery time
Meet patient demand with limited staff and beds
Allocate assets weighing risk and return
Get the sample you need on a limited survey budget (07 Survey sampling)
What to decide, what to make as good as possible, and what to respect
Optimization isn’t simply making a number bigger or smaller. You first have to be clear about what you count as best and which conditions you are choosing within.
The three elements of an optimization problem
Decision variables
Decision variables are the values we set ourselves. If we are deciding how many units of products A and B to make, the decision variables are x = units of product A produced and y = units of product B produced. These aren’t observed data; they are values the optimization will decide. They may take continuous values, like production quantities; only whole numbers, like the number of employees or vehicles; or only 0 or 1, like whether to install a piece of equipment.
Objective function
The objective function expresses, as a formula, what you want to make as good as possible. If each unit of product A earns a profit of $60 and each unit of B earns $100, it is:
max: maximize · Z: total profit ($)
Some problems maximize something, like profit, output or utility; others minimize something, like cost, travel distance or waiting time. When goals conflict, as when you want to cut costs and raise quality at the same time, you also have to decide which goal comes first or how to judge the balance between them.
Constraints
Constraints are limits or rules that must be respected. If one unit of product A takes 2 hours and one unit of B takes 1 hour, and 100 hours are available in total, then along with the condition that production can’t be negative, the plan must satisfy:
If raw materials are limited too, you add a separate constraint for them. In this way, constraints set the range of choices that are possible in reality.
Building an optimization problem from an example
Profit $60 per unit · 2 hours to make
Profit $100 per unit · 1 hour to make
100 hours in total
If we consider only production time and assume no other factors such as raw materials, demand limits or fixed costs, the model is max Z = 60x + 100y, 2x + y ≤ 100, x, y ≥ 0. Let’s compare a few possible production combinations.
| Product A (x) | Product B (y) | Hours used | Total profit |
|---|---|---|---|
| 0 | 0 | 0 hours | $0 |
| 20 | 40 | 80 hours | $5,200 |
| 30 | 40 | 100 hours | $5,800 |
| 40 | 20 | 100 hours | $4,400 |
| 50 | 0 | 100 hours | $3,000 |
Among the combinations in the table, 30 of A and 40 of B ($5,800) is best, but comparing just a few combinations can’t tell you whether it is the best overall. If you draw every possible combination (the blue area), as in Figure 2, and push a line joining points of equal profit (an iso-profit line) toward the upper right, the last point that still touches the area is 0 of A and 100 of B ($10,000). B earns more than A and uses only half the time, so if time is the only constraint, making only B is best. Linear programming (Ch. 02) covers how to find optimal solutions systematically like this.
Feasible and optimal solutions
A feasible solution is a set of values for the decision variables that satisfies every constraint. 30 of A and 40 of B uses 2(30) + 40 = 100 hours and stays within the limit, so it is feasible. 40 of A and 30 of B needs 2(40) + 30 = 110 hours and goes over the limit, so it isn’t allowed, even though its profit ($5,400) looks large.
An optimal solution is the feasible solution that makes the objective function value best. In a maximization problem it is the one with the largest objective value; in a minimization problem, the smallest.
Feasible only means it respects the conditions.
If it breaks a constraint, it isn’t a solution, however high its objective value.
It must respect every constraint and achieve the goal best.
Types of optimization problems
| Type | Feature | Example uses |
|---|---|---|
| Linear programming | Linear objective and constraints | Production quantities · budget allocation |
| Integer programming | Variables must be integers | Number of employees · number of vehicles |
| Mixed-integer programming | Uses continuous and integer variables together | Equipment installation and production quantities |
| Nonlinear optimization | Nonlinear objective or constraints | Complex cost · revenue models |
| Multi-objective optimization | Considers several goals together | Balancing cost and quality |
| Robust · stochastic optimization | Builds uncertainty into the model | Handling demand swings · supply risk |
These types aren’t mutually exclusive. A problem can include integer variables and consider several goals at once. What matters is not picking a name first, but describing the real situation accurately and then choosing a method that fits its structure.
An optimal solution doesn’t always exist
Setting up an optimization problem doesn’t always get you an optimal solution. If the constraints conflict with each other, there is no feasible solution; if there aren’t enough constraints, the objective function improves without limit. If nothing at all limits production and every unit made adds profit, the mathematical model has no finite maximum profit. Even when an optimal solution exists, several solutions may give the same value.
In reality there are limits such as demand, production capacity and funds. Leave these conditions out and the calculation may still run, but the result won’t match reality. In the earlier example, the conclusion that "100 of B only" is optimal also changes if demand for B is under 100 units. When you interpret the results, check not only the computed numbers but also whether the model’s conditions reflect reality properly.
How does optimization proceed?
Rather than carrying out whatever an optimization tool computes as is, it’s important to review whether the inputs and assumptions are sound and, if needed, revise the conditions and analyze again.
We built an optimization model with the production quantities of products A and B as decision variables, total profit (60x + 100y, in dollars) as the objective function, and 100 hours of production time (2x + y ≤ 100) plus non-negativity as constraints. With production time as the only constraint, the optimal solution was to produce only product B, 100 units, for a total profit of $10,000; adding demand limits or raw-material constraints could change this result.
Start by defining what you decide, what counts as best, and what you must respect.
- Optimization: A way to find the choice that best fits your goal under the given conditions.
- Three elements: Decision variables are the values to decide, the objective function is the goal to achieve, and constraints are the limits to respect.
- Feasible and optimal solutions: A feasible solution satisfies every constraint; the optimal solution is the one among them that achieves the objective best.
- Types: Depending on the problem’s structure, you use different methods such as linear, integer, nonlinear, multi-objective or stochastic optimization.
- Model quality: Sometimes there is no solution or the objective grows without bound, and the result depends on how accurately you defined the objective and constraints.
Up next: 02 Linear Programming. We’ll look at how to find the optimal solution systematically when the objective function and constraints are linear.