34A. Elementary Analysis of Algorithms

Analysis of algorithms

---

We will be looking at computational problems with the goal of finding efficient ways to solve them. That requires that we can compare two algorithms to determine which is better, which involves finding out just efficient each algorithm is.

Algorithm analysis is concerned with determining how much of a resource, such as time or memory, an algorithm uses as a function of some characteristic of the input to the algorithm, usually the size of the input.

It is customary practice to use variable n for the number of characters that it takes to write down the input to an algorithm. The time or memory is given as a function of n. For example, a particular algorithm might take time n².

Big-O and Θ notation

---

Usually it is not worth determining the exact cost of running an algorithm. An approximation is all we need. In fact, we usually want to know how the cost depends on n as n grows, not what the exact cost is; it is enough to know that the cost is proportional to some function of n. For expressing that, we use O and Θ notation.

Let f(n) and g(n) be two functions that take an integer n and yield an integer or real number. f(n) and g(n) are mathematical functions, not C++ functions.

For example,

• 5n² is O(n²).
  (Choose c = 5 and n₀ = 0. Observe that 5n² < cn² for all n.)

• n² is O(5n²).
  (Choose c = 1 and n₀ = 0. Observe that n² < cn² for all n.)

• 5n² is Θ(n²) since
   5n² is O(n²) and
   n² is O(5n²).

• 3n² + 2n + 10 is O(n²).
  (Choose n₀ = 10 and c = 5. For all n ≥ 10, 3n² + 2n + 10 ≤ 5n².)

• 3n² is O(n² + 2n + 10).
  (What values of n₀ and c will work?)

• 3n² + 2n + 10 is Θ(n²) since
   n² is O(3n² + 2n + 10) and
   3n² + 2n + 10 is O(n²).

• 2n is O(n²).
  (Choose n₀ = 2 and c = 1. n² grows much faster than 2n as n increases. For example, if n = 8, 2n = 16 and n² = 64.)

• n² is not O(2n).
  Function g(n) = n² grows much faster than f(n) = 2n.

When comparing functions using O and Θ, polymomials depend only on their degree. If f(n) is a polynomial of degree d and g(n) is a polynomial of degree e, then

• f(n) is O(g(n))
  if d ≤ e.

• f(n) is Θ(g(n))
  if d = e.

Exercises

---

1. True or false: n² is O(n³). Answer

2. True or false: n³ is O(n²). Answer

3. True or false: n³ + 2n² − n is Θ(n³). Answer

4. True or false: n³ + 2n² − n is Θ(n⁵). Answer