Searching an Array

Linear search

A common problem is to find out whether a value occurs in an array. Keeping the logical size separate from the physical size, let's write a method contains(v, a, n) that returns true if one of a[0], a[1], ..., a[n-1] contains v. (So the array has logical size n.) A natural approach is called linear search. The idea is to look at each value in the array, stopping as soon as the value v is seen, or when the algorithm runs out of things to look at.

  public static boolean contains(int v, int[] a, int n)
  {
    for(int k = 0; k < n; k++)
    {
      if(a[k] == v) return true;
    }
    return false;
  }

The cost of linear search

In the worst case, computing contains(v, a, n) requires you to look at all n values in the occupied part of array a. (In fact, you always need to look at all of the values in the array when v does not occur in the array.) When n is a small number, the method runs very fast. But when n is really large, it can take a long time to look at everything in the array.

The amount of time that it takes to compute contains(v, a, n) is proportional to the number of values that the method looks at, so we say that it takes time proportional to n. (Phrase proportional to indicates that the time is some constant times that amount. The constant takes into account the number of basic steps needed to go once around the loop and the time that the computer needs to perform each step. For example, if it takes 15 basic instructions to go around the loop once, and each instruction takes s seconds to perform, then the time used is close to 15sn, in the worst case.)

Binary search

Think about the problem of looking up a name in a telephone book. You obviously do not start at the beginning and search through the names until you find the one you are looking for. Instead, you take advantage of the fact that the names are listed in alphabetical order, and that speeds up the search fantastically.

You can do a similar thing when searching for a number in an array of numbers, provided you know that the numbers in the array are in ascending order. The idea is to start in the middle. Compare the number v that you are looking for with the value in the middle of the array. In the unlikely event that v is equal to the middle value, then the search is finished. If v is smaller than the middle value, then v must be in the first half of the array (if it is in the array at all). But if v is larger than the middle value, then it will have to be in the second half. Keep going with the same idea, taking the middle value in the part of the array that is left. Each time you compare v to a value in the array, you cut the number of remaining candidate values approximately in half.

Method binarySearch(v, a, low, high) returns true if v is equal to one of the values a[low], a[low+1], ..., a[high]. So it searches in a part of the array, from index low to index high. A precise statement of what binarySearch does is as follows.

binarySearch(v, a, low, high) returns true if and only if there exists an integer k where

1. low ≤ k ≤ high
2. a[k] = v

One way to write binarySearch is to use recursion. That means breaking down the problem into cases, and (in some cases) letting another copy of the method finish the job.

  public static boolean binarySearch(int v, int[] a, int low, int high)
  {
    //-----------------------------------------------------------
    // Case 1.  When low > high, there are no indices to look at.
    //-----------------------------------------------------------

    if(low > high) return false;

    //-----------------------------------------------------------
    // Get the middle index (the average of low and high) and the
    // value in the array at that index.
    //-----------------------------------------------------------

    int midindex = (low + high)/2;
    int midval   = a[midindex];

    //-----------------------------------------------------------
    // Case 2.  If v = midval, then v obviously occurs in this
    // part of the array.
    //-----------------------------------------------------------
 
    if(v == midval) return true;

    //-----------------------------------------------------------
    // Case 3.  If v < midval, then search the array from
    // index low to index midval - 1.  The result of that search
    // is exactly what we want to return as the result of this search.
    //-----------------------------------------------------------

    if(v < midval) return binarySearch(v, a, low, midval - 1);

    //-----------------------------------------------------------
    // Case 4.  If v > midval (the only remaining possibility), 
    // then search the array from index midval + 1 to index high.
    //-----------------------------------------------------------

    return binarySearch(v, a, midval + 1, high);
  }

That method definition might look long, but it is quite short without the comments that clarify what it is doing. Here it is with comments removed.

  public static boolean binarySearch(int v, int[] a, int low, int high)
  {
    if(low > high) return false;

    int midindex = (low + high)/2;
    int midval   = a[midindex];

    if(v == midval) return true;
    if(v < midval) return binarySearch(v, a, low, midval - 1);
    return binarySearch(v, a, midval + 1, high);
  }

An alternative is to use a loop. Just keep decreasing the index range, either by changing the low point or the high point of the range. If we happen upon value v in the array, return true immediately. If the index range becomes empty, then exit the loop and return false.

  public static boolean binarySearch(int v, int[] a, int low, int high)
  {
    int lo = low;
    int hi = high;
    while(lo <= hi) 
    {
      int midindex = (lo + hi)/2;
      int midval = a[midindex];
      if(v == midval) return true;
      else if(v < midval) hi = midval - 1;
      else lo = midval + 1;
    }
    return false;
  }

The cost of binary search

We saw that linear search takes time proportional to n, where n is the number of values in the array. Based on the telephone book analogy, you would expect binary search to be faster. But how much faster?

For binary search, define n = high - low + 1 to be the number of values in the section of the array to look at. For our purposes here, it is probably easiest to understand the version that uses a loop. Each time around the loop (except for the last time where either the method finds v and returns true or it exits the loop and returns false), the size of the range is approximately cut in half. For example, suppose lo = 100 and hi = 400, so there are hi - lo + 1 = 301 values in the range. The method computes midval = (100 + 400)/2 = 250. If v < midval, then the method changes hi to be 249, and the range is now from 100 to 249, containing 150 values. If v > midval, then the method changes lo to be 251, and the range is now from 251 to 400, also containing 150 values. In either event, the size of the range has approximately been cut in half (from 301 to 150). When there are an odd number of things in the array, the round down when dividing by 2.

Imagine starting with some number of values to search. Cut that in half, then cut it in half again and again. The worst case occurs when the number being sought is not in the array. When the size becomes 1, the very next step we get out of the loop, so keep cutting in half until you reach 1. For example, imagine that there are 100 values in the array to search. Successively halving (and rounding down at odd values) gives the following sequence of numbers.

  100
   50
   25
   12
    6
    3
    1

  1000
   500
   250
   125
    62
    31
    15
     7
     3
     1

Function log₂(n) is defined to be the solution x to the equation 2^x = n. For example

When n is not a power of 2, log₂(n) is not an integer. For example, log₂(3) is about 1.5849625. But here we are really only interested in approximate values, and we will be happy saying that log₂(3) is between 1 and 2.

If you start at n and successively do halving steps until you get to 1, then it takes about log₂(n) halving steps to finish. For example, 9 < log₂(1000) < 10, and we found that it took 9 halving steps to reach 1, starting at 1000. (In fact, if you round down at each halving of an odd number, then you round log₂(n) down too. If you round up at the halving steps, you also round log₂(n) up to get the number of halving steps.)

Since the number of halving steps tells how many times you go around the loop for binary search, the time taken by binary search is proportional to log₂(n).

Comparison of linear search and binary search

So linear search takes time proportional to n and binary search takes time proportional to log₂(n). Is that a big difference?

When comparing algorithms, it is always a good idea to imagine large problems. For small problems, almost any algorithm will work quickly, and it is the large problems where you really care. So imagine a somewhat large value of n, say n = 10,000. Ignoring the constants of proportionality (which in this case are mostly influenced by the speed of the computer), linear search takes 10,000 steps and binary search takes about 14 steps. Obviously, the difference is huge.

Sometimes people who are inexperienced with computers get overly impressed with computer speeds, and fall back on the phrase "computers are fast" to imagine that your choice of algorithm does not matter. Imagine that you have a really large collection of things to look at, say a billion of them. (Think of an internet search engine, which has a huge set of web pages to search — much more than a billion.) Imagine that your computer can check a million of them per second, or one microsecond per check. Since log₂(1 billion) is about 30, you would expect binary search to be done in about 30 microseconds. Using linear search would take about a billion microseconds, which is about 17 minutes.

Searching for strings

For simplicity, we have looked at the searching problem using arrays of integers. But a more realistic problem is to search for a string, such as somebody's name. To deal with that, you might expect it just to be a matter of changing int to String in some places. And it almost is.

But there is another thing to watch out for. You do not want to ask whether two strings are the same using operator ==. (Remember that s == t is true if strings s and t are the same object, not if they contain the same characters.) Instead, the correct expression is s.equals(t). Also, you cannot ask whether one string is less than another (alphabetically) using the < operator. The correct expression is s.compareTo(t).

Here are conversions of linear search and of the looping version of binary search to use strings.

  public static boolean contains(String v, String[] a, int n)
  {
    for(int k = 0; k < n; k++)
    {
      if(a[k].equals(v)) return true;
    }
    return false;
  }

  public static boolean binarySearch(Sting v, String[] a, int low, int high)
  {
    int lo = low;
    int hi = high;
    while(lo <= hi) 
    {
      int midindex = (lo + hi)/2;
      String midval = a[midindex];
      int cmp = v.compareTo(midval);
      if(cmp == 0) return true;
      else if(cmp < 0) hi = midval - 1;
      else lo = midval + 1;
    }
    return false;
  }

Storing a telephone book

When you look up a name in a telephone book, it is not enough to learn that the name is there. You also want to know the person's telephone number. How can you handle that?

One idea is to store two arrays, one holding the names and the other holding the telephone numbers. The idea is that the telephone number for the person whose name is names[k] is found in numbers[k]. That is, find the name you are looking for in the names array, and you will find the corresponding telephone number at the same index in the numbers array. Here is a method (lookup) that employs binary search to look up someone's telephone number, assuming (1) arrays names and numbers have already been created, (2) there are n names, and (3) the names array is in alphabetical order. If the name is not found, then lookup returns null.

  public static String lookup(Sting name, String[] names, String[] numbers, int n)
  {
    int lo = 0;
    int hi = n-1;
    while(lo <= hi) 
    {
      int midindex = (lo + hi)/2;
      String midval = names[midindex];
      int cmp = name.compareTo(midval);
      if(cmp == 0) return numbers[midindex];
      else if(cmp < 0) hi = midval - 1;
      else lo = midval + 1;
    }
    return null;
  }

Problems

1. [solve] How long does linear search take, in terms of the size n of the array to search?

2. [solve] How long does binary search take, in terms of the size n of the array to search?

3. [solve] What two integers is log₂(40) between?

Summary

Linear search searches an array of size n in time proportional to n.

Binary search searches an array of size n in time proportional to log₂(n), but it requires the array to be sorted.