11.3 Application: Analysis of Algorithm Efficiency I

Transkrypt

1 11.3 Application: Analysis of Algorithm Efficiency I Prove that if f (x) and g(x) are both o(h(x)), then for all real numbers a and b, af(x) + bg(x) is o(h(x)). 61. Prove that for any positive real numbers a and b, ifa < b then x a is o(x b ). Answers for Test Yourself 1. f (x) is (g(x)). f (x) is O(g(x)) 3. f (x) is (g(x)) 4. >; > 5. the degree of p(x) 6. n 11.3 Application: Analysis of Algorithm Efficiency I As soon as an Analytical Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will then arise by what course of calculation can these results be arrived at by the machine in the shortest time? Charles Babbage, 1864 Charles Babbage ( ) Bettmann/CORBIS Charles Babbage s Analytical Engine was similar in concept to a modern computer, and the quotation shown above suggests that well over a hundred years ago he anticipated the importance of analyzing the efficiencies of computer algorithms. Starting in the late 1940s, a number of mathematicians and computer scientists contributed to the development of algorithm analysis. Alan Turing may have been the first to suggest a concrete way for doing this. In a 1948 paper he wrote: It is convenient to have a measure of the amount of work involved in a computing process, even though it be a very crude one.... We might, for instance, count the number of additions, subtractions, multiplications, divisions, recording of numbers... In the early 1960s, Donald Knuth started writing The Art of Computer Programming, a multivolume work, which provides a solid and extensive foundation for the subject that is both elegant and mathematically rigorous. Note For more about the work of Alan Turing, see Sections 6.4 and 1.. The Sequential Search Algorithm The object of a search algorithm is to hunt through an array of data in an attempt to find a particular item x. In a sequential search, x is compared to the first item in the array, then to the second, then to the third, and so on. The search is stopped if a match is found at any stage. On the other hand, if the entire array is processed without finding a match, then x is not in the array. An example of a sequential search is shown diagrammatically in Figure a[1] a[] a[3] a[4] a[5] a[6] a[7] no no no no a[1] = x? a[] = x? a[3] = x? a[4] = x? a[5] = x? Done Figure Sequential Search of a[1], a[],..., a[7] for x where x = a[5] yes Quarterly Journal of Mechanics and Applied Mathematics, vol. 1 (1948), pp Donald E. Knuth, The Art of Computer Programming, vol.1:fundamental Algorithms, 3rd ed. (1997); vol. : Seminumerical Algorithms, 3rd ed., (1997); vol. 3: Searching and Sorting, nd ed. (1998) (Reading, MA: Addison-Wesley).

2 740 Chapter 11 Analysis of Algorithm Efficiency Example Best- and Worst-Case Orders for Sequential Search Find best- and worst-case orders for the sequential search algorithm from among the set of power functions. Solution Suppose the sequential search algorithm is applied to an input array a[1], a[],...,a[n] to find an item x. In the best case, the algorithm requires only one comparison between x and the items in a[1], a[],...,a[n]. This occurs when x is the first item in the array. Thus in the best case, the sequential search algorithm is (1). (Note that (1) = (n 0 ).) In the worst case, however, the algorithm requires n comparisons. This occurs when x = a[n] or when x does not appear in the array at all. Thus in the worst case, the sequential search algorithm is (n). The Insertion Sort Algorithm Insertion sort is an algorithm for arranging the items in an array into ascending order. Initially, the second item is compared to the first. If the second item is less than the first, their values are interchanged, and as a result the first two array items are in ascending order. The idea of the algorithm is gradually to lengthen the section of the array that is known to be in ascending order by inserting each subsequent array item into its correct position relative to the preceding ones. When the last item has been placed, the entire array is in ascending order. Figure illustrates the action of step k of insertion sort on an array a[1], a[], a[3],...,a[n]. sorted subarray a[1], a[], a[3],..., a[k 1], a[k], a[k + 1],..., a[n] Step k: Insert the value of a[k] into its proper position relative to a[1], a[],..., a[k 1]. At the end of this step a[1], a[],..., a[k] is sorted. Donald Knuth (born 1938) Courtesy of Donald Knuth Figure Step k of Insertion Sort Understanding the relative efficiencies of algorithms designed to do the same job is of much more than academic interest. In industrial and scientific settings, the choice of an efficient over an inefficient program may result in the saving of many thousands of dollars or may make the difference between being able or not being able to do a project at all. Two aspects of algorithm efficiency are important: the amount of time required to execute the algorithm and the amount of memory space needed when it is run. In this chapter we introduce basic techniques for calculating time efficiency. Similar techniques exist for calculating space efficiency. Occasionally, one algorithm may make more efficient use of time but less efficient use of memory space than another, forcing a trade-off based on the resources available to the user. Time Efficiency of an Algorithm How can the time efficiency of an algorithm be calculated? The answer depends on several factors. One is the size of the set of data that is input to the algorithm; for example, it takes longer for a sort algorithm to process 1,000,000 items than 100 items. Consequently, the execution time of an algorithm is generally expressed as a function of its input size. Another factor that may affect the run time of an algorithm is the nature of the input data. For instance, a program that searches sequentially through a list of length n to find a

3 11.3 Application: Analysis of Algorithm Efficiency I 741 data item requires only one step if the item is first on the list, but it uses n steps if the item is last on the list. Thus algorithms are frequently analyzed in terms of their best case, worst case, and average case performances for an input of size n. Roughly speaking, the analysis of an algorithm for time efficiency begins by trying to count the number of elementary operations that must be performed when the algorithm is executed with an input of size n (in the best case, worst case, or average case). What is classified as an elementary operation may vary depending on the nature of the problem the algorithms being compared are designed to solve. For instance, to compare two algorithms for evaluating a polynomial, the crucial issue is the number of additions and multiplications that are needed, whereas to compare two algorithms for searching a list to find a particular element, the important distinction is the number of comparisons that are required. As is common, we will classify the following as elementary operations: addition, subtraction, multiplication, division, and comparisons that are indicated explicitly in an if-statement using one of the relational symbols <,, >,, =, or =. When algorithms are implemented in a particular programming language and run on a particular computer, some operations are executed faster than others, and, of course, there are differences in execution times from one machine to another. In certain practical situations these factors are taken into account when we decide which algorithm or which machine to use to solve a particular problem. In other cases, however, the machine is fixed, and rough estimates are all that we need to determine the clear superiority of one algorithm over another. Since each elementary operation is executed in time no longer than the slowest, the time efficiency of an algorithm is approximately proportional to the number of elementary operations required to execute the algorithm. Consider the example of two algorithms, A and B, designed to do a certain job. Suppose that for an input of size n, the number of elementary operations needed to perform algorithm A is between 10n and 0n (at least for large n) and the number of elementary operations needed to perform algorithm B is between n and 4n. Note that 0n < n whenever n > 10, which means that the maximum number of operations required to execute A is less than the minimum number of operations required to execute B whenever n > 10. In fact, 0n is very much less than n when n is large. For instance, if n = 1000, then 0n = 0,000, whereas n =,000, 000. We say that in the worst case, algorithm A is (n) (or has worst-case order n) and that in the worst case, algorithm B is (n ) (or has worst-case order n ). Definition Let A be an algorithm. 1. Suppose the number of elementary operations performed when A is executed for an input of size n depends on n alone and not on the nature of the input data; say it equals f (n).if f (n) is (g(n)), we say that A is (g(n)) or A is of order g(n).. Suppose the number of elementary operations performed when A is executed for an input of size n depends on the nature of the input data as well as on n. a. Let b(n) be the minimum number of elementary operations required to execute A for all possible input sets of size n. Ifb(n) is (g(n)), we say that in the best case, A is (g(n)) or A has a best-case order of g(n). b. Let w(n) be the maximum number of elementary operations required to execute A for all possible input sets of size n. Ifw(n) is (g(n)), we say that in the worst case, A is (g(n)) or A has a worst-case order of g(n).

4 74 Chapter 11 Analysis of Algorithm Efficiency Some of the orders most commonly used to describe algorithm efficiencies are shown in Table As you see from the table, differences between the orders of various types of algorithms are more than astronomical. The time required for an algorithm of order n to operate on a data set of size 100,000 is approximately 10 30,076 times the estimated 15 billion years since the universe began (according to one theory of cosmology). On the other hand, an algorithm of order log n needs at most a fraction of a second to process the same data set. Table Time Comparisons of Some Algorithm Orders Approximate Time to Execute f (n) Operations Assuming One Operation per Nanosecond f (n) n = 10 n = 1,000 n = 100,000 n = 10,000,000 log n sec 10 8 sec sec sec n 10 8 sec 10 6 sec sec 0.01 sec n log n sec 10 5 sec sec 0.3 sec n 10 7 sec sec 10 sec 7.8 min n sec 1sec 11.6 days 31,688 yr n 10 6 sec yr yr yr one nanosecond = 10 9 second Example Computing an Order of an Algorithm Segment Assume n is a positive integer and consider the following algorithm segment: p := 0, x := for i := to n p := (p + i) x a. Compute the actual number of additions and multiplications that must be performed when this algorithm segment is executed. b. Use the theorem on polynomial orders to find an order for this algorithm segment. Solution a. There are one multiplication and one addition for each iteration of the loop, so there are twice as many multiplications and additions as there are iterations of the loop. Now the number of iterations of the for-next loop equals the top index of the loop minus the bottom index plus 1; that is, n + 1 = n 1. Hence there are (n 1) = n multiplications and additions. b. By the theorem on polynomial orders, and so this algorithm segment is (n). n is (n), The next example looks at an algorithm segment that contains a nested loop.

5 11.3 Application: Analysis of Algorithm Efficiency I 743 Example An Order for an Algorithm with a Nested Loop Assume n is a positive integer and consider the following algorithm segment: s := 0 for i := 1 to n for j := 1 to i s := s + j (i j + 1) a. Compute the actual number of additions, subtractions, and multiplications that must be performed when this algorithm segment is executed. b. Use the theorem on polynomial orders to find an order for this algorithm segment. Solution a. There are two additions, one multiplication, and one subtraction for each iteration of the inner loop, so the total number of additions, multiplications, and subtractions is four times the number of iterations of the inner loop. Now the inner loop is iterated one time when i = 1, two times when i =, three times when i = 3, n times when i = n.. You can see this easily if you construct a table that shows the values of i and j for which the statements in the inner loop are executed. There is one iteration for each column in the table. i n j n }{{}} {{ }} {{ }} {{ } } {{ } n Hence the total number of iterations of the inner loop is n(n + 1) n = by Theorem 5.., and so the number of additions, subtractions, and multiplications is n(n + 1) 4 = n(n + 1). An alternative method for computing the number of columns of the table uses an approach discussed in Example Observe that the number of columns in the table is the same as the number of ways to place two s in n categories, 1,,...,n, where the location of the s indicates the values of i and j with j i. By Theorem 9.6.1, this number is ( ) n 1 + = ( ) n + 1 = (n + 1)! (n + 1)n(n 1)! = =!((n + 1) )! (n 1)! n(n + 1). Although, for this example, the alternative method is more complicated than the one preceding it, it is simpler when the number of loop nestings exceeds two. (See exercise 19.)

6 744 Chapter 11 Analysis of Algorithm Efficiency b. By the theorem on polynomial orders, n(n + 1) = n + n is (n ),andsothis algorithm segment is (n ). Example When the Number of lterations Depends on the Floor Function Assume n is a positive integer and consider the following algorithm segment: for i := n/ to n a := n i a. Compute the actual number of subtractions that must be performed when this algorithm segment is executed. b. Use the theorem on polynomial orders to find an order for this algorithm segment. Solution a. There is one subtraction for each iteration of the loop, and the loop is iterated n n n + 1times.Ifn is even, then = n, and so the number of subtractions is n n + 1 = n n + 1 = n +. n If n is odd, then n n b. By the theorem on polynomial orders, = n 1, and so the number of subtractions is + 1 = n n 1 n (n 1) = n + is (n) and n + 3 is (n) = n + 3. also. Hence, regardless of whether n is even or odd, this algorithm segment is (n). The following is a formal algorithm for insertion sort. Algorithm Insertion Sort [The aim of this algorithm is to take an array a[1], a[], a[3],...,a[n], wheren 1, and reorder it. The output array is also denoted a[1], a[], a[3],...,a[n]. It has the same values as the input array, but they are in ascending order. In the kth step, a[1], a[], a[3],...,a[k 1] is in ascending order, and a[k] is inserted into the correct position with respect to it.] Input: n [a positive integer], a[1], a[], a[3],...,a[n] [an array of data items capable of being ordered]

7 11.3 Application: Analysis of Algorithm Efficiency I 745 Algorithm Body: for k := to n [Compare a[k] to previous items in the array a[1], a[], a[3],...,a[k 1], starting from the largest and moving downward. Whenever a[k] is less than a preceding array item, increment the index of the preceding item to move it one position to the right. As soon as a[k] is greater than or equal to an array item, insert the value of a[k] to the right of that item. If a[k] is greater than or equal to a[k 1], then leave the value of a[k] unchanged.] x := a[k] j := k 1 while ( j = 0) if x < a[j] then a[ j + 1] :=a[ j] j := j 1 end if end while a[ j + 1] :=x next k Output: a[1], a[], a[3],...,a[n] [in ascending order] Figure shows the result of each step when insertion sort is applied to the particular array a[1] =6, a[] =3, a[3] =5, a[4] =7, a[5] =. Initial Result of step 1 Result of step Result of step 3 Result of step 4 a[1] a[] a[3] a[4] a[5] The top row of the table shows the initial values of the array, and the bottom row shows the final values. The result of each step is shown in a separate row. For each step, the sorted section of the array is shaded. Figure Action of Insertion Sort on an Array Example develops a trace table for the action of insertion sort on a particular array. Example A Trace Table for Insertion Sort Construct a trace table showing the action of insertion sort on the array a[1] =6, a[] =3, a[3] =5, a[4] =7, a[5] =. Solution The first column on the next page shows the state of the variables before the first iteration of the for-next loop. When the for-next loop is first iterated, k is assigned the value ; x the value of a[], which is 3; and j the value of k 1, which is 1. Because j = 0, the while loop is entered and the condition for the if-then-else statement is tested. Because a[1] > x, then a[] is assigned the value of a[1], which is 6, j is assigned the value of j 1, which is 0, and a[1] is assigned the value of x, which is 3. The condition governing the while loop is tested again, but since j = 0, it is not satisfied, and so the while loop is not entered. Thus the value of k is incremented by 1 (so that it equals 3),

8 746 Chapter 11 Analysis of Algorithm Efficiency n 5 and the for-next loop is entered a second time. This process continues until the value of k has been incremented to 6. Because 6 is greater than the top value in the for-next loop, execution of the algorithm ceases, and the array items are seen to be in ascending order. a[1] 6 3 a[] a[3] a[4] a[5] 7 k x j Example Finding a Worst-Case Order for Insertion Sort a. What is the maximum number of comparisons that are performed when insertion sort is applied to the array a[1], a[], a[3],...,a[n]? b. Use the theorem on polynomial orders to find a worst-case order for insertion sort. Solution a. In each attempted iteration of the while loop, two explicit comparisons are made: one to test whether j = 0 and the other to test whether a[ j] > x. During the time that a[k] is put into position relative to a[1], a[],...,a[k 1], the maximum number of attempted iterations of the while loop is k. This happens when a[k] is less than every a[1], a[],...,a[k 1]; on the kth attempted iteration, the condition of the while loop is not satisfied because j = 0. Thus the maximum number of comparisons for a given value of k is k. Because k goes from to n, it follows that the maximum total number of comparisons occurs when the items in the array are in reverse order, and it equals n = ( n) by factoring out the = [( n) 1] by adding and subtracting 1 ( ) n(n + 1) = 1 by Theorem 5.. = n(n + 1) = n + n by algebra. b. By the theorem on polynomial orders, n + n is (n ), and so the insertion sort algorithm has worst-case order (n ). The definition of expected value that was introduced in Section 9.8 can be used to find an average-case order for insertion sort. Example Finding an Average-Case Order for Insertion Sort a. What is the average number of comparisons that are performed when insertion sort is applied to the array a[1], a[], a[3],...,a[n]? b. Use the theorem on polynomial orders to find an average-case order for insertion sort.

9 11.3 Application: Analysis of Algorithm Efficiency I 747 Solution a. Let E n be the average, or expected, number of comparisons used to sort a[1], a[],..., a[n] with insertion sort. Note that for each integer k =, 3,...,n, the expected number of comparisons used to sort a[1], a[],...,a[k] the expected number of the expected number of comparisons = comparisons used to sort a[1], a[],...,a[k 1] + used to place a[k] into position relative to a[1], a[],...,a[k 1]. Thus the expected number of comparisons E k = E k 1 + used to place a[k] into position. relative to a[1], a[],...,a[k 1] Also, E 1 = 0 because when there is just one item in the array, n = 1 and no iterations of the outer loop are performed. Now at the time a[k] is placed relative to a[1], a[],...,a[k 1], a reasonable assumption is that it is equally likely to belong in any one of the first k positions. Thus the probability of its belonging in any particular position is 1/k. If it actually belongs in position j, then (k j + 1) comparisons will be used in moving it, because there will be k j + 1 attempted iterations of the while loop and there are comparisons per attempted iteration. According to the definition of expected value given in Section 9.8, the expected number of comparisons used to place a[k] relative to a[1], a[],..., a[k 1] is therefore k j=1 1 k (k j + 1) = by [k + (k 1) ] k = ( ) k(k + 1) k Hence writing the summation in expanded form by Theorem 5.. = k + 1 by algebra. E k = E k 1 + k + 1 for all integers k, and E 1 = 0. Exercise 7 at the end of the section asks you to solve this recurrence relation to show that E n = n + 3n 4 for each integer n 1. b. By the theorem on polynomial orders, n + 3n 4 = 1 n + 3 n is (n ), and so the average-case order of insertion sort is also (n ). Test Yourself 1. When an algorithm segment contains a nested for-next loop, you can find the number of times the loop will iterate by constructing a table in which each column represents.. In the worst case for an input array of length n, the sequential search algorithm has to look through elements of the array before it terminates. 3. The worst-case order of the insertion sort algorithm is, and its average-case order is.

10 748 Chapter 11 Analysis of Algorithm Efficiency Exercise Set Suppose a computer takes 1 nanosecond (= 10 9 second) to execute each operation. Approximately how long will it take for the computer to execute the following numbers of operations? Convert your answers into seconds, minutes, hours, days, weeks, or years, as appropriate. For example, instead of 50 nanoseconds, write 13 days. a. log 00 b. 00 c. 00 log 00 d. 00 e f. 00. Suppose an algorithm requires cn operations when performed with an input of size n (where c is a constant). a. How many operations will be required when the input size is increased from m to m (where m is a positive integer)? b. By what factor will the number of operations increase when the input size is doubled? c. By what factor will the number of operations increase when the input size is increased by a factor of ten? 3. Suppose an algorithm requires cn 3 operations when performed with an input of size n (where c is a constant). a. How many operations will be required when the input size is increased from m to m (where m is a positive integer)? b. By what factor will the number of operations increase when the input size is doubled? c. By what factor will the number of operations increase when the input size is increased by a factor of ten? Exercises 4 5 explore the fact that for relatively small values of n, algorithms with larger orders can be more efficient than algorithms with smaller orders. 4. Suppose that when run with an input of size n, algorithm A requires n operations and algorithm B requires 80n 3/ operations. a. What are orders for algorithms A and B from among the set of power functions? b. For what values of n is algorithm A more efficient than algorithm B? c. For what values of n is algorithm B at least 100 times more efficient than algorithm A? 5. Suppose that when run with an input of size n, algorithm A requires 10 6 n operations and algorithm B requires n 3 operations. a. What are orders for algorithms A and B from among the set of power functions? b. For what values of n is algorithm A more efficient than algorithm B? c. For what values of n is algorithm B at least 100 times more efficient than algorithm A? For each of the algorithm segments in 6 19, assume that n is a positive integer. (a) Compute the actual number of additions, subtractions, multiplications, divisions, and comparisons that must be performed when the algorithm segment is executed. For simplicity, however, count only comparisons that occur within if-then statements; ignore those implied by for-next loops. (b) Use the theorem on polynomial orders to find an order for the algorithm segment. 6. for i := 3 to n 1 a := 3 n + i 1 7. max := a[1] for i := to n if max < a[i] then max := a[i] 8. for i := 1 to n/ a := n i 9. for i := 1 to n for j := 1 to n a := n + i j 10. for k := to n for j := 1 to 3n x := a[k] b[ j] next k 11. for k := 1 to n 1 for j := 1 to k + 1 x := a[k]+b[ j] next k 1. for k := 1 to n 1 max := a[k] for i := k + 1 to n if max < a[i] then max := a[i] a[k] := max next k 13. for i := 1 to n 1 for j := i to n if a[ j] > a[i] then do temp := a[i] a[i] :=a[ j] a[ j] :=temp end do

11 11.3 Application: Analysis of Algorithm Efficiency I 749 H t := 0 for i := 1 to n s := 0 for j := 1 to i s := s + a[ j] t := t + s 15. r := 0 for i := 1 to n 1 p := 1 q := 1 for j := i + 1 to n p := p c[ j] q := q (c[ j]) r := p + q 16. t := 0 for i := 1 to n s := 0 for j := 1 to i 1 s := s + j (i j + 1) r := s 17. for i := 1 to n for j := 1 to (i + 1)/ a := (n i) (n j) 18. for i := 1 to n for j := (i + 1)/ to n x := i j for i := 1 to n for j := 1 to i for k := 1 to j x := i j k next k 0. Construct a table showing the result of each step when insertion sort is applied to the array a[1] =6, a[] =, a[3] =1, a[4] =8, and a[5] =4. 1. Construct a table showing the result of each step when insertion sort is applied to the array a[1] =7, a[] =3, a[3] =6, a[4] =9, and a[5] =5.. Construct a trace table showing the action of insertion sort on the array of exercise Construct a trace table showing the action of insertion sort on the array of exercise How many comparisons between values of a[ j] and x actually occur when insertion sort is applied to the array of exercise 0? 5. How many comparisons between values of a[ j] and x actually occur when insertion sort is applied to the array of exercise 1? 6. According to Example , the maximum number of comparisons needed to perform insertion sort on an array of length five is =. Find an array of length five that requires the maximum number of comparisons when insertion sort is applied to it. H 7. Consider the recurrence relation that arose in Example : E 1 = 0andE k = E k 1 + k + 1, for all integers k. a. Use iteration to find an explicit formula for the sequence. b. Use mathematical induction to verify the correctness of the formula. Exercises 8 35 refer to selection sort, which is another algorithm to arrange the items in an array in ascending order. Algorithm Selection Sort [Starting with an array a[1], a[], a[3],..., a[n], this algorithm sorts the array by selecting the correct item to place in each position by moving sequentially through the elements of the array. In general, for each k = 1 to n 1, the kth step of the algorithm finds the index of the array item with minimum value from among a[k + 1], a[k + ], a[k + 3],..., a[n]. Once this index is found, the value of the corresponding array item is interchanged with the value of a[k]. At the end of execution the array elements are in order.] Input: n [a positive integer], a[1], a[], a[3],..., a[n] [an array of data items capable of being ordered] Algorithm Body: for k := 1 to n 1 IndexOfMin := k for i := k + 1 to n if (a[i] < a[indexofmin]) then IndexOfMin := i if IndexOfMin = k then Temp := a[k] a[k] := a[indexofmin] a[indexofmin] := Temp next k Output:,a[1], a[], a[3],..., a[n] [in ascending order]

12 750 Chapter 11 Analysis of Algorithm Efficiency The action of selection sort can be represented pictorially as follows: a[1] a[] a[k] a[k + 1] a[n] kth step: Find the index of the array element with minimum value from among a[k + 1],...,a[n] and interchange its value with the value of a[k]. 8. Construct a table showing the interchanges that occur when selection sort is applied to the array a[1] =5, a[] = 3, a[3] =4, a[4] =6, and a[5] =. 9. Construct a table showing the interchanges that occur when selection sort is applied to the array a[1] =6, a[] = 4, a[3] =5, a[4] =8, and a[5] = Construct a trace table showing the action of selection sort on the array of exercise Construct a trace table showing the action of selection sort on the array of exercise When selection sort is applied to the array of exercise 8, how many times is the comparisonintheif-then statement performed? 33. When selection sort is applied to the array of exercise 9, how many times is the comparison in the if-then statement performed? 34. When selection sort is applied to an array a[1], a[], a[3], a[4], how many times is the comparison in the if-then statement performed? 35. Consider applying selection sort to an array a[1], a[], a[3],...,a[n]. a. How many times is the comparison in the if-then statement performed when a[1] is compared to each of a[], a[3],...,a[n]? b. How many times is the comparison in the if-then statement performed when a[] is compared to each of a[3], a[4],...,a[n]? c. How many times is the comparison in the if-then statement performed when a[k] is compared to each of a[k 1], a[k + ],...,a[n]? H d. Using the number of times the comparison in the if-then statement is performed as a measure of the time efficiency of selection sort, find an order for selection sort. Use the theorem on polynomial orders. Exercises refer to the following algorithm to compute the value of a real polynomial. Algorithm Term-by-Term Polynomial Evaluation [This algorithm computes the value of the real polynomial a[n]x n + a[n 1]x n 1 + +a[]x + a[1]x + a[0] by computing each term separately, starting with a[0], and adding it on to an accumulating sum.] Input: n [a nonnegative integer] a[0], a[1], a[],...,a[n] [an array of real numbers], x [a real number] Algorithm Body: polyval := a[0] for i := 1 to n term := a[i] for j := 1 to i term := term x polyval := polyval + term [At this point polyval= a[n]x n + a[n 1]x n 1 + +a[]x + a[1]x + a[0].] Output: polyval [a real number] 36. Trace Algorithm for the input n = 3, a[0] =, a[1] =1, a[] = 1, a[3] =3, and x =. 37. Trace Algorithm for the input n =, a[0] = 5, a[1] = 1, a[] =, and x = Let s n = the number of additions and multiplications that must be performed when Algorithm is executed for a polynomial of degree n. Express s n as a function of n. 39. Use the theorem on on polynomial orders to find an order for Algorithm Exercises refer to another algorithm, known as Horner s rule, for finding the value of a real polynomial. Algorithm Horner s Rule [This algorithm computes the value of the real polynomial a[n]x n + a[n 1]x n 1 + +a[]x + a[1]x + a[0] by nesting successive additions and multiplications as indicated in the following parenthesization: (( ((a[n]x + a[n 1])x + a[n ])x + +a[])x + a[1])x + a[0]. At each stage, starting with a[n], the current value of polyval is multiplied by x and the next lower coefficient of the polynomial is added on.] Input: n [a nonnegative integer] a[0], a[1], a[],..., a[n] [an array of real numbers], x [a real number] Algorithm Body: polyval := a[n] for i := 1 to n polyval := polyval x + a[n i] [At this point polyval= a[n]x n + a[n 1]x n 1 + +a[]x + a[1]x + a[0].] Output: polyval [a real number]

13 11.4 Exponential and Logarithmic Functions: Graphs and Orders 751 H 40. Trace Algorithm for the input n = 3, a[0] =, a[1] =1, a[] = 1, a[3] =3, and x =. 41. Trace Algorithm for the input n =, a[0] = 5, a[1] = 1, a[] =, and x = Let t n = the number of additions and multiplications that must be performed when Algorithm is executed for a polynomial of degree n. Express t n as a function of n. 43. Use the theorem on polynomial orders to find an order for Algorithm How does this order compare with that of Algorithm ? Answers for Test Yourself 1. one iteration of the innermost loop. n 3. n ; n 11.4 Exponential and Logarithmic Functions: Graphs and Orders We ought never to allow ourselves to be persuaded of the truth of anything unless on the evidence of our own reason. René Descartes, Exponential and logarithmic functions are of great importance in mathematics in general and in computer science in particular. Several important computer algorithms have execution times that involve logarithmic functions of the size of the input data (which means they are relatively efficient for large data sets), and some have execution times that are exponential functions of the size of the input data (which means they are quite inefficient for large data sets). In addition, since exponential and logarithmic functions arise naturally in the descriptions of many growth and decay processes and in the computation of many kinds of probabilities, these functions are used in the analysis of computer operating systems, in queuing theory, and in the theory of information. Graphs of Exponential Functions As defined in Section 7., the exponential function with base b > 0 is the function that sends each real number x to b x. The graph of the exponential function with base (together with a partial table of its values) is shown in Figure Note that the values of this function increase with extraordinary rapidity. If we tried to continue drawing the graph using the scale shown in Figure , we would have to plot the point (10, 10 ) more than 1 feet above the horizontal axis. And the point (30, 30 ) wouldbelocated more than 610,080 miles above the axis well beyond the moon! x x y 7 y = x x Figure The Exponential Function with Base