We will revisit the issue of correctness of algorithms throughout the semester. Remember that termination of an algorithm is just as important a requirement for its correctness as that of producing correct results. There are some exceptions to this rule, but we will not mention those until we start discussing advanced topics. Until then, we will require correct algorithms to always terminate.
Some people will argue that algorithms are what define computer science or computing. Whether or not you agree with that view, it is clear that algorithms are at the heart of computing. Understanding algorithms, their design and analysis, is not only practically useful, but also scientifically interesting. Study of algorithms also gives rise to fundamental questions about what is computing and what are the limits of computing, which is a deep and interesting subject in its own right (even discounting its numerous practical implications). Do not underestimate the value of human curiosity!
Issues other than running time include memory footprint, energy consumption, number of I/O operations (when an algorithm involves explicit I/O), and size of the code.
Note that approximation works best when approximate solutions are acceptable. In other words, when there is not a unique solution to the problem, but there is a cost criterion that makes certain solutions more desirable than others. If you want sorted numbers then there is a unique solution. However, if you want the least cost path through a set of places then there might be multiple solutions that achieve the least cost and it makes sense to talk about an approximate solution that is a valid path, but may have higher than the least possible cost.
Insertion-Sort (A)
1 for j = 2 to A.length
2 key = A[j]
3 // Insert A[j] into the sorted sequence A[1..j-1].
4 i = j-1
5 while i > 0 and A[i] > key
6 A[i+1] = A[i]
7 i = i-1
8 A[i+1] = key
We are going to do this in three steps (note similarity with induction).
Insertion-Sort (A)
|
cost | times |
1 for j = 2 to A.length |
c1 | n |
2 key = A[j] |
c2 | n−1 |
3 // Insert A[j] into the sorted sequence A[1..j−1]. |
0 | n−1 |
4 i = j-1 |
c4 | n−1 |
5 while i > 0 and A[i] > key |
c5 | ∑nj=2 tj |
6 A[i+1] = A[i] |
c6 | ∑nj=2 (tj−1) |
7 i = i-1 |
c7 | ∑nj=2 (tj−1) |
8 A[i+1] = key |
c8 | n−1 |
| T(n) = | c1n + c2(n−1) + c4(n−1) + |
| c5 ∑nj=2 tj + c6 ∑nj=2 (tj−1) + c7 ∑nj=2 (tj−1) + | |
| c8(n−1) |
| Best case running time | ||
| T(n) | = | c1n + c2(n−1) + c4(n−1) + c5(n−1) + c8(n−1) |
| = | (c1 + c2 + c4 + c5 + c8)n - (c2 + c4 + c5 + c8) |
| Worst case running time | ||
| T(n) | = | c1n + c2(n−1) + c4(n−1) + c5(n(n+1)2 − 1) + |
| c6(n(n−1)2) + c7(n(n−1)2) + c8(n−1) | ||
| = | (c52 + c62 + c72) n2 | |
| + (c1 + c2 + c4 + c62 + c62 − c72 + c8) n | ||
| − (c2 + c4 + c5 + c8) | ||
| = | O(n2) |
The kind of detailed step-by-step analysis we did for this example is often an overkill. For example, we can simplify the calculation by assuming that each constant is 1. Another simplification is to only consider the highest degree term and ignore the rest. Asymptotically, the highest degree term dominates the rest.
| T(n) = { | Θ(1) | if n = 1 |
| 2T(n/2) + Θ(n) | if n > 1 |