CSE-250 Fall 2022 - Section B - Inductive Proofs, Divide and Conquer

Inductive Proofs, Divide and Conquer

CSE-250 Fall 2022 - Section B

Sept 26, 2022

Textbook: Ch. 15

Warm up...

Fibonacci

What's the complexity? (in terms of n)


    def fibb(n: Int): Long =
      if(n < 2){ 1 }
      else { fibb(n-1) + fibb(n-2) }
  

Fibonacci

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n < 2\\ T(n-1) + T(n-2) + \Theta(1) & \textbf{otherwise} \end{cases}$$

Test Hypothesis: $T(n) \in O(2^n)$

Divide and Conquer

Remember the Towers of Hanoi...

  1. You can move $n$ blocks, if you know how to move $n-1$ blocks
  2. You can move $n-1$ blocks, if you know how to move $n-2$ blocks
  3. You can move $n-2$ blocks, if you know how to move $n-3$ blocks
  4. You can move $n-3$ blocks, if you know how to move $n-4$ blocks

...

  • You can always move $1$ block

Divide and Conquer

To solve the problem at $n$:

  • Divide the problem into a problem at $n-1$
  • Conquer the size $n-1$ problems
  • Combine the size $n-1$ solutions

Merge Sort

Input: An array with elements in unknown order.

Output: An array with elements in sorted order.

Merge Sort

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Merge Sort


  def merge[A: Ordering](left: Seq[A], right: Seq[A]): Seq[A] = {
    val output = ArrayBuffer[A]()
  
    val leftItems = left.iterator.buffered
    val rightItems = right.iterator.buffered
  
    while(leftItems.hasNext || rightItems.hasNext) {
      if(!left.hasNext)          { output.append(right.next) }
      else if(!right.hasNext)    { output.append(left.next) }
      else if(Ordering[A].lt( left.head, right.head ))
                                 { output.append(left.next) }
      else                       { output.append(right.next) }
    }
    output.toSeq
  }
  

Merge Sort

Each time though loop advances either left or right.

Total Runtime: $\Theta(|\texttt{left}| + |\texttt{right}|)$

Merge Sort

Observation: Merging two sorted arrays can be done in $O(n)$.

Idea: Split the input in half, sort each half, and merge.

Merge Sort

Merge Sort

... but how do you sort each half?

Merge Sort

Divide and Conquer

To solve the problem at $n$:

  • Divide the problem into two problems of size $\frac{n}{2}$
  • Conquer the size $\frac{n}{2}$ problems
  • Combine the size $\frac{n}{2}$ solutions

Merge Sort

  1. If the sequence has $1$ or $0$ values: Sorted!
  2. If the sequence has $n > 1$ values:
    1. Sort values $1$ to $\left\lfloor\frac{n}{2}\right\rfloor-1$
    2. Sort values $\left\lfloor\frac{n}{2}\right\rfloor$ to $n$
    3. Merge sorted halves

Merge Sort


    def sort[A: Ordering](data: Seq[A]): Seq[A] = 
    {
      if(data.length <= 1) { return data }
      else {
        val (left, right) = data.splitAt(data.length / 2)
        return merge(
          sort(left),
          sort(right)
        )
      }
    }
  

Divide and Conquer

If we solve a problem of size $n$ by:

  • Dividing it into $a$ sub-problems
  • ...where each problem is of size $\frac{n}{b}$ (usually $b=a$)
  • ...and stop recurring at $n \leq c$
  • ...and the cost of dividing is $D(n)$
  • ...and the cost of combining is $C(n)$

The total cost will be: $$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq c \\ a\cdot T(\frac{n}{b}) + D(n) + C(n) & \textbf{otherwise} \end{cases}$$

Merge Sort

Divide: Split the sequence in half
$D(n) = \Theta(n)$ (can do in $\Theta(1)$)
Conquer: Sort left and right halves
$a = 2$, $b = 2$, $c = 1$
Combine: Merge halves together
$C(n) = \Theta(n)$

Merge Sort

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

How can we find a closed-form hypothesis?

Idea: Draw out the cost of each level of recursion.

Merge Sort: Recursion Tree

$$T(n) = \begin{cases} \Theta(1) & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise} \end{cases}$$

Each node of the tree shows $D(n) + C(n)$

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

Merge Sort: Recursion Tree

At level $i$ there are $2^i$ tasks, each with runtime $\Theta(\frac{n}{2^i})$,
and there are $\log(n)$ levels.

$\sum_{i=0}^{\log(n)}$ $\sum_{j=1}^{2^i}$ $\Theta(\frac{n}{2^i})$

Merge Sort: Recursion Tree

$$\sum_{i=0}^{\log(n)} \sum_{j=1}^{2^i} \Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} (2^i+1-1)\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} 2^i\Theta(\frac{n}{2^i})$$

$$\sum_{i=0}^{\log(n)} \Theta(n)$$

$$(\log(n) - 0 + 1) \Theta(n)$$

$$\Theta(n\log(n)) + \Theta(n)$$

$$\Theta(n\log(n))$$

Merge Sort: Proof By Induction

Now use induction to prove that there is a $c, n_0$
such that $T(n) \leq c \cdot n\log(n)$ for any $n > n_0$

$$T(n) = \begin{cases} c_0 & \textbf{if } n \leq 1 \\ 2\cdot T(\frac{n}{2}) + c_1 + c_2\cdot n & \textbf{otherwise} \end{cases}$$

Merge Sort: Proof By Induction

Base Case: $T(1) \leq c \cdot 1$

$$c_0 \leq c$$

True for any $c > c_0$

Merge Sort: Proof By Induction

Assume: $T(\frac{n}{2}) \leq c \frac{n}{2} \log\left(\frac{n}{2}\right)$

Show: $T(n) \leq c n \log\left(n\right)$

$$2\cdot T(\frac{n}{2}) + c_1 + c_2 n \leq c n \log(n)$$

By the assumption and transitivity, showing the following inequality suffices:
$$2 c \frac{n}{2} \log\left(\frac{n}{2}\right) + c_1 + c_2 n \leq c n \log(n)$$

$$c n \log(n) - c n \log(2) + c_1 + c_2 n \leq c n \log(n)$$

$$c_1 + c_2 n \leq c n \log(2)$$

$$\frac{c_1}{n \log(2)} + \frac{c_2}{\log(2)} \leq c$$

True for any $n_0 \geq \frac{c_1}{\log(2)}$ and $c > \frac{c_2}{\log(2)}+1$

Next time...

  • Quick Sort
  • Average Runtime