What happens when you don't know your data precisely?

      SELECT * FROM Posts WHERE image_class = 'Cat';
    

      SELECT COUNT(*) FROM Posts WHERE image_class = 'Cat';
    

      SELECT user_id FROM Posts
      WHERE image_class = 'Cat'
      GROUP BY user_id HAVING COUNT(*) > 10;
    

Incomplete Databases
↓

Probabilistic Databases

1. Representing Incompleteness
2. Querying Incomplete Data
3. Implementing It

Incomplete Database ($\mathcal D$): A set of possible worlds

Possible World ($D \in \mathcal D$): One (of many) database instances

(Require all possible worlds to have the same schema)

What does it mean to run a query on an incomplete database?

$Q(\mathcal D) = ?$

$Q(\mathcal D) = \{\;Q(D)\;|\;D \in \mathcal D \}$

$$Q_1 = \pi_{Name}\big( \sigma_{state = \texttt{'NY'}} (R \bowtie_{zip} ZipLookups) \big)$$

$$Q_2 = \pi_{Name}\big( \sigma_{region = \texttt{'Northeast'}} (R \bowtie_{zip} ZipLookups) \big)$$

$$Q_2 = \pi_{Name}\big( \sigma_{region = \texttt{'Northeast'}} (R \bowtie_{zip} ZipLookups) \big)$$

Challenge: There can be lots of possible worlds.

Observation: Possibilities for database creation break down into lots of independent choices.

Factorize the database.

Alice appears in both databases.
The only differences are Bob and Carol's zip codes.

List Out Choices

• $\texttt{bob}$$ \in \{ 4, 9 \}$ (Bob's zip code digit)
• $\texttt{carol}$$ \in \{ 3, 8 \}$ (Carol's zip code digit)

Pick one of each: $\big[\;\{a\},\; \{b, c\},\; \{d, e\}\;\big]$

Set those variables to $T$ and all others to $F$

$R_1 \equiv \big[a \rightarrow T, b \rightarrow T, d \rightarrow T, * \rightarrow F\big]$

Use provenance as before...

... but what about aggregates?

                SELECT COUNT(*) 
                FROM R NATURAL JOIN ZipCodeLookup 
                WHERE State = 'NY'
    

$$= \begin{cases} 1 & \textbf{if } \texttt{bob} = 9 \wedge \texttt{carol} = 8\\ 2 & \textbf{if } \texttt{bob} = 4 \wedge \texttt{carol} = 8 \\&\; \vee\; \texttt{bob} = 9 \wedge \texttt{carol} = 3\\ 3 & \textbf{if } \texttt{bob} = 4 \wedge \texttt{carol} = 3 \end{cases}$$

Problem: A combinatorial explosion of possibilities

Idea: Simplify the problem

1. Is a particular tuple Possible?
2. Is a particular tuple Certain?

Certain Tuple

A tuple that appears in all possible worlds

$\forall D \in \mathcal D : t \in D$

Possible Tuple

A tuple that appears in at least one possible world

$\exists D \in \mathcal D : t \in D$

Non-aggregate queries

Is a tuple Certain?

Is the provenance polynomial a tautology?

Is a tuple Possible?

Is the provenance polynomial a contradiction?

Pick your favorite SAT solver, plug in and go

Aggregate queries

As before, factorize the possible outcomes

$$1 + \{\;1\;\textbf{if}\;\texttt{bob} = 4\;\} + \{\;1\;\textbf{if}\;\texttt{carol} = 3\;\}$$

Not bigger than the aggregate input...

...but at least it only reduces to bin-packing
(or a similarly known NP problem.)