Vizier

Download

Data-Centric Notebooks

Vizier is a notebook that puts your data front-and-center.

Whether you prefer to use spreadsheets, notebook scripting, or databases, Vizier makes it easy to to explore the data to find out what you have, validate that the data makes sense, and transform it to fix bugs and mold it into a form your tools can use.

There is a lot of hate for some popular notebooks. Unlike most popular notebooks, Vizier is multi-lingual and multi-modal, letting you edit your data through the best interface for what you're trying to do. On top of that, it tracks provenance of your data and automatically versions your workflows. Vizier also uses dependency analysis to make sure you're never looking at stale outputs.

Screenshots Video Demo Docs Install

Run with Bootstrap

(Requires Java JDK 8)

# Install Vizier
$> wget https://maven.mimirdb.info/info/vizierdb/vizier
$> chmod +x vizier
$> sudo mv vizier /usr/local/bin

# Run Vizier
$> vizier

Detailed Instructions / FAQ

Run With Docker

(Requires Docker)

$> docker run \
          -p 5000:5000 \
          --name vizier \
          okennedy/vizier:latest

(Connect at http://localhost:5000)

A truly multi-lingual notebook.

Seamlessly switch between languages within the same notebook. Vizier currently supports Python, SQL, and Spark. Alternatively, add one of Vizier's pre-canned data widgets to plot data or repair errors with only a few clicks.

Staged Execution

User-friendly, rendered task output in between steps, like in a notebook...

Multi-Lingual

... but mix and match languages in the same workflow (SQL, Python, or Scala). Communicate between languages through spark-backed data frames.

Modules

... or use Vizier's library of user-friendly modules to make cleaning and exploration easy.

For small fixes, you can directly edit your data frames through Vizier's spreadsheet mode. Changes in spreadsheet mode are immediately reflected back into the notebook.

Spreadsheets

Edit data frames directly ... like with a spreadsheet.

Auditable

Your edits get recorded as a new script cell in the workflow.

Big-Data Friendly

Got big data? No problem! Vizier's spreadsheet is backed by Apache Spark.

The power of provenance

Vizier tracks where your data comes from and what's happened to it. This means Vizier knows which cells depend on which other cells. When you make a change, Vizier re-runs exactly those cells that are needed to make sure you're not looking at stale data. Vizier also tracks possible errors in your data tagging your data values when something goes wrong.

In-Order Execution

Vizier's dependency analysis makes sure you're never looking at stale results.

Lenses

Vizier includes Mimir, with support for data validating lenses.

Error Reporting

Get a report of all of the errors Mimir found in your data.

Versioning without the sweat

Everything you do in Vizier is versioned. Made a mistake? No problem, just go back to an earlier version. Share snapshots of your workflow with collaborators, or branch your workflow to try something different.

Time Travel

Vizier records where you've been so you can easily go back.

Snapshots

Checkpoint your work to get feedback from others.

Branch

Just trying out something new and different? Create a branch to keep the projects linked

Video Tour

Support

Vizier is supported by NSF Awards ACI-1640864 and IIS-1750460.

The Team

Mike Brachmann (UB)
Heiko Mueller (NYU)
Sonia Castelo (NYU)
Munaf Arshad Qazi (NYU)
Juliana Freire (NYU)
Boris Glavic (IIT)
Oliver Kennedy (UB)

Publications

Data Debugging and Exploration with Vizier SIGMOD 2019 (Demo) ( paper )

Uncertainty Annotated Databases - A Lightweight Approach for Approximating Certain Answers SIGMOD 2019 ( paper )

Adaptive Schema Databases CIDR 2017 ( paper )

Communicating Data Quality in On-Demand Curation QDB 2016 ( paper )

The Exception That Improves The Rule HILDA 2016 ( paper )

Provenance-aware Versioned Dataworkspaces TaPP 2016 ( paper )

Lenses: An On-Demand Approach to ETL VLDB ( paper )