Vizier

Download

Data On-boarding

Your client (or collaborator, or boss) just sent you a brand new dataset. You have hours to make sense of the data and bring it into the system.

Whether you prefer to use a spreadsheet, notebook, scripting language, or database, Vizier makes it easy to to explore the data to find out what you have, validate that the data makes sense, and transform it to fix bugs and mold it into a form your system can handle.

Data Life-cycles

You just got an updated dataset and your boss wants the data visible in your system yesterday. Just load it into your ETL workflows... what could go wrong?

Data dictionaries change all the time! There's no guarantee that today's data will look like yesterday's. Vizier makes it easy to re-use and adapt workflows for new data. It helps you validate new data, and quickly debug anything that goes wrong. Vizier even provides provenance for everything it creates, so it's got your back even if you discover problems downstream.

Notebooks like you've never seen them before

User-friendly, rendered task output in between steps, like in a notebook...

Staged Execution

User-friendly, rendered task output in between steps, like in a notebook...

... but mix and match languages in the same workflow (SQL, Python, or Scala). Communicate between languages through spark-backed data frames.

Multi-Lingual

... but mix and match languages in the same workflow (SQL, Python, or Scala). Communicate between languages through spark-backed data frames.

... or use Vizier

Modules

... or use Vizier's library of user-friendly modules to make cleaning and exploration easy.

Spreadsheets for those pesky, hard to reach values...

Edit data frames directly ... like with a spreadsheet.

Spreadsheets

Edit data frames directly ... like with a spreadsheet.

Your edits get recorded as a new script cell in the workflow.

Auditable

Your edits get recorded as a new script cell in the workflow.

Got big data? No problem! Vizier

Big-Data Friendly

Got big data? No problem! Vizier's spreadsheet is backed by Apache Spark.

Versioning without the sweat

Vizier records where you

Time Travel

Vizier records where you've been so you can easily go back.

Checkpoint your work to get feedback from others.

Snapshots

Checkpoint your work to get feedback from others.

Just trying out something new and different? Create a branch to keep the projects linked

Branch

Just trying out something new and different? Create a branch to keep the projects linked

Data unit testing baked in

Vizier includes Mimir, with support for data validating lenses.

Lenses

Vizier includes Mimir, with support for data validating lenses.

Get a report of all of the errors Mimir found in your data..

Error Reporting

Get a report of all of the errors Mimir found in your data..

... and jump right in to fix the error in Vizier

Jump-to-Error

... and jump right in to fix the error in Vizier's spreadsheet mode.

Video Tour

The Team

Mike Brachmann (UB)
Heiko Mueller (NYU)
Sonia Castelo (NYU)
Carlos Bautista (NYU)
Juliana Freire (NYU)
Boris Glavic (IIT)
Oliver Kennedy (UB)

Publications

Communicating Data Quality in On-Demand Curation QDB 2016 ( paper )

The Exception That Improves The Rule HILDA 2016 ( paper )

Provenance-aware Versioned Dataworkspaces TaPP 2016 ( paper )

Lenses: An On-Demand Approach to ETL VLDB ( paper )