Introduction
Important Note: this project now lives in the
quiltdata/quilt
repository.
Overview
Rethinking S3: Announcing T4, a team data hub.
A team data hub for S3
T4 adds search, content preview, versioning, and a Python API to any S3 bucket
Every file in T4 is versioned and searchable
T4 is for data scientists, data engineers, and data-driven teams
Use cases
Collaborate - get everyone on the same page by pointing them all to the same immutable data version
Experiment faster - blob storage is schemaless and scalable, so iterations are quick
Recover, rollback, and reproduce with immutable packages
Understand what's in S3 - plaintext and faceted search over S3
Key features
Browse, search any S3 bucket
Preview images, Jupyter notebooks, Vega visualizations - without downloading
Read/write Python objects to and from S3
Immutable versions for objects, immutable packages for collections of objects
Components
/catalog
(JavaScript) - Search, browse, and preview your data in S3/api/python
- Read, write, and annotate Python objects in S3
Roadmap
Last updated