As explained in "Building a Package", packages are managed using registries. There is a one local registry on your machine, and potentially many remote registries elsewhere "in the world". Use
list_packages to see the packages available on a registry:
$ python>>> import t4>>> t4.list_packages() # list local packagesPACKAGE TOPHASH CREATED SIZEnamespace/packagename:latest cac145b9c3dc just now 2.4 GBothernamespace/packagename:latest 95a134c80z48 14 days ago 2.4 GB>>> t4.list_packages("s3://my-bucket") # list remote packagesPACKAGE TOPHASH CREATED SIZEuser1/seattle-weather:latest cac145b9c3dc 1 hour ago 2.4 GBuser2/new-york-ballgames:latest 95a134c80z48 6 days ago 2.4 GB
To make a remote package and all of its data available locally,
import t4p = t4.Package.install("username/packagename","s3://your-bucket",)
Installing a package downloads all of the data and populates an entry for the package in your local registry.
You can omit
registry if you configure a default remote registry (this will persists between sessions):
t4.config(default_remote_registry='s3://your-bucket')# this now 'just works't4.Package.install("username/packagename")
Data files that you download are written to a folder in your local registry by default. You can specify an alternative destination using
Finally, you can install a specific version of a package by specifying the corresponding top hash:
An alternative to
browse downloads a package manifest without also downloading the data in the package.
import t4# load a package manifest from a remote registryp = t4.Package.browse("username/packagename", "s3://your-bucket")# load a package manifest from the default remote registryp = t4.Package.browse("username/packagename")# load a package manifest from the local registryp = t4.Package.browse("username/packagename", "local")
browse is advantageous when you don't want to download everything in a package at once. For example if you just want to look at a package's metadata.
You can import a local package from within Python:
from t4.data.username import packagename
This allows you to manage your data and code dependencies all in one place in your Python scripts or Jupyter notebooks.