4.1. Bambry CommandΒΆ

usage: bambry [-h] [-i ID] [-D] [-L] [-e] [-E] [-T] [-m] [-p PROCESSES]
              {new,variant,config,dump,set,info,sync,duplicate,clean,ingest,schema,run,build,finalize,package,checkin,update,install,exec,view,repopulate,import,export,extract,colmap,test,log,ui,notebook,copy}
              ...
Options:
-i, --id Bundle reference. May be an id, name, vid or vname. See also, bundle_ref
-D, --debug THE USR1 signal will break to interactive prompt
-L, --limited-run
 Enable bundle-specific behavior to reduce number of rows processed
-e, --echo Echo database queries.
-E, --exceptions
 Don’t capture and reformat exceptions; show the traceback on the console
-T, --trace Trace every line of program execution
-m, --multi Run in multiprocessing mode
-p, --processes
 Number of multiprocessing processors. implies -m
Sub-commands:
new

Create a new bundle

BEFORE NEW

usage: bambry new [-h] -s SOURCE -d DATASET [-b SUBSET] [-t TIME] [-p SPACE]
                  [-v VARIATION] [-n] [-k KEY]
                  ...
Positional arguments:
args Undocumented
Options:
-s, --source Source, usually a domain name
-d, --dataset Name of the dataset
-b, --subset Name of the subset
-t, --time Time period. Use ISO Time intervals where possible.
-p, --space Spatial extent name
-v, --variation
 Name of the variation
-n, --dryrun Dry run
-k, --key Number server key. One of ‘self’, ‘unregistered’, ‘registered’, ‘authority’ Use ‘self’ for a random, self-generated key.
variant

Create a variant of an existing bundle

usage: bambry variant [-h] [-s SOURCE] [-d DATASET] [-b SUBSET] [-t TIME]
                      [-p SPACE] [-v VARIATION] [-n] [-k KEY]
                      ref
Positional arguments:
ref Reference to an existing bundle
Options:
-s, --source Source, usually a domain name
-d, --dataset Name of the dataset
-b, --subset Name of the subset
-t, --time Time period. Use ISO Time intervals where possible.
-p, --space Spatial extent name
-v, --variation
 Name of the variation
-n, --dryrun Dry run
-k, --key Number server key. One of ‘self’, ‘unregistered’, ‘registered’, ‘authority’ Use ‘self’ for a random, self-generated key.
config

Operations on the bundle configuration file

usage: bambry config [-h] {} ...
dump

Dump records from the bundle database

usage: bambry dump [-h] [-c | -t | -f | -i | -s | -T | -p | -P | -m] [ref]
Positional arguments:
ref Specific object ID to dump
Options:
-c, --config Dump configs
-t, --dest_tables
 Dump destination tables, but not the table columns
-f, --files Dump bundle definition files
-i, --ingested List ingested data sources
-s, --sources Dump data source records
-T, --source_tables
 Dump source tables
-p, --partitions
 Dump partitions
-P, --pipes Dump destination tables
-m, --metadata Dump metadata as json
set

Set configuration and state values

usage: bambry set [-h] [-s STATE | -S]
Options:
-s, --state Set the build state
-S, --source-dir
 Set the source directory to the current directory
info

Print information about the bundle

usage: bambry info [-h] [-w] [-s] [-b] [-S] [-p] [-q] [-H] [-T] [ref]
Positional arguments:
ref Bundle reference
Options:
-w, --which Report the reference of the bundles that will be accessed by other commands
-s, --source_dir
 Display the source directory
-b, --build_dir
 Display the build directory
-S, --stats Also report column stats for partitions
-p, --partitions
 Also report partition details
-q, --quiet Just report the minimum information,
-H, --history When locating a bundle for –which, use only the edit history.
-T, --touch Update the edit history to this bundle
sync

Sync with a source representation

usage: bambry sync [-h] [-i] [-F] [-f] [-r] [-o] [-O] [-c] [file_name]
Positional arguments:
file_name File reference
Options:
-i, --in Sync from files to records, and records to objects
-F, --files Sync from files to records
-f, --force Force sync
-r, --records Sync from records to objects
-o, --out Sync from records to files
-O, --objects Sync from objects to records
-c, --code Sync bundle.py, bundle.yaml and other code files in, but not sources or schemas.
duplicate

Increment a bundles version number and create a new bundle

usage: bambry duplicate [-h] [-s] [ref]
Positional arguments:
ref Bundle reference
Options:
-s, --new-source-dir
 Assign a new source directory. Otherwise, use the current source directory
clean

Return bundle to state before build, prepare and extracts

usage: bambry clean [-h] [-a] [-s] [-F] [-i] [-t] [-p] [-b] [-B] [-y] [-f]
                    [ref]
Positional arguments:
ref Bundle reference
Options:
-a, --all Clean everything: metadata, partitions, tables, config, everything.
-s, --source Clean the source tables schema, but not ingested source files.
-F, --files Clean build source files
-i, --ingested Clean the ingested files
-t, --tables Clean destination tables
-p, --partitions
 Clean any built partitions
-b, --build Clean the build directory
-B, --build-state
 Clean the build state configuration
-y, --sync Sync in after cleaning
-f, --force Clean even built and finalized bundles
ingest

Build or install download and convert data to internal file format

usage: bambry ingest [-h] [-f] [-c] [-m] [-t TABLE] [-s SOURCE] [-S STAGE]
                     [-y]
                     [ref]
Positional arguments:
ref Bundle reference
Options:
-f, --force Force ingesting already ingested files
-c, --clean Clean ingested files first
-m, --load_meta
 If only one source, and it supports metadata, load the bundle title and summary from the source’s metadata
-t, --table Only run the schema for the named tables.
-s, --source Sources to ingest, instead of running all sources
-S, --stage Ingest sources at this stage
-y, --sync Sync first
schema

Generate destination schemas from the source schemas

usage: bambry schema [-h] [-s [SOURCE]] [-S] [-d] [-D] [-b] [-t TABLE] [-y]
                     [-o]
                     [ref]
Positional arguments:
ref Bundle reference
Options:
-s, --source Sources to build source schema for, instead of building all soruces
-S, --source-clean
 Build the source schema, cleaning the existing schema first
-d, --dest Build the destination schema, merging with the existing schema.
-D, --dest-clean
 Build the destination schema, cleaning the existing schema first
-b, --build For the destination schema, use the build process to determine the schema, not the source tables
-t, --table Build only the destination schemfor these tables.
-y, --sync Sync in first
-o, --sync-out Sync generated file out
run

Ingest, crate a schema, and build.

usage: bambry run [-h] [-f] [-c] [-y] [-q] [-s SOURCE] [-t TABLE] [-S STAGE]
                  [ref]
Positional arguments:
ref Bundle reference
Options:
-f, --force Build even built or finalized bundles
-c, --clean Clean and synchronize before running
-y, --sync Synchronize before and after
-q, --quick Just rebuild; don’t clean beforehand
-s, --source Sources to build, instead of running all sources
-t, --table Build only sources that output to these destination tables
-S, --stage Ingest sources at this stage
build

Build the data bundle and partitions

usage: bambry build [-h] [-f] [-y] [-c] [-s SOURCE] [-t TABLE] [-S STAGE]
                    [ref]
Positional arguments:
ref Bundle reference
Options:
-f, --force Build even built or finalized bundles
-y, --sync Synchronize before and after
-c, --clean Equivalent to bambry clean -y
-s, --source Sources to build, instead of running all sources
-t, --table Build only sources that output to these destination tables
-S, --stage Ingest sources at this stage
finalize

Finalize the bundle, preventing further changes

usage: bambry finalize [-h]
package

Package the bundle into a sqlite file.

usage: bambry package [-h] [-f] [-s] [-i]
Options:
-f, --force Rebuild if is already exists
-s, --source Only package source files
-i, --incver Increment the revision number
checkin

Commit the bundle to the remote store

usage: bambry checkin [-h] [-n] [-r REMOTE] [-s] [-f]
Options:
-n, --no-partitions
 Don’t check in partitions
-r, --remote Specify remote, rather than using default for bundle
-s, --source Only package source files
-f, --force Overwrite already uploaded files
update

Build the data bundle and partitions from an earlier version

usage: bambry update [-h] [-c] [-f]
Options:
-c, --clean Clean first
-f, --force Force build. ( –clean is usually preferred )
install

Install bundles and partitions to the library

usage: bambry install [-h] [-c] [-l LIBRARY] [-f--force]
Options:
-c, --clean Clean first
-l, --library Name of the library, defined in the config file
-f--force Force storing the file
exec

Execute a method on the bundle

usage: bambry exec [-h] [-c] [-f] [-y] Method [args [args ...]]
Positional arguments:
method Name of the method to run
args additional arguments
Options:
-c, --clean Clean first
-f, --force Force running on a built or finalized bundle
-y, --sync Syncrhonize before and after
view

View the datafile for a source or partition, using the ampr command

usage: bambry view [-h] [-m] [-s] [-S] [-t] [-H] [-T] [-r] [-R] [-j] [-c CSV]
                   [-l LIMIT]
                   path
Positional arguments:
path File path
Options:
-m, --meta Show metadata
-s, --schema Show the schema
-S, --stats Show the statistics
-t, --types Show type resolution information
-H, --head Display the first 10 records. Will only display 80 chars wide
-T, --tail Display the first last 10 records. Will only display 80 chars wide
-r, --records Output the records in tabular format
-R, --raw For the sample output, use the raw iterator
-j, --json Output the entire file as JSON
-c, --csv Output the entire file as CSV
-l, --limit The number of rows to output for CSV or JSON
repopulate

Load data previously submitted to the library back into the build dir

usage: bambry repopulate [-h]
import

Import a source bundle.

usage: bambry import [-h] [source]
Positional arguments:
source Bundle source directory or file
export

Export a source bundle.

usage: bambry export [-h] [-a] [-d] [source]
Positional arguments:
source Bundle source directory or file
Options:
-a, --append Append the source and bundle name to the path
-d, --defaults Write default files when there is no other content for file.
extract

Extract data from a bundle

usage: bambry extract [-h] [-l LIMIT] [partition] [directory]
Positional arguments:
partition Partition to extract
directory Output directory
Options:
-l, --limit Limit on number of rows per file
colmap

Create or load a column map

usage: bambry colmap [-h] (-c | -b | -l)
Options:
-c, --create Create the individual table maps
-b, --build Build the combined map from the table maps
-l, --load Load the combined map into the source tables
test

Run the bundle test code

usage: bambry test [-h] [tests [tests ...]]
Positional arguments:
tests Tests to run
log

Print out various logs

usage: bambry log [-h] [-e] [-p] [-s] [-a]
Options:
-e, --exceptions
 Print exceptions from the progress log
-p, --progress Display progress logs
-s, --stats Display states and counts of partitions and sources
-a, --all Display all records
ui

If the ambry-ui package is installed, start the web user interface

usage: bambry ui [-h] [-p PORT]
Options:
-p, --port Server port
notebook

If the ambry-ui package is installed, run Jupyter notebook on the bundle

usage: bambry notebook [-h]
copy

Copy partitions from another bundle into references in this one

usage: bambry copy [-h] ref
Positional arguments:
ref Reference to an existing bundle