Storage — bulk upload & download - Minerva Documentation

mc.storage is a thin wrapper around your private S3 storage area with two well-known directories:

Incoming/ — files you send in.
Outgoing/ — files placed for you to read.

upload* defaults to Incoming/; download*, list*, and delete default to Outgoing/ so the natural list-then-act loop targets the same direction. Override per call with direction="incoming" / direction="outgoing".

pip install "minerva-sdk[storage]"

from minerva import Minerva

mc = Minerva()  # MINERVA_API_KEY from env

Single file

# Upload — defaults to Incoming/
mc.storage.upload("data/customers.csv")
# → Incoming/customers.csv

mc.storage.upload("data/customers.csv", remote_name="batch-001.csv")
# → Incoming/batch-001.csv

mc.storage.upload("data/customers.csv", subdir="2026-06-29")
# → Incoming/2026-06-29/customers.csv

# Download — defaults to Outgoing/
mc.storage.download("results.csv")
# → ./results.csv

mc.storage.download("sub/inner.csv", local_path="out/inner.csv")
# → ./out/inner.csv  (creates parent dirs)

# Delete — defaults to Outgoing/. Pass direction="incoming" to clean up your own uploads.
mc.storage.delete("results.csv")
mc.storage.delete("batch-001.csv", direction="incoming")

upload returns the relative remote key. download returns the local Path that was written.

Listing

for obj in mc.storage.list():           # Outgoing/ by default
    print(obj.key, obj.size, obj.last_modified)

for obj in mc.storage.list(subdir="2026-06"):
    ...

Each ObjectMeta carries key (relative — pass back to download / delete as-is), size, last_modified, and etag.

Bulk

Bulk methods run uploads/downloads in parallel — defaults to 8 workers, tune with max_concurrency=. Order is preserved; the first error in a batch fails the call.

# Many files at once
mc.storage.upload_many(["a.csv", "b.csv", "c.csv"])

# Whole directory tree, relative paths preserved
mc.storage.upload_directory("./batch-2026-06-29")
# Sends ./batch-2026-06-29/x.csv     → Incoming/x.csv
# Sends ./batch-2026-06-29/sub/y.csv → Incoming/sub/y.csv

mc.storage.upload_directory("./day", subdir="2026-06-29")
# → Incoming/2026-06-29/...

# Pull a batch by keys (typically the output of list())
mc.storage.download_many(
    ["results-1.csv", "sub/inner.csv"],
    local_dir="./out",
)

# List, then download everything — single call
mc.storage.download_all(local_dir="./out")

Credentials

Storage credentials are fetched lazily on first call and cached on the client. The SDK auto-refreshes them when they’re within ~2 minutes of expiry, so a long-running process can hold one Minerva() instance and keep working across hour-long sessions without manual renewal. Construct one Minerva() per process and reuse it — every fresh instance pays the credential round-trip again.

creds = mc.storage.get_credentials()
print(creds.bucket_name, creds.prefix, creds.aws_session_expiration)
# Secrets (aws_secret_access_key, aws_session_token) are hidden from repr
# but accessible by attribute.

Access scope

Storage credentials are scoped to your org’s directory — the underlying session policy only permits s3:GetObject / s3:PutObject / s3:DeleteObject on {your_org}/* keys, and s3:ListBucket filtered to the same prefix. Other orgs’ data is unreachable regardless of what you pass to the SDK. The SDK additionally rejects path-traversal-shaped keys (.. segments, absolute paths, backslashes) at the call site for clean local errors — but the real isolation is server-side.

Errors

All storage errors derive from MinervaStorageError:

Condition	Raises
`[storage]` extra not installed (`boto3` missing)	`MinervaStorageExtraNotInstalledError`
Credentials endpoint returned an unexpected shape, missing fields, or unparsable expiry	`MinervaStorageCredentialError`
Object doesn’t exist (404)	`MinervaStorageObjectNotFoundError` — carries `.key`
Access denied (403)	`MinervaStorageAccessDeniedError` — carries `.key`
Network / transient S3 failure (5xx, timeouts, …)	`MinervaStorageTransferError` — carries `.original`
Bad input (non-file path, `..` in key, invalid `direction`, …)	`MinervaValidationError`

from minerva import (
    MinervaStorageError,
    MinervaStorageObjectNotFoundError,
    MinervaStorageAccessDeniedError,
    MinervaStorageTransferError,
)

try:
    mc.storage.download("missing.csv")
except MinervaStorageObjectNotFoundError as e:
    print(f"no such key: {e.key}")
except MinervaStorageError:
    raise

The [storage] extra pulls in boto3 so the base wheel stays small. If you already have boto3 in your environment, the extra still wires up the SDK’s lazy-import guard cleanly — install it either way.

Getting started

Endpoints

Power features

Reference

Storage — bulk upload & download

Single file

Listing

Bulk

Credentials

Access scope

Errors

​Single file

​Listing

​Bulk

​Credentials

​Access scope

​Errors

Single file

Listing

Bulk

Credentials

Access scope

Errors