> ## Documentation Index
> Fetch the complete documentation index at: https://docs.minerva.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Storage — bulk upload & download

> Move files in and out of your private storage area without writing boto3 by hand. Gated on the [storage] extra.

`mc.storage` is a thin wrapper around your private S3 storage area with two
well-known directories:

* `Incoming/` — files **you send** in.
* `Outgoing/` — files **placed for you** to read.

`upload*` defaults to `Incoming/`; `download*`, `list*`, and `delete` default
to `Outgoing/` so the natural list-then-act loop targets the same direction.
Override per call with `direction="incoming"` / `direction="outgoing"`.

```bash theme={null}
pip install "minerva-sdk[storage]"
```

```python theme={null}
from minerva import Minerva

mc = Minerva()  # MINERVA_API_KEY from env
```

## Single file

```python theme={null}
# Upload — defaults to Incoming/
mc.storage.upload("data/customers.csv")
# → Incoming/customers.csv

mc.storage.upload("data/customers.csv", remote_name="batch-001.csv")
# → Incoming/batch-001.csv

mc.storage.upload("data/customers.csv", subdir="2026-06-29")
# → Incoming/2026-06-29/customers.csv

# Download — defaults to Outgoing/
mc.storage.download("results.csv")
# → ./results.csv

mc.storage.download("sub/inner.csv", local_path="out/inner.csv")
# → ./out/inner.csv  (creates parent dirs)

# Delete — defaults to Outgoing/. Pass direction="incoming" to clean up your own uploads.
mc.storage.delete("results.csv")
mc.storage.delete("batch-001.csv", direction="incoming")
```

`upload` returns the relative remote key. `download` returns the local
`Path` that was written.

## Listing

```python theme={null}
for obj in mc.storage.list():           # Outgoing/ by default
    print(obj.key, obj.size, obj.last_modified)

for obj in mc.storage.list(subdir="2026-06"):
    ...
```

Each `ObjectMeta` carries `key` (relative — pass back to `download` /
`delete` as-is), `size`, `last_modified`, and `etag`.

## Bulk

Bulk methods run uploads/downloads in parallel — defaults to 8 workers,
tune with `max_concurrency=`. Order is preserved; the first error in a
batch fails the call.

```python theme={null}
# Many files at once
mc.storage.upload_many(["a.csv", "b.csv", "c.csv"])

# Whole directory tree, relative paths preserved
mc.storage.upload_directory("./batch-2026-06-29")
# Sends ./batch-2026-06-29/x.csv     → Incoming/x.csv
# Sends ./batch-2026-06-29/sub/y.csv → Incoming/sub/y.csv

mc.storage.upload_directory("./day", subdir="2026-06-29")
# → Incoming/2026-06-29/...

# Pull a batch by keys (typically the output of list())
mc.storage.download_many(
    ["results-1.csv", "sub/inner.csv"],
    local_dir="./out",
)

# List, then download everything — single call
mc.storage.download_all(local_dir="./out")
```

## Credentials

Storage credentials are fetched lazily on first call and cached on the
client. The SDK auto-refreshes them when they're within \~2 minutes of
expiry, so a long-running process can hold one `Minerva()` instance and
keep working across hour-long sessions without manual renewal.

Construct **one `Minerva()` per process and reuse it** — every fresh
instance pays the credential round-trip again.

```python theme={null}
creds = mc.storage.get_credentials()
print(creds.bucket_name, creds.prefix, creds.aws_session_expiration)
# Secrets (aws_secret_access_key, aws_session_token) are hidden from repr
# but accessible by attribute.
```

## Access scope

Storage credentials are scoped to your org's directory — the underlying
session policy only permits `s3:GetObject` / `s3:PutObject` /
`s3:DeleteObject` on `{your_org}/*` keys, and `s3:ListBucket` filtered to
the same prefix. Other orgs' data is unreachable regardless of what you
pass to the SDK.

The SDK additionally rejects path-traversal-shaped keys
(`..` segments, absolute paths, backslashes) at the call site for clean
local errors — but the real isolation is server-side.

## Errors

All storage errors derive from `MinervaStorageError`:

| Condition                                                                               | Raises                                               |
| --------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| `[storage]` extra not installed (`boto3` missing)                                       | `MinervaStorageExtraNotInstalledError`               |
| Credentials endpoint returned an unexpected shape, missing fields, or unparsable expiry | `MinervaStorageCredentialError`                      |
| Object doesn't exist (404)                                                              | `MinervaStorageObjectNotFoundError` — carries `.key` |
| Access denied (403)                                                                     | `MinervaStorageAccessDeniedError` — carries `.key`   |
| Network / transient S3 failure (5xx, timeouts, …)                                       | `MinervaStorageTransferError` — carries `.original`  |
| Bad input (non-file path, `..` in key, invalid `direction`, …)                          | `MinervaValidationError`                             |

```python theme={null}
from minerva import (
    MinervaStorageError,
    MinervaStorageObjectNotFoundError,
    MinervaStorageAccessDeniedError,
    MinervaStorageTransferError,
)

try:
    mc.storage.download("missing.csv")
except MinervaStorageObjectNotFoundError as e:
    print(f"no such key: {e.key}")
except MinervaStorageError:
    raise
```

<Tip>
  The `[storage]` extra pulls in `boto3` so the base wheel stays small. If
  you already have `boto3` in your environment, the extra still wires up
  the SDK's lazy-import guard cleanly — install it either way.
</Tip>