> ## Documentation Index > Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt > Use this file to discover all available pages before exploring further. # Configuring Cloud Storage in LanceDB > Configure LanceDB to use S3, GCS, Azure Blob, and S3-compatible object stores with environment variables or storage options. When using LanceDB OSS, you can choose where to store your data. The tradeoffs between storage options are covered in the [storage architecture guide](https://docs.lancedb.com/storage.md). This page shows how to configure each backend. > [!NOTE] > > **LanceDB Enterprise storage configuration** > > In LanceDB Enterprise, you connect with `db://...` and the cluster owns the storage credentials, so `storage_options` are not passed at runtime. Cloud auth is set at deployment time. For federated databases, the namespace service vends per-request credentials automatically. See the [quickstart](https://docs.lancedb.com/quickstart.md), [Enterprise overview](https://docs.lancedb.com/enterprise.md), and [Azure deployment guide](https://docs.lancedb.com/enterprise/deployment/azure.md) for the Enterprise flow. ## Object stores LanceDB supports AWS S3 (and compatible stores), Azure Blob Storage, and Google Cloud Storage. The URI scheme in your `connect` call selects the backend. db = lancedb.connect("s3://bucket/path") async function storageConnectS3() \{ const db = await lancedb.connect("s3://bucket/path"); return db; } db = lancedb.connect("gs\://bucket/path") async function storageConnectGcs() \{ const db = await lancedb.connect("gs\://bucket/path"); return db; } db = lancedb.connect("az://bucket/path") async function storageConnectAzure() \{ const db = await lancedb.connect("az://bucket/path"); return db; } ### Configuration options When running inside the target cloud with correct IAM bindings, LanceDB often needs no extra configuration. When running elsewhere, provide credentials via environment variables or `storage_options`. db = lancedb.connect( "s3://bucket/path", storage\_options=\{"timeout": "60s"}, ) async function storageConnectTimeout() \{ const db = await lancedb.connect("s3://bucket/path", \{ storageOptions: \{ timeout: "60s" }, }); return db; } > [!NOTE] > > **Storage option casing** > > Keys are case-insensitive. Use lowercase in `storage_options` and uppercase in environment variables. Table-level `storage_options` inherit every key from the connection and override on a per-key basis. Pass them to `create_table` or `open_table` for options that should apply to a single table: table = db.create\_table( "table", \[\{"a": 1, "b": 2}], storage\_options=\{"timeout": "60s"}, ) async function storageTableTimeout() \{ const db = await lancedb.connect("s3://bucket/path"); const table = await db.createTable( "table", \[\{ a: 1, b: 2 }], \{ storageOptions: \{ timeout: "60s" } }, ); return table; } > [!TIP] > > **Inspect the effective options** > > On `AsyncTable`, `await table.initial_storage_options()` returns the options the table was opened with, and `await table.latest_storage_options()` returns the current options after any provider-driven refresh. The deprecated `table.storage_options()` method will be removed in a future release. #### General object store options | Key | Description | | :- | :- | | `allow_http` | Allow non-TLS connections. | | `allow_invalid_certificates` | Skip certificate validation for TLS connections. | | `connect_timeout` | Timeout for the connect phase. | | `timeout` | Timeout for the full request. | | `user_agent` | User agent string sent with requests. | | `proxy_url` | Proxy URL to route requests through. | | `proxy_ca_certificate` | PEM-formatted CA certificate for proxy connections. | | `proxy_excludes` | Comma-separated hosts that bypass the proxy (domains or CIDR). | | `download_retry_count` | Number of retries when downloading objects. | | `client_max_retries` | Maximum retries for object-store client requests. | | `client_retry_timeout` | Total retry timeout (seconds) for object-store client requests. | > [!NOTE] > > **Option support varies by backend** > > These are commonly used options. Cloud-specific keys (for example `region`, `endpoint`, `service_account`, and Azure credential keys) are backend-dependent and can be provided in `storage_options` as needed. #### New table configuration These options control the Lance file format and features used when creating new tables. Pass them via `storage_options` at connection or table level. They are evaluated only at table creation; setting them on an existing connection does not rewrite or alter tables that already exist. | Key | Values | Default | Description | | :- | :- | :- | :- | | `new_table_data_storage_version` | `legacy`, `stable` | `stable` | Lance file format version for new tables. Use `legacy` for backward compatibility with older clients, or `stable` for the current format with better performance. | | `new_table_enable_v2_manifest_paths` | `true`, `false` | `false` | Use v2 manifest path naming. Requires LanceDB >= 0.10.0 to read. | | `new_table_enable_stable_row_ids` | `true`, `false` | `false` | Keep row IDs stable across compaction, delete, and merge operations. | ```mermaid theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}} flowchart TD A[Creating a new table] --> B{Need backward compatibility\nwith older LanceDB clients?} B -->|Yes| C[new_table_data_storage_version: legacy] B -->|No| D[new_table_data_storage_version: stable\nDefault — recommended] D --> E{Need stable row IDs\nacross compaction and deletes?} E -->|Yes| F[new_table_enable_stable_row_ids: true] E -->|No| G[Default: false] D --> H{All clients on LanceDB >= 0.10.0?} H -->|Yes| I[new_table_enable_v2_manifest_paths: true] H -->|No| J[Default: false] ``` ```python Python icon="python" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}} import lancedb # Set the Lance file format version at connection level db = lancedb.connect( "s3://bucket/path", storage_options={ "new_table_data_storage_version": "stable", }, ) ``` ```typescript TypeScript icon="square-js" theme={"theme":{"light":"vitesse-light","dark":"catppuccin-mocha"}} import * as lancedb from "@lancedb/lancedb"; // Set the Lance file format version at connection level const db = await lancedb.connect("s3://bucket/path", { storageOptions: { newTableDataStorageVersion: "stable", }, }); ``` > [!WARNING] > > **Deprecated parameter** > > The `data_storage_version` parameter on `create_table()` is deprecated. Use `new_table_data_storage_version` in `storage_options` instead. ## AWS S3 ![](https://mintcdn.com/lancedb-bcbb4faf/0sS6vrpmM3KSVyss/static/assets/images/storage/aws.jpg?fit=max\&auto=format\&n=0sS6vrpmM3KSVyss\&q=85\&s=e5ca36574c5c78fb379e1ffb0a6836bc) Set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optionally `AWS_SESSION_TOKEN` as environment variables or pass them in `storage_options`. Region is optional for AWS but required for most S3-compatible stores. Minimum permissions usually include `s3:PutObject`, `s3:GetObject`, `s3:DeleteObject`, `s3:ListBucket`, and `s3:GetBucketLocation` scoped to the relevant bucket/prefix. ### S3-compatible stores db = lancedb.connect( "s3://bucket/path", storage\_options=\{ "region": "us-east-1", "endpoint": "http\://minio:9000", }, ) async function storageS3Minio() \{ const db = await lancedb.connect("s3://bucket/path", \{ storageOptions: \{ region: "us-east-1", endpoint: "http\://minio:9000", }, }); return db; } If the endpoint is `http://` (common in local development), also set `ALLOW_HTTP=true` or pass `allow_http=True` in `storage_options`. ### S3 Express db = lancedb.connect( "s3://my-bucket--use1-az4--x-s3/path", storage\_options=\{ "region": "us-east-1", "s3\_express": "true", }, ) async function storageS3Express() \{ const db = await lancedb.connect( "s3://my-bucket--use1-az4--x-s3/path", \{ storageOptions: \{ region: "us-east-1", s3Express: "true", }, }, ); return db; } Consult AWS networking requirements for S3 Express before enabling. > [!TIP] > > **Clean up failed multipart uploads** > > LanceDB aborts multipart uploads on graceful shutdown, but crashes can leave incomplete uploads. Add an S3 lifecycle rule to delete in-progress uploads after a few days. ### Server-side encryption with KMS To encrypt at rest with an AWS KMS key, set `aws_server_side_encryption` to `aws:kms` and `aws_sse_kms_key_id` to the key ID or ARN. The same options apply at connection or table level and combine with bucket-level default encryption. db = lancedb.connect( "s3://bucket/path", storage\_options=\{ "aws\_server\_side\_encryption": "aws:kms", "aws\_sse\_kms\_key\_id": "\", }, ) async function storageS3SseKms() \{ const db = await lancedb.connect("s3://bucket/path", \{ storageOptions: \{ awsServerSideEncryption: "aws:kms", awsSseKmsKeyId: "\", }, }); return db; } The IAM principal needs `kms:Encrypt`, `kms:Decrypt`, and `kms:GenerateDataKey` on the configured KMS key. ## Google Cloud Storage ![](https://mintcdn.com/lancedb-bcbb4faf/0sS6vrpmM3KSVyss/static/assets/images/storage/gcp.jpg?fit=max\&auto=format\&n=0sS6vrpmM3KSVyss\&q=85\&s=c6e235e8dc41623579d507a8dd7d60ef) Provide credentials via `GOOGLE_SERVICE_ACCOUNT` (path to JSON) or include the path in `storage_options`. GCS defaults to HTTP/1; set `HTTP1_ONLY=false` if you need HTTP/2. db = lancedb.connect( "gs\://my-bucket/my-database", storage\_options=\{ "service\_account": "path/to/service-account.json", }, ) async function storageGcsServiceAccount() \{ const db = await lancedb.connect( "gs\://my-bucket/my-database", \{ storageOptions: \{ serviceAccount: "path/to/service-account.json", }, }, ); return db; } ## Azure Blob Storage ![](https://mintcdn.com/lancedb-bcbb4faf/0sS6vrpmM3KSVyss/static/assets/images/storage/azure.jpg?fit=max\&auto=format\&n=0sS6vrpmM3KSVyss\&q=85\&s=2ab2a9af6d996eb0cd0ca1273738f360) Set `AZURE_STORAGE_ACCOUNT_NAME` and `AZURE_STORAGE_ACCOUNT_KEY` as environment variables, or pass them via `storage_options`. db = lancedb.connect( "az://my-container/my-database", storage\_options=\{ "account\_name": "some-account", "account\_key": "some-key", }, ) async function storageAzureAccount() \{ const db = await lancedb.connect( "az://my-container/my-database", \{ storageOptions: \{ accountName: "some-account", accountKey: "some-key", }, }, ); return db; } For SAS-token auth, set `azure_storage_account_name` and `azure_storage_sas_token`: db = lancedb.connect( "az://my-container/my-database", storage\_options=\{ "azure\_storage\_account\_name": "some-account", "azure\_storage\_sas\_token": "\", }, ) async function storageAzureSas() \{ const db = await lancedb.connect( "az://my-container/my-database", \{ storageOptions: \{ azureStorageAccountName: "some-account", azureStorageSasToken: "\", }, }, ); return db; } Other supported keys include service principal credentials (`azure_client_id`, `azure_client_secret`, `azure_tenant_id`), managed identities, and custom endpoints. ## Tigris Object Storage ![](https://mintcdn.com/lancedb-bcbb4faf/0sS6vrpmM3KSVyss/static/assets/images/storage/tigris.jpg?fit=max\&auto=format\&n=0sS6vrpmM3KSVyss\&q=85\&s=572dbfe7dd0a3e125b36f73b33df5fda) Tigris exposes an S3-compatible API. Configure the endpoint and region: db = lancedb.connect( "s3://your-bucket/path", storage\_options=\{ "endpoint": "https\://t3.storage.dev", "region": "auto", }, ) async function storageTigrisConnect() \{ const db = await lancedb.connect( "s3://your-bucket/path", \{ storageOptions: \{ endpoint: "https\://t3.storage.dev", region: "auto", }, }, ); return db; } Environment variables `AWS_ENDPOINT=https://t3.storage.dev` and `AWS_DEFAULT_REGION=auto` achieve the same configuration. ## Tencent COS [Tencent Cloud Object Storage (COS)](https://www.tencentcloud.com/products/cos) is the primary object store for workloads running in the China region. Use the `cos://` URI scheme to connect directly to a COS bucket. db = lancedb.connect( "cos\://my-bucket/my-database", storage\_options=\{ "secret\_id": "\", "secret\_key": "\", "region": "ap-guangzhou", }, ) Supported keys include `secret_id`, `secret_key`, `region`, and `endpoint`. You can also authenticate via the `TENCENTCLOUD_SECRET_ID` and `TENCENTCLOUD_SECRET_KEY` environment variables. > [!NOTE] > > **Availability** > > COS is bundled in the Python wheel by default. To use it from Rust or the Node binding, build with the `cos` Cargo feature enabled. ## GooseFS [GooseFS](https://www.tencentcloud.com/document/product/1424) is Tencent Cloud's distributed cache acceleration layer for COS and S3. It is a common choice when the same hot dataset is read repeatedly, such as vector search and AI training workloads. Connect using the `goosefs://` URI scheme. db = lancedb.connect("goosefs\://my-namespace/my-database") async function storageGoosefsConnect() \{ const db = await lancedb.connect("goosefs\://my-namespace/my-database"); return db; } GooseFS reads credentials and endpoint configuration from the GooseFS client environment. See the [GooseFS documentation](https://www.tencentcloud.com/document/product/1424) for cluster setup. > [!NOTE] > > **Availability** > > GooseFS is bundled by default in the Python wheel and the Node binding. To use it from Rust, build with the `goosefs` Cargo feature enabled. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.