> ## Documentation Index > Fetch the complete documentation index at: https://docs.lancedb.com/llms.txt > Use this file to discover all available pages before exploring further. # Multimodal Data (Blobs) > Learn how to store and query multimodal data (images, audio, video) directly in LanceDB using binary columns. LanceDB handles multimodal data—images, audio, video, and PDF files—natively by storing the raw bytes in a binary column alongside your vectors and metadata. This approach simplifies your data infrastructure by keeping the raw assets and their embeddings in the same database, eliminating the need for separate object storage for many use cases. This guide demonstrates how to ingest, store, and retrieve image data using standard binary columns, and also introduces the **Lance Blob API** for optimized handling of larger multimodal files. ## Store binary data To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust). ### 1. Setup and imports First, import the necessary libraries for LanceDB and Arrow in your SDK. import lancedb import pyarrow as pa import pandas as pd import numpy as np import io from PIL import Image import \* as arrow from "apache-arrow"; import \{ Buffer } from "node:buffer"; import \* as lancedb from "@lancedb/lancedb"; use std::collections::HashMap; use std::sync::Arc; use arrow\_array::types::Float32Type; use arrow\_array::\{ BinaryArray, FixedSizeListArray, Int32Array, Int64Array, LargeBinaryArray, RecordBatch, RecordBatchIterator, StringArray, }; use arrow\_schema::\{DataType, Field, Schema}; use futures\_util::TryStreamExt; use lancedb::connect; use lancedb::database::CreateTableMode; use lancedb::query::\{ExecutableQuery, QueryBase}; ### 2. Prepare data For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object. \# Create some dummy images def create\_dummy\_image(color): img = Image.new('RGB', (100, 100), color=color) buf = io.BytesIO() img.save(buf, format='PNG') return buf.getvalue() \# Create dataset with metadata, vectors, and image blobs data = \[ \{ "id": 1, "filename": "red\_square.png", "vector": np.random.rand(128).astype(np.float32), "image\_blob": create\_dummy\_image('red'), "label": "red" }, \{ "id": 2, "filename": "blue\_square.png", "vector": np.random.rand(128).astype(np.float32), "image\_blob": create\_dummy\_image('blue'), "label": "blue" } ] const createDummyImage = (color: string): Uint8Array => \{ const pngHeader = Uint8Array.from(\[137, 80, 78, 71, 13, 10, 26, 10]); return Buffer.concat(\[Buffer.from(pngHeader), Buffer.from(color, "utf8")]); }; const data = \[ \{ id: 1, filename: "red\_square.png", vector: Array.from(\{ length: 128 }, (\_, i) => (i % 16) / 16), image\_blob: createDummyImage("red"), label: "red", }, \{ id: 2, filename: "blue\_square.png", vector: Array.from(\{ length: 128 }, (\_, i) => ((i + 8) % 16) / 16), image\_blob: createDummyImage("blue"), label: "blue", }, ]; let create\_dummy\_image = |color: u8| -> Vec\ \{ let mut png\_like = vec!\[137, 80, 78, 71, 13, 10, 26, 10]; png\_like.push(color); png\_like }; let data = vec!\[ ( 1\_i32, "red\_square.png", vec!\[0.1\_f32; 128], create\_dummy\_image(1), "red", ), ( 2\_i32, "blue\_square.png", vec!\[0.2\_f32; 128], create\_dummy\_image(2), "blue", ), ]; ### 3. Define the schema When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list. \# Define schema explictly to ensure image\_blob is treated as binary schema = pa.schema(\[ pa.field("id", pa.int32()), pa.field("filename", pa.string()), pa.field("vector", pa.list\_(pa.float32(), 128)), pa.field("image\_blob", pa.binary()), # Important: Use pa.binary() for blobs pa.field("label", pa.string()) ]) const schema = new arrow\.Schema(\[ new arrow\.Field("id", new arrow\.Int32()), new arrow\.Field("filename", new arrow\.Utf8()), new arrow\.Field( "vector", new arrow\.FixedSizeList( 128, new arrow\.Field("item", new arrow\.Float32(), true), ), ), new arrow\.Field("image\_blob", new arrow\.Binary()), new arrow\.Field("label", new arrow\.Utf8()), ]); let schema = Schema::new(vec!\[ Field::new("id", DataType::Int32, false), Field::new("filename", DataType::Utf8, false), Field::new( "vector", DataType::FixedSizeList(Arc::new(Field::new("item", DataType::Float32, true)), 128), false, ), Field::new("image\_blob", DataType::Binary, false), Field::new("label", DataType::Utf8, false), ]); ### 4. Ingest data Now, create the table using the data and the defined schema. tbl = db.create\_table("images", data=data, schema=schema, mode="overwrite") const multimodalData = lancedb.makeArrowTable(data, \{ schema }); const tbl = await db.createTable("images", multimodalData, \{ mode: "overwrite", }); let schema = Arc::new(schema); let image\_batch = RecordBatch::try\_new( schema.clone(), vec!\[ Arc::new(Int32Array::from\_iter\_values(data.iter().map(|row| row\.0))), Arc::new(StringArray::from\_iter\_values(data.iter().map(|row| row\.1))), Arc::new( FixedSizeListArray::from\_iter\_primitive::\( data.iter() .map(|row| Some(row\.2.iter().copied().map(Some).collect::\>())), 128, ), ), Arc::new(BinaryArray::from\_iter\_values( data.iter().map(|row| row\.3.as\_slice()), )), Arc::new(StringArray::from\_iter\_values(data.iter().map(|row| row\.4))), ], ) .unwrap(); let image\_reader: Box\ = Box::new(RecordBatchIterator::new(vec!\[Ok(image\_batch)].into\_iter(), schema.clone())); let table = db .create\_table("images", image\_reader) .mode(CreateTableMode::Overwrite) .execute() .await .unwrap(); ## Retrieve and use blobs When you search your LanceDB table, you can retrieve the binary column just like any other metadata. \# Search for similar images query\_vector = np.random.rand(128).astype(np.float32) results = tbl.search(query\_vector).limit(1).to\_pandas() const queryVector = Array.from(\{ length: 128 }, (\_, i) => (i % 16) / 16); const results = await tbl.search(queryVector).limit(1).toArray(); let query\_vector = vec!\[0.1\_f32; 128]; let results = table .query() .nearest\_to(query\_vector) .unwrap() .limit(1) .execute() .await .unwrap() .try\_collect::\>() .await .unwrap(); ### Convert bytes back to objects Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer). \# Convert back to PIL Image for \_, row in results.iterrows(): image\_bytes = row\['image\_blob'] image = Image.open(io.BytesIO(image\_bytes)) print(f"Retrieved image: \{row\['filename']}, Size: \{image.size}") \# You can now use 'image' with other libraries or display it for (const row of results) \{ const imageBytes = row\.image\_blob as Uint8Array; console.log( \`Retrieved image: \$\{row\.filename}, Byte length: \$\{imageBytes.length}\`, ); } for batch in \&results \{ let filenames = batch .column\_by\_name("filename") .unwrap() .as\_any() .downcast\_ref::\() .unwrap(); let images = batch .column\_by\_name("image\_blob") .unwrap() .as\_any() .downcast\_ref::\() .unwrap(); for row in 0..batch.num\_rows() \{ let image\_bytes = images.value(row); println!( "Retrieved image: \{}, Byte length: \{}", filenames.value(row), image\_bytes.len() ); } } ## Large Blobs (Blob API) For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront. ### 1. Define a blob schema To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata. import pyarrow as pa \# Define schema with Blob API metadata for lazy loading schema = pa.schema(\[ pa.field("id", pa.int64()), pa.field( "video", pa.large\_binary(), metadata=\{"lance-encoding:blob": "true"} # Enable Blob API ), ]) const blobSchema = new arrow\.Schema(\[ new arrow\.Field("id", new arrow\.Int64()), new arrow\.Field( "video", new arrow\.LargeBinary(), true, new Map(\[\["lance-encoding:blob", "true"]]), ), ]); let blob\_metadata = HashMap::from(\[( "lance-encoding:blob".to\_string(), "true".to\_string(), )]); let blob\_schema = Schema::new(vec!\[ Field::new("id", DataType::Int64, false), Field::new("video", DataType::LargeBinary, true).with\_metadata(blob\_metadata), ]); ### 2. Ingest large blobs You can then ingest data normally, and Lance will handle the optimized storage. import lancedb db = lancedb.connect(db\_path\_factory("blob\_db")) \# Create sample data data = \[ \{"id": 1, "video": b"fake\_video\_bytes\_1"}, \{"id": 2, "video": b"fake\_video\_bytes\_2"} ] \# Create the table tbl = db.create\_table("videos", data=data, schema=schema) const blobData = lancedb.makeArrowTable( \[ \{ id: 1, video: Buffer.from("fake\_video\_bytes\_1") }, \{ id: 2, video: Buffer.from("fake\_video\_bytes\_2") }, ], \{ schema: blobSchema }, ); const blobTable = await db.createTable("videos", blobData, \{ mode: "overwrite", }); let blob\_rows = vec!\[ (1\_i64, b"fake\_video\_bytes\_1".to\_vec()), (2\_i64, b"fake\_video\_bytes\_2".to\_vec()), ]; let blob\_schema = Arc::new(blob\_schema); let blob\_batch = RecordBatch::try\_new( blob\_schema.clone(), vec!\[ Arc::new(Int64Array::from\_iter\_values(blob\_rows.iter().map(|row| row\.0))), Arc::new(LargeBinaryArray::from\_iter\_values( blob\_rows.iter().map(|row| row\.1.as\_slice()), )), ], ) .unwrap(); let blob\_reader: Box\ = Box::new(RecordBatchIterator::new(vec!\[Ok(blob\_batch)].into\_iter(), blob\_schema)); let blob\_table = db .create\_table("videos", blob\_reader) .mode(CreateTableMode::Overwrite) .execute() .await .unwrap(); > For more advanced usage, including random access and file-like reading of blobs, see the > Lance format's [blob API documentation](https://lance.org/guide/blob/). ### 3. Convert blob tables to pandas When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`. `blob_mode` accepts: * `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path. * `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process. * `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob. `"bytes"` and `"descriptions"` require a filesystem-backed Lance dataset and are not supported on in-memory tables. Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`: \# Default: blob columns come back lazily df\_lazy = tbl.to\_pandas() \# Materialize blob bytes eagerly df\_bytes = tbl.to\_pandas(blob\_mode="bytes") \# Return descriptors instead of payloads df\_desc = tbl.to\_pandas(blob\_mode="descriptions") \# Forward extra kwargs to PyArrow's to\_pandas df\_typed = tbl.to\_pandas(split\_blocks=True, self\_destruct=True) Query builders also accept `blob_mode` on their `to_pandas()` method: * Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`. * Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes. * This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded. \# Plain scan query: blob\_mode is supported end to end df\_lazy = ( tbl.search() .where("id = 1") .select(\["id", "video"]) .to\_pandas(blob\_mode="lazy") ) \# Same call shape works on async query builders df\_bytes = await ( tbl\_async.query() .where("id = 1") .select(\["id", "video"]) .to\_pandas(blob\_mode="bytes") ) \# Vector / FTS / hybrid queries can't materialize blob columns, \# so omit them from the projection df\_vec = ( tbl.search(query\_vector) .limit(10) .select(\["id", "vector"]) .to\_pandas(split\_blocks=True, self\_destruct=True) ) ## Other modalities The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data: * **Audio:** Read `.wav` or `.mp3` files as bytes. * **Video:** Store video transitions or full clips using the Blob API. * **PDFs/Documents:** Store the raw file content for document search. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.