Async API¶

The remote_store.aio module provides native async/await support for store operations. See Store for the synchronous counterpart.

AsyncStore¶

AsyncStore ¶

AsyncStore(
    backend: AsyncBackend | Backend, root_path: str = ""
)

An async logical remote folder scoped to a root path.

All path arguments are validated and prefixed with root_path before being delegated to the backend. Supports the async context-manager protocol (async with AsyncStore(...) as s:) which calls aclose() on exit.

Parameters:

backend (AsyncBackend | Backend) –

Async or sync backend instance. Sync backends are auto-wrapped via SyncBackendAdapter.
root_path (str, default: '' ) –

Prefix prepended to every path. "" means the backend root.

Async counterpart to Store

Same methods, same errors, same capability model. See the Async Store Guide for usage patterns and Store for the synchronous counterpart.

Thread safety

AsyncStore is immutable after construction and can be shared across tasks on the same event loop. Backend thread safety depends on the backend implementation.

Reading¶

Requires Capability.READ

All read methods raise CapabilityNotSupported on backends that do not declare this capability. Most backends declare it.

read ¶

read(path: str) -> AsyncIterator[bytes]

Return an async iterator of byte chunks for path.

The caller is responsible for consuming the iterator. Validation (capability check, path check) happens eagerly on call, not lazily on first iteration.

Parameters:

path (str) –

Store-relative file path.

Returns:

AsyncIterator[bytes] –

Async iterator of byte chunks.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path is empty, or if path names a directory.

Quality flag: Capability.LAZY_READ

When declared, data is fetched lazily — partial reads avoid loading the whole file. Without it, the backend may buffer content before returning the stream.

read_bytes `async` ¶

read_bytes(path: str) -> bytes

Read the entire file into memory and return bytes.

Parameters:

path (str) –

Store-relative file path.

Returns:

bytes –

The file content as bytes.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path is empty, or if path names a directory.

Equivalent to collecting all chunks from read(path).

read_text `async` ¶

read_text(
    path: str,
    *,
    encoding: str = "utf-8",
    errors: str = "strict",
) -> str

Read the entire file and decode it as text.

Parameters:

path (str) –

Store-relative file path.
encoding (str, default: 'utf-8' ) –

Text encoding, any name accepted by codecs.
errors (str, default: 'strict' ) –

Error handler: "strict", "ignore", "replace", "backslashreplace". See codecs.register_error for custom handlers.

Returns:

str –

The file content as str.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path is empty, or if path names a directory.
UnicodeDecodeError –

If decoding fails with errors="strict".

Equivalent to (await read_bytes(path)).decode(encoding, errors).

Writing¶

Requires Capability.WRITE

write() and write_text() raise CapabilityNotSupported on backends that do not declare this capability. Most backends declare it. write_atomic() additionally requires Capability.ATOMIC_WRITE.

Quality flag: Capability.WRITE_RESULT_NATIVE

When declared, the returned WriteResult fields (etag, version_id, last_modified, digest) are populated from the backend's write response. Without it, only locally computable fields are set.

write `async` ¶

write(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write binary content to path. Creates parent folders implicitly.

Parameters:

path (str) –

Store-relative file path.
content (AsyncWritableContent) –

bytes or async iterator of bytes.
overwrite (bool, default: False ) –

If False, raises AlreadyExists when path exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

WriteResult with at least path and size populated.

Raises:

ValueError –

If metadata contains invalid keys or values (see Store.write() for validation rules).
CapabilityNotSupported –

If metadata is non-empty and the backend lacks USER_METADATA.
AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path is empty.

Backend-conditional argument: metadata=

Passing metadata raises CapabilityNotSupported on backends that do not declare Capability.USER_METADATA. Passing None or {} is safe on all backends.

write_text `async` ¶

write_text(
    path: str,
    text: str,
    *,
    encoding: str = "utf-8",
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write a string to path, encoded with the given encoding.

Parameters:

path (str) –

Store-relative file path.
text (str) –

The string to write.
encoding (str, default: 'utf-8' ) –

Text encoding.
overwrite (bool, default: False ) –

If False, raises AlreadyExists when path exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Raises:

ValueError –

If metadata contains invalid keys or values (see Store.write() for validation rules).
CapabilityNotSupported –

If metadata is non-empty and the backend lacks USER_METADATA.
AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path is empty.

Returns:

WriteResult –

WriteResult with at least path and size populated.

Equivalent to await write(path, text.encode(encoding), overwrite=overwrite, metadata=metadata).

Backend-conditional argument: metadata=

Passing metadata raises CapabilityNotSupported on backends that do not declare Capability.USER_METADATA. Passing None or {} is safe on all backends.

write_atomic `async` ¶

write_atomic(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write binary content to path atomically.

If the write fails or is interrupted, path is not left in a partial state.

Parameters:

path (str) –

Store-relative file path.
content (AsyncWritableContent) –

bytes or async iterator of bytes.
overwrite (bool, default: False ) –

If False, raises AlreadyExists when path exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Raises:

ValueError –

If metadata contains invalid keys or values (see Store.write() for validation rules).
CapabilityNotSupported –

If metadata is non-empty and the backend lacks USER_METADATA, or if backend lacks ATOMIC_WRITE.
AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path is empty.

Returns:

WriteResult –

WriteResult with at least path and size populated.

Requires Capability.ATOMIC_WRITE

Raises CapabilityNotSupported on backends that do not declare this capability.

Backend-conditional argument: metadata=

Passing metadata raises CapabilityNotSupported on backends that do not declare Capability.USER_METADATA. Passing None or {} is safe on all backends.

Deleting¶

Requires Capability.DELETE

All delete methods raise CapabilityNotSupported on backends that do not declare this capability.

delete `async` ¶

delete(path: str, *, missing_ok: bool = False) -> None

Delete a single file.

Parameters:

path (str) –

Store-relative file path.
missing_ok (bool, default: False ) –

If True, silently succeeds when path does not exist.

Raises:

NotFound –

If the file is missing and missing_ok is False.
InvalidPath –

If path is empty, or if path names a directory (regardless of missing_ok).

delete_folder `async` ¶

delete_folder(
    path: str,
    *,
    recursive: bool = False,
    missing_ok: bool = False,
) -> None

Delete a folder.

Parameters:

path (str) –

Store-relative folder path. Must not be "" (root).
recursive (bool, default: False ) –

If True, delete all contents first. If False, raises DirectoryNotEmpty when folder is non-empty.
missing_ok (bool, default: False ) –

If True, silently succeeds when path does not exist.

Raises:

NotFound –

If the folder is missing and missing_ok is False.
DirectoryNotEmpty –

If the folder is non-empty and recursive is False.
InvalidPath –

If path is empty (cannot delete the store root), or if path names a file (use delete instead).

Listing and Iteration¶

Requires Capability.LIST

All listing methods raise CapabilityNotSupported on backends that do not declare this capability.

list_files ¶

list_files(
    path: str,
    *,
    recursive: bool = False,
    pattern: str | None = None,
    max_depth: int | None = None,
) -> AsyncIterator[FileInfo]

Yield FileInfo objects for files under path.

Validation (capability check, max_depth) happens eagerly on call, not lazily on first iteration.

Parameters:

path (str) –

Store-relative folder path.
recursive (bool, default: False ) –

Descend into subfolders. Ignored when max_depth is set.
pattern (str | None, default: None ) –

Glob pattern to filter filenames (e.g. "*.csv"). Matched against each file's name (basename only). For full path-based patterns, use ext.glob.glob_files().
max_depth (int | None, default: None ) –

Maximum folder depth to include. 0 means files directly in path only; 1 adds files in its immediate subfolders, and so on. None (default) defers to recursive. When set, recursive is ignored.

Returns:

AsyncIterator[FileInfo] –

Async iterator of FileInfo with store-relative paths.

Raises:

ValueError –

If max_depth is negative.

Backend-conditional argument: max_depth=

Backends with native depth limiting prune traversal early. Backends that do not support it still return correct results — the Store applies client-side filtering as a safety net.

list_folders ¶

list_folders(
    path: str,
    *,
    pattern: str | None = None,
    max_depth: int | None = None,
) -> AsyncIterator[FolderEntry]

Yield subfolders of path as FolderEntry objects.

Validation (capability check, max_depth) happens eagerly on call, not lazily on first iteration.

Parameters:

path (str) –

Store-relative folder path.
pattern (str | None, default: None ) –

Glob pattern to filter folder names (e.g. "raw_*"). Matched against each folder's name (basename only) via fnmatch.fnmatch. Filters yielded results only — does not prune BFS traversal, so non-matching folders are still descended into.
max_depth (int | None, default: None ) –

Maximum folder depth to include. None or 0 returns immediate children only (default). 1 adds grandchildren, and so on. BFS traversal runs first; pattern filters what is yielded.

Returns:

AsyncIterator[FolderEntry] –

Async iterator of FolderEntry with .name and .path (store-relative).

Raises:

ValueError –

If max_depth is negative.

Backend-conditional argument: max_depth=

Backends with native depth limiting prune traversal early. Backends that do not support it still return correct results — the Store applies client-side filtering as a safety net.

iter_children ¶

iter_children(
    path: str,
) -> AsyncIterator[FileInfo | FolderEntry]

Yield all immediate children (files and folders) of path in a single pass.

Files are yielded as FileInfo, folders as FolderEntry. Both have .name and .path attributes (satisfying the PathEntry protocol) so callers can iterate uniformly.

Validation (capability check) happens eagerly on call, not lazily on first iteration.

Parameters:

path (str) –

Store-relative folder path.

Returns:

AsyncIterator[FileInfo | FolderEntry] –

Async iterator of FileInfo (files) and FolderEntry (folders).

glob ¶

glob(pattern: str) -> AsyncIterator[FileInfo]

Yield files matching a glob pattern, using the backend's native glob implementation.

Requires Capability.GLOB. Validation (capability check) happens eagerly on call, not lazily on first iteration.

Parameters:

pattern (str) –

Glob pattern (e.g. "data/**/*.parquet").

Returns:

AsyncIterator[FileInfo] –

Async iterator of FileInfo with store-relative paths.

Raises:

CapabilityNotSupported –

If the backend lacks GLOB.

Requires Capability.GLOB

glob() raises CapabilityNotSupported on backends that do not declare this capability. Check store.supports(Capability.GLOB) before calling.

Ordering and laziness

Ordering is backend-defined and may vary between backends (e.g. lexicographic on S3, OS-dependent on local filesystems). Callers must not depend on any particular order.

Results are yielded lazily. Backends may use pagination internally. Memory usage stays bounded for large directories.

File Operations¶

Requires Capability.MOVE / Capability.COPY

move() requires Capability.MOVE; copy() requires Capability.COPY. Each raises CapabilityNotSupported on backends that do not declare the respective capability.

move `async` ¶

move(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Move (rename) a file from src to dst.

File-only -- to move a folder, iterate its contents.

Parameters:

src (str) –

Source file path.
dst (str) –

Destination file path.
overwrite (bool, default: False ) –

If False, raises AlreadyExists when dst exists.

Raises:

NotFound –

If src does not exist.
AlreadyExists –

If dst exists and overwrite is False.
InvalidPath –

If src or dst is empty, or if src names a directory.

Requires Capability.MOVE

Raises CapabilityNotSupported on backends that do not declare this capability.

Atomicity

Atomicity is backend-dependent. Local uses os.replace (atomic on same filesystem). S3 and Azure use copy-then-delete (not atomic). SFTP atomicity depends on the server. Check store.supports(Capability.ATOMIC_MOVE) to query this at runtime.

copy `async` ¶

copy(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Copy a file from src to dst.

File-only -- to copy a folder, iterate its contents.

Parameters:

src (str) –

Source file path.
dst (str) –

Destination file path.
overwrite (bool, default: False ) –

If False, raises AlreadyExists when dst exists.

Raises:

NotFound –

If src does not exist.
AlreadyExists –

If dst exists and overwrite is False.
InvalidPath –

If src or dst is empty, or if src names a directory.

Requires Capability.COPY

Raises CapabilityNotSupported on backends that do not declare this capability.

Metadata preservation

Metadata preservation is backend-dependent. S3 copies metadata; local preserves metadata (copy2); SFTP does not (stream copy).

Metadata¶

Partially requires Capability.METADATA

head() and get_file_info() require Capability.METADATA. get_folder_info() requires Capability.METADATA without max_depth, or Capability.LIST when max_depth is set. exists(), is_file(), and is_folder() are always available.

head `async` ¶

head(path: str) -> WriteResult

Return a WriteResult snapshot of path via a metadata lookup.

Async equivalent of Store.head(). Gated on Capability.METADATA. Returns source="sidecar".

Parameters:

path (str) –

Store-relative file path.

Returns:

WriteResult –

WriteResult with source="sidecar".

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path is empty, or if path names a directory.
CapabilityNotSupported –

If the backend lacks METADATA.

Requires Capability.METADATA

Raises CapabilityNotSupported on backends that do not declare this capability.

exists `async` ¶

exists(path: str) -> bool

Return True if path exists (file or folder).

Parameters:

path (str) –

Store-relative path.

is_file `async` ¶

is_file(path: str) -> bool

Return True if path exists and is a file.

Parameters:

path (str) –

Store-relative path.

is_folder `async` ¶

is_folder(path: str) -> bool

Return True if path exists and is a folder.

Parameters:

path (str) –

Store-relative path.

get_file_info `async` ¶

get_file_info(path: str) -> FileInfo

Return a FileInfo with size, modification time, and content type for a single file.

Parameters:

path (str) –

Store-relative file path.

Returns:

FileInfo –

FileInfo.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path is empty, or if path names a directory.

Requires Capability.METADATA

Raises CapabilityNotSupported on backends that do not declare this capability.

get_folder_info `async` ¶

get_folder_info(
    path: str, *, max_depth: int | None = None
) -> FolderInfo

Return a FolderInfo with aggregated size and file count for a folder.

Parameters:

path (str) –

Store-relative folder path.
max_depth (int | None, default: None ) –

Maximum folder depth to aggregate. 0 means files directly in path only; 1 adds files in its immediate subfolders, and so on. None (default) performs a full recursive traversal via the backend.

Returns:

FolderInfo –

FolderInfo.

Raises:

NotFound –

If the folder does not exist.
InvalidPath –

If path names a file (use get_file_info instead).
ValueError –

If max_depth is negative.

Capability depends on max_depth

Without max_depth: requires Capability.METADATA. With max_depth set: requires Capability.LIST — works on backends that lack METADATA.

Backend-conditional argument: max_depth=

Backends with native depth limiting prune traversal early. Backends that do not support it still return correct results — the Store applies client-side filtering as a safety net.

Introspection¶

resolve ¶

resolve(key: str) -> ResolutionPlan

Return a ResolutionPlan describing how key maps to storage.

Delegates to the backend's resolve() and rebases the key so that plan.key is the store-relative key, not the backend-relative path.

Parameters:

key (str) –

Store-relative path. "" resolves the store root.

Returns:

ResolutionPlan –

A frozen ResolutionPlan.

Info

resolve() is a pure introspection method — it performs no I/O and is never called implicitly by other Store methods. The returned ResolutionPlan describes how a key maps to its storage location.

Lifecycle¶

ping `async` ¶

ping() -> None

Verify that the backend is reachable.

Raises:

PermissionDenied –

If credentials are invalid.
NotFound –

If the bucket, container, or root path does not exist.
BackendUnavailable –

If the backend cannot be reached.

aclose `async` ¶

aclose() -> None

Release backend resources.

Called automatically when used as an async context manager.

child ¶

child(subpath: str) -> AsyncStore

Return a new AsyncStore scoped to subpath under the current root.

The child shares the same backend instance.

Parameters:

subpath (str) –

Path segment to append to the current root.

Returns:

AsyncStore –

AsyncStore.

Raises:

InvalidPath –

If subpath is empty, contains .. segments, or includes null bytes.

data = store.child("data/2024")
async for fi in data.list_files(""):
    ...  # lists files under <root>/data/2024/

Interop (Backend-Specific)¶

Backend-specific methods

Methods in this section expose backend internals. Using them ties your code to a specific backend. For portable alternatives, see Store or the Async Store Guide.

unwrap ¶

unwrap(type_hint: type[T]) -> T

Return the backend's native client object, cast to type_hint.

Parameters:

type_hint (type[T]) –

The expected type of the native client (e.g. pyarrow.fs.FileSystem).

Returns:

T –

The native client.

Raises:

CapabilityNotSupported –

If the backend cannot provide the requested type.

arrow_fs = store.unwrap(pyarrow.fs.FileSystem)

native_path ¶

native_path(key: str) -> str

Convert a store-relative key to the backend's native path representation.

Inverse of to_key().

Parameters:

key (str) –

Store-relative path.

Returns:

str –

Backend-native path (e.g. S3 object key, local filesystem path).

to_key ¶

to_key(path: str) -> str

Convert a backend-native path to a store-relative key.

Inverse of native_path().

Parameters:

path (str) –

Backend-native path string.

Returns:

str –

Store-relative key.

Raises:

InvalidPath –

If the path does not belong to this store.

supports ¶

supports(capability: Capability) -> bool

Check whether the backend supports a given Capability.

Parameters:

capability (Capability) –

A Capability enum member.

Returns:

bool –

True if the backend declares this capability.

if store.supports(Capability.GLOB):
    async for fi in store.glob("**/*.csv"):
        ...

Info

supports() itself is portable — it works on all backends. Only the capability-gated methods it guards are backend-specific.

AsyncBackend¶

AsyncBackend ¶

Bases: ABC

Abstract base class for all async storage backends.

Every backend must implement all abstract methods. Backend-native exceptions must never leak -- they must be mapped to remote_store errors.

Implementing an async backend

Subclass AsyncBackend and implement all abstract methods. Map every backend-native exception to a remote_store error — native exceptions must never leak to callers.

Identity¶

name `abstractmethod` `property` ¶

name: str

Unique identifier for this backend type (e.g. 'local', 's3').

capabilities `abstractmethod` `property` ¶

capabilities: CapabilitySet

Declared capabilities of this backend.

Existence¶

exists `abstractmethod` `async` ¶

exists(path: str) -> bool

Check if a file or folder exists. Never raises NotFound.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if a file or folder exists at path.

is_file `abstractmethod` `async` ¶

is_file(path: str) -> bool

Return True if path is an existing file.

Parameters:

path (str) –

Backend-relative key.

Returns:

bool –

True if path exists and is a file.

is_folder `abstractmethod` `async` ¶

is_folder(path: str) -> bool

Return True if path is an existing folder.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if path exists and is a folder.

Reading¶

Requires Capability.READ

All read methods raise CapabilityNotSupported on backends that do not declare this capability. Most backends declare it.

read `abstractmethod` `async` ¶

read(path: str) -> AsyncIterator[bytes]

Open a file for reading and return an async iterator of byte chunks.

Parameters:

path (str) –

Backend-relative key.

Returns:

AsyncIterator[bytes] –

An async iterator yielding byte chunks.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

Quality flag: Capability.LAZY_READ

When declared, data is fetched lazily — partial reads avoid loading the whole file. Without it, the backend may buffer content before returning the stream.

read_bytes `abstractmethod` `async` ¶

read_bytes(path: str) -> bytes

Read the full content of a file as bytes.

Parameters:

path (str) –

Backend-relative key.

Returns:

bytes –

The file content.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

Writing¶

Requires Capability.WRITE

write() raises CapabilityNotSupported on backends that do not declare this capability. Most backends declare it. write_atomic() additionally requires Capability.ATOMIC_WRITE.

Quality flag: Capability.WRITE_RESULT_NATIVE

When declared, the returned WriteResult fields (etag, version_id, last_modified, digest) are populated from the backend's write response. Without it, only locally computable fields are set.

write `abstractmethod` `async` ¶

write(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content to a file.

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write.
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

A WriteResult with size, path, and optional native fields.

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory, or if any slash-aligned ancestor of path exists as a regular file. Flat-namespace backends (S3, Azure non-HNS, SQL) cannot detect a file ancestor in O(1) and skip the check by default; the per-backend reject_write_under_file_ancestor opt-in enables it.

Backend-conditional argument: metadata=

Passing metadata raises CapabilityNotSupported on backends that do not declare Capability.USER_METADATA. Passing None or {} is safe on all backends.

write_atomic `abstractmethod` `async` ¶

write_atomic(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content atomically via temp file + rename.

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write.
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

A WriteResult with size, path, and optional native fields.

Raises:

CapabilityNotSupported –

If backend lacks ATOMIC_WRITE.
AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory, or if any slash-aligned ancestor of path exists as a regular file (see write).

Requires Capability.ATOMIC_WRITE

Raises CapabilityNotSupported on backends that do not declare this capability.

Backend-conditional argument: metadata=

Passing metadata raises CapabilityNotSupported on backends that do not declare Capability.USER_METADATA. Passing None or {} is safe on all backends.

Deleting¶

Requires Capability.DELETE

All delete methods raise CapabilityNotSupported on backends that do not declare this capability.

delete `abstractmethod` `async` ¶

delete(path: str, *, missing_ok: bool = False) -> None

Delete a file.

Parameters:

path (str) –

Backend-relative key.
missing_ok (bool, default: False ) –

If True, do not raise when the file is absent.

Raises:

NotFound –

If the file is missing and missing_ok is False.
InvalidPath –

If the path is empty, or if path names a directory (regardless of missing_ok).

delete_folder `abstractmethod` `async` ¶

delete_folder(
    path: str,
    *,
    recursive: bool = False,
    missing_ok: bool = False,
) -> None

Delete a folder.

Parameters:

path (str) –

Backend-relative key.
recursive (bool, default: False ) –

If True, delete all contents first.
missing_ok (bool, default: False ) –

If True, do not raise when absent.

Raises:

InvalidPath –

If path names an existing file (regardless of missing_ok — a type mismatch is not a missing file).
NotFound –

If the folder is missing and missing_ok is False.
DirectoryNotEmpty –

If non-empty and recursive is False.

Listing and Iteration¶

Requires Capability.LIST

All listing methods raise CapabilityNotSupported on backends that do not declare this capability.

list_files `abstractmethod` `async` ¶

list_files(
    path: str,
    *,
    recursive: bool = False,
    max_depth: int | None = None,
) -> AsyncIterator[FileInfo]

List files under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.
recursive (bool, default: False ) –

If True, include files in all subdirectories.
max_depth (int | None, default: None ) –

Optional maximum folder depth to traverse. When set, backends that support native depth limiting prune traversal early. Backends that ignore this parameter still produce correct results -- the Store applies client-side filtering as a safety net. None (default) defers to recursive.

Returns:

AsyncIterator[FileInfo] –

An async iterator of FileInfo objects.

Backend-conditional argument: max_depth=

Backends with native depth limiting prune traversal early. Backends that do not support it still return correct results — the Store applies client-side filtering as a safety net.

list_folders `abstractmethod` `async` ¶

list_folders(path: str) -> AsyncIterator[FolderEntry]

List immediate subfolders under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FolderEntry] –

An async iterator of FolderEntry objects with .name and .path.

iter_children `async` ¶

iter_children(
    path: str,
) -> AsyncIterator[FileInfo | FolderEntry]

Yield both files and folders under path in a single pass.

Files are yielded as FileInfo objects, folders as FolderEntry objects. The default implementation chains list_files() and list_folders(). Backends that can fetch both in a single I/O call should override this for efficiency.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FileInfo | FolderEntry] –

An async iterator of FileInfo (files) and FolderEntry (folders).

glob `async` ¶

glob(pattern: str) -> AsyncIterator[FileInfo]

Match files against a glob pattern.

Non-abstract -- backends with native glob support override this and add Capability.GLOB to their capability set.

Parameters:

pattern (str) –

Glob pattern (e.g., "data/*.csv", "**/*.txt").

Raises:

CapabilityNotSupported –

If the backend lacks GLOB.

Requires Capability.GLOB

glob() raises CapabilityNotSupported on backends that do not declare this capability.

Metadata¶

Requires Capability.METADATA

get_file_info() requires Capability.METADATA. get_folder_info() requires Capability.METADATA without max_depth, or Capability.LIST when max_depth is set.

get_file_info `abstractmethod` `async` ¶

get_file_info(path: str) -> FileInfo

Get metadata for a file.

Parameters:

path (str) –

Backend-relative key.

Returns:

FileInfo –

A FileInfo with size, modification time, etc.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

Requires Capability.METADATA

Raises CapabilityNotSupported on backends that do not declare this capability.

get_folder_info `abstractmethod` `async` ¶

get_folder_info(path: str) -> FolderInfo

Get metadata for a folder.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

FolderInfo –

A FolderInfo with file count, total size, etc.

Raises:

InvalidPath –

If path names an existing file.
NotFound –

If the folder does not exist.

Capability depends on max_depth

Without max_depth: requires Capability.METADATA. With max_depth set: requires Capability.LIST — works on backends that lack METADATA.

File Operations¶

Requires Capability.MOVE / Capability.COPY

move() requires Capability.MOVE; copy() requires Capability.COPY. Each raises CapabilityNotSupported on backends that do not declare the respective capability.

move `abstractmethod` `async` ¶

move(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Move or rename a file.

src == dst is a no-op (the file is preserved unchanged).

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

InvalidPath –

If src names a directory, dst names an existing directory, or any slash-aligned ancestor of dst exists as a regular file.
NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.

Requires Capability.MOVE

Raises CapabilityNotSupported on backends that do not declare this capability.

Quality flag: Capability.ATOMIC_MOVE

When declared, move() is guaranteed atomic under concurrent access. Check store.supports(Capability.ATOMIC_MOVE) to query at runtime.

copy `abstractmethod` `async` ¶

copy(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Copy a file.

src == dst is a no-op (the file is preserved unchanged).

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

InvalidPath –

If src names a directory, dst names an existing directory, or any slash-aligned ancestor of dst exists as a regular file.
NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.

Requires Capability.COPY

Raises CapabilityNotSupported on backends that do not declare this capability.

Lifecycle¶

aclose `async` ¶

aclose() -> None

Release resources. Default is a no-op.

check_health `async` ¶

check_health() -> None

Verify the backend is reachable and credentials are valid.

The default implementation is a no-op (always succeeds). Backends override this to perform a lightweight, non-destructive connectivity check using the cheapest possible read-only operation.

Raises:

PermissionDenied –

If credentials are invalid.
NotFound –

If the bucket, container, or root path does not exist.
BackendUnavailable –

If the backend cannot be reached.

Introspection¶

resolve ¶

resolve(path: str) -> ResolutionPlan

Return a ResolutionPlan describing how path maps to storage.

Pure introspection -- no I/O is performed. Backends override this to populate details with backend-specific context.

Parameters:

path (str) –

Backend-relative key.

Returns:

ResolutionPlan –

A frozen ResolutionPlan with kind, backend,
ResolutionPlan –

key, native_path, and details.

Interop (Backend-Specific)¶

Backend-specific methods

Methods in this section expose backend internals. Using them ties your code to a specific backend. For portable alternatives, use the methods above.

unwrap ¶

unwrap(type_hint: type[T]) -> T

Return the native backend handle if it matches the requested type.

Parameters:

type_hint (type[T]) –

The expected type (e.g., fsspec.AbstractFileSystem).

Raises:

CapabilityNotSupported –

If backend cannot provide the requested type.

native_path ¶

native_path(path: str) -> str

Convert a backend-relative key to the backend-native path.

The inverse of to_key(). The default implementation is the identity function -- backends with a native root (bucket, base_path) override this to prepend their prefix.

Parameters:

path (str) –

Backend-relative key.

Returns:

str –

Backend-native path usable with the native handle from unwrap().

to_key ¶

to_key(native_path: str) -> str

Convert a backend-native path to a backend-relative key.

Strips the backend's own root/prefix from the path. The default implementation is the identity function -- backends with a native root (filesystem path, bucket prefix, base_path) override this.

Parameters:

native_path (str) –

Absolute or backend-native path string.

Returns:

str –

Path relative to the backend's root.

SyncBackendAdapter¶

Wraps any synchronous Backend as an AsyncBackend by dispatching each blocking call to the default executor via asyncio.to_thread. AsyncStore auto-wraps sync backends on construction; explicit construction is only required when you want to introspect the adapter.

SyncBackendAdapter ¶

SyncBackendAdapter(backend: Backend)

Wraps a synchronous Backend as an AsyncBackend.

Every blocking call is dispatched to the default executor via asyncio.to_thread, keeping the event loop responsive.

Parameters:

backend (Backend) –

The synchronous backend instance to wrap.

name `property` ¶

name: str

Backend identifier, forwarded from the wrapped backend.

capabilities `property` ¶

capabilities: CapabilitySet

Capability set, forwarded from the wrapped backend.

to_key ¶

to_key(native_path: str) -> str

Convert a native path to a backend-relative key.

native_path ¶

native_path(path: str) -> str

Convert a backend-relative key to the native path.

resolve ¶

resolve(path: str) -> ResolutionPlan

Return a resolution plan for path.

unwrap ¶

unwrap(type_hint: type[T]) -> T

Return the native backend handle.

exists `async` ¶

exists(path: str) -> bool

Check if a file or folder exists.

is_file `async` ¶

is_file(path: str) -> bool

Return True if path is an existing file.

is_folder `async` ¶

is_folder(path: str) -> bool

Return True if path is an existing folder.

read_bytes `async` ¶

read_bytes(path: str) -> bytes

Read the full content of a file as bytes.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

get_file_info `async` ¶

get_file_info(path: str) -> FileInfo

Get metadata for a file.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

get_folder_info `async` ¶

get_folder_info(path: str) -> FolderInfo

Get metadata for a folder.

Raises:

InvalidPath –

If path names an existing file.
NotFound –

If the folder does not exist.

move `async` ¶

move(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Move or rename a file.

Raises:

InvalidPath –

If src names a directory or dst names an existing directory.
NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.

copy `async` ¶

copy(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Copy a file.

Raises:

InvalidPath –

If src names a directory or dst names an existing directory.
NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.

delete `async` ¶

delete(path: str, *, missing_ok: bool = False) -> None

Delete a file.

Raises:

NotFound –

If the file is missing and missing_ok is False.
InvalidPath –

If the path is empty, or if path names a directory (regardless of missing_ok).

delete_folder `async` ¶

delete_folder(
    path: str,
    *,
    recursive: bool = False,
    missing_ok: bool = False,
) -> None

Delete a folder.

Raises:

InvalidPath –

If path names an existing file (regardless of missing_ok — a type mismatch is not a missing file).
NotFound –

If the folder is missing and missing_ok is False.
DirectoryNotEmpty –

If non-empty and recursive is False.

check_health `async` ¶

check_health() -> None

Verify the backend is reachable.

read `async` ¶

read(path: str) -> AsyncIterator[bytes]

Open a file for reading and yield chunks asynchronously.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

write `async` ¶

write(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content to a file.

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

write_atomic `async` ¶

write_atomic(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content atomically via temp file + rename.

Raises:

CapabilityNotSupported –

If backend lacks ATOMIC_WRITE.
AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

list_files `async` ¶

list_files(
    path: str,
    *,
    recursive: bool = False,
    max_depth: int | None = None,
) -> AsyncIterator[FileInfo]

List files under path.

list_folders `async` ¶

list_folders(path: str) -> AsyncIterator[FolderEntry]

List immediate subfolders under path.

glob `async` ¶

glob(pattern: str) -> AsyncIterator[FileInfo]

Match files against a glob pattern.

iter_children `async` ¶

iter_children(
    path: str,
) -> AsyncIterator[FileInfo | FolderEntry]

Yield both files and folders under path.

aclose `async` ¶

aclose() -> None

Release resources held by the wrapped backend.

AsyncBackendSyncAdapter¶

Wraps any AsyncBackend as a synchronous Backend by running a private event loop on a dedicated daemon thread for the adapter's lifetime. See Async-sync bridges for the full behaviour contract and the async-to-sync adapter decision record for the design rationale.

AsyncBackendSyncAdapter ¶

AsyncBackendSyncAdapter(async_backend: AsyncBackend)

Wraps an AsyncBackend as a synchronous Backend.

One private asyncio event loop per adapter instance, running on a dedicated daemon thread for the adapter's lifetime. Sync methods submit coroutines via asyncio.run_coroutine_threadsafe and block on the returned concurrent.futures.Future.

Construction does not enter the wrapped backend's async context manager -- callers that need __aenter__ semantics should use AsyncStore directly.

Parameters:

async_backend (AsyncBackend) –

The async backend instance to wrap.

name `property` ¶

name: str

Backend identifier, forwarded from the wrapped async backend.

capabilities `cached` `property` ¶

capabilities: CapabilitySet

Capabilities reported by the wrapped backend, with stream-shape translation applied.

SEEKABLE_READ is masked off unconditionally — the chunk-pull stream this adapter returns is forward-only. USER_METADATA and WRITE_RESULT_NATIVE are forwarded from the inner async backend unchanged; the write*() methods accept and forward metadata=.

unwrap ¶

unwrap(type_hint: type[T]) -> T

Return a sync-safe native handle if the wrapped backend provides one.

By default raises CapabilityNotSupported because async-SDK handles are bound to the adapter's private event loop and cannot be used safely from the caller's thread.

Wrapped backends that can expose a sync-safe handle should implement _SyncSafeHandleProvider and return it from _SyncSafeHandleProvider.sync_safe_unwrap.

Parameters:

type_hint (type[T]) –

The type of handle to retrieve; passed through to _SyncSafeHandleProvider.sync_safe_unwrap for backends that support the exemption.

Returns:

T –

A sync-safe handle of the type requested via type_hint.

Raises:

CapabilityNotSupported –

If the wrapped backend does not implement _SyncSafeHandleProvider.

Example

class _SafeBackend(AsyncBackend, _SyncSafeHandleProvider):
    def sync_safe_unwrap(self, type_hint):
        return self._sync_client

adapter = AsyncBackendSyncAdapter(_SafeBackend())
client = adapter.unwrap(SyncClient)

check_health ¶

check_health() -> None

Submit a connectivity probe to the wrapped async backend.

Not a no-op: the probe is forwarded to the wrapped AsyncBackend, and any connectivity error it raises reaches the sync caller unchanged.

Returns:

None –

None on success.

Raises:

BackendUnavailable –

Or another backend-specific error if the wrapped backend's health check fails.
RuntimeError –

If the adapter is closed or called from a running event loop.

Example

try:
    adapter.check_health()
except BackendUnavailable:
    ...  # handle connectivity failure

read ¶

read(path: str) -> BinaryIO

Open path for reading and return a forward-only chunk-pull stream.

The returned stream pulls one async chunk per read() call; the file is never fully materialised in memory. At most one __anext__ is in-flight at a time.

The stream is forward-only: seekable() returns False and no seek / tell / fileno methods are exposed. Closing via the context manager or close() submits aclose() on the underlying async iterator so backend resources are released promptly.

Parameters:

path (str) –

Backend-relative key of the file to read.

Returns:

BinaryIO –

A forward-only io.RawIOBase stream over the file
BinaryIO –

contents.

Raises:

RuntimeError –

If the adapter is closed or called from a running event loop.
NotFound –

If path does not exist (propagated verbatim from the wrapped backend).

Example

with adapter.read("data/report.csv") as stream:
    header = stream.read(512)

close ¶

close(timeout: float | None = 30.0) -> None

Drain in-flight work, stop the loop, and join the daemon thread.

Drain order: submit aclose on the wrapped backend → loop-drain in-flight tasks (repeating while new tasks appear, see below) → stop the private loop → join the daemon thread.

The drain step repeats _drain_tasks until the private loop is quiet, narrowing (not eliminating) the window where a caller that passed _guard before the closed flag was set can still submit a coroutine. Each pass snapshots outstanding tasks and waits for them; new tasks that arrive between snapshot and completion trigger another pass. A residual TOCTOU gap remains: a thread that passes _guard after the final empty-snapshot check but before the loop is stopped will have its coroutine silently discarded when the loop stops — eliminating this gap would require serialising all submits against a shutdown lock.

If timeout expires before the loop is drained, a single WARNING record is emitted (message stem "AsyncBackendSyncAdapter close timed out") and the daemon thread is left for process-exit reaping. Idempotent: subsequent calls are no-ops.

Parameters:

timeout (float | None, default: 30.0 ) –

Maximum seconds to wait for in-flight work to finish and the background thread to join. None means wait indefinitely. Defaults to 30.0.

AsyncMemoryBackend¶

In-memory async backend using a tree-indexed data structure. Zero dependencies, no filesystem access, no network. Designed as a drop-in async backend for unit testing, interactive exploration, and documentation examples. Supports all capabilities except GLOB.

AsyncMemoryBackend ¶

AsyncMemoryBackend()

In-memory async backend using a tree-indexed data structure.

Zero dependencies, no filesystem access, no network. Designed as a drop-in async backend for unit testing, interactive exploration, and documentation examples.

Supports all capabilities except GLOB. The full conformance suite passes with zero skips.

Note

LAZY_READ is included here but absent from MemoryBackend (the sync mirror). The sync backend buffers fully in memory; the async backend can yield chunks incrementally. The mirrors graph edge between the two does not imply capability parity.

exists `async` ¶

exists(path: str) -> bool

Check if a file or folder exists.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if a file or folder exists at path.

is_file `async` ¶

is_file(path: str) -> bool

Return True if path is an existing file.

Parameters:

path (str) –

Backend-relative key.

Returns:

bool –

True if path exists and is a file.

is_folder `async` ¶

is_folder(path: str) -> bool

Return True if path is an existing folder.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if path exists and is a folder.

read `async` ¶

read(path: str) -> AsyncIterator[bytes]

Open a file for reading and return an async iterator of byte chunks.

Parameters:

path (str) –

Backend-relative key.

Returns:

AsyncIterator[bytes] –

An async iterator yielding a single byte chunk (the full content).

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

read_bytes `async` ¶

read_bytes(path: str) -> bytes

Read the full content of a file as bytes.

Parameters:

path (str) –

Backend-relative key.

Returns:

bytes –

The file content.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

write `async` ¶

write(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content to a file.

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write (bytes or async iterator of bytes).
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

WriteResult with path, size, last_modified, and
WriteResult –

source="native" populated.

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

write_atomic `async` ¶

write_atomic(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content atomically (same as write for in-memory backend).

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write.
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

WriteResult with path, size, last_modified, and
WriteResult –

source="native" populated.

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

delete `async` ¶

delete(path: str, *, missing_ok: bool = False) -> None

Delete a file.

Parameters:

path (str) –

Backend-relative key.
missing_ok (bool, default: False ) –

If True, do not raise when the file is absent.

Raises:

NotFound –

If the file is missing and missing_ok is False.
InvalidPath –

If the path is empty, or if path names a directory (regardless of missing_ok).

delete_folder `async` ¶

delete_folder(
    path: str,
    *,
    recursive: bool = False,
    missing_ok: bool = False,
) -> None

Delete a folder.

Parameters:

path (str) –

Backend-relative key.
recursive (bool, default: False ) –

If True, delete all contents first.
missing_ok (bool, default: False ) –

If True, do not raise when absent.

Raises:

NotFound –

If the folder is missing and missing_ok is False.
DirectoryNotEmpty –

If non-empty and recursive is False.
InvalidPath –

If the path is empty or names an existing file (regardless of missing_ok).

list_files `async` ¶

list_files(
    path: str,
    *,
    recursive: bool = False,
    max_depth: int | None = None,
) -> AsyncIterator[FileInfo]

List files under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.
recursive (bool, default: False ) –

If True, include files in all subdirectories.
max_depth (int | None, default: None ) –

Optional maximum folder depth to traverse.

Returns:

AsyncIterator[FileInfo] –

An async iterator of FileInfo objects.

list_folders `async` ¶

list_folders(path: str) -> AsyncIterator[FolderEntry]

List immediate subfolders under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FolderEntry] –

An async iterator of FolderEntry objects.

iter_children `async` ¶

iter_children(
    path: str,
) -> AsyncIterator[FileInfo | FolderEntry]

Yield both files and folders under path in a single pass.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FileInfo | FolderEntry] –

An async iterator of FileInfo (files) and FolderEntry (folders).

get_file_info `async` ¶

get_file_info(path: str) -> FileInfo

Get metadata for a file.

Parameters:

path (str) –

Backend-relative key.

Returns:

FileInfo –

A FileInfo with size, modification time, etc.

Raises:

InvalidPath –

If path names an existing directory.
NotFound –

If the file does not exist.

get_folder_info `async` ¶

get_folder_info(path: str) -> FolderInfo

Get metadata for a folder.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

FolderInfo –

A FolderInfo with file count, total size, etc.

Raises:

InvalidPath –

If path names an existing file.
NotFound –

If the folder does not exist.

move `async` ¶

move(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Move or rename a file.

src == dst is a no-op (the file is preserved unchanged).

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.
InvalidPath –

If src or dst is empty, src names a directory, or dst names an existing directory.

copy `async` ¶

copy(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Copy a file.

src == dst is a no-op (the file is preserved unchanged).

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

NotFound –

If src does not exist.
AlreadyExists –

If dst exists, src != dst, and overwrite is False.
InvalidPath –

If src or dst is empty, src names a directory, or dst names an existing directory.

AsyncAzureBackend¶

Native async Azure Storage backend. Uses the async Blob SDK for non-HNS accounts (plain Blob Storage, Azurite) and the async DataLake SDK for HNS accounts (ADLS Gen2) to get atomic rename and real directory support.

AsyncAzureBackend ¶

AsyncAzureBackend(
    container: str,
    *,
    account_name: str | None = None,
    account_url: str | None = None,
    account_key: str | Secret | None = None,
    sas_token: str | Secret | None = None,
    connection_string: str | Secret | None = None,
    credential: Any | None = None,
    client_options: dict[str, Any] | None = None,
    retry: RetryPolicy | None = None,
    max_concurrency: int = 1,
    reject_write_under_file_ancestor: bool = False,
)

Async Azure Storage backend.

Uses the async Blob SDK for non-HNS accounts (plain Blob Storage, Azurite) and the async DataLake SDK for HNS accounts (ADLS Gen2) to get atomic rename and real directory support.

Parameters:

container (str) –

Azure Storage container name (required, non-empty).
account_name (str | None, default: None ) –

Storage account name.
account_url (str | None, default: None ) –

Full account URL (e.g. https://myaccount.dfs.core.windows.net).
account_key (str | Secret | None, default: None ) –

Storage account key.
sas_token (str | Secret | None, default: None ) –

Shared Access Signature token.
connection_string (str | Secret | None, default: None ) –

Azure Storage connection string.
credential (Any | None, default: None ) –

Any credential object (e.g. DefaultAzureCredential()).
client_options (dict[str, Any] | None, default: None ) –

Additional options passed to service clients. The library sets max_single_put_size, max_block_size, and min_large_block_upload_threshold defaults for streaming memory discipline; user-supplied values take precedence.
retry (RetryPolicy | None, default: None ) –

Retry policy for transient failures.
max_concurrency (int, default: 1 ) –

Maximum number of parallel connections for uploads and downloads (default 1 -- sequential).
reject_write_under_file_ancestor (bool, default: False ) –

If True, write / write_atomic / move / copy HEAD each slash-aligned ancestor of the target path on non-HNS accounts and raise InvalidPath on the first regular-file hit, matching the cross-backend contract that hierarchical filesystems enforce natively. On HNS accounts the kwarg short-circuits: hdi_isfolder rejects the operation natively, and the backend detects the file ancestor on that rejection and re-raises it as InvalidPath, so HNS delivers the cross-backend contract with or without the kwarg set. Default False.

name `property` ¶

name: str

Unique identifier for this backend type.

capabilities `property` ¶

capabilities: CapabilitySet

Declared capabilities of this backend.

check_health `async` ¶

check_health() -> None

Verify the backend is reachable and credentials are valid.

Raises:

PermissionDenied –

If credentials are invalid.
NotFound –

If the container does not exist.
BackendUnavailable –

If the backend cannot be reached.

to_key ¶

to_key(native_path: str) -> str

Convert a backend-native path to a backend-relative key.

Parameters:

native_path (str) –

Absolute or backend-native path string.

Returns:

str –

Path relative to the backend's root.

native_path ¶

native_path(path: str) -> str

Convert a backend-relative key to the backend-native path.

Parameters:

path (str) –

Backend-relative key.

Returns:

str –

Backend-native path (container/path).

resolve ¶

resolve(path: str) -> ResolutionPlan

Return a ResolutionPlan with Azure-specific details.

Parameters:

path (str) –

Backend-relative key.

Returns:

ResolutionPlan –

Plan with kind="async-azure" and details containing
ResolutionPlan –

container and account_url.

exists `async` ¶

exists(path: str) -> bool

Check if a file or folder exists.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if a file or folder exists at path.

is_file `async` ¶

is_file(path: str) -> bool

Return True if path is an existing file.

Parameters:

path (str) –

Backend-relative key.

Returns:

bool –

True if path exists and is a file.

is_folder `async` ¶

is_folder(path: str) -> bool

Return True if path is an existing folder.

Parameters:

path (str) –

Backend-relative key, or "" for the root.

Returns:

bool –

True if path exists and is a folder.

read `async` ¶

read(path: str) -> AsyncIterator[bytes]

Open a file for reading and return an async iterator of byte chunks.

Parameters:

path (str) –

Backend-relative key.

Returns:

AsyncIterator[bytes] –

An async iterator yielding byte chunks.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path names a directory (HNS accounts only).

read_bytes `async` ¶

read_bytes(path: str) -> bytes

Read the full content of a file as bytes.

Parameters:

path (str) –

Backend-relative key.

Returns:

bytes –

The file content.

Raises:

NotFound –

If the file does not exist.
InvalidPath –

If path names a directory (HNS accounts only).

write `async` ¶

write(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content to a file.

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write (bytes or async iterator of bytes).
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

WriteResult with native Azure fields (etag, last_modified,
WriteResult –

etc.) populated from the SDK upload response.

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

write_atomic `async` ¶

write_atomic(
    path: str,
    content: AsyncWritableContent,
    *,
    overwrite: bool = False,
    metadata: Mapping[str, str] | None = None,
) -> WriteResult

Write content atomically via temp file + rename.

For non-HNS accounts, direct upload is atomic (PUT semantics). For HNS accounts, write to temp file via DFS then atomic rename.

Parameters:

path (str) –

Backend-relative key.
content (AsyncWritableContent) –

Data to write.
overwrite (bool, default: False ) –

If False, raise if file already exists.
metadata (Mapping[str, str] | None, default: None ) –

Optional user-defined string metadata.

Returns:

WriteResult –

WriteResult with native Azure fields populated from the SDK
WriteResult –

response (non-HNS) or from get_file_properties() after rename
WriteResult –

(HNS).

Raises:

AlreadyExists –

If the file exists and overwrite is False.
InvalidPath –

If path names a directory.

delete `async` ¶

delete(path: str, *, missing_ok: bool = False) -> None

Delete a file.

Parameters:

path (str) –

Backend-relative key.
missing_ok (bool, default: False ) –

If True, do not raise when the file is absent.

Raises:

NotFound –

If the file is missing and missing_ok is False.
InvalidPath –

If path names a directory (HNS accounts only).

delete_folder `async` ¶

delete_folder(
    path: str,
    *,
    recursive: bool = False,
    missing_ok: bool = False,
) -> None

Delete a folder.

Parameters:

path (str) –

Backend-relative key.
recursive (bool, default: False ) –

If True, delete all contents first.
missing_ok (bool, default: False ) –

If True, do not raise when absent.

Raises:

NotFound –

If the folder is missing and missing_ok is False.
InvalidPath –

If path names a file (use delete instead).
DirectoryNotEmpty –

If non-empty and recursive is False.

list_files `async` ¶

list_files(
    path: str,
    *,
    recursive: bool = False,
    max_depth: int | None = None,
) -> AsyncIterator[FileInfo]

List files under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.
recursive (bool, default: False ) –

If True, include files in all subdirectories.
max_depth (int | None, default: None ) –

Optional maximum folder depth to traverse.

Returns:

AsyncIterator[FileInfo] –

An async iterator of FileInfo objects.

list_folders `async` ¶

list_folders(path: str) -> AsyncIterator[FolderEntry]

List immediate subfolders under path.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FolderEntry] –

An async iterator of FolderEntry objects.

iter_children `async` ¶

iter_children(
    path: str,
) -> AsyncIterator[FileInfo | FolderEntry]

Yield both files and folders under path in a single pass.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

AsyncIterator[FileInfo | FolderEntry] –

An async iterator of FileInfo (files) and FolderEntry (folders).

glob `async` ¶

glob(pattern: str) -> AsyncIterator[FileInfo]

Match files against a glob pattern.

Parameters:

pattern (str) –

Glob pattern (e.g., "data/*.csv", "**/*.txt").

Returns:

AsyncIterator[FileInfo] –

An async iterator of matching FileInfo objects.

get_file_info `async` ¶

get_file_info(path: str) -> FileInfo

Get metadata for a file.

Parameters:

path (str) –

Backend-relative key.

Returns:

FileInfo –

A FileInfo with size, modification time, etc.

Raises:

InvalidPath –

If path names a directory (HNS: hdi_isfolder=true).
NotFound –

If the file does not exist.

get_folder_info `async` ¶

get_folder_info(path: str) -> FolderInfo

Get metadata for a folder.

Parameters:

path (str) –

Backend-relative folder key, or "" for the root.

Returns:

FolderInfo –

A FolderInfo with file count, total size, etc.

Raises:

NotFound –

If the folder does not exist.
InvalidPath –

If path names a file (use get_file_info instead).

move `async` ¶

move(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Move or rename a file.

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

NotFound –

If src does not exist.
InvalidPath –

If src or dst names a directory (HNS only).
AlreadyExists –

If dst exists and overwrite is False.

copy `async` ¶

copy(
    src: str, dst: str, *, overwrite: bool = False
) -> None

Copy a file.

Parameters:

src (str) –

Backend-relative source key.
dst (str) –

Backend-relative destination key.
overwrite (bool, default: False ) –

If True, replace any existing file at dst.

Raises:

NotFound –

If src does not exist.
InvalidPath –

If src or dst names a directory (HNS only).
AlreadyExists –

If dst exists and overwrite is False.

aclose `async` ¶

aclose() -> None

Release all Azure SDK client resources.

unwrap ¶

unwrap(type_hint: type[T]) -> T

Return the native async FileSystemClient if it matches the requested type.

Parameters:

type_hint (type[T]) –

The expected type.

Returns:

T –

The native async client instance matching type_hint.

Raises:

CapabilityNotSupported –

If backend cannot provide the requested type.

AsyncWritableContent¶

AsyncWritableContent `module-attribute` ¶

AsyncWritableContent = bytes | AsyncIterator[bytes]

Async API¶

AsyncStore¶

AsyncStore ¶

Reading¶

read ¶

read_bytes async ¶

read_text async ¶

Writing¶

write async ¶

write_text async ¶

write_atomic async ¶

Deleting¶

delete async ¶

delete_folder async ¶

Listing and Iteration¶

list_files ¶

list_folders ¶

iter_children ¶

glob ¶

File Operations¶

move async ¶

copy async ¶

Metadata¶

head async ¶

exists async ¶

is_file async ¶

is_folder async ¶

get_file_info async ¶

get_folder_info async ¶

Introspection¶

resolve ¶

Lifecycle¶

ping async ¶

aclose async ¶

child ¶

Interop (Backend-Specific)¶

unwrap ¶

native_path ¶

to_key ¶

supports ¶

AsyncBackend¶

AsyncBackend ¶

Identity¶

name abstractmethod property ¶

capabilities abstractmethod property ¶

Existence¶

exists abstractmethod async ¶

is_file abstractmethod async ¶

is_folder abstractmethod async ¶

Reading¶

read abstractmethod async ¶

read_bytes abstractmethod async ¶

Writing¶

write abstractmethod async ¶

write_atomic abstractmethod async ¶

Deleting¶

delete abstractmethod async ¶

delete_folder abstractmethod async ¶

Listing and Iteration¶

list_files abstractmethod async ¶

list_folders abstractmethod async ¶

iter_children async ¶

glob async ¶

Metadata¶

get_file_info abstractmethod async ¶

get_folder_info abstractmethod async ¶

File Operations¶

move abstractmethod async ¶

copy abstractmethod async ¶

Lifecycle¶

aclose async ¶

check_health async ¶

Introspection¶

resolve ¶

Interop (Backend-Specific)¶

unwrap ¶

native_path ¶

to_key ¶

SyncBackendAdapter¶

SyncBackendAdapter ¶

read_bytes `async` ¶

read_text `async` ¶

write `async` ¶

write_text `async` ¶

write_atomic `async` ¶

delete `async` ¶

delete_folder `async` ¶

move `async` ¶

copy `async` ¶

head `async` ¶

exists `async` ¶

is_file `async` ¶

is_folder `async` ¶

get_file_info `async` ¶

get_folder_info `async` ¶

ping `async` ¶

aclose `async` ¶

name `abstractmethod` `property` ¶

capabilities `abstractmethod` `property` ¶

exists `abstractmethod` `async` ¶

is_file `abstractmethod` `async` ¶

is_folder `abstractmethod` `async` ¶

read `abstractmethod` `async` ¶

read_bytes `abstractmethod` `async` ¶

write `abstractmethod` `async` ¶

write_atomic `abstractmethod` `async` ¶

delete `abstractmethod` `async` ¶

delete_folder `abstractmethod` `async` ¶

list_files `abstractmethod` `async` ¶

list_folders `abstractmethod` `async` ¶

iter_children `async` ¶

glob `async` ¶

get_file_info `abstractmethod` `async` ¶

get_folder_info `abstractmethod` `async` ¶

move `abstractmethod` `async` ¶

copy `abstractmethod` `async` ¶

aclose `async` ¶

check_health `async` ¶

name `property` ¶

capabilities `property` ¶

exists `async` ¶

is_file `async` ¶

is_folder `async` ¶

read_bytes `async` ¶

get_file_info `async` ¶

get_folder_info `async` ¶

move `async` ¶

copy `async` ¶

delete `async` ¶

delete_folder `async` ¶

check_health `async` ¶

read `async` ¶

write `async` ¶

write_atomic `async` ¶

list_files `async` ¶

list_folders `async` ¶

glob `async` ¶

iter_children `async` ¶

aclose `async` ¶

name `property` ¶

capabilities `cached` `property` ¶

exists `async` ¶

is_file `async` ¶

is_folder `async` ¶

read `async` ¶

read_bytes `async` ¶

write `async` ¶

write_atomic `async` ¶

delete `async` ¶

delete_folder `async` ¶

list_files `async` ¶

list_folders `async` ¶

iter_children `async` ¶

get_file_info `async` ¶

get_folder_info `async` ¶

move `async` ¶

copy `async` ¶

name `property` ¶

capabilities `property` ¶

check_health `async` ¶