Skip to content

SFTP Backend Specification

Overview

SFTPBackend implements the Backend ABC for SSH File Transfer Protocol (SFTP) servers using pure paramiko internally. It maps the Backend contract onto a real remote filesystem accessed over SSH/SFTP.

Unlike fsspec's SFTPFileSystem (which hardcodes AutoAddPolicy), this backend provides explicit host key policy control via a HostKeyPolicy enum, PEM key sanitization for Azure Key Vault compatibility, and tenacity-based retry for transient SSH errors.

Dependencies: paramiko, tenacity (optional extra: pip install "remote-store[sftp]")

Construction

SFTP-001: Constructor Parameters

Invariant: SFTPBackend is constructed with a required host and optional connection/authentication parameters. Signature: 

SFTPBackend(
    host: str,
    *,
    port: int = 22,
    username: str | None = None,
    password: str | None = None,
    pkey: Any = None,                   # paramiko.PKey, lazy-typed
    base_path: str = "/",               # root on remote server
    host_key_policy: HostKeyPolicy = HostKeyPolicy.STRICT,
    known_host_keys: str | None = None,
    host_keys_path: str | None = None,  # defaults to ~/.ssh/known_hosts
    config: dict | None = None,         # may contain "known_host_keys"
    timeout: int = 10,
    connect_kwargs: dict | None = None, # extra SSHClient.connect() kwargs
)
Postconditions: The backend stores configuration but does not connect during construction (see SFTP-004).

SFTP-002: Backend Name

Invariant: name property returns "sftp".

SFTP-003: Capability Declaration

Invariant: SFTPBackend declares capabilities: READ, WRITE, DELETE, LIST, MOVE, COPY, ATOMIC_WRITE, METADATA, WRITE_RESULT_NATIVE. Does not declare GLOB (no native pattern matching; use list_files(pattern=…) or ext.glob for client-side fallback). Rationale: - WRITE_RESULT_NATIVE: write() and write_atomic() populate size (counted during upload) and source from the write path itself. Every rich field (etag / version_id / last_modified / digest) is None — SFTP's write response carries no metadata at all, and the backend does not stat after upload/rename to fetch any (BK-313: that round-trip was paid on every write). WR-001a permits None for a field the write response omits, so the declaration rests on size / source; callers needing the metadata call get_file_info(). - ATOMIC_WRITE: Simulated via temp file + rename (see SFTP-014). Orphan temp files are possible on connection failure — documented caveat. - MOVE: Implemented via posix_rename with fallback (see SFTP-018). - COPY: Implemented via read + write (no server-side copy in SFTP, see SFTP-019). - Read-path directory rejection is lazy for read_bytes (BK-313: the eager stat was the round-trip removed): a directory target raises InvalidPath only because reading it fails. This assumes the server either refuses to open a directory for reading or reports a non-zero directory st_size — both hold on OpenSSH, where a directory reports st_size == 4096. A non-standard server that opens a directory for reading and reports st_size == 0 would make read_bytes return empty bytes rather than raising InvalidPath. read (streaming) keeps an eager check instead, because a streaming read never issues the in-band I/O that would surface the directory. Accepted as a documented server assumption (audit-020 M3).

SFTP-004: Lazy Connection

Invariant: No network call occurs during __init__. The SSH/SFTP connection is established lazily on first operation. Rationale: Fail-fast at construction is undesirable — the backend may be created during application wiring before the network is available. Automatic reconnection on staleness is also supported (see SFTP-010).

SFTP-005: Construction Validation

Invariant: host must be a non-empty string. Passing an empty or whitespace-only host raises ValueError at construction time. Postconditions: No network validation of host reachability at construction time.

Connection

SFTP-006: HostKeyPolicy Enum

Invariant: HostKeyPolicy controls how unknown remote host keys are handled: - STRICT (default): Reject unknown hosts. Requires host key in known_hosts. - TRUST_ON_FIRST_USE: Accept and save on first connect, verify on subsequent connects. - AUTO_ADD: Accept any key. Development/testing only — not safe for production.

String values ("strict", "tofu", "auto") passed from TOML/YAML config are coerced to the enum in __init__ via HostKeyPolicy(value). The enum-name forms ("STRICT", "TRUST_ON_FIRST_USE", "AUTO_ADD") are also accepted, case-insensitive on the name ("auto_add" and "Auto_Add" both resolve to AUTO_ADD); value-form aliasing (e.g. "AUTO" for canonical value "auto") is not folded and continues to raise ValueError. Invalid strings, and any non-string input, raise ValueError. See 020-credential-hygiene.md SEC-005.

SFTP-007: Host Key Resolution Chain

Invariant: Known host keys are resolved with first-match precedence: 1. known_host_keys constructor parameter (code-level override) 2. config["known_host_keys"] dict value 3. SFTP_KNOWN_HOST_KEYS environment variable 4. host_keys_path file on disk (default: ~/.ssh/known_hosts)

Postconditions: If none of the above yield keys and the policy is STRICT, connection will fail with a host key verification error.

SFTP-008: PEM Key Sanitization

Invariant: _sanitize_pem() normalizes PEM line separators, handling the Azure Key Vault quirk where newlines may be replaced with spaces or other characters. Postconditions: The sanitized PEM string has standard \n line separators within the Base64 payload. Invalid PEM structures (not 5 parts) raise ValueError.

SFTP-009: Tenacity Retry on Connect

Invariant: The _connect() method retries on transient SSH errors using tenacity. When no RetryPolicy is provided, uses defaults: 3 attempts, exponential backoff (2s min, 10s max). When a RetryPolicy is provided via the retry constructor parameter, maps its fields to tenacity: max_attempts -> stop_after_attempt, backoff_base -> wait_exponential(min=), backoff_max -> wait_exponential(max=), jitter -> wait_random(0, jitter), timeout -> stop_after_delay. See also: spec 025-retry-policy.md (RET-010). Retried exceptions: paramiko.SSHException, OSError, EOFError. Postconditions: After all retries are exhausted, the original exception is reraised.

SFTP-010: Staleness Detection and Reconnect

Invariant: Staleness is detected in two tiers, neither of which spends a per-operation round-trip. (1) The lazy _sftp property reads the SSH transport's is_active() flag — a local check, no bytes on the wire; a transport that has gone inactive is reconnected before the operation runs. (2) A drop that leaves the transport flag True but the SFTP channel dead (idle-channel timeout, subsystem restart, half-open partition) is invisible to tier 1; it is caught on the operation itself and mapped to BackendUnavailable, which invalidates the cached client so the next _sftp access reconnects. The trigger is the conclusion that the connection is unusable, not any single signal: every mapping that concludes BackendUnavailable clears the client, across the full dead-connection signal set (EOFError, OSError('Socket is closed'), the socket-teardown errnos, socket.timeout, and the paramiko SFTPError / SSHException / ChannelException families — see SFTP-023). Anchoring recovery to that conclusion rather than an enumerated list is what keeps a signal the list forgot (e.g. a socket.timeout from the channel timeout) from wedging the long-lived backend. Operations outside the default _errors() scope must still route through this mapping for the guarantee to hold: the listing operations route their failure through _map_exception, and open_atomic's streamed-write phase — which yields the handle outside _errors() — routes its backend failures (a dead channel or any paramiko SSH / protocol error surfaced during the caller's writes) through _map_exception too, while its temp-file open and promote steps run inside _errors(). Rationale: The property is accessed several times per operation, so the former stat('.') liveness probe multiplied each operation's RTT count — but it doubled as a universal self-heal, reconnecting a dead client of any kind on the next op. The transport flag costs nothing on the wire; tier 2 restores that self-heal for a channel-only death by clearing the client on every BackendUnavailable, without adding a probe to the happy path. Postconditions: A healthy connection is reused with no per-operation probe round-trip. A dropped connection surfaces as BackendUnavailable and the following call re-establishes it — recovery may take one failed call when the drop is channel-only (tier 2).

Filesystem Model

SFTP-011: Real Directories

Invariant: SFTP operates on a real remote filesystem with actual directories, unlike S3's virtual prefix-based folders. is_folder() uses stat() + S_ISDIR. Postconditions: Folders exist independently of their contents.

SFTP-012: Write Creates Intermediate Directories

Invariant: write("a/b/c.txt", content) creates intermediate directories a/ and a/b/ if they do not exist. Rationale: SFTP servers reject writes to non-existent directories. Creating them automatically matches the convenience of local and S3 backends.

SFTP-013: Empty Folders Persist

Invariant: Unlike S3 (where folders vanish when empty), empty directories on an SFTP server persist after their contents are deleted. Postconditions: is_folder("dir") returns True even after all files under dir/ are deleted.

Operations

SFTP-014: Atomic Write (Simulated)

Invariant: write_atomic writes to a temporary file .~tmp.<name>.<uuid8> in the same directory as the target, then renames to the target via posix_rename. Caveat: If the connection drops between write and rename, the orphan temp file remains. This is simulated atomicity, not true atomicity — the capability is declared to enable the write-then-rename pattern, but the caveat must be documented. Postconditions: On success, the temp file is gone and the target contains the new content. On failure, the backend makes a best-effort temp-file cleanup that never reconnects: when the failure is itself a dropped-connection signal (or the client is already invalidated), the cleanup unlink is deliberately skipped rather than triggering a fresh connect against a possibly-down server inside the error-handling path — so the orphan-temp caveat above holds and the original error propagates without a multi-second reconnect stall. An abnormal exit of an open_atomic block (including a GeneratorExit / KeyboardInterrupt) removes the temp file under the same best-effort guard.

SFTP-015: Atomic Write Overwrite Semantics

Invariant: write_atomic(path, content, overwrite=False) raises AlreadyExists if the target already exists. With overwrite=True, the existing file is replaced.

SFTP-016: delete_folder Recursive

Invariant: delete_folder(path, recursive=True) walks the directory tree bottom-up, deleting files then directories. Raises: NotFound if the folder does not exist and missing_ok=False.

SFTP-017: delete_folder Non-Recursive

Invariant: delete_folder(path, recursive=False) succeeds only if the directory is empty. Raises: NotFound if missing. RemoteStoreError if the directory is not empty.

SFTP-018: Move Via posix_rename

Invariant: move(src, dst) attempts posix_rename (atomic overwrite), falls back to rename, and falls back to copy + delete if rename fails entirely. Raises: NotFound if src does not exist. AlreadyExists if dst exists and overwrite=False.

SFTP-019: Copy Via Read + Write

Invariant: copy(src, dst) reads the source file and writes it to the destination. There is no server-side copy operation in SFTP — data passes through the client. Raises: NotFound if src does not exist. AlreadyExists if dst exists and overwrite=False.

Error Mapping

SFTP-020: NotFound Mapping

Invariant: IOError with errno.ENOENT (errno 2) and FileNotFoundError are mapped to NotFound. Postconditions: path and backend attributes are set on the error.

SFTP-021: PermissionDenied Mapping

Invariant: IOError with errno.EACCES (errno 13) is mapped to PermissionDenied.

SFTP-022: AlreadyExists Mapping

Invariant: IOError with errno.EEXIST (errno 17) is mapped to AlreadyExists.

SFTP-023: BackendUnavailable Mapping

Invariant: paramiko.SSHException and its subclasses (authentication failures, ChannelException, etc.) are mapped to BackendUnavailable. So are the dropped- connection signals that are not SSHException subclasses: EOFError, OSError('Socket is closed') (no errno), OSError with errno in ECONNRESET / EPIPE / ECONNABORTED / ETIMEDOUT / ESHUTDOWN / ENOTCONN / EBADF, socket.timeout / TimeoutError (matched by type, since a half-open instance often carries no matching errno), and paramiko.SFTPError (an SFTP-protocol failure that subclasses neither OSError nor SSHException). Every BackendUnavailable this mapping returns — the SSHException family included — invalidates the cached SFTP client so the next operation reconnects (see SFTP-010, tier 2). The list is not the guarantee: recovery is anchored to the BackendUnavailable conclusion, so a dead-connection signal the list has not enumerated still clears the client rather than wedging the backend. One exception keeps its own branch first: IncompatiblePeer (a connect-time SSHException) is mapped with a diagnostic hint before the generic SSHException mapping, so the hint is not lost.

SFTP-024: No Native Exception Leakage

Invariant: No paramiko, socket, or OS exception raised by the backend — an operation's own I/O, including open_atomic's temp-file open, the caller-facing handle's flush/close, and the promote — propagates to callers; all are mapped to remote_store error types per BE-021. The only non-mapped exceptions are those the caller raises inside an open_atomic yield block (their with body) that are not themselves dead-connection signals: those are not the backend's and propagate unchanged, leaving the target untouched. open_atomic distinguishes by scope — the temp open, flush, and promote run inside _errors(), while the yielded write does not; a dead-connection signal surfacing from that write is still mapped to BackendUnavailable (it is indistinguishable from a real drop). Postconditions: backend attribute is set to "sftp" on all mapped errors.

Resource Management

SFTP-025: close()

Invariant: close() closes both the SFTP client and the underlying SSH transport. Postconditions: Safe to call multiple times (idempotent). After close, further operations will trigger a new connection via lazy init.

SFTP-026: unwrap(SFTPClient)

Invariant: unwrap(paramiko.SFTPClient) returns the underlying SFTP client. Raises: CapabilityNotSupported for any other type hint. Rationale: Escape hatch for users who need paramiko-specific features (per ADR-0003).

SFTP-027: Idempotent Close

Invariant: Calling close() multiple times must not raise. Internal state is set to None after close, and the next operation will reconnect lazily.

SFTP-028: TOFU Host Key Persistence

Invariant: When host_key_policy is TRUST_ON_FIRST_USE and keys are resolved from the file-based path (not from inline known_host_keys, config, or environment), the backend persists newly accepted host keys to disk on disconnect.

Preconditions:

  • _resolved_host_keys is None (no inline keys).
  • Policy is TRUST_ON_FIRST_USE.

Postconditions:

  • The known_hosts file (default ~/.ssh/known_hosts or host_keys_path) and its parent directory are created if absent, with 0o700 directory / 0o644 file permissions (best-effort on Windows).
  • load_host_keys(path) is always called so paramiko records the filename internally.
  • save_host_keys(path) is called in _close_clients() before SSH client closure.
  • On reconnection, keys saved during the previous session are loaded back.
  • Save failures are suppressed — they must not prevent connection teardown.
  • Inline keys (known_host_keys parameter, config dict, or env var) are never persisted to disk.

Concurrency

SFTP-029: Concurrent-Use Posture

Invariant: SFTPBackend is single_connection (the BE-028 non-default posture): a single instance drives one paramiko SFTPClient over one SSH channel, which is not safe for concurrent use. Concurrent operations on one instance race on the shared channel and may interleave or corrupt protocol state.

Remedy: Use one instance per thread, or drive it through AsyncBackendSyncAdapter (which funnels concurrent callers onto a single private loop and serializes them — ASYNC-089). A single_connection backend wrapped by SyncBackendAdapter and driven with asyncio.gather is not safe (ASYNC-094). This pins the caveat previously stated only in the class docstring and docs-src/guides/async.md.

See also: 003-backend-adapter-contract.md (BE-028), 029-async-store-backend-api.md (ASYNC-094).