zig/dedup.zig

Página generada desde conduit@conduit-0.1.0-f7J_7zg5DwCsHBHItwdOz9ZVq8cvH7ceFsk7-m3-T61t (zig/root.zig). No se edita a mano: bun run docs:gen la regenera y bun run docs:check falla si difiere.

Dedup domains and the index a v2 server consults for have (spec/wire-v2.md §6).

A have answer tells the client whether the server already holds some bytes. Unrestricted, that is a confirmation-of-file oracle (does anybody here hold this exact file?), a learn-the-remaining-information oracle (which of 10⁶ variants of a template "already exists"?) and, if a have let a client attach bytes it never sent, a hash used as a read capability. The rules this module makes structural:

• the Authorizer grants a Domain with each credential: none, the subject (default) or a shared opaque id whose members can already read every object of the domain (a styx household). scope resolves it to an injective Scope (the kind is part of the key, so a subject and a shared id with the same bytes never meet);
• every Index call takes exactly one Scope; there is no call that searches "anywhere". MemoryIndex keeps one table per scope and a lookup touches only the caller's table: other domains change no answer, no error and no timing on the request path;
• what a domain may hold is bounded (MemoryIndex.Limits): objects and blobs per scope, scopes in total, bytes of bookkeeping. A domain that fills its share loses dedup for itself only (record refuses), never space of other domains;
• the index never announces what its store can no longer give back: the server reports every failed read with invalidate, the index forgets that location (MemoryIndex: the whole object) and a later upload that resends those bytes records them afresh. An object announced and lost before complete is a retryable 409 incomplete, never a failed or wrong file (spec §4.4, §5.5);
• the index never decides quotas: the server charges every upload its full logical size, held bytes included (a smaller charge would itself be an oracle, and referencing a held blob 65 536 times must cost what a 65 536-blob file costs).

The index is an interface because the authority over "what does this domain hold" belongs to the consumer (styx: the catalog of a household). MemoryIndex is the reference implementation (CLI serve, tests): objects are entries of a zkit.safety.fs.Root, read fd-relative (styx dec-0117 I7).

dedup.IdCopiar enlace

const · línea 49

pub const Id = [32]u8

B3(bytes): a blob id or the root of a file.

dedup.hashCopiar enlace

fn · línea 51

pub fn hash(bytes: []const u8) Id

Sin ///.

dedup.DomainCopiar enlace

type · línea 58

pub const Domain = union

The dedup domain a grant gives (spec §6.1).

dedup.ScopeCopiar enlace

type · línea 69

pub const Scope = struct

A resolved dedup domain: the key every index operation is confined to.

dedup.Scope.KindCopiar enlace

type · línea 73

pub const Kind = enum

Sin ///.

dedup.Scope.eqlCopiar enlace

fn · línea 75

pub fn eql(a: Scope, b: Scope) bool

Sin ///.

dedup.scopeCopiar enlace

fn · línea 81

pub fn scope(domain: Domain, subject: [32]u8) ?Scope

null = no dedup for this grant.

dedup.BlobRefCopiar enlace

type · línea 90

pub const BlobRef = struct

A blob by content: its id and its exact length.

dedup.ObjectCopiar enlace

type · línea 93

pub const Object = struct

A completed object, as recorded after complete.

dedup.StaleCopiar enlace

type · línea 108

pub const Stale = union

What the store failed to give back (Index.invalidate).

dedup.ReadErrorCopiar enlace

const · línea 115

pub const ReadError = error{ /// Not held (any more) in that scope. NotFound, /// The object could not be read in full. ReadFailed, }

Sin ///.

dedup.RecordErrorCopiar enlace

const · línea 122

pub const RecordError = error{ /// The scope already holds all its share (`MemoryIndex.Limits`). QuotaExceeded, /// Bookkeeping budget exhausted. OutOfMemory, /// Inconsistent object (lengths, spans o …

Sin ///.

dedup.IndexCopiar enlace

type · línea 133

pub const Index = struct

What a server asks about a domain. Every call is confined to s. Implementations must be thread-safe and must not look at other scopes.

dedup.Index.VTableCopiar enlace

type · línea 137

pub const VTable = struct

Sin ///.

dedup.Index.hasObjectCopiar enlace

fn · línea 162

pub fn hasObject(ix: Index, s: Scope, root: Id, size: u64) bool

Sin ///.

dedup.Index.hasBlobsCopiar enlace

fn · línea 166

pub fn hasBlobs(ix: Index, s: Scope, blobs: []const BlobRef, held: []bool) void

Sin ///.

dedup.Index.readBlobCopiar enlace

fn · línea 171

pub fn readBlob(ix: Index, s: Scope, b: BlobRef, offset: u64, dest: []u8) ReadError!void

Sin ///.

dedup.Index.readObjectCopiar enlace

fn · línea 175

pub fn readObject(ix: Index, s: Scope, root: Id, size: u64, offset: u64, dest: []u8) ReadError!void

Sin ///.

dedup.Index.invalidateCopiar enlace

fn · línea 179

pub fn invalidate(ix: Index, s: Scope, stale: Stale) void

Sin ///.

dedup.Index.recordCopiar enlace

fn · línea 183

pub fn record(ix: Index, s: Scope, o: Object) RecordError!void

Sin ///.

dedup.CopyErrorCopiar enlace

const · línea 188

pub const CopyError = error{ /// The store no longer gives the object's exact bytes. The object is /// invalidated already: answer `409 incomplete` and go on as `have: /// "none"` (nothing stored; the …

Sin ///.

dedup.copyHeldObjectCopiar enlace

fn · línea 202

pub fn copyHeldObject(ix: Index, s: Scope, root: Id, size: u64, file: zfs.File, buf: []u8) CopyError!void

Finalize from held (§4.4, have: "all"): copies the object root/size of s into file from offset 0, hashing it as it goes, and checks the hash is root. hasObject answered at create; the object may have gone or changed since, so the copy is what proves it. On Lost the object is already invalidated, so the next create answers have: "none".

dedup.MemoryIndexCopiar enlace

type · línea 231

pub const MemoryIndex = struct

In-memory index over the objects of a Root (reference server, tests). Sessions do not survive a restart and neither does this index: a consumer that wants dedup across restarts implements Index over its catalog.

dedup.MemoryIndex.LimitsCopiar enlace

type · línea 232

pub const Limits = struct

Sin ///.

dedup.MemoryIndex.max_key_lenCopiar enlace

const · línea 243

pub const max_key_len = 256

Sin ///.

dedup.MemoryIndex.initCopiar enlace

fn · línea 269

pub fn init(self: *MemoryIndex, gpa: Allocator, root: *const Root, limits: Limits) error{InvalidLimits}!void

self must not move after init (index() points at it).

dedup.MemoryIndex.deinitCopiar enlace

fn · línea 279

pub fn deinit(self: *MemoryIndex) zkit.safety.LeakReport

Frees everything; the report says whether the bookkeeping was clean.

dedup.MemoryIndex.indexCopiar enlace

fn · línea 291

pub fn index(self: *MemoryIndex) Index

Sin ///.

dedup.MemoryIndex.UsageCopiar enlace

type · línea 309

pub const Usage = struct

Sin ///.

dedup.MemoryIndex.usageCopiar enlace

fn · línea 312

pub fn usage(self: *MemoryIndex, s: Scope) Usage

Objects and distinct blobs recorded for s (tests, metrics).

blobs.IdCopiar enlace

const · línea 49

pub const Id = [32]u8

B3(bytes): a blob id or the root of a file.

blobs.BlobRefCopiar enlace

type · línea 90

pub const BlobRef = struct

A blob by content: its id and its exact length.