zig/blobs.zig

Página generada desde conduit@conduit-0.1.0-f7J_7zg5DwCsHBHItwdOz9ZVq8cvH7ceFsk7-m3-T61t (zig/root.zig). No se edita a mano: bun run docs:gen la regenera y bun run docs:check falla si difiere.

Mode blobs of conduit wire v2 (spec/wire-v2.md §5, §6): the manifest, the record bitmaps, the per-session blob state machine of the receiver and the sender's plan of what to PUT.

The file is the concatenation of blobs cut by the client wherever it wants (GOP, FastCDC, fixed); each is named by B3(blob), which does not depend on its position. The client sends a manifest (blobId ‖ length LE per record), the server answers which records it already has (have, confined to the subject's dedup domain: dedup.zig) and the client PUTs only the rest.

Receiver side, sans-IO except for the staging file it is handed:

• ManifestParser: streamed record by record (never more than one record buffered), every rule of §5.2 checked as it arrives, overflow checked, bounded by Limits before allocating;
• Session: set-once manifest, per distinct blob missing → writing → stored (or held from the domain), window, idempotency, chunk_busy;
• BlobWriter: hashes the body once while writing it to every offset the manifest lists for that blob; nothing is marked until the id matches;
• Session.assemble: copies the held blobs from the index (verifying each against its id: a store that changed under us is a lost blob, not a corrupt file; every lost blob is found in the same pass and invalidated in the index) and computes the root in one sequential pass — the declared cost of this mode (§5.5).

The caller (a server) serializes calls on one Session (its lock), pins the session while a BlobWriter streams, and charges the staging quota the full totalSize at create, held bytes included (§6.2).

Rules (styx dec-0117 B4/I5): exact lengths, lowercase hex only, no unreachable on input, every allocation through the caller's allocator (the server's session budget) and released on every path.

blobs.record_lenCopiar enlace

const · línea 47

pub const record_len = 40

One manifest record: blobId (32) ‖ length (u64 LE).

blobs.LimitsCopiar enlace

type · línea 49

pub const Limits = struct

Sin ///.

blobs.validCreateCopiar enlace

fn · línea 61

pub fn validCreate(l: Limits, total_size: u64, total_blobs: u64) bool

Is create's {totalSize, totalBlobs} acceptable under l? (§5.1: totalBlobs ≤ totalSize, ≤ maxBlobs, 0 only with totalSize 0, and the size reachable with blobs of at most maxBlob.)

blobs.manifestLenCopiar enlace

fn · línea 69

pub fn manifestLen(total_blobs: u32) u64

Exact body length of the manifest (Content-Length, checked before reading).

blobs.hexIdCopiar enlace

fn · línea 74

pub fn hexId(id: Id) [64]u8

Hex form of an id (paths and JSON).

blobs.parseHexIdCopiar enlace

fn · línea 79

pub fn parseHexId(s: []const u8) wire.ParseError!Id

64 lowercase hex characters, nothing else.

blobs.RouteCopiar enlace

type · línea 92

pub const Route = union

Sin ///.

blobs.routeCopiar enlace

fn · línea 100

pub fn route(base: []const u8, method: []const u8, path: []const u8) wire.RouteError!?Route

The two routes v2 adds, both PUT. null = not one of them (the v1 router decides: other methods and shapes stay not_found there). A matching shape with a bad id or blob id is BadRequest.

blobs.writeManifestResponseCopiar enlace

fn · línea 125

pub fn writeManifestResponse(w: *Writer, upload_id: []const u8, s: *const Session) Writer.Error!void

{"uploadId":"…","have":"<bitmap>","missing":n} from the session's state.

blobs.writeBlobResponseCopiar enlace

fn · línea 132

pub fn writeBlobResponse(w: *Writer, upload_id: []const u8, id: Id, bytes: u64) Writer.Error!void

{"uploadId":"…","blobId":"…","bytesReceived":n}.

blobs.ManifestResponseCopiar enlace

type · línea 136

pub const ManifestResponse = struct

Sin ///.

blobs.decodeManifestResponseCopiar enlace

fn · línea 146

pub fn decodeManifestResponse(arena: Allocator, bytes: []const u8, records: u32) wire.DecodeError!ManifestResponse

Client side of the manifest response. have is checked against the number of records the client sent and missing against have (a server that contradicts itself is not believed). have lives in arena.

blobs.bitmapLenCopiar enlace

fn · línea 166

pub fn bitmapLen(n: u32) usize

⌈n/8⌉ bytes.

blobs.bitIsSetCopiar enlace

fn · línea 170

pub fn bitIsSet(bitmap: []const u8, i: u32) bool

Sin ///.

blobs.writeBitmapCopiar enlace

fn · línea 179

pub fn writeBitmap(w: *Writer, bitmap: []const u8) Writer.Error!void

Base64 (standard, padded) of a packed bitmap, streamed in 48-byte pieces.

blobs.decodeBitmapCopiar enlace

fn · línea 190

pub fn decodeBitmap(dest: []u8, s: []const u8, n: u32) wire.ParseError!void

Decodes s into dest (bitmapLen(n) bytes): exact length, padded standard alphabet, spare bits zero; anything else is Invalid.

blobs.ManifestErrorCopiar enlace

const · línea 202

pub const ManifestError = error{ Invalid, OutOfMemory }

Sin ///.

blobs.ManifestCopiar enlace

type · línea 207

pub const Manifest = struct

A validated manifest. Owned by whoever holds it (deinit).

blobs.Manifest.deinitCopiar enlace

fn · línea 223

pub fn deinit(m: *Manifest, gpa: Allocator) void

Sin ///.

blobs.Manifest.recordsCopiar enlace

fn · línea 234

pub fn records(m: *const Manifest) u32

Sin ///.

blobs.Manifest.occurrencesCopiar enlace

fn · línea 239

pub fn occurrences(m: *const Manifest, b: u32) Occurrences

Records of blob b, in file order.

blobs.Manifest.OccurrencesCopiar enlace

type · línea 243

pub const Occurrences = struct

Sin ///.

blobs.Manifest.Occurrences.nextCopiar enlace

fn · línea 247

pub fn next(it: *Occurrences) ?u64

Sin ///.

blobs.ManifestParserCopiar enlace

type · línea 258

pub const ManifestParser = struct

Validates a manifest body as it streams in. feed any split of the bytes, then finish; deinit always (it frees whatever finish did not hand out).

blobs.ManifestParser.initCopiar enlace

fn · línea 278

pub fn init(gpa: Allocator, limits: Limits, total_size: u64, total_blobs: u32) ManifestError!ManifestParser

Sin ///.

blobs.ManifestParser.deinitCopiar enlace

fn · línea 289

pub fn deinit(p: *ManifestParser) void

Sin ///.

blobs.ManifestParser.feedCopiar enlace

fn · línea 301

pub fn feed(p: *ManifestParser, bytes: []const u8) ManifestError!void

Sin ///.

blobs.ManifestParser.finishCopiar enlace

fn · línea 354

pub fn finish(p: *ManifestParser) ManifestError!Manifest

The manifest, if the body was exactly totalBlobs records adding up to totalSize. On success the parser is left empty.

blobs.parseManifestCopiar enlace

fn · línea 381

pub fn parseManifest(gpa: Allocator, limits: Limits, total_size: u64, total_blobs: u32, bytes: []const u8) ManifestError!Manifest

Parses a whole manifest held in memory (client tests, vectors).

blobs.writeManifestCopiar enlace

fn · línea 390

pub fn writeManifest(w: *Writer, records: []const BlobRef) Writer.Error!void

The manifest body for records (file order). The client's side of §5.2.

blobs.BlobStateCopiar enlace

type · línea 401

pub const BlobState = enum

Sin ///.

blobs.SlotCopiar enlace

type · línea 412

pub const Slot = struct

Sin ///.

blobs.BeginCopiar enlace

type · línea 414

pub const Begin = union

Sin ///.

blobs.SetManifestCopiar enlace

type · línea 423

pub const SetManifest = enum

Sin ///.

blobs.SessionCopiar enlace

type · línea 430

pub const Session = struct

Sin ///.

blobs.Session.initCopiar enlace

fn · línea 442

pub fn init(gpa: Allocator, limits: Limits, total_size: u64, total_blobs: u32) error{Invalid}!Session

Sin ///.

blobs.Session.deinitCopiar enlace

fn · línea 447

pub fn deinit(s: *Session) void

Sin ///.

blobs.Session.parserCopiar enlace

fn · línea 454

pub fn parser(s: *const Session) ManifestError!ManifestParser

A parser for this session's manifest body.

blobs.Session.setManifestCopiar enlace

fn · línea 460

pub fn setManifest(s: *Session, m: Manifest) error{ Conflict, OutOfMemory }!SetManifest

Takes ownership of m in every case. A second, different manifest is error.Conflict (409 chunk_conflict).

blobs.Session.blobRefsCopiar enlace

fn · línea 476

pub fn blobRefs(s: *const Session) []const BlobRef

Distinct blobs of the manifest (to ask the index), &.{} before it.

blobs.Session.applyHeldCopiar enlace

fn · línea 484

pub fn applyHeld(s: *Session, held: []const bool) void

Marks as held the missing blobs the index holds in the session's domain (held.len == blobRefs().len). Never downgrades a blob that is being written or already stored.

blobs.Session.missingCopiar enlace

fn · línea 506

pub fn missing(s: *const Session) u32

Records the client still has to send (the manifest response's missing).

blobs.Session.confirmedCopiar enlace

fn · línea 511

pub fn confirmed(s: *const Session) u32

Records with their blob present (status confirmed).

blobs.Session.readyCopiar enlace

fn · línea 516

pub fn ready(s: *const Session) bool

Every record present: complete may assemble.

blobs.Session.writePresentCopiar enlace

fn · línea 523

pub fn writePresent(s: *const Session, w: *Writer) Writer.Error!void

The bitmap over the records with their blob held or stored: the manifest response's have and the status's stored (§5.2, §5.3). Before the manifest, all zero over totalBlobs.

blobs.Session.beginCopiar enlace

fn · línea 547

pub fn begin(s: *Session, id: Id, content_length: ?u64) Begin

Admission of PUT blob/{id} before its body is read (§5.3).

blobs.Session.endCopiar enlace

fn · línea 566

pub fn end(s: *Session, slot: Slot, verified: bool) void

Ends a begin(.write): verified marks the blob stored at every offset; otherwise it is missing again (bytes written are overwritten by the next attempt and never visible: staging).

blobs.Session.settleCopiar enlace

fn · línea 583

pub fn settle(s: *Session, lost: []const bool) void

After assemble answered HeldLost (under the session lock): the held blobs marked in lost become missing (complete answers 409 incomplete and the client, after one status, sends all of them); the other held blobs were copied into staging and verified in that same pass, so they become stored and the next assemble does not copy them again.

blobs.Session.AssembleErrorCopiar enlace

const · línea 596

pub const AssembleError = error{ /// Not every record is present. NotReady, /// Some held blobs (marked in `lost`) could not be given back with /// their exact bytes; they are invalidated in the index …

Sin ///.

blobs.Session.assembleCopiar enlace

fn · línea 615

pub fn assemble(s: *const Session, ix: dedup.Index, sc: ?dedup.Scope, file: zfs.File, buf: []u8, lost: []bool) AssembleError!Id

Copies the held blobs into file at every offset (verifying each against its id), then hashes the assembled file once: its root. A held blob the store does not give back is marked in lost (blobRefs().len entries, all set by this call), invalidated in the index (Index.invalidate) and the pass goes on with the next one, so one HeldLost reports every lost blob: one 409 round, never one per blob. Runs without the session lock but with the session pinned and no writer in flight (every blob is held or stored, so begin admits none).

blobs.Session.objectCopiar enlace

fn · línea 671

pub fn object(s: *const Session, root: Id, key: []const u8) dedup.Object

What to record in the index once the object is final under key.

blobs.BlobWriterCopiar enlace

type · línea 680

pub const BlobWriter = struct

Streams one blob's body: hashed once, written at every offset the manifest lists for it. The session must stay pinned (its manifest is immutable once set, so the writer reads it without the lock).

blobs.BlobWriter.initCopiar enlace

fn · línea 685

pub fn init(slot: Slot) BlobWriter

Sin ///.

blobs.BlobWriter.WriteErrorCopiar enlace

const · línea 689

pub const WriteError = error{ TooLong, Io }

Sin ///.

blobs.BlobWriter.writeCopiar enlace

fn · línea 691

pub fn write(w: *BlobWriter, m: *const Manifest, file: zfs.File, bytes: []const u8) WriteError!void

Sin ///.

blobs.BlobWriter.verifiedCopiar enlace

fn · línea 700

pub fn verified(w: *const BlobWriter) bool

true = the whole blob arrived and its hash is its id.

blobs.PlanErrorCopiar enlace

const · línea 710

pub const PlanError = error{ Invalid, OutOfMemory }

Sin ///.

blobs.toSendCopiar enlace

fn · línea 716

pub fn toSend(gpa: Allocator, records: []const BlobRef, have: []const u8) PlanError![]u32

Which records to PUT given the server's have bitmap (decoded) over records: the first occurrence of each blob not present. A have that marks one occurrence of a blob and not another is inconsistent (the server marks blobs, not positions) and is Invalid. Caller frees the result.