Preparing and transferring data

Preparing data for storage

Before a system file (e.g. puppy.gif) can be stored on the Filecoin network, it must first be transformed into a Filecoin Piece.

In the first stage of this transformation, the system file is chunked up with UnixFS to create an IPLD DAG (Directed Acyclic Graph). You can learn more about DAGs (a form of merkle tree) in our Content Addressing on the Decentralized Web tutorial. This IPLD DAG has a payload CID, identical to an IPFS CID, which represents the root of the DAG.

The IPLD DAG is then serialized to a CAR file and bit padded to make a Filecoin Piece. (Bit padding adds extra bits to make the piece conform to a standard size.) This piece has a unique piece CID, also known as a CommP (Piece Commitment).

Since payload CIDs and piece CIDs are cryptographic hashes of the data itself, they're unique, with identical CIDs guaranteeing identical content. Identical IPLD DAGs will produce identical payload CIDs and identical pieces will produce identical piece CIDs, no matter who's going to store or retrieve them.

Negotiating a storage deal and transferring data

When a client negotiates a storage deal with a miner, they're hiring them to store a piece of data, which might be a whole or partial file. Miners store these pieces from one or more clients in sectors, the fundamental storage unit used by Filecoin. Sectors come in a variety of sizes, and a client can store data up to the largest sector size per deal. (Learn more about sector sizes and storing large files.)

A piece CID is wrapped with other deal parameters to create a Deal Proposal. The deal CID contains information about the data itself, in the form of the piece CID, the identities of the miner and client, and other important transaction details.

The client sends this deal proposal to a miner, who agrees to store their data. Once the miner has confirmed, the client transfers their data to the miner. Once the miner has the data and verifies that it matches the piece CID noted in the deal proposal, they publish the deal proposal on Filecoin's blockchain, committing both parties to the deal.