In the Filecoin network, storage providers (SPs) play a vital role by storing data and serving it back to clients upon request. To make the data publically discoverable, the client can choose to advertise the storage deal to the Interplanetary Network Indexer (IPNI), which contains a CID-to-SP mapping. This mapping enables clients to query the IPNI and discover where content is located on the Filecoin network. In this article, we will discuss the functions of storage providers in the Filecoin network, the role of the indexer, and the retrieval process for publicly available data.
When a storage deal is originally made, the client can opt to make the data publically discoverable. If this is the case, the storage provider must publish an advertisement of the storage deal to the Interplanetary Network Indexer (IPNI). The IPNI contains a CID-to-SP mapping. This mapping allows clients to query the IPNI to discover where content is on Filecoin.
The IPNI also tracks which data transfer protocols you can use to retrieve specific CIDs. Currently, Filecoin SPs have the ability to serve retrievals over graphsync, bitswap, and HTTP. This is dependent on the SP setup.
If a client wants to retrieve publicly available data from the Filecoin network, then they generally follow this process.
Query the IPNI
Before the client can submit a retrieval deal to a storage provider, they first need to find which providers hold the data. To do this, the client sends a query to the Interplanetary Network Indexer.
Select a provider
Assuming the IPNI returns more than one storage provider, the client can select which provider they’d like to deal with. Here, they will also get additional details (if needed) based on the retrieval protocol they want to retrieve the content over.
The client then attempts to retrieve the data from the SP over bitswap, graphsync, or HTTP. Note that currently, clients can only get full-piece retrievals using HTTP.
When attempting this retrieval deal using graphsync, payment channels are used to pay FIL to the storage provider. These payment channels watch the data flow and pay the storage provider after each chunk of data is retrieved successfully.
Finalize the retrieval
Once the client has received the last chunk of data, the connection is closed.