IPFS (InterPlanetary File System)

Table of Contents

1. IPFS

IPFS(InterPlanetary File System,星际文件存储系统)是一种 p2p 的分布式文件系统。

当文件被添加到 IPFS 节点后,会产生一个新名字(它是根据文件内容计算出来的哈希值)。当用户想从 IPFS 网络中获取某个文件时,用户就向 IPFS 网络中查询的是该文件的哈希值是否存在,如果存在则去拥有该数据的节点获取对应数据。这是通过一个分布式哈希表(采用了 Kademlia 算法)来实现的,可以简单地理解为分布式哈希表的 Key 为文件哈希值,而 Value 为拥有该文件数据的节点地址。

参考:
IPFS白皮书

2. IPFS 基本使用

2.1. 初始化运行环境

运行 ipfs init ,可以初始化运行环境,生成 ~/.ipfs 目录以及当前节点的 peer identity,如:

$ ipfs init
initializing IPFS node at /home/cig01/.ipfs
generating 2048-bit RSA keypair...done
peer identity: QmNxZBt6z92QJLJ2ewnSxr343af1c6GaTQWaF9BsB4aMYU
to get started, enter:

	ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme

在上面例子中,当前节点的 identity 为“QmNxZBt6z92QJLJ2ewnSxr343af1c6GaTQWaF9BsB4aMYU”。

初始化完成后,运行 ipfs id 可以查看当前节点的 peer identity 等基本信息,如:

$ ipfs id
{
	"ID": "QmNxZBt6z92QJLJ2ewnSxr343af1c6GaTQWaF9BsB4aMYU",
	"PublicKey": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDXkrHfdm6KIEq6FEGUZJB73M4X56eYneU+gyMZeVTRgES4nRZ4Xp+UWvJIo0/ZnqCrF5tdqFiR3Ysr8JynwZFQn5Pw2Ch03aK1nWBb2rg1i5/Y/dayI1Ezm58zOgQknHEdiw39rv/1Dn2ZbC9yLuqA0PVEwpaqQfIOIVuB4ooyaEstQQditDXszpnZcTol6Y54Rj5Z3WcuEQadTzgzyIXtudIO7B8fPXwnlHBbBpQPRGC03yQNgckiCJjDXwuw1wkdryZHhnaVzd+3OdPvGhssuiaMWVFZGel/bfXZtNWL+jQPTRg6Ov3EgMnFZMuCKeNyws3MJL4HC+8js9CK/It1AgMBAAE=",
	"Addresses": null,
	"AgentVersion": "go-ipfs/0.4.17/",
	"ProtocolVersion": "ipfs/0.1.0"
}

2.2. 添加和查看文件

2.2.1. 通过命令行操作

假设文件 1.txt 的内容为:

test file

执行命令 ipfs add 可以添加文件到 IPFS 网络中,执行成功后会返回 hash 值。如:

$ ipfs add 1.txt
added QmShBmhvEZ1dDwPvLvpjimRW76AQrVz3fbpZYwsqNJN2Xh 1.txt
 10 B / 10 B [=========================================================] 100.00%

执行命令 ipfs cat <hash> 可以查看 hash 对应文件的内容。如:

$ ipfs cat QmShBmhvEZ1dDwPvLvpjimRW76AQrVz3fbpZYwsqNJN2Xh
test file

2.2.2. 通过 HTTP API 操作

IPFS 的 HTTP API Endpoint 默认监听在 127.0.0.1:5001,如果修改过,则可以通过 ipfs config show 查看当前配置。

下面是使用 HTTP API 添加文件到 IPFS 网络中的例子:

$ curl -F file=@1.txt "http://localhost:5001/api/v0/add"
{"Name":"1.txt","Hash":"QmShBmhvEZ1dDwPvLvpjimRW76AQrVz3fbpZYwsqNJN2Xh","Size":"18"}

下面是使用 HTTP API 查看 hash 所对应文件内容的例子:

$ curl "http://localhost:5001/api/v0/cat?arg=QmShBmhvEZ1dDwPvLvpjimRW76AQrVz3fbpZYwsqNJN2Xh"
test file

参考:https://docs.ipfs.io/reference/api/http/

2.2.3. 通过 Gateway 查看文件

除了使用 HTTP API 查看文件外,还可以使用 Gateway 端口(默认为 8080,可以通过 ipfs config Addresses.Gateway 查看配置)查看文件,测试如下:

$ curl http://localhost:8080/ipfs/QmShBmhvEZ1dDwPvLvpjimRW76AQrVz3fbpZYwsqNJN2Xh
test file

2.3. 查看和修改配置

使用 ipfs config <key> [<value>] 可以查看和修改 IPFS 配置。如:

$ ipfs config Addresses.API                            # 查看配置(省略value)
/ip4/127.0.0.1/tcp/5001
$ ipfs config Addresses.API /ip4/127.0.0.1/tcp/6001    # 修改配置(指定value)

参考: ipfs config --help

2.3.1. 查看所有配置

运行 ipfs config show 可以查看所有配置,如:

$ ipfs config show
{
  "API": {
    "HTTPHeaders": {
      "Server": [
        "go-ipfs/0.4.17"
      ]
    }
  },
  "Addresses": {
    "API": "/ip4/127.0.0.1/tcp/5001",
    "Announce": [],
    "Gateway": "/ip4/127.0.0.1/tcp/8080",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4001",
      "/ip6/::/tcp/4001"
    ]
  },
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/ipfs/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/ipfs/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/ipfs/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb",
    "/dnsaddr/bootstrap.libp2p.io/ipfs/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.236.179.241/tcp/4001/ipfs/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
    "/ip4/128.199.219.111/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
    "/ip4/104.236.76.40/tcp/4001/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
    "/ip4/178.62.158.247/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd",
    "/ip6/2604:a880:1:20::203:d001/tcp/4001/ipfs/QmSoLPppuBtQSGwKDZT2M73ULpjvfd3aZ6ha4oFGL1KrGM",
    "/ip6/2400:6180:0:d0::151:6001/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu",
    "/ip6/2604:a880:800:10::4a:5001/tcp/4001/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64",
    "/ip6/2a03:b0c0:0:1010::23:1001/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd"
  ],
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "path": "blocks",
            "shardFunc": "/repo/flatfs/shard/v1/next-to-last/2",
            "sync": true,
            "type": "flatfs"
          },
          "mountpoint": "/blocks",
          "prefix": "flatfs.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true,
      "Interval": 10
    }
  },
  "Experimental": {
    "FilestoreEnabled": false,
    "Libp2pStreamMounting": false,
    "ShardingEnabled": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "PathPrefixes": [],
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "QmNxZBt6z92QJLJ2ewnSxr343af1c6GaTQWaF9BsB4aMYU"
  },
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "all"
  },
  "Routing": {
    "Type": "dht"
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {
      "GracePeriod": "20s",
      "HighWater": 900,
      "LowWater": 600,
      "Type": "basic"
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "DisableRelay": false,
    "EnableRelayHop": false
  }
}

3. Pinning

通过 IPFS 某个节点(记为 A)的命令行(或 API)增加数据后,通过另外节点(记为 B)的命令行(或 API)访问该数据时,会先把节点 A 上的该数据缓存到节点 B 中,不过这些缓存是临时的,可能会被定期清理掉(称为 GC)。我们可以通过 Pinning 操作来防止 GC 操作清理掉缓存的数据。

通过 ipfs pin add <ipfs-path> 可以防止缓存的数据被 GC 掉,其它 pin 相关命令可以参考:https://docs.ipfs.io/reference/api/cli/#ipfs-pin

4. Block

IPFS 默认会把大文件拆分成若干个 256KB 大小的块(block),使用 ipfs ls <hash> 可以查看对象包含哪些块,如:

$ ls -l ripgrep_0.10.0_amd64.deb   # 文件ripgrep_0.10.0_amd64.deb超过了256KB
-rw-rw-r-- 1 jack jack 1461290 Oct 26 10:03 ripgrep_0.10.0_amd64.deb
$ ipfs add ripgrep_0.10.0_amd64.deb
added QmS6BjZ5RvmwSpNd1XycqidNuJ2eCNxUUdwhLz9ASDe24u ripgrep_0.10.0_amd64.deb
 1.39 MiB / 1.39 MiB [=================================================] 100.00%
$ ipfs ls QmS6BjZ5RvmwSpNd1XycqidNuJ2eCNxUUdwhLz9ASDe24u   # 上面文件分为了6个块
QmWTWG2eDDCoZyzmduFMQNvxFg6EoU7Y5JjedbUnV2Hcoo 262158
QmexCKzioztpQtgiURVKgAKwNtEDEZugWu3o9vBn1W9ucK 262158
QmPn4siTHJYLcB22FhfRdhznWBJ4AzSPCUfQmVnFN7oxme 262158
QmfCiwR25Q93pTdKEjrwFtARzdG9sBZ3SKWnkdrAHQDKYA 262158
QmaH9RXNKehPtcpnJAiVJurGcvQLmmEm9cUWsirmfA3FHN 262158
QmfUyvJGxVqS99vnk2oBfnoguuwoJAN2mMKZsgurJxG7c8 150584

参考:Dealing With Blocks

5. IPFS 私有网络

Author: cig01

Created: <2018-08-26 Sun>

Last updated: <2020-03-17 Tue>

Creator: Emacs 27.1 (Org mode 9.4)