From 1b9ca884ead29a978e8cdc44b9ba2ad70f33725b Mon Sep 17 00:00:00 2001 From: Daniel N <2color@users.noreply.github.com> Date: Thu, 25 Jul 2024 17:57:01 +0200 Subject: [PATCH 1/6] feat: new ipfs on the web docs page --- docs/.vuepress/config.js | 6 ++-- docs/.vuepress/redirects | 1 + docs/how-to/browser-tools-frameworks.md | 32 ----------------- docs/how-to/ipfs-on-the-web.md | 46 +++++++++++++++++++++++++ 4 files changed, 50 insertions(+), 35 deletions(-) delete mode 100644 docs/how-to/browser-tools-frameworks.md create mode 100644 docs/how-to/ipfs-on-the-web.md diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index b2f7b4978..ebd54ba3c 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -255,12 +255,12 @@ module.exports = { ] }, { - title: 'IPFS in the browser', + title: 'IPFS on the web', sidebarDepth: 1, collapsable: true, children: [ + '/how-to/ipfs-on-the-web', '/how-to/address-ipfs-on-web', - '/how-to/browser-tools-frameworks' ] }, { @@ -277,7 +277,7 @@ module.exports = { collapsable: true, children: [ '/how-to/gateway-best-practices', - '/how-to/gateway-troubleshooting' + '/how-to/gateway-troubleshooting', ] }, { diff --git a/docs/.vuepress/redirects b/docs/.vuepress/redirects index e2954dfd7..92b46067a 100644 --- a/docs/.vuepress/redirects +++ b/docs/.vuepress/redirects @@ -40,6 +40,7 @@ /guides/guides/addressing/ /how-to/address-ipfs-on-web /guides/guides/install/ /install /how-to/host-single-page-site /how-to/websites-on-ipfs/single-page-website +how-to/browser-tools-frameworks /how-to/ipfs-on-the-web/ /how-to/troubleshoot-file-transfers /how-to/troubleshooting /how-to/run-ipfs-inside-docker /install/run-ipfs-inside-docker /install/command-line-quick-start/ /how-to/command-line-quick-start diff --git a/docs/how-to/browser-tools-frameworks.md b/docs/how-to/browser-tools-frameworks.md deleted file mode 100644 index 9ed1e49d7..000000000 --- a/docs/how-to/browser-tools-frameworks.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Browser tools and frameworks -description: Information on how to use IPFS in combination with your favorite frameworks or browser implementation tools. ---- - -# Browser Tools and Frameworks - -Want to learn how to use IPFS in combination with your favorite framework or browser implementation tool? See below for some common examples, including hints and boilerplate code. Don't see your framework or tool of choice here? Let us know by [filing an issue](https://github.com/ipfs/ipfs-docs/issues/new?assignees=&labels=OKR+3%3A+Content+Improvement%2C+docs-ipfs&template=content-request.md&title=%5BCONTENT+REQUEST%5D+%20Browser%20Implementation%20Tools%20and%20Frameworks) in the IPFS docs GitHub repo. - -## Browserify - -[Example and boilerplate](https://github.com/ipfs-examples/js-ipfs-browser-browserify) you can use to guide yourself into bundling js-ipfs with browserify, so that you can use it in your own web app. - -## Parcel.js - -[Example and boilerplate application](https://github.com/ipfs-examples/js-ipfs-examples/tree/master/examples/browser-parceljs) that connects to IPFS using js-ipfs and is bundled with Parcel.js, so that you can follow it for creating Parcel.js bundled js-ipfs dapps. - -## Vue - -[Minimal demonstration](https://github.com/ipfs-examples/js-ipfs-examples/tree/master/examples/browser-vue) of how to use js-ipfs with Vue. - -## Webpack - -[Example and boilerplate](https://github.com/ipfs-examples/js-ipfs-examples/tree/master/examples/browser-webpack) you can use to guide yourself into bundling js-ipfs with webpack, so that you can use it in your own web app. - -## With a ` + +## Providing data + +In the examples above you saw how to address data by CID. But what you do with it depends on your use case. + +If you want to data to be retrievable by other peers on [Mainnet](../concepts/glossary.md#ipfs-mainnet) you will want to upload it to a pinning service or an IPFS node you run. + +### You probably don't want to provide data from a browser + +Browsers make for lousy servers. It's difficult to make a Web page "dialable", i.e. allow network incoming connections from other computers. There's one exception, namely WebRTC, however, it has many caveats. + +For this reason, you should almost never count on providing data from a browser to work. + +Instead, you should provide data from a long-running server that runs reliably and has a public IP. That can be a Kubo node that you run, or a [pinning service](../concepts/persistence.md#pinning-services). + + +### CAR files + +### Example: Uploading a file to a pinning service + +TODO: + +## Retrieval + +### With Verified-fetch + +

+ See the Pen + Fetch an image on IPFS Mainnet @helia/verified-fetch by Daniel Norman (@2color) + on CodePen. +

+ diff --git a/docs/how-to/ipfs-on-the-web.md b/docs/how-to/ipfs-on-the-web.md deleted file mode 100644 index ef8f15de7..000000000 --- a/docs/how-to/ipfs-on-the-web.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -title: IPFS on the web -description: How to use IPFS on the web, including IPFS retrieval and pinning in browsers using implementations. ---- - -# IPFS on the web - -## Challenges with IPFS on the Web - -One of the main benefits of IPFS is that CIDs allow you to fetch data from anyone **trustlessly** through hash verification. This foundation enables IPFS to function as a permissionless peer-to-peer network. - -However, making all of this work on the web is tricky. While web browsers have the widest proliferation as an application runtime, they impose many restrictions on what web pages can do. For example, a web page cannot just open TCP/UDP connections. It is mostly constrained to HTTP, WebSockets, WebRTC, and most recently WebTransport. There are good reasons for this like security and resource management, but ultimately, it means that **using IPFS on the Web is different to IPFS in native binaries.** - -Thus, the Web is both a **resource-constrained and API-constrained** environment. So practically everything on this page is applicaable - -## Onboarding, Retrieving, and Providing - -Ultimately, there are three things that you do with IPFS: -- Onboard data, i.e. take arbitrary data, chunk it, and structure it as a Merkle DAG represented by a single CID. -- Retrieve files and arbitrary data. Given a CID, you use IPFS to find providers (peers who share the block), connect to them *somehow*, fetch the blocks, and verify from peers who have the CID. Once that's all done, you save the block locally. -- Provide the block to others. - -### You probably don't want to rely on the web for providing - -Browsers make for lousy servers. It's almost impossible to make a web page "dialable", i.e. allow other computers to connect to it (the exception is WebRTC with many caveats). - -For this reason, for CIDs to it's recommended to delegate providing to an external server that runs reliably and has a public IP. That can be a Kubo node that you run, or a [pinning service](../concepts/persistence.md#pinning-services). - -## Onboarding - -Even when relying on another server for providing, it's recommended to handle chunking and Merkle-DAG construction locally in the browser and uploading a [CAR file](../concepts/glossary.md#car). This reduces the trust surface and allows you to know the CID ahead of the upload. - -## Retrieval - -### With Verified-fetch - - - - - - - From 3f6545c5fd7c92c9a995e672ae1fcd47004ed937 Mon Sep 17 00:00:00 2001 From: Daniel N <2color@users.noreply.github.com> Date: Mon, 16 Dec 2024 13:02:21 +0100 Subject: [PATCH 3/6] fix: web apps guide --- docs/.vuepress/config.js | 2 +- docs/how-to/ipfs-in-web-apps.md | 24 +++++++++--------------- 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/docs/.vuepress/config.js b/docs/.vuepress/config.js index ebd54ba3c..81b596c8e 100644 --- a/docs/.vuepress/config.js +++ b/docs/.vuepress/config.js @@ -259,7 +259,7 @@ module.exports = { sidebarDepth: 1, collapsable: true, children: [ - '/how-to/ipfs-on-the-web', + '/how-to/ipfs-in-web-apps', '/how-to/address-ipfs-on-web', ] }, diff --git a/docs/how-to/ipfs-in-web-apps.md b/docs/how-to/ipfs-in-web-apps.md index 5296700c2..107cc94db 100644 --- a/docs/how-to/ipfs-in-web-apps.md +++ b/docs/how-to/ipfs-in-web-apps.md @@ -1,6 +1,6 @@ --- title: IPFS in web applications -description: How to use IPFS in web applications, including IPFS retrieval and pinning in browsers using implementations such as Helia. +description: How to develop applications that use IPFS in web browsers, including IPFS retrieval and pinning in browsers using implementations such as Helia. --- # IPFS in web-applications and resource-constrained environments @@ -25,7 +25,6 @@ As a developer, IPFS exposes three main operations for interacting with the netw - **Providing data by CID**: making data addressed by a CID retrievable by other peers, either by running a node or with a pinning service. - **Retrieving data by CID**: given a CID, IPFS finds providers (peers who share the block), connects to them, fetches the blocks, and verifies. - ## Addressing by CID @@ -60,15 +59,14 @@ console.info(cid) ### Example: Addressing a file by CID with unixfs -

- See the Pen + ## Providing data -In the examples above you saw how to address data by CID. But what you do with it depends on your use case. +In the examples above you saw how to address data by CID. But what you do with it depends on your use case. If you want to data to be retrievable by other peers on [Mainnet](../concepts/glossary.md#ipfs-mainnet) you will want to upload it to a pinning service or an IPFS node you run. @@ -80,20 +78,16 @@ For this reason, you should almost never count on providing data from a browser Instead, you should provide data from a long-running server that runs reliably and has a public IP. That can be a Kubo node that you run, or a [pinning service](../concepts/persistence.md#pinning-services). - ### CAR files -### Example: Uploading a file to a pinning service +CAR files offer a serialized representation of content-addressed resources in one single concatenated stream, alongside a header that describes that content. This makes them a great way to store content-addressed data in a way that is easy to transport and store. -TODO: ## Retrieval ### With Verified-fetch -

- See the Pen + From a4c4c07925d6a783acf35bb37a7e9d19b0f26d7f Mon Sep 17 00:00:00 2001 From: Daniel N <2color@users.noreply.github.com> Date: Mon, 16 Dec 2024 13:38:43 +0100 Subject: [PATCH 4/6] chore: add words to dictionary --- .github/styles/pln-ignore.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/styles/pln-ignore.txt b/.github/styles/pln-ignore.txt index a4025c9ae..818634931 100644 --- a/.github/styles/pln-ignore.txt +++ b/.github/styles/pln-ignore.txt @@ -128,6 +128,7 @@ mainnet markdown(lint) markdownlint merkle +merklizing metadata('s) metamask minimalistic @@ -140,6 +141,7 @@ multiaddrs multibase multicast multicodec +multicodec(s) multiformats multihash multihashes @@ -217,6 +219,7 @@ trustlessly uncensorable undialable uniswap +unixfs unreachability untrusted upgradeability From bc4bd628c4d3e1b0eafcd5887629add79bcdd350 Mon Sep 17 00:00:00 2001 From: Daniel N <2color@users.noreply.github.com> Date: Thu, 19 Dec 2024 15:54:52 +0100 Subject: [PATCH 5/6] wip --- docs/concepts/lifecycle.md | 21 +++++++++++---------- docs/how-to/ipfs-in-web-apps.md | 2 +- 2 files changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/concepts/lifecycle.md b/docs/concepts/lifecycle.md index bd0b5af16..3834bd6ef 100644 --- a/docs/concepts/lifecycle.md +++ b/docs/concepts/lifecycle.md @@ -5,17 +5,18 @@ description: Learn about the lifecycle of data in IPFS. # The lifecycle of data in IPFS -- [1. Content-addressable representation](#1-content-addressable-representation) -- [2. Pinning](#2-pinning) -- [3. Retrieval](#3-retrieval) -- [4. Deleting](#4-deleting) +- [1. Content-addressing](#1-content-addressing) +- [2. Providing](#2-providing) +- [3. Retrieving](#3-retrieving) - [Learn more](#learn-more) -## 1. Content-addressable representation +## 1. Content-addressing / Merkelizing -The file is transformed into a content-addressable representation using a CID. The basic idea is that this representation makes files and directories **content-addressable** via CIDs by chunking files into smaller blocks, calculating their hashes, and constructing a [Merkle DAG](./merkle-dag.md). +The first stage in the lifecycle of data in IPFS is to address it by CID. This is a local operation that takes arbitrary data and encodes it so it can be addressed by a CID. -## 2. Pinning +The exact process depends on the type of data. For files and directories, this is done by constructing a [UnixFS](./file-systems.md#unix-file-system-unixfs) [Merkle DAG](./merkle-dag.md). For other data types, such as dag-cbor, this is done by encoding the data with [dag-cbor](https://ipld.io/docs/codecs/known/dag-cbor/) which is hashed to produce a CID. + +## 2. Providing In this stage, the blocks of the CID are saved on an IPFS node (or pinning service) and made retrievable to the network. Simply saving the CID on the node does not mean the CID is retrievable, so pinning must be used. Pinning allows the node to advertise that it has the CID, and provide it to the network. @@ -23,7 +24,7 @@ In this stage, the blocks of the CID are saved on an IPFS node (or pinning servi - **Providing:** The content-addressable representation of the CID is persisted on one of web3.storage's IPFS nodes (servers running an IPFS node) and made publicly available to the IPFS network. -## 3. Retrieval +## 3. Retrieving In this stage, an IPFS node fetches the blocks of the CID and constructs the Merkle DAG. This usually involves several steps: @@ -35,13 +36,13 @@ In this stage, an IPFS node fetches the blocks of the CID and constructs the Mer - **Local access:** Once all blocks are present, the Merkle DAG can be constructed, making the file or directory underlying the CID successfully replicated and accessible. -## 4. Deleting + ## Learn more diff --git a/docs/how-to/ipfs-in-web-apps.md b/docs/how-to/ipfs-in-web-apps.md index 107cc94db..8b729d5ee 100644 --- a/docs/how-to/ipfs-in-web-apps.md +++ b/docs/how-to/ipfs-in-web-apps.md @@ -21,7 +21,7 @@ There are good reasons for this like security and resource management, but ultim As a developer, IPFS exposes three main operations for interacting with the network: -- **Addressing data with CIDs** (also known as merkelising): taking arbitrary data and encoding so its addressable by CID. For example, given a file and encoding it so it can be addressed by a CID. +- **Addressing data with CIDs** (also known as merklizing): taking arbitrary data and encoding so its addressable by CID. For example, given a file and encoding it so it can be addressed by a CID. - **Providing data by CID**: making data addressed by a CID retrievable by other peers, either by running a node or with a pinning service. - **Retrieving data by CID**: given a CID, IPFS finds providers (peers who share the block), connects to them, fetches the blocks, and verifies. From 00b720f2b117c9508e74afeeff2847512302784a Mon Sep 17 00:00:00 2001 From: Daniel N <2color@users.noreply.github.com> Date: Tue, 7 Jan 2025 13:14:23 +0100 Subject: [PATCH 6/6] refine examples and add note about websites --- docs/how-to/ipfs-in-web-apps.md | 43 ++++++++++++++------------------- 1 file changed, 18 insertions(+), 25 deletions(-) diff --git a/docs/how-to/ipfs-in-web-apps.md b/docs/how-to/ipfs-in-web-apps.md index 8b729d5ee..ec959b0e6 100644 --- a/docs/how-to/ipfs-in-web-apps.md +++ b/docs/how-to/ipfs-in-web-apps.md @@ -5,17 +5,19 @@ description: How to develop applications that use IPFS in web browsers, includin # IPFS in web-applications and resource-constrained environments -This guide covers the key operations of IPFS in the context of web applications, including addressing, retrieving, and providing. +In this guide you will learn how to use IPFS in web applications, including addressing, retrieving, and providing. -For demonstration purposes, this guide uses [Helia](https://github.com/ipfs/helia), the most actively maintained library for using IPFS on the web and is the recommended library for most use cases. +In this guide, you will use [Helia](https://github.com/ipfs/helia), the most actively maintained TypeScript IPFS library for use on the web and the recommended library for most use cases. -## Challenges with IPFS on the Web +> **Note:** this guide is focused solely on using IPFS for data within a web application. It does _not_ cover using IPFS for static website distribution with IPFS Gateways. + +## Challenges with IPFS on the web IPFS allows you to fetch data by CID from multiple providers without being reliant on a single authoritative server. -However, making all of this work on the Web is tricky due to networking constraints. Browsers impose many restrictions on Web apps, for example, opening TCP/UDP connections is not possible. Instead, Web apps are constrained to HTTP, WebSockets, WebRTC, and most recently WebTransport. +However, making all of this work on the web is tricky due to networking constraints. Browsers impose many restrictions on web apps, for example, opening TCP/UDP connections is not possible. Instead, web apps are constrained to HTTP, WebSockets, WebRTC, and most recently WebTransport. -There are good reasons for this like security and resource management, but ultimately, it means that using IPFS on the Web is different to native binaries. +There are good reasons for this like security and resource management, but ultimately, it means that using IPFS on the web is different to native binaries. ## Key IPFS operations: Addressing, Retrieving, and Providing @@ -25,8 +27,7 @@ As a developer, IPFS exposes three main operations for interacting with the netw - **Providing data by CID**: making data addressed by a CID retrievable by other peers, either by running a node or with a pinning service. - **Retrieving data by CID**: given a CID, IPFS finds providers (peers who share the block), connects to them, fetches the blocks, and verifies. - -## Addressing by CID +## Addressing data by CID As mentioned above, the first step in the [lifecycle of data in IPFS](../concepts/lifecycle.md) is to address it by CID. @@ -37,29 +38,22 @@ When addressing data by [CIDs](https://proto.school/anatomy-of-a-cid/03) you wil - [unixfs](../concepts/file-systems.md#unix-file-system-unixfs) for files and directories. - [dag-cbor](../concepts/glossary.md#dag-cbor) for json-like structured data with binary encoding. DAG-CBOR is an extension of CBOR that adds a "link" type for CIDs, allowing for the creation of interlinked CBOR objects (which can be used to form larger linked data structures). -As you can see, there are multiple ways to address data by CID. +### CID Determinism -One important thing to note is that **the same data can result in different CIDs** depending on a number of factors, including the hash function, the multicodec you use, and the way you encode the data. **This is especially true for files**, where the same file, hash function and multicodec can still result in different CIDs depending on the different options that UnixFS supports. See the [forum discussion](https://discuss.ipfs.tech/t/should-we-profile-cids/18507) for more details and a possible solution. +One important thing to note is that **the same data can result in different CIDs** depending on a number of factors, including the hash function, the multicodec you use, and the multicodec. **This is especially true for files**, where the same file, hash function and multicodec can still result in different CIDs depending on the different options that UnixFS supports. See the [forum discussion](https://discuss.ipfs.tech/t/should-we-profile-cids/18507) for more details. ### Example: Addressing an object by CID with dag-cbor For example, to address an object by CID with the `dag-cbor` multicodec and `sha2-256` hash function, you can use the following code using [Helia](https://github.com/ipfs/helia): -```ts -import { createHelia } from 'helia' -import { dagCbor } from '@helia/dag-cbor' - -const helia = await createHelia() -const d = dagCbor(helia) -const cid = await d.add({ hello: 'world' }) - -console.info(cid) -// CID(bafyreidykglsfhoixmivffc5uwhcgshx4j465xwqntbmu43nb2dzqwfvae) -``` + ### Example: Addressing a file by CID with unixfs - @@ -72,7 +66,7 @@ If you want to data to be retrievable by other peers on [Mainnet](../concepts/gl ### You probably don't want to provide data from a browser -Browsers make for lousy servers. It's difficult to make a Web page "dialable", i.e. allow network incoming connections from other computers. There's one exception, namely WebRTC, however, it has many caveats. +Browsers make for lousy servers. It's difficult to make a Web page a server, i.e. allow network incoming connections from other computers. There's one exception, namely WebRTC, however, it has many caveats. For this reason, you should almost never count on providing data from a browser to work. @@ -82,12 +76,11 @@ Instead, you should provide data from a long-running server that runs reliably a CAR files offer a serialized representation of content-addressed resources in one single concatenated stream, alongside a header that describes that content. This makes them a great way to store content-addressed data in a way that is easy to transport and store. - ## Retrieval ### With Verified-fetch -