Recently, I discovered an interesting database project called OrbitDB. I want to create a decentralized public anime metadata database using OrbitDB's decentralized implementation through libp2p. However, understanding and setting up the libp2p-related content has been the most difficult part of the development process. Fortunately, the official website has recently released a high-quality beginner's tutorial, which provides a lot of relevant knowledge. It's worth summarizing the content in a blog post.
WebRTC with js-libp2p
Why Choose WebRTC#
What is WebRTC#
WebRTC, short for Web Real-Time Communication, is an API that supports real-time voice and video communication between web browsers. With WebRTC, browsers can establish peer-to-peer connections, allowing direct data transfer between two browsers. WebRTC protocol is supported by mainstream browsers, and when combined with js-libp2p, it allows browsers to act as independent nodes in the libp2p network.
JS-libp2p Connectivity and Capability#
libp2p has multiple language implementation options, but the implementation progress and limitations vary for each option. The official website provides information on the implementation progress and limitations of each language.
libp2p Implementation Progress
Due to browser limitations, browsers cannot directly transmit data using TCP and QUIC protocols. Currently, there are only three protocols available for creating independent libp2p nodes in browsers: WebSocket, WebTransport, and WebRTC. The official website also provides information on the connectivity between different libp2p protocols and explains why browsers do not support TCP and QUIC protocols.
Connectivity Between libp2p Protocols
Based on the connectivity results provided by the official website, WebSocket and WebTransport protocols can only connect to servers and do not support direct browser-to-browser connections. Among all the implementation options, only js-libp2p combined with WebRTC can treat browsers as independent nodes.
Limitations and Solutions of WebRTC#
The most important issue when establishing a P2P connection is how to make two browsers discover each other so that nodes can establish subsequent connections. Two solutions are provided for this: STUN servers and TURN servers.
- STUN servers: Due to the presence of many NAT structures in networks, it is difficult for browsers to discover each other directly through public IP addresses. STUN servers help browsers discover each other's nodes, facilitating the establishment of connections. There are many free public STUN servers available.
Free STUN Public Servers - TURN (Relay) servers: If direct P2P connections cannot be established between browsers, even after using STUN services to discover nodes, TURN servers can be used to relay all the traffic of the P2P connection. However, using TURN servers is costly and not an ideal choice.
When browsers establish P2P connections, they may need to exchange metadata or signaling information. The WebRTC standard does not specify these contents, so developers need to implement their own signaling exchange solutions.
In libp2p, a new solution is designed to solve the above two problems, using a libp2p relay node. The libp2p relay node plays two roles in the network:
- Circuit Relay V2: When browsers cannot establish direct P2P connections, using TURN services is costly. Circuit Relay V2 can solve this problem effectively because it is decentralized. It allows active nodes in the network to help browsers discover each other and relay the required traffic when P2P connections cannot be established.
- PubSub Peer Discovery: The above solution solves the issue of connectivity between nodes, while PubSub Peer Discovery solves the problem of automatic connection between nodes. GossipSub is an implementation of PubSub Peer Discovery. Nodes subscribe to a topic, and whenever a new node joins and subscribes to the same topic, PubSub Peer Discovery broadcasts the new node's address to all nodes subscribed to that topic. Each node then attempts to connect to the new node. However, it is important to note that broadcasting in a large network consumes a significant amount of resources. Improper usage can lead to congestion and loss of reliability, so this solution may not be suitable for production environments.
Connection Process#
Regarding how two nodes establish a P2P connection and the role of the libp2p relay node in the process, the official website provides a flowchart that describes the workflow.
In terms of the workflow, when two nodes establish a connection, they need to know the address of the Relay node first. They initiate WebSocket connections with the Relay node. After successfully establishing stable connections with the Relay node and subscribing to the same GossipSub topic, the Relay node informs other nodes in the network that have subscribed to the same topic about the multiaddress (multiaddr) of the other node through Circuit Relay V2. A multiaddress is an address in the following format:
// Multiaddress of a node
// Using WebSocket on port 9001 with WebRTC support
/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/webrtc/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Not using WebSocket on port 9002 with WebRTC support
/ip4/127.0.0.1/tcp/9002/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/webrtc/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Using WebSocket on port 9001 without WebRTC support
/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
// Not using WebSocket on port 9002 without WebRTC support
/ip4/127.0.0.1/tcp/9002/p2p/12D3KooWDpJEwVdPBrQf6ZcwRYPzynBU2PZNGTQs3X8uh6FpxRTZ/p2p-circuit/p2p/12D3KooWR3oBwBzZxSTd5RTKGJC2WsCxqWumpBUP1WjsixjeMQ9s
Finally, the two nodes attempt to establish a WebRTC connection using the multiaddress, completing the P2P connection.
The official website also provides a more detailed flowchart of the WebRTC connection process.
After discovering each other through the libp2p relay node, the WebRTC connection can be established. First, each node retrieves its public IP and port from the STUN server. Then, the initiating node creates an RTCPeerConnection object (used to manage the WebRTC connection state), DataChannel, and SDP (Session Description Protocol, used to describe session information) based on the information returned by the STUN server to prepare for the connection. The initiating node creates a signaling message defined in libp2p (webrtc-signaling) and sends the SDP to the receiving node through the libp2p relay node. When the receiving node receives the SDP, it also creates an RTCPeerConnection and SDP to reply to the sending node. The sending node records the received SDP, and thus, the two nodes complete the handshake and exchange SDP information.
Then, the receiving node adds the other node to the connection management object RTCPeerConnection using ICE (Interactive Connectivity Establishment, used to find the best connection path in P2P connections). Finally, the connection is encrypted using Noise (a communication protocol), completing the connection.
This concludes the theoretical article. To gain a deeper understanding of the content, you can continue reading the practical article:
Creating a libp2p Network and Establishing Connections Using JS and WebRTC (Practice)