Skip to content

Getting Failed to send message to peer: TimeoutError: signal timed out when sending a message from b2b relayed peers #3092

@akef-malomatia

Description

@akef-malomatia
  • Version:
    libp2p: "^2.8.3"

  • Platform:
    Windows 11 64-bit

  • Subsystem:
    @libp2p/circuit-relay-v2: "^3.2.9"
    react: "^18.3.1"
    Browser: Edge v 135.0.3179.98

Severity:

High

Description:

  • What you did
  • I created two browser clients on different windows machine each one is connected to different networks. Also, I have a relay that is working on a machine with a private static IP. I tried to send a message from one client to another through the relay, I got a message that the two peers connected each peer sees the other, but the message that is sent from one peer doesn't seem to be deliver to the second peer and the peer that sends the message return the following error:
 Failed to send message to peer: TimeoutError: signal timed out

Steps to reproduce the error:

The config of the relay

const options = {
    privateKey: privateKey,
    addresses: {
      listen: ["/ip4/0.0.0.0/tcp/9001/ws", "/ip4/0.0.0.0/tcp/9002"],
      // announce: [
      //   "/dns4/relay-node.bfhkdgcdc9bxfebs.westus2.azurecontainer.io/tcp/9001/ws",
      // ],
    },
    transports: [webSockets(), tcp()],
    connectionEncrypters: [noise()],
    streamMuxers: [yamux()],
    connectionGater: {
      // Allow private addresses for local testing
      denyDialMultiaddr: () => false,
    },
    services: {
      identify: identify(),
      autoNat: autoNAT(),
      relay: circuitRelayServer(),
      ping: ping(),
    },
    // datastore,
  };

The config of the client

const options: Libp2pOptions = {
    privateKey: privateKey,
    addresses: {
      listen: ["/ip4/0.0.0.0/tcp/9001", "/p2p-circuit", "/webrtc"],
    },
    transports: [
      webSockets(),
      webTransport(),
      webRTC(),
      circuitRelayTransport(),
    ],
    peerDiscovery: [
      bootstrap({
        list: [
          "/ip4/RELAY_PUBLIC_IP/tcp/9001/ws/p2p/12D3KooWF6ttBYsoHXQs9FbCUeafdJabVQpYJYPcuHkMrRv9zoFY"
        ],
      }),
    ],
    connectionEncrypters: [noise()],
    streamMuxers: [yamux()],
    connectionGater: {
      // Allow private addresses for local testing
      denyDialMultiaddr: () => false,
    },
    connectionManager: {
      inboundStreamProtocolNegotiationTimeout: 1e4,
      inboundUpgradeTimeout: 1e4,
      outboundStreamProtocolNegotiationTimeout: 1e4,
      outboundUpgradeTimeout: 1e4,
    },
    services: {
      identify: identify(),
      ping: ping(),
      dcutr: dcutr()
      // keychain: keychain(keychainInit),
    },
    
    // datastore,
  };

The package.json file in client

"dependencies": {
    "@chainsafe/libp2p-noise": "latest",
    "@chainsafe/libp2p-yamux": "latest",
    "@libp2p/bootstrap": "^11.0.33",
    "@libp2p/circuit-relay-v2": "^3.2.9",
    "@libp2p/dcutr": "^2.0.29",
    "@libp2p/echo": "^2.1.19",
    "@libp2p/identify": "latest",
    "@libp2p/interface": "^2.8.0",
    "@libp2p/interface-peer-id": "^2.0.2",
    "@libp2p/kad-dht": "^15.0.0",
    "@libp2p/keychain": "^5.2.0",
    "@libp2p/mdns": "latest",
    "@libp2p/peer-id-factory": "^4.2.4",
    "@libp2p/ping": "^2.0.29",
    "@libp2p/tcp": "latest",
    "@libp2p/webrtc": "^5.2.10",
    "@libp2p/webrtc-star": "^7.0.0",
    "@libp2p/websockets": "^9.2.9",
    "@libp2p/webtransport": "^5.0.38",
    "@multiformats/multiaddr": "^12.3.4",
    "axios": "^1.8.4",
    "browser-readline": "^0.0.1-security",
    "browserify-fs": "^1.0.0",
    "buffer": "^6.0.3",
    "clsx": "^2.1.0",
    "crypto-browserify": "^3.12.1",
    "datastore-core": "^10.0.2",
    "datastore-idb": "^3.0.1",
    "dexie": "^4.0.11",
    "idb": "^8.0.2",
    "it-length-prefixed": "^9.1.0",
    "it-map": "^3.1.1",
    "it-pipe": "^3.0.1",
    "libp2p": "^2.8.3",
    "libp2p-gossipsub": "^0.13.0",
    "lucide-react": "^0.344.0",
    "p-defer": "^4.0.1",
    "qrcode.react": "^4.2.0",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "stream-browserify": "^3.0.0",
    "tailwind-merge": "^2.2.1",
    "uint8arrays": "^5.1.0",
    "util": "^0.12.5",
    "wrtc": "^0.4.7"
  },

The package.json file in relay

 "dependencies": {
    "@chainsafe/libp2p-gossipsub": "^14.1.1",
    "@chainsafe/libp2p-noise": "latest",
    "@chainsafe/libp2p-yamux": "latest",
    "@libp2p/autonat": "^2.0.29",
    "@libp2p/circuit-relay-v2": "^3.2.9",
    "@libp2p/identify": "latest",
    "@libp2p/keychain": "^5.2.0",
    "@libp2p/ping": "^2.0.29",
    "@libp2p/tcp": "latest",
    "@libp2p/websockets": "^9.2.9",
    "datastore-idb": "^3.0.1",
    "datastore-level": "^11.0.1",
    "level": "^9.0.0",
    "libp2p": "^2.8.3",
    "cross-env": "^7.0.3"
  },

The log of the relay when the first client connects to it using bootstrap:

libp2p:websockets:listener new inbound connection /ip4/CLIENT_1_IP/tcp/12791 +0ms
  libp2p:websockets:maconn encrypting inbound connection to /ip4/CLIENT_1_IP/tcp/12791 using /noise +0ms
  libp2p:websockets:maconn successfully upgraded inbound connection +89ms
Connected to peer: 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU
  libp2p:connection-manager:connection-pruner checking max connections limit 1/300 +0ms
  libp2p:connection:inbound:bfpqq71745825622167 incoming stream opened on /ipfs/id/1.0.0 +0ms
  libp2p:identify identify completed for peer 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU and protocols [ '/chat/1.0.0', '/ipfs/id/1.0.0', '/ipfs/ping/1.0.0', '/libp2p/circuit/relay/0.2.0/stop', '/libp2p/dcutr', '/webrtc-signaling/0.0.1' ] +0ms
  libp2p:identify received identify from 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +3ms
  libp2p:identify 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU did not send a signed peer record +1ms
  libp2p:connection:inbound:bfpqq71745825622167 incoming stream opened on /libp2p/circuit/relay/0.2.0/hop +182ms
  libp2p:circuit-relay:server received circuit v2 hop protocol stream from 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +0ms
  libp2p:circuit-relay:server received RESERVE +46ms
  libp2p:circuit-relay:server received hop message +7ms
  libp2p:circuit-relay:server hop reserve request from 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +0ms
  libp2p:circuit-relay:server:reservation-store creating new reservation for client 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +0ms
  libp2p:circuit-relay:server sent confirmation response to 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +11ms 

The log of the relay when the second client connects to it using boostrap:

libp2p:websockets:listener new inbound connection /ip4/CLIENT_2_IP/tcp/51766 +1m
  libp2p:websockets:maconn encrypting inbound connection to /ip4/CLIENT_2_IP/tcp/51766 using /noise +0ms
  libp2p:websockets:maconn successfully upgraded inbound connection +554ms
Connected to peer: 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS
  libp2p:connection-manager:connection-pruner checking max connections limit 2/300 +1m
  libp2p:connection:inbound:c16h4d1745825699119 incoming stream opened on /ipfs/id/1.0.0 +0ms
  libp2p:identify identify completed for peer 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS and protocols [ '/chat/1.0.0', '/ipfs/id/1.0.0', '/ipfs/ping/1.0.0', '/libp2p/circuit/relay/0.2.0/stop', '/libp2p/dcutr', '/webrtc-signaling/0.0.1' ] +1m
  libp2p:identify received identify from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +1ms
  libp2p:identify 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS did not send a signed peer record +0ms
  libp2p:connection:inbound:c16h4d1745825699119 incoming stream opened on /libp2p/circuit/relay/0.2.0/hop +210ms
  libp2p:circuit-relay:server received circuit v2 hop protocol stream from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +1m
  libp2p:circuit-relay:server received RESERVE +52ms
  libp2p:circuit-relay:server received hop message +1ms
  libp2p:circuit-relay:server hop reserve request from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +12ms
  libp2p:circuit-relay:server:reservation-store creating new reservation for client 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +1m
  libp2p:circuit-relay:server sent confirmation response to 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +7ms 

The log of the relay when the CLIENT_2 sends a message to CLIENT_1

libp2p:connection:inbound:3n4u841745825835789 incoming stream opened on /ipfs/id/1.0.0 +0ms
  libp2p:connection:inbound:3n4u841745825835789 incoming stream opened on /libp2p/circuit/relay/0.2.0/hop +58ms
  libp2p:circuit-relay:server received circuit v2 hop protocol stream from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +13s
  libp2p:identify identify completed for peer 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS and protocols [ '/chat/1.0.0', '/ipfs/id/1.0.0', '/ipfs/ping/1.0.0', '/libp2p/circuit/relay/0.2.0/stop', '/libp2p/dcutr', '/webrtc-signaling/0.0.1' ] +13s
  libp2p:identify received identify from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +1ms
  libp2p:circuit-relay:server received CONNECT +16ms
  libp2p:circuit-relay:server received hop message +0ms
  libp2p:circuit-relay:server hop connect request from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS +1ms
  libp2p:circuit-relay:server starting circuit relay v2 stop request to 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU +0ms
  libp2p:circuit-relay:server stop request to 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU was successful +22ms
  libp2p:circuit-relay:server connection from 12D3KooWKwewVcnTea4asuLFHVgDaKAPpGvPx9R3fgEu5tWqdsYS to 12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU established - merging streams +1ms
  libp2p:circuit-relay:server limiting relayed connection duration to 120000ms +20ms

The function that is called to send a message to the peer:

 public async sendMessageToPeer(peerMultiaddr: string, message: string) {
    try {
      console.log("Sending message to peer:", peerMultiaddr);
      const stream = await this.node.dialProtocol(
        multiaddr(peerMultiaddr),
        "/chat/1.0.0"
      );

      const encoder = new TextEncoder();
      const encodedMessage = encoder.encode(message);

      await pipe(
        [encodedMessage], // Source (an iterable of Uint8Array)
        stream.sink // Sink (stream write)
      );

      console.log(`Message sent to ${peerMultiaddr}: ${message}`);
      return true;
    } catch (e) {
      console.error("Failed to send message to peer:", e);
      return false;
    }
  }

The listener that listen to the incoming messages from a peer:

this.node.handle("/chat/1.0.0", async ({ stream, connection }) => {
        const publicKey = connection.remotePeer.toString();
        console.log(`Received a connection from ${publicKey}`);

        for await (const chunk of stream.source) {
          const text = new TextDecoder().decode(chunk.subarray()); // Uint8ArrayList to Uint8Array

          // check if this sender is new
          if (!(await MyLocalStorage.contact.isContactExist(publicKey))) {
            console.log("New contact added:", publicKey);
            MyLocalStorage.contact.addContact(publicKey, "New Contact");
          } else {
            console.log("Contact already exists");
          }

          this.messageCallback(publicKey, text);
        }
      });

The multiaddress that the CLIENT_2 uses to send a message to the CLIENT_1:
Sending message to peer: /ip4/RELAY_PUBLIC_IP/tcp/9001/ws/p2p/12D3KooWBVtp5xk4eo5Ztc8b3iG7Nbco3btbtp8VzdKzHsSKL3Ly/p2p-circuit/webrtc/p2p/12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU

The error that I got in the CLIENT_2 when sending a message to the CLIENT_1:
Failed to send message to peer: TimeoutError: signal timed out

If the CLIENT_2 uses the following multiaddress of the CLIENT_1:
Sending message to peer: /ip4/RELAY_PUBLIC_IP/tcp/9001/ws/p2p/12D3KooWBVtp5xk4eo5Ztc8b3iG7Nbco3btbtp8VzdKzHsSKL3Ly/p2p-circuit/p2p/12D3KooWBVsTi7XZz8cMp8v7fTgNVwCsPtDc6hq5FgBJ3WfB9rNU

I got the following error:
Failed to send message to peer: LimitedConnectionError: Cannot open protocol stream on limited connection

Metadata

Metadata

Assignees

No one assigned

    Labels

    need/triageNeeds initial labeling and prioritization

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions