[bug]: Issue with Paying Sats Invoices Using Taproot Assets via sendPayment with rfqId

[pranav-punjabi]

Background

• We are working with a setup where we have Litd nodes (node a, b, c) and a Sats Node (as per the attached diagram). Node b acts as the Edge Node.

• Currently, we use the Taproot Assets sendPayment API to pay invoices from Litd Nodes.

• By design, sendPayment first places an AddAssetSellOrder and then routes the payment to the destination.

• Since we run our own Price Oracle Server to fetch BTC ↔️ TaprootAsset exchange rates, we have faced the following problem:

• When calling sendPayment, the API attempts to place a SellOrder, but we hit a "connection reset by peer" error.

• This is consistent with the issue described here: lightninglabs/taproot-assets#1267.

The root cause seems to be a network layer error due to idleness between the Edge Node and the Price Oracle server. The channel closes without sending a GO_AWAY message, leaving the client unaware of the broken connection, which results in payment failure.

Our Approach to Mitigate
To avoid this error, we split the process into two steps:

  1. Manually place SellOrder for the msats amount specified in the invoice.
  2. We handle retries and timeouts explicitly.
  3. This ensures reliability and always gives us a PeerAcceptedSellOrder.
  4. Call sendPayment with the rfqId obtained from the PeerAcceptedSellOrder response.

Current Problem:

• This approach works in most cases. However, we hit a specific issue:

  **When paying Sats invoices with TaprootAssets using sendPayment with the manually provided rfqId, we consistently receive NO_ROUTE errors.**
  

The strange part is that:

• A valid path does exist.
• If we retry the same payment without setting the rfqId, it succeeds.

• any other relevant environment details
  we are using tapd v0.6.1
  "lightning-terminal version : v0.15.1-alpha"

Steps to reproduce

1. Create an invoice on the Sats Node.
2. Decode the invoice to get the msats amount.
3. On Node A(LitdNode), manually place a SellOrder for the decoded msats amount.
4. Use the returned acceptedPeerQuoteId in the rfqId field of the sendPayment API.
5. Call sendPayment.

Observed Result:
The payment fails with NO_ROUTE.

Payment with hash 86ae98dc44ffdeac5097b15f62f863f6309e73e454f53339156ad94d1f52ff5f failed under this scenario. I have attached the relevant log file for this failed attempt.

output (1).log

[ffranr]

If we retry the same payment without setting the rfqId, it succeeds.

If the RFQ ID is not set, then tapd should run the RFQ process before sending the payment. If that succeeds, I don’t see the issue, just leave it unset?

Is there anymore to the log filed? I see that the last attempt (where the RFQ ID was provided to the SendPayment tapd RPC endpoint) seems to have progressed to LND path finding. So the quote with the peer was known to tapd. I would be interested to see the results of the LND path finding.

[Roasbeef]

The root cause seems to be a network layer error due to idleness between the Edge Node and the Price Oracle server. The channel closes without sending a GO_AWAY message, leaving the client unaware of the broken connection, which results in payment failure.

Perhaps we need to explicitly set the various keep alive parameters to ensure that a stream will continue to persist, even when there's low/no traffic?

On the client side:

• https://pkg.go.dev/google.golang.org/grpc#WithKeepaliveParams

On the server side:

• https://pkg.go.dev/google.golang.org/grpc#KeepaliveEnforcementPolicy

[Roasbeef]

Here're lnd's settings for reference: https://github.com/lightningnetwork/lnd/blob/df46d1862662b9ae2cdb19cf9425986982fd2cd5/lnd.go#L364

[pranav-punjabi]

Hi @Roasbeef,

We already configured the following keepalive properties on our Price Oracle server as per the gRPC documentation:
https://grpc.io/docs/guides/keepalive/#keepalive-configuration-specification

KEEPALIVE_TIMEOUT → 10 seconds

KEEPALIVE_TIME → 1 minute

Despite this, we are still seeing issues that seem to stem from network layer idleness between the Edge Node and the Price Oracle server. Specifically, the channel closes silently (no GO_AWAY message), leaving the client unaware of the broken connection. As a result, the Edge Node continues to use the broken connection to fetch quotes and waits ~30 seconds before failing with “connection reset by peer.”

After this failure, if we retry, the Edge Node establishes a new connection to the Price Oracle server and everything works fine again.
i dont know why my keep alive configuration not working can we do keepalive from client too

[Roasbeef]

I think this PR should fix your issue @pranav-punjabi: #1834.

We didn't configure any keep-alive settings for the client at all. This fixes that, makes them very forgiving, and adds a similar set of settings for the server.

[Hunterstech]

when v0.7

[Roasbeef]

🔜