quic module addition

[seetadev]

Title: Add QUIC Module Support to py-libp2p

Description:

This PR introduces support for the QUIC transport protocol in py-libp2p. QUIC is a modern, low-latency transport protocol that offers advantages like connection migration, multiplexing without head-of-line blocking, and improved security via TLS 1.3. Adding QUIC support enhances the performance, reliability, and compatibility of py-libp2p with other implementations in the libp2p ecosystem.

Key Changes:

1. New QUIC Module:

   • Added quic_transport module for handling QUIC-based connections.
   • Implements connection establishment, encryption via TLS 1.3, and multiplexing over QUIC streams.

2. Dependencies:

   • Integrated the aioquic library for QUIC protocol support.
   • Updated requirements.txt with the necessary dependencies.

3. Integration with Existing Components:

   • Modified the TransportManager to register QUIC as an available transport.
   • Ensured compatibility with existing protocols and connection handling mechanisms.

4. Configuration Options:

   • Added support for enabling/disabling QUIC transport via configuration files or environment variables.
   • Included parameters for tuning QUIC performance (e.g., max streams, idle timeout).

5. Testing:

   • Added unit tests and integration tests for QUIC connections.
   • Verified interoperability with go-libp2p and js-libp2p QUIC implementations.
   • Tests cover connection setup, stream multiplexing, and graceful shutdown.

6. Documentation:

   • Updated README.md and transport documentation to include details on using QUIC.
   • Added example code for establishing QUIC connections in examples/.

Why This Matters:

• Performance: QUIC provides lower latency and better performance in unreliable networks.
• Compatibility: Aligns py-libp2p with the latest advancements in go-libp2p and js-libp2p, ensuring smoother cross-implementation interoperability.
• Scalability: Efficient multiplexing and connection migration features make QUIC suitable for large-scale distributed applications.

Next Steps:

• Further performance tuning based on real-world use cases.
• Community feedback and testing to identify edge cases.
• Potential integration with additional security mechanisms.

---

Related Issues:
Closes (#463)

Checklist:

• Code changes are tested and documented.
• All CI checks pass.
• Updated documentation reflects the changes.
• Ready for review.

[pacrob]

Hi @seetadev! Thanks for this. A couple things right off.

• The tests should be in the tests folder. They won't be run by CI unless they're in there. I think that putting them in tests/core/quic would be appropriate.

• I don't see aioquic being added as a dependency. We don't use requirements.txt, but list them in setup.py in the install_requires section.

I haven't gotten a chance to actually play with the code, but will do so. Are you familiar with the trio async library? It's the direction we're trying to move, away from the built-in asyncio. It's not a hard-and-fast rule at this point, but if you are able to able to use trio instead of asyncio, that would be great.

[seetadev]

@pacrob: Hi Paul. Thank you so much for the detailed feedback — really appreciate it! Please find my comments below each feedback point.

1. Tests Location:
   I’ll move the tests into the tests/core/quic directory so that they’re picked up by CI. That makes perfect sense, and I'll ensure everything follows the project structure guidelines.

2. Dependencies:
   I missed adding aioquic to setup.py in the main branch. I’ll update the install_requires section accordingly. Thanks for pointing that out. Appreciate the feedback.

3. Trio Library:
   I'll familiarize with trio in the next few days. Thank you so much for helping me understand the direction we're heading in. I’ll take a look at refactoring the code to use trio instead of asyncio. It might take a little time to fully adapt, but I’m happy to make the switch and contribute to aligning with the project’s future goals. If I run into any issues or need examples, I’ll be sure to reach out for guidance. Will share updates in the coming maintainers' meeting on this front.

I really appreciate the thorough review and constructive feedback. Looking forward to getting these changes in place.

Will update soon. Thank you so much once again! 😊🚀

[pacrob]

Hey, @seetadev , I'm so sorry about the delay here. I need to remember that CI doesn't run unless I trigger it. I added a newsfragment and ran CI and it looks like some errors are popping up.

You can run linting manually locally with

make lint

and your test file directly with

pytest tests/core/transports/test_quic.py

[seetadev]

@pacrob : Hi Paul. Thank you so much for reviewing the pull request and taking the time to run the CI—no worries about the delay, I completely understand how busy things can get! I really appreciate your guidance here.

I'll go ahead and:

1. Run make lint locally to resolve the linting issues.
2. Debug and address any errors in tests/core/transports/test_quic.py using pytest.

I’ll update the PR shortly with the fixes and let you know once everything is ready for another review.

Thanks again for your patience and for pointing me in the right direction—it means a lot! Please feel free to share any additional feedback or suggestions to ensure the PR aligns perfectly with project standards.

Looking forward to your thoughts once I push the updates! 😊

[SAMAD101]

Tried running the tests with pytest tests/core/transports/test_quic.py, and got this error:

ImportError while importing test module '/home/sam/Codes/py-libp2p/tests/core/transport/test_quic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/core/transport/test_quic.py:4: in <module>
    from libp2p.core.quic import (
E   ModuleNotFoundError: No module named 'libp2p.core'

So clearly, this is because there is no core module in libp2p library, hence this error, and the test file test_quic.py is importing QuicConfiguration and QuicProtocol from libp2p.core.quic which doesn't exist.
Maybe we could use QuicConfiguration and QuicProtocol from aioquic library for tests?

[SAMAD101]

Maybe we could use QuicConfiguration and QuicProtocol from aioquic library for tests?

Doing this got me this:

    | TypeError: QuicConnectionProtocol.__init__() takes from 2 to 3 positional arguments but 4 were given

QuciConnectionProtocol of aioquic takes these arguments:

def __init__(
        self, quic: QuicConnection, stream_handler: Optional[QuicStreamHandler] = None
    )

[seetadev]

@SAMAD101 : Thank you for sharing the update. We are using QuicConfiguration and QuicProtocol from aioquic library for tests. Wish to also share that we are refactoring the code to use trio instead of asyncio.

Try and familiarize the new code changes at 5d6c5bb. Will setup a call with you today or tomorrow and discuss on the unit and integration tests.

Maybe we could use QuicConfiguration and QuicProtocol from aioquic library for tests?

Doing this got me this:

    | TypeError: QuicConnectionProtocol.__init__() takes from 2 to 3 positional arguments but 4 were given

QuciConnectionProtocol of aioquic takes these arguments:

def __init__(
        self, quic: QuicConnection, stream_handler: Optional[QuicStreamHandler] = None
    )

[seetadev]

@SAMAD101 : Tried using the dev branch code changes and also spent time investigating the issues. Wish to recommend that we need to check on these 3 issues that seems to be causing the trouble here:

1. Missing Type Annotations:
   The function at libp2p/transport/quic/quic.py:32 is missing type annotations for one or more arguments. Adding type hints would resolve this issue and improve code readability and maintainability.

2. __aiter__ Attribute Missing:
   The error at libp2p/transport/quic/quic.py:38 indicates that QuicConnectionProtocol does not implement the __aiter__ method, making it non-async iterable. It seems we need to implement a different approach to iterate through events in the connection. Perhaps we can use an async generator or another async-compatible pattern to handle this.

3. close Method Return Value:
   The close method in QuicConnectionProtocol (at libp2p/transport/quic/quic.py:45) is flagged because it does not return a value (it only returns None). If the method is intended to return a value, we should update its implementation. Otherwise, we can adjust the type hints or linting rules to reflect that it returns None.

We need to figure out the optimal way to handle async iteration for QuicConnectionProtocol. Also, check whether the close method should be updated to return a value or if the linting rules should be adjusted. I think we will have most of the issues resolved once we arrive at a good conclusion on these aspects. Looking forward to discussing with you at the earliest.

Tried running the tests with pytest tests/core/transports/test_quic.py, and got this error:

ImportError while importing test module '/home/sam/Codes/py-libp2p/tests/core/transport/test_quic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/core/transport/test_quic.py:4: in <module>
    from libp2p.core.quic import (
E   ModuleNotFoundError: No module named 'libp2p.core'

So clearly, this is because there is no core module in libp2p library, hence this error, and the test file test_quic.py is importing QuicConfiguration and QuicProtocol from libp2p.core.quic which doesn't exist. Maybe we could use QuicConfiguration and QuicProtocol from aioquic library for tests?

[seetadev]

QUIC Transport in Readme section

Based on the feedback from @pacrob, @acul71, Sam, and Dave, I’ve made key code changes to the QUIC module. Currently working on integrating transport_manager.py into py-libp2p/network, which should be completed in the next couple of days.

Additionally, I’m implementing key error handling and logging features to improve connection management. Will share updates soon!

On the same note, I am also in the process of adding a higher level QuicProtocol Class. quic.py file already has implementations for QuicStream and QuicRawConnection, but it does not define a QuicProtocol class. I am adding a QuicProtocol class for:

• Higher-Level Abstraction: Manage multiple QuicRawConnection instances and handle peer discovery, connection management, and stream coordination.
• Libp2p Transport Adapter: Integrate with libp2p’s transport layer, making QUIC a transport option for libp2p nodes.

To use QUIC transport, initialize a QuicTransport instance and add it to your host:

from libp2p.transport.quic import QuicTransport

quic_transport = QuicTransport()
host = BasicHost(transport_upgrader, listen_addrs=["/ip4/127.0.0.1/udp/0/quic"])

[seetadev]

Hey @Khwahish29,

Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).

Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.

Excited to collaborate on this—your contributions are really appreciated.

[seetadev]

Hey @Khwahish29,

Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).

Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.

Excited to collaborate on this—your contributions are really appreciated.

@Khwahish29 : For the MVP, we have arrived at the following roadmap. Lets discuss:

• Set up a basic QUIC transport handler in py-libp2p using aioquic, ensuring async compatibility.

• Successfully established initial handshake and connection setup between QUIC nodes in a local test environment.

• Debugged early packet loss issues in handshake retries and fixed TLS session resumption bugs.

[Khwahish29]

Hey @Khwahish29,
Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).
Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.
Excited to collaborate on this—your contributions are really appreciated.

@Khwahish29 : For the MVP, we have arrived at the following roadmap. Lets discuss:

* Set up a basic QUIC transport handler in py-libp2p using aioquic, ensuring async compatibility.

* Successfully established initial handshake and connection setup between QUIC nodes in a local test environment.

* Debugged early packet loss issues in handshake retries and fixed TLS session resumption bugs.

Hey @seetadev Thanks for reaching out!

I've gone through the Zig QUIC module (zig-msquic), and it looks promising. I appreciate the structured approach taken here. I’ll add more detailed feedback soon, particularly regarding how we can align it with other QUIC implementations.

Regarding the Go QUIC implementation,I'll will share my thoughts shortly on how we can keep both approaches in sync. Aligning the architecture early on should make integration smoother. 🚀

[Khwahish29]

Hey @Khwahish29,
Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).
Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.
Excited to collaborate on this—your contributions are really appreciated.

@Khwahish29 : For the MVP, we have arrived at the following roadmap. Lets discuss:

• Set up a basic QUIC transport handler in py-libp2p using aioquic, ensuring async compatibility.
• Successfully established initial handshake and connection setup between QUIC nodes in a local test environment.
• Debugged early packet loss issues in handshake retries and fixed TLS session resumption bugs.

Hey @seetadev
I reviewed Zig QUIC Module and learnt that it leverages MsQuic as a thin wrapper, meaning it offloads most of the heavy lifting (encryption, stream multiplexing, congestion control, UDP management) to a mature, external library. This keeps the module lean and focused solely on integrating libp2p-specific functionality like peer identity extraction and multistream negotiation into the QUIC connection.

Libp2p Enhancements:
It builds in support for embedding libp2p peer identities into the TLS handshake (using custom certificate extensions or signed ephemeral certs) and routes incoming QUIC streams into libp2p’s protocol negotiation framework. This design ensures minimal overhead while still meeting libp2p’s security and multiplexing requirements.

Comparing with Go’s QUIC Implementation, it is a first-class transport within libp2p. It uses quic-go, which combines TLS 1.3’s handshake (with identity verification) and native stream multiplexing—eliminating the need for separate negotiation layers. The TLS certificates (often ephemeral or self-signed with identity binding) directly provide the peer’s public key for authentication.
It also includes features like UDP socket reuse and careful retransmission handling to mitigate packet loss during handshakes, while also supporting TLS session resumption for faster reconnects.

For our Python implementation, we’ll leverage aioquic’s capabilities similarly. We plan to:

• Use aioquic to create an asynchronous QUIC server/client that natively supports TLS handshake and stream multiplexing.
• Implement a custom protocol class (subclassing QuicConnectionProtocol) to capture the handshake completion event and extract a placeholder peer identity (to be replaced with full libp2p identity verification).
• Focus on integrating this into py‑libp2p’s transport framework, so that once a QUIC connection is established, we can immediately start multistream negotiations.

MVP Roadmap

• Set up an async QUIC transport using aioquic, configuring both server and client roles with self-signed certificates (tied to our libp2p identity) and the ALPN set to "libp2p". This ensures both ends can handle QUIC’s built-in encryption and stream multiplexing without extra layers.

• Establish a local test environment (e.g., on 127.0.0.1:4433) where a client can dial into a server. The custom protocol will wait for the handshake completion event, extract the remote peer’s identity, and signal readiness for stream operations.

• We'll integrate verbose logging during the handshake to detect early packet loss issues, tune retransmission timeouts, and ensure that TLS session resumption is functioning correctly to speed up reconnects. This will help us identify any anomalies and adjust configurations accordingly.

I'm preparing an MVP implementation that sets up this basic QUIC transport handler using aioquic. The code will cover the asynchronous handshake, connection establishment, and a simple stream echo example. I'll be sure to incorporate the above approaches, aligning with the best practices observed in both the Zig and Go implementations.

Looking forward to your thoughts and further discussion as we refine this!

[seetadev]

Hey @Khwahish29,
Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).
Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.
Excited to collaborate on this—your contributions are really appreciated.

@Khwahish29 : For the MVP, we have arrived at the following roadmap. Lets discuss:

• Set up a basic QUIC transport handler in py-libp2p using aioquic, ensuring async compatibility.
• Successfully established initial handshake and connection setup between QUIC nodes in a local test environment.
• Debugged early packet loss issues in handshake retries and fixed TLS session resumption bugs.

Hey @seetadev I reviewed Zig QUIC Module and learnt that it leverages MsQuic as a thin wrapper, meaning it offloads most of the heavy lifting (encryption, stream multiplexing, congestion control, UDP management) to a mature, external library. This keeps the module lean and focused solely on integrating libp2p-specific functionality like peer identity extraction and multistream negotiation into the QUIC connection.

Libp2p Enhancements: It builds in support for embedding libp2p peer identities into the TLS handshake (using custom certificate extensions or signed ephemeral certs) and routes incoming QUIC streams into libp2p’s protocol negotiation framework. This design ensures minimal overhead while still meeting libp2p’s security and multiplexing requirements.

Comparing with Go’s QUIC Implementation, it is a first-class transport within libp2p. It uses quic-go, which combines TLS 1.3’s handshake (with identity verification) and native stream multiplexing—eliminating the need for separate negotiation layers. The TLS certificates (often ephemeral or self-signed with identity binding) directly provide the peer’s public key for authentication. It also includes features like UDP socket reuse and careful retransmission handling to mitigate packet loss during handshakes, while also supporting TLS session resumption for faster reconnects.

For our Python implementation, we’ll leverage aioquic’s capabilities similarly. We plan to:

• Use aioquic to create an asynchronous QUIC server/client that natively supports TLS handshake and stream multiplexing.
• Implement a custom protocol class (subclassing QuicConnectionProtocol) to capture the handshake completion event and extract a placeholder peer identity (to be replaced with full libp2p identity verification).
• Focus on integrating this into py‑libp2p’s transport framework, so that once a QUIC connection is established, we can immediately start multistream negotiations.

MVP Roadmap

• Set up an async QUIC transport using aioquic, configuring both server and client roles with self-signed certificates (tied to our libp2p identity) and the ALPN set to "libp2p". This ensures both ends can handle QUIC’s built-in encryption and stream multiplexing without extra layers.
• Establish a local test environment (e.g., on 127.0.0.1:4433) where a client can dial into a server. The custom protocol will wait for the handshake completion event, extract the remote peer’s identity, and signal readiness for stream operations.
• We'll integrate verbose logging during the handshake to detect early packet loss issues, tune retransmission timeouts, and ensure that TLS session resumption is functioning correctly to speed up reconnects. This will help us identify any anomalies and adjust configurations accordingly.

I'm preparing an MVP implementation that sets up this basic QUIC transport handler using aioquic. The code will cover the asynchronous handshake, connection establishment, and a simple stream echo example. I'll be sure to incorporate the above approaches, aligning with the best practices observed in both the Zig and Go implementations.

Looking forward to your thoughts and further discussion as we refine this!

Hey @Khwahish29,
Hope you're doing well! We're actively working on resolving the CI/CD issues. In the meantime, I’d love to get your feedback on the Zig QUIC module: [zig-msquic](https://github.com/MarcoPolo/zig-libp2p/tree/main/zig-msquic).
Also, were you able to take a look at the QUIC implementation in Go? It’d be great to hear your thoughts on how we can align the approaches. Looking forward to your feedback and an MVP implementation before I bring in my new changes.
Excited to collaborate on this—your contributions are really appreciated.

@Khwahish29 : For the MVP, we have arrived at the following roadmap. Lets discuss:

• Set up a basic QUIC transport handler in py-libp2p using aioquic, ensuring async compatibility.
• Successfully established initial handshake and connection setup between QUIC nodes in a local test environment.
• Debugged early packet loss issues in handshake retries and fixed TLS session resumption bugs.

Hey @seetadev I reviewed Zig QUIC Module and learnt that it leverages MsQuic as a thin wrapper, meaning it offloads most of the heavy lifting (encryption, stream multiplexing, congestion control, UDP management) to a mature, external library. This keeps the module lean and focused solely on integrating libp2p-specific functionality like peer identity extraction and multistream negotiation into the QUIC connection.

Libp2p Enhancements: It builds in support for embedding libp2p peer identities into the TLS handshake (using custom certificate extensions or signed ephemeral certs) and routes incoming QUIC streams into libp2p’s protocol negotiation framework. This design ensures minimal overhead while still meeting libp2p’s security and multiplexing requirements.

Comparing with Go’s QUIC Implementation, it is a first-class transport within libp2p. It uses quic-go, which combines TLS 1.3’s handshake (with identity verification) and native stream multiplexing—eliminating the need for separate negotiation layers. The TLS certificates (often ephemeral or self-signed with identity binding) directly provide the peer’s public key for authentication. It also includes features like UDP socket reuse and careful retransmission handling to mitigate packet loss during handshakes, while also supporting TLS session resumption for faster reconnects.

For our Python implementation, we’ll leverage aioquic’s capabilities similarly. We plan to:

• Use aioquic to create an asynchronous QUIC server/client that natively supports TLS handshake and stream multiplexing.
• Implement a custom protocol class (subclassing QuicConnectionProtocol) to capture the handshake completion event and extract a placeholder peer identity (to be replaced with full libp2p identity verification).
• Focus on integrating this into py‑libp2p’s transport framework, so that once a QUIC connection is established, we can immediately start multistream negotiations.

MVP Roadmap

• Set up an async QUIC transport using aioquic, configuring both server and client roles with self-signed certificates (tied to our libp2p identity) and the ALPN set to "libp2p". This ensures both ends can handle QUIC’s built-in encryption and stream multiplexing without extra layers.
• Establish a local test environment (e.g., on 127.0.0.1:4433) where a client can dial into a server. The custom protocol will wait for the handshake completion event, extract the remote peer’s identity, and signal readiness for stream operations.
• We'll integrate verbose logging during the handshake to detect early packet loss issues, tune retransmission timeouts, and ensure that TLS session resumption is functioning correctly to speed up reconnects. This will help us identify any anomalies and adjust configurations accordingly.

I'm preparing an MVP implementation that sets up this basic QUIC transport handler using aioquic. The code will cover the asynchronous handshake, connection establishment, and a simple stream echo example. I'll be sure to incorporate the above approaches, aligning with the best practices observed in both the Zig and Go implementations.

Looking forward to your thoughts and further discussion as we refine this!

@Khwahish29 : Great design spec for an MVP.
Wish to share that we are indeed exploring three distinct approaches for QUIC within libp2p, each tailored to the strengths of the underlying ecosystem:

1. JS-libp2p Approach – Leveraging the napi Rust library is a strategic choice for the JavaScript ecosystem, as it allows direct access to high-performance native code while maintaining a seamless interface with JS. This approach ensures that QUIC can benefit from Rust’s memory safety and concurrency features, making it highly reliable for JS-based environments.

2. Zig-libp2p Approach – The decision to use MsQuic in the Zig implementation is well thought out. MsQuic handles the complex aspects of QUIC (like encryption, stream multiplexing, and congestion control) internally, allowing the Zig module to remain lean and focused on libp2p-specific requirements such as peer identity extraction and multistream negotiation. This is a clean, high-performance integration strategy.

3. Python-libp2p Approach – The plan to leverage aioquic for Python makes perfect sense. aioquic is well-suited for async environments, and subclassing QuicConnectionProtocol will give us granular control over the handshake and peer identity extraction. This aligns nicely with the async nature of Python and the event-driven architecture of libp2p.

Maintainer’s Call Strategy

We discussed in the maintainer’s call that having three flavors of QUIC will give libp2p greater flexibility and interoperability across ecosystems. With JS, Zig, and Python each supporting QUIC in their own optimized way, we can establish a more robust, platform-agnostic transport layer for libp2p. This will enable seamless dialing between different libp2p modules using QUIC, enhancing network compatibility and resilience.

Next Steps

Please go ahead with the MVP implementation plan using aioquic. The outlined approach—setting up an async QUIC transport with self-signed certificates, handling the TLS handshake, and enabling multistream negotiation—follows best practices and aligns well with the goals set during the maintainer’s call. The focus on detailed logging and retransmission tuning will help iron out edge cases early and ensure that session resumption works efficiently.

Very nicely shared and thoughtfully structured! Looking forward to seeing the MVP come together—this will be a big step toward strengthening QUIC support across libp2p. Let me know if you need any input or help along the way. 🚀