We associate a transport when [listening](https://github.com/libp2p/go-libp2p/blob/1f6232db191f07c4de212dea7b7fa13fb524ddc6/p2p/transport/quicreuse/connmgr.go#L184), but we aren't cleaning up the association when the listener is closed.
