Issue
I have a client and server that uses SSL_set_fd() to start a handshake on an existing connection. Basically the server does:
auto ctx = SSL_CTX_new(TLS_server_method());
SSL_CTX_set_options(ctx, SSL_OP_ENABLE_KTLS);
SSL_CTX_use_certificate_chain_file(ctx, "cert.pem");
SSL_CTX_use_PrivateKey_file(ctx, "priv.pem", SSL_FILETYPE_PEM);
auto ssl = SSL_new(ctx);
SSL_set_fd(ssl, sock);
SSL_accept(ssl); // client does SSL_connect()
std::cerr << BIO_get_ktls_send(SSL_get_wbio(ssl)) << "\n";
std::cerr << BIO_get_ktls_recv(SSL_get_rbio(ssl)) << "\n";
I've excluded the error handling and extra cert checking, but my question is why does it only enable ktls on TX, not RX. In other words this prints a 1 and then a 0.
When I strace I see that OpenSSL is indeed activating TLS on the socket, and enabling it for TX:
setsockopt(4, SOL_TCP, TCP_ULP, [7564404], 4) = 0
setsockopt(4, SOL_TLS, TLS_TX, "…algorithm and key material…") = 0
That's "tls" in integer form, enabling the ability to start using KTLS. The kernel module tls gets its usage incremented while the socket is open (seen in lsmod | grep tls).
For TX this looks like it's doing both steps of the kernel documentation, but why no TLS_RX?
I don't really understand BIO in OpenSSL, so I suspect that I'll need to attach a BIO, which will activate it? At least I infer that I'm missing a KTLS-compatible BIO reading this comment. But then again it works without it for TX.
My question is: What do I add to the above to make KTLS "kick in" for RX?
Of course I will also accept any other suggestions for changes, such as "oh you're using the old interface, this is better". But while there may be higher level APIs, surely this is also doable by modifying what I have so far?
Maybe I can extract the crypto parameters from OpenSSL and call setsockopt(…… TLX_RX, …) myself? Though it seems like OpenSSL should be able to just turn it on in a more portable way.
OpenSSL version is the one that comes with Ubuntu 22.04.1 LTS: 3.0.2-ubuntu1.7. openssl version says OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022).
Inspecting apt-get source libssl-dev source it looks like it should support RX just as well as TX.
Both client and server get KTLS on TX but not RX, so it should not be that they've negotiated different algorithms, only one of which is KTLS compatible.
uname -a is Linux myhost 5.15.0-46-generic #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 x86_64 x86_64 GNU/Linux
Solution
KTLS on the receive path was not enabled for TLS 1.3 until this commit, which doesn't seem to have been integrated into any OpenSSL version as of this writing (latest being 1.1.1 and 3.0.7).
Forcing TLS version to be 1.2 enables KTLS for both RX and TX:
SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION);
SSL_CTX_set_max_proto_version(ctx, TLS1_2_VERSION);
After this fix the test program returns 1 and 1 for TX and RX KTLS.
Answered By - Thomas Answer Checked By - Pedro (WPSolving Volunteer)