This is a forward-port of release/3.5 commit 566f4678
* Add generic direction enum for negotiation setup.
* Rename a rather long wrapper function to a shorter one consistent with
what it does.
* Rework context setup function.
* Don't check for handshake state before beginning handshaking.
The old backend began a handshake and then stepped into the callback
function if it was interrupted; the current one just jumps right into
it, so there is no need to check if it has previously succeeded,
because it hasn't been attempted yet.
* Add missing errno assignment to one of the handshake wrappers.
* Don't bother checking if SSL_P(F) is NULL when we already checked if
F->ssl is NULL -- this should be impossible.
* Don't bother checking if SSL_C(F) is NULL -- this was a no-op.
* Change the socket send and recv functions to not peer into a foreign
ratbox structure -- use the correct function to get the socket fd.
* Rewrap some lines and function arguments.
Other backends will be brought into line with this backend soon.
This will enable easier maintenance of the backends, by reducing the
diffs between them, which should make different behaviour easier to
spot.
At the moment, if a link quits in just the right (wrong [1]) way,
the quit reason will resemble:
<-- foo (~bar@baz) has quit (Read error: (-0x0) )
This should resolve that.
[1] Peers should send a close_notify alert before abruptly shutting
down their socket. This will result in a sane quit message:
<-- foo (~bar@baz) has quit (Read error: (-0x7880) SSL -
The peer notified us that the connection is going to be closed)
[ci skip]
If initialising the server context fails, but the client one succeeds,
we will not only leak memory, but the error message reported for
initialising the server context might not make sense, because we
initialise the client context after and that could erase or change the
list of queued errors.
This scenario is considered rare. Nevertheless, we now initialise the
client context after *successfully* initialising the server context.
Commit cf12678 introduced a fix for issue #186 by freeing the old SSL_CTX
structure before constructing a new one, which could disconnect existing
clients otherwise.
Unfortunately, the freeing is done first, which means that if setting up
a new structure fails for any reason, there will be no usable structures
left, but they are still referenced.
This fix moves the freeing to the end of the function, using intermediate
new variables in the meantime. This problem was discovered while testing
against OpenSSL 1.1.0 RC6.
These include warnings about "break" statements that will never be
executed (because they are after "return" statements), unused macros
(lost to code refactoring or never even used in the first place),
functions that call abort() or loop indefinitely but aren't marked with
the "noreturn" attribute, and use of variables possibly uninitialised
(a false positive).
LibreSSL does not have the new version macros & functions that OpenSSL
1.1.0 implements. This causes a compile-time failure against LibreSSL.
Further, the runtime function for returning the library version returns
the wrong number (the hardcoded constant number SSLEAY_VERSION_NUMBER
aka OPENSSL_VERSION_NUMBER, instead of LIBRESSL_VERSION_NUMBER).
Add more ifdef soup to remedy the situation.
The code already assumes the presence of fopen(3) and errno, and, by
extension, fclose(3) and strerror(3), so just use those instead of the
BIO wrappers.
Additionally, don't fail to initialise if the DH file does exist but
parsing it fails, as per the pre-existing comment about them being
optional.
* Move certificate, key, DH parameters and configuration to heap
(Documentation states that setting new configuration, e.g.
during a rehash, is unsupported while connections using that
configuration are active)
This is the same approach as the fix for #186
Refcount these structures so as to not introduce a memory leak
On rehash, it will use new structures only if there are no
errors in constructing them
* Make fingerprint generation work for TLS connections
See the comments in the newly created file for an explanation
* Fix memory leak when generating a fingerprint from a file
* Add better error-reporting (strings in addition to numbers)
where possible
* Coalesce several connection memory allocations into one function
* Reduce boilerplate where possible (Charybdis targets C99)
* Support private key being in certificate file, and having no
DH parameters file
* Correct erroneous closing comment
* Don't manually initialise libssl 1.1.0 -- it does this automatically
* SSL_library_init() should be called first otherwise
* Move SSL_CTX construction to rb_setup_ssl_server()
* Test for all required files (certificate & key) before doing anything
* Free the old CTX before constructing a new one (Fixes#186)
* Don't try to set options / ciphers etc on a NULL CTX
* Clean up ifdef indentation
* Fix DH parameters memory leak
The OpenSSL backend is the only one that assigns a non-constant
value to the length variable. Use the correct type for its
pointer and cast instead.
[ci skip]
* Certificate fingerprint length functions return an "int", so use an
int when calculating the length
* Clean up the OpenSSL certificate fingerprint if() and indentation mess
These operate on the SubjectPublicKeyInfo of the certificate, which does
change unless the private key is changed. This allows the fingerprint to
stay constant even if the certificate is reissued.
(The same fingerprint is also used by DANE)
Provider ID's are now assigned dynamically at load-time. To accomodate
this, there is now a lookup system for finding providers by name (all
providers have names as well).
