LibreSSL update

A long list of recent LibreSSL commits by Theo Buehler (tb@) culminated in bumps to library versions:

CVSROOT:	/cvs
Module name:	src
Changes by:	tb@cvs.openbsd.org	2022/01/14 02:15:08

Modified files:
	lib/libcrypto  : shlib_version 
	lib/libssl     : shlib_version 
	lib/libtls     : shlib_version 

Log message:
bump libcrypto, libssl, libtls majors after struct visibility changes
and Symbol addition and removal in libcrypto.

Undeadly reached out to Theo asking whether he would share with readers an explanation of the changes. He kindly responded:

Today's commits were nothing super exciting, just a lot of work, most of it boring and mechanical…

One major issue with the OpenSSL code base is that many things that should have remained library internals are exposed in public headers. This means that some application is inevitably making use of the most obscure thing. Changes in public structs and function signatures usually mean that all programs need to be rebuilt and many of them need to be fixed. This strongly constrains our ability of making internal changes, i.e., taking care of much needed rewrites and cleanups. It is easier for everyone involved -- library maintainers, application authors and porters -- if only things that application programmers need to touch are exposed to them.

OpenSSL made many structs opaque in the transition from 1.0 to 1.1. This means that they hid the struct internals from the applications. While this is a good thing, it would have been even better if it had been accompanied by a well thought out, usable API and a transition plan.

So far, LibreSSL adopted as much of the OpenSSL 1.1 API as was really needed to keep the ports builds working. We will continue to do so. We did not follow the changes making the structs opaque. The reason for this was twofold:

First, we intended to keep backwards compatibility with 1.0 as far as reasonable and possible. At this point such worries are moot since OpenSSL 1.0 support has long been only available to paying customers of OpenSSL, so it is mostly dead. The ecosystem has long moved on to expecting and using OpenSSL 1.1.

Second, it is a lot of work and we lacked resources and infrastructure to do it. In spring last year, the OpenBSD foundation offered me a beefy box that allows me to do ports bulk builds on my own. This way I don't need to bother naddy or sthen with requests for bulk builds. This makes it possible to iterate quickly and make progress. I used this machine for making libssl's structs opaque during the OpenBSD 7.0 cycle, i.e., in LibreSSL 3.4.

In the final quarter of last year I had some time on my hands, so one goal I set myself for the OpenBSD 7.1 release (LibreSSL 3.5) is to make our struct visibility match OpenSSL's as far as possible and to improve compatibility with OpenSSL in general. This work allowed me to identify and eliminate a lot of the special casing for LibreSSL in ports. This has some security implications, since many code bases force LibreSSL to use some sort of legacy code path which is barely maintained and tested if at all. The biggest step for getting there was accomplished with today's flurry of commits. These were just the things I couldn't commit before. There were probably a couple hundred prior commits that led to this point.

There is still some work to be done in some remaining headers, but I don't expect it to be nearly as intrusive and painful. Hopefully I will be able to use my spare time also on more interesting and rewarding things before the next release.

inoguchi and jsing patiently reviewed my seemingly never-ending stream of boring diffs. ajacoutot and sthen were extremely helpful and tackled some of the bigger stumbling blocks in ports. Many thanks!

As you can see, donations can be very helpful, so please donate if you can! Many thanks, Theo, for both the work and the explanation!