CESNET
Versions don't match anymore
A new tagged release was made where the SO version was move from 2 to 3; however, the tagged project version appears to remain at 2.
Having the versions diverge like this just makes things really complex and hard for projects that are trying to use libyang. Why shouldn't the project version number also increment to indicate the non-backward compat changes as well? Packaging should also change from libyang2 to libyang3, I would think, if it hasn't.
Yes, SO version has increased to 3 (because it had to) but project version only to 2.2 (from 2.1) because the changes were not that significant. Why is that a problem? The packages should use the major SO version as suffix and the included scripts do, I also know about other similar packages, which have a completely unrelated version and SO version, never had any issues with that.
Who is the consumer of this 2.2 version string (i.e., who is using or cares about the somewhat arbitrary "not that significant" information)? Users and project owners care about whether things work or not, and the major version update informs on that. Also, if you look at pkg-config tests in autoconf I think they normally use the version string you are not changing the major of.
SO version is now 3, that is a significant change, even if it were a single code line change. A major version number pump is cheap. Why not just do that and avoid the confusion? It is not like you are going to run out of digits to use for major numbers any time soon.
from https://semver.org/ :
Given a version number MAJOR.MINOR.PATCH, increment the:
MAJOR version when you make incompatible API changes MINOR version when you add functionality in a backward compatible manner PATCH version when you make backward compatible bug fixes
In the world of software management there exists a dreaded place called “dependency hell.” The bigger your system grows and the more packages you integrate into your software, the more likely you are to find yourself, one day, in this pit of despair.
In systems with many dependencies, releasing new package versions can quickly become a nightmare. If the dependency specifications are too tight, you are in danger of version lock (the inability to upgrade a package without having to release new versions of every dependent package). If dependencies are specified too loosely, you will inevitably be bitten by version promiscuity (assuming compatibility with more future versions than is reasonable). Dependency hell is where you are when version lock and/or version promiscuity prevent you from easily and safely moving your project forward.
Of course, Libyang as a project can just ignore all of that and leave it to downstream projects, package maintainers, and system integrator to sort out the dependencies and give libyang whatever version number they want. Or Libynag can from the start adhere to common versioning practices and make everyone's life easier.
The motivation to use project and SO version is to have a version for both use-cases. To learn the details of API/ABI compatibility between versions and to learn the general significance of changes, respectively.
I have looked at the libraries installed on my system and all except one (libxml2) had a completely different SO version (in the name of the library file) than the version of the package they were in. So I am confused why we are even discussing this and why it should be a problem.
@michalvasko : even if technically, you are right, so versions and library/code version can be uncorrelated, we have to think about the maintenance streams and the visibility for those who will just consume the deb/rpm packages. Since we create a major change of the ABI which leads to libyang so 3, let's align the source code to this version too, please ?
But can you explain why this is a problem when the great majority of projects follow similar double versioning schemes? Why should we now suddenly drop the project version? And for packages I think it is no problem at all, you can specify dependencies for exact packages and exact (or ranges of) versions, users could not care less since the dependencies are downloaded automatically.
Michal Vasko @.***> writes:
But can you explain why this is a problem when the great majority of projects follow similar double versioning schemes? Why should we now suddenly drop the project version? And for packages I think it is no problem at all, you can specify dependencies for exact packages and exact (or ranges of) versions, users could not care less since the dependencies are downloaded automatically.
If the packaging system only has an FRR package that works with version 2 of libyang, and this is guaranteed since we have not released a version of FRR that works with your new "minor update" that has a new major SO version 3 -- it won't even compile with it -- and the libyang package is updated in that package system, the user now has no way to install a working FRR.
I can't claim to understand every packaging system out there, debian/ubuntu (apt), redhat/centos (rpm/yum), alpine (apk), archlinux (pacman?), gentoo (dunno), netbsd (pkgsrc), freebsd (ports) -- can you?
I'm pretty sure that almost all of them don't allow 2 versions of the same package to be installed simultaneously -- this means we need a "libyang2" and a "libyang3" package for things to actually work -- Now do you also expect them all to create a "libyang2" and a "libyang3" both based on a version 2.x.x of the software -- I don't I think they will be confused and mistakes will be made.
I do know that some of these systems have split groupings where libyang might be in the main infra category and FRR might be in a user supported one, and that it's conceivable the main maintainers might update libyang w/o testing the huge number of user supported packages b/c it was a "minor" version bump in the sofware version.
Maybe it all works like you say in every one of those pacakge systems, and there will be no issue; however, since libyang has already caused API breakages with incorrect versioning in the past, consider me highly skeptical.
Christian Hopps @.***> writes:
[[PGP Signed Part:Good signature from 2E1D830ED7B83025 Christian Hopps @.***> (trust ultimate) created at 2024-04-10T09:05:20-0400 using RSA]]
Michal Vasko @.***> writes:
But can you explain why this is a problem when the great majority of projects follow similar double versioning schemes? Why should we now suddenly drop the project version? And for packages I think it is no problem at all, you can specify dependencies for exact packages and exact (or ranges of) versions, users could not care less since the dependencies are downloaded automatically.
If the packaging system only has an FRR package that works with version 2 of libyang, and this is guaranteed since we have not released a version of FRR that works with your new "minor update" that has a new major SO version 3 -- it won't even compile with it -- and the libyang package is updated in that package system, the user now has no way to install a working FRR.
Perhaps this is an important point. The SO version is about the ABI compatibility -- a binary linked against SO version 2.x.0 will dynamically load and be expected to run with SO version 2.y.0 but it will not try and load and run with SO version 3.z.0.
The software version is perhaps more about the API compatibility. Here you've only changed the minor component of the version yet FRR will no longer even compile with this new version of the software.
If the packaging system only has an FRR package that works with version 2 of libyang, and this is guaranteed since we have not released a version of FRR that works with your new "minor update" that has a new major SO version 3 -- it won't even compile with it -- and the libyang package is updated in that package system, the user now has no way to install a working FRR.
The package libyang2 will not be updated because there is no update (unless some maintainers decide to update it despite API incompatibility and our scripts using libyang3). And libyang3 package should fine alongside libyang2.
I'm pretty sure that almost all of them don't allow 2 versions of the same package to be installed simultaneously -- this means we need a "libyang2" and a "libyang3" package for things to actually work -- Now do you also expect them all to create a "libyang2" and a "libyang3" both based on a version 2.x.x of the software -- I don't I think they will be confused and mistakes will be made.
I suppose that can happen, will not be our fault, though. We did all we could to make sure version 2.2.x package is named libyang3.
I do know that some of these systems have split groupings where libyang might be in the main infra category and FRR might be in a user supported one, and that it's conceivable the main maintainers might update libyang w/o testing the huge number of user supported packages b/c it was a "minor" version bump in the sofware version.
I would hope they would at least read the changelog on every release and I tried to highlight there the information about new SO version.
Maybe it all works like you say in every one of those pacakge systems, and there will be no issue; however, since libyang has already caused API breakages with incorrect versioning in the past, consider me highly skeptical.
Whatever the case in the past, any problems now it will directly be our fault. In any case, the release has been made. If some major problems appear as you anticipate, we can discuss this again.
The software version is perhaps more about the API compatibility. Here you've only changed the minor component of the version yet FRR will no longer even compile with this new version of the software.
Yes, project version carries no information about API/ABI compatibility since that is expressed in the SO version.
Michal Vasko @.***> writes:
The package libyang2 will not be updated because there is no update (unless some maintainers decide to update it despite API incompatibility and our scripts using libyang3). And libyang3 package should fine alongside libyang2.
Both your header install directory and your shared-object name do not include the major version number, they both use "libyang" not "libyang[23]" so I don't see how those things live together on the same system.
So if the user updates from libyang 2.1.x to 2.2.y they can no longer build FRR.
Can you give a couple examples of other these projects you mentioned that have successfully deployed major SO version changes w/o updating their project version? It would perhaps be useful to look closer at what they did.
Both your header install directory and your shared-object name do not include the major version number, they both use "libyang" not "libyang[23]" so I don't see how those things live together on the same system.
Yes, you should always have only the latest devel package and applications should not link against libraries without SO versions, I am actually not sure why a symlink with such name is being installed.
So if the user updates from libyang 2.1.x to 2.2.y they can no longer build FRR.
If using the source directly then yes.
Can you give a couple examples of other these projects you mentioned that have successfully deployed major SO version changes w/o updating their project version? It would perhaps be useful to look closer at what they did.
I have really just went through the libraries I found on my system, OpenSUSE (RPM). For example, libssh4 (package 0.10.6, SO 4.9.6), libcmocka0 (package 1.1.7, SO 0.8.0), libcurl4 (package 8.7.1, SO 4.8.0), libpcre2-8-0 (package 10.43, SO 0.12.0). Such was approximately the bigger half of the libraries I checked, the rest had either matching major versions but the rest different or a few even using the exact SO version as the package version.
Just to add my two cents, I don't think there's any actual breaking problem with having the versions as they are right now, but it's just confusing.
First, people don't usually expect breaking changes without a major version update. So it might be a surprise that the project doesn't build anymore after you upgraded the library from 2.1 to 2.2.
Second, there's a confusion with the naming now. How do we call the current libyang? Is it libyang3 now? But why then the version is 2.2? Or is it still the same libyang2? But why did my build broke then?
As I said, all this is not a huge problem, especially because it's all development-related stuff and actual end-users are not affected. But having at least the same major version for the project and the SO would be much more easier to understand.
And BTW it's version 3 in the code (LY_VERSION_MAJOR). So basically in the build system (autoconf/cmake/other) we'll have a dependency like libyang >= 2.2 and in the code it will be LY_VERSION_MAJOR >= 3 which is pretty confusing as well.
+1 with @idryzhov : for example, https://github.com/FRRouting/frr/pull/15608/files#diff-35a25e3a633413a63808f112260bca3ac308fd18325d4044c95e709899640981R3687
Anyone who will review this code will be confused:
#if (LY_VERSION_MAJOR < 3)
LY_ERR err = ei->no;
#else
LY_ERR err = ei->err;
#endif
whether it shall apply on libyang 2.2.8 using a so version 3.
The macros can be renamed (added new ones and at some point the current ones deleted) if you think it will help because it should explicitly say it is SO version.
I have really just went through the libraries I found on my system, OpenSUSE (RPM). For example, libssh4 (package 0.10.6, SO 4.9.6), libcmocka0 (package 1.1.7, SO 0.8.0), libcurl4 (package 8.7.1, SO 4.8.0), libpcre2-8-0 (package 10.43, SO 0.12.0). Such was approximately the bugger half of the libraries I checked, the rest had either matching major versions but the rest different or a few even using the exact SO version as the package version.
So the only package you have here that has an SO major versions that is less than the package version (i.e., they incremented the SO major but not the package major at some point like you have done) is libssh4 and it's version is 0 which is well understood to be a special version number allowing for surprises.
I don't see any non-zero major examples above where the package version major is less than the SO major.
This is the first time you mention the problem is that the package version is lower than the SO version, we were talking about it being different. In any case, we are stuck and not moving anywhere in the discussion so what I last said stands. If there are major problems with our versioning in the near future, there can be some changes such as moving the project major version to 3. Otherwise it stays the way it is.
Michal Vasko @.***> writes:
This is the first time you mention the problem is that the package version is lower than the SO version, we were talking about it being different. In any case, we are stuck and not moving anywhere in the discussion so what I last said stands. If there are major problems with our versioning in the near future, there can be some changes such as moving the project major version to 3. Otherwise it stays the way it is.
No, I (and everyone else) have been saying all along that you need to rev the major of the package version along with revving the major of the SO.
I am pointing out to you that all your example projects you give as justification of your action have in fact probably never done what you did -- they have more frequently increased their project version, not their SO version, and I'm guessing if you go and look they have always increased their project major whenever their SO major increased.
SO version and package version don't need to match, we never said that. In all of the example you gave, and as @choppsv1 pointed out, the package version is higher than the SO version. The SO version needs only to be incremented when there are backward incompatible changes. Package version might be incremented multiple times to reflect major new features in the software without breaking anything related to SO. The library/SO is still part of the whole software, so when you increment the SO version, you MUST increment the package version if you want to adhere to common/standard practices. It is not like the SO development lives on its own, it is still part of your project, so the major version of the project should reflect a big change in the SO file, it does not have to match it.
Second, there's a confusion with the naming now. How do we call the current libyang? Is it libyang3 now? But why then the version is 2.2? Or is it still the same libyang2? But why did my build broke then?
When Libyan version 3.0 eventually comes out. Package maintainers would be forced to call that libyang4 given the history of how versions have been tracked so far in this project.
Just to add my two cents, I don't think there's any actual breaking problem with having the versions as they are right now, but it's just confusing.
Versions can be anything you want. Version numbers are there for a reason. If there don't serve a purpose, might as well have random numbers and they would still "work".
When Libyan version 3.0 eventually comes out. Package maintainers would be forced to call that libyang4 given the history of how versions have been tracked so far in this project.
What is the reason? I mean, in all likelihood, if libyang project version 3.0 ever comes out, it will be accompanied with a new major SO version, but on the off chance it is not (major SO version does not change), why should the package name change?
Versions can be anything you want. Version numbers are there for a reason. If there don't serve a purpose, might as well have random numbers and they would still "work".
Well, our project version has its meaning but it is the first time I hear that it is supposed to have some special meaning (since it is used for the packages, too). I was hoping SO version with its exact definition is enough. I still see no real issue with the current state, except "not adhering to common/standard practices", so you say. Could you maybe elaborate, is this documented anywhere?
https://github.com/CESNET/libyang/issues/2220#issuecomment-2040400616
https://github.com/CESNET/libyang/issues/2220#issuecomment-2050301830
libyang releases in general (https://github.com/CESNET/libyang/releases) don't follow it. The patch level should be reserved for bug fixes which isn't the case. And Semver isn't just for SO version, it should cover the whole project.
Libyang releases use the project version, the SO version of those releases follows semver. And why should project version follow semver, why have 2 same versions?
They don't have to be the same. What is the project version? the project version should reflect changes in any of its components. Any major change in any component is a major change in the project as a whole. What is the product of this project?
And why should project version follow semver
Please see https://github.com/CESNET/libyang/issues/2220#issuecomment-2040400616
We have increased the major project version when we have rewritten the whole library from scratch and I wanted to stick to that, increase it only if there was a major change to a significant part of the whole library. It seems to me the main problem is that the project version is used for packages, perhaps it should have been the SO version instead. I think it could be changed this way.