Cascade bumping of prerequisite version

Introducing backward-incompatible change to a piece of code, especially if that code has a lot of dependants (i.e. located more upstream in the river, if we’re using the river of CPAN analogy), will cause pain. But sometimes you need or want to do it anyway.

Suppose you have this tree of dependencies:

Aa (0.01)
    Bb (0.01, requires Aa=0)
    Cc (0.01, requires Aa=0)
        Dd (0.01, requires Cc=0)
        Ee (0.01, requires Cc=0)
            Ff (0.01, requires Ee=0)
            Gg (0.01, requires Ee=0)
    Hh (0.01, requires Aa=0.01)

Now a backward-incompatible change is introduced in Aa, and you release Aa 0.02. Let’s say this change happens to affect Bb and Cc but not Hh.

If a system updates Aa to 0.02, suddenly Bb and Cc will break. And Dd, Ee, Ff, Gg will break too because Bb and Cc break. Aa can be updated due to a variety of causes, from manually for testing (like in a CPAN Testers machine) or because user installs something else like say Ii which needs Aa=0.02.

So you now release Bb 0.02 and Cc 0.02 to cope with the changes introduced in Aa. And these updates are not backward-incompatible so users of Bb and Cc can still specify Bb=0 or Cc=0.01. The dependency tree becomes like this:

Aa (0.02)
    Bb (0.02, requires Aa=0.02)
    Cc (0.02, requires Aa=0.02)
        Dd (0.01, requires Cc=0)
        Ee (0.01, requires Cc=0)
            Ff (0.01, requires Ee=0)
            Gg (0.01, requires Ee=0)
    Hh (0.01, requires Aa=0.01)

If a system happens to update just Bb or Cc, Aa will be correctly updated automatically to 0.02.

If a system just updates Aa, the same situation still happens: Bb & Cc will break, Dd, Ee, Ff, Gh will break too because Bb and Cc break.

Now suppose you add a new feature to Ff and release new version of Ff (0.02, requires Ee=0). Even though this does not have to do with backward-incompatible change of A 0.02, breakage might still happen. Let’s say a CPAN Testers machine tries to install Ff 0.02. The machine won’t automatically upgrade Bb and Cc because the specified dependency of Ff 0.02 doesn’t require it too. Now the test will fail when there is a new Aa (0.02) installed but old versions of Bb and Cc.

This is exactly what happens to me a few times, most recently in the case of Data::Sah 0.79. After I released Data::Sah 0.79, and then a couple of weeks after that release some other distributions that do not directly depend on it, some CPAN Testers machines will start reporting failure for these distributions. This is because the machines happen to have the updated Data::Sah but some older direct dependants which break under the new Data::Sah.

So back to our Aa example, to properly induce a cascade update, after we release Bb 0.02 and Cc 0.02, we also need to release Dd and Ee just to bump the prerequisite version of Cc to 0.02 even though Dd and Ee don’t exactly require Cc 0.02 (they can live with Cc 0.01). And repeat the process recursively: update Ff and Gg just to bump prerequisite version of Ee, and so on.

Thus, if a system updates Gg 0.02, Ee will automatically be upgraded to 0.02, Cc automatically upgraded to 0.02, and Aa automatically upgraded to 0.02.

To reiterate: after we introduce a backward-incompatible update to a module, we must update all the direct dependants of that module that are affected by the change, and also recursively update all their dependants just to bump the minimum prerequisite version and force pulling the module’s and direct dependants’ update.

In the case of Data::Sah, this involves hundreds of distributions because Perinci::CmdLine::Lite is a direct dependant that is affected. And Perinci::CmdLine::Lite (via Perinci::CmdLine::Any) is used by many of my App:: distributions. But fortunately, on a production system, Data::Sah typically won’t be updated without Perinci::CmdLine::Lite also being updated.

Advertisements

7 thoughts on “Cascade bumping of prerequisite version

  1. Even that is not enough. Consider what happens if a user has Bb 0.01 and Cc 0.01 already installed, and then they install Gg. This causes Ee to be installed or upgraded, which causes Cc to be upgraded, which causes Aa to be upgraded.

    But it does not cause Bb to be upgraded.

    So now their perl installation is broken.

    Or, likewise, they don’t install Gg, they install Xx, which requires Aa 0.02. Or, they have code that uses Aa itself and they upgrade Aa directly. In every case, the already-installed Bb and Cc are now both broken.

    To handle that you need something like Dist::CheckConflicts on top of all the things you are already doing.

    Like

    • True, either Dist::CheckConflicts or the client performs a complete “system” upgrade (something like cpan-outdated) which will upgrade all of Aa, Bb, Cc, Dd and the rest.

      In my case (of just preventing bogus CPANTesters failure reports) the above method already works, because a CPANTesters machine only tests the distributions it installs (directly).

      Like

      • And… you consider that a good thing? When a user can find themselves with a broken perl in a situation where CPAN Testers will give you the all-green PASS, then what’s the point of CPAN Testers?

        Like

        • I should correct myself when I said that the CPAN Testers failure reports described in the post are “bogus”: they are not and they point out a real problem that does need to be solved. The problem is, a) a module (Aa) has been updated which is backward incompatible with older version; b) there are direct dependants of Aa that are affected by this change (Bb and Cc) which already have updated releases to cope with the change but are not automatically updated when an indirect dependant of Aa (Ff) is installed and tested by a CPAN Testers machine. By doing updated releases of indirect dependants just to bump minimum version and force cascading update from the direct prerequisites of Ff up to Bb and Cc, I solve this problem. And CPAN Testers stops reporting this kind of failure reports.

          The case that you described in the first comment is: user already has Bb and Cc then installs Gg and causes Ee, Cc, and Aa to be installed/upgraded. But Bb is not upgraded automatically. Now when the user is a CPAN Testers machine doing this (install Gg), it will not detect this nor it can/should. When it installs Gg, it means it wants to test Gg and Gg alone, not Aa nor Bb or Cc and so on. In fact, Bb is not related/involved at all as it is not in the dependency chain of Gg.

          But a CPAN Testers machine can install and test a new release of Bb separately (which it normally does).

          Of course this does not solve other problems that might be caused by a backward-incompatible Aa. There might be other code e.g. application code that are not released as a CPAN distribution that depends on the old Aa which got affected that are not tested by CPAN Testers. But as far as CPAN Testers goes, it does its job.

          Now please tell me again how, by doing the cascade bumping as I described above, I will somehow cause a false negative to CPAN Testers as you mentioned?

          Like

          • Sorry it took me so long to respond. You’re right that it doesn’t cause false negatives. It’s still the case that prereq bumping alone won’t solve the problem (while at the same time in some ways oversolves it – as you pointed out, it has to force upgrades that aren’t strictly necessary), and that something like the conflict checker is necessary to address that. But, you’re right that it does not cause false negative CPAN Testers reports, and you’re right that that is because breaking a perl installation by upgrading a reverse dependency of multiple modules to an incompatible version is not a failure that fits into the CPAN Testers model. (It’s not an easily detected failure in general… even though it is an important one.)

            Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s