[RFC] Updating boot loaders in lenny and squeeze

[Stephen Powell]

Stephen Powell recently reminded the kernel team that LILO is not
automatically updated on installation of a new kernel version in lenny.
In fact there is a general problem that there are several different ways
a boot loader may be updated automatically and currently no guarantee
that this does happen whenever it should.
There are two major cases where a boot loader should be updated
1. Where the boot loader relies on block lists rather than reading the
filesystem, these block lists need to be updated whenever a kernel image
or an initramfs is updated or removed.
2. Where the boot loader allows selection between arbitrarily many
installed versions, the configuration should be updated whenever a
kernel package is newly installed or removed.
A. Packages built with older versions of kernel-package, including the
official kernel packages in etch, run a platform-specific default boot
loader if it is installed and if it falls into case 1 above.
B. The maintainer scripts of kernel packages invoke hook scripts
installed in appropriate subdirectories under /etc/kernel/.
C. The maintainer scripts of kernel packages invoke hook commands
specified in /etc/kernel-img.conf.
D. "update-initramfs -u" will run certain boot loader update programs if
installed.
In lenny, route A was effectively disabled in official kernel packages.
However, no boot loader uses route B and debian-installer enables route
C only for GRUB (as far as I know). Disabling route A in lenny was
clearly premature and I intend to restore it in a stable update.
However, given that route B is present in both lenny and squeeze (and
even in etch, I think) there is no good reason for packages not to rely
on it now. (Well, except lack of documentation.) All boot loader
packages that fall into case 1 or 2 should be installing hook scripts,
but currently only extlinux does.

That's a well-organized summary, Ben.

I intend to remove the vestigial code
for route A, and file bugs on all boot loaders in case 1 or 2 that do
not use route B.

I believe that bug number 585856 will qualify for this purpose for the
lilo boot loader. If you decide to extend this maintainer script
"feature" to the s390 platform also (for Squeeze and later releases)
then you should, at the same time, file a bug report against the
s390-tools package, which includes the zipl boot loader, which has the
same basic design as lilo. (It reads the kernel and the initrd as
a list of blocks, which must be kept in sync with the file system.)

In case 1, the boot loader should also be called by update-initramfs
when it is called outside of kernel package installation and removal.
This is normally covered by route D, but this seems a little fragile. I
would prefer to replace this with a hook system (as already exists for
building the initramfs itself), but I'm not that involved in
initramfs-tools development.

That does seem like a more general-purpose solution, rather than having
lilo and zipl treated as special cases. But please keep the appropriate
parties informed of any future design changes to update-initramfs.
I myself have never used yaird, but I assume that to be consistent it
should have a similar hook system.
--
.''`. Stephen Powell
: :' :
`. `'`
`-

[Ben Hutchings]

[...]

Post by Stephen Powell

In case 1, the boot loader should also be called by update-initramfs
when it is called outside of kernel package installation and removal.
This is normally covered by route D, but this seems a little fragile. I
would prefer to replace this with a hook system (as already exists for
building the initramfs itself), but I'm not that involved in
initramfs-tools development.

That does seem like a more general-purpose solution, rather than having
lilo and zipl treated as special cases. But please keep the appropriate
parties informed of any future design changes to update-initramfs.
I myself have never used yaird, but I assume that to be consistent it
should have a similar hook system.

yaird is no longer present in testing or unstable, so it is a non-issue
as far as I'm concerned.

Ben.
--
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
- Albert Camus

[Stephen Powell]

Post by Ben Hutchings
yaird is no longer present in testing or unstable, so it is a non-issue
as far as I'm concerned.

I see it is gone from testing, but it is still present in unstable, as of
this moment, according to the Debian web site.
--
.''`. Stephen Powell
: :' :
`. `'`
`-

[maximilian attems]

Post by Stephen Powell

Post by Ben Hutchings
yaird is no longer present in testing or unstable, so it is a non-issue
as far as I'm concerned.

I see it is gone from testing, but it is still present in unstable, as of
this moment, according to the Debian web site.

it has no support for any recent linux-2.6.

[Jonas Smedegaard]

Post by Stephen Powell
That does seem like a more general-purpose solution, rather than having
lilo and zipl treated as special cases. But please keep the appropriate
parties informed of any future design changes to update-initramfs.
I myself have never used yaird, but I assume that to be consistent it
should have a similar hook system.

A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and kernel-package.

Sure would be nice with a stable ABI again, and getting informed if it
changes.


- Jonas

[Ben Hutchings]

Post by Jonas Smedegaard

Post by Stephen Powell
That does seem like a more general-purpose solution, rather than having
lilo and zipl treated as special cases. But please keep the appropriate
parties informed of any future design changes to update-initramfs.
I myself have never used yaird, but I assume that to be consistent it
should have a similar hook system.

A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and kernel-package.
Sure would be nice with a stable ABI again, and getting informed if it
changes.

That is a separate issue. What we need here is an interface for the
initramfs builder to update the boot loader if necessary. No such
interface exists yet, AFAIK.

I suggest something like the following:

1. Boot loaders that maintain block lists install a script under
/etc/mkinitramfs/post-update.d which takes two arguments: the kernel ABI
version (uname -r) and the absolute path to an initramfs.

2. Initramfs builders call the scripts in this directory after creating,
updating or deleting an initramfs by running:
run-parts --verbose --exit-on-error --arg=$version --arg=$path /etc/mkinitramfs/post-update.d
or similar.

We could alternately use multiple directories or an argument to
distinguish creation, update and deletion. However, I suspect that
these scripts will need to invoke the same command in all cases.

Ben.

[Stephen Powell]

Post by Ben Hutchings

Post by Jonas Smedegaard
A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and kernel-package.
Sure would be nice with a stable ABI again, and getting informed if it
changes.

That is a separate issue. What we need here is an interface for the
initramfs builder to update the boot loader if necessary. No such
interface exists yet, AFAIK.
1. Boot loaders that maintain block lists install a script under
/etc/mkinitramfs/post-update.d which takes two arguments: the kernel ABI
version (uname -r) and the absolute path to an initramfs.
2. Initramfs builders call the scripts in this directory after creating,
run-parts --verbose --exit-on-error --arg=$version --arg=$path /etc/mkinitramfs/post-update.d
or similar.
We could alternately use multiple directories or an argument to
distinguish creation, update and deletion. However, I suspect that
these scripts will need to invoke the same command in all cases.

Sounds reasonable to me. This is for Squeeze+1, right?
--
.''`. Stephen Powell
: :' :
`. `'`
`-

[Jonas Smedegaard]

Post by Ben Hutchings

Post by Jonas Smedegaard

Post by Stephen Powell
That does seem like a more general-purpose solution, rather than
having lilo and zipl treated as special cases. But please keep the
appropriate parties informed of any future design changes to
update-initramfs. I myself have never used yaird, but I assume that
to be consistent it should have a similar hook system.

A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and
kernel-package.
Sure would be nice with a stable ABI again, and getting informed if
it changes.

That is a separate issue. What we need here is an interface for the
initramfs builder to update the boot loader if necessary. No such
interface exists yet, AFAIK.

Agreed, this is a different ABI. The wish for such ABI being treated as
a cross-package ABI still exist.

One approach would be to create a page at wiki.debian.org which all
interested parties could then subscribe to.

I would dislike if (as in the past) we simply rely on whatever internal
routines implemented by the most popular packages (initramfs-tools and
minux-2.6) which others then need to track sources of.

Post by Ben Hutchings
1. Boot loaders that maintain block lists install a script under
/etc/mkinitramfs/post-update.d which takes two arguments: the kernel ABI
version (uname -r) and the absolute path to an initramfs.
2. Initramfs builders call the scripts in this directory after creating,
run-parts --verbose --exit-on-error --arg=$version --arg=$path /etc/mkinitramfs/post-update.d
or similar.
We could alternately use multiple directories or an argument to
distinguish creation, update and deletion. However, I suspect that
these scripts will need to invoke the same command in all cases.

Seems reasonable to me.


- Jonas

[Ben Hutchings]

Post by Jonas Smedegaard

Post by Ben Hutchings

Post by Jonas Smedegaard

Post by Stephen Powell
That does seem like a more general-purpose solution, rather than
having lilo and zipl treated as special cases. But please keep the
appropriate parties informed of any future design changes to
update-initramfs. I myself have never used yaird, but I assume that
to be consistent it should have a similar hook system.

A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and
kernel-package.
Sure would be nice with a stable ABI again, and getting informed if
it changes.

That is a separate issue. What we need here is an interface for the
initramfs builder to update the boot loader if necessary. No such
interface exists yet, AFAIK.

Agreed, this is a different ABI. The wish for such ABI being treated as
a cross-package ABI still exist.
One approach would be to create a page at wiki.debian.org which all
interested parties could then subscribe to.
I would dislike if (as in the past) we simply rely on whatever internal
routines implemented by the most popular packages (initramfs-tools and
minux-2.6) which others then need to track sources of.

I agree.

Post by Jonas Smedegaard

Post by Ben Hutchings
1. Boot loaders that maintain block lists install a script under
/etc/mkinitramfs/post-update.d which takes two arguments: the kernel ABI
version (uname -r) and the absolute path to an initramfs.
2. Initramfs builders call the scripts in this directory after creating,
run-parts --verbose --exit-on-error --arg=$version --arg=$path /etc/mkinitramfs/post-update.d
or similar.
We could alternately use multiple directories or an argument to
distinguish creation, update and deletion. However, I suspect that
these scripts will need to invoke the same command in all cases.

Seems reasonable to me.

There is a minor problem with this, which is that it will likely result
in updating the boot loader twice during a kernel installation or
upgrade. We could avoid that by specifying that:

3. Boot loaders must install kernel hook scripts named beginning with
'zz-'. All other packages must use names that sort before this. (This
ensures that the boot loader update happens last.)

4. Initramfs builders may omit calling initramfs hook scripts when they
are invoked from a kernel hook script.

We're still left with the question of how to transition from the current
mess.

Ben.

[Stephen Powell]

Post by Ben Hutchings
There is a minor problem with this, which is that it will likely result
in updating the boot loader twice during a kernel installation or
3. Boot loaders must install kernel hook scripts named beginning with
'zz-'. All other packages must use names that sort before this. (This
ensures that the boot loader update happens last.)

I'm glad to see that someone is finally beginning to see the need for
a naming convention for hook scripts to control execution order.
Franz Pop brought up this topic in May of 2009, but no-one seemed
to think it was important then.
(See http://lists.debian.org/debian-kernel/2009/03/msg00611.html)

Post by Ben Hutchings
4. Initramfs builders may omit calling initramfs hook scripts when they
are invoked from a kernel hook script.
We're still left with the question of how to transition from the current
mess.

--
.''`. Stephen Powell
: :' :
`. `'`
`-

[Ben Hutchings]

Post by Ben Hutchings

Post by Jonas Smedegaard
A great while back initramfs-tools and kernel packages broke the ABI
coordinated across initramfs-tools, linux-2.6, yaird and kernel-package.
Sure would be nice with a stable ABI again, and getting informed if it
changes.

That is a separate issue. What we need here is an interface for the
initramfs builder to update the boot loader if necessary. No such
interface exists yet, AFAIK.
1. Boot loaders that maintain block lists install a script under
/etc/mkinitramfs/post-update.d which takes two arguments: the kernel ABI
version (uname -r) and the absolute path to an initramfs.
2. Initramfs builders call the scripts in this directory after creating,
run-parts --verbose --exit-on-error --arg=$version --arg=$path /etc/mkinitramfs/post-update.d
or similar.
We could alternately use multiple directories or an argument to
distinguish creation, update and deletion. However, I suspect that
these scripts will need to invoke the same command in all cases.

Sounds reasonable to me. This is for Squeeze+1, right?

No, we need something like this for squeeze.

Ben.

[Stephen Powell]

Post by Ben Hutchings

Sounds reasonable to me. This is for Squeeze+1, right?

No, we need something like this for squeeze.

As for "update-initramfs -u", it *will* invoke lilo if lilo is installed
and "do_bootloader = yes" is specified in /etc/kernel-img.conf, which I
highly recommend.

this fall back will be gone as soon as squeeze is out.
so you'd really need to gear up.

(The above quotes are from the bug log for Debian bug number 505609.)
This led me to believe that, for lilo and zipl anyway, specifying

do_bootloader = yes

in /etc/kernel-img.conf would be sufficient to get the boot loader
run when "update-initramfs -u" is executed at least through and
including the Squeeze release. But in Squeeze+1 this "fallback", as
Max put it, will no longer work and therefore a new architecture
will be needed. Did I misunderstand something?
--
.''`. Stephen Powell
: :' :
`. `'`
`-

[Ben Hutchings]

Post by Stephen Powell

Post by Ben Hutchings

Sounds reasonable to me. This is for Squeeze+1, right?

No, we need something like this for squeeze.

As for "update-initramfs -u", it *will* invoke lilo if lilo is installed
and "do_bootloader = yes" is specified in /etc/kernel-img.conf, which I
highly recommend.

this fall back will be gone as soon as squeeze is out.
so you'd really need to gear up.

(The above quotes are from the bug log for Debian bug number 505609.)
This led me to believe that, for lilo and zipl anyway, specifying
do_bootloader = yes
in /etc/kernel-img.conf would be sufficient to get the boot loader
run when "update-initramfs -u" is executed at least through and
including the Squeeze release. But in Squeeze+1 this "fallback", as
Max put it, will no longer work and therefore a new architecture
will be needed. Did I misunderstand something?

Maybe we can scrape along without changing this, but since we already
need to change the boot loader packages for squeeze we may as well sort
this out. This also gives Jonas a chance to make yaird just work
without having to introduce the same sort of kluge.

Ben.