Adding DASD to a Debian guest

[Howard V. Hardiman]

Hello,

I am configuring a golden image that will live on one piece of dasd with LVM. When I clone it I will need to add more dasd to certain of the cloned guests. So, now I am attempting to do a fresh install that has LVM for the golden image, because my current golden image does not use LVM. As you mentioned, in this process I am letting the 'installer' do the job. But, during one of the last steps of the install I get the error about zipl bootloader not being able to download and I have to skip that step. The install finishes but I cannot boot with the ipl command.

Here is a run down of what I have done relating to the LVM.

(1.) Placed a 100M partition on the dasd and made it the /boot partition
(2.) Placed the remainder of dasd in a virtual group vg1
(3.) Created logical volume lv1 = '/' and lv2=swap, using vg1
(4.) Wrote partitions and moved on with install process


If I proceed in the installation it says that I can manually boot with the /vmlinuz kernel on partition /dev/dasda1 and root /dev/mapper/vg1-lv1 passed as kernel argument. How do I do that? If that works, can I then load zipl or some bootloader that will allow me to be able to ipl the OS like normal?

Howard

[Stephen Powell]

Post by Howard V. Hardiman
Hello,
I am configuring a golden image that will live on one piece of dasd
with LVM.

OK.

Post by Howard V. Hardiman
When I clone it I will need to add more dasd to certain of
the cloned guests. So, now I am attempting to do a fresh install that
has LVM for the golden image, because my current golden image does not
use LVM. As you mentioned, in this process I am letting the
'installer' do the job. But, during one of the last steps of the
install I get the error about zipl bootloader not being able to
download and I have to skip that step. The install finishes but I
cannot boot with the ipl command.
Here is a run down of what I have done relating to the LVM.
(1.) Placed a 100M partition on the dasd and made it the /boot partition
(2.) Placed the remainder of dasd in a virtual group vg1
(3.) Created logical volume lv1 = '/' and lv2=swap, using vg1
(4.) Wrote partitions and moved on with install process
If I proceed in the installation it says that I can manually boot
with the /vmlinuz kernel on partition /dev/dasda1 and root
/dev/mapper/vg1-lv1 passed as kernel argument. How do I do that?
If that works, can I then load zipl or some bootloader that will allow
me to be able to ipl the OS like normal?

First of all, the /boot partition cannot be in an LVM2 logical volume.
The "/boot" partition must be a partition on a "physical" DASD volume,
that is, a volume which appears as a physical volume to an operating
system running in the z/VM virtual machine. In reality, this can be
a z/VM minidisk, but not an LVM2 logical volume.

I'm not sure if the "/" partition can be in a logical volume either.
But even if it can, I don't recommend it. There are a number of
maintenance situations where one wants to (or needs to) unmount
the file system in order to perform maintenance on it. Since "/" is
always mounted, that presents a problem. I highly recommend that
the "/" file system be a partition on a physical volume as well.

Here is a scenario that has worked well for me. I create four
minidisks in z/VM, with virtual device numbers 200, 201, 202, and 203.
During Debian installation I create a single partition on each device
which occupies the entire device (except for the required metadata
at the beginning: track 0 reserved for the IPL records and the
volume label, and track 1 reserved for the VTOC). I use the partition
on 200 as the "/" filesystem. I use the partition on 201 as the
"/boot" filesystem. I use the partition on 202 as the "/home"
filesystem, and I use the partition on 203 as a swap partition.

After installation, I boot the machine with "IPL 201".

I would probably make /boot about 100 cylinders. Now if you want
to use LVM2, you can. For example, if you're going to be building
packages from source, you might want to create a logical volume
and mount it on /usr/src. Or maybe you are going to be needing
a lot of space for SQL data, such as with mariadb or postgresql.
You might want to create a logical volume for that and mount it
on /dbase, or whatever mount point the database wants to use.
You can put /home on a logical volume too, if you want. But I
wouldn't put /boot or / or a swap partition on a logical volume.
You can have multiple swap partitions anyway, so expanding swap
space is not an issue. I would make / big enough so that after
installation is complete it is no more than 50% full, to allow
for future growth.

Let me know how it goes.

If you add new DASD devices to your system after installation,
whether you mount them directly or add them to a logical volume,
be sure to create files in /etc/sysconfig/hardware so that
sysconfig-hardware will bring them online at the next boot.

[Philipp Kern]

Post by Howard V. Hardiman
I am configuring a golden image that will live on one piece of dasd
with LVM. When I clone it I will need to add more dasd to certain of
the cloned guests. So, now I am attempting to do a fresh install that
has LVM for the golden image, because my current golden image does not
use LVM. As you mentioned, in this process I am letting the
'installer' do the job. But, during one of the last steps of the
install I get the error about zipl bootloader not being able to
download and I have to skip that step. The install finishes but I
cannot boot with the ipl command.
Here is a run down of what I have done relating to the LVM.
(1.) Placed a 100M partition on the dasd and made it the /boot partition
(2.) Placed the remainder of dasd in a virtual group vg1
(3.) Created logical volume lv1 = '/' and lv2=swap, using vg1
(4.) Wrote partitions and moved on with install process

I encountered the same problem recently on a crash-and-burn machine and
it needed some coercing. I see the following in dmesg:

[ 0.472658] dasd-eckd 0.0.0100: New DASD 3390/0C (CU 3990/01) with 20000 cylinders, 15 heads, 224 sectors
[ 0.484985] dasd-eckd 0.0.0100: DASD with 4 KB/block, 14400000 KB total size, 48 KB/track, compatible disk layout
[ 0.488496] dasda:VOL1/ 0X0100: dasda1 dasda2

And the partitions are set up like this:

# fdasd -p /dev/dasda

WARNING: Your DASD '/dev/dasda' is in use.
If you proceed, you can heavily damage your system.
If possible exit all applications using this disk
and/or unmount it.

reading volume label ..: VOL1
reading vtoc ..........: ok


Disk /dev/dasda:
cylinders ............: 20000
tracks per cylinder ..: 15
blocks per track .....: 12
bytes per block ......: 4096
volume label .........: VOL1
volume serial ........: 0X0100
max partitions .......: 3

------------------------------- tracks -------------------------------
Device start end length Id System
/dev/dasda1 2 5462 5461 1 Linux native
/dev/dasda2 5463 299999 294537 2 Linux lvm
exiting...

# cat /proc/mounts | grep dasd
/dev/dasda1 /boot ext2 rw,relatime 0 0
# pvs
PV VG Fmt Attr PSize PFree
/dev/dasda2 sysvg lvm2 a-- 13.48g 0

However it then turned out that I needed to hack the zipl config to make the
kernel see the DASD from within the initrd.

[ 0.066844] Kernel command line: root=/dev/sysvg/root dasd_mod.dasd=0.0.0100 BOOT_IMAGE=0

When the zipl-installer part failed you can always do two things: Go to
a shell (debian-installer) offers you this in the menu. Then run `sh -x
/var/lib/dpkg/info/zipl-installer.postinst'. That should show you where
the bootloader installation actually fails. (I.e. calling zipl, maybe
with a few log lines of what's missing.) Then you can also manually run
it via `chroot /target /bin/bash' and then running `zipl' in there.

How does it complain and what do the above commands show?

Post by Howard V. Hardiman
If I proceed in the installation it says that I can manually boot with
the /vmlinuz kernel on partition /dev/dasda1 and root
/dev/mapper/vg1-lv1 passed as kernel argument. How do I do that? If
that works, can I then load zipl or some bootloader that will allow me
to be able to ipl the OS like normal?

You will always need zipl. You could in theory punch the files from
within the chroot during installation and load that up, but the correct
way is to get zipl to install.

Kind regards and thanks
Philipp Kern

[Stephen Powell]

Post by Philipp Kern
...
However it then turned out that I needed to hack the zipl config to
make the kernel see the DASD from within the initrd.
[ 0.066844] Kernel command line: root=/dev/sysvg/root dasd_mod.dasd=0.0.0100 BOOT_IMAGE=0

That's because sysconfig-hardware isn't in the initial RAM file system,
and therefore doesn't bring DASD volumes online until the permanent root
file system has been mounted. If the permanent root file system is a
partition on a physical volume, there is exception code in

/usr/share/initramfs-tools/scripts/init-premount/sysconfig_hardware

to bring that specific volume online early, but only if the root device
is specified in zipl via the form

root=/dev/disk/by-path/ccw-0.0.xxxx-partz

where xxxx is the device number and z is the partition number. It must
be in this form so that

/usr/share/initramfs-tools/scripts/init-premount/sysconfig_hardware

knows the device number and can bring it online via

echo 1 >/sys/bus/ccw/devices/0.0.xxxx/online

This is one reason, but not the only reason, why I advised the OP not
to make the root file system a logical volume.

On my systems, I add sysconfig-hardware to the initial RAM file system,
using the method described in

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621080

but of course this cannot be done until *after* installation.
The main reason that I do it is so that I can specifiy the root
file system in zipl as

root=UUID=...

which will only work if all the DASD volumes have been brought online
already, and therefore udev has found the volumes and their partitions
and has created symbolic links for them in /dev/disk. This makes
the boot process much closer to how it works on all other hardware
platforms that Debian supports.

[Philipp Kern]

Post by Stephen Powell
That's because sysconfig-hardware isn't in the initial RAM file system,
and therefore doesn't bring DASD volumes online until the permanent root
file system has been mounted.

As much as I dislike sysconfig-hardware, it's probably the thing we
should do: Include a hook script that includes enough of
sysconfig-hardware to bring up hardware early in the initrd. (Best to
not fail if any of it is missing at this point. Debugging from within
the initrd is awkward.)

I was under the impression -- mostly because of the presence of the
mentioned hook script -- that this already happened.

Post by Stephen Powell
If the permanent root file system is a partition on a physical volume,
there is exception code in
/usr/share/initramfs-tools/scripts/init-premount/sysconfig_hardware
to bring that specific volume online early, but only if the root device
is specified in zipl via the form
root=/dev/disk/by-path/ccw-0.0.xxxx-partz
where xxxx is the device number and z is the partition number. It must
be in this form so that
/usr/share/initramfs-tools/scripts/init-premount/sysconfig_hardware
knows the device number and can bring it online via
echo 1 >/sys/bus/ccw/devices/0.0.xxxx/online
This is one reason, but not the only reason, why I advised the OP not
to make the root file system a logical volume.

The correct thing is to fix that. There are multiple facets to that.

I guess the easiest is to include the list of /etc/sysconfig/hardware
in the initrd, in the hope that the installer writes it out already
with all configured DASDs. Otherwise you need to figure out the root
disks, which is hard to generalize on Linux. (No "give me all underlying
disks for this mount point" command.)

Post by Stephen Powell
On my systems, I add sysconfig-hardware to the initial RAM file system,
using the method described in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621080
but of course this cannot be done until *after* installation.
The main reason that I do it is so that I can specifiy the root
file system in zipl as
root=UUID=...
which will only work if all the DASD volumes have been brought online
already, and therefore udev has found the volumes and their partitions
and has created symbolic links for them in /dev/disk. This makes
the boot process much closer to how it works on all other hardware
platforms that Debian supports.

I'm sympathetic to making it more alike other platforms. The route SUSE
went with running grub2 from a kernel image booted by zipl and then
kexec()ing into the right kernel and initrd is a bit too much, though.
(OTOH, if someone wants to implement that for Debian that'd be great.)

Kind regards
Philipp Kern

[Stephen Powell]

Post by Philipp Kern
As much as I dislike sysconfig-hardware, it's probably the thing we
should do: Include a hook script that includes enough of
sysconfig-hardware to bring up hardware early in the initrd. (Best to
not fail if any of it is missing at this point. Debugging from within
the initrd is awkward.)

I don't dislike sysconfig-hardware, per se, but I do dislike the lack
of documentation. It was apparently written by SUSE. They used it
themselves for a while, then abandoned it. (I don't know what they
put in its place.) Furthermore, the version we use differs from the
version SUSE last used in several ways, including naming conventions.
So for all practical purposes, Debian is now its own "upstream" for
sysconfig-hardware. So we can do whatever we want to with it.

Post by Philipp Kern
I was under the impression -- mostly because of the presence of the
mentioned hook script -- that this already happened.

No. The hook script was written by me years ago, and I put it in the
bug report, but it was never incorporated into the official package.

Post by Philipp Kern
The route SUSE
went with running grub2 from a kernel image booted by zipl and then
kexec()ing into the right kernel and initrd is a bit too much, though.

I agree.

[Philipp Kern]

Post by Stephen Powell
No. The hook script was written by me years ago, and I put it in the
bug report, but it was never incorporated into the official package.

You were of course right all along. I'll add the hook script to the
package ASAP (I just tested it). WAIT_FOR_SYSFS probably has to go,
because it's long deprecated, so I'll retry testing without it. But
apart from that it works.

Root on LVM should work afterwards as long as you have /boot outside
of it (zipl requirement). It requires another patch to zipl-installer to
make it work, but I have and tested that part already. (Essentially
taken from lilo-installer.)

Kind regards and thanks
Philipp Kern

[Stephen Powell]

Post by Philipp Kern

Post by Stephen Powell
No. The hook script was written by me years ago, and I put it in the
bug report, but it was never incorporated into the official package.

You were of course right all along. I'll add the hook script to the
package ASAP (I just tested it). WAIT_FOR_SYSFS probably has to go,
because it's long deprecated, so I'll retry testing without it. But
apart from that it works.
Root on LVM should work afterwards as long as you have /boot outside
of it (zipl requirement). It requires another patch to zipl-installer to
make it work, but I have and tested that part already. (Essentially
taken from lilo-installer.)

Another thing I would check on, both in zipl-installer and lilo-installer,
is to make sure that the /boot partition, whether it is a separate
partition or whether it is included in /, is not formatted with the btrfs
file system. I'm pretty sure that neither boot loader supports btrfs.
Only boot loaders which have file system drivers, such as grub and extlinux,
support btrfs, IIRC. Neither lilo nor zIPL understand the format of the
file system and require files to be in the "extent" format. Furthermore,
I don't believe that btrfs supports "mapping" a file to physical blocks.

Note that if /boot is a separate partition, then / can be btrfs; but if /boot
is included in /, then / cannot be btrfs.