Future of the s390 port

[STEPHEN POWELL]

The s390 port was released with Lenny. However it is not in the best
condition. There are mainly two problems which needs attention, lack of
manpower and a 64 bit userland.
The first problem is the worst. Currently only Frans Pop and I do work
on it. Frans only does the Debian-Installer part and I simply have not
enough time to do the rest. The s390 architecture is quite different to
anything else, so it needs several specialized packages to work[1] and
they need lasting attention. So if anyone wants to help (especially
Debian developers) for the continuity of this port please speak up.
The second problem is not that critical. No other distribution still
supports a complete 31 bit s390 userland and even Debian dropped the 31
bit kernel support in the meantime[2]. Strategies for an upgrade to a 64
bit userland was discussed lately[3].
I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.
Bastian

That would be awful! As you mentioned, Debian is the only remaining
viable, non-commercial distribution of Linux for s390. And it would be
a blow to Debian's goal of being "The Universal Operating System", as
they advertise on their home page. I'd like to help. What can I do?
I'm not a Debian developer or package maintainer. I'm an IBM mainframe
system programmer, with both MVS and VM responsibilities. I'm good with
S/390 assembler language, but I can't even spell C. I can test things
for you. And I have some time on my hands. (Don't tell my boss I said that!)
How can I help?

[STEPHEN POWELL]

I just joined this list server yesterday; so I apologize for
the un-timeliness of my replies to earlier posts.

Is this really an important problem?
Does a significant number of people actually use Debian/s390 on
production servers? And if they exist, why they are not helping?
--
ciao,
Marco

This logic makes an unwarranted assumption. It assumes that
those who use Linux on S/390 don't want to help. In most cases,
the problem is not a willingness to help but a lack of the requisite
skills to help. I have been using IBM mainframes for 34 years
as end user, application programmer, and system programmer. I know
a number of traditional mainframe programming languages, such as
PL/I, FORTRAN, and S/390 assembler language. And I know the
primary mainframe scripting language (REXX). But Linux is a
different animal. Linux is written in things like C, shell scripts,
perl scripts, etc. Those are not skill sets that a typical
mainframe developer would have. Similarly, the Linux gurus typically
know very little about the mainframe. The number of people with
advanced technical skills in both arenas is a very limited talent
pool. And many of them work for IBM, Suse, or Red Hat and may be
restricted by employment contract from helping with Debian.

I am willing to help. But I lack the requisite skills. I think
I have basic computer programming aptitudes, but I will need to
learn new skill sets before I can be useful. That probably won't
be in time for getting Squeeze into production. But I'm willing to
do whatever I can.

One simple way to help a bit which will make it more visible to the
Debian community at large that the s390 port is used, is to install
and activate popularity-contest on these machines and thus make sure
236 machines show up on <URL: http://popcon.debian.org/ > as running
the s390 port. :)
At the moment, only 8 machines are reporting to popularity-contest,
which put s390 in line with hurd-i386, kfreebsd-amd64 and m68k as
ports with very small user groups. That number made me and probably
others believe that very few are using the s390 port - at least until
your email came along. :)

As a number of other responders have already said, this is a security
problem in the eyes of most network managers. There are a lot of
Debian s390 host images out there. But most of them are either
internal-use-only servers with no access to the internet or are
restricted by policy from participating in popcon.

We cannot allow the s390 port of Debian to die. That would be a
great tragedy. I've been extolling the virtues of free and open source
software for a long time and management is finally beginning to listen.
I just got approval this week for a new Debian s390 server for an in-house
pilot project. One of the reasons I got the approval is Debian's
"Universal Operating System" appeal. If the s390 port dies, management
is likely to either switch their focus back to Windows or else choose
another Linux distribution to work with, not just on the mainframe,
but on Intel boxes as well. Either way, Debian loses on all platforms.

[Bastian Blank]

Post by STEPHEN POWELL
That would be awful! As you mentioned, Debian is the only remaining
viable, non-commercial distribution of Linux for s390. And it would be
a blow to Debian's goal of being "The Universal Operating System", as
they advertise on their home page. I'd like to help. What can I do?
I'm not a Debian developer or package maintainer. I'm an IBM mainframe
system programmer, with both MVS and VM responsibilities.

Run it, try the different installation methods, especially with testing
and unstable. Report the findings either as bugs or on the mailing-list.
Show things that can be made easier to understand/use for the mainframe
people.

Maybe you know how to properly IPL systems from CD/DVD and can even test
that. Maybe you have the possibility to run it on a LPAR instead of VM.

Bastian

[Stephen Powell]

Post by Bastian Blank
Run it, try the different installation methods, especially with testing
and unstable. Report the findings either as bugs or on the mailing-list.
Show things that can be made easier to understand/use for the mainframe
people.
Maybe you know how to properly IPL systems from CD/DVD and can even test
that. Maybe you have the possibility to run it on a LPAR instead of VM.
Bastian

I shall be happy to do all of the above. I do have access to a real
mainframe. It's a z/890 (2086). It's not one of the newest machines.
It's not a z9 or z10. But it is 64-bit. And I do have a spare LPAR I
can test with. And yes, I do know how to IPL Linux in an LPAR from CD.
I don't know how to create an ISO image that the mainframe considers
bootable; but if you point me to an ISO image I can burn a copy onto CD
and IPL the thing and let you know if it came up OK. I also have z/VM
5.4.0 and can install in a virtual machine under z/VM. That's how I
normally do it. z/VM's LPAR has a dedicated IFL processor, and that is
it's only processor. The other LPARs, including my test LPAR,
share a standard processor.

I'm currently working on a "how to" document that explains how to get
the high-performance DASD DIAG driver to work under Debian. I'll post
a link to it on this list when I'm done.

[Frans Pop]

Post by Stephen Powell
I shall be happy to do all of the above. I do have access to a real
mainframe. It's a z/890 (2086). It's not one of the newest machines.
It's not a z9 or z10. But it is 64-bit. And I do have a spare LPAR I
can test with. And yes, I do know how to IPL Linux in an LPAR from CD.
I don't know how to create an ISO image that the mainframe considers
bootable; but if you point me to an ISO image I can burn a copy onto CD
and IPL the thing and let you know if it came up OK.

That would be great. Please see [1] for details. If you need additional
info, just ask.

Cheers,
FJP

[1] http://lists.debian.org/debian-s390/2009/09/msg00027.html

[Stephen Powell]

Post by Frans Pop
That would be great. Please see [1] for details. If you need additional
info, just ask.
Cheers,
FJP
[1] http://lists.debian.org/debian-s390/2009/09/msg00027.html

It sounds to me like you are asking for information on how to create a
mainframe-bootable CD. I'm afraid I don't know the answer. If you
have a specific .iso image that you want me to download and try to
see if it will boot, I can do that for you. But I don't pretend
to know how to create one.

I can see, though, from the link above, that you are using the
"generic" sub-architecture files. Have you tried using the "tape"
files? The reason I ask is that, on the Multiprise 3000, the CD-ROM
could be used as an emulated tape or via the "Load from CD-ROM or
Server" method. I just wonder if the tape files are what you
should use in that .ins file. Hey it's just a guess, but it's
worth a try.

[Frans Pop]

Post by Stephen Powell
It sounds to me like you are asking for information on how to create a
mainframe-bootable CD. I'm afraid I don't know the answer.

That's a pity. Because that is *exactly* the information we need. And I'm
very surprised that nobody in the community is able (or willing) to
provide that info.

Post by Stephen Powell
If you have a specific .iso image that you want me to download and try to
see if it will boot, I can do that for you. But I don't pretend
to know how to create one.

I can do that myself on Hercules (using the method described in the README
file on current CD images).

Post by Stephen Powell
I can see, though, from the link above, that you are using the
"generic" sub-architecture files. Have you tried using the "tape"
files? The reason I ask is that, on the Multiprise 3000, the CD-ROM
could be used as an emulated tape or via the "Load from CD-ROM or
Server" method. I just wonder if the tape files are what you
should use in that .ins file. Hey it's just a guess, but it's
worth a try.

I doubt it will make a difference as the problem is not in the kernel, but
in loading the initrd.

But maybe I'm wrong and the method you describe really is different from
what I can do in hercules. If you think it may help, please give it a
shot. It should be trivial to create a CD image yourself as all you need
is the D-I images and boot files. You don't need any packages on the CD to
do an installation on s390.

Besides, you have the hardware, I don't. So you can play around with
different options, I can't.

Cheers,
FJP

[Adam Thornton]

Post by Frans Pop

Post by Stephen Powell
It sounds to me like you are asking for information on how to
create a
mainframe-bootable CD. I'm afraid I don't know the answer.

That's a pity. Because that is *exactly* the information we need. And I'm
very surprised that nobody in the community is able (or willing) to
provide that info.

Post by Stephen Powell
If you have a specific .iso image that you want me to download and try to
see if it will boot, I can do that for you. But I don't pretend
to know how to create one.

I can do that myself on Hercules (using the method described in the README
file on current CD images).

Post by Stephen Powell
I can see, though, from the link above, that you are using the
"generic" sub-architecture files. Have you tried using the "tape"
files? The reason I ask is that, on the Multiprise 3000, the CD-ROM
could be used as an emulated tape or via the "Load from CD-ROM or
Server" method. I just wonder if the tape files are what you
should use in that .ins file. Hey it's just a guess, but it's
worth a try.

I doubt it will make a difference as the problem is not in the
kernel, but
in loading the initrd.
But maybe I'm wrong and the method you describe really is different from
what I can do in hercules. If you think it may help, please give it a
shot. It should be trivial to create a CD image yourself as all you need
is the D-I images and boot files. You don't need any packages on the CD to
do an installation on s390.
Besides, you have the hardware, I don't. So you can play around with
different options, I can't.

Does the .ins file specify load addresses? The short version I would
suggest is "see how SuSE does it" and do that.

I don't have access to s390(x) hardware anymore either, I'm afraid.

Adam

[Frans Pop]

Post by Adam Thornton
Does the .ins file specify load addresses?

Yes, it does.
See: http://lists.debian.org/debian-s390/2009/09/msg00027.html

Post by Adam Thornton
The short version I would suggest is "see how SuSE does it" and do that.

Right, so who has a *recent* SuSE CD he/she can disect? I personally don't
feel like registering with Novell just for this.

Main questions are:
- what does the .ins file they use look like?
- what kind of initrd do they use: initramfs or "oldfashioned" initrd?
- if oldfashioned, then what filesystem do they use?
- if initramfs, then how do they manage to boot it?
- what is the kernel config they use in their installer?

I've had CD boot working with an ext2 initrd and I suspect that would still
work (and testing that would be fairly straightforward). But the Debian
kernels have all moved away from that. So we really want to find out how
to boot from CD using an initramfs initrd.

If we can't I'd have to try to convince Bastian to compile ext2 into the
s390 kernels again.

Cheers,
FJP

[Adam Thornton]

Post by Frans Pop

Post by Adam Thornton
Does the .ins file specify load addresses?

Yes, it does.
See: http://lists.debian.org/debian-s390/2009/09/msg00027.html

Post by Adam Thornton
The short version I would suggest is "see how SuSE does it" and do that.

Right, so who has a *recent* SuSE CD he/she can disect? I personally don't
feel like registering with Novell just for this.
- what does the .ins file they use look like?
- what kind of initrd do they use: initramfs or "oldfashioned" initrd?
- if oldfashioned, then what filesystem do they use?
- if initramfs, then how do they manage to boot it?
- what is the kernel config they use in their installer?
I've had CD boot working with an ext2 initrd and I suspect that would still
work (and testing that would be fairly straightforward). But the Debian
kernels have all moved away from that. So we really want to find out how
to boot from CD using an initramfs initrd.
If we can't I'd have to try to convince Bastian to compile ext2 into the
s390 kernels again.

I have a recentish (SLES 10, not SLES 11) SuSE image I can get to
fairly easily, I think. I think their initrds were old-style, though.

I'll see what I can do with the rest of the weekend.

Adam

[Bastian Blank]

| #ifndef __s390x__
| #define IPL_DEVICE (*(unsigned long *) (0x10404))
| #define INITRD_START (*(unsigned long *) (0x1040C))
| #define INITRD_SIZE (*(unsigned long *) (0x10414))
| #else /* __s390x__ */
| #define IPL_DEVICE (*(unsigned long *) (0x10400))
| #define INITRD_START (*(unsigned long *) (0x10408))
| #define INITRD_SIZE (*(unsigned long *) (0x10410))
| #endif /* __s390x__ */
| #define COMMAND_LINE ((char *) (0x10480))
So the addresses for the initrd differs between the 31 and 64bit kernel.

Flawed observation. This way you can load the 32bit long addresses
always at the same location, regardless of the type.

Bastian

[Frans Pop]

Post by Frans Pop
Right, so who has a *recent* SuSE CD he/she can disect? I personally
don't feel like registering with Novell just for this.

Well, I have an account and got an image.

Thanks a lot Bastian.
I received basically the same info in a private mail from Adam (private
probably because he forwarded it from a contact at SuSE). So thanks to
them too.

And with the help of google to fill in the missing bits I now have it
working in Hercules again.

Attached the patch for debian-cd (for squeeze). I've tested it for both
Lenny and Squeeze (the latter using current daily D-I images), so the same
changes work for both 31-bit and 64-bit kernels.

Post by Frans Pop
- what does the .ins file they use look like?

| * SuSE Linux for zSeries Installation/Rescue System
| boot/s390x/vmrdr.ikr 0x00000000
| boot/s390x/initrd.off 0x0001040c
| boot/s390x/initrd.siz 0x00010414
| boot/s390x/initrd 0x00800000
| boot/s390x/parmfile 0x00010480

We did not yet specify initrd offset and size.

Post by Frans Pop
- what kind of initrd do they use: initramfs or "oldfashioned" initrd?
- if oldfashioned, then what filesystem do they use?
- if initramfs, then how do they manage to boot it?

This are just memory loads, so this should work regardless of the type.
The kernel handles both types in the same way.

Well, it worked without the size and offset for ext2 initrds. And at least
that defines block you quoted has been unchanged since the start of git
(2.6.12). Maybe there has been a change in the kernel, or maybe the
autodetection differs between initramfs and a "real" file system.

Post by Frans Pop
- what is the kernel config they use in their installer?

The VM reader kernel.

Same as us then.


So with this weekly built CDs should work again very soon if you boot using
d390.ins. And Lenny CDs should work again with the next point release.

Note that I have not modified d390.tdf, which I assume is tape emulation.
I have no idea if that is supposed to be possible from CD. If someone knows
and wants to test that, please do.

Cheers,
FJP

diff --git a/data/squeeze/s390/d390.ins b/data/squeeze/s390/d390.ins
index 5e82599..78c5f11 100644
--- a/data/squeeze/s390/d390.ins
+++ b/data/squeeze/s390/d390.ins
@@ -1,4 +1,6 @@
* Debian GNU/Linux for S/390 (boot from CD-ROM or FTP-Server)
linux_vm 0x00000000
+root.off 0x0001040c
+root.siz 0x00010414
parmfile 0x00010480
root.bin 0x00800000
diff --git a/data/squeeze/s390/d390oco.ins b/data/squeeze/s390/d390oco.ins
index 03da8f6..b3f11dd 100644
--- a/data/squeeze/s390/d390oco.ins
+++ b/data/squeeze/s390/d390oco.ins
@@ -1,5 +1,7 @@
* Debian GNU/Linux for S/390 (boot from CD-ROM or FTP-Server with
OCO-Modules)
linux_vm 0x00000000
+root.off 0x0001040c
+root.siz 0x00010414
parmfile 0x00010480
root.bin 0x00800000
oco.bin 0x00c00000
diff --git a/tools/boot/squeeze/boot-s390 b/tools/boot/squeeze/boot-s390
index 6b3796f..34b8351 100755
--- a/tools/boot/squeeze/boot-s390
+++ b/tools/boot/squeeze/boot-s390
@@ -87,6 +87,10 @@ done
# - d390oco.tdf : same, using object-code-only-modules-ramdisk (example)
cp $BASEDIR/data/$CODENAME/s390/d390* "$imagedir/"

+# Create the files specifying offset and size of the initrd
+perl -e "print pack('N', 0x800000)" >"$imagedir/root.off"
+perl -e "print pack('N', -s '$imagedir/root.bin')" >"$imagedir/root.siz"
+
# Copy the README file
cp $BASEDIR/data/$CODENAME/s390/README.boot "boot$N/"

[Bastian Blank]

Post by Frans Pop
diff --git a/data/squeeze/s390/d390oco.ins b/data/squeeze/s390/d390oco.ins

This file can be removed.

If I can find the repo, I will do some cleanup in a few days.

Bastian

[Bastian Blank]

Post by Frans Pop
Right, so who has a *recent* SuSE CD he/she can disect? I personally don't
feel like registering with Novell just for this.

Well, I have an account and got an image.

Post by Frans Pop
- what does the .ins file they use look like?

| * SuSE Linux for zSeries Installation/Rescue System
| boot/s390x/vmrdr.ikr 0x00000000
| boot/s390x/initrd.off 0x0001040c
| boot/s390x/initrd.siz 0x00010414
| boot/s390x/initrd 0x00800000
| boot/s390x/parmfile 0x00010480

This looks like absolute addresses, so the contents of the initrd.off,
initrd.siz and parmfile files are written into the kernel image.

Post by Frans Pop
- what kind of initrd do they use: initramfs or "oldfashioned" initrd?
- if oldfashioned, then what filesystem do they use?
- if initramfs, then how do they manage to boot it?

This are just memory loads, so this should work regardless of the type.
The kernel handles both types in the same way.

Post by Frans Pop
- what is the kernel config they use in their installer?

The VM reader kernel.

More informations:

| $ l boot/s390x/{vmrdr.ikr,initrd*,parmfile*}
| -r--r--r-- 1 root root 12897335 8. MÀr 2009 boot/s390x/initrd
| -r--r--r-- 1 root root 4 8. MÀr 2009 boot/s390x/initrd.off
| -r--r--r-- 1 root root 4 8. MÀr 2009 boot/s390x/initrd.siz
| -r--r--r-- 1 root root 71 8. MÀr 2009 boot/s390x/parmfile
| -r--r--r-- 1 root root 102 8. MÀr 2009 boot/s390x/parmfile.cd
| -r--r--r-- 1 root root 6761472 8. MÀr 2009 boot/s390x/vmrdr.ikr
| $ hexdump -C boot/s390x/initrd.off
| 00000000 00 80 00 00 |....|
| 00000004
| $ hexdump -C boot/s390x/initrd.siz
| 00000000 00 c4 cc 37 |...7|
| 00000004
| $ cat boot/s390x/parmfile
| ramdisk_size=65536 root=/dev/ram1 ro init=/linuxrc TERM=dumb
| manual=1
|
| $ cat suse.ins
| * SuSE Linux for zSeries Installation/Rescue System
| boot/s390x/vmrdr.ikr 0x00000000
| boot/s390x/initrd.off 0x0001040c
| boot/s390x/initrd.siz 0x00010414
| boot/s390x/initrd 0x00800000
| boot/s390x/parmfile 0x00010480
| $ cat boot/s390x/suse.ins
| * SuSE Linux for zSeries Installation/Rescue System
| vmrdr.ikr 0x00000000
| initrd.off 0x0001040c
| initrd.siz 0x00010414
| initrd 0x00800000
| parmfile 0x00010480

The addresses are specified this way:
| #ifndef __s390x__
| #define IPL_DEVICE (*(unsigned long *) (0x10404))
| #define INITRD_START (*(unsigned long *) (0x1040C))
| #define INITRD_SIZE (*(unsigned long *) (0x10414))
| #else /* __s390x__ */
| #define IPL_DEVICE (*(unsigned long *) (0x10400))
| #define INITRD_START (*(unsigned long *) (0x10408))
| #define INITRD_SIZE (*(unsigned long *) (0x10410))
| #endif /* __s390x__ */
| #define COMMAND_LINE ((char *) (0x10480))

So the addresses for the initrd differs between the 31 and 64bit kernel.

Bastian

[Stephen Powell]

Post by Frans Pop
Thanks a lot Bastian.
I received basically the same info in a private mail from Adam (private
probably because he forwarded it from a contact at SuSE). So thanks to
them too.
And with the help of google to fill in the missing bits I now have it
working in Hercules again.

Well, I'm sorry I couldn't be of much help on this one. I'm afraid I'm
spoiled by z/VM. Since I have access to z/VM, I don't mess with LPARs much.
Perhaps my main contribution this time was to demonstrate that there are
people out there in the user community who care passionately about
the continued viability of the Debian s390 port. And perhaps that motivated
others who had the requisite knowledge to come forth. My next suggestion
was going to be "look at the install CD for another distribution and see
how they do it". But unfortunately I don't have access to such a CD myself.
And apparently neither did you. Fortunately, someone out there did.

:soapbox.
Documentation for IBM hardware really stinks these days. In the "old days",
when they were still operating under the consent decree, you could get
documentation for just about anything. But since the consent decree has been
canceled, they have been really tight with everything. For example, I can't
get documentation on the channel commands or sense bytes for 3590 tape control
units. I can't get documentation on the SERVC interface to the processor
controller (for things like I/O to the integrated system console, integrated
3270 console, obtaining the LOADPARM, etc.). And I'd be willing to bet that
you can't get documentation on the internals of "LOAD from a CD ROM or FTP
Server" either. All of that came out after the consent decree was canceled.
How ironic that the world's best server consolidation platform for open source
servers is so tight with its own documentation!
:esoapbox.

I might suggest that you modify the s390 install script to put in a plug
for this mailing list. I have been using Debian on s390 for years but only
joined this list last week. Perhaps there are others out there like me
who could be of service in some way if they were connected.

As I mentioned in an earlier post, I am working on a "how to" document for
getting the high-performance DASD DIAG driver to work for Debian on s390.
Perhaps that will be useful to someone. If I can be useful in some other
way, please let me know.

Regards,
Steve