Future of the s390 port

[Bastian Blank]

Hi folks

The s390 port was released with Lenny. However it is not in the best
condition. There are mainly two problems which needs attention, lack of
manpower and a 64 bit userland.

The first problem is the worst. Currently only Frans Pop and I do work
on it. Frans only does the Debian-Installer part and I simply have not
enough time to do the rest. The s390 architecture is quite different to
anything else, so it needs several specialized packages to work[1] and
they need lasting attention. So if anyone wants to help (especially
Debian developers) for the continuity of this port please speak up.

The second problem is not that critical. No other distribution still
supports a complete 31 bit s390 userland and even Debian dropped the 31
bit kernel support in the meantime[2]. Strategies for an upgrade to a 64
bit userland was discussed lately[3].

I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.

Bastian

[1]: Mainly the kernel, generic tools (s390-tools), hardware support
(sysconfig-hardware) and a whole bunch of debian-installer packages.
[2]: <***@wavehammer.waldi.eu.org>
[3]: <***@droopy.oc.cox.net>

[Marco d'Itri]

Post by Bastian Blank
I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.

Is this really an important problem?
Does a significant number of people actually use Debian/s390 on
production servers? And if they exist, why they are not helping?

[Michael Casadevall]

Post by Marco d'Itri

Post by Bastian Blank
I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.

Is this really an important problem?
Does a significant number of people actually use Debian/s390 on
production servers? And if they exist, why they are not helping?

I think a bigger question is where do you find hardware where you can
get remote root on; I'm not very familiar with s390 or mainframes in
general, but its not a piece of hardware one individual person would
own. I'm aware of the Hercules emulator, but that doesn't seem like it
would be useful for general development of a port.
Michael

Post by Marco d'Itri
--
ciao,
Marco
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkqb/koACgkQFGfw2OHuP7G4NACfSPn2hWvMsEU1DhfSCk0M9boh
vi0AnR2IzJJyChe76F4vORV79qdP/9hi
=QbZi
-----END PGP SIGNATURE-----

[Adam Thornton]

Post by Michael Casadevall
I think a bigger question is where do you find hardware where you can
get remote root on; I'm not very familiar with s390 or mainframes in
general, but its not a piece of hardware one individual person would
own. I'm aware of the Hercules emulator, but that doesn't seem like it
would be useful for general development of a port.

The Debian project has access to a couple of machines hosted by OSDL,
or at least it used to (I haven't actually checked the status
recently). These are older machines (z800?) but still 64-bit.

The death of Flex-ES and the lack of a P/390, Integrated Server, or
even H50/H70 equivalent for zSeries has left a hole in the market for
lower-end developers (not just in the Debian or even Linux space).
Hercules is actually pretty useful except in that it doesn't emulate a
lot of modern peripheral hardware, particularly the QDIO OSA
interfaces and the Fibre Channel interfaces which are a fact of life
on modern z boxes.

Adam

[The Fungi]

I've been meaning to set up a used representative for testing in my
lab at home, and would appreciate suggestions on some of the
"smaller" options (from a space and power/thermal perspective). I've
worked with a larger S/390 sysplex in the past (unfortunately not
running Linux natively), but am curious if there's anything spported
in the ballpark of an AS/400, size-wise. Preferably something which
will run on 110V/15A power...
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(***@yuggoth.org); IRC(***@irc.yuggoth.org#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(***@yuggoth.org);
MUD(***@katarsis.mudpy.org:6669); WWW(http://fungi.yuggoth.org/); }

[The Fungi]

A Multiprise H50 or H70 would fit your needs, maybe....it's about
that size, probably pretty cheap on the used market, could run
z/VM 4.4. It is, however, 31-bit only. There is nothing at all
64-bit yet that fits the bill.

How thorough is the 31-bit Linux support, and does it take the same
userspace binaries as 64-bit? (Feel free to RTFM me--didn't see what
I was looking for online, but then again I'm not entirely certain I
grok some of the platform specifics I was reading about either.)

All real z boxes are basically 2 rack cabinets' worth of stuff and
don't have internal disk.

[...]

Right--those are the same size as the ones I helped manage MVS and
USS on some years back. Definitely not something I have room or
electrical/cooling for, not to mention budget (think we paid around
US$5M for that sysplex at the time).
--
{ IRL(Jeremy_Stanley); PGP(9E8DFF2E4F5995F8FEADDC5829ABF7441FB84657);
SMTP(***@yuggoth.org); IRC(***@irc.yuggoth.org#ccl); ICQ(114362511);
AIM(dreadazathoth); YAHOO(crawlingchaoslabs); FINGER(***@yuggoth.org);
MUD(***@katarsis.mudpy.org:6669); WWW(http://fungi.yuggoth.org/); }

[Adam Thornton]

Post by The Fungi
I've been meaning to set up a used representative for testing in my
lab at home, and would appreciate suggestions on some of the
"smaller" options (from a space and power/thermal perspective). I've
worked with a larger S/390 sysplex in the past (unfortunately not
running Linux natively), but am curious if there's anything spported
in the ballpark of an AS/400, size-wise. Preferably something which
will run on 110V/15A power...

A Multiprise H50 or H70 would fit your needs, maybe....it's about that
size, probably pretty cheap on the used market, could run z/VM 4.4.
It is, however, 31-bit only. There is nothing at all 64-bit yet that
fits the bill. All real z boxes are basically 2 rack cabinets' worth
of stuff and don't have internal disk.

For a 64-bit box currently your only viable option in that footprint
is Hercules.

Adam

[Bastian Blank]

Post by Michael Casadevall
I think a bigger question is where do you find hardware where you can
get remote root on;

I know at least two posibilites
- IBM provides access for evaluation purposes and
- OSDL provides access for project work.

Bastian

[Frans Pop]

Post by Marco d'Itri

Post by Bastian Blank
I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.

Is this really an important problem?
Does a significant number of people actually use Debian/s390 on
production servers?

That's hard to say, but I've seen 4 or 5 separate reports from people
who've wanted to install stable on s390 systems in the past 2 months [1].
That was more than I'd expected for s390.

Cheers,
FJP

[1] We got the reports because the stable kernel for s390 was broken.

[Adam Thornton]

Post by Bastian Blank
The first problem is the worst. Currently only Frans Pop and I do work
on it. Frans only does the Debian-Installer part and I simply have not
enough time to do the rest. The s390 architecture is quite
different to
anything else, so it needs several specialized packages to work[1] and
they need lasting attention. So if anyone wants to help (especially
Debian developers) for the continuity of this port please speak up.

I'd like to help; my time has become much more limited than during
previous release cycles, though, and my access to modern zSeries
hardware has also become more limited.

I can test installation, but on nothing anywhere near a full
complement of device types, and I can make recommendations and
amendments to a fair number of the sysconfig shell scripts and the
shell scripts in the d-i packages. And given the pressures of my day
job I can't really guarantee that I will be able to respond to any
specific item in a timely fashion.

I wish I could do more.

Adam

[Martin Grimm]

Post by Bastian Blank
Hi folks
The s390 port was released with Lenny. However it is not in the best
condition. There are mainly two problems which needs attention, lack of
manpower and a 64 bit userland.
The first problem is the worst. Currently only Frans Pop and I do work
on it. Frans only does the Debian-Installer part and I simply have not
enough time to do the rest. The s390 architecture is quite different to
anything else, so it needs several specialized packages to work[1] and
they need lasting attention. So if anyone wants to help (especially
Debian developers) for the continuity of this port please speak up.
The second problem is not that critical. No other distribution still
supports a complete 31 bit s390 userland and even Debian dropped the 31
bit kernel support in the meantime[2]. Strategies for an upgrade to a 64
bit userland was discussed lately[3].
I doubt that I would be able to push this port through another release
in the current state. The consequence would by that the port dies
completely and with it the only free and released distribution for this
machines.
Bastian
[1]: Mainly the kernel, generic tools (s390-tools), hardware support
(sysconfig-hardware) and a whole bunch of debian-installer packages.

Hi

we'd like to help.

We've currently running 240+ Linux guests on 2 IBM System z10 EC, that's
the newest hardware of this kind for those not so familiar with this
architecture. 236 of these are running Debian, mostly etch, some still
sarge and some lenny. Amongst these are zelenka.debian.org and a build
system for security updates.

We've no debian developers here and our staff is rather small but we are
willing to test new packages and contribute to existing tools in when
some debian developer can tell us where to start and what's necessary.

I've also clearance to provide more systems like zelenka to developers,
so access to current hardware should be possible.

Greetings
Martin
--
Martin Grimm
Zentrum für Informationsverarbeitung und Informationstechnik
Dienstsitz Bonn
An der Küppe 2
53225 Bonn
Tel.: [phone]
e-mail: ***@zivit.de

[Petter Reinholdtsen]

[Martin Grimm]

Post by Martin Grimm
We've currently running 240+ Linux guests on 2 IBM System z10 EC,
that's the newest hardware of this kind for those not so familiar
with this architecture. 236 of these are running Debian, mostly
etch, some still sarge and some lenny. Amongst these are
zelenka.debian.org and a build system for security updates.

One simple way to help a bit which will make it more visible to the
Debian community at large that the s390 port is used, is to install
and activate popularity-contest on these machines and thus make sure
236 machines show up on <URL: http://popcon.debian.org/ > as running
the s390 port. :)

At the moment, only 8 machines are reporting to popularity-contest,
which put s390 in line with hurd-i386, kfreebsd-amd64 and m68k as
ports with very small user groups. That number made me and probably
others believe that very few are using the s390 port - at least until
your email came along. :)

Happy hacking,

[Martin Grimm]

Post by Petter Reinholdtsen
[Martin Grimm]

Post by Martin Grimm
We've currently running 240+ Linux guests on 2 IBM System z10 EC,
that's the newest hardware of this kind for those not so familiar
with this architecture. 236 of these are running Debian, mostly
etch, some still sarge and some lenny. Amongst these are
zelenka.debian.org and a build system for security updates.

One simple way to help a bit which will make it more visible to the
Debian community at large that the s390 port is used, is to install
and activate popularity-contest on these machines and thus make sure
236 machines show up on <URL: http://popcon.debian.org/ > as running
the s390 port. :)
At the moment, only 8 machines are reporting to popularity-contest,
which put s390 in line with hurd-i386, kfreebsd-amd64 and m68k as
ports with very small user groups. That number made me and probably
others believe that very few are using the s390 port - at least until
your email came along. :)
Happy hacking,

I'm aware of popcon and as much as I'd appreciate it to see our systems
counted there this will not happen because these are mainly production
systems behind firewalls or in internal networks with no internet access
and I've generally a bad feeling when thinking of software that's
talking to outside systems when there is sensitive data on my server ;-)

So as long as there is no easy manual way to provide anonymized figures
without installing software on our production servers we can't deliver
such data :-( and yes, I know there are many reasons why a manual upload
form would be a bad idea regarding accuracy and actuality.

Greetings,
Martin
--
Martin Grimm
Zentrum für Informationsverarbeitung und Informationstechnik
Dienstsitz Bonn
An der Küppe 2
53225 Bonn
Tel.: [phone]
e-mail: ***@zivit.de

[Bastian Blank]

Post by Martin Grimm
So as long as there is no easy manual way to provide anonymized figures
without installing software on our production servers we can't deliver
such data :-(

Hmm. You could collect the /var/lib/dpkg/status files and do a
mass submit with the data out of this files. It would lack the usage
data, but at least shows something.

Okay, this also depends on the condition that you don't consider the
package names themself sensitive.

Bastian

[Mike Hommey]

Post by Bastian Blank

Post by Martin Grimm
So as long as there is no easy manual way to provide anonymized figures
without installing software on our production servers we can't deliver
such data :-(

Hmm. You could collect the /var/lib/dpkg/status files and do a
mass submit with the data out of this files. It would lack the usage
data, but at least shows something.
Okay, this also depends on the condition that you don't consider the
package names themself sensitive.

That could surely grant a wishlist bug for popcon, to allow to, say, only
report package names that are found in the debian archive.

Mike

[Russ Allbery]

Post by Martin Grimm
I'm aware of popcon and as much as I'd appreciate it to see our systems
counted there this will not happen because these are mainly production
systems behind firewalls or in internal networks with no internet access
and I've generally a bad feeling when thinking of software that's
talking to outside systems when there is sensitive data on my server ;-)

We're in a similar situation, although not with s390. In specific, our
information security office (rightfully) considers the relationship
between system and list of installed packages to be confidential data
because of the potential use of such data in determining which systems to
attack following a publicly announced vulnerability. I can therefore only
report popcon results for a handful of personal and test systems, rather
than ~300 production servers.

Is there an easy way (read: the software already exists and I can just
install it) for all of the systems to report to an internal proxy that
then resubmits the data so that no one else can know where it's coming
from exactly other than from our servers somewhere?

[Petter Reinholdtsen]

[Russ Allbery]

Post by Russ Allbery
Is there an easy way (read: the software already exists and I can
just install it) for all of the systems to report to an internal
proxy that then resubmits the data so that no one else can know
where it's coming from exactly other than from our servers
somewhere?

I expect you will get this if you use HTTP to submit and any HTTP
proxy specified using the HTTP_PROXY variable in
/etc/popularity-contest.conf. The only identity submitted would be
the random ID generated by popcon to make sure the weekly resubmission
of information replaces the last one. Or one could use a SMTP
remailer stripping headers, I guess. But that seem to be more work
than just using a HTTP proxy.

The information submitted is generated and available in
/var/log/popularity-contest for those that want to have a look. There
is no information recorded about the source, except for the unique ID
of the host which is generated using

dd if=/dev/urandom bs=1k count=1 2>/dev/null | md5sum

when the package is installed.

Happy hacking,

[Christian Perrier]

Post by Petter Reinholdtsen
I expect you will get this if you use HTTP to submit and any HTTP
proxy specified using the HTTP_PROXY variable in
/etc/popularity-contest.conf. The only identity submitted would be
the random ID generated by popcon to make sure the weekly resubmission
of information replaces the last one. Or one could use a SMTP
remailer stripping headers, I guess. But that seem to be more work
than just using a HTTP proxy.

FWIW, I have personnally always considered this as way enough to
guarantee the level of "anonymization" mentioned by Russ.

Enough to have all "my" production Debian servers at ONERA
(www.onera.fr) to report by popcon through our HTTP proxy. ONERA is a
governmental research agency which activity is focused on both
Defense-oriented research and industry-related research in aeronautics
and space. You probably get the drill: we *are* concerned about
security and confidentiality and our servers do report to popcon.

I would of course be much more reluctant to do this for servers
located directly on the Internet but we have (sigh) no Debian servers
there. We create our own security holes by using Solaris
everywhere..:-)

[Samuel Thibault]

In specific, our information security office (rightfully) considers
the relationship between system and list of installed packages to
be confidential data because of the potential use of such data in
determining which systems to attack following a publicly announced
vulnerability.

Could perhaps an almost empty popcon report be considered as valid at
least just for the number of installed systems?

Samuel

[Adam Thornton]

Post by Martin Grimm
I'm aware of popcon and as much as I'd appreciate it to see our systems
counted there this will not happen because these are mainly production
systems behind firewalls or in internal networks with no internet access
and I've generally a bad feeling when thinking of software that's
talking to outside systems when there is sensitive data on my
server ;-)

I'm running about 20 Debian guests on z, but like Martin's, mine
cannot actually get to the outside world directly, and that is not
going to change.

Apt-proxy is a wonderful thing, though.

Adam