Bug#673590: libxml-libxml-perl: FTBFS on s390x: Failed 1/51 test programs. 0/2419 subtests failed.

[Cyril Brulebois]

Source: libxml-libxml-perl
Version: 1.98+dfsg-1
Severity: important

Hi,

your package FTBFS on s390x, in the test suite:
| Test Summary Report
| -------------------
| t/12html.t (Wstat: 256 Tests: 41 Failed: 0)
| Non-zero exit status: 1
| Parse errors: Bad plan. You planned 43 tests but ran 41.
| Files=51, Tests=2419, 9 wallclock secs ( 0.50 usr 0.11 sys + 4.82 cusr 0.64 csys = 6.07 CPU)
| Result: FAIL
| Failed 1/51 test programs. 0/2419 subtests failed.
| make[2]: *** [test_dynamic] Error 255

Full build logs:
https://buildd.debian.org/status/package.php?p=libxml-libxml-perl&suite=sid

I noticed this FTBFS because it is responsible for the uninstallability
of the following packages in testing, on the s390x architecture:
libxml-atom-fromowl-perl
libnet-amazon-s3-tools-perl
xmltv
torrus-common
jirc
ripit
libhtml-html5-parser-perl
libxml-simpleobject-libxml-perl
xmltv-util
movabletype-opensource
lemonldap-ng
libxml-atom-service-perl
librdf-microturtle-perl
libcgi-xmlapplication-perl
libgeo-googleearth-pluggable-perl
[…]

I won't list them all, too many of them. ;)

I'm putting the s390 list in X-Debbugs-Cc, maybe somebody will know.

Mraw,
KiBi.

[Niko Tyni]

Post by Cyril Brulebois
Source: libxml-libxml-perl
Version: 1.98+dfsg-1
Severity: important
| Test Summary Report
| -------------------
| t/12html.t (Wstat: 256 Tests: 41 Failed: 0)
| Non-zero exit status: 1
| Parse errors: Bad plan. You planned 43 tests but ran 41.
| Files=51, Tests=2419, 9 wallclock secs ( 0.50 usr 0.11 sys + 4.82 cusr 0.64 csys = 6.07 CPU)
| Result: FAIL
| Failed 1/51 test programs. 0/2419 subtests failed.
| make[2]: *** [test_dynamic] Error 255
https://buildd.debian.org/status/package.php?p=libxml-libxml-perl&suite=sid

FWIW, the test is dying with 'Out of memory!':

t/10ns.t ............................ ok
t/11memory.t ........................ skipped: developers only (set MEMORY_TEST=1 to run these tests)
Out of memory!
# Looks like you planned 43 tests but ran 41.
# Looks like your test exited with 1 just after 41.
t/12html.t ..........................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 2/43 subtests
t/13dtd.t ........................... ok

Post by Cyril Brulebois
I'm putting the s390 list in X-Debbugs-Cc, maybe somebody will know.

It happens on ppc64 and sparc64 as well, as seen at
buildd.debian-ports.org. I suppose that means it's broken on all big
endian 64-bit platforms.

[Niko Tyni]

Post by Niko Tyni

Post by Cyril Brulebois
Source: libxml-libxml-perl
Version: 1.98+dfsg-1
Severity: important
https://buildd.debian.org/status/package.php?p=libxml-libxml-perl&suite=sid

Out of memory!
# Looks like you planned 43 tests but ran 41.
# Looks like your test exited with 1 just after 41.
t/12html.t ..........................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 2/43 subtests
It happens on ppc64 and sparc64 as well, as seen at
buildd.debian-ports.org. I suppose that means it's broken on all big
endian 64-bit platforms.

It's a pointer cast in XML::LibXML::Document::toStringHTML, in LibXML.xs
around line 2939.

STRLEN len = 0;
[...]
htmlDocDumpMemory(self, &result, (int*)&len);
[...]
RETVAL = newSVpvn((char *)result, (STRLEN)len);

(STRLEN is defined as a size_t via /usr/lib/perl/5.14/CORE/perl.h)

See the attached patch, which just makes 'len' an int and removes the
problematic pointer cast. I wonder if the STRLEN cast on becomes an
issue, though. Is it possible that an int doesn't fit into a size_t
variable somewhere?

[Philipp Kern]

Niko,

Post by Niko Tyni
It's a pointer cast in XML::LibXML::Document::toStringHTML, in LibXML.xs
around line 2939.
STRLEN len = 0;
[...]
htmlDocDumpMemory(self, &result, (int*)&len);
[...]
RETVAL = newSVpvn((char *)result, (STRLEN)len);
(STRLEN is defined as a size_t via /usr/lib/perl/5.14/CORE/perl.h)
See the attached patch, which just makes 'len' an int and removes the
problematic pointer cast. I wonder if the STRLEN cast on becomes an
issue, though. Is it possible that an int doesn't fit into a size_t
variable somewhere?

you cannot reinterpret a size_t as an int. size_t might be unsigned, might
have another length, etc. On 64bit big endian you fill the top bits and
leave the lower ones untouched, because size_t is 64bit. So yeah, that code
was broken. (nbd had something similar, glib too. I don't know why it only
turns up with 64bit big endian.)

Kind regards
Philipp Kern

[Bastian Blank]

Post by Philipp Kern
you cannot reinterpret a size_t as an int. size_t might be unsigned, might
have another length, etc. On 64bit big endian you fill the top bits and
leave the lower ones untouched, because size_t is 64bit. So yeah, that code
was broken. (nbd had something similar, glib too. I don't know why it only
turns up with 64bit big endian.)

Because on little endian 64bit, it touches the lower bytes.

Bastian

[Philipp Kern]

Post by Bastian Blank

Post by Philipp Kern
you cannot reinterpret a size_t as an int. size_t might be unsigned, might
have another length, etc. On 64bit big endian you fill the top bits and
leave the lower ones untouched, because size_t is 64bit. So yeah, that code
was broken. (nbd had something similar, glib too. I don't know why it only
turns up with 64bit big endian.)

Because on little endian 64bit, it touches the lower bytes.

Yeah, but that assumes that the other ones are 0? Or ok, maybe it's just
working when going from size_t to int, because it's truncation, but it
shouldn't work for the other way round?

Kind regards
Philipp Kern

[Cyril Brulebois]

Hi Niko,

and thanks for the quick follow-up.

Post by Niko Tyni
See the attached patch, which just makes 'len' an int and removes the
problematic pointer cast. I wonder if the STRLEN cast on becomes an
issue, though. Is it possible that an int doesn't fit into a size_t
variable somewhere?

If you were asking for a test-build, the answer is: no more FTBFS on s390x.

On the int/size_t thingy, quoting [1]:
| ISO/IEC 9899:1990, Programming Languages - C (ISO C) left the definition
| of the short int, the int, the long int, and the pointer deliberately
| vague to avoid artificially constraining hardware architectures that
| might benefit from defining these data types independent from the other.
| The only constraints were that ints must be no smaller than shorts, and
| longs must be no smaller than ints, and size_t must represent the
| largest unsigned type supported by an implementation. It is possible,
| for instance, to define a short as 16 bits, an int as 32 bits, a long as
| 64 bits and a pointer as 128 bits. The relationship between the
| fundamental data types can be expressed as:
|
| sizeof(char) <= sizeof(short) <= sizeof(int) <= sizeof(long) = sizeof(size_t)

1. http://www.unix.org/whitepapers/64bit.html

Thanks for your time.

Mraw,
KiBi.