Document all Testsuite Issues

[Jim Gifford]

We need to document all Testsuite issues and place them either in the Wiki or some other page where they are accessable

[chris@…]

Glibc 2.4 has a failure in one of the nptl tests (tst-cancel24), reporting that it can't find libstdc++.so.6. Creating a /usr/lib/libstdc++.so.6 -> /tools/lib symlink seems to make it work.

[chris@…]

Glibc 2.4 has an expected (ignored) failure in posix/annexc. Hmm, I'm surprised it isn't mentioned in the book...I'm doing a Google search on that error and seeing reports of the "ignored" failure as far back as glibc 2.2.3 in 2001.

[ken]

I think there is still a bit of work to pin down some of the glibc test failures on architectures other than x86. Meanwhile, I'm noticing a degree of variability/unpredictability about which tests fail. I suspect that bash, at least on 64-bit machines, might still be the cause of some glibc test failures. Other than glibc and occasional failures in gcc, I think everything else should pass all its tests.

The book documents known glibc failure problems, and I think tests that sometimes fail can probably be mentioned at the end of this on an arch-specific basis. However, I'm unconvinced about keeping the text about math tests ('at least on i686') in for all architectures, and similarly I don't think we should be referring to gettext issues from the host, those surely are relevant to LFS unless we are more closely coupled to the host than I thought ?

[ken]

Now that the big failures in multilib glibc are mostly under control, detailed error messages in the log are uncommon (things like forgetting the libstdc++ symlink). Should we mention that any messages from failing tests will be in the associated '.out' file ? Also, we mention the sort of errors that can be encountered, but we aren't logging the tests - makes it kind of difficult for people to see which tests failed if some wre several hundred lines earlier.

[ken]

To clarify the comment that we aren't logging the results - it only applies to the 64-bit versions in multilib/ and ppc64/ so it's probably an accidental difference. I'll fix that up unless anybody objects.

[ken]

Actually the logging is for everything that doesn't use plain glibc.xml. See r2361.

Will do separate changes to mention posix/annexc, to comment on the .out files, math tests, gettext. For the moment I'm not touching the "these sometimes fail" tests, I suspect they maybe only fail when the build is broken.

[ken]

Moved posix/annexc out of the math tests paragraph, and added more explanation of where to look for failures in r2362.

[ken]

Missed corresponding text in ppc64, pointed to multilib master text in r2363.

Comment on gettext test removed in r2364.

[ken]

Moved the i686 references into x86/glibc.xml and added a couple of missing paragraphs about the tests to pure64 books, r2365.

With the exception of occasional arch-specific failures in the gcc tests, I'm hoping that all the test failures have now been nailed.

[Jim Gifford]

It appears this ticket will always remain open in some fashion. Most of us building on different systems and even the same systems find testsuites failures different on every build. I think we need to address them on a one on one basis.

The Glibc testsuites are the main culprits on this issue, so we need to handle them on a one of basis.

Binutils, GCC, and GLIBC testsuites are also going to show different results for each architecutre. We can not know what each architecture is going to do, in PPC, MIPS, and Sparc there are various different subarchitectures which can show different results. My feeling here is to tell everyone who thinks they have a problem to continue on, until something is visiibly broken.

All the program testsuites have shown to be exactly the same on all architectures, so a failure here, needs to be investigated if one should arise.

We need to track errors that are posted in the wiki, so we don't get duplicate answers.

[Jim Gifford]

Testsuite issues are going to be different for all builds, we will note them in the mailing list as them come in.