From diy-linux-dev@diy-linux.org Thu Sep 6 08:32:38 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Thu, 6 Sep 2007 18:32:38 +1000 Subject: util-linux-ng-2.13 In-Reply-To: <20070828012751.GA5557@eyo32.local> References: <20070828012751.GA5557@eyo32.local> Message-ID: <20070906083238.GA15142@eyo32.local> On Tue, Aug 28, 2007 at 11:27:51AM +1000, Greg Schafer wrote: > This new version insists upon linking against blkid (e2fsprogs) or > volume_id (udev) so we may have to add either one of those (or just the > libs thereof) to the temptools phase. Alternatively, it might be possible > to hack around the requiremnt Hacking around it is not easy. I'm just going to add e2fsprogs (only the static libs, no shared) to the Temptools phase with `make libs; make install-libs' Some other things: - I've added `--enable-arch'. arch(1) is pretty useless but some old scripts might need it. `config.guess' doesn't need it but will use it if available. Apparently it's moving to coreutils anyway so we'll drop this switch when upgrading to next coreutils. - I've added `--enable-write' to maintain compat with the older release. Don't really know if anyone uses this kind of thing anymore. - I've added `--disable-wall' because wall is already included in sysvinit. - `elvtune' is no longer installed. It's useless according to DEPRECATED. - `rdev' and related crud is no longer installed. Useless. - `raw' is no longer installed. Useless. - `setfdprm' is gone completely. - some new stuff is installed, including `setarch' which is great for faking i386 on x86_64 - i18n is probably broken in this release due to a screwup with LOCALEDIR, but I won't let that hold up installing it to the Refbuild. Regards Greg From diy-linux-dev@diy-linux.org Tue Sep 11 01:25:19 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Mon, 10 Sep 2007 19:25:19 -0600 Subject: Buffer overflows with newest toolchain Message-ID: <200709101925.19479.alf@mypals.org> I've been playing around with the latest toolchain combo (Glibc 2.6.1, GCC 4.2.1, and Binutils 2.18). Unfortunately, not everything works. I haven't done much research into this, except rebuilt the system a few times to see if the problem could be duplicated. As an introduction, I have an Athlon-XP system. My "standard" CFLAGS are as follows: -Os -march=athlon-xp -pipe -fstack-protector -D_FORTIFY_SOURCE=2 \ -Wl,-O1,--as-needed,--hash-style=both For Glibc, I use: -O3 -march=athlon-xp -pipe -Wl,-O1,--as-needed,--hash-style=both For GCC I use: -O2 -march=athlon-xp -pipe -Wl,-O1,--as-needed,--hash-style=both For Binutils I use: -Os -march=athlon-xp -pipe -fstack-protector \ -Wl,-O1,--as-needed,--hash-style=both As you can see, there is a lot that can go wrong with those flags. However, those flags worked well with the previous incarnations of GCC 4.x. I've had a bit of problems with -Os in the past but things have been built. Anyway, the first problem I noticed was during the build of m4. During ./configure, conftest was aborted due to a buffer overflow (caught by -fstack-protector). The dump is as follows: *** buffer overflow detected ***: ./conftest terminated ======= Backtrace: ========= /lib/libc.so.6(__chk_fail+0x41)[0xb7effb81] /lib/libc.so.6[0xb7eff3c8] /lib/libc.so.6(_IO_default_xsputn+0xae)[0xb7e7f45e] /lib/libc.so.6(__printf_fp+0x100e)[0xb7e5ca5e] /lib/libc.so.6(_IO_vfprintf+0x3c4)[0xb7e57364] /lib/libc.so.6(__vsprintf_chk+0xab)[0xb7eff47b] /lib/libc.so.6(__sprintf_chk+0x30)[0xb7eff3b0] ./conftest[0x8048acd] /lib/libc.so.6(__libc_start_main+0xe0)[0xb7e2ff80] ./conftest[0x80483b1] ======= Memory map: ======== 08048000-08049000 r-xp 00000000 08:06 15004 /build/m4/source /m4-1.4.10/conftest 08049000-0804a000 rw-p 00001000 08:06 15004 /build/m4/source /m4-1.4.10/conftest 0804a000-0806b000 rw-p 0804a000 00:00 0 [heap] b7e0e000-b7e18000 r-xp 00000000 08:01 25165953 /usr/lib/libgcc_s.so.1 b7e19000-b7e1a000 rw-p b7e19000 00:00 0 b7e1a000-b7f59000 r-xp 00000000 08:01 12584695 /lib/libc-2.6.1.so b7f59000-b7f5b000 r--p 0013f000 08:01 12584695 /lib/libc-2.6.1.so b7f5b000-b7f5c000 rw-p 00141000 08:01 12584695 /lib/libc-2.6.1.so b7f5c000-b7f60000 rw-p b7f5c000 00:00 0 b7f63000-b7f7f000 r-xp 00000000 08:01 12584688 /lib/ld-2.6.1.so b7f7f000-b7f80000 r--p 0001b000 08:01 12584688 /lib/ld-2.6.1.so b7f80000-b7f81000 rw-p 0001c000 08:01 12584688 /lib/ld-2.6.1.so bfb02000-bfb18000 rw-p bfb02000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] The program seem to build fine. However, when trying to use vi, another buffer overflow occurred: *** buffer overflow detected ***: vi terminated ======= Backtrace: ========= /lib/libc.so.6(__chk_fail+0x41)[0xb7f25b81] /lib/libc.so.6(__strcpy_chk+0x43)[0xb7f25053] vi[0x8066f8c] vi[0x80691e1] vi[0x806c1d4] vi[0x8083309] vi[0x808e50e] vi[0x80a0e0d] /lib/libc.so.6(__libc_start_main+0xe0)[0xb7e55f80] vi[0x804a101] ======= Memory map: ======== 08048000-0812f000 r-xp 00000000 08:01 21052006 /usr/bin/vim 0812f000-08135000 rw-p 000e7000 08:01 21052006 /usr/bin/vim 08135000-081a0000 rw-p 08135000 00:00 0 [heap] b7e24000-b7e2e000 r-xp 00000000 08:01 25165953 /usr/lib/libgcc_s.so.1 b7e1f000-b7e20000 rw-p 00009000 08:01 25165953 /usr/lib/libgcc_s.so.1 b7e20000-b7e29000 r-xp 00000000 08:01 12595878 /lib/libnss_files-2.6.1.so b7e29000-b7e2a000 r--p 00008000 08:01 12595878 /lib/libnss_files-2.6.1.so b7e2a000-b7e2b000 rw-p 00009000 08:01 12595878 /lib/libnss_files-2.6.1.so b7e2b000-b7e2d000 rw-p b7e2b000 00:00 0 b7e2d000-b7e30000 r-xp 00000000 08:01 12623565 /lib/libattr.so.1.1.0 b7e30000-b7e31000 rw-p 00002000 08:01 12623565 /lib/libattr.so.1.1.0 b7e31000-b7f70000 r-xp 00000000 08:01 12584695 /lib/libc-2.6.1.so b7f70000-b7f72000 r--p 0013f000 08:01 12584695 /lib/libc-2.6.1.so b7f72000-b7f73000 rw-p 00141000 08:01 12584695 /lib/libc-2.6.1.so b7f73000-b7f76000 rw-p I haven't Googled this much. I was hoping someone here would immediately jump up and say "your problem lies in . . . . " (please fill in the blank). I also wanted to pass the warning that this new toolchain may not be entirely stable (sorry, Greg if that bursts your bubble). Here are my thoughts. These warnings are being caused by the -fstack-protector flag. It is possible that GCC is miscompiling code. I am not sure if Glibc is to blame. I was thinking of trying out GCC 4.1.2, since that compiler worked well. Does anyone know if this is OK for Glibc 2.6.1? If GCC 4.1.2 works, then I would try to isolate GCC's fault (either in the -march flag or the -O? flag). If I could isolate the problem, then we could come up with a good bug report for the maintainers. FYI, I have the build logs for Glibc, GCC and Binutils in case someone wants them. (Yes, they contain errors. I haven't had time to fully analyze them yet.) Thanks in advance. -- @ - Alf From diy-linux-dev@diy-linux.org Tue Sep 11 01:49:00 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Mon, 10 Sep 2007 19:49:00 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709101925.19479.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> Message-ID: <200709101949.00797.alf@mypals.org> On Monday 10 September 2007 07:25:19 pm Alf mel wrote: > I was thinking of trying out GCC 4.1.2, since that compiler worked well. > Does anyone know if this is OK for Glibc 2.6.1? If GCC 4.1.2 works, then > I would try to isolate GCC's fault (either in the -march flag or the -O? > flag). If I could isolate the problem, then we could come up with a good > bug report for the maintainers. I almost forgot. If I run vi inside the chroot environment where it was built, it crashes. If I finish the system, reboot into the new system and try vi, it still crashes. But, if I run the new vi outside the chroot environment it works fine. Does that mean the problem could also lie in Glibc? -- @ - Alf From diy-linux-dev@diy-linux.org Wed Sep 12 14:48:06 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Wed, 12 Sep 2007 08:48:06 -0600 Subject: util-linux-ng-2.13 In-Reply-To: <46D3BB70.6050204@ums.usu.ru> References: <20070828012751.GA5557@eyo32.local> <46D3BB70.6050204@ums.usu.ru> Message-ID: <200709120848.06454.alf@mypals.org> On Tuesday 28 August 2007 12:06:40 am Alexander E. Patrakov wrote: > Is it possible to mount the virtual filesystems from outside the chroot, > thus completely bypassing the need for temporary "mount" program? Dumb question. From util-linux we currently use 2 programs in the tools phase: mount and getopt. I take it the e2fsprogs requirement only applies to mount and not getopt. Do we still need getopt as part of the toolchain? If we don't could util-linux be optional along with e2fsprogs in the tools phase? I can see where an old mount may not be kosher. However, I only build new systems from newer systems so my mount is probably safe. My scripts also mount things before going into chroot, which means I don't need mount at all. Can util-linux simply be taken out of the equation in such cases? -- @ - Alf From diy-linux-dev@diy-linux.org Thu Sep 13 00:13:58 2007 From: diy-linux-dev@diy-linux.org (Jeremy Huntwork) Date: Wed, 12 Sep 2007 20:13:58 -0400 Subject: util-linux-ng-2.13 In-Reply-To: <20070828065417.GA22514@eyo32.local> References: <20070828012751.GA5557@eyo32.local> <46D3BB70.6050204@ums.usu.ru> <20070828065417.GA22514@eyo32.local> Message-ID: <46E880C6.8010805@linuxfromscratch.org> Greg Schafer wrote: > It most certainly is, and that's exactly what current LFS does. But I do > recall some discussion from years ago where it was decided that using the > host's mount binary was considered not kosher. I searched around and couldn't find this. (Of course, I assumed the discussion took place on the LFS lists. I just realized this might be a bogus assumption.) I'm interested in the details as to why it's not 'kosher'. If you happen to come across it again, I'd be grateful for a link. -- JH From diy-linux-dev@diy-linux.org Thu Sep 13 14:20:22 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Thu, 13 Sep 2007 08:20:22 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709101925.19479.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> Message-ID: <200709130820.22989.alf@mypals.org> Another followup to my own post. On Monday 10 September 2007 07:25:19 pm Alf mel wrote: > Anyway, the first problem I noticed was during the build of m4. > During ./configure, conftest was aborted due to a buffer overflow (caught > by -fstack-protector). I rebuild the entire system using GCC 4.1.2 instead. M4 configure still overflowed at conftest. GCC may not be the culprit here. It might be the conftest program itself, or a bad combination of conftest and glibc. I'll check more on that. > when trying to use vi, another buffer overflow occurred: VI did not overflow this time around and seems to work just fine. So it seems GCC is partly to blame for the VI overflow. I will change my configure flags to -O2 -march=i686 and see how that affects the build. As always, any suggestions on how to isolate these problems would be greatly appreciated. I feel I'm way over my head. -- @ - Alf From diy-linux-dev@diy-linux.org Thu Sep 13 15:17:53 2007 From: diy-linux-dev@diy-linux.org (Jon Grosshart) Date: Thu, 13 Sep 2007 11:17:53 -0400 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709130820.22989.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> Message-ID: <9cf823c60709130817j1840ceafvfd5c74c6aad3f49b@mail.gmail.com> ------=_Part_31245_19651051.1189696673957 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On 9/13/07, Alf mel wrote: > > > > I will change my > configure flags to -O2 -march=i686 and see how that affects the build. > > As always, any suggestions on how to isolate these problems would be > greatly > appreciated. Your own suggestion is probably the best one your likely to hear. I doubt you'll run into problems using supported/safe CFLAGS. That was my thought from your first email anyway; CFLAGS usage. ------=_Part_31245_19651051.1189696673957 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline

On 9/13/07, Alf mel <alf@mypals.org> wrote:


I will change my
configure flags to -O2 -march=i686 and see how that affects the build.

As always, any suggestions on how to isolate these problems would be greatly
appreciated.


Your own suggestion is probably the best one your likely to hear. I doubt you'll run into problems using supported/safe CFLAGS. That was my thought from your first email anyway; CFLAGS usage.


------=_Part_31245_19651051.1189696673957-- From diy-linux-dev@diy-linux.org Thu Sep 13 15:26:04 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Thu, 13 Sep 2007 09:26:04 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <9cf823c60709130817j1840ceafvfd5c74c6aad3f49b@mail.gmail.com> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> <9cf823c60709130817j1840ceafvfd5c74c6aad3f49b@mail.gmail.com> Message-ID: <200709130926.05602.alf@mypals.org> On Thursday 13 September 2007 09:17:53 am Jon Grosshart wrote: > Your own suggestion is probably the best one your likely to hear. I doubt > you'll run into problems using supported/safe CFLAGS. That was my thought > from your first email anyway; CFLAGS usage. I admit my CFLAGS are a bit on the wild side, but I also think it is important to test them and get the wrinkles out. If people don't report problems with them, they will always remain buggy. Additionally, other than using --hash-style=both, I used those exact CFLAGS on many other occasions without any problems. In fact, both my desktop and laptop are running full KDE systems with those CFLAGS without any problems. The change to Glibc 2.6.1 and and GCC 4.2.1 is what is breaking things for me. Thanks for the tip. I hope to be able to figure out if the problem lies with -Os or with -march=athlon-xp. -- @ - Alf From diy-linux-dev@diy-linux.org Fri Sep 14 04:17:40 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Thu, 13 Sep 2007 22:17:40 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709130820.22989.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> Message-ID: <200709132217.41051.alf@mypals.org> On Thursday 13 September 2007 08:20:22 am Alf mel wrote: > M4 configure still > overflowed at conftest. GCC may not be the culprit here. It might be > the conftest program itself, or a bad combination of conftest and glibc. Well, I found the culprit for this problem. Here is a better snippet of where things go wrong: checking whether printf supports size specifiers as in C99... yes checking whether printf supports 'long double' arguments... yes checking whether printf supports infinite 'double' arguments... yes checking whether printf supports infinite 'long double' arguments... *** buffer overflow detected ***: ./conftest terminated ======= Backtrace: ========= /lib/libc.so.6(__chk_fail+0x41)[0xb7ed4191] /lib/libc.so.6[0xb7ed3998] /lib/libc.so.6(_IO_default_xsputn+0xb7)[0xb7e53d47] /lib/libc.so.6(__printf_fp+0x6a9)[0xb7e30559] /lib/libc.so.6(_IO_vfprintf+0x3c7)[0xb7e2b877] /lib/libc.so.6(__vsprintf_chk+0xad)[0xb7ed3a4d] /lib/libc.so.6(__sprintf_chk+0x30)[0xb7ed3980] ./conftest[0x8048d11] /lib/libc.so.6(__libc_start_main+0xe0)[0xb7e03f90] ./conftest[0x8048311] ======= Memory map: ======== 08048000-0804a000 r-xp 00000000 08:06 6295768 /var/local/slim/build/m4/source/m4-1.4.10/conftest 0804a000-0804b000 rw-p 00001000 08:06 6295768 /var/local/slim/build/m4/source/m4-1.4.10/conftest 0804b000-0806c000 rw-p 0804b000 00:00 0 [heap] b7de2000-b7dec000 r-xp 00000000 08:01 134 /usr/lib/libgcc_s.so.1 b7dec000-b7ded000 rw-p 00009000 08:01 134 /usr/lib/libgcc_s.so.1 b7ded000-b7dee000 rw-p b7ded000 00:00 0 b7dee000-b7f2d000 r-xp 00000000 08:01 20983511 /lib/libc-2.6.1.so b7f2d000-b7f2f000 r--p 0013f000 08:01 20983511 /lib/libc-2.6.1.so b7f2f000-b7f30000 rw-p 00141000 08:01 20983511 /lib/libc-2.6.1.so b7f30000-b7f34000 rw-p b7f30000 00:00 0 b7f37000-b7f53000 r-xp 00000000 08:01 20983504 /lib/ld-2.6.1.so b7f53000-b7f54000 r--p 0001b000 08:01 20983504 /lib/ld-2.6.1.so b7f54000-b7f55000 rw-p 0001c000 08:01 20983504 /lib/ld-2.6.1.so bfad6000-bfaec000 rw-p bfad6000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] no This error is triggered when using Glibc 2.6.1 with either GCC 4.1.2 or GCC 4.2.1. Using Glibc 2.5 and GCC 4.1.2 does not trigger this problem. The CFLAGS that trigger this problem are as follows: -O2 -fstack-protector -D_FORTIFY_SOURCE Of course, both stack protector and fortify source have to be enabled. However, just using -fstack-protector -D_FORTIFY_SOURCE does not trigger the error. Some sort of optimization (-O2, -Os, etc.) has to be enabled. Knowing there are plenty of people smarter than me on this one, would this be a GCC problem or a Glibc problem? I'm trying to isolate the code in the configure script to create a better bug report. But who should I tell? Also, could someone else try building a system with those flags and see if the problem is reproducible in another environment? -- @ - Alf From diy-linux-dev@diy-linux.org Fri Sep 14 14:16:52 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Fri, 14 Sep 2007 08:16:52 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709132217.41051.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> <200709132217.41051.alf@mypals.org> Message-ID: <200709140816.52953.alf@mypals.org> On Thursday 13 September 2007 10:17:40 pm Alf mel wrote: > This error is triggered when using Glibc 2.6.1 with either GCC 4.1.2 or > GCC 4.2.1. Using Glibc 2.5 and GCC 4.1.2 does not trigger this problem. > The CFLAGS that trigger this problem are as follows: > > -O2 -fstack-protector -D_FORTIFY_SOURCE VI doesn't crash anymore when compiled without -D_FORTIFY_SOURCE. I'll work more on isolating the piece of code that's causing the problems and find the real culprit. So far the suspects are: - Glibc 2.6.1 - GCC 4.1 or 4.2 - The packages themselves Although it would be easy to say that -D_FORTIFY_SOURCE is unstable, at the same time it is designed to protected against unknown buffer overflows. In a way it is a last-resort debugging tool. I personally think this is worth investigating. If you don't think so, or if you're getting tired of my messages on this list, just let me know. -- @ - Alf From diy-linux-dev@diy-linux.org Fri Sep 14 19:49:41 2007 From: diy-linux-dev@diy-linux.org (Leon Woestenberg) Date: Fri, 14 Sep 2007 21:49:41 +0200 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709140816.52953.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> <200709132217.41051.alf@mypals.org> <200709140816.52953.alf@mypals.org> Message-ID: <46EAE5D5.1030008@mailcan.com> Hello Alf, Alf mel wrote: > On Thursday 13 September 2007 10:17:40 pm Alf mel wrote: > > a way it is a last-resort debugging tool. I personally think this is worth > investigating. If you don't think so, or if you're getting tired of my > messages on this list, just let me know. I think I am not speaking for myself along when I agree that this is worth investigating, otherwise I wouldn't be subscribed to something like DIY Linux. However, my head is stuck in some other pigeonhole bug in some other package. Keeping us updated is very kind of you, although the lack of replies might not confirm our appreciation that much :-) Thanks a bunch, keep up the good work! Regards, Leon. From diy-linux-dev@diy-linux.org Fri Sep 14 23:38:45 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 15 Sep 2007 09:38:45 +1000 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709101925.19479.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> Message-ID: <20070914233845.GA25428@eyo32.local> On Mon, Sep 10, 2007 at 07:25:19PM -0600, Alf mel wrote: > I've been playing around with the latest toolchain combo (Glibc 2.6.1, GCC > 4.2.1, and Binutils 2.18). Unfortunately, not everything works. I haven't > done much research into this, except rebuilt the system a few times to see > if the problem could be duplicated. > > As an introduction, I have an Athlon-XP system. My "standard" CFLAGS are as > follows: > > -Os -march=athlon-xp -pipe -fstack-protector -D_FORTIFY_SOURCE=2 \ > -Wl,-O1,--as-needed,--hash-style=both First some comments about your CFLAGS: * -Os : non-standard, should be mostly safe but we know it has caused problems in the past. Use carefully. * -march=athlon-xp : -march is overrated according to the Red Hat / Fedora toolchain gurus. They prefer to use -mtune. Of course -march implies -mtune but still use carefully. (IMHO -march is most useful for kernel and glibc) * -fstack-protector -D_FORTIFY_SOURCE=2 : Most of Fedora is built like this so I'd be looking there for hints. * -Wl,-O1,--as-needed,--hash-style=both : The -O1 linker flag should be safe but I'm not so sure about --as-needed. There used to be bugs when trying to compile a whole distro with it but maybe they're now fixed. --hash-style=both is just wasteful for a self built system. Glibc is automatically built --hash-style=both for compatibility with older binaries but everything else can just use --hash-style=gnu (as per the current specs hack). > For Glibc, I use: > > -O3 -march=athlon-xp -pipe -Wl,-O1,--as-needed,--hash-style=both Those linker flags are definitely redundant for Glibc. The Glibc build system is very well self contained and takes care to add those linker flags itself when needed. I wouldn't build Glibc like that. > Anyway, the first problem I noticed was during the build of m4. > The program seem to build fine. However, when trying to use vi, another > buffer overflow occurred: > I haven't Googled this much. I was hoping someone here would immediately > jump up and say "your problem lies in . . . . " (please fill in the blank). > I also wanted to pass the warning that this new toolchain may not be > entirely stable (sorry, Greg if that bursts your bubble). It's definitely stable for non-hardened builds. Your use of -D_FORTIFY_SOURCE is what's causing the grief. One very important point that needs to be kept in mind is that Red Hat / Fedora have been the driving force behind this -fstack-protector -D_FORTIFY_SOURCE=2 hardening. Unfortunately, for reasons I find a little mystifying, Fedora are still on GCC-4.1.x (admittedly, with LOADS of backported stuff from 4.2 and 4.3). So I suspect they haven't yet hit some of these issues that you are seeing. I'll do some more investigations.. Regards Greg From diy-linux-dev@diy-linux.org Sat Sep 15 00:17:58 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 15 Sep 2007 10:17:58 +1000 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709132217.41051.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709130820.22989.alf@mypals.org> <200709132217.41051.alf@mypals.org> Message-ID: <20070915001758.GA21524@eyo32.local> On Thu, Sep 13, 2007 at 10:17:40PM -0600, Alf mel wrote: > On Thursday 13 September 2007 08:20:22 am Alf mel wrote: > > M4 configure still > > overflowed at conftest. GCC may not be the culprit here. It might be > > the conftest program itself, or a bad combination of conftest and glibc. > > Well, I found the culprit for this problem. Here is a better snippet of > where things go wrong: This m4 issue can be reproduced on a "normal" (ie: non-hardened) system as simply as: ./configure CFLAGS="-O2 -D_FORTIFY_SOURCE" > checking whether printf supports infinite 'long double' arguments... *** > buffer overflow detected ***: ./conftest terminated Interestingly, it doesn't change the result of the configure test, it's still "no" in the normal case. So maybe it's just a buggy configure test? The program still builds and passes its test suite. I couldn't find much on google. Might be worth shooting a quick bug report upstream to bug-m4@gnu.org Re: the vim issue, hlfs have a patch that might be worth looking at: http://www.linuxfromscratch.org/patches/downloads/vim/vim-7.0-fortify_warnings-1.patch Regards Greg From diy-linux-dev@diy-linux.org Sat Sep 15 00:31:47 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 15 Sep 2007 10:31:47 +1000 Subject: util-linux-ng-2.13 In-Reply-To: <200709120848.06454.alf@mypals.org> References: <20070828012751.GA5557@eyo32.local> <46D3BB70.6050204@ums.usu.ru> <200709120848.06454.alf@mypals.org> Message-ID: <20070915003147.GA26504@eyo32.local> On Wed, Sep 12, 2007 at 08:48:06AM -0600, Alf mel wrote: > On Tuesday 28 August 2007 12:06:40 am Alexander E. Patrakov wrote: > > Is it possible to mount the virtual filesystems from outside the chroot, > > thus completely bypassing the need for temporary "mount" program? > > Dumb question. Yes :-) > From util-linux we currently use 2 programs in the tools > phase: mount and getopt. I take it the e2fsprogs requirement only applies > to mount and not getopt. Do we still need getopt as part of the toolchain? Rationale for getopt is explained in the Refbuild text. Apparently you missed it. Regards Greg From diy-linux-dev@diy-linux.org Sat Sep 15 01:08:50 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Fri, 14 Sep 2007 19:08:50 -0600 Subject: util-linux-ng-2.13 In-Reply-To: <20070915003147.GA26504@eyo32.local> References: <20070828012751.GA5557@eyo32.local> <200709120848.06454.alf@mypals.org> <20070915003147.GA26504@eyo32.local> Message-ID: <200709141908.50920.alf@mypals.org> On Friday 14 September 2007 06:31:47 pm Greg Schafer wrote: > > Dumb question. > > Yes :-) > > Rationale for getopt is explained in the Refbuild text. Apparently you > missed it. My darn scrollitis is acting up again. I told you it was dumb. :-) That sure answers my question, though. Since I use the mount in the host, I don't need it at all and won't have to compile e2fsprogs libraries in the tools phase. Thanks for pointing me in the right direction. -- @ - Alf From diy-linux-dev@diy-linux.org Sat Sep 15 04:54:12 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 15 Sep 2007 14:54:12 +1000 Subject: Buffer overflows with newest toolchain In-Reply-To: <20070914233845.GA25428@eyo32.local> References: <200709101925.19479.alf@mypals.org> <20070914233845.GA25428@eyo32.local> Message-ID: <20070915045412.GA10678@eyo32.local> On Sat, Sep 15, 2007 at 09:38:45AM +1000, Greg Schafer wrote: > It's definitely stable for non-hardened builds. Your use of > -D_FORTIFY_SOURCE is what's causing the grief. One last thing. The more I look at this stuff, the more I believe there is nothing wrong with the toolchain. The anecdotal evidence on the web suggests that almost always -D_FORTIFY_SOURCE problems lie in the application code. BTW, I just tried to reproduce this with vim (with latest patches). It seems fine with -D_FORTIFY_SOURCE=1, but it misbehaved with -D_FORTIFY_SOURCE=2. I didn't get the same output as Alf, just some crazy aborts when running the test suite. Regards Greg From diy-linux-dev@diy-linux.org Sat Sep 15 14:14:02 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Sat, 15 Sep 2007 08:14:02 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <20070915001758.GA21524@eyo32.local> References: <200709101925.19479.alf@mypals.org> <200709132217.41051.alf@mypals.org> <20070915001758.GA21524@eyo32.local> Message-ID: <200709150814.03187.alf@mypals.org> On Friday 14 September 2007 06:17:58 pm Greg Schafer wrote: > This m4 issue can be reproduced on a "normal" (ie: non-hardened) system Alright! I'm not crazy! > I couldn't find much on google. Might be worth shooting a quick bug > report upstream to bug-m4@gnu.org I'll do that. > Re: the vim issue, hlfs have a patch that might be worth looking at: > > http://www.linuxfromscratch.org/patches/downloads/vim/vim-7.0-fortify_war >nings-1.patch That particular patch is just for 7.0. The current HFLS SVN version has a sed for fixing the immediate overflow problem and a new patch for 7.1. The patch for 7.0 contains the change done by the sed. I've tried the sed and it fixed the overflow at startup. Apparently VIM is designed to intentionally overflow due to its use of C89 code. Thanks for the pointer. I don't know why it didn't occur to me to go there for help on this issue. My SLIM project (based heavily on DIY-Linux) is my hobby. Some people build cars for hobby, some people build furniture as hobby. I try (the operative word) to build a Linux Distribution. I'm not the smartest, and work stresses me out quite a bit sometimes. At times like that I can't think straight as pointed out by my util-linux post. Thanks Greg, Jon and Leon for you help on this issue. It's always a big plus to know people that are smarter than me *and* are willing to help. Greg, I agree with you. It seems the latest toolchain is very stable. It even pointed out the problems in other packages. I'll continue now with other packages (Apache, MySQL, etc. for servers and X, KDE for workstations) and see how this new toolchain fairs. Thanks for your help. -- @ - Alf From diy-linux-dev@diy-linux.org Sun Sep 16 21:17:27 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Mon, 17 Sep 2007 07:17:27 +1000 Subject: Buffer overflows with newest toolchain In-Reply-To: <200709150814.03187.alf@mypals.org> References: <200709101925.19479.alf@mypals.org> <200709132217.41051.alf@mypals.org> <20070915001758.GA21524@eyo32.local> <200709150814.03187.alf@mypals.org> Message-ID: <20070916211727.GA18208@eyo32.local> On Sat, Sep 15, 2007 at 08:14:02AM -0600, Alf mel wrote: > On Friday 14 September 2007 06:17:58 pm Greg Schafer wrote: > > This m4 issue can be reproduced on a "normal" (ie: non-hardened) system > > Alright! I'm not crazy! > > > I couldn't find much on google. Might be worth shooting a quick bug > > report upstream to bug-m4@gnu.org > > I'll do that. Found your report. Sorry dude, but I kinda hijacked it :-/ Thread starts here: http://lists.gnu.org/archive/html/bug-gnulib/2007-09/msg00108.html Bottom line - buggy configure test. Regards Greg From diy-linux-dev@diy-linux.org Sun Sep 16 22:03:21 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Sun, 16 Sep 2007 16:03:21 -0600 Subject: Buffer overflows with newest toolchain In-Reply-To: <20070916211727.GA18208@eyo32.local> References: <200709101925.19479.alf@mypals.org> <200709150814.03187.alf@mypals.org> <20070916211727.GA18208@eyo32.local> Message-ID: <200709161603.21272.alf@mypals.org> On Sunday 16 September 2007 03:17:27 pm Greg Schafer wrote: > Found your report. Sorry dude, but I kinda hijacked it :-/ That's fine. Thanks for isolating the piece of code and sending it upstream. -- @ - Alf From diy-linux-dev@diy-linux.org Mon Sep 17 23:36:47 2007 From: diy-linux-dev@diy-linux.org (Andrew Fyfe) Date: Tue, 18 Sep 2007 00:36:47 +0100 Subject: Rational for linux-kernel-headers commands Message-ID: <46EF0F8F.9000403@neptune-one.net> Two questions about installing the kernel headers... - why are the scsi headers removed? - why are the headers installed into a temporary directory then copied to usr/include? why not install them straight into usr/include? Andrew From diy-linux-dev@diy-linux.org Mon Sep 17 23:46:49 2007 From: diy-linux-dev@diy-linux.org (Dan Nicholson) Date: Mon, 17 Sep 2007 16:46:49 -0700 Subject: Rational for linux-kernel-headers commands In-Reply-To: <46EF0F8F.9000403@neptune-one.net> References: <46EF0F8F.9000403@neptune-one.net> Message-ID: <91705d080709171646u464c3201la4ec79400d1bf7ee@mail.gmail.com> On 9/17/07, Andrew Fyfe wrote: > Two questions about installing the kernel headers... > > - why are the scsi headers removed? Glibc installs its own scsi headers, and generally it's better to prefer the system headers from the C library. > - why are the headers installed into a temporary directory then copied > to usr/include? why not install them straight into usr/include? Because it blows away existing files in the target directory. Not a problem if /usr/include is empty, but a big problem if you're reinstalling the headers for any reason. -- Dan From diy-linux-dev@diy-linux.org Tue Sep 18 06:00:08 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Tue, 18 Sep 2007 16:00:08 +1000 Subject: Pacman 3 (was Re: Rational for linux-kernel-headers commands) In-Reply-To: <46EF0F8F.9000403@neptune-one.net> References: <46EF0F8F.9000403@neptune-one.net> Message-ID: <20070918060008.GA18750@eyo32.local> Hi Andrew, Sorry to hijack your post but I notice you are active in Pacman development. I just wanted your thoughts on whether I should upgrade the Refbuild to Pacman 3.x or not. Pacman-2.9.8 still works very well for me so I haven't had a real need to upgrade. Is the new version now part of an Arch Linux release and therefore in widespread usage and considered stable? Am I going to be able to patch it to enable the fake_install() stuff? This is really the killer feature for me. Thanks for any insight. Regards Greg From diy-linux-dev@diy-linux.org Wed Sep 19 05:06:44 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Wed, 19 Sep 2007 15:06:44 +1000 Subject: texinfo-4.11 issues Message-ID: <20070919050644.GA27651@eyo32.local> Hi Guys, Just a headsup about a few issues with this version. 1. It fails 2 tests because `tex' is missing. The failing tests should be skipped in this case so I sent a patch upstream which will likely appear in next release: http://lists.gnu.org/archive/html/bug-texinfo/2007-09/msg00038.html For now, I'm just going to note these as expected fails in the Refbuild. 2. Binutils and GCC do not recognize 4.11 as a valid version. Unfortunately, due to earlier mentioned bug with the Binutils-2.18 `make dist' release, this problem now breaks the Binutils build instead of just skipping the texinfo page generation like it should. A patch was posted here to fix the issue for both Binutils and GCC (recall they both share the top level): http://gcc.gnu.org/ml/gcc-patches/2007-09/msg01271.html For Binutils, I'll use the following workaround instead of the patch: make LDFLAGS=${LDFLAGS} tooldir=/usr MAKEINFO=makeinfo For GCC, more is needed. The build doesn't break, but info pages do not get installed. This does the trick: make LDFLAGS=${LDFLAGS} MAKEINFO=makeinfo gcc_cv_prog_makeinfo_modern=yes On a slightly related note, grep-2.53 fails to build if makeinfo is not present. It's not a major problem at the moment because makeinfo is needed for the Binutils build anyway. But we could maybe look at moving texinfo before grep in the temptools phase if problem persists. Regards Greg From diy-linux-dev@diy-linux.org Fri Sep 21 23:57:03 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 22 Sep 2007 09:57:03 +1000 Subject: texinfo-4.11 issues In-Reply-To: <20070919050644.GA27651@eyo32.local> References: <20070919050644.GA27651@eyo32.local> Message-ID: <20070921235703.GA24838@eyo32.local> On Wed, Sep 19, 2007 at 03:06:44PM +1000, Greg Schafer wrote: > For Binutils, I'll use the following workaround instead of the patch: > > make LDFLAGS=${LDFLAGS} tooldir=/usr MAKEINFO=makeinfo In the case of binutils-2.16, need to also pass MAKEINFO=makeinfo to the `make install' invocation to get all the info pages. > On a slightly related note, grep-2.53 fails to build if makeinfo is not > present. It's not a major problem at the moment because makeinfo is needed > for the Binutils build anyway. But we could maybe look at moving texinfo > before grep in the temptools phase if problem persists. A workaround for this bug is to configure Grep like this: ./configure MISSING=true Regards Greg From diy-linux-dev@diy-linux.org Sun Sep 23 21:48:30 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Mon, 24 Sep 2007 07:48:30 +1000 Subject: texinfo-4.11 issues In-Reply-To: <20070921235703.GA24838@eyo32.local> References: <20070919050644.GA27651@eyo32.local> <20070921235703.GA24838@eyo32.local> Message-ID: <20070923214830.GA30702@eyo32.local> On Sat, Sep 22, 2007 at 09:57:03AM +1000, Greg Schafer wrote: > On Wed, Sep 19, 2007 at 03:06:44PM +1000, Greg Schafer wrote: > > > For Binutils, I'll use the following workaround instead of the patch: > > > > make LDFLAGS=${LDFLAGS} tooldir=/usr MAKEINFO=makeinfo > > In the case of binutils-2.16, need to also pass MAKEINFO=makeinfo to the > `make install' invocation to get all the info pages. Actually, rather than pass "MAKEINFO=makeinfo" twice, it seems more efficient to just do this after the configure step: echo "MAKEINFO = makeinfo" >> Makefile Which is what I'll use for the chroot binutils. But of course, neither of the above options is going to cut it for the temptools phase. What if there is no makeinfo on the host? I've come up with this solution: echo "MAKEINFO = :" >> Makefile The beauty of this hack is that it solves all makeinfo related problems. ie: it eliminates the dependence on texinfo entirely, even for the pesky HJL binutils case. Sweet. Regards Greg From diy-linux-dev@diy-linux.org Thu Sep 27 04:38:45 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Thu, 27 Sep 2007 14:38:45 +1000 Subject: to bootstrap, or not to bootstrap Message-ID: <20070927043845.GA12605@eyo32.local> Hi Guys, How would folks feel if we were to drop the GCC bootstrap? To clarify, I mean downgrade the current GCC Pass 1 from a 3-stage bootstrap to just a normal 1-stage build. The topic came up recently on the LFS dev list when Alexander P. hit 2 separate issues on x86_64: http://thread.gmane.org/gmane.linux.lfs.devel/5885 Those 2 issues were eventually explained and understood: http://article.gmane.org/gmane.linux.lfs.devel/6011 http://article.gmane.org/gmane.linux.lfs.devel/6227 but they highlighted an area of weakness in the current native build method ie: that there is an open window where the new tools link against the host Glibc, specifically in stage 2 of the GCC 3-stage bootstrap. Omitting the bootstrap would mostly side step the issue (not to mention result in much quicker builds). I've been thinking about it quite a lot lately and have now finally gotten around to some actual testing and research. Bottom line - it seems to work fine and the end result is identical. It works even from an ancient Red Hat 6.2 host! When I say identical, I mean byte-for-byte identical as per ICA-style binary diffing techniques. During the earlier pointed to threads, it was suggested moving the bootstrap to either pass 2 or even the chroot phase, but if there is no difference in the end product, why even bother? The more I look into it the more I feel we can justify it *in the context of our 2 phase build method* Anyhoo, just wanted to float the idea and solicit some feedback. I will put together a thorough rationale detailing the technicalities before going ahead with any changes. Regards Greg From diy-linux-dev@diy-linux.org Thu Sep 27 13:51:16 2007 From: diy-linux-dev@diy-linux.org (Jeremy Huntwork) Date: Thu, 27 Sep 2007 07:51:16 -0600 Subject: to bootstrap, or not to bootstrap In-Reply-To: <20070927043845.GA12605@eyo32.local> References: <20070927043845.GA12605@eyo32.local> Message-ID: <20070927135116.GA13520@linuxfromscratch.org> On Thu, Sep 27, 2007 at 02:38:45PM +1000, Greg Schafer wrote: > but they highlighted an area of weakness in the current native build method > ie: that there is an open window where the new tools link against the host > Glibc, specifically in stage 2 of the GCC 3-stage bootstrap. Omitting the > bootstrap would mostly side step the issue (not to mention result in much > quicker builds). Actually, correct me if I'm wrong, but it's not really the linking against the host's Glibc that is the problem. Just to clarify, the first pass of Binutils and GCC have to link against the host's libc. Where the window opens and problems occur is when we try to use our new tools in conjunction with the host's libc to do the linking. We get around this by using 'gcc -B/usr/bin' for Binutils and GCC. But since this only affects stage1 of a bootstrapped GCC, there is a potential for trouble because the second stage is linked with our new tools, which when it comes down to it, may be incompatible with the host's libc. > I've been thinking about it quite a lot lately and have now finally gotten > around to some actual testing and research. Bottom line - it seems to work > fine and the end result is identical. It works even from an ancient Red Hat > 6.2 host! When I say identical, I mean byte-for-byte identical as per > ICA-style binary diffing techniques. Glad to hear the tests came back positive. I'd be fine with dropping the bootstrap so long as the devs make sure that they do a similar ICA-style test each time they go to upgrade the toolchain. -- JH From diy-linux-dev@diy-linux.org Thu Sep 27 13:58:58 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Thu, 27 Sep 2007 07:58:58 -0600 Subject: to bootstrap, or not to bootstrap In-Reply-To: <20070927043845.GA12605@eyo32.local> References: <20070927043845.GA12605@eyo32.local> Message-ID: <200709270758.58177.alf@mypals.org> On Wednesday 26 September 2007 10:38:45 pm Greg Schafer wrote: > How would folks feel if we were to drop the GCC bootstrap? > > Omitting the bootstrap would mostly side step the issue (not to mention > result in much quicker builds). > > Bottom line - it seems > to work fine and the end result is identical. It works even from an > ancient Red Hat 6.2 host! When I say identical, I mean byte-for-byte > identical as per ICA-style binary diffing techniques. Pros: Quicker builds, no change in end result Cons: We change the way we do things. I opt for the changes. The Pass 1 GCC is going to get thrown away 30 min. later (depending on the speed of our machine). As long as it can still build a proper Glibc and GCC to replace itself, we should be in good shape. -- @ - Alf From diy-linux-dev@diy-linux.org Thu Sep 27 14:25:24 2007 From: diy-linux-dev@diy-linux.org (Jeremy Huntwork) Date: Thu, 27 Sep 2007 08:25:24 -0600 Subject: to bootstrap, or not to bootstrap In-Reply-To: <20070927135116.GA13520@linuxfromscratch.org> References: <20070927043845.GA12605@eyo32.local> <20070927135116.GA13520@linuxfromscratch.org> Message-ID: <20070927142524.GB13520@linuxfromscratch.org> On Thu, Sep 27, 2007 at 07:51:16AM -0600, Jeremy Huntwork wrote: > Glad to hear the tests came back positive. I'd be fine with dropping the > bootstrap so long as the devs make sure that they do a similar ICA-style > test each time they go to upgrade the toolchain. Sorry, the above comments don't really fit this list. I was reading my mail with a client that doesn't filter the mailing list contents, and I thought this was an lfs-dev post. Of course, your findings will probably make their way to LFS land at some point... -- JH From diy-linux-dev@diy-linux.org Sat Sep 29 13:56:45 2007 From: diy-linux-dev@diy-linux.org (naruto canada) Date: Sat, 29 Sep 2007 21:56:45 +0800 Subject: recommend this patch for "scriptlets/temptools/glibc" In-Reply-To: <9dde3d700709290635s36804883kd56bbec1fa967fce@mail.gmail.com> References: <9dde3d700709290635s36804883kd56bbec1fa967fce@mail.gmail.com> Message-ID: <9dde3d700709290656m2c2f46fepf6b712c39c88642e@mail.gmail.com> On 9/29/07, naruto canada wrote: > hi > > I recommend adding the "sed" statement for "scriptlets/temptools/glibc" > This helps with some strange circumstances where timezone failed to build, > As this patch won't affect the final chroot tools, this can only help. > > CFLAGS="-O2 $MARCH -pipe" \ > ../glibc-${GLIBC_VER}/configure \ > --disable-profile \ > --enable-add-ons \ > --with-binutils=${TT_PFX}/bin \ > --with-headers=${TT_PFX}/include \ > --without-selinux > sed -i "s/localedata timezone rt/localedata rt/" > /temptools/src/glibc-2.3.6/Makeconfig > make AUTOCONF=no > From diy-linux-dev@diy-linux.org Sat Sep 29 14:09:15 2007 From: diy-linux-dev@diy-linux.org (naruto canada) Date: Sat, 29 Sep 2007 22:09:15 +0800 Subject: tar 1.18 test failed Message-ID: <9dde3d700709290709n3e9d99r8e534cf724ece5e1@mail.gmail.com> hi test failed at: .... ... 66 truncate ... <---- .... on second run, it went away. From diy-linux-dev@diy-linux.org Sat Sep 29 14:18:06 2007 From: diy-linux-dev@diy-linux.org (naruto canada) Date: Sat, 29 Sep 2007 22:18:06 +0800 Subject: tar 1.18 test failed In-Reply-To: <9dde3d700709290709n3e9d99r8e534cf724ece5e1@mail.gmail.com> References: <9dde3d700709290709n3e9d99r8e534cf724ece5e1@mail.gmail.com> Message-ID: <9dde3d700709290718i17368d56s7ccd4ea4e7a0287e@mail.gmail.com> On 9/29/07, naruto canada wrote: > hi > > test failed at: > .... > ... > 66 truncate ... <---- that should be: .... 65: truncate FAILED (truncate.at:32) > > on second run, it went away. >