From diy-linux-dev@diy-linux.org Sat Nov 3 04:51:09 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Sat, 3 Nov 2007 14:51:09 +1100 Subject: Multilib build method In-Reply-To: <20071030023220.GA2003@eyo32.local> References: <20071027064843.GA8402@eyo32.local> <20071030023220.GA2003@eyo32.local> Message-ID: <20071103035109.GA3006@eyo32.local> On Tue, Oct 30, 2007 at 01:32:20PM +1100, Greg Schafer wrote: > There are 2 new environment variables, "$TARGET" and "$BUILD_HOST", both of > which should be self-explanatory. Actually, $TARGET is a bit too generic and might conflict with something. Therefore I'll go with "$DIY_TARGET" and "$BUILD_HOST". I'm also going to need a way to distinguish between "pure-64 x86_64" and "multilib x86_64". "Multilib" is not really the technically correct term for this situation so I think I'll create a new variable, "$BIARCH" which is set to either "YES" or "NO". > It should be easy to automate the setting of a > faked $BUILD_HOST by installing config.guess in $HOME as part of the > environment setup and putting some logic in the bash startup files of the > build user. Will sort this detail out later.. Ok, this is what I'm proposing here. Instruct the user to install a config.guess to $HOME by unpacking the binutils tarball and cp'ing it in. Then add something like this to the existing contents of .bashrc: ############################## DIY_TARGET=i686-pc-linux-gnu BIARCH=NO if [ -f "$HOME/config.guess" ]; then H=`sh $HOME/config.guess` BUILD_HOST=$(echo $H | sed "s/$(echo $H | cut -d- -f2)/fake/") else echo Warning: Please install $HOME/config.guess fi if ! echo $DIY_TARGET | grep ^x86_64 >/dev/null; then [ "$BIARCH" = YES ] && echo Warning: no BIARCH support for target $DIY_TARGET ! else [ "$BIARCH" = YES ] && export DIR_SUFFIX=64 fi export DIY_TARGET BIARCH BUILD_HOST ############################## It's starting to bloat the .bashrc a little but what the hell. One other thing, the existing "$DIY_ARCH" variable does not translate well to the myriad of new build scenarios supported by the new build method. "$DIY_TARGET" conveys the true meaning much better so I will be phasing out usage of "$DIY_ARCH". While I'm at it, I might change the name of the current "$SYSROOT" variable. It was a mistake choosing that name because it's not really a "sysroot" at all in the GCC sense of the word. $DIY_ROOT is probably a better choice. > Not sure yet whether I'll somehow integrate all this into the current > Refbuild or write a separate doc. Comments? Questions? For now, I'm just going to add a new Temptools phase page before the Appendix (xml/temptools2.xml) titled "Alternate Build Method" or maybe even "Next Gen Build Method". Then same for the Chroot phase. Over time, if it becomes clear that the new method is superior to the old, I'll just retire the existing method and replace it with the new. BTW, I have bi-arch x86_64 almost working correctly now. But I've hit the same bug (GCC searching host lib dirs) that I hit back when first adding pure x86_64 to the current build. The current hack of sedding out MULTILIB_OSDIRNAMES is clearly not going to fly for a bi-arch build. I still think it's a bug in the GCC driver (gcc.c) but I cannot figure it out. I've come up with a workaround solution which seems to do the trick: echo ' #define STANDARD_STARTFILE_PREFIX_1 "" #define STANDARD_STARTFILE_PREFIX_2 ""' >> gcc/config/$DL_HEADER The bug description is - GCC-Pass2, once installed, will search for libs on the host because it adds `-L' directives eg: "-L/lib/../lib64" and "-L/usr/lib/../lib64" which can be seen in the `gcc -v' output. It will only add these directives if the /lib64 and /usr/lib64 dirs actually exist. This bug affects pure x86_64 and the `-m32' part of bi-arch x86_64. It's got to be some kind of bug in the multilib handling because the above workaround is not needed for i686. If I were to hazard a guess, I'd say the bug is in the for_each_path() function of gcc.c. I'll probably add a sanity check like this at the end of GCC-Pass2: # Sanity check. echo 'main(){}' | cc -x c -lbogus -v -Wl,--verbose - &> foo || : if grep "^attempt to open /usr" foo; then echo Oops; exit 1 fi rm -fv a.out foo Regards Greg From diy-linux-dev@diy-linux.org Wed Nov 7 17:16:53 2007 From: diy-linux-dev@diy-linux.org (Jon Grosshart) Date: Wed, 7 Nov 2007 11:16:53 -0500 Subject: binutils-2.18 & makeinfo Message-ID: <9cf823c60711070816q21706d71k14015b0c01048ee0@mail.gmail.com> ------=_Part_34765_14963398.1194452213099 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hey all. Running into the same issue that Greg posted on sourceware and I'm not sure why at present. http://www.sourceware.org/ml/binutils/2007-08/msg00472.html Only been looking at it for a few minutes but thought I'd bounce it off the list before I went to work to see if anyone has ran into it. Using 2.6.1/4.2.2/2.18 combo on x86.. root-in-chroot: # which makeinfo /temptools/bin/makeinfo # echo $PATH /bin:/usr/bin:/sbin:/usr/sbin:/temptools/bin # /temptools/bin/makeinfo --version makeinfo (GNU texinfo) 4.11 Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Dunno off hand. Will investigate when I get home tho. Thanks. Sorry for posting this without trouble shooting further than what is listed above. Just thought I'd see if anyone else hit this before I re-invent the wheel as it were. Jon ------=_Part_34765_14963398.1194452213099 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline Hey all. Running into the same issue that Greg posted on sourceware and I'm not sure why at present.
http://www.sourceware.org/ml/binutils/2007-08/msg00472.html

Only been looking at it for a few minutes but thought I'd bounce it off the list before I went to work to see if anyone has ran into it. Using 2.6.1/4.2.2/2.18 combo on x86..

root-in-chroot:

# which makeinfo
/temptools/bin/makeinfo
# echo $PATH
/bin:/usr/bin:/sbin:/usr/sbin:/temptools/bin
# /temptools/bin/makeinfo --version
makeinfo (GNU texinfo) 4.11

Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


Dunno off hand. Will investigate when I get home tho. Thanks. Sorry for posting this without trouble shooting further than what is listed above. Just thought I'd see if anyone else hit this before I re-invent the wheel as it were.

Jon
------=_Part_34765_14963398.1194452213099-- From diy-linux-dev@diy-linux.org Wed Nov 7 22:29:34 2007 From: diy-linux-dev@diy-linux.org (Jon Grosshart) Date: Wed, 7 Nov 2007 16:29:34 -0500 Subject: binutils-2.18 & makeinfo In-Reply-To: <9cf823c60711070816q21706d71k14015b0c01048ee0@mail.gmail.com> References: <9cf823c60711070816q21706d71k14015b0c01048ee0@mail.gmail.com> Message-ID: <9cf823c60711071329wbca84c4hb58357d8664ec74e@mail.gmail.com> ------=_Part_36188_5572826.1194470974897 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Nov 7, 2007 11:16 AM, Jon Grosshart wrote: > > Found various fixes, mainly to configure.ac regarding $MAKEINFO --version in a matter of seconds, so again, sorry for posting. Still not at home but it seems to be a trivial matter. I guess what I was really driving at by posting, is why it appears to be a non-issue with you Greg? You imply it's an older texinfo problem, from what I gathered, and there is no mention of it within the ref build (that I saw)... Frugalware and countless others seem to be hitting this 'snag' however. Anyway.... I'll patch it and move on when I get home. Just looking for a little DIY specific info on the matter. Jon ------=_Part_36188_5572826.1194470974897 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline

On Nov 7, 2007 11:16 AM, Jon Grosshart <jgrosshart@gmail.com> wrote:

<blah>


Found various fixes, mainly to configure.ac regarding $MAKEINFO --version in a matter of seconds, so again, sorry for posting. Still not at home but it seems to be a trivial matter. I guess what I was really driving at by posting, is why it appears to be a non-issue with you Greg? You imply it's an older texinfo problem, from what I gathered, and there is no mention of it within the ref build (that I saw)...

Frugalware and countless others seem to be hitting this 'snag' however. Anyway.... I'll patch it and move on when I get home. Just looking for a little DIY specific info on the matter.

Jon

------=_Part_36188_5572826.1194470974897-- From diy-linux-dev@diy-linux.org Wed Nov 7 23:16:33 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Thu, 8 Nov 2007 09:16:33 +1100 Subject: binutils-2.18 & makeinfo In-Reply-To: <9cf823c60711071329wbca84c4hb58357d8664ec74e@mail.gmail.com> References: <9cf823c60711070816q21706d71k14015b0c01048ee0@mail.gmail.com> <9cf823c60711071329wbca84c4hb58357d8664ec74e@mail.gmail.com> Message-ID: <20071107221633.GA30180@eyo32.local> On Wed, Nov 07, 2007 at 04:29:34PM -0500, Jon Grosshart wrote: > seems to be a trivial matter. I guess what I was really driving at by > posting, is why it appears to be a non-issue with you Greg? You imply it's > an older texinfo problem, from what I gathered, and there is no mention of > it within the ref build (that I saw)... Jon, I previously put some workarounds in the Refbuild. grep for MAKEINFO. Are they not working for you? More info in thread starting here: http://www.diy-linux.org/pipermail/diy-linux-dev/2007-September/001113.html Regards Greg From diy-linux-dev@diy-linux.org Thu Nov 8 00:22:41 2007 From: diy-linux-dev@diy-linux.org (Jon Grosshart) Date: Wed, 7 Nov 2007 18:22:41 -0500 Subject: binutils-2.18 & makeinfo In-Reply-To: <20071107221633.GA30180@eyo32.local> References: <9cf823c60711070816q21706d71k14015b0c01048ee0@mail.gmail.com> <9cf823c60711071329wbca84c4hb58357d8664ec74e@mail.gmail.com> <20071107221633.GA30180@eyo32.local> Message-ID: <9cf823c60711071522u263af288v77d0bdde128efe52@mail.gmail.com> ------=_Part_36523_10206208.1194477761032 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline On Nov 7, 2007 5:16 PM, Greg Schafer wrote: > Jon, I previously put some workarounds in the Refbuild. grep for MAKEINFO. > Are they not working for you? More info in thread starting here: > > I just saw that Greg. I was making it a point to sync my scripts to DIY current, atleast I thought I was. Feel free to bitch slap me. Serious.... Sorry for all the static. ------=_Part_36523_10206208.1194477761032 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline

On Nov 7, 2007 5:16 PM, Greg Schafer <gschafer@zip.com.au> wrote:
 
Jon, I previously put some workarounds in the Refbuild. grep for MAKEINFO.
Are they not working for you? More info in thread starting here:


I just saw that Greg. I was making it a point to sync my scripts to DIY current, atleast I thought I was. Feel free to bitch slap me. Serious.... Sorry for all the static.


------=_Part_36523_10206208.1194477761032-- From diy-linux-dev@diy-linux.org Thu Nov 8 04:45:05 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Thu, 8 Nov 2007 14:45:05 +1100 Subject: Binutils --with-lib-path Message-ID: <20071108034505.GA5228@eyo32.local> Hi Guys, While looking at the Multilib build method stuff, it has occurred to me that the way we handle the "where ld searches for -lfoobar libs" issue is somewhat bogus and unnecessary. I'm talking about the *current* build method used in the Refbuild, LFS, etc. It's true that we need to prevent ld from finding libs on the host, but installing an adjusted linker and then repeating same in the Chroot phase is just nonsense. The facts are that *both* GCC and ld have a say in finding libs when linking. GCC has some built-in search paths (signified by -L directives seen in -v output) and ld uses SEARCH_DIR statements (seen in the ld --verbose output and also in the installed linker scripts). A better solution would be to just stop ld from using SEARCH_DIR at all, and just let GCC do the talking. This can be achieved by using the not-well-documented: --with-lib-path=: (read ld/genscripts.sh for details) I've used it to good effect in the new build method. I'll test it with the old method and if all is well, make the obvious Refbuild changes to simplify the build cmds. Regards Greg From diy-linux-dev@diy-linux.org Thu Nov 15 04:09:48 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Thu, 15 Nov 2007 14:09:48 +1100 Subject: Binutils --with-lib-path In-Reply-To: <20071108034505.GA5228@eyo32.local> References: <20071108034505.GA5228@eyo32.local> Message-ID: <20071115030948.GA16317@eyo32.local> On Thu, Nov 08, 2007 at 02:45:05PM +1100, Greg Schafer wrote: > It's true that we need to prevent ld from finding libs on the host, but > installing an adjusted linker and then repeating same in the Chroot phase is > just nonsense. Not quite. The current build does actually need it because of the Chroot phase build order ie: GCC before Binutils. However, the above holds true if Binutils is built before GCC. > --with-lib-path=: > > (read ld/genscripts.sh for details) > > I've used it to good effect in the new build method. I'll test it with the > old method and if all is well, make the obvious Refbuild changes to simplify > the build cmds. No. I'll just leave the current build as is and make changes in the new method. ie: Binutils will come before GCC in the Chroot phase. One of the incentives for changing the order a few years ago is no longer valid due to upstream fixes (wrong HOSTING_CRT0 in the ld testsuite). All will become clear when I add the new build method to the Refbuild, hopefully soonish. Regards Greg From diy-linux-dev@diy-linux.org Thu Nov 15 17:40:14 2007 From: diy-linux-dev@diy-linux.org (Pierre Labastie) Date: Thu, 15 Nov 2007 17:40:14 +0100 Subject: Binutils --with-lib-path In-Reply-To: <20071115030948.GA16317@eyo32.local> References: <20071108034505.GA5228@eyo32.local> <20071115030948.GA16317@eyo32.local> Message-ID: <473C766E.3040506@club-internet.fr> Greg Schafer a =E9crit : > [...] > No. I'll just leave the current build as is and make changes in the new > method. ie: Binutils will come before GCC in the Chroot phase. One of t= he > incentives for changing the order a few years ago is no longer valid du= e to > upstream fixes (wrong HOSTING_CRT0 in the ld testsuite). > =20 Actually, I have always been wondering why the building of binutils and=20 gcc is not done for both together (that is extracting both in the same=20 directory), as per the recommandations of the developpers (or at least=20 of their documentation). (Sorry if I am hitting an old debate I have not=20 been able to sort out in the mailing lists). Regards Pierre I have tried the build with glibc-2.7, it builds fine. There is a new=20 error in g++ test and also two more in glibc test. See below the list of=20 errors (the new grep is not happy either, but I am not sure whether it=20 is not locale related). The error on sha512 is somewhat scarry : does it mean that a crypted=20 connection may fail? I have not kept the directory so I cannot tell=20 exactly what it is... will try to find somme time to investigate it furth= er. ------------------------------------ glibc : /temptools/src/logs/chroot-1/glibc-2.7.log:make[2]: ***=20 [/home/pkgmgr/src/glibc-build/math/test-double.out] Error 1 /temptools/src/logs/chroot-1/glibc-2.7.log:make[1]: *** [math/tests] Erro= r 2 /temptools/src/logs/chroot-1/glibc-2.7.log:make[2]: ***=20 [/home/pkgmgr/src/glibc-build/crypt/sha512c-test.out] Error 1 /temptools/src/logs/chroot-1/glibc-2.7.log:make[1]: *** [crypt/tests]=20 Error 2 /temptools/src/logs/chroot-1/glibc-2.7.log:make: *** [check] Error 2 -------------------------------------- g++ : Running /home/pkgmgr/src/gcc-4.2.2/gcc/testsuite/g++.dg/dg.exp ... FAIL: g++.dg/other/unused1.C scan-assembler=20 (string|ascii?)z?\t"class2("|\\\\000) XPASS: g++.dg/tree-ssa/ivopts-1.C scan-tree-dump-not offset: -4B XPASS: g++.dg/tree-ssa/ivopts-1.C scan-tree-dump-not &x\[5\] ------------------------------------------ grep : Testing: ../src/grep Word -o -i input: "WordA/wordB/WORDC/" output: "" expect: "Word/word/WORD/" FAIL Testing: ../src/grep WORD -o -i input: "WordA/wordB/WORDC/" output: "" expect: "Word/word/WORD/" FAIL Testing: ../src/grep Word --color=3Dalways -i input: "WordA/wordb/WORDC/" output: "WordA/wordb/WORDC/" expect:=20 "=1B[01;31m=1B[KWord=1B[m=1B[KA/=1B[01;31m=1B[Kword=1B[m=1B[Kb/=1B[01;31m= =1B[KWORD=1B[m=1B[KC/" FAIL Testing: ../src/grep WORD --color=3Dalways -i input: "WordA/wordb/WORDC/" output: "WordA/wordb/WORDC/" expect:=20 "=1B[01;31m=1B[KWord=1B[m=1B[KA/=1B[01;31m=1B[Kword=1B[m=1B[Kb/=1B[01;31m= =1B[KWORD=1B[m=1B[KC/" FAIL FAIL: foad1.sh SKIP: fmbtest.sh Test #11: { ../src/grep -F -n -b -m 5 -C 1 yes; echo "?$?"; sed=20 's!^!X!'; } output: "2-10-[B02 no ]/3:20:[C03 yes]/4:30:[D04 yes]/5:40:[E05=20 yes]/6-50-[F06 no ]/7-60-[G07 no ]/8:70:[H08 yes]/9:80:[I09=20 yes]/10-90-[J10 no ]/?0/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13=20 yes]/X[N14 yes]/" expect: "2-10-[B02 no ]/3:20:[C03 yes]/4:30:[D04 yes]/5:40:[E05=20 yes]/6-50-[F06 no ]/7-60-[G07 no ]/8:70:[H08 yes]/9:80:[I09=20 yes]/?0/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/" FAIL Test #27: { ../src/grep -F -n -b -m 2 -v -C 1 yes; echo "?$?"; sed=20 's!^!X!'; } output: "1:0:[A01 no ]/2:10:[B02 no ]/3-20-[C03 yes]/?0/X[C03=20 yes]/X[D04 yes]/X[E05 yes]/X[F06 no ]/X[G07 no ]/X[H08 yes]/X[I09=20 yes]/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/" expect: "1:0:[A01 no ]/2:10:[B02 no ]/?0/X[C03 yes]/X[D04 yes]/X[E05=20 yes]/X[F06 no ]/X[G07 no ]/X[H08 yes]/X[I09 yes]/X[J10 no ]/X[K11 no=20 ]/X[L12 no ]/X[M13 yes]/X[N14 yes]/" FAIL Test #28: { ../src/grep -F -n -b -m 2 -v -C 1 -o yes; echo "?$?"; sed=20 's!^!X!'; } output: "3-25-yes/?0/X[C03 yes]/X[D04 yes]/X[E05 yes]/X[F06 no=20 ]/X[G07 no ]/X[H08 yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 no=20 ]/X[M13 yes]/X[N14 yes]/" expect: "?0/X[C03 yes]/X[D04 yes]/X[E05 yes]/X[F06 no ]/X[G07 no=20 ]/X[H08 yes]/X[I09 yes]/X[J10 no ]/X[K11 no ]/X[L12 no ]/X[M13=20 yes]/X[N14 yes]/" FAIL FAIL: yesno.sh =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D 2 of 13 tests failed (1 tests were not run) Please report to bug-grep@gnu.org From diy-linux-dev@diy-linux.org Thu Nov 15 22:47:49 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Fri, 16 Nov 2007 08:47:49 +1100 Subject: Binutils --with-lib-path In-Reply-To: <473C766E.3040506@club-internet.fr> References: <20071108034505.GA5228@eyo32.local> <20071115030948.GA16317@eyo32.local> <473C766E.3040506@club-internet.fr> Message-ID: <20071115214749.GA7333@eyo32.local> On Thu, Nov 15, 2007 at 05:40:14PM +0100, Pierre Labastie wrote: > Actually, I have always been wondering why the building of binutils and > gcc is not done for both together (that is extracting both in the same > directory), as per the recommandations of the developpers (or at least > of their documentation). (Sorry if I am hitting an old debate I have not > been able to sort out in the mailing lists). It's been touched on before. Combined trees are ideal for developers actually hacking on the toolchain itself. Our use case is more like that of a distro maker using tarballs. More comment here: http://www.mail-archive.com/lfs-dev@linuxfromscratch.org/msg10829.html > I have tried the build with glibc-2.7, it builds fine. There is a new > error in g++ test and also two more in glibc test. See below the list of > errors (the new grep is not happy either, but I am not sure whether it > is not locale related). > The error on sha512 is somewhat scarry : does it mean that a crypted > connection may fail? I have not kept the directory so I cannot tell > exactly what it is... will try to find somme time to investigate it further. > ------------------------------------ > glibc : > [/home/pkgmgr/src/glibc-build/math/test-double.out] Error 1 I see this one (same as 2.6.x) > [/home/pkgmgr/src/glibc-build/crypt/sha512c-test.out] Error 1 But I don't see this one. You should check if it's reproducible then try to understand the issue. > /temptools/src/logs/chroot-1/glibc-2.7.log:make[1]: *** [crypt/tests] > Error 2 > /temptools/src/logs/chroot-1/glibc-2.7.log:make: *** [check] Error 2 > -------------------------------------- > g++ : > Running /home/pkgmgr/src/gcc-4.2.2/gcc/testsuite/g++.dg/dg.exp ... > FAIL: g++.dg/other/unused1.C scan-assembler Yes, it's new. But others on the GCC test results list are seeing it too so I don't think it's a big problem. > grep : > Testing: ../src/grep Word -o -i > input: "WordA/wordB/WORDC/" > output: "" > expect: "Word/word/WORD/" > FAIL This is covered in the Refbuild notes. Regards Greg From diy-linux-dev@diy-linux.org Tue Nov 20 00:30:33 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Tue, 20 Nov 2007 10:30:33 +1100 Subject: Multilib build method In-Reply-To: <20071030023220.GA2003@eyo32.local> References: <20071027064843.GA8402@eyo32.local> <20071030023220.GA2003@eyo32.local> Message-ID: <20071119233033.GA12359@eyo32.local> On Tue, Oct 30, 2007 at 01:32:20PM +1100, Greg Schafer wrote: > Because we want to use cross compilation even when targetting > i686-pc-linux-gnu from i686-pc-linux-gnu, we need to "fake" the build > host. Actually, I've discovered a fundamental problem with this approach. Essentially, the cross tools built and installed into $PATH in pass1 (eg: i686-pc-linux-gnu-ld etc) can end up being found by the native configure scripts later on (even inside the chroot) which can cause weird side effects and even build failure in some scenarios. In order to avoid this, the solution is to fake the *target* instead of the build host. For example: DIY_TARGET=i686-diy-linux-gnu This means there is zero chance that `i686-diy-linux-gnu-ld' etc can be found later on when we are "native". Doing it this way makes a whole lot more sense. It even simplifies the build commands because we don't need to specify BUILD_HOST as often, which means we might be able to avoid the installation of `config.guess' as discussed earlier. In summary, the environment setup instuctions will tell the reader to specify a "fake" target for the purpose of forcing the pass 1 tools to be a cross compilation toolchain: Real Target $DIY_TARGET i686-pc-linux-gnu i686-diy-linux-gnu x86_64-unknown-linux-gnu x86_64-diy-linux-gnu (pure or biarch) I now have the 3 main builds working well as far as the toolchain goes (up to chroot sed, e2fsprogs etc.), pure i686, pure x86_64 and bi-arch x86_64. Watch out for a Refbuild commit over the next few days which will add the new method. It will be a bit rough initially, as I've just copied the current xml and modified the build commands only. The rationales will therefore be wrong but they'll be fixed over time. Still haven't addressed testsuites, PM, ICA, ppc, etc.. but I've hacked my build scripts to at least build the new setup. Regards Greg From diy-linux-dev@diy-linux.org Tue Nov 20 16:41:27 2007 From: diy-linux-dev@diy-linux.org (=?ISO-8859-1?Q?J=FCrg?= Billeter) Date: Tue, 20 Nov 2007 16:41:27 +0100 Subject: Multilib build method In-Reply-To: <20071119233033.GA12359@eyo32.local> References: <20071027064843.GA8402@eyo32.local> <20071030023220.GA2003@eyo32.local> <20071119233033.GA12359@eyo32.local> Message-ID: <1195573287.5022.18.camel@juerg-pd.bitron.ch> On Tue, 2007-11-20 at 10:30 +1100, Greg Schafer wrote: > On Tue, Oct 30, 2007 at 01:32:20PM +1100, Greg Schafer wrote: >=20 > > Because we want to use cross compilation even when targetting > > i686-pc-linux-gnu from i686-pc-linux-gnu, we need to "fake" the build > > host. >=20 > Actually, I've discovered a fundamental problem with this approach. > Essentially, the cross tools built and installed into $PATH in pass1 (eg: > i686-pc-linux-gnu-ld etc) can end up being found by the native configure > scripts later on (even inside the chroot) which can cause weird side effe= cts > and even build failure in some scenarios. >=20 > In order to avoid this, the solution is to fake the *target* instead of t= he > build host. For example: >=20 > DIY_TARGET=3Di686-diy-linux-gnu Doesn't this mean that you use cross compilation more often than before? The basic idea before was to build a cross compiler from a fake host architecture to the target architecture in pass1, then build a target native compiler in pass2 (using the cross compiler from pass1) and then build the final root with the compiler from pass2 without cross compilation. If you now build a cross compiler from the real host to a fake target in pass1, then build a cross compiler from the fake target to the real target in pass2, you're not only cross compiling in pass2 (using the cross compiler from pass1) but you're also cross compiling the final root with the cross compiler from pass2, i.e. from i686-diy-linux-gnu to i686-pc-linux-gnu. Maybe you avoid that by reverting the fake in other places but that doesn't make a lot of sense to me. Or am I just missing something and this works without breaking the logic? The way we avoid the issue in paldo[1] is to just separate pass1 and pass2 by installing them in different directories, not just one tools directory. The directory of pass1 will be deleted (resp. removed from the path) at the end of pass2, so it's not possible to run a binary of pass1 in the chroot phase. This works very well for us for years with pure x86 and x86_64 builds. J=C3=BCrg [1] http://www.paldo.org/ From diy-linux-dev@diy-linux.org Wed Nov 21 04:29:32 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Wed, 21 Nov 2007 14:29:32 +1100 Subject: Multilib build method In-Reply-To: <1195573287.5022.18.camel@juerg-pd.bitron.ch> References: <20071027064843.GA8402@eyo32.local> <20071030023220.GA2003@eyo32.local> <20071119233033.GA12359@eyo32.local> <1195573287.5022.18.camel@juerg-pd.bitron.ch> Message-ID: <20071121032932.GA1612@eyo32.local> On Tue, Nov 20, 2007 at 04:41:27PM +0100, J=FCrg Billeter wrote: > Doesn't this mean that you use cross compilation more often than before= ? No. > The basic idea before was to build a cross compiler from a fake host > architecture to the target architecture in pass1, then build a target > native compiler in pass2 (using the cross compiler from pass1) and then > build the final root with the compiler from pass2 without cross > compilation. Correct. It's still mostly the same, except that we build a cross compile= r from a real host to a fake target. The fake target is not really a fake. = It only differs from the real target in the vendor field of the target tripl= et eg: i686-XXX-linux-gnu which is enough to coax cross compilation mode (bu= t you already know that). > If you now build a cross compiler from the real host to a fake target i= n > pass1, then build a cross compiler from the fake target to the real > target in pass2, ??? This is where I think you went slightly wrong. We are building a *native* pass2 compiler using the cross compiler we built in pass1. ie: CC=3D"$DIY_TARGET-gcc -isystem etc, etc." The logic works, trust me :-) > The way we avoid the issue in paldo[1] is to just separate pass1 and > pass2 by installing them in different directories, not just one tools > directory. The directory of pass1 will be deleted (resp. removed from > the path) at the end of pass2, so it's not possible to run a binary of > pass1 in the chroot phase. That would certainly work too. But of course then you need to somewhat ha= ck the sources to change search paths etc. I'm trying to minimise hackery in this new method by using the normal and documented command line user interfaces where possible (-B, -isystem, -L, etc). Regards Greg From diy-linux-dev@diy-linux.org Fri Nov 23 18:51:29 2007 From: diy-linux-dev@diy-linux.org (Pierre Labastie) Date: Fri, 23 Nov 2007 18:51:29 +0100 Subject: New Error in glibc-2.7 test suite In-Reply-To: <20071115214749.GA7333@eyo32.local> References: <20071108034505.GA5228@eyo32.local> <20071115030948.GA16317@eyo32.local> <473C766E.3040506@club-internet.fr> <20071115214749.GA7333@eyo32.local> Message-ID: <47471321.8040504@club-internet.fr> Dear all, Not long ago I reported a new glibc-testsuite failure: ---------------------- GCONV_PATH=/temptools/src/glibc-build/iconvdata LC_ALL=C /temptools/src/glibc-build/elf/ld-linux.so.2 --library-path /temptools/src/glibc-build:/temptools/src/glibc-build/math:/temptools/src/glibc-build/elf:/temptools/src/glibc-build/dlfcn:/temptools/src/glibc-build/nss:/temptools/src/glibc-build/nis:/temptools/src/glibc-build/rt:/temptools/src/glibc-build/resolv:/temptools/src/glibc-build/crypt:/temptools/src/glibc-build/nptl /temptools/src/glibc-build/crypt/sha512c-test > /temptools/src/glibc-build/crypt/sha512c-test.out Timed out: killed the child process make[2]: *** [/temptools/src/glibc-build/crypt/sha512c-test.out] Error 1 make[2]: Target `tests' not remade because of errors. make[2]: Leaving directory `/temptools/src/glibc-2.7/crypt' make[1]: *** [crypt/tests] Error 2 -------------------------------- This is a new test in glibc-2.7, testing new functions in the crypt directory. As you can see, it is a timeout issue (I am running a laptop with a pentium-m processor at 1.8 GHz). I kept the build directory and I can do a few tests (notice that TIMEOUTFACTOR can be set in the environment to increase the timeout limit): ---------------------------------- build@b07-cluster$ time TIMEOUTFACTOR=1 ../elf/ld-linux.so.2 --library-path ..:. ./sha512c-test Timed out: killed the child process real 0m2.106s user 0m1.800s sys 0m0.002s build@b07-cluster$ time TIMEOUTFACTOR=2 ../elf/ld-linux.so.2 --library-path ..:. ./sha512c-test real 0m3.677s user 0m2.281s sys 0m0.008s ----------------------------------------- So, it does not look too scarry. Only the function does not return early enough with the normal timeout. Do you think I should fill a bug report ? Regards Pierre From diy-linux-dev@diy-linux.org Sat Nov 24 16:15:42 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Sat, 24 Nov 2007 08:15:42 -0700 Subject: Call for Assistance - udev In-Reply-To: <20070824012308.GA18387@eyo32.local> References: <20070824012308.GA18387@eyo32.local> Message-ID: <200711240815.42780.alf@mypals.org> On Thursday 23 August 2007 07:23:08 pm Greg Schafer wrote: > Anyhoo, it seems the latest release (udev-115) finally installs something > that looks like a sane default config. This message is probably late, and it's probably worthless, but in my testing, the default udev rules suck. They can't seem to find half the hardware and permissions are not correct. I actually found that using the LFS rules works much better than udev's own rules. Just my thoughts. -- @ - Alf From diy-linux-dev@diy-linux.org Sun Nov 25 13:42:19 2007 From: diy-linux-dev@diy-linux.org (Pierre Labastie) Date: Sun, 25 Nov 2007 13:42:19 +0100 Subject: Call for Assistance - udev In-Reply-To: <200711240815.42780.alf@mypals.org> References: <20070824012308.GA18387@eyo32.local> <200711240815.42780.alf@mypals.org> Message-ID: <47496DAB.8060704@club-internet.fr> Alf mel a =E9crit : > On Thursday 23 August 2007 07:23:08 pm Greg Schafer wrote: > =20 >> Anyhoo, it seems the latest release (udev-115) finally installs so= mething >> that looks like a sane default config. >> =20 > > This message is probably late, and it's probably worthless, but in = my=20 > testing, the default udev rules suck. They can't seem to find half= the=20 > hardware and permissions are not correct. I actually found that us= ing the=20 > LFS rules works much better than udev's own rules. Just my thought= s. > =20 Well, I have been using udev-115 and then -116 for 2 months now,=20 together with the LFS bootscripts, without changing a line in the rul= es,=20 except that I add the rules in the directory etc/udev/packages from = the=20 tarball to the rules in rules.d. See below for a tentative scriptlet.= It=20 finds "all the hardware" on my laptop, which might not be a thorough= =20 test, because the hardware is only PCI or USB (*). I mean that the /d= ev=20 directory obtained is mostly the same as the one with the LFS rules,= =20 save for these bizarre X0R related things. (*) I have tested with a nonmodular kernel as well as with almost all= =20 the drivers as modules. -------------------------- *|sed -i.bak 's/uucp/dialout/' $(grep -rl 'uucp' etc) sed -i.bak 's/XOR/X0R/' $(grep -rl 'XOR' etc) make EXTRAS=3D"$(echo extras/*)"|* *|make DESTDIR=3D$PM_DEST EXTRAS=3D"$(echo extras/*)" install install -v etc/udev/packages/* $PM_DEST/etc/udev/rules.d install -dv $PM_DEST/lib/{firmware,udev/devices/{pts,shm}} mknod -m666 $PM_DEST/lib/udev/devices/null c 1 3 ln -sv /proc/self/fd $PM_DEST/lib/udev/devices ln -sv fd/0 $PM_DEST/lib/udev/devices/stdin ln -sv fd/1 $PM_DEST/lib/udev/devices/stdout|**| ln -sv fd/2 $PM_DEST/lib/udev/devices/stderr|* ----------------------------- I just need to change the group uucp to dialout because uucp is not i= n=20 /etc/group. The line with the sed from XOR to X0R (X,zero,R) is maybe not necessa= ry.=20 I sticked to the recommandations in=20 (http://www.eu.kernel.org/pub/linux/docs/device-list/devices-2.6+.txt= ). Regards Pierre From diy-linux-dev@diy-linux.org Sun Nov 25 15:20:09 2007 From: diy-linux-dev@diy-linux.org (Alf mel) Date: Sun, 25 Nov 2007 07:20:09 -0700 Subject: Call for Assistance - udev In-Reply-To: <47496DAB.8060704@club-internet.fr> References: <20070824012308.GA18387@eyo32.local> <200711240815.42780.alf@mypals.org> <47496DAB.8060704@club-internet.fr> Message-ID: <200711250720.09401.alf@mypals.org> On Sunday 25 November 2007 05:42:19 am Pierre Labastie wrote: > Well, I have been using udev-115 and then -116 for 2 months now, > together with the LFS bootscripts, without changing a line in the rules, > except that I add the rules in the directory etc/udev/packages from the > tarball to the rules in rules.d. See below for a tentative scriptlet. It > finds "all the hardware" on my laptop, which might not be a thorough > test, because the hardware is only PCI or USB (*). I mean that the /dev > directory obtained is mostly the same as the one with the LFS rules, > save for these bizarre X0R related things. > (*) I have tested with a nonmodular kernel as well as with almost all > the drivers as modules. Well, I have to give you the credit, you took the time to figure it out their rules. I had a hard time trying to get everything I wanted to work (CD/DVD device aliases, persistent network device naming, group permissions, etc.), so I just used the latest LFS rules and they worked. I have to say, however, that using udev 115 and udev 117 seem very stable. I'm still torn, however, on whether it should be included as part of the DIY-Linux instructions. -- @ - Alf From diy-linux-dev@diy-linux.org Sun Nov 25 15:31:43 2007 From: diy-linux-dev@diy-linux.org (Alexander E. Patrakov) Date: Sun, 25 Nov 2007 19:31:43 +0500 Subject: Call for Assistance - udev In-Reply-To: <200711250720.09401.alf@mypals.org> References: <20070824012308.GA18387@eyo32.local> <200711240815.42780.alf@mypals.org> <47496DAB.8060704@club-internet.fr> <200711250720.09401.alf@mypals.org> Message-ID: 2007/11/25, Alf mel : > I have to say, however, that using udev 115 and udev 117 seem very stable. > I'm still torn, however, on whether it should be included as part of the > DIY-Linux instructions. I think, they shouldn't, for the same reason I stated originally: udev is closely tied to the bootscripts, and bootscripts are not part of DIY-Linux. Also, udev-117 still has a bug with the rule generator for persistent network interfaces (already reported to linux-hotplug list) and thus cannot be forced on DIY users. Even more (but this is _strictly_ my personal opinion, I don't force it on anyone), DIY-Linux without bootscripts essentially produces a "chrootable system" and thus things like kbd, udev, module-init-tools, inetutils, grub and sysvinit don't belong there. I.e., I want to see them in a separate book or at least a separate chapter. -- Alexander E. Patrakov From diy-linux-dev@diy-linux.org Sun Nov 25 23:49:18 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Mon, 26 Nov 2007 09:49:18 +1100 Subject: New Error in glibc-2.7 test suite In-Reply-To: <47471321.8040504@club-internet.fr> References: <20071108034505.GA5228@eyo32.local> <20071115030948.GA16317@eyo32.local> <473C766E.3040506@club-internet.fr> <20071115214749.GA7333@eyo32.local> <47471321.8040504@club-internet.fr> Message-ID: <20071125224918.GA6747@eyo32.local> On Fri, Nov 23, 2007 at 06:51:29PM +0100, Pierre Labastie wrote: > build@b07-cluster$ time TIMEOUTFACTOR=2 ../elf/ld-linux.so.2 > --library-path ..:. ./sha512c-test > > real 0m3.677s > user 0m2.281s > sys 0m0.008s > ----------------------------------------- > So, it does not look too scarry. Only the function does not return early > enough with the normal timeout. > Do you think I should fill a bug report ? No, I don't think so. TIMEOUTFACTOR was introduced for exactly this purpose. The default timeout is apparently 2 seconds, so you could maybe argue for the default to be increased. But I wouldn't like your chances.. I'm sure I read somewhere recently (kernel list probably) when it comes to Intel arch, Ulrich doesn't run any x86 testing boxes at all anymore (only x86_64) which probably explains why we continue to see math testsuite failures on x86. Regards Greg From diy-linux-dev@diy-linux.org Mon Nov 26 00:14:32 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Mon, 26 Nov 2007 10:14:32 +1100 Subject: Binutils --with-lib-path In-Reply-To: <20071115030948.GA16317@eyo32.local> References: <20071108034505.GA5228@eyo32.local> <20071115030948.GA16317@eyo32.local> Message-ID: <20071125231432.GA6858@eyo32.local> On Thu, Nov 15, 2007 at 02:09:48PM +1100, Greg Schafer wrote: > On Thu, Nov 08, 2007 at 02:45:05PM +1100, Greg Schafer wrote: > > > It's true that we need to prevent ld from finding libs on the host, but > > installing an adjusted linker and then repeating same in the Chroot phase is > > just nonsense. > > Not quite. The current build does actually need it because of the Chroot > phase build order ie: GCC before Binutils. However, the above holds true if > Binutils is built before GCC. Well, that statement was a load of rubbish (just like the one above it). I failed to take into account the "deps from DT_NEEDED" issue ie: when a shared lib depends on another shared lib, ld itself needs to find the dependent lib for the actual link without using -L directives. Bottom line - we definitely need to keep on using an adjusted linker. The reason I was trying to get rid of this is because of bi-arch x86_64. The problem is that `--with-lib-path' does not add the lib64 variant dirs to the ld SEARCH_DIR vars. CLFS have a patch for this but I do wonder why it hasn't been submitted upstream.. Not that it really matters as I've since determined that we can get away without a patch because we don't need to care at all whether the -m32 part of the bi-arch x86_64 toolchain is fully functional or not. Think about it - during the Temptools phase, we only care about the 32-bit Glibc and the 32-bit portions of the GCC build. We are not compiling any normal -m32 user code, so why bother with it? The only anomaly I've found with this approach is a failed configure test in 32-bit libgomp (it links to -lrt which then fails to find the dependent libpthread) but I don't think this is a problem for the final system (ICA will confirm it). Regards Greg From diy-linux-dev@diy-linux.org Mon Nov 26 07:07:48 2007 From: diy-linux-dev@diy-linux.org (Greg Schafer) Date: Mon, 26 Nov 2007 17:07:48 +1100 Subject: Next Generation build method Message-ID: <20071126060748.GA4765@eyo32.local> Hi Guys, I've added the new build method to the Refbuild as an alternative. It is still *far* from finished but it's all coming together nicely as I continue to work on it in my spare time. Having said that, x86 and pure x86_64 should work fine. There are still quite a few issues to work through on bi-arch x86_64. I've placed appropriate big fat warnings at the head of each of the new chapters. Overall, I'm quite pleased with the concept. And it really must be said, because this is just a variation of the current method, for the goal of building a bi-arch x86_64 system this thing blows CLFS out of the water in the simplicity and understandability stakes. But like I said earlier, CLFS clearly has vastly different goals. There are 2 main things I'm unhappy with at the moment. 1) the earlier mentioned inability of Binutils' `--with-lib-path' to add lib64 dirs to ld's search paths (not a major problem anyway) and 2) an apparent bug in the GCC driver multilib processing which I'm forced to work around by twiddling with STANDARD_STARTFILE_PREFIX_{1,2}. I'm out of my depth debugging non-trivial C code with gdb, so if there is anyone out there with these kind of skills, I could *REALLY* do with your help on this one as it would be nice to not need the workaround. Also, during the bi-arch Temptools phase some of the 64-bit libs end up in $TT_PFX/lib. This is untidy but the toolchain can still find'em, but I may end up fixing it. If building bi-arch x86_64, the core toolchain is working up to and including Chroot GCC, but I haven't yet addressed any lib64 issues beyond that. One other thing that's only recently dawned upon me, I need to add the -m32 portion of the GCC testsuite (which BTW, is missing from CLFS and therefore constitutes a gaping hole). Anyhoo, please look it all over and give feedback. Feel free to ask any questions about rationales etc. Regards Greg