From: UnixOS2 Archive To: "UnixOS2 Archive" Date: Fri, 22 Nov 2002 04:41:08 EST-10EDT,10,-1,0,7200,3,-1,0,7200,3600 Subject: [UnixOS2_Archive] No. 376 ************************************************** Thursday 21 November 2002 Number 376 ************************************************** Subjects for today 1 Re: config.site : Andreas Buening 2 Re: REGEX.DLL : Andreas Buening 3 Re: installpkg : Andreas Buening 4 Re: PKGINFO (was: installpkg) : Andreas Buening 5 Re: installpkg : Sebastian Wittmeier (ShadoW)" 6 Re: PKGINFO (was: installpkg) : Sebastian Wittmeier (ShadoW)" 7 Re: gzip/tar max path length : Thomas Dickey 8 Re: gzip/tar max path length : John Poltorak 9 Re: gzip/tar max path length : Lyn St George" 10 Re: gzip/tar max path length : Dan Drake"
11 Re: REGEX.DLL : John Poltorak 12 Re: REGEX.DLL : Andreas Buening 13 Re: installpkg : Andreas Buening **= Email 1 ==========================** Date: Fri, 22 Nov 2002 00:03:21 +0100 From: Andreas Buening Subject: Re: config.site Lyn St George wrote: > > On Tue, 19 Nov 2002 22:01:02 +0100, Andreas Buening wrote: [config.site] > Is there something that can be added into config.site so as get > around that annoying "cannot create while cross-compiling" > error? Ie. to tell the config file that gcc is *not* a cross-compiler? config.site works like a config.cache file. This means you can use this file to tell configure that you have a function foo() or a header file foo.h or a -lfoo library. However, I don't know what causes your error. In principle, it should be possible to turn off cross compilation ($host, $host_alias?), but I have no idea, how. Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 2 ==========================** Date: Fri, 22 Nov 2002 00:03:37 +0100 From: Andreas Buening Subject: Re: REGEX.DLL John Poltorak wrote: > > What is the current preferred method of building regex.dll and where do I > get the source for it? I moved the regex code into unixos2.dll to keep the number of additional libraries at a low level. However, I guess, the latest changes of the new GNU regex can be found in the glibc CVS. ;-) Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 3 ==========================** Date: Fri, 22 Nov 2002 00:08:25 +0100 From: Andreas Buening Subject: Re: installpkg Michael Zolk wrote: > > On Sun, Nov 17, 2002 at 01:25:56AM +0100, Andreas Buening wrote: > > > The idea of /bin, /lib and /sbin is that _all_ stuff that is > > necessary for the installation has to be there. If you really need > > another rexx script for the installation we must add it to /bin > > or /sbin. > > Yes. According to the FHS, such "internal binaries" should be placed in > /usr/lib/ (or /lib/ in this case), but this would require another > addition to PATH. So the right place would indeed be /bin or /sbin. No, only /bin is in PATH. All essential binaries and scripts that are of "general interest" for the user like installpkg.cmd itself belong to /bin and /bin is in PATH. All essential binaries and scripts that are NOT of "general interest" for the user (but maybe for the admin) belong to /sbin (Currently, /sbin is empty) and /sbin is NOT in PATH. If you have some special installation scripts that are necessary to activate a certain package put them into /var/lib. If the same script is required by two or three packages put it into each of them (-> /var/lib/unixos2/scripts/package1-foo.cmd, /var/lib/unixos2/scripts/package2-foo.cmd). But if it's required by several packages it's better to put that script into /sbin. > [ install scripts that require additional packages ] > > > > This is indeed a problem. I don't know how all the Linux distros handle > > > this. > > > > Hmm, good question. Does anybody know? > > I know that the Debian policy requires that all programs that are marked > 'Essential' must work properly even without configuration. We can require the same. However, I think at least postinstall scripts won't be a problem. > > > This is the reason why all the scripts in ux2_base are Rexx scripts - > > > they only need the stuff that's already present on an OS/2 system. I'm not > > > sure if we should simply define that install scripts _have to be_ Rexx > > > scripts. > > > > No, not in general. My idea for the installation process is as follows: > > > > 0) - Change OS/2 variables (PATH, LIBPATH, ...) > > - Add Unix/Posix variables (HOME, LOGNAME, ...) > > - Add UnixOS/2 variables (UNIXROOT, TMPDIR) > > - Reboot if necessary > > All this is done by unzipping the ux2_base package and running doinst.cmd. How safe is doinst.cmd? E.g. if anbody installs the UnixOS/2 stuff several times will the directories be added several times to config.sys? Can the backup config.sys accidentally be deleted? Is it guaranteed that those paths are available when the first install scripts are executed? The installation procedure must be bulletproof. In doubt, we can leave this issue (still) to the user because adding some variables once isn't that difficult. > > 1) installpkg puts the UnixOS/2 core packages (/bin and /lib) > > in place. > > All installation scripts and file lists are stored > > somewhere in /var without executing them. After this has been > > done we have all necessary installation tools available > > (That's the definition of /bin and /lib: necessary for installation > > and maintainance). > > > 2) Run "ux2-update" the first time which a) executes those scripts > > and b) updates some databases/whatever. > > > 3) Now the base system is working and it's possible to install > > any number of other packages by installpkg and run "ux2-update" > > again. > > Hmm... the more I think about it... :) The only Linux distro that I *really* > know good enough, Debian, seems to do it in a similar way. First all packages > are unpacked, when the files are in place the packages are configured. > > It would only be necessary to delay the configuration for all the stuff > that's in /bin, for everything else ux2-update could be called directly by > installpkg. Yes. If and only if the essential packages have already been installed. > > Because of that all install scripts can rely on every tool that > > is in /bin. I.e. sh or even sed scripts are allowed but perl scripts > > are not (because perl is in /usr/bin). > > This set of "essential" packages should be kept as small as possible. > Furthermore, it's important to choose an install method that makes life > easier for package maintainers. It would be good to get some more input > on this issue. [snip] > > SCRIPT {PREINST|POSTINST|POSTDEL} scriptname > > At least the POSTINST scripts can be sh scripts. Where are these > > scripts stored? How do you avoid name collisions? > > The scripts are copied to /var/lib/unixos2/scripts/. The scripts there > are named .postinst. The at the end is used so > that it's possible to have more than one script of the same type. What happens if a package has two scripts (foo1, foo2) and foo1 runs foo2? Would it be possible to use ..postinstall instead so that the script gets a fixed name? (and an error if there are two scripts with the same name) [snip] > > FILE pathname [owner group mode] > > Adding hundreds of file to PKGINFO could be very time consuming. > > What about "FILE ALL" which installs all files automatically? > > It's good to have some redundant information like this in the PKGINFO file. > It could be used, among other things, to detect corrupted packages. > > Of course it would be good to have a 'mkpkginfo' command that adds all the > files in a directory tree to a PKGINFO file. The problem is it won't be possible to create PKGINFO file for large packages without a "mkpkginfo" if we don't have "FILE ALL". It might be possible to read the zip file content by "unzip -Z -1" (e.g. into a queue). This command prints all file names to stdout. > > What is the sense of "owner"? Currently, we have no file "owner". > > If we had this feature it would be possible to install files for > > another user (e.g. "root") but after that we might have no access > > to these files anymore, i.e. we wouldn't even be able to uninstall > > them. > > It has been discussed some time ago, that Holger's libemu could be able > to emulate things like file owners, symlinks, device nodes IIRC. At the > moment this is not working, it's mainly there for future updates :) As long as we don't have those features it has no effect but if this ever happens we have a problem: Assume a file is installed for a user "joe". There might be no such user on the system (unless "joe" is "root"), the current user (i.e. you) may have no access to "joe"'s files, i.e. you might be able to install these files but the postinstall scripts may fail because they can't even modify a config file. The symlink feature make sense. It allows to reduce the size of some packages, e.g. create a "symlink" awk.exe for gawk.exe and you save several 100 K in the zip file. The device feature is misleading because it has just no effect. Btw. do we want to have another file extension for the package files instead of ".zip"? This would allow to use wps associations for that file extension. > > How do I specify "mode"? E.g. how to produce a readonly file? > > Access permissions are specified using the 3-digit octal notation known > from Unix. Ouch. ;-) > > What about a "NOINSTALL file" keyword. A package might contain > > some files that shouldn't be installed (a COPYING file or a special > > README). PKGINFO has the "NOINSTALL" attribute by default. > > All files that are not listed in one of the "FILE ..." lines have > this attribute. [snip] > > CONFFILE pathname [owner group mode] > > What exactly is the difference between CONFFILE and FILE? > > Will the user be prompted for every CONFFILE as if it were readonly? > > That's the plan :) The keyword CONFFILE should be used for configuration > files that can be modified by the user. installpkg should avoid to simply > overwrite these files when possibly the user has invested lots of time to > fiddle with his configuration. What happens if the whole distro with dozens of config files is installed at the same time? Is there an option to overwrite all config files (-f option?) or to rename the old ones to *.backup? Pressing "y" or "n" several hundred times can become very annoying. ;-) Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 4 ==========================** Date: Fri, 22 Nov 2002 00:09:30 +0100 From: Andreas Buening Subject: Re: PKGINFO (was: installpkg) Michael Zolk wrote: > > On Sun, Nov 17, 2002 at 01:25:56AM +0100, Andreas Buening wrote: [snip] > > VERSION version-string > > Make it mandatory. It would cause the maintainers to use a version > > number. Installing an old version over a new one could cause an error. > > The problem here is that there are so many different formats the version > string can have so that I don't know how to compare version numbers. Debian > can do it somehow, but I don't know how. I have seen version strings like > "962", "0.1", "1.2PL2", "2.4.19", "20011018" etc. What about a lexical comparison? "abc" is less then "abd" and "1.2.3" is less then "1.2.4". [snip] > > DESC description > > Is this info printed? A message like "Do not install this package > > if ..." might be helpful. If this is the case then a comment that > > is not printed (REM?) might helpful, too. > > ? E.g. things that had better be printed: DESC This is a beta release DO NOT INSTALL unless you really know DESC what you are doing! Things that had better not be printed: REM This package was compiled with -lfoo --enable-bar REM Do not use -lbla. It screws up the dll in combination REM with --enable-bar. [snip] Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 5 ==========================** Date: Fri, 22 Nov 2002 03:57:26 +0100 (CET) From: "Sebastian Wittmeier (ShadoW)" Subject: Re: installpkg On Fri, 22 Nov 2002 00:08:25 +0100, Andreas Buening wrote: >Btw. do we want to have another file extension for the package >files instead of ".zip"? This would allow to use wps associations >for that file extension. Just a note: You don't need a new extension for wps associations. They work with every kind of filter. For example: uxos2*.zip, or ????xyz*.rh* Sebastian **= Email 6 ==========================** Date: Fri, 22 Nov 2002 04:02:05 +0100 (CET) From: "Sebastian Wittmeier (ShadoW)" Subject: Re: PKGINFO (was: installpkg) On Fri, 22 Nov 2002 00:09:30 +0100, Andreas Buening wrote: >> The problem here is that there are so many different formats the version >> string can have so that I don't know how to compare version numbers. Debian >> can do it somehow, but I don't know how. I have seen version strings like >> "962", "0.1", "1.2PL2", "2.4.19", "20011018" etc. >What about a lexical comparison? "abc" is less then "abd" >and "1.2.3" is less then "1.2.4". is not enough: 10.0 is newer than 2.0. So you need to separately compare the parts between periods. But what about 1.2PL2? Should it be translated into "1.2,PL,2"? (That means: insert "," between letters and numbers.) With "." the first level of comparision, and "," the second? Sebastian **= Email 7 ==========================** Date: Fri, 22 Nov 2002 07:24:00 -0500 From: Thomas Dickey Subject: Re: gzip/tar max path length On Thu, Nov 21, 2002 at 10:43:31PM -0600, xyzyx wrote: > But, being strictly POSIX, the limit was still 100 characters. For > various other purposes, GNU tar used areas left unassigned in the POSIX > draft. POSIX later revised P1003.1 ustar format by assigning previously > unused header fields, in such a way that the upper limit for file name > length was raised to 256 characters. However, the actual POSIX limit > oscillates between 100 and 256, depending on the precise location of > slashes in full file name (this is rather ugly). Since GNU tar use the > same fields for quite other purposes, it became incompatible with the > latest POSIX standards. > > For longer or non-fitting file names, we plan to use yet another set of note that the "plan" dates back to around 1996. -- Thomas E. Dickey http://invisible-island.net ftp://invisible-island.net **= Email 8 ==========================** Date: Fri, 22 Nov 2002 11:11:57 +0000 From: John Poltorak Subject: Re: gzip/tar max path length On Thu, Nov 21, 2002 at 10:43:31PM -0600, xyzyx wrote: > On Thu, 21 Nov 2002 14:11:44 +0000, John Poltorak wrote: > > >On Thu, Nov 21, 2002 at 11:52:17AM +0000, Lyn St George wrote: > > > >> If I unpack the tarball on Linux, zip it up and then unzip here on OS/2, > >> all is OK. My remote gzip is 1.2.4, GNUtar 1.13.17, local ones on OS/2 > >> are 'gzip 1.2.4' and 'tar 1.10 - AK 2.58'. Hmm, maybe I'll look at the > >> options that Paul mentioned. If you want to see this John, then the tarball > >> is any of those at ftp://ftp.icdevgroup.org/pub/interchange/nightly_build > > > >I've had a look and the problem seems to be related to occurences of:- > > > >././ at LongLink > > > >After UNGZIPing the compressed tarball, the tar file has a lot of files > >with shortened names and they are immediately prefixed with the at LongLink > >label, which I've never previously encountered. > > I think the culprit is the TAR being used to CREATE the archive, not > the one being used to extract it. But it sounded as though it was possible to extract it correctly on Linux... If so, then there must be sufficient information held somewhere in the tar file to be able to extract the full name but OS/2's TAR cannot process this info. Without having a Linux system handy I can't confirm this. > regards, > paul -- John **= Email 9 ==========================** Date: Fri, 22 Nov 2002 11:57:21 +0000 From: "Lyn St George" Subject: Re: gzip/tar max path length On Fri, 22 Nov 2002 11:11:57 +0000, John Poltorak wrote: >On Thu, Nov 21, 2002 at 10:43:31PM -0600, xyzyx wrote: >> On Thu, 21 Nov 2002 14:11:44 +0000, John Poltorak wrote: >> >> >On Thu, Nov 21, 2002 at 11:52:17AM +0000, Lyn St George wrote: >> > >> >> If I unpack the tarball on Linux, zip it up and then unzip here on OS/2, >> >> all is OK. My remote gzip is 1.2.4, GNUtar 1.13.17, local ones on OS/2 >> >> are 'gzip 1.2.4' and 'tar 1.10 - AK 2.58'. Hmm, maybe I'll look at the >> >> options that Paul mentioned. If you want to see this John, then the tarball >> >> is any of those at ftp://ftp.icdevgroup.org/pub/interchange/nightly_build >> > >> >I've had a look and the problem seems to be related to occurences of:- >> > >> >././ at LongLink >> > >> >After UNGZIPing the compressed tarball, the tar file has a lot of files >> >with shortened names and they are immediately prefixed with the at LongLink >> >label, which I've never previously encountered. >> >> I think the culprit is the TAR being used to CREATE the archive, not >> the one being used to extract it. > >But it sounded as though it was possible to extract it correctly on >Linux... If so, then there must be sufficient information held somewhere >in the tar file to be able to extract the full name but OS/2's TAR cannot >process this info. > >Without having a Linux system handy I can't confirm this. Just ran another check on this. Unzipped the tarball to a *.tar on Linux and then looked inside for these at LongLink files - none visible. Copied this *tar down to this OS/2 box, again looked inside, and it's now full of at LongLink files. As tar is trying to be backwards compatible, it may be possible that it always creates the at LongLink files to hold file info greater than 100 characters, but only lists or writes these out if the tar being used cannot write the full path/file names. Or it only creates the at LongLink files whenever it realizes that it won't be able to unpack the full path/file name. Either way, it's a nuisance. The source code is in GTAR258, but as Andy MacIntyre has mentioned that he is looking into star (which can handle path/file names of 1024 characters) then I would rather offer to help out with testing or whatever on that one. >> regards, >> paul > > >-- >John > - Cheers Lyn St George +--------------------------------------------------------------------------------- + http://www.zolotek.net .. eCommerce hosting, consulting + http://www.os2docs.org .. some 'How To' stuff ... +---------------------------------------------------------------------------------- **= Email 10 ==========================** Date: Fri, 22 Nov 2002 12:23:55 -0800 (PST) From: "Dan Drake"
Subject: Re: gzip/tar max path length On Fri, 22 Nov 2002 07:24:00 -0500, Thomas Dickey wrote: >On Thu, Nov 21, 2002 at 10:43:31PM -0600, xyzyx wrote: > >> But, being strictly POSIX, the limit was still 100 characters. For >> various other purposes, GNU tar used areas left unassigned in the POSIX >> draft. POSIX later revised P1003.1 ustar format by assigning previously >> unused header fields, in such a way that the upper limit for file name >> length was raised to 256 characters. However, the actual POSIX limit >> oscillates between 100 and 256, depending on the precise location of >> slashes in full file name (this is rather ugly). Since GNU tar use the >> same fields for quite other purposes, it became incompatible with the >> latest POSIX standards. >> >> For longer or non-fitting file names, we plan to use yet another set of > >note that the "plan" dates back to around 1996. Interesting, though. I didn't know about that variability in Posix max name lengths. What I do know, and I hope is taken into account if anyone manages to work on TAR, is there are even weirder problems. Using GTAK a few years ago, I found the mAngled stuff to be hopelessly unfunctional, so I moved to Posix format for my backups. Seemed fine for a long time. But finally it tripped me. Reloading the Desktop directory (this is with the last version of GTAK), the program will crash at unpredictable times. The problem is with certain nested folders with disgustingly stupidly long names, largely those created by Netscape and Java installations. Some of these are well within 256 bytes, but at some point *after* loading the folder correctly, Tar crashes. I'd assume this is on a tape-buffer boundary or something. Editing the QFA file to get rid of these directories will allow the load to proceed without error. Finding out what to delete is an infuriatingly slow blind iterative process. THe funny thing is, it probably wouldn't be terribly hard to debug this, at least enough to stop the crashes if not to cure the underlying problems. That is, if it were possible to build the executable, which it isn't. -- Dan Drake dd at dandrake.com http://www.dandrake.com/index.html If I knew then what I know now, I would have said, 'I don't recall.' --Frank Doyle, FBI agent, testifying under oath about his previous deposition under oath. **= Email 11 ==========================** Date: Fri, 22 Nov 2002 22:33:28 +0000 From: John Poltorak Subject: Re: REGEX.DLL On Fri, Nov 22, 2002 at 10:55:46PM +0100, Andreas Buening wrote: > John Poltorak wrote: > > > > On Fri, Nov 22, 2002 at 12:03:37AM +0100, Andreas Buening wrote: > > > John Poltorak wrote: > > > > > > > > What is the current preferred method of building regex.dll and where do I > > > > get the source for it? > > > > > > I moved the regex code into unixos2.dll to keep the number > > > of additional libraries at a low level. > > > However, I guess, the latest changes of the new GNU regex > > > can be found in the glibc CVS. ;-) > > > > I thought you were putting together a small libary of routines for use > > with UnixOS/2... > > Yes. I see no contradiction. All I am trying to do is build it for myself, and ensure that anyone is able to do so using a standard set of tools, rather than something which is unique to an individual developer... > > I would like to put together some scripts for building some basic apps, > > and don't think it justifies grabbing glibc which is around 70MB or more > > when you only want around 10kB. > > Sure. If you just want to _use_ the regex functionality you > can use unixos2.dll. Is the source for this available? > Bye, > Andreas > > -- > One OS to rule them all, One OS to find them, > One OS to bring them all and in the darkness bind them > In the Land of Mordor where the Shadows lie. -- John **= Email 12 ==========================** Date: Fri, 22 Nov 2002 22:55:46 +0100 From: Andreas Buening Subject: Re: REGEX.DLL John Poltorak wrote: > > On Fri, Nov 22, 2002 at 12:03:37AM +0100, Andreas Buening wrote: > > John Poltorak wrote: > > > > > > What is the current preferred method of building regex.dll and where do I > > > get the source for it? > > > > I moved the regex code into unixos2.dll to keep the number > > of additional libraries at a low level. > > However, I guess, the latest changes of the new GNU regex > > can be found in the glibc CVS. ;-) > > I thought you were putting together a small libary of routines for use > with UnixOS/2... Yes. I see no contradiction. > I would like to put together some scripts for building some basic apps, > and don't think it justifies grabbing glibc which is around 70MB or more > when you only want around 10kB. Sure. If you just want to _use_ the regex functionality you can use unixos2.dll. Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 13 ==========================** Date: Fri, 22 Nov 2002 22:56:04 +0100 From: Andreas Buening Subject: Re: installpkg Sebastian Wittmeier (ShadoW) wrote: > > On Fri, 22 Nov 2002 00:08:25 +0100, Andreas Buening wrote: > > >Btw. do we want to have another file extension for the package > >files instead of ".zip"? This would allow to use wps associations > >for that file extension. > > Just a note: You don't need a new extension for wps associations. > They work with every kind of filter. > > For example: uxos2*.zip, or ????xyz*.rh* If I understood correctly we're going to use just *.zip without any other prefix. You can't have two associations (unzip, installpkg) for the same extension. (Okay you can have but only the first one works with double click). Bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie.