From: UnixOS2 Archive To: "UnixOS2 Archive" Date: Sun, 18 Aug 2002 04:35:24 EST-10EDT,10,-1,0,7200,3,-1,0,7200,3600 Subject: [UnixOS2_Archive] No. 302 ************************************************** Saturday 17 August 2002 Number 302 ************************************************** Subjects for today 1 Some Ideas : Andreas Buening 2 Distribution formats : Jeff Robinson 3 Re: Some Ideas : Christian Hennecke" 4 Re: Some Ideas : Franz Bakan" 5 Re: Some Ideas : Andreas Buening 6 Re: Some Ideas : Andreas Buening **= Email 1 ==========================** Date: Sun, 18 Aug 2002 14:49:40 +0200 From: Andreas Buening Subject: Some Ideas Hello! It has been rather quiet here for a while. May be it's now a good time to start a discussion how the UnixOS/2 project should move forward. I guess, all of us want to make the UnixOS/2 project becoming a success. Therefore I thought about which problems may occur from the point we are now to the point we can release an ISO image of the first UnixOS/2 distribution. I'd like to see a lot of feedback. If you agree or disagree or if you have other suggestions, please tell. If you have a solution to one of the problems, please tell. If you want to help by doing one of the outstanding jobs, please tell, too. I apologize for the case some of the problems have already been addressed or even solved. 1. Source Codes First thing we need is an organization of the source code. Having a ftp server is only one part. A flat and simple directory structure for the source (and binary) packages is also necessary. If one searches for the sources of foo then he could find it easier in src/foo or src/gnu/foo than in os2/unix/devtools/foo because he might not know that foo is considered as an development tool. That directory structure has to contain all old versions. Further a simple and unique naming scheme like foo-3_14_15.zip is required. foo.zip doesn't contain any info about what's really in that package. Additionally every developer need access to "his" directories. He needs the permission to add, rename or remove files. And every developer needs ftp access even if all anonymous accounts are in use at the moment. If these requirements can't be met it might be suitable to start a UnixOS/2 project at SourceForge. summary: - powerful ftp server - flat and simple directory structure - simple and unique naming scheme - write permissions for developers - if this is not possible start project at SourceForge 2. How to build packages There isn't only the question which compiler flags/whatever are required to build a package, but also which configure/whatever options are required/ recommended/discouraged. We need a package build guide that contains all this info: compiler flags (CFLAGS, LDFLAGS, ...), env. vars. (UNIXROOT, ...), configure options (--prefix=xxx, --without-included-gettext, ...) and so on. All exceptions from this rule (e.g. perl) have to be listed on the UnixOS/2 website. This includes also the question _what_ has to be installed, e.g. do we want a static or a import library /usr/lib/foo.a, do we want gzipped man pages? OS/2 .INF files? And so on. The next question is _where_ has it to be installed, e.g. do we want grep or the file utilities in /bin or /usr/bin? /bin seems to be the default on most unix systems. What about docs and other stuff of these packages in /bin? Do we want their .info files also in /info and manpages in /man? I think a lot of people wouldn't like if UnixOS/2 would use lots of directories in their root directory (e.g. bin, lib, share, info, man, doc, libexec). Furthermore we need space for future extensions like a libc (I prefer /lib for the basic survival dll's like emx, intl, regex) or some UnixOS/2 related stuff (e.g. package management tools), e.g. /usr/unixos2. Yet another thing is the libc itself. Currently this means emx. There are several code fragments (like gettext or regex) that are part of glibc but not of emx. At least gettext is provided as a standalone library so that's no problem. Do we want those mini libraries like regex (there may be others) to be standalone libraries (which means we may need LIBS="-lregex" for a lot of packages or we have to change the packages themselves) or do we want them as part of the libc itself (then we need someone who builds them into emx)? There are a lot of Posix (or GNUish) functions that are not in emx. If one of those functions is used in a package then there is either a #ifdef __EMX__ that implements that function or the package doesn't provide a specific feature for OS/2. It's a complete waste of time to implement the same stuff ever and ever again for every single package. If one of those functions has been implemented once this implementation has to be put into one single library for general use. I would prefer to put them directly into emx than using LIBS="-lux2" or LIBS="-lposix2" for every package. This not a trial to revive the posix/2 or libext projects. What I want to say: If we add foo to the UnixOS/2 distro and foo requires a function (e.g. uname()) then this function has to be implemented. If this has already been done by the posix/2 or libext project or if Holger spends his LIBEMU implementation of the function then it's fine. Otherwise that function has to be rewritten from scratch (and also the according header file sys/utsname.h). summary: - standard build guide containing flags, env. vars, configure options - _what_ has to be installed (static versus import libraries, ...) - _where_ has it to be installed (/bin|/info versus /usr/bin|/usr/info) - unixos2 specific stuff into /usr/unixos2 - put all new Posix (or GNUish) functions into a ux2 (or better emx) library 3. Installation tools In order to make a "real" UnixOS/2 distro we need some tools for package management. First we have to decide which file format we want to use: a) zip (simple, everbody knows it, not as flexible as a "real" package installer) b) tar.gz (simple, less options than zip) c) rpm (more complicated, more options) d) something else (e.g. the corresponding debian tool?) As I don't have experience with rpm and others I'd really like to know what are its advantages and what are its disadvantages? Next we need some installation tools because we need tools with a simple syntax every user can use. I'm thinking about four tools that do the job: - ux2-install: syntax: ux2-install xyz Installs a package xyz.foo (foo may be the package extension .zip or .rpm) from the current directory into the $UNIXROOT tree and prepares the files for an update (e.g. if a file intl.dll is locked, it stores the new file as intl.dll.update). - ux2-update: syntax: ux2-update Performes the real update: may use the locked file system driver to update locked file, installs new .info files into the index file info/dir, compresses new installed manpages, creates OS/2 .INF files from whatever, and so on. This may require a reboot. Because ux2-install and ux2-update change the UnixOS/2 tree they must not rely on any of those programs or dlls. I suggest to use REXX for these tools. - ux2-check: syntax: ux2-check Checks whether the UnixOS/2 stuff is working properly. This should be a sh script. - ux2-package: syntax: ux2-package name_of_new_package Creates a binary package name_of_new_package.foo from the output of "make install" from a fresh prefix tree. This scripts knows which files have to be considered (e.g. /usr/info/dir must not be put into the package file). This can be either a sh or REXX script. How it works to create a (binary) package xyz: ./configure --prefix=c:/new_prefix make make install cd c:/new_prefix ux2-package xyz How it works to install a (binary) package xyz: ux2-install g:/download/xyz ... install other packages ... ux2-update reboot (if necessary) ux2-check summary: - which package format (zip/tar.gz/rpm) - install tools (preferebly REXX): ux2-install.cmd, ux2-update.cmd, ux2-package.cmd, ux2-check.sh 4. Docs Of course, we need docs for all that stuff above. 5. Package maintainance Last but not least is package maintainance. We need people who do that stuff. We need an distro maintainer (he has to keep track of the binary packages and to release the distro), somebody who maintains the website, somebody who writes and maintains the install tools, somebody who keeps track of all the library stuff and of course some people who make the packages ready for UnixOS/2. We need a priority list for packages that are important but have no active maintainer yet. Not all of them need to be hardcore programmers. If a package xyz once has been adapted to UnixOS/2 it is unlikely that there will be severe changes to that software in the near future. This means the job of the maintainer of xyz is basically to be on the xyz specific mailing list (if there is any) and to test beta and pre releases of xyz that are provided by the Unix maintainers of xyz. Any feedback would be really appreciated. :-) bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 2 ==========================** Date: Sun, 18 Aug 2002 16:55:05 -0500 From: Jeff Robinson Subject: Distribution formats One of the subjects that seems to come up every now and then is how the distribution should be packaged. There are proponents for and against the different methods, but maybe there doesn't have to be one be-all-end-all method. Seeing as some of the existing UnixOS2 packages have already been setup by John P. as .zip files, I think this could be a good start. I personally, prefer using something like WarpIN just to make for easy adding and removing of packges as well as validating the files on my system, to see if anything has changed unbeknownst to me. So instead of reinventing the wheel or trying to come up with something else I did what seemed like the next best thing. I developed a REXX script that will take the info in a UnixOS2 pkginfo file and build a (generally simple) WarpIN archive out of it. That way all the convenience of a .zip is still preserved (such as extraction of a single file) but having all the convenience of a WarpIN file... but the maintainers do not need to keep two separate archives of the same thing. Taking the repackaging concept one step further, as my first script seemed to be rather limited in what it can do, I sort've got my hooks into the BSD-style "ports", and grabbed ahold of the basic concept. I started creating my own pkginfo files (making them totally compatible with existing UnixOS2 files) that not only contain the details the current pkginfo files do, but also the location of master servers where these archives are, md5 checksums of the packages and some other info. The idea is that the user can have all these pkginfo files made available to them, and with a simple set of commands like: fetch sed pack sed ... the archive is grabbed from a master server and then repackaged as a WarpIN archive which is then (at the user's option) installed. The one thing I wish to be very cautious of in the whole bit of packaging is to try and make things easy for people that don't want to spend days just tinkering around with their system to get things in running order. Individual .zip files are great, but when I had the option between downloading, unpacking, and configuring Python from a .zip or using the WarpIN archive I chose the latter. This way, though, the original zip files are still there for everyone to use if they want so there isn't really any "lock-in" to a certain format. If anyone is interested in trying my 'ports' concept I can make it available. Jeff **= Email 3 ==========================** Date: Sun, 18 Aug 2002 21:37:31 +0200 (CEST) From: "Christian Hennecke" Subject: Re: Some Ideas On Sun, 18 Aug 2002 14:49:40 +0200, Andreas Buening wrote: >The next question is _where_ has it to be installed, e.g. do we want grep >or the file utilities in /bin or /usr/bin? /bin seems to be the default >on most unix systems. What about docs and other stuff of these packages in >/bin? Do we want their .info files also in /info and manpages in /man? >I think a lot of people wouldn't like if UnixOS/2 would use lots of >directories in their root directory (e.g. bin, lib, share, info, man, doc, >libexec). For the sake of keeping PATH and LIBPATH at an acceptable length that does not cause problems with pipes and such, I strongly vote for using the root directory of a drive. People can always set up a dedicated partition for the UnixOS2 distribution. >d) something else (e.g. the corresponding debian tool?) >As I don't have experience with rpm and others I'd really like to know >what are its advantages and what are its disadvantages? The advantages are a certain degree of control concerning revisions and dependencies on other packages. Something like Debian's package tool or the FreeBSD ports stuff would be nice, too. Christian Hennecke **= Email 4 ==========================** Date: Sun, 18 Aug 2002 21:53:49 +0200 (CEST) From: "Franz Bakan" Subject: Re: Some Ideas On Sun, 18 Aug 2002 14:49:40 +0200, Andreas Buening wrote: ... >1. Source Codes > >A flat and simple directory structure for >the source (and binary) packages is also necessary. If one searches >for the sources of foo then he could find it easier in src/foo or src/gnu/foo I vote for /src/foo ... >do we want a static or a import library /usr/lib/foo.a, /emx/lib/foo.a is also a possible place for the libs. So you dont have to use -L statements or extensions to LIBRARY_PATH >do we want gzipped man pages? yes >OS/2 .INF files? when possible. Is the a conversion tool man>inf? >The next question is _where_ has it to be installed, e.g. do we want grep >or the file utilities in /bin or /usr/bin? I vote for /usr/bin /usr/lib /usr/man and so on >3. Installation tools > >In order to make a "real" UnixOS/2 distro we need some tools for package >management. First we have to decide which file format we want to use: >a) zip (simple, everbody knows it, not as flexible as a "real" package > installer) >b) tar.gz (simple, less options than zip) From my point of view (as a user of fc/2) a simple zip-file which unpacks to /usr and a /readme.os2 plus some kind of /list_of_installed_files.foo is the best solution. This is the way most of the XFree86-apps are distributed now. Locked dll's can simply be unlocked with the unlock-tool included in lxlite. No need for reboots. Any kind of installers make things more complicated. >How it works to create a (binary) package xyz: >./configure --prefix=c:/new_prefix Please NEVER use a drive letter in configure --prefix= Because of this I had to manually patch my PERL-executables and dll's or my Window-manager wmaker.exe with some HEX-Editor before they were usable. >make >make install >cd c:/new_prefix >ux2-package xyz Easiest way for me seems to be something like: ./configure --prefix=/usr make ren usr usr.temp md /usr make install zip -r foo.zip /usr rm -rf usr ren usr.temp usr then add /readme.os2 to the the package Of course this only works if the tools needed for 'make install' are available outside of /usr. In /emx/bin etc. for example. This way works fine for packing /XFree86 applications. Perhaps this is possible with /usr apps, but I never tried it. Franz **= Email 5 ==========================** Date: Sun, 18 Aug 2002 23:01:55 +0200 From: Andreas Buening Subject: Re: Some Ideas IanM wrote: > [snip] > >requirements can't be met it might be suitable to start a UnixOS/2 project > >at SourceForge. > > If we can get space at SourceForge, that would be great. There are hundred (thousands?) of projects at SourceForge. There must be a way to get some space. ;-) > >- flat and simple directory structure > >- simple and unique naming scheme > > I'm starting a cleanup of os2site.com for unixported stuff at > http://www.os2site.com/sw/unixport/ > > The file naming system on unixos2.com, to be blunt, sucks ! > > This will be rearranged, and have a lot more added shortly > when I'm back home, then once I've figured out whats what > with UnixOS2.org, I'll try to merge the various packages etc > so there is one coherant directory structure, before I do this > I will ask John & the list about preferences. It shouldnt be to > hard to have two views of structures, ie, the UnixOS2 structure, > and a structure similiar to the os2site one (but cleaned up), or > even a totaly new idea put forward by list members ? Great. I think a top level structure like "sources", "binaries", "distro" and maybe "sources-alpha", "binaries-alpha" will make sense. The lowest level should be the package name like "grep" or "make". The question is whether we need a second directory level. The current debian distro consists of more than 8000 packages. Even if most of them are Linux specific or redundant we are talking about a huge number of possible packages. Therefore I tend to say that putting packages in (very few) different groups sound reasonably. E.g. keeping all packages from gnu.org in one gnu subdirectory and all UnixOS/2 specific tools into unixos2. At the beginning we will have to deal basically only with the well known GNU tools from gnu.org. This suggestion would lead to a directory structure like: /pub/sources/unixos2/tools package maintainance tools /pub/sources/unixos2/emx emx dlls /pub/sources/gnu/grep /pub/sources/gnu/make /pub/sources/gnu/... other tools from gnu.org /pub/sources/other/... do we currently have others? same structure for binaries: /pub/sources/unixos2/... /pub/sources/gnu/... However, I think we shouldn't put old or abandoned packages without an active maintainer into that structure. We'd better keep them in their own /pub/old/... directory tree. > >- write permissions for developers > > I can do this with the location of the present unixos2.com with > individual FTP logins Great. > but SourceForge sounds like a much > better idea, then we can have mirror sites. Personally I would prefer if we had our own ftp server. If our stuff were good enough we would be automatically mirrored. ;-) But if you are the ftp maintainer you may have to decide whether we need SourceForge or not. > >2. How to build packages [snip] > >The next question is _where_ has it to be installed, e.g. do we want grep > >or the file utilities in /bin or /usr/bin? /bin seems to be the default > >on most unix systems. What about docs and other stuff of these packages in > >/bin? Do we want their .info files also in /info and manpages in /man? > > I see no reason some directories cant be the same as OS/2, ie, > I leave the etc directory in x:/mptn/etc for convenience, and having > a seperate system and user bin directory is a moot point under > OS/2, as they dont have seperate privileges BUT it can add in > seperating dev binaries, and application binaries. I use x:/usr/bin, > , x:/usr/sbin, x:/usr/dll to unify my unixos2 stuff. > > I do put all info and man files under man, as its pointless having > the extra directory for this part. The basic idea of a distro is that you don't need to put single files somewhere. info is the default for .info files and man for man pages, everything else would require additional effort. Same is for dlls. Default install directory is lib, everything else needs extra maintainance. > >I think a lot of people wouldn't like if UnixOS/2 would use lots of > >directories in their root directory (e.g. bin, lib, share, info, man, doc, > >libexec). > > Which is why I like the idea of > > x:/usr > x:/usr/bin > x:/usr/sbin (emx and development binaries) > x:/usr/dll > x:/usr/man Yes, for applications, 100% ACK. I meant something else: Even Unix systems have the "basic survival kit" like sh, grep, cp, gzip in /bin. I think we should do this, too. Further, even this "basic survival kit" accounts for 100-200 executables (the file and shell utilities!). It makes sense to give them their own directory. This had also another advantage: By simply copying /bin and /lib (provided you have all the "basic survival library kit" there) to another computer you have a working Unix like environment without touching several 100 MB or even GB in /usr. Another question is whether we want also /man, /info, /share and so on. I think it would be doable with a reasonable maintainance effort (because these are only few packages) to put executables and libraries of this "basic survival kit" into /bin and /lib but their docs and other stuff into /usr/info, /usr/man and so on. [snip] > >3. Installation tools [snip] > >summary: > >- which package format (zip/tar.gz/rpm) > > zip (or WPI as it has full installation capabilities) We need full control over the archive, i.e. we must be able to add or remove or extract single files or exclude single files from being unpacked. When I last played with WPI it wasn't possible. Another thing: Do you want to click several thousand times with your mouse to install several hundred packages? In practice one would write one single script that calls ux2-install several hundred times. [snip] bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie. **= Email 6 ==========================** Date: Sun, 18 Aug 2002 23:27:43 +0200 From: Andreas Buening Subject: Re: Some Ideas Franz Bakan wrote: > > On Sun, 18 Aug 2002 14:49:40 +0200, Andreas Buening wrote: > > ... > >1. Source Codes > > > >A flat and simple directory structure for > >the source (and binary) packages is also necessary. If one searches > >for the sources of foo then he could find it easier in src/foo or src/gnu/foo > > I vote for /src/foo > > ... > > >do we want a static or a import library /usr/lib/foo.a, > > /emx/lib/foo.a > is also a possible place for the libs. So you dont have to use -L statements > or extensions to LIBRARY_PATH It's outside of the prefix directory, i.e. it needs extra maintainance but gives no real advantages. I think we'll have to put all those /usr/{bin|lib|include|man|info} and /usr/X11R6/... and maybe also /{bin|lib|...} into PATH, LIBPATH, C_INCLUDE_PATH, LIBRARY_PATH, MANPATH, and so on. [snip] > >The next question is _where_ has it to be installed, e.g. do we want grep > >or the file utilities in /bin or /usr/bin? > > I vote for > /usr/bin > /usr/lib > /usr/man > and so on As I told in another posting it sounds reasonably to use at least also /bin and /lib (and of course the /usr/X11R6 tree for X11). > >3. Installation tools > > > >In order to make a "real" UnixOS/2 distro we need some tools for package > >management. First we have to decide which file format we want to use: > >a) zip (simple, everbody knows it, not as flexible as a "real" package > > installer) > >b) tar.gz (simple, less options than zip) > > >From my point of view (as a user of fc/2) a simple zip-file which unpacks to > /usr > and a /readme.os2 plus some kind of > /list_of_installed_files.foo > is the best solution. > > This is the way most of the XFree86-apps are distributed now. At the beginning, yes. However, on the long term we might want to have some version or package management which is difficult (but doable) with simple .zip files. > Locked dll's can simply be unlocked with > the unlock-tool included in lxlite. No need for reboots. Okay, for dlls this would work. But I would prefer a more failure safe solution on the long term. In the worst case somebody has opened a file in a DOS editor for writing. > Any kind of installers make things more complicated. Maybe. Maybe not. I don't know the real capabilities of rpm or the debian installer. > >How it works to create a (binary) package xyz: > >./configure --prefix=c:/new_prefix > > Please NEVER use a drive letter in configure --prefix= > Because of this I had to manually patch my PERL-executables and dll's or > my Window-manager wmaker.exe with some HEX-Editor before they were usable. It depends. The program has to consider $UNIXROOT. If it doesn't any prefix will lead to problems. Therefore we need a build guide which one has to be the default prefix (this might be c:/usr or /usr) but also a list of all packages that need another prefix. > >make > >make install > >cd c:/new_prefix > >ux2-package xyz > > Easiest way for me seems to be something like: > > ./configure --prefix=/usr > > make > ren usr usr.temp Works only if you have no locked files or directories in /usr. > md /usr > make install > zip -r foo.zip /usr > rm -rf usr > ren usr.temp usr > > then add /readme.os2 to the the package > > Of course this only works if the tools needed for 'make install' > are available outside of /usr. In /emx/bin etc. for example. Okay, my prefix was wrong: ./configure --prefix=c:/usr make make install prefix=c:/new_prefix cd c:/new_prefix ux2-package xyz [snip] bye, Andreas -- One OS to rule them all, One OS to find them, One OS to bring them all and in the darkness bind them In the Land of Mordor where the Shadows lie.