From: UnixOS2 Archive To: "UnixOS2 Archive" Date: Tue, 17 Jun 2003 14:07:06 EST-10EDT,10,-1,0,7200,3,-1,0,7200,3600 Subject: [UnixOS2_Archive] No. 130 ************************************************** Monday 16 June 2003 Number 130 ************************************************** Subjects for today 1 libc (was: Re: [Ux2bs] baseline .. some thoughts) : Adrian Gschwend" 2 Re: libc : Stefan Neis 3 Re: libc : Adrian Gschwend" 4 Re: [EMX] Re: libc : Holger Veit 5 Re: libc : Andreas Buening 6 Re: libc : Knut St. Osmundsen" **= Email 1 ==========================** Date: Tue, 17 Jun 2003 17:45:52 +0200 (CDT) From: "Adrian Gschwend" Subject: libc (was: Re: [Ux2bs] baseline .. some thoughts) On Tue, 17 Jun 2003 17:17:49 +0200 (CEST), Stefan.Neis at t-online.de wrote: >BTW, anybody knows what exactly InnoTek is doing for >"their" gcc-port in that respect? While I like the idea >of a CVS for libc, I'd also insist on having _ONE_ libc. >So we should maybe contact those people as well. And >Holger also isn't subsribed to this list, I suppose... > >Maybe moving that discussion to UnixOS2 or EMX list would >be more appropriate... I vote for UnixOS2 list. Just some comments from Achim (taken from #netlabs irc, btw hint hint, IRC is a *good* way to communicate :-) well, our libc is done to fulfill our needs it is incompatible with emx (note by ktk: due to licensing problems with EMX) our GCC is targeted to be a VAC/Watcom replacement not a Unix porting environment just wait for our first distribution then we can discuss how to combine the various efforts For Unixos2-only people: Henry proposed to create a CVS for libc, now the question is what do we want to take as base (EMX, LIBEMU and Innotek-GCC libc is in dicussion, there is also one from IBM btw) It would be nice if we really could discuss that all together and agree on a sollution that is fine for everyone and allows enhancements in the future (no dead end). cu Adrian -- Adrian Gschwend at netlabs.org ktk [a t] netlabs.org ------- Free Software for OS/2 and eCS http://www.netlabs.org **= Email 2 ==========================** Date: Tue, 17 Jun 2003 18:49:52 +0200 (CEST) From: Stefan Neis Subject: Re: libc On Tue, 17 Jun 2003, Adrian Gschwend wrote: > >Maybe moving that discussion to UnixOS2 or EMX list would > >be more appropriate... > > I vote for UnixOS2 list. In view of Ilya's recent post to EMX list > Can't we find a new volunteer for maintainance of EMX.DLL and > EMXCRTM.DLL? The job is to *collect* contributed patches, weed out > the weeds, and decide when we have a snapshot good enough to be named > golded. I decided to also involve emx mailing list into this discussion... I was asking about InnoTek and their gcc (and its libc), Adrian replied: > Just some comments from Achim (taken from #netlabs irc, btw hint hint, > IRC is a *good* way to communicate :-) I like the asynchronity (??) of mail. Everyone can read/reply when he can spare some minutes. ;-) > well, our libc is done to fulfill our needs > it is incompatible with emx (note by ktk: due to licensing > problems with EMX) Huh? Which ones? I always had the impression that EMX's licence is rather permissive... > our GCC is targeted to be a VAC/Watcom replacement not a Unix > porting environment > just wait for our first distribution then we can discuss how > to combine the various efforts To be honest, that doesn't sound to promising from the porting point of view. There are some things that _have_ to be deep inside libc, otherwise e.g. closing sockets by close instead of closesocket or pthread support won't work... :-( > For Unixos2-only people: Henry proposed to create a CVS for libc, now > the question is what do we want to take as base (EMX, LIBEMU and > Innotek-GCC libc is in dicussion, there is also one from IBM btw) I think we should start with EMX, add Posix/2, some link support, pthreads, libintl and maybe some other nice little extensions and then start making it all work together and fixing bugs (e.g. pthread library and library for symlink support (via EA) both modify certain functions for accessing files - BTW, large file support is the third candidate which "wants" specific changes in that same area). Whether the CVS is rooted at netlabs or sourceforge (we could start by abusing the Posix/2 CVS) is rather irrelevant to me, however it would be great to automatically send all CVS commits (as a diff) to some dedicated mailing list (I know that wxWindows did such a thing also while the CVS was hosted by sourceforge, so it's possible to set it up like this even there - but I don't know, how to do it), so whoever is interested has a chance to check things easily. The biggest problem for starting however is probably the build system: EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want I don't know what etc. Unifying this and "porting" the Makefiles to some current GNUmake is going to require much time for not much to gain... :-( Ideally, we would also stay backward compatible, which implies keeping the right ordering for the already existing symbols in the DLL's. (though I'd suggest to make the import libraries for the new DLL's using "import by name"). OTOH, I _suppose_ large file support if put into the "normal" C routines is going to cause incompatibilities anyway (file positions no longer fit into an int, do they?), so maybe it would even be a good idea to be intentionally incompatible (old EMX DLL's handle small files, in the new runtime, everything would be large file enabled - Hmm, I'll have to get a new harddisk to create a large enough JFS partition for testing...). > It would be nice if we really could discuss that all together and agree > on a sollution that is fine for everyone and allows enhancements in the > future (no dead end). What I don't know at all is, how this could fit into Holger's long term plans. Ideally, replacing a couple of low level C routines by his more complete/efficient implementations - once they are ready - would be sufficient, but for this, it would be a big plus to have a rather detailed list of what functions those are going to be, I suppose. E.g. an improved link implementation is going to affect some other file related functions ((f)open, (f)stat, ...), but if we know this in advance, we can have some internal function like __dereference_link that's called by all those, so the changes from a temporary E/A based link emulation to "real" links are minimized... Regards, Stefan -- Micro$oft is not an answer. It is a question. The answer is 'no'. **= Email 3 ==========================** Date: Tue, 17 Jun 2003 20:39:16 +0200 (CDT) From: "Adrian Gschwend" Subject: Re: libc On Tue, 17 Jun 2003 18:49:52 +0200 (CEST), Stefan Neis wrote: >In view of Ilya's recent post to EMX list >> Can't we find a new volunteer for maintainance of EMX.DLL and >> EMXCRTM.DLL? The job is to *collect* contributed patches, weed out >> the weeds, and decide when we have a snapshot good enough to be named >> golded. >I decided to also involve emx mailing list into this discussion... ok, I'm not subscribed to the EMX list so it's possible that my posting won't make it there. >I like the asynchronity (??) of mail. Everyone can read/reply when he can >spare some minutes. ;-) yes, fine for a lot of things but it would still be nice if at least some people could show up on our #unixos2 channel. Would help in some cases too :) As a reminder: Server is irc.anduin.net, Mozilla got an IRC client built in. >> well, our libc is done to fulfill our needs >> it is incompatible with emx (note by ktk: due to licensing >> problems with EMX) > >Huh? Which ones? I always had the impression that EMX's licence is rather >permissive... Well I won't go into details, some Innotek people should answer that ( Knut promised me to go through the mails later) > >To be honest, that doesn't sound to promising from the porting point of >view. There are some things that _have_ to be deep inside libc, otherwise >e.g. closing sockets by close instead of closesocket or pthread support >won't work... :-( I'm sure they will help us if we talk to each other. >I think we should start with EMX, add Posix/2, some link support, >pthreads, libintl and maybe some other nice little extensions and then >start making it all work together and fixing bugs (e.g. pthread library >and library for symlink support (via EA) both modify certain functions >for accessing files - BTW, large file support is the third candidate which >"wants" specific changes in that same area). fine for me. >Whether the CVS is rooted at netlabs or sourceforge (we could start by >abusing the Posix/2 CVS) is rather irrelevant to me, however it would be >great to automatically send all CVS commits (as a diff) to some dedicated >mailing list (I know that wxWindows did such a thing also while the CVS >was hosted by sourceforge, so it's possible to set it up like this even >there - but I don't know, how to do it), so whoever is interested has a >chance to check things easily. I can't provide the diff, so if you want that sourceforge is the server. But to be honest I don't see the sense in that if we use CVS anyway. But the cvs server is not really an important question as long as it works :) >The biggest problem for starting however is probably the build system: >EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want >I don't know what etc. Unifying this and "porting" the Makefiles to some >current GNUmake is going to require much time for not much to gain... :-( ok, can't comment that due the lack of know how in this area. But IMHO it needs to be done, will help everyone at the end. >What I don't know at all is, how this could fit into Holger's long term >plans. Ideally, replacing a couple of low level C routines by his more >complete/efficient implementations - once they are ready - would be >sufficient, but for this, it would be a big plus to have a rather >detailed list of what functions those are going to be, I suppose. ACK, probably Holger can give some details about that? cu Adrian -- Adrian Gschwend at netlabs.org ktk [a t] netlabs.org ------- Free Software for OS/2 and eCS http://www.netlabs.org **= Email 4 ==========================** Date: Tue, 17 Jun 2003 21:08:14 +0200 From: Holger Veit Subject: Re: [EMX] Re: libc On Tue, Jun 17, 2003 at 06:49:52PM +0200, Stefan Neis wrote: > On Tue, 17 Jun 2003, Adrian Gschwend wrote: [...] > I like the asynchronity (??) of mail. Everyone can read/reply when he can > spare some minutes. ;-) I prefer mailinglists over IRC as well for the reason of asynchronicity. > > well, our libc is done to fulfill our needs > > it is incompatible with emx (note by ktk: due to licensing > > problems with EMX) > > Huh? Which ones? I always had the impression that EMX's licence is rather > permissive... The old GPL debate. GPL is not free, it is rather restricted. Too restricted to be acceptable for people who need tailored version of the software. > > our GCC is targeted to be a VAC/Watcom replacement not a Unix > > porting environment > > just wait for our first distribution then we can discuss how > > to combine the various efforts > > To be honest, that doesn't sound to promising from the porting point of > view. There are some things that _have_ to be deep inside libc, otherwise > e.g. closing sockets by close instead of closesocket or pthread support > won't work... :-( I don't see a problem so far, unless "VAC/Watcom" compatibility is meant in a way to copy all of their quirks and bugs and anachronisms one by one. In such a case one would have to separate close and closesocket, for instance. OTOH, Unix compatibility also means to copy numerous broken concepts of the Unix API which are better solved in the OS/2 API. EMX tried some balance there with the effect that some Unix things work, and some OS/2 things work, and special cases don't, in both areas. > > For Unixos2-only people: Henry proposed to create a CVS for libc, now > > the question is what do we want to take as base (EMX, LIBEMU and > > Innotek-GCC libc is in dicussion, there is also one from IBM btw) > > I think we should start with EMX, add Posix/2, some link support, > pthreads, libintl and maybe some other nice little extensions and then > start making it all work together and fixing bugs (e.g. pthread library > and library for symlink support (via EA) both modify certain functions > for accessing files - BTW, large file support is the third candidate which > "wants" specific changes in that same area). > > Whether the CVS is rooted at netlabs or sourceforge (we could start by > abusing the Posix/2 CVS) is rather irrelevant to me, however it would be > great to automatically send all CVS commits (as a diff) to some dedicated > mailing list (I know that wxWindows did such a thing also while the CVS > was hosted by sourceforge, so it's possible to set it up like this even > there - but I don't know, how to do it), so whoever is interested has a > chance to check things easily. Meanwhile, with rather limited time, I second that fully. > The biggest problem for starting however is probably the build system: > EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want > I don't know what etc. Unifying this and "porting" the Makefiles to some > current GNUmake is going to require much time for not much to gain... :-( I had already started to move EMX to GNUmake (don't ask me which version, I am typically off the recent porting attempt), but eventually messed this up by trying, at the same time, to improve the EMX environment, e.g. by throwing out DOSisms and the syscall interface through the INT21 like register calling mechanism. What make the dmakefile build mechanism complicated was the IMHO too complex interwoven system of dependencies in order to build the various variants of linraries from a single source tree. It needs some time to untangle this, but this is something that has to be done before any further work. > Ideally, we would also stay backward compatible, which implies keeping the > right ordering for the already existing symbols in the DLL's. (though I'd > suggest to make the import libraries for the new DLL's using "import by You'll have to give up full compatibility eventually, no doubt about this. Leave the old EMX*.DLLs in the system, and name the new ones EMU*.DLL or anything else, and don't attempt to keep old code where you have better new code. You do know that if you want to integrate libemu, you'll get a clash on certain APIs. And then there is no need to keep the DLL ordinals consistent with old EMX. They may be compatible in the first release, but will diverge sooner or later. So might it right from the beginning (means: name imports). > name"). OTOH, I _suppose_ large file support if put into the "normal" C > routines is going to cause incompatibilities anyway (file positions no You can move file I/O into a separate DLL, and provide large file I/O by DosLoadModule where present, and normal I/O where not. This is a similar thing like EM's TCP/IP support (in those days it was not clear whether every OS/2 system had SO32DLL.DLL, now you would rely on this). > longer fit into an int, do they?), so maybe it would even be a good idea > to be intentionally incompatible (old EMX DLL's handle small files, in the > new runtime, everything would be large file enabled - Hmm, I'll have to > get a new harddisk to create a large enough JFS partition for testing...). > > > It would be nice if we really could discuss that all together and agree > > on a sollution that is fine for everyone and allows enhancements in the > > future (no dead end). > > What I don't know at all is, how this could fit into Holger's long term > plans. Ideally, replacing a couple of low level C routines by his more > complete/efficient implementations - once they are ready - would be > sufficient, but for this, it would be a big plus to have a rather > detailed list of what functions those are going to be, I suppose. The API I am targeting is the man(2) API. This means there should be still a separation of man(2) and man(3). This is more or less already implemented in EMX, however with the mentioned syscall interface. > E.g. an improved link implementation is going to affect some other file > related functions ((f)open, (f)stat, ...), but if we know this in advance, > we can have some internal function like __dereference_link that's called > by all those, so the changes from a temporary E/A based link emulation to > "real" links are minimized... Look at the way it is done in "real" Unix. If the "link" code touches any stream-I/O function, e.g. fstat() or fopen(), it is doing something wrong. The link code should be based on lstat()/stat() and open(), and in particular as close as possible to DosOpen and friends, because it is very likely that a future DosOpen will already understand and handle links. Implementing it in this way is IMHO the main challenge of cleaning up EMX code. And don't worry about EA-based vs. "real" links; once the symlink code is stabilized, there is a way to do the same in kernel mode (with the side effect that these EAs become restricted and protected unless handled by the correct functions). Holger **= Email 5 ==========================** Date: Tue, 17 Jun 2003 21:48:05 +0200 From: Andreas Buening Subject: Re: libc Stefan Neis wrote: > > On Tue, 17 Jun 2003, Adrian Gschwend wrote: [snip] > > our GCC is targeted to be a VAC/Watcom replacement not a Unix > > porting environment > > just wait for our first distribution then we can discuss how > > to combine the various efforts > > To be honest, that doesn't sound to promising from the porting point of > view. There are some things that _have_ to be deep inside libc, otherwise > e.g. closing sockets by close instead of closesocket or pthread support > won't work... :-( You're right. That sounds ... strange. > > For Unixos2-only people: Henry proposed to create a CVS for libc, now > > the question is what do we want to take as base (EMX, LIBEMU and > > Innotek-GCC libc is in dicussion, there is also one from IBM btw) > > I think we should start with EMX, add Posix/2, some link support, > pthreads, libintl and maybe some other nice little extensions and then > start making it all work together and fixing bugs (e.g. pthread library > and library for symlink support (via EA) both modify certain functions I think we'll have to start with more basic tasks like updating the emx headers/libs for new standard functions. ;-) I'd like to keep the standalone libs like intl and pthreads out of emx. Lots of Unix system also have standalone versions of them so it's more important to get them working than to integrate them into emx. However, the details have to be discussed. > for accessing files - BTW, large file support is the third candidate which > "wants" specific changes in that same area). How do other systems solve this problem? Do they use an internal fseek_64() replacement which is used by default instead of fseek()? [snip] > The biggest problem for starting however is probably the build system: > EMX libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want > I don't know what etc. Unifying this and "porting" the Makefiles to some > current GNUmake is going to require much time for not much to gain... :-( Yes, this could become a serious problem. Maybe we have to port emx to a new build system first. ;-) [snip] > > It would be nice if we really could discuss that all together and agree > > on a sollution that is fine for everyone and allows enhancements in the > > future (no dead end). > > What I don't know at all is, how this could fit into Holger's long term > plans. Nobody knows, I guess. [snip] Bye, Andreas **= Email 6 ==========================** Date: Tue, 17 Jun 2003 23:06:18 +0200 From: "Knut St. Osmundsen" Subject: Re: libc Andreas Buening wrote: > Stefan Neis wrote: > >> On Tue, 17 Jun 2003, Adrian Gschwend wrote: > > > [snip] > > >>> our GCC is targeted to be a VAC/Watcom replacement not a Unix >>> porting environment just wait for our first distribution then >>> we can discuss how to combine the various efforts >> >> To be honest, that doesn't sound to promising from the porting point of >> view. There are some things that _have_ to be deep inside libc, otherwise >> e.g. closing sockets by close instead of closesocket or pthread support >> won't work... :-( > > You're right. That sounds ... strange. The shared socket and file handlespace is phun. :-) >>> For Unixos2-only people: Henry proposed to create a CVS for libc, now the >>> question is what do we want to take as base (EMX, LIBEMU and Innotek-GCC >>> libc is in dicussion, there is also one from IBM btw) >> >> I think we should start with EMX, add Posix/2, some link support, pthreads, >> libintl and maybe some other nice little extensions and then start making >> it all work together and fixing bugs (e.g. pthread library and library for >> symlink support (via EA) both modify certain functions > > > I think we'll have to start with more basic tasks like updating the emx > headers/libs for new standard functions. ;-) I'd like to keep the standalone > libs like intl and pthreads out of emx. Lots of Unix system also have > standalone versions of them so it's more important to get them working than > to integrate them into emx. However, the details have to be discussed. From the compiler and libc side there is a major piece of work in updating headers (and implementation in libc) to later C standards. The tcpip headers have allready been updated a bit. And not to forget that we updated the os2 emx headers with correct calling convention. That's all I can tell now. > >> for accessing files - BTW, large file support is the third candidate which >> "wants" specific changes in that same area). > > > How do other systems solve this problem? Do they use an internal fseek_64() > replacement which is used by default instead of fseek()? fseek64 if anything, but what the 'standard' extensions in this respect is could easily be found out by look at glibc, freebsd and the single unix specs on opengroup. > [snip] > > >> The biggest problem for starting however is probably the build system: EMX >> libs expect dmake, IIRC. Posix/2 wants some GNUmake, pthreads want I don't >> know what etc. Unifying this and "porting" the Makefiles to some current >> GNUmake is going to require much time for not much to gain... :-( > > > Yes, this could become a serious problem. Maybe we have to port emx to a new > build system first. ;-) > Build system isn't an issue anylonger, and even if it were, there are millions of them (allready responsible for 10+ (closed and open) myself). (If anything, a working GNU make is of much better use than yet another build system - the latest port is leaking a file/pipe handle everytime it's spawning a child, and -j is broken.) > [snip] > > >>> It would be nice if we really could discuss that all together and agree >>> on a sollution that is fine for everyone and allows enhancements in the >>> future (no dead end). >> >> What I don't know at all is, how this could fit into Holger's long term >> plans. > > > Nobody knows, I guess. > Now, I just wanna say that I do not see any reasons why there should be several efforts provding gcc and libc for OS/2. Exactly how to combine efforts we must talk together about. I'm personally willing to offer some spare time (that's the time between 2400 and 0600) working on such a project. For now, I can only repeat what Achim said on IRC; wait for us to come out with a distro, then we'll talk where to go from there. Kind Regards, knut PS. Holger, by the by, named exports sucks performance wise on OS/2 - period - . Any libc dll should make use of versioning (in the name) and ordinal import library for performance. In the ideal UNIX(tm) EMU world we should of course use ELF, ldos2.so, and ELF style load/dynload symbol resolving.