[am, 19991106] The UNIX and OS/2 EMX Porting Companion FAQ Preface This is a simple collection of tips & tricks (without any warranty) for porting software from unix to OS/2 It has a few sections but really find something you need remember that the FIND function of your HTML-browser is your friend: Since we list quite a few APIs/concepts/packages which you will run across, just look for a keyword and hope we already have a helpful comment on it. In the following we assume the reader is already familiar with EMX and has access to the docs. Since porting requires a very broad knowledge of many topics this list has been extended to include general information about programming techniques, unix, X11, ... Parts of this docs are stolen/taken from various people. To avoid that a single one is neglected in a reference list, I omit all and even put this doc into PD w/o asking all those. PD (public domain) means you can get this doc and do with it whatever you want. However the best usage is correcting bugs and publishing it! In case you may want to correct an error or add an entry, please do so and send it back to me. Thanks in advance! ------------------------------------------------------------------------ Breaking News I have a preliminary porting.inf available! Not really helpful due to missing external links and other problems, though. There are nice tools, but no one does produce "perfect" IPF output yet. In addition it's not on the current level of this document :-( ------------------------------------------------------------------------ Useful & dirty workarounds These are not necessarily "canonical" and clean approaches but often they will fullfill their purpose very well. Attention: using these redefinitions will make debugging more complicated since you have to remember all of them! #define chdir _chdir2 /* see below !*/ #define chown(x,y,z) (0) #define drand48() drand() /* see below */ #define endgrent() /* int fchmod(int fildes, mode_t mode ); get the file name for the handle and call chmod() */ #define FNDELAY O_NONBLOCK #define getcwd _getcwd2 /* see below !*/ #define getdtablesize() (1024) #define getgrent() ((group*)0) #define lstat stat #define mkfifo(p,m) (0) /* see below */ #define mknod(p,m,d) (0) #define readlink(x,y,z) (0) #define readlink(s,t,l) (strcpy(t,s),strlen(t)) #define setgrent() #define strcasecmp stricmp #define strncasecmp strnicmp #define symlink(x,y) (-1) #define S_ISLNK(x) (0) #define S_IFLNK (0) #define S_ISBLK(x) (0) #define S_IFBLK (0) #define S_ISVTX (0) #define usleep(t) _sleep2( ((t)+500) / 1000 ) /* _sleep2 is an EMXism */ ------------------------------------------------------------------------ APIs that need special care exec() Replace exec*() with spawn*() and exit() if the parent process waits for the termination of the new process (by calling wait() or by waiting for SIGCLD). This is required to keep the process ID of the child process. In a forked process, however, you don't have to do this because emx.dll does it for you. fork Replace fork() and exec*() with spawn*(). Under OS/2, fork() is inefficient. You may not care if it's really being called, but read the docs for any specific information about the fork() implementation. popen If you are lazy and keep the un*x-like forward slashes in paths, be aware that arguments to popen() have to pass cmd.exe which can't handle them! rename rename doesn't replace existing targets. Check your favourite resource for it's specification on un*x :-) It also fails on open files. remove remove fails on open files. signal By default, signal processing is different when using signal(): SIG_ACK should be used instead of the signal handler address to re-enable a signal by calling signal() when the signal handler has been called. This behavior can be changed with the -Zbsd-signals and -Zsysv-signals options of GCC. If you use POSIX.1 functions for signal handling, SIG_ACK is not required. Comment out /replace code which bases on non-implemented signals. (note that some are only on few systems availabe) o SIGCONT (Continue if stopped) o SIGDANGER (Danger signal) o SIGINFO (Information request) o SIGIOT (IOT trap) o SIGLOST(Resource lost) o SIGNOFP (Floating point co-processor not available) o SIGPHONE (?) o SIGPOLL (I/O possible) o SIGPROF (Profiling timer expired) o SIGPWR (Power failure) o SIGSTOP (Stop process) o SIGTSTP (Stop typed at tty) o SIGTTIN (tty input for background process) o SIGTTOU (tty output for background process) o SIGURG (Urgent I/O condition) o SIGVTALRM (Virtual timer expired) o SIGWINCH (Window resize signal; see below) o SIGWIND (?) o SIGXCPU (CPU time limit exceeded) o SIGXFSZ (File size limit exceeded) system If you are lazy and keep the un*x-like forward slashes in paths, be aware that arguments to system() have to pass cmd.exe which can't handle them! unlink On un*x you can (re)move open files. On OS/2 you can't and therefore this call might fail. ------------------------------------------------------------------------ Ported APIs/APIsets Collection of known ports of APIs/APIsets, i.e. stuff which is more complex than a simple #define foo bar. We restrict ourselves to those of general usage, which can be used for "everything". Libs for special purposes may be collected elsewhere. Well, this should be a list with links. For now we start w/o ... Sources are Hobbes, LEO and the sites listed at the end of this doc. * dlopen()/dlfcn (see below) * drand48(), srand48() * mmap: a primitive but open source implementation and a closed-source one which is more sophisticated ... * pthread * (GNU) readline * regex * scandir * setitimer/getitimer o H. Veit posted some instructions and an example. o I tried to write a wrapper for this. But with the current approach it's not 100% possible. Especially one is limited to the thread #1 for doing X requests. All is work in progress, watch the X/2 mailinglist! Currently only setitimer(ITIMER_REAL) is implemented and only up to the resolution provided by _sleep2(). o hrtimer.sys (version >= 1.1) from Hobbes, with new simple open()/read() interface should make timing easy and enable writing an itimer compatible interface * shm (see below) * syslog: various(?) ports on Hobbes or LEO ------------------------------------------------------------------------ un*x process stuff [Process groups, fork, ...] Process handling is a bit different on un*x. However often you won't notice a difference, since the most frequently used APIs are starting of process via system(), exec(), etc. which are available on OS/2. (also see fork()). The term threads has no unique meaning in the un*x world as opposed to OS/2. Those unix systems which had offerede somthign like that often developped their own standard for this. pthreads (POSIX threads) is now a standard. A wrapper from this API set to OS/2 exists (see Hobbes). ------------------------------------------------------------------------ un*x filesystem stuff Un*x filesystems are "slightly" different from our OS/2 one. We try to collect some major differences which we will run across. Also be aware that things you might not expect to be found here may be listed: process handling for example! * case-sensitive names * no drive letters * There are standard paths which always exist. We may also include some which are not mandatory / root directory. Only a single one due a "lack" of drive latters ... /bin contains system executables /dev virtual interface to devices of all kinds /etc configuration data /home contains directories of the users /opt optional installed stuff /proc Interface to processes on the system. By read/write access to this virtual file system gives you control over all processes. However there's no portable standard for this. /tmp a place for temporary data /usr binaries, data for the users /var partition which contains stuff of varying size during run time (logs, mails, data caches, ...) * Distributed over a few places of the system you will find the man pages, the standard un*x online help format (despite some projects have chosen Texinfo based systems nowadays). The several subdirectories contain various man pages which usually carry the section number as a suffix (like foo.2) Section 1 user commands Section 2 system calls Section 3 libc calls Section 4 devices (e.g., hd, sd) Section 5 file formats and protocols (e.g., wtmp, /etc/passwd, nfs) Section 6 games Section 7 conventions, macro packages, etc. Section 8 system administration Section n [Tcl7Tk, Perl, ?] * They support links (see TVFS) * The separator of path components is a slash '/' as opposed to OS/2's backslash '\'. Note that many APIs understand the '/' on OS/2, but cmd.exe and most apps don't. * Multiple paths (e.g. in environment variables like PATH) are separated by a colon ':' * The root dir on every un*x system is '/'. Often code checks whether a given path is an absolute one by comparing the beginning with '/'. So grep for this string! (EMX: see _fullpath() and _abspath() can also be useful) * Using #define getcwd _getcwd2 #define chdir _chdir2 may help to support drive letters. Read the EMX docs. * Note that ///abc is a valid Unix filename. It's equivalent to /abc. * Note that chdir ("..") is a no-op under Unix if the current working directory is the root directory. Under emx, chdir ("..") fails in the root directory. ------------------------------------------------------------------------ Misc stuff Collections of various helpful suggestions. * On un*x static libraries follow an unique naming convention to carry a leading lib prefix, like libfoo.a. If you want to link against it however, the right command is -lfoo. On EMX unfortunatly this prefix has been dropped, i.e. -lfoo links against foo.a instead. * Shared libraries use the suffix .so on most un*x systems. Note that there's no need for an import library. * On un*x you can start executables with any name - given the executable file bit is set properly. On OS/2 things are a bit more complicated. You're on the safe side if you adopt the cmd.exe convention that you can only execute files with the endings ".com", ".exe", ".cmd", ".bat" (in exactly this order they are searched, BTW) * for 'fast' moving targets like libpng or nameclash examples like xaw, xaw3d [and xpm] don't be shy to link statically ... * Check for those un*xisms (listed here) by using grep or an equivalent command to do a recursive search. Look for all standard paths (see above), and all commands which are mentioned in this introduction. * famous problems are related to the binary/text mode mess on OS/2, especially when using stdin/stdout. See _fsetmode(), fopen(), ... in EMX docs * Note that on un*x plain text files have a format different from those used on DOS, OS/2. A short list of text formats: Operating systems line delimiters (hex) DOS, OS/2 0D 0A un*x 0A Macintosh 0D * Check out what's already available. Many ports are not available from Hobbes or LEO. Try to search on altavista (or similar search engines) and ask on the emx or XFree86 OS/2 mailinglists (see below). * Most un*x shells expand the wildcard characters '*' (matches every set of characters) and '?' (matches a single character). To get this done in your code on OS/2 you have to do it "manually" in your code. However be aware that even people on OS/2 may use shells which already expand. Hmm, this shouldn't hurt?! int main (int argc, char *argv[]) { _wildcard (&argc, &argv); /* ... the program ... */ } (See EMX docs for details on _wildcard() and _response()) * There are some environment variables which are frequently used on un*x, and often they are also in use on OS/2: EDITOR Your favourite editor. In case you don't want to end up inside vi [a famous un*x editor] and don't know how to get out :q! HOME points to a directory which contains user specific data. Always non-NULL on un*x PAGER standard tool to page output. Something like "more" or "less" PATH same as on OS/2 PRINTER default printer. Might be used in conjunction with printing tools like "lp", "lpr" SHELL standard shell to be used. Often used by Makefiles, ... TERM Name of the terminal currently used. (see termcap, terminfo). Make sure you're software runs in an OS/2 window as well as in an xterm TMP Usually not necessary, since all systems have a directory for temporary stuff in /tmp. On OS/2 unfortunately one sometimes runs across TEMP, TEMPDIR and other garbage. WINDOWID Id of current X11 window (terminals session only?!) * Never name an executable "test". test is a program on un*x, which does nothing - at least if you don't have a closer look ... * Unfortunately many OS/2 tools, including built-in cmd.exe commands have their counterparts on un*x with exactly the same name but different semantics/syntax. An incomplete list: echo, find, install, more, patch, sort, unpack * To merge objects try ld -X -r -o darin.o $(OBJS) * You will run across many new file formats. Get the file utility for OS/2 which tries to identify formats using a database of file signatures. The most important ones are archiv formats. We list a few and the according tools to extract them (see Hobbes, LEO) Note that on un*x it's standard to use cumulative suffixes, so hello.tar.gz refers to a file hello which was produced by tar and finally by gzip. extension related tool .gz (GNU) gzip (gzip -d, gunzip) .rpm RPM; OS/2 port of this tool! .shar sh (shell archive) (sh foo.shar) .tar (GNU) tar .uue uudecode/uudeview .z pack/unpack .Z decompress/gunzip ------------------------------------------------------------------------ In & out [misc section about IO related stuff; this is rather complex and I can't even nearly fill/discuss all sections; we start with the simple ones...] 1. Printing Printing on un*x is quite different to OS/2: no sophisticated system interfaces exist (read: may exist but are rarely used) so all applications have to this on their own. Lowest denominator is that a Postscript-capable printer is attached to the system. If you're lucky enough to have a Postscript printer on your OS/2 system you may use the lpr and lpd tools (shipped with Warp 4) to access the Postscript printer queues from the commandline. On my system "lp" is a batch script, a kind of an alias for "lpr.exe -b -p HP5MPPSQ -s localhost" where HP5MPPSQ is one my Postscript printer queues. If can't work out something similar try out starting ghostview (ghostscript) or print to a file first. You may want to use "lportd" or tools like printmon.zip (from Hobbes or LEO.) 2. Sound: [OSS, digital audio, ...] There's a /dev/audio emulation available from Hobbes and LEO (devaudio*.zip) ------------------------------------------------------------------------ EMX specifics [many part of this chapter should be removed and the content placed at the proper sections of this document. Started now, moved stuff is just deleted!!] * Excerpts from the EMX docs: from 5.1 Porting Unix applications When porting Unix applications, please note the following restrictions: - Socket handles are not inherited across exec*() and spawn*(). (Note that a process created by fork() inherits the socket handles of its parent process.) Inheriting socket handles is planned for a future release of emx. - The size of messages is restricted to 32767 bytes (this is a limitation of IBM TCP/IP for OS/2). - Initially, sockets are in binary mode. Use setmode() to switch to text mode. Text mode applies to read() and write(), only. recv() and send() always use binary mode. - The functions recvmsg(), sendmsg(), and socketpair() are not implemented. from 5 Hints for porting Unix programs to emx - Change all open(), fopen(), fdopen() and freopen() calls to use O_BINARY or "b", respectively, for binary files. If a file contains both binary and textual data, read the file in binary mode and do the conversion yourself. - Though fseek() and ftell() now work on text files, the offsets are different from what Unix programs expect. You may have to open the files in binary mode and ignore carriage returns (this has been done in GDB). - Programs reading a.out files should be changed to call _seek_hdr() or _fseek_hdr() before reading the header to support .exe files. More changes are usually required. - The null device is called /dev/null under Unix. The __open() system call translates the filenames "/dev/null" and "/dev/tty" (lower case, with slashes) to "nul" and "con", respectively. However, system ("whatever >/dev/null"); won't work as the standard OS/2 and DOS command interpreters don't recognize /dev/null. - Use termio or termios or read the keyboard with _read_kbd() if you don't want to get input line by line. - Do not use the PTRACE_TRACEME request of ptrace(): use P_DEBUG instead when starting the process with spawn*(). - The shell isn't called /bin/sh. Use system(). system() and popen() don't expand wildcards (unless COMSPEC points to a shell which expands wildcards). - Printing single characters is inefficient. A solution is to use setvbuf (stdout, NULL, _IOLBF, BUFSIZ) and to use fflush (stdout) if you need the output immediately (flushing is required only after displaying prompting texts before reading input or displaying progress reports that don't end with a newline character). GDB output has been made much faster by using line buffering. - Note that VEOF != VMIN and VEOL != VTIME. Programs which use VEOF and VEOL to access VMIN and VTIME, respectively, should be changed to use VMIN and VTIME. emx uses separate fields for VEOF, VEOL, VMIN and VTIME. - To use termio, you have to reset the IDEFAULT bit of c_lflag. This does not apply to termios. * If you need to check which preprocessor defines are in use you can dig them out according to the example batch script below. Note that it makes use of 2 un*x utils (touch, rm). touch __shd.c gcc -dM -E %1 %2 %3 %4 %5 %6 %7 %8 %9 __shd.c rm __shd.c ------------------------------------------------------------------------ Configuration stuff Various build mechanisms are used in the un*x world: simple/complex Makefiles, crazy batch files, autoconf, Imakefiles, ... Anyway you should get all available un*x tools which are already ported, most of them being from GNU. More details can be found in the tools section Makefiles Get a version of GNUmake from Hobbes/LEO. Watch out for embedded shell commands! In case you use XFree86 OS/2 you have a make.cmd in your path which is helpful to some extent and a problem in some other cases. You can call the underlying binary executable by x11make.exe. The batch file defines SHELL="" and may do other things (currently it doesn't but ...) batch files Try to run them in an un*x shell like ash, bash, ksh, tcsh, ... autoconf In case there's a configure.in you can/should use the autoconf port (link below). Else try this script from Ilya Imakefile Used for X11 software only. Run xmkmf (mxmkmf). Unfortunately it may also contain non-portable shell commands. libtool used to build libraries. Usually only for shared libs, since producing static ones is usually straight forward. (see below) ranlib ranlib is not available. Use the s command of ar instead (or forget about it completly) ------------------------------------------------------------------------ Tools To compile for OS/2 you should have except the mandatory complete(!) set of EMX installed: * GNU Bash or ash * GNU Make * GNU file utils (ls, rm, mv etc) * GNU text utils (cat) * GNU sed * GNU grep * GNU bison * GNU perf * GNU flex * patch * diff * GNU flex * tar * gzip Don't forget to put 'sh.exe' into /bin subdirectory to keep many scripts/Makefiles running w/o changes. Other helpful stuff * TVFS: Toronto Virtual File System Available from Hobbes, LEO it creates a virtual drive and create links on it. So you can acess all your partitions on a single one and don't need to use drive letters! * OS2TRACE (OS/2 API tracer; see Hobbes, LEO) can trace down most OS/2 API calls. Obviously not helpful upon starting an un*x port ;-) * Get chk4dlls somewhere. It lists all DLLs required by an executable/DLL and is an equivalent to the ldd available on some un*x systems * Get some tools to process the various documentation formats. Converters are available for o HTML to IPF o man to HTML o Texinfo to html (new makeinfo) o Texinfo to IPF o ... * Get Concurrent Versions System (CVS): A tool used for many open source projects on the net. The given link may carry outdated executables; check Hobbes, LEO) ------------------------------------------------------------------------ X11 specifics In general you don't need to make changes related to X11. The existing XFree86 OS/2 is very, very close to the level which is supplied for un*x platforms, like linux. In turn XFree86 is based on the official releases from the OpenGroup and therefore conforming to the standards. In fact most problems arising are already covered in the XFree86 OS/2 FAQ and on the mailinglist, which is has an archive online. How to get a single key pressed in an xterm? This is FAQ for un*x programming. By default terminals are in a mode which blocks until RETURN is pressed. [Feel free to finish this section. Or have a look at key.c for now ...] How to work around the non-working SIGWINCH? Have a look at this library What to do if an app refuses to build due to missing symbols like gethostname or connect ? Link against the socket library (-lsocket). How to debug an X11 application? See below for some general comments. Remember that an X11 application isn't running straight through some main() routine (except for the begin or end, perhaps) but most of the time it's running in a loop which waits for events (keyboard, mouse, ...) and then calls the appropriate routines which have been installed to react (so called callbacks). How to resolve X errors? If the app foo is linked against Xt.dll run foo -sync inside a debugger and set a breakpoint on exit() (see below). There are usually static X11 libs available. Linking against these makes things easier. ------------------------------------------------------------------------ Debugging * There are some debuggers available for free: o gdb/pmgdb (for a.out objects; part of EMX) o sd386 (Hobbes, LEO) (for omf objects) * To set a hardcoded breakpoint you may use this: #define BREAKPOINT __asm__("int3"); * Breakpoint on exit() If a binary is linked against emxlibcm.dll, exit() is not a known symbol to the debugger. Unfortunately this is usually the case if building X11 apps ... exit() is in EMXLIBCM.DLL, built with LINK386. GDB doesn't know how to read symbols from DLLs built with LINK386. Therefore, it doesn't know the address of exit(). Therefore, it cannot set a breakpoint on exit(). Look up the offset of exit() in \emx\etc\emxlibcm.map and set the breakpoint by address. (Use set show-dlls.) Warning: Due to a bug in OS/2, the breakpoint will apply to _all_ programs using EMXLIBCM.DLL. * Resolving segmentation faults (SIGSEGVs): Watch out for debugging malloc implementations like dbmalloc, etc. (no, libefence isn't available on OS/2) Turn off all optimizing flags upon compilation. ------------------------------------------------------------------------ Resources in the web This section is unfortunately rapidly growing. It isn't meant to contain hundreds of links. One day it will be shrinked down to a reasonable size again ... * Related stuff o a document specific to Imake(files) * General info, standards o C9x standard drafts o C standard drafts o C++ standard drafts * Un*x & other API references o un*x man pages o un*x man pages (for/from linux) o API, standards reference table o miscellaneous Linux dox o Information about various un*x flavours o Xlib manual o Xlib Manual (online) * Sources; libraries o FreeBSD sources o linux source code reference o Freely Distributable LIBM (fdlibm) Missing math stuff can be found here o cephes Math library, too * OS/2 software, ports o Hobbes major OS/2 software repository o LEO major OS/2 software repository o XFree86 for OS/2 o misc stuff ported libs; port of autoconf, shm, GNU gettext o misc stuff e.g. a PD dlfcn() implementation (there are others) * OS/2 specific INFO o IBM redbooks online o EDM/2 Electronic developer magazine for OS/2 o TT's OS/2 Programming Page (many useful links) * Misc o GNU project; those guys with a rather ill definition of "free software". More information about the common license types GPL and LGPL can be found here. o Introduction to various license types o Concurrent Versions System (CVS) A tool used for many open source projects on the net. * Mailing lists o emx mailinglist o XFree86 OS/2 mailinglist ------------------------------------------------------------------------ Send email to Alexander Mai ------------------------------------------------------------------------ Back to OS/2 page of Alexander Mai