I'm looking for any documents on developing interposing calls for libraries,
particular for libc, but possibly for any library.

The problem is vfork.
[snip]
it's impossible to wrap vfork since the wrapper is
not allowed to return. I've "solved" this by intercepting vfork() calls
and turning them into a real fork(). This works most of the time but is
not ideal. On a given platform and CPU it may be possible to work around
this with handcrafted assembler code but I haven't tried yet. If you
solve this I'd appreciate it if you'd post the solution.

My interposing is meant to be transparent production.  That is, once it is
implemented and deployed, it will be left in place.  It is not for debugging,
but that might be done as well for other things.

Bernie Ohls wrote:

The problem is vfork.
[snip]
it's impossible to wrap vfork since the wrapper is
not allowed to return. I've "solved" this by intercepting vfork() calls
and turning them into a real fork(). This works most of the time but is
not ideal. On a given platform and CPU it may be possible to work around
this with handcrafted assembler code but I haven't tried yet. If you
solve this I'd appreciate it if you'd post the solution.

You already discovered the solution: replace vfork with fork.
Everyone has agreed that vfork is a performance optimization only.
If replacing vfork with real fork creates any semantic difference,
then that is a certified bug in the code which [mis-]uses vfork.

John Reiser wrote:
Bernie Ohls wrote:

The problem is vfork.
[snip]
it's impossible to wrap vfork since the wrapper is
not allowed to return. I've "solved" this by intercepting vfork() calls
and turning them into a real fork(). This works most of the time but is
not ideal. On a given platform and CPU it may be possible to work around
this with handcrafted assembler code but I haven't tried yet. If you
solve this I'd appreciate it if you'd post the solution.
You already discovered the solution: replace vfork with fork.
Everyone has agreed that vfork is a performance optimization only.
If replacing vfork with real fork creates any semantic difference,
then that is a certified bug in the code which [mis-]uses vfork.

There's a lot of history around vfork, in c.u.p as well as other
places. I don't want to go over it, and it's not topical here anyway,
but the short version is that when a system is running low on virtual
memory a vfork/exec sequence might succeed where a fork/exec would
fail.

In comp.unix.programmer John Reiser <jreiser at (no spam) bitwagon.com> wrote:
....
| glibc also has symbols such as:
| 3785: 005eb32c 205 FUNC LOCAL HIDDEN 11 __GI___open64
| 4354: 005eb2b0 122 FUNC LOCAL DEFAULT 11 __GI_open
....
Potentially a need to interpose every open could interpose these other
function entry points as well.

It is issues like these I'm wanting to read about. And if that means the
solution ultimately needs to be an OS syscall level intercept like ptrace
in Linux, than I think the document would need to cover that, too (and how
on at least a few major platforms like BSD, Linux, and Solaris).

In comp.unix.programmer Paul Pluzhnikov <ppluzhnikov-nsp at (no spam) gmail.com> wrote:
| phil-news-nospam at (no spam) ipal.net writes:
|
|> In comp.unix.programmer John Reiser <jreiser at (no spam) bitwagon.com> wrote:
| ...
|> | glibc also has symbols such as:
|> | 3785: 005eb32c 205 FUNC LOCAL HIDDEN 11 __GI___open64
|> | 4354: 005eb2b0 122 FUNC LOCAL DEFAULT 11 __GI_open
| ...
|> Potentially a need to interpose every open could interpose these other
|> function entry points as well.
|
| Except you can't interpose them without "extreme hackery": the
| symbols are local, which means they don't go through "normal"
| lazy binding. Rather, there are direct 'call 0x5eb2b0' instructions
| inside glibc, and if you want to "steal" them, you have to patch
| .text, or place breakpoints (ala "ltrace").

Ah, yes, they would be statically linked within the linking of the
library itself. So this means that library interposing might have
to be limited to API scope. Beyond that is ptrace() intercepting
(or whatever is equivalent to that Linux call on other systems).


|> It is issues like these I'm wanting to read about. And if that means the
|> solution ultimately needs to be an OS syscall level intercept like ptrace
|> in Linux, than I think the document would need to cover that, too (and how
|> on at least a few major platforms like BSD, Linux, and Solaris).
|
| I have been doing large amount of libc interposing on Linux, Solaris,
| HP-UX and AIX for the last 10 years, and have not seen a document
| you are seeking.

OK. I've pretty much ended the search, anyway.


| From my experience, the issues you are likely to encounter are:
|
| 1. Partial interception (e.g. interposing mmap() does not get you
| intra-glibc calls to mmap() from e.g. fopen()).
|
| This is the issue John raised with open(); there are many more
| such "internal" calls in glibc, and some in Solaris libc as well.

One would have to be wanting to interpose only what the application
is directly calling for this to be useful or clean, then.


| 2. Startup dependencies: libc startup is a complicated dance,
| where things work only partially while libc is starting itself up,
| and calling things in the "wrong" order often leads to a crash
| either due to NULL pointer dereference (some global pointer
| is expected to have been set up, but hasn't been set yet),
| or infinite recursion (discover that some global pointer hasn't
| been set up yet, try to set it, hit interposer, discover that
| the global pointer hasn't been set up yet, spin into the ground ...).
|
| In addition, libc startup of threaded programs is even more
| complicated, as various pthread_* routines expect parts of libc
| they depend on to have been initialized beforehand, and
| interposing can severely change the order of initialization.
|
| 3. Interposing symbols that are called during dlsym(): on Linux,
| for programs linked against libpthread, dlsym() calls calloc().
|
| If you interpose calloc(), you can't find the "original" glibc
| calloc() via 'dlsym(RTLD_NEXT, "calloc")' because that will
| result in infinite recursion.

A new call to calloc() via the interposers call ... a nice gotcha to
know about. One would have to be sure they are not interposing the
library itself (e.g. calls inside the library don't get interposed).
Not sure how that is done.


| 4. Symbol versioning: there are two incompatible versions of
| e.g. pthread_cond_init() inside glibc, and if you have 3rd
| party libraries in the picture, it's quite possible for both of
| these versions to be actually used. Correctly interposing both
| symbols requires some tricks, and the use of dlvsym().
|
| Cheers,

These are the kinds of things I was looking for. Thanks for the starter clues.

--
|WARNING: Due to extreme spam, googlegroups.com is blocked. Due to ignorance |
| by the abuse department, bellsouth.net is blocked. If you post to |
| Usenet from these places, find another Usenet provider ASAP. |
| Phil Howard KA9WGN (email for humans: first name in lower case at ipal.net) |