The last time Hackerfall tried to access this page, it returned a not found error. A cached version of the page is below, or clickhereto continue anyway

Code Confessional - Patching an application from a device driver

Patching an application from a device driver

I confess: I once fixed a bug in a device driver by having the driver patch the application calling it so that the bug was masked. Here’s my story…

Long ago I worked on DOS and (early) Windows device drivers. In particular, I wrote device drivers for network adapter cards for Token Ring networks. To make these work on Novell networks (where Ethernet was technology of choice) that company made out device drivers “API” compatible with Novell’s Ethernet cards.

Specifically, the cards I worked on emulated Novell’s IPX/SPX network stack (IPX is a little like IP, SPX a little like TCP; IPX is connectionless, SPX guarantees delivery).

A particular application I worked on a did a “liveness” check of the network stack and card by sending itself a packet and seeing if it was received. This worked as follows.

The application would register an Event Service Routine (ESR) which would be called by the network stack when a packet was received. In that ESR it would simply set a flag (literally flip a bit from 0 to 1 in a byte). The application then sent itself a packet. 

The Novell driver had a quirk: if you sent yourself a packet it never hit the wire and would be received immediately. As a consequence the ESR would get called before the call to the send function returned to the application. This particular application exploited that by immediately checking the flag after the send returned. It could then tell if the network driver was working.

Unfortunately, our driver actually sent a packet on the wire and so the ESR got called after the send had returned. That meant the application always thought the network driver was dead. It turned out that solving this was a nightmare within our code because we were heavily optimized for speed and the driver was actually split in two between a part running in the host computer (and did little processing) and a part running independently on the network adapter card on its own processor (which, just to add to the fun, had the opposite endianness!).

So, the conclusion was that we should “patch the app”. And that’s what I did. I altered the driver to “take a look” at the application that was registering the ESR. The driver would pop take a look at the ESR that was to be called and if it contained the code that flipped the single bit then it would fix the bug by setting that bit immediately as the routine was registered.

Thus when the packet was sent to itself the ESR was irrelevant and the sending code could check the flag and discover that the network was still functioning.

PS Welcome to Code Confessional: a place for programmers to confess their sins. Feel free to submit!

  1. ourwitching reblogged this from codeconfessional

  2. hmagemusic liked this

  3. codeconfessional posted this

Continue reading on codeconfessional.tumblr.com