Golang's builtin net/http/httptest package allows you to test code that implements an HTTP server meeting the http.Handler interface. It does this by listening for connections on the loopback interface on the first available socket. This works fine, but presumes that you can get a listening socket on the loopback interface of your test machine. This is generally the case, but we can do better.

Making an in-memory listener

The net package now has the net.Pipe function that gives two connected sockets in memory. To test a listening socket server in memory this needs to be adapted to function as a net.Listener. That interface is as follows

type Listener interface {
        // Accept waits for and returns the next connection to the listener.
        Accept() (c Conn, err error)

        // Close closes the listener.
        // Any blocked Accept operations will be unblocked and return errors.
        Close() error

        // Addr returns the listener's network address.
        Addr() Addr
}

So, we need Accept() to return a connection to a client that is trying to connect. The other two functions need to exist, but their purpose is less important for unit testing.

The Accept() function is blocking until a client connects or an error occurs. To create this sort of behavior, we can use a channel.

func (ml *MemoryListener) Accept() (net.Conn,error) {
    select {
        case newConnection := <- ml.connections:
            return newConnection, nil
        case <- ml.state:  
            return nil,errors.New("Listener closed")
    }
}   

By using a channel, the listening server blocks until a new connection is passed over ml.connections. When ml.state is closed Accept() immediately returns an error indicating the listener has been closed. In order to get connections to the server blocking in Accept() we'll need another function to open new connections.

func (ml *MemoryListener) Dial(network, addr string) (net.Conn,error) {
    //Check for being closed
    select {
        case <- ml.state:
            return nil, errors.New("Listener closed")
        default:
    }
    //Create the in memory transport
    serverSide, clientSide := net.Pipe()
    //Pass half to the server
    ml.connections <- serverSide
    //Return the other half to the client
    return clientSide, nil
}

This allows a connection to be established to the server using the same MemoryListener instance. For testing a simple socket server, this is sufficient. The normal net.Listener used by the server is replaced with an instance of MemoryListener. For the client, you'll need to be able to explicitly specify an already existing net.Conn to use or be able to set a function the client uses to open a new connection. Here is an example in github.com/garyburd/redigo of what I mean.

Testing an HTTP servers

The goal of this is test HTTP servers using an in memory listener. There is no easy way to marry this to existing test framework net/http/httptest unfortunately. It is completely possible to use this with package net/http. All we need to do is construct an instance of http.Server that uses an instance of MemoryListener. This can be wrapped up as follows

func NewInMemoryServer(h http.Handler) *MemoryServer {
    retval := &MemoryServer{}
    retval.Listener = NewMemoryListener()
    retval.Server = &http.Server{}
    retval.Handler = h
    go retval.Serve(retval.Listener)
    return retval
}

In order to make a request to the HTTP server, you'll need to use an instance of http.Client that opens a connection to the MemoryListener. The http.Client type relies on an implementation of http.RoundTripper to select the appropriate network transport for connecting to the server. The type http.Transport typically fills this role. The implementation of (*MemoryListener).Dial takes two string arguments which are ignored. This allows the (*MemoryListener).Dial function to be used as the value of the member Dial of http.Transport. This results in all requests regardless of the domain in the URL going to the server under test.

All of the behavior required to instantiate an http.Client that makes HTTP requests to the server under test can be wrapped up into a single function.

func (ms *MemoryServer) NewTransport() *http.Transport {
    transport := &http.Transport{}
    transport.Dial = ms.Listener.Dial
    return transport
}

func (ms *MemoryServer) NewClient() *http.Client {
    client := &http.Client{}
    client.Transport = ms.NewTransport()
    return client
}

Requests can be made to the server under test by using ms.NewClient().Get("http://server.test/foo"). This creates a complete environment to test your HTTP server without relying on the network stack of the machine running the tests.

Putting it all together

I've put all of this together in a complete library github.com/hydrogen18/memlistener. There is an example of how to use it for the purposes of testing an http.Handler in the package github.com/hydrogen18/memlistener/example.