How to Disable FileServer Directory Listings

A nice feature of Go's http.FileServer is that it automatically generates navigable directory listings, which look a bit like this:

But for certain applications you might want to prevent this behavior and disable directory listings altogether. In this post I’m going to run through three different options for doing exactly that:

• Using index.html files
• Using middleware
• Using a custom filesystem

Using index.html files

Before http.FileServer generates a directory listing it checks for the existence of an index.html file in the directory root. If an index.html file exists, then it will respond with the contents of the file instead.

So it follows that a simple way to disable directory listings is to add a blank index.html file to your root static file directory and all sub-directories, like so:

.
├── main.go
└── static
    ├── css
    │   ├── index.html
    │   └── main.css
    ├── img
    │   ├── index.html
    │   └── logo.png
    ├── index.html
    └── robots.txt

If you've got a lot of sub-directories an easy way to do that is with a one-line command like this:

$ find ./static/ -type d -exec touch {}/index.html \;

Any requests for a directory should now result in an empty 200 OK response for the user, instead of a directory listing. For example:

$ curl -i http://localhost:4000/static/img/
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/html; charset=utf-8
Last-Modified: Tue, 13 Mar 2018 12:41:10 GMT
Date: Tue, 13 Mar 2018 12:42:35 GMT

Or without the trailing slash, the user should get a 301 Redirect like so:

$ curl -i http://localhost:4000/static/img
HTTP/1.1 301 Moved Permanently
Location: /static/img/
Date: Tue, 13 Mar 2018 12:43:13 GMT
Content-Length: 43
Content-Type: text/html; charset=utf-8

<a href="/static/img/">Moved Permanently</a>.

This is a good-enough solution if you can't (or don't want to) make any changes to your Go application itself.

But it's not perfect. You'll need to remember to add a blank index.html file for any new sub-directories in the future, and — if we're being pedantic — sending a 403 Forbidden or 404 Not Found status code would be more appropriate than sending the user an empty 200 OK response.

Using middleware

Both of these imperfections can be addressed if we take a different approach and implement some custom middleware to intercept requests before they reach the http.FileServer.

Essentially, we want the middleware to check if the request URL ends with a / character, and if it does, return a 404 Not Found response instead of passing on the request to the http.FileServer. Here's a basic implementation:

package main

import (
    "log"
    "net/http"
    "strings"
)

func main() {
    mux := http.NewServeMux()

    fileServer := http.FileServer(http.Dir("./static"))
    mux.Handle("/static/", http.StripPrefix("/static", neuter(fileServer)))

    err := http.ListenAndServe(":4000", mux)
    log.Fatal(err)
}

func neuter(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if strings.HasSuffix(r.URL.Path, "/") {
            http.NotFound(w, r)
            return
        }

        next.ServeHTTP(w, r)
    })
}

This approach would result in a user getting responses like these:

$ curl -i http://localhost:4000/static/img/
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 13 Mar 2018 12:46:20 GMT
Content-Length: 19

404 page not found

$ curl -i http://localhost:4000/static/img
HTTP/1.1 301 Moved Permanently
Location: /static/img/
Date: Tue, 13 Mar 2018 12:46:55 GMT
Content-Length: 43
Content-Type: text/html; charset=utf-8

<a href="/static/img/">Moved Permanently</a>.

To me, this feels like a cleaner and easier-to-maintain way to disable directory listings than using blank index.html files. But again, it's still not perfect.

Firstly, requests for any directories without the trailing slash will be 301 redirected only to receive a 404 Not Found response. It's an extra, unnecessary, request for both the client and server to deal with.

Secondly, if one of your directories does contain an index.html file then it won't ever be used. For example, if you have the directory structure...

.
├── main.go
└── static
    ├── css
    │   ├── index.html
    │   └── main.css
    ├── img
    │   └── logo.png
    └── robots.txt

... any request to http://localhost:4000/static/css/ will result in a 404 Not Found response instead of returning the contents of the /static/css/index.html file.

$ curl -i http://localhost:4000/static/css/
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 13 Mar 2018 12:51:09 GMT
Content-Length: 19

404 page not found

Using a custom filesystem

The final option we're going to look at is creating a custom filesystem and passing that to your http.FileServer.

There are a couple of approaches described by Brad Fitzpatrick and George Armhold you might want to consider, but I would personally suggest doing something like this:

package main

import (
    "log"
    "net/http"
    "path/filepath"
)

func main() {
    mux := http.NewServeMux()

    fileServer := http.FileServer(neuteredFileSystem{http.Dir("./static")})
    mux.Handle("/static", http.NotFoundHandler())
    mux.Handle("/static/", http.StripPrefix("/static", fileServer))

    err := http.ListenAndServe(":4000", mux)
    log.Fatal(err)
}

type neuteredFileSystem struct {
    fs http.FileSystem
}

func (nfs neuteredFileSystem) Open(path string) (http.File, error) {
    f, err := nfs.fs.Open(path)
    if err != nil {
        return nil, err
    }

    s, err := f.Stat()
    if s.IsDir() {
        index := filepath.Join(path, "index.html")
        if _, err := nfs.fs.Open(index); err != nil {
            closeErr := f.Close()
            if closeErr != nil {
                return nil, closeErr
            }

            return nil, err
        }
    }

    return f, nil
}    

In this code we're creating a custom neuteredFileSystem type which embeds the standard http.FileSystem. We then implement an Open() method on it — which gets called each time our http.FileServer receives a request.

In our Open() method we Stat() the requested file path and use the IsDir() method to check whether it's a directory or not. If it is a directory we then try to Open() any index.html file in it. If no index.html file exists, then this will return a os.ErrNotExist error (which in turn we return and it will be transformed into a 404 Not Found response by http.Fileserver). We also call Close() on the original file to avoid a file descriptor leak. Otherwise, we just return the file and let http.FileServer do its thing.

Putting this to use with the directory structure...

.
├── main.go
└── static
    ├── css
    │   ├── index.html
    │   └── main.css
    ├── img
    │   └── logo.png
    └── robots.txt

...would result in responses like:

$ curl -i http://localhost:4000/static/img/
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 13 Mar 2018 16:53:21 GMT
Content-Length: 19

404 page not found

$ curl -i http://localhost:4000/static/img
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Tue, 13 Mar 2018 16:53:22 GMT
Content-Length: 19

404 page not found

$ curl -i http://localhost:4000/static/css/
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 37
Content-Type: text/html; charset=utf-8
Last-Modified: Tue, 13 Mar 2018 12:49:00 GMT
Date: Tue, 13 Mar 2018 16:53:27 GMT

<h1>This is my custom index page</h1>

This is now working pretty nicely:

• All requests for directories (with no index.html file) return a 404 Not Found response, instead of a directory listing or a redirect. This works for requests both with and without a trailing slash.
• The default behavior of http.FileServer isn't changed any other way, and index.html files work as per the standard library documentation.