Send files faster with X-Sendfile
Rails' send_file has a beautiful, simple interface. Unfortunately, it ties up your Rails process with the rather mundane task of reading a file from disk. If you use Mongrel, you'll quickly notice that the :stream option doesn't seem to make any difference in memory usage and your mongrel process stubbornly doesn't want to let go of that memory once the file has been sent. This is a very, very bad thing.
Please don’t let this post scare you away from Mongrel. I have found it much more stable and easier to set up than FCGI. Mongrel + Capistrano saved me from the brink of insanity when deployment came around. The memory issue with send_file is indicative of a bigger issue with Ruby’s garbage collection—not a problem with Mongrel itself.
There’s this slick HTTP header called “X-Sendfile” that Apache and Lighttpd support. It was made to address this kind of problem (isn’t that convenient?). Now, when you send an HTTP response, all you have to do is tack on an X-Sendfile header with the path of the file you need to send—don’t worry about actually reading the file or sending any of those bytes yourself. The web server will load the file you specified and send it downstream.
This is loads easier on your Rails process (it just sets the headers and gets on with life), and you get all the cool file transfer functionality already baked into your web server (like proper caching, resuming, etc). Of course, you can send any file that the web server can read, not just files in your normal public directory. This makes it perfect for sending private or protected content to specific users of your app.
Setting those magical headers
You shouldn’t be too surprised that Rails makes it easy to set this header correctly. You could do something like this:
This code sample has been excerpted from the Ruby on Rails Wiki.
filename = "/var/www/myfile.xyz"
response.headers['Content-Type'] = "application/force-download"
response.headers['Content-Disposition'] = "attachment; filename=\"#{File.basename(filename)}\""
response.headers["X-Sendfile"] = filename
response.headers['Content-length'] = File.size(filename)
render :nothing => true
Apache users will need mod_xsendfile. Lighttpd users should look into mod_fastcgi and mod_proxy_core.
Now, as long as your server is correctly configured, you file should be zipping along at record speeds. If you need this functionality in more than one action or more than one app it’s not terribly DRY… and you should probably write some functional tests for this… ugh. Someone really ought to write a plugin.
The XSendfile plugin
The advanced or impatient may want to jump straight to the rdoc or source.
I wrapped up all this goodness in a plugin. I’ve had this running on a production server (Apache2 + Mongrel) for a couple of weeks with no issues. For the good of humanity, it comes complete with rdoc and tests.
Installation
You can install the plugin using Rails’ built-in plugin command.
ruby script/plugin install http://john.guen.in/svn/plugins/x_send_file/
Usage
The interface is as close as possible to Rails’ send_file command. If your server requires it, you can override the HTTP header used for X-Sendfile with the new :header option (the default is X-Sendfile).
x_send_file('/path/to/file')
x_send_file('/path/to/image.jpg', :type => 'image/jpeg', :disposition => 'inline')
x_send_file('/path/to/file/', :header => 'X-LIGHTTPD-SEND-FILE')
If you want x_send_file jump in and take over any time send_file is used,
add this to your environment.rb:
That ! stands for “danger!” Use this method responsibly.
XSendFile::Plugin.replace_send_file!
More information
If you’re ready to dig further, check out these resources:
- The rdoc
- The source
- RoR wiki’s How To Send Files Fast
- Apache’s mod_xsendfile
- Lighttpd’s mod_fastcgi and mod_proxy_core
Comments
There are 16 comments on this post. Post yours →