I finally bit the bullet and rewrote eigenclass using the Ocsigen web server + framework for OCaml. It is simpler, faster, more reliable, and easier to extend than the customized wiki implementation in Ruby (Hiki) I'd been using. It is also easier to deploy because it's self-contained: a single (native code) executable contains both the Ocsigen web server and the application code, so I don't have to use any special Apache modules, FastCGI or any sort of adapter. (The ability to create standalone, native-code executables was added recently to Ocsigen and is thus available on the devel branch, soon to be released as Ocsigen 1.2.)
I'd read somewhere that the Ocsigen server hadn't received much (any?) optimization work, so I benchmarked it against Lighttpd, Apache and mongrel, both at static file serving and dynamic contents (a minimal "hello world" service), to see if that could represent a problem. It turns out it isn't: the OCaml+Ocsigen combo is very fast. It serves minimal dynamic requests an order of magnitude faster than Rails with a pack of mongrels behind nginx, and uses 40 times less memory. More surprisingly, it handles more requests per core than lighttpd with a minimal FastCGI server written in C! (lighttpd wasn't able to handle ab's load with max_procs = 1, and generated way too many 5xx errors, so I had to use several FastCGI processes). It also serves static files at rates exceeding Apache's (per core).
The following figures were obtained using ApacheBench (ab) locally, on a 3GHz, dual core Athlon64 64 X2.
Dynamic contents
| Reqs/sec | Mem usage (resident memory, RSS) | |
|---|---|---|
| Rails with mongrel, 1 process | 260 | 49MB |
| Rails with mongrel via nginx (rev proxy), 1 proc | 220 | ~51MB |
| Rails with mongrel, 4 processes via nginx | 430 | ~200MB |
| Ocsigen (1 process) | 5800 | 4.5MB |
| lighttpd with FastCGI app in C, 20 procs | 9300 | 4.5MB |
Obviously, these figures represent only upper bounds, since the "dynamic" content was but "hello world", and few sites (certainly not eigenclass.org) need to handle thousands of requests per second. The interesting thing is that, if anything, the difference is going to become even more favorable for Ocsigen+OCaml if the page involves any significant amount of computation, as OCaml is typically 100 times faster than interpreted languages like Ruby. For instance, the OCaml code that processes the markdown-like markup used for this very page is fast enough to sustain over 2000 requests per second without caching the generated HTML. A quick test shows that Ruby's bluecloth library is around 200 times slower, so I would be getting maybe 20 reqs/sec (using both cores) on the AMD64 box (much faster than the one running eigenclass.org) with Rails + Mongrel + nginx. Of course, caching would solve this.; this is not a panacea, though, as it introduces other problems (expiration, invalidation, resource limitation, etc.) and is not always applicable.
At the end of the day, this means that OCaml + Ocsigen allow me to write code that can be deployed trivially (I can even link the executable statically so that it doesn't depend on libs like SQLite or libssl), and is more than fast enough with a single process (no load balancing needed) and no caching (no memcached or whatever).
File serving
(Small 13-byte file.)
| Reqs/sec | |
|---|---|
| mongrel | ~1000 |
| Ocsigen (1 process) | ~4500 |
| Apache2 (multiple workers) | ~8500 |
| lighttpd | ~12000 |
This shows that even though Ocsigen could use some optimization on the static serving front (dynamic contents are served faster), it's still quite reasonable. As I said, I've heard Ocsigen has undergone few if any optimizations, so the outlook is quite positive.
Comments
test
That is really awesome, like the new layout as well.
Can you please publish the eigenclass source? I'd love to see how Ocsigen works...
I got intrigued and ran ab with an tntnet app. I am writing. Here are some results for comparison. I didn't capture ram usage completely for apache but it was 1-2MB residnt, for tntnet. Apache served same file but statically.
Regards Shridhar
Bob, I'm releasing it once I document it a bit (gotta add the configuration file and copyright notices to the git repos).
Fortunately, there's a nice tutorial for Eliom (Ocsigen's web framework) at http://ocsigen.org/tutorialdev1 that illustrates Ocsigen/Eliom's features better than my code does (it doesn't use Ocsigen's more advanced functionality).
Essentially, you can declare services, specify their parameters (and their types, allowing Eliom to perform static type checking and to "overload" services for the same base URL but different parameters), attach them to an URL and give their handler at once.
Here's a very small example taken from my code, for the service than handles
/R2/writings/xxx, which is declared as follows (I'm changing it a bit for clarity of exposition):let rec page_service = register_new_service ~path:["writings"] ~get_params:(suffix (string "page")) serve_pageThis indicates that the service attached at
writingstakes astringparameter given as a suffix (otherwise, it could only be given with?page=xxxx; usingsuffixallows both styles).serve_pageis a function taking the request ino and astringparameter (and no POST parameters) which returns the response. Eliom/OCaml will check statically thatserve_pagehas got the right type. Moreover, Eliom can ensure statically that all links are valid (i.e. that you're linking to an existent service and that you're giving it the right parameters).Now, you can use OCaml to its full power to structure the handlers, using higher-order functions, partial application, functors...
Another feature of Ocsigen is that it includes a typed HTML/XHTML module that ensures (again, statically) that the generated markup is valid. The code will look a bit like the XML builders you often find in dynamic languages, the key difference being that it's typed and wrong markup just doesn't compile.
This is a function that generates an HTML page (with some extras) containing the supplied title and body:
let rec page_with_title sp thetitle thebody = html (head (title (pcdata thetitle)) [css_link css_uri (); ctype_meta; rss2_link sp]) (body (thebody @ analytics))The compiler knows which elements are valid inside
bodyand any other tags, and will complain if you try to generate invalid XHTML/HTML 4.01 (here I show the error you get in the toplevel aka. REPL; the compiler will behave similarly):# page_with_title () "the title" [ol (pcdata "foo") []];; -------------- This expression has type [> `PCDATA ] XHTML.M.elt but is here used with type [< `Li ] XHTML.M.elt The second variant type does not allow tag(s) `PCDATAThe request handler looks like this:
and serve_page sp page () = match Pages.get_entry pages page with (* ^ ^ *) (* | this () represents (empty) post parameters *) (* this holds additional request info *) None -> not_found () | Some node -> let thetitle = Node.title node in let toplink = a ~service:toplevel_service ~sp [pcdata !toplevel_link] () in page_with_title sp thetitle [div_with_id "article_body" (div_with_id "header" [h1 [a ~service:page_service ~sp [pcdata thetitle] page]; with_class p "date" [pcdata (format_date (Node.date node))]; p [toplink]] :: (node_body_with_comments ~sp node @ [footer]))]That
a ~service:page_service ~sp [pcdata thetitle] pageis a "permalink". Eliom knows that the service takes only one parameter of typestring, and the compiler would complain if I tried to give it no or more parameters with the wrong types --- it would also tell me if I were trying to link to a non-existent service (page_servicewould not be defined).I think something is wrong here, all of these numbers are REALLY low. Apache and lighttpd should be more or less identical for small static files (with lighty pulling ahead for large static files served off nfs or via proxy), as both will be limited by bandwidth, and both should be way higher unless you are on a 1mbit hub.
Does this also apply to the "dynamic" test, i.e., should I be getting thousands of reqs/s out of nginx + mongrel + RoR? If so, how? Some quick googling yields a few results that seem in line with mine (mongrel + RoR serving at most a few hundred requests per core on comparable machines).
How many requests per second should I be getting, what would be a normal ballpark figure (Linux 2.6.26-1-amd64, 3GHz dual core)? I'm running the tests locally, not over the wire, and the servers are far from being limited by bandwidth, they're CPU-bound.
I have to confess I'm not that interested in static file serving performance; dynamic contents are normally the limiting factor.
hey, i like that you are trying to get performance. however, the above code in the comments is quite noisy. It is easy to outperform rails on a web performance basis. It is hard to compete with it on a 'get things done quickly and nicely' basis. I cannot state 'programmer performance', because some of the people who work on rails code should not be called programmers.
How's it compare against, say, Erlyweb?
http://erlyweb.org/
Dan, I didn't know I was signing up for a web framework shootout :)
I read that Erlyweb is 4 to 6 times faster than Rails. If that's true, it'd be in the ~2000-2500 reqs/sec range on my box, which sounds very good! If the page involves substantial amounts of computation, Erlang's performance shortcomings might become a problem, though. String manipulation is known to be one of Erlang's major weakness (e.g., by default strings use 16 bytes per character(!)), which could also prove challenging.
The thing implementations based on Erlang should own everything else at is handling massive numbers of simultaneous connections, e.g. for streaming. (Ocsigen also uses lightweight threads which can be spawned by the millions, but still employs
selectto monitor connections --- I reckon it'd take a few dozen lines of code to switch to something likeepoll; in fact I have that half-written).Dru, I'm sure Rails is almost unbeatable regarding devel speed for many (most?) CRUD applications thanks to its convention over configuration philosophy. It seems to me that the head start disappears quickly once you deviate from the default behavior, though.
The code you're seeing corresponds mostly to HTML generation using Rails'
Builder. In Ocsigen, this is called XHTML.M, and has got two advantages over Rails' counterpart:it is typed and guarantees statically that the generated markup is valid (invalid markup does not compile)
it is much faster
For instance,
let rec page_with_title sp thetitle thebody = html (head (title (pcdata thetitle)) [css_link css_uri (); ctype_meta; rss2_link sp]) (body (thebody @ analytics))corresponds roughly to this in Ruby/Rails:
def page_with_title(title, buffer = "", &body) xm = Builder::XmlMarkup.new(buffer) xm.body { xm.head { xm.title title xm.link("href" => @css_uri, "type" => "text/css", "rel" => "stylesheet") xm.meta("content" => "text/html; charset=UTF-8", "http-equiv" => "Content-Type") rss2_link(xm) } xm.body { body.call(xm) analytics(xm) } } xm.target! endCould this be done using something equivalent to
erb? Certainly, Ocsigen allows you to use whatever templating engine you choose, and there are some similar in spirit toerb(only much faster :).Continuing with the analogies,
a ~service:page_service ~sp [pcdata thetitle] pageis equivalent tolink_to thetitle, { :controller => "pages", :action => "show", :page => page }.Granted,
register_new_service ~path:["writings"] ~get_params:(suffix (string "page")) serve_pageis "noise" compared to an implicit Rails route, but not so much if you consider it corresponds to something like (I had to google for this so it might be a bit off)
map.connect 'writings/:page`, :controller => 'pages', :action => 'serve_page', :requirements => { :page => /\S+/ }At the end of the day, you have to map URLs to actions and generate HTML, so the code isn't that different if you're deviating from the defaults.