Domino and gzip compression
by Volker Weber
Here is a challenge: Try to find out why Domino uses gzip compression for mail databases and not for other applications served by the Domino http task.
There has been some disappointment that Domino 7.0 did not provide for gzip compression over http, a feature I am using for all of this site. It greatly decreases the amount of traffic between browser and server. Your only choice so far is to use a reverse proxy in front of Domino and have that compress the data stream.
Now there is a surprise. Domino does use gzip, but only for mail databases. There must be a switch somewhere which causes the http task to compress this data stream. A few people went looking but nobody could find it.
Have you succeeded in finding it? Or in case you work on the mothership, can you drop us a hint?
Update: Manfred has nailed it. Hire this guy. Or do you need more proof of how good he is?
Comments
At first guess I’d hazard that the http stack looks at the “Type”attribute in database properties. If this is set to “Mailfile” then it does the gzip thing.
I guess I should build a server one day to test this stuff... ;o)
... As to why only mailfiles, maybe this is a new bleeding edge feature for Domino, and IBM don’t want to hose the relevant box by gzipping everything? i.e. roll the feature out a little at a time? Mailfiles are the obvious candidates due to the unpredictable nature of the content being served-up, and the fact that users expect a decent response for mail.
Ah, I did not think of that. I was guessing it is either in the code (which I use on vowe.net), read template, or in the directory path. But the type would make sense. This is easy to test (and abuse).
There is a setting in the Domino Directory in the Domino Web Access area.
Compress HTTP response data
But as this is also in 7.0 I do not think that this is what you were looking for, isn't it?
Ask Brendon Upson of Puakma :-)
Volker, what you need is Puakma Web Booster. Is the http accelerator from Brendon Upson.
Puakma Web Booster Link
.::AleX::.
Alex, web booster is reverse caching proxy. That is not what I need/want. What I need is a good explanation, why Domino supports gzip in mail but not elsewhere. Both Ben and Heini have given valuable hints.
Domino has been supporting GZIP for iNotes web access at least since Domino 6.5x. It was supposed to be a R7 feature but did not make it into the release. Knowbody knows why it is missing. I raised the issue during the "Ask the Developers" session at Lotusphere 2006. Kevin Cavanaugh promied to look into it.
@Ben Thanks for the suggestion. I have been tearing my hears out to get gzip operational, but unfortunately I couldn't get your suggestion to work.
On an existing file, I changed the type to mailfile, restarted the http task and checked the compression with HttpWatch. No gzip. I checked the type of the forms7.nsf file in iNotes, where many of the gzipped pages seem to come from, and it is of the type "standard".
Well, if you really want gzip in Domino, you can still do it using a custom solution (java.util.zip.GZIPOutputStream!)
Of course, this then means using a Java agent. And if the whole point of gzip in your site is to compress content due to heavy use, then that defeats the object.
But it can be done ;o)
This is silly. Domino has gzip built in, but won't let you have it. Kevin Cavanaugh is evading a question at "Beat the developers" and does not follow up in, what, six weeks?
Volker this is not silly. What's silly is that Domino has "programmability" built in but Designer won't let you have it :-/
Developing web applications with Domino has always been something that works, in spite of best efforts by Lotus to screw it up. :-)
Is it possible that this is another case of IBM protecting us from ourselves (I *really* hate it when they do that...)? There are some bugs (or perhaps 'features') in the way IE, at least, handles GZipped javascript and css files. The onLoad event fires before the files have been unzipped, or so I seem to remember. Maybe IBM didn't turn the feature on because testing indicated it breaks too many existing applications? In which case, one hopes they're preparing to make it a per-database (or per-design element) property for 7.x...
Volker,
currently (after circa 1.5 hours of investigation), I also have no idea what separates the mailfile from any other database related to your question on GZip compression.
But I have really good news: It is easily possible to turn GZip compression on for all NSF files on a Domino Server within one minute (ok, lets say 5 minutes). :-)
Read my just finished article (with step-by-step instructions) for the whole story...
ND7 undocumented: Using HTTP GZip compression for all your NSF files
@Manfred, tnx for information.
It's nice but like most is quite 'defeatured' by IBM.
I wonder how this gzip implementation behaves. Maybe the Content-Length header reflects the length of the incoming data from the client and not the byte count of the decompressed data stream. So there is some predictable browser confusion.
Even being seen as a backseatdriver i have to mention that the combination of a gzipping domino and a reverse proxy in front does not work. Because mod_proxy can't parse and rewrite gzipped html files.
The gzip compression has to do with the inotes enhancements in the http-stack. Reformat some code from the formsX.nsf, then you will see the reason! ;-)))
@Manfred: Thanks!
Followed Manfred's article and then tried access DWA via redirect.nsf and was prompted to save the downloaded file. So I am not sure what the heck happend but have disabled GZIP for now. Nice idea though!
Same for me here. I am prompted to download the nsf-file. But only with IE. With Firefox everything is fine!
application/* would include the X-Lotus-NSF (nsf) mimetype. I would not be surprised if this confuses IE. My exclude list is
application/zip
application/X-Lotus-NSF
applicationX-Lotus-NTF
application/x-shockwave-flash
There is no apparent reason that I exclude swf files (it is missing in the default file identifications view anyway).
