Detecting site performance anomalies using cURL

I recently participated in an IBM Webshpere Commerce Server deployment and was tasked with hunting down the source of some performance anomalies. cURL is a tool that has the ability to easily probe and collect detailed timing data using its many options. Below are the links I referenced to write my scripts that provided critical information and insight into the problem. I will add more detail here soon about what exact steps I used. 


http://newestindustry.org/2006/10/03/baseline-testing-with-curl-2/

http://josephscott.org/archives/2011/10/timing-details-with-curl/

http://www.jonefox.com/blog/2011/10/10/find-the-time-to-first-byte-using-curl/

http://curl.haxx.se/docs/manual.html

Front-end enginerring: Image compression

I recently participated in an IBM Webshpere Commerce Server deployment and was surprised when I ran Google's PageSpeed against the site, that it suggests the use of image optimizers on the stock widget graphics. The analysis for one of our pages was as follows:

The detailed message under the Learn more link was helpful in that it linked to the tools they suggest using - bookmarking it here for reference:

Use an image compressor.

Several tools are available that perform further, lossless compression on JPEG and PNG files, with no effect on image quality. For JPEG, we recommend jpegtran or jpegoptim (available on Linux only; run with the --strip-all option). For PNG, we recommend OptiPNG or PNGOUT.

Tip: When you run Page Speed against a page referencing JPEG and PNG files, it automatically compresses the files and saves the output to a configurable directory.

I would expect an off the shelf tool to do these kinds of optimizations on their own graphics before shipping, no? Some of the absolute numbers seem small, but every 1.5K is a packet that doesn't need to be sent, and this is a small step to make for such an easy gain.

Passive House links

Found some good links on the topic of passive house building. Would have just subscribed to feeds if they had them, but some of them didn't, hence this post.


http://www.minnephithouse.com


http://bruteforcecollaborative.com/bfc/blog/


http://www.passivehouse.us


http://www.greenbuildingadvisor.com/blogs/dept/green-building-news/passivhaus-institut-germany-disowns-its-us-satellite


http://www.greenbuildingadvisor.com/blogs/dept/green-building-news/american-passive-house-institute-responds-dr-feist

RAMDisk options for Windows

IBM Rational Application Developer can be real hard on the disk, especially when debugging in the test environment for Websphere 6. In lieu of convincing my employer that every developer should get a speedy SSD or RAM-based disk to minimize wait times between WAS6 restarts, I have been investigating the available options and have been looking at software based RAMDisks. These could even be faster than most SSD disks, in theory.

I was pleasantly surprised to find these options below are available for Windows 7:

• This one is payware, but the free version allows a RAMDisk of up to 4g in size, which is enough for me. http://memory.dataram.com/products-and-services/software/ramdisk
• This one is free, haven't tried it yet. http://www.ltr-data.se/opencode.html/#ImDisk
• This one looks promising, especially for its ability to transparently write ram disk changes to a backing file on a physical disk. http://www.superspeed.com/desktop/ramdisk.php

For now, I need to test these out and see if there really is a benefit for some of the IBM RAD slowness I see on a daily basis. I will post my findings here at a later date.

Graywater Heat Reclaimation

Just some links/notes on a topic I became interested recently. Why do I find graywater heat reclaimation interesting?

• cost saving: reduced energy bill
• cost saving: extends the life of the water heater
• cost avoidance: average water heater recovery times are markedly improved
• environmental goodness
• low/no risk: passive system with no moving parts

See

• http://www.watercycles.ca/
• http://www.ecoinnovation.ca/residential-solutions/thermodrain-connection-options/
• http://www.theenergyconscious.com/wat7005.html
• http://www.ecoinnovation.ca/residential-solutions/
• http://gfxtechnology.com/contents.html
• http://dailyhomerenotips.com/2008/02/11/dwhr-drain-water-heat-recovery-follow-up/
• http://www.ecoshift.ca/Drain-Water-Heat-Recovery-DWHR-Overview.html

Seems like the price is around $750 for a bit of copper pipe that is supposed to last about 50 years.

Javascript: An easy to make day-of-week calculation bug

It would be nice if unit tests could save one from every silly mistake it is possible to make when writing software. These were my thoughts when I found the following bit of javascript code on a small website I wrote years ago. This rather obvious time-related bug only surfaced now because of the current date under which the code was running.

Give the following javascript method a yyyymmdd date string and you will only sometimes get the correct output:

function makeDate(rawDate) {

  var dayNum=rawDate.substr(6,2);

  var monNum=new Number(rawDate.substr(4,2));

  var yrNum=new Number(rawDate.substr(0,4));

  var monName=months[monNum-1];

  var dateObj=new Date();

  dateObj.setDate(new Number(dayNum));

  dateObj.setMonth(monNum-1);

  dateObj.setYear(yrNum);

  var dayOfWeekNum=dateObj.getDay();

  var dayOfWeekName=days[dayOfWeekNum];

  return dayOfWeekName+" "+dayNum+" "+monName+" "+yrNum;

}

For instance, on Sep 24, 2011, feed the string "20110731" to the function above and you will get a result that claims 7/31/2011 is a Friday. That day is actually a Sunday.

The problem is that because the month stored in the date reference corresponds to a 30 day month when you set the day to 31, the month value increments as you set the day (while the day of month changes from 31 to 1). Stepwise explanation below:

function makeDate(rawDate) {

  var dayNum=rawDate.substr(6,2);

  var monNum=new Number(rawDate.substr(4,2));

  var yrNum=new Number(rawDate.substr(0,4));

  var monName=months[monNum-1];

  var dateObj=new Date(); 

// dateObj at this point is today's date, Sat 24 Sep 2011

// since rawDate is "20110731" the dayNum we use next is "31"

  dateObj.setDate(new Number(dayNum)); // boom!

// dateObj is now Fri 1 Oct 2011, how?

// because there is no such thing as September 31st, 

// the date object rolls over to the next month.

  dateObj.setMonth(monNum-1); 
// we still set this correctly to July

  dateObj.setYear(yrNum); // and this is fine to 2011

  var dayOfWeekNum=dateObj.getDay(); 
// but here the problem comes to light

// dayOfWeekNum reflects a day of Friday, since
// July 1, 2011 is a Friday

  var dayOfWeekName=days[dayOfWeekNum];

  return dayOfWeekName+" "+dayNum+" "+monName+" "+yrNum;

}

The way I chose to solve this is to set the dateObj from most significant denomination to least significant to avoid rollover problems. This order ensures that before we attempt to set any date like the 29th, 30th or 31st that the correct month has already been set, avoiding rollovers:

var dateObj=new Date(); 

dateObj.setYear(yrNum);

dateObj.setMonth(monNum-1);

dateObj.setDate(new Number(dayNum)); 

  // as long as the month is already right, we're ok here

var dayOfWeekNum=dateObj.getDay();

After fixing this, I did find a good reference here for someone experiencing a similar problem:

http://stackoverflow.com/questions/1214234/javascript-date-parsing-bug-fails-for-dates-in-june

Windows batch script timestamps from gAWK

# This uses GNU Awk to emit a script to set an environment variable to the current date and time in a format suitable for use as a filename.

# See http://www.gnu.org/software/gawk/manual/

# You should supply two variables:

# 1) output_filename

# 2) env_var

BEGIN {

format = "%Y%m%d_%H%M%S"

print "set " env_var "=" strftime(format) > output_filename

}

Invoke with something like

gawk -v "output_filename=tsgenerated.bat" -v "env_var=TS" -f dt.awk
call tsgenerated.bat