Programming 34m0

Installing and configuring Cband for Apache2 on Ubuntu

2011-12-07T06:28:00.001-08:00

If you need to manage bandwidth usage on Apache web server (particularly on a per virtualhost basis) CBand could be that hero you were hoping for. Below are the steps I followed to get up and running with a basic CBand setup.

Installation

Install additional apache libraries. Only one of these two maybe required.
apt-get install apache2-prefork-dev
apt-get install apache2-threaded-dev

Download the module
wget http://dembol.org/downloads/cband/mod-cband-0.9.7.5.tgz

Untar the file
tar -xzf mod-cband-0.9.7.5.tgz

Enter newly created directory
cd mod-cband-0.9.7.5

Add -lm to the APXS OPTS in Makefile.in like so:
APXS_OPTS=-lm -Wc,-Wall -Wc,-DDST_CLASS=@DST_CLASS@

Configure and install
./configure
make
make install

Restart Apache
/etc/apache2/restart

Hopefully you should now have successfully installed Cband on your Ubuntu server.
Now let's configure it.

Configuration

In order to improve performance of Cband, add the following to /etc/apache2/mods-available/cband.conf
CBandScoreFlushPeriod 1
CBandRandomPulse On

Add sym link in mods-enabled folder
ln -s ../mods-available/cband.conf cband.conf

Make directory to store the accumulated data
mkdir /var/www/scoreboard

Change ownership of directory to the apache user
chown www-data:www-data /var/www/scoreboard/

Configure access to cband-status. Cband provides a status page, which you must activate via the apache configuration. It is likely you should restrict access to this page by some means. For unrestricted access to this page add the following to cband.conf
<Location /cband-status>
SetHandler cband-status
</Location>
<Location /cband-status-me>
SetHandler cband-status-me
</Location>

In your browser, navigate to http://server/cband-status and observe your new cband status page

Now add cband configuration for one or more virtual hosts, restart apache, and refresh your cband status page
<IfModule mod_cband.c>
CBandLimit 6G
CBandPeriod 4W
CBandPeriodSlice 1W
CBandSpeed 500kbps 10 30
CBandExceededSpeed 128kbps 5 15
CBandScoreboard /var/www/scoreboard/domain.com.scoreboard
</IfModule>

If you would like to do something more with all the data Cband is collecting you can retrieve an XML representation by using a url of the form:
http://server/cband-status?xml

Security

You don't want to let just anyone find intimate details about a site your hosting. You need to password-protect that data!

One of the simplest ways to password-protect is using the http auth module of apache. To get started on this path, reopen the file /etc/apache2/mods-available/cband.conf

Add the following lines to both of your Location elements replacing knownSystemUser appropriately:
AuthType Basic
AuthName "Restricted"
AuthUserFile /usr/local/apache/passwd/passwords
Require user knownSystemUser

Now create a password for the specified user in the specified file
mkdir -p /usr/local/apache/passwd/passwords
htpasswd -c /usr/local/apache/passwd/passwords knownSystemUser

Credits

http://blog.lael.be/290
http://dembol.org/mod_cband
http://pessetto.com/question2answer/index.php?qa=3&qa_1=why-does-mod_cband-have-a-truncf-symbol-undefined
http://www.howtoforge.com/mod_cband_apache2_bandwidth_quota_throttling
http://www.uno-code.com/?q=node/64
http://www.sourcefiles.org/Internet/WebServers/Modules/Bandwidth/mod-cband-0.9.7.2.tgz.shtml
http://httpd.apache.org/docs/2.0/howto/auth.html

Groovy safe navigation gotcha

2011-12-05T07:17:00.001-08:00

I just came across this Groovy gotcha today while accessing a list within a chain of attributes. I had a statement like this in a gsp:

${invoice?.invoiceItems[0]?.name}

This is the (erroneous) recommended style when data binding many ended associations in Grails, but it could easily arise in other scenarios. If I try this "safe" statement where invoiceItems is null, I'll see a null pointer exception. In fact I'll even see it when invoice is null. The reason for the exception is a combination of how lists and the safe navigation work in Groovy.

Lists are much more elegant in Groovy than Java, as they share the square brackets that primitive arrays use. Creating a list is as easy as writing:

def x = [1,2,3]

Just like an array you can access the first element of this list like so:

println x[0]

However, unlike a primitive array the [] operator on a list is mapped (by some Groovy magic) behind the scenes to the getAt method of the particular List object. We can expand our example as follows:

${invoice?.invoiceItems.getAt(0).name}

Safe navigation is a beautiful, and elegant means of protecting your code for scenarios where a variable could be null. It doesn't work as you might expect though. In my example above if invoice is null you might expect that after discovering invoice to be null execution would move to the next statement. Instead what happens is that each part of the chain is executed in turn, and the ? is equivalent to a try/catch block, where if a null pointer exception is caught, the exception will be swallowed, and null returned.

The end result of these two Groovy behaviors is that in my example (and indeed the examples in the grails documentation) we will see the attempted execution of

null.getAt(0)

The solution is to avoid using the [] operator when accessing lists and to place a ? before every full stop in your statement chain.

${invoice?.invoiceItems?.getAt(0)?.name}

Credits

http://grails.org/doc/latest/guide/6.%20The%20Web%20Layer.html#6.1.6 Data Binding

Mass find and replace on Mac OSX with SED

2011-11-02T01:38:00.000-07:00

Lets say you are doing some grand refactoring, and you want to change a class name from Dog to Pooch in multiple files.
There are a couple of ways you can do it with the sed command. One uses the -exec argument and the other uses the xargs program. Though I'm not sure why, I favor the xargs strategy. In Linux to do a mass find and replace for our Dog to Pooch example you could do something like:
find . -name "*.php" -print | xargs sed -i 's/Dog/Pooch/g'

But if you try this same command In Mac OSX you will see an error like this:
extra characters at the end of g command

The reason for this is the command in Mac OSX the command doesn't trust you. I wonder if rm -rf / works in Mac OSX? Well I'm not going to find out ;-) Mac OSX will create a backup file in case you mess up your find and replace, and it requires an extension for this file. With that in mind, the following command will work in Mac OSx:
find . -name "*.php" -print | xargs sed -i.bak 's/Dog/Pooch/g'

I recommend you choose a unique backup extension so that you can easily delete all the backup files. Once your happy with your change of course you can run something like: find . -name '*.bak' | xargs rm

Credits

http://rushi.wordpress.com/2008/08/05/find-replace-across-multiple-files-in-linux/
http://www.unix.com/emergency-unix-linux-support-help-me/137864-sed-mac-os-versus-linux.html

Debugging dependencies in Grails

2011-11-01T07:51:00.001-07:00

In Grails, and indeed large Java projects generally, dependency resolution can become quite a headache. Tools like Maven and Ivy are a huge help of course, but even they can't prevent tedious days trying to fix that conflict where two classes of the same name are loaded by the same class loader. Grails provides a feature that helps navigate through the complex web of depencies that your web application will no doubt accrue.

Issue the command: grails dependency-report
or if your using the maven-grails plugin: mvn grails:exec -Dcommand="dependency-report"

This will generate a number of reports in the /target/dependency-reports folder for you. The html reports can of course be opened in the browser of your choice. The graph reports (in graphml format) however need a special and - to me at least - uncommon tool. The tool that seems most popular is yEd. Fortunately yEd is available for Linux, Mac, and Windows.

After you have download and installed yEd, open one of the generated graphml files. The graph won't look like much to start, but after you tweak the view it becomes a very useful resource of information. The Ivy site provides a quick how-to on how to make the graph digestible.

Credits

http://burtbeckwith.com/blog/?p=624

Groovy XML Parser Gotcha - it's not a Map your parsing!

2011-10-22T07:55:00.000-07:00

When I first looked at examples of using the Groovy XmlParser it appeared that the parser took some nice raw XML and turned it into a map for easy traversing. I could take the following source:


<person>
    <face>
         <eyes>
             blue
         </eyes>
    </face>
</person>

And parse it find the eye color like this:


 def xml = new XmlParser().parseText(xmlSrc)
 assert xml.person.face.eyes.text() == 'blue'

Confident in my understanding, I thought it would be trivial to handle the following XML source I was receiving from a domain registrar:


<?xml version="1.0"?>
<interface-response>                       
     <ErrCount> 1</ErrCount >
     <errors>
         &ly;Err1> Do main name not available </Err1>
      </errors>                        
</interface-response>

I parsed the source in order to extract the errors from the response as follows:


    def xml = new XmlParser().parseText(xmlSrc)
    def errMsg = "Errors:"
    xml.errors.each { err ->
        errMsg += "\n ${err.text()}"
    }

This made perfect sense to me, but I wasn't getting the expected output in my error message. The problem was my "understanding". I had not grasped just what the
"xml.person.face.eyes" groovy magic was actually being done behind the scenes. If the parser had created a Map representation of my xml, my logic would have been right, but it wasn't. While being able to write xml.errors or xml.person.face.eyes might give the impression you can bring the ubiquitous map paradigm here it is a "Faux Amis" (a false friend as we say here in France). The associative dot notation is actually translated into XPath, and then this XPath query is run against your XML. So while sometimes the XPath can be equivalent to retrieving data from a Map often times it is not. Now here is code that follows the XPath paradigm (and it works!):


    def xml = new XmlParser().parseText(xmlSrc)
    def errMsg = "Errors:"
    xml.errors.'*'.each { err ->
        errMsg += "\n ${err.text()}"
    }

Configuring Logging of Bind DNS in Ubuntu

2011-09-28T23:50:00.000-07:00

After installing the Bind9 DNS server in Ubuntu, you might find the need to play with the logging settings. Logging is essential when your trying to figure out why the hell DNS setup is not working as expected. Without any explicit logging configuration (from you or your colleagues) Bind will log messages of the category "default" to "/var/log/syslog". In order to customize the configuration to your own needs try the following steps.

Step 1: Create a new file to write your logs to
The location of this file is up to you, but due to AppArmor restrictions in Ubuntu the easiest strategy is to place it in "/var/log/named". If you need to place it somewhere else you might need to alter your AppArmor configuration.

mkdir /var/log/named
touch /var/log/named/debug.log
touch /var/log/named/query.log

Step 2: Define the configuration
You can place your logging configuration in either "/etc/bind/named.conf" or "/etc/bind/named.conf.local". If there is a best practice on where to place it I couldn't find it - do let me know if you have any insights! Which ever file you choose, the content to add would be something like the following:


logging {

    channel debug_log {
         file "/var/log/named/debug.log";
        severity debug 3;
        print-category yes;
        print-severity yes;
        print-time yes;
    };

    channel query_log {
        file "/var/log/named/query.log";
        severity dynamic;
        print-category yes;
        print-severity yes;
        print-time yes;
    };

    category resolver { debug_log; };
    category security { debug_log; }; 
    category queries { query_log; };
   
};

The severity option is the logging level. The possible levels are critical, error, warning, notice, info, debug [ level ] , and dynamic. In the example above you can see I'm using both debug and dynamic. Note that debug is subdivided into levels 1, 2, and 3.

Logs in Bind are divided into categories. In our example we are printing logs from from 3 of the categories. For a complete list of the categories available see the Bind reference

Logs are output to channels. In our example we have two channels, each corresponding to a file. Note from the example how it is possible to send the logs of multiple categories to the one channel.

Step:3 Restart Bind
Don't forget to restart Bind for your changes to take effect.

sudo /etc/init.d/bind9 restart

Credits:

https://help.ubuntu.com/community/BIND9ServerHowto
http://www.bind9.net/manual/bind/9.3.2/Bv9ARM.ch06.html#id2552867

A Custom Dialog Box in JQuery Tutorial

2011-05-18T05:07:00.000-07:00

In the Jquery world, constructing a basic custom dialog box can be quite trivial. Using Jquery UI, creating a dialog box like the above can be done with this little snippet of code:


<div id="dialog" title="Basic dialog">
 <p>This is the default dialog which is useful for displaying information. The dialog window can be moved, resized and closed with the 'x' icon.</p>
</div>
<script>
$( "#dialog" ).dialog();
</script>

Web development is easy right? Well in a perfect world yes, but as any developer will tell you there are two worlds: the ideal world, and the real world. JQuery UI is a fine toolkit when one is looking for a standard off the shelf styled widget, but not when something more custom is desired - even with help from the excellent Themeroller.

The recent Twitter UI redesign does everything it can to ensure users will use their browser's back button less. Many of the actions possible do not change the principle view for the user, but merely extend it. One such measure was the compose a tweet dialog box. The JQuery UI dialog widget won't give you a Twitter inspired dialog box - you have to create your own.

Analysis of the Twitter dialog box reveals the following:

Non modal - actions possible on the originating view
Fixed position - dialog box is fixed in center of window even as window scrolls
Dialog box has a simple structure, but the nature of opaque inheritance could cause some problems.

The dialog box really has a very simple structure, and the HTML required reflects this.


<div class="dialog">
<div class="dialog-frame">
  <header>
    <h1>Our first Dialog</h1>
  </header>
</div>
<div class="dialog-content">
   <p>A pleasure to open our first dialog with you</p>
</div>
</dialog>

Note:
The dialog-frame div is a sibling of the dialog-content div rather than being its parent. The reason for this is to accommodate a
semi-transparent dialog frame. Once you reduce the opacity of an element, its children will automatically inherit this opacity. You cannot increase the opacity of an element from the value it has inherited.

Most of the work is in the javascript: you don't know the size of the user's browser window when writing your css, so you need to set dimensions and coordinates of the dialog box dynamically.


    var dialog = $('.dialog') // cache the dialog element

   // Initial dialog setup
    var dialogFrameWidth = '10px'
    dialog.css({position: 'fixed', padding:dialogFrameWidth})

    // Set dialog coordinates 
    dialogHeight = dialog.outerHeight()
    dialogWidth = dialog.outerWidth()
    dialogTop = ($(window).height() - dialogHeight)/2
    dialogLeft = ($(window).width() - dialogWidth)/2
    dialogFrameWidth = "10px"
    dialog.css({top: dialogTop, left: dialogLeft, position: fixed,  padding:dialogFrameWidth}})

    // Set coordinates and dimensions of dialog frame
    dialogFrame = dialog.find('.dialog-frame')
    dialogFrame.css({height: dialogHeight, width: dialogWidth, position: 'absolute', top: 0, left: 0, opacity: 0.8, z-index:1000}})

   // Set coordinates and dimensions of dialog content
    dialogFrameHeaderHeight = dialogFrame.find('header').outerHeight(true)
    dialogContent = dialog.find('.dialog-content')
    dialogContent.css({'margin-top': dialogFrameHeaderHeight, position: 'relative', top: 0, left: 0, z-index: 3000})

Add some style like background colors, fonts, etc. and your in business!

Credits:

http://twitter.com
http://jqueryui.com/demos/dialog/

Simplifying a javascript jump menu

2011-05-16T02:33:00.000-07:00

The jump menu is a reasonably common UI design pattern on the web. It is used by big guns such as Facebook and Twitter, and has been dissected and reproduced many times over.

When I set about implementing my own jump menu I found opening and closing the menu when it is clicked on was rather trivial.


$('.jump-menu').click(function(e){
   toggleMenu();
}

The difficulty came in closing the menu when there is a click outside the jump menu. Googling for answers, I found the scenario of closing a menu on clicking outside it to be a well trodden path. Stackoverflow had plenty of examples and possible solutions such as here and here. I even found a plugin created by Ben Alman that extended Jquery to provide a custom clickoutside event. Many of the proposed solutions were fragile, and I was slow to add a new plugin to my web application - as good and all as the implementation appeared. I really wanted to understand the problem before opting to take a proposed solution off the shelf.

The state diagram above illustrates the four possible actions for the two menu states of open and closed. We can see that one of the four actions results in no change of state, while another two result in the same change of state. Clearly this diagram could be simplified, and along with it my perception of the problem.

"Make things as simple as possible, but not simpler" - Albert Einstein

Given this simplified state transition diagram, there was no need for an event register, a global array, comparing with event.target, a handler that would be called for each and every document click, or the jquery plugin. All that I needed was something like the following:


    isMenuOpen = function(menu) {
      return menu.hasClass('ui-active-state');
    };
    openMenu = function(event) {
      var documentClicks, menu, self;
      menu = $(event.target).parents('.jump-menu');
      if (isMenuOpen(menu)) {
        menu.removeClass('ui-default-state');
        menu.addClass('ui-active-state');
        self = this;
        documentClicks = 0;
        $(document).bind("click", __bind(function(e) {
          documentClicks++;
          if (documentClicks === 2) {
            $(this).unbind(e);
            this.closeMenu(menu);
          }
          return this;
        }, this));
      }
      return this;
    };
    closeMenu = function(menu) {
      menu = $(this.el).find('.jump-menu');
      menu.removeClass('ui-active-state');
      menu.addClass('ui-default-state');
    };

There is always another way to look at a problem.
Happy Coding!

Credits:
http://davidwalsh.name/twitter-dropdown-jquery
http://stackoverflow.com/questions/152975/how-to-detect-a-click-outside-an-element
http://stackoverflow.com/questions/2868582/click-outside-menu-to-close-in-jquery
http://www.filamentgroup.com/lab/developer_your_own_jquery_themeroller_ready_components/

Compiling brunch from source

2011-04-21T02:17:00.000-07:00

Background:

Brunch is a client side web app framework built using a number of open source tools and libs from the world of javascript and node.js: CoffeeScript, Backbone, Eco templates, and more.

The steps

Pull source from Github
git pull https://github.com/brunch/brunch
check if you have all the brunch dependencies:
./bin/brunch --version
Building from source means you don't get the dependency management of npm as easily.
You must have node installed, and the required packages:
Note: at time of writing Stitch was not defined in npm dependencies but required
npm install stitch
Compile brunch with Cake:
cake watch
To create a project:
./bin/brunch new --output $locationOfNewProject

Nerd Tree: A File Explorer with Mac Vim

2011-04-15T00:34:00.000-07:00

The one thing missing from the great text editor that is Mac Vim is a project / file explorer side bar. Competing text editors such as Sublime and Text Mate have one, so why not MacVim? It doesn't seem like it would require too much additional work of the developers. Reading through debates the community has had on the issue, the reason appears to be the elitist and idealistic nature of the Mac Vim community. Please don't read that last sentence pejoratively; these guys are purists, and very good at what they do. There were quite a few in the community who believed NERDTree was more than enough for any code jocky's needs, so I decided to checkout what all the fuss was about.

I use pathogen to manage vim plugins, and it is especially useful in conjunction with a public source control system like github. Once you have pathogen installed simply pull the NerdTree Github project to your vim plugins directory.


cd ~/.vim/bundle
git clone git://github.com/scrooloose/nerdtree.git

Next add a shortcut to make opening your NERDTree more convenient by adding the following to your vim config file (~/.vimrc)


" Give a shortcut key to NERD Tree
map <F2> :NERDTreeToggle<CR>

Now fire up your MacVim, open a window, and press (fn+F2).
With NerdTree now started, the following commands should get you going:

Use the natural vim navigation keys hjkl to navigate the files, or just the arrow keys

Press o to open the file in a new buffer or open/close directory.

Press t to open the file in a new tab.

Press i to open the file in a new horizontal split.

Press s to open the file in a new vertical split.

Press p to go to parent directory.

Press r to refresh the current directory.

Tip:
Open a project with mvim + nerdtree from the command line like this:
mvim ~/Dev/path/to/project

Happy Coding!

References:
http://www.catonmat.net/blog/vim-plugins-nerdtree-vim
http://tammersaleh.com/posts/the-modern-vim-config-with-pathogen
http://wiredupandfiredup.blogspot.com/2008/09/pimp-your-macvim.html

Accessing appName property from controller in Grails

2010-07-28T08:37:00.000-07:00

It is trivial to access the appName property from within the config.groovy file, and it is a common practice to do so. Here is an example of specifying an external config file based on the name of the grails application:


grails.config.locations = ["file:$userHome/.osaas/$appName.groovy"

I could find no where obvious that described how to access this appName property from a constructor. You would expect the following to work but it does not:


grailsApplication.config.appName

The following does work however


grailsApplication.metadata['app.name']

Tracing DNS resolution

2010-07-05T02:49:00.001-07:00

When you need to trace how dns is resolving your server here is a way to do it.
First run this nslookup query:


nslookup -q=ns example.com a.root-servers.net
Authoritative answers can be found from:
com nameserver = h.gtld-servers.net.
com nameserver = d.gtld-servers.net.
com nameserver = k.gtld-servers.net.
com nameserver = g.gtld-servers.net.
com nameserver = c.gtld-servers.net.
com nameserver = f.gtld-servers.net.
com nameserver = j.gtld-servers.net.

Now choose one of the results and run the query again like this:


nslookup -q=ns example.com a.gtld-servers.net
Authoritative answers can be found from:
example.com nameserver = ns1.example.com.
example.com nameserver = ns2.example.com.
example.com nameserver = ns3.example.com.
ns1.example.com internet address = 10.12.12.12

You can repeat the above if you still haven't reached your destination

A Javascript best practice: one special semicolon

2010-06-07T15:13:00.000-07:00

It is becoming common practice to concatenate javascript files. Such concatenation means that only one HTTP request is required to download the necessary javascript rather than several. This reduction in HTTP overhead reduces the time taken to download the particular web application. This concatenation can introduce an unforeseen problem: if, in one file, some code is not properly finished (i.e. with a semicolon), and the proceeding file begins with a closure
being opened, the interpreter will break.

Jquery plugins often begin with closures and so are susceptible to this problem. Beginning the file with a semicolon will prevent this kind of problem. Where a file begins with a closure, best practice should dictate that it begins with a semicolon. An example of this practice in action is source code of the jquery form plugin.

References:
http://groups.google.com/group/jquery-dev/browse_thread/thread/2ce10bc1bcf6a8d1?pli=1

Overcomming the virtual hosts/HTTPs conflict on Amazon AWS

2010-05-28T02:32:00.000-07:00

Previously I've described the conflict between virtual hosts and HTTPs. For those that use Amazon's EC2 cloud, the below describes an elegant remedy for that conflict .

Assumed Basic Setup

For the purposes of this example, I'll assume you have an EC2 instance of Ubuntu Lucid with an apache server serving two websites: example.com and eazybusiness.com. Both have their own virtual host definitions, each containing it's corresponding site's specific configuration. If you need more information on how to configure your virtual hosts I direct you to apache's comprehensive documentation on the subject.

The virtual host definitions would look something like:


<VirtualHost *:80>
        ServerName www.eazybusiness.com
       
        ServerAdmin admin@eazybusiness.com
        DocumentRoot /var/sites/eazybusiness.com/
</VirtualHost>
<VirtualHost *:80>
        ServerName www.example.com
       
        ServerAdmin admin@example.com
        DocumentRoot /var/sites/example.com/
</VirtualHost>

The above definitions allow insequre, plain HTTP connections to both sites. The hostname provided by the browser client is matched against the servername's in the virtual hosts and the correct configuration is used accordingly.

Configure Amazon's Elastic Load Balancer

In order to secure our two websites, we'll first create the necessary proxy via Amazon's load balancer facility.

First configure your ec2 image to use the Amazon command line tools. Ubuntu this images make this easier thanks to the power of apt-get.

Seperate command line tools are required for ELB. Download the tools from Amazon, and unzip to somewhere on your image.

Set an environment variable AWS_ELB_HOME to point to where you unzipped the tools.

Now, let's create our first loadbalancer:

Here we configured loadbalacer to listen on port 443 and port 80. All requests to port 80, are to be forwarded to port 80, and requests to 443 are to be forwarded to 8445
Note: amazon's api tools default to the region us-east-1. If you wish to use any other region, you must provide the --region parameter

Create a second loadbalancer for example.com:

Note: the example.com loadbalancer will forward requests to port 443 to a different port than the eazybusiness.com loadbalancer

Next, register our instance with both load balancers:


elb-register-instances-with-lb eazybusiness --region us-west-1 --instances $instanceId
elb-register-instances-with-lb example --region us-west-1 --instances $instanceId

Configure Apache

Apache must start to list on the ports that the load balancers will forward to. To this, edit /etc/apache2/ports.conf and add the lines:


Listen 8443 https
Listen 8445 https

To confirm the ports are being listened on try the following commands:


netstat -na | more
nmap -sT -O localhost

Ensure mod_headers is enabled

sudo a2enmod headers

Configure Virtual Hosts

The virtual hosts need to be configured for SSL, and reflect the ports defined for the loadbalancers. In our example, the virtual hosts become:


<VirtualHost *:80>
        ServerName www.eazybusiness.com
       
        ServerAdmin admin@eazybusiness.com
        DocumentRoot /var/sites/eazybusiness.com/
        SSLEngine On
        SSLCertificateFile /etc/ssl/eazybusiness.com.crt
        SSLCertificateKeyFile /etc/ssl/eazybusiness.com.key
        RequestHeader set X_FORWARDED_PROTO 'https'
</VirtualHost>
<VirtualHost *:80>
        ServerName www.example.com
       
        ServerAdmin admin@example.com
        DocumentRoot /var/sites/example.com/
        SSLEngine On
        SSLCertificateFile /etc/ssl/example.com.crt
        SSLCertificateKeyFile /etc/ssl/example.com.key
        RequestHeader set X_FORWARDED_PROTO 'https'
</VirtualHost>

Note: the RequestHeader directive requires that you enable mod_headers for Apache.

sudo a2enmod headers

Configure DNS

The AWS console displays a list of load balancers. From this list, glean the relevant DNS names, and make the relevant alterations to the dns server like this:


eazybusiness.com. IN CNAME eazybusiness-2028819496.us-west-1.elb.amazonaws.com.
example.com. IN CNAME example-2028229496.us-west-1.elb.amazonaws.com.

Note: don't forget the proceeding dot when using full domains.

Configure Amazon Firewall

Ensure the specified ports open, which in our example are 8445 and 8443.

To confirm the port is open with Telnet try:


telnet $public_dns_of_image 8445

After completing all of the steps above you should now be able to support multiple secure websites sharing the same webserver.

References

Major credit goes to this post

Multiple HTTPs websites on one webserver

2010-05-27T08:40:00.001-07:00

A long time ago, servers were very slow and a web server could only handle one web site. In time, increased computing power and virtual hosts technology allowed one webserver to handle multiple web sites. At the same time as websites started to share the one webserver, HTTPS was coming to the fore as the defato means of securing web communication from fraudsters, snoopers, and other nefarious trades. Unfortunately, due to the nature of the technologies HTTPS and virtual hosts were not fully compatible.

Virtual hosts work based on host/server names. Examples of hostnames are example.com, eazybusiness.com, and blogger.com. The hostname provided by the client browser determined what one of its web sites the webserver servered to that client.

In order for a website to be able to offer the security of HTTPS, it must provide its own unique SSL certificate. This SSL cert is proof of the site's authenticity, and forms part of the protocol used to secure the communication between a user's browser client and the webserver.

The conflict between virtual hosts and HTTPs lies in the fact that the SSL protocol (the 's' in HTTPs) does not include the need for the client to pass the hostname to the server during initiation. This missing information means that if a webserver hosts multiple websites, each with their own SSL certificate, it does not know which SSL certificate to use.

Recently browsers and websites have starting supporting a technology called SNI. SNI enables the hostname to be included part of the SSL protocol. SNI works, and solves the conflict perfectly, but unfortunately the prevelance of older browsers makes using it impractical.

An alternative solution is to use multi-domain certificates. Using multi-domain certs means that all the web sites sharing a webserver can also share the one certificate. The webserver then only has the one certificate to use, and so does not need to know information about the hostname when establishing an SSL connection. This solution works, but these certificates can be expensive and come with a number of limitations:

There is generally a limit to the number of websites a multi-domain cert can accomodate (usually around 100)

Multi-domain certificates do not accomodate wild-card certificates

An third possible solution to this problem is to use a proxy, and have each website operate on a different port. My next post will contain a solution to the multiple https websites on Amazon's AWS.

Error sending Mail: 501 Syntax: HELO hostname

2010-05-25T23:53:00.001-07:00

The exact error I found was:


javax.mail.MessagingException: 501 Syntax: HELO hostname

To fix it I had to set the hostname of the server


hostname $hostname.tld

Grails/Groovy deleting an element in a collection while iterating over it

2010-05-25T04:47:00.000-07:00

I ran into a recent problem whereby I wanted to delete a sub-set of the elements in a list after processing all elements in the list. The general gist of the logic was:


apples.each { apple ->
   if(apple.red) apple.delete()   
}

which resulted in a java.util.ConcurrentModificationException exception.

The solution is to remove the element from the interator before deleting it.


Iterator itr = apples.iterator()
while(itr.hasNext()){
    def apple = itr.next() 
    itr.remove()
    apple.delete()
}

For more a little more detail see this thread

Dynamic methods with javascript

2010-04-20T06:42:00.001-07:00

I recently had an implementation requirement to be able to call a method given a string with the name of the method.

I came across two strategies for achieving this:

1. Eval


function a(){
alert('a called');
}
eval('a');

2. Map


function a(){
alert('a called');
}
window['a']();

Daylight Saving Time for Ubunu Linux

2010-03-30T08:35:00.000-07:00

If you find the time of your Ubuntu machine out by an hour due to DST, then the following steps might sort it out for you:

locate the name of the timezone that is suitable for you
```
         ls /usr/share/zoneinfo
    
```

Use zic to set the zone correctly for your localtime
```
         zic -l $zoneName
     
```

Copying EC2 Instances Between Regions

2010-03-04T05:58:00.001-08:00

On Amazon AWS EC2, there are different regions. Presently there are three regions. EU-West, US-West, and US-East. To move/copy instances between regions is not trivial, and is not possible using the GUI (either elasticfox or amazon console).

The following is what has worked for me to copy from EU-WEST to US-WEST:

Determine the name of the bucket your current instance is stored at

The bucket name is usually part of the manifest url and is something of the form:
$bucket_name/$manifest_name.xml

Determine suitable kernel and ramdisk ids in the target region

Each AMI image requires references to suitable kernels and ramdisks. This references differ from region to region, so suitable ids need to be researched for your target region. For best results, select akis and aris that are as close as possible to those in the original region


ec2-describe-images -o amazon --region $target_region | grep $kernel_description

Using the API tools provided by Amazon, run the following Command


./ec2-migrate-image -o $Access_Key -w $Access_Password --region eu-west-1 --bucket $name_of_existing_bucket --destination-bucket $new_bucket_in_us --manifest $manifest_name.manifest.xml --location us-west-1 --kernel=$suitable_kernel_id --ramdisk=$suitable_ram_disk_id

Note: Transferring ECB blocks currently needs to be done out of band. This involves using rsync, dd, or equivalent.

Resources

Amazon API Tools Reference

Migration Tutorial

Running an instance with a different kernel

Jar Scanning

2010-02-05T08:46:00.000-08:00

Sometimes it can be helpful to track down where exactly a class is within a library of jars.

One means of doing it is to run something like the following from the command line:


find . -name \*jar -print -exec jar -tvf {} \; | nawk '/YOURSEARCHSTRING/ || /jar/ {print} '

Another means is to use a java based jarscan tool.
The tool comes with a .bat file, which is not exactly ideal for linux development.
I created an equivalent .sh file with the following contents:


java -jar $JARSCAN_HOME/jarscan.jar $1 $2

and added an alias to my ~/.bashrc file of the form:


alias jarscan='~/dev/apps/jarscan/jarscan.sh'

Now you have a command line jarscanner. Navigate to any directory, and to run a recursive jar search enter a short command like the following:


jarscan MBeanServer

Maven Tricks

2010-02-05T05:30:00.001-08:00

see all the libs that end up on the classpath of your maven project:

mvn dependency:build-classpath | sed 's/:/\n/g' > cp.txt

Jar search:
find . -name \*jar -print -exec jar -tvf {} \; | nawk '/YOURSEARCHSTRING/ || /jar/ {print} '

or use jarscan tool

bash script:
java -jar $JARSCAN_HOME/jarscan.jar $1 $2

alias added to .bashrc:
alias jarscan='~/dev/apps/jarscan/jarscan.sh'

and your good to go. Now you can run a recursive search of jars in any directory with a command like:
jarscan MBean

The Wonder of GIT

2010-02-05T05:09:00.000-08:00

Setup your identity:


git config --global user.name "FirstName LastName"
git config --global user.email "user@example.com"

Make git a little prettier:


git config --global color.ui "auto"

The basic format of the push operation is:


git push $repository-location $source-branch:$dest-branch

Publish a local branch, remotely:


git push origin $local-branch:$remote-branch

Delete a remote branch:


git push origin :$remote-ref

Once a remote branch is deleted, other team members must delete references to the deleted branches by running:


git remote prune origin

Create a local branch to track a remote branch:


git branch --track local-branch-2 origin/remote-branch-2

Work on a remote branch other than master


git fetch // to ensure git local knows about remote branch
git checkout --track -b $local-branch origin/$remote-branch

To find some code you previously had in a file


// Find the relevant commit
git log /path/to/file
// Now view the changes in that file since that particular commit
git diff $commitHash /path/to/file

To revert just one file to a previous commit
git checkout $commitHASH /path/to/file

To see who is responsible for the current state of a particular file:


git blame $fileName

Sources:

http://www.sourcemage.org/Git_Guide

Working With HTML's Underrated UL and LI Tags

2009-12-14T02:09:00.000-08:00

I think it is important to note that the HTML tag ul is short for unordered list. Many novice html coders think ul and li (and indeed ol) tags are for bullet points, but the main purpose of these tags is to define lists. A list can run verticaly, or horizontally, and elements can be separated by any manner of means - the default rendering of a list just happens to be that of a series of bullet points.

Tutorials illustrating how to use css to turn a list into a top navigation bar for a website abound on the web, so I'm no need to go there. Instead, I'm going to illustrate how to use the ul and li tags to get a nice bottom navigation, that would typically include a privacy policy, and a copyright declaration.

First lets define the list of elements to go in our bottom navigation in HTML:



    © Copyright 2009 
        EazyBusiness
    

    Company

    Support

    Privacy Policy

    Terms of Service

    Affiliates

Now that we have defined the content in a structured way, we have to turn our attentions to the style.


#footer-nav li{float:left;list-style:none;}
#footer-nav li:before{content:"\2014\a0";}

The above css removes the default list style of the HTML elements, and instead adds a dash before each element. Combining the style, and the data structure, we get the following list:

Now we have our navigation list running horizontally, separated by dashes, and all that is needed is for us to tidy it up a little bit. In doing that our HTML became:




    © Copyright 2009 
        EazyBusiness
    

    Company

    Support

    Privacy Policy

    Terms of Service

    Affiliates

And our CSS became:


#footer-nav li{float:left;list-style:none;font-size:10pt;color:#fff;}
#footer-nav li:before{content:"\2014\a0";color:#fff;margin-left:2px;}
#footer-nav li.first:before{ content:"";margin-left:0px;}
#footer-nav a:link, #footer-nav a:hover, #footer-nav a:visited{color:#fff;text-decoration:none;}
#footer {float:left;padding:10px;background-color:#039;}

And our menu became:

Using Groovy Scripting To Remove Unwanted MySQL Databases

2009-12-09T07:03:00.001-08:00

Doing some Autumn cleaning on a server, I wanted to delete a lot of redundant mysql databases. There were over 900 databases, of which about 90% could be deleted or archived.

To create a list of databases from a terminal window first create a file show_db.sql
:


echo "show databases" > show_db.sql

Then use this file to generate a text file containing a list of your databases


mysql -p < show_db.sql > dblist.txt

The resulting text file had unwanted characters and spaces. Fortunately file manipulation is easy in groovy.


def file = '/tmp/dblistOriginal.txt'
def targetFile = new File('/tmp/dblistClean.txt')
if (targetFile.exists()) targetFile.write("")
new File(file).text.eachLine { line ->
    targetFile.append( line.replace('|', '').trim() )
}

I reasoned it was better to keep the archieve and deletion tasks separate incase something went wrong in the process. So, firstly I created an archieve script to dump all the databases, and tar them into one file.


def dbsToPreserve = new File('/tmp/dbsToSave.txt') // list of databases to exclude
def dbsList = new File('/tmp/dblistClean.txt')
def archiveScript = new File('/tmp/archiveAndDeleteDBs.sh')
def password = 'pwd12345'
def scriptBackupFolder = 'archived_dbs'

dbsToPreserveList = []
dbsToPreserve.text.eachLine { line ->
    dbsToPreserveList << line.trim()    
}

// Archive Script
if (archiveScript.exists()) archiveScript.write("")
archiveScript.append('#!/bin/sh \n')
dbsList.eachLine { line ->
    if(!dbsToPreserveList.contains(line)){
        def dbName = line.trim()
        archiveScript.append("mysqldump -p$password $dbName > /tmp/$scriptBackupFolder/${dbName}.sql \n")
    }
}
archiveScript.append('cd /tmp \n')
archiveScript.append("tar -czf dbArchives${(new Date()).getTime()}.tgz $scriptBackupFolder \n")

Next, I created the sql to delete all the relevant databases


// SQL Script
if (sqlScript.exists()) sqlScript.write("")
dbsList.eachLine { line ->
    if(!dbsToPreserveList.contains(line) && !line.contains('-')){
        def dbName = line.trim()        
        sqlScript.append("DROP DATABASE IF EXISTS $dbName;\n")
    }
}

Using my newly created files, has saved me so much time, and kept me from doing oh so tedious work. Thanks again Groovy!

Overriding Domain Class Getters and Setters in Grails

2009-12-08T07:41:00.001-08:00

One might assume that it is possible to override getters and setters in the following manner:


def setName(String name){
//do stuff
}
def getName(){
//do stuff
return name
}

but infact static typing is required


public void setName(String name){
//do stuff
}

public String getName(){
//do stuff
return name
}

My Data Nest Registration Error

2009-12-05T11:02:00.000-08:00

mydatanest can't handle us Irish, or at least our names.
This error appeared to me, on my first attempt at registration:


MySQL Error: You have an error in your SQL syntax;
 check the manual that corresponds to your MySQL server version 
for the right syntax to use near 'Connell',`address`='Dublin',`refferal_id`=NULL,`phone`='',
`max_messages`='20000'' at line 1 on line 244 
in /home/datan3st/public_html/account.php

Grails doesn't handle gstrings in controllers

2009-12-04T01:19:00.000-08:00

You would expect Groovy gstrings and plain strings to be interchangable in Grails, but today I discovered that it is not so. In a Grails controller calling the redirect operation like this works:


redirect(action:'step1') // redirects to step1

but like this does not, and it redirects to the default index action


redirect(action:"step${getNextStep()}") //redirects to next step if it worked

the temporary solution to achieve dynamic actions is to convert the gstring into a string like so:


redirect(action:"step${getNextStep()}".toString())

Kris Kindle / Secret Santa the Groovy Way

2009-12-03T07:00:00.000-08:00

In efforts to explain to folks in the office how secret santa works I directed them to take a look at this secret santa definition or alternatively to examine the following groovy code:


def ezbPeeps  = User.findByCompanyName('eazybusiness')
def hatOfNames = ezbPeeps.collect{it.name}
ezbPeeps.each { peep ->

        def goodPick = null
        while(!goodPick){
                def randomInteger = RandomInteger.getGenerate()
                assert randomInteger > 0
                def index = randomInteger%hatOfNames.size()
                switch( hatOfNames[index] ){
                        case peep.name:
                                goodPick = false
                                break
                        default:
                                goodPick = true
                                print "${peep.name} gets ${hatOfNames[index]}"
                                hatOfNames.remove(index)
                                break                        
                }
        }       
}

mistakes on a postcard to Eamonn O'Connell, Saasplex, 3015 Lake View
Drive, Citywest, Dublin, Ireland

Ubuntu adds did you mean to command line

2009-12-03T04:22:00.000-08:00

Just miss spell a command in the the terminal window to see what I mean.

I mistyped pico as pioc and recieved the following response:


pioc
No command 'pioc' found, did you mean:
 Command 'pico' from package 'nano' (main)
 Command 'pico' from package 'alpine-pico' (universe)
 Command 'pic' from package 'groff-base' (main)
pioc: command not found

This appears to be one of the added benefits of the Karmic release. Pretty cool!

Git Pack Error

2009-11-30T07:30:00.000-08:00

I believe this error occurred when a pack operation was interupted during a push operation:


Counting objects: 338, done.
warning: suboptimal pack - out of memory
fatal: Out of memory, malloc failed   
error: pack-objects died with strange error
error: failed to push some refs to 'ssh://build@192.168.0.17/~build/dev/git-public/platform/billing/billing'

The solution that worked for me:


git repack -adf --window-memory=100M

Hibernate Exception in Grails when dealing with exceptions

2009-11-30T05:29:00.000-08:00

HibernateException: Found shared references to a collection

Like me, you may have have caused this error by having something like the following


sale2.discounts = sale1.discounts
sale2.save()

The solution in my case was to do the following:


sale1.discounts.each { d->
    sale2.discounts << d
}

Quick Solution to Godaddy Certs not working in Java

2009-11-30T04:35:00.000-08:00

If your experiencing problems with godaddy certs (generally an SSLHandshakes error) when using Java (perhaps with HttpClient), the following may solve your problem quickly.

Note: this was done on Ubuntu, so you may need to adjust it to your OS.


wget 'http://blogs.sun.com/andreas/resource/InstallCert.java'

javac InstallCert.java

java InstallCert $mydomainname 
#respond to prompt with return

sudo cp jssecacerts $JAVA_HOME/jre/lib/security

grails remotefunction limitation

2009-11-13T04:56:00.000-08:00

I stumbled apon a limitation of the grails remote function today. I have two drop down lists Products and Prices, with the Prices dependent on the particular Product selected, and also the customer in question. In order to achieve this, I needed to pass two params to the remote method.

There are two ways of using the params attribute of the remotefunction.

Option 1:

You can pass a string


<select onchange="${remoteFunction(action:'getRightPriceForProduct',update:'price', params:'\'productId=\' + this.value\'' )}">

    
</select>

Option 2:

you can pass a map


<select onchange="${remoteFunction(action:'getRightPriceForProduct',update:'price', params:[customerId:customer.id])}">
    
    
</select>

Option 1 allows me to use 'this' in javascript to send the product selected, and option 2 allows me to pass the customer's id at the page rendering time, but neither option allows me to do both at the same time.

One solution to choose option 1, and use document.getElementById to retrieve the value of the customer id


<select onchange="${remoteFunction(action:'getRightPriceForProduct',update:'price', params:'\'productId + \'&document.getElementById(\'customerId\')'
params:'\'customerI=\'document.getElementById(\'customerid\').value\'&productId= \' + this.value\')}">


</select>

A better solution is to implement your own taglib funtion like this


def ezbRemoteFuction= { attrs, body ->
    def params = attrs['params']?:null
    if(params){
    String pString = "\'"
    params.each { key, value ->
    pString += "${key}=${value}&"
    }
    pString += "\'"
    attrs['params'] = pString;
    }
    out << g.remoteFunction(attrs, body)
    }
}