Recently at work we’ve been in heavy development mode on a new thick-client application architecture.  If you aren’t familiar with the concept, thick-client essentially means putting most of the work onto the client’s processor, and removing it from the server.  The benefits of this are many, not the least of which is a much faster site response time for your users, and a much lighter load for your server.

To accomplish this basically means using javascript and a lot of AJAX, as well as implementing the application such that that a) search engines can still index your site and b) the user can both bookmark and use the back button wherever they are in your app.

In order to facilitate this heavy use of ajax and management of state, a number of MVC(ish) patterned javascript libraries have been created, and more appear seemingly every week.  After a little bit of research, we settled on Backbone due to its barebones nature, low overhead, lesser learning curve, and the size of the community.  It always helps to be able to reach out when you get stuck.

In combination with jQuery (we are already using it heavily), and Require.js (a dynamic resource loader which has also been in use for quite a while), we had the perfect trinity of tools to get the job done.

Basic App Structure

There are a lot of things to consider when architecting a large application.  For HMS, this is compounded by the fact that a) we have a lot of users with a lot of different site configurations, b) we have different variations on the main backend application, c) there is a lot of legacy code and newer front-end logic that needs to maintain compatibility so that pieces can be added one at a time.   This is where Require and its support of AMD (asynchronous module definition) really shines.

All of our thick-client application code is managed through AMD, and there are two things about it that are very awesome:

1. It allows seamless integration on a module-by-module basis with the current application code
2. Even portions of modules can be broken out and used before their parent module is complete – for example, we already have the autocomplete module live, even though the search module is still in development. When the time comes, it will be trivial to move it back into its proper place.

The general application structure goes like this:

1. Event Aggregator
   Outlined here, we have a single global object, and its job is to manage communication between modules.
2. Outer Router
   We have two routers.  This helps keep the application router a lot simpler, and keeps module related logic inside the module. The outer router manages routes for the entire application, and determines which modules to load.
3. User Module
   Right now, this is the only module that sits everywhere in the application. As such, it is loaded by the outer router.
4. Modules
   Each module contains its own router as well as templates and views. The router loads the views, the views load the templates (we’re using Hogan).  When we compile with require, each module (including templates) becomes a single file (the module router).
5. Models
   The models are the glue between the client and the server,  and they sit in their own folder because multiple modules may use the same models.
6. Form Mediator
   This is actually one of the modules, but its worth describing here.  We use a special Backbone view to handle every form, enhancing functionality as we go.  This view simplifies forms immensely, managing the model, validation, and state with ease.  Its special power is to allow multiple modules to combine into a single form definition – most usefully for search / advanced search.
7. UI Modules
   Most DOM manipulation (outside of insertions/removals)  and all UI effects are offloaded into separate UI modules.  The reasons for this are a) it separates presentation concerns and b) it allows for a much easier time should we decide to switch from jQuery for future manipulation (one can dream of full standards support across browsers)
8. Backbone.Store (localStorage)
   Of course we’re using this for our app.  Currently it persists forms, helps maintain the user session client-side, and caches model data.

Obviously, there are many ways to architect a thick-client application.  Our approach intends to prevent module dependency (outside of UI Modules), separate DOM manipulation from data logic, keep communication centralized in an event aggregator, load quickly (through smart module loading and use of localStorage, among other things), and preserve the global namespace through the use of AMD.  There are a lot of details I’ve left out (for instance reuse for mobile), but I do hope to dive a little deeper in future posts.

1. Backbone Fundamentals
   Open source e-book, started, curated, edited by Addi Osmani.  Still being written, but in time sure to become the defacto free reference to Backbone development.  There are lots of other good resources at the bottom of this page.
2. Event Aggregator using Backbone
   An elegant solution to a common application problem.
3. Data-binding with Backbone
   Derek  Bailey (see #2) is awesome.
4. Understanding MVC And MVP (For JavaScript And Backbone Developers)
   Addi Osmani’s post on the history of the aforementioned patterns, and how they relate to the Backbone world.
5. Recipes With Backbone ($24)
   Okay, this one isn’t free, but it goes through some useful Backbone patterns for dealing with common application architecture concerns.
6. Peepcode Backbone Screencasts ($12)
   If you are a total Backbone beginner, these screencasts are worth every one of those twelve dollars.  Peepcode has a few Backbone related screencasts on their site, so don’t just settle for the basics (which is linked here)

And here is a bonus link because it’s worth looking at and following the links that branch off of it:

Javascript Patterns Collection  by Shi Chuan – a list of patterns and anti-patterns to make your javascript awesome.

There was already a great library for Backbone to utilize HTML5 localStorage functionality; however, there were a few drawbacks to the library that caused me to look in another direction, and ultimately led me to creating Backbone.store.

My two greatest needs were:

1. The ability to utilize the storage mechanism as a cache.
2. A greater range of compatibility outside of localStorage support.

To meet these needs, I turned to Lawnchair, a JSON storage library with a wide range of adapters for different storage types, mixed in a little bit of the aforementioned backbone.localstorage plugin, and came out with the beginnings of a (hopefully) useful storage middleware.

While the overall project is in very early stages, its not too soon to look at, play with, and provide feedback on.

Download / clone from Github.

Features:

1. Can be used with or without a backend server
2. Configurable cache time to live
3. Configure when to use server vs cached model – great for list  vs. detail of a record
4. Does not require localStorage support, though a Lawnchair adapter may be needed to get the support you want.

Planned enhancements:

1. Batch model upload to server (online/offline or time delayed)
2. Parameter based caching (for caching paging / search results / etc)
3. Remove Require (AMD) dependency
4. ??? (accepting ideas)

Also included in the project is a basic ‘Todo’ style application including a CodeIgniter (php) based RESTful web server and structure, and a Backbone/Require/jQuery front-end consisting of a list/detail view with complete CRUD control and proper routing using both direct (bookmarkable) URL access and pushState routing.

Again, this project is nowhere near complete, so I’m sure there are some bugs… Still on the todo list is to add test cases using Sinon and Jasmine, and there is a known issue with the list -> detail -> list navigation.

DynoMenu was conceived to solve a problem on HomesConnect.  Since HomesConnect is a CMS that allows users to add new pages and new menu items to their navigation, and since the navigation is horizontal across the top of the page, the need arose to be able to maintain the layout of the templates while removing the burden of keep the navigation usable from the agent.

The solution was this plugin, which will first try to shrink the navigation font to a determined size, and if failing will systematically remove navigation items until it achieves the best fit. These items are placed in a ‘more’ link which becomes the last main navigation item, and can be accessed from there.  Since the agent can determine the order of their navigation via a drag and drop interface on the backend, it is easy for them to decide which items will fall under the ‘more’ link by pushing them to the end of the navigation.

In any case this may not have much use outside of CMS systems where you cannot control the total number of nav items, but who knows.. maybe there is a use case I’m not considering.

Download it from Github here.
See a demo.

So… I’ve redesigned the main portion of my site (its only 1 page!), and will shortly get the updated design into the blog. Until then, its plenty usable, it just doesn’t match (but then, according to my wife I rarely do either).

Luckily, there is a fairly easy workaround for this situation as long as you are not trying to do too much with the pngs. It will even fix pngs in IE6 for you too (in fact, that is the real purpose of the script). That workaround is the jQuery ifixpng2 plugin, written by Yereth and available here.

All you need to do is install the plugin and use it on the PNG containing objects (images or backgrounds) that you would like to fade. There are two changes you will need to make however:

Line 64 looks like this:

ltie7	: $.browser.msie && $.browser.version < 7

You’ll want to change the 7 to a 9 (in hopes that this is fixed for good by IE 9, otherwise you’ll need to go back and change it to 10 later).

The other change you need to make is on Line 149, which looks like this:

for (prop in expr) elem.style.setExpression(prop, expr[prop], 'JavaScript');

Internet Explorer 8 doesn’t support setExpression, so it will throw an error, unless you add an if statement to keep it from running:

if( $.browser.msie && $.browser.version < 8 ) {
for (prop in expr) elem.style.setExpression(prop, expr[prop], 'JavaScript');
}

That’s all there is to it. Happy fading!

While this tutorial is geared towards Mac users, most of the steps should be similar for Windows users.

Step 1

Download and Install MAMP

MAMP is the OSX version of the one-click Apache-MySQL-PHP install.  If you’re on a PC, you want to try out XAMPP instead (XAMPP also comes in an OSX flavor, but I haven’t personally tried it).  In any case, you’ll want to download one of these two programs and follow the instructions for installation.

Step 2

Download Code Igniter

Code Igniter is an MVC based PHP framework created by the makers of Expression Engine – a CMS/blogging platform. What (I think) sets CI apart from other frameworks like Cake or Zend is its soft learning curve and strong documentation. The involved community doesn’t hurt either. You’ll want to download it in order to continue to step 3.

Step 3

Install Code Igniter

Once you’ve downloaded and unzipped CI, you’ll want to copy the folder over to your server root. If you’re using MAMP – the default is /Applications/MAMP/htdocs. At that point, you should change the folder name from CodeIgniter_1.7.1 to whatever you want to use to describe your site. I changed my folder name to ‘ci’. That’s it for installation.

Step 4

Setup Your MySQL database

Odds are, you’re following this tutorial so you can build a dynamic site in PHP. Dynamic nearly always means database, so you’ll have to create one. You can do this in MAMP by switching over to your running instance and clicking on ‘Open start page’. From the links on top, you can select ‘phpMyAdmin’. Otherwise, you can probably just copy and paste this: http://localhost:8888/MAMP/frame.php?src=%2FphpMyAdmin%2F%3Flang%3Den-iso-8859-1&language=English. From there, create a new database (I’ve called mine ‘ci’).

A note here: CI can handle sessions either with or without the database. My personal preference is to use the database, because it provides greater functionality, and you don’t have to worry about hitting your 4k cookie size limit. Enabling database sessions is fairly simple in CI, and we’ll get to that further down. However, you’ll first need to add the sessions table to the database. The default name of the sessions table is ‘ci_sessions’ – you can change this here and in the settings, but remember to do it in both places or this won’t work. In this case, I’m going to keep it the same. Code Igniter provides the SQL you’ll need here, so you can click on the ‘SQL’ tab in phpMyAdmin and copy/paste the code in. Here it is:

CREATE TABLE IF NOT EXISTS  `ci_sessions` (
session_id varchar(40) DEFAULT '0' NOT NULL,
ip_address varchar(16) DEFAULT '0' NOT NULL,
user_agent varchar(50) NOT NULL,
last_activity int(10) unsigned DEFAULT 0 NOT NULL,
user_data text NOT NULL,
PRIMARY KEY (session_id)
);

Step 5

Edit your Code Igniter Settings

From this point, you’ll want to edit your CI settings to match your environment. The first thing we’ll do is open up the system->application->config folder. In here are all the files that run the configuration settings.

config.php

On line 14, you’ll see this:

$config['base_url']	= "http://example.com/";

You’ll want to change it to your own install directory. You can also edit your vhosts file to have it match your live url… but that’s a more advanced procedure and not part of this tutorial. So in this case, I’ve changed it to:

$config['base_url']	= "http://localhost:8888/ci/";

The next portion of code you might need to change is from lines 234-242, and it looks like this:

$config['sess_cookie_name']		= 'ci_session';
$config['sess_expiration']		= 7200;
$config['sess_encrypt_cookie']	= FALSE;
$config['sess_use_database']	= FALSE;
$config['sess_table_name']		= 'ci_sessions';
$config['sess_match_ip']		= FALSE;
$config['sess_match_useragent']	= TRUE;
$config['sess_time_to_update'] 	= 300;

This is where you can setup your sessions to use the database, as outlined above. Just change $config['sess_use_database'] to true.

There are other settings in config.php you can modify to suit the framework to your needs, but those are the only ones I typically use.

database.php

This is the file where we’ll set our database access – location, username, password, and database name. It looks like this:

$active_group = "default";
$active_record = TRUE;

$db['default']['hostname'] = "localhost:8889";
$db['default']['username'] = "root";
$db['default']['password'] = "root";
$db['default']['database'] = "ci";
$db['default']['dbdriver'] = "mysql";
$db['default']['dbprefix'] = "";
$db['default']['pconnect'] = TRUE;
$db['default']['db_debug'] = TRUE;
$db['default']['cache_on'] = FALSE;
$db['default']['cachedir'] = "";
$db['default']['char_set'] = "utf8";
$db['default']['dbcollat'] = "utf8_general_ci";

MAMP automatically sets your username and password to root, and your MySQL port to 8889, so can copy and paste the above for working locally. You can also create a second set of these variables called $db['production'] for example, with your live settings in it. When you are ready to move the site, all you’ll need to do is change your $active_group variable to ‘production’.

autoload.php

Code Igniter gives you the ability to add only the libraries and functionality you need for your project. The autoload.php file lets you name which files you want across the application. You can read more about it on the CI site, but one thing you’ll likely want to do right away is enable your database and session libraries to load. This is done on line 42:

$autoload['libraries'] = array('database','session');

Odds are you’ll also want to load the URL helper, that allows you to enter shortcut functions for some common locations (like the base url). You can do this on line 54:

$autoload['helper'] = array('url');

.htaccess

CI doesn’t come with an .htaccess file, but odds are you’ll want to add additional folders to your root ‘ci’ folder while keeping the rest of your site secure by redirecting most requests through the main index.php file. So, create a file named .htaccess in your main ‘ci’ folder. Within it, you’ll want to do this:

RewriteEngine on
RewriteCond $1 !^(index\.php?|images|css|js)
RewriteRule ^(.*)$ http://localhost:8888/ci/index.php?/$1 [L]

What this does in tell the Apache URL-rewrite module to redirect any request that isn’t in the above list to the index.php file of our ‘ci’ directory. I’ve added images,css, and js folders to the ‘ci’ directory so I can start adding assets there. When you want to link to your files, you’ll be able to do so like this:

That’s all there is to it. At this point, you should have all of the basic resources you need to start developing a sweet app in Code Igniter. When you are ready to move your site to a production server, just remember to change your database settings, base url in config.php, and the RewriteRule in your .htaccess file, and you’ll be up and running in no time.

In the old days (2 years ago), working the Javascript magic to create a cool AJAX based event took a fairly decent working knowledge of the mechanisms behind the process.

With the increasing popularity of Javascript libraries however, this type of functionality became available to the web site hobbyist, and was made much easier for the web site professional.

The following step-by-step tutorial will show you how to combine the power of JQuery (a javascript library that weighs in at about 20k) with Code Igniter (a PHP framework based on the MVC design pattern) to quickly and painlessly pass a record ID through the javascript and over to the server, where it will be passed to a mysql database, used to retrieve some data, and sent back to the page for display.

Step 1
We begin by assuming that you have a div with an id of content, which is where you would like your freshly retrieved data to display, once it has been freshly retrieved. For this exercise, you have already taken an action to call your javascript function with a record ID parameter.

The first thing you need to do, is make sure JQuery is being loaded, and to create a function for your AJAX request.

Step 2:
Next, youll use the JQuery function load, and attach it to your content div:

function get_record_id(record_id) {
     $('#content').load()
}

Step 3:
The load function accepts three arguments. The page to be called on the other side of the HTTPRequest, the array to pass through the POST, and a callback function. It looks like this:

function get_record_id(record_id) {
     $('#content').load(/controller/method,p,function(str){

	 });
}

Lets go back to that. Code Igniter URLs are created by calling the name of your controller, followed by the function inside the controller class that will handle your request. If your server does not support mod-rewrite, you may also need to append an index.php to the beginning. The str inside the callback function is the results of your AJAX request. There isnt much use for the str when using the .load function, but it does come in handy using the other JQuery AJAX functions – $.post and $.get, which I assume are self explanatory.

Step 4

var p = {}; //instantiate the array
p[record_id] = record_id //assign your record_id variable to it.

Thats all there is to it. Your final javascript function looks like this:

function get_record_id(record_id) {
     var p = {};
     p[record_id] = record_id
     $('#content').load(/controller/method,p,function(str){

     });
}

Step 5
On the CI side, you have a controller and method setup something like this:

class Controller
{
   function Controller()
   {
       parent::CI;
   }

   function method()
   {
   }
}

The important part is the method() function, as it will contain some of the code we need to make things happen.

Step 6
The first thing you need to do on the CI side is retrieve the value passed through the request object. This is simple enough, using $_POST[record_id]. You also want to load up your database model so you can get the record out of your database. So, well load the database library, and then load the actual model. Then, we want to send the record ID to the database, get the resulting data, and pass it back out to the request. our function starts to look like its doing something useful pretty quickly.

function method()
{
   $record_id = $_POST[record_id];
   //set the record ID
   $this->load->library(database);
   //load the database library to connect to your database
   $this->load->model(records);
   //inside your system/application/models folder, create a model based on the procedure
   //outlined in the CI documentation
   $results = $this->records->get_record($record_id);
   //get the record from the database
}

Step 7
At this point, we need to go into our records.php file in the model folder. Since Code Igniter uses a Model-View-Controller structure, database activity, server-side processing, and client-side display should be as separate from one another as possible. You dont NEED to do this for Code Igniter to do its thing, but its good practice.

Inside the records.php file, well create a method called get_record to match the method referenced above. Well use it to get a record by its primary key of ID, put the resulting data into an array, and send it back to the controller, out to the view, and ultimately into the content div we started with.

function get_record($record_id)
{
   $this->db->where(ID,$record_id);
   //we want the row whose ID matches the value were passing in
   $query = $this->db->get(record_table);
   //get the table and put it into an object named $query
   $row = $query->row();
   //gets the first row of the resulting dataset.  In this case, only 1 row will ever be returned
   $results[record][$row->ID][name] = $row->name;
   //here, we create a multi-dimensional array holding the returned values
   //based on the key.
   return $results;
   //send the record back to the controller
}

The trickiest part of this section is the array. It seems pretty complex from here, but youll see soon enough how it breaks down into something more manageable as we go along.

Step 8
Were back to the controller again, and we have one more line to add – this time to pass the resulting data into a view to be formatted and printed to the content div. The whole method() function now looks like this:

function method()
{
   $ID = $_POST[record_id];  //set the record ID
   $this->load->library(database);
   //load the database library to connect to your database
   $this->load->model(records);
   //inside your system/application/models folder, create a model based on
   //the procedure outlined in the CI documentation
   $results = $this->records->get_record($record_id);
   //get the record from the database
   $this->load->view(AJAX_record,$results);
}

Step 9
The AJAX_record.php file should be in your system/application/views folder. Keep in mind, that when you pass an array to a view (in this case the $results array), it will be exploded inside the view. So, the path to your record is now $record, instead of $results[record]. Also inside will be your standard HTML markup, and something like this:

< ?php foreach($record as $id=>$value) { ?>
     The name associated with this record is: < ?php print $value[name];?>
< ?php } ?>

This output is what php is sending to the request object, and is also what gets loaded into the content div. Code Igniter and JQuery make it that easy to dynamically load data using AJAX.

Initially, I was planning to use Amazon’s Cloudfront CDN, because it is very low cost and has Amazon backing it. However, during my research I discovered that Cloudfront is not website optimization friendly. Specifically, it isn’t a simple process to gzip files, or to set expires headers on them so they get cached. For the sake of time, I wanted something simple. I saw a few comments about SimpleCDN – which is actually lower cost than Cloudfront, and supports those optimization methods out of the box. One short account later, and I was ready to rock.

SimpleCDN offers three modes of getting your files onto their service. I chose to use their upload tool, because while it is the most manual labor, it also offer the most fine-grained control over the files being uploaded. All you need to do is login to their admin tool, create a ‘bucket’ – which is really just a folder that holds your files, and them use the provided ftp info to login to the ‘bucket’ and upload your files. You can then CNAME a domain (in my case cdnc.mrforbes.com) to the service. They give you plenty of options on how you want to serve your files – zipped or not zipped, headers set to expire after a day, 3 days, etc… At that point, its just a matter of pointing your files to the right place in your code, and your good to go.

Results

If you recall, we ended Part 1 with a YSlow score of 61, and an average load time of around 2 seconds.  This is pretty fast, but we’ll see how a CDN helps us out.  Our grade on this YSlow metric is currently an F.

I followed the procedure above, and got all of my files up onto the CDN. The one thing to keep in mind here is if you change any of the files, you generally need to rename them, relink them in the code, and re-upload them.. otherwise you’ll be waiting for CDN to re-propagate, which can take up to 24 hours. Completing this, I ran YSlow again.

[singlepic id=9 w=295 h=150 float=right] If you expand the image, you’ll see that my site now received a Yslow score of 98. Before you start thinking a CDN is the answer to all things optimized – keep in mind that the CDN is also gzipping and caching the files it is serving, which are other recommendations for website optimization. Essentially, by adding SimpleCDN I changed 3 scores, not one. My only question is, if I have all As, why did I get a 98? How do I get 100?

1. Initial YSlow Score:	D(61)	Final YSlow Score:	A(98)
   Initial Page Size:	500k	Final Page Size:	400k
   Initial Load Time:	2+ seconds	Final Load Time	1.5+ seconds

*Update: I have since added a few WordPress plugins which are loading their own images and stylesheets, so if you run YSlow on my site you won’t get the same numbers I have here. I plan to optimize again once I have decided on final functionality for the site.

1. Combined Files

   It makes sense that the first thing I should do is combine those javascript and css files. I certainly don’t need 17 when 2 or 3 will do just fine. So, what I’m going to do is take all of my separate javascript files, open them up, and copy and paste the code into one large file. Yes, this makes editing a little more time consuming, since you need to navigate a whole huge file to get the piece you want. Luckily, there are plugins available for different programming languages which will do this for you (pack:tag for java is an example). That way you can edit your original code, save it, and let the server handle this step. My site is simple, and if you aren’t that technically inclined, this method works so I’m outlining it here.

   Next, I’m going to do the same with the stylesheets. When I load YSlow and click on ‘components’, I see that one of my wordpress plugins is loading 2 stylesheets, so even with combining my own two I will still be loading three. I don’t really want this, so I did a search for the file in question, commented it out of the autoloading in the plugin (yes, it will come back if I upgrade and I’ll have to do it again), and pasted it into my style.css.

   Now that my files are combined, I’ll move on to the next recommended step.

2. CSS Sprites, Image Maps, Inline Images

   Okay, I cheated a little bit because I did this when I exported the original PSD. There are a few non-css-sprited images on the homepage, but they are there of necessity. The background, because its very wide and needs to repeat. The ocean in the footer for the same reason (which is also a png so the island can be seen behind it – even though a gif would be much smaller in file size), and the latest creations graphic (which is also currently a png, but I will likely change to a JPG as part of the optimization). The clouds and balloons are also graphics, but out of necessity – they need to be resizable.

   Image maps wouldn’t have really helped me on this design, and to be honest – I’m not a fan of inline images. Until there is a tool that automates their inclusion, adding that kind of code to an html page feels totally contrary to the entire value of CSS.

3. Wait, let’s do a little more.

   YSlow Step 10, minimize js, and Step 6 – put all javascript at the bottom are separate steps, but related to this exercise, so I’ll take care of them now. Currently, the site.js file is 173k, quite large. However, we can compress this by removing comments and whitespace, and shortening variable names. Again, we don’t have to do this ourselves. In fact, we can even do it online here. After minimization, that 173k file gets shrunk down to 93k.

   As for moving the javascript to the bottom of the page, I’ll just take my script and put it into the footer. I see that there are two javascript files being autoloaded into the header by the nggallery plugin. I want to get those to the bottom too, but will do so at a later time.

Results

1. Initial YSlow Score:	F(39)	Final YSlow Score:	D(61)
   Initial Requests:	23	Final Requests:	15: 3 js, 2 css, 9 images
   Initial Page Size:	600k	Final Page Size:	500k
   Initial Load Time:	6+ seconds	Final Load Time	2+ seconds

[singlepic id=9 w=298 h=150 float=right]

As you can see from these results, reducing the number of HTTP requests and minimizing our javascripts has increased load times by around 300%. We also raised our YSlow score by over 22 points. Not bad for about 30 minutes worth of optimizing.

Coming up next – adding Amazon Cloudfront as a Content Delivery Network