Category: Software Engineering

What are some popular myths in software development?

Posted by – November 23, 2011

The following one is my favorite and I have seen this in practice in most of the places I have workd.

The Mythical Man Month, the idea that adding people to a development team makes it more efficient in a linear fashion.

via What are some popular myths in software development? – Quora.

One of my cow-orker said managers thing if you can not deliver a  baby in 9 months alone they will make you do it by adding another 8 people in your team and giving you one month. The problem doesn’t end after delivering baby, none of the 8 people who helped in delivering the baby will care about the baby. It is not their baby at all. Another one which I noticed is at OSI Days talks is that OO a silver bullet, a golden hammer. Well, I work on web technologies and AFAIK, Google is bland but faster than any other known web app. More abstractions means performance hit. I want my page to load faster and if my code takes a minute to spit a text string my code is wrong how good OO patterns it may follow. Add to it the purists views of not adding markup to the script.

The Big Pipe – Faster Facebook

Posted by – July 20, 2010

Facebook is a big time killer. You can harvest farm without owning an inch of land, kill your buddies and still brag about it, save unknown species of zoo animals and poke any girl you know and she wont mind.
But Facebook is a case study for web developers like me. A typical Facebook page, depending on network and host, loads within 5 seconds. Facebook team has set this time to 2.5 seconds. Anything beyond 2.5 seconds is a big no no. For me it loads 19 CSS files, just one JS file, all in all a 50KB page w/out images. If you didnt pay attention to last line, it FB loads just one JS file and if you look at the byte size its just 10K file. Like I mentioned earlier Gzipped content is much lighter i.e. low byte size, it can be sent over network quickly. Facebook and every other website follows and compress the content they send over the network. But this is not all.
Before I move ahead, lets understand how a page is “rendered” in browser.

  • You send a request to FB servers.
  • FB server looks at your request and prepares a page, if you were logged in, it will send your profile page otherwise a nice login page. This job is done by PHP, MySQL, Apache and some in house tools developed by FB.
  • FB sends you back the page, the HTML
  • The browser reads or “parses” the HTML and finds out well it also needs CSS, images and JS and sends separate requests for these assets.

Now here is the tricky part. Browsers need all the possible assets that can affect the rendering of page and JS is one thing that can hide images, change CSS applied on an element, add text or can change the complete layout. So when browsers encounter a JS file they stop loading anything and concentrate only on JS. Till the time JS is not loaded and processed nothing else is done. This have been, more or less, same for nearly all the browsers till date. Web developers try to deceive browsers in many ways. The one way suggested by Yahoo! is to load JS at the bottom of page or develop the page in such a way that JS is loaded just before body tag is closed. This is not feasible in many cases. Say for example, on my webpage a menu appears in top header not in bottom of the page so it is rendered first and I need it to be active. There are way outs.

  • RequireJS
  • labJS
  • Load JS using AJAX
  • Create Script tag dynamically
  • Load scripts in iFrame
  • and others

Facebook employs some thing it calls a BigPipe. Interestingly FB divides its page into smaller chunks, named pagelets and every pagelet has its own JS associated with it. Have a look at following BigPipe fragment

<script> big_pipe.onPageletArrive({
 "id": "pagelet_nav_lite",
 "phase": 1,
 "is_last": false,
 "append": false,
 "display_dependency": [
 "pagelet_navigation"
 ],
 "bootloadable": {
 "editable-side-nav-js": [
 "CiNUi",
 "WTiBX",
 "Nahdv",
 "vstFf"
 ]
 },
 "css": [
 "p0tHf",
 "HOV0r",
 "FLbGk",
 "mWAGo",
 "e2Nmn"
 ],
 "js": [
 "CiNUi",
 "WTiBX",
 "k29le",
 "kJEXe",
 "5CZKw",
 "S4cYi",
 "3iRY2"
 ],
 "resource_map": {
 "e2Nmn": {
 "name": "css\/sprite\/autogen\/a2gtd9_duri.css",
 "type": "css",
 "nonblocking": 1,
 "src": "http:\/\/static.ak.fbcdn.net\/rsrc.php\/z8ZLB\/hash\/40d4o2t0.css"
 },
 "vstFf": {
 "name": "js\/ui\/xhp\/page\/editable_side_nav.js",
 "type": "js",
 "src": "http:\/\/static.ak.fbcdn.net\/rsrc.php\/z5QP6\/p\/hash\/eoj7km8p.js"
 }
 },
 "requires": [

 ],
 "provides": [

 ],
 "onload": [

 ],
 "onafterload": [

 ],
 "onpagecache": [

 ],
 "onafterpagecache": [

 ],
 "refresh_pagelets": [

 ],
 "invalidate_cache": [

 ],
 "content": {
 "pagelet_nav_lite": "HTML removed, view source your own FB profile page :-) "
 },
 "page_cache": true
})</script>

Further spying on the DOM using Firebug, I could see an iframe named “bootloader_iframe” which loads JS files.

BigPipe has really changed the way Facebook loads and works. Facebook is showing generosity and is sharing what it does. Unlike Yahoo! and Google, Facebook is not so much developer friendly and is not well known to share its knowledge. I hoping to see more documentation on BigPipe. Here is some info: http://www.facebook.com/notes/facebook-engineering/bigpipe-pipelining-web-pages-for-high-performance/389414033919

The JSON object was formatted using JSONLint.

Error Reporting

Posted by – March 25, 2009

PHP has come a long way. I used to program with Global vars then $HTTP_POST_VARS and then $_POST and even did a foreach to convert all Posted/Get-ed vars to their respective variable names. But I never faced any issue till the time I started using error_reporting(E_ALL).
My life changed. Initially I started pulling my hairs and then I realised that I will be bald very soon and then started concentrating on writing better code. Avoiding on the fly variables. Yesterday at work, I again realised the value of error_reporting(E_ALL).I created a small piece of code and tested it in a temp php file. It looked superb. I ported the code to production environment and it started breaking. I thought for a while. May be I have introduced some parse errors. I ran a php -l test and didnt see any syntax error. I read the whole code line by line. Even did a diff. Still the temporary php code was running fine and the code in production environment was not working fine. :-/

I decided to look at the code bit later. After a break of 15 minutes I decided to write a unit test and capture everything. And then I found a notice message telling me I was accessing an index that didn’t exist. I looked at my data set. It was a long range of number, 6184 to be precise and an index value 280 was not set. I looked at my temp file and found my code was not going thru error reporting and production environment was set up to halt at all notices and warnings. Damn it!