Author:


find & replace in multiple files using egrep & sed

Posted by – July 1, 2011

Programmers are lazy creatures. Any given programmer would love get things done with lesser keystrokes. For instance, find and replace all in multiple files. I was setting up a project on my localhost that some other guy had done. That guy wasn’t a real programmer otherwise what I did would have not been required at first place. The great soul hard-coded all the hyperlinks. What now? I thought of firing an editor and doing a find and replace all by creating a project. Simple. No. I can use grep/egrep to find things then obviously I can chain it to some other command and replace that text. Yes. I am lazy, so instead of experimenting I did some googling. Here is how to do this. Find all files, recursively of course, in current working directory that contain “teh” and replace them with “the”.

egrep -lRZ "teh" ./ | xargs -0 -l sed -i -e 's/teh/the/g'

egrep/grep takes further arguments like type of files to look for, example egrep “teh” *.txt ./ will search for all text files in current working direcotry that contain “teh”. Cool!

So, you can safely say, I can haz teh Linux power!

M4 – My top secret learning project :-)

Posted by – June 5, 2011

Yahoo! is now going through a very tough time and is being ignored for mostly. There is hardly any attention being paid to Yahoo! products. I recently tried to play with Yahoo! Application Platform and YQL. One can develop web based and standalone applications using YAP SDK and YQL. These apps are more or less similar to Facebook Apps in architecture. I used YQL and quickly created a small site that fetches Sensex data and displays it. Please visit http://www.indianbizdirectory.com/.

The domain IndianBizDirectory.com was lying idle for more than 4-5 years. I ran a small example fetch weather data for Beverly using YQL console. The example is so simple and self explaining. I was encouraged and decided to use YQL in some real life stuff. I used CodeIgniter to set up a project. Codeigniter is my framework of choice for rapidly developing prototypes. I searched for some feeds and web pages for data and as usual Yahoo! and Google were there to help me. Apart from this I hooked up BSEIndia.com, NDTV.com, Bing.com and MoneyControl.com for data. Initially I decided to run real time calls to these data sources. I setup a very basic logger with CI that mailed me every time a call to any data source or feed failed. I asked my friends to start hitting my site. They hammered it for two weeks. It was slow. And occasionally the feed from BSEIndia.com was failing. I studied logs an realised that the speed was slow due to multiple calls, there were at least 4 to 5 calls to different data sources, were making the page slow. I decided to cache the data. This increased the speed. So

  • on every page load, CI checks if cache is available and loads it, if not then it updates cache for that particular data source/feed.
  • there is a background process that caches data for Sensex during trading hours. The cache is refreshed every 15 minutes.
  • there are chances that a feed may fail during the background process, in that case the process leaves the cache in its original state. The cache is updated by CI.

I found there that BSE feed failure log entries were reduced though the data was not real time. After fetching data I used Xpath to transform data into HTML. Result, I could develop a standalone self sustaining website that displays useful Sensex, Forex and Commodity data. One can see Sensex top looser and gainers, clicking on a scrip name shows further detail about that scrip. On top of it, I added Bing search to fetch and display news related to that particular scrip. All in all, it was a great experiment and learning.

To monetize this effort, I added Google Ads on all pages. I wanted to add more ads but don’t have experience with other platforms.

After this, I am planning to dirty my hands with Python, PyGTK specifically to solve a problem I usually face. :-)

Playing with YUI3

Posted by – March 1, 2011

I have been playing with YUI3 for a while. Some one with a background in prototype.js and jQuery will find YUI3 bit complicated. To me YUI always sounded like JS toolkit developed by some c++/Java programmer who hated web developers to the core. Well YUI3 tries to be friendly. It took me just 15 minutes to write down the following code:

<html>
<head>
<title>YUI says Hello World!</title>
</head>
<body id="doc-body">
<div id="rgbvalues"> </div>
<script src="http://yui.yahooapis.com/3.3.0/build/yui/yui-min.js" charset="utf-8"></script>
<script type="text/javascript">
//use node and event modules
YUI().use("node", "event", function(Y){
    var handleClick = function(e) {
        var docBody = Y.one("#doc-body"); //equivalent to $(element) in prototype and jQuery(element)
        var winHeight = docBody.get('winHeight');//get "viewport" height
        var winWidth = docBody.get('winWidth');//get "viewport" width
        var red = parseInt (e.pageX * (255/winWidth));
        var green = parseInt(e.pageY * (255/winHeight));
        var blue = parseInt (red*green*0.004);
        var bgstyle = "rgb("+red+","+green+"," +blue+")";
        docBody.setStyle("background-color", bgstyle);//same as $(element).setStyle() from prototype and jQuery(element).css()
        Y.one("#rgbvalues").set("innerHTML", bgstyle);//set innerHTML
    };
    Y.on("mousemove", handleClick, "#doc-body");
});
</script>
</body>
</html>

Easy!
As I am not so good at maths I wasn’t able to write a formula or function that could translate mouse motion into color values, so I simply wrote down my stupid conversion :) .

Dynamic subdomains using .htaccess or mod_rewrite

Posted by – December 7, 2010

The man page documentation page for Apache mod_rewrite module says, “Welcome to mod_rewrite, the Swiss Army Knife of URL manipulation!”

Believe me, mod_rewrite lets you do cool things with URLs. You can have one script for every URI request. For uninformed a URI is simply a request for a page or a website like: http://www.kumarchetan.com/blog/all-about-lamp-web-development-blog/ or http://www.shameless-self-promotion.org/about/. In both cases, the websites run WordPress blogging platform and use mod_rewrite. mod_rewrite sends URIs like http://www.shameless-self-promotion.org/about/ or http://www.shameless-self-promotion.org/get los to one server side script or program which knows how to deal with good and bad requests.

But mod_rewrite also lets you play with your domain name and you can have sub-domains on the fly. So you can, without messing up your DNS entries, create sub-domains automagically.  Look at the following mod_rewrite which was added to a .htaccess.

Options +FollowSymLinks
#Turn on the engine
RewriteEngine On
RewriteCond %{HTTP_HOST} ([^.]+)\.domain\.com
RewriteCond %{SCRIPT_FILENAME} !-f
RewriteCond %{SCRIPT_FILENAME} !-d
RewriteRule ^(.*)$ /%1/$1

Event Delegation: Save thou webpage

Posted by – October 3, 2010

To make a web page interactive web developers rely on Events on the page. Event handling could be as easy as reloading the page on click of a button or as complex as following mouse pointer and responding to it. The usual way to do this is onEvent do some thing.

<button name="aButton" id="aButton" onclick="alert('Hello World!')">Cliché</button>

Easy.

There are so many ways to handle events and then do something. Inline, shown above, unobtrusive, shown below.

<button name="anotherButton" id="anotherButton">Another Cliché</button>
<script>
document.getElementById('anotherButton').addEventListener('click', function(){alert('Hello World!')}, false);
</script>

One can add as many as event listeners required for an event occurring for an element. Well, not really a wise idea. Think of Gmail inbox set to display 100 emails in a row. Does that mean100 addEventListener? Not a good idea. Here comes a very simple technique called Event Delegation. Example

<ol id="olElem">
 <li id="li1">One</li>
 <li id="li2">Two</li>
 <li id="li3">Three</li>
 <li id="li4">Four</li>
 <li id="li5">Five</li>
 <li id="li6">Six</li>
 <li id="li7">Seven</li>
 <li id="li8">Eight</li>
 <li id="li9">Nine</li>
 <li id="lin">n</li>
</ol>
<script>
document.getElementById('olElem').onclick = function(e){
 e = e || window.event;
 var target = e.target || e.srcElement;
 var textClicked = document.getElementById(target.id).innerHTML;
 alert('You clicked ' + textClicked);
}
</script>

This is all due to Event Bubbling which simply means that an event bubbles up to document root. So, when I click on “Nine”, the element li9 tells its parent that it got clicked, which in turn informs its parent and so on.