Here’s what happen: Let’s assume that your Firefox browser is opened with plenty of tabs, but it is not currently your active application (you might be doing some work in Ms. Office). When you hover your mouse above the Firefox icon in the taskbar, it will show a preview of all the opened tabs. This is good and fine as I can now pick the tab that I want to go to. The bad part is, I don’t want to manually select the tab everytime. When I click on the Firefox icon, I would expect it to switch to the browser immediately and load my last active tab, not to make me select the tab.
Imagine if you have 10 or more tabs opened and you need to manually select your last active tab before it would switch to the browser, how counter-productive it has become. If you are experiencing the same annoyance as me, here’s how you can fix it. Open a new tab and type “about:config”. You will see the void warranty page. Just click the “I’ll be careful, I promise!” button. In the Filter field at the top, enter the term “taskbar” (without the quote). Scroll down to find the entry “browser.taskbar.previews.enable”. Double click on it to toggle it to “false”.

Close the tab. That’s it. the Tab Preview feature is now disabled. Update: There is actually an easier method to disable the tab preview. In Firefox 4, go to Options page. Under the Tab tab. Uncheck the box “Show tab preview in the Windows taskbar”.

Thanks to @liamjdavison for providing the tips.