There is an excellent article over on Precision Nutrition.
Can eating too little actually damage your metabolism?
On the surface, it's just about the concept of "metabolic damage." But in fact it's got an excellent look at the difficulty of determining your energy balance (energy in vs. energy out.)
It also explains and illustrates what happens when you modify this balance with either eating less or exercising more.
Really worth a look if you want to get a better of idea what "Calories in vs. Calories out" really means within you body.