I think it depends on how far you want to go with your photography
If you're a serious amateur, then I definitely think it's worth spending the extra on the D80, otherwise you are likely to outgrow a D40 before too long, which is not going to benefit you financially.