Why does the Constitution and due process matter?

Ok, I have been hearing a lot of discussion about the NSA program and why it should exist. The basic theory runs something like this:

I think that regardless of whether or not it is legal it should be done. After 911, we learned that oceans cannot protect us. There are people out there who want to kill us all. This is a small price to pay. Frankly, I think it would be best if the government listened to all of our phone calls, foreign and domestc. We are at war. Those who oppose having their phone calls monitored -- what are they hiding? The government is welcome to listen to all of my phone calls anytime it wants to -- all they will learn is that I love my country and hate terrorists.

Leaving the emotional argument aside, I must say lots of discussions about government are structural (due process right etc.) in precisely the sense that I argue below--we worry about concentrating certain powers into the hands of certain governmental entities without sufficient oversight and regulation, because we are worried about the potential for abuse of that power. And those worries are grounded in our long history with governments, which does in fact suggest that sweeping governmental powers without oversight and regulation are frequently abused by the people who hold such powers.

Accordingly, we try to place structures in place to prevent that sort of thing from happening. Indeed, that is what much of our Constitution is about, and much of our laws as well. And in that sense we are trying to stop abuses of power before they happen, not simply address abuses of power after they happen.

So, when people in government start ignoring those structural limits, and try to concentrate power without oversight and regulation into their own hands, we reasonably get worried. And those worries need not depend on actually knowing that they have already abused those powers, because the point of those structures was to prevent abuse before it happens, not simply address it after the fact.