A very casual reading of economics may leave you with the impression that Keynesian economics believes in 'budget deficits' Whilst conservative economic thought believes in responsibility and the reduction of budget deficits.
There is always a danger of simplifying economics, and this view would be misleading. Keynesian theory advocates government borrowing only in cyclical downturns when there is a rise in private sector saving and period of unemployment. In a cyclical upturn, there should be the reverse of borrowing - governments should take the opportunity to reduce the budget deficit, and if appropriate run a surplus. See (principles of borrowing)
For example, it was a mistake for the Labour party to increase borrowing in the period 2003-04 by raising public spending. The rise in debt to GDP seemed innocuous enough at the time ( a rise in debt from 29% to 35% of GDP hardly seemed cause of much concern, and there wasn't much criticism at the time.) However, in retrospect, it was a mistake to run a bigger deficit in a period of economic boom.
It was a similar story in the late 1980s, when the chancellor responded to an economic boom, by er cutting income tax to put even more fuel on the fire of an overheating economy (Lawson boom)
It is the same story in the US. In 2001, conservatives were deeply worried about the prospect of budget surpluses they had inherited from the Clinton administration. They feared the prospect of the government having too much budget surplus and having to do things like invest to provide for long term pension commitments and other 'socialist' nonsense. It's hard to believe only a few years ago, American conservatives were worrying about having a budget surplus. But, it was this ideology which justified huge income tax cuts for the middle class, at a time when the housing bubble was just getting underway. (see: forgotten Conservative campaign to avoid budget surplus
It is of small irony, that Republicans, such as, Senator Ben Nelson, who recently voted to cut unemployment benefits (worth a small $77 billion on grounds economy couldn't afford it) was the same senator instrumental in passing the first Bush tax cut, which cost a cool $1.3 trillion. It is a case of getting everything the wrong way round.