James Cartlidge is the Conservative candidate for South Suffolk, a Director of Share to Buy and Babergh District Councillor.

Ed Miliband may have pledged in his conference speech to ‘double the number of first time buyers’, but in the real world we should be worried about a much more likely surge in the number of buy to let investors. I believe that the housing market is now genuinely unbalanced, with buy to let mortgage applicants facing ever laxer rules at a time when aspirant homeowners are facing much tighter borrowing restrictions, and we should consider two measures to level the playing field between residential purchasers and those buying property to rent out. First, insisting all buy to let mortgages are on a capital repayment basis. Secondly, implementing higher rates of stamp duty for purchases of property other than one’s primary residence.

The growth of buy to let shows no sign of abating: lending on such loans rose a whopping 31 per cent last year. Most importantly, it is estimated that the impact of the Chancellor’s new pension rules could boost buy to let investment by £10bn a year – or a staggering 50 per cent of current gross mortgage lending to landlords. This is not to criticise those pension reforms, which are part of a wider picture of free access to hard earned savings, but that wall of money has got to go somewhere and most experts believe a significant chunk of it is going to be chasing properties, the same properties sought by first time buyers and those trying to move their family home to larger accommodation. At the very least, the playing field between these contrasting buyers should surely be as level as possible, but at the moment it is clearly tilted towards the investor.

The key issue is mortgage lending rules. The tightening of rules for residential mortgages has been well documented of late – even with a large deposit, say 25 per cent or more, someone applying for a mortgage to buy a home to live in can only do so on a capital repayment basis. That is entirely right; the surge in interest only mortgages was one of the many examples of how the last property boom was built on unjustifiably lax lending rules. Yet many readers may be amazed to learn that for the very same property and from the very same bank, were you to apply for a buy to let mortgage today, you could still do so on an interest only basis with no repayment vehicle other than eventual sale of the property.

Allowing interest only applications as standard for buy to let is justified by the old adage that buy to let is ‘commercial’ lending; it’s ‘different’; it’s unregulated. There is one problem with this argument. It’s the same banks lending the cash. The fundamental point is that since the crash we now know it’s the state which, either directly or indirectly, underwrites every single bank and therefore every single mortgage – whether buy to let or residential. We talk about banks separating their casino and retail sides but we seem less bothered about the parallel universe of risk that applies in mortgage lending.

If new buy to let mortgages were required to be on a capital repayment basis the impact could be dramatic. The key with any buy to let application is ‘coverage’. The wannabe landlord, or more likely, established owner of multiple properties for rent, applies for the new buy to let mortgage having to prove that the likely rent will cover the mortgage payment by a set percentage – say, 125 per cent. This is based on covering the mortgage on an interest only basis. If the applicant were required to prove that their rent could cover a capital repayment amount by the same clearance, they would find the application somewhat harder to pass, inevitably requiring a much higher deposit to get the mortgage payments below the clearance threshold – and inevitably making buy to let mortgages hard to come by but safer for society in the process.

But what about the many investors unlikely to require a mortgage? This is likely to apply to much of the overseas money which has been the biggest factor in driving London house prices to stratospheric new highs, and will also be the case for the next wave of cash to enter the equation via the relaxation of pension rules. One weapon would, I believe, secure huge public support: having two rates of stamp duty, one for those buying a property which will be their only residence to live in; and a higher rate for those buying for any other reason – investment (from overseas or otherwise), holiday homes, second homes etc.

There are many weaknesses to stamp duty – a clunky, market-distorting tax at its best. But I have yet to find a single person who disagrees that it is patently unfair that a first time buyer pays the stamp duty as an investor purchasing their eighteenth buy to let property. I would go further and argue that this is one of the least fair aspects of our entire tax system. Basing stamp duty not just on the property price but also on the circumstances of the buyer is not unprecedented – there was a first time buyer stamp duty holiday earlier in this parliament. Such a change is relatively easy to co-ordinate because the conveyancing solicitor acts as the point of ‘verification’, and thereby, ensures that the appropriate rate is paid.

The beauty of this differential duty rule is that it is almost impossible to find a circumstance in which it is regressive. If property is the key part of measuring wealth, by definition someone who owns more than one is likely to be wealthier than someone who will only own one. There are not many popular tax rises, but I would be prepared to bet that a majority even of Conservative voters would support this reform, since most property-owning older voters nevertheless worry about the next generation’s access to the property ladder.

The fact is that there is no easy way to manage the property market, if this should be countenanced at all, but it becomes hard to argue against intervening to dampen demand when you are a Government that has made so much effort to boost it. When all is said and done, the combination of cheap money and scarce supply can only lead to one outcome – rising prices, but it is wrong to argue that the only answer is a vast building programme; and delusional to think that there is some magic reform of planning out there which would deliver such an outcome, not least when thousands of existing consents have not been built out.

It is perfectly possible to use mortgage criteria and other rules of the system to dampen demand. And there is always the threat of higher interest rates, but higher rates will not help exporters, and we have to remember the context of rebalancing the economy. Raising rates to ward off inflation is one thing, raising them purely to act as a break on the housing market is quite another. We need a rebalancing within the housing market itself that makes property investment less attractive, thus ensuring that those who aspire to put their own roof over their own head might still believe they have the glimmer of a chance.