David Cameron cannot be trusted to implement serious political reform. This in ten words is why Douglas Carswell decided to leave the Conservative Party and join UKIP. Yet during an hour-long interview yesterday with Carswell in his constituency of Clacton, where he is asking voters to ratify his change of allegiance by backing him in a by-election, it was difficult at times to follow the process by which he suffered such a profound loss of faith in Cameron that it became necessary to change parties.

Admittedly the interview was carried out under less than ideal conditions, at first in a café where voters kept rushing up to Carswell and saying things like:

“Well done!”

“I’m coming over to you!”

“I just want to shake your hand!”

“You’ve done well for me since you’ve come down here! Well done, mate!”

Judging by these brief encounters, the people of Clacton warmly approve of Carswell’s act of defiance.

Carswell himself displayed an evident relish for talking to his constituents. As he put it during our conversation: “I’ve never felt more comfortable than when walking round Clacton, meeting constituents. I feel far more comfortable having a cup of tea here than in the Commons tea room.”

For Carswell is by temperament an outsider, while Cameron is an insider, a person who feels at home in the heart of the Establishment. If one wishes to understand why Carswell finds Cameron intolerable, incompatibility of character is perhaps more important than incompatibility of principle. And Carswell’s affinity with Clacton could also have something to do with it being a town of outsiders: of people who used to live in London, but have retired to this relatively remote spot on the Essex coast, connected to the capital by a slow and infrequent train service.

Carswell certainly possesses an outsider’s ability to be deeply shocked when he discovers how the world works. In March this year, he attended the Königswinter Conference in Cambridge, an annual meeting of senior German and British policymakers. It was the top people from Whitehall, not Berlin, who horrified Carswell by expressing contempt for the policy, set out in Cameron’s Bloomberg speech in January 2013, of obtaining reform in Europe and then allowing the British people to accept or reject the deal in a referendum:

“Yes, I was shocked, I thought the Government was serious about change. And I sat in rooms listening to some very senior people, who unlike me are not democratically accountable, who clearly decide things, and who were just contemptuously dismissive. And when I raised very mild suggestions and put some questions, or even when I was just listening, they were smirkingly, eyeball-rollingly contemptuous of even the most modest treaty reforms. There was almost a sense of ‘we know best’. And yes we’ve got to say these things to the voters because of course you know the Prime Minister has got to win the next election, because you know, rest assured, we don’t mean it and it’s not going to happen. As I say, it’s trust. The Bloomberg trust went. That was a key moment, and it’s a key moment I can talk about in fairly specific terms without breaching confidences. If I give you other key moments you’ll know who I’m talking about and that wouldn’t be fair.”

In January this year, Carswell still believed Cameron was sincere in pursuing genuine reform followed by a referendum. Now Carswell thinks the Prime Minister’s attitude can be summed up by a line from Giuseppe di Lampedusa’s novel The Leopard: “If we want things to stay as they are, things will have to change.” The Prime Minister, Carswell reckons, is intent on keeping us in the EU by making some minor changes.

But although Carswell has lost faith in Cameron, it can still be quite hard to understand why he regards this as a sufficient reason to leave the Conservative Party. For Carswell is at pains to emphasise the good opinion that he has of very many Conservatives: “I’m not implying anything disparaging. Bear in mind that I have a lot of friends in the Conservative Party. I agree with a huge amount of things the Conservative Party stands for. And its activists and most of its MPs believe in the things I believe in. Decent, good, honourable people.”

ConHome: “One does revert to the question of why you left, given that you believe in the essential soundness of rank-and-file Conservatives. One can get rid of the hierarchy. Thatcher was got rid of. The Tory Party’s quite good at getting rid of leaders.”

Carswell: “The people in the upper echelons of the party are not on the side of those good, decent people. I think they are not serious about real change.” He pointed to Charles Moore’s piece in Saturday’s Telegraph, and particularly to this remark:

Yesterday – too late – the high command organised a ring-round trying to persuade prominent Eurosceptics to talk the referendum up. Why are they surprised if people do not trust their good faith?”

But Carswell is determined to fight the by-election in a respectful rather than a rancorous tone: “I am going to make absolutely certain that I and my team retain the right tone, because I have a huge amount of respect for the conservative family, conservative with a small “c” family. I’ve left the Conservative Party in sorrow, not in anger. I’m going to make absolutely certain that there’s no personal animus, make sure none of that comes into it. And I’m sure people will reciprocate.”

Carswell speaks well of the man who will be running the Conservative by-election campaign, and adds that this person understands how the internet has changed politics: “The party Chairman, Grant Shapps, he’s one of the good guys, I like him, he gets a lot of this, but unfortunately, despite there being one or two good people who get this, but a lot of people at the top of the party, their business model is basically it’s their property and it’s the property of a small clique and it’s not really in any meaningful sense a mass membership movement…We had an action day on Saturday and we had 84 young people and they had all come here from Facebook, and that was pretty phenomenally impressive.”

ConHome: “But you were doing this sort of thing, using new media, as a Tory.”

Carswell: “I don’t want to sound ‘Everything Tory bad, new to me at UKIP good’. Grant understands this and in fairness to him he’s been pretty phenomenal at mobilising resources from the internet, and in fact the full weight of that machine is about to bear down on me like a ton of bricks.” Carswell gave a rueful laugh, which became less rueful when he was interrupted by a supporter who told him he is bound to win.

Carswell: “Well there’s plenty of slips between cup and lip, as I’m about to find out with this cup of tea.”

Supporter [speaking about Cameron]: “That man will not have a referendum. It’s all rhetoric.”

The subject of the NHS came up, and the supporter claimed it was being exploited by visitors to this country: “When you’ve got the National Health Service, let’s face it, you can’t pay for the influx of immigrants any longer.”

Carswell: “I don’t think that is the fundamental question.”

Supporter: “I’m not allowed to say these things. But I’m speaking what everyone else is saying.”

Carswell: “Well you shouldn’t be able to say that. Remember the Olympics, the opening ceremony, I don’t know about you but that made me feel so good about this country. We were all together and we’ve got to have that sense of we’re all together.”

Supporter: “It was actually organised by an arch-socialist.”

Carswell: “Come on, there are some very nice socialists. If it wasn’t for socialism we wouldn’t have the NHS. They have done some good things, the Left in this country.”

ConHome: “Carswell’s getting a bit wishy-washy.”

Carswell [anxious to go to a quieter café]: “Shall we move on?”

Once we reached the quieter café, he said: “I dislike that nativism.”

ConHome: “It is very widespread.”

Carswell: All of the things that people worry about in Clacton – a shortage of GPs, loss of control of our borders, a fall in living standards, the banking crisis, the public debt situation – all of them can be fixed if we get political reform.”

ConHome: “Some Tories think you’ve behaved disgracefully, because they think you’ve deserted the party. What would you say to them?”

Carswell: “I feel very sad at doing what I had to do. And I can understand why some people might think that. But I’m fundamentally and ultimately accountable to the people you see me talking to today.”

ConHome: “What’s your idea about what’s going to become of UKIP? Is the idea to supplant the Tories or in due course to do a deal with them?”

Carswell: “I’m not even going to go there. I’m trying to win this by-election.”

ConHome: “The essential Cameroon argument is that you’re in danger of handing the election to Miliband and you’re destroying the chances of having a referendum. Maybe you’re frightened of having a referendum and losing it? How do you respond to this line of attack?”

Carswell: “I can promise you if I’m a Member of Parliament in the run-up to 2017, Mr Cameron and his Whips if he’s still Prime Minister can count on me to help ensure the legislation goes through. I give him that guarantee now. The Carswell guarantee. As for letting Labour in: in Clacton if you vote UKIP you’re going to get UKIP. So that argument doesn’t stack up here.”

ConHome: “I appreciate you’re not going to get Miliband in Clacton. But you might get Miliband in Number Ten if UKIP damages the Tories enough.”

Carswell: “I don’t fundamentally believe that on most of the macro issues, an Ed Miliband government would be that different from a Cameron government. Different cliques, same sofa. Purnell started welfare reform under Labour. On Education we mustn’t forget that it was actually Andrew Adonis who initiated some reform.”

ConHome: “Perhaps you should join the Labour Party.”

Carswell [laughing]: “I’m not a Labour man in any way, shape or form.”

ConHome: “Do you want to be UKIP leader?”

Carswell: “Absolutely not. In the short time that I’ve got to know him well, I’ve huge admiration and respect for Nigel. Without meaning to sound rude, your question is daft. I’m not temperamentally suited to that sort of thing.”

As we strolled back to the new UKIP headquarters near Clacton station, we passed the Conservative office, and Carswell said: “They’re good people. A sizeable number are coming with me. It confirms in my mind that I’m doing the right thing.”