Immigration is currently the number one issue in British politics. Voters blame immigration for depressed wages and high house prices. They also worry that many immigrants are criminals or extremists. Immigration is not so toxic an issue in Australia or New Zealand—two other island nations, also with conservative prime ministers but with immigration per head running at a much higher level. I met John Key, the New Zealand prime minister earlier this year and he offered me a one-word explanation: “control”.

Voters, Mr. Key continued, will support even quite high levels of immigration if they think their politicians have a good idea of who is entering the country and also if they know that numbers can be reduced whenever their democratically-elected politicians want numbers reduced. The Australian and New Zealand people have that confidence—so long as the United Kingdom remains a member of the European Union the British people cannot.

Immigration is at the top of UK voters’ concerns because David Cameron has been unable to reduce net immigration to under 100,000 per year – as he promised in his manifesto. The current level is three times as high at over 300,000. Although Mr Cameron’s government has enacted many measures to reduce immigration from outside of the EU it has no power to limit numbers from within the EU because of the “freedom of movement” principle. There are legitimate worries that net annual immigration might surge to 400,000 or 500,000 in the years ahead. So long as unemployment is so high in eurozone countries and wages are so uneven in the EU (the gap between per capita GDP in some European countries and that of Britain is nearly as big as Mexico’s is different from the USA’s) the UK, Europe’s fastest growing economy, is going to be a magnet to anyone with get-up-and-go.

The simmering immigration issue has bubbled over in the last few days because of a particular issue at the French port of Calais. On a clear day refugees from Sudan, Syria and other war or poverty-stricken nations can actually see the famous white cliffs of Dover. Some have been lured into thinking they can swim across the English Channel from Calais and have drowned. In recent days thousands assembled in shanty town-type camps—dubbed “The Jungle”—in and around the French town have attempted to storm the tunnel that connects mainland Europe to England. The French authorities are struggling to stop the very desperate people doing so. If you’ve escape barrel bombing in Syria, given all your worldly wealth to people smugglers and have survived crossing the Mediterranean in unseaworthy boats, you are not going to be deterred by a handful of relatively powerless European police officers.

Britain wants the French to do more to stop the refugees crossing France to get into Britain. EU law requires the country that first receives non-EU citizens to process their claims for asylum. France has not been fulfilling this legal responsibility adequately but it complains that, without identity cards and with lax employment laws, Britain is a soft touch for immigrants and it needs to tighten its procedures.

In the meantime most Britons watch refugees clambering aboard British lorries or breaching pretty insubstantial fences and that word “control”—or the lack of it—is at the front of their minds.

Given global income disparities and the spread of conflict across broken Middle Eastern and African states we are not going to see any reduction in refugee flows any time soon. While crackdowns on people smugglers and the £7 million razor-topped fence planned by the British government might provide some security the underlying causes of the misery and problem will persist. Britain gives more foreign aid to war-torn nations than any other EU country but it accepts a miserably small number of refugees. If it had control of its borders that might just possibly change. If the British people felt that the government was choosing and controlling who entered Britain it might be more relaxed. That would be bad news for low income Romanians and Bulgarians—but with net immigration at more modest and therefore acceptable levels it could be good news for the genuinely desperate from warzones. But so long as Britain is a member of an unreformed EU any hope of true border control is out of the question and that’s a great pity—and not just for British eurosceptics.    

Tim Montgomerie is a columnist for the London Times and founder of the website ConservativeHome.com.