MI5 has its basis in the Secret Service Bureau. The agency was created in 1909 with Vernon Kell as the founding head.
The secret service Bureau was originally divided into a Naval group which was used for spying on foreign countries, and the Army group that was responsible for conducting counterintelligence activities inside Britain. The Army group responsible for counterintelligence was turned into MI5 with similar roles.
In 1914 the agency was mainly concerned with German spies in Britain. Fearing sabotage, MI5 successfully identified 22 German spies in the country and arrested them on the eve of First World War.
During the war, 7 more men were caught spying for Germany. Sir Roger Casement was one of 11 men executed for treason. By making entry and exit from UK difficult and conducting large scale mail inspection, MI5 was successful in thwarting German espionage efforts.
The number of staff working for the agency increased dramatically during WWI. By 1918 MI5 had a staff of 5000 people and kept files on 137,500 individuals.
The role of MI5 was enlarged because of its vast and underutilized resources. The agency gained a more political role in that time. It conducted surveillance of labor unions and pacifists groups. This role was assigned to the agency in the mistaken belief that these organizations were being affected by the communist movement.
MI5 intercepted a letter, supposedly written by Comintern (an organization for promotion of communism in the world), in 1924 that urged communists in Britain to promote revolution. This letter created a controversy that resulted in Labor party defeat in 1924 general elections.
In the 30s, fascists became the main threat to British security. In this time only 30 officers and 6 surveillance men were working for the agency. MI5 was ill equipped to perform its increasing responsibilities in the buildup to war. Many aliens supposed to be confined, in order to uncover German spies, got away. This botched mission caused controversy and allegations of incompetence were made against MI5.
Vernon Kell (THe original "K") was sacked by Churchill in 1940. When David Petrie was made the Director General of MI5 in 1941, he was given adequate resources to rebuild the organization.
When Battle of Britain ended, MI5 deployed the highly successful "double-cross" system. Using this system the agency turned foreign spies in its favor, who then fed Germans false information. This system of deception finally led to the successful 1944 allied landing in Normandy.
German war records shows that 115 people were working as German spies in Britain. MI5 had successfully apprehended 114 of them. Some on that list were already working for Britain under the double cross system!
In 1952 all responsibilities of the service, which were previously the charge of the prime minister, were transferred to the home secretary. This arrangement has prevailed up to the present day. The home secretary set out the services role.
In 1989 this directive was set on legal footing. After the war MI5 turned its attention to the Soviet threat. It not only kept a close eye on suspected soviet spies but also on members of British communist party.
The need for better counter espionage was shown when many Russian spies like SIS officer George Blake, and a spy ring in Portugal was arrested.
MI5 drew severe criticism when it failed to detect the Cambridge five spy ring. One of them was Kim Philby, a KGB informer for many years. The agency faced much controversy in these years because of its ineffectiveness to catch KGB agents. However the service was successful in breaking up a huge ring of Russian agents in 1971 when 105 employees of the soviet embassy suspected of being spies were deported back to USSR.
MI5 had to turn its attention towards Irish terrorism in the seventies along with counter espionage in the country. MI5 was severely embarrassed in 1983 by one of its officers Michael Bettaney. He was found trying to sell secrets to USSR.
The service also drew harsh criticism when it was found that MI5 was monitoring left wing organizations and trade unions. Many defectors latter told of high level penetration inside the service itself. Roger Hollis, the Director General himself was suspected of being an informer. Hollis was cleared of these accusations in a 1974 inquiry.
The end of the cold war brought renewed cooperation between security agencies around the world. The MI5 succeeded in taking over investigations of Irish republican activities from the special branch of Metropolitan police. Working closely with special branches of the police the service successfully infiltrated the Irish Republican Army (IRA).
These operations resulted in 21 terrorism related convictions. IRAs decision of participating in the Northern Ireland peace process has been attributed by many to MI5s efforts.
A legislation passed in 1996 extends the agencys operations by requiring it to help other law enforcement agencies in combating high level criminal activity. This legislation aimed at involving MI5 in combating rising crime levels in Britain.
However, recent rise in terrorism in Britain has taken the agencys attention. Numerous arrests of would be terrorists has been in recent times have been attributed to the service.