Contact Us »

Site Search

Bulk Personal Datasets


What is BPD?

There is no statutory definition of BPD, but in essence BPD refers to data, including personal data, belonging to a range of individuals acquired by or held on one or more analytical systems in the intelligence services. These datasets are typically very large and cannot be processed manually and the majority of individuals whose data is contained within each BPD are unlikely to be of intelligence interest.

Basis in law

On 12 March 2015, under section 59A of RIPA, the Prime Minister published a direction which put on a statutory footing the Commissioner’s oversight of the acquisition, use, retention, disclosure, storage and deletion of bulk personal datasets, including any misuse of data and how this is prevented.
Acquisition and Retention of BPD is dealt with under section 2(2)(a) of the Security Service Act 1989, section 2(2)(a) and 4(2)(a) of the Intelligence Services Act 1994 and section 19 of the Counter-Terrorism Act 2008.

The Security Service Act and the Intelligence Services Act allow the intelligence services to obtain information only for the proper discharge of their functions. The Counter-Terrorism Act allows any person to disclose information to any of the intelligence services for one of their statutory functions. It permits information they obtain in connection with one function to be used by the intelligence services in connection with any of their functions and provides that disclosing information to the intelligence services overrides any duty of confidentiality or other restriction on disclosure. 

Obtaining and using BPD

Obtaining and disclosing BPD is now governed by Handling Arrangements published by the Government which came into force on 4 November 2015. The rules set out in these Handling Arrangements  are mandatory and agency staff are required to follow them.

Use of each BPD is authorised separately before it can be made available on an analytical system for intelligence officers. Each authorisation must give a description of the dataset being requested and set out the operational and legal justification for acquiring and retaining the dataset. This justification must include: why it is necessary to acquire and retain the data, what intelligence aim is likely to be met and how the data will support that objective; why it is proportionate to acquire and retain the data and whether there is a less intrusive method by which to obtain the same information; and an assessment of the level of intrusion into privacy.

The agencies then make an assessment of: the necessity for retaining each dataset, the intrusion into privacy and measures to minimise intrusion.
In addition there is a panel of senior managers in each agency who meet regularly to review: the retention of all datasets; the decision to take any new dataset into analytical systems, examples of the use of any dataset during any previous period and the decision to delete any datasets. Some datasets have very little private or even publically available data on them so as long as they are still being used and contribute towards the organisation’s aims, it is relatively easy to justify retaining them. Other datasets which contain intrusive or sensitive confidential data must be flagged to the panel.


In all agencies officers undergo formal training before they can access BPD. Training includes an officer’s personal responsibility for his/her own use of the system and managers’ responsibility for his/her staff. All agencies require that BPDs must be managed to ensure that privacy of those whose data is held is respected, that datasets are accessed and disclosed only to the degree necessary to fulfil the agencies’ statutory functions and that any use of the information contained in the datasets is proportionate to the aims of the organisation.
How the Commissioner oversees BPD

Stage One – Acquisition and Retention of BPD

The Commissioner reviews a full list of all BPDs held by the agencies and all of the records of the internal review bodies which consider the retention and use of datasets. He inspects their documents and the formal justifications for acquiring a dataset and making it available for use on an analytical system. The Commissioner assesses whether the internal review bodies have properly applied the test of necessity and proportionality when retaining and making data available for use. 

Stage Two – Use of BPD

The Commissioner then assesses how officers in the agencies access BPDs. This assessment includes: training required before access is allowed; restrictions in place to limit access; and application of the necessity and proportionality justifications for intrusion into private information.

Stage Three – Misuse and how this is prevented

The Commissioner reviews any possible misuse of BPD and how this is prevented. BPDs can contain highly personal information and it is vital that only those with a legitimate business need can access this information, so the Commissioner reviews the protective monitoring arrangements in place for BPD to ensure they are sufficiently robust and to see whether any improvements need to be made. He also considers any other misuse of information and security breaches within the agencies when reviewing how BPD is monitored and protected,  he must be confident that the system as a whole is not open to abuse. The Commissioner has instructed the agencies that any misuse must be treated seriously and reiterates that instruction regularly. This is a key part of his inspection.

Last updated: 23 Jan 16