1. Systematic dataset of cyber-attacks for IDS benchmarking a) Dataset 1 - For the company which has external interface such as website and mobile app, they would be more likely to deal with Probing and DoS Attack. b) Dataset 2 - For the company which has internal infrastructure, they would be more likely to deal with Remote to Local and Privileges attack. c) For performance of the dataset, our dataset covers 4 categories with 21 cyber- attacks while other common datasets use 2 categories with less than 10 cyber- attacks. The size of our dataset is around 10GB which is more suitable than other common datasets (e.g. 734MB or 6.5TB). In term of features, our dataset includes 6 features [1) Simulating real world situation, 2) Labelling, 3) Attack diversity, 4) Anonymity, 5) Heterogeneity and 6) Metadata] while other common datasets normally include only 2 or 3 features (e.g. Simulating real world situation, Labelling, Metadata). For detail, please refer to Part C.II.1. 2. System Log Anonymizer a) Technical Specification b) Program of Anonymizer c) Testing Report 3. Testing report of 2 selected IDS with benchmark dataset (Test 1) a) Focusing on Attack Detection Rate b) Focusing on Attack Detection Accuracy and Total Time taken c) Recommended actions 4. Testing report of 2 selected IDS with benchmark dataset (Test 2) a) Focusing on Attack Detection Rate b) Focusing on Attack Detection Accuracy and Total Time taken c) Recommended actions In total, we will test 4 IDS for all testing scenarios. 5. Case Study of major vendors 6. Incident reports in recent two years
Capital Delight Inc Limited
Hong Kong Police Force
Office of the Government Chief Information Officer
There are many Intrusion Detection Systems (IDS) that claim to detect various cyber- attacks. For validation, a comprehensive set of attacks is needed as the benchmarking tool.
We aim to create a dataset based on comprehensive attack scenarios, which comprises probing, denial of service, remote to local and privileges escalation. These attack scenarios are referenced from OWASP Top 10 which is an updated report outlining web application security concerns, focusing on the 10 most critical risks.
We will use a red team approach from an attacker's perspective which can cover more realistic attack simulation than a blue team approach (defender).
We will also use an anonymizer to allow external parties to contribute their datasets without disclosing their identities. During anonymization, the connection information is kept but all identities will be replaced.
The testing will leverage the Cyber-Range facilities to simulate attack using network appliances such as firewall, IDS and capturing server.
This will be the first benchmarking dataset in Hong Kong/Greater Bay Area region. It can
stimulate the security product industry to use standard benchmarking dataset for product
comparison and lead to enhancement of cyber security protection as a whole.
This is a Government Initiated Project supported by Hong Kong Police Force and Office of the Government Chief Information Officer (OGCIO)