« UC7 : Suspicious security events detection » : différence entre les versions

Version du 12 novembre 2024 à 15:14

Développement du Use Case pédagogique "Suspicious security events detection" dans le cadre du GT IA et Cyber

Catégorie : Commun Statut : Production 1 : Idée - 2 : Prototype - 3 : Validation - 4 : Production

&gt:translate>

Overview[modifier | modifier le wikicode]

Project Objectives[modifier | modifier le wikicode]

The overall objective of the Cyber project is to propose pedagogical notebooks presenting a set of algorithms on a precise cyber application.

Business Objectives[modifier | modifier le wikicode]

The use case (UC7) aims at detecting abnormal security events in web events.

Data & methodologies[modifier | modifier le wikicode]

By exploring web activities, it uses two stastistical methods (interquartile range - IQR - and Isolation Forest) to identify "outlier" behaviors that are rare compared to the most standard behaviors.

Results[modifier | modifier le wikicode]

This section is not really applicable since the current objective is not targeted towards production or POC. Nevertheless, the two models are highlighting the most abnormal IP addresses (and potentially related users). These lists of addresses could be used as input for further investigation by an operational expert.

Authors : Nicolas Stucki & Thomas Levy
Keywords: Unsupervised detection, clustering, outlier detection, Isolation Forest, interquartile range (IQR)

Data[modifier | modifier le wikicode]

Online Shopping Store[modifier | modifier le wikicode]

This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).

The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site. Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses. Finally, 3 csv files have been ingested into the platform:

Small size dataset: logs_sub_2.csv (211 335 lines)
Medium size dataset: logs_sub_5.csv (495 997 lines)
Large size dataset: logs_sub_10.csv (1 030 453 lines)

Main columns used are:


Name	Description
SourceIP	Source IP address
datetime	Local date&time of the reception of the request by the web server
method	HTTP method of the request (i.e GET, POST..)

Specific feature engineering[modifier | modifier le wikicode]

Not applicable

Notebooks[modifier | modifier le wikicode]


Notebook	Data Science step
UseCase7_DataPrep.ipynb	Data preparation (executed outside the platform)
UseCase7_Detection.ipynb	Security events detection (including simple feature engineering)

Risks & Compliance[modifier | modifier le wikicode]


Type	Applicable	Comment(s) (if applicable)
Bias	No	Has a complete bias study been carried out ? Comments if applicable
Ethics committee	No	Does the project needs to be screened by an ethics committee and if so has this been achieved ? Comments if applicable
RSSI	No	Does the project needs to inform the RSSI about its outcome and activity, and if so has this been achieved ? Comments if applicable
DPO	No	Does the project needs to inform the DPO about its outcome and activity, and if so has this been achieved ? Comments if applicable
RGPD	No	Are all the data collected necessary to perform the project objective and is there a necessity to collect consent from individuals ? Comments if applicable
CNIL	No	Does the project needs to inform the CNIL ? Comments if applicable

Requirements[modifier | modifier le wikicode]

Python (3.6 or +)
scikit-learn (1.0.2 or +)
seaborn (0.11.2 or +)

Groupe de travail

IA et cybersécurité

@@ Ligne 27 : / Ligne 27 : @@
 This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).
-The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site.
+The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site. Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses. Finally, 3 csv files have been ingested into the platform:
-Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses.
-Finally, 3 csv files have been ingested into the platform:
+* Small size dataset: logs_sub_2.csv (211 335 lines)
-- Small size dataset: logs_sub_2.csv (211 335 lines)
+* Medium size dataset: logs_sub_5.csv (495 997 lines)
-- Medium size dataset: logs_sub_5.csv (495 997 lines)
+* Large size dataset: logs_sub_10.csv (1 030 453 lines)
-- Large size dataset: logs_sub_10.csv (1 030 453 lines)
 Main columns used are:
+{| class="wikitable"
-| Name | Description |
+|+
-|:--- | :---|
+!Name
-| SourceIP | Source IP address |
+!Description
-| datetime | Local date&time of the reception of the request by the web server  |
+!
-| method | HTTP method of the request (i.e GET, POST..)  |
+|-
+|SourceIP
+|Source IP address
+|
+|-
+|datetime
+|Local date&time of the reception of the request by the web server
+|
+|-
+|method
+|HTTP method of the request (i.e GET, POST..)
+|
+|}
 ===  Specific feature engineering===
 Not applicable
 ===  Notebooks===
+{| class="wikitable"
-| Notebook | Data Science step                                 |
+|+
-|:--- |:---:|
+!Notebook
-| UseCase7_DataPrep.ipynb          | *Data preparation (executed outside the platform)* |
+!Data Science step
-| UseCase7_Detection.ipynb    | *Security events detection (including simple feature engineering)* |
+!
+|-
-===  Risks & Compliance===
+|UseCase7_DataPrep.ipynb
+|''Data preparation (executed outside the platform)''
-| Type | Applicable | Comment(s) (if applicable) |
+|
-|:--- |:---:|:---|
+|-
-| Bias | No | Has a complete bias study been carried out ? Comments if applicable |
+|UseCase7_Detection.ipynb
-| Ethics committee | No | Does the project needs to be screened by an ethics committee and if so has this been achieved ? Comments if applicable |
+|''Security events detection (including simple feature engineering)''
-| RSSI | No | Does the project needs to inform the RSSI about its outcome and activity, and if so has this been achieved ? Comments if applicable |
+|
-| DPO | No | Does the project needs to inform the DPO about its outcome and activity, and if so has this been achieved ? Comments if applicable |
+|}
-| RGPD | No | Are all the data collected necessary to perform the project objective and is there a necessity to collect consent from individuals ? Comments if applicable |
-| CNIL | No | Does the project needs to inform the CNIL ? Comments if applicable |
+===  Risks & Compliance===
+{| class="wikitable"
+|+
+!Type
+!Applicable
+!Comment(s) (if applicable)
+|-
+|Bias
+|No
+|Has a complete bias study been carried out ? Comments if applicable
+|-
+|Ethics committee
+|No
+|Does the project needs to be screened by an ethics committee and if so has this been achieved ? Comments if applicable
+|-
+|RSSI
+|No
+|Does the project needs to inform the RSSI about its outcome and activity, and if so has this been achieved ? Comments if applicable
+|-
+|DPO
+|No
+|Does the project needs to inform the DPO about its outcome and activity, and if so has this been achieved ? Comments if applicable
+|-
+|RGPD
+|No
+|Are all the data collected necessary to perform the project objective and is there a necessity to collect consent from individuals ? Comments if applicable
+|-
+|CNIL
+|No
+|Does the project needs to inform the CNIL ? Comments if applicable
+|}
 ===  Requirements===
-- Python (3.6 or +)
+* Python (3.6 or +)
-- scikit-learn (1.0.2 or +)
+* scikit-learn (1.0.2 or +)
-- seaborn (0.11.2 or +)
+* seaborn (0.11.2 or +)
----
-&gt:/translate>
 {{PageSubHeader Commun
 |WorkGroup=IA et cybersécurité
 }}

« UC7 : Suspicious security events detection » : différence entre les versions