UC7 : Suspicious security events detection - Historique des versions

Juliette : Cette version a été marquée à traduire

2025-01-02T13:48:34Z

Cette version a été marquée à traduire

← Version précédente		Version du 2 janvier 2025 à 15:48
Ligne 6 :		Ligne 6 :
	}}		}}
	<translate>		<translate>
	== Overview ==		== Overview == <!--T:1-->

	=== Project Objectives===		=== Project Objectives=== <!--T:2-->
	The overall objective of the Cyber project is to propose pedagogical notebooks presenting a set of algorithms on a precise cyber application.		The overall objective of the Cyber project is to propose pedagogical notebooks presenting a set of algorithms on a precise cyber application.

	=== Business Objectives===		=== Business Objectives=== <!--T:3-->
	The use case (UC7) aims at detecting abnormal security events in web events.		The use case (UC7) aims at detecting abnormal security events in web events.

	=== Data & methodologies===		=== Data & methodologies=== <!--T:4-->
	By exploring web activities, it uses two stastistical methods (interquartile range - IQR - and Isolation Forest) to identify "outlier" behaviors that are rare compared to the most standard behaviors.		By exploring web activities, it uses two stastistical methods (interquartile range - IQR - and Isolation Forest) to identify "outlier" behaviors that are rare compared to the most standard behaviors.

	=== Results===		=== Results=== <!--T:5-->
	This section is not really applicable since the current objective is not targeted towards production or POC.		This section is not really applicable since the current objective is not targeted towards production or POC.
	Nevertheless, the two models are highlighting the most abnormal IP addresses (and potentially related users). These lists of addresses could be used as input for further investigation by an operational expert.		Nevertheless, the two models are highlighting the most abnormal IP addresses (and potentially related users). These lists of addresses could be used as input for further investigation by an operational expert.

			<!--T:6-->
	* Authors : Nicolas Stucki & Thomas Levy		* Authors : Nicolas Stucki & Thomas Levy
	* Keywords: Unsupervised detection, clustering, outlier detection, Isolation Forest, interquartile range (IQR)		* Keywords: Unsupervised detection, clustering, outlier detection, Isolation Forest, interquartile range (IQR)

	== Data==		== Data== <!--T:7-->
	=== Online Shopping Store===		=== Online Shopping Store===
	This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).		This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).

			<!--T:8-->
	The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site. Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses. Finally, 3 csv files have been ingested into the platform:		The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site. Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses. Finally, 3 csv files have been ingested into the platform:

			<!--T:9-->
	* Small size dataset: logs_sub_2.csv (211 335 lines)		* Small size dataset: logs_sub_2.csv (211 335 lines)
	* Medium size dataset: logs_sub_5.csv (495 997 lines)		* Medium size dataset: logs_sub_5.csv (495 997 lines)
	* Large size dataset: logs_sub_10.csv (1 030 453 lines)		* Large size dataset: logs_sub_10.csv (1 030 453 lines)

			<!--T:10-->
	Main columns used are:		Main columns used are:
	{\| class="wikitable"		{\| class="wikitable"
Ligne 54 :		Ligne 58 :
	\|}		\|}

	=== Specific feature engineering===		=== Specific feature engineering=== <!--T:11-->
	Not applicable		Not applicable

	=== Notebooks===		=== Notebooks=== <!--T:12-->
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Ligne 73 :		Ligne 77 :
	\|}		\|}

	=== Risks & Compliance===		=== Risks & Compliance=== <!--T:13-->
	{\| class="wikitable"		{\| class="wikitable"
	\|+		\|+
Ligne 105 :		Ligne 109 :
	\|}		\|}

	=== Requirements===		=== Requirements=== <!--T:14-->

			<!--T:15-->
	* Python (3.6 or +)		* Python (3.6 or +)
	* scikit-learn (1.0.2 or +)		* scikit-learn (1.0.2 or +)
	* seaborn (0.11.2 or +)		* seaborn (0.11.2 or +)

	== Notebooks ==		== Notebooks == <!--T:16-->
	Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads		Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads

Juliette le 2 janvier 2025 à 13:48

2025-01-02T13:48:15Z

← Version précédente		Version du 2 janvier 2025 à 15:48
Ligne 1 :		Ligne 1 :
	{{Commun		{{Commun
	\|ShortDescription FR=Développement du Use Case pédagogique "Suspicious security events detection" dans le cadre du GT IA et Cyber		\|ShortDescription FR=Développement du Use Case pédagogique "Suspicious security events detection" dans le cadre du GT IA et Cyber
			\|ShortDescription EN=Development of the educational Use Case ‘Suspicious security events detection’ as part of the IA and Cyber WG
	\|ImageStr=Illu wiki2.png		\|ImageStr=Illu wiki2.png
	\|Status=Production		\|Status=Production
	}}		}}
			<translate>
	== Overview ==		== Overview ==

Ligne 111 :		Ligne 113 :
	== Notebooks ==		== Notebooks ==
	Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads		Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads

			</translate>
	{{PageSubHeader Commun		{{PageSubHeader Commun
	\|WorkGroup=IA et cybersécurité		\|WorkGroup=IA et cybersécurité
	}}		}}

Juliette le 4 décembre 2024 à 17:29

2024-12-04T17:29:35Z

← Version précédente		Version du 4 décembre 2024 à 19:29
Ligne 1 :		Ligne 1 :
	{{Commun		{{Commun
	\|ShortDescription FR=Développement du Use Case pédagogique "Suspicious security events detection" dans le cadre du GT IA et Cyber		\|ShortDescription FR=Développement du Use Case pédagogique "Suspicious security events detection" dans le cadre du GT IA et Cyber
	\|ImageStr=~~Use case-GTIA~~.png		\|ImageStr=Illu wiki2.png
	\|Status=Production		\|Status=Production
	}}		}}

	== Overview ==		== Overview ==

Angèle le 12 novembre 2024 à 14:38

2024-11-12T14:38:17Z

← Version précédente		Version du 12 novembre 2024 à 16:38
Ligne 4 :		Ligne 4 :
	\|Status=Production		\|Status=Production
	}}		}}
	~~&gt:translate>~~
	== Overview ==		== Overview ==

Juliette le 12 novembre 2024 à 14:20

2024-11-12T14:20:22Z

← Version précédente		Version du 12 novembre 2024 à 16:20
Ligne 109 :		Ligne 109 :
	* scikit-learn (1.0.2 or +)		* scikit-learn (1.0.2 or +)
	* seaborn (0.11.2 or +)		* seaborn (0.11.2 or +)

			== Notebooks ==
	Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads		Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads
	{{PageSubHeader Commun		{{PageSubHeader Commun
	\|WorkGroup=IA et cybersécurité		\|WorkGroup=IA et cybersécurité
	}}		}}

Juliette le 12 novembre 2024 à 14:19

2024-11-12T14:19:45Z

← Version précédente		Version du 12 novembre 2024 à 16:19
Ligne 109 :		Ligne 109 :
	* scikit-learn (1.0.2 or +)		* scikit-learn (1.0.2 or +)
	* seaborn (0.11.2 or +)		* seaborn (0.11.2 or +)

	Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads		Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads
	{{PageSubHeader Commun		{{PageSubHeader Commun
	\|WorkGroup=IA et cybersécurité		\|WorkGroup=IA et cybersécurité
	}}		}}

Juliette le 12 novembre 2024 à 14:19

2024-11-12T14:19:33Z

← Version précédente		Version du 12 novembre 2024 à 16:19
Ligne 110 :		Ligne 110 :
	* seaborn (0.11.2 or +)		* seaborn (0.11.2 or +)

			Retrouvez tous les éléments du Use Case sur le GitLab du Campus Cyber : https://gitlab.com/campuscyber/gt-ia-et-cyber/-/tree/main/UC7%20Suspicious%20security%20events%20detection?ref_type=heads
	{{PageSubHeader Commun		{{PageSubHeader Commun
	\|WorkGroup=IA et cybersécurité		\|WorkGroup=IA et cybersécurité
	}}		}}

Juliette le 12 novembre 2024 à 14:14

2024-11-12T14:14:45Z

← Version précédente		Version du 12 novembre 2024 à 16:14
Ligne 27 :		Ligne 27 :
	This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).		This use case relies on the dataset ["Online Shopping Store - Web Server Logs"](https://doi.org/10.7910/DVN/3QBYB5).

	The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site.		The dataset has been processed to convert it from a raw log file format into tabular elements corresponding to client requests to the web site. Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses. Finally, 3 csv files have been ingested into the platform:
	Then, to reduce its size, it has been sampled to keep only requests coming from a sub-part of the client IP adresses.
	Finally, 3 csv files have been ingested into the platform:		* Small size dataset: logs_sub_2.csv (211 335 lines)
	- Small size dataset: logs_sub_2.csv (211 335 lines)		* Medium size dataset: logs_sub_5.csv (495 997 lines)
	- Medium size dataset: logs_sub_5.csv (495 997 lines)		* Large size dataset: logs_sub_10.csv (1 030 453 lines)
	- Large size dataset: logs_sub_10.csv (1 030 453 lines)

	Main columns used are:		Main columns used are:
			{\| class="wikitable"
	\| Name \| Description \|		\|+
	\|~~:--- \| :~~-~~--\|~~		!Name
	\| SourceIP \| Source IP ~~address~~ \|		!Description
	\| datetime \| Local date&time of the reception of the request by the web ~~server~~ \|		!
	\| method \| HTTP method of the request (i.e GET, POST..) \|		\|-
			\|SourceIP
			\|Source IP address
			\|
			\|-
			\|datetime
			\|Local date&time of the reception of the request by the web server
			\|
			\|-
			\|method
			\|HTTP method of the request (i.e GET, POST..)
			\|
			\|}

	=== Specific feature engineering===		=== Specific feature engineering===
	Not applicable		Not applicable

	=== Notebooks===		=== Notebooks===
			{\| class="wikitable"
	\| Notebook \| Data Science ~~step \|~~		\|+
	\|~~:--- \|:~~-~~--:\|~~		!Notebook
	\| UseCase7_DataPrep.ipynb \| Data preparation (executed outside the platform) \|		!Data Science step
	\| UseCase7_Detection.~~ipynb~~ \| Security events detection (including simple feature engineering) \|		!
			\|-
	~~=== Risks & Compliance===~~		\|UseCase7_DataPrep.ipynb
			\|''Data preparation (executed outside the platform)''
	~~\| Type \| Applicable \| Comment(s) (if applicable)~~ \|		\|
	\|~~:--- \|:---:\|:---\|~~		\|-
	~~\| Bias \| No \| Has a complete bias study been carried out ? Comments if applicable \|~~		\|UseCase7_Detection.ipynb
	~~\| Ethics committee \| No \| Does the project needs to be screened by an ethics committee and if so has this been achieved ? Comments if applicable \|~~		\|''Security events detection (including simple feature engineering)''
	~~\| RSSI \| No \| Does the project needs to inform the RSSI about its outcome and activity, and if so has this been achieved ? Comments if applicable \|~~		\|
	~~\| DPO \| No \| Does the project needs to inform the DPO about its outcome and activity, and if so has this been achieved ? Comments if applicable \|~~		\|}
	~~\| RGPD \| No \| Are all the data collected necessary to perform the project objective and is there a necessity to collect consent from individuals ? Comments if applicable \|~~
	~~\| CNIL \| No \| Does the project needs to inform the CNIL ? Comments if applicable \|~~

			=== Risks & Compliance===
			{\| class="wikitable"
			\|+
			!Type
			!Applicable
			!Comment(s) (if applicable)
			\|-
			\|Bias
			\|No
			\|Has a complete bias study been carried out ? Comments if applicable
			\|-
			\|Ethics committee
			\|No
			\|Does the project needs to be screened by an ethics committee and if so has this been achieved ? Comments if applicable
			\|-
			\|RSSI
			\|No
			\|Does the project needs to inform the RSSI about its outcome and activity, and if so has this been achieved ? Comments if applicable
			\|-
			\|DPO
			\|No
			\|Does the project needs to inform the DPO about its outcome and activity, and if so has this been achieved ? Comments if applicable
			\|-
			\|RGPD
			\|No
			\|Are all the data collected necessary to perform the project objective and is there a necessity to collect consent from individuals ? Comments if applicable
			\|-
			\|CNIL
			\|No
			\|Does the project needs to inform the CNIL ? Comments if applicable
			\|}

	=== Requirements===		=== Requirements===

	- Python (3.6 or +)		* Python (3.6 or +)
	- scikit-learn (1.0.2 or +)		* scikit-learn (1.0.2 or +)
	- seaborn (0.11.2 or +)		* seaborn (0.11.2 or +)

	~~---~~

	~~&gt:/translate>~~
	{{PageSubHeader Commun		{{PageSubHeader Commun
	\|WorkGroup=IA et cybersécurité		\|WorkGroup=IA et cybersécurité
	}}		}}

Juliette le 12 novembre 2024 à 14:10

2024-11-12T14:10:36Z

← Version précédente		Version du 12 novembre 2024 à 16:10
Ligne 20 :		Ligne 20 :
	Nevertheless, the two models are highlighting the most abnormal IP addresses (and potentially related users). These lists of addresses could be used as input for further investigation by an operational expert.		Nevertheless, the two models are highlighting the most abnormal IP addresses (and potentially related users). These lists of addresses could be used as input for further investigation by an operational expert.

	Authors: Nicolas Stucki & Thomas Levy		* Authors : Nicolas Stucki & Thomas Levy
			* Keywords: Unsupervised detection, clustering, outlier detection, Isolation Forest, interquartile range (IQR)
	Keywords: Unsupervised detection, clustering, outlier detection, Isolation Forest, interquartile range (IQR)

	== Data==		== Data==

Juliette : Juliette a déplacé la page Use Case Suspicious security events detection vers UC7 : Suspicious security events detection

2024-11-12T14:09:16Z

Juliette a déplacé la page Use Case Suspicious security events detection vers UC7 : Suspicious security events detection

← Version précédente		Version du 12 novembre 2024 à 16:09
(Aucune différence)

UC7 : Suspicious security events detection - Historique des versions

Juliette : Cette version a été marquée à traduire

Juliette le 2 janvier 2025 à 13:48

Juliette le 4 décembre 2024 à 17:29

Angèle le 12 novembre 2024 à 14:38

Juliette le 12 novembre 2024 à 14:20

Juliette le 12 novembre 2024 à 14:19

Juliette le 12 novembre 2024 à 14:19

Juliette le 12 novembre 2024 à 14:14

Juliette le 12 novembre 2024 à 14:10

Juliette : Juliette a déplacé la page Use Case Suspicious security events detection vers UC7 : Suspicious security events detection

Juliette : Cette version a été marquée à traduire

Juliette : Juliette a déplacé la page Use Case Suspicious security events detection vers UC7 : Suspicious security events detection