Yassine Yousfi

I am passionate about machine intelligence.

I work as a head of Machine Learning at comma.ai where my focus is learning driving agents from data and in an end-to-end fashion.

To do so, we’re training World Models for driving. These World Models can be seen as “simulators” generating realistic driving video given driving inputs (steering, gas, etc.). World Models are useful for many reasons:

They are trained with very little supervision (self-supervised training)
They can be used to generate synthetic data for other models to learn on-policy
They can be used as powerful feature extractors
They can be used as policies

I hold a PhD in Electrical and Computer Engineering from Binghamton University (SUNY) where I researched digital media security topics such as Steganography/Steganalysis or Watermarking Deep Learning.

The take-away from my PhD was that when carefully scaling Machine Learning architecture and data, digital media security tasks can be learned end-to-end, with little domain specific preprocessing or tricks.

I also hold a MSc in Computer Science/Machine Learning from École Centrale de Lille (France).

News

Jun 25, 2023	I’m giving a talk at comma_con titled “Learning a Driving Simulator.” Blog post coming soon.
Jun 1, 2023	I’m serving as a Technical Program Chair for the 11th ACM Workshop on Information Hiding and Multimedia Security (IH&MMSec)
Jun 28, 2022	Our new paper “Detector-Informed Batch Steganography and Pooled Steganalysis” won the Best Student Paper Award at the ACM Workshop on Information Hiding and Multimedia Security (IH&MMSEC)!
May 10, 2022	I successfully defended my Ph.D. dissertation 🎉
May 1, 2022	Excited about our new paper on Batch Steganography and Pooled Steganalysis accepted at the ACM Workshop on Information Hiding and Multimedia Security (IH&MMSEC).

Selected publications

thesis

DEEP LEARNING FOR IMAGE STEGANOGRAPHY AND STEGANALYSIS. CHALLENGES, ADVANCES, AND OPPORTUNITIES

Yousfi, Y.

2022

Abs PDF

Deep learning has proven incredibly successful in a plethora of fields. In computer vision, deep neural networks are now the state-of-the-art for a variety of tasks. At the first glance, steganography and steganalysis appear to be very much different tasks than classical computer vision tasks, yet deep learning, especially convolutional neural networks, popularized by the computer vision field, have outperformed all classical feature-based approaches for detecting steganography. Intuitively, steganographic embedding changes are weak, noise-like signals executed primarily in complex content, such as textures and edges. Since computer vision classifies and categorizes content, it is also suitable for detecting the presence of noise-like stego signals modulated by content. In this dissertation, we focus on refactoring steganography detectors with more modern and general components, qualitatively understanding their strengths and failure cases, and using them algorithmically to improve steganography. First, we show that many custom ingredients long believed to be necessary for successfully training a deep neural network for steganography detection can be omitted in favor of more general-purpose convolutional architectures with very few domain-specific changes. Next, we focus on understanding what makes deep neural networks superior to their classical feature-based predecessors. Lastly, we use these powerful steganography detectors as a feedback loop in novel batch steganography algorithms, which allocate more payload in images where state-of-the-art detectors fail to detect steganography.
IH

Detector-Informed Batch Steganography and Pooled Steganalysis

Yousfi, Y., Dworetzky, E., and Fridrich, J.

In The 10th ACM Workshop on Information Hiding and Multimedia Security 2022

Abs PDF

Abstract We study the problem of batch steganography when the senders use feedback from a steganography detector. This brings an additional level of complexity to the table due to the highly non-linear and non-Gaussian response of modern steganalysis detectors as well as the necessity to study the impact of the inevitable mismatch between senders’ and Warden’s detectors. Two payload spreaders are considered based on the oracle generating possible cover images. Three different pooling strategies are devised and studied for a more comprehensive assessment of security. Substantial security gains are observed with respect to previous art – the detector-agnostic image-merging sender. Close attention is paid to the impact of the information available to the Warden on security.
WIFS21

CNN Steganalyzers Leverage Local Embedding Artifacts

Yousfi, Y., Butora, J., and Fridrich, J.

In IEEE International Workshop on Information Forensics and Security 2021

Abs PDF Slides

While convolutional neural networks have firmly established themselves as the superior steganography detectors, little human-interpretable feedback to the steganographer as to how the network reaches its decision has so far been obtained from trained models. The folklore has it that, unlike rich models, which rely on global statistics, CNNs can leverage spatially localized signals. In this paper, we adapt existing attribution tools, such as Integrated Gradients and Last Activation Maps, to show that CNNs can indeed find overwhelming evidence for steganography from a few highly localized embedding artifacts. We look at the nature of these artifacts via case studies of both modern content-adaptive and older steganographic algorithms. The main culprit is linked to “content creating changes” when the magnitude of a DCT coefficient is increased (Jsteg, –F5), which can be especially detectable for high frequency DCT modes that were originally zeros (J-MiPOD). In contrast, J-UNIWARD introduces the smallest number of locally detectable embedding artifacts among all tested algorithms.
IH

Improving EfficientNet for JPEG Steganalysis

Yousfi, Y., Butora, J., Fridrich, J., and Fuji-Tsang, C.

In The 9th ACM Workshop on Information Hiding and Multimedia Security 2021

Abs PDF

In this paper, we study the EfficientNet family pre-trained on ImageNet when used for steganalysis using transfer learning. We show that certain “surgical modifications” aimed at maintaining the input resolution in EfficientNet architectures significantly boost their performance in JPEG steganalysis, establishing thus new benchmarks. The modified models are evaluated by their detection accuracy, the number of parameters, the memory consumption, and the total floating point operations (FLOPs) on the ALASKA II dataset. We also show that, surprisingly, EfficientNets in their “vanilla form” do not perform as well as the SRNet in BOSSbase+BOWS2. This is because, unlike ALASKA II images, BOSSbase+BOWS2 contains aggressively subsampled images with more complex content. The surgical modifications in EfficientNet remedy this underperformance as well.
WIFS20

ImageNet Pre-trained CNNs for JPEG Steganalysis

Yousfi, Y., Butora, J., Khvedchenya, E., and Fridrich, J.

In IEEE International Workshop on Information Forensics and Security 2020

Abs PDF Code Video

In this paper, we investigate pre-trained computervision deep architectures, such as the EfficientNet, MixNet, and ResNet for steganalysis. These models pre-trained on ImageNet can be rather quickly refined for JPEG steganalysis while offering significantly better performance than CNNs designed purposely for steganalysis, such as the SRNet, trained from scratch. We show how different architectures compare on the ALASKA II dataset. We demonstrate that avoiding pooling/stride in the first layers enables better performance, as noticed by other top competitors, which aligns with the design choices of many CNNs designed for steganalysis. We also show how pre-trained computer-vision deep architectures perform on the ALASKA I dataset.
SPL

An Intriguing Struggle of CNNs in JPEG Steganalysis and the OneHot Solution

Yousfi, Y., and Fridrich, J.

IEEE Signal Processing Letters 2020

Abs PDF Code

Deep convolutional neural networks (CNNs) have become the tool of choice for steganalysis because they outperform older feature-based detectors by a large margin. However, recent work points at cases where feature-based detectors perform better than CNNs due to their failure to compute simple statistics of DCT coefficients. We introduce a shallow “OneHot” CNN, which encodes DCT coefficients using clipped one-hot encoding into a binary volumetric representation of the DCT plane fed to a convolutional block designed to learn relevant intra-block and inter-block relationships using vanilla and dilated convolutions. Methodology for plugging the “OneHot” network into conventional steganalysis CNNs is also introduced for an end-to-end learnable detector with improved performance
EI

JPEG steganalysis detectors scalable with respect to compression quality

Yousfi, Y., and Fridrich, J.

In Proceedings IS&T, Electronic Imaging, Media Watermarking, Security, and Forensics 2020

Abs PDF

Practical steganalysis inevitably involves the necessity to deal with a diverse cover source. In the JPEG domain, one key element of the diversification is the JPEG quality factor, or, more generally, the JPEG quantization table used for compression. This paper investigates experimentally the scalability of various steganalysis detectors w.r.t. JPEG quality. In particular, we report that CNN detectors as well as older feature-based detectors have the capacity to contain the complexity of multiple JPEG quality factors within a single model when the quality factors are properly grouped based on their quantization tables. Detectors trained on multiple JPEG qualities show no loss of detection accuracy when compared with dedicated detectors trained for a specific JPEG quality factor. We also demonstrate that CNNs (but not so much feature-based classifiers) trained on multiple qualities can generalize to unseen custom quantization tables compared to detectors trained for specific JPEG qualities. Their ability to generalize to very different quantization tables, however, remains a challenging task. A semi-metric comparing quantization tables is introduced and used to interpret our results.
IH

Breaking ALASKA: Color Separation for Steganalysis in JPEG Domain

Yousfi, Y., Butora, J., Giboulot, Q., and Fridrich, J.

In The 7th ACM Workshop on Information Hiding and Multimedia Security 2019

Abs PDF

This paper describes the architecture and training of detectors developed for the ALASKA steganalysis challenge. For each quality factor in the range 60–98, several multi-class tile detectors implemented as SRNets were trained on various combinations of three input channels: luminance and two chrominance channels. To accept images of arbitrary size, the detector for each quality factor was a multi-class multi-layered perceptron trained on features extracted by the tile detectors. For quality 99 and 100, a new “reverse JPEG compatibility attack” was developed and also implemented using the SRNet via the tile detector. Throughout the paper, we explain various improvements we discovered during the course of the competition and discuss the challenges we encountered and trade offs that had to be adopted in order to build a detector capable of detecting steganographic content in a stego source of great diversity.