Publications

Conference Publications

Inter-modality Discordance for Multimodal Fake News Detection Permalink

Published in ACM Multimedia Asia, 2021

Authors: Singhal, Shivangi and Dhawan, Mudit and Shah, Rajiv Ratn and Kumaraguru, Ponnurangam
Keywords: Multimodal Fake News, Inter-modality Discordance, Contrastive Loss, Multitask learning, Metric learning
code, pdf

Abstract

The paradigm shift in the consumption of news via online platforms has cultivated the growth of digital journalism. Contrary to traditional media, lowering entry barriers and enabling everyone to be part of content creation have disabled the concept of centralized gatekeeping in digital journalism. This in turn has triggered the production of fake news. Current studies have made a significant effort towards multimodal fake news detection with less emphasis on exploring the discordance between the different multimedia present in a news article. We hypothesize that fabrication of either modality will lead to dissonance between the modalities, and resulting in misrepresented, misinterpreted and misleading news. In this paper, we inspect the authenticity of news coming from online media outlets by exploiting relationship (discordance) between the textual and multiple visual cues. We develop an inter-modality discordance based fake news detection framework to achieve the goal. The modal-specific discriminative features are learned, employing the cross-entropy loss and a modified version of contrastive loss that explores the inter-modality discordance. To the best of our knowledge, this is the first work that leverages information from different components of the news article (i.e., headline, body, and multiple images) for multimodal fake news detection. We conduct extensive experiments on the real-world datasets to show that our approach outperforms the state-of-the-art by an average F1-score of 6.3%.

HLDC: Hindi Legal Documents Corpus Permalink

Published in Findings of the Association for Computational Linguistics: ACL-IJCNLP, 2022

Authors: Kapoor, A., Dhawan, M., Goel, A., Arjun, T.H, Bhatnagar, A., Agrawal, V., Agrawal, A., Bhattacharya, A., Kumaraguru, P., Modi, A.
Keywords: AI for Social Good, Judicial AI, MultiTask Learning, Low-Resource Language, Hierarchical Transformers
code, pdf

Abstract

Populous countries (e.g., India) are burdened with a considerable backlog of legal cases. This calls for the development of automated systems that could process legal documents and augment legal practitioners. To develop such data-driven systems, there is a dearth of high-quality corpora. The problem gets even more pronounced in the case of low resource language (e.g., Hindi). In this resource paper, we introduce the Hindi Legal Documents Corpus (HLDC), a corpus of 900K legal documents in Hindi. The documents are cleaned and structured to enable the development of downstream applications. Further, as a usecase for the corpus, we introduce the task of Bail Prediction. We experiment with a battery of models and propose a multi-task learning (MTL) based model. MTL models use summarization as an auxiliary task along with bail prediction as the main task. Results on different models are indicative of the need for further research in this area.

Pre-Prints/ Under-Review

GAME-ON: Graph Attention Network based Multimodal Fusion for Fake News Detection Permalink

Published in Accepted in Social Network Analysis and Mining Journal 2024, 2022

Authors: Mudit Dhawan^, Shakshi Sharma^, Aditya Kadam, Rajesh Sharma, Ponnurangam Kumaraguru
Keywords: Graph Neural Networks, Multimodedia Representation Learning, Heterogeneity Gap, Multimodal Fake News Detection
pdf

Abstract

Social media in present times has a significant and growing influence. Fake news being spread on these platforms have a disruptive and damaging impact on our lives. Furthermore, as multimedia content improves the visibility of posts more than text data, it has been observed that often multimedia is being used for creating fake content. A plethora of previous multimodal-based work has tried to address the problem of modeling heterogeneous modalities in identifying fake content. However, these works have the following limitations: (1) inefficient encoding of inter-modal relations by utilizing a simple concatenation operator on the modalities at a later stage in a model, which might result in information loss; (2) training very deep neural networks with a disproportionate number of parameters on small but complex real-life multimodal datasets result in higher chances of overfitting. To address these limitations, we propose GAME-ON, a Graph Neural Network based end-to-end trainable framework that allows granular interactions within and across different modalities to learn more robust data representations for multimodal fake news detection. We use two publicly available fake news datasets, Twitter and Weibo, for evaluations. Our model outperforms on Twitter by an average of 11% and keeps competitive performance on Weibo, within a 2.6% margin, while using 65% fewer parameters than the best comparable state-of-the-art baseline.

Enhancing Tail Performance in Extreme Classifiers by Label Variance Reduction Permalink

Published in Under review at International Conference on Learning Representations 2024 (ICLR), 1900

Authors: Anirudh Buvanesh, Rahul Chand, Jatin Prakash, Bhawna Paliwal, Mudit Dhawan, Neelabh Madan, Deepesh Hada, Vidit Jain, Sonu Mehta, Yashoteja Prabhu, Manish Gupta, Ramachandran Ramjee, Manik Varma
Keywords: Extreme Classification, Supervised Representation Learning, Label Smoothing
pdf

Abstract

Extreme Classification (XC) architectures, which utilize a massive one-vs-all classifier layer at the output, have demonstrated remarkable performance on problems with large label sets. Nonetheless, these have also been observed to falter on tail labels with few representative samples. This phenomenon has been attributed to factors such as classifier over-fitting and missing label bias, and solutions involving regularization and loss re-calibration have been developed. This paper explores the impact of label variance, a previously unexamined factor, on the tail performance in extreme classifiers. Label variance refers to the imprecision introduced in the ground truth when sampling it from a complex underlying distribution - a common phenomenon in most XC datasets. This compromises the quality of trained models, with a pronounced impact on the classifiers for infrequently sampled tail labels. This paper presents a method to systematically reduce label variance in XC by effectively utilizing the capabilities of an additional, tail-robust teacher model. It proposes a principled knowledge distillation framework, \model, which enhances tail performance in extreme classifiers, with formal guarantees on generalization. Finally, we introduce an effective instantiation of this framework that employs a specialized Siamese teacher model. This model excels in tail accuracy and significantly enhances the quality of student one-vs-all classifiers. Comprehensive experiments are conducted on a diverse set of XC datasets which demonstrate that \model can enhance tail performance by around 5% and 6% points in PSP and Coverage metrics respectively when integrated with leading extreme classifiers. Moreover, when added to the top-performing Renée classifier, it establishes a new state-of-the-art. Extensive ablations and analysis substantiate the efficacy of our design choices. Code and datasets will be released for research purposes.

Accurate and Efficient Cross-encoders for Ranking

Published in Under review at NeurIPS 2024, 1900

Authors: Bhawna Paliwal^, Deepak Saini^, Mudit Dhawan, Siddarth Asokan, Nagarajan Natarajan, Surbhi Aggarwal, Pankaj Malhotra, Jian Jiao, Manik Varma
Keywords: Recommendation Systems, Cross-Encoder Architecture, Efficient Machine Learning

Abstract

Ranking a set of items based on their relevance to a given query is a core problem in search and recommendation pipelines. Transformer-based cross-encoders are state-of-the-art models for ranking tasks. These models process and assign a relevance score for each query-item pair separately, independent of other relevant items to be ranked. In this work, we identify a critical oversight in the cross-encoder ranking models, i.e., their inability to model multiple items jointly leads to sub-optimal ranking accuracy and high computational costs. In response, we propose Cross-encoders with Joint Efficient Modeling, referred to as CROSS-JEM. This new ranking approach allows the transformer-based architectures to rank multiple items for a given query together, maximizing parameter utilization. By leveraging redundancies and token overlaps in the joint ranking of multiple items and a novel training objective modeling ranking probabilities, CROSS-JEM achieves state-of-the-art accuracy and supports over 30x lower ranking latency than standard cross-encoders. Our contributions extend to three key aspects: (i) highlighting the disparity between ranking application's demand for scoring multiple (order of thousands) items per query and the limited capabilities of current cross encoders; (ii) introducing CROSS-JEM for efficient joint scoring of multiple items per query, and (iii) achieving state-of-the-art accuracy on standard public datasets as well as a proprietary dataset. CROSS-JEM opens up new directions for the design of tailored early-attention-based ranking models that incorporate strict production constraints such as item multiplicity and latency.

Thesis

Multimodal Fake News Analysis and Detection Permalink

Published in B.Tech. Thesis, IIIT Delhi, 2021

Mudit Dhawan, Bachelor’s Thesis, IIIT Delhi
Advisors: Prof. Ponnurangam Kumaraguru and Prof. Rajiv Ratn Shah
Keywords: Multimodal Fake News, Indian Reginal Languages, Explainable System, Multiple Images, Contrastive Learning
code, pdf

Abstract

Fake News has become the curse of our time. Online social media networks provide a low-cost platform to facilitate information and fact sharing, but it fails to offer any quality control. As the number of people receiving their daily news through these platforms increases, it becomes a significant problem for the government and other organizations. Fake News articles leverage the multimedia content posted on the platforms and mislead the reader through fabricated image(s) or text (title and text body) accompanying it. Many organizations have started an initiative to provide de-bunked fake news, i.e., fact-checked and verified counterfeit news items floated on various social media platforms by human fact-checkers. Though this human intervention is a good start towards eradicating this evil, it can not be feasible at a larger scale providing human fact-checked information for every post on social media. The scalability of this human fact-checked information isn’t the only issue, but the promptness of such accurate information becomes crucial in this digital age. To address this problem, we aim to analyze multimodal fake content from platforms supporting online journalism (including various social media platforms) to extract meaningful features and design an all-inclusive early-stage Automated Fake News Detection System.