Please wait ...
0% Complete
فارسی
Home
/
شانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش
Robustness Gap in NLP Models for Vulnerability Descriptions: Benchmarking and Data Augmentation
Authors :
AmirHossein Majd
1
Mahdi Yousefikia
2
Saghar Ghasemzadeh
3
Amirreza Asari
4
Arya Khoshnavataher
5
Seyedeh Leili Mirtaheri
6
1- University of Calabria
2- دانشگاه خوارزمی
3- دانشگاه خوارزمی
4- دانشگاه خوارزمی
5- دانشگاه خوارزمی
6- University of Calabria
Keywords :
Software Vulnerabilities،Natural Language Processing،Robustness Benchmark،Noise Injection،Exploitability Prediction،Data Augmentation،Cybersecurity
Abstract :
Software vulnerability descriptions from CVE/NVD are the primary corpus for analysis, prioritization, and risk management in cybersecurity. Yet natural noise (typos, synonym substitutions, lexical variety) and adversarial perturbations undermine the accuracy and trustworthiness of NLP models. This paper presents, to our knowledge, the first systematic benchmark of NLP robustness on vulnerability descriptions. We train nine diverse architectures—lightweight transformers (MiniLM, MPNet, SBERT), hybrid models (BERT-LSTM, TextRCNN), and classical recurrent networks (BiLSTM, LSTM)—on a balanced dataset of over 56,000 real-world records from NVD and Exploit-DB, and fine-tune them for exploitability prediction. For comprehensive evaluation, we inject three noise families into test sets at levels from 10% to 80%: character-level edits (substitutions/swaps), synonym replacements using WordNet, and composite adversarial attacks generated with TextAttack. Performance declines across all models as noise rises, but vulnerability profiles differ: MiniLM attains the strongest clean-data score (F1 ≈ 0.933) yet is most brittle under character noise, whereas TextRCNN, despite a lower baseline, preserves comparatively higher stability in heavily perturbed conditions. Finally, we test a pragmatic hardening strategy—data augmentation with noisy variants followed by retraining—which consistently narrows robustness gaps across architectures without materially sacrificing clean-data accuracy. The benchmark and code enable reproducible evaluation and future robust modeling in cybersecurity.
Papers List
List of archived papers
Embedded speech encoder for low-resource languages
Alireza A.Tabatabaei - Pouria Sameti - Ali Bohlooli
Design and Simulation of a New Multiplexer with Energy Analysis in Quantum Cellular Automata Technology
- - -
طراحی و پیاده سازی بستر اجرای بازی جنگ سایبری
مریم نصراصفهانی - بهروز ترک لادانی - بهروز شاهقلی قهفرخی - حسین قجاوند بلتیجه - نوید شیرمحمدی - مهدی شمس - محمدامین آقاکبیری
Combinatorial Auction Based on Social Choice in the Internet of Things
Maede Esmaeili - Faria Nassiri-Mofakham - Fatemeh Hassanvand
A Swarm Intelligence Approach to Design Optimal Repeaters in Multilayer Graphene Nanoribbon Interconnects
Majid Sanaeepur - Maryam Momeni
Distributed Learning Automata-based Algorithm for Finding K-Clique in Complex Social Networks
Mohammad Mehdi Daliri Khomami - Alireza Rezvanian - Ali Mohammad Saghiri - Mohammad Reza Meybodi
آرتمیا: پروتکل مسیریابی مبتنی بر انجمن و آگاه به نظم تماس در شبکة اجتماعی متحرک تأخیرپذیر
سعید مرادی - جمشید باقرزاده محاسفی
پیشبینی حجم ترافیک شهری با استفاده از دادههای سرویس نشان مورد مطالعاتی: خیابان کمال اصفهان
مهسا لطیفی - جمشید مالکی
Automatic Analysis of Inconsistencies in Inter-Enterprise Business Processes: Introducing a Formal Adaptation Patterns Catalog
Somayeh Ashourian - Shohreh َAjoudanian
AI-based Secure Intrusion Detection Framework for Digital Twin-enabled Critical Infrastructure
Tanisha Patel - Nilesh Kumar Jadav - Tejal Rathod - Sudeep Tanwar - Deepak Garg - Hossein Shahinzadeh
more
Samin Hamayesh - Version 45.0.1