لطفا منتظر بمانید ...
0% Complete
English
صفحه اصلی
/
شانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش
Design of low-latency Floating-Point units for Softmax Computation in Transformer-based Large Language Models
نویسندگان :
Hoda Ghabeli
1
Amir Sabbagh Molahosseini
2
1- دانشکاه آزاد کرمان
2- دانشکاه آزاد کرمان
کلمات کلیدی :
LLM،transformer،softmax،speculative،floating-point
چکیده :
Large Language Models (LLMs) have emerged as one of the most desirable and widely used interactive digital tools in the world in the last decade. Softmax is one of the key steps in LLMs where the output is a vector of probabilities for each token in the model dictionary. The softmax computations are time-consuming due to the large vocabulary size, which can significantly increase the exponential computations and normalization, impacting the overall speed of the model. Given the importance of accuracy and speed, some of the main operations and computations of softmax are performed on the floating-point units. Arithmetic speculative computations are considered when the result of the computations can be estimated from a path shorter than the critical path, with improved speedup. In this paper, speculative 32-bit floating-point computation is proposed by merging two formats, 32-bit and 16-bit, for softmax computations. Both the floating-point adder and the floating-point multiplier use this strategy. The proposed design, based on the input data of the softmax function, speculates that the 32-bit floating-point computations can be obtained by concatenating the result of 16-bit format and a part of the 32-bit format result, that gives correct results most of the time with less delay. If speculation is unsuccessful, the longer path from through the conventional 32-bit floating-point unit is activated at the cost of a slightly longer critical path. Experimental results show that speculative floating-point units lead to a reduction in delay with only marginal overhead in area and power consumption.
لیست مقالات
لیست مقالات بایگانی شده
Improved Weighting in the Automated Texts Classification using Fuzzy Method
Hamidreza Sadrarhami - S. Mohammadali Zanjani - Ghazanfar Shahgholian
A Mathematical Optimization Approach for Preference Learning in Movie Recommender Systems with Shared Accounts
Milad Khademali - Fazlollah Aghamohammadi - Marjan Kaedi - Alireza Nasiri
Sentiment Analysis of the Amazon Customers Using the BiGRU Neural Network Enhanced by Attention Mechanism
Sara Sinan Salman al-Abedi - Keyvan Mohebbi
استخراج ویژگی مجموعه دادههای پزشکی دارای ابعاد بالا با استفاده از برنامه نویسی ژنتیک چند منظوره
سحر فقیهی راد - دکتر سیده نفیسه آل محمد سحر فقیهی راد - سیده نفیسه آل محمد -
تشخیص بیماری مزمن کلیوی با استفاده از یادگیرندههای گروهی و انتخاب ویژگیهای مؤثر مبتنی بر الگوریتم بهینهسازی تبادل حرارتی
صبا عارفنیا - مهدی هاشمزاده - امین گلزاری اسکوئی
Distributed Learning Automata-based Algorithm for Finding K-Clique in Complex Social Networks
Mohammad Mehdi Daliri Khomami - Alireza Rezvanian - Ali Mohammad Saghiri - Mohammad Reza Meybodi
Knowledge gap extraction based on the learner click behavior in interaction with videos using the association rule algorithm
Yosra Bahrani - Omid Fatemi
روش مهاجرت خوشهای برای بهبود بستربندی به مشتری در گردشکارهای بدون سرویسدهنده
محمدامین قسوری جهرمی - مهرداد آشتیانی - فاطمه بخشی
Binary water stream algorithm: a new meta-heuristic optimization technique
Faezeh Rahimi Sebdani - Mehdi Nasri
A Topic Based Method to Classify the Question Clarity in CQA Networks
Alireza Khabbazan - Dr Ahmad Ali Abin
بیشتر
ثمین همایش، سامانه مدیریت همایش ها و جشنواره ها - نگارش 45.0.1