title: Dynamic Few-shot Learning for Computational Social Science

authors: R Malla, TG Coan, V Srinivasan, C Boussalis

snippet: Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) and have recently demonstrated remarkable potential in social science research through their capacity to efficiently perform a wide range of text-as-data tasks …


title: Easy Problems That LLMs Get Wrong

authors: S Williams, J Huckle - arXiv preprint arXiv:2405.19616, 2024

snippet: We introduce a comprehensive Linguistic Benchmark designed to evaluate the limitations of Large Language Models (LLMs) in domains such as logical reasoning, spatial intelligence, and linguistic understanding, among others. Through a series of …


title: Echoes of culture: Relationships of implicit and explicit attitudes with contemporary English, historical English, and 53 non-English languages

authors: TES Charlesworth, K Morehouse, V Rouduri… - Social Psychological and …, 2024

snippet: Attitudes are intertwined with culture and language. But to what extent? Emerging perspectives in attitude research suggest that cultural representations in language are more related to implicitly measured (vs. explicitly measured) attitudes, and that …


title: Edge-LLMs: Edge-Device Large Language Model Competition

authors: S Liu, K Han, A Fernandez-Lopez, A Jaiswal…

snippet: … The C4 dataset is a colossal, cleaned version of Common Crawl’s web crawl corpus, which is mainly intended to pre-train language models and word representations. Language models like MPT-7B and T5 are pre-trained with the C4 …


title: Effektivisering av fakturaklassificering enligt UNSPSC-standarden: en maskininlärningslösning

snippet: Procurement requirements and procurement analysis are approaches used by the Swedish National Agency for Public Procurement to ensure and maintain a sustainable societal development. The aim is to safeguard tax funds and ensure that …


title: Efficiency Comparison of Dataset Generated by LLMs using Machine Learning Algorithms

authors: P Pawade, M Kulkarni, S Naik, A Raut, KS Wagh - 2024 International Conference on …, 2024

snippet: … • Bing Chat was trained on web text data, including Common Crawl, Wikipedia, news articles, social media posts, and other publicly available text, via a technique called denoising autoencoder (DAE). In DAE, the model is given a noisy version of a …


title: Efficient learning in spiking neural networks

authors: A Rast, MA Aoun, EG Elia, N Crook - Neurocomputing, 2024

snippet: … Training data required is similarly massive: the CommonCrawl dataset is 242 TB in size, and hyperparameter tuning remains an expensive, search-based process that can require ∼ 1000 − 10 , 000 + sweeps. After evaluating a number of possible …


title: Efficient Model-Relational Data Management: Challenges and Opportunities

authors: V Sanca, A Ailamaki - IEEE Transactions on Knowledge and Data …, 2024

snippet: As modern data pipelines continue to collect, produce, and store various data formats, extracting and combining value from traditional and context-rich sources becomes unsuitable for RDBMS. To tap into the dark data, domain experts analyze …


title: Efficient query clustering and information retrieval using Sequenced User Search Pattern Query Optimization

authors: S Surya, P Sumitra - Multimedia Tools and Applications, 2024

snippet: The rapid growth of the internet has ushered in an era of unprecedented information availability. However, this expansion has also given rise to the formidable challenge of efficiently extracting relevant data from the vast expanse of web pages …


title: Efficient Training and Inference: Techniques for Large Language Models Using Llama

snippet: … The training dataset consisted of a mixture of the Common Crawl, Wikipedia, and BooksCorpus datasets, encompassing a diverse range of linguistic structures and vocabularies. For evaluation, the GLUE benchmark suite was utilized, providing a …


title: Efficient URL and URI Compression

authors: F Savins, K Saric, GS Ramachandran, R Jurdak - 2024 33rd International Conference …, 2024

snippet: … The dataset explored was taken from the Common Crawl project and consists of 5 million URIs in 5 sets. The URIs were found to be 99.4% URLs with the rest being other URI formats such as “Javascript:” and “Tel:”. These URIs were selected …


title: Eir: Thai Medical Large Language Models

authors: Y Thiprak, R Ngodngamthaweesuk… - arXiv preprint arXiv …, 2024

snippet: We present Eir Thai Medical LLM, a large language model with 8 billion parameters, specifically designed to enhance the accuracy of handling medical tasks in the Thai language. This model focuses on providing clear and easy-to-understand answers …


title: Eliciting Big Five Personality Traits in Large Language Models: A Textual Analysis with Classifier-Driven Approach

authors: A Hilliard, C Munoz, Z Wu, AS Koshiyama - arXiv preprint arXiv:2402.08341, 2024

snippet: … GPT-3 followed, with 175 billion parameters and training on datasets like Common Crawl [15]. GPT-3.5 Turbo was introduced to enhance real-time performance, and the latest, GPT-4, boasts 1.76 trillion parameters and capabilities …


title: EMBA: Entity Matching using Multi-Task Learning of BERT with Attention-over-Attention

snippet: … The WDC Product Data Corpus for Large-scale Product Matching [34], was built by extracting product offers from Common Crawl. The WDC datasets serve as a popular benchmark and have been used for evaluation in DITTO, JointBERT, and …


title: Embedding-based Query Spelling Correction

snippet: For many retrieval systems, correcting spelling errors in the queries that searchers submit is an essential step of query understanding. Inspired by a blog post on spelling correction from 2018, we implement and analyze a simple embedding-based …


title: Emergent Abilities in Reduced-Scale Generative Language Models

authors: S Muckatira, V Deshpande, V Lialin, A Rumshisky - arXiv preprint arXiv:2404.02204, 2024

snippet: Large language models can solve new tasks without task-specific fine-tuning. This ability, also known as in-context learning (ICL), is considered an emergent ability and is primarily seen in large language models with billions of parameters. This …


title: Emerging AI Tools for Education and Research: Perspective and Policies for IISc

snippet: The Director, IISc, constituted a committee on August 21, 2023 with the following mandate:(a) Explore the challenges and benefits of emerging AI tools in the context of academic teaching and learning,(b) provide guidance and recommendations to …


title: Emotion Analysis in Narrative Economics

snippet: The purpose of this quantitative cross-sectional design using a structural equation model was to determine if emotions mediate the viral spread of narratives with potentially far-reaching economic and political consequences. This dissertation …


title: Emotional significance in Cross-Cultural Semantic Crossmodal Correspondences

authors: J Alvarado - Proceedings of the Annual Meeting of the Cognitive …, 2024

snippet: Crossmodal correspondences are associations between perceptual features from different senses that aid in crossmodal binding. The semantic coding of these correspondences is expected to capture and mediate the emergence of perceptual …


title: Employing Siamese MaLSTM Model and ELMO Word Embedding for Quora Duplicate Questions Detection

authors: A Altamimi, M Umer, D Hanif, S Alsubai, TH Kim… - IEEE Access, 2024

snippet: … 3) FastText Subword FastText Subword comprises a collection of 2 million word vectors trained on the Common Crawl dataset, which consists of a massive 600 billion tokens. In contrast to traditional word embeddings, sub-word embeddings …


title: Empowering hate speech detection: leveraging linguistic richness and deep learning

authors: IGBJ Abasan, EB Setiawan - Bulletin of Electrical Engineering and Informatics, 2024

snippet: … What they don’t realize is the use of pre-trained models, namely CommonCrawl and Wiki, in extraction using FastText or GloVe. The pre-trained model used is not specific for hate speech detection and, of course, consists of many languages, and …


title: Empowering Humans with Machines: From Explanations to Teaching

snippet: … This dataset contains hundreds of thousands of online biographies from the Common Crawl corpus. The task is to predict a person’s profession given a biography. The original dataset consists of 29 professions, and we narrow it down to …


title: Empowering knowledge through AI: open scholarship proactively supporting well trained generative AI

snippet: … The bulk of the content consists of the common crawl dataset, a free-to-use dataset consisting of over 250 billion webpages from the last 17 … Both the common crawl dataset and the WebText2 dataset rely heavily on well-linked and well-described …


title: Enabling action crossmodality for a pretrained large language model

authors: A Caesar, O Özdemir, C Weber, S Wermter - Natural Language Processing Journal, 2024

snippet: Natural language processing and vision tasks have seen large improvements recently through the rise of Transformer architectures. The high performing large language models (LLMs) benefit from large textual datasets that are numerously …


title: Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment

authors: A Agarwalla, A Gupta, A Marques, S Pandit, M Goin… - arXiv preprint arXiv …, 2024

snippet: Large language models (LLMs) have revolutionized Natural Language Processing (NLP), but their size creates computational bottlenecks. We introduce a novel approach to create accurate, sparse foundational versions of performant LLMs that achieve full …


title: Enabling self-identification in intelligent agent: insights from computational psychoanalysis

authors: L Li, C Li - arXiv preprint arXiv:2403.07664, 2024

snippet: Building upon prior framework of computational Lacanian psychoanalysis with the theory of active inference, this paper aims to further explore the concept of self-identification and its potential applications. Beginning with two classic paradigms in psychology …


title: End to End Urdu Abstractive Text Summarization with Dataset and Improvement in Evaluation Metric

snippet: Urdu, being a common language in South Asia, has not received significant attention in terms of language processing compared to more advanced languages. In the field of Natural Language Processing (NLP), the task of text summarization …


title: Energy and AI

authors: J Göpfert, JM Weinand, P Kuckertz, D Stolten

snippet: … Because models can be trained on this objective in a self-supervised setting, large unlabeled corpora (such as Wikipedia, book corpora, and Common Crawl data) can be used for training. The computational advantages of the transformer …


title: English-language abstract text summarization using the T5 model

authors: RD Darshan, I Surya, G Malarselvi - AIP Conference Proceedings, 2024

snippet: … The Common Crawl public web scrape is the source for the 750GB of English-language text that makes up the C4 dataset. The news summary dataset that we've used in this instance has 4515 samples and includes the following information: Headlines, Short …


title: EnhancedBERT: A Feature-rich Ensemble Model for Arabic Word Sense Disambiguation with Statistical Analysis and Optimized Data Collection

authors: S Kaddoura, R Nassar - Journal of King Saud University-Computer and …, 2024

snippet: Accurate assignment of meaning to a word based on its context, known as Word Sense Disambiguation (WSD), remains challenging across languages. Extensive research aims to develop automated methods for determining word senses in …


title: Enhancing 3D Asset Retrieval with Semantic Search

snippet: Semantic search goes beyond exact keyword matching and instead focuses on the comprehension of search queries' purpose and the surrounding context to deliver relevant search results. In this work, we focus on integrating hybrid search, which …


title: Enhancing AI Tutoring in Robotics Education: Evaluating the Effect of Retrieval-Augmented Generation and Fine-Tuning on Large Language Models

authors: S Kahl, F Löffler, M Maciol, F Ridder, M Schmitz… - 2024

snippet: … The majority of data is gained from CommonCrawl with about 850 billion tokens. CommonCrawl is a free and open of data gathered by web crawls.In addition, the corpus is based on data from C4 (about 190 billion tokens), Github (about 100 billion …


title: ENHANCING AUTOMATIC INVOICE CODING PERFORMANCE WITH UNSTRUCTURED DATA

snippet: … The published word vectors are 300 dimensional and trained on the internet data from Common Crawl3. The pretrained word vectors can be further fine-tuned using the provided code and datasets more similar to the final usage scenario to increase …


title: Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation

authors: T Wang, N Zhou, Z Chen - arXiv preprint arXiv:2407.05437, 2024

snippet: Large language models (LLMs) and prompt engineering hold significant potential for advancing computer programming education through personalized instruction. This paper explores this potential by investigating three critical research questions: the …


title: Enhancing Cyber Security Enhancement Through Generative AI

snippet: … GPT-1: Released in 2018, GPT-1 was trained on the Common Crawl and BookCorpus datasets. It demonstrated proficient language comprehension but limited conversational coherence and retention of context. GPT-2: Trained on …


title: Enhancing Cybersecurity: A Review and Comparative Analysis of Convolutional Neural Network Approaches for Detecting URL-Based Phishing Attacks

authors: M Nanda, M Saraswat, PK Sharma - e-Prime-Advances in Electrical Engineering …, 2024

snippet: Phishing attempts to mimic the official websites of businesses, including banks, e-commerce, government offices, and financial institutions. Phishing websites aim to collect and retrieve sensitive data from users, including passwords, credit card numbers, email …


title: Enhancing Ecological Knowledge Discovery Using Large Language Models

snippet: Earth is home to an astonishing diversity of life forms. Among these, vascular plants stand as one of the most vital and ubiquitous groups, accounting for approximately 80% of global biomass [3]. The census of vascular plants, which presently exceeds …


title: Enhancing English Translation Quality Assessment through Knowledge Transfer in Artificial Intelligence Context

snippet: Abstract Machine translation technology, which employs computers to autonomously convert text between source and target languages, represents a pivotal realm within artificial intelligence and natural language processing research. This paper …


title: Enhancing knowledge graphs with microdata and LLMs: the case of Schema. org and Wikidata in touristic information

authors: L Gonzalez-Garcia, G González-Carreño… - The Electronic Library, 2024

snippet: … We have used the Common Crawl (CC) corpus of crawled Web pages to extract metadata published in the wild, and then used a mapping of Schema.org annotations to Wikidata to get estimates of the added value (in terms of additional …


title: Enhancing Large Language Models for Structured Data Extraction

snippet: This thesis examines the application of large language models for structured data extraction using microdata-annotated webpages from the Web Data Commons (WDC). It compares the performance of models trained on WDC data with those trained on …


title: Enhancing Misogyny Detection in Bilingual Texts Using FastText and Explainable AI

authors: E Hashmi, MM Yamin, S Imran, SY Yayilgan, M Ullah - 2024 International Conference …, 2024

snippet: … It features an extensive dictionary with 2 million words drawn from the Common Crawl database. Every word is depicted in a 300-dimensional vector space, amassing an enormous total of 600 billion word vectors. 1) Unsupervised FastText …


title: Enhancing Multilingual Hate Speech Detection: From Language-Specific Insights to Cross-Linguistic Integration

authors: E Hashmi, SY Yayilgan, IA Hameed, MM Yamin… - IEEE Access, 2024

snippet: … It features a vast vocabulary of 2 million words from the Common Crawl dataset, mapping each to a 300-dimensional vector space. What … This model was built using Common Crawl and Wikipedia using FastText’s unsupervised learning …


title: Enhancing Parameter Efficiency in Model Inference Using an Ultralight Inter-Transformer Linear Structure

snippet: … It was constructed from the Chuweb21 generated based on the April 2021 block of the Common Crawl dataset. The WWW-4 dataset comprises 50 queries. Robust04 dataset is also an English web corpus generated from TREC disks 4 and 5, and …


title: Enhancing Phishing Detection, Leveraging Deep Learning Techniques

authors: A Ullah, RA Shah, SA Nawaz, N Ahmad, MH Malik - Journal of Computing & …, 2024

snippet: … To achieve this, we utilized the Common Crawl Corpus, which aggregates extensive data collected over seven years. This corpus comprises raw web page data, metadata, and textual content. Legitimate domains with substantial backlink …


title: Enhancing Product Design through AI-Driven Sentiment Analysis of Amazon Reviews Using BERT

authors: MK Shaik Vadla, MA Suresh, VK Viswanathan - Algorithms, 2024

snippet: Understanding customer emotions and preferences is paramount for success in the dynamic product design landscape. This paper presents a study to develop a prediction pipeline to detect the aspect and perform sentiment analysis on review …


title: Enhancing Sentiment Analysis Accuracy in Borobudur Temple Visitor Reviews through Semi-Supervised Learning and SMOTE Upsampling

authors: C Agustina, P Purwanto, F Farikhin - Journal of Advances in Information Technology, 2024

snippet: The level of visitor satisfaction with tourist destinations can be known from reviews on social media. One method used is to carry out sentiment analysis on comments given by visitors on social media or related websites. This study was envisioned as a …


title: Enhancing Sentiment Analysis on Social Media Data with Advanced Deep Learning Techniques.

authors: HH Nguyen - International Journal of Advanced Computer Science & …, 2024

snippet: This paper introduces a comprehensive methodology for conducting sentiment analysis on social media using advanced deep learning techniques to address the unique challenges of this domain. As digital platforms play an increasingly pivotal …


title: Enhancing Text Summarization in a Limited Data and Long Sample Setting

snippet: This thesis addresses the challenges of automatic text summarization in a limited amount of data and a significantly large sample size setting. The study introduces a novel dataset of Ukrainian legislative documents professionally translated into …


title: Enhancing Text Summarization in a Limited Data and Long Sample

snippet: In our rapidly digitizing world, the volume of online information is growing at an unprecedented rate. Legal systems worldwide do not lag behind and generate massive amounts of documents daily. Lawyers are tasked with navigating an ever-growing …


title: Enhancing Vision-Language Pre-training with Rich Supervisions

authors: Y Gao, K Shi, P Zhu, E Belval, O Nuriel, S Appalaraju… - arXiv preprint arXiv …, 2024

snippet: … This is expected as our data collected from Common Crawl has different distribution against the original Pix2struct data due to the data size (2M vs. 80M) and potentially different website filtering strategies. Specifically, we filter out website …


title: Enriching Interactive Explanations with Fuzzy Temporal Constraint Networks

authors: M Canabal-Juanatey, JM Alonso-Moral, A Catala… - International Journal of …, 2024

snippet: … such as BERT [2] (Bidirectional Encoder Representations from Transformers), GPT [26] (Generative Pre-trained Transformer) and their successors (ALBERT [4], RoBERTa [3], BART [7], GPT-3 [5] or GPT-4 [27]), which were trained with huge …


title: Enriching satellite image annotations of forests with keyphrases from a specialized corpus

authors: N Neptune, J Mothe - Multimedia Tools and Applications, 2024

snippet: … We test fastText embeddings trained on Common Crawl Footnote 4 and Wikipedia Footnote 5 with dimension 300 and a context size of 5 … These fastText models were pre-trained on Common Crawl and Wikipedia. None of the models was …


title: Entrant: A large financial dataset for table understanding

authors: E Zavitsanos, D Mavroeidis, E Spyropoulou… - Scientific Data, 2024

snippet: Tabular data is a way to structure, organize, and present information conveniently and effectively. Real-world tables present data in two dimensions by arranging cells in matrices that summarize information and facilitate side-by-side comparisons …


title: EpilepsyLLM: Domain-Specific Large Language Model Fine-tuned with Epilepsy Medical Knowledge

authors: X Zhao, Q Zhao, T Tanaka - arXiv preprint arXiv:2401.05908, 2024

snippet: With large training datasets and massive amounts of computing sources, large language models (LLMs) achieve remarkable performance in comprehensive and generative ability. Based on those powerful LLMs, the model fine-tuned with domain-specific …


title: Error Patterns in Vision Transformers: A Focused Study on InternVL's Real-World Performance

snippet: This study focuses on the errors in Vision Transformers, particularly examining the realworld performance of InternVL, an opensource multimodal model designed to rival commercial systems. The primary tasks explored include image captioning and …


title: Establishing Global AI Accountability: Training Data Transparency, Copyright, and Misinformation

snippet: As artificial intelligence (AI) technologies continue advancing at a rapid pace, the systems' growing capabilities as well as their expanding integration into vital social functions are raising complex questions around trust and accountability. AI models …


title: Establishing the importance of co-creation and self-efficacy in creative collaboration with artificial intelligence

authors: J McGuire, D De Cremer, T Van de Cruys - Scientific Reports, 2024

snippet: The emergence of generative AI technologies has led to an increasing number of people collaborating with AI to produce creative works. Across two experimental studies, in which we carefully designed and programmed state-of-the-art human–AI …


title: Ethical aspects of ChatGPT: An approach to discuss and evaluate key requirements from different ethical perspectives

authors: M Steen, J Greeff, M Boer, C Veenman - AI and Ethics, 2024

snippet: … Many LLMs have been trained with data from Common Crawl, Footnote 9 which contains inaccuracies, errors, and bias. Another concern is whether people can access ChatGPT and, for example, change parameters or delete data, so that the …


title: Ethical Considerations for Generative AI in Social Science Research

authors: BA Brown, KL Heitner - Generative AI and Implications for Ethics, Security, and …, 2024

snippet: Social science research embodies the inquiry into people as individuals and their interpersonal interactions with each other in communities and varied societies, with due consideration for their natural, technological, and constructed environments …


title: Ethics of AI in the Teaching of English

snippet: New technologies have always impacted literacy, and artificial intelligence (AI) is no exception (Tyner, 1998). English teachers find themselves having to consider how AI may impact how they teach. NCTE’s Beliefs for Integrating Technology Into the …


title: EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation

authors: AL Tonja, IA Azime, TD Belay, MG Yigezu… - arXiv preprint arXiv …, 2024

snippet: Large language models (LLMs) have gained popularity recently due to their outstanding performance in various downstream Natural Language Processing (NLP) tasks. However, low-resource languages are still lagging behind current state-of-the-art …


title: EvalQuiz: self-assessment generated through language transformer models

snippet: This thesis explores the constraints of self-assessment creation in higher education, focusing on the lack of tools, standardization, and time. We conduct didactic field expert interviews to investigate how teaching evolves towards learning management …


title: Evaluating and improving lexical language understanding in neural machine translation

snippet: Lexical understanding is an inalienable component of the translation process. In order to correctly map the meaning of a linguistic unit to the appropriate target language expression, the meaning of its constituent words has first to be identified …


title: Evaluating and Improving the Reasoning Abilities of Language Models

snippet: This thesis focuses on evaluating and improving the reasoning abilities of Smaller Language Models (SLMs) and Large Language Models (LLMs). It explores SLMs’ performance on complex tasks and their limitations with simpler ones. This thesis …


title: Evaluating and Mitigating Limitations of Large Language Models in Clinical Decision Making

authors: P Hager, F Jungmann, K Bhagat, I Hubrecht, M Knauer… - medRxiv, 2024

snippet: Clinical decision making is one of the most impactful parts of a physician's responsibilities and stands to benefit greatly from AI solutions and large language models (LLMs) in particular. However, while LLMs have achieved excellent …


title: Evaluating Differential Privacy Approaches for Query Obfuscation in Information Retrieval

snippet: Protecting the privacy of a user while they interact with an Information Retrieval (IR) system is crucial. This becomes more challenging when the IR system is not cooperative in satisfying the user’s privacy needs. Recent advancements in Natural …


title: Evaluating embedded semantics for accessibility description of web crawl data

authors: R Navarrete, D Martinez-Mosquera, L Recalde… - AHFE (2023) International …, 2023

snippet: … The dataset for analysis is provided by Web Data Commons (WDC), an organization that releases extracted data from Common Crawl (CC), the largest web corpus available to the public. The dataset was released in October 2021 …


title: Evaluating Embedding Models for Clustering Italian Political News: A Comparative Study of Text-Embedding-3-Large and UmBERTo

snippet: … This study compares two embedding models: OpenAI's text-embedding-3-large model and umberto-commoncrawl-cased-v1 (hereafter referred to as UmBERTo) (Parisi et al., 2020). UmBERTo, a BERT-based model (more specifically RoBERTa-based) …


title: Evaluating English-language morphological awareness assessments

authors: CL Hudson Kam, E Sadlier-Brown, S Clark, C Jang… - First Language, 2024

authors: CLH Kam, E Sadlier-Brown, S Clark, C Jang, CD Epp… - First Language, 2024

snippet: … These corpora were selected because they contain language that children are exposed to, and so, provide a more realistic picture of children’s potential knowledge than an adult-language based corpus would (eg the Enron email dataset …


title: Evaluating Generative Language Models in Japanese Using Question-Answering Templates: A Synthetic Data Generation Approach

snippet: In the last five years, transformer-based large language models have not only made tremendous advancements in fluent and accurate text generation, but also show unparalleled performance as zero-or few-shot shot learners of numerous …


title: Evaluating GPT and BERT models for Protein-Protein interaction identification in biomedical text

authors: H Rehana, NB Çam, M Basmaci, J Zheng, C Jemiyo… - Bioinformatics Advances, 2024

snippet: Detecting protein-protein interactions is crucial for understanding genetic mechanisms, disease pathogenesis, and drug design. As biomedical literature continues to grow rapidly, there is an increasing need for automated and accurate …


title: Evaluating Large Language Model Performance on Haskell

snippet: I introduce HaskellEval, a Haskell evaluation benchmark for Large Language Models. HaskellEval’s curation leverages a novel synthetic generation framework, streamlining the process of dataset curation by minimizing manual intervention. The …


title: Evaluating Large Language Models for Generalization and Robustness via Data Compression

authors: Y Li, Y Guo, F Guerin, C Lin - arXiv preprint arXiv:2402.00861, 2024

snippet: Existing methods for evaluating large language models face challenges such as data contamination, sensitivity to prompts, and the high cost of benchmark creation. To address this, we propose a lossless data compression based evaluation …


title: Evaluating Large Language Models on Academic Literature Understanding and Review: An Empirical Study among Early-stage Scholars

authors: J Wang, H Hu, Z Wang, Y Song, Y Sheng, D He - 2024

snippet: The rapid advancement of large language models (LLMs) such as ChatGPT makes LLM-based academic tools possible. However, little research has empirically evaluated how scholars perform different types of academic tasks with LLMs …


title: Evaluating LLMs on large contexts: a RAG approach on text comprehension

snippet: While the latest Large Language Models (LLMs) continue to expand in size and context window capacity, their knowledge base remains constrained by their training corpus. Retrieval Augmented Generation (RAG) offers a solution to this limitation by …


title: Evaluating Multilingual Abstractive Dialogue Summarization in Indian Languages using mT5-small & IndicBART

authors: M Sharma, G Goyal, A Gupta, R Rani, A Sharma, A Dev - 2024 IEEE 9th International …, 2024

snippet: … The mC4 dataset consists of natural text in 101 languages including Indian languages like Hindi, Mararti, Punjabi and others, that was collected from the public Common Crawl web scrape. Introduced as a multilingual, sequence-to-sequence pre-trained …


title: Evaluating Natural Monopoly Conditions in the AI Foundation Model Market

snippet: The authors of this report examined the economic and production attributes of pre-trained artificial intelligence (AI) foundation models to answer the following questions: Does the market for foundation models have the characteristics of a natural monopoly, and …


title: Evaluating Prediction-Based Theories of Bilingual Comprehension of Spanish/English Codeswitches.

snippet: This dissertation investigates how bilinguals use their two grammars to comprehend written intra-sentential codeswitches. I focus on adjective/noun constructions in Spanish and English where I manipulate the congruence of grammatical word order …


title: Evaluating Shortest Edit Script Methods for Contextual Lemmatization

authors: O Toporkov, R Agerri - arXiv preprint arXiv:2403.16968, 2024

snippet: Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES), namely, the number of edit operations to transform a word form into its lemma. In fact, different methods of computing SES have been proposed as an …


title: Evaluating the Capabilities of GPT-4 in Full-Stack Web Development: A Practical Approach

snippet: With the rapid advancements in Artificial Intelligence, leveraging machine learning models for various domains has become increasingly prevalent. This thesis explores the capabilities of OpenAI’s Generative Pre-trained Transformer 4 (GPT-4) within full-stack …


title: Evaluating the Experience of LGBTQ+ People Using Large Language Model Based Chatbots for Mental Health Support

authors: Z Ma, Y Mei, Y Long, Z Su, KZ Gajos - arXiv preprint arXiv:2402.09260, 2024

snippet: LGBTQ+ individuals are increasingly turning to chatbots powered by large language models (LLMs) to meet their mental health needs. However, little research has explored whether these chatbots can adequately and safely provide tailored support …


title: Evaluating the Factuality of Zero-shot Summarizers Across Varied Domains

authors: S Ramprasad, K Krishna, ZC Lipton, BC Wallace - arXiv preprint arXiv:2402.03509, 2024

snippet: Recent work has shown that large language models (LLMs) are capable of generating summaries zero-shot (ie, without explicit supervision) that, under human assessment, are often comparable or even preferred to manually composed …


title: Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

authors: S Kahl, F Löffler, M Maciol, F Ridder, M Schmitz… - arXiv preprint arXiv …, 2024

snippet: This study evaluates the performance of Large Language Models (LLMs) as an Artificial Intelligence-based tutor for a university course. In particular, different advanced techniques are utilized, such as prompt engineering, Retrieval-Augmented-Generation …


title: Evaluating the impact of design decisions on passive DNS-based domain rankings

authors: V Le Pochat, S Fernandez, T Van Goethem…

snippet: … They recommend that researchers select a random sample of websites among Common Crawl hosts. Ruth et al. [6] compared top lists to Cloudflare traffic data, concluding that the Chrome User Experience Report [11] most accurately represents …


title: Evaluating the Quality of AI-Generated Items for a Certification Exam

authors: AD Mead, C Zhou - Journal of Applied Testing Technology, 2024

snippet: OpenAI’s GPT-3 model can write multiple-choice exam items. This paper reviewed the literature on automatic item generation and then described the recent history of OpenAI GPT models and their operation, and then described a methodology for …


title: Evaluating Transformers and Linguistic Features integration for Author Profiling tasks in Spanish

authors: JA García-Díaz, G Beydoun, R Valencia-García - Data & Knowledge Engineering, 2024

snippet: … This model is a training from scratch of RoBERTa, trained on the Spanish texts from mC4, compiled from the public Common Crawl web … This is a multilingual version of RoBERTa, pre-trained with about 2.5 TB of data from the 100 different …


title: Evaluation and Adaptation of Neural Language Models for Under-Resourced Languages

snippet: Abstract Language models are now commonly used by researchers, industry, and anyone interested. However, language models of all sizes and types are primarily developed for the English language while efforts on other languages lag behind …


title: Evaluation of AI-generated Responses by Different Artificial Intelligence Chatbots to the Clinical Decision-Making Case-Based Questions in Oral and Maxillofacial …

authors: A Azadi, F Gorjinejad, H Mohammad-Rahimi, R Tabrizi… - Oral Surgery, Oral Medicine …, 2024

snippet: Objectives This study aims to evaluate the correctness of the generated answers by Google Bard, GPT-3.5, GPT-4, Claude-Instant, and Bing chatbots to decision-making clinical questions in the oral and maxillofacial surgery (OMFS) area. Study Design A …


title: Evaluation of Geographical Distortions in Language Models: A Crucial Step Towards Equitable Representations

authors: R Decoupes, R Interdonato, M Roche, M Teisseire… - arXiv preprint arXiv …, 2024

snippet: Language models now constitute essential tools for improving efficiency for many professional tasks such as writing, coding, or learning. For this reason, it is imperative to identify inherent biases. In the field of Natural Language Processing …


title: Evaluation of word embedding models used for diachronic semantic change analysis

authors: Y Maslennikova, V Bochkarev - Journal of Physics: Conference Series, 2024

snippet: … As a reference word2vec model, we used pre-trained word vectors that were trained on English versions of Common Crawl dataset using fastText library [14]. fastText is a library for efficient learning of word representations and sentence …


title: Evidence for a digital divide? Measuring DNS dependencies in the context of the indigenous population of Australia

authors: R Holz, N Nazemi, O Tavallaie, AY Zomaya

snippet: We recently presented a work-in-progress paper at the Workshop on Transparency, Accountability and User Control for a Responsible Internet (TAURIN 2023). Our paper investigates the relationship between the digital divide, Internet transparency …


title: EVLM: An Efficient Vision-Language Model for Visual Understanding

authors: K Chen, D Shen, H Zhong, H Zhong, K Xia, D Xu… - arXiv preprint arXiv …, 2024

snippet: In the field of multi-modal language models, the majority of methods are built on an architecture similar to LLaVA. These models use a single-layer ViT feature as a visual prompt, directly feeding it into the language models alongside textual tokens …


title: Evolution and Prospects of Foundation Models: From Large Language Models to Large Multimodal Models

authors: Z Chen, L Xu, H Zheng, L Chen, A Tolba, L Zhao, K Yu… - Computers, Materials and …, 2024

snippet: Since the 1950s, when the Turing Test was introduced, there has been notable progress in machine language intelligence. Language modeling, crucial for AI development, has evolved from statistical to neural models over the last two decades …


title: EXAMINING ACCURACY HETEROGENEITIES IN CLASSIFICATION OF MULTILINGUAL AI-GENERATED TEXT

snippet: Tools for detection of AI-generated texts are used globally, however, the nature of the apparent accuracy disparities between languages must be further observed. This paper aims to examine the nature of these differences through testing OpenAI’s …


title: Experimental Evaluation of Possible Feature Combinations for the Detection of Fraudulent Online Shops

authors: A Janavičiūtė, A Liutkevičius, G Dabužinskas… - Applied Sciences, 2024

snippet: Online shopping has become a common and popular form of shopping, so online attackers try to extract money from customers by creating online shops whose purpose is to compel the buyer to disclose credit card details or to pay money for …


title: Expert Phishing Detection System

authors: A Alsarhan, I Al-Aiash, D Al-Fraihat, M Aljaidi… - 2024 IEEE International …, 2024

snippet: … and Common Crawl datasets. The highest accuracy reported in this work was 83.00%, achieved with the RF classifier. In the study conducted by [16], the authors employed Machine Learning (ML) and Heuristic methods on three datasets—PhishTank …


title: Expert-Like Systems Engineering Artifacts? Insights from an Empirical

authors: M Husain, P Wach, TG Topcu - The Proceedings of the 2024 Conference on Systems …

snippet: The old saga of complex system development continues as the vast majority of government and industry programs continue to result in cost and schedule overruns [1–7]. While numerous researchers and government reports attribute this trend to the ever-increasing …


title: Explainable Generative AI (GenXAI): A Survey, Conceptualization, and Research Agenda

authors: J Schneider - arXiv preprint arXiv:2404.09554, 2024

snippet: Generative AI (GenAI) marked a shift from AI being able to recognize to AI being able to generate solutions for a wide variety of tasks. As the generated solutions and applications become increasingly more complex and multi-faceted, novel needs …


title: Exploring AI-driven approaches for unstructured document analysis and future horizons

authors: SV Mahadevkar, S Patil, K Kotecha, LW Soong… - Journal of Big Data, 2024

snippet: In the current industrial landscape, a significant number of sectors are grappling with the challenges posed by unstructured data, which incurs financial losses amounting to millions annually. If harnessed effectively, this data has the potential to …


title: Exploring Emergent Phenomena in AI: A Pantheistic Approach to the Underlying Source of Information

snippet: Emergent phenomena in artificial intelligence (AI) systems, where creative, novel, and unexpected outputs arise, have long intrigued researchers. While these behaviors are typically seen as products of complex computation, this paper …


title: Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russian

authors: A Glazkova, D Morozov - arXiv preprint arXiv:2409.10640, 2024

snippet: Keyphrase selection plays a pivotal role within the domain of scholarly texts, facilitating efficient information retrieval, summarization, and indexing. In this work, we explored how to apply fine-tuned generative transformer-based models to the …


title: Exploring Language Models and Question Answering in Biomedical and Arabic Domains

snippet: Despite the success of the Transformer model and its variations (eg, BERT, ALBERT, ELECTRA, T5) in addressing NLP tasks, similar success is not achieved when these models are applied to specific domains (eg, biomedical) and limited-resources …


title: Exploring large language models for microstructure evolution in materials

authors: P Satpute, S Tiwari, M Gupta, S Ghosh - Materials Today Communications, 2024

snippet: … These models are based on massive machine learning (ML) architectures (specifically transformer neural networks [3], [4]) pre-trained on a massive corpus of text data from diverse internet sources like Common Crawl [5] and Wikipedia [6], each …


title: Exploring LLMs' Capabilities for Error Detection in Dutch L1 and L2 Writing Products

authors: J Kruijsbergen, S Van Geertruyen, V Hoste… - Computational Linguistics in …, 2024

snippet: This research examines the capabilities of Large Language Models for writing error detection, which can be seen as a first step towards automated writing support. Our work focuses on Dutch writing error detection, targeting two envisaged end-users …


title: Exploring Low-Resource Machine Translation: Case Study of Lao-Vietnamese Translation

authors: QD Tran - 2024 International Conference on Multimedia Analysis …, 2024

snippet: Despite ongoing research efforts in recent years, machine translation in low-resource settings remains challenging. This paper explores recent methods to enhance low-resource machine translation (LoResMT), especially for both-side low-resource language …


title: Exploring Machine Learning for Malware Detection With Feature Selection, Explainable AI, and Generative Adversarial Networks

snippet: This research focuses on the detection of malware in sample datasets using machine learning algorithms. As malware becomes more advanced against detection in the cybersecurity sector, researchers have looked for other methods to …


title: Exploring the competence of ChatGPT for customer and patient service management

authors: A Haleem, M Javaid, RP Singh - Intelligent Pharmacy, 2024

snippet: The modern language generation model ChatGPT, created by Open Artificial Intelligence (AI), is recognised for its capacity to comprehend context and produce pertinent content. This model is built on the transformer architecture, which enables …


title: Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models

authors: AMK Mamaghan, S Papa, KH Johansson, S Bauer… - arXiv preprint arXiv …, 2024

snippet: Object-centric (OC) representations, which represent the state of a visual scene by modeling it as a composition of objects, have the potential to be used in various downstream tasks to achieve systematic compositional generalization and facilitate …


title: Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning

authors: Y Wang, W Chen, X Han, X Lin, H Zhao, Y Liu, B Zhai… - arXiv preprint arXiv …, 2024

snippet: Strong Artificial Intelligence (Strong AI) or Artificial General Intelligence (AGI) with abstract reasoning ability is the goal of next-generation AI. Recent advancements in Large Language Models (LLMs), along with the emerging field of Multimodal Large …


title: Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties

authors: E Artemova, V Blaschke, B Plank - arXiv preprint arXiv:2402.02078, 2024

snippet: Mainstream cross-lingual task-oriented dialogue (ToD) systems leverage the transfer learning paradigm by training a joint model for intent recognition and slot-filling in English and applying it, zero-shot, to other languages. We address a gap in prior …


title: Exploring the Role of ChatGPT in Cardiology: A Systematic Review of the Current Literature

authors: A Sharma, T Medapalli, M Alexandrou, E Brilakis… - Cureus, 2024

snippet: Chat Generative Pre-Trained Transformer (ChatGPT) is a chatbot based on a large language model that has gained public interest since its release in November 2022. This systematic review examines the current literature on the potential applications …


title: Exploring the role of ChatGPT in higher education: opportunities, challenges and ethical considerations

authors: A Zeb, R Ullah, R Karim - The International Journal of Information and Learning …, 2024

snippet: Purpose This paper aims to examine the opportunities and challenges of using ChatGPT in higher education. Furthermore, it is also discuss the potential risks and plunders of these tools. Design/methodology/approach The paper discuss the use of …


title: Exploring transformer models in the sentiment analysis task for the under-resource Bengali language

authors: MN Hoque, U Salma, MJ Uddin, MM Ahamad, S Aktar - Natural Language Processing …, 2024

snippet: In the sentiment analysis (SA) task, we can obtain a positive or negative-typed comment or feedback from an online user or a customer about any object, such as a movie, drama, food, and others. This user’s sentiment may positively impact various …


title: Exploring very low-resource translation with llms: The university of edinburgh's submission to americasnlp 2024 translation task

authors: V Iyer, B Malik, W Zhu, P Stepachev, P Chen… - Proceedings of the 4th …, 2024

snippet: This paper describes the University of Edinburgh’s submission to the AmericasNLP 2024 shared task on the translation of Spanish into 11 indigenous American languages. We explore the ability of multilingual Large Language Models (LLMs) to …


title: EXPRESS: AI-Human Hybrids for Marketing Research: Leveraging LLMs as Collaborators

authors: N Arora, I Chakraborty, Y Nishimura - Journal of Marketing, 2024

snippet: The authors’ central premise is that a human-LLM hybrid approach leads to efficiency and effectiveness gains in the marketing research process. In qualitative research, they show that LLMs can assist in both data generation and analysis; …


title: EXPRESS: Malay Lexicon Project 3: The Impact of Orthographic-Semantic Consistency on Lexical Decision Latencies

authors: M Maziyah Mohamed, D Jared - Quarterly Journal of Experimental Psychology, 2024

snippet: Theories of word processing propose that readers are sensitive to statistical co-occurrences between spelling and meaning. Orthographic-Semantic Consistency (OSC) measures provide a continuous estimate of the statistical regularities between …


title: Extending the Comparative Argumentative Machine: Multilingualism and Stance Detection

authors: I Nikishina, A Bondarenko, S Zaczek, OL Haag…

snippet: The comparative argumentative machine CAM can retrieve arguments that answer comparative questions—questions that ask which of several to-be-compared options should be favored in some scenario. In this paper, we describe how we equipped …


title: Extending Translate-Train for ColBERT-X to African Language CLIR

authors: E Yang, DJ Lawrie, P McNamee, J Mayfield - arXiv preprint arXiv:2404.08134, 2024

snippet: … We used Common Crawl documents in Afriberta Corpus [6] for the four African languages and collected additional English Common Crawl documents to match the genre. We fine-tune the model for 200,000 update steps using a learning rate of 1×10-5 …


title: External Attack-Surface of Modern Organizations

authors: N Gelernter, H Schulmann, M Waidner - Proceedings of the 19th ACM Asia …, 2024

snippet: Navigating the maze of contemporary organizational attack surfaces is paramount in fortifying our defenses against the relentless tide of cyber incidents. However, existing network reconnaissance and security measurements, which enumerate IP …


title: Extracting emotion from resource poor language through transfer learning

authors: A Majeed, U Imtiaz, MA Nseem, M Aleem, W Shahzad… - Multimedia Tools and …, 2024

snippet: … Common Crawl materials in over 100 languages are used to train the algorithm. XLM-Roberta has seen a significant increase in training data since its first release. Several cross-language benchmarks show that XLM-R performs the best. …


title: Extracting intersectional stereotypes from embeddings: Developing and validating the Flexible Intersectional Stereotype Extraction procedure

authors: TES Charlesworth, K Ghate, A Caliskan, MR Banaji - PNAS Nexus, 2024

snippet: … text corpora, ranging from static GloVe embeddings trained on 840 billion words from the Common Crawl to contextualized BERT embeddings trained on a combination of Wikipedia and Common Crawl text. Additionally, in supplemental …


title: Factuality of Large Language Models in the Year 2024

authors: Y Wang, M Wang, MA Manzoor, G Georgiev, RJ Das… - arXiv preprint arXiv …, 2024

snippet: Large language models (LLMs), especially when instruction-tuned for chat, have become part of our daily lives, freeing people from the process of searching, extracting, and integrating information from multiple sources by offering a …


title: FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication

authors: E Slyman, S Lee, S Cohen, K Kafle - arXiv preprint arXiv:2404.16123, 2024

snippet: Recent dataset deduplication techniques have demonstrated that content-aware dataset pruning can dramatically reduce the cost of training Vision-Language Pretrained (VLP) models without significant performance losses compared to …


title: Fairness and Bias in Multimodal AI: A Survey

authors: T Adewumi, L Alkhaled, N Gurung, G van Boven… - arXiv preprint arXiv …, 2024

snippet: The importance of addressing fairness and bias in artificial intelligence (AI) systems cannot be over-emphasized. Mainstream media has been awashed with news of incidents around stereotypes and bias in many of these systems in recent years. In …


title: Fake news detection in low-resource languages: A novel hybrid summarization approach

authors: J Alghamdi, Y Lin, S Luo - Knowledge-Based Systems, 2024

snippet: … Pre-training encompasses a vast dataset exceeding 2TB of CommonCrawl data. The overarching concept involves mapping any given language to a language-agnostic vector space, ensuring that inputs in various languages converge to corresponding …


title: Falcon2-11B Technical Report

authors: Q Malartic, NR Chowdhury, R Cojocaru, M Farooq… - arXiv preprint arXiv …, 2024

snippet: We introduce Falcon2-11B, a foundation model trained on over five trillion tokens, and its multimodal counterpart, Falcon2-11B-vlm, which is a vision-to-text model. We report our findings during the training of the Falcon2-11B which follows a multi-stage …


title: Feasibility of Big Data Analytics in Disaster Psychiatry: Impact of Seoul Itaewon Tragedy on Sentiment Distribution on Twitter

authors: J Kim, M Ki, J Yang, C Na, J Kim, C Han - Social Science & Medicine, 2024

snippet: Numerous studies have highlighted the significant impact of disasters on mental health, often leading to psychiatric disorders among affected individuals. Timely identification of disaster-related mental health problems is crucial to prevent long-term …


title: Feature-augmented model for multilingual discourse relation classification

authors: E Metheniti, C Braud, P Muller - Proceedings of the 5th Workshop on Computational …, 2024

snippet: Discourse relation classification within a multilingual, cross-framework setting is a challenging task, and the best-performing systems so far have relied on monolingual and mono-framework approaches. In this paper, we introduce transformer-based …


title: Federated learning for privacy-preserving depression detection with multilingual language models in social media posts

authors: SS Khalil, NS Tawfik, M Spruit - Patterns, 2024

snippet: The incidences of mental health illnesses, such as suicidal ideation and depression, are increasing, which highlights the urgent need for early detection methods. There is a growing interest in using natural language processing (NLP) models to analyze …


title: Few shot clinical entity recognition in three languages: Masked language models outperform LLM prompting

authors: M Naguib, X Tannier, A Névéol - arXiv preprint arXiv:2402.12801, 2024

snippet: Large Language Models are becoming the go-to solution for many natural language processing tasks, including in specialized domains where their few-shot capacities are expected to yield high performance in low-resource settings. Herein, we aim to …


title: Final Thoughts: Digital Humanities Looking at Generative AI

authors: M Aguiar, S Araújo - Digital Humanities Looking at the World: Exploring …, 2024

snippet: In this chapter, we examine generative artificial intelligence in the context of Digital Humanities. We commence by providing a concise overview of this technology and its prevalent models. Following that, we provide a brief survey of the existing …


title: Final Words

authors: M McTear, M Ashurkina - Transforming Conversational AI: Exploring the Power …, 2024

snippet: Conversational AI is a dynamic and fast moving field, and a lot has happened in the six months or so since we began writing this book. We have tried to ensure that what we have covered in the preceding chapters provides a sufficiently general and …


title: Fine grain emotion analysis in Spanish using linguistic features and transformers

authors: A Salmerón-Ríos, JA García-Díaz, R Pan… - PeerJ Computer Science, 2024

snippet: Mental health issues are a global concern, with a particular focus on the rise of depression. Depression affects millions of people worldwide and is a leading cause of suicide, particularly among young people. Recent surveys indicate an increase in …


title: Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

authors: K VM, H Warrier, Y Gupta - arXiv preprint arXiv:2404.10779, 2024

snippet: … The datasets are massive text corpus like Common crawl, The Pile and code repositories from Github. Further the model can be fine tuned for specific tasks using specialised datasets. One such series of datasets are called Instruction datasets like …


title: Fine Tuning LLMs for Low Resource Languages

authors: S Joshi, MS Khan, A Dafe, K Singh, V Zope, T Jhamtani - 2024 5th International …, 2024

snippet: Large Language Models (LLMs) hold immense potential, but their data hunger can limit its performance in processing languages with limited resources. This research study explores the techniques for fine-tuning LLMs specifically for low-resource …


title: Fine-Tuned Large Language Models for Symptom Recognition from Spanish Clinical Text

authors: MA Shaaban, A Akkasi, A Khan, M Komeili, M Yaqub - arXiv preprint arXiv …, 2024

snippet: … XLM-RL (561M parameters) and XLM-RB (279M parameters) are multilingual masked language models, trained on a filtered CommonCrawl dataset of over two terabytes. We leverage BBS and BBES both pretrained on Spanish clinical data …


title: Fine-Tuned Understanding: Enhancing Social Bot Detection with Transformer-based Classification

authors: A Sallah, S Agoujil, MA Wani, M Hammad… - IEEE Access, 2024

snippet: … fined common crawl data, XLM-RoBERTa is purposefully crafted as a multilingual version of RoBERTa. The acronym "XLM" denotes its capabilities in crosslingual language modeling; 3… It is trained on 2.5 TB of freshly curated, clean …


title: FINE-TUNING BERT, DISTILBERT, XLM-ROBERTA AND UKR-ROBERTA MODELS FOR SENTIMENT ANALYSIS OF UKRAINIAN LANGUAGE REVIEWS

snippet: … It is pre-trained on 2.5TB of filtered CommonCrawl data containing 100 languages. The architecture of the XLMRoBERTa base model consists of 12 layers, 12 attention heads with 768 dimensions, and a feed-forward network [19]. This model is trained …


title: Fine-tuning BERT-based Models for Negative Content Identification on Indonesian Tweets

authors: AF Hidayatullah, K Kalinaki, MM Aslam, RY Zakari…

snippet: Social media platforms like Twitter have become substantial sources of user-generated content, enabling people to easily express their emotions and opinions. However, this freedom has increased the spread of harmful content, such as abusive language …


title: Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection

authors: F Xiong, T Markchom, Z Zheng, S Jung, V Ojha… - arXiv preprint arXiv …, 2024

snippet: SemEval-2024 Task 8 introduces the challenge of identifying machine-generated texts from diverse Large Language Models (LLMs) in various languages and domains. The task comprises three subtasks: binary classification in monolingual …


title: Fine-tuning large language models: from accuracy enhancement to bias mitigation

snippet: As the frontier of artificial intelligence (AI) continues to expand, Large Language Models (LLMs) have emerged as pivotal tools across a broad spectrum of applications. These sophisticated AI systems, powered by vast amounts of data and …


title: FINETUNING COMMERCIAL LARGE LANGUAGE MODELS WITH LORA FOR ENHANCED ITALIAN LANGUAGE UNDERSTANDING

authors: J Hartsuiker, P Torroni, AE Ziri, DF Alise, F Ruggeri

snippet: The world of Artificial intelligence is booming, especially the fields of natural language processing (NLP) and Computer Vision (CV). Recently the chatGPT models from OpenAI have caused another shock-wave. This was a clear sign to the …


title: Finite State Automata on Multi-Word Units for Efficient Text-Mining

snippet: Text mining is crucial for analyzing unstructured and semi-structured textual documents. This paper introduces a fast and precise text mining method based on a finite automaton to extract knowledge domains. Unlike simple words, multi-word …


title: FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

authors: G Bhatia, EMB Nagoudi, H Cavusoglu… - arXiv preprint arXiv …, 2024

snippet: We introduce FinTral, a suite of state-of-the-art multimodal large language models (LLMs) built upon the Mistral-7b model and tailored for financial analysis. FinTral integrates textual, numerical, tabular, and image data. We enhance FinTral with domain-specific …


title: Foregrounding Artist Opinions: A Survey Study on Transparency, Ownership, and Fairness in AI Generative Art

authors: J Lovato, J Zimmerman, I Smith, P Dodds, J Karson - arXiv preprint arXiv:2401.15497, 2024

snippet: Generative Artificial Intelligence (AI) tools are used to create art-like outputs and aid in the creative process. While these tools have potential benefits for artists, they also have the potential to harm the art workforce and infringe upon artistic and intellectual …


title: Fostering the Ecosystem of Open Neural Encoders for Portuguese with Albertina PT* Family

authors: R Santos, J Rodrigues, L Gomes, J Silva, A Branco… - arXiv preprint arXiv …, 2024

snippet: … The OSCAR subset for Portuguese we use is based on November/December 2022 version of Common Crawl, which is an automatic crawl from the web. Despite being a crawl, the final dataset is of relatively good quality due the filtering performed …


title: Foundation Models for Autonomous Robots in Unstructured Environments

authors: H Naderi, A Shojaei - arXiv preprint arXiv:2407.14296, 2024

snippet: Automating activities through robots in unstructured environments, such as construction sites, has been a long-standing desire. However, the high degree of unpredictable events in these settings has resulted in far less adoption compared to …


title: Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship

authors: M Baturu, M Solakhan, TG Kazaz, O Bayrak - International Journal of Impotence …, 2024

snippet: The present study assessed the accuracy of artificiaI intelligence-generated responses to frequently asked questions on erectile dysfunction. A cross-sectional analysis involved 56 erectile dysfunction-related questions searched on Google …


title: From Bytes to Biases: Investigating the Cultural Self-Perception of Large Language Models

authors: W Messner, T Greene, J Matalone - arXiv preprint arXiv:2312.17256, 2023

snippet: Large language models (LLMs) are able to engage in natural-sounding conversations with humans, showcasing unprecedented capabilities for information retrieval and automated decision support. They have disrupted human-technology …


title: From Cosine Similarity to Likelihood Ratio: Coupling Representations From Machine Learning (and Other Sources) With Cognitive Models

snippet: Modern machine learning models yield vector representations that capture similarity relations between complex items like text and images. These representations can help explain and predict how individuals respond to those items in particular tasks …


title: From Data Creator to Data Reuser: Distance Matters

authors: CL Borgman, PT Groth - arXiv preprint arXiv:2402.07926, 2024

snippet: … Common Crawl, launched in 2007, now spends roughly $200,000 per year to maintain petabytes of openly available web crawl data (Common Crawl, 2023; Roberts, 2023). The use of these data to train large language models has …


title: From Data Quality to Model Performance: Navigating the Landscape of Deep Learning Model Evaluation

authors: M Akram, WH Moosa - Deep Learning for Multimedia Processing Applications, 2024

snippet: Deep learning (DL) technologies have revolutionized the analysis of multimedia data, with natural language processing, visual data analytics, and audio recognition being just a few examples of multimedia applications that have effectively utilized DL. This …


title: From Development to Dissemination: Social and Ethical Issues with Text-to-Image AI-Generated Art

snippet: … Model developed by the Computer Vision & Learning Group (CompVis) lab at Ludwig Maximilian University of Munich was trained with a subset of the LAION-5B dataset [6], comprising of about 2.3 billion CLIP-filtered image-text pairs by parsing …


title: From Form (s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

authors: X Ohmer, E Bruni, D Hupkes - arXiv preprint arXiv:2404.12145, 2024

snippet: … We use the current common crawl statistics6 to compute an estimate of how low- or high-resource these languages are in web-based corpora. Of this corpus, English constitutes 46% of the data, German 5.8%, Italian 2.7%, Dutch 2.2%, and Swedish …


title: From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients

authors: A Jaiswal, L Yin, Z Zhang, S Liu, J Zhao, Y Tian… - arXiv preprint arXiv …, 2024

snippet: … The C4 dataset is a massive collection of Common Crawl’s web crawl corpus, meticulously filtered and cleaned to ensure high-quality language modeling and training. For downstream task finetuning of compressed models, we consider a good …


title: From Graph Theory for Robust Deep Networks to Graph Learning for Multimodal Cancer Analysis

snippet: This dissertation explores the intersection of graph theory and deep learning, focusing on enhancing the robustness of deep neural networks (DNNs) and applying these advancements to complex problems like cancer diagnosis and …


title: From ML to LLM: Evaluating the Robustness of Phishing Webpage Detection Models against Adversarial Attacks

authors: A Kulkarni, V Balachandran, DM Divakaran, T Das - arXiv preprint arXiv:2407.20361, 2024

snippet: Phishing attacks attempt to deceive users into stealing sensitive information, posing a significant cybersecurity threat. Advances in machine learning (ML) and deep learning (DL) have led to the development of numerous phishing webpage detection …


title: From News to Summaries: Building a Hungarian Corpus for Extractive and Abstractive Summarization

authors: B Barta, D Lakatos, A Nagy, MK Nyist, J Ács - arXiv preprint arXiv:2404.03555, 2024

snippet: … We use the freely available Common Crawl dataset4 as a basis for constructing the corpus. It contains petabytes of crawled web pages from the past 25 years and it is available on Amazon S3 in WARC format. Retrieval and deduplication of the raw …


title: From Pixels to Prose: A Large Dataset of Dense Image Captions

authors: V Singla, K Yue, S Paul, R Shirkavand… - arXiv preprint arXiv …, 2024

snippet: Training large vision-language models requires extensive, high-quality image-text pairs. Existing web-scraped datasets, however, are noisy and lack detailed image descriptions. To bridge this gap, we introduce PixelProse, a comprehensive dataset …


title: From Populism to Platforms: Antitrust Law and the AI Revolution

snippet: This chapter, from my book manuscript “The Political Economy of the Fourth Industrial Revolution,” examines how changes to US antitrust law catalyzed the rise of digital platforms and the Artificial Intelligence (AI) Revolution. It traces the …


title: From Rule-Based Models to Deep Learning Transformers Architectures for Natural Language Processing and Sign Language Translation Systems: Survey, Taxonomy …

authors: N Shahin, L Ismail - arXiv preprint arXiv:2408.14825, 2024

snippet: With the growing Deaf and Hard of Hearing population worldwide and the persistent shortage of certified sign language interpreters, there is a pressing need for an efficient, signs-driven, integrated end-to-end translation system, from sign to gloss to …


title: From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility

authors: P Kaur, GS Kashyap, A Kumar, MT Nafis, S Kumar… - arXiv preprint arXiv …, 2024

snippet: This groundbreaking study explores the expanse of Large Language Models (LLMs), such as Generative Pre-Trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) across varied domains ranging from …


title: From'Showgirls' to'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs

authors: M Bartl, S Leavy - arXiv preprint arXiv:2407.04434, 2024

snippet: Gender bias is not only prevalent in Large Language Models (LLMs) and their training data, but also firmly ingrained into the structural aspects of language itself. Therefore, adapting linguistic structures within LLM training data to promote gender-inclusivity …


title: Frozen or Fine-tuned? Analyzing Deep Learning Models and Training Strategies for Optimizing Big Five Personality Traits Prediction from Text

authors: M Soleimani, HB Kashani - 2024 10th International Conference on Artificial …, 2024

snippet: In today’s digital age, the comprehension and prediction of human personality traits have assumed paramount significance. This study embarks on the task of forecasting the Big Five personality traits through textual data, harnessing the …


title: FuLG: 150B Romanian Corpus for Language Model Pretraining

authors: VA Bădoiu, MV Dumitru, AM Gherghescu, A Agache… - arXiv preprint arXiv …, 2024

snippet: … billion-token Romanian corpus extracted from CommonCrawl. We present our methodology for … , we delve into filtering Romanian content from CommonCrawl, documenting the process and … Second, we document the process of filtering data …


title: Fundus: A Simple-to-Use News Scraper Optimized for High Quality Extractions

authors: M Dallabetta, C Dobberstein, A Breiding, A Akbik - arXiv preprint arXiv:2403.15279, 2024

snippet: … Specifically, we support the CC-NEWS3 dataset provided by the COMMONCRAWL initiative. At the time of writing, this dataset comprises around 40 terabytes of WARC-formatted data, containing millions of news articles dating back …


title: Fusing Domain-Specific Content from Large Language Models into Knowledge Graphs for Enhanced Zero Shot Object State Classification

authors: F Gouidis, K Papantoniou, KPT Patkos, A Argyros… - arXiv preprint arXiv …, 2024

snippet: Domain-specific knowledge can significantly contribute to addressing a wide variety of vision tasks. However, the generation of such knowledge entails considerable human labor and time costs. This study investigates the potential of Large Language …


title: Fusion of Deep Learning with Advanced and Traditional Embeddings in Sentiment Analysis

authors: K Kiran, PD Shenoy - 2024 IEEE 9th International Conference for …, 2024

snippet: In todays changing landscape the abundance of unprocessed data plays a vital role. It is crucial to curate this dataset and conduct a thorough analysis of the sentiments it captures. The aim of this research study is to explore learning techniques such as Bi …


title: Fusion of Knowledge: Enhancing AI Reasoning Through Language Models and Knowledge Graphs

snippet: Large Language Models (LLMs) and Knowledge Graphs (KGs) have rapidly emerged as important areas in Artificial Intelligence (AI). LLMs leverage vast amounts of unstructured text to understand and generate natural language. KGs are …


title: FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data

authors: H Sun, R Jin, S Xu, L Pan, M Cui, J Dui, Y Lei, L Yang… - arXiv preprint arXiv …, 2024

snippet: Large language models (LLMs) have demonstrated prowess in a wide range of tasks. However, many LLMs exhibit significant performance discrepancies between highand low-resource languages. To mitigate this challenge, we present FuxiTranyu, an …


title: GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

authors: J Zhao, Z Zhang, B Chen, Z Wang, A Anandkumar… - arXiv preprint arXiv …, 2024

snippet: Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank adaptation (LoRA), add a trainable …


title: Gauging Novelty in Crowdfunding Projects: A Theory-Driven Text Analysis Approach

snippet: The explosion of crowdfunding projects has called for new ways to screen for novel projects. Building on the multidimensional nature of novelty, we draw on the literature to propose two measures of project novelty that are theoretically sound and …


title: GEB-1.3 B: Open Lightweight Large Language Model

authors: J Wu, Y Zhu, L Shen, X Lu - arXiv preprint arXiv:2406.09900, 2024

snippet: … Data Processing The Common Crawl dataset is crawled from the web and often suffers from quality issues, because of including garbled … Rule Cleaning First, we extract a Chinese corpus from the Common Crawl dataset, by applying specialized …


title: GenDec: A robust generative Question-decomposition method for Multi-hop reasoning

authors: J Wu, L Yang, Y Ji, W Huang, BF Karlsson, M Okumura - arXiv preprint arXiv …, 2024

snippet: Multi-hop QA (MHQA) involves step-by-step reasoning to answer complex questions and find multiple relevant supporting facts. However, Existing large language models'(LLMs) reasoning ability in multi-hop question answering remains exploration, which is …


title: Gender Bias in Natural Gender Language and Grammatical Gender Language within Children's Literature

snippet: There has been much research on the connection between language and gender bias but there is little comparing natural gender language, grammatical gender language, and gender bias. This research is important because it can offer an …


title: Gender Identity and Representation in the Context of Economic Development in India

snippet: … It is a large multi-lingual language model, trained on 2.5TB of filtered CommonCrawl data. It represents some of the most sophisticated pre-trained models prevalently being used in natural language processing for text classification …


title: Gendered Grammar or Ingrained Bias? Exploring Gender Bias in Icelandic Language Models

authors: SR Friðriksdóttir, H Einarsson - Proceedings of the 2024 Joint International …, 2024

snippet: Large language models, trained on vast datasets, exhibit increased output quality in proportion to the amount of data that is used to train them. This data-driven learning process has brought forth a pressing issue where these models may not only reflect …


title: Gendered Language in Job Advertisements Relates to Gender Sorting in Public Labor Markets: A Multi-Source Analysis

snippet: Increasing gender diversity constitutes a desirable goal for policymakers and recruiters in public organizations. However, contemporary research lacks focus on gender sorting, referring to structural self-selection among job seekers in the public …


title: General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

authors: H Wei, C Liu, J Chen, J Wang, L Kong, Y Xu, Z Ge… - arXiv preprint arXiv …, 2024

snippet: … For the document-level data, we first collect open-source PDF-style files from the Common Crawl and employ the Fitz Python package to extract corresponding dense text content. In such a process, we gain 1.2M full-page PDF-style image-text pairs …


title: Generating Apparel Images by Using Stable Diffusion with Prompt Recommendation

authors: XZ Liu, HM Hung, LH Chen - 人工知能学会全国大会論文集 第 38 回 (2024), 2024

snippet: Fashion is a profound expression of personal identity, style, and culture in daily life. Making oneself look attractive, which often equates to being fashionable or stylish, is always a priority for many people. In this paper, we proposed a clothing …


title: Generating Character Lines in Four-Panel Manga

authors: M Inaba - Proceedings of the 37th Pacific Asia Conference on …, 2023

snippet: Automatic content generation based on natural language processing is an active research area, especially for story generation. Research on story generation has focused on generating consistent text pertaining to characters’ actions and events; …


title: Generating Probabilistic Scenario Programs from Natural Language

authors: K Elmaaroufi, D Shankar, A Cismaru… - arXiv preprint arXiv …, 2024

snippet: … Most of these model’s are fine-tuned variants of a foundational model that was trained on a wider body of text such as Common Crawl. For instance Codex is ”a GPT language model fine-tuned on publicly available code from GitHub” and it’s …


title: Generation of API Documentation using Large Language Models–Towards Self-explaining APIs

snippet: Application Programming Interfaces (APIs) play a crucial role in modern software development. They are key in not only in enabling software reuse and adaptation, which greatly increase productivity, but also in facilitating exploration of new …


title: Generative adversarial network-based phishing URL detection with variational autoencoder and transformer

authors: JK Sasi, A Balakrishnan - Int J Artif Intell, 2024

snippet: Phishing attacks pose a constant threat to online security, necessitating the development of efficient tools for identifying malicious URLs. In this article, we propose a novel approach to detect phishing URLs employing a generative …


title: Generative AI and Large Language Models for Cyber Security: All Insights You Need

authors: MA Ferrag, F Alwahedi, A Battah, B Cherif, A Mechri… - arXiv preprint arXiv …, 2024

snippet: This paper provides a comprehensive review of the future of cybersecurity through Generative AI and Large Language Models (LLMs). We explore LLM applications across various domains, including hardware design security, intrusion detection …


title: Generative AI and the Continuing Importance of Information Literacy

snippet: GAI models are embedded in a variety of applications. These models are trained using existing information, which is then formatted and delivered within the context of the GAI application. Bias, misand disinformation, and ethical use of information …


title: Generative AI for Math: Part I--MathPile: A Billion-Token-Scale Pretraining Corpus for Math

authors: Z Wang, R Xia, P Liu - arXiv preprint arXiv:2312.17120, 2023

snippet: … expanse of Common Crawl snapshots, a venture we aim to pursue in our future work. … We provide a near-duplicate example found in Common Crawl, as shown in 2 (See Table 8 to … StackExchange that appeared in a document from Common …


title: Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunities

authors: G Liu, N Van Huynh, H Du, DT Hoang, D Niyato, K Zhu… - arXiv preprint arXiv …, 2024

snippet: With recent advances in artificial intelligence (AI) and robotics, unmanned vehicle swarms have received great attention from both academia and industry due to their potential to provide services that are difficult and dangerous to perform by humans …


title: Generative AI from Theory to Practice: A Case Study of Financial Advice

snippet: We identify some of the most pressing issues facing the adoption of large language models (LLMs) in practical settings and propose a research agenda to reach the next technological inflection point in generative AI. We focus on three challenges …


title: Generative AI in Education From the Perspective of Students, Educators, and Administrators

snippet: This research explores how advanced artificial intelligence (AI), like the technology that powers tools such as ChatGPT, is changing the way we teach and learn in schools and universities. Imagine AI helping to summarize thick legal documents …


title: Generative AI

snippet: … • CommonCrawl News: This is a dataset of news stories from websites across the globe. …


title: Generative AI, Plagiarism, and Copyright Infringement in Legal Documents

authors: AB Cyphert - Minnesota Journal of Law, Science & Technology, 2024

snippet: … OpenAI acknowledged in the GPT-3 research paper that GPT-3 was trained on the Common Crawl dataset, “which includes everything from traditional news sites like the New York Times to sites like Reddit.” Cyphert, supra note 1, at 407, citing Liz …


title: Generative AI-Augmented Decision-Making for Business Information Systems

authors: E Kromidha, RM Davison - IFIP International Conference on Human Choice and …, 2024

snippet: … This is only influenced by the data it has been trained on, and its users which is one of its main limitations as ChatGPT was trained on datasets like Wikipedia, BookCorpus for books, and Common Crawl for web content which are mainly in …


title: Generative AI-Based Text Generation Methods Using Pre-Trained GPT-2 Model

authors: R Pandey, H Waghela, S Rakshit, A Rangari, A Singh… - arXiv preprint arXiv …, 2024

snippet: This work delved into the realm of automatic text generation, exploring a variety of techniques ranging from traditional deterministic approaches to more modern stochastic methods. Through analysis of greedy search, beam search, top-k …


title: GENERATIVE AI-POWERED FRAMEWORK

authors: K VAYADANDE - Deep Learning Model Optimization, Deployment and …, 2024

snippet: Artificial intelligence (AI) tools encompass a diverse array of applications and software designed to simulate intelligent behavior, facilitate decision making, and automate tasks across various domains. These tools leverage artificial intelligence …


title: Generative Artificial Intelligence (AI) for the Data Economy: Use Cases in Nursing

snippet: … The core recent AIGC advances stem from training more sophisticated generative models on larger datasets; for example, while the main framework maintains the same as GPT-2, GPT-3’s pre-training data size grows from WebTex (38GB) to …


title: GENERATIVE ARTIFICIAL INTELLIGENCE (AI) IN EDUCATION: A CROSS-NATIONAL SURVEY ON UNIVERSITY TEACHERS'PERCEPTIONS ON THE USE OF …

snippet: … A year and three months later, the GPT-3 emerged with the Common Crawl as the main source of its data. The current GPT-4 started in March 2023 with the “unpublished” data source (Wu et al., 2023, p. 1123). …


title: Generative Artificial Intelligence-Guided User Studies: An Application for Air Taxi Services

authors: S Xiao, J Li, T Fushimi, Y Ochiai - arXiv preprint arXiv:2406.12296, 2024

snippet: User studies are crucial for meeting user needs. In user studies, real experimental scenarios and participants are constructed and recruited. However, emerging and unfamiliar studies face limitations, including safety concerns and iterative efficiency …


title: Generative Artificial Intelligence: Models, Benefits, Dangers and Detection of AI-Generated Text on Specialized Domains

snippet: … However, Radford et al. found that a large amount of document within the hashed pages of Common Crawl was unintelligible. A new approach could be to simply use a curated subset of Common Crawl, but they also created a web scrape from Reddit …


title: Generative Knowledge Management for Financial Inclusion Through Financial Literacy: A Systematic Review.

authors: S Vadari, C Malladi - IUP Journal of Knowledge Management, 2024

snippet: … OpenAI utilized an openly available corpus of webpages called Common Crawl, which in turn derives from a bot that crawls the entire Internet. This Common Crawl dataset contains billions of webpages and is one of the colossal textual datasets …


title: GeniL: A Multilingual Dataset on Generalizing Language

authors: AM Davani, S Gubbi, S Dev, S Dave, V Prabhakaran - arXiv preprint arXiv …, 2024

snippet: … For each language, we queried the Multilingual Common Crawl (mC4) language corpus to collect naturally occurring sentences that contain both the terms in the tuples present in SGM. To ensure a diverse representation of different associations …


title: German Text Embedding Clustering Benchmark

authors: S Wehrli, B Arnrich, C Irrgang - arXiv preprint arXiv:2401.02709, 2024

snippet: This work introduces a benchmark assessing the performance of clustering German text embeddings in different domains. This benchmark is driven by the increasing use of clustering neural text embeddings in tasks that require the grouping of texts (such …


title: GFWeb: Measuring the Great Firewall's Web Censorship at Scale

authors: NP Hoang, J Dalek, M Crete-Nishihata, N Christin…

snippet: … Aggregating the ranking information provided by the Tranco list [62] and the Common Crawl dataset [1], we analyze the popularity of the base censored domains discovered by GFWeb for HTTP(S) filters in comparison to the GFWatch (DNS filter) …


title: Ghost Sentence: A Tool for Everyday Users to Copyright Data from Large Language Models

authors: S Zhao, L Zhu, R Quan, Y Yang - arXiv preprint arXiv:2403.15740, 2024

snippet: Web user data plays a central role in the ecosystem of pre-trained large language models (LLMs) and their fine-tuned variants. Billions of data are crawled from the web and fed to LLMs. How can \textit{\textbf{everyday web users}} confirm if LLMs …


title: Goldfish: Monolingual Language Models for 350 Languages

authors: TA Chang, C Arnett, Z Tu, BK Bergen - arXiv preprint arXiv:2408.10441, 2024

snippet: For many low-resource languages, the only available language models are large multilingual models trained on many languages simultaneously. However, using FLORES perplexity as a metric, we find that these models perform worse than …


title: GPT‐3‐and DALL‐E‐Powered Applications: A Complete Survey

authors: K Vayadande, CB Pednekar, PA Khune… - How Machine Learning is …, 2024

snippet: In the new millennium, the advancement of language and image processing technologies has witnessed remarkable progress, and various applications have demonstrated promising results by utilizing sophisticated models such as GPT‐3 …


title: GPTs: Concerns, Limitations and (Some) Responses

snippet: … The ones of these that are large and containing substantial source material from the Internet (eg from Common Crawl) will have some harms of representation bias. Then, if the training data has bias then so too will LLMs. To address this, there seem …


title: Grammatical versus Spelling Error Correction: An Investigation into the Responsiveness of Transformer-Based Language Models Using BART and MarianMT

authors: R Raju, PB Pati, SA Gandheesh, GS Sannala… - Journal of Information & …, 2024

snippet: Text continues to remain a relevant form of representation for information. Text documents are created either in digital native platforms or through the conversion of other media files such as images and speech. While the digital native text is …


title: Grammatical vs Spelling Error Correction: An Investigation into the Responsiveness of Transformer-based Language Models using BART and MarianMT

authors: R Raju, PB Pati, SA Gandheesh, GS Sannala, S KS - arXiv preprint arXiv:2403.16655, 2024

snippet: … C4 Dataset is an Open-Source dataset obtained with Common Crawl web scrape [30]. This dataset contains millions of collected sentences along with their target sentences. The collected statements, referred to as input sentences in this work …


title: Granite Code Models: A Family of Open Foundation Models for Code Intelligence

authors: M Mishra, M Stallone, G Zhang, Y Shen, A Prasad… - arXiv preprint arXiv …, 2024

snippet: Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and …


title: Granularity is crucial when applying differential privacy to text: An investigation for neural machine translation

authors: DNL Vu, T Igamberdiev, I Habernal - arXiv preprint arXiv:2407.18789, 2024

snippet: … The computation of b is carried out using Common Crawl11 monolingual datasets. For each language and contextual embedding model, one million candidate-reference pairs are created by grouping two random sentences. Due to the random pairing …


title: Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions

authors: YG Hsieh, CY Hsieh, SY Yeh, L Béthune, HP Ansari… - arXiv preprint arXiv …, 2024

snippet: … In a similar vein, Meta-CLIP [68] reproduces the processing of the seminal CLIP paper [50] on a subset of the Common Crawl dataset, … to filter data and remove duplicates, while DFN [17] uses filtering networks trained on high quality data to …


title: Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients

authors: A Muhamed, O Li, D Woodruff, M Diab, V Smith - arXiv preprint arXiv:2406.17660, 2024

snippet: Large language model (LLM) training and finetuning are often bottlenecked by limited GPU memory. While existing projection-based optimization methods address this by projecting gradients into a lower-dimensional subspace to reduce optimizer …


title: GreekT5: Sequence-to-Sequence Models for Greek News Summarization

authors: N Giarelis, C Mastrokostas, N Karacapilidis - IFIP International Conference on …, 2024

snippet: … mBART was pre-trained for multiple languages using the CC25 dataset [15], which contains documents from 25 different languages from Common Crawl. The authors have also published an updated mBART model, which supports 50 natural …


title: Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation

authors: S Xia, X Wang, J Liang, Y Zhang, W Zhou, J Deng, F Yu… - arXiv preprint arXiv …, 2024

snippet: Retrieval-Augmented Generation (RAG) has been widely adopted to enhance Large Language Models (LLMs) in knowledge-intensive tasks. Recently, Attributed Text Generation (ATG) has attracted growing attention, which provides citations to …


title: Grounded Content Automation: Generation and Verification of Wikipedia in Low-Resouce languages.

snippet: In this thesis, we work towards improving the representation of low-resource languages in the digital world by easing the access and participation of these communities to reliable information hubs like Wikipedia. Although the internet has …


title: Grounding Language in Images and Videos

snippet: While machine learning research has traditionally explored image, video and text understanding as separate fields, the surge in multi-modal content in today’s digital landscape underscores the importance of computation models that adeptly navigate …


title: Growing Smaller Language Models Using Knowledge Distillation from Larger Models

authors: M Featherstone, E Cuthbertson, D Appleyard, S Gittins

snippet: … Publicly available datasets, such as The Pile and Common Crawl, were employed to ensure a comprehensive and representative sample of language usage across various domains. Data preprocessing involved several crucial steps …


title: Hacker News new| past| comments| ask| show| jobs| submit login

snippet: I don't think these dichotomies in quality are going to go away. It's the exact same thing in every neural net domain. Self driving is probably the most sophisticated domain besides chat, and there too it's the exact same problem. They can drive a …


title: Hallucinating (or poorly fed) LLMs? The problem of data accuracy

snippet: … This is the case of “Common Crawl”, which consists in a massive collection of web pages and … To give a picture, the Common Crawl dataset constitutes nearly a trillion words. … were publicly available online and memorised in the Common …


title: Harder Tasks Need More Experts: Dynamic Routing in MoE Models

authors: Q Huang, Z An, N Zhuang, M Tao, C Zhang, Y Jin, K Xu… - arXiv preprint arXiv …, 2024

snippet: In this paper, we introduce a novel dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adjusting the number of activated experts based on input difficulty …


title: Hardware Phi-1.5 B: A Large Language Model Encodes Hardware Domain Specific Knowledge

authors: W Fu, S Li, Y Zhao, H Ma, R Dutta, X Zhang, K Yang…

snippet: … The first segment includes data from specialized platforms such as Arxiv , Books, Wikipedia, and StackExchange; the second segment is derived from broader internet content via CommonCrawl and C4. Table II provides a comprehensive …


title: Harmony in the Australian Domain Space

authors: X Gong, PX McCarthy, MA Rizoiu, P Boldi - arXiv preprint arXiv:2404.10006, 2024

snippet: … Though not focused on localized networks, the extensive and all-encompassing web graph from Common Crawl utilized in our study enables the extraction of implicit information on a broader scale. … We utilize the datasets mainly from …


title: Harnessing Artificial Intelligence in Bariatric Surgery: Comparative Analysis of ChatGPT-4, Bing, and Bard in Generating Clinician-Level Bariatric Surgery …

authors: Y Lee, T Shin, L Tessier, A Javidan, J Jung, D Hong… - Surgery for Obesity and Related …

snippet: Background The formulation of clinical recommendations pertaining to bariatric surgery is essential in guiding healthcare professionals. However, the extensive and continuously evolving body of literature in bariatric surgery presents considerable …


title: Harnessing generative AI for overcoming labeled data challenges in social media NLP

snippet: With the introduction of Transformers and Large Language Models, the field of NLP has significantly evolved. Generative AI, a prominent transformer-based technology for crafting human-like content, has proven powerful skills across numerous NLP …


title: Harnessing Small AI Model Collaboration and Debate Mechanisms in 6G Networks: Distributed Architectures and Layered Privacy Protection

authors: C Han, T Yang, X Sun, Z Cui - ICC 2024-IEEE International Conference on …, 2024

snippet: … was used for classification tasks, while OpenSubtitles and CommonCrawl were used for text classification tasks. Our primary evaluation metrics revolve around model accuracy, convergence speed, and computational and communication …


title: Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings

authors: L Gao - arXiv preprint arXiv:2408.15650, 2024

snippet: … For features that require embedding words, we use the 300-dimensional GloVe word embeddings [100] pretrained on the 42 billion token Common Crawl corpus. The GloVe embeddings are provided in decreasing order by frequency, and some …


title: Harnessing the Power of Metadata for Enhanced Question Retrieval in Community Question Answering

authors: S Ghasemi, A Shakery - IEEE Access, 2024

snippet: … GPT-3 was trained on an open source dataset called ‘CommonCrawl’ and other texts from OpenAI such as Wikipedia entries. GPT-4 is the newest version of OpenAI’s language model systems. In this article, we fine-tune GPT-2 and GPT-3 pretrained …


title: Hate speech detection in low-resourced Indian languages: An analysis of transformer-based monolingual and multilingual models with cross-lingual experiments

authors: K Ghosh, A Senapati - Natural Language Processing

snippet: Warning: This paper is based on hate speech detection and may contain examples of abusive/ offensive phrases.Cyberbullying, online harassment, etc., via offensive comments are pervasive across different social media platforms like ™Twitter, ™Facebook …


title: HCI International 2024 Posters: 26th International Conference on Human-Computer Interaction, HCII 2024, Washington, DC, USA, June 29–July 4, 2024, Proceedings …

snippet: Preliminary scientific results, professional news, or work in progress, described in the form of short research papers (4–11 pages long), constitute a popular submission type among the International Conference on Human-Computer …


title: Headline Generation for Indian Languages

snippet: … mT5: We conducted experiments on our dataset by fine-tuning the pre-trained mT5 [35] model, a multilingual variant of T5 [25], which was originally trained on the common crawl dataset, encompassing 101 languages. We use mT5-small model for …


title: HGT: Leveraging Heterogeneous Graph-enhanced Large Language Models for Few-shot Complex Table Understanding

authors: R Jin, Y Li, G Qi, N Hu, YF Li, J Chen, J Wang, Y Chen… - arXiv preprint arXiv …, 2024

snippet: Table understanding (TU) has achieved promising advancements, but it faces the challenges of the scarcity of manually labeled tables and the presence of complex table structures.To address these challenges, we propose HGT, a framework with a …


title: Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding

authors: H Xu, L Chen, Z Zhao, D Ma, R Cao, Z Zhu, K Yu - arXiv preprint arXiv:2402.18262, 2024

snippet: … Common Crawl is a publicly available web crawl dataset that collects webpages from the internet. Instead of using the source code, we collect webpage links from Common Crawl. We traverse all links in a dataset snapshot 4 and categorize and …


title: Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania

authors: J Rudolph, MF Ismail, S Popenici - Journal of University Teaching and Learning …, 2024

snippet: Higher education is currently under a significant transformation due to the emergence of generative artificial intelligence (GenAI) technologies, the hype surrounding GenAI and the increasing influence of educational technology business …


title: Historical Dutch Spelling Normalization with Pretrained Language Models

authors: A Wolters, A Van Cranenburgh - Computational Linguistics in the Netherlands …, 2024

snippet: … These texts are from the common crawl corpus, which contains petabytes of data scraped from the internet. Further pretraining on in-domain data can make the model more suited for the downstream task it will be finetuned for, and since our task …


title: Historical German Text Normalization Using Type-and Token-Based Language Modeling

authors: A Ehrmanntraut - arXiv preprint arXiv:2409.02841, 2024

snippet: Historic variations of spelling poses a challenge for full-text search or natural language processing on historical digitized texts. To minimize the gap between the historic orthography and contemporary spelling, usually an automatic orthographic …


title: History, Development, and Principles of Large Language Models-An Introductory Survey

authors: Z Chu, S Ni, Z Wang, X Feng, C Li, X Hu, R Xu, M Yang… - arXiv preprint arXiv …, 2024

snippet: Language models serve as a cornerstone in natural language processing (NLP), utilizing mathematical methods to generalize language laws and knowledge for prediction and generation. Over extensive research spanning decades, language …


title: HMoE: Heterogeneous Mixture of Experts for Language Modeling

authors: A Wang, X Sun, R Xie, S Li, J Zhu, Z Yang, P Zhao… - arXiv preprint arXiv …, 2024

snippet: Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous experts, each with identical capacity. However, varying …


title: Holographic Embeddings for Text and Graphs A Master's Thesis Presented to The Faculty of the Graduate School of Arts and Sciences

snippet: … Static word embeddings such as word2vec, Global Vectors for Word Representation (GloVe), and fastText are trained on large amounts of natural language text sources like Common Crawl and Wikipedia dumps. These models …


title: Homonym Sense Disambiguation in the Georgian Language

authors: D Melikidze, A Gamkrelidze - arXiv preprint arXiv:2405.00710, 2024

snippet: This research proposes a novel approach to the Word Sense Disambiguation (WSD) task in the Georgian language, based on supervised fine-tuning of a pre-trained Large Language Model (LLM) on a dataset formed by filtering the Georgian …


title: How Chinese are Chinese Language Models? The Puzzling Lack of Language Policy in China's LLMs

authors: AW Wen-Yi, UES Jo, LJ Lin, D Mimno - arXiv preprint arXiv:2407.09652, 2024

snippet: … Additionally, similar to the training of international models, sources of pretrained data rely heavily on web pages, with three reports specifically mentioning Common Crawl as their primary data source… Common Crawl 8 (80%), encyclopedia, books …


title: How Do We Learn What We Cannot Say?

snippet: The contributions of this thesis are two-fold. First, this thesis presents UDTube, an easily usable software developed to perform morphological analysis in a multi-task fashion. This work shows the strong performance of UDTube versus the current state-of-the-art …


title: How Does Code Pretraining Affect Language Model Task Performance?

authors: J Petty, S van Steenkiste, T Linzen - arXiv preprint arXiv:2409.04556, 2024

snippet: Large language models are increasingly trained on corpora containing both natural language and non-linguistic data like source code. Aside from aiding programming-related tasks, anecdotal evidence suggests that including code in pretraining corpora may …


title: How Effective are State Space Models for Machine Translation?

authors: H Pitorro, P Vasylenko, M Treviso, AFT Martins - arXiv preprint arXiv:2407.05489, 2024

snippet: … In order to obtain a small high-quality subset for training, we exclude ParaCrawl and CommonCrawl samples from the original dataset and clean the remaining data. Our cleaning process includes three steps. First, we identify and remove samples in …