Publication Type

Conference Proceeding Article

Version

publishedVersion

Publication Date

7-2024

Abstract

Llama 2, an open-source large language model developed by Meta, offers a versatile and high-performance solution for natural language processing, boasting a broad scale, competitive dialogue capabilities, and open accessibility for research and development, thus driving innovation in AI applications. Despite these advancements, there remains a limited understanding of the underlying principles and performance of Llama 2 compared with other LLMs. To address this gap, this paper presents a comprehensive evaluation of Llama 2, focusing on its application in in-context learning — an AI design pattern that harnesses pre-trained LLMs for processing confidential and sensitive data. Through a rigorous comparative analysis with other open-source LLMs and OpenAI models, this study sheds light on Llama 2’s performance, quality, and potential use cases. Our findings indicate that Llama 2 holds significant promise for applications involving in-context learning, with notable strengths in both answer quality and inference speed. This research offers valuable insights for the fields of LLMs and serves as an effectivereference for companies and individuals utilizing such large models. The source codes and datasets of this paper are accessible at https://github.com/inflaton/Llama-2-eval.

Keywords

large language model, in-context learning, generative pre-trained transformer, model evaluation

Discipline

Artificial Intelligence and Robotics | Databases and Information Systems

Research Areas

Intelligent Systems and Optimization

Areas of Excellence

Digital transformation

Publication

Proceedings of the 2024 IEEE Conference on Artificial Intelligence (IEEE CAI 2024), Singapore, June 25-27

First Page

1

Last Page

5

Publisher

IEEE

City or Country

Los Alamitos, CA

Copyright Owner and License

Authors

Download

Share

COinS