Location
School of Law Seminar Room 3.09
Start Date
4-6-2026 4:00 PM
End Date
4-6-2026 4:30 PM
Description
As AI-powered tools are emerging in all aspects of research, it’s challenging for researchers, students, and librarians alike to assess the benefit, usefulness, and risks associated with a new AI-powered tool or AI-enhanced features of an existing one. To address this challenge, Carnegie Mellon University Libraries is developing a Tool Evaluation Framework to help users understand how AI-powered tools work, initially focused on AI-enhanced academic search engines. By evaluating criteria such as mechanisms for document retrieval, processes for content generation, and the quality of outputs, this framework guides users to carefully consider the strengths and weaknesses of a tool before adoption. Furthermore, a key consideration of our framework is the inclusion of sustainability as a factor in tool assessment. Data collection is underway to understand the environmental impact of AI Summary and AI Assistant functions embedded in AI-powered tools used in scholarly communications. Work from the project will create practical criteria and checklists that users may use to navigate the complex nature of adopting AI-powered tools while maintaining critical lenses in scholarship.
Included in
Developing an assessment framework to support critical evaluation of AI-powered academic search engines
School of Law Seminar Room 3.09
As AI-powered tools are emerging in all aspects of research, it’s challenging for researchers, students, and librarians alike to assess the benefit, usefulness, and risks associated with a new AI-powered tool or AI-enhanced features of an existing one. To address this challenge, Carnegie Mellon University Libraries is developing a Tool Evaluation Framework to help users understand how AI-powered tools work, initially focused on AI-enhanced academic search engines. By evaluating criteria such as mechanisms for document retrieval, processes for content generation, and the quality of outputs, this framework guides users to carefully consider the strengths and weaknesses of a tool before adoption. Furthermore, a key consideration of our framework is the inclusion of sustainability as a factor in tool assessment. Data collection is underway to understand the environmental impact of AI Summary and AI Assistant functions embedded in AI-powered tools used in scholarly communications. Work from the project will create practical criteria and checklists that users may use to navigate the complex nature of adopting AI-powered tools while maintaining critical lenses in scholarship.