October 12 schedule

Student Panel

A Comparative Analysis of Multimodal LLMs for Grading Handwritten Fractions Assignments

Abstract: This project develops an AI-assisted grading pipeline to assess mathematical accuracy and classify errors in elementary students’ handwritten fraction assignments. Using a teacher-in-the-loop approach, the system aims to provide personalized feedback that supports student learning. We evaluated three multimodal large language models (Claude, ChatGPT, and Gemini) on a dataset of 320 problems. Claude performed best overall, achieving 86.9% agreement with human graders and the most accurate error-family classifications, although it required longer processing times. Its reliable parsing and strong grading performance suggest that multimodal LLMs can support automated mathematics grading and provide educators with actionable insights into students’ learning gaps. However, the models occasionally flagged correct answers as errors, underscoring the importance of teacher oversight.

A Label-Free Framework for Evaluating RAG Chatbots with a 16-Month Production Case Study

Abstract: Retrieval-augmented generation (RAG) chatbots answer user questions using relevant passages retrieved from a knowledge base. They are commonly evaluated using curated benchmarks with labeled data. In real-world deployments, however, chatbot interactions are typically unlabeled, and ground-truth answers are often unavailable for evaluation. To support real-world evaluation, we propose a label-free framework that organizes existing evaluation methods into a reusable workflow without requiring labeled data or ground-truth answers. We demonstrate the framework through a production case study analyzing 112,474 real user interactions collected over 16 months from AskCody, a RAG-based chatbot deployed at CSUSB. The proposed workflow uses embedding-based cosine similarity to measure how well a response addresses the user’s question (relevance) and how closely it aligns with the available retrieved evidence (groundedness). It then examines the distributions of these scores, their relationship at the interaction level, and how they change over time. It also examines reward-model scores, fallback behavior, and interaction patterns to provide additional context for interpreting the automatic scores. Together, the findings demonstrate how the framework combines relevance, groundedness, reward-model scores, and system context to provide complementary screening signals for evaluating large, unlabeled production datasets.

Center for Cyber and AI

Coming Soon

Gen AI for Teaching and Learning
How are CSUSB faculty responding to student use of AI for coursework? What pedagogical practices help address AI concerns or create opportunities to promote student learning outcomes and student engagement? Moderator TRC Director Mihaela Popescu will facilitate a discussion with scheduled panelists.

Faculty Panel: 
Dr. Jesus Canelon, Associate Professor, School of Cyber & Decision Sciences
Dr. Katy Wimberly, Assistant Professor, Physics and Astronomy
Dr. Mahbuba Hammad, Lecturer, College of Arts & Letters
Dr. Sailesh Maharjan, Lecturer, Psychology
Dr. Samantha Juan, Assistant Professor, Nursing
Dr. Vincent Nestler, Professor, School of Cyber & Decision Sciences

AI and Faculty Research
How are CSUSB faculty using AI in their research? What do they see as the opportunities and concerns? Moderator TRC Director Mihaela Popescu will facilitate a discussion with scheduled panelists.

Faculty Panel: 
Dr. Samia Amin, Assistant Professor, Health Science and Human Ecology
Dr. Ahmed Elnoshokaty, Assistant Professor, School of Cyber & Decision Sciences
Dr. Daniel MacDonald, Professor, Economics
Dr. Jeremy Murray, Professor, History
Justin Swanson, Lecturer, Kinesiology
Dr. Sishi Wu, Assistant Professor, Criminal Justice
Dr. Stephen Bronack, Professor, Education Leadership and Technology

Embodied AI Systems: Building Intelligent Humanoid Robots (Yote)
Student Demo (IST 5930 - Embodied AI Systems)

Researchers—Intelligence and Interaction: 
Explore how Yotie can understand questions, communicate with students, and support classroom learning. 

Engineers—Movement and Integration: 
Connect the robot’s hardware, sensors, software, and movement so Yotie can interact with its physical environment.

Security Officers—Security and Privacy: 
Investigate how to protect the robot’s network connections, manage access, and handle classroom data responsibly.