Evaluating Intelligent Tutoring Systems for Teaching Japanese Mahjong

Marshall An

Additional authors: He, Mufei, Human-Computer Interaction Institute; Yao, Wei, Human-Computer Interaction Institute; Stamper, John, Human-Computer Interaction Institute.

We examine the effectiveness of Intelligent Tutoring Systems (ITS) in the CMU student-taught course "Introduction to Japanese Mahjong" (StuCo 98179). Developing expertise in Japanese Mahjong is challenging, as each move requires the integration and application of various Knowledge Components (KCs). Specifically, the same observable action may result from different combinations of diverse KCs, depending on the context. The ITS we developed breaks down complex cognitive processes into manageable subgoals and detailed steps, providing students with hints and immediate feedback to facilitate learning. Our in-vivo experiment conducted in Spring 2024 utilized pre- and post-tests to evaluate the system's impact. The results indicated a statistically significant improvement in median scores post-intervention. This pilot study demonstrates ITS's potential to enhance learning in domains that demand extensive cognitive integration and application of multiple KCs.

GPTutor: Great Personalized Tutor with AI for Personalized Learning Content Generation

Eason Chen

Additional authors: Lin, Jionghao, Koedinger, Kenneth, Tang, Xinyi, Xiao, Zimo

GPTutor is a pioneering web application designed to revolutionize personalized learning by leveraging Generative AI at scale. It adapts educational content and practice exercises to individual students' interests and career goals, enhancing engagement and understanding of critical academic concepts. GPTutor uses a serverless architecture to deliver scalable learning experiences and integrates advanced Chain-of-Thoughts prompting methods. This personalized educational journey addresses unique student interests and prepares them for future professional success. This demo paper showcases the design, functionality, and potential of GPTutor to foster a more engaging and effective educational environment.

Encouraging Review of Foundational Concepts through Student Generated Visual Diagrams

Lynley Doonan

Additional author: Willard, Amanda, Biological Sciences

​​Frontiers, Analysis, and Discovery in Biological Sciences is a first-year, research based laboratory course. Students often have difficulty connecting the individual experimental components with the overall project goals, especially when components needed at the end of the semester are generated at the beginning. To address this, we implemented a hands-on activity where students create their own visual diagram of the cellular experiment using craft supplies. We revisit this diagram multiple times to add details as our project progresses. This gives students the autonomy to visually display the concepts and redefine key terms in their own way. This also allowed instructors to identify areas of confusion and correct them in real time. This strategy can be adapted to any class, project, or assignment that requires students to review concepts that build on each other. We will share examples of our activity and how it is presented and structured in class.

Moving a Simulation-Based, Flipped-Classroom Course into an Online Environment

Nick Frollini

Simulation-based, flipped-classroom courses offer the potential for enhanced student learning, but are often viewed as challenging to execute. The idea of moving such courses into an online environment while maintaining the essential elements of experiential learning can seem to be even more daunting. We present the process of moving a negotiations course from the physical to the virtual world, while working to ensure that students were not disadvantaged by the change in course modality. We review the initial considerations in terms of course design, as well as changes made to accommodate the remote experience for both the students and the instructor. We also note several unexpected challenges posed by a student base distributed around the world and some serendipitous outcomes as well. Finally, we highlight some key lessons learned that can be applied by other instructors contemplating moving their own simulation-based courses into an online modality.

Inclusive Machine Learning: Promoting Belongingness and Empathy in Diverse Classrooms

Gabriela Gongora-Svartzman

As part of a 2023 Provost’s Inclusive Teaching Fellowship (PITF), Machine Learning Foundations with Python was redesigned to include teaching strategies to better address student motivation and engagement, and to explore the intersection of ethics, bias, and societal impact in machine learning education for non-technical students. The research questions for this project were: 1) Do group discussions of case studies impact students' growth in ethics awareness? 2) Do students feel connected and motivated toward machine learning? and 3) Can students recognize how machine learning is connected with their careers and related to their future? This poster will discuss the importance of group discussions of case studies from diverse fields, such as public policy and healthcare, and their impact on students' ethical awareness and motivation towards machine learning. Although the case studies were vital to this project, other teaching strategies, such as TILTing assignments, refining learning objectives, and adjusting communication and transparency, significantly enhanced student engagement. The success of this project was measured through surveys and written answers to individual and group case studies, among other assignments. I hope to share with others, across disciplines and fields, how to make small, sustainable, and inclusive changes over time.

Closing the Loop on Interdisciplinary Perspectives Outcomes in Dietrich College First-Year Seminars

Baron Glanvill

Additional authors: Laudenbach, Michael, English; Dickert, Joanna Core Competencies Initiative and Dietrich College Dean's Office; Leverett-Ptak, Chelsea, Core Competencies Initiative and Dietrich College Dean's Office; Carver, Sharon M., Dietrich College Dean's Office; Ballesteros Soria, Nuria, Dietrich College Dean's Office

This presentation reports on the second phase of artifact collection for Dietrich College’s Grand Challenge (GC) first-year seminars. Following consultations with GC instructors about the limited evidence of Interdisciplinary Perspectives (IP) across 2021-2022 seminars, the rubric was revised to account for preliminary stages of IP development among first-year students. Following recommendations from the report, a customizable assignment was developed that explicitly asked students to display disciplinary knowledge in the context of a societal challenge. After testing the revised rubric and discussing the assignment with GC instructors, new artifacts from 2022-2024 were assessed to determine whether the evidence of IP remained limited. The customizable assignments scored very highly, demonstrating the importance of explicit instruction and prompts to support students’ learning and application of IP. The presentation will address the project design details so other programs targeting interdisciplinary learning can replicate the study if so desired. Finally, the presentation will provide links to the report, the rubric, and the assignments, which may be leveraged in any undergraduate course and/or program targeting interdisciplinary learning.

AI-Enhanced Writing Studio for Writing in the Disciplines and Professions

Suguru Ishizaki

Additional authors: Kaufer, David, Department of English

We present here our working prototype of our AI-enhanced online writing environment with a suite of generative and assessment tools supporting the writing process from initial-phase writing to final draft. Despite significant research conducted on writing instruction over the past 60+ years, there have been no scalable breakthrough solutions to help college graduates meet standards of written proficiency. Our project seeks to close this gap through the application of research-based principles derived from the literature on writing process and pedagogy.

Does Generative AI Change How Students Brainstorm (for Startup Ideas)?

Carrington Motley

Students in 70-415 (Introduction to Entrepreneurship) completed a brainstorming session for ideas for startups to use in the main project for the course. An intervention was implemented to determine how students’ ability to use generative AI would influence the brainstorming process & students’ experience. There are two sections of the course. In one section, students were encouraged to use generative AI to assist with idea generation and selection and were given some basic instructions on prompt engineering & brainstorming. In the second section, students were given the same instructions about brainstorming but were not allowed to use generative AI. By comparing the two sections, I aimed to better understand the impact of generative AI during a class session on students’ ability to generate & evaluate startup ideas, satisfaction with brainstorming, and confidence in using GenAI tools. To answer this question, I administered surveys, collected trace data, and evaluated student assignment outcomes. The initial analysis of this exploratory study indicates that students across the two groups received similar final grades on the assignments. However, the students who used generative AI reported creating more ideas in the brainstorming session while also reporting being less satisfied with the session (relative to students who did not use generative AI). The study’s findings provide useful insights to people interested in the effects of generative AI on the creative process and idea generation, in particular.

Is Citizenship the Right Model to Engage Students?

Raelin Sawka Musuraca

Whether it is their digital native upbringing or the long-lasting impact of the pandemic, it has been difficult to engage students. Attendance, class participation, office hours, and team-based projects have all been a challenge. In Service Design, a mixed undergraduate and graduate course, I used a Citizenship Score model to drive engagement beyond the course materials and elicit contributions as a classroom citizen. This model involved a detailed rubric awarding points for attendance, on-time arrival, engagement, and peer surveys; it allowed for bonus points for exceeding expectations. Anecdotal results were mixed – while two-thirds scored above 90%, verbal feedback was more carrot/stick driven rather than a desire to be a classroom citizen. I am proposing this discussion to engage with my peers and explore ways to improve this method. I can share the Assignment and Rubric with others to start the conversation.

AI-Driven Insights from Data Analytics Education

Fethiye Ozis

Additional author: Takara Matthew, Civil and Environmental Engineering

Our presentation will focus on an educational research project that addresses two primary research questions: RQ1: Does the use of genAI tools influence students’ ability to work with, clean, and visualize large datasets compared to the pre-GenAI era? RQ2: What are students’ attitudes, perceptions, and experiences regarding genAI-powered tools for data processing and visualization? Students were juniors in a Civil Engineering course who were offered the opportunity to utilize Perplexity AI and other free generative AI tools for processing and visualizing a large dataset. The instructional approach also included providing additional subtasks, clarifying expectations, and transparent evaluation criteria. Measures included student learning outcomes, perceptions, experiences, and attitudes. We will describe our research results as well as lessons learned from doing a teaching-as-research (TAR) project.

Empowering Student Voices: The Role of the Student Advisory Council in Shaping CMU-Africa's Messaging

Monica Sumbi, Associate Director of Marketing & Communications, CMU-Africa

Engaging the student body is crucial to creating a dynamic and inclusive environment at CMU-Africa. The Student Advisory Council (SAC) serves as a vital platform to ensure that student voices are actively represented in the university’s messaging. This abstract explores the role of the SAC in fostering communication between students and the administration, highlighting the importance of student input in shaping the narrative and outreach of CMU-Africa. By integrating diverse perspectives, the SAC not only enhances the relevance and authenticity of institutional messaging but also strengthens the sense of community and belonging among students. The council’s efforts contribute to a more engaged student body, aligning institutional goals with the needs and aspirations of the students. This approach underscores CMU-Africa's commitment to inclusivity, transparency, and collaborative growth, ensuring that the university’s communications resonate with and reflect the diverse voices of its student population.

Designing Intelligent Tutoring Systems in Community College IT Courses

Leah Teffera

Additional authors: An, Marshall, HCII; Mehrvarz, Mahboobeh, HCII; Purtell, Krystof, HCII; Li, Bruce, HCII; Bogart, Chris, Computer Science Department; McLaren, Bruce M., HCII; Branstetter, Lee, Heinz College; Burkhardt, Bryan, Des Moines Area Community College (DMACC); Hoffman, Jeremy, DMACC; Howe, Jeffrey, DMACC; Rinsem, Michael, Community College of Allegheny County (CCAC)

This poster presents our research on a hands-on cloud administration course co-designed with community colleges, utilizing project-based learning (PBL) and intelligent tutoring systems (ITS). We explore the role of ITS in addressing challenges associated with PBL and examine whether the effectiveness of ITS is primarily attributed to the teaching method or the content. Using the Knowledge-Language-Instruction (KLI) framework, we analyzed the course content to identify skills that would benefit most from repeated practice through an ITS. We then designed an ITS based on those skills, conducting two pilot studies in introductory computing courses. The studies were randomized control trials, using the ITS and a multimodal reading including text, graphs, and multiple choice questions. Based on data collected from the ITS and learning management system, we will share analyses of students’ performance and behavior, and discuss future directions for designing new educational tools applicable in multiple learning contexts.

A Novel Approach to Student Team Formation

Andrew Twigg

Additional author: Valluri, Nikita, graduate research assistant

Proposing a novel method for student team formation using combined student-provided information, instructor guidance, data visualization, and real-time classroom facilitation. Performance of student teams formed by instructor, by student preference, or by random assignment is debated; this approach hybridizes student input with instructor guidance and in-class dialogue. Students complete a survey declaring preferences for collaboration style, as well as strengths and skill development goals. Survey data allows the creation of visualizations used to build potential teams; these are shared with students, alongside discussion on advantages and disadvantages of each teaming strategy. Through instructor-facilitated dialogue, teams may be revised in real time, based on student input, to create final teams. This instructor-guided, student-driven process aims to result in high-performing team composition, and cultivate student buy-in due to their direct involvement. The focus of this presentation is the methodology, opportunities for improvement, its significance and limitations, and potential future directions.

Hei Hai Come On: incorporating song learning into a community engagement event

Sue-mei Wu

Incorporating songs into language and culture learning can not only enhance learners’ language skills but also inspire their critical thinking and creativity and create an interdisciplinary and collaborative environment for learners. This presentation will showcase the “Hei Hai Come On” song and its music videos, which were created by the 82-139 class (82-139: Chinese Learning Through Cultural Practices and Community Engagement). The instructor will showcase how students enhanced their learning outcomes by contributing to community engagement events, which made their learning process more entertaining, engaging and impactful.