When it comes to detecting humans, artificial intelligence tools can be fooled by simple tricks. U.S. Marines were able to deceive a robot trained to identify humans(opens in new window) by doing somersaults, hiding in a cardboard box or pretending to be a tree. Funny? Maybe in this scenario. But in the real world, where human detection technology is used for security and occupational safety purposes, failures like these can have serious consequences.

In a team project based on this premise, high school seniors from across the state of Pennsylvania spent part of their summer analyzing human detection technologies at Carnegie Mellon University. The five-week project, which the students named “Hide and Seek: A Multi-modal Approach to Human Detection,” was offered free of cost through the Pennsylvania Governor’s School for the Sciences(opens in new window) (PGSS). Guided by instructor Andrew McGuier, the students put a suite of cameras and sensors together to make and test their own machine learning model.

Starting with facial recognition software(opens in new window) developed at CMU, the students developed baseline data by testing recognition at different distances from the camera and seeing how accurately the software could detect individuals if the face was partially covered with an office folder. Then, they incorporated a radar sensor, an infrared range-finding camera from an Xbox and a thermal camera to see what kind of successful recognition percentage they could obtain using these types of sensors.

“I share some expertise with them and nudge them in a direction when I see them getting stalled, but otherwise I let them take the ball and run with it, letting them choose which direction the project unfolds and what kinds of avenues of research they pursue,” Andrew McGuier said. “Honestly, I think having the freedom to make those choices is as important a skill to develop as the actual underlying science we’re doing. For many of them, this project is their first opportunity to take the reins and make decisions that sometimes don’t work out, and I think that’s an important experience for high achieving, striving students like them to have.”

Anshi Paul from Hershey, Pennsylvania, was interested in this team project because her science fair projects at school have been focused on assistive technology for dementia patients and have involved facial recognition technology. 

“I like the intersection between hardware and software, and how you can turn lines of computer code into something that’s moving and actually doing something. That is what really excites me,” Paul said. She was also excited to learn Carnegie Mellon has a flying club(opens in new window), as she’s passionate about aviation and robotics and has been working toward getting her pilot’s license. “In the future, I would maybe like to work in robotics in aerospace or biomedical engineering, wherever this path takes me.”

Lukas Dimitroff, who lives about half an hour from the university, said he wants to take the machine learning route in his career and is actually looking into attending Carnegie Mellon after graduation. The project is helping him to get into machine learning and understand how it works.

“It’s nice that there are so many students here who you can collaborate with,” Dimitroff said. “There’s not much I can do for research through my high school. Even if I tried to talk to one of my teachers about doing something, it would still just be me doing the research on my own.” 

His teammate on the project, Shivam Maheshwari, agreed.

“One of the things that Carnegie Mellon naturally stresses is collaboration, and you can see, we have so many different groups here, and they’re all working together towards a common goal. That’s something I never get to experience in high school because we don’t have much research at our school,” Maheshwari said. “Here we’re learning from each other, and then we’re combining all our skills, too. Anshi has expertise in hardware, while Lukas and I have expertise in coding and our whole sensor analysis portion. So, it’s really cool to see how all of us with diverse experiences can work together on a project like this.”

Andrew McGuier said many of these students come from smaller schools where they are the smartest kid in their class, and this is the first time they might be the one who doesn’t quite understand what’s going on.

“For many of them, this is the first time they’ve experienced a project where they are the one who has to ask for help, and I think that is also an incredibly important aspect of this program,” he said. 

Natalie McGuier, director of the PGSS, said creating an opportunity for exceptional students to explore their diverse scientific interests, collaborate with like-minded peers, conduct meaningful research and be challenged academically is at the core of PGSS, which Carnegie Mellon has hosted since its inception in 1982. There is no cost to the students other than transportation to and from campus. Food, housing and the entire experience is a full scholarship program. 

“Carnegie Mellon is and always has been a champion of interdisciplinary work and training the next generation of STEM (science, technology, engineering and mathematics) professionals while holding its community to the highest standards,” she said. “One hundred percent of our students from this program go on to college, 93% pursue careers in STEM, 86% earn postgraduate degrees and 60% earn the highest degree in their field.”

Andrew McGuier said instructing the project is his favorite thing he does all year.

“These students are constantly finding new ways to surprise you with how bright they are and also their novel way of looking at problems,” he said.

Andrew McGuier was a student in the PGSS program himself in 2004, after which he became a member of the residence life staff. He went on to earn his undergraduate degree from Carnegie Mellon and has been teaching in the program since 2017. From his perspective, technology is evolving very rapidly and getting rolled out and released into the wild without enough thought dedicated to the potential impacts on people who are underserved, underprivileged, or potentially or historically discriminated against. 

“I think those challenges require a scientifically literate group of both workers and academics who have a sensitivity to those things, and who can tackle those challenges,” he said. “That’s why it’s so important these students get this experience and have this background, because they’re the ones we’re going to hand these challenges off to.”