Exploring uncharted skies
The students began work on the project after meeting
Krista Kinnard (MSPPM ‘17), a Heinz College alumna and the deputy chief AI officer at NASA. Kinnard was visiting CMU to discuss her career, and the students pitched her the idea of a capstone with the space agency. With Kinnard on board as their capstone client, the group decided to focus the project on AI governance.
“AI governance is a wide-open space. It’s not really solved in any industry, especially aerospace, where the consequences are very high if there are errors or hallucinations,” Kinnard said. “We want to create policies and protocols that allow us to embrace AI while continuing to set up our missions for success.”
Developing the AI framework involved hours of research over much of the fall semester. The team learned about the federal AI regulations NASA must follow. They reviewed the guidelines of foreign space agencies, international organizations and private companies such as SpaceX and Blue Origin. They even interviewed senior leadership members from several private companies, thanks to Kinnard’s industry connections.
Given their limited experience in the aerospace industry, the students needed to learn new terminology and the inner workings of an unfamiliar policy landscape. To get their bearings, the team gathered insights from more than a dozen people at NASA, including cyber, data and engineering experts. They frequently consulted with Kinnard and sought guidance from
Ramayya Krishnan and
Anand Rao, both AI experts and Heinz College professors.
“AI governance is an emerging field. New federal policies would come out, or breaking news stories would develop, and as a result, we would continuously adapt our project,” Willhite said.
With some team members
studying in Washington, D.C., the students could further their research by connecting with the policymakers and leading minds who frequently convene in the nation’s capital. Willhite received funding from CMU to attend a conference from
IAPP, a nonprofit focused on AI governance. Other team members volunteered at AI conferences to network and ask questions.
In the end, the students put forth recommendations for NASA across three governance categories: data, performance and operations. Among the recommendations:
- Add AI-specific reviews to existing cyber practices
- Identify thresholds at which models pose cyber risks
- Stress-test all mission-embedded AI in simulated environments
- Perform continuous monitoring and accuracy testing of AI systems.
“We didn’t want NASA to reinvent the wheel because they're already doing a lot of continuous risk assessment, and they have a lot of engineers already monitoring the technology,” Willhite said. “We aimed to say, ‘Here are the existing problems, here's how other existing policies address them, and here's where NASA can address new risks.”
Building on that analysis, Chavez developed an implementation framework to help translate the team’s governance recommendations into practical steps NASA could apply.
For the project’s technical element, team members Zwijacz and Jhaveri used Python and tools such as Ollama and PageIndex to develop a local, artificial intelligence-based tool that could efficiently search large procedural manuals for information. The two built and tested the AI tool iteratively, improving its citations and safety checks when early versions failed to provide correct information.
The tool was vetted and refined through weekly meetings between the students and NASA employees. The students had to refine their project scope several times as they better understood both NASA’s needs and the limitations of different technical approaches. Overall, Zwijacz said the project taught him how to solve a complex, unfamiliar technical problem in a limited timeframe.
“Rather than approaching the project strictly as a technical exercise, I learned to consider the broader operational, policy and governance implications of the system we were building,” Zwijacz said. “I also benefited from being able to consult previous professors and faculty for guidance when we encountered unfamiliar challenges and incorporated their feedback.”
Collaboration with NASA
With both the technical and policy components of the project completed, the team presented their work to NASA’s human spaceflight group and senior leadership. The project was well-received. “I was blown away by the quality of the work that they were able to deliver, by the depth of research and critical thinking they were able to apply to the project,” Kinnard said.
“It was not an academic exercise so that they could check the box to say, ‘I did my systems project. This is a real, valuable project where they produced deliverables that we can use as an organization,” Kinnard added. “What we got from them was delivered at a higher quality, in a faster time, than I could have paid a big consultancy to deliver.”
In the future, NASA’s human spaceflight mission team envisions piloting the students’ prototyped large language model, according to Kinnard. As part of the pilot, NASA is exploring opportunities for deploying the model and ways of expanding its baseline knowledge. On top of that, NASA’s Office of Safety and Mission Assurance is considering the students’ recommendations in drafting policies for evaluating AI systems.
The Heinz College experience
Working on the capstone project was challenging but exciting, Williams said. The students were all taking classes while pursuing the project, and some were also working full-time. Despite those demands, the team met multiple times a week to dole out responsibilities and track progress.
Coming out of undergraduate studies with a political science degree, Williams said Heinz College taught him how to analyze data, understand complex research papers and translate findings to a wide range of audiences. The Washington, D.C., track also taught him time management, as he quickly learned he needed to balance his class work and full-time job responsibilities.
“Having that amount of work in the first semester was genuinely a life-changing experience for me because I found that all of the things that I'm doing matter at the moment. There was nothing I could throw to the wayside,” he said. “That preparation really helped us be able to work with an organization like NASA and keep our poise.”
The Heinz College curriculum and alumni network also supported the students in their work, Willhite said. Through a class on federal budget policy, the students learned how federal organizations successfully pitch and earn funding for projects. Along the way, the Heinz College reputation gave the students credibility when presenting their work, asking to connect with leaders in AI and networking at conferences.
For Chavez, the project strengthened his interest in applying emerging technology and policy analysis to national security challenges. He’s now training to become an intelligence officer for the U.S. Air Force, where that same emphasis on synthesizing complex information and understanding its operational implications carries into a national security context.
The project opened a lot of doors, Willhite said. The work sparked her interest in AI governance research and gave her a competitive edge. Now, she’s an AI safety researcher at the University of Cambridge through ERA, a global research organization that aims to ensure a safe transition to advanced AI.
“CMU is really good at identifying where emerging tech is headed and getting students involved before it explodes, so that when opportunities arise, students are well-equipped to pursue them,” she said.
Williams added, “CMU is uniquely positioned to go into organizations like NASA, especially with the curriculum and the focus on AI and data. We were very passionate students around very passionate people, we all had a similar goal and mindset going into the project, and we came out with a lot of amazing takeaways.”