Stephanie Milani

I am a research assistant in the Institute for Software Research and the Robotics Institute at Carnegie Mellon University. I work with Fei Fang on integrating deep reinforcement learning and game theory for societal challenges, and with David Held on reinforcement learning and learning from demonstration for robotics. I am also an incoming Ph.D. student in the Machine Learning Department at Carnegie Mellon University.

I graduated in 2019 with a B.S. in Computer Science and a B.A. in Psychology from the University of Maryland, Baltimore County. There, I was advised by Marie desJardins and Cynthia Matuszek.

I am open to and excited about collaborating with others. Please email me if you have any questions about a machine learning (or psychology) project, or if you are interested in a research collaboration.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Twitter


  • August: Joining Carnegie Mellon's Machine Learning Department as a Ph.D. student.
  • July - August: Working with Fei Fang at Carnegie Mellon University for the second half of the summer.
  • July: Presented some new work at RLDM 2019.
  • June: Attended ICML.
  • June - July: Working with Dave Held at Carnegie Mellon University for the first half of the summer.
  • May: Graduated from UMBC!
Research Interests

I'm interested in sequential decision making, reinforcement learning, and neuroscience.


The MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors
William Guss, Cayden Codel*, Katja Hofmann*, Brandon Houghton*, Noburo Sean Kuno*, Stephanie Milani*, Sharada Mohanty*, Diego Perez-Liebana*, Ruslan Salakhutdinov*, Nicholay Topin*, Manuela Veloso*, Phillip Wang*
NeurIPS Competition Track, 2019

Penalty-Modified Markov Decision Processes: Efficient Incorporation of Norms into Sequential Decision Making Problems
Stephanie Milani, Nicholay Topin, Katia Sycara
RLDM, 2019

Perceptions of Domestic Robots' Normative Behavior Across Cultures
Huao Li, Stephanie Milani, Vigneshram Krishnamoorthy, Michael Lewis, Katia Sycara
AI, Ethics, and Society, 2019

Towards Planning with Hierarchies of Learned Markov Decision Processes
John Winder, Shawn Squire, Matthew Landen, Stephanie Milani, Marie desJardins
ICAPS Workshop on Integrated Planning, Acting, and Execution, 2017

R-AMDP: Model-Based Learning for Abstract Markov Decision Process Hierarchies
Shawn Squire, John Winder, Matthew Landen, Stephanie Milani, Marie desJardins
RLDM, 2017

I got this great website here.