I build code¹ generating systems inspired by human communications, enabling them to follow instructions with both flexibility and precision. Human communication is flexible in modality, situations, and conversation partners. Remarkably, these flexible instructions can convey precise tasks such as writing a program, refining a floor-plan, or operating a machinery. Current instruction-following systems are either inflexible (e.g. programming, user interface) or imprecise (e.g. LLM, generative AI). My work aims to develop computational models of human communication, and applying these models to build code generating agents that interact with people. I elicit instruction datasets, develop sample efficient instruction following algorithms, and evaluate interactive systems with user studies.

I am a senior research scientist at Autodesk AI Lab, where I work on code-generation for human-machine collaboration in CAD, and industry scale instruction-following dataset annotation. My academic collaborations focus on building (human) sample efficient instruction-following algorithms in program synthesis, embodied agents, and collaborative design. I collaborate broadly across Cogsci, NLP, PL, Graphics, and HCI. I received my PhD under Armando Solar-Lezama and Leslie Kaelbling at MIT.

Representative Works

• Program Synthesis with Pragmatic Communication (NeuRIPS 2020)
  Yewen Pu, Kevin Ellis, Marta Kryven, Josh Tenenbaum, Armando Solar-Lezama
• Communicating Natural Programs to Humans and Machines (NeuRIPS 2022)
  Samuel Acquaviva *, Yewen Pu *, Marta Kryven, Theodoros Sechopoulos, Catherine Wong, Gabrielle E Ecanow, Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum
• Communicating Design Intent Using Drawing and Text (Creativity & Cognition 2024)
  William P. McCarthy, Justin Matejka, Karl D.D. Willis, Judith E. Fan, and Yewen Pu

Additional Information

• CV
• publication (under Yewen Pu)
• collaborators
• mentoring statement
• updates / news on twitter
• email

Shoot me a twitter message or email any time if you want to chat !

1. Code as in any interactions with a computer. For example: programming, API calls, and acting in a simulated environment. ↩