For decades, the emphasis in engineering design has been to make design decision support tools more intelligent, in the sense that they can learn and adapt to different problems or to different phases of the search process for a given problem. Some researchers have also looked at the idea of creating ways in which humans and computers can collaborate in an interactive design process. This second view casts design tools as autonomous and adaptive agents that collaborate with humans in mixed design teams. This project build upon this second view and human-robot interaction to develop the foundations of what we called a mixed-initiative approach to the design of complex systems. Our vision is illustrated in the figure below.
One of the key steps is to cast the design decision support tool as an intelligent agent that can sense the state of the world and plan a sequence of actions to execute. The first task is thus to apply this model to intelligent design agents, as shown below.
The next key step to enable our mixed-initiative vision is to provide agents with the ability to explain their actions. You should be able to ask your AI design companion why she suggested that change to the design. This requires a natural language interface. The Daphne cognitive assistant was a perfect framework for this NSF project and a lot of its development happened to support the science behind this project. You can read more about Daphne here.
Once we had a prototype of Daphne running, the next step in our vision was to experiment with role allocation. Specifically, we experimented with two different roles for the assistant, that we called the Analyst and the Critic. The Analyst is the traditional paradigm of cognitive assistants, by which they remain idle unless they are asked a question at which point they process the request, gather the information required to address it, and build and answer. The Analyst can perform tasks such as searching the design space, applying data mining to datasets, or querying databases. The Critic, on the other hand, is a different role, closer to that of a peer designer. The Critic can answer more complex questions such as “What do you think of this design?”
We are currently performing experiments with human subjects to measure the effect of the roles (critic vs Analyst) for the Daphne cognitive assistant on human performance and learning during design space exploration tasks.
A third goal of this project was to experiment with non-verbal communication, tangible interfaces, and embodied agents. This part of the project was led by our collaborator Guy Hoffman from Cornell University. Guy’s team developed a tangible interface for our design space exploration problems, namely a table top where cubes represent different satellite instruments, and their positioning on the table represents their orbits.
We then conducted a study that showed that the search performance of the human-agent team on the tangible interface was higher than that of the human or the agent alone.
Finally, an important part of our vision is that the agent should be able to detect and adapt to individual user differences and preferences. We performed an initial study in this direction looking at cognitive style differences as well as level of expertise. For this study, we made extensive use of our current work on feature space exploration and compared two design support interfaces, one based on traditional design space exploration and the other based on our proposed feature space exploration ideas. Analysis results show significant impacts of field knowledge and visual processing style on both learning and user experience. Potential interaction effects with the design support system interface type and cognitive styles were also observed.
You can learn more about this project by reading the papers below.
Viros Martin, A., & Selva, D. (2019). From Design Assistants to Design Peers: Turning Daphne into an AI Companion for Mission Designers. AIAA Information Systems-AIAA Infotech at Aerospace, 2019. https://doi.org/10.2514/6.2019-0402
Bang, H., Viros, A., Prat, A., & Selva, D. (2018). Daphne : An Intelligent Assistant for Architecting Earth Observing Satellite Systems. 2018 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA SciTech Forum, 1–14. https://doi.org/10.2514/6.2018-1366
Law, M., Dhawan, N., Bang, H., Yoon, S.-Y., Selva, D., & Hoffman, G. (2018). Side-by-side Human-Computer Design using a Tangible User Interface. Design Computing and Cognition ’18.
Lily Shi, Bang, H., Hoffman, G., Selva, D., & Yoon, S.-Y. (2018). Cognitive style and field knowledge in complex design problem solving: A comparative case study of decision support systems. Design Computing and Cognition ’18.
Bang, H., Shi, Y. L. Z., Yoon, S.-Y., Hoffman, G., & Selva, D. (2018). Exploring the Feature Space to Aid Learning in Design Space Exploration. Design Computing and Cognition’18.
This work is funded by NSF CMMI grant number 1635253.