Designing Human and Generative AI Collaboration

We examined the effectiveness of human-AI collaboration designs in creative work. Through a human subjects experiment in the context of creative writing, we found that while AI assistance improved productivity across all models, collaboration design significantly influenced output quality, user satisfaction, and content characteristics. Models incorporating human creative input delivered higher content interestingness and overall quality as well as greater task performer satisfaction compared to conditions where humans were limited to confirming AI outputs. Increased AI involvement encouraged creators to explore beyond personal experience but also led to greater story and genre similarities among participants. However, this effect was mitigated through human creative input. These findings underscore the importance of preserving the human creative role to ensure quality, satisfaction, and creative diversity in human-AI collaboration.