InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs

Visual programming has the potential of providing novice programmers with a low-code experience to build customized processing pipelines. Existing systems typically require users to build pipelines from scratch, implying that novice users are expected to set up and link appropriate nodes from a blank workspace. In this paper, we introduce InstructPipe, an AI assistant for prototyping machine learning (ML) pipelines with text instructions. We contribute two large language model (LLM) modules and a code interpreter as part of our framework. The LLM modules generate pseudocode for a target pipeline, and the interpreter renders the pipeline in the node-graph editor for further human-AI collaboration. Both technical and user evaluation (N=16) shows that InstructPipe empowers users to streamline their ML pipeline workflow, reduce their learning curve, and leverage open-ended commands to spark innovative ideas.

@article{zhou2025_2312.09672,
  title={ InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs },
  author={ Zhongyi Zhou and Jing Jin and Vrushank Phadnis and Xiuxiu Yuan and Jun Jiang and Xun Qian and Kristen Wright and Mark Sherwood and Jason Mayes and Jingtao Zhou and Yiyi Huang and Zheng Xu and Yinda Zhang and Johnny Lee and Alex Olwal and David Kim and Ram Iyengar and Na Li and Ruofei Du },
  journal={arXiv preprint arXiv:2312.09672},
  year={ 2025 }
}