MARE: Multi-Agents Collaboration Framework for Requirements Engineering

Leverage collaboration among large language models (LLMs) throughout the entire requirement engineering (RE) process.

Mare (image by Mourad Saadi)

Requirements engineering (RE) is a critical phase in project development across various domains, serving as the foundation for translating stakeholders’ needs into comprehensive requirements specifications. RE is characterized by discussions and brainstorming to define the scope, functionality, and quality of a product. Traditionally, requirements engineers as well as stakeholders actively iteratively collaborate with each other to establish a common understanding of the envisioned product. This RE process is challenging because it usually consists of multiple tasks.

Solution

Nowadays, deep learning (DL) techniques have been successfully applied to automate various RE tasks [8]. For example, DL has been used to mine stakeholders’ needs [9], extract requirements model [11], and deal with ambiguity in requirements [12]. Fantechi et al. used ChatGPT to detect inconsistency in natural language requirements [13], and Rodriguez et al. provided prompt strategies for requirements traceability [14].

While recent advancements in DL techniques have enhanced specific RE tasks, generating high-quality requirements remains a complex challenge due to the inherently collaborative nature of the process. The RE tasks need to be interwoven and iterated. The automation of only a few of these tasks limits further increase in effectiveness and efficiency.

To address this challenge, we investigated the possibility of automating the end-to-end RE process from a novel perspective by applying a generative approach. We found MARE (Multi-Agent collaboration for Requirements Engineering) to be such a promising collaborative framework [6].

MARE systematically divides the RE process into four essential tasks:

• Elicitation
• Modeling
• Verification
• Specification

MARE seeks to fully automate the RE process by fostering collaboration among various tasks and roles. Starting with a vague set of initial requirements, MARE independently and repeatedly carries out the fundamental RE tasks to ultimately produce comprehensive end-to-end requirements specifications.

• Elicitation: Given a rough idea of requirements X, this task collects stakeholders’ needs U and generates a requirements draft D.
• Modeling: Based on the generated requirements draft D, this task generates a requirements model M as required by the ‘Metamodel’.
• Verification: The task detects requirements smells S in the requirements draft D and requirements model M with ‘accept criteria’.
• Specification: If requirements smells S don’t exist, this task generates a requirement specification R based on the requirements draft D, and requirements model M by following a ‘Template’. Otherwise, this task generates an Error report E.

Each task is accomplished with at least one specific agent, and MARE contains five agents:

• stakeholders
• collector
• modeler
• checker
• documenter

Overview of MARE (image by author)

Each agent is responsible for a requirements task and performs predefined actions to accomplish that task. Jin et al. [6] identified nine actions designed for this purpose. The role definition of agents are:

Agents’ role definitions (image by author)

MARE enhances teamwork by providing a purpose-built collaborative environment. This shared digital space allows agents to upload and store interim requirements documents they’ve created and easily access and retrieve essential information. The key aspects of this shared workspace include centralized storage for all generated artifacts, access to needed information, version control to track changes and real-time updates to ensure all agents work with the most current information. This collaborative platform streamlines the requirements engineering process by facilitating smooth information exchange and cooperation among agents.

Agents act on artifacts shared on work space (image by author)

The input and output artifacts for each action are as follows:

Action triggered output generation (image by author)

This is how the action migration diagram in MARE looks like:

Action migration diagram in MARE (image by author)

The communication framework is built upon stored artifacts. These agents engage in an iterative and cooperative process to generate high-quality requirements specifications, mirroring the approach of a human requirements engineering team. Through repeated cycles of collaboration and refinement, the agents collectively navigate the requirements engineering journey, producing deliverables that adhere to established quality benchmarks.

Outlook

With the rapid advancements in LLMs, the field of RE is poised for exciting transformations. MARE’s ability to streamline and improve every phase of RE, from elicitation to specification is an exhaustive approach.

While each new iteration of open and closed-source LLMs brings increased parameters and improved capabilities, we question if this will significantly improve MARE’s current output status. To maximize the benefits of LLMs in RE, shifting the focus toward leveraging proprietary corporate data is key. Fine-tuning LLMs with this data can customize the model’s output to align closely with specific organizational needs. Additionally, integrating MARE with Retriever-Augmented Generation (RAG) applications can provide a powerful means of deriving requirements based on established standards, such as ISO standards. This approach ensures that the solutions are not only precise but also compliant with industry benchmarks.

By orchestrating agents that are finely tuned to your unique data and requirements, organizations can achieve highly tailored solutions that address their specific RE challenges. This individualized approach not only enhances the relevance and accuracy of RE but also positions companies to leverage the full potential of LLMs for innovative and effective RE processes.

Are you interested in the topic of RE as well? I look forward to connecting with you!

References

• 1. Alves, A. P. S., Kalinowski, M., Giray, G., Mendez, D., Lavesson, N., Azevedo, K., … & Gorschek, T. 9 (2023). Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an 10 International Survey. In International Conference on Product-Focused Software Process 11 Improvement (pp. 159‒174). Springer Nature Switzerland.
• 2. Arora, C., J. Grundy, and M. Abdelrazek, “Advancing requirements engineering through generative AI: assessing the role of llms,” CoRR, vol. abs/2310.13976, 2023.
• 3. Bertram, V., Boß, M., Kusmenko, E., Nachmann, I. H., Rumpe, B., Trotta, D., & Wachtmeister, L. 16 (2022). Technical report on neural language models and few-shot learning for systematic 17 requirements processing in mdse. arXiv preprint arXiv:2211.09084.
• 4. Dehn, S., Jacobs, G., Zerwas, T., Berroth, J., Hötter, M., Korten, M., … & Fleischer, D. (2023). On 28 identifying possible artificial intelligence applications in requirements engineering processes. 29 Forschung im Ingenieurwesen, 87(1), 497‒506.
• 5. Feng, J., W. Miao, H. Zheng, Y. Huang, J. Li, Z. Wang, T. Su, B. Gu, G. Pu, M. Yang, and J. He, “FREPA: an automated and formal approach to requirement modeling and analysis in aircraft control domain,” in 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pp. 1376–1386.
• 6. Jin, D., Jin, Z., Chen, X., & Wang, C. (2024). MARE: Multi-Agents Collaboration Framework for 1 Requirements Engineering. arXiv preprint arXiv:2405.03256.
• 7. Mu, F., L. Shi, W. Zhou, Y. Zhang, and H. Zhao, “NERO: A text-based tool for content annotation and detection of smells in feature requests,” in 28th IEEE International Requirements Engineering Conference, 2020, pp. 400–403.
• 8. Naumcheva,M., “Deep learning models in software requirements engineering,” CoRR, vol. abs/2105.07771, 2021. [Online]. Available: https://arxiv.org/abs/2105.07771
• 9. Shi, L., C. Chen, Q. Wang, and B. W. Boehm, “Automatically detecting feature requests from development emails by leveraging semantic sequence mining,” Requir. Eng., vol. 26, no. 2, pp. 255–271, 2021. [Online]. Available: https://doi.org/10.1007/s00766-020-00344-y
• 10. Tikayat Ray, A., Cole, B. F., Pinon Fischer, O. J., Bhat, A. P., White, R. T., & Mavris, D. N. (2023). Agile 26 Methodology for the Standardization of Engineering Requirements Using Large Language Models. 27 Systems, 11(7), 352.
• 11. Wang,C., L. Hou, and X. Chen, “Extracting requirements models from natural-language document for embedded systems,” in 30th IEEE International Requirements Engineering Conference Workshops, 2022, pp. 18–21.
• 12. Ezzini,S., S. Abualhaija, C. Arora, M. Sabetzadeh, and L. C. Briand, “Using domain-specific corpora for improved handling of ambiguity in requirements,” in 43rd IEEE/ACM International Conference on Software Engineering, 2021, pp. 1485–1497
• 13. Fantechi,A., S. Gnesi, L. C. Passaro, and L. Semini, “Inconsistency detection in natural language requirements using chatgpt: a preliminary evaluation,” in 31st IEEE International Requirements Engineering Conference Workshop, 2023, pp. 335–340.
• 14. Rodriguez,A. D., K. R. Dearstyne, and J. Cleland-Huang, “Prompts matter: Insights and strategies for prompt engineering in automated software traceability,” in 31st IEEE International Requirements Engineering Conference Workshops, 2023, pp. 455–464.
• 15. AI assistant monitors teamwork to promote effective collaboration | MIT News | Massachusetts Institute of Technology