Making AI Agents Accountable on Shared Research Computing

The challenge is bigger than writing code

AI coding agents can generate scientific workflows, but a workflow that looks correct on a researcher’s computer may still fail on a shared cluster or research cloud. The agent needs up-to-date knowledge of the facility’s software and site rules. It also needs to know what is happening after a distributed job is submitted. And the facility needs to ensure that an agent’s actions can be traced to a real user and stay within that user’s permissions.

PegasusAI Studio, described in a new preprint, tackles this operational problem. It is a browser-based environment built around the Pegasus workflow system and HTCondor scheduler. Rather than replacing those systems, Studio connects AI agents to their existing interfaces and records. The authors’ central idea is to make infrastructure legible to agents and agent activity accountable to infrastructure.

Context, live status and guardrails

Studio gathers operational guidance in one curated, versioned knowledge base: instructions, examples, reusable skills and system documentation. Small adapters translate that shared material into the formats used by different coding agents. This is intended to keep guidance consistent when users switch tools and to let facility administrators update it centrally.

The system also makes workflows and execution sites easier to inspect. It reads a workflow generator’s command-line interface to build a user-facing parameter form, without running the generator code. During execution, a monitoring service combines information from the Pegasus workflow database and HTCondor scheduler into a structured event stream. That stream shows job states and other diagnostic details to both the researcher and the agent. When a run fails, Studio can supply the agent with relevant logs and scheduler information to help it investigate and propose a fix.

To limit risk, an agent acts inside the researcher’s environment, using that user’s identity and scheduler credentials rather than a shared agent account. Studio also has approval controls for actions that change state. In its deployed default, edits within a project workspace can proceed, but workflow submissions and writes elsewhere require user approval. The design also includes usage monitoring and makes collection of conversation text dependent on user consent.

What the case studies show

The authors describe four kinds of workflow work: modernizing legacy weather-radar analyses, reanalyzing gravitational-wave data, building an EEG analysis with an open-weight model, and reconstructing images of the black hole M87 from Event Horizon Telescope data.

In the CASA weather-workflow case, structured run information helped identify several infrastructure and configuration problems that were not clear from terminal output alone. After repairs, the modernized workflow completed all 86 jobs. In the gravitational-wave case, Studio generated a parameter form from an agent-written workflow interface. The first launch exposed a mismatch between how the form passed a list and how the workflow expected to receive it; after correction, the workflow completed 350 jobs with no failures.

For the EEG analysis, the agent and model stack differed from those used in another case. The workflow recovered from a problem packaging its software environment and completed 127 jobs. In the M87 reconstruction, two runs failed during data staging because required software images had not been built successfully. The agent diagnosed the issue, repaired the image definitions with user approval and resubmitted the workflow. The third run completed with no failures or retries.

Together, these examples support a limited conclusion: the system’s knowledge, monitoring and approval mechanisms operated in the cases the authors examined. They do not show that the AI independently validated the scientific conclusions; the paper emphasizes that researchers remain responsible for scientific intent, review and validation.

What remains uncertain

This is a systems demonstration, not a controlled test of whether AI assistance saves time or effort. The authors say they have not quantitatively compared Studio’s debugging process with other approaches. The cases were selected by the research team, and Studio has mainly been used by its developers and collaborators already familiar with Pegasus. That means the results may not reflect the experience of a broader group of researchers.

The same application was tested in a JupyterHub-based research-cloud gateway and as an Open OnDemand application on a test cluster. The latter has not yet been exercised at a production site. Although the design is intended to extend to other workflow systems, it has so far been implemented with Pegasus and HTCondor, alongside the tested gateway technologies. The authors also note that agent and model versions change, so the examples are not a benchmark of model capability.

The paper is an arXiv preprint, and its claims should be read as the authors’ reported findings rather than as independently established evidence. Its useful contribution is a concrete account of how AI agents might be connected to shared research computing without handing them unbounded access. Whether this approach makes workflows easier to operate for researchers generally remains an open question.