For most of the last decade, “AI for data science” meant autocomplete: a model that suggested the next line of pandas or finished a SQL join. That era is over. In 2025 and 2026 every major data platform shipped agents that take a goal in plain language, write a plan, generate and run code, read the errors, fix them and hand back a notebook. Databricks, Snowflake and Google have all made this a headline feature, and analyst firms now treat agentic analytics as a top data and analytics trend.
For data science leaders, the question is no longer whether agents can write useful code. They clearly can. The harder questions are which parts of the workflow should be delegated, how to verify what an agent produced, and how to change team practices so that faster output does not become faster mistakes. This article explains what has changed, what the evidence says about agent reliability, and how to adopt data science agents in a way that improves both speed and rigor.
What Changed: From Code Completion to Goal-Driven Agents
The difference between a copilot and an agent is the loop. A copilot responds to one request at a time and leaves execution to you. An agent receives an objective, decomposes it into steps, executes code against real data, inspects the results and decides what to do next. Several product launches made this loop mainstream:
- Databricks Assistant Data Science Agent (announced September 2025) added an Agent Mode inside notebooks and the SQL editor. According to Databricks, it can run exploratory analysis on a referenced table, train and evaluate models with MLflow, and repeatedly diagnose and fix errors. An optional planner drafts a step-by-step plan, asks clarifying questions and waits for approval before each step, and code execution requires explicit user permission.
- Databricks Genie Code (March 2026) extended the idea to broader data engineering and analytics work. Databricks described the goal, as reported by SiliconANGLE, as systems that plan and execute data tasks under human supervision, using organizational context such as query history and business definitions, with Unity Catalog as the governance boundary.
- Snowflake Cortex Code launched in February 2026 as a coding agent for the data stack. By April 2026 Snowflake reported that more than half of its customers had used it, and it was extended to external systems such as AWS Glue, Databricks and Postgres, with a VS Code extension, an Agent SDK and sandboxes in Snowsight.
- Data Science Agent in Colab, Google’s Gemini-powered agent, generates complete, runnable notebooks from a description of the analysis and an uploaded file, from library imports to visualizations and models.
The common thread is that the agent works inside the governed data platform, not in a separate chat window. That matters: the agent sees real schemas, real permissions and real execution errors, which is what lets it iterate rather than guess.
What the Evidence Says About Agent Reliability
Vendor demos show agents completing clean, well-specified tasks. Real analytical work is rarely clean. The most useful public evidence comes from DABStep, the Data Agent Benchmark for Multi-step Reasoning released by Adyen and Hugging Face. It contains more than 450 analysis tasks based on realistic financial operations data. Easy tasks use a single structured dataset; hard tasks require combining several datasets with domain documentation, and generally cannot be solved by one-shot code generation.
When the benchmark launched, the best reasoning agent tested reached only 16 percent accuracy on the hard tasks, according to the Hugging Face announcement. Google later reported that the Colab agent ranked fourth on the DABStep leaderboard at the time of its post. Models have improved since, but the lesson holds: agents handle the mechanics of analysis far better than the judgment of analysis. They are strong at loading data, writing transformations, producing charts and fitting standard models. They are weaker when the correct answer depends on business rules buried in documentation, on subtle join semantics, or on noticing that a question is ill-posed.
Data science agents are excellent at producing analysis and unreliable at deciding whether the analysis is right. Design your workflow so that speed comes from the agent and correctness comes from explicit, human-owned checks.
Where Agents Add the Most Value in the Data Science Lifecycle
Not every step of a data science project benefits equally. Mapping agent strengths to the lifecycle helps teams decide where to start.
High value, lower risk
- Exploratory data analysis. Profiling columns, spotting nulls and outliers, plotting distributions and summarizing relationships is repetitive and easy to verify visually. Agents compress hours into minutes here.
- Boilerplate and scaffolding. Feature pipelines, train-test splits, cross-validation loops, MLflow logging and evaluation reports follow well-known patterns.
- Debugging. Reading stack traces, fixing type mismatches and resolving library version issues is exactly the iterate-and-retry loop agents are built for.
- Documentation. Turning a finished notebook into a readable narrative, docstrings or a model card draft.
Moderate value, needs review
- Baseline modeling. Agents can quickly produce a gradient-boosted baseline and a few alternatives. Reviewers must still check for target leakage, inappropriate splits for time-dependent data and metric choices that do not match the business decision.
- Feature engineering. Agents propose plausible features but cannot know which ones are available at prediction time in production unless you tell them.
Low value or high risk without strong context
- Problem framing. Deciding what to predict, for whom and at what decision point remains a human responsibility.
- Metric definitions. If “active customer” or “net revenue” has a specific definition, an agent without access to a governed semantic layer will improvise one.
- Causal claims. Agents readily produce correlations and will describe them in causal language unless constrained.
A Governance Model for Agentic Data Science
Because agents execute code against real data, governance has to cover actions, not just outputs. A practical model has four layers.
1. Permissions follow the human
The agent should act with the invoking user’s identity and inherit their access policies in the catalog. Avoid service accounts with broad read access “to make the agent work.” Platforms such as Unity Catalog and Snowflake’s role model already support this; the risk is that teams bypass them during pilots.
2. Approval gates for side effects
Reading data and running analysis in a sandbox is low risk. Writing tables, registering models, scheduling jobs or calling external APIs should require explicit approval. Databricks’ own design, with per-run, per-thread or always-allow choices, is a useful pattern; organizations should decide which of those options are permitted for production workspaces.
3. Context as a governed asset
Agent quality depends heavily on context: table descriptions, column comments, certified metric definitions, glossary terms and example queries. Treat these as maintained assets with owners, not as optional metadata. The fastest way to improve agent accuracy is often to document the twenty tables the team uses most.
4. Traceability
Keep the agent’s plan, prompts, generated code and execution logs alongside the notebook. When a result is challenged in three months, reviewers need to reconstruct how it was produced. Version control for notebooks and MLflow experiment tracking already provide most of the plumbing.
A Step-by-Step Adoption Playbook
Teams that get value from agents tend to follow a staged rollout rather than switching the feature on for everyone.
- Pick two or three bounded use cases. Good candidates are recurring EDA on new data drops, baseline model refreshes and notebook debugging. Avoid high-stakes regulatory models in the first phase.
- Curate the context. Add descriptions and comments to the core tables, publish certified metric definitions and write a short “house style” instruction file covering libraries, naming conventions, validation splits and plotting standards.
- Define review checklists. For each use case, write the five to ten checks a reviewer must perform before an agent-produced result is used (see the checklist below).
- Run a side-by-side pilot. For four to six weeks, have practitioners solve real tasks with and without the agent. Record time spent, number of review corrections and any errors that reached stakeholders.
- Decide policy from evidence. Based on the pilot, set approval defaults, which workspaces allow agents, and which tasks require a second reviewer.
- Scale with training. Teach practitioners to write good objectives, to ask the agent to explain its plan before executing, and to request alternative approaches when results look surprising.
- Measure continuously. Track review rework and downstream incidents, not just usage. Usage alone rewards speed and hides quality problems.
Reviewer checklist for agent-generated analysis
- Was the correct table and version used, and are filters consistent with the question?
- Are joins at the right grain, with no unintended row multiplication?
- Do metric definitions match the certified definitions?
- For predictive models: is the split appropriate (time-based where needed), and are all features available at prediction time?
- Are evaluation metrics aligned with the business decision and cost of errors?
- Are claims phrased as associations unless a causal design supports them?
- Can the result be reproduced from the logged code?
Worked Example: Churn Analysis With an Agent in the Loop
Consider a mid-sized subscription business that wants to understand churn drivers before a pricing change. A data scientist opens a notebook and asks the agent to analyze churn over the last eighteen months using the subscriptions and usage tables, and to propose a baseline churn model.
With planning enabled, the agent first proposes a plan: profile both tables, define churn as non-renewal within 30 days of term end, join usage aggregates at the account-month level, explore drivers, then train a baseline classifier. The data scientist corrects one assumption before execution: the company’s certified churn definition excludes accounts moved to a partner reseller. Because that rule is in the glossary but not in the table comments, the agent would otherwise have missed it.
The agent then runs EDA, fixes a timezone mismatch that caused duplicate monthly rows, and produces charts showing churn concentrated among accounts with falling weekly active users. It trains a gradient-boosted model with a random split and reports strong AUC.
Review catches two problems. First, a random split leaks future behavior into training; the reviewer asks the agent to redo validation with a time-based split, and performance drops to a more realistic level. Second, one top feature, support tickets in the final month, is not known at the point when retention offers must be sent. The reviewer removes it and asks the agent to recompute. The final notebook took a fraction of the time a manual build would have, and the two review corrections are precisely the kind of judgment calls agents still miss. The team adds both rules to its house-style instructions so future runs start from better defaults.
Pitfalls to Avoid
- Treating fluent output as validated output. Clean charts and confident summaries do not mean correct joins.
- Skipping context work. Agents on undocumented tables invent definitions. Context curation is the main lever for accuracy.
- Allowing write access too early. Keep agents read-only in production until review practices are mature.
- Measuring only productivity. Track rework and incidents, or quality problems will be invisible until a stakeholder finds them.
- Deskilling junior staff. If juniors only prompt and accept, they never learn to spot leakage or bad grain. Pair agent use with explicit review training.
- Tool sprawl. Running separate agents in every notebook environment with different context sources fragments definitions. Prefer agents that read from the same governed catalog and semantic layer.
How the Data Scientist Role Changes
Agents do not eliminate the need for data scientists; they shift the scarce skill. Less time goes into writing transformation code and more into framing problems, specifying constraints, designing validation and reviewing results. In practice, senior practitioners become editors and reviewers of agent output, while the team’s collective knowledge, encoded in documentation, metric definitions and instruction files, becomes a productivity asset that compounds over time.
This also changes hiring and training. Statistical judgment, domain understanding and the ability to articulate assumptions become more valuable than fluency in a particular library. Teams should expect to spend deliberate effort on review skills, because the failure modes of agent-produced work, plausible-looking but subtly wrong, are different from the failure modes of hand-written code.
Outlook: Agents as Collaborators on a Shared Context Layer
Over the next two years, expect three developments. First, agents will become more interoperable: Snowflake already exposes Cortex Code through MCP and an Agent SDK, and other vendors are moving the same way, so a single agent may operate across several platforms. Second, evaluation of agents will move inside the enterprise, with teams maintaining their own task suites, much like DABStep but drawn from their own data and rules, to test new agent versions before rollout. Third, the semantic and governance layer will become the real competitive asset. Gartner has predicted that by 2030 universal semantic layers will be treated as critical infrastructure alongside data platforms and cybersecurity. For data science teams, that means investment in definitions, documentation and lineage pays off twice: once for human analysts and again for every agent that relies on them.
Conclusion
Data science agents are now capable enough to change how analysis is produced, and the productivity gains on routine work are real. But public benchmarks still show agents struggling on hard, multi-step problems that depend on domain rules. The winning approach is not to resist agents or to trust them blindly, but to redesign the workflow: curate context, gate side effects, standardize review and measure quality alongside speed. Teams that do this will ship more analysis with fewer errors, and they will build a reusable context layer that makes every future agent more useful.
How Hanoi Technology Can Help
Hanoi Technology helps data teams pilot and govern data science agents, from context curation and review workflows to MLOps on Databricks and Snowflake.
Further Reading
- Introducing Databricks Assistant Data Science Agent – Databricks
- DABStep: Data Agent Benchmark for Multi-step Reasoning – Hugging Face
- Data Science Agent in Colab – Google for Developers
- Databricks introduces Genie Code – SiliconANGLE
- Snowflake Intelligence and Cortex Code become the agentic AI control layer – Techzine