Blog

Best Practices for Automating Data Workflows with AI

 

By Vladimir Williams

03/07/2026

 

Workflow automation is one of AI's most promising use cases, however, deploying AI in production workflows requires a lot of discipline. AI models can be used as powerful tools to take on many of the tasks involved in data workflows, but they should never replace human judgement. Potential applications include data classification, anomaly detection, report and narrative generation, synthesizing from large texts, and database query generation. Based on current recommendations and my own experience, here are some best practices for integrating AI into automated data workflows while minimizing hallucinations and maintaining quality.

 

Design Solutions Around Human Review

The most crucial principle is to design workflows so that anything handled or produced by the AI can be revised by a human. While the AI handles the brunt of the effort—the repetitive, tedious, and synthesis-heavy tasks—humans provide the context and validation. A well-designed AI SQL query generator pipeline, for example, might have four distinct stages:

  1. Schema definition to tell the AI what the database structure should look like, helping it to produce narratives and descriptions while improving the accuracy of subsequent query generations

  2. Component association to link data components mentioned in subsequent prompts to the correct tables and columns in the database

  3. Prompt construction and query generation to design a prompt to solve the problem at hand with well-defined constraints on the generated query

  4. Human review and execution to validate the query (e.g., syntax checking) before execution

 

Of course, the final human review stage is non-negotiable for high-stakes activities, ensures accuracy, and may take far less time than writing it from scratch.

 

Use AI for What It's Good At (and Nothing Else)

Play to AI's strengths and compensate for its weaknesses. AI excels at synthesizing information, cleaning up messy data, generating code templates, and identifying patterns. It's not reliable for accurate numerical calculations, accessing data it hasn't been given, or making judgement calls that require domain expertise.

 

The "garbage in, garbage out" principle is amplified in AI workflows: we cannot upload raw data to an LLM and ask it to compute the results. Instead, we should use traditional data pipelines to calculate statistical metrics, percentages, and period-over-period changes, etc., then pass that clean, deterministically-processed data to the AI to synthesize a narrative. Here, the calculations happen in the deterministic layer and the AI provides the interpretation.

 

Minimizing Hallucinations Through Structure and Context

The main challenge affecting reliability and accuracy in AI models is hallucinations. Hallucinations can broadly be categorized into data- and reasoning-driven, stemming from flawed training data and logical inconsistencies at inference time. To mitigate these risks, there are several proven strategies:

  • Lower the model temperature: First things first, drop the model temperature. This reduces randomness which is important when you want consistent, reliable outputs rather than creative text. 
  • Give explicit boundaries: Tell the model what you want and how you want it. For instance, tell it to never fabricate data, produce data from a specified distribution and in specific formats, or even to flag inconsistencies for human review.
  • Provide step-by-step guidance: Instead prompting the model with a broad question, guide it through the process to arrive at the proper output. For example, instead of "Write a report about this data", apply more detail and structure: "Use the data in the attached file to describe, for each metric, the current value, the change over time, and any notable patterns. Then, write a summary including descriptions of each metric and their behavior."
  • Ground the model with examples: Use few-shot learning techniques to show the AI what good output looks like. By providing a small number of labeled examples, the model learns to perform a task, such as classification or report generation, more reliably.

 

A Real-World Example

I recently built an AI-powered workflow automation for a client. The problem was straightforward: automate the generation of a Gantt chart using a text document as input. This automation knocked hours off of their workflows, while eliminating human error introduced by manually filling charts. One could have proposed to input the text document to an LLM and ask it directly to output a Gantt chart, however, this route would have limited the capacity for human review and modification.

Instead, I designed a pipeline which would load the text document into the LLM, using an API to extract the relevant data and export it as a JSON file, which would then be fed into a deterministic Python code. The Python code used the JSON file to generate an Excel spreadsheet formatted as a Gantt chart. In this case, the expected data structure and its rules were well-known, and so basic validations could be made by the Python code, for example, checking to make sure the input's event start dates came before their end dates. The result was a hybrid solution which joined a probabilistic AI step with a deterministic step to run basic validation on the AI-produced data and generate the XLSX file which could be reviewed and edited by a person. The final product was wrapped in a user-friendly GUI which allowed for human validation and customization.

Path to Production: Iterate and Test

When building an automated data workflow, expect multiple iterations: a crucial initial time investment is necessary to reduce human effort in the future. This time might be spent engineering prompts and uploading sample data to test the AI's response and functionality of the entire workflow. Once finished, the benefits of workflow automation will be dramatic: reducing hours of tedious labor into mere seconds.

 

The key takeaway is that AI in data workflow automation is not about removing people entirely, but rather about shifting their role from authoring to editing and providing strategic interpretation. By designing systems that leverage AI's strengths while mitigating its weaknesses through structure and human oversight, data teams can achieve both efficiency and reliability.