The Complete Agent Stack · Part 08 of 14 · Distribution layer

This guide is one part of a 14-part series on what a production data agent needs, from the data model it reads to the agent that keeps it accurate. Each part stands on its own, and they build on each other in order.

← Previous: Part 07: MCP Servers for Data Agents · Next: Part 09: AI Report Builder →

Part 08 opens the distribution layer: getting agent answers in front of the people who need them.

Key Takeaways

• Chat history is a poor filing system: In a Nielsen Norman Group study of eight people using ChatGPT, participants repeatedly went back to earlier answers to build their next prompt, a behaviour the researchers named "apple picking".

• Screenshots strip out meaning: A review of 83 dashboards by Sarikaya and colleagues notes that BI users often paste dashboard screenshots into slides just to add the context the dashboard lacks.

• AI charts still need checking: On the VisEval benchmark, the best model tested produced a chart that rendered and matched the request for 75.27% of queries, so roughly one chart in four needed correction.

• Sharing needs two modes: A snapshot shows exactly what the author saw, while a live view should re-run under each viewer's own permissions, which is the principle of least privilege applied to dashboards.

The 45-second version:

What Is an AI-Generated Dashboard?

An AI-generated dashboard is a collection of charts that a data agent produced in response to natural-language questions, saved and arranged so other people can return to them. Conversational BI covers the first half of that loop. Someone asks "what is our total spend trend over the last 12 months?", and the agent writes the query, runs it and draws the chart. The dashboard is the second half, where those answers stop being chat messages and become shared, durable assets.

Traditional dashboards are designed up front. An analyst decides which metrics matter, builds the views and publishes them, so the questions a dashboard can answer are fixed when it is built. An AI-generated dashboard is assembled the other way round, from answers that have already proved useful in conversation.

Dashboards serve many purposes. Sarikaya et al. coded a corpus of 83 dashboards into a design space of 15 visual and functional aspects, with types ranging from operational monitoring to communication and learning. Charts built from conversation can serve any of those purposes, provided they keep enough context to be trusted later.

Why Do Chat Answers Disappear?

Chat is a good way to ask questions and a poor way to keep answers. Every response lands in a linear thread, newer messages push older ones out of view, and a useful chart from Monday is buried under twenty follow-ups by Thursday.

The Nielsen Norman Group observed this directly. In 90-minute sessions with ChatGPT, participants kept referencing earlier responses to build their next prompt, and the researchers identified the endlessly scrolling chat window as the source of the friction. Three problems follow for analytics teams.

• Answers can't be revisited. A chart that exists only inside a conversation has no stable address. Finding it again means remembering which chat it was in and scrolling until it turns up.

• Screenshots lose the question. The usual workaround is a screenshot pasted into a slide or a message. Sarikaya and colleagues found that BI users take screenshots of dashboards and annotate them in slides, which suggests dashboards often capture the data without its meaning. A screenshot of an AI-generated chart loses even more: the question, the definitions the agent used and the follow-ups that shaped the result.

• Teams start over. When a colleague wants to extend an analysis, there is nothing to extend. They ask the question again in their own chat, get a slightly different chart, and the organisation ends up with two versions of the same metric.

In each case the chart has been separated from its reasoning. Keeping the two together is the core design problem for AI-generated dashboards.

What Should a Saved AI Chart Keep?

Visual analytics research has a name for the history behind an analysis: provenance. Ragan et al. organised provenance into five types of information (data, visualisation, interaction, insight and rationale) and six purposes for capturing it: recall, replication, action recovery, collaborative communication, presentation and meta-analysis. Their framework works well as a checklist for what a saved AI chart should carry.

The question and the conversation

The natural-language question and the turns that refined it are the chart's rationale. Keeping them lets the author recall why the chart looks the way it does. It also helps a colleague understand the state of the analysis, which Ragan et al. describe as establishing common ground among analysts.

The query and the data it ran against

An AI-generated chart is the output of a generated query, and generated queries are sometimes wrong. VisEval, a benchmark of 2,524 natural-language queries across 146 databases, found that GPT-4 produced charts that rendered and met the request 75.27% of the time. Another 21.44% of its outputs rendered but did not meet the query, for example by selecting the wrong columns or plotting inaccurate legends. A chart that links back to its query and conversation can be checked and corrected. A screenshot cannot.

The permissions it ran under

Every chart was produced for a particular person with particular access. If that context is lost, sharing the chart can quietly expose data that later viewers were never meant to see. A saved chart should record whose permissions produced it, so the system can decide what to show each viewer.

A way to continue

The most valuable property is that the chart can be picked up again. Reopening a saved chart should restore its conversation, so the next question, such as "break this down by supplier", starts where the last one ended.

Governed definitions make all of this more reliable. If the agent computes "spend" from a shared semantic layer or data model, two people extending the same chart get comparable numbers instead of two private interpretations.

How Do You Turn Chat Answers Into a Shared Dashboard?

A four-step workflow has emerged for conversational BI. Each step is simple on its own, and the value comes from the links between them.

1. Favourite the answers worth keeping

Saving should take a single action at the moment a chart proves useful, and the saved item should keep its full conversation. A personal list of favourites becomes the raw material for every dashboard that follows.

2. Arrange favourites on a canvas

A canvas is a drag-and-drop board where saved charts can be positioned, resized and grouped. Because favourites accumulate over weeks of analysis, one canvas can combine charts from many separate conversations. Versioning matters here, because when someone rearranges a shared board, earlier layouts should remain recoverable.

3. Share with the right audience

Sharing with an organisation, with named colleagues or through a public link are separate decisions with separate risks. The next section covers how to choose between them.

4. Fork instead of asking again

Forking is borrowed from software development. In GitHub's definition, a fork is a separate repository that starts as a copy of another and stays connected to it. Applied to analysis, forking a chart copies the conversation behind it into the viewer's own thread. The viewer can drill further without changing the original, then add the new chart back to the shared canvas. A dashboard built this way becomes a working space that several people extend, instead of a page one person publishes.

This pattern matters most for products that embed analytics for their customers. When end users can save and share their own dashboards inside the product, the vendor no longer has to anticipate every report in advance. Our guides to white-label analytics and self-service analytics cover that shift in more depth.

Snapshot or Live: How Should You Share an AI Dashboard?

The hardest question in sharing an AI-generated dashboard is which numbers the viewer should see: the ones the author saw, or the ones the viewer is allowed to see. Both answers are legitimate, so a good system offers both and makes the difference obvious.

A snapshot freezes the data at the moment each chart was created. It suits a record of a decision, a weekly review or anything sent outside the organisation, because it never queries the database again. The trade-off is that a snapshot passes on whatever the author could see. If the author has wider access than the viewer, the snapshot shares that wider view.

A live view re-runs each chart's query when it is opened, under the viewer's own permissions. NIST describes least privilege as granting each entity the minimum system resources and authorisations it needs, and a live view applies that rule chart by chart. A regional buyer opening a live view of a company-wide spend chart should see only the suppliers in their own region. That only works if row-level security is enforced outside the agent, as described in our guide to row-level security for AI agents.

Public links need extra care. The W3C Technical Architecture Group's guidance on capability URLs points out that anyone holding such a link gains access, that URLs are not generally kept secret, and that they leak through browser history, referrer headers, shared screens and search crawlers. A public dashboard link should therefore expose a read-only snapshot and never a live connection to the database.

How Upsolve Approaches AI-Generated Dashboards

BI Canvases is Upsolve's implementation of this workflow. Any chart the agent makes can be favourited, and each favourite links back to the conversation that produced it. Favourites go onto drag-and-drop canvases that can be saved in versions, shared with an organisation or published as a read-only snapshot. Canvases also work inside the embedded iframe, so a SaaS product's own end users can save and share their dashboards without leaving the product.

The demo uses a sample procurement project with two sample users, Grace Tan and Carlos Ruiz, each opened through a preview link of the embedded agent. Grace asks which suppliers make up the biggest share of spend and gets a top-10 bar chart. Carlos asks for the total spend trend and favourites the 12-month chart. Carlos then reopens an older 24-month favourite, sees when it was asked and how the conversation went, and could continue it with a new question. Finally, Carlos creates a canvas called Trends, adds both charts and shares it with the organisation.

When Grace opens the shared canvas, the chat history behind Carlos's chart is visible. Grace forks that conversation, asks for a breakdown by supplier, favourites the result and adds it to the shared canvas. After rearranging and resizing the layout, Grace saves a new version, and Carlos sees the updated canvas on the next visit.

The demo also shows the snapshot and live distinction. Snapshot view shows what the creator saw when each chart was made. Live view re-runs every chart under the viewer's permissions, so Carlos sees only the suppliers that Carlos's access allows. Public links are snapshots and never query the database live. One decision stays with the author, as Serguei Balanovich, Upsolve, explained in the demo:

"Any charts [you add] to shared canvases or shared favourites are going to share snapshots of that data with users who might not normally have access to it."

Canvases are also where the next part of the series starts. Any canvas can be rebuilt as a branded HTML app, covered in Part 09 on branded AI reports. For the full feature list, see the BI Canvases feature page.

Frequently Asked Questions

What is an AI-generated dashboard?

An AI-generated dashboard is a set of charts created by a data agent from natural-language questions, then saved and arranged on a shared board. Unlike a traditional dashboard, which an analyst designs before anyone uses it, an AI-generated dashboard is assembled from answers that have already proved useful. The best implementations keep the conversation behind every chart, so anyone viewing the dashboard can see the original question, check how the number was produced and ask a follow-up from the same starting point.

What is conversational BI?

Conversational BI is business intelligence driven by dialogue instead of menus and drag-and-drop builders. A user asks a question in plain language, a data agent translates it into a query against governed data, and the answer comes back as a number, a table or a chart. Follow-up questions refine the result. Conversational BI becomes far more useful when its answers can be saved, shared and arranged into dashboards, because otherwise every insight stays trapped in one person's chat history.

How do I save charts from an AI chat?

Look for a tool that lets you favourite a chart in one action and keeps the full conversation attached to it. A saved chart should store the question, the query, the time it ran and the permissions it ran under. That way you can reopen it later, continue the analysis and place it on a dashboard. Screenshots are a weak substitute, because they lose the question, the definitions behind the numbers and any way to check or extend the result.

Is it safe to share an AI-generated dashboard publicly?

It can be, if the public link exposes only a read-only snapshot. The W3C's guidance on capability URLs warns that shareable links leak through browser history, referrer headers and search crawlers, so a public link should never hold a live connection to your database. Before publishing, check what the snapshot contains. A snapshot shows everything the author could see, which may include data some recipients would not normally have access to.

What is the difference between a snapshot and a live dashboard?

A snapshot freezes each chart's data at the moment it was created, so every viewer sees exactly what the author saw. A live dashboard re-runs each chart's query when it is opened, filtered to what the current viewer is allowed to see. Snapshots suit records, reviews and external sharing. Live views suit ongoing monitoring inside an organisation, provided row-level security is enforced on the server so each viewer receives only their own rows.

Can customers build their own dashboards inside an embedded analytics product?

Yes, if the analytics layer supports favourites and canvases inside the embedded view. In that model, each end user asks questions of the embedded agent, saves the charts that matter and arranges them on personal or shared canvases, all within the host product. Shared canvases then need the same snapshot and live controls as internal dashboards, so one customer's users never see another customer's data through a live view.

Sources

1. Nielsen Norman Group: Accordion Editing and Apple Picking: Early Generative-AI User Behaviors. https://www.nngroup.com/articles/accordion-editing-apple-picking/

2. Sarikaya, Correll, Bartram, Tory and Fisher: What Do We Talk About When We Talk About Dashboards? IEEE Transactions on Visualization and Computer Graphics (InfoVis). https://tableau.com/sites/default/files/2023-01/DashboardsConspiracy_final.pdf

3. Ragan, Endert, Sanyal and Chen: Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes. IEEE Transactions on Visualization and Computer Graphics (VAST). https://faculty.cc.gatech.edu/~aendert3/resources/ragan-vast-2015.pdf

4. Chen, Zhang, Xu et al.: VisEval: A Benchmark for Data Visualization in the Era of Large Language Models. IEEE Transactions on Visualization and Computer Graphics (VIS). https://arxiv.org/html/2407.00981v2

5. GitHub Docs: About forks. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks

6. NIST Computer Security Resource Center: Glossary, least privilege. https://csrc.nist.gov/glossary/term/least_privilege

7. W3C Technical Architecture Group: Good Practices for Capability URLs. https://www.w3.org/2001/tag/doc/capability-urls/

Turn your agent's best answers into dashboards your team shares. Start free at upsolve.ai with 2,000 free AI credits, or talk to our team about embedding BI Canvases in your product. Next in the series: Part 09: AI Report Builder.