Data Quality Framework

Trust the Numbers: Data-Quality Framework for Shopify Profit Tools

Why Every Profit Tool Guesses (and How to Know When Yours Is)

The most common complaint about Shopify profit tracking apps and dashboards is that the numbers don't match reality. This isn't a bug. It's a design choice every tool makes, usually without telling you. This guide gives you the framework to know whether any profit number you're looking at is ACTUAL data or an ESTIMATED model, and five questions to ask any vendor before trusting their margin reports.

Who this is for: $3-12M DTC founders and finance leads evaluating or currently using a profit reporting tool, and anyone who has ever looked at a margin number and thought "that can't be right."

Last updated: May 2026

TL;DR: The bottom line

  • Every profit tool estimates something: No tool has direct access to your carrier invoices, your 3PL billing files, and your supplier POs simultaneously. Where they don't have real data, they model it, using averages, blended rates, or industry benchmarks. The problem isn't that they estimate. It's that most don't tell you what they're estimating.
  • ACTUAL and ESTIMATED are two different numbers: A metric labeled ACTUAL was pulled from a verified source document. A metric labeled ESTIMATED was calculated from a model or assumption. Treating both as equivalent is how you make a bad pricing or inventory decision with high confidence.
  • The fix is labeling, not perfection: No profit tool will ever have 100% ACTUAL data. The goal isn't perfect data. It's knowing which data you can act on and which requires verification before a big decision.

The test: Can your profit tool tell you, for any given margin number, whether it's ACTUAL or ESTIMATED? If not, every number in the dashboard carries unknown confidence.

The recurring complaint: "The numbers don't match"

Read the G2 reviews for any Shopify profit tracking app used by DTC brands (TrueProfit, BeProfit, Triple Whale, Polar Analytics, Sellerboard) and you'll find the same pattern repeated across hundreds of reviews: "The numbers don't match my Shopify data," "Margins are significantly off compared to our books," "I've been using this for 6 months and still can't trust the profit number."

Based on public G2 reviews for tools like these, last checked 2026-05.

These are not edge cases from unsophisticated users. These are signals from operators running $3-15M brands who plugged in a tool, followed the setup instructions, and still got numbers they couldn't reconcile.

The reviews share a common frustration: the tool produces a confident-looking number (a clean dashboard, a margin percentage, a profit trend chart) that turns out not to reflect reality when tested against bank statements, 3PL invoices, or actual carrier costs.

The tool isn't lying. It's estimating. And the problem is that it doesn't say so.

Why every profit tool estimates at least some of the time

A complete profit number for a DTC brand requires data from multiple disconnected sources:

  • Shopify (revenue, order data, basic COGS)
  • Carrier invoices or a shipping aggregator (actual outbound shipping cost per order)
  • 3PL billing files (per-order pick/pack/fulfillment, storage, receiving)
  • Freight forwarder invoices (inbound freight and duties per PO)
  • Payment processor statements (actual processing fees per transaction)
  • Returns platform or 3PL returns data (return cost per unit)

In practice, no profit tool has live, first-party connections to all of these systems at once, especially carrier invoices and 3PL billing files, which are often PDFs or CSVs emailed monthly. Most of these systems simply don't expose reliable APIs: carrier invoices arrive weekly as PDFs, 3PL billing files are often emailed as CSVs, and freight costs live in a PO system that doesn't talk to Shopify.

So tools fill the gaps. They use the COGS you entered in Shopify (often a round number or an estimate). They use an average shipping cost you entered during setup. They use industry-average processing fees. They use a blended return rate across your whole catalog.

None of this is wrong in principle. The problem is when these estimates are presented with the same visual confidence as actual data (same font, same dashboard widget, same "profit" label) with no indication of how they were calculated or how confident you should be in them.

The result is a dashboard full of numbers that look precise but carry unknown amounts of estimation. You don't know which margins are based on real invoices and which are based on the $5 average shipping cost you entered in month one and never updated.

Actual-vs-Estimated tagging: the framework

The solution to the unknown-confidence problem is labeling. Every metric in a profit reporting system should carry one of two tags:

In practice: a green ACTUAL badge means "this number came from a real document." An amber ESTIMATED badge means "this number came from a model or manual input."

What ACTUAL means

A metric tagged ACTUAL was derived from a verified source document: a carrier invoice, a 3PL billing file, a Shopify transaction record, a payment processor statement, or a supplier PO. The specific source document and the calculation method can be traced.

ACTUAL data has three properties:

  • Source: The specific document or API that produced the number (e.g., "UPS invoice 2026-04-15, order #12345").
  • Granularity: The number applies to a specific transaction, order, or SKU, not a blended average across all orders.
  • Verification date: The number was current as of a specific date and can be refreshed when the source document updates.

ACTUAL data is the right basis for high-stakes decisions: SKU discontinuations, price changes, channel mix shifts, investor reporting.

What ESTIMATED means

A metric tagged ESTIMATED was derived from a model, assumption, or blended average rather than a specific source document. This includes:

  • Shipping costs calculated from a flat rate entered in setup rather than actual carrier invoices.
  • COGS based on a manually entered number that hasn't been updated since the last supplier PO.
  • Return cost calculated as a % of revenue using an industry average rather than actual 3PL return processing data.
  • 3PL costs entered as a monthly flat fee divided by total orders, rather than per-order billing data.
  • Royalties or influencer / affiliate commissions modeled as a flat % of revenue when actual per-order payouts are not yet integrated.

ESTIMATED data is not useless. It's often good enough for directional decisions, trend monitoring, and early-stage planning when real source data isn't yet available. But it should be treated with appropriate skepticism for high-stakes decisions, and it should be replaced with ACTUAL data as quickly as data sources become available.

The methodology trace

Every ESTIMATED metric should carry a methodology trace: a short note documenting how the number was calculated, what assumptions were made, and when it was last verified.

Example methodology trace for an estimated shipping cost metric:

// Methodology: Outbound shipping cost (ESTIMATED)
Source: Manual input (setup)
Value: $6.50 per order (flat rate)
Last verified: 2026-01-10
Actual source: Not connected (carrier invoice not integrated)
Risk: Carrier rate increases since Jan 2026 not reflected. DIM weight surcharges not captured.
Upgrade path: Connect carrier billing files via API or upload to replace this estimate with ACTUAL per-shipment cost.

A methodology trace turns an opaque number into an auditable one. It tells you not just what the number is, but how confident you should be in it and what it would take to upgrade it to ACTUAL.

What ACTUAL vs ESTIMATED looks like in practice

Most profit dashboards blur estimates and facts together. MarginOS does the opposite. It makes the status of every number obvious on the surface, then lets you open the receipts behind it. Three views work together: a status pill on every cost layer, an evidence drawer that shows the source documents, and an integration panel that shows where the evidence comes from and how fresh it is.

1. The cost stack: every layer carries a status

In the Cost Stack & Overrides panel, each cost layer for a SKU (base product, freight, inbound handling, storage, fulfillment, gateway fees) sits next to a status pill:

  • Evidence-backed means the rate is computed from real WMS / 3PL documents and billing files. Hovering the pill explains exactly how. For example: computed from your real 3PL bill, floor-smoothed on a 10-unit sample so a few odd batches can't whiplash the rate, with a link to Data & Lineage for the raw math.
  • Estimated means the number comes from an explicit model or operator input, such as inbound handling modeled at a fixed rate per carton until invoices are connected.
  • Mixed means the rate blends evidence and modeled logic, for example real 3PL documents plus a safety floor.
  • Missing flags layers that have no cost data at all yet.

On the right, the Force Math Exceptions box lets you override specific layers, such as base product cost or fulfillment cost, when you have hard evidence for a SKU that doesn't match the catalog default. Overrides are saved and treated as first-class inputs in the cost stack, not side notes in a spreadsheet.

MarginOS Cost Stack and Overrides panel: each layer shows a status pill, from Base Product Estimated and Freight Missing to Fulfillment Evidence-backed and Gateway Fees Mixed, beside a Force Math Exceptions override box

Figure 1: Every cost layer carries a status pill. Evidence-backed layers are computed from real documents, Estimated layers come from a model, Mixed layers blend both, and Missing layers have no data yet. The Force Math Exceptions box lets you override a layer when you have hard evidence for a specific SKU.

2. The evidence drawer: show me the documents

Clicking an evidence-backed layer opens the Evidence & Model drawer. For fulfillment, you see:

  • A 30-day rollup of WMS / 3PL documents (in this example, 133 units for $1,006.79 total) and the raw per-unit rate.
  • The effective rate the model is using (here, a sample floored to 133 units for $7.57 fulfillment per unit), with the smoothing rule stated in plain language.
  • A table of source documents: external IDs, source system (here, ShipStation), dates, amounts, and units for every label in the sample window.

This is the show-your-work layer. A founder, CFO, or auditor can see exactly which documents produced the number on screen and how the model turned them into a stable per-unit rate.

MarginOS Fulfillment Cost Evidence and Model drawer: a 30-day rollup of 133 units for $1,006.79, an effective modeled rate of $7.57 per unit, and a table of ShipStation source documents with external IDs, dates, amounts, and units

Figure 2: The evidence drawer for fulfillment. A 30-day rollup (133 units, $1,006.79) becomes a modeled $7.57 per unit, and every source document in the sample is listed by external ID, date, amount, and units. This is the number and the receipts on the same screen.

3. The integration: where the evidence starts

The integration panel (here, ShipStation via Direct API) shows where the evidence comes from and how fresh it is:

  • Connection status, API key, and the last successful sync timestamp.
  • Registered live events, such as real-time capture on every new shipping label, so new costs land in minutes rather than at month-end.
  • A clear list of the data being synced, such as shipping and label costs: the real carrier charge and insurance on every label, attributed down to each SKU.
MarginOS ShipStation Direct API integration panel: Connected status with API key and last sync time, a registered real-time cost-capture event, and a Data We Sync list showing shipping and label costs attributed to each SKU

Figure 3: The integration panel shows connection status, last sync time, the live events that push new costs in real time, and exactly which data is synced. This is where the evidence chain starts, with read-only access that never modifies your ShipStation orders or settings.

Together, these three views turn "we think your fulfillment cost is about $7.57" into a traceable statement: here is the API that brings in every label, here are the raw ShipStation documents behind the number, here is how they were smoothed into a per-unit rate, and here is the badge that tells you this layer is evidence-backed, not guessed.

The CFO defense angle: when data quality becomes a liability

For brands that are raising capital, preparing for an acquisition, or bringing on a CFO, the ACTUAL vs ESTIMATED distinction stops being a dashboard preference and becomes a due diligence issue.

An acquirer or investor performing financial due diligence will want to reconcile your reported margins against your actual cost documents. If your profit tool has been using estimated shipping costs and estimated 3PL costs for 18 months, the gap between reported margin and actual margin may be material, and it will surface during diligence.

The more common and immediate version of this problem: you hire a fractional CFO or finance director, hand them access to the profit dashboard, and they ask where the numbers come from. If the answer is "some of it is real and some of it is estimates we set up two years ago," you've handed them a month of reconciliation work before they can do anything strategic.

Actual-vs-Estimated tagging is also a protection against internal decisions made with false confidence. A category manager who makes a SKU discontinuation decision based on an ESTIMATED margin that turns out to be 8 points too optimistic has made a different decision than if they'd known the number was estimated. Labeling doesn't slow decisions down. It makes them better calibrated.

A simple test: if your profit tool cannot show an auditor, on screen, the source document and methodology for a single order's margin calculation, assume every rolled-up margin chart will require manual reconciliation.

5 questions to ask any profit tool vendor

Before trusting any profit tool's margin numbers (including MarginOS), run it through these five questions. A vendor who can't answer them clearly is telling you something important about the confidence level of their data.

  1. For each cost component (shipping, 3PL, COGS), what is the specific data source you pull from, and how often does it sync?

    The right answer names a specific API, file format, or integration, not "we connect to Shopify." Shopify doesn't have carrier invoices or 3PL billing data. If the answer for shipping is "the shipping cost from the Shopify order," that's an estimated number based on what the customer paid for shipping, which may not equal what you paid the carrier.

  2. When you don't have real data for a cost component, what do you use instead, and how is that labeled in the UI?

    Every tool makes assumptions when real data isn't available. The question is whether those assumptions are visible. If the answer is "we use a default you can customize," ask: how does a user know whether they're looking at the default or real data?

  3. Can you show me the methodology trace for a specific metric (e.g., what produced the outbound shipping cost for order #12345)?

    This is the drill-down test. If a tool can't trace any specific metric back to a source document or clearly named assumption, it can't be audited. That means you can't catch errors, and neither can anyone reviewing your financials.

  4. What's the gap, on average, between your estimated and actual numbers for shipping and 3PL costs, for a typical brand at our volume and SKU count?

    A vendor who knows their product can answer this. "Our estimated shipping cost is within 8% of actual for brands using average box sizes" is an honest, useful answer. "Our numbers are very accurate" is not.

  5. If I enter a COGS in Shopify that turns out to be wrong, how do I know which historical margins to recalculate?

    This tests data lineage. If COGS is updated in Shopify, does the profit tool retroactively recalculate margins for all affected orders, or does historical data remain stale? If historical data stays stale with no flag, every trend analysis you've done may be comparing apples to oranges.

How MarginOS implements Actual-vs-Estimated tagging

MarginOS is built on the premise that showing a confident margin number without labeling its confidence level is more dangerous than showing an accurate margin number with an ESTIMATED tag. Every metric in the platform carries an explicit tag, a data source citation, and a methodology trace accessible in the UI.

ACTUAL data sources in MarginOS

  • Shopify: Revenue and order data via Shopify API. ACTUAL, synced on a frequent schedule.
  • Payment processors (Shopify Payments and others): Actual processing fees per transaction from statements or APIs. ACTUAL once connected and ingested for the relevant period.
  • Carrier costs (direct or via shipping platforms): UPS, FedEx, USPS billing files or shipping-platform label data parsed per order to produce per-shipment actual cost. ACTUAL, synced on a regular schedule when billing files or label data are available.
  • 3PL API / EDI: Per-order pick/pack/fulfillment fees and monthly storage allocation from 3PL billing data. ACTUAL for integrated 3PL partners.
  • Supplier POs: Landed cost per unit including duties, imported from your PO or freight forwarder file. ACTUAL once the file is uploaded.

ESTIMATED fallbacks, labeled

When a real data source isn't yet connected, MarginOS shows an ESTIMATED metric with an amber badge, the assumption being used, and a prompt to connect the real data source to upgrade it. You always know what you're looking at.

This is how the full cost-layer model becomes actionable: each of the 8 layers starts as ESTIMATED when you connect and upgrades to ACTUAL as you wire in each data source. MarginOS shows you the upgrade path, so closing the gap between estimated and actual is a project with a clear to-do list, not an abstract data quality initiative. Once you trust the numbers, the LTGP:CAC model shows you what to do with them.

Actual-vs-estimated tagging only matters once you have a cost model worth trusting and a way to use it. The cost-layer model shows true contribution margin per SKU, and the LTGP:CAC model shows how far you can push paid acquisition on top of that. This data-quality framework sits between them: it tells you which profit numbers are safe to bet the business on, and which still need verification.

Use this framework when...

  • Your profit tool's margin numbers don't match your actual bank deposits or carrier invoices.
  • You're evaluating a new profit reporting tool and need to know what to ask.
  • You're preparing for investor due diligence or a CFO hire and need to audit how solid your financial data is.
  • You're making a SKU discontinuation or price change and need to know how much to trust the margin number driving it.

Key Definitions & Metrics

ACTUAL

Sourced from a verified document or real-time API. Can be traced to a specific record.

ESTIMATED

Derived from a model, assumption, or average. Useful for direction. Verify before a high-stakes decision.

Methodology Trace

The source, assumption, and last-verified date for any metric. Turns an opaque number into an auditable one.

Vendor Evaluation Checklist

  • Names the specific data source for each cost component
  • Labels ACTUAL vs. ESTIMATED in the UI
  • Can trace any metric to a source document
  • Quantifies the gap between estimates and actuals
  • Recalculates historical data when source data changes

Get Your Profit & Inventory Command Center

See actual-vs-estimated tagging in real time, not in spreadsheets.

Get your Profit & Inventory Command Center

Or email sales@marginos.com to see actual-vs-estimated tagging on your own data.