From Shopify webhook payload to data checks in 90 seconds
Profiling nested webhook payloads (orders/create from Shopify) and exporting Soda + JSON Schema checks — entirely in the browser.
Webhook payloads are where most profilers fall over. They are deeply nested, sparsely populated, and the shape mutates across event types. A Shopify orders/create body has line items inside line_items[], shipping addresses nested under shipping_address, and discount codes that may or may not exist depending on the channel.
1. Save a single payload as NDJSON
Grab a webhook delivery from your logs (Shopify, Stripe, Segment, whatever) and save it as a single.json or .ndjson file. One payload is enough to bootstrap — you can come back with a richer sample later and use the drift view to diff.
2. Drop it on the home page
The in-browser profiler unrolls nested arrays into per-leaf paths like line_items[].price and shipping_address.zip. Presence is computed against the right denominator (per-record for top-level, per-element for in-array fields), so a field that appears on every line item shows 100% presence even when half your orders have no line items.
3. Accept the high-confidence checks
The confidence band groups proposals into High / Medium / Low. For a webhook sample, almost everything in High is safe to keep: not_null on the IDs, allowed-values on the currency, range on the totals. The Medium band is where you should slow down and inspect — these are usually format inferences (email, ISO date) that need a human nod.
4. Export JSON Schema for the ingestion layer
For webhook validation, JSON Schema is the right target because it preserves the nested shape. Wire it into your ingress (Ajv in Node, jsonschema in Python) and quarantine payloads that fail. Use SodaCL for the warehouse-side checks after the data lands in your fact table.
5. Come back next week and diff
Save the .profile.json bundle. When you re-profile a week later, attach the saved file as a baseline — the drift panel surfaces new fields, type changes, and presence shifts. That's how you catch the silent vendor-side schema migrations that break every webhook integration in production.