Shopify's webhook delivery is reliable. That's not the problem.
The problem is what happens after the webhook lands on your side.
Shopify fires orders/create. Your Flask endpoint receives it. Returns 200. Shopify marks it delivered. And somewhere between that 200 and the database write, something silently fails. Order never recorded. No error thrown. No alert fired. Customer emails you 3 hours later asking where their order confirmation is.
That's the webhook processing gap — and it's invisible to every standard monitoring tool because nothing failed by definition.
This is the Flask boilerplate I built to close it.
What this monitors
- Order created — real-time alert with amount, items, customer
- Order paid — payment confirmation with amount
- Order cancelled — immediate warning with cancellation reason
- Refund issued — instant alert with refund amount
- Checkout started — abandoned checkout signal
Every event sends actual numeric amounts in the meta object. NotiLens ML reads these to learn your revenue baseline automatically — your normal Wednesday order volume, your typical refund rate, your checkout-to-order conversion pattern. When reality diverges from baseline, the anomaly alert fires. No thresholds to configure.
The broader concept
This boilerplate is Shopify-specific but the pattern applies to any webhook source — Stripe, GitHub, custom systems, internal services.
The concept is simple:
- Receive the webhook
- Verify the signature — confirms it's genuinely from the source
- Send a notification immediately — you know it arrived and was processed
- Send actual business metrics in the meta — enables ML baseline learning
- Use
force_send=Truefor critical events — bypasses ML, fires immediately regardless The gap this closes: webhook delivered on the sender's side, processed on your side, NotiLens knows both happened. Iforders/createwebhooks stop arriving during business hours — silence alert fires. That's the "webhook processing silently stopped" detection that Shopify's own dashboard will never show you.
Setup
Step 1 — Install dependencies
pip install flask notilens
Step 2 — Get credentials
- Shopify webhook secret — Shopify Dashboard → Settings → Notifications → Webhooks → Signing secret
- NotiLens token + secret — app.notilens.com → Create New Topic → Token + Secret
Step 3 — Configure
SHOPIFY_WEBHOOK_SECRET = "YOUR_SHOPIFY_WEBHOOK_SECRET"
NOTILENS_TOKEN = "YOUR_NOTILENS_TOKEN"
NOTILENS_SECRET = "YOUR_NOTILENS_SECRET"
APP_NAME = "shopify-monitor"
PORT = 5000
Step 4 — Run
python notilens-shopify-webhook-monitor.py
Step 5 — Expose publicly
Local development:
ngrok http 5000
Production — deploy to Railway, Render, or any server. Your webhook endpoint:
POST https://your-domain.com/webhook/shopify
Step 6 — Register in Shopify
Shopify Dashboard → Settings → Notifications → Webhooks → Add webhook → paste your endpoint URL → select events.
The code — section by section
Signature verification
def verify_shopify_signature(payload: bytes, hmac_header: str) -> bool:
"""Verify webhook is genuinely from Shopify"""
digest = hmac.new(
SHOPIFY_WEBHOOK_SECRET.encode("utf-8"),
payload,
hashlib.sha256
).digest()
computed = base64.b64encode(digest).decode("utf-8")
return hmac.compare_digest(computed, hmac_header or "")
Every incoming webhook gets verified before processing. Shopify signs every webhook with your secret using HMAC-SHA256. If the signature doesn't match — 401, no processing, no NotiLens notification.
Never skip this. Any public endpoint without signature verification is open to spoofed webhooks.
Initialize NotiLens
nl = notilens.init(name=APP_NAME, token=NOTILENS_TOKEN, secret=NOTILENS_SECRET)
One line. The nl object is used for all notifications throughout the handler.
Order created
if topic == "orders/create":
total = price(data.get("total_price"), currency)
customer = data.get("email") or data.get("contact_email", "guest")
items = len(data.get("line_items", []))
nl.notify(
"order.created",
f"New order — {total} · {items} item(s) · {customer}",
force_send=False, # route through ML — reduces noise on high volume
meta={
"order_id": data.get("id"),
"order_number": data.get("order_number"),
"total": float(data.get("total_price", 0)),
"currency": currency,
"items": items,
"customer": customer,
"is_new": data.get("customer", {}).get("orders_count", 1) == 1,
},
tags="shopify,order",
)
force_send=False — on high volume stores, every order firing an immediate push notification becomes noise fast. With force_send=False, NotiLens routes the notification through ML. Normal order arrives — logged, no push. Anomaly detected — push fires. This is how you get alerted when order volume drops 80% from your Wednesday baseline without getting pinged on every single order.
is_new flag — tracks whether this is a first-time customer. Useful for conversion monitoring.
Refund and cancellation — force send
elif topic == "refunds/create":
nl.notify(
"refund.created",
f"Refund issued — {amt}",
level="warning",
meta={
"order_id": data.get("order_id"),
"refund_id": data.get("id"),
"amount": total_refund,
"currency": currency,
"items_refunded": len(refund_lines),
},
tags="shopify,refund",
)
No force_send flag here — defaults to True for warning level events. Refund issued and order cancelled are things you want to know about immediately, not filtered through ML. Push fires regardless of baseline.
Checkout started — abandoned checkout signal
elif topic == "checkouts/create":
nl.notify(
"checkout.created",
f"Checkout started — {total} · {customer}",
force_send=False,
meta={
"checkout_id": data.get("id"),
"total": float(data.get("total_price", 0)),
"currency": currency,
"customer": customer,
"items": len(data.get("line_items", [])),
},
tags="shopify,checkout",
)
Tracking checkouts/create alongside orders/create gives you a checkout-to-order signal. NotiLens learns the normal ratio between checkouts started and orders completed. When that ratio changes significantly — checkout volume stays the same but orders drop — that's the silent failure signal. Payment processing broke somewhere between checkout and order confirmation.
This is the broken flow detection concept applied to e-commerce — not monitoring individual events in isolation but the relationship between them.
The webhook handler — full flow
@app.route("/webhook/shopify", methods=["POST"])
def shopify_webhook():
payload = request.data
hmac_header = request.headers.get("X-Shopify-Hmac-Sha256", "")
topic = request.headers.get("X-Shopify-Topic", "")
if not verify_shopify_signature(payload, hmac_header):
return jsonify({"error": "Invalid signature"}), 401
data = request.get_json()
currency = data.get("currency", "USD")
# ... event handlers ...
return jsonify({"success": True}), 200
Always return 200 after successful processing. Returning non-200 causes Shopify to retry delivery — which creates duplicate processing if your actual handler succeeded. The 200 confirms receipt. NotiLens confirms what happened after receipt.
What the monitoring layer adds
Without this boilerplate — Shopify fires webhooks, your endpoint processes them, you find out something went wrong when a customer emails you.
With this boilerplate:
Every event processed — NotiLens logs it. You have a record of every order, payment, refund, and checkout that arrived and was processed. Not just what Shopify sent — what your backend actually handled.
Silence detection — if orders/create webhooks stop arriving during your normal business hours, NotiLens fires a silence alert. Your backend might be healthy. Your Shopify webhook registration might have expired. The endpoint URL might have changed after a deployment. NotiLens catches the gap.
Revenue anomaly detection — NotiLens ML learns your normal order volume and average order value. Sudden drop in order frequency, spike in refund volume, checkout-to-order ratio collapsing — anomaly alert fires before you check the end-of-day report.
Refund and cancellation spikes — immediate push notification. Not filtered. Not batched. Fires the moment it happens.
Adapting for other webhook sources
The same pattern works for any webhook source. Replace the Shopify signature verification with the equivalent for your source:
Stripe:
import stripe
event = stripe.Webhook.construct_event(
payload, sig_header, stripe_webhook_secret
)
GitHub:
signature = "sha256=" + hmac.new(
secret.encode(), payload, hashlib.sha256
).hexdigest()
Custom internal service:
# Use a shared secret in the request header
token = request.headers.get("X-Webhook-Token")
if token != WEBHOOK_SECRET:
return 401
The NotiLens notification pattern — nl.notify() with numeric meta values — works the same regardless of source. The ML baseline learning works on whatever numeric values you send. Order amounts, payment values, record counts, response times — all become signals NotiLens learns from.
Full boilerplate
Copy the complete script from the gist:
👉 gist.github.com/notilens/703cd96c1d08ec441f1f102d8b001249
Why this pattern matters
Shopify's webhook logs show delivery. Your server logs show receipt. Neither shows you whether the business logic that runs after receipt actually worked correctly.
That gap — between webhook delivered and business outcome confirmed — is where silent failures live. A webhook monitor that only tracks delivery is watching the wrong thing.
The pattern this boilerplate implements watches the outcome: order processed, payment recorded, refund logged, checkout started. When those outcomes stop arriving — that's the alert.
notilens.com — 7-day free trial, no credit card required.