Policy vs Reality: How GDPR Audits Actually Work

Most compliance vendors sell policy verification. Written assertions, signed boxes, attested controls, an annual review against a checklist. None of that is what Integritetsskyddsmyndigheten (IMY) actually examines when it opens a tillsyn against a Swedish controller. The gap between what compliance vendors verify and what the regulator inspects is the entire reason fines keep landing on companies that already passed an audit.

This article explains the gap. Then it explains how a behavioral audit closes it.

The audit nobody runs

Every company in IMY's tracker-and-consent enforcement cluster had a privacy policy. They had cookie banners. They had been audited — internally, externally, or both. They were still fined a combined 85,000,000 SEK across five final IMY decisions: Apoteket (IMY-2022-3270, 29 Aug 2024) and Apohem (IMY-2022-3272, 29 Aug 2024) for Meta Pixel transmissions of health-related browsing; Avanza (DI-2021-5544, 24 Jun 2024) for a Meta Pixel security failure (Art 5(1)(f) + Art 32(1)); Bonnier News / Dagens Industri (DI-2022-2178, 26 Jun 2023) for profiling customers and visitors for direct marketing without valid consent (NOT a Meta Pixel case); and Tele2 (DI-2022-2175, 30 Jun 2023) for Google Analytics third-country transfers without adequate Schrems II safeguards.

Why did the audits not catch what the regulator caught? Because the audits never tested the live website. They reviewed documents — policies, vendor contracts, banner configurations, screenshot evidence packages — and signed off when the documentation looked complete. The Pixel still ran. The reject button still failed. The browser still talked to facebook.com/tr before consent. None of that was in the document review.

The same gap exists in nearly every compliance product on the market. Vendors describe what should happen. The browser does what it actually does. When those two diverge, the regulator only cares about the second one.

What IMY actually does in a tillsyn

Read any of the five Meta Pixel decisions and the evidence pattern is identical: the authority loaded the live site in a browser, observed outbound network traffic, and matched the requests against the controller's claimed consent state. When requests to facebook.com/tr fired before any consent action — or after the user clicked reject — the violation was demonstrated by reproducible network logs.

This is observation, not document review. The regulator does not read your DPO's annual statement. The regulator does not audit your CMP vendor's security questionnaire. The regulator opens DevTools. The browser tells the truth, and the truth is what gets cited in the decision.

The technique scales beyond Meta Pixel. The Trygg-Hansa breach (DI-2021-1905, decided 2023-08-28, 35,000,000 SEK) was demonstrated by direct URL manipulation — IMY found that approximately 650,000 customers' sensitive data sat behind an IDOR vulnerability that any authenticated visitor could trigger by editing the URL. No internal access required. The Region Stockholm 1177 decision (2021-06-07, 500,000 SEK) involved approximately 2.7 million recorded patient calls sitting on an unsecured internet-accessible storage server — found by following external pointers to public infrastructure. The Spotify access-rights decision (DI-2019-6696, 2023-06-12, 58,000,000 SEK) was built on filing actual Article 15 requests and observing what came back.

In every case, the regulator gathered evidence the same way an external attacker would. No privileged access. No insider knowledge. Just the public surface, observed methodically.

Three things policy can't predict

There are three categories of compliance failure that no document review can catch, because the failure lives in runtime behavior the document does not describe.

Whether your reject button actually rejects

Two of the five cases (Apoteket, Apohem) were Meta Pixel Article 6(1)(a) consent failures involving health data. The cookie banner existed. The reject button was visible. But the reject signal did not propagate to the Pixel — either because the CMP did not block third-party scripts before consent, or because the reject handler did not actually unset the consent flag the Pixel checked, or because the Pixel was loaded outside the CMP's tag manager scope. Avanza was a separate Meta Pixel security failure (Art 5(1)(f) + Art 32(1)) — accidentally activated sub-functions transmitted financial data without adequate technical measures. Bonnier News was profiling for direct marketing, telemarketing and direct mail without valid consent (Art 6(1)(a)), not a pixel case. Tele2's case was Google Analytics under Art 44 — distinct from the Meta Pixel cluster.

A policy review cannot detect this. A banner screenshot cannot detect this. The only thing that detects it is opening the live site, clicking reject, and observing what happens to outbound network traffic afterwards. Either the request to facebook.com/tr stops, or it doesn't. Document review has no opinion on the answer.

What your tracker is configured to send

Avanza is the textbook case. The tillsyn path was different from the rest of the cluster: Art 32 + Art 33, ~500,000 customers, financial data, fine 15M SEK. The banner was not the issue. The Pixel was misconfigured to transmit financial browsing data — amounts, product searches — that should never have left the controller. The leak ran from November 2020 to June 2021 before being noticed, and the company did not file a breach notification at the time, which compounded the fine under Art 33.

No policy review catches this. The Pixel was, on paper, the same Pixel every other site uses. The custom data fields it transmitted were the problem — and those are only visible by inspecting the actual payload of an actual outbound request, decoded, with the controller's data classification scheme on hand. That is a behavioral test, not a paperwork test.

What runs after the page loads that your CMP didn't catalog

Modern websites load scripts the CMP never inventoried. Common sources: server-side tagging that proxies through a first-party subdomain (so the CMP sees only the controller's own domain and lets it through); CNAME-cloaked tracking endpoints that resolve to third-party infrastructure but appear as first-party DNS records; late-arriving scripts injected by A/B testing tools or marketing automation platforms that bypass the CMP queue; third-party embeds (chat widgets, video players, social embeds) that load their own dependencies recursively.

Every one of these is a real-world finding pattern in the IMY enforcement record. The CMP does not block them because the CMP does not see them. The compliance audit does not flag them because the audit does not enumerate them. Only network-level observation of the live site catches what is actually executing.

Why behavioral evidence is asymmetric

When the regulator and the vendor have access to the same evidence — the live, public website — the vendor selling policy attestations has zero edge. The regulator can reproduce any claim by loading the site themselves. If the vendor's evidence is a signed PDF and the regulator's evidence is a HAR file showing the violation, the HAR file wins.

This is what makes behavioral testing structurally different from policy verification. Behavioral evidence:

• Is reproducible. Anyone with a browser can run the same test and get the same result.
• Is falsifiable. If the test passes today and fails tomorrow, that is a real regression, not a documentation drift.
• Survives adversarial inspection. The regulator cannot dismiss a network log as 'your interpretation' — the bytes are the bytes.
• Tracks runtime drift. Marketing tag deployments, CDN configuration changes, third-party script updates, and consent vendor regressions all show up as observable behavior changes.

The browser tells the truth. The privacy policy describes intent. When those two disagree, only one of them is admissible evidence in an IMY decision.

The asymmetry runs the other way too. A controller that produces behavioral evidence — signed scan reports, HAR files, before/after cookie diffs — has evidence the regulator cannot easily challenge. Documentation can be disputed as inadequate, ambiguous, or self-serving. Reproducible runtime evidence is just data.

What the Vakteye mechanism does differently

Vakteye is an audit, not a policy generator. The mechanism is concrete: the scanner opens a real Chromium browser pointed at the customer's live site. It loads the page. It records every outbound network request to a HAR 1.2 file (forensic-quality, replayable). It detects the consent banner and clicks reject. It waits. It re-records the network state. It diffs cookies before reject, after reject, and after a forced page reload to test for zombie cookies that respawn after withdrawal.

Every finding is mapped to (a) the GDPR or ePrivacy article cited by the violation pattern and (b) the matching Swedish enforcement precedent — the actual IMY decision that establishes what the regulator considers the violation worth. When the scanner finds a Pixel firing pre-consent, the report cites Art 6(1)(a), LEK 9 kap. §28, ePrivacy Art 5(3), and the Apoteket/Apohem/Bonnier/Tele2 decisions as precedent. When the scanner finds an exposed admin endpoint or an IDOR pattern, the report cites Art 32 and Trygg-Hansa.

Where a proof package is issued, the report can include signature metadata and a SHA-256 manifest for bundled evidence. If a regulator opens an inspection later, the customer has organized, article-mapped, dated behavioral evidence ready for counsel and compliance review.

Three audit modes the policy world doesn't offer

Once the audit is behavioral, three operational modes become possible that policy attestation simply cannot deliver.

Pre-audit: run documented behavioral checks before a formal review

If a future review centers on a HAR file showing your Pixel firing before consent, you want comparable runtime evidence in your hands first. Pre-audit mode runs documented behavioral checks on your schedule and surfaces observable defects before a formal tillsyn opens.

Post-deployment: catch the consent regression introduced by last week's marketing tag deploy

Compliance does not stay compliant. Marketing teams deploy new tags. Vendors push silent updates to embedded widgets. CDN configurations change. Third-party scripts add dependencies. Each of these can introduce a consent regression that was not present at the last annual audit. Behavioral testing catches the regression the day it ships, not 11 months later when the regulator notices.

Cross-reference: when your DPO says 'we don't use Meta Pixel anywhere,' the scanner says 'we found it on the checkout page'

Internal documentation drifts from reality. Vendors get added, decommissioned, and re-added under different names. A scan that enumerates every third-party domain making outbound requests on every key page produces a definitive answer to 'what is actually running' — and surfaces the gap when the answer differs from what the data inventory claims. That gap is itself a finding the regulator will care about: an inaccurate Article 30 record of processing activities.

Bottom line

The gap between policy and reality is structural. Policy attestations describe intent. Behavior reveals reality. IMY only cares about reality, because reality is what the public sees, and the public is who the regulation protects.

Every fined company in the Meta Pixel cluster had documentation. The Trygg-Hansa breach happened to a company that was insurance-regulated and presumably audited. Region Stockholm had ISO frameworks. None of that paperwork stopped the fine, because none of it matched what the live system actually did.

The companies best prepared for the next review are the ones that audit live behavior, not only documents. That is not a guarantee of outcome; it is what the IMY enforcement record rewards when read end to end. Five Meta Pixel decisions, one Trygg-Hansa, one 1177, one Spotify. Eight final decisions. Same evidence pattern in every one.

If your compliance program produces only documents, it is verifying the wrong thing. Add behavioral evidence to the audit and the asymmetry improves: now you have the runtime facts first.