Saudi deepfake campaign: build verification into products

Saudi Arabia’s SDAIA and Communications, Space and Technology Commission launched the “It’s AI” awareness campaign on October 15, 2026 to help people recognize deepfakes, protect visual and voice identity and verify content before believing or sharing it. For product and commerce teams, the operational lesson is larger than spotting strange eyes or mismatched lips: media must be treated as untrusted evidence until its source, integrity, context and business claim are checked through independent controls.

The campaign recommends awareness, labels for AI-generated content, advanced detection tools, regulation and international cooperation through initiatives such as C2PA and the Content Authenticity Initiative. Those are complementary controls, not a single “fake or real” button. C2PA can make provenance tamper-evident; it cannot decide whether the depicted event or attached claim is factually true. Detection models are probabilistic and can drift after compression, editing or model changes. High-risk actions still need authenticated channels and step-up approval.

The launch date, campaign guidance and examples below are official Saudi facts. The product architecture, evidence model, thresholds and operating controls are engineering analysis.

What the Saudi campaign says

The campaign covers visual, audio and textual deepfakes. Its public guidance says that seeing a person in a video or hearing their voice is not sufficient proof, and that views or reposts do not give content credibility. It asks people to inspect visual and audio inconsistencies, compare language and timing with the claimed source, and return to official or trusted sources.

The campaign distinguishes ordinary editing from deceptive fabrication. Improving lighting or cropping a scene does not by itself change the truth; fabricated media creates an event or statement that did not happen and presents it as real. It lists fraud, identity theft, reputational harm, verification bypass and misinformation among the risks.

That distinction matters to software. A product should not label every edited asset “fake,” and it should not accept a realistic clip as authorization. The decision must be tied to what the media is being used to prove.

Start from the business claim

The same file can be low risk in a public gallery and high risk when used to unlock money, identity or administrative access. Policy should therefore evaluate asset evidence and requested action separately.

A verification pipeline for product teams

### 1. Quarantine and preserve the original

Store the received bytes before transcoding, resizing, stripping metadata or generating thumbnails. Assign an internal asset ID and calculate a cryptographic hash. Record the authenticated account, source channel, receipt time, declared creator, linked order or case and retention policy.

Parse media in an isolated worker with size, duration, codec and resource limits. A malformed file is a security problem even before deepfake analysis. Derived previews should point back to the immutable original rather than replacing it.

### 2. Verify Content Credentials without flattening the result

C2PA Content Credentials are cryptographically bound provenance records. A conforming validator examines the manifest, assertions, ingredients, signature, trust chain, timestamps or revocation information and the binding between the manifest and asset.

Keep distinct outcomes:

• absent: no credential was found;
• unsupported: the format or feature cannot be evaluated by this validator;
• invalid: the manifest, signature or asset binding failed validation;
• valid but untrusted: the data is structurally valid but the signer is not anchored in an accepted trust list;
• trusted: validation succeeded and the signer chains to an accepted trust anchor.

Do not convert “absent” into “fake.” Many legitimate files have no Content Credentials, and platforms can remove metadata. Do not convert “trusted” into “the claim is true.” C2PA verifies that signed provenance is associated with the asset and has not been silently altered; the consumer still decides whether to trust the signer and the factual claim.

### 3. Recover context outside the file

Check whether the content exists on the claimed organization’s official channel. Compare the earliest known publication, caption, time, location and surrounding event. Reverse search can reveal older uses or a real image paired with a false caption. When soft bindings or a remote manifest are supported, resolve them through an allowlisted service and preserve the resolution evidence.

For commerce evidence, compare against authoritative systems: order events, shipment scans, payment records, support history and account ownership. Media may support a claim but should not overwrite transactional facts.

### 4. Run detectors as measured signals

Visual, audio and text detectors can contribute scores, localization maps or anomaly labels. Store the model name, version, threshold, input transformation and calibration set beside every result. A score without this context cannot be audited later.

Route by uncertainty instead of pretending the detector is an oracle:

• low-risk content with no adverse signals can continue through ordinary moderation;
• conflicting provenance or detector signals require human review;
• high-risk requested actions require independent authentication regardless of the score;
• invalid credentials, impersonation indicators or coordinated abuse should open an investigation.

Test detectors on Arabic content, Saudi accents, screen recordings, forwarded messaging-app files, aggressive compression and legitimate edits. A threshold validated on studio-quality English video is not evidence for real support traffic.

### 5. Apply action policy and human review

The policy engine should receive a normalized evidence record, not raw vendor scores. It combines business action, account trust, provenance state, source checks, detector results, prior abuse and transaction context. It returns a bounded outcome such as allow publication, request stronger proof, hold for review, deny the requested action or open an incident.

Human reviewers need the original asset, derived previews, provenance chain, source comparison, detector limitations and the exact decision being requested. Hide unnecessary personal data and log every access and decision.

A minimal evidence record

{
  "asset_id": "media_01...",
  "original_sha256": "...",
  "received_at": "2026-10-28T17:00:00Z",
  "source_channel": "authenticated_support_upload",
  "subject_ref": "order_or_case_id",
  "claimed_creator": "merchant_account_id",
  "c2pa": {
    "status": "trusted|valid_untrusted|invalid|absent|unsupported",
    "signer": "...",
    "trust_anchor": "...",
    "assertions": []
  },
  "detectors": [
    {"model": "...", "version": "...", "score": 0.0, "threshold": 0.0}
  ],
  "source_checks": [],
  "requested_action": "change_payout_account",
  "decision": "step_up_and_separate_approval",
  "reviewer": null
}

The schema should contain references rather than duplicate sensitive material. Encrypt originals, use short retention where evidence is no longer needed, separate review permission from ordinary support access and make deletion propagate to derivatives and indexes.

Example: a merchant payout-change video

Suppose a marketplace receives a video of a merchant owner asking to replace the settlement bank account. The video looks natural and has a valid Content Credential from a known camera. That proves useful provenance about the file; it does not prove that the person is currently authorized, that the request is voluntary or that the destination account belongs to the merchant.

A safe workflow is:

1. hold the payout change and preserve the original video;
2. require sign-in from an existing privileged account with step-up authentication;
3. confirm the new bank account through the regulated payment or banking verification path;
4. require a second authorized approver or a cooling-off period;
5. notify existing trusted contacts through a separate channel;
6. log the evidence and release the change only after all controls agree.

No detector score or realistic voice should bypass these controls. Media verification reduces uncertainty; it does not replace authorization.

Sign and label outgoing content

Product teams should also improve the supply side. Sign official images, video and audio at creation or publication, include truthful AI-use and edit assertions, protect signing keys in managed hardware or a dedicated signing service, rotate and revoke credentials and keep unsigned originals out of the publishing path.

Display Content Credentials in a way users can inspect, not as a vague green badge. Show who signed, what changed, whether validation succeeded and what the credential does not prove. If an export, CDN or social workflow strips manifests, test soft binding or provide an official verification page with the original asset hash.

Evaluation and monitoring

Measure the complete decision system:

• credential coverage on official outbound media;
• validation results split by absent, unsupported, invalid, valid-untrusted and trusted;
• false-accept and false-reject rates by media type, language, compression and channel;
• human-review agreement and reversal rate;
• time to resolve high-risk holds;
• percentage of financial or admin requests blocked from media-only authorization;
• detector and policy version used for each historical decision;
• deletion completion for originals and derivatives.

Replay a fixed, rights-cleared evaluation set after changing a detector, media pipeline, trust list or threshold. Shadow a new model before it can change outcomes, and keep a rollback path to the previous policy version.

Best practices

• Preserve originals before any transformation.
• Separate provenance validation, source verification, detection and authorization.
• Treat C2PA status as structured evidence, not a truth score.
• Bind media to an authenticated account, case and business event.
• Require independent controls for money, identity and administrative actions.
• Calibrate detectors on Arabic and real delivery channels.
• Sign and label official outbound media with protected keys.
• Record uncertainty and provide an appeal or review path.

Anti-patterns

• Accepting a face or voice as sufficient authorization.
• Declaring content fake because credentials are absent.
• Declaring a claim true because a credential is trusted.
• Using one detector threshold for every language, codec and risk level.
• Transcoding before preserving the original and its metadata.
• Uploading sensitive customer media to unapproved public verification tools.
• Keeping biometric-like media indefinitely “for future AI training.”
• Exposing detector internals as a public exploit oracle.

Limits

Visual cues such as blinking, lighting or lip synchronization can help people pause, but generation quality improves and legitimate compression can produce similar artifacts. Provenance can be removed, incomplete or signed by an entity the consumer does not trust. Detectors can be evaded and can fail on groups or languages absent from calibration data. Human review also makes mistakes.

The defensible design is layered: preserve evidence, validate provenance, verify the claimed source, measure detectors, bind decisions to business risk and require strong authorization for consequential actions.

Executive takeaway

The “It’s AI” campaign puts verification ahead of belief and sharing. Product teams should turn that principle into an evidence pipeline. Content Credentials, detectors and human review each answer different questions; none authorizes a payout, account or identity change alone. Build the controls around the claim and requested action, preserve uncertainty and make every decision auditable.

Continue with the AI, RAG & Vector Search hub, Applied AI service, Saudi Data Regulatory Sandbox readiness, OpenAI textGrain provenance guide and Secure RAG ingestion.

Primary references

• CST — SDAIA and CST launch the “It’s AI” campaign
• CST — Deepfake and fabricated clips awareness guide
• C2PA — Content Credentials technical specification 2.5
• C2PA — Content Credentials explainer
• NIST AI 100-4 — Reducing Risks Posed by Synthetic Content
• Wikimedia Commons — Deepfake.jpg
• Creative Commons Attribution-ShareAlike 2.0

Prepared by: Noor Yasser

Prepared by: Noor Yasser