Docs: Add client-side media processing documentation by adamsilverstein · Pull Request #75895 · WordPress/gutenberg

adamsilverstein · 2026-02-25T10:16:32Z

Summary

Adds comprehensive documentation for the Client-Side Media Processing feature shipping in WordPress 7.1. The underlying infrastructure landed during the 7.0 cycle when the feature graduated from a Gutenberg experiment; the 7.1 release adds HEIC support, AVIF end-to-end uploads, performance improvements, and finishes hardening before the public release.

Architecture doc (docs/explanations/architecture/client-side-media-architecture.md): How the system works - the three-package split (@wordpress/upload-media, @wordpress/vips, @wordpress/media-utils), upload queue and operation chaining, WebAssembly/libvips pipeline, Cross-Origin Isolation and SharedArrayBuffer requirements, browser feature detection, and graceful fallback to server-side processing.
How-to guide (docs/how-to-guides/client-side-media.md): Practical guide for plugin/theme developers - disabling the feature, customizing image processing via existing WordPress filters, handling compatibility with metadata hooks, and debugging.
Editor filters reference (docs/reference-guides/filters/editor-filters.md): New "Client-side media processing" section documenting the wp_client_side_media_processing_enabled PHP filter, the generate_sub_sizes and convert_format REST API parameters, and the existing core filters that the client-side pipeline respects (big_image_size_threshold, image_editor_output_format, image_save_progressive, wp_image_maybe_exif_rotate).
Diagrams (docs/assets/client-side-media-*.svg): Four SVG illustrations covering the upload flow, architecture overview, processing pipeline, and GIF-to-video conversion.
Manifest and TOC updates: docs/manifest.json and docs/toc.json regenerated to surface the new pages in the handbook.

Closes #75111

Test plan

Verify documentation renders correctly in the handbook
Review technical accuracy of architecture explanations
Confirm code examples are correct and follow WordPress coding standards
Check all internal cross-references and links resolve

Document the upcoming capability detection areas including credentialless iframe support, device memory, network conditions, and Web Worker checks. Notes that browsers without credentialless support will have client-side media disabled by default, with a server-side filter opt-in.

Architecture explanation, developer how-to guide, editor filters reference, and make/core announcement draft for the client-side media processing feature shipping in WP 7.0.

- Fix wasm-vips links to point to canonical kleisauke/wasm-vips repo - Add wp_ prefix to filter name in feature-detection docblock - Use wp.apiFetch instead of hardcoded /wp-json/ path in sideload example Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Document the finalize REST endpoint, editor.media.imageQuality JS filter, and server-side hook compatibility added in PR #74913 across architecture, how-to, and Make Core post docs. Also fixes a typo ("eprocessing") in the Make Core post.

github-actions · 2026-03-01T05:33:38Z

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: juanmaguitar <juanmaguitar@git.wordpress.org>
Co-authored-by: aaronjorbin <jorbin@git.wordpress.org>
Co-authored-by: westonruter <westonruter@git.wordpress.org>
Co-authored-by: andrewserong <andrewserong@git.wordpress.org>
Co-authored-by: swissspidy <swissspidy@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

- Replace COEP/COOP cross-origin isolation with Document-Isolation-Policy (Chromium 137+ only; Firefox/Safari not supported) - Remove outdated "not yet wired through" note on imageQuality hook - Add client_side_supported_mime_types PHP filter documentation - Add replace_file parameter to sideload endpoint docs - Update browser compatibility table and troubleshooting section - Update feature detection table to match current implementation

Adopt trunk's feature-detection docblock listing — it accurately reflects implemented checks (hardware concurrency). Drop the credentialless-iframe bullet since no such check exists in the code.

github-actions · 2026-04-23T22:53:20Z

Flaky tests detected in e9a2ab9.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/29544340374
📝 Reported issues:

[Flaky Test] should paste block styles using keyboard #78759 in /test/e2e/specs/editor/various/list-view.spec.js

The feature shipping target moved from 7.0 to 7.1 (per #76756). Bring the architecture explanation and how-to guide up to date with the post-7.0-cycle state of the code: - Reposition introductions around 7.1 with 7.0 as the groundwork cycle. - HEIC/HEIF: document the canvas-based fallback path (createImageBitmap, HTMLImageElement+OffscreenCanvas, HEIC container parsing + WebCodecs VideoDecoder), JPEG companion file, and isHeicCanvasSupported() gate. - AVIF: note the wp_prevent_unsupported_mime_type_uploads bypass when generate_sub_sizes=false (#76371). - Batch thumbnail generation via image.copyMemory() / thumbnailImage() (#76979). - Sub-size deduplication via image_size: string|string[] on the sideload route (#77036). - Single VIPS instance via promise caching (#76780). - Build-output trimming: remove vips-jxl.wasm (#76639), skip non-min worker (#76615), skip WASM-inlined sourcemaps (#75993). - COI: <img> excluded from cross-origin attribute injection (#76618). - Loosened feature-detection thresholds: 2 cores, 3g (#76616). - Post-saving lock fix: Save Draft now respects the lock (#76973). - convert_format declared as boolean on sideload route (#77565). - Fix incorrect REST index settings list (only image_sizes and image_size_threshold are exposed). - Remove references to client_side_supported_mime_types filter (PR #76549 was closed unmerged); state the supported MIME set is fixed at CLIENT_SIDE_SUPPORTED_MIME_TYPES. Refs #75111.

The feature ships in 7.1 instead of 7.0 (per #76756). Reframe the post as a 7.1 announcement with a dedicated 'What's new in 7.1' section listing HEIC support, end-to-end AVIF uploads, batch thumbnail generation, sub-size deduplication, single VIPS instance, save lock fix, loosened device requirements, <img> COI exclusion, smaller build output, and the convert_format boolean coercion fix. Each item is linked to its merged PR. Move the original 7.0 cycle deliverables under 'What changed during 7.0' so the historical context stays available for the announcement audience. Update technical overview to use the actual function name (gutenberg_is_client_side_media_processing_enabled), reflect that DIP replaces COEP/COOP, and remove the inaccurate note that the imageQuality hook isn't wired up. Drop the make-believe client_side_supported_mime_types filter; PR #76549 was closed unmerged. State that the supported MIME set is fixed at CLIENT_SIDE_SUPPORTED_MIME_TYPES. Update browser compatibility table to reflect the DIP-based gating (Chromium 137+, no Firefox/Safari) and add a feature detection thresholds section.

The handbook generator derives slugs from filename basenames, so docs/how-to-guides/client-side-media.md and docs/explanations/architecture/client-side-media.md collided on the slug "client-side-media" and broke docs:gen in CI. Rename the architecture file to client-side-media-architecture.md and update all internal cross-references plus the public-URL references in the Make/Core draft post.

Run docs:build to add manifest entries for the new how-to guide and architecture explanation pages, fixing the check-local-changes CI step.

Adds a Frequently Asked Questions section addressing common public misconceptions raised on issue #76756 — bandwidth-vs-CPU/RAM framing, the 'never trust the client' security objection, fallback behavior, hook compatibility, format conversion, and browser support.

GitHub user-attachments URLs can break over time in published WP docs. Store the diagram images in docs/assets and reference them via raw.githubusercontent HEAD URLs, matching the existing docs convention.

adamsilverstein · 2026-07-07T16:01:25Z

Good call, thanks! Moved the diagrams into docs/assets and switched the references to raw.githubusercontent.com/.../HEAD/docs/assets/... URLs, matching the existing convention in the docs. Fixed in faa8582.

andrewserong

Thanks for the updates, this is reading well to me and we can always polish in follow-ups 👍

The goal of client-side media is to provide the highest quality media processing to all WordPress sites.

Nice opening sentence 🎉

westonruter · 2026-07-07T23:44:48Z

Question: Are SVGs for the assets available? This would be better, if possible.

adamsilverstein · 2026-07-07T23:50:41Z

Question: Are SVGs for the assets available? This would be better, if possible.

good suggestion, I'll work on adding those.

Replace the four PNG diagrams with hand-authored SVGs so they stay crisp at any zoom, carry selectable text, and weigh less. Content, layout, and styling match the originals; markdown references updated to the .svg files.

adamsilverstein · 2026-07-08T00:01:45Z

Good call, @westonruter. I've replaced all four PNGs with hand-authored SVGs in b938d7f - crisp at any zoom, real selectable text, and smaller files. Content, layout, and styling match the originals, and the markdown references now point at the .svg files (raw.githubusercontent serves them as image/svg+xml, so they render inline).

The REST API panel's heading and endpoint lines set light fills via presentation attributes (fill="#ffffff"/"#e8eaed"), but the .ct and .body classes set dark fills in the <style> block. Stylesheet class rules outrank presentation attributes in the CSS cascade, so the dark fills won and the text rendered nearly invisible on the #1e1e1e panel. Move the light fills into .ct-inv/.body-inv classes declared after .ct/.body so they win the cascade, matching how the sibling diagrams use fill-setting classes (.pill/.num) for text on dark backgrounds.

Summarize the findings of the hooks audit in #80210: which hooks fire on each REST request of a client-side upload, which filters now fire at settings-computation time instead of during encoding, the three hooks that never fire on the client path and their replacement signals, and how EXIF rotation ownership moves to the client. Also document the image_strip_meta and image_max_bit_depth REST index fields and their customization examples (#80216, honored client-side via #80218).

adamsilverstein · 2026-07-13T21:38:42Z

Added hooks documentation based on the audit in #80210 (faf7a74):

Architecture doc: new "Hook firing: client path vs server path" section - which hooks fire on each REST request (create / per-file sideload / finalize), which filters now fire at settings-computation time instead of during encoding, the three hooks that never fire on the client path (wp_image_editors, image_make_intermediate_size, image_memory_limit) with replacement signals, and the EXIF rotation handoff. This covers the documentation-only items (3-5) of Some server-side image-processing hooks never fire on the client upload path #80216.
How-to guide: per-request hook summary under "Server-side plugin compatibility", "Hooks that no longer apply" guidance, customization examples for image_strip_meta and image_max_bit_depth, and a troubleshooting row.
Filters reference: added image_strip_meta / image_max_bit_depth and a note on the never-firing hooks.

Note: the image_strip_meta / image_max_bit_depth bits describe the behavior added in #80218 (items 1-2 of #80216), which should land before or alongside this PR.

swissspidy · 2026-07-15T07:51:55Z

@adamsilverstein This is amazing! Thank you for putting all of this together.

adamsilverstein · 2026-07-15T23:20:54Z

@adamsilverstein This is amazing! Thank you for putting all of this together.

Thanks! Now that we hit Beta1 and not much is likely to change, I will merge it soon. We can always update anything we change during beta. I also have a draft make post I will share with you separately.

andrewserong · 2026-07-15T23:46:53Z

This is amazing! Thank you for putting all of this together.

Couldn't agree more!

adamsilverstein · 2026-07-16T23:50:29Z

I'm going to give this a final review then get it merged. We can always update the documentation as we land new items.

PR #80072 made the editor block switch user-initiated via a "Display as video" toolbar control on the Image block, replacing the automatic swap-on-upload. Update the architecture doc, how-to guide, filters reference, and pipeline diagram to match, and remove references to the "Display as original GIF" toggle and preserveAnimatedGif attribute, which no longer exist.

Drop the WASM MIME-type troubleshooting row: the WASM binaries are inlined as base64 data URLs at build time (as the architecture doc states), so no separate .wasm fetch exists and a server MIME type can never be the cause. Reword the low-spec device row since the detection gates are not configurable. Join the architecture index bullet to the existing list so it renders as one tight list.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org> Co-authored-by: juanmaguitar <juanmaguitar@git.wordpress.org> Co-authored-by: aaronjorbin <jorbin@git.wordpress.org> Co-authored-by: westonruter <westonruter@git.wordpress.org> Co-authored-by: andrewserong <andrewserong@git.wordpress.org> Co-authored-by: swissspidy <swissspidy@git.wordpress.org>

adamsilverstein and others added 3 commits February 24, 2026 21:07

Add client-side media processing documentation

b1f05e9

Architecture explanation, developer how-to guide, editor filters reference, and make/core announcement draft for the client-side media processing feature shipping in WP 7.0.

adamsilverstein self-assigned this Feb 26, 2026

adamsilverstein added [Type] Task Issues or PRs that have been broken down into an individual action to take [Feature] Client Side Media Media processing in the browser with WASM labels Feb 26, 2026

adamsilverstein added this to WordPress 7.0 Editor Tasks Feb 26, 2026

github-project-automation Bot moved this to 🔎 Needs Review in WordPress 7.0 Editor Tasks Feb 26, 2026

adamsilverstein moved this from 🔎 Needs Review to 🏗️ In Progress in WordPress 7.0 Editor Tasks Feb 26, 2026

adamsilverstein added the [Status] In Progress Tracking issues with work in progress label Feb 26, 2026

adamsilverstein added 2 commits February 28, 2026 17:35

fine tine docs 1

7e529cc

adamsilverstein marked this pull request as ready for review March 1, 2026 05:33

adamsilverstein requested review from ajitbohra, fabiankaegy and juanmaguitar as code owners March 1, 2026 05:33

adamsilverstein mentioned this pull request Mar 2, 2026

Client Side Media iteration for WordPress 7.0 #74333

Closed

t-hamano mentioned this pull request Mar 2, 2026

7.0 Dev Note Tracking #75815

Closed

14 tasks

adamsilverstein mentioned this pull request Mar 23, 2026

Client Side Media iteration for WordPress 7.1: stabilization, docs, and Core backports #76756

Open

Merge branch 'trunk' into 75111-documentation

77f3b7e

Adopt trunk's feature-detection docblock listing — it accurately reflects implemented checks (hardware concurrency). Drop the credentialless-iframe bullet since no such check exists in the code.

juanmaguitar requested changes Apr 24, 2026

View reviewed changes

Comment thread docs/contributors/client-side-media-make-core-post.md Outdated

adamsilverstein and others added 6 commits April 28, 2026 14:07

Merge branch 'trunk' into 75111-documentation

b6054e2

Docs: Regenerate manifest for client-side media docs

d6b565c

Run docs:build to add manifest entries for the new how-to guide and architecture explanation pages, fixing the check-local-changes CI step.

adamsilverstein added 3 commits July 7, 2026 08:46

Clean up; emphasize the why in intro

5d72fe2

more cleanup

4e1e2af

Docs: Move diagrams into docs/assets

faa8582

GitHub user-attachments URLs can break over time in published WP docs. Store the diagram images in docs/assets and reference them via raw.githubusercontent HEAD URLs, matching the existing docs convention.

adamsilverstein requested a review from andrewserong July 7, 2026 16:02

andrewserong approved these changes Jul 7, 2026

View reviewed changes

Docs: Recreate client-side media diagrams as SVG

b938d7f

Replace the four PNG diagrams with hand-authored SVGs so they stay crisp at any zoom, carry selectable text, and weigh less. Content, layout, and styling match the originals; markdown references updated to the .svg files.

westonruter reviewed Jul 8, 2026

View reviewed changes

Comment thread docs/assets/client-side-media-architecture-overview.svg

adamsilverstein added 2 commits July 7, 2026 23:28

Merge remote-tracking branch 'origin/trunk' into 75111-documentation

8590110

Merge remote-tracking branch 'origin/trunk' into 75111-documentation

35aca8a

adamsilverstein added 2 commits July 16, 2026 17:16

adamsilverstein merged commit 038ebbd into trunk Jul 17, 2026
55 of 57 checks passed

adamsilverstein deleted the 75111-documentation branch July 17, 2026 02:36

github-actions Bot added this to the Gutenberg 23.7 milestone Jul 17, 2026

adamsilverstein mentioned this pull request Jul 17, 2026

Media: Add an opt-in filter to generate animated image sub-sizes #80385

Open

Uh oh!

Conversation

adamsilverstein commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

github-actions Bot commented Mar 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

adamsilverstein commented Jul 7, 2026

Uh oh!

andrewserong left a comment

Choose a reason for hiding this comment

Uh oh!

westonruter commented Jul 7, 2026

Uh oh!

adamsilverstein commented Jul 7, 2026

Uh oh!

adamsilverstein commented Jul 8, 2026

Uh oh!

Uh oh!

adamsilverstein commented Jul 13, 2026

Uh oh!

swissspidy commented Jul 15, 2026

Uh oh!

adamsilverstein commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewserong commented Jul 15, 2026

Uh oh!

adamsilverstein commented Jul 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

adamsilverstein commented Feb 25, 2026 •

edited

Loading

github-actions Bot commented Mar 1, 2026 •

edited

Loading

github-actions Bot commented Apr 23, 2026 •

edited

Loading

adamsilverstein commented Jul 15, 2026 •

edited

Loading