Skip to content

A Look at the WordPress Media Experiments Plugin in an Interview With Pascal Birchler

Hello everyone! Today, we want to welcome you to another original interview (in our series). This time, we’re thrilled to feature Pascal Birchler, a trailblazer in the WordPress community.

Pascal has been working on an innovative project – the WordPress Media Experiments plugin – designed to help achieve better image optimization within the Block Editor. We’ll get his insights into how the plugin came to be, some of the inherent challenges Pascal faced during development, and a glimpse of how the plugin operates in real-world scenarios.

Additionally, we’ll delve into whether this functionality might become a staple in the WordPress core. ⚙️

Pascal Birchler Interview.

But first, let’s dive deeper into what the WordPress Media Experiments plugin is all about and why it’s making waves. 🏄

WordPress Media Experiments: in a nutshell 🐿️

Before we give you a full rundown, let’s give you a quick overview of what the WordPress Media Experiments plugin does and how to install it.

The plugin is a way to optimize images directly within the Block Editor. It’s one area that isn’t yet possible in WordPress, and we will all usually use third-party solutions to do image optimization site-wide (for instance, via Optimole). However, this plugin carries out all the optimization behind the scenes, which is what makes it original.

The plugin can do much more than simple image optimization, though. For instance, it lets you generate thumbnails, convert GIFs to video, and much more. In fact, WordPress Media Experiments is aiming to achieve a lot, and it could be that plenty of this eventually ends up in the WordPress core.

However, there are also some caveats to note. First off, it’s an experimental plugin, which means it’s not in the WordPress plugin directory.

If you want to try it out, you’ll have to get the archive from GitHub – plus probably do a test run on a dev site instead of your production website (but you probably know that).

Click here if you're not sure how to get a GitHub plugin installed on WordPress:

Before you start, you’ll need to use PHP 8.1 or higher, as you won’t be able to activate the plugin on a lower version.

Go to the plugin’s repo at https://github.com/swissspidy/media-experiments.

The GitHub page for the WordPress Media Experiments plugin.

Download and extract the ZIP file from the dedicated GitHub page. While there’s a build process for the more stable version, we recommend the ‘nightly’ build, as you can download and install it in a simpler fashion:

The Code drop-down menu from GitHub for the WordPress Media Experiments plugin.

From here, log into your site using Secure File Transfer Protocol (SFTP), and drop the plugin folder into your wp-content/plugins directory:

Dragging and dropping the Media Experiments directory into the plugins folder within WordPress using SFTP.

You can activate the plugin in the same way as any other and begin to try it out!

👉 In case you want to learn more on how to install WordPress themes and plugins from GitHub, read this.

Now that we’ve got the installation out of the way, here’s what’s waiting for you on the inside:

Touring the WordPress Media Experiments plugin with Pascal Birchler

The WordPress Media Experiments plugin comes courtesy of Pascal Birchler.

He’s a Swiss Google engineer who contributes to WordPress in a number of ways.

Pascal Birchler, developer of WordPress Media Experiments.

For instance, he had hands-on involvement in embedding content within WordPress, the WP-CLI, and a whole host of other facets. If you don’t already know about it, his periodic table of WordPress plugins is well worth a look:

Pascal Birchler's periodic table of WordPress plugins.

As such, it’s no surprise that he also had a way to optimize media within the Block Editor near completion. In fact, this is a great place to start in our Pascal Birchler interview, so let’s do it!

On building things for future scenarios

“How did you start working on this project, and what’s the development context of the plugin?”

Pascal Birchler.

I’ve been a long-time WordPress contributor to core, translations and everything related to multilingual, including the Core Performance Team. We look at ways to improve the front end performance of WordPress for the millions of sites that use it. Image compression and formats are a part of that.

I had experience working on another WordPress plugin with similar media features. Matt Mullenweg expressed some interest about compressing the images directly in the Block Editor. I already built this! After some polish and bug fixes, I put out an explanatory video.

What Media Experiments does at its core

“How does the plugin handle the image processing and compression?”

Pascal Birchler.

In a nutshell, the plugin moves all image processing from the server to the client. When you add a new image to a post (or want to compress an existing one), all optimizations will happen in the browser before the final image reaches the server. The same goes for all the thumbnail versions of an image.

This is possible thanks to a new, custom upload queue system I’ve built. There are also ports of some open-source image libraries to WebAssembly so you can use them on the web.

Note that once you compress or optimize an image, it’s a permanent change. However, you can retrieve the original version if you change your mind.

You can find any options within the Media Experiments metabox within the Block or Site Editor:

A close up of the WordPress Media Experiments sidebar within the Block Editor.

This will change in a dynamic way based on the media you’re using or whether you’re loading an Image Block. Once you complete an image upload, you get to confirm whether you want to use an optimized version on your site:

The Compare Media Quality dialog that shows once you upload an image to WordPress.

We’ll showcase a few more of these specific screens through our interview with Pascal Birchler, so keep your eyes peeled for them.

Developing for a near-infinite number of devices, displays, and standards

“What were the most significant technical challenges faced during the development of this plugin, especially in implementing the client-side image processing feature?”

Pascal Birchler.

Most of the challenges arise from having to support so many browsers, devices, and image formats – all with their subtle differences. Ensuring that the compression works cross-browser, cross-viewport, and for a broad set of file types is challenging. It’s also very time-consuming.

Inspecting media on the front end of a site, showing the different dimensions the Media Experiments plugin generates.

Another challenge was the introduction of ‘cross-origin isolation’ to the Block Editor, as the WebAssembly image libraries require this security feature. Because the Block Editor is highly customizable and extensible through plugins, it is important that the security hardening doesn’t break any other on-site plugins.

The benefits to the end user when using image compression directly within the Block Editor

Right now, we have a good system for optimizing images using tools such as Optimole or others. However, the ability to achieve all of this behind the scenes within the Block Editor has clear advantages.

Here’s Pascal to explain:

“What’s the impact on loading times from the user’s point of view as opposed to using a dedicated media optimization plugin?”

Pascal Birchler.

Client-side image compression like this impacts the load time in two areas:

  1. In the editor, image uploads will be much faster as you don’t have to wait for the server to finalize the upload and correlate all the thumbnails. You can continue editing your post while the browser is doing the heavy lifting for you. Other plugins still do everything on the server, whether that’s your own or via an API.
  2. When you read a post on the front end, the load time will improve thanks to smaller image file sizes. The improvements will be similar regardless of which plugin you use, as the output image will always have optimization.

Of course, this is a positive for those who wish to try it out. However, in the future, the functionality will be even better – especially if this gets into the WordPress core.

How security is vital for WordPress Media Experiments

“What security measures are in place to protect against potential vulnerabilities, given that the plugin handles external media imports and transcoding operations?”

Pascal Birchler.

Right now, there are multiple safety nets in place. Cross-origin isolation instructs the browser to isolate the website from browsing contexts and lets you block the loading of unwanted resources.

Second, there are multiple client-side file type checks in JavaScript to be able to filter out any unwanted media as early as possible. The plugin also relies on the existing strict media handling in WordPress itself. This will reject anything suspicious on the server if needed.

On whether the Media Experiments functionality will make it into the WordPress core

Most WordPress features don’t just appear in a point release. It takes months (even years) of hard work to develop the functionality, which then goes through a thorough review and beta testing process.

Pascal has his own thoughts on whether WordPress Media Experiments will ever become a native feature:

“Do you think the image compression feature is a good candidate for the WordPress core?”

Pascal Birchler.

I think it should be possible to build something that is valuable and useful for all WordPress sites. Of course, this means elements such as the video recording or other things I’ve shown in the demo are probably not good candidates for inclusion in core. However, the image compression to me is a ‘no-brainer’: it’s useful, it makes Images faster, it makes your site faster, it reduces stress for the server.

This is arguably one area where WordPress lacks, so Pascal’s thoughts make a lot of sense. Who knows: we may see this end up in a WordPress release within the next year or two.

What it will take for Media Experiments to get into core

“How does a feature get into the WordPress core, along with the approval and committal process?”

Pascal Birchler.

…It depends on the size of the feature, I would say. For very small changes (like an improvement) you open a ticket on Trac and someone may feel as though it’s a good idea. For bigger changes, you should maybe write a proposal blog post.

With the Block Editor, there’s another approach because the Gutenberg plugin is a great way to experiment with new features. They may not even be fully ready yet, such as these media experiments. Over multiple Gutenberg plugin releases, we can further improve the feature and eventually add it to the core if it’s good enough.

User Interface (UI) design is a vital component of any plugin

The Media Experiments settings page within the Block Editor.
Media Experiments comes with its own set of options to help customize the experience to your needs.

While it should be that functionality wins when it comes to whether the team accepts new features, the reality is that visuals matter too. As such, Pascal had to consider UI design as well as feature design.

As a developer first and foremost, he admits this was a tough job:

“Which was the most challenging part of developing the UI, and what was your most difficult decision?”

Pascal Birchler.

I am not a designer! When I began, I first had to get up to speed on the latest changes in the Gutenberg design system, which goes through constant refinement.

Overall, I tried my best to make the whole experience as seamless as possible. There’s a lot of complexity involved with things such as image compression, conversion, recording videos within the editor, or uploading via a QR code. I attempted to build this in a way that feels very natural and ‘WordPress-like’ to users, rather than something ‘foreign.’

What’s exciting about WordPress Media Experiments, besides the image compression?

WordPress Media Experiments packs in a lot of functionality for what is an image compressor at its core. However, there’s plenty to consider, such as its thumbnail generation and GIF conversion.

In fact, it’s the former feature Pascal believes is a clear winner:

“Out of all the plugin features – besides the image compression – which ones excite you the most?”

Pascal Birchler.

That’s a tough one – it’s like asking who your favorite child is or something! There are many features in [the plugin], and I don’t think I’ve even shown all of them in the demo.

The client-side thumbnail generation is probably the nicest one because it’s basically invisible. You just upload the image, and everything happens for you.

The thumbnail generation that Media Experiments creates when uploading images to the WordPress Block Editor.

This facet is something Matt Mullenweg talks about in the State of the Word as an example of what we can still achieve with WordPress – especially the impact on server stress.

Combining Artificial Intelligence (AI) with Media Experiments

WordPress Media Experiments uses AI with regards to its speech-to-text recognition. However, this isn’t all. The plugin will use AI to create video captions and ‘alt’ text too.

Even this simple implementation offers complexity, as Pascal explains:

“What AI does the speech-to-text recognition use?”

Pascal Birchler.

It’s basically a company that built open-source machine learning models for speech-to-text generation. They have these open source models for multiple languages…I’m using those to generate the subtitles.

What I really like about it is that it does not talk to any server: it’s all in your browser. [There are] no API keys, and it works offline. Of course, there are limitations because if you have a server that you can talk to, there’s more you can do.

I think it shows what’s possible, and maybe the possibilities in browser is an area where more people will invest some time in the WordPress space, rather than just using ChatGPT.

Of course, AI is something that many plugins are starting to implement, so any new WordPress features also need to consider whether they can offer a benefit. We like the idea of AI alt text generation as it is often a time sink and requires some knowledge to get right for accessibility reasons. This is definitely one area we are excited about, and it could provide a tangible and superb positive for practically all users.

Improving WordPress Media Experiments to capture attention now

WordPress Media Experiments is akin to a ‘live document,’ in that it won’t likely see completion until it becomes part of the WordPress core.

As such, there is a lot to implement and improve upon with this proof of concept plugin:

“Are you planning to continue adding features to the plugin, or will you focus on trying to bring specific features into the WordPress core?”

Pascal Birchler.

There are still a lot of things that need polishing or adding to the plugin, but at the same time, I want to bring this to as many people as possible. Doing the former helps to further improve the architecture and uncover any roadblocks that might still exist.

That said, not all the current features necessarily make sense for core inclusion, at least not for an initial version. So right now, I’m trying to figure out what that initial version in the core could look like.

Future functionality within WordPress Media Experiments

Along with updating the plugin in order to make improvements, the feature set will also need additions. This is something Pascal is constantly working on:

“What other exciting things are you planning to bring to the plugin?”

Pascal Birchler.

I did get one feature request/idea over the weekend after my Tweet, and I’m now building it. So, let’s say you write a new blog post in WordPress on your computer. Then, you want to add an image to it that’s on your phone. How do you get the image to the computer?

My solution is to click a button in the editor and get a QR code. You scan this with your phone, select the picture, and the plugin will upload it to WordPress. The editor will refresh and insert the image. You could even send the link or code to someone else, for example. Maybe this will become my favorite feature!

Uploading an image using a QR code within the WordPress Block Editor.
[QR code uploading] is not something I’ve seen done before, whereas there are already solutions for compression. I think this is definitely new.

However, where does Pascal see the future of WordPress Media Experiments heading? It’s no surprise that he is thinking about the end-user experience:

Pascal Birchler.

There are so many other things you could build, such as enhanced video editing features or a way to optimize your entire media library. First though, I think I am going to start with something simpler: improved thumbnail cropping using so-called ‘saliency detection.’

Today, when you upload a photo of an animal or a person where the object is not nicely in the center, it could be completely missing from the thumbnail version WordPress generates. It’s much better to focus on the most salient region of an image instead, such as a person’s face. A tiny detail like this could make thumbnails much more pleasing.

Wrapping up 🎁

The WordPress Media Experiments plugin brings a potentially revolutionary piece of functionality to the Block Editor. Image compression and thumbnail generation are things you often need to take extra steps to achieve. However, as our interview with Pascal Birchler shows, this technology could soon be within the WordPress core. Certainly, the developer is working hard to refine and revise the plugin, with the result being something we will all benefit from in years to come. Even better, because it’s an open-source solution, anyone can contribute and improve upon the plugin’s functionality!

What was your favorite part about our Pascal Birchler interview, and what excites you about the WordPress Media Experiments plugin? Share your thoughts with us in the comments section below!

Don’t forget to join our crash course on speeding up your WordPress site. Learn more below:

 
Yay! 🎉 You made it to the end of the article!
Tom Rankin
Share:

0 Comments
Inline Feedbacks
View all comments

Or start the conversation in our Facebook group for WordPress professionals. Find answers, share tips, and get help from other WordPress experts. Join now (it’s free)!

0
Would love your thoughts, please comment.x