Accepted paper :)

Accepted workshop paper

Thank you for submitting your paper, Six Layers Less: Encoder Pruning for Whisper with Label-Free Recovery, to GLOW 2026. We are delighted to inform you that your submission has been accepted. Congratulations!

I had been working on an idea on the side for some time. Some pretty random experimentation lead to the observation that we could remove any layer from the Whisper-large-v3-turbo encoder without much degradation. This motivated the question: How far can we take this? Some initial analysis on layer similarity showed that the signal that traverses the encoder stack stayed highly similar apart from a few places where a significant change occurred.

We also need to find some principled way of ranking the layers in terms of their “removability”. After looking around in the literature I found that LLM research had similar findings and a recent result indicated that leave-one-layer-out downstream performance as a reliable metric.

I packaged the results up as an extended abstract and sent it to the GLOW Workshop at IJCAI and it was accepted! I’ll be going to Bremen in August to present the poster.

Internal use-case for local transcription

A known risk about the project as a whole has always been the usefulness to the business-side of things. What if I can’t translate my knowledge and findings into something that the company finds valuable? We have spent a non-trivial amount of time and effort thinking and planning around this but I have always had the (perhaps naive) thought at the back of my mind that local AI deployment makes so much sense that use-cases simply must spawn and present themselves on their own out of needs from the product.

This was the case with one of our products which is at a scale where centralized transcription on a cloud service would be costly compared to the value-add that it brings. However, the value-add is meaningful so the product team wanted to explore more options for transcription services. The product owner had made an isolated demo in Lovable that I could freely play around with. Downloading that and instructing Claude Code to use Transformers.js and whisper-base for the transcription feature worked seamlessly, even on edge devices such as the Android tablets that are used for testing the product.

Summer school preparation

I will be going to The European Summer School on Artificial Intelligence in July. It’ll be in Vienna and I’m very much looking forward to it. I think it’ll be good to get out a bit and experience a broader circle of people. I’m particularly looking forward to the course in The Art of Compressing LLMs: Pruning, Distillation, and Quantization Demystified by Nvidia.

I will also be taking some personal vacation in Vienna - solo! That will be interesting. I’m honestly dreading it a bit because I feel like there’s an endless amount of details to plan for and things to think about, but I tell myself that I will manage things as they come.