Summer school in Vienna and starting student projects

While half of July was booked for some much needed vacation it was still felt jam-packed as I attended my first summer school outside of Denmark.

European Summer School on Artificial Intelligence

There were a few firsts for me during this summer school. It was the furthest I had gone by myself and I didn’t know anyone at the school in advance. In a way that was a liberating feeling as I didn’t have any social plans and things could just happen. On the other hand it was slightly anxiety inducing to have to make an active effort to socialize. However, as I arrived in Vienna on the 1st of July and the summer school starting on the 5th, I had gone a few days completely solo, depraving me of social interactions to the point where I really felt like I was socially “in the zone” and it all came to me without much effort.

Looking back on it now it’s kind of amazing how much variance there must be at such events and how it depends on small social interactions. My week was heavily defined by the randomness of the seating during the first lecture and the fact that the person sitting next to me had a Taylor Swift tote bag that I asked about. Feels a little surreal to sit here and analyze.

Academic content

While it was fantastic socially it also featured some excellent courses. I was a little taken aback by the expectation to follow 4x1,5 hour lectures each day with an additional event or keynote at the end of the day. That felt like a bit much.

My favorite course was definitely “The Art of Compressing LLMs: Pruning, Distillation, and Quantization Demystified” by people from NVIDIA as it’s highly aligned with my interests and project. I’m not so sure that I learned all that much as it was more of an introduction to the techniques that I have been familiarizing myself with over the past year or so, but it was incredibly nice as a confirmation that my efforts haven’t been wasted and that I understood most of the material without much effort.

Supervising student projects

We’re starting up two student projects this semester. Both have to do with quantization of LLMs and lean heavily towards benchmarking. I’m quite excited about this as they both seem like self-driven and motivated students that will definitely get a lot done during the next couple of months.

The first project is about filling a gap in practical usage of LLMs. Official benchmarks are usually just reported as a bar chart with limited or no details on software or hardware configuration, quantization, etc. As the performance can differ significantly after quantizing, and quantization being widely used as it’s what allows practitioners to actually use the models on reasonable hardware configs, there is a need to collect empirical benchmarks where the software/hardware config is a central part of the metadata.

The second project is a master’s thesis, so the scope is broader and with more unknowns at this point in time. The student is interested in getting more familiar with general benchmarking of LLMs but also developing intuition for Post-Training Quantization (PTQ) methods that will allow him to experiment with novel approaches.

I’m excited to see what comes out of these projects!