Sparse attention might be explanation.
Last January I participated in the Fairness, Accountability, Confidentiality, and Transparency course. We were tasked with replicating a recent paper on one...
About me
I'm a PhD candidate at the Vrije Universiteit Amsterdam (CLTL), supervised by Piek Vossen, Peter Bloem, and Ilia Markov.
My main research interests involve evaluating whether models have strong capabilities like understanding, reasoning, intentionality, etc. I enjoy trying to make precise what exactly such capabilities can look like, taking into account philosophy to understand different ways these terms are used. In practice, I try to tackle these questions as problems of (mechanistic) interpretability. Currently, my focus is on understanding how LLMs represent whether to treat a sentence as true/false, and how this interacts with (in-context) information through reasoning.
Recent Posts
Care2Report: Automated Medical Reporting
During my bachelor thesis project I worked in a team that developed a prototype for a research program called Care2Report at Utrecht University. The project ...