TCAV Introduction

Understanding a deep learning model is an open topic and is quite subjective. In part, I’m still unsure what exactly understanding a model means and this is brought up in many papers as understanding can be very different depending on your technical understandings.

What makes a model interpretable? Some good blog posts that similarly touch on this topic are:

• How neural networks build up their understanding of images
• The building blocks of interpretability

While those are great and talk about different ways of understanding models, the TCAV score is based around the idea that specific layers will be activated by features or “concepts” more than others.