Introduction Recently I have become interested in better workflows for machine learning projects. One of the main issues I’ve run into is developing locally but not being able to test locally for various reasons. Generally this is from my local computer being a bit slower and older for the types of models and datasets I am working on and not architecting the code from the start to accommodate a slower computer than the server I may actually be training on.

TCAV Introduction Understanding a deep learning model is an open topic and is quite subjective. In part, I’m still unsure what exactly understanding a model means and this is brought up in many papers as understanding can be very different depending on your technical understandings. What makes a model interpretable? Some good blog posts that similarly touch on this topic are: How neural networks build up their understanding of images The building blocks of interpretability While those are great and talk about different ways of understanding models, the TCAV score is based around the idea that specific layers will be activated by features or “concepts” more than others.

fastText fastText is a useful tool that allows us to use text data and train supervised and unsupervised models. While this is possible without fastText using sklearn, spacy, etc. there are plenty of guides on those and not much information on fastText. It allows you to use it from the command line very straightforward or there is a python libary included. Part of why it’s appeling is that it is incredibly quick, straightforward and doesn’t require much knowledge to use.

Running Postgres on kubernetes locally While this may be overkill, its better than configuring a kubernetes cluster on gcloud or whatever else and if done correctly will translate to a cloud service we can later use in a production system while allowing us to focus on micro services individually. To start Kubernetes locally I am using Docker for mac which comes with Kubernetes v1.9.8 as of this time of writing and while it may not perfectly replicate a development/staging/production environment, I find it to be much more straightforward to develop in this manner due to many of the new Kubernetes tooling.

Issue Often there is no simple way to get the files from kaggle to a remote server. While previously I had used either a cookies extension or a python command line module that allowed me to specify the competition, neither of these work efficiently or at all for various reasons. I had been meaning to write a script to do this for some time now had never gotten around to it.