One of the side effects of deep learning models becoming increasingly large is the amount of data that they intermediately generate which is then often ignored. Researchers have used this information to create explainability methods which show importance relative to a specific input it is connected with, but often the intermediate values are disregarded and mostly underutilized. Although this intermediate data can become increasingly unwieldily in size as the networks grow, the ability to monitor specific layers is a valuable tool that provides insight into how the model is learning as well as vague generalities about how the model performs overall (e.g. where are specific features begun to be extracted or where is noise filtered out).

Updates

1/24/23 Ran into this library cloudpickle and it reminded me of this post.

3/18/21: Found this outrun which seems to be basically a better and more thought out way to do a lot of this.

Introduction

Recently I became interested in better workflows for machine learning projects. One of the main issues I’ve experienced is developing locally but not being able to easily test locally. Generally this is from a local computer being a bit slower and older for the types of models and datasets I am working with and initial code decisions that resulted in poor performance on a slower computer than the server used to train. Another instance I have had this happen is if the remote server is behind a VPN and a relay hop and there is a bit of a latency between saving the file and running it if I am using something such as VSCode (although you can try something such as a ProxyJump in your ssh config). Regardless of the problem, I have yet to find an actual workflow that I find versatile and simple.

TCAV Introduction

Understanding a deep learning model is an open topic and is quite subjective. In part, I’m still unsure what exactly understanding a model means and this is brought up in many papers as understanding can be very different depending on your technical understandings.

What makes a model interpretable? Some good blog posts that similarly touch on this topic are:

• How neural networks build up their understanding of images
• The building blocks of interpretability

While those are great and talk about different ways of understanding models, the TCAV score is based around the idea that specific layers will be activated by features or “concepts” more than others.

fastText: A Short Guide

fastText is a powerful, open-source library designed to facilitate both supervised and unsupervised learning of text representations and classifiers. Unlike other tools like sklearn or spaCy, fastText is particularly praised for its speed and ease of use, which make it accessible even for those with limited technical background. The official fastText website describes it as “lightweight” and capable of running on “standard, generic hardware,” with the potential to optimize models for mobile devices.

Running Postgres on kubernetes locally

While this may be overkill, its better than configuring a kubernetes cluster on gcloud or whatever else and if done correctly will translate to a cloud service we can later use in a production system while allowing us to focus on micro services individually.

To start Kubernetes locally

I am using Docker for mac which comes with Kubernetes v1.9.8 as of this time of writing and while it may not perfectly replicate a development/staging/production environment, I find it to be much more straightforward to develop in this manner due to many of the new Kubernetes tooling. While we could use docker swarm or compose, kubernetes seems to be a more versatile use case for the hope of not only running an app but creating pipelines to ingest and maintain a stable configuration that could be used in the cloud (and much more resembles how you can develop locally then roll out to kubernetes systems).

Issue

Often there is no simple way to get the files from kaggle to a remote server. While previously I had used either a cookies extension or a python command line module that allowed me to specify the competition, neither of these work efficiently or at all for various reasons. I had been meaning to write a script to do this for some time now had never gotten around to it. Finally I did it and here it is posted, this is also incredibly useful for downloading files that are locked away by login/password on some site. For this, I am using RoboBrowser.

Setting up a Telegram Bot in Python and Docker

This post is my personal introduction to using Telegram and Bluemix and while it is incredibly simple, it is useful to see how to do the basics before integrating peripheral API’s or extraneous processes.

First, set up a telegram account however you’d like. I used Desktop client and it worked great. Once you have an account, you need to interact with @BotFather to set up a bot. Message @BotFather /start and you will receive these options:

edit: everything has since been updated to Tensorflow 0.7.0 which I based off of my base cuda dockerfile to use with tensorflow or theano (depending on my goals, Keras allows great flexibility in between training vs compiling)

TensorFlow

In 2015 Google came out with a new deep learning framework/tensor library similar in many ways to Theano and I enjoy using it a lot more than Theano simply due to long compile times of Theano when using Keras and TensorBoard. This will not go into detail about using Theano or TensorFlow or Keras but instead is how I built a docker image that uses a slightly older nvidia card (which for my purposes is capable of using multiple gpu’s in isolation and exiting a model on one card and not effecting the other).

NOTE 6/6/23: DON’T USE

Someone recently emailed me asking a question about this post. This code is not usable in any manner and the code itself was written when I had very little programming experience. Keeping this post up for posterity but otherwise it is useless.

Note 4/10/17: almost all the python modules have changed quite a bit since this original post and there are issues with youtube-dl and keras, if you would like to work on an updated version or have an updated version please let me know!

blank if not used!

add text after this:

afterSwift ->