Updates 1/24/23 Ran into this library cloudpickle and it reminded me of this post. 3/18/21: Found this outrun which seems to be basically a better and more thought out way to do a lot of this. Introduction Recently I have become interested in better workflows for machine learning projects. One of the main issues I’ve run into is developing locally but not being able to test locally for various reasons. Generally this is from my local computer being a bit slower and older for the types of models and datasets I am working on and not architecting the code from the start to accommodate a slower computer than the server I may actually be training on.

Issue Often there is no simple way to get the files from kaggle to a remote server. While previously I had used either a cookies extension or a python command line module that allowed me to specify the competition, neither of these work efficiently or at all for various reasons. I had been meaning to write a script to do this for some time now had never gotten around to it.