Updates 1/24/23 Ran into this library cloudpickle and it reminded me of this post. 3/18/21: Found this outrun which seems to be basically a better and more thought out way to do a lot of this. Introduction Recently I became interested in better workflows for machine learning projects. One of the main issues I’ve experienced is developing locally but not being able to easily test locally. Generally this is from a local computer being a bit slower and older for the types of models and datasets I am working with and initial code decisions that resulted in poor performance on a slower computer than the server used to train.

Issue Often there is no simple way to get the files from kaggle to a remote server. While previously I had used either a cookies extension or a python command line module that allowed me to specify the competition, neither of these work efficiently or at all for various reasons. I had been meaning to write a script to do this for some time now had never gotten around to it.