Jon Krohn, Cajoler of Datums

Deep Learning

First Steps in Deep Learning

For people in New York, I founded a Deep Learning Study Group. If you're further afield, you can track our progress via GitHub.

Based on my experience with the study group, I have recorded seventeen hours of interactive introductory tutorials:

The notebooks of code built over the course of the videos are available for free in GitHub.  In addition, I offer a part-time in-classroom Deep Learning course at the NYC Data Science Academy. 

Otherwise, get a lay of the land from: 

  • the sequence of courses suggested by Greg Brockman, or

  • this (more comprehensive) introductory resource post from Ofir Press, or

  • this (even more comprehensive) guide from YerevaNN Research Lab

Deep Learning Textbooks

Relative to viewing lectures, I prefer reading and working through problems. The stand-out resources for this, in the order I recommend tackling them are: 

Interactive Deep Learning Demos

Top-drawer interactive demos you can develop an intuitive sense of neural networks from are provided by: 

  • Distill, the academic publication for visualising machine learning research

  • Chris Olah

  • the illustrious Andrej Karpathy

  • fun, concise, browser-based (i.e., JavaScript) self-driving cars

  • ML-Showcase, a curated collection of remarkable deep-learning focused demos

  • ...in addition, I've curated introductory Jupyter notebooks across the popular libraries TFLearn, Keras, Theano, and TensorFlow here

Applications of Deep Learning

Scroll further down the page down to see my recommendations for high-quality data sources as well as global issues in need of solutions. Problems worth solving with deep learning approaches in particular are curated by OpenAI. In addition, if you're at the stage that you'd like to test a deep reinforcement learning algorithm across a range of applications (e.g., games), work with: 

Predicting Time Series with Deep Learning, e.g., Financial Applications

Academic Deep Learning Papers

If you're looking for the latest deep learning research, check out: 

Deep Learning Hardware

Here is the part list for a deep learning server that I built

Cloud Infrastructure, useful for Deep Learning

Histories of Deep Learning

The Future of Deep Learning

Open Data Sources

To train a powerful model, the larger the data set, the better -- if it's well-organised and open, that's ideal. The following repositories are standouts that meet all these criteria: 

For machine learning models that require a lot of labelled data, check out:

If none of the above data sources suit your needs, Google provides a dataset-specific search tool.

Problems Worth Solving

Medical Applications of Deep Learning

Charitable Projects

  • DataKind is a well-respected platform for finding humanitarian causes to apply your data science skills to. 

 

General Data Scientist Tools

As initially outlined in my post on Data Scientist Skills and Salaries, here is a list of key data science tools. With a focus on coding in Python wherever possible, they are:

It's also helpful to develop familiarity with:

Note that these tools generally appear in the open-source Hadoop cluster in the O'Reilly Data Science Salary Survey. Based on demand and relative compensation, it appears that valuable next steps to becoming a unicorn-variety data scientist would be to equip oneself with parallel-processing tools (e.g., SparkHivePig). 

Fun Online Primers for Data Science Techniques

 

Lay Primers on Software and Artificial Intelligence

Excellent Lay Books on Statistics

 

 

Meetups

News

 

 

References

 

 

Clarity and Productivity

 

 

List of Additional Tools

  • LaTeX for creating beautiful documents, including Beamer for slideshows and Pandoc for conversion to countless other formats (e.g., word processor formats for sharing with coworkers)

  • Amazon AWS, especially S3 buckets, EC2, and Redshift

  • I love the Mathematica-based Wolfram Alpha web interface for learning about mathematical concepts interactively

  • Plotly is a free, easy-to-use GUI for collaboratively creating aesthetically-pleasing visualisations

  • if you would like a slick, professional tool for mining data from patents, companies and/or the news, check out Quid, which I used extensively for a political project

network.jpg

Eudaemonia

For a life of flourishing -- a life of beauty, truth, justice, play and love -- choose mathematics