Jon Krohn, Ph.D.
jon-krohn-deep-learning-teaching.jpg

Deep Learning

First Steps in Deep Learning

For people in New York, I founded a Deep Learning Study Group. If you're further afield, you can track our progress via GitHub.

Based on my experience with the study group, I have recorded seventeen hours of interactive introductory tutorials:

The notebooks of code built over the course of the videos are available for free in GitHub.  In addition, I offer a part-time in-classroom Deep Learning course at the NYC Data Science Academy. 

Otherwise, get a lay of the land from: 

  • the sequence of courses suggested by Greg Brockman, or

  • this (more comprehensive) introductory resource post from Ofir Press, or

  • this (even more comprehensive) guide from YerevaNN Research Lab



Deep Learning Textbooks

Relative to viewing lectures, I prefer reading and working through problems. The stand-out resources for this, in the order I recommend tackling them are: 



Interactive Deep Learning Demos

Top-drawer interactive demos you can develop an intuitive sense of neural networks from are provided by: 

  • Distill, the academic publication for visualising machine learning research

  • Chris Olah

  • the illustrious Andrej Karpathy

  • fun, concise, browser-based (i.e., JavaScript) self-driving cars

  • ML-Showcase, a curated collection of remarkable deep-learning focused demos

  • ...in addition, I've curated introductory Jupyter notebooks across the popular libraries TFLearn, Keras, Theano, and TensorFlow here



Applications of Deep Learning

Scroll further down the page down to see my recommendations for high-quality data sources as well as global issues in need of solutions. Problems worth solving with deep learning approaches in particular are curated by OpenAI. In addition, if you're at the stage that you'd like to test a deep reinforcement learning algorithm across a range of applications (e.g., games), work with: 



Time Series Prediction, e.g., Financial Applications



Academic Deep Learning Papers

If you're looking for the latest deep learning research, check out: 



Deep Learning Hardware

Here is the part list for a deep learning server that I built



Cloud Infrastructure, useful for Deep Learning



Histories of Deep Learning



The Future of Deep Learning







Open Data Sources

To train a powerful model, the larger the data set, the better -- if it's well-organised and open, that's ideal. The following repositories are standouts that meet all these criteria: 

For machine learning models that require a lot of labelled data, check out:

If none of the above data sources suit your needs, Google provides a dataset-specific search tool.







Problems Worth Solving

Medical Applications of Deep Learning

Charitable Projects

  • DataKind is a well-respected platform for finding humanitarian causes to apply your data science skills to. 

 





General Data Scientist Tools

As initially outlined in my post on Data Scientist Skills and Salaries, here is a list of key data science tools. With a focus on coding in Python wherever possible, they are:

It's also helpful to develop familiarity with:

Note that these tools generally appear in the open-source Hadoop cluster in the O'Reilly Data Science Salary Survey. Based on demand and relative compensation, it appears that valuable next steps to becoming a unicorn-variety data scientist would be to equip oneself with parallel-processing tools (e.g., SparkHivePig). 







Fun Online Primers for Data Science Techniques

 





Lay Primers on Software and Artificial Intelligence

Excellent Lay Books on Statistics



 

 

Meetups

News

 

 



References

 

 



Clarity and Productivity

 



 

List of Additional Tools

  • LaTeX for creating beautiful documents, including Beamer for slideshows and Pandoc for conversion to countless other formats (e.g., word processor formats for sharing with coworkers)

  • Amazon AWS, especially S3 buckets, EC2, and Redshift

  • I love the Mathematica-based Wolfram Alpha web interface for learning about mathematical concepts interactively

  • Plotly is a free, easy-to-use GUI for collaboratively creating aesthetically-pleasing visualisations

  • if you would like a slick, professional tool for mining data from patents, companies and/or the news, check out Quid, which I used extensively for a political project



 

Eudaemonia

For a life of flourishing -- a life of beauty, truth, justice, play and love -- choose mathematics