DML : Chapter 1. Introduction¶

• [손고리즘] middle learning - 파이썬을 이용한 기계학습 알고리즘 기초
• 김무성

DEEP LEARNING¶

Table of Contents¶

• 1 Introduction
• PART I Applied math and machine learning basics
  • 2 Linear Algebra
  • 3 Probability and Information Theory
  • 4 Numerical Computation
  • 5 Machine Learning Basics
• PART II Modern practical deep networks
  • 6 Feedforward Deep Networks
  • 7 Regularization
  • 8 Optimization for Training Deep Models
  • 9 Convolutional Networks
  • 10 Sequence Modeling: Recurrent and Recursive Nets
  • 11 Large scale deep learning
  • 12 Practical methodology
  • 13 Applications
• PART III Deep learning research
  • 14 Structured Probabilistic Models : A Deep Learnig Perspective

 - 15 Monte Carlo Methods - 16 Linear Factor Models and Auto-Encoders - 17 Representation Learning - 18 The Manifold Perspective on Representation Learning - 19 Confronting the Partition Function - 20 Approximate inference - 21 Deep Generative Models

Chapter 1. Introduction [1]¶

Contents¶

• 1.1 Who Should Read This Book?
• 1.2 Historical Trends in Deep Learning

Keywords¶

• artificial intelligence (AI)
• knowledge base approach
• machine learning
• feature
• hand-designed representations
• representation learning
• factors of variation
• Deep learning

1.1 Who Should Read This Book?¶

• One of these target audiences is university students (undergraduate or graduate) learning about machine learning, including those who are beginning a career in deep learning and artificial intelligence research.
• The other target audience is software engineers who do not have a machine learning or statistics background, but want to rapidly acquire one and begin using deep learning in their product or platform.

• This book has been organized into three parts in order to best accommodate a variety of readers.
  • Part 1 introduces basic mathematical tools and machine learning concepts.
  • Part 2 describes the most established deep learning algorithms that are essentially solved technologies.
  • Part 3 describes more speculative ideas that are widely believed to be important for future research in deep learning.

1.2 Historical Trends in Deep Learning¶

• artificial neural networks, parallel distributed processing, connectionism
• inspiration from the brain
  • Beyond that, it is difficult to say how similar the two are, and unlikely that they have many similarities, and our knowledge of the brain does not give very specific guidance for improving deep learning.
• Deep learning has now drawn considerably useful insights from many fields other than neuroscience, including
  • structured probabilistic models
  • manifold learning
  • and the modern terminology aims to avoid implying that only one field has inspired the current algorithms.
• the age of “Big Data”
• GPUs
• Sart! Perceptron and ADALINE etc.. -> back-propagation algorithm
• Winter! VS hand-designing domain-specific features became the typical approach to practical applications : Kernel machines + graphical models
• Boom! Research groups led by Geoffrey Hinton at University of Toronto, Yoshua Bengio at University of Montreal, and Yann LeCun at New York University re-popularized neural networks re-branded as “deep learning” beginning in 2006.

참고자료¶

• [1] DEEP LEARNING - http://www.iro.umontreal.ca/~bengioy/dlbook/