Complete guide on deriving and implementing word2vec, GloVe, word embeddings, and sentiment analysis with recursive nets
What you'll learn
·
Understand and implement word2vec
·
Understand the CBOW method in word2vec
·
Understand the skip-gram method in word2vec
·
Understand the negative sampling optimization in word2vec
·
Understand and implement GloVe using gradient descent and
alternating least squares
·
Use recurrent neural networks for parts-of-speech tagging
·
Use recurrent neural networks for named entity recognition
·
Understand and implement recursive neural networks for sentiment
analysis
·
Understand and implement recursive neural tensor networks for
sentiment analysis
·
Use Gensim to obtain pretrained word vectors and compute
similarities and analogies
Requirements
·
Install Numpy, Matplotlib, Sci-Kit Learn, and Theano or
TensorFlow (should be extremely easy by now)
·
Understand backpropagation and gradient descent, be able to
derive and code the equations on your own
·
Code a recurrent neural network from basic primitives in Theano
(or Tensorflow), especially the scan function
·
Code a feedforward neural network in Theano (or Tensorflow)
·
Helpful to have experience with tree algorithms
Description
In this course we are going to look at NLP
(natural language processing) with deep learning.
Previously, you learned about some of the
basics, like how many NLP problems are just regular machine learning and data
science problems in disguise, and simple, practical methods like bag-of-words and
term-document matrices.
These allowed us to do some pretty cool
things, like detect spam emails, write poetry, spin
articles, and group together similar words.
In this course I’m going to show you how to do
even more awesome things. We’ll learn not just 1, but 4 new
architectures in this course.
First up is word2vec.
In this course, I’m going to show you exactly
how word2vec works, from theory to implementation, and you’ll see that it’s
merely the application of skills you already know.
Word2vec is interesting because it magically
maps words to a vector space where you can find analogies, like:
- king - man = queen - woman
- France - Paris = England -
London
- December - Novemeber = July -
June
For those beginners who find algorithms tough
and just want to use a library, we will demonstrate the use of the Gensim library
to obtain pre-trained word vectors, compute similarities and analogies, and
apply those word vectors to build text classifiers.
We are also going to look at the GloVe method,
which also finds word vectors, but uses a technique called matrix
factorization, which is a popular algorithm for recommender systems.
Amazingly, the word vectors produced by GLoVe
are just as good as the ones produced by word2vec, and it’s way easier to
train.
We will also look at some classical NLP
problems, like parts-of-speech tagging and named
entity recognition, and use recurrent neural networks to
solve them. You’ll see that just about any problem can be solved using neural
networks, but you’ll also learn the dangers of having too much complexity.
Lastly, you’ll learn about recursive
neural networks, which finally help us solve the problem of negation
in sentiment analysis. Recursive neural networks exploit the fact
that sentences have a tree structure, and we can finally get away from naively
using bag-of-words.
All of the materials required for this course
can be downloaded and installed for FREE. We will do most of our work in Numpy, Matplotlib, and Theano.
I am always available to answer your questions and help you along your data
science journey.
This course focuses on "how to build
and understand", not just "how to use". Anyone can learn to
use an API in 15 minutes after reading some documentation. It's not about
"remembering facts", it's about "seeing for
yourself" via experimentation. It will teach you how to visualize
what's happening in the model internally. If you want more than
just a superficial look at machine learning models, this course is for you.
See you in class!
"If you can't implement it, you don't
understand it"
- Or as the great physicist
Richard Feynman said: "What I cannot create, I do not
understand".
- My courses are the ONLY courses
where you will learn how to implement machine learning algorithms from
scratch
- Other courses will teach you
how to plug in your data into a library, but do you really need help with
3 lines of code?
- After doing the same thing with
10 datasets, you realize you didn't learn 10 things. You learned 1 thing,
and just repeated the same 3 lines of code 10 times...
Suggested Prerequisites:
- calculus (taking derivatives)
- matrix addition, multiplication
- probability (conditional and
joint distributions)
- Python coding: if/else, loops,
lists, dicts, sets
- Numpy coding: matrix and vector
operations, loading a CSV file
- neural networks and
backpropagation, be able to derive and code gradient descent algorithms on
your own
- Can write a feedforward neural
network in Theano or TensorFlow
- Can write a recurrent neural
network / LSTM / GRU in Theano or TensorFlow from basic primitives,
especially the scan function
- Helpful to have experience with tree algorithms
WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:
- Check out the lecture
"Machine Learning and AI Prerequisite Roadmap" (available
in the FAQ of any of my courses, including the free Numpy course)
Who this course is
for:
·
Students and professionals who want to create word vector
representations for various NLP tasks
·
Students and professionals who are interested in state-of-the-art
neural network architectures like recursive neural networks
·
SHOULD NOT: Anyone who is not comfortable with the
prerequisites.
Show less
0 Comments