What to Expect

The assignment comes in two sections:

a ‘math’ bit, in which you mostly compute the gradient of various objective functions related to word2vec (this is where the latex template comes in), and
a ‘code’ bit, in which you use your definitions to implement word2vec.

You’ll pretty much have to do the first part before you can do the second.

Submission and Grading

Their notes reference an autograder at Gradescope, which unfortunately we won’t have access to.

Please submit your answers to the math section using the latex template, and run their collect_submission.sh script to package up your code for us. Use the submission link in Discord to drop your stuff (preferred), or email Liam and I.

I’ll use their grading points as a rough guide, but I’ll be grading this assignment simply as an A, B, C, or F. We’ll be comparing your answers and your code to a reference implementation.

Tips

I think this is the first challenging assignment. The concepts in NLP and machine learning are often quite simple, but then there comes a time when we sit down at the keyboard to write the implementation, and our mind returns a vast, empty desert indicating the extent of our ignorance. Time to fill that desert.

You will have to learn or relearn your differential calculus to get the right answers. I think you will find the cs224 notes helpful, because the assignment was written with them in mind.

I imagine that most of these questions have answers available online, and I expect that some of you will reference these as you go through the assignment. That is okay. Please try to make it a good learning experience for yourself, though. Push yourself to see what you can do without help. Write down questions to bring to us at office hours. Then take a peek at the answers and give yourself just enough boost to get to the next question.

time burden Please spend no more than six hours on this (not including training time). Give the best answers you can, then move on to your readings and your research project.