Fresh offering on our site: A Beginner's Tutorial in Natural Language Processing using Coding Techniques
A new course is making waves in the world of Natural Language Processing (NLP), offering a blend of traditional techniques, newer neural net approaches, and urgent issues of bias and disinformation. The course, which was originally taught in the University of San Francisco's Master of Science in Data Science program, covers a wide range of topics, from transfer learning for NLP to the Transformer and text generation algorithms.
Prerequisites for the course include familiarity with working with data in Python, machine learning concepts, and some experience with PyTorch and neural networks. The course is taught using Python in Jupyter Notebooks, with PyTorch and the fastai library.
One of the highlights of the course is transfer learning for NLP, a technique that allows models to leverage pre-trained models to learn from a large amount of data. The course also delves into multilingual NLP, focusing on languages other than English. Attention mechanisms and Transformers, key components in modern NLP, are also covered extensively.
Ethical issues in NLP are a significant focus of the course. Topics such as bias and disinformation are explored in depth, with a special guest lecture by Nikhil Garg, a Stanford PhD student, discussing word embeddings and their relation to gender and ethnic stereotypes. The risks of disinformation and the dynamics of its creation are also covered.
For those seeking resources to learn more about NLP, a curated list has been compiled. MIT’s Introduction to Deep Learning (6.S191) is a comprehensive course covering deep learning foundations, transformers, attention mechanisms, and language models. It includes discussions on ethical AI and societal impacts. While not PyTorch/fastai specific, it is highly relevant for content.
Zero to Mastery Academy offers courses on machine learning, deep learning, NLP, and deep dives into Hugging Face (transfer learning, transformers, multilingual models). PyTorch courses are available, and fastai is occasionally featured. Some courses and community discussions touch on ethical issues.
Large Language Models: A Self-Study Roadmap provides step-by-step guides to building and deploying large language models, including multilingual capabilities and practical projects. It features Python, LangChain, Hugging Face (which supports PyTorch and has fastai community overlap). Some resources include discussions on responsible AI.
500 AI/ML Projects offers a vast collection of NLP projects, with a focus on transfer learning, attention, transformers, and some multilingual models. All projects have code on GitHub and are often supported by YouTube tutorials.
By combining these resources, learners can cover all the requested topics—including transfer learning, multilingual models, attention, transformers, ethics, and Python, PyTorch, and fastai—through a self-directed, project-based learning approach. The course covers translation using Seq2Seq models, applications such as topic modeling, classification, language modeling, and translation, and new material on transfer learning for NLP with languages other than English. A playlist of all the videos from the course is available on YouTube.
- This new NLP course utilizes machine learning concepts and incorporates Python for implementation within Jupyter Notebooks.
- The course highlights transfer learning for NLP, enabling models to capitalize on pre-trained models for learning from vast amounts of data.
- The fastai library, along with PyTorch, is used in the course for building and understanding neural networks.
- Attention mechanisms and Transformers, crucial components in modern NLP, are thoroughly explored in the course.
- The course delves into multilingual NLP, focusing on languages other than English for a more global approach.
- Ethical issues in NLP are significantly addressed, with topics like bias and disinformation being discussed in detail.
- For further learning, MIT's Introduction to Deep Learning course covers deep learning fundamentals, transformers, attention mechanisms, and language models, including discussions on ethical AI and societal impacts.
- Zero to Mastery Academy provides courses on various AI/ML topics, including NLP, transfer learning, and transformers, with some resources addressing ethical issues.
- Large Language Models: A Self-Study Roadmap offers comprehensive guides for building and deploying language models, featuring Python, Hugging Face, and LangChain, with select resources discussing responsible AI.
- 500 AI/ML Projects offers numerous NLP projects focusing on transfer learning, attention, transformers, and multilingual models, with code available on GitHub and tutorials on YouTube.
