教學大綱表 (110學年度 第2學期)
請遵守智慧財產權,勿使用非法影印教科書,避免觸法。
課程名稱
Course Title
(中文) 自然語言處理
(英文) Natural Language Processing
開課單位
Departments
資訊工程研究所
課程代碼
Course No.
I5840B
授課教師
Instructor
葉慶隆
學分數
Credit
3.0 必/選修
core required/optional
選修 開課年級
Level
研究所
先修科目或先備能力(Course Pre-requisites):程式設計、基礎數學(微積分、機率、離散數學)
課程概述與目標(Course Overview and Goals): 自然語言處理和電腦視覺是人工智慧做為智慧型代理人(Intelligent Agent)的溝通及感知的主要能力,資料驅動的深度學習技術是促成電腦這些能力的重要基礎。近來深度學習技術已經成為實現溝通及感知的基礎建設,並且廣泛整合在課程中。

一直以來人工智慧是電腦科學裡最複雜的領域之一,學生需要相當紮實的資訊及數學基礎,才能學好這門課。這門課將針對只具備基本程式設計能力,及基礎數學的學生,以問題導向為主軸,結合各種深度學習技術,引領學生認識自然語言處理。本課程將依以下順序進行:

自然語言處理介紹
基礎字串處理:Regular Expressions, Text Normalization, Edit Distance
以邏輯斯分類(logistic classification)為基礎的情感分析(sentiment analysis)
以Naive Bayes 為基礎的情感分析
向量空間模型
向量語意及詞嵌入(Vector Semantics and Word Embedding)
以神經網路為基礎的情感分析
遞迴神經網路(RNN)和語言模型(language modeling)
序列對序列神經網路及機器翻譯
自然語言處理與關注模型(NLP and Attention Model)
教科書(Textbook) Dan Jurafsky and James H. Martin, Speech and Language Processing, 3rd ed.
參考教材(Reference) Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016
Tensorflow Tutorial, tensorflow.org
Charles Severance, Python for Everybody
Introduction to AI, udacity.com
AI for Everyone, coursera.org
Introduction to TensorFlow for Deep Learning, udacity.com
課程大綱 Syllabus 學生學習目標
Learning Objectives
單元學習活動
Learning Activities
學習成效評量
Evaluation
備註
Notes

No.
單元主題
Unit topic
內容綱要
Content summary
1 Welcome and introduction to Natural Language Processing NLP background
Knowledge in Speech and Language Processing
Ambiguity
Models and Algorithms
Language, Thought, and Understanding
Some Brief History
NLP basis  
2 Regular Expressions, Text Normalization, Edit Distance (1) Regular Expressions
Word and Corpora
Word Tokenization
Word Normalization
Regular Expressions, Text Normalization  
3 Regular Expressions, Text Normalization, Edit Distance (2) Python RE
Python NLTK
Python programming: re and nltk  
4 Sentiment Analysis Using Logistic Classification(1) Supervised ML and Sentiment Analysis
Vocabulary & Feature Extraction
Negative and Positive Frequencies
Feature Extraction with Frequencies
Sentiment Analysis Using Logistic Classification  
5 Sentiment Analysis Using Logistic Classification(2) Preprocessing
Putting it All Together
Logistic Regression Overview
Logistic Regression: Training
Logistic Regression: Testing
Sentiment Analysis Using Logistic Classification 專題分組開始。 專題第一階段:python re, nltk; Sentiment Analysis using Logistic Classification  
6 Sentiment Analysis with Naïve Bayes (1) Probability and Bayes’ Rule
Naïve Bayes Introduction
Laplacian Smoothing
Log-Likelihood
Sentiment Analysis with Naïve Bayes  
7 Sentiment Analysis with Naïve Bayes (2) Training Naïve Bayes
Testing Naïve Bayes
Applications of Naïve Bayes
Naïve Bayes Assumptions
Sentiment Analysis with Naïve Bayes 專題分組開始。 專題第二階段:Sentiment Analysis using Naive Bayes Method  
8 Vector Space Models Intro. to Vector Space Models
Word by Word and Word by Doc.
Euclidean Distance
Cosine Similarity
Manipulating Words in Vector Spaces
Visualization and PCA
Vector Space Models  
9 期中考週 期中測驗 期中測驗  
10 Vector Semantics and Embeddings (1) Overview
Basic word Representation
Word Embeddings
How to Create Word Embeddings
Word Embedding Methods
Continuous Bag-of-Words Model
Vector Semantics and Embeddings  
11 Vector Semantics and Embeddings (2) Cleaning and Tokenization
Sliding Window of Words in Python
Transforming Words into Vectors
Architecture of the CBOW Model
Architecture of the CBOW Model: Activation Functions
Vector Semantics and Embeddings  
12 Vector Semantics and Embeddings (3) Training a CBOW Model
Extracting Word Embedding Vectors
Evaluating Word Embeddings
Examples of Word Embedding
Vector Semantics and Embeddings  
13 Neural Networks for Sentiment Analysis Neural Networks and Forward Propagation
Trax: Neural Networks Trax: Layers
Dense and ReLU Layers
Serial Layer
Other Layers
Traiining
Neural Networks for Sentiment Analysis  
14 Recurrent Neural Networks for Language Modeling Traditional Language Models
Recurrent Neural Networks
Applications of RNNs
Math in Simple RNNs
Cost Function for RNNs
Gated Recurrent Units
Recurrent Neural Networks for Language Modeling  
15 Neural Machine Translation and Transformer Seq2Seq
Alignment
Attention
Setup for Machine Translation
Introduction to Transformer
Neural Machine Translation and Transformer  
16 期末報告 (1) 期末報告 期末報告  
17 期末報告(2) 期末報告 期末報告  
18 期末考週 期末測驗 期末測驗  


教學要點概述:
1.自編教材 Handout by Instructor:
□ 1-1.簡報 Slids
□ 1-2.影音教材 Videos
□ 1-3.教具 Teaching Aids
□ 1-4.教科書 Textbook
□ 1-5.其他 Other
□ 2.自編評量工具/量表 Educational Assessment
□ 3.教科書作者提供 Textbook

成績考核 Performance Evaluation: 專題:20%   :20%   :30%   作業:30%  

教學資源(Teaching Resources):
□ 教材電子檔(Soft Copy of the Handout or the Textbook)
□ 課程網站(Website)
扣考規定:http://eboard.ttu.edu.tw/ttuwebpost/showcontent-news.php?id=504