CES 510 Intelligent System Design                           Fall 2008

Dept of Engineering Science, Sonoma State University

Instructor: B. Ravikumar          Office:  116 I, Darwin Hall

Office phone: 664 3335              Email: ravi93@gmail.com

Office hours:  T Th 9:30 to 10:30 AM, F 12 – 1 PM


Class Time:   W 6 to 8:45 PM, 2008, Salazaar Hall


Prerequisites for the course:


  • Data Structures and programming – ability to implement algorithms that use recursion and recursively defined data structures (such as trees) in c++
  • Mathematics – Calculus, Probability Theory.


 Chris Manning and Hinrich Schόtze, Foundations of Statistical Natural Language Processing, MIT Press. Cambridge, MA: May 1999.




       For natural language and speech processing


  • Jurafsky & Martin, SPEECH and LANGUAGE PROCESSING: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition


For general Artificial Intelligence



Course Goals


 Intelligent systems are hardware/software systems that exhibit intelligent behavior – at least in narrow contexts such as playing a game or predicting weather. There are at least two different approaches to designing intelligent systems. One is the symbolic approach in which the task is modeling as a search problem and efficient algorithms to explore the search space are devised. The other is to model the decision making as a statistical process in which statistical models are derived from the data that arises from the domain. This dichotomy can also be understood in terms of whether the solution is pre-programmed or is learnt by experience. We will primarily use the latter approach (machine-learning based approach that uses statistical models) in the context of one problem domain – namely natural language processing. This area has become important in view of the extensive applications of WWW and information retrieval problems in which the data is encoded in natural language. Although our focus is in the area of natural language processing, we will focus on techniques and models that are useful in other domains (such as speech recognition, bio-informatics etc.)

Topics covered

·         Mathematical tools

o        Information theory, entropy

o        Bayesian statistics

o        Statistical inference

o        Maximum likelihood estimation

·         General AI techniques

o        hidden Markov model

o        clustering

o        support vector machines, decision trees

·         Domain specific techniques

o        stochastic context-free grammars, parsing

o        weighted finite automata

·         applications

o        document classification

o        information retrieval from Web pages

Parts of chapters 1 to 7, 9, 10, 11, 13, 14, 15


Assigned work will include HW assignments, a programming project, an in-class mid-term test and an in-class final exam. The HW assignments will usually not involve programming or implementation. It may, however, involve using software tools.

Weights for these components are (approximately) as follows:

·         HW assignments                                  –   20%

·         programming project                          –   30%

·         Mid-term test                                        –   15%

·          Final Exam                                        –  35%