Department of Mathematics



Math 03.412 Stochastic Models in Operations Research


Catalogue Description:


Math 03.412 Stochastic Models in Operations Research                                                             3 s.h.


(Prerequisites: STAT 02.360 Probability and Random Variables, and either [(Math 01.230 Calculus III or Math 01.141 Accelerated Calculus II) and (Math 01.210 Linear Algebra or Math 01.235 Math for Engineering Analysis I)] or Math 03.411 Deterministic Models in Operations Research with a grade of C- or better in all prerequisites.


This course is an introduction to mathematical modeling, analysis, and solution procedures applicable to decision-making problems in an uncertain (stochastic) environment. Methodologies covered include dynamic programming, Markov chains, queuing theory, decision trees, and system reliability and inventory theory. Solutions will be obtained using theoretical methods and software packages.



Objectives in Relation to Student Outcomes


Students in this course will become familiar with the process of Operations Research: learning how to create and validate a mathematical model, as well as the processes and optimization/sub-optimization. They will be able to create and solve Markovian and general queuing models. They will also learn how to use decision trees to determine optimal policies in the face of uncertainty. They will learn how to determine optimal inventory policies under the assumption of variable demand. They will complete this process for a variety of model types; however, all of the types of modeling covered in this course will be stochastic, that is, including uncertainty.  Reliance on the tools in the Calculus, Linear Algebra and Probability will be substantial, but we will also examine the reasons why these tools provide us with an optimal solution in each scenario.  In addition, we will examine how multiple modeling procedures can be used to arrive at the same result, as well as the benefits and pitfalls of the different techniques. Furthermore, students will learn a procedure called sensitivity analysis, which is used to determine what types of changes are necessary for our optimal solution to become sub-optimal. Use of some of the leading software in the field, which is included in the text, will be required.


a)      Topical Outline


1.       Markov Chains


Stochastic Processes

Discrete Time Markov Chains

Chapman-Kolmogorov Equations

Transition Matrices

Steady-State Behavior

Passage Times

Absorbing and Transient States

Continuous Time Markov Chains (Markov Processes)


2.       Queuing Theory


Exponential Distribution

Birth-Death Processes

Single Server Queues

Finite, Multiple Server Queues

Little’s Law

Finite and Infinite Capacity Queues


3.       Decision Trees


4.       Stochastic Inventory Theory


Continuous Review Models

Periodic Review Models

Models Involving Perishables


5.       Stochastic Dynamic Programming and Markov Decision Processes


6.       Reliability Theory


Parallel Systems

Series Systems

Mixed Systems


b)      Evaluation and Grading


Students will be evaluated by traditional methods of homework, which will include analytic and computer-based problems, and written exams.  Students will also prepare solutions to class projects and be required to make a brief presentation at the end of the semester.  Additional methods, such as journal reviews, may also be used.


c)      Course Evaluation


The course will be evaluated through customary student evaluations as well as regular departmental review.