Applied Probability and Stochastic Processes

Preface

Contents

Basic Probability Review

Basic Definitions

Random Variables and Distribution Functions

Mean and Variance

Important Distributions

Multivariate Distributions

Combinations of Random Variables

Fixed Sum of Random Variables

Random Sum of Random Variables

Mixtures of Random Variables

Appendix

Problems

References

Basics of Monte Carlo Simulation

Simulation by Hand

Generation of Random Numbers

Multiplicative Linear Congruential Generators

A Multiple Recursive Generator

Composite Generators

Generation of Random Variates

Discrete Random Variates

Continuous Random Variates

Bivariate Continuous Random Variates

Random Variates from Empirical Distributions

Appendix

Problems

References

Basic Statistical Review

Collection of Data

Preliminary Definitions

Graphical Representations

Parameter Estimation

Method of Moments

Maximum Likelihood Estimation

Confidence Intervals

Means

Proportions

Variances

Correlation Coefficient

Fitting Distributions

The Chi-Square Test

The Kolmogorov-Smirnov Test

Fitting Models

Ordinary Least Squares Regression

Maximum Likelihood Estimates for Linear Models

Appendix

Problems

References

Poisson Processes

Basic Definitions

Properties and Computations

Extensions of a Poisson Process

Compound Poisson Processes

Non-stationary Poisson Process

Poisson Regression

Poisson Regression with One Independent Variable

Poisson Regression with Several Independent Variables

Appendix

Problems

References

Markov Chains

Basic Definitions

Multistep Transitions

Classification of States

Steady-State Behavior

Computations

Appendix

Problems

References

Markov Processes

Basic Definitions

Steady-State Properties

Revenues and Costs

Time-Dependent Probabilities

Appendix

Problems

Queueing Processes

Basic Definitions and Notation

Single Server Systems

Infinite Capacity Single-Server Systems

Finite Capacity Single Server Systems

Multiple Server Queues

Approximations

Appendix

Problems

References

Queueing Networks

Jackson Networks

Open Jackson Networks

Closed Jackson Networks

Network Approximations

Deterministic Routing with Poisson Input

Deterministic Routing with non-Poisson Input

Appendix

Problems

References

Event-Driven Simulation and Output Analyses

Event-Driven Simulations

Statistical Analysis of Output

Terminating Simulations

Steady-State Simulations

Comparing Systems

Problems

References

Inventory Theory

The News-Vendor Problem

Single-Period Inventory

No Setup Costs

Setup Costs

Multi-Period Inventory

Problems

References

Replacement Theory

Age Replacement

Discrete Life Times

Continuous Life Times

Minimal Repair

Minimal Repairs without Early Replacements

Minimal Repairs with Early Replacements

Block Replacement

Problems

Markov Decision Processes

Basic Definitions

Expected Total Discounted Cost Criterion

Average Long-Run Cost Criterion

Stationary Policies

Discounted Cost Algorithms

Value Improvement for Discounted Costs

Policy Improvement for Discounted Costs

Linear Programming for Discounted Costs

Average Cost Algorithms

Policy Improvement for Average Costs

Linear Programming for Average Costs

The Optimal Stopping Problem

Problems

Advanced Queues

Difference Equations

Batch Arrivals

Quasi-Birth-Death Processes

Batch Arrivals (continued)

Phase-Type Distributions

Systems with Phase-Type Service

The M/Ph/1 Queueing System

The M/Ph/c Queueing System

Systems with Phase-Type Arrivals

Problems

References

Matrix Review

Matrix Addition and Subtraction

Matrix Multiplication

Determinants

Determinants by Cofactor Expansion

Nonsingular Matrices

Inversion-in-Place

Derivatives

References

Index