Extending OSI to Network Security

Solutions in this chapter:

Introduction

“Everything old becomes new again.” The goal of this chapter is to take the well-known Open Systems Interconnect (OSI) model and use it to present security topics in a new and unique way. While each of the subsequent chapters focuses on one individual layer, this chapter offers a high-level overview of the entire book.

Our Approach to This Book

This book is compiled of issues and concerns that security professionals must deal with on a daily basis. We look at common attack patterns and how they are made possible. Many attacks occur because of poor protocol design; others occur because of poor programming or lack of forethought when designing code. Finally, the tools that are useful for identifying and analyzing exploits and exposures are discussed—the tools you will return to time and time again.

Tools of the Trade

The following sections examine “protocol analyzers” and the Intrusion Detection Systems (IDSes), which are the two main tools used throughout this book.

Protocol Analyzers

Protocol analyzers (or sniffers) are powerful programs that work by placing the host system’s network card into promiscuous mode, thereby allowing it to receive all of the data it sees in that particular collision domain. Passive sniffing is performed when a user is on a hub. When using a hub, all traffic is sent to all ports; thus, all a security professional or attacker has to do is start the sniffer and wait for someone on the same collision domain to begin transmitting data. A collision domain is a network segment that is shared but not bridged or switched; packets collide because users are sharing the same bandwidth.

Sniffing performed on a switched network is known as active sniffing, because it switches segment traffic and knows which particular port to send traffic to. While this feature adds much needed performance, it also raises a barrier when attempting to sniff all potential switched ports. One way to overcome this impediment is to configure the switch to mirror a port. Attackers may not have this capability, so their best hope of bypassing the functionality of the switch is through poisoning and flooding (discussed in subsequent chapters).

Sniffers operate at the data link layer of the OSI model, which means they do not have to play by the same rules as the applications and services that reside further up the stack. Sniffers can capture everything on the wire and record it for later review. They allow user’s to see all of the data contained in the packet. While sniffers are still a powerful tool in the hands of an attacker, they have lost some of their mystical status as many more people are using encryption.

The sniffer used in this book is called Ethereal, which is free and works well in both a Windows and a Linux environment. (Chapter 3 provides a more in-depth review of how to install and use Ethereal.) If you’re eager to start using Ethereal, more details about the program can be found at www.ethereal.com. (Ethereal’s name has been changed to Wireshark.)

Intrusion Detection Systems

Intrusion detection systems (IDSes) play a critical role in protecting the Information Technology (IT) infrastructure. Intrusion detection involves monitoring network traffic, detecting attempts to gain unauthorized access to a system or resource, and notifying the appropriate individuals so that counteractions can be taken. The ability to analyze vulnerabilities and attacks with a sniffer and then craft a defense with an IDS is a powerful combination. The IDS system used in this book is Snort, which can be used with both Linux and Windows and has industry wide support.

Snort is a freeware IDS developed by Martin Roesch and Brian Caswell. It’s a lightweight, network-based IDS that can be set up on a Linux or Windows host. While the core program uses a Command Line Interface (CLI), graphical user interfaces (GUIs) can also be used. Snort operates as a network sniffer and logs activity that matches predefined signatures. Signatures can be designed for a wide range of traffic, including Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Internet Control Message Protocol (ICMP).

Snort consists of two basic parts:

■ Header Where the rules “actions” are identified

■ Options Where the rules “alert messages” are identified

To learn more about Snort, go to www.Snort.org.

Organization of This Book

This book is arranged in the same manner as the layers of the OSI model, which was developed to provide organization and structure to the world of networking. In 1983, the International Organization for Standardization (ISO) and the International Telegraph and Telephone Consultative Committee (CCITT) merged documents and developed the OSI model, which is based on a specific hierarchy where each layer builds on the output of each adjacent layer (see ISO 7498). Today, it is widely used as a guide for describing the operation of a networking environment, and also serves as a teaching model for hacks, attacks, and defenses.

The OSI model is a protocol stack where the lower layers deal primarily with hardware, and the upper layers deal primarily with software. The OSI model’s seven layers are designed so that control is passed down from layer to layer. The seven layers of the OSI model are shown in Table 1.1

The OSI model functions as follows:

1. Information is introduced into the application layer and passed down until it ends up at the physical layer.

2. Next, it is transmitted over the physical medium (i.e., wire, coax, or wireless) and sent to the target device.

3. Once at the target device, it proceeds back up the stack to the application layer.

For this book, an eighth layer has been added to the OSI model that is called the “people” layer (or “social” layer). Figure 1.1 shows the eight layers and interprets the services of each.

The People Layer

Layer 8 is known as the people layer, and while not an official layer of the OSI model, it is an important consideration; therefore, it has been added to the OSI model for this book. People are often the weakest link. We can implement the best security solutions known at the lower layers of the OSI model and still be vulnerable through people and employees. Social...