The network is monitored by the network monitoring
systems. These systems need to be robust and must be capable of inevitably
coping with the situations in which the overload occurs. The network gets
overloaded because of the nodes generating large volumes of data at high rates.
Overload might also occur because of the burstiness of the traffic in its
normal course of operation. For reducing the load of the network, load shedding
techniques are applied.
- Load shedding techniques have to be followed if the network
is under a lot of stress.
- This has to be done while monitoring the network for
avoiding the packet loss that otherwise might be uncontrollable.
- Load
shedding involves sampling the incoming traffic.
- CoMo or continuous monitoring has
been developed to serve this purpose.
- It uses such a load shedding scheme
which can infer the query’s cost using the relation between the set of features
of the traffic and the actual resource usage without having any knowledge of
the plug-ins.
- Here, traffic feature can be defined as a counter describing the
incoming traffic’s particular property.
The property might be any of the
following:
Ø Number of
packets
Ø Number of
bytes
Ø Flows
Ø Unique IP
destination address and so on.
- The CoMo consists of a prediction and the load
shedding sub-system for intercepting the packets prior to sending them to the
plug-in from the filter.
- A traffic query is implemented by this plug-in.
- The
system completes the process in 4 phases.
- In the first phase, it forms a batch
of packets for each 100ms of the traffic. - It then processes each of these
batches for extracting a predefined traffic features’ set that is quite large.
- From
these, the most relevant sets are selected by the feature selection sub-system based up on the present stats of the CPU usage of the query.
- The
selected subset is then supplied as input for the “multiple linear regression
subsystem”.
- This is done for the prediction of the CPU cycles that the query
requires for processing the whole batch.
- If the prediction is greater than the
capacity of the system, the batch is pre-processed by the load shedding subsystem for discarding the packet’s portion.
- The batch is discarded through
packet or flow sampling.
Load shedding is now being seen as an effective method
for curbing the overload situations even in the real time systems.
- It
involves shedding excess of the load in such a way that the stability of the
system is not disturbed and also the system buffers do not experience any
overflows.
- The idea for applying the technique of load shedding in the field of
networking has been adopted from the concept of the electric power management.
- Here, the electric current is intentionally disconnected on particular lines
when the demands for the power supply are higher than what is being supplied.
- CoMo
is an open source system and can be quickly implemented and can be used for
further deploying other network monitoring applications.
- The system has been
written using C language and uses a feature rich API.
- The system works by
predicting the CPU usage of the system and thus anticipates about the resource
requirements bursts that might occur in future.
- The load shedding scheme used
by the CoMo has the capability of automatically identifying the features using
which the resource usage can be best modeled for each monitoring
application.
- This identification is made
according to the previous resource usage measurements.
- These measurements are
then used for determining the system’s overall load and by what percentage the
load must be shed.
