What is meant by load shedding?

The network is monitored by the network monitoring systems. These systems need to be robust and must be capable of inevitably coping with the situations in which the overload occurs. The network gets overloaded because of the nodes generating large volumes of data at high rates. Overload might also occur because of the burstiness of the traffic in its normal course of operation. For reducing the load of the network, load shedding techniques are applied. 

- Load shedding techniques have to be followed if the network is under a lot of stress. 

- This has to be done while monitoring the network for avoiding the packet loss that otherwise might be uncontrollable. 

- Load shedding involves sampling the incoming traffic. 

- CoMo or continuous monitoring has been developed to serve this purpose. 

- It uses such a load shedding scheme which can infer the query’s cost using the relation between the set of features of the traffic and the actual resource usage without having any knowledge of the plug-ins. 

- Here, traffic feature can be defined as a counter describing the incoming traffic’s particular property. 

The property might be any of the following:

Ø  Number of packets

Ø  Number of bytes

Ø  Flows

Ø  Unique IP destination address and so on.

- The CoMo consists of a prediction and the load shedding sub-system for intercepting the packets prior to sending them to the plug-in from the filter.

- A traffic query is implemented by this plug-in. 

- The system completes the process in 4 phases. 

- In the first phase, it forms a batch of packets for each 100ms of the traffic. - It then processes each of these batches for extracting a predefined traffic features’ set that is quite large. 

- From these, the most relevant sets are selected by the feature selection sub-system based up on the present stats of the CPU usage of the query. 

- The selected subset is then supplied as input for the “multiple linear regression subsystem”. 

- This is done for the prediction of the CPU cycles that the query requires for processing the whole batch. 

- If the prediction is greater than the capacity of the system, the batch is pre-processed by the load shedding subsystem for discarding the packet’s portion. 

- The batch is discarded through packet or flow sampling. 

Load shedding is now being seen as an effective method for curbing the overload situations even in the real time systems. 

- It involves shedding excess of the load in such a way that the stability of the system is not disturbed and also the system buffers do not experience any overflows. 

- The idea for applying the technique of load shedding in the field of networking has been adopted from the concept of the electric power management.

- Here, the electric current is intentionally disconnected on particular lines when the demands for the power supply are higher than what is being supplied.

- CoMo is an open source system and can be quickly implemented and can be used for further deploying other network monitoring applications. 

- The system has been written using C language and uses a feature rich API. 

- The system works by predicting the CPU usage of the system and thus anticipates about the resource requirements bursts that might occur in future. 

- The load shedding scheme used by the CoMo has the capability of automatically identifying the features using which the resource usage can be best modeled for each monitoring application.

- This identification is made according to the previous resource usage measurements. 

- These measurements are then used for determining the system’s overall load and by what percentage the load must be shed.