What is meant by load shedding?

The network is monitored by the network monitoring systems. These systems need to be robust and must be capable of inevitably coping with the situations in which the overload occurs. The network gets overloaded because of the nodes generating large volumes of data at high rates. Overload might also occur because of the burstiness of the traffic in its normal course of operation. For reducing the load of the network, load shedding techniques are applied. 

- Load shedding techniques have to be followed if the network is under a lot of stress. 

- This has to be done while monitoring the network for avoiding the packet loss that otherwise might be uncontrollable. 

- Load shedding involves sampling the incoming traffic. 

- CoMo or continuous monitoring has been developed to serve this purpose. 

- It uses such a load shedding scheme which can infer the query’s cost using the relation between the set of features of the traffic and the actual resource usage without having any knowledge of the plug-ins. 

- Here, traffic feature can be defined as a counter describing the incoming traffic’s particular property. 

The property might be any of the following:

Ø  Number of packets

Ø  Number of bytes

Ø  Flows

Ø  Unique IP destination address and so on.

- The CoMo consists of a prediction and the load shedding sub-system for intercepting the packets prior to sending them to the plug-in from the filter.

- A traffic query is implemented by this plug-in. 

- The system completes the process in 4 phases. 

- In the first phase, it forms a batch of packets for each 100ms of the traffic. - It then processes each of these batches for extracting a predefined traffic features’ set that is quite large. 

- From these, the most relevant sets are selected by the feature selection sub-system based up on the present stats of the CPU usage of the query. 

- The selected subset is then supplied as input for the “multiple linear regression subsystem”. 

- This is done for the prediction of the CPU cycles that the query requires for processing the whole batch. 

- If the prediction is greater than the capacity of the system, the batch is pre-processed by the load shedding subsystem for discarding the packet’s portion. 

- The batch is discarded through packet or flow sampling. 

Load shedding is now being seen as an effective method for curbing the overload situations even in the real time systems. 

- It involves shedding excess of the load in such a way that the stability of the system is not disturbed and also the system buffers do not experience any overflows. 

- The idea for applying the technique of load shedding in the field of networking has been adopted from the concept of the electric power management.

- Here, the electric current is intentionally disconnected on particular lines when the demands for the power supply are higher than what is being supplied.

- CoMo is an open source system and can be quickly implemented and can be used for further deploying other network monitoring applications. 

- The system has been written using C language and uses a feature rich API. 

- The system works by predicting the CPU usage of the system and thus anticipates about the resource requirements bursts that might occur in future. 

- The load shedding scheme used by the CoMo has the capability of automatically identifying the features using which the resource usage can be best modeled for each monitoring application.

- This identification is made according to the previous resource usage measurements. 

- These measurements are then used for determining the system’s overall load and by what percentage the load must be shed. 

Describe some Caching Issues?

We all are familiar with what a cache is? 

"A cache can be defined as a memory component that is held for storing the data transparently in order to speed up the future serving of the data requests."

The data stored in a cache might be the data that has been required earlier for some operations. 

There are two events related to cache as mentioned below:

1. Cache hit: When the requested data is available in cache and

2. Cache miss: When the requested data is not available in cache and has to be looked up in to the RAM.

- The speed of processing is directly proportional to the number of requests that can be served via the cache. 

- Cache memories are quite costly and hence to make it cost efficient and keep the data usage as efficient as possible, smaller caches are used. 

- But since the time of its advent, cache has proven itself in the field of computing. 

"Caching can be thought of as a technique that is aimed at increasing the computing performance by keeping in itself the frequently accessed data."

There are basically three kinds of caching as we have stated below:

1. Caching output caching: In this type of caching the dynamic output that had been generated up on a request.

2. Fragment caching: In this type of caching the portion of the page that is generated by the request is cached since in many situations it is not practical to cache the whole page at once.

3. Data caching: In this type of caching the objects are cached pro grammatically.

What are different caching issues?

In this article we have taken in to discussion some very prominent caching issues. Most of the people experience problem in server caching of certain files. 

There are four major caching issues have been recognized which have been mentioned below:

1. Designing of a custom cache.

2. Securing of a custom cache.

3. Monitoring of a custom cache and lastly.

4. Synchronization of the caches in a server farm.

Besides these four major caching issues, there are many other minor caching issues.

- Some times it happens that the package delivery fails or an object or element appears like it has been corrupted and seems like such a failure has not got anything to do with the connection! 

- In such cases you can go on with a cache clear up. 

- If the situation is worse, you may also require clearing up the proxy cache!

- Web sites and browsers are looked up as a means of optimizing the resources which is done by them so well that they end up breaking down your dynamic web site content. 

- Your web site is not updated as you thought it will be done. 

Let us see an example, suppose you own a music web site which you frequently update with new music. Your clients come to your site every day so what happens is that the cache forces the web site to list the cached version of the play list and so the clients would never be able to listen to new and the latest music.

Such situation though enhances your internet experience, can also cause many other problems! Some times the cache will stash up an old page of the web site instead of showing up the latest one. You should make it a point to empty your browser’s cache from time to time. There are many internet service providers that cache pages to speed up the internet access like AOL. All the web pages that you visit are stashed up in the cache. 

What types of documents one need for QA, QC, and Software Testing?

For every process, some documents are vital and in this article we have discussed about the type of documents required in the below mentioned three processes:

       1. QA or quality assurance

       2. QC or quality control and

       3. Software testing

First let us see what are these three processes? 

Quality Assurance

"Quality assurance is a process that involves implementation of planned and systematic activities in a quality software system or application so that all the quality requirements of the software system or application under question are met". 

It has also got the following attributions:

1. Systematic measurement
2. Comparison with a standard
3. Monitoring of processes
4. Associated feedback loop
5. Error prevention

This whole process deals with two principles:

1. Fit for purpose and
2. Right first time

Quality Control Process

- It is a process involving a wholesome review of the quality of all the factors that have direct as well as indirect involvement in the production of the software or application. 

- This process is one of the best practices that are used for inspection of the software systems as well as applications. 

- The software products and artifacts are put through a visual examination.

- The developers or testers who are to examine the software system or application are provided with a list containing the description of the unacceptable software defects.

- The products and artifacts are visually tested and the defects are dug out and are reported to the management who are responsible for taking the action against software product release.

- This process plays a great role in the stabilization of the software production process.

Software Testing

- Software testing is a self justifying term and is just like an investigation seeking out defects and flaws in software systems and applications. 

- All the stake holders get to know about the quality of the software system or application under test.

- Tools here used are nothing but normal testing techniques that are intended to dig out the bugs and errors. 

- This process verifies:

1. Meets the requirements as stated in its documentation.
2. Works in the desired way.
3. Whether its implementation with same characteristics is possible or not.
4. Satisfaction of the stake holders.

Now let us mention what all documents are required for all the above discussed three processes:

1. First main document is the software requirements specifications document.
2. Use  cases document
3. Solution document
4. Software design documents
5. Test plan document: this document should contained detail description of the following:

(a)  Scope of the functionality the test case will test.

(b)  Expected outcome

(c)  Technique used

6. Test cases documentation containing procedure as well as the obtained results.
7. Business requirements documents
8. Functional specifications documents
9. Project member details documents containing information about the team members including testers, test lead, pm etc.
10. Software testing schedule document.
11. Traceability matrix: this document is used to check whether or not the test cases match with the requirements stated in the SRS.
12. Documents which are specific to a particular organization for quality control.
13. Discovery documents (only for quality control): this document states the business needs.
14. Test reports
15. Bug reports: this report includes all the missing, additional, wrong deviations in the functionalities or features of the software system or application.
16.  Release report: obtained at the end of testing.
17. Test scenarios
18. Test case templates
19. Test case form
20. Logs
21. Weekly status reports
22. Test scripts
23. Resolution
24. Test bed

Business requirements specification (BRS) and software requirements specification (SRS) are a must for the quality assurance and quality control processes. 

What are tools and techniques to monitor test progress?

Why do we need to monitor the test progress? What is the purpose of monitoring the tests we are performing? 

- The monitoring of the test progress is aimed at giving feed back and visibility regarding the test activities. 

- The information which has to be put under monitoring is collected either automatically or manually and this information might be used for making the measurements of the exit criteria like coverage criteria etc. 

- In some cases, metrics are also used to make an assessment of the progress being made by the tests against a very well planned project budget and development schedule. 

Tools and Techniques to Monitor Test Progress

Test metrics: These include the following:

-  Percentage of work done during preparation of the test.

-  Percentage of work done during the preparation of the test environment.

- Number of test cases run and not run.

-  Number of test cases passed and failed.

-  Details of the defects.

-  Test coverage of code, risk and requirements etc.

-  A subjective measure of the confidence of the testers in the software product.

-  Dates of the testing phase milestones and deadline.

- Testing costs

1. Soon after you have drawn out your test plan, you need to constantly monitor the test progress along with the activities and time progress. 

2. Monitoring the time scales and the test activities together is termed as test monitoring. 

3. The purpose of test monitoring is to obtain a constant feed back.

4. The test monitoring makes all the test activities visible to the developers and testers.

5. The data that has to be monitored apart form being tested by counting the test cases can also be obtained using the sophisticated test management tools. 

6. With such sophisticated tools, it has also become possible to collect the data like an automatic output from another tool and that has already been formatted in to form of reports or data files. 

7. These data files as well as reports can be manipulated to present a flow of the test progress. 8. The above mentioned test metrics are usually used for tracking the progress of the tests towards the end of the testing phase which is marked by the validation of the test criteria.

9. Nowadays, test dashboards are in use because of their easy to use and understand interface which represents all of the relevant test metrics on single screen page and thus providing the maximum impact. 

10. This gained too much popularity in a short span of time since it saved a lot of time of the users who do not want to go through the large piles of data searching for just one key information they want. 

11. These test metrics are often presented in a graphical form.

12. While monitoring a testing process you need to look at the test progress through both a micro as well as macro perspective. 

13. The macro perspective involves determining whether or not the individual devices and servers are running smoothly as expected.

14. On the other hand, the micro perspective involves making sure that the customers experience pleasure while accessing and using your software system or product. 

15. These two types of perspectives can also be termed as the “engineering monitoring” and “business monitoring” respectively. 

16. Both these kinds of monitoring are equally important. 

17. Good testers always make it a point to monitor the progress of the tests they have put in to play on the software product. 

18. They do not consider their software system or application to be complete until and unless they have ran their software product through a couple of monitoring checks. 

What is Long Session Soak Testing ?

When an application is used for long periods of time each day, the above approach should be modified, because the soak test driver is not logins and transactions per day, but transactions per active user for each user each day. This type of situation occurs in internal systems, such as ERP and CRM systems, where user logins and stay logged in for many hours, executing a number of business transactions during that time. A soak test for such a system should emulate multiple days of activity in a compacted time frame rather than just pump multiple days worth of transactions through the system.

Long session soak tests should run with realistic user concurrency, but the focus should be on the number of transactions processed. VUGen scripts used in long session soak testing may need to be more sophisticated than short session scripts, as they must be capable of running a long series of business transactions over a prolonged period of time.

The duration of most soak tests is often determined by the available time in the test lab. There are many applications that require extremely long soak tests. Any application that must run, uninterrupted for extended periods of time, may need a soak test to cover all of the activity for a period of time that is agreed to by the stakeholders. Most systems have a regular maintenance window, and the time between such windows is usually a key driver for determining the scope of soak test.

Overview of Soak testing.

Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day than would be expected in a busy day, to identify any performance problems that appear after a large number of transactions have been executed. Also, it is possible that a system may stop working after a certain number of transactions have been processed due to memory leaks or other defects. Soak tests provide an opportunity to identify such defects, whereas load tests and stress tests may not find such problems due to their relatively short duration.A soak test would run for as long as possible, given the limitations of the testing situation. For example, weekends are often an opportune time for a soak test.

There are some typical problems identified during soak tests are:
- Serious memory leaks that would eventually result in memory crisis.
- Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
- Failure to close database cursors under some conditions which would eventually result in the entire system stalling.
- Gradual degradation of response time of some functions as internal data structures becomes less efficient during a long test.

Apart from monitoring response time, it is also important to measure CPU usage and available memory. If a server process needs to be available for the application to operate, it is often worthwhile to record its memory usage at the start and end of the soak test. It is also important to monitor internal memory usages of facilities such as Java virtual machines, if applicable.