Sharing of test data with vendors along with test cases ..

In several previous posts, I have been outlining some details with respect to interaction with vendors, for example - (Outlining focus areas of testing, Process for answering queries from vendors, Extracting relevant test cases); all of these outline some areas of interaction with the vendors. Working with vendors cannot be just one single area, there is a lot related to information regarding processes, tools to be used, focus areas to be decided, and so on. Yet another area of interaction between the vendors and the core testing team is the data to be used for the various areas of testing.
Having testing data is very important. You may have all the required test cases for doing a complete feature testing of the application, but without the test data, all this does not mean anything. For example, if you are testing a video application, then you need to be able to test for the different video formats that the application would support, and there are a large number of such video formats that are needed to be tested (unless you are in the video domain, you would not really believe the number of formats that exist because of the large number of companies that do something for video). For each such different type of video format, you would need to have the required data in this regard - multiple number of video files in each format (as another example, you could have a video file that is just a collection of images, another without audio, and yet another that has a combination of video and audio - there could be a defect that is only found if the video file does not have any audio - we once found a defect like this and it was a big pain to figure out what the problem was; after this, we had to add files with and without audio to our testing matrix).
And video files are large. The total size of the video test data that we used for complete testing was more than 50 GB in size, and it was guarded with great care. Because of the sheer size of the test data, we had decided not to put this test data in the source safe we used (because the data backup of the source safe meant that the guys running the source safe were not happy over backing up so much data that was binary rather than being coding text files) and hence had made multiple disk copies of the this test data, and there was some amount of effort required to ensure that there was a master copy of the data and all copies were synchronized to this copy.
Are you getting an idea of the problem we faced when we added a vendor testing team to this matrix. We had to tell them the focus area of testing, we had to prepare the extraction of the required test cases and make sure that these made sense for somebody wanting to do the testing who did not have the same amount of experience as the core team, and then we had to also pass on the large test data to the vendor while linking this test data to the test cases. Even though we had a fast connection to the vendor, there were some permission requests that also had to be processed since some of the test data that we had was from the vendors with no permissions to pass on, and hence every time we needed to pass them onto a vendor, we needed permission (no monetary problem, just the paperwork and time required), and since the test data was changing with new video cameras entering into the market, there were synchronization problems. And in an unusual case, we found that a particular set of high end video files required a very high end machine and this information was not captured properly. So when the vendor tries to use those files, things were not working.
The challenges may differ depending of the type of test data, but there is a need to ensure that there is a strategy to prepare for the passing on the required test data to the vendor.

Explain the page replacement algorithms - FIFO, LRU, and Optimal

- Paging is used by most of the computer operating systems for the purpose of virtual memory management. 

- Whenever a page fault occurs, some pages are swapped in and swapped out. Who decides which pages are to be replaced and how? 

- This purpose is served by the page replacement algorithms. 

- Page replacement algorithms only decide which pages are to be page out or written to the disk when a page of the memory has to be allocated. 

- Paging takes place only upon the occurrence of a page fault.

- In such situations a free cannot suffice because either one is not available or because the number of available pages is less than threshold. 

- If a previously paged out page is reference again, then it has to be read in from the disk again. 

- But for doing this, the operating system has to wait for the completion of the input/ output operation. 

- The quality of a page replacement algorithm is denoted by the time it takes for a page in. 

- The lesser it is, the better the algorithm is. 

- The information about the access to page as provided by the hardware is studied by the page replacement algorithm and then it decides which pages should be replaced so that the number of page faults can be minimized. 

In this article we shall see about some of the page replacement algorithms.

FIFO (first – in, first – out): 

- This one being the simplest of all the page replacement algorithms has the lowest overhead and works by book – keeping in place of the OS. 

- All pages are stored in a queue in the memory by the operating system.

- The ones that have recently arrived are kept at the back while the old ones stand at the front end of the queue.

- While making replacement, the oldest page is selected and replaced. 

- Even though this replacement algorithm is cheap as well as intuitive, practically it does not perform well. 

- Therefore, it is rarely used in its original form. 

- VAX/ VMS operating systems make use of the FIFO replacement algorithm after making some modifications. 

- If a limited number of entries are skipped you get a partial second chance. 

Least recently used (LRU): 

- This replacement algorithm bears resemblance in name with the NRU. 

- However, difference is there which is that this algorithm follows that the page usage is tracked for a certain period of time. 

- It is based on the idea that the pages being used many times in current set of instructions will be used heavily in the next set of the instructions also. 

- Near optimal performance is provided by LRU algorithm in the theory however in practical it is quite expensive to be implemented. 

- For reducing the costs of the implementation of this algorithm, few implementation methods have been devised. 

- Out of these the linked list method proves to be the costliest. 

- This algorithm is so expensive because it involves moving the items about the memory which is indeed a very time consuming task. 

- There is another method that requires hardware support.

Optimal page replacement algorithm: 

- This is also known as the clairvoyant replacement algorithm. 

- The page that has to be swapped in, it is placed in place of the page which according to the operating system will be used in future after a very long time. - In practical, this algorithm is impossible to be implemented in case of the general purpose OS because approximate time when a page will be used is difficult to be predicted. 

Dealing with dependencies that can impact your project schedule ..

Dependencies are the bane of a project schedule, or as one of the developers in a team told me, dependencies and other risks are the reason that a team needs a Project or a Program Manager. I have had informal discussions with some of the senior members of the team, as well as the members of the management team, and there is a lot of discomfort in handling the various dependencies that bedevil a project; as well as doing all the negotiations that are required to mitigate the risk of a dependency, or when the dependency causes a problem to the project.
What am I talking about ? Well, consider the fact that in today's world, most projects have a lot of dependency on items outside the control of the core team. You may be getting some work done from outside the core team by an external team, you may be picking up a component that has been created by an external team (which can be somebody who you pay for delivering the component, or you may be picking up a component that is being done by a open source type of team which produces a component and you pick up the component as and when it is prepared).
However, anytime you are using an external component, you are exposing yourself to a dependency and to a high amount of risk (actually the risks depends to the extent on which the dependency can screw your project). Let me list down a few examples:
- You use a component that is fairly stable (say, being done from another team in your organization). The component is fairly stable, and even though new versions of the component are released from time to time, there is no major necessity of picking up a new version of the component. The dependency is there, but the risk is low and so the mitigation required is also low. Even if there is a problem with the latest version of the component, you can stay with the version that you already have.
- You use a component where there is a bug fix that is needed for solving a problem in your application. In such a case, the risk to your product or project schedule depends on the critical nature of the defect in your application. If the defect is critical to your application, then you have a high amount of dependency and need some mitigation plans. If the component has some problem, then you have to consider the impact on your project and have mitigation plans accordingly. If you are able to survive with the defect in your software, then you would not imperil your schedule if there is any problem in the delivery or the quality of the component.
- Now consider the case of a component that is critical to your product, or delivers a functionality that is critical to your component. For example, when you consider the Adobe Camera Raw and the products that incorporate it, one of the critical advantages that a new version of the Camera Raw provides is support for new Digital SLR's that have been released. So if you were releasing a new version of Photoshop or a new version of Lightroom, it is important for them to integrate the latest version of Adobe Camera Raw because of this support for new cameras. In such cases, if a component is delayed, it causes significant problems for the product schedule and needs a high degree of mitigation. In this case, the mitigation that is done by these products is by enabling this component to be updated later. However, the mitigation strategy for every such component or dependency may not be so easily solved, and the team needs to ensure that there is a lot of thought to how to mitigate these problems.

Explain the various File Operations

A number of operations can be carried out on a file. However, there are 6 basic file operations. As we know a file is an ADT or abstract data type. Certain operations need to be considered for defining a file. Operating systems makes calls to these operations. 

Following are the six basic operations:

1. Creation of a file: 

- This operation involves two steps. 

- Firstly, a sufficient space has to be found for storing the file in the file system. - Secondly, for this new file an entry should be made in the directory.

2. Writing to a file: 

- For writing data to a file, a system call has to be made with name and the data to be written as its arguments. 

- A writer pointer is kept by the system at the location in the file where the next write operation is to be carried out. 

- This pointer is updated whenever a write operation occurs.

3. Reading from a file: 

- Just like the write operation, in order to read information from a file, a system call has to be generated along with the name of the file and the location of the content to be read as its arguments. 

- Here, instead of a write pointer there is a read pointer that will indicate the location where the next read operation is to take place. 

- The location at which the current operation is being carried out is kept as a “per – process current – file – position” pointer since the process is either writing to or reading from the file. 

- The same pointer can be used by both the read and write operations in order to reduce the complexity of the system as well as for saving space.

4. Re-positioning within a file: 

- System carries out search in the directory looking for the appropriate entry. 

- When found, the current file position pointer is re-pointed to this position. 

- This file operation does not require carrying out any input or output operation in actual. 

- Another name for this file operation is the file seek.

5. Deletion of a file: 

- For deletion of the file, the system searches through the directory to find the appropriate entry. 

- When found, the space held by this file is released and the entry in the directory is destroyed so that this space can be reused by other files.  

6. Truncating a file: 

- Sometimes you may require deleting only the contents of a file while keeping it attributes. 

- Deleting the file and recreating it is not an efficient solution. 

- This file operation lets you to erase the contents of the file but save its attributes.

- But here the length attribute of the file will be changed to zero after truncation. 

- The file space is released after truncating.

The above mentioned six basic file operations constitute the minimal file operations set. These operations are primary ones and if combined can perform some other secondary file operations such as copying. A table known as the open file table is maintained by the operating system that stores all the information about the files that are currently open. When the file is closed, its entry is deleted from the open file table. Some files have to be opened explicitly with the function open() before using it. The name of the file is passed as an argument to this function. Then it looks in the directory for this file and an entry is made in the open file table.  Each file has some access rights. It is in these access modes that a process uses the file. A process can perform only those operations which are permitted by the access rights of the file. 

Working with vendors: Telling them the points of focus for the particular version ..

In some of the previous posts, I have been talking about the issues and processes involved in ensuring that the interaction with the vendors (especially when they are involved in testing) is fine. In the last post (Working with vendors to ensure that tools are synchronized), I talked about how the team should be spending time on defining a process for transferring knowledge about the tools used by the team during testing. Now, this is easy for tools that are in generic usage, such as tools for Defect Management, Source Safe, or for capturing the details of Test Plans and Test cases. However, this is more complicated when the tools are specialized such as Fiddler, and other such tools that take time for vendors to learn more.
In this line of thought, there are several other processes that the team needs to transfer to the vendor team and do this in a systematic way to increase the level of efficiency. Some of the processes that the team might need to convey to the vendor team are:
- Certain specific processes such as the dealing with defects that have existed for some time. Consider the case of a defect that has been present in the product for more than one version, and the team has decided that they do not want to make the fix. Similarly, there may be cases where there is a difference of opinion with respect to the functionality, but anybody who needs to do this testing will be able to log a defect. In such cases, the team members know about these defects and do not try to file them, but somebody who is new to that area such as a vendor team member will not know, and will file defects in such areas. It is for the product team to ensure that they have transferred instructions regarding how to handle these defects to the vendor. If this does not happen, there will be wastage of time when the vendors actually do these testing. I saw such a case whereby there were some new members to the vendor team and there was a delay in passing on such information, and suddenly there was a buzz in the team about the vendor finding a number of defects that the team members had not found. There was some pressure on the testing team, but then it was found that most of these defects were of the types that the team already knew about, and there was a sigh of relief in the testing team (they would have come under a lot of pressure if the vendors found a number of defects that the core testing team was not able to find).
- Similarly, there are certain points of focus for every release. Or it could be the case where this is a patch or dot release that is focusing on a certain area, and it is incumbent on the product team to pass on the application focus onto the vendor team. This is needed to ensure that the testing done by the vendor team focuses on the relevant area instead of spending time on other areas which are not relevant. The team needs to decide on the best way to do this, one way to do this is to take the application where test plans and test cases are captured and create a plan that shows the vendor the focus areas where testing needs to be done. However, this pre-supposes that the test cases are written in a way that a subset can be used by the vendors and would make sense.

Working with vendors: Ensuring that tools are synchronized ..

This is part of a series of discussions on how to work with people outside the software development core team, which in most cases are typically vendors. You would be working with vendors for a variety of reasons:
- Part of the testing needed is done by the vendors
- There is a sudden work to be done besides the core work, and the additional work testing is needed to be done by vendors
- The product is being localized into many languages and the testing in various languages is being done by vendors
- Multiple other reasons such as these, all of which raise the need for vendors to take part for testing.

Now, when you have vendors working with the core team for testing purposes, you will be lucky if there is a continuity in terms of the personnel from the vendor side over the years. Our experience has been that unless we are able to provide business to the vendors the year round, people on the vendor side move onto different projects and would not be so easily available the next time when there is a need for the vendor. In addition, there are times when there is a sudden need for testing, and at that time, testers from the vendor side who have experience in the product would not be available.
In previous posts (Process setting, Kickoff and knowledge transfer), I have already been talking about some processes involved in working with vendors who assist in testing, and how to ensure that getting them trained is done as easily as possible. However, there is another area where there is a need for ensuring that knowledge transition happens efficiently, and that is in the area of tools that are used for the process of preparing the data as well as doing the testing.
Applications such as Defect Management systems and source repository systems are easy for vendors to understand, given that even if the vendors have been using a different tool for these functions, the basic concept of using these tools does not change too much, and hence it should be fairly easy for them to pick up a new tool.
However, it gets problematic when the team is using specialized tools, and there is a high chance that the personnel from the vendor site would not have the experience in using such tools. We had this in a previous project, where we were using specific tools such as 'Fiddler' for testing out the flow between the desktop application and an online application. When we spoke to the vendor team around the start of the project, they had no experience in a similar tool, and yet it was necessary for them to do the testing (we could not do the testing of this tool in all the areas where it was the vendor responsibility, and hence it was important for the vendor to learn this tool). So, finally we had to run a project whereby we did a few rounds of web conferencing with the vendor and explained how to use this and other such tools. However, it would have been more efficient to have already prepared such a document for the various tools that the teams use, along with something like a self-learning document.

Read more about working with vendors in the next post (Processes when dealing with vendors)

Explain about the Paged Memory and Segmentation techniques?

Paging and segmentation, both are memory management techniques. 

What is Paging?

- This technique has been designed so that the system can store or retrieve data from the virtual memory or secondary memory of the system to be loaded in the main memory and used. 

- In this scheme, the data from the secondary memory is retrieved by the operating system in blocks of same size commonly known as the paging. 

- This is why the technique has been called the paging memory – management scheme. 

- This memory management scheme has a major advantage over the segmentation scheme. 

- The advantage is that non-contiguous address spaces are allowed. 

- In segmentation, non-contiguous physical address spaces are not allowed. 

- Before the paging actually came in to use, the whole program had to be fitted in to the contiguous memory space by the systems. 

- This in turn led to a number of issues related to fragmentation and storage. 

- Paging is very important for the implementation of the virtual memory in many of the operating systems that are general purpose. 

- With the help of paging memory management technique, the data that cannot be fitted in to the physical memory i.e., RAM can be easily used. 

- Paging actually comes in to play whenever a program makes an attempt for accessing the pages that have not been presently mapped to the main memory (RAM). 

- Such situation is termed as the page fault. 

- At this point the control is handed over to the operating system for handling the page fault.

- This is done in a way that it is not visible to the interrupt raising program. 

The operating system has to carry out the following instructions:

1. Determining the location of the requested data from the auxiliary storage.
2. Obtaining a page frame in the main memory that is empty to be used for storing the requested data.
3. Loading the data requested in to the empty page obtained above.
4. Making updates to the page table so that new data is only available.
5. Returning the control interrupting program and retrying to execute the same instruction that caused the fault.

What is Segmentation?

- This memory management technique involves dividing the main memory in to various sections or segments.

- In the system that makes use of this management technique, a value identifying the segment and its offset is contained in the reference to that memory location. 

- Object files that are produced during the compilation of the programs make use of the segments when they have to be linked together to form an image of the program and this image has to be loaded in to the memory.  

- For different program modules, different segments might be created. 

- Some programs may even share some of the segments.

- In one way, memory protection is implemented by means of memory segmentation only.

- Paging and segmentation can be combined together for memory protection. 

- The size of memory segment is not always fixed and can be as small as a byte. 

- Natural divisions such as the data tables or the individual routines are represented by the segments.

- This is to make the segmentation visible to the programmer. 

- With every segment, a set of permissions and length is associated. 

- A segment can be referred to by the process only in a way that is permitted by this set of permissions. 

- If this is not done, a segmentation fault is raised by the operating system. 

- Segments also consist of a flag that indicates the presence of the segment in the main memory of the system.