Sharing of test data with vendors along with test cases ..

In several previous posts, I have been outlining some details with respect to interaction with vendors, for example - (Outlining focus areas of testing, Process for answering queries from vendors, Extracting relevant test cases); all of these outline some areas of interaction with the vendors. Working with vendors cannot be just one single area, there is a lot related to information regarding processes, tools to be used, focus areas to be decided, and so on. Yet another area of interaction between the vendors and the core testing team is the data to be used for the various areas of testing.
Having testing data is very important. You may have all the required test cases for doing a complete feature testing of the application, but without the test data, all this does not mean anything. For example, if you are testing a video application, then you need to be able to test for the different video formats that the application would support, and there are a large number of such video formats that are needed to be tested (unless you are in the video domain, you would not really believe the number of formats that exist because of the large number of companies that do something for video). For each such different type of video format, you would need to have the required data in this regard - multiple number of video files in each format (as another example, you could have a video file that is just a collection of images, another without audio, and yet another that has a combination of video and audio - there could be a defect that is only found if the video file does not have any audio - we once found a defect like this and it was a big pain to figure out what the problem was; after this, we had to add files with and without audio to our testing matrix).
And video files are large. The total size of the video test data that we used for complete testing was more than 50 GB in size, and it was guarded with great care. Because of the sheer size of the test data, we had decided not to put this test data in the source safe we used (because the data backup of the source safe meant that the guys running the source safe were not happy over backing up so much data that was binary rather than being coding text files) and hence had made multiple disk copies of the this test data, and there was some amount of effort required to ensure that there was a master copy of the data and all copies were synchronized to this copy.
Are you getting an idea of the problem we faced when we added a vendor testing team to this matrix. We had to tell them the focus area of testing, we had to prepare the extraction of the required test cases and make sure that these made sense for somebody wanting to do the testing who did not have the same amount of experience as the core team, and then we had to also pass on the large test data to the vendor while linking this test data to the test cases. Even though we had a fast connection to the vendor, there were some permission requests that also had to be processed since some of the test data that we had was from the vendors with no permissions to pass on, and hence every time we needed to pass them onto a vendor, we needed permission (no monetary problem, just the paperwork and time required), and since the test data was changing with new video cameras entering into the market, there were synchronization problems. And in an unusual case, we found that a particular set of high end video files required a very high end machine and this information was not captured properly. So when the vendor tries to use those files, things were not working.
The challenges may differ depending of the type of test data, but there is a need to ensure that there is a strategy to prepare for the passing on the required test data to the vendor.

Infrastructure: Testing, VPN and network issues relating to external teams

When dealing with testing or development by external teams, there are a lot of infrastructural issues to deal with, and in a number of cases, teams have not prepared some kind of process documentation to take care of such needs and processes. Some of the external teams that can be involved in the process of development or testing for a software application include the following:
- External teams that are working as an extension of the testing team. This is getting more and more common, whereby the testing team can be expanded as necessary by adding vendors to the testing resources. However, when the these external vendors need to start their work, they need access to the testing infrastructure such as test cases repository and the defect management software.
- External teams working on the localization (conversion of the software into different languages). Such teams typically work on both development and testing of the application in different languages and need access to the source repository, as well as the testing infrastructure (test plan and case repository, defect management software, and so on).
- Teams working on the documentation of the product typically need access to the defect management software, since there can be several issues that need to be documented which are typically present in the defect management software, and the documentation team would be provided access to the defect management software.

For these 3 teams and others working in a similar area, if they are located outside the organization, then there may be a necessity for them to be provide the required VPN access to get access to all these tools. In modern secure organizations, such access policies would go through an approval process, and to ensure that such a process is accelerated, it makes sense to prep the approving authorities about these requests for approval.

- External testers / pre-release testers. Such pre-release testers typically do not need any access to source code repository or the testing infrastructure. However, these testers do need a platform where the defects they have logged get into the defect management software in an easy and transparent way (and where the testers are not expected to do any additional work - in some cases, it is as simple as providing them a web page where they can enter the defects that they have found, and some additional parameters that testers  are expected to enter but pre-release testers are not expected to enter are entered by default).

For all these teams, there may be other access issues that teams may not have thought about. Typically when a team has any kind of feature that provides access to an online feature, such features are available in staged servers that need some kind of setting or special access. For ensuring that all the teams listed above and similar teams do not get stuck, it is required to plan for such an access. We had a problem in the past whereby an important online feature was rolled out to the press reviewing team, but there was a big screw up relating to the access for these features, and the team was not able to provide access for around 3 days, in which time, some of the press withdrew - a big disaster. So, a great team has plans for all these before they look to execute such features.