However, using such components also adds a large amount of risk to the software development project. External components typically do not follow the same schedule as the software application. In addition, external components can have a release for which the quality is suspect and unknown, and it is a problem taking such a component. We were incorporating some open source components for which we knew that quality could be a problem sometimes on the latest release (and hence we would always take a previous stable release), but for a couple of years, we were running into a problem even in a case where we were paying a vendor for the component delivery.
The component was having quality issues which indicated that a comprehensive quality scan was not happening, and this would only be clear only when some detailed testing was happening at our end (in one case, a significant problem only happened during the testing conducted by a Beta tester). Even while discussion was happening with the vendor to synchronize our testing policies, there was a risk associated with the component that needs to be handled.
In our case, given the number of components that we integrated and in some cases, our lack of control on the schedule of the external component, we needed to figure out the way to manage these risks. The only way we could do this was to do a much more comprehensive testing at our end. However, given that we had an iterative cycle where we would receive a component, report issues, get a new version of the component, and so on, it was very expensive in terms of testing time to keep on doing a comprehensive testing of these components every time we received a new version of this component.
We did have a budget for automation of some of our testing cases, and we decided to focus on these components. The execution plan was simple - prioritize the components that were received multiple times and automate the major testing cases for these components. Running these automation cases took significantly much less time time than the manual testing and found defects much earlier. This also gave a learning to use that we shared with other teams, since the benefits of automation of testing in these cases was much higher - after all, the cycle of testing-defect-fix-test was much more efficient when the code was written by the developers in the product rather than when this was related to a component written by an external team.