You are providing a service to some clients, and they have come to totally depend on you. Like the post before this one, if you are providing analytics information to external teams, you have a lot of responsibility on your head. They would come to depend on you and if you are doing a good job for them, more and more, they will not think about any problems at your end and will depend on you to inform them if you are running into any problem. And that places an awful lot of responsibility on your end. Towards this purpose, you have to have a strategy to figure out what to do if something goes wrong at your end, and informing your clients quickly should be a part of such a strategy,
We recently had a big shock when it came to our analytics program, where we were using an external component. We were integrating such a component, and passing it parameters from within our code. What this ended up doing was passing data from our user base to the database maintained by the external team, and they would let us mine this information through a web front end, and they would also help us if we needed some new queries or new reports.
All this was perfect, and we did not bother anymore to check whether there could be a problem of any kind (so part of the problem was at our end before of a lack of checking for variables that could impact the program). We were looking at some data pursuant to the launch of a new version of the application, and in the second to fourth week of the launch, we saw data that indicated that the number of people using the older version of the program had reduced. This enthused the Product Management and other senior stakeholders, and a decision was made to target the older users through an email campaign sent to those users who were on our email list and had purchased previous versions of the program. This campaign would build on this conversion and offer some benefits if they did an upgrade, the idea being that if there are people doing an upgrade, it would help to push even more people.
This campaign was made ready in record time, and pushed live for a week. Like any other campaign, this program had some large costs because of the resources used, and the infrastructure cost for tracking of the email campaign, including measuring the conversion numbers. However, at the end of the week, it was very surprising that the conversion rate did not show any uptick; and in a coincidence, a day later we received an email from the analytics team that they were suspecting some configuration issues on their side, and they have probed, and finally figured out that some data had been lost, which also overlapped the weeks that made us decide that people using the older versions had dropped.
This was like a big blow, and since the analytics team was part of the same organization, we bit back the strong words, put in the issue for being more careful the next time that we did a campaign based on data. However, senior stakeholders did not hold back and had an interesting discussion with the head of the analytics team about not sending out a message to the various client teams when they suspected some data issues. The analytics team wanted to be sure before sending out a message to the client teams, but it was a learning to all of us. Since none of us really knows when another team is in a critical position and are dependent on us, as soon as we run into some problem, it is imperative to quantify the risk position and send it out the client teams that are dependent on us. This helps them avoid business decisions that are impacted if the data turns out to be problematic; even if it was a false alarm, there is some loss, but in general, the loss might be less than making a decision based on data that is incorrect.
We recently had a big shock when it came to our analytics program, where we were using an external component. We were integrating such a component, and passing it parameters from within our code. What this ended up doing was passing data from our user base to the database maintained by the external team, and they would let us mine this information through a web front end, and they would also help us if we needed some new queries or new reports.
All this was perfect, and we did not bother anymore to check whether there could be a problem of any kind (so part of the problem was at our end before of a lack of checking for variables that could impact the program). We were looking at some data pursuant to the launch of a new version of the application, and in the second to fourth week of the launch, we saw data that indicated that the number of people using the older version of the program had reduced. This enthused the Product Management and other senior stakeholders, and a decision was made to target the older users through an email campaign sent to those users who were on our email list and had purchased previous versions of the program. This campaign would build on this conversion and offer some benefits if they did an upgrade, the idea being that if there are people doing an upgrade, it would help to push even more people.
This campaign was made ready in record time, and pushed live for a week. Like any other campaign, this program had some large costs because of the resources used, and the infrastructure cost for tracking of the email campaign, including measuring the conversion numbers. However, at the end of the week, it was very surprising that the conversion rate did not show any uptick; and in a coincidence, a day later we received an email from the analytics team that they were suspecting some configuration issues on their side, and they have probed, and finally figured out that some data had been lost, which also overlapped the weeks that made us decide that people using the older versions had dropped.
This was like a big blow, and since the analytics team was part of the same organization, we bit back the strong words, put in the issue for being more careful the next time that we did a campaign based on data. However, senior stakeholders did not hold back and had an interesting discussion with the head of the analytics team about not sending out a message to the various client teams when they suspected some data issues. The analytics team wanted to be sure before sending out a message to the client teams, but it was a learning to all of us. Since none of us really knows when another team is in a critical position and are dependent on us, as soon as we run into some problem, it is imperative to quantify the risk position and send it out the client teams that are dependent on us. This helps them avoid business decisions that are impacted if the data turns out to be problematic; even if it was a false alarm, there is some loss, but in general, the loss might be less than making a decision based on data that is incorrect.
No comments:
Post a Comment