Startup slogans are inescapable in tech: Move fast. Break things. Minimum viable product. These are exciting ideas, for sure, but to put them to use in IT, you’ll need to tailor them to your context.
For IT executives running established businesses, the risk-reward scenario is different. Thousands or millions of customers depend on your infrastructure. And if a change takes down the online banking platform, for instance, significant losses may occur.
So, while the drumbeat for CIOs to be innovative is only growing louder, and digital transformations are under way across nearly every industry in pursuit of top-line growth, tech execs would be wise not to discount the risks inherent in their innovation efforts.
Following is advice on how to reduce the amount of risk your company will face in pursuing new opportunities — without blocking your organization’s ability to move forward.
Provide a safety net with real backups
The first word in mitigating IT risk is security. Hacking incidents continue to pose a significant challenge, as evidenced by Equifax’s epic breach in 2017. Such high-profile incidents mean cybersecurity has become a top priority, well-funded by many organizations. But what about backups?
Say your programmers just found a great new way to increase efficiency. No problems come up during the testing process, but once you go live, the process inexplicably corrupts your data. If you have a reliable backup, you’re safe. But if you haven’t verified the viability of your backups, your organization is at significant risk.
“An untested backup is not a backup. IT infrastructure is constantly being tweaked and changed and updated, and seemingly unrelated changes can affect backups,” explains David Colgan, a SaaS reliability consultant. “In the past, I saw a backup process that stopped running entirely for three weeks because we installed a new server package that made our backup scheduler crash without showing any errors. If we had not discovered this until we needed the backups, we would have been in a pickle. It takes five minutes to check that your backups are still running, and doing that regularly can mitigate a wide range of failure modes.”
Automated testing cannot be entirely trusted. An actual test event to find out whether your backup risk is appropriately managed is essential. If you have enterprise-critical suppliers, periodically questioning their “green” status reports for validity is a good move.
Ensure risk-monitoring tools keep pace with risks as your environment changes
Monitoring tools play a significant role in keeping you informed about risks and incidents in your organization. But monitoring tools have limitations, especially as your environment changes. If your organization has adopted new technologies such as containers, your monitoring processes may miss a significant source of risk.
“Many IT monitoring tools, like traditional APM platforms, are not able to effectively integrate with modern applications and understand the complex interdependencies between applications and the underlying infrastructure that support them,” explains Antonio Piraino, CTO of ScienceLogic, a provider of IT monitoring services and products. “Traditional APM solutions simply don’t go deep enough. The rise of containerization and multi-cloud systems has contributed to this enormously and has rendered many traditional monitoring tools unable to keep up.”
How can you evaluate whether your monitoring tools are performing? Experimentation is one answer to the problem — experiment with turning certain services on and off to see if these events are reported. Further, revisit your application inventories to see whether they are truly comprehensive.
Beware emerging and evolving regulations — aka the GDPR case
In May 2018, General Data Protection Regulation (GDPR) will take effect in the European Union. With the prospect of large fines, GDPR has many marketers and technologists concerned. How do you balance your need to market and grow the business with the unknowns of navigating a new regulatory regime?
IT leaders can address this risk by analyzing their service providers, current data on EU residents and business goals. You might decide to take a conservative, risk-adverse approach. Take Drip, an email marketing service, as an example. For small companies likely to find the cost of compliance prohibitive, Drip has a simple solution: Disengage with EU prospects and customers. However, this disengagement approach does not eliminate risk entirely because it relies on IP detection tools and related techniques to turn customers away.
If your organization has a higher risk tolerance or greater emphasis on European growth, a different approach may be needed. How can IT leaders help support this change? You may decide to invest in more-powerful tracking tools to make sure EU resident data is tracked correctly. Or you may propose that IT assess the company’s marketing agencies and tools to determine the scope of the GDPR risk exposure.
Watch out for the ‘OpEx surprise’ of cloud services
Given in part the large, upfront capital expenditures of investing in on-premises software and hardware, CIOs have found an attractive solution in the pay-as-you-go cloud model. But increasing cloud adoption means that you have to become skillful with what Vijay Raghavan, chief technology officer at LexisNexis Risk Solutions, calls “the OpEx surprise.”
“The ability to easily scale an application (or even just a pilot of proof of concept) in the cloud has financial pros and cons. The pro is that a large-scale pilot can be conducted in the cloud without a huge upfront CapEx investment,” Raghavan added. “The potential con, if not managed carefully, is that a runaway pilot that auto-scales across thousands of nodes for several days can rack up unforeseen or unplanned OpEx costs.” Mitigating this risk via cloud cost management requires robust vendor management, reporting and IT controls.
Know your vendor lock-in risk
Vendor lock-in risk is a longstanding challenge in IT. The cloud complicates this. The first cloud services were simple services like basic file storage. In that situation, switching to a new vendor is easy to manage. More sophisticated cloud services, however, make switching difficult, raising your risk of vendor lock-in.
“Look at Amazon as an example of vendor lock-in. As Amazon makes it incredibly easy to build and deploy systems within AWS offering a plethora of AWS-proprietary tools for developers and DevOps personnel to care and feed their applications, it becomes effortless to put oneself in a position where it’s extremely difficult or cost-prohibitive to move applications to an alternative cloud vendor should the need arise,” Raghavan explains.
Mitigating vendor lock-in risk is tough. Some organizations may choose to accept the risk and hope for the best. Others may decide to split their operations between multiple vendors. Managing numerous vendor relationships does take more effort, but it may be worth it.
IT audits are your friend — even in the cloud
Other stakeholders can assist in managing risk. Auditors — especially internal auditors focused on risk and IT controls — can help you write contracts and oversee programs to address IT risk. Sometimes audit responses to IT risk are driven by your end customers.
“In our work with large enterprises, our clients have a significant say in how we manage our security and compliance. Some of them have concerns about our migration to AWS,” comments Mark Goldin, CTO at Cornerstone OnDemand. “We are mitigating the risk in several ways: adding AWS into our audit scope, updating all policies and procedures to include services used at AWS and employing high-security measures, such as advanced encryption, to increase security preparedness in the AWS environment.”
Beware the insider threat risk of SaaS
Executives have a delicate tightrope to walk when it comes to people. Increasing use of SaaS services, personal and business, increases the probability of data loss. On the other hand, if you prohibit SaaS services entirely, your staff may perceive you as anti-innovation.
“The insider threat has become one of the greatest challenges for companies,” says Prakash Linga, CTO of Vera, a data-security platform for tracking and monitoring access to corporate data. “With the rise of SaaS, it is almost impossible for companies to keep employees and partners from taking corporate documents with them if they leave the company.”
Prioritize strong data governance and data quality
Big data and analytics only produce results if you have reliable data to use. “The biggest risk to innovation is bad data,” says Piraino. “As such, enterprises must spend the effort up front to ensure the data they collect is high quality and reliable.”
If data quality is poor, your analytics dreams will come to naught. At an early stage, organize a working group to study your data quality and governance issues. If you’re further along, appoint a manager or executive to be responsible for the effort.
Keep up the balancing act
A CIO at a major Canadian bank once said that “keeping the lights on is table stakes.” That is the right way to think about IT risk management. Recognize that proper risk management will tend to be ignored by most other executives because few people notice the absence of a failure. However, your success in managing the risks you understand buys credibility to invest further in innovation.
This article was written by Bruce Harpham from CIO and was legally licensed through the NewsCred publisher network. Please direct all licensing questions to legal@newscred.com.
