The goal of data integration is to get data to where it's needed so it can do its job. Long ago, people did this work. Armies of clerks copied receipts, made accounting entries, wrote letters, and did the work we use technology for today.
If I were to tell you that, a year from now, your data collection needs will be exactly the same as they are today, you'd probably tell me I was crazy. And you'd be right!
"Are legacy integration technologies plotting your demise?" That's the title of the webinar I'm hosting next week. A little dramatic, I know, and most definitely the result of binge watching an entire season of Game of Thrones over the holidays.
In today’s competitive environment, it is vital to the success of an organization to be able to quickly integrate new business applications, link trading partners for electronic data interchange (EDI), transform data between specific formats to fulfill regulatory record keeping requirements, and manage increasingly complicated and distributed supply chains. As complicated as all these tasks are, it's my opinion that we have only scratched the surface of complexity.
As we head into 2016, here is a roundup of trends that I believe will impact the integration and data management technology choices made by CIOs this year.
Trend #1: IoT and other technology disruptions
The Internet of Things (IoT), cloud, and 3Vs of big data (volume, variety, and velocity) are all interrelated and all impacting integration in a major way. CIOs need to figure out how to bring all these new devices, applications, and data sources together in a way that allows them to be seamlessly integrated, analyzed in real time, and mined for insights. And they need to do it quickly because the potential of these disruptive technologies is tied to revenue growth and there's a need to achieve results in a narrow window of time.
If your organization deals with regulated data such as credit card or health data, then you know how difficult of an undertaking compliance is. Not only did you face huge upfront costs to assess and meet governing compliance standards such as HIPAA or PCI DSS, but also ongoing costs in order to maintain compliance with these stringent standards, which often require annual third party certifications.
And while the upfront compliance costs are usually anticipated and budgeted, many organizations underestimate the significant cost of maintaining compliance, which can reach hundreds of thousands or even millions of dollars as enterprises struggle to keep up with ever-changing regulations that require ongoing investments in new technologies and expertise. As one small example, last year's 3.1 release of the Payment Card Industry Data Security Standard (PCI DSS) has merchants and other organizations that handle payment card data under the gun to migrate from SSL and early cryptographic protocols to newer, more secure protocols such as TLS 1.2.
Further keeping your Chief Security Officer and/or Chief Compliance Officer up at night are the massive consequences of non-compliance. Breaches as a result of non-compliance can result in fines, lawsuits, lost revenue, and a severely damaged reputation. Just ask Target whose 2014 fourth-quarter profits dropped $440 million as a result of a large-scale credit card breach. Or health insurer giant, Anthem, who is facing more than 50 class-action lawsuits as a result of a 2015 customer data breach.
When my son was nine, he loved to ask anyone and everyone this classic question: if you were a superhero, which superpower would you choose—invisibility or flight?
It's a fun question that always led to a lively conversation of pros, cons, and what-ifs. I mean, how to choose? Any superhero worth his or her salt would want both of these superpowers.
Of course, when we're talking about data, arguably the superhero at your organization, invisibility is anything but a desirable superpower. Nevertheless, all too many organizations have an invisibility cloak draped over their data activities that, much like kryptonite, is sapping their data's strength.
Many believe that the key to getting what you want is to visualize it. With my 2016 New Year's resolutions still fresh in my mind, I've admittedly spent some time visualizing that perfect handstand I've vowed to achieve—while being cheered on by a room full of sweaty yogis no less!
But what if what you want in 2016 is to mine insights from your organization's data?
Well, visualization works well here too. But inspiring mental imagery alone is not going to cut it. Organizations need a different kind of visualization technique: one that employs sophisticated data tools to organize, query, report on, and visually represent their data.
If there are two life lessons that I have learned from my experience with long distance running, they are this:
- If you find yourself doing something that is painful but necessary, do it more
- Trade long term debilitating painful events for more frequent, but shorter and less debilitating painful events
Of course, these lessons aren't unique to running. In fact, both of these tenets are driving two of the most popular trends in software development today: continuous integration (CI) and continuous delivery (CD).
Continuous integration, as defined by Wikipedia, is the practice of "merging all developer working copies to a shared mainline several times a day." The idea behind this best practice is for a developer to develop, build, and test code as often as possible to avoid major integration issues with other developers’ code down the road.
Along similar lines, continuous delivery is the notion that new features and functionalities should be in a state of deployment readiness so that the actual deployment process does not require bringing the system down or disrupting users.
In my previous blog regarding transplant data, I outlined the difficult decision that blood and marrow transplant centers face when it comes to retrieving data from CIBMTR: do they choose to retrieve data that is easily accessible or data that is comprehensive? Pick just one!
But putting aside this longstanding conundrum for a moment, let's assume that your center has access to all the data it could possibly want. Now what? How will you maintain it so that it can ultimately be used to improve patient care?
Many transplant centers manage their clinical data in spreadsheets. When you consider that a patient intake form alone can consist of 700 fields of data, cobbled together from disparate systems by dedicated data specialists, the drawbacks quickly become apparent. Spreadsheets—and homegrown systems—can be cumbersome to maintain, inflexible at such a large scale, and have no built-in organization to leverage.
Every day, knowledge in the field is expanding, meaning every day your dataset could get bigger. As a result, transplant centers that would like to use their highly relevant clinical data to jumpstart quality improvement initiatives find that they simply don't have the back-end organization (or manpower) to translate data into discovery.
I go to this fantastic gym. The trainers put together varied and effective workout classes, know exactly how to get the most out of me, and instill just the right level of healthy competition to keep us all enthused and motivated. They work with local high school athletic departments to improve their training regimens and they mentor trainers in training. In short, my gym is a Center of Excellence.
Where does your business excel? If it’s a successful one, then it most likely excels at delivering a unique product or service—perhaps an experience—to a unique set of customers. And within that context, your organization has expertise and experience that no other organization can offer. Consider too that these assets (expertise and experience) are nurtured by best practices, training programs, support teams, and efficient processes—all aimed at ensuring that your organization continues to excel.
What am I getting at? A successful company is, by its very nature, a Center of Excellence (COE).
And there’s no shortage of companies capitalizing on this concept—as they should. A Google news search on “Center of Excellence” returns a plethora of new launch announcements. Here’s a small sampling of recent headlines: