Skim the day’s headlines, and it’s clear: The need for a new generation database is only growing. The older relational databases that both public and private sectors relied on for decades are no longer a match for today’s “tsunami of data,” as Kevin Shelly, Group Vice President, Global Public Sector at MarkLogic, calls it.
What new technologies can ingest and make sense of the tidal wave of data to come? Recently, Shelly offered his thoughts on the complicated intersection of data and new mission needs, and how MarkLogic, and its NoSQL Enterprise database, is stepping in to help solve some of the day’s most complex problems – and make the the world a safer place along the way.
WashingtonExec: What are the top data challenges your customer are facing?
Kevin Shelly: Customers, whether in intel, healthcare, DoD, or state and local, are all creating and consuming more data than they have in the past. We will create more data in the next five years than we have in our lifetime.
How do you mine that data and turn it into information? How do you extract value from it? Organizations are looking for ways to leverage new technologies to improve services, mission effectiveness and arrive at better decision making. Mission leaders and customers are demanding accurate and up-to-date information rapidly. But outdated technologies are being used in many instances to try to harness this tsunami of data. There is a bit of a mismatch.
WashingtonExec: What new technologies can meet the growing data challenge?
Kevin Shelly: For the first time in decades, the industry has seen a disruption in the database market. If you look at relational databases – certainly they have evolved over time — but the relational model hasn’t changed; it’s the same as it was 40 years ago. While the relational model, which is suitable for structured data, hasn’t changed, the data has.
Today 80% of the world’s data is unstructured. There’s a new generation of enterprise software, which can aggregate all of the data, structured and unstructured, in real time, and deliver the right information, to the right people, at the right time.
There are advancements in storage, data analysis tools, and capability in the cloud. The key to harnessing big data is the database – in which it resides. Given that the data is coming at us in such volume and variety – that is why there is such interest in NoSQL databases.
WashingtonExec: What differentiates NoSQL databases from the old ways of doing business?
Kevin Shelly: MarkLogic’s Enterprise NoSQL is- a very unique combination. It combines all of those things that you like in relational technology; the hardened enterprise features like high availability, disaster recovery and data consistency and integrity, as well as government grade security, while differentiating itself with the flexibility and the agility of a NoSQL database. You can pull in and ingest all different data types, whether they are structured, unstructured or poly-structured. In today’s world a lot of the data that is unstructured might be – photos, video files, social media, contracts, case files, emails, documents and none of those lend themselves to relational technology.
Furthermore, MarkLogic has a very powerful integrated search so you don’t have to develop complex queries to find out about the data. You can use a Google-like search to find out different things in the MarkLogic database. It is very easy to use. It is like using Google on the internet but it is more powerful and can be used for your business.
WashingtonExec: How have you helped agencies make sense of today’s tidal wave of unstructured data?
Kevin Shelly: With Healthcare.gov, for example, MarkLogic was able to pull together a variety of data from different sources, whether from healthcare payers, insurance companies or state exchanges, to determine the eligibility of the recipient. That was a massive undertaking, and through all of that, we supported 280,000 concurrent users, we are operational in 38 simultaneous sites, and there were over 5,500 transactions per second processed, much of it from a variety of different data sources and data types.
There are also opportunities where we’ve been able to work with the FAA to provide them with a real-time situation awareness of weather, social media, air traffic on the ground and on the runways and flight schedules so the FAA can see what’s going on in real-time to make decisions. Also in the financial markets – MarkLogic is very prevalent on Wall Street. There is one financial institution that runs about 35% of the world’s derivatives through MarkLogic.
WashingtonExec: What growth areas do you see ahead for MarkLogic in the data arena?
Kevin Shelly: The growth areas are really limitless as far as we see. Everybody will be a part of the Internet of Things. There will be sensors from appliances in your house, smart watch, networks, engineering and medical instruments and your car. All of these items and more will provide data that can be utilized to provide preventive maintenance, increase security, improve health and a whole host of other benefits.
However, the Internet of Things really increases the surface area for cyber attacks so it is critical that the infrastructure is secure, especially the database. This is a terrific use of MarkLogic’s Enterprise NoSQL Database as it has “government grade” security as evidenced by its certifications to run in the Intelligence community on classified programs.
The Internet of Things will be part of the “Big Data” tsunami across all industries and our ability to consume data and make it useful will depend on utilizing new technologies, none more important than the database. MarkLogic’s Enterprise NoSQL Database is the right technology, at the right time, as it can ingest a variety of data types and the data is immediately searchable and can be leveraged by your applications. It’s all about making better decisions faster and in a secure fashion. That is what MarkLogic does and will continue to do for our customers in government, healthcare, financial services and other markets.
On a personal level, I’m extremely proud to represent MarkLogic. Whether we’re protecting our homeland or providing better access to healthcare for millions of people – my work at MarkLogic is a mission, it’s a calling, for the greater good of government, business and people around the globe.
Related: What’s on Your Summer Reading List? -Kevin Shelly of MarkLogic, MarkLogic Public Sector Guru Kevin Shelly Talks Solving Challenges: Helping in Haiti to Big Data Innovation
