If you read a lot of development blogs nowadays, you’ll probably notice a common theme: developers don’t want to deal with databases. They want to focus on designing, building, testing, and deploying applications that deliver value to the business as quickly as possible.
Because of this, developers often instinctively reach for whichever database platform minimizes the amount of time needed to design a schema, and allows them to get started fast.
Focusing on the speed of development and the ability to “fail fast” is commendable, but this approach doesn’t eliminate the need to pay close attention to the database layer. In fact, it only postpones the tough decisions and head-scratching to a later stage in the project, when the new application needs to move into production.
The problem is that by the time an application is ready for production deployment, it may already be too late to reverse poor database design decisions that were made at the beginning of the project.
If you build your application on a database that can’t scale, or can’t be backed up efficiently, or doesn’t meet security and encryption standards, or fails to protect data from corruption when an outage occurs, then the only option may be to go back to the drawing board. And that is an option that few enterprises can afford to take.
I spoke to Jon Lind, Product Manager for Db2 for Linux, UNIX, and Windows, about how companies can avoid these pitfalls, and whether it’s possible to get the best of both worlds – a database that makes it easy for developers to experiment with new projects, while still offering the resilience, scalability and security that enterprises need.
Dr. Jon A. Lind is Product Manager for Db2 at IBM. He has spent most of his career in the database management and data warehousing space, working as a software engineer, researcher and development manager before moving into product management. His broad experience gives him a unique perspective on the challenges of architecting database platforms with the performance, resilience and scalability required to meet the needs of large enterprises.
Chris Gillar: Jon, in the old days, selecting databases for enterprise applications used to be simple: you would choose from the options that the database administration team was able to support. What has changed?
Jon Lind: There has been a shift in the way enterprises are running their development projects – it’s a much more decentralized model, where line-of-business teams are allowed or even encouraged to develop applications for themselves.
That’s a really good thing in many ways, because it allows for much more innovation, creativity and agility in application development. However, it also comes at a price. By allowing the line of business too free a choice of technical architectures, you can end up with a situation where time and resources are invested in building something that isn’t fit for use at an enterprise scale.
Chris Gillar: Why is it that developers often make bad decisions when choosing database platforms?
Jon Lind: I think there are several factors at play here. First, developers are often under significant pressure to deliver value quickly. There’s a lot of hype about agile development. So, when business stakeholders learn that a prototype can be put together in a few weeks, they aren’t willing to wait any longer. This means the quickest option always seems like the best option at the start of a new project, even if it’s going to cause problems down the line.
Second, there’s a lack of awareness about what real enterprise databases are, and what they offer that’s different. Open source relational databases like PostgreSQL and MySQL have made enormous progress in the past decade, and there has been a lot of exciting work on NoSQL databases like MongoDB and CouchDB too.
As a result, there’s a perception among young developers that you don’t need Db2 or other databases when you can get an open source database for free. Moreover, most line-of-business developers are just that – developers. They’re not experienced DBAs, and they don’t necessarily understand that a lot of the tools and capabilities that you need for an enterprise deployment either don’t exist or aren’t mature enough in many of these open-source projects.
Third, in the past, there haven’t always been a lot of good options for solo developers, small development teams, or students who want to get some experience working with enterprise databases. In many cases, the big database vendors either didn’t release a version that was free for non-production use, didn’t make it easy to download and install on inexpensive servers, or imposed limitations that prevented users from experimenting with the enterprise features available.
And so there is now a whole new generation of talented developers out there who don’t know the benefits of an enterprise database platform, don’t have the skills to use one – and can’t see the challenges being created for deployment until it’s too late.
“There’s a whole new generation of talented developers out there who don’t know the benefits of an enterprise database platform, don’t have the skills to use one – and can’t see deployment challenges until it’s too late.”-Jon Lind
Chris Gillar: What are some of the advantages of enterprise databases that these developers are missing out on?
Jon Lind: I’ll use some of the features of Db2 as examples, since that’s the database I know best.
When you move an application into production and it becomes mission-critical, that often means that the volume of data you’re dealing with is substantially larger than you expected when you built your initial prototype. Technologies that work perfectly well on a developer’s laptop or a small server can fall apart when you need to run them across dozens or even hundreds of machines.
By contrast, the Db2 Data Partitioning Feature (DPF) provides the ability to scale up database environments, bringing many physical or virtual machines together to process very large, complex queries much more quickly.
Or if the problem is not the complexity of the queries, but the sheer volume of requests that need to be handled, Db2 pureScale allows users to scale out their environment, using many physical or virtual machines to process large numbers of transactions concurrently.
And if the workload is more analytical than transactional, the Db2 BLU Acceleration feature provides yet another way to scale, using columnar table structures to boost online analytical processing (OLAP) performance dramatically.
Security is another central issue: because cyber crime is becoming more prevalent and sophisticated, data protection regulations are becoming more stringent. It’s increasingly important to ensure that all of your data is properly encrypted – both in transit and at rest – and that encryption keys are stored in a location that is completely separated from the database itself. Db2 makes it easy to manage encryption keys with an external key manager or hardware security module (HSM), so it fits in easily with the kind of enterprise-grade security architectures that large companies want or are required to build.
Even something as simple as backup and recovery can be a weak point with non-enterprise platforms. Of course, basic backup tools are generally available, but for most businesses today, that’s not enough. With Db2 you have capabilities like HADR, FlashCopy and log shipping, which aim to get you back online in a matter of minutes or seconds, with zero data loss.
Compare that to an open-source database I was working on a few weeks ago, where the data center had a power outage and most of the data ended up being corrupted. It’s much easier to avoid these kinds of problems when the tools you need to ensure resilience are built into the fabric of the product, instead of offered as community-maintained add-ons.
“Enterprise databases like Db2 give you the security, reliability and scalability that you need to build solutions that don’t just work as cool prototypes – they can also handle whatever your production environment throws at them.” -Jon Lind
Chris Gillar: A lot of developers aren’t just turning their back on enterprise databases – they’re moving away from relational database management systems altogether, and embracing the freedom of a NoSQL approach. Does Db2 have any value for them too?
Jon Lind: Absolutely! We’re now making JSON a first-class citizen on the Db2 platform, so if you want to move away from tabular data and adopt the more flexible schema-free approach that databases like MongoDB offer, you can.
Whether you’re a NoSQL enthusiast working with data from the web, or a more traditional developer working with relational data from back-end systems, enterprise databases like DB2 give you the security, reliability and scalability you need to build solutions that work as cool prototypes and can also handle whatever your production environment throws at them.
“For the first time, we’re putting the full power of Db2 in the hands of whoever wants to use it – and the hope is that we’ll awaken a new generation of students and developers to the value that enterprise databases bring.” -Jon Lind
Chris Gillar: But none of this matters if users can’t get hold of a copy of Db2 in the first place. How can IBM put Db2 in the hands of these new developers, and persuade them that it’s the right tool to reach for when they’re starting a new project?
Jon Lind: That’s the best part: we’ve just announced the general availability of a completely new edition called Db2 Developer-C.
Unlike the free versions we’ve offered in the past, Db2 Developer-C gives you access to all of the enterprise features, including DPF, pureScale, BLU Acceleration, compression and encryption. It’s easy to install and use, and there’s nothing to pay until you decide to move a database you’ve created into production.
For the first time, we’re putting the full power of Db2 into the hands of whoever wants to use it – and the hope is that we’ll awaken a new generation of students and developers to the value that enterprise-level databases bring. That way, when they start their first job, they’ll be ready to build enterprise-class software from day one.
To learn more about IBM Db2 Developer-C, read the product announcement – or download a copy and get started right away! If you have any questions, there are dozens of tutorials and resources on ibm.com/developerworks, as well as a friendly community to help you get started.