As hardware and memory prices continue to decline, in memory databases (IMDB) are gaining traction for real-time analytics, applications, and platforms. There are good reasons for the accelerated adoption of IMDBs for big data, with the most obvious being performance (as in very fast delivery of large data, and on the fly aggregations of multi-dimensional data delivered quickly to the end user).
From an architecture perspective there are a few key differences, looking at HANA as an example, that power such enhanced performance.
- Memory vs disk as primary storage (much faster performance)
- Column and row store vs tables (no indices, materialized views, or cubes - aggregations on the fly)
- Data compression
- Parallelism (both on hardware and processing)
These changes are redefining the roles of data modelers, database developers and DBAs which are way more blurred in IMDB environments vs traditional databases.
Use Cases for IMDBs include: real-time analytics on operational databases, data warehousing, and predictive analysis on big data, as well as real-time applications and platforms (mobile, cloud)
Drawbacks for IMDBs include: Maturity of most product offerings, cost and availability of skills / expertise.
Enterprise IMDB Vendors and Products
- SAP HANA (to be merged in the future with Sybase IQ offering). Is the leader in the enterprise space. I am involved with an SAP HANA POC/Pilot to be compared vs a traditional DW and fully expect the IMDB to easily outperform the traditional system. Noteworthy that SAP HANA comes as an appliance (option of leading HW vendors) and only with certified configurations by SAP.
- Oracle TimesTen / Exadata X3
- IBM Solid DB (noteworthy that IBM is selling HW bundled with SAP HANA)
- Microsoft (updated per announcement on Oct 24th 2012)
- Microsoft HDInsight Server for Windows
- HDInsight Server for Windows connects to Microsoft SQL Server, Microsoft Excel, PowerPivot (for high-scale in-memory data exploration), Power View (data visualizatio) and works with Microsoft's virtualization platform. Microsoft is the first to deliver Hadoop on virtualized infrastructure. Office 2013 Excel includes: PowerPivot and Power View.
- http://www.informationweek.com/software/information-management/microsoft-releases-hadoop-on-windows/240009632
- Opensource
- Project Serengeti is an open-source effort that is coming right behind Microsoft from VMWare
- MySQL, other smaller players here
- Feedback welcome on opensource IMDB enterprise solutions you have had success with
Expecting the adoption of IMDBs as well as product maturity to continue to accelerate and the vendor battle to heat up, as large enterprise clients climb on-board. At this time everyone seems to be chasing SAP HANA.
Other Links
- IMDBs Overview: http://en.wikipedia.org/wiki/In-memory_database
- Rise of IMDBs http://slashdot.org/topic/datacenter/the-rise-of-in-memory-databases/
- IMDB Q and A: http://www.mcobject.com/in_memory_database
- Comparison of IBM Solid DB vs Oracle TimesTen http://bigdatamatters.com/bigdatamatters/2009/12/oracle_ibm_database_comparison.html
No comments:
Post a Comment