Over the past couple of years, the ODP’s effort to produce a basic Hadoop distribution have been deemed controversial by many. Now it seems that these efforts are finally beginning to bear fruit, but it’s highly unlikely that it comes up to the critics’ standards.
Two key players in the SQL in Hadoop endeavors by ODP, Hortonworks and Pivotal, have joined forces in order to certify Hawq by Pivotal, an SQL layer for operating on Hadoop Data, on the Hortonworks Data Platform (HDP) distribution of Hadoop.
Hawq was, and still is a very much key part of the Big Data Suite by Pivotal, a set of tools intended for Hadoop, which Pivotal had previously made only available as a patented product. That said, Pivotal conceded and open sourced the components of the Big Data Suite (Hawq, Greenplum Analytical System and the NoSQL Gemfire database) under different licenses.
Now Pivotal is ensuring that every single one of these components will work as they are intended to with HDP and they are claiming that this shifts the focus from the proprietary configuration to an open source native environment, in order to provide with a lower TCO and a better SQL in Hadoop solution. Pivotal is now pitching Hawq to enterprises which are already investing in HDP at the same time showing a strong interest in SQL engines in order to build offload tasks from traditional organizational warehouses, use cases for analytical purposes and execute at a huge Hadoop scale.
Satisfying the organizations is only a small part of the big picture with this coalition. It is also meant to serve as a great example of building towards a better SQL on Hadoop solution. Since all the Hadoop distributions based on the ODP share some underlying parts, it is much easier for engineers to build on top of the Hadoop Platform with a mission to extend on it, lifting the restrictions of any of the Hadoop distributions.
Since its inception, the Open Data Platform has inspired a lot of controversy and dissension as much as its acquisitions of contributors and adherents. Some very prominent platforms have opted to stay out of this coalition, for instance there is Cloudera and MapR (not a very prominent but a significant distribution of Hadoop). In MapR’s opinion, the existing efforts of Apache Software Foundation makes the existence of Open Data Platform redundant if nothing else.
One of the Apache-sponsored projects, Ambari is a CMS (configuration Management System) for all the single components of Hadoop which Hawq integrates itself with. Claims made by Ambari reveal that MapR is being used by less than 25% of the users in the market, and thus falls short of offering a substantial offering than the other SQL in Hadoop solutions.
MapR said in a recent blog post that the interoperability of projects and subprojects in Hadoop is very good. Applications which are built on any one specific distribution can now be easily migrated with almost zero switching costs to the other distributions.
The Pivotals’ and Hortonworks’, ODP (Open Data Platform) yields some of the first tangible results, but it is highly unlikely that they’ll dispel controversy over the mission of the Open Data Platform.
Big Data Analytics 101 