Hadoop 3.0 Release Dateis 13-Dec-2017 and General Availability (GA) binaries can be downloaded from its official website. General availability (GA) marks a point of quality and stability for the release series that indicates its ready for production use. You can also download the source code for this release which is around ~25Mb in size. Hadoop 3.0 binaries will be somewhere around ~250Mb in size.
Hadoop 3.0 alpha was released in Sep-2016 and since then they have made 6 releases and the final Hadoop 3.0 release date was Dec 2017. If you want to follow more closely, you can click on their JIRA dashboard for patch availability.
Congratulations to the Apache Hadoop community on this major milestone for the project.
Downstream Compatibility with Other Apache Project
Following are the version compatibility matrix sheet indication the version of different Apache projects and their unit test status including basic functionality testing. This was done as part of Hadoop 3.0 Beta 1 release in Oct 2017.
Overview of Hadoop 3.0 Release Highlight
- Improved storage efficiency using HDFS Erasure Coding
- Java 1.8 Support
- Improved Yarn Timeline Services v2
- Yarn resource types which allows scheduling for user defined resources like GPU
- Support for Opportunistic Containers
Hadoop 3.0 Bundling with Cloudera (CDH)
Cloudera 5.x release does not bundle Hadoop 3.0. Cloudera 6.x release is planning to have Hadoop 3.0 and itsfeatures and improvements listed above will be eventually be incorporated into CDH after our usual rigorous testing and integration process.
Though Hadoop is an open-source distributed computing technology, Cloudera strongly influcences the direction its development. Moreover Doug Cutting who founded Hadoop and mamber of Apache Hadoop Foundation.
Hadoop 3.0 Bundling with Hortonworks (HDP)
The current HDP version does not support Hadoop 3.0 and there are lot of discussion in hortonworks community by when this support will be available (refer discussion). No update is available as on posting of this blog.
Hadoop 3.0 Release Date Journey
Apache Hadoop Community has planned to release early and release often to quickly iterate the feedback from the downstream projects. You can see the release history and future plan below.