Flume 1.7.0 User Guide (also in pdf) Flume 1.7.0 Developer Guide (also in pdf) Flume 1.7.0 API Documentation Changes. This book starts with an architectural overview of Flume and its logical components. It is used to stream logs from application servers to HDFS for ad hoc analysis. Apache Flume 1.7.0 is production-ready software. Apache Flume is a distributed, reliable, and available service used to efficiently collect, aggregate, and move large amounts of log data. Flume 1.8.0 User Guide (also in pdf) Flume 1.8.0 Developer Guide (also in pdf) Flume 1.8.0 API Documentation Changes. (examples below) But it does not do data manipulation. These data feeds include streaming logs, network traffic, Twitter feeds, etc. Flume is a tool that is very dependable, distributed, and customizable. Apache Flume reads a data source and writes it to storage at incredibly high volumes and without losing any events. Apache Flume 1.8.0 is production-ready software. Apache Flume is a tool/service/data ingestion mechanism for gathering, aggregating, and delivering huge amounts of streaming data from diverse sources, such as log files, events, and so on, to centralized data storage. By the end of this book, you will be fully equipped to construct a series of Flume agents to dynamically transport your stream data and logs from your systems into Hadoop.Ī step-by-step book that guides you through the architecture and components of Flume covering different approaches, which are then pulled together as a real-world, end-to-end use case, gradually going from the simplest to the most advanced features. Apache Flume 1.7.0 is the tenth release of Flume as an Apache top-level project (TLP). Apache Flume 1.8.0 is the eleventh release of Flume as an Apache top-level project (TLP). It explores channels, sinks, and sink processors, followed by sources and channels. Once installed, make sure the maven executable mvn is in your path and test it out using the following command: mvn -version Apache Maven 3.0. Download the latest update of Maven 3.x and install it locally on your system. Design and implement a series of Flume agents to send streamed data into HadoopĪpache Flume is a distributed, reliable, and available service used to efficiently collect, aggregate, and move large amounts of log data. The build tool used for building and testing Flume is Apache Maven.