Spark The Definitive Guide Spark权威指南中文笔记

Welcome to this first edition of Spark: The Definitive Guide! We are excited to bring you the most complete resource on Apache Spark today, focusing especially on the new generation of Spark APIs introduced in Spark 2.0. Apache Spark is currently one of the most popular systems for large-scale data processing, with APIs in multiple programming languages and a wealth of built-in and third-party libraries. Although the project has existed for multiple years—first as a research project started at UC Berkeley in 2009, then at the Apache Software Foundation since 2013—the open source community is continuing to build more powerful APIs and high-level libraries over Spark, so there is still a lot to write about the project. We decided to write this book for two reasons. First, we wanted to present the most comprehensive book on Apache Spark, covering all of the fundamental use cases with easy-to-run examples. Second, we especially wanted to explore the higher-level “structured” APIs that were finalized in Apache Spark 2.0—namely DataFrames, Datasets, Spark SQL, and Structured Streaming—which older books on Spark don’t always include. We hope this book gives you a solid foundation to write modern Apache Spark applications using all the available tools in the project. In this preface, we’ll tell you a little bit about our background, and explain who this book is for and how we have organized the material. We also want to thank the numerous people who helped edit and review this book, without whom it would not have been possible.

Work with all aspects of batch processing in a modern Java environment using a selection of Spring frameworks. This book provides up-to-date examples using the latest configuration techniques based on Java configuration and Spring Boot. The Definitive Guide to Spring Batch takes you from the “Hello, World!” of batch processing to complex scenarios demonstrating cloud native techniques for developing batch applications to be run on modern platforms. Finally this book demonstrates how you can use areas of the Spring portfolio beyond just Spring Batch 4 to collaboratively develop mission-critical batch processes. You’ll see how a new class of use cases and platforms has evolved to have an impact on batch-processing. Data science and big data have become prominent in modern IT and the use of batch processing to orchestrate workloads has become commonplace. The Definitive Guide to Spring Batch covers how running finite tasks on cloud infrastructure in a standardized way has changed where batch applications are run. Additionally, you’ll discover how Spring Batch 4 takes advantage of Java 9, Spring Framework 5, and the new Spring Boot 2 micro-framework. After reading this book, you’ll be able to use Spring Boot to simplify the development of your own Spring projects, as well as take advantage of Spring Cloud Task and Spring Cloud Data Flow for added cloud native functionality. Includes a foreword by Dave Syer, Spring Batch project founder. What You’ll Learn Discover what is new in Spring Batch 4 Carry out finite batch processing in the cloud using the Spring Batch project Understand the newest configuration techniques based on Java configuration and Spring Boot using practical examples Master batch processing in complex scenarios including in the cloud Develop batch applications to be run on modern platforms Use areas of the Spring portfolio beyond Spring Batch to develop mission-critical batch processes Who This Book Is For Experienced Java and Spring coders new to the Sp

Spark：权威指南 这是与相关的所有材料的中央资料库 Bill Chambers和Matei Zaharia撰写。 这个资料库目前仍在进行中，随着时间的推移会增加新的资料。 书中的代码 您可以从code子文件夹的书中找到代码，在该code中，可以按语言和章节对code进行细分。 如何运行代码 在本地计算机上运行 要在本地计算机上运行该示例，请将data子文件夹中的所有数据拉到计算机上的/data或指定本地计算机上该特定数据集的路径。 在Databricks上运行 要在Databricks上运行这些模块，您将需要做两件事。 注册新账号。 你可以在做。 导入单个笔记本以在平台上运行 Databricks是一个零管理云平台，可提供： 完全托管的Spark集群 一个用于探索和可视化的交互式工作区 生产管道调度程序 一个为您喜欢的基于Spark的应用程序提供动力的平台 导入说明 导航到您要

Spark: The Definitive Guide Big data processing made simple