{"id":411316,"date":"2024-06-29T21:54:04","date_gmt":"2024-06-29T21:54:04","guid":{"rendered":"http:\/\/savepearlharbor.com\/?p=411316"},"modified":"-0001-11-30T00:00:00","modified_gmt":"-0001-11-29T21:00:00","slug":"","status":"publish","type":"post","link":"https:\/\/savepearlharbor.com\/?p=411316","title":{"rendered":"Big Data Tools Update 11 Is Out<\/span>"},"content":{"rendered":"
<\/div>\n
\n
\n
\n
\n

EAP 11 of the Big Data Tools plugin for IntelliJ IDEA Ultimate, PyCharm, and DataGrip is available starting today. You can install it from the JetBrains Plugin Repository<\/a> or inside your IDE.<\/p>\n

<\/p>\n

Big Data Tools is a new JetBrains plugin that allows you to connect to Hadoop and Spark clusters and monitor nodes, applications, and jobs. It also brings support for editing and running Zeppelin notebooks inside IntelliJ IDEA and DataGrip, so you can create, edit, and run Zeppelin notebooks without ever having to leave your favorite IDE. The plugin offers smart navigation, code completion, inspections, quick-fixes, and refactoring inside notebooks.<\/p>\n

<\/p>\n

<\/p>\n

<\/a> <\/p>\n

New run\/debug configuration for spark-submit<\/h2>\n

<\/p>\n

One of the most significant improvements in this release is the ability to submit Spark applications from your IDE without using a console. This feature is available for all supported IDEs. The spark-submit script in the Spark bin directory is used to launch applications on a cluster. It can use all supported cluster managers<\/a> through a single interface, so you don\u2019t have to configure your application. <\/p>\n

<\/p>\n

The major drawback of using the spark-submit script is that there\u2019s a number of rituals you have to perform. You have to manually build all artifacts, copy them to the destination server over SSH, and run spark-submit with a special command or, in the worst-case scenario, write a separate bash script to run it. This new feature eliminates all these tedious tasks. All you need to do is create a new run configuration and fill in the parameters; that\u2019s it.<\/p>\n

<\/p>\n

<\/p>\n

<\/p>\n

Apache Zeppelin support for DataGrip<\/h2>\n

<\/p>\n

Also, you can now use the Zeppelin integration inside the DataGrip IDE. The Big Data Tools plugin provides advanced data visualization and context-sensitive code completion inside Zeppelin notebooks to help you write SQL code faster. Please note that Zeppelin support is limited to the SQL language, for example, Matplotlib or Scala code inspections will not work.<\/p>\n

<\/p>\n

Change log<\/h2>\n

<\/p>\n

This update also brings the following fixes and improvements:<\/p>\n

<\/p>\n

Features<\/h3>\n

<\/p>\n