PySparkJob (Cloud Dataproc API v1 (Rev. 150) 1.25.0)

java.lang.Object
- java.util.AbstractMap<java.lang.String,java.lang.Object>
- - com.google.api.client.util.GenericData
  - - com.google.api.client.json.GenericJson
    - - com.google.api.services.dataproc.model.PySparkJob

All Implemented Interfaces:

java.lang.Cloneable, java.util.Map<java.lang.String,java.lang.Object>
```
public final class PySparkJob
extends com.google.api.client.json.GenericJson
```
A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python- programming-guide.html) applications on YARN.
This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the Cloud Dataproc API. For a detailed explanation see: https://developers.google.com/api-client-library/java/google-http-java-client/json

Author:

Google, Inc.

Nested Class Summary
- Nested classes/interfaces inherited from class com.google.api.client.util.GenericData
  com.google.api.client.util.GenericData.Flags
- Nested classes/interfaces inherited from class java.util.AbstractMap
  java.util.AbstractMap.SimpleEntry<K,V>, java.util.AbstractMap.SimpleImmutableEntry<K,V>
- Nested classes/interfaces inherited from interface java.util.Map
  java.util.Map.Entry<K,V>

Constructor Summary

Constructors
Constructor and Description

PySparkJob()

Constructors
Constructor and Description
`PySparkJob()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`PySparkJob`	`clone()`
`java.util.List<java.lang.String>`	`getArchiveUris()` Optional.
`java.util.List<java.lang.String>`	`getArgs()` Optional.
`java.util.List<java.lang.String>`	`getFileUris()` Optional.
`java.util.List<java.lang.String>`	`getJarFileUris()` Optional.
`LoggingConfig`	`getLoggingConfig()` Optional.
`java.lang.String`	`getMainPythonFileUri()` Required.
`java.util.Map<java.lang.String,java.lang.String>`	`getProperties()` Optional.
`java.util.List<java.lang.String>`	`getPythonFileUris()` Optional.
`PySparkJob`	`set(java.lang.String fieldName, java.lang.Object value)`
`PySparkJob`	`setArchiveUris(java.util.List<java.lang.String> archiveUris)` Optional.
`PySparkJob`	`setArgs(java.util.List<java.lang.String> args)` Optional.
`PySparkJob`	`setFileUris(java.util.List<java.lang.String> fileUris)` Optional.
`PySparkJob`	`setJarFileUris(java.util.List<java.lang.String> jarFileUris)` Optional.
`PySparkJob`	`setLoggingConfig(LoggingConfig loggingConfig)` Optional.
`PySparkJob`	`setMainPythonFileUri(java.lang.String mainPythonFileUri)` Required.
`PySparkJob`	`setProperties(java.util.Map<java.lang.String,java.lang.String> properties)` Optional.
`PySparkJob`	`setPythonFileUris(java.util.List<java.lang.String> pythonFileUris)` Optional.

Methods inherited from class com.google.api.client.json.GenericJson
getFactory, setFactory, toPrettyString, toString

Methods inherited from class com.google.api.client.util.GenericData
entrySet, get, getClassInfo, getUnknownKeys, put, putAll, remove, setUnknownKeys

Methods inherited from class java.util.AbstractMap
clear, containsKey, containsValue, equals, hashCode, isEmpty, keySet, size, values

Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map
compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, putIfAbsent, remove, replace, replace, replaceAll

- Constructor Detail
  - PySparkJob
```
public PySparkJob()
```
- Method Detail
  - getArchiveUris
```
public java.util.List<java.lang.String> getArchiveUris()
```
    Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
    
    Returns:
    
    value or null for none
  - setArchiveUris
```
public PySparkJob setArchiveUris(java.util.List<java.lang.String> archiveUris)
```
    Optional. HCFS URIs of archives to be extracted in the working directory of .jar, .tar, .tar.gz, .tgz, and .zip.
    
    Parameters:
    
    archiveUris - archiveUris or null for none
  - getArgs
```
public java.util.List<java.lang.String> getArgs()
```
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    
    Returns:
    
    value or null for none
  - setArgs
```
public PySparkJob setArgs(java.util.List<java.lang.String> args)
```
    Optional. The arguments to pass to the driver. Do not include arguments, such as --conf, that can be set as job properties, since a collision may occur that causes an incorrect job submission.
    
    Parameters:
    
    args - args or null for none
  - getFileUris
```
public java.util.List<java.lang.String> getFileUris()
```
    Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
    
    Returns:
    
    value or null for none
  - setFileUris
```
public PySparkJob setFileUris(java.util.List<java.lang.String> fileUris)
```
    Optional. HCFS URIs of files to be copied to the working directory of Python drivers and distributed tasks. Useful for naively parallel tasks.
    
    Parameters:
    
    fileUris - fileUris or null for none
  - getJarFileUris
```
public java.util.List<java.lang.String> getJarFileUris()
```
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    
    Returns:
    
    value or null for none
  - setJarFileUris
```
public PySparkJob setJarFileUris(java.util.List<java.lang.String> jarFileUris)
```
    Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Python driver and tasks.
    
    Parameters:
    
    jarFileUris - jarFileUris or null for none
  - getLoggingConfig
```
public LoggingConfig getLoggingConfig()
```
    Optional. The runtime log config for job execution.
    
    Returns:
    
    value or null for none
  - setLoggingConfig
```
public PySparkJob setLoggingConfig(LoggingConfig loggingConfig)
```
    Optional. The runtime log config for job execution.
    
    Parameters:
    
    loggingConfig - loggingConfig or null for none
  - getMainPythonFileUri
```
public java.lang.String getMainPythonFileUri()
```
    Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    
    Returns:
    
    value or null for none
  - setMainPythonFileUri
```
public PySparkJob setMainPythonFileUri(java.lang.String mainPythonFileUri)
```
    Required. The HCFS URI of the main Python file to use as the driver. Must be a .py file.
    
    Parameters:
    
    mainPythonFileUri - mainPythonFileUri or null for none
  - getProperties
```
public java.util.Map<java.lang.String,java.lang.String> getProperties()
```
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    
    Returns:
    
    value or null for none
  - setProperties
```
public PySparkJob setProperties(java.util.Map<java.lang.String,java.lang.String> properties)
```
    Optional. A mapping of property names to values, used to configure PySpark. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf and classes in user code.
    
    Parameters:
    
    properties - properties or null for none
  - getPythonFileUris
```
public java.util.List<java.lang.String> getPythonFileUris()
```
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    
    Returns:
    
    value or null for none
  - setPythonFileUris
```
public PySparkJob setPythonFileUris(java.util.List<java.lang.String> pythonFileUris)
```
    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py, .egg, and .zip.
    
    Parameters:
    
    pythonFileUris - pythonFileUris or null for none
  - set
```
public PySparkJob set(java.lang.String fieldName,
                      java.lang.Object value)
```
    Overrides:
    
    set in class com.google.api.client.json.GenericJson
  - clone
```
public PySparkJob clone()
```
    Overrides:
    
    clone in class com.google.api.client.json.GenericJson

Cloud Dataproc API v1 (revision 150)

Class PySparkJob

Nested Class Summary

Nested classes/interfaces inherited from class com.google.api.client.util.GenericData

Nested classes/interfaces inherited from class java.util.AbstractMap

Nested classes/interfaces inherited from interface java.util.Map

Constructor Summary

Method Summary

Methods inherited from class com.google.api.client.json.GenericJson

Methods inherited from class com.google.api.client.util.GenericData

Methods inherited from class java.util.AbstractMap

Methods inherited from class java.lang.Object

Methods inherited from interface java.util.Map

Constructor Detail

PySparkJob

Method Detail

getArchiveUris

setArchiveUris

getArgs

setArgs

getFileUris

setFileUris

getJarFileUris

setJarFileUris

getLoggingConfig

setLoggingConfig

getMainPythonFileUri

setMainPythonFileUri

getProperties

setProperties

getPythonFileUris

setPythonFileUris

set

clone