Commit 3e27c990 authored by Peter Parente's avatar Peter Parente

Remove os.environ step from spark READMEs

(c) Copyright IBM Corp. 2016
parent 155fdea5
...@@ -32,7 +32,7 @@ This configuration is nice for using Spark on small, local data. ...@@ -32,7 +32,7 @@ This configuration is nice for using Spark on small, local data.
1. Open a Python 2 or 3 notebook. 1. Open a Python 2 or 3 notebook.
2. Create a `SparkContext` configured for local mode. 2. Create a `SparkContext` configured for local mode.
For example, the first few cells in a Python 3 notebook might read: For example, the first few cells in a notebook might read:
```python ```python
import pyspark import pyspark
...@@ -43,15 +43,6 @@ rdd = sc.parallelize(range(1000)) ...@@ -43,15 +43,6 @@ rdd = sc.parallelize(range(1000))
rdd.takeSample(False, 5) rdd.takeSample(False, 5)
``` ```
In a Python 2 notebook, prefix the above with the following code to ensure the local workers use Python 2 as well.
```python
import os
os.environ['PYSPARK_PYTHON'] = 'python2'
# include pyspark cells from above here ...
```
### In a R Notebook ### In a R Notebook
0. Run the container as shown above. 0. Run the container as shown above.
......
...@@ -27,7 +27,7 @@ This configuration is nice for using Spark on small, local data. ...@@ -27,7 +27,7 @@ This configuration is nice for using Spark on small, local data.
2. Open a Python 2 or 3 notebook. 2. Open a Python 2 or 3 notebook.
3. Create a `SparkContext` configured for local mode. 3. Create a `SparkContext` configured for local mode.
For example, the first few cells in a Python 3 notebook might read: For example, the first few cells in the notebook might read:
```python ```python
import pyspark import pyspark
...@@ -38,15 +38,6 @@ rdd = sc.parallelize(range(1000)) ...@@ -38,15 +38,6 @@ rdd = sc.parallelize(range(1000))
rdd.takeSample(False, 5) rdd.takeSample(False, 5)
``` ```
In a Python 2 notebook, prefix the above with the following code to ensure the local workers use Python 2 as well.
```python
import os
os.environ['PYSPARK_PYTHON'] = 'python2'
# include pyspark cells from above here ...
```
## Connecting to a Spark Cluster on Mesos ## Connecting to a Spark Cluster on Mesos
This configuration allows your compute cluster to scale with your data. This configuration allows your compute cluster to scale with your data.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment