Load Multipe CSV files in PySpark
spark= SparkSession.builder \
.master(“local”) \
.appName(“Data Exploration”) \
.getOrCreate()
#load data as Spark DataFrame
data2=spark.read.format(“csv”) \
.option(“header”,”true”) \
.option(“mode”,”DROPMALFORMED”) \
.load(‘/home/Desktop//input/*.csv’)
Convert Many Columns to Float in PySpark
from pyspark.sql.functions import col
for col_name in data7.columns:
data7 = data7.withColumn(col_name, col(col_name).cast(‘float’))
data7.printSchema()
How to run PySpark through Jupyter notebook via Docker
docker run -it -p 8888:8888 jupyter/pyspark-notebook
Install Docker before that
source -https://levelup.gitconnected.com/using-docker-and-pyspark-134cd4cab867
for A few songs more
Few Movie Songes
After Life TV Series Review
A story well written tells itself more than any acting, directing or choreography will do. After life is a sweet, tangy, pretty and yet honest tale of grieving by the brilliant Ricky G.
Now I hope it goes to second season, but it looks tough
DECISION STATS 