Analytic Professionals — Please participate in the 2020 Data Science Survey.Please use and share these links:— Access Code: ZVWX3It is OK to share this Access Code with others: It can be used by multiple people— Our main Data Science Survey page also has more survey information & FREE downloads of the 2007-2017 Survey Summary ReportsRexer Analytics has been conducting the Data Science Survey since 2007. Each survey explores the analytic behaviors, views and preferences of data scientists and analytic professionals. Over 1100 people from around the globe participated in the 2015 and 2017 surveys. A report summarizing the 2020 Survey findings will be available free to everyone in 4Q-2020.The survey should take approximately 20 minutes to complete. Your responses are completely confidential.
Category: Analytics
Time series forecasting and mortality rate
Seems like many different types of time series curves that can be used for forecasting hospital resources across different countries
Equivalent of pandas info in PySpark
Use df.printSchema()
Machine Learning Week with 15% Discount Code
A project is more than just a Kaggle dataset
a few criteria that define a good data science project
- Learnability- What did you learn in the Project
- Capability – What capabilities were showcased in the project
- Difficulty- How difficult or easy was the project
- Potential Hireability- How likely are you going to be hired based on that project
- Ability- What creative approaches did you bring to the solution
A few datasets I liked only from a teaching purpose- iris, Boston, mtcars, Titanic, German Credit and mnist handwriting
A project is more than just a Kaggle dataset. hashtagdatascience hashtagdatasets hashtagkaggle hashtagmachinelearning
Saving Dataframe as a table
- ModelData2=ModelData.toPandas() #CONVERTS SPARK DF TO PANDAS DF
- table_model = spark.createDataFrame(ModelData2) # CREATES SPARK DF
- table_model.write.saveAsTable(‘LIBRARYPATH.model_data’) #SAVES AS TABLE
AND
new_df = transformed_chrn2[[‘Var1’, ‘Var2’, ‘Var3’, ‘Var4′,’Var5’]]
table_df = spark.createDataFrame(new_df)
table_df.write.saveAsTable(‘directory_name.table_name’)
SOURCE
https://stackoverflow.com/questions/30664008/how-to-save-dataframe-directly-to-hive
https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-connect-to-sql-database
https://docs.microsoft.com/en-us/azure/databricks/getting-started/spark/dataframes
