On Azure Databricks you can create experiments using MLFlow https://mlflow.org/
notebook_path = ‘/Users/Ajay/Folder’
notebook_path = notebook_path
mlflow.set_experiment(notebook_path + ‘_experiments’)
width = 5
%pythondf=spark.sql(“select * from name_csv”)
pandas_df = df.toPandas()
Creating SQL Table using Spark
acc_1=spark.sql(“create table test_spark as select columns, column,columnc from table where to_date(ac_opn_dt) < ‘2012-07-01’ )”)
# Given pandas dataframe, return a spark’s dataframe.
columns = list(pandas_df.columns)
types = list(pandas_df.dtypes)
struct_list = 
for column, typo in zip(columns, types):
p_schema = StructType(struct_list)
return sqlContext.createDataFrame(pandas_df, p_schema)
News from Rexer Analytical ics
The 2020 Data Science Survey is live!!
Please take the survey. And let me know what you think of it.
If you have time, please also help spread the word about the survey.
- Tell the analytic people in your network about the survey
- Forward this email
- Anyone doing anything in the field of analytics is welcome to take the survey
- Consider posting something about it in online analytic communities or social media
Please use and share these links:
It is OK to share this Access Code with others: It can be used by multiple people