Generative AI and LLMs


What is Generative AI?:

checkGenerative AI is a type of artificial intelligence (AI) that can create new content, such as text, images, audio, and video. It does this by learning from existing data and then using that knowledge to generate new and unique outputs.

Hallucinations are words or phrases that are generated by the model that are often nonsensical or grammatically incorrect. What are some factors that can cause hallucinations? Select three options.


The model is not given enough context.

The model is trained on noisy or dirty data.

The model is not trained on enough data

What are foundation models in Generative AI?

A foundation model is a large AI model pretrained on a vast quantity of data that was “designed to be adapted” (or fine-tuned) to a wide range of downstream tasks, such as sentiment analysis, image captioning, and object recognition.

What is a prompt?

A prompt is a short piece of text that is given to the large language model as input, and it can be used to control the output of the model in many ways.

What is an example of both a generative AI model and a discriminative AI model?

A generative AI model could be trained on a dataset of images of cats and then used to generate new images of cats. A discriminative AI model could be trained on a dataset of images of cats and dogs and then used to classify new images as either cats or dogs

What are large language models (LLMs)?:

An LLM is a type of artificial intelligence (AI) that can generate human-quality text. LLMs are trained on massive datasets of text and code, and they can be used for many tasks, such as writing, translating, and coding.

What are some of the challenges of using LLMs? Select three options.

They can be biased.

They can be used to generate harmful content.

They can be expensive to train.

What are some of the applications of LLMs?

LLMs can be used for many tasks, including:

  1. Writing
  2. Translating
  3. Coding
  4. Answering questions
  5. Summarizing text
  6. Generating creative content

What are some of the benefits of using large language models (LLMs)?

LLMs have many benefits, including:

  1. They can generate human-quality text.
  2. They can be used for a variety of tasks.
  3. They can be trained on massive datasets of text and code.
  4. They are constantly improved.

2023 Data Science Survey: Please Participate & Help spread the word

Rexer Analytics has been conducting the Data Science Survey since 2007. Each survey explores the analytic behaviors, views, and preferences of data scientists and analytic professionals. This year we are excited to work with Eric Siegel and his Machine Learning Week organization to design, promote, and analyze this Data Science Survey.

Summary reports from previous surveys are available FREE to download from the Rexer Analytics website.

Karl Rexer and Eric Siegel will present preliminary highlights of the 2023 survey results at the Machine Learning Week conference in Las Vegas in June 2023, and the presentation will be posted online shortly afterward. Come join us! A full summary report will be available for download from the Rexer Analytics website later in 2023.

Movie Review -The Batman is the worst Batman movie sans Ben Affleck

We have seen Batman movies before. The Gold Standard of the Nolan Trilogy. The original series of Keaton-Kilmer-Clooney (Clooney was terrible too)

Ben Affleck was a bad Batman , but better than Daredevil

This one is an attempt to make Batman the Greatest Detective some angst and brooding. It fails

All you have is a wasted effort and a long movie

I could be a better Batman. Its like that

MLFlow on Azure Databricks

On Azure Databricks you can create experiments using MLFlow https://mlflow.org/

notebook_path = ‘/Users/Ajay/Folder’

notebook_path = notebook_path
mlflow.set_experiment(notebook_path + ‘_experiments’)

with mlflow.start_run(run_name=”ExperimentRun”+curr_ts):
mlflow.log_params({
‘RSME’: RSME,
‘AUC’: AUC,

})
mlflow.end_run()

https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/

https://docs.microsoft.com/en-us/azure/databricks/applications/mlflow/quick-start-python

Extract date from datetime in Pandas column

use .dt.date

df[‘column’] = pd.to_datetime(df[‘column’], format=’%Y-%m-%d’).dt.date

Source-

https://stackoverflow.com/questions/16176996/keep-only-date-part-when-using-pandas-to-datetime