DECISION STATS

Recurrent Neural Networks (RNNs) are a specialized class of deep learning models designed to process sequential data, where the order of information is as important as the information itself. Unlike traditional feed-forward neural networks that treat each input independently, RNNs maintain a hidden state that acts as memory, enabling them to retain information from previous time steps while processing new inputs. As discussed in this presentation, this capability makes RNNs particularly suitable for applications involving text, speech, time-series data, sensor readings, and video sequences, where contextual information plays a crucial role in making accurate predictions.

The fundamental building block of an RNN is the recurrent cell, which combines the current input with the hidden state from the previous time step to produce a new hidden state and output. Since the same set of weights is reused at every step, RNNs can process sequences of varying lengths while maintaining a fixed number of parameters. During training, the network is unrolled through time, allowing Backpropagation Through Time (BPTT) to compute gradients across all time steps. However, standard RNNs often suffer from vanishing and exploding gradient problems, making it difficult to learn long-range dependencies in lengthy sequences. Techniques such as truncated BPTT and gradient clipping help improve training stability and computational efficiency.

To address the limitations of vanilla RNNs, more advanced architectures such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks were introduced. LSTMs use forget, input, and output gates to regulate the flow of information through a dedicated cell state, enabling the model to preserve important information over long time intervals. GRUs simplify this design by combining gates and eliminating the separate cell state, resulting in fewer parameters and faster training while maintaining competitive performance on many sequence-learning tasks. The presentation also explores bidirectional and stacked RNN architectures, along with practical implementation using PyTorch, demonstrating how padded sequences, LSTM classifiers, and gradient clipping are incorporated into real-world deep learning workflows.

Although Transformers have become the dominant architecture for modern natural language processing because of their ability to process sequences in parallel and capture long-range dependencies more effectively, RNNs continue to play an important role in applications requiring streaming data, low-latency inference, or resource-constrained edge devices. They remain widely used in speech recognition, sentiment analysis, handwriting recognition, time-series forecasting, anomaly detection, music generation, and sequential sensor analysis. Understanding RNNs, LSTMs, and GRUs provides a strong foundation for learning more advanced sequence models and appreciating the evolution of deep learning architectures that power today’s AI systems.

https://docs.google.com/presentation/d/e/2PACX-1vRJPePDuaheqMrnJ1KjpWpzJrXsUt-2fkaL_jjVW7ow0M-U3k3bISIdsgxen6LT8A/pub?start=true&loop=true&delayms=10000

Convolutional Neural Networks (CNNs) are among the most influential deep learning architectures, specifically designed to process and analyze image and visual data. Unlike traditional fully connected neural networks, CNNs automatically learn hierarchical features directly from raw images, eliminating the need for manual feature engineering. By exploiting the spatial relationships between neighboring pixels, CNNs can recognize edges, textures, shapes, and complex objects with remarkable accuracy. As highlighted in the presentation, this ability has made CNNs the backbone of modern computer vision, powering applications ranging from image classification and object detection to facial recognition and medical image analysis.

The core strength of a CNN lies in its specialized architecture. The Convolutional Layer applies learnable filters (kernels) that slide across an image to extract meaningful features while preserving spatial information. These feature maps are passed through activation functions, commonly ReLU (Rectified Linear Unit), introducing non-linearity so the network can model complex visual patterns. Pooling layers, such as Max Pooling and Average Pooling, reduce the dimensionality of feature maps, lowering computational requirements while improving robustness to small translations and distortions. By stacking multiple convolution and pooling layers, CNNs progressively learn increasingly abstract representations, from simple edges in early layers to complete objects in deeper layers.

After feature extraction, the learned representations are flattened and passed to fully connected layers, where the network performs classification or regression based on the extracted features. CNNs are trained using forward propagation, backpropagation, and optimization algorithms such as Stochastic Gradient Descent (SGD) or Adam to minimize a suitable loss function. The presentation also discusses popular CNN architectures including LeNet, AlexNet, VGGNet, GoogLeNet (Inception), ResNet, DenseNet, EfficientNet, and MobileNet, each introducing architectural innovations that improve accuracy, computational efficiency, and scalability. Modern deep learning frameworks such as TensorFlow, PyTorch, and Keras make it straightforward to build, train, and deploy CNN models for real-world applications.

Today, CNNs are widely used in image classification, object detection, semantic segmentation, autonomous driving, facial recognition, medical diagnostics, satellite image analysis, industrial quality inspection, handwriting recognition, and video analytics. Although Vision Transformers (ViTs) have recently gained popularity for large-scale vision tasks, CNNs remain the preferred choice for many practical applications because of their computational efficiency, strong inductive biases, and excellent performance on limited datasets. Their ability to automatically learn robust visual features has established Convolutional Neural Networks as one of the most important breakthroughs in deep learning and a cornerstone of modern artificial intelligence.

https://docs.google.com/presentation/d/e/2PACX-1vSJsg0TarlJBTwFVtZBOk2RxDdFBHh864DgPX2LU5WRDjZKa9n8swH4s3qPTUtO5Q/pub?start=true&loop=true&delayms=10000

Large Language Models (LLMs) such as GPT, Llama, and Mistral are pre-trained on massive datasets and possess strong general-purpose capabilities. However, many real-world applications require these foundation models to follow a specific writing style, understand domain-specific terminology, or perform specialized tasks. Fine-tuning addresses this challenge by continuing the training of a pre-trained model on a smaller, task-specific dataset, enabling it to adapt its behavior while retaining the broad knowledge acquired during pretraining. As the presentation emphasizes, fine-tuning is most effective for teaching behavior, format, and tone, whereas Retrieval-Augmented Generation (RAG) remains the preferred solution for incorporating frequently changing or proprietary knowledge.

The presentation compares three approaches for improving LLM performance: prompt engineering, RAG, and fine-tuning. Prompting should generally be the first choice because it requires no additional training and allows rapid experimentation. When the limitation is missing or dynamic knowledge, RAG retrieves relevant external documents at inference time without modifying the model. Fine-tuning becomes the preferred option when consistent behavior, structured outputs, domain-specific language, or reduced prompt complexity are required. Traditional full fine-tuning updates every parameter in the model, offering maximum flexibility but demanding substantial GPU memory, storage, and computational resources while increasing the risk of catastrophic forgetting.

To overcome these limitations, modern LLM development increasingly relies on Parameter-Efficient Fine-Tuning (PEFT) techniques such as LoRA (Low-Rank Adaptation) and QLoRA. LoRA freezes the original model weights and trains only a small set of low-rank adapter matrices, often updating less than 1% of the total parameters. QLoRA extends this idea by quantizing the frozen base model to 4-bit precision while training LoRA adapters in higher precision, making it possible to fine-tune very large language models on a single consumer GPU with minimal loss in performance. These methods significantly reduce memory requirements, training costs, and storage while preserving the model’s general capabilities, making them the practical standard for modern LLM adaptation.

Successful fine-tuning depends as much on high-quality data as on the training algorithm itself. The presentation highlights the importance of using clean, well-formatted datasets, maintaining consistent prompt-response templates, covering difficult edge cases, balancing class distributions, and reserving unseen data for evaluation. Hyperparameters such as learning rate, training epochs, LoRA rank, alpha, and target modules must be selected carefully to avoid overfitting and preserve model performance. Finally, every fine-tuned model should be evaluated against the original base model using held-out datasets, task-specific metrics, LLM-as-a-Judge, and human evaluation. By combining well-designed datasets with efficient techniques like LoRA and QLoRA, organizations can customize powerful language models while keeping computational costs manageable, making fine-tuning one of the most important techniques in modern Generative AI.

https://docs.google.com/presentation/d/e/2PACX-1vRioryuqP8W6-6qv0Kc0OSEsWFhjJHBzKxAUgcBJ_7_aSM7FKBO6c79JxLcDt71pQ/pub?start=true&loop=true&delayms=10000

Large Language Models (LLMs) have transformed artificial intelligence by powering applications such as conversational AI, code generation, document summarization, and question answering. However, evaluating these models is far more complex than measuring traditional machine learning algorithms because there is rarely a single “correct” answer. As highlighted in this presentation, LLM evaluation requires a combination of probability-based metrics, overlap metrics, semantic similarity scores, benchmark evaluations, human judgments, and operational metrics to obtain a comprehensive understanding of model quality. No single metric can fully capture fluency, factual correctness, reasoning ability, and user satisfaction.

The presentation first introduces probability metrics, which measure how well a model predicts the next token during training. Cross-Entropy Loss serves as the primary optimization objective, while Perplexity (PPL) measures how “surprised” a model is by real text—the lower the perplexity, the better the language model. Additional diagnostics such as Bits-per-Token and Expected Calibration Error (ECE) provide insights into compression efficiency and confidence calibration. For generated text evaluation, metrics like BLEU, ROUGE, and METEOR compare model outputs with reference answers using n-gram overlap, whereas BERTScore and BLEURT leverage contextual embeddings to better capture semantic similarity, making them more suitable for evaluating paraphrases and natural language generation tasks.

Beyond automatic metrics, the presentation discusses task-specific evaluation through measures such as Accuracy, Precision, Recall, F1 Score, and pass@k, along with benchmark datasets including MMLU, HumanEval, GSM8K, HellaSwag, and TruthfulQA. These benchmarks assess different capabilities ranging from reasoning and coding to commonsense understanding and factual reliability. However, benchmark scores should always be interpreted carefully because factors such as data contamination, prompt sensitivity, benchmark saturation, and limited domain coverage can produce misleading conclusions about a model’s true real-world performance.

The presentation concludes by emphasizing that evaluating production-grade LLMs extends beyond model accuracy. Human evaluation remains the gold standard for assessing open-ended responses, while LLM-as-a-Judge has emerged as a scalable alternative for large-scale evaluation despite known biases. For Retrieval-Augmented Generation (RAG) systems, metrics such as Faithfulness, Answer Relevance, Context Precision, and Context Recall are essential for measuring retrieval quality. Finally, operational metrics including latency, throughput, cost per request, hallucination rate, toxicity, fairness, and jailbreak resistance determine whether an LLM is practical, safe, and deployable in production environments. The presentation recommends combining multiple evaluation methods, preventing benchmark leakage, and continuously monitoring models after deployment to achieve trustworthy and reliable AI systems.

https://docs.google.com/presentation/d/e/2PACX-1vS1RWy70p0GKCuFoIeQHExJk3xK6Y80ziICV-HFmQh88ygl5tv9sXAlf3Of5-U6jA/pub?start=true&loop=true&delayms=10000

Deep learning models have transformed artificial intelligence by enabling computers to recognize images, understand language, generate content, and solve complex prediction problems. At the heart of every neural network are two essential processes: Forward Propagation and Backpropagation. Forward propagation is responsible for generating predictions, while backpropagation enables the model to learn from its mistakes by adjusting its parameters. Together, these two processes form the complete learning cycle that powers modern neural networks and deep learning applications.

During forward propagation, data enters the input layer and passes through one or more hidden layers before reaching the output layer. At each neuron, the inputs are multiplied by weights, combined with a bias, and passed through an activation function such as ReLU, Sigmoid, or Tanh. These activation functions introduce non-linearity, allowing neural networks to model complex relationships beyond simple linear equations. Once the final output is produced, the prediction is compared with the actual target using a loss function, such as Mean Squared Error (MSE) for regression or Cross-Entropy Loss for classification, to measure the model’s prediction error.

The next stage is backpropagation, which enables the neural network to improve its predictions. Using the Chain Rule of Calculus, the algorithm computes gradients that indicate how much each weight and bias contributed to the prediction error. These gradients are propagated backward from the output layer through every hidden layer, allowing the model to determine how its parameters should be updated. An optimization algorithm such as Gradient Descent, Stochastic Gradient Descent (SGD), or Adam then adjusts the network’s parameters in the direction that minimizes the loss. This forward-and-backward cycle is repeated over many training epochs until the model converges and achieves better predictive performance.

Modern deep learning frameworks such as PyTorch and TensorFlow automate this entire learning process through automatic differentiation (Autograd), eliminating the need for manual gradient calculations. Functions like loss.backward() compute gradients automatically, while optimizers update the model parameters efficiently. Forward propagation and backpropagation are therefore the mathematical foundation of virtually every deep learning model used today, including computer vision systems, speech recognition, recommendation engines, autonomous vehicles, medical diagnostics, natural language processing, and Generative AI models. A thorough understanding of these concepts is essential for anyone beginning their journey into artificial intelligence and neural network development.

https://docs.google.com/presentation/d/e/2PACX-1vSnJh9WRADznL597jBIJN1_MK5qmA0LbyzaLlz5yF7s0ThagCfqdmPMSIv85bmu3g/pub?start=true&loop=true&delayms=1000

The Transformer Encoder is one of the most fundamental components of modern deep learning and artificial intelligence, serving as the backbone of many state-of-the-art language understanding models. Introduced in the groundbreaking paper “Attention Is All You Need,” the encoder is designed to transform an input sequence into rich contextual representations by allowing every token to attend to every other token simultaneously. This capability enables Transformer-based models to capture complex linguistic relationships far more effectively than traditional recurrent neural networks (RNNs) or long short-term memory (LSTM) networks.

Unlike sequential models that process one word at a time, the Transformer Encoder processes the entire input sequence in parallel. This parallelization dramatically reduces training time while improving the model’s ability to capture long-range dependencies between words. At the heart of the encoder lies the self-attention mechanism, where each token generates a Query (Q), Key (K), and Value (V) vector. By computing attention scores between all tokens, the encoder learns which words are most relevant to one another, producing context-aware embeddings that better represent the meaning of the input.

Each Transformer Encoder is composed of multiple identical encoder blocks, with every block containing two major components: Multi-Head Self-Attention and a Position-wise Feed-Forward Neural Network (FFN). The multi-head attention mechanism allows the model to focus on multiple contextual relationships simultaneously, while the feed-forward network further refines the learned representations. To improve training stability and preserve information, each sub-layer incorporates Residual Connections followed by Layer Normalization, enabling the construction of very deep neural networks without suffering from vanishing gradients.

Transformer Encoders are trained using backpropagation with optimization algorithms such as Adam or AdamW, typically minimizing Cross-Entropy Loss for language understanding tasks. Regularization techniques including dropout, learning rate scheduling, and label smoothing further improve model generalization and training stability. Once trained, the encoder produces high-quality contextual embeddings that can be fine-tuned for a wide variety of downstream applications.

The Transformer Encoder forms the foundation of numerous modern AI models, including BERT, RoBERTa, DistilBERT, ALBERT, and many other encoder-based architectures. These models excel at text classification, sentiment analysis, named entity recognition, semantic search, document retrieval, question answering, information extraction, and language understanding. Beyond natural language processing, encoder architectures have also been successfully adapted for computer vision, speech recognition, protein structure prediction, and multimodal learning.

Although Transformer Encoders deliver exceptional performance, they require substantial computational resources because the standard self-attention mechanism has quadratic time and memory complexity (O(n²)) with respect to sequence length. Researchers continue to develop more efficient attention mechanisms to improve scalability for longer documents and larger datasets. Despite these computational challenges, the Transformer Encoder remains one of the most influential innovations in artificial intelligence and serves as the foundation for many of today’s most advanced machine learning systems.

https://docs.google.com/presentation/d/e/2PACX-1vTC6oZa53REmAuyGpBC9b_W1JbAtYB1N9wl0RgOFs5xV5NdmMyS7dYn7i6UrrH_7Q/pub?start=true&loop=true&delayms=10000