Machine Learning - Simplified

In the context of linear algebra and machine learning, an affine transformation refers to a transformation that preserves points, straight lines, and planes. Vectors are transformed, and translations are allowed. Mathematically, an affine transformation of a vector x can be represented as w.x + b , where:  - w is a weight vector (matrix in higher dimensions),  - x is the input vector,  - b is a bias vector (a translation vector).   The operation w.x represents a linear transformation. When you add the bias b, it becomes an affine transformation. The reason w.x is considered part of an affine transformation is because affine transformations include both linear transformations and translations (shifts).   Why is it Affine?   - Linear Transformation:  The term w.x alone (without the bias) constitutes a linear transformation. A linear transformation maps vectors to other vectors without changing the origin; it involves scaling and rotating...

Linear Activation Functions For classification problems, linear functions like the identity function can have Separating Plane  to be linear hyperplane. With a single layer of ANN can only it can produce a line. A model composed of only linear functions, no matter how many layers it has, cannot learn non-linear boundaries because the composition of linear functions is itself a linear function. However, the introduction of non-linear activation functions between layers enables neural networks to learn non-linear separators. They are suitable for regression tasks, especially in the output layer for problems expecting a continuous output range. Sigmoid Functions Sigmoid Functions are mathematical function with S shaped curve(sigmoid curve) Types of Sigmoid functions 1. Logistic function Output between 0 and 1 f(x) = 1/(1+𝑒^(−𝑥) ) 2. Hyperbolic tangent Output between -1 and 1 f(x) = (𝑒^𝑥− 𝑒^(-𝑥 ))/(𝑒^𝑥+ 𝑒^(-𝑥) ) Effect of w and b on logistic function When we use the logistic ...

The learning rate is a hyperparameter that controls how much we are adjusting the weights of our network with respect to the loss gradient. It plays a critical role in the training of neural networks and other machine learning algorithms that use gradient-based optimization methods, such as gradient descent. The learning rate determines the size of the steps that the optimization algorithm takes towards the minimum of the loss function . It controls the size of the update steps. A higher learning rate makes larger changes to the weights, potentially overshooting the optimal values, while a lower learning rate makes finer adjustments. Python Code for Calculating Gradient and Weight Updation: ml-course/gd.ipynb at main · lovelynrose/ml-course (github.com) Importance of the Learning Rate:   Convergence:  A properly set learning rate helps the model to converge to a minimum loss efficiently. If the learning rate is too high, the model might overshoot the minimum, potentially l...

What is Gradient Descent? Gradient Descent is a fundamental optimization algorithm used in machine learning and artificial intelligence to minimize the cost function, which is a measure of how far off a model's predictions are from the actual values. The goal of gradient descent is to find the model parameters (typically weights in neural networks) that minimize the cost function, thereby improving the model's predictions. With numeric example, let us see how to find the model parameters using Gradient Descent where the activation function is the identity function f(x) = x. Python Code for Calculating Gradient and Weight Updation: ml-course/gd.ipynb at main · lovelynrose/ml-course (github.com) How Gradient Descent Works: Summary Initialization : Start with initial values for the parameters to be optimized (weights in the case of neural networks). Compute Gradient : Calculate the gradient of the cost function with respect to each parameter. The gradient is a vector that repre...

  Both loss functions and cost functions are central to the training and optimization of ANNs. They guide the adjustment of model parameters to improve performance. The loss function applies to a single data point, while the cost function is an aggregate measure over the entire dataset. Minimizing the cost function, which involves adjusting the model's parameters, is achieved through optimization algorithms like gradient descent. The following example finds the loss function and cost function for a hypothetical dataset solving the regression problem.

An Artificial Neural Network (ANN) is a computational model inspired by the networks of biological neurons found in animal brains. It is a cornerstone of artificial intelligence (AI) and machine learning, designed to simulate the way a human brain analyzes and processes information. ANNs are capable of learning and making intelligent decisions on their own by recognizing patterns in data. Structure of an ANN An ANN is composed of nodes (also called neurons) arranged in layers: an input layer, one or more hidden layers, and an output layer. Each node in one layer is connected to nodes in the next layer through pathways that are associated with weights. These weights adjust as the network learns from data, essentially tuning the ANN to perform a specific task. Structure of an ANN An ANN is composed of nodes (also called neurons) arranged in layers: an input layer, one or more hidden layers, and an output layer. Each node in one layer is connected to nodes in the next layer through pathwa...

We create models making approximations on the equation that would fit a training set. Some common approximations we make are linear(y=W.X), polynomial(given in notes). How do we validate our model approximation? A good model should neither underfit(simple models) nor overfit(complex models).  Bias Bias shows if a model has a tendency to underfit. To measure bias we create models on different training datasets using k-fold cross validation and measure how much the predicted values of the models created differ from the true value. If they underfit, the models created will be similar and the mean of the predicted values will vary a lot from the true value. That is, they have high bias. A model that fits well will have low bias. Variance On the other hand, variance measures if a model has the tendency to overfit. It checks how much the models created in k-fold cross-validation differ from one another. If they overfit, the variance between the predicted models will be high. This normall...