Lesson4.5 The Final Design - Designing A Learning System

-

 Central components of Learning System

1. **Performance System**:

Input : New Problem

Output : Trace of the solution

- play against itself to get the sequence of moves

- Uses the learned evaluation function, denoted as V', to select its next move.

- As the evaluation function becomes more accurate, the quality of the AI’s move selection should improve.

2. **Critic**:

Input : Trace of game

Output : Finds (b,Vtrain(b))

i.e. estimates Vtrain(b) = V'(successor(b))

3. **Generalizer**:

Input : Training examples or dataset
Output : Estimate of weights
- the machine learning model

   - Takes the training examples provided by the Critic and creates a generalized hypothesis for the target function.

   - Applies an algorithm, like the LMS (Least Mean Squares) algorithm, to generalize from specific examples to a broader understanding that can be applied to unseen game states.

4. **Experiment Generator**:

Input : The linear function got as output from generalizer
Output : A new board state which the performance system plays

   - Uses the current hypothesis (the current state of the learned function \( p \)) to generate new problems or initial board states for the Performance System to solve.

   - Its goal is to select problems that will effectively improve the learning rate of the AI, thereby enhancing its playing ability.

   - In the given example, the strategy of the Experiment Generator is basic: it repeatedly proposes the same starting board configuration to begin a new game. However, more complex strategies could be implemented to present a variety of starting positions or scenarios, forcing the AI to adapt and learn from a broader range of situations.

Overall Strategy

The overall strategy of the learning system is cyclical and iterative. 

1. The Performance System plays the game based on its current knowledge. 

2. The Critic reviews the performance and generates training data. 

3. The Generalizer updates the AI’s knowledge based on that data. 

4. Finally, the Experiment Generator provides new challenges, propelling the cycle to start anew, each iteration aimed at refining the AI's ability to evaluate and select moves. 

This process is designed to incrementally improve the AI's performance in playing checkers.

Note that this iteration is required cause the training experience itself was based on heuristics. For examples that have defined labelled datasets, this repetition might not be required.

Comments

Post a Comment

Popular posts from this blog

ANN Series - 10 - Backpropagation of Errors

-
Image
Watch the video to understand the forward pass in an ANN. Backpropagation, short for "backward propagation of errors," is a fundamental algorithm used for training artificial neural networks. It efficiently computes the gradient of the loss function with respect to the weights of the network, which is essential for adjusting the weights and minimizing the loss through gradient descent or other optimization techniques. The process involves two main phases: a forward pass and a backward pass.     Forward Pass   1. Input Layer:  The input features are fed into the network. 2. Hidden Layers:  Each neuron in these layers computes a weighted sum of its inputs (from the previous layer or the input layer) plus a bias term. This sum is then passed through an activation function to produce the neuron's output. This process repeats layer by layer until the output layer is reached. 3. Output Layer:  The final output of the network is computed, which is then used to calculate the loss
Read more

Regression 10 - Pseudo-Inverse Matrix Approach - Relationship between MLE and LSA - Derivation

-
Pseudo-Inverse Matrix Approach: The pseudo-inverse matrix approach is essentially the OLS approach. It provides a solution to the linear regression problem by minimizing the sum of squared residuals between the observed and predicted values. This approach is a direct algebraic method that does not make any probabilistic assumptions about the residuals; it simply finds the best fit in the least squares sense.   For simple linear regression, the MLE estimates of the coefficients will actually be the same as the Ordinary Least Squares (OLS) estimates, which are also the same as what you get from the pseudo-inverse matrix method, under the assumption of i.i.d. normal errors. In this blog we show how to prove that  Maximization of likelihood function under conditional Gaussian noise for a linear model is similar to the minimization of the sum of the squares error function.  The complete derivation along with formulae used are provided. Ensure that you go through the previous blogs to get a
Read more

Regression 9 - Maximum Likelihood Estimation(MLE) Approach for Regression - Derivation

-
  MLE Approach: When performing linear regression using MLE under the assumption that the residuals (errors) are independently and identically distributed (i.i.d.) with a normal distribution, we estimate the regression coefficients that maximize the likelihood of observing the given data. In this blog we see how to perform regression on a dataset that applies MLE for model fitting. The mathematical assumptions and derivations are given in detail.   The MLE approach gives you point estimates for the coefficients (mean of the likelihood distribution), and you can also compute the variance-covariance matrix of these estimates, which gives you the variances (and covariances) of the estimates. These variances are a measure of the uncertainty or the spread of the likelihood distribution of the parameter estimates. For simple linear regression, the MLE estimates of the coefficients will actually be the same as the Ordinary Least Squares (OLS) estimates, which are also the same as what you get
Read more