Constructing Kernels

-

 Properties of Kernel Functions

To perform this transformation we need to chose proper kernel functions. Functions that satisfy the following properties make good kernel functions.

1. Symmetry

K(x,y) = K(y,x)

That is, the similarity measure between x and y is the same as the similarity measure between y and x. The kernel matrix will be a symmetric matrix.

2. Positive Semidefinitiveness

The kernel matrix formed must be positive semidefinite.


Eigenvalues and Eigenvectors



Mercer's Theorem

It states that a continuous, symmetric and positive semidefinite function is a valid kernel. It also states that the kernel function corresponds to the dot product(inner product in Euclidean space) in higher dimensional space.

Constructing Common Kernel functions


The following pdf shows how to construct kernels and check for valid kernels.



Comments

Post a Comment

Popular posts from this blog

ANN Series - 10 - Backpropagation of Errors

-
Image
Watch the video to understand the forward pass in an ANN. Backpropagation, short for "backward propagation of errors," is a fundamental algorithm used for training artificial neural networks. It efficiently computes the gradient of the loss function with respect to the weights of the network, which is essential for adjusting the weights and minimizing the loss through gradient descent or other optimization techniques. The process involves two main phases: a forward pass and a backward pass.     Forward Pass   1. Input Layer:  The input features are fed into the network. 2. Hidden Layers:  Each neuron in these layers computes a weighted sum of its inputs (from the previous layer or the input layer) plus a bias term. This sum is then passed through an activation function to produce the neuron's output. This process repeats layer by layer until the output layer is reached. 3. Output Layer:  The final output of the network is computed, which is then used to cal...
Read more

Naive Bayesian Classifiers - Multinomial, Bernoulli and Gaussian with Solved Examples and Laplace Smoothing

-
Image
 Learn about the Naive Bayes Classifier in the following notes. Numeric Example with Dataset (Transactional Data) Sum1 Sum2 Consider the following dataset. Apply the Naïve Bayes classifier to the following frequency table and predict the type of fruit given it is {Yellow, Sweet, Long}. Solution can be viewed in the following pdf. Numeric Example with Text Data Multinomial Naive Bayes vs Bernoulli Naive Bayes Multinomial Naive Bayes and Bernoulli Naive Bayes are both variations of the Naive Bayes algorithm, and they are used for different types of data distributions: 1. Multinomial Naive Bayes:    - The Multinomial Naive Bayes classifier is used for data that is multinomially distributed, which typically means it is used for discrete data.    - It is particularly suitable for text classification problems where features (or words) can occur multiple times. For example, it can be used for document classification where the features are the frequencies...
Read more

Clustering - K means Clustering

-
K-means clustering is a popular unsupervised learning algorithm used to partition a dataset into a set of distinct, non-overlapping groups (or clusters) based on similarity. The goal is to organize the data into clusters such that data points within a cluster are more similar to each other than to those in other clusters. The "K" in K-means represents the number of clusters to be identified from the data, and it must be specified a priori. This method is widely used in data mining, pattern recognition, image analysis, and machine learning for its simplicity and efficiency, especially in handling large datasets.   How K-means Clustering Works   K-means clustering follows a straightforward iterative procedure to partition the dataset:   1. Initialization:  Choose K initial centroids randomly or based on a heuristic. Centroids are the center points of the clusters.   2. Assignment Step:  Assign each data point to the nearest centroid. The "nearest" is ...
Read more