Bayesian Networks

-

A Bayesian network, also known as a belief network, probabilistic directed acyclic graphical model, or Bayes net, is a statistical model that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). 

Key components and concepts:

1. Nodes: 

Each node in the graph represents a random variable. These variables can be observable quantities, latent variables, unknown parameters, or hypotheses.

2. Edges: 

The edges between the nodes represent conditional dependencies; an edge from node A to node B indicates that B is dependent on A. The absence of an edge indicates conditional independence between variables.

3. Conditional Probability Tables (CPTs): 

Each node is associated with a probability function that takes a particular set of values for the node's parent variables and gives the probability of the variable represented by the node. For nodes without parents, the CPT reduces to the prior probability of the node.

4. Joint Probability Distribution: 

Using the chain rule for Bayesian networks, the joint probability distribution over all the variables can be decomposed into a product of conditional distributions:

5. Inference: 

Bayesian networks can be used to perform inference, which allows you to compute the probability of unknown variables given known variables. 

The following notes are problems solved for inferencing from Bayesian Networks.

Numerical Problem 1


Numerical Problem 2



Numerical Problem 3



6. Learning: 

There are two major tasks in learning a Bayesian network from data: structure learning and parameter learning. Structure learning involves finding the most suitable graph structure that represents the dependencies between variables. Parameter learning is about estimating the values of the CPTs, often through methods like maximum likelihood estimation or Bayesian estimation.

Create and train a Bayesian network from a dataset in Python

Data Preprocessing: 

Prepare your dataset, ensuring that it is clean and formatted correctly. Each column in your dataset should correspond to a node in the Bayesian network.

Structure Learning: 

Determine the structure of the Bayesian network. This can be done using domain knowledge or by employing algorithms that discover dependencies between variables.

Parameter Learning: 

Once you have a structure, you need to learn the parameters (i.e., the conditional probability distributions) from the data. This can be done through various methods such as Maximum Likelihood Estimation or Bayesian Estimation.

Inference: 

After the structure and parameters are learned, you can perform inference on the network to answer probabilistic queries about the data.

Sample Code

from pgmpy.models import BayesianNetwork

from pgmpy.estimators import HillClimbSearch, BicScore, MaximumLikelihoodEstimator

import pandas as pd


# Load and preprocess your dataset

data = pd.read_csv('your_dataset.csv')


# Step 1: Structure Learning

# Using Hill Climb Search with the BIC Score as scoring method

hc = HillClimbSearch(data, scoring_method=BicScore(data))

best_model_structure = hc.estimate()

print("Structure of the model:", best_model_structure.edges())


# Create a Bayesian Network with the structure learned from the dataset

model = BayesianNetwork(best_model_structure.edges())


# Step 2: Parameter Learning

# Using Maximum Likelihood Estimator to learn the CPDs from the data

model.fit(data, estimator=MaximumLikelihoodEstimator)


# Now, the Bayesian network has been trained on your dataset.

# You can now perform inference on the trained model:

from pgmpy.inference import VariableElimination

inference = VariableElimination(model)

# Query a variable

result = inference.query(variables=['HeartDisease'], evidence={'Exercise': 'Yes'})

print(result)


Comments

Post a Comment

Popular posts from this blog

ANN Series - 10 - Backpropagation of Errors

-
Image
Watch the video to understand the forward pass in an ANN. Backpropagation, short for "backward propagation of errors," is a fundamental algorithm used for training artificial neural networks. It efficiently computes the gradient of the loss function with respect to the weights of the network, which is essential for adjusting the weights and minimizing the loss through gradient descent or other optimization techniques. The process involves two main phases: a forward pass and a backward pass.     Forward Pass   1. Input Layer:  The input features are fed into the network. 2. Hidden Layers:  Each neuron in these layers computes a weighted sum of its inputs (from the previous layer or the input layer) plus a bias term. This sum is then passed through an activation function to produce the neuron's output. This process repeats layer by layer until the output layer is reached. 3. Output Layer:  The final output of the network is computed, which is then used to calculate the loss
Read more

Regression 10 - Pseudo-Inverse Matrix Approach - Relationship between MLE and LSA - Derivation

-
Pseudo-Inverse Matrix Approach: The pseudo-inverse matrix approach is essentially the OLS approach. It provides a solution to the linear regression problem by minimizing the sum of squared residuals between the observed and predicted values. This approach is a direct algebraic method that does not make any probabilistic assumptions about the residuals; it simply finds the best fit in the least squares sense.   For simple linear regression, the MLE estimates of the coefficients will actually be the same as the Ordinary Least Squares (OLS) estimates, which are also the same as what you get from the pseudo-inverse matrix method, under the assumption of i.i.d. normal errors. In this blog we show how to prove that  Maximization of likelihood function under conditional Gaussian noise for a linear model is similar to the minimization of the sum of the squares error function.  The complete derivation along with formulae used are provided. Ensure that you go through the previous blogs to get a
Read more

Regression 9 - Maximum Likelihood Estimation(MLE) Approach for Regression - Derivation

-
  MLE Approach: When performing linear regression using MLE under the assumption that the residuals (errors) are independently and identically distributed (i.i.d.) with a normal distribution, we estimate the regression coefficients that maximize the likelihood of observing the given data. In this blog we see how to perform regression on a dataset that applies MLE for model fitting. The mathematical assumptions and derivations are given in detail.   The MLE approach gives you point estimates for the coefficients (mean of the likelihood distribution), and you can also compute the variance-covariance matrix of these estimates, which gives you the variances (and covariances) of the estimates. These variances are a measure of the uncertainty or the spread of the likelihood distribution of the parameter estimates. For simple linear regression, the MLE estimates of the coefficients will actually be the same as the Ordinary Least Squares (OLS) estimates, which are also the same as what you get
Read more