← Return to Utils

Cellular Learning Automata

Emergent intelligence through distributed reinforcement learning

Epoch: 0
Global Objective (Rewards): 0.0%

1 Discrete Actions

The physical choice (Blue vs Orange) made by each cell this epoch, sampled from its probability distribution.

2 Internal Probabilities

The cell's confidence. Brighter blue = strongly prefers Blue. Brighter orange = strongly prefers Orange. Grey = uncertain.

3 Local Environment Score

Green cells satisfied the local objective (Reward). Red cells failed (Penalty). This is the immediate feedback signal.

4 Global Objective Over Time

The total percentage of rewarded cells. As local cells minimize their own penalties, the system maximizes this global metric.

How It Works: The Math & Theory

🧠 1. Learning Automata (LA)

A Learning Automaton is a fundamental reinforcement learning agent. It maintains a probability vector over a set of actions (here, Blue and Orange).

In each epoch, it:
1. Selects an action based on its probabilities.
2. Receives a response from the environment (Reward or Penalty).
3. Updates its probabilities to favor actions that yield rewards.

🔄 2. The LR-P Update Rule

This simulation uses the Linear Reward-Penalty algorithm. It dictates how probabilities change based on feedback.

If action i gets a REWARD (α):
P(i) = P(i) + α * (1 - P(i))
P(j) = P(j) - α * P(j)
Action i approaches 1.0, others shrink.
If action i gets a PENALTY (β):
P(i) = P(i) - β * P(i)
P(j) = P(j) + β * (1 / (r-1) - P(j))
Action i shrinks, others grow.

🎯 3. The Local Reward (Score)

In a Cellular LA, the "environment" is simply the neighboring cells. The Local Score is how a single cell evaluates its choice against its neighbors.

Score = Σ δ(Action, Neighbor_j)
Reward if Score ≥ 4 δ = 1 if colors match, 0 otherwise.

For Global Consensus, a cell is rewarded if it chooses the same color as the majority of its 8 neighbors (Moore neighborhood).

🌐 4. The Global Objective

The magic of CLA is that no cell knows the global state. They only see their immediate neighbors. However, a Global Objective Function emerges.

Objective(J) = Maximize Total Matches System naturally forms large color domains.

By greedily trying to maximize their own local rewards, the cells collectively minimize the energy of the entire grid. This is how decentralized, distributed agents solve complex optimization tasks like graph coloring, image processing, or pathfinding.