What Is Reinforcement Learning and How Does an Agent Learn from Rewards?

Reinforcement learning (RL) is a machine learning approach in which an agent learns by interacting with an environment, taking actions, and adjusting its behavior based on the rewards it receives. Instead of being told the correct answer for every input, the agent tries things, observes outcomes, and gradually favors actions that lead to better cumulative reward. RL is the right choice when you can define a goal as a reward signal but cannot easily label the correct action for every situation—game playing, control tasks, and sequential decision-making are typical fits. It is a poor fit when you already have a large labeled dataset of correct answers, where supervised learning is simpler and more reliable.

The agent–environment loop

Every RL problem reduces to a repeating loop:

1. The agent observes the current state of the environment.
2. It selects an action according to its current policy.
3. The environment transitions to a new state and returns a reward.
4. The agent uses that reward to update its policy or value estimates.
5. Repeat until the episode ends or training stops.

The agent's objective is not to maximize the next reward but the cumulative reward over time, which is why RL problems are described as sequential. A move that scores nothing now may set up a much larger payoff later—this delayed-credit structure is what separates RL from ordinary prediction tasks.

How RL differs from supervised and unsupervised learning

A useful rule of thumb: if you can write down the right answer for each example, use supervised learning. If you can only say whether an outcome was good or bad after the fact, and the agent's choices affect what happens next, RL is worth considering.

Core concepts you need to recognize

Policy

The policy is the agent's strategy: a mapping from states to actions (or a probability distribution over actions). Training an RL agent essentially means improving this mapping.

Value

A value function estimates how much total reward the agent can expect from a state, or from taking an action in a state. Value estimates let the agent reason about long-term consequences rather than only immediate reward, and many algorithms learn a value function as an intermediate step toward a good policy.

Exploration vs. exploitation

The agent must balance trying new actions (exploration) to discover better strategies against repeating known-good actions (exploitation) to collect reward. Pure exploitation can lock the agent into a mediocre strategy; pure exploration never cashes in what it has learned. Most practical algorithms use a schedule or randomness to manage this trade-off.

Reward design

The reward function is the specification of the problem. A sparse reward (given only on success) makes learning slow because the agent rarely stumbles onto the goal. A misshapen reward can be worse: the agent will optimize exactly what you wrote, including unintended shortcuts. Designing rewards that are dense enough to guide learning but faithful to the actual objective is one of the most consequential and error-prone parts of an RL project.

Common algorithms and tooling

• Q-learning and Deep Q-Networks (DQN): learn action values and are suited to discrete action spaces, such as choosing among a fixed set of moves.
• Policy gradient methods: directly adjust the policy's parameters, which extends naturally to continuous actions like steering or joint torques.
• Actor–critic methods: combine a policy (actor) with a value estimate (critic) to reduce the variance of policy updates.

In practice, these are implemented in Python, often with deep learning frameworks such as TensorFlow or PyTorch, and run against simulated environments so the agent can take many thousands of trial-and-error steps safely and quickly. Greg Surma's portfolio lists reinforcement learning among its topics and includes small game-oriented projects—an α-β Minimax tic-tac-toe variant, a Conway's Game of Life implementation, and a prisoner's-dilemma research environment—which illustrate the same underlying idea: an agent choosing moves in a defined environment and being scored on outcomes. Note that minimax is a classic search technique rather than a learned policy, so it is a useful contrast case: it shows what you can do when the environment's rules are fully known and searchable, versus learning from experience when they are not.

A small concrete example

Consider a grid-world agent that must reach a goal tile. The state is its position; the actions are up, down, left, right; the reward is −1 per step and +10 on reaching the goal. At first the agent moves randomly and accumulates penalties. Over many episodes, value estimates propagate backward from the goal, so tiles near the goal become attractive, and the policy shifts toward the shortest path. If you instead rewarded every step with +1, the agent would learn to wander forever—a direct demonstration of why reward design matters more than algorithm choice in many projects.

Common pitfalls

• Sparse rewards: the agent never gets a learning signal. Consider shaping rewards or starting with easier versions of the task.
• Reward hacking: the agent finds a loophole that scores well without solving your real problem. Test the learned behavior, not just the training curve.
• Unstable training: RL results are sensitive to hyperparameters, random seeds, and environment details. Run multiple seeds and report variance rather than a single lucky run.
• Too much compute for the question: if a simple rule or supervised model solves the task, RL adds complexity without benefit.
• Evaluation gaps: performance during training can differ sharply from performance under slightly changed conditions, so evaluate on held-out scenarios.

Deciding whether to use RL

Choose RL when the problem is sequential, feedback is a reward rather than a label, and the agent's actions influence future states. Choose supervised learning when labeled examples exist, and classical search or planning when the environment's rules are fully known and the state space is manageable. If you want to try RL hands-on, start with a small simulated environment and a discrete action space, verify that your reward produces the behavior you intend, and only then scale up the algorithm or the environment.