Genetic Algorithm in Machine Learning: A Step-by-Step Breakdown

Most genetic algorithm explainers spend three paragraphs on Darwin and then hand-wave the actual mechanism. This one skips the biology lecture and does the arithmetic instead.

A genetic algorithm is an optimisation technique inspired by natural selection. It evolves a population of candidate solutions across generations using three operators, selection, crossover, and mutation, gradually favouring solutions that score better on whatever you're trying to optimise. No derivatives required, which is the entire reason it exists.

Below: the mechanism in five phases, one full generation worked by hand with real numbers you can check yourself, runnable Python, and an honest answer to where genetic algorithms actually earn their keep in ML (spoiler: not training your neural network's weights).

What Is a Genetic Algorithm in Machine Learning?

A genetic algorithm (GA) searches for a good solution by evolving a population of candidates instead of calculating a gradient and following it downhill. Each candidate gets scored by a fitness function, the better performers get to "reproduce," and their offspring, combined and occasionally mutated, form the next generation. Repeat enough times and the population drifts toward better solutions, the same way natural selection drifts a species toward traits that survive.

The method was formalised by John Holland in his 1975 book, Adaptation in Natural and Artificial Systems, and it's held up remarkably well for a fifty-year-old idea. In ML terms, it's a gradient-free optimisation method, useful precisely in the cases where you can't take a derivative of your objective function, or where the search space is discrete and lumpy rather than smooth. That's a narrower niche than the marketing around "evolutionary AI" sometimes implies, and we'll get specific about that niche later in this piece.

For the broader landscape this sits inside, our machine learning tutorial hub is worth a browse if GAs are your first stop into optimisation methods generally.

How a Genetic Algorithm Works: 5 Phases

Snippet version first, detail after: initialise a population, evaluate fitness, select parents, apply crossover, apply mutation, repeat until you hit a stopping condition.

1. Initial Population & Encoding

You start by generating a batch of random candidate solutions, each encoded some consistent way:

• Binary encoding, a string of 0s and 1s. The classic choice, and what we'll use for the worked example below.

• Real-valued encoding, actual numbers (floats), common for continuous optimisation problems like tuning a learning rate.

• Permutation encoding, an ordered sequence, used for problems like the travelling salesman where order is the whole point.

Encoding choice matters more than people expect early on. Get it wrong and crossover starts producing nonsense offspring that don't even represent valid solutions.

2. Fitness Function

The fitness function is the GA's equivalent of a loss function, just flipped. Where gradient descent minimises loss, a GA typically maximises fitness (you can always flip the sign if your problem is naturally framed as minimisation).

A good fitness function needs to actually discriminate between decent and bad solutions across the whole population, not just cleanly identify the one perfect answer. If every candidate scores roughly the same, selection has nothing to work with, and the algorithm ends up wandering randomly instead of evolving anywhere.

3. Selection (Roulette Wheel & Tournament)

Selection decides who gets to reproduce, and it's weighted so fitter individuals get picked more often, without completely locking out the weaker ones.

• Roulette wheel selection, each individual gets a slice of a wheel proportional to its fitness, then you spin. Higher fitness, bigger slice, better odds, not a guarantee.

• Tournament selection, pick a few individuals at random, let the fittest of that small group win. Simpler to implement and tends to control selection pressure more predictably.

We'll use roulette wheel selection in the worked example below, mostly because watching the actual probabilities is the fastest way to understand why fitter doesn't mean guaranteed.

Hiring Partners:

GoogleAmazonMicrosoftFlipkartAdobe1200+ more

4. Crossover (Single-Point, Two-Point, Uniform)

Crossover combines two parent chromosomes into offspring, this is where a GA actually exploits what's already working in the population.

• Single-point crossover, pick one cut point, swap everything after it between the two parents.

• Two-point crossover, pick two cut points, swap the middle segment.

• Uniform crossover, each gene gets independently inherited from either parent, coin-flip style.

Single-point is the one we'll walk through by hand next, it's the easiest to verify with a calculator and a bit of patience.

5. Mutation & Termination

Mutation randomly flips a gene, usually with a low probability per bit, somewhere in the 0.1% to 5% range depending on the problem. It's not there to improve any single individual. It's there to keep genetic diversity alive so the population doesn't converge prematurely on a merely-okay solution and get stuck.

Termination happens on whichever comes first:

• A fixed number of generations has run

• Fitness has plateaued for several generations in a row, no meaningful improvement left to squeeze out

• A target fitness value has actually been reached

Worked Example: One Full Generation by Hand

Here's the classic toy problem, maximise f(x) = x² for x between 0 and 31, encoded as 5-bit binary strings (since 2⁵ = 32 covers the full range). Population size 4.

Generation 0, four random chromosomes and their fitness:

Total fitness across the population is 1,170, average 292.5. Chromosome 11000 (x=24) is the strongest performer here by a wide margin, which is exactly why it gets nearly half the roulette wheel.

Roulette selection, weighted by those probabilities, produces this mating pool: 11000, 10011, 11000, 01101. Notice 11000 got picked twice (its 49% slice made that likely) and 01000, the weakest chromosome, didn't get picked at all this round. That's the mechanism working as intended, not a bug.

Pair the mating pool up and apply single-point crossover, using a different cut point per pair to show the mechanic clearly:

Walking Pair A by hand: parent 11000 and parent 10011, cut after the second bit. Parent 1 contributes its first two bits (11), parent 2 contributes everything from its third bit onward (011). Glue them together and you get 11011, which decodes to 27. Swap the contribution direction for the second child and you get 10000, decoding to 16.

Average fitness of the four children, before any mutation, is 438.5, already a solid jump from 292.5. That's crossover alone, recombining the genes of fitter parents tends to produce fitter offspring, though not every single time.

Now one mutation, flip the last bit of the second child, 10000 becomes 10001. That single bit flip moves its decoded value from 16 to 17, fitness from 256 to 289. Small nudge, but it's exactly the kind of small nudge that occasionally rescues a population from getting stuck on a local optimum.

One generation, best fitness up from 576 to 729, average up from 292.5 to 446.75. Run this loop another fifteen or twenty times and the population converges on 11111 (x=31, fitness 961), the actual maximum. Small example, but the mechanism scales to problems where you genuinely can't just take a derivative and walk downhill.

Where Genetic Algorithms Are Used in Machine Learning

This is the part most GA explainers skip entirely, and it's the actual answer to why anyone searching this term should care.

Hyperparameter Tuning

Learning rate, number of layers, tree depth, regularisation strength, these live in a search space that's often discrete, non-smooth, and expensive to evaluate. A GA can treat each hyperparameter combination as a chromosome, train-and-score each one as its fitness function, and evolve toward better configurations without ever needing a gradient through the training process itself. Our neural network hyperparameter tuning page covers the broader landscape of methods here, GA is one option among several, not the default.

Feature Selection

Deciding which subset of features actually helps a model, out of dozens or hundreds of candidates, is a combinatorial problem, not a differentiable one. Encode a chromosome as a bit string where each bit represents "include this feature or not," fitness equals model performance on that subset, and a GA searches the combination space directly. Our feature engineering page covers where this fits into the broader modelling workflow.

Neural Architecture Search

Choosing the number of layers, filter sizes, and connection patterns for a neural network is exactly the kind of discrete, non-differentiable search where GAs have shown real results. Google's AmoebaNet work (Real et al., 2019) used evolutionary methods to search architecture space and found configurations competitive with, and in some cases beating, architectures designed by hand or found via reinforcement learning. It's a genuinely modern proof point for a fifty-year-old algorithm, not just a textbook footnote.

Genetic Algorithms vs Gradient-Based Optimization

This is the honest comparison most GA content avoids, mainly because "evolutionary AI" sells better as a headline than "a slower method for a narrow set of problems." Here's where each one actually wins:

Neither replaces the other, they're not competing for the same job. Training a neural network's millions of weights is gradient descent's job, full stop, a GA would take approximately forever on that scale. But picking the architecture, or the hyperparameters, or which features even go in, that's where gradients don't exist to follow in the first place, and a GA (or similar gradient-free method) is a legitimate tool instead of a novelty. Our optimizers in deep learning page covers the gradient-based side of this comparison in depth, and the MathWorks GA documentation is a solid reference if you want to see the operator variants implemented in a production toolbox rather than a teaching example.

Advantages, Limitations & Key Takeaways

What GAs get right:

• No gradient required, works on discrete, non-differentiable, or genuinely messy search spaces

• Searches globally, less prone to getting permanently stuck in one local optimum than a purely local method

• Naturally parallelisable, evaluating a population's fitness is an embarrassingly parallel problem

Where they fall short:

• Computationally expensive, evaluating fitness for an entire population every generation adds up fast

• Sensitive to its own hyperparameters, population size, mutation rate, crossover rate all need tuning, which is a little ironic for an optimisation method

• No convergence guarantee, a GA can run for a long time and still not find the true optimum, only something reasonably good

If the probability and optimisation math underneath all this feels shaky, Scaler's free Maths for Machine Learning course is worth a detour, it covers the foundations that make both GAs and gradient-based methods actually click.

Want to master both classic and modern optimisation for ML? Explore Scaler's AI & ML Program.