Kullback-Leibler Divergence Calculator for Probability Distributions

Supply two equally sized arrays whose probabilities sum to one, and receive the directed divergence in nats. The calculation is deterministic, uses the natural logarithm, and treats zero probabilities in the first distribution correctly. It also validates the distributions before calculating, so mismatched lengths, invalid totals, negative values, and zero probabilities in the reference distribution are reported clearly instead of producing a misleading infinite or undefined result.

Understand what KL divergence measures

Kullback-Leibler divergence compares a distribution P with a reference distribution Q by calculating the expected logarithmic ratio between their corresponding probabilities. It answers a directed question: how much information is lost when Q is used to represent outcomes that actually follow P? A result of zero means the distributions agree at every position, while a larger positive value indicates a greater discrepancy. The direction matters because swapping P and Q generally changes the answer. This calculator reports D(P || Q), so the first array is the distribution being evaluated and the second is the reference. It uses natural logarithms, which means the output is measured in nats rather than bits. KL divergence is not a distance metric: it is asymmetric and does not satisfy the triangle inequality. Those properties are intentional and make it useful for information theory, statistical inference, model fitting, and comparisons where one distribution has a clear reference role. Read the result as relative information discrepancy, not as an ordinary geometric distance.

Prepare valid probability arrays

Enter the same number of probabilities in both arrays and keep their outcome order aligned. If the first position represents rain in P, the first position must also represent rain in Q. Every value must be a finite number, probabilities cannot be negative, and each complete array must sum to one. The calculator allows a tiny tolerance around one to accommodate normal floating-point representation, but it does not silently normalize arbitrary weights. Normalize counts or scores before submitting them when they are not already probabilities. A zero in P is valid because an impossible observed outcome contributes zero to the sum. A zero in the reference distribution is rejected explicitly. A reference probability of zero makes the logarithmic ratio unavailable for any positive corresponding P value and can imply infinite divergence; this capability uses a strict contract and rejects every zero reference entry so callers never receive a disguised infinity. Clear validation also helps find transposed categories, truncated arrays, percentages entered as whole numbers, and distributions assembled from incompatible label sets before they enter a report or pipeline.

Interpret and use the result responsibly

The returned object contains the divergence and the unit nats. Values near zero indicate that P is well represented by Q, but there is no universal threshold separating a good model from a bad one. The importance of a value depends on the number of categories, the application, the sampling process, and the cost of errors. Compare results produced with the same category definitions and direction. In machine-learning evaluation, you might track divergence across model versions or data windows; in monitoring, you might compare the latest categorical distribution with an approved baseline; in research, you might report it alongside other measures and uncertainty estimates. Do not interpret the result as a probability, percentage, correlation, or symmetric distance. Sampling noise can also create divergence even when two samples come from the same underlying process, so small changes deserve context rather than an automatic alarm. Browser calculation is convenient for individual checks, while an API request costs $0.002 and supports repeatable automation. Because the algorithm is deterministic, identical validated inputs produce identical rounded JSON results across repeated calls.