Artificial intelligence now beats some of the best human forecasters

Meaning:Parity is the state of being equal — same level, same standing, neither side ahead. It is the word you reach for at the moment a gap closes, which is exactly why the article uses it: on the Forecasting Research Institute's set of questions, AI systems are no longer trailing the very best human forecasters, they are level with them. Notice how precise that is, and how easy it would be to overstate — parity is not superiority. The machines caught up; the article is careful not to say they pulled ahead.

Meaning:Something esoteric is understood by, or available to, only a small circle with special knowledge — specialist, obscure, inside-baseball. In the article it describes data rather than ideas: the numbers Mantic feeds its bot are locked behind paywalls, so the chatbots everyone else can use have never seen them. That is the whole edge being described. When every competitor is reading the same public internet, the advantage moves to whoever holds the esoteric stuff.

Meaning:A charlatan is someone who falsely claims skill, knowledge or special powers in order to impress people or take their money — a fraud wearing an expert's costume. The word carries a whiff of the fairground and the fortune-teller, which is why it lands so well here, right after the article's parade of astrologers and racing tipsters. The point being made is about accountability: once a prediction gets a number and a score, a charlatan's record can be checked and found wanting, whereas vague prophecies never can be.

Anyone can say "I think it'll rain tomorrow." Say it every day for a year and at the end nobody — including you — can tell whether you were any good. Hundreds of people entered the Metaculus Cup and came out ranked in order, so something must have solved that problem. The solution comes from statistics, and it is called a scoring rule.

The first move is to stop asking for a yes or a no and ask for a number. Not "will it rain?" but "what's the chance of rain?" — 70%, say. When tomorrow arrives, reality is either a 1 (it rained) or a 0 (it didn't), so now there is a gap you can measure: that is the "distance of their prediction from the true answer" in the quote. The most common rule, the Brier score, takes that gap and squares it. Say 70% and it rains: you were off by 0.30, squared is 0.09. Say 70% and it stays dry: off by 0.70, squared is 0.49. Low is good.

The squaring is the clever part, because it is what stops you gaming the system. Being wrong after claiming 90% costs you 0.81, more than three times what being wrong after a cautious 50% costs you (0.25) — so bluffing confidence at everything is punished hard. But hedging at 50% forever is no escape either: that scores a mediocre 0.25 every single time, rain or shine, and never beats anyone. The only way to score well is to say what you actually believe, as precisely as you actually believe it. (Metaculus adds a wrinkle the article names — points for being "closest for longest" — which rewards getting there early, not just being right at the end.)

Run this over hundreds of questions and luck averages out, leaving skill behind. That is why numbers "weed out the charlatans": an astrologer's prediction is worded loosely enough to fit whatever happens, while a scored forecaster's record is a column of numbers anyone can add up. So whenever somebody tells you they called it, ask the two questions a scoring rule asks: what else did you call, and what number did you put on it beforehand?

The article closes on a genuinely odd thought: as forecasters get better, they may end up proving how much of the future cannot be forecast at all. That only makes sense if there is a hard wall out there somewhere — and in mathematics and physics there is one, with a name. It is called chaos.

Chaos here does not mean randomness, and it does not mean disorder, which is the everyday sense of the word. A chaotic system obeys exact rules with no dice in them anywhere; what makes it chaotic is that it amplifies tiny differences. The atmosphere is the textbook case. To forecast it, you load a computer with today's temperature, pressure and wind from weather stations, balloons and satellites, and let the equations of fluid physics run forward. But no thermometer is exact, and there is no weather station over most of the Pacific, so the starting picture is off by a hair in a million places at once. In the atmosphere those hair-sized errors roughly double every day or two. A fraction of a degree wrong today is a whole degree wrong within the week; a storm that should sit off the coast is placed inland; and by around two weeks the forecast has drifted so far that it beats nothing — you would do just as well looking up what the weather is normally like there in September. That is the "15 days" in the quote: not a shortage of computing power, and not something a bigger model fixes, but a limit sitting inside the air itself.

The formal name for the mechanism is sensitive dependence on initial conditions, and the pop-culture version is the butterfly whose wings supposedly change the weather a continent away. The useful part travels well beyond weather. When a prediction gets steadily worse the further out it reaches, more data and a smarter model may not be the answer: in some systems the errors in any starting measurement grow until none of the original precision survives. So before asking who can forecast a thing best, it is worth asking how far out anyone can forecast it at all.