What is the LabDerIA Predictions Observatory?

It is a research tool developed by the Laboratory of Law and Artificial Intelligence at Universidad Panamericana. It verifies, one by one, the public predictions made by AI and Law experts and scores their accuracy using the Brier Score. The observatory began with U.S. data (141 predictions by 68 experts published in The National Law Review ) and is now extending its methodology to Mexico.

What is the Brier Score and why does it matter for legaltech?

The Brier Score (Brier, 1950) is a standard metric for evaluating the accuracy of probabilistic predictions: it penalizes both overconfidence and excessive caution. Applied to legaltech, it enables an objective comparison of who actually predicts the future of AI in Law correctly, independent of marketing narratives or the forecaster's institutional prestige.

How does this ranking differ from standard 'predictions for 2026' surveys?

Standard surveys collect opinions without requiring explicit probabilities or verifiable criteria. The LabDerIA Prediction Ranking requires four elements: an explicit probability (5%–95%), a resolution deadline, an objective verification criterion, and a specificity score to penalize the Barnum effect (predictions so vague that almost any outcome would confirm them). Without all four, a prediction does not enter the ranking.

Who Gets AI and Law Predictions Right?

It was during a session titled “Innovation, Ethics, and Agent-Based AI,” organized jointly by ANADE’s Digital Law and Privacy Committee, Microsoft, and the Law and Artificial Intelligence Laboratory at the Universidad Panamericana, that I presented a new line of research from LabDerIA: a Ranking of AI-Law Predictions for Mexico.

The premise can be summed up in a single line; it is precisely its simplicity that makes it troubling: in the world of AI and law, everyone predicts the future. That’s easy. But who verifies who was right?

Predictions Without Accountability

Every year, legaltech experts publish their lists of trends, sometimes setting a date by which AI will “transform” law firms, announcing that regulation will come “before 2027,” or that large language models will “dominate contract review” within a certain timeframe. These claims circulate, are cited in corporate presentations, and are used to justify tool budgets and hiring decisions.

Legitimacy goes to the loudest voice, but it should go to the most reliable one.

The Study in the United States

To validate the methodology, we collected 141 predictions made by 68 experts in 2024 for the year 2025, published in The National Law Review, and verified them one by one. The results are available in the Prediction Observatory.

This exercise confirmed what the literature on forecasting has established since Tetlock: the predictions made with the greatest confidence are not the most accurate; the author’s institutional prestige is not correlated with the accuracy of their predictions.

The Methodology

The Prediction Ranking is not a survey of perceptions. It is based on three validated frameworks.

The Brier Score (Brier, 1950) as the central metric: it evaluates the accuracy of probabilistic predictions and penalizes both overconfidence and excessive caution. A prediction such as: “With a 90% probability, X will occur before December” is verifiable and scoreable; whereas one stating “AI will likely change law firms” is not.

The proper scoring rules (Gneiting and Raftery, 2007, JASA): these require the author to be honest about their uncertainty, as any attempt to circumvent the score results in a worse outcome for those who take the risk.

Philip Tetlock’s Good Judgment Project framework, validated by IARPA through more than twenty-five peer-reviewed articles: it demonstrates that the best forecasters are not the most self-assured, but those who frequently revise their beliefs and calibrate their confidence based on available data. It sounds like a description of the perfect son-in-law, but this is precisely what is required in science and academia.

Four Conditions for Inclusion in the Rankings

Unlike typical “predictions for 2026” polls, the tool we’ve developed imposes four non-negotiable conditions.

A quantified probability, between 5% and 95%. Simply stating that something “will likely happen” is insufficient.
A deadline for resolution. Without a deadline, verification is impossible.
An objective verification criterion. The prediction must be verifiable using publicly available information.
A specificity score, to discourage the Barnum effect: the tendency to make predictions so vague that almost any outcome would confirm them (hello, astrologers!).

Why This Is Important for Mexico

The Mexican legaltech ecosystem is particularly fragile: data on the actual adoption of AI tools in law firms and legal departments is scarce; when it does exist, it most often comes from the providers of those very tools. Narratives therefore often circulate without any empirical counterbalance, and almost always within an ecosystem that is overly eager to follow the economic trends of the country’s northern neighbors.

A Prediction Ranking built on a solid methodology and grounded in verifiable data offers precisely this counterbalance: not just another opinion on the future, but a way to determine who has made sound predictions in the past.

The result will be a public ranking where the visibility and reputation of the best forecasters will be based on evidence, not marketing. For those working at the intersection of AI and law in Mexico, contributing to this project from its inception is an opportunity to strengthen both the rigor of their own analyses and the collective credibility of the sector. Not bad, right? So, join the adventure!

Originally published on LinkedIn on February 25, 2026, at the launch session of the Prediction Ranking: ANADE + Microsoft + LabDerIA UP.