Chapter 10
Where to drill next
Everything so far produced beliefs. This chapter turns beliefs into actions — the part of the field with the clearest theory, the most obvious economic value, and the least deployment. If you only read one chapter twice, read this one.
Prediction is not the product
A company does not buy a probability map. It buys answers to: Should we drill? Where? How deep? What do we do after we see the result? When do we stop and walk away? Those are decisions under uncertainty with a budget constraint, and there is a mature formalism for them.
Start with the static case. Actions a, uncertain state s, utility u(a, s). You pick the action maximising expected utility:
a* = argmaxa Es ~ p(s) [ u(a, s) ]
Now the interesting question: an experiment e — a drill hole, a survey — will return an observation o before you must choose. What is that experiment worth?
Value of information
VOI(e) = Eo [ maxa Es ~ p(s | o) [ u(a, s) ] ] − maxa Es ~ p(s) [ u(a, s) ]
Expected utility acting with the information, minus expected utility acting without it. Note the max is INSIDE the expectation in the first term — that is the whole point: information is valuable because it lets you condition your action on what you learn.
Three properties, each of which reorganises how you think about a modelling project:
- VOI ≥ 0, always. By Jensen — you can always ignore the observation. Information never hurts a rational agent (though paying for it certainly can, which is why you compare VOI to the cost of the experiment).
- VOI = 0 whenever the observation cannot change your action. This is the killer. If you would drill regardless of the survey result, the survey is worth nothing no matter how informative it is. Precision without decision-relevance has no value.
- Value of perfect information bounds everything. Compute VOPI first — it is cheap, and if it is less than the price of the survey you can stop immediately and never build the model.
VOI is the decision-theoretic parent of every acquisition function in Bayesian optimisation. Expected improvement, upper-confidence-bound, and expected-information-gain are all cheap myopic approximations to it, chosen because the real thing is expensive. Exploration is the domain where paying for the real thing is justified: your "function evaluation" costs $250,000 and takes six weeks, so spending a day of compute choosing it is trivially worthwhile.
Why myopic VOI is not enough
Standard VOI evaluates one experiment at a time. Real campaigns are long sequences, and each hole informs where the next should go. A greedy one-step-lookahead policy will systematically under-value an observation whose payoff is that it enables a better next observation — the classic failure of myopic active learning.
Optimising the whole closed-loop sequence directly is a combinatorial search over an exponentially branching tree of (action, observation) histories. It is intractable in the literal sense. The way out is to recognise the structure for what it is.
The POMDP formulation
A partially observable Markov decision process is the tuple ⟨S, A, O, T, Z, R, γ⟩. You cannot see the true subsurface s, so you maintain a belief b(s) — a distribution over subsurface models — and update it by Bayes' rule after every observation:
b′(s′) ∝ Z(o | s′, a) · Σs T(s′ | s, a) · b(s)
The optimal policy satisfies a Bellman equation over beliefs rather than states — which converts a partially observed problem into a fully observed one on a much nastier (continuous, infinite-dimensional) space:
V*(b) = maxa [ R(b, a) + γ · Σo P(o | b, a) · V*(ba,o) ]
b^{a,o} is the belief after taking action a and observing o. The max-inside-expectation structure of VOI is visible here as the recursive case: VOI is a one-step POMDP.
| State s | The true 3-D distribution of mineralisation, plus which locations have already been measured |
| Belief b | An ensemble of geological realisations consistent with everything seen so far — exactly the simulation output of Chapter 4 |
| Actions A | Every candidate drill location, plus the two terminal actions: MINE and ABANDON |
| Observations O | The assays that come back from a hole |
| Reward R | Negative cost per measurement; zero for abandoning; expected profit for mining |
Including MINE and ABANDON as actions is what makes this a decision system rather than a mapping system. The agent is not asked "where is the ore" — it is asked "keep spending, or commit, or walk away," which is the question the board is actually asking.
The Intelligent Prospector
John Mern and Jef Caers built exactly this, and published it as The Intelligent Prospector v1.0 (Geoscientific Model Development 16, 289–313, 2023). It is the cleanest entry point into the field and the code is open.
They solve the POMDP with POMCPOW — Partially Observable Monte Carlo Planning with Observation Widening, a Monte Carlo tree search variant that handles continuous observation spaces via progressive widening (without it, MCTS in a continuous observation space never revisits a node, and the tree degenerates into a set of disconnected paths). Belief is carried by a particle filter; the reported configuration runs 10,000 simulations per decision.
The headline is that second-to-last number in context: the sequential policy reached grid-level knowledge in roughly half the measurements. In a domain where a measurement is a drill hole, halving the count is the entire value proposition. Performance degraded in the hardest configuration (two ore bodies at unknown positions, ~68% decision accuracy) and the authors attribute that to particle-filter convergence — an honest and instructive limitation, since belief representation is the hard part of any real POMDP.
v2.0: when the prior itself is wrong
Intelligent Prospector v2.0 (Mern, Corso, Burch, House & Caers, 2024) attacks the problem Chapter 7 flagged: optimal Bayesian design requires a prior, and in exploration the prior is a human story about ore genesis that the data will often falsify outright.
Their framing is precise: geologists state several competing hypotheses, each a different model of spatial variability. Frequently all of them turn out wrong. Planning acquisition under a wrong prior is worse than useless — you efficiently reduce uncertainty about a variable that does not describe reality. So v2.0 (a) plans optimally across multiple hypotheses simultaneously, and (b) equips the agent with a method to detect early that the stated hypotheses are all incorrect, so the campaign can stop before more money goes into a dead model.
This is the cleanest applied example of the distinction you will find. Aleatoric uncertainty is variability within a hypothesis — grade varies from place to place; more drilling reduces it. Epistemic uncertainty is uncertainty about which hypothesis holds; more drilling under the wrong hypothesis does not reduce it and may not even reveal the error. Systems that report a single posterior variance conflate the two and will confidently mislead. Detecting model misspecification early is worth more than sharpening estimates under a model that is wrong.
Two details give v2.0 unusual weight. The author list includes Kurt House and Damian Burch of KoBold Metals — this is not purely academic. And the abstract states the approach was tested on a sediment-hosted copper deposit and that the algorithm "aided in the characterization of an ultra high-grade deposit in Zambia in 2023." That is a decision-theoretic AI agent contributing to a real tier-one copper story, which is about as strong a demonstration as this field currently has.
Drill to falsify, not to fill a grid
The philosophical shift underneath all of this, and the one Stanford's Mineral-X group pushes hardest: conventional practice drills a grid to estimate grade. Decision-theoretic practice drills to discriminate between hypotheses — put the hole where the competing models disagree most, because that is where an observation carries the most information about which world you are in.
Jef Caers has argued publicly that this kind of planning could cut exploration drilling by around a factor of five. Treat that as a directional claim from an advocate rather than a measured industry-wide result — but the direction is right, and the mechanism is exactly the one above.
A POMDP planner needs a generative model of the subsurface it can sample from thousands of times per decision. That model is the geostatistical simulation machinery of Chapter 8, conditioned on data whose integrity is the subject of Chapter 6. The planner is only as good as the belief it plans against — which is why the unglamorous chapters are load-bearing and not preamble.
How you would actually build one
- Generative model. A sampler over subsurface realisations consistent with current data — SGS, a process-based geological simulator, or increasingly a learned generative model conditioned on observations.
- Belief representation. Particle filter over realisations. Watch for particle depletion; this is the failure mode the v1.0 paper reports in its hardest case.
- Forward/observation model. Given a realisation and a proposed hole, simulate the assays that hole would return — including the noise and the QA/QC failure modes from Chapter 6.
- Reward. Real drilling costs, real metal prices, real processing recovery. Rewards are where domain economics enter, and getting them roughly right matters far more than getting the solver exactly right.
- Solver. POMCPOW or a modern successor. Progressive widening on both actions and observations if your action space is continuous locations rather than a grid.
- Evaluate against baselines that geologists respect: a fixed grid, and an experienced geologist's actual programme. Beating a grid is table stakes; beating a good geologist is the result worth publishing.
The obvious frontier is replacing hand-built generative models and particle filters with learned ones — conditional deep generative models for belief-state planning are an active research direction, and they attack precisely the particle-degeneracy bottleneck that limits v1.0. If you want a thesis-scale problem with real economic stakes, this is it.
Sources for this chapter
- Mern & Caers, "The Intelligent Prospector v1.0", Geoscientific Model Development 16, 289–313 (2023) — open access. Code at sisl/MineralExploration (MIT).
- Mern, Corso, Burch, House & Caers, "Intelligent prospector v2.0: exploration drill planning under epistemic model uncertainty", arXiv:2410.10610 (2024). Multiple hypotheses and early falsification detection.
- Kaelbling, Littman & Cassandra, "Planning and acting in partially observable stochastic domains", Artificial Intelligence 101 (1998). The canonical POMDP formulation.
- Sunberg & Kochenderfer, "Online algorithms for POMDPs with continuous state, action, and observation spaces", ICAPS 2018. The POMCPOW paper — where progressive widening on observations comes from, and why you need it. Preceded by Silver & Veness, "Monte-Carlo planning in large POMDPs", NeurIPS 2010 (POMCP).
- Kochenderfer, Wheeler & Wray, Algorithms for Decision Making (MIT Press, 2022) — free PDF. Belief states, POMDPs, online planning and value of information with clean pseudocode.
- Howard, "Information Value Theory", IEEE Transactions on Systems Science and Cybernetics 2 (1966). The origin of VOI, and still the clearest statement of why information is only worth what it changes.
- Eidsvik, Mukerji & Bhattacharjya, Value of Information in the Earth Sciences (Cambridge, 2015). Book-length treatment of VOI for exactly this class of spatial decision problem. The most directly applicable reference in this list.
- Chaloner & Verdinelli, "Bayesian experimental design: a review", Statistical Science 10 (1995). The statistical parent of the whole approach.