Energy Estimation

Overview

In variational quantum algorithms, energy estimation refers to the process of estimating the expectation value of the Hamiltonian (or another observable) with respect to a given quantum state prepared on the quantum computer. This is a crucial step in any variational quantum eigensolver (VQE), where the goal is to find the ground state energy of a quantum system. In Kvantify Qrunch, energy estimation is performed by estimators (see Create an Estimator). Unfortunately, the energy cannot simply be read out directly from the quantum state, and different strategies must instead be employed. In Kvantify Qrunch, the estimation strategy is handled by two components called groupers and shot_distributors. General for all strategies is that the Hamiltonian is written as

\[H = \sum_i H_i\]

where \(H_i\) can all individually be measured directly on the quantum computer. Each \(H_i\) is measured a number of times (shots) \(s_i\) with a total number of measurements or shots \(S = \sum_i s_i\). The resulting estimate and variance of the energy are then given by:

\[\begin{split}\langle H \rangle &= \sum_i \langle H_i \rangle \\ \mathrm{Var}(H) &= \sum_{i,\, s_i \neq 0} \frac{\mathrm{Var}(H_i)}{s_i}\end{split}\]

The grouper determines how the Hamiltonian is decomposed into measurable terms \(H_i\) while ideally minimizing \(\mathrm{Var}(H_i)\). The shot distributor determines how the total number of shots \(S\) is distributed among the terms \(H_i\) to minimize the overall variance of the energy estimate. Doing both of these optimally is a hard problem. In Kvantify Qrunch, we target chemically relevant Hamiltonians where the problem size is too large for exact optimization, and we instead rely on heuristics to find good groupings and shot distributions.

Pauli Grouping

Most grouping strategies, including all strategies in Kvantify Qrunch, rely on grouping Pauli strings. Here the Hamiltonian is expressed as a sum of Pauli strings \(H = \sum_i c_i P_i\) where \(P_i\) are tensor products of Pauli operators. The grouping strategy partitions the Pauli strings into groups \(G_1, G_2, \ldots, G_m\) such that all strings within each group can be measured simultaneously. A Pauli string may appear in more than one group, provided its coefficient is split across those groups. Writing \(c_{k,i}\) for the contribution of string \(P_k\) to group \(G_i\), the only requirement is that the contributions sum to the original coefficient:

\[\sum_{i:\, k \in G_i} c_{k,i} = c_k \qquad \forall\, k\]

Each group then defines a measurable fragment:

\[H_i = \sum_{k \in G_i} c_{k,i} P_k\]

The energy estimate and variance from the previous section then specialise to:

\[\begin{split}\langle H \rangle &= \sum_i \sum_{k \in G_i} c_{k,i} \langle P_k \rangle \\ \mathrm{Var}(H) &= \sum_{i,\, s_i \neq 0} \frac{\mathrm{Var}(H_i)}{s_i} \\ &= \sum_{i,\, s_i \neq 0} \frac{1}{s_i} \sum_{k, l \in G_i} c_{k,i}\, c_{l,i} \mathrm{Cov}(P_k, P_l)\end{split}\]

where \(\mathrm{Cov}(P_k, P_l) = \langle P_k P_l \rangle - \langle P_k \rangle \langle P_l \rangle\). Because the Pauli strings within a group commute, all covariance terms are in principle accessible from the same measurement basis. As grouping occurs before the measurement, the covariance terms are not known at the time of grouping, but the final error is supplied with the estimated energy.

With this brief introduction to energy estimation, we can more easily describe the grouping and shot distribution strategies in Kvantify Qrunch.

Grouping strategies

Identity grouper

The identity grouper (identity()) is the trivial baseline: every Pauli string \(P_k\) forms its own group \(G_k = \{P_k\}\). No two strings are ever measured together, so each group contains exactly one term and the number of groups equals the number of Pauli strings in the Hamiltonian. This is simple and unbiased, but wasteful. It ignores all opportunities to measure compatible strings simultaneously and typically requires many more shots to achieve a given precision than the other groupers.

Qubit-wise commuting grouper

The qubit-wise grouper (qubit_wise()) groups Pauli strings that are qubit-wise commuting: two strings are compatible if, on every qubit, their single-qubit Pauli operators commute (i.e. they are either equal or at least one is the identity). This is a stricter compatibility condition than full commutativity, but it has the advantage that the joint measurement basis can always be constructed as a simple tensor product of single-qubit rotations which are cheap.

The grouper uses a greedy distance-based heuristic that assigns each Pauli string to the existing group whose members are most compatible with it, opening a new group only when no compatible group exists. This strategy is fast and scales well.

Shadow grouper

The shadow grouper (shadow()) implements the Shadow Grouping algorithm described in Gresch et al., Nature Communications (2024) and SF Jørgensen, RE Barfknecht, P Ettenhuber, NT Zinner arXiv preprint arXiv:2603.20029, 2026. Instead of forming groups purely by commutativity, it iteratively builds measurement groups while re-weighting all Pauli strings according to how well they are already covered by the groups formed so far. The re-weighting uses the Bernstein bound as a priority function, so that strings contributing most to the residual variance are placed into new groups first.

The number of groups is predefined and can be specified. This makes the shadow grouper predictable in the number of groups and allows the grouper to better and more precisely exploit the available measurement budget.

BEAST grouper

The BEAST grouper (beast()) is a specialised grouper designed for the BEAST-VQE chemistry algorithm. It exploits the specific structure of the BEAST ansatz circuit to construct a small, fixed set of measurement groups that exactly covers all Pauli strings in a molecular Hamiltonian optimally.

Shot distribution strategies

Given a fixed grouping \(G_1, \ldots, G_m\), the shot distributor decides how to split the total shot budget \(S\) across the groups. The optimal allocation is \(s_i \propto \sqrt{\mathrm{Var}(H_i)}\), but the variances are not known before measurement. The strategies below use tractable proxies.

Homogeneous distributor

The homogeneous distributor (homogeneous()) allocates shots equally across all groups:

\[s_i = \frac{S}{m}\]

This is the simplest strategy and requires no information about the Hamiltonian beyond the number of groups. It is a reasonable default when group variances are expected to be similar, but can be far from optimal when group sizes or coefficient magnitudes differ substantially.

Group-size distributor

The group-size distributor (group_size()) weights each group by the number of Pauli strings it contains:

\[s_i \propto |G_i|\]

Larger groups, which tend to have more terms and therefore higher variance, receive proportionally more shots. This is a cheap heuristic that only requires counting terms and does not need coefficient information.

L2-norm distributor

The L2-norm distributor (l2_norm()) weights each group by the \(\ell^2\) norm of its original (un-split) coefficients:

\[s_i \propto \sqrt{\sum_{k \in G_i} c_k^2}\]

Under the assumption that all Pauli strings are uncorrelated, each has variance of order 1, and no Pauli strings appear in more than one group, this is the analytically optimal allocation. These assumptions are not valid, but covariances are not known at the time of shot distribution, and this can serve as a useful heuristic.

Minimal posterior variance distributor

The minimal posterior variance distributor (minimal_posterior_variance()) finds the shot allocation that minimizes the total variance directly, via constrained optimisation. Assuming all Pauli strings are uncorrelated and each has unit variance, the variance of \(H\) due to string \(P_k\) is \(c_k^2\) divided by the total number of shots spent measuring it across all groups. The objective is therefore:

\[\begin{split}\min_{\{s_i\}} \quad & \sum_k \frac{c_k^2}{\displaystyle\sum_{i:\, k \in G_i} s_i} \\[6pt] \text{subject to} \quad & \sum_i s_i = S, \quad s_i \geq 0\end{split}\]

For non-overlapping groups, the minimization converges to the same allocation as the L2-norm distributor. For overlapping groups, the optimal allocation is more complex and can differ significantly from L2.

Recommendations

The best grouping and shot distribution strategy depends on the specific Hamiltonian, the measurement budget, and the desired precision. In general, the shadow grouper with the L2-norm distributor is a strong default choice that performs well across a wide range of scenarios. The qubit-wise grouper with the L2-norm distributor is faster and will scale to larger systems. The excitation gate only supports the qubit-wise grouper with a constant number of shots per group.

Summary

Energy estimation decomposes a Hamiltonian into measurable fragments via a grouper and then distributes a finite shot budget across those fragments via a shot distributor. The grouper controls how many Pauli strings can be measured simultaneously, and therefore, how large and how few the fragments are; the shot distributor controls how precisely each fragment is estimated. Choosing both well is essential for achieving chemical accuracy within a practical measurement budget on near-term quantum hardware.