For a bit of early morning compression fun why not go to

tinyurl.com/IMNN-AICosmo

or even better...

pip install IMNN jupyter matplotlib
git clone https://github.com/tomcharnock/IMNN-LFI_Taskforce.git (branch AICosmo)
cd IMNN-LFI_Taskforce
jupyter notebook

... then you'll have the IMNN on your own machine - you lucky people

Lossless data compression for cosmological surveys (and everything else!)

Tom Charnock
Institut d'Astrophysique de Paris

Guilhem Lavaux, Benjamin D. Wandelt
Notebook: presentations.charnock.fr/IMNN/AICosmo19

Sorbonne Université ANR IAP CNRS Aquila

How can we trust predicted parameters?

Do we need to worry about the exactness of simulations?

Can we bring scientific (statistical) rigor to our results?

YES!

Information maximising neural networks

Find the function which gives summaries which are Gaussianly distributed

$$-2\ln\mathcal{L}({\bf x}|\boldsymbol{\theta}) = ({\bf x}-\boldsymbol{\mu}_\mathscr{f})^T{\bf C}^{-1}_\mathscr{f}({\bf x}-\boldsymbol{\mu}_\mathscr{f})$$

Optimise to maximise the
Fisher information!

$$\textsf{Loss} = -\ln\det{\bf F}_{\alpha\beta} + \lambda||{\bf C}_\mathscr{f}-\mathbb{I}||_2$$
$${\bf F}_{\alpha\beta}=\textrm{Tr}\left[\boldsymbol{\mu}_\mathscr{f},_\alpha^T{\bf C}_\mathscr{f}^{-1}\boldsymbol{\mu}_\mathscr{f},_\beta\right]$$

What do we need?

A neural network (obvs.)

Some simulations ${\bf d}^i$

  • a single fiducial parameter value!
  • $N$ enough for ${\bf C}_\mathscr{f}$ - (1000 simulations)

Some derivatives of the simulations ${\bf d},_{\alpha}^i$

Train it!

Pass all simulations through network and calculate covariance of outputs, ${\bf C}_\mathscr{f}$

Calculate derivative of outputs with respect to parameters and find their mean

$$\frac{\partial\mu_\mathscr{f}}{\partial\theta_\alpha} = \frac{1}{n_\textrm{p}}\sum_{i=1}^{n_\textrm{p}}\frac{\partial{\bf x}_i}{\partial{\bf d}_i}\frac{\partial{\bf d}_i}{\partial\theta_\alpha}$$

Calculate the Fisher information and backpropagate

So what do we actually end up with?

Outputs that are sufficient statistics

they saturate the Cramér-Rao bound!

and unbiased parameter estimates

$$\hat{\boldsymbol{\theta}}_\alpha = \boldsymbol{\theta}^\textrm{fid}_\alpha + {\bf F}^{-1}_{\alpha\beta}{\bf C}_\mathscr{f}^{-1}\boldsymbol{\mu}_{\mathscr{f},\beta}({\bf x} - \boldsymbol{\mu}_\mathscr{f})$$

And how should we use the summaries?

Within a likelihood-free inference framework

We work in the space of data ${\bf d}$ and model parameters $\boldsymbol{\theta}$

Posterior is a slice through this space at some given data

ABC posterior

Simulate the data and accept simulations close to the true data

DELFI

Learn the joint space of ${\bf d}$ and $\boldsymbol{\theta}$ with neural density estimators

Reduce the dimensionality with our summaries

We pass all simulations through the network to get summaries.

We pass the real data through the network to get its summary.

Measure how far the summaries of the simulations are from the summary of the real data

The simulations can be bad!

Network becomes part of the data aquisition

the data products are now the summaries



If simulations are bad, the summaries are less informative

BUT NOT BIASED!

Finish the example

1000 noisy CMB E-mode realisations at $\tau=0.055$

50 numerical derivatives with respect to $\tau$

A network to best exploit the data

Fast convolutions on the sphere to learn real space kernels in the correct topology

An observed CMB E-mode map at $\tau=0.65$ (not fiducial)

Keep eyes peeled for application to Planck data!

Information maximising neural networks

We can massively compress our data to optimal summaries

Using LFI we don't have to worry as much about the simulations

Its exceptionally cheap!

Discussion topics

We know physics - how much should we insert into our networks?

  • massive reduction in training data
  • massive increase in interpretability
  • orders of magnitude fewer paramaters

Up to what point can we trust regression to parameter values?

Convolutions for translationally invariant data
(images or local signals)

We can build the network to work on summaries with unknown likelihoods

Finding that one extra summary...

We can combine the IMNN with our known and loved summaries to extract just that little bit extra information

Of course we can still do score compression

Inferring the ionisation rate from quasar absorption spectra

Use fully connected neural network

Couple it with DELFI to get another convincing posterior!

Multi-dimensional cosmological parameter inference from tomographic cosmic sheap maps of a Euclid-like survey

Summarise the maps via the power spectrum and compress that with the IMNN

Use DELFI for inference

In [ ]: