Multimodal registration#

A simple example to register two images acquired with different modalities

import matplotlib as mpl
import matplotlib.pyplot as plt
import numpy
import spam.datasets
import spam.deformation

import spam.DIC
import spam.helpers
Traceback (most recent call last):
  File "/home/debian/spam/examples/DIC/plot_multiModalRegistration.py", line 11, in <module>
    import spam.datasets
  File "/home/debian/spam/src/spam/datasets.py", line 1, in <module>
    import pkg_resources as pkg
ModuleNotFoundError: No module named 'pkg_resources'

Load the two images#

xr = spam.datasets.loadConcreteXr().astype("<f4")
ne = spam.datasets.loadConcreteNe().astype("<f4")

#
#
# plt.figure()
# plt.imshow(xr[halfSlice, :, :])
# plt.figure()
# plt.imshow(ne[halfSlice, :, :])

Preparation of the images#

We start to crop the images to:
  1. select a region of interest: crop

  2. keep some margin to feed the transformation: margin

cropRatio = 0.1
crop = (
    slice(int(cropRatio * xr.shape[0]), int((1 - cropRatio) * xr.shape[0])),
    slice(int(cropRatio * xr.shape[1]), int((1 - cropRatio) * xr.shape[1])),
    slice(int(cropRatio * xr.shape[2]), int((1 - cropRatio) * xr.shape[2])),
)
cropPx = int(cropRatio * numpy.mean(xr.shape[0]))
marginRatio = 0.1
marginPx = int(marginRatio * numpy.mean(xr.shape[0]))
cropWithMargin = (
    slice(
        int((cropRatio + marginRatio) * xr.shape[0]),
        int((1 - (cropRatio + marginRatio)) * xr.shape[0]),
    ),
    slice(
        int((cropRatio + marginRatio) * xr.shape[1]),
        int((1 - (cropRatio + marginRatio)) * xr.shape[1]),
    ),
    slice(
        int((cropRatio + marginRatio) * xr.shape[2]),
        int((1 - (cropRatio + marginRatio)) * xr.shape[2]),
    ),
)

We rescale the two images between 0 and the number of bins int the joint histogram. For sake of efficiency they have to be saved as 8bits integers (<u1).

bins = 128
xrMin = xr.min()
xrMax = xr.max()
neMax = ne.max()
neMin = ne.min()
xr = numpy.array(bins * (xr - xrMin) / (xrMax - xrMin)).astype("<u1")
ne = numpy.array(bins * (ne - neMin) / (neMax - neMin)).astype("<u1")

To see the spatial incoherency we can plot them both on the same image with a checkerbord pattern, where two adjacent squares represents the two images.

halfSlice = xr.shape[0] // 2
# checker = spam.helpers.checkerBoard(xr[halfSlice], ne[halfSlice], n=3)
# plt.figure()
# plt.imshow(checker, cmap="Greys")
# plt.colorbar()

We apply and initial guess transformation.

PhiGuess = spam.deformation.computePhi({"t": [0.0, 0.0, 0.0], "r": [15.0, 0.0, 0.0]})
tmp = spam.deformation.decomposePhi(PhiGuess)
neTmp = spam.DIC.applyPhi(ne.copy(), Phi=PhiGuess).astype("<u1")
print("Translations: {:.4f}, {:.4f}, {:.4f}".format(*tmp["t"]))
print("Rotations   : {:.4f}, {:.4f}, {:.4f}".format(*tmp["r"]))

Registration algorithm#

We first get the parameters of the fitted gaussians and the joint histogram. x axis corresponds to the first image grey levels. y axis corresponds to the second image grey levels. The Gaussian parameters are parameters of the two ellipsoids that fit the two peaks.

print("STEP 1: Get gaussian parameters")
nPhases = 2
gaussianParameters, jointHistogram = spam.DIC.gaussianMixtureParameters(xr[cropWithMargin], neTmp[cropWithMargin], BINS=bins, NPHASES=nPhases)
plt.figure()
tmp = jointHistogram.copy()
tmp[jointHistogram <= 0] = numpy.nan
tmp = numpy.log(tmp)
plt.imshow(tmp.T, origin="lower", extent=[0.0, bins, 0.0, bins])
plt.xlabel("x-ray grey levels")
plt.ylabel("neutron grey levels")
plt.colorbar()
for gp in gaussianParameters:
    plt.plot(gp[1], gp[2], "b*")

Then we create the phase diagram based on the joint histogram. Each peak corresponds to a phase (1 and 2). The grey background (points to far away from a peak) is ignored (0).

print("STEP 2: Create phase repartition")
phaseDiagram, actualVoxelCoverage = spam.DIC.phaseDiagram(gaussianParameters, jointHistogram, voxelCoverage=0.99, BINS=bins)
plt.figure()
plt.imshow(
    phaseDiagram.T,
    origin="lower",
    extent=[0.0, bins, 0.0, bins],
    vmin=-0.5,
    vmax=nPhases + 0.5,
    cmap=mpl.cm.get_cmap("Set1_r", nPhases + 1),
)
plt.xlabel("x-ray grey levels")
plt.ylabel("neutron grey levels")
plt.colorbar(ticks=numpy.arange(0, nPhases + 1))
for gp in gaussianParameters:
    plt.plot(gp[1], gp[2], "b*")

And and we use both Gaussian parameters and phase diagram as an input of the registration algorithm

print("STEP 3: Registration")
registration = spam.DIC.multimodalRegistration(
    xr,
    ne,
    phaseDiagram,
    gaussianParameters,
    BINS=bins,
    PhiInit=PhiGuess,
    verbose=True,
    margin=marginPx,
    maxIterations=50,
    deltaPhiMin=0.005,
)

Final transformation#

We can now apply the final transformation

neReg = spam.DIC.applyPhi(ne, Phi=registration["Phi"])
print("Translations: {:.4f}, {:.4f}, {:.4f}".format(*registration["transformation"]["t"]))
print("Rotations   : {:.4f}, {:.4f}, {:.4f}".format(*registration["transformation"]["r"]))

And check the validity of the result with a checkerboard pattern mixing the two images

checker = spam.helpers.checkerBoard(xr[halfSlice], neReg[halfSlice], n=3)
plt.figure()
plt.imshow(checker, cmap="Greys")
plt.colorbar()

From the phase diagram a segemntation can also directly be obtained We can check that phase 1 corresponds to the mortar matrix and phase 2 to the aggregates

phaseField = registration["phaseField"]
plt.figure()
plt.imshow(
    phaseField[halfSlice, :, :],
    vmin=-0.5,
    vmax=nPhases + 0.5,
    cmap=mpl.cm.get_cmap("Set1_r", nPhases + 1),
)
plt.colorbar(ticks=numpy.arange(0, nPhases + 1))
plt.show()

Total running time of the script: (0 minutes 0.003 seconds)

Gallery generated by Sphinx-Gallery