Integrating animation in DIPY to help in intrusive quality assessment by domain experts

       

Hey,

I will start with the good news, the image registration workflow is in the process of being merged. 

The code went under several iterations of improvement and optimization (after the PR was created), thanks to the community developer for sharing their useful comments.

PR Link: Image Registration Workflow

Wonder, what the image is saying ? read on to learn more about it.

For the past few days, I have been brainstorming and experimenting with the following objectives,

The Objectives

1) Adding an intuitive component to help in the assessment of the quality of registered images. 

Please see my earlier posts about visualizing the results of Image Registration (due to some reason the http://blog.python.org is not reachable since yesterday, so you may have to try a few attempts to see my older posts:)

2) Trying different options about what possibly is the optimal and platform independent way of communicating the results for quality assessment.

Adding an intuitive Visualization Component

I have already discussed about the usage and application of 'Mosaics' in assessing the visual quality of the registered data. A mosaic presents a holistic view of the registered image by displaying all the slices (from a specific plane) parallelly so that the change in the voxels can be observed in a sequential manner.

What are mosaics? and their limitations

Mosaics can be handy when a user wants to view the slices and not the entire volume. This provides for more fine-grained control over the quantity of information presented.

However, there is one thing lacking in the mosaic based approach (not that it is not useful), in case of large volumes the number of slices can be overwhelming.  A large number of slices present difficulty not only in creating the mosaic but also makes it hard for the user to understand the data/results easily. Just imagine looking at hundreds of slices while trying to track the changes from one to the other.

Old Commit Link: Mosaics

Animations to the rescue

So, I needed another method to go with the mosaics and complement the quality assessment arsenal in DIPY. After discussing with the project mentors, it was decided to use an animation (preferably a GIF) to show the changes in the moving image as the registration goes on.

I started experimenting in a separate branch for creating the animations to visually convey the quality of the registered data.

Commit Link: Adding the animation 

After multiple rounds of debugging and data specific and exploration (adjusting the code for the specific number of slices and volumes), I finally was able to create an animation for showing the progress of image registration. 

Animating the 'Center of Mass' based Image Registration

The static image (in RED) and the moving image (in GREEN) shown to be overlapping
. The dots of yellow indicates the areas where the overlap is good.

Animating the 'Translation' based Registration

The animation below shows the slices (from both the static and the moving images) in the axial plane overlapping with each other. This particular image is a bit blurry due to the enlarged size, but I am working on to improve the quality of the results further.

Translation based registration using Mutual Information Metric: The static image (in RED) and the
moving image (in GREEN) shown to be overlapping. 

Animating the 'Rigid body' based registration

The following animations show the effect of rigid body based registration on the moving image when the rigid registration is done using the Mutual Information metric.

Rigid registration using Mutual Information Metric: The static image (in RED) and the
moving image (in GREEN) shown to be overlapping. The portion in yellow are the ones
that aligned well.

Animating the 'Affine' registration

The best results were obtained in terms of the affine registration as is being shown in the following animation. The portions of 'RED' are most superimposed by the GREEN in the AFFINE registration (relatively better when compared to the other modes).

Full Affine registration using Mutual Information Metric: The static image (in RED) and the
moving image (in GREEN) shown to be overlapping. 

Correcting the orientation

By interpolating the moving image and normalizing it (for the color range of a GIF), the animations were reproduced in the correct orientation. 

The MP4 video below shows the animation in the correct orientation (this is center of mass based for demonstration but the similar results can be produced for other modes too.)

Commit Link: Visualising with the correct orientation

The animation for 'COM' registration 

presented in the correct orientation.

The Code

Since the initial experiments with the animations yielded satisfactory output so I started adding the changes in the branch from my local repository and also making the code more generalized.

Commit Link: Complete Code Affine Visualizations

These changes are still under testing and will be merged with the experimental branch of DIPY for large data (nipy/dipy_qa).

That's about it for now. More details will follow about the various design decisions that went into the making of the DIPY animated visualizations and about developing new workflows with DIPY.


Adios!

Parichit.