09 - Alignment and Atlases: Part 1 of 2
Date Posted:
August 7, 2018
Date Recorded:
May 29, 2018
Speaker(s):
Daniel Glen, NIMH
All Captioned Videos AFNI Training Bootcamp
Description:
Daniel Glen, NIMH
Related documents:
For more information and course materials, please visit the workshop website: http://cbmm.mit.edu/afni
We recommend viewing the videos at 1920 x 1080 (Full HD) resolution for the best experience. A lot of code and type is displayed that may be hard to read otherwise.
DANIEL GLEN: Today I'm going to be talking about alignment and Atlas in the AFNI software package. So I'm going to start off talking about what we mean by alignment and why do we want to do it. So what we want to do with alignment is we're trying to bring two objects into some kind of correspondence. And so this idea of correspondence is where we have an xyz location in one, a correspondence to an xyz location in another data set.
And so we're going to do that for lots of different reasons. We're going to do it across time, while we do motion correction. We're going to do it between our EPI data. It has our functional signal in there to our anatomical data.
We're going to do it made for longitudinal studies. We can compare data maybe across scanners. And we're going to put all of our data into some kind of standard space so we can compare all of our subjects together, or even with subjects from other studies.
And these are the software programs that we have available to do these kinds of things. We'll spend some time talking about 3DVolreg. So that's for motion correction. This does a rigid body correction, alignment to some sort of base data set.
We can also use 3DWarp Drive and add auto teller rack. If we're aligning data to a template, those do affine transformations. We can use 3dQwarp and auto warp and now @SSwarper for doing alignment non-linearly to a template. We'll talk about what that means. @SSwarper is a very new package program. And that one also includes skull stripping as part of its procedure.
We use 3DAllineate, which has a variety of cost functions to align data across modalities. And then we have align_epi_anat.py that calls 3DAllineate. And it's a general alignment script for them. Most of you will probably not be using any of these software programs by themselves. You'll probably use afni_proc.py because it will call the appropriate programs with the options for you.
And we have other things for aligning slices and even for images. We have a plug-in in the AFNI GUI. And yesterday I showed you where the plugins were hiding. And there's a nudge plug-in. And the nudge plug-in lets you manually move one data set over another.
Everyone thinks that this ought to be a kind of gold standard, if you just align it by yourself that it would be good enough. But it's almost impossible to do this manually well. The computer has infinite patience and will try every possibility. But we don't.
And finding just the right combinations of rotations and translations is really hard for us. So it's good for getting a rough approximation. But it's not good for actually great alignment.
If you know that you're your data is rotated by a certain amount, or translated, you could use 3DRotate. We have also 3DWarp. You would use 3DWarp for matching obliquities between data sets or applying a specific transformation.
One of the first procedures that we do with data sets that comes come from multiple scanners is we will align the centers of the data set. So to give it a good starting point, we want to put the one data set over the other. And so we can use add a line centers or even 3dCM to align the centers of mass.
If you have specific regions that you want to align, specific markers, you can use 3DTagalign and mark off each point that corresponds across multiple data sets. Then it will compute an affine transformation to match these data sets together. So that's just the kind of overview of some of these tools we will go into, some detail for some of these, OK.
So how are we doing it? There is a little description here of what we're doing with these transformations. But I'm trained as an engineer and I like flow charts.
So I prefer to see this more diagrammatically. So this is what we're doing in almost all of our alignment programs. We're taking some data set that we'll do, some sort of pre-processing. And we'll mask that data set, compute a wait for all the voxels. And then we'll calculate some kind of initial cost for it.
And if we use this option called topass, which is basically a code for saying, let's guess a lot of different options, a lot of different parameters, we'll do all those guesses separately and then see which guesses work out the best. And then we tweak the parameters in the process called optimization. And with each tweak, we transform our data set, our source data set to match our base data set.
And we interpolate into the grid of the base data set. We measure the cost of that alignment, see how well those two data sets match. And if the cost is less than some tolerance, then we save the final data. Otherwise we repeat this loop. It's pretty simple, all right.
So what do we mean by the cost? Well, if we've got two images and we want to calculate some cost of alignment, one of the simplest ways is to just look at the difference. It just says, how different are these two intensities? So let's say my hands are two images, and I want to see if they're aligned. Well, they're aligned when they're roughly about over each other, right?
And so we'll take the intensity of one minus the intensity of the other. But we'll do this in a little bit more sophisticated way. So we'll square the difference. Because we want to look. It doesn't matter which direction it is. So we just want to get that difference. We'll square it.
And then we we will apply some sort of weighting system. So higher intensity voxels will contribute more to that cost than lower intensities. That's one way to calculate a weight for it. And that's the overall cost function is weighted least squares.
And there are lots of different ways to calculate cost functions. This way is good if you have two data sets that are very similar, they have similar intensities. So both of them, T1, SPGR area, or both of them are EPI data sets. But it's not good if we are matching two different kinds of data sets. In that case, that system doesn't work. And we'll talk about alternatives to that.
And when we put the data onto the grid, we've got choices on how we interpolate our data too. So there is 4A interpolations, linear interpolations. This data has to be moved. So we have to find our source voxel and put it into the new location.
What we do is we pull from a set of sources. So we say the location in the original data set that goes into our final grid is going to be at this location. And we'll get an interpolation of the voxels around it. We can do it linear, cubic, quintic, heptic, all different kinds of ways, OK.
So for the weighted least squares with method, we use that method with 3DVolreg and with 3DWarp Drive. And we're going to do that with 3DVolreg where you are doing that for motion correction. And yesterday we looked at how we see motion in the graphs, in the images, how to find it. And this is an example of what motion actually looks like. Yesterday we looked at a kind of synthesized sample.
And motion in an EPI data set looks like a striped pattern very often. As the person moves out of the image where they were originally acquired, the intensities are changing. So things aren't in the same place. And when we look at the results, we'll see rings of activation and deactivation.
So around the ventricles, you'll see high activation and low activation. You'll see it around the head. So these are clues that you still have some motion in your analysis.
The program 3DVolreg will do this motion correction for you. And it works with six parameters. So it has three translations, shifts in the x,y and z, along those axes. And then rotations about each of those axes.
And then we have the program 3DWarp Drive which will extend this. And this has got 12 parameters. So besides the translations and rotations, we've added three scaling factors and three shearing factors. So scaling is just an enlargement, or shrinking around each of the axes. And the shears are kind of sliding.
So those are those 12 parameters. These make up the 12 parameters of what's called an affine transformation. And these are applied with just a single matrix multiplication. And it's done in the same way for every voxel in a data set. All voxels are treated equivalently. This has advantages and disadvantages. And we'll see some of these, OK.
So how do you use 3DVolreg? This is an example command. 3DVolreg, you give it the base sub-brick that you're using, the kind of interpolation that you want to use, whether you want to have any zero padding, additional zero padding around the data set.
AUDIENCE: [INAUDIBLE]
DANIEL GLEN: So zero padding is adding slices of zeros around a data set to allow for rotation. So if something is tilted a lot, you can add some more zeros to accommodate for that rotation out of the data set. Mostly, you don't have to worry about that, but in some cases you will.
The prefix, this is standard across. Almost all AFNI programs have a prefix. And this describes the output. So our output data set will be called Fred 1 EPI VR plus a ridge in this case.
If you wanted a nifty data set, you would add, after the VR you could put dot knee or dot knee dot gz. And it would be saved as a nifty data set. And then you could bring it into some other software program, OK?
And then here, we're saving our motion parameters as a 1D file. And we're going to use that in our regression as a regressor of no interest here. And this is our input data set. So this is a pretty typical format for an AFNI command in general.
So lots and lots of options. There are a lot more options that I haven't talked about. Those are available through the Help of course, OK. So in general, we don't use heptic. I think the default inside 3DVolreg is fourier. And we don't actually recommend that.
So generally, cubic or quintic is what we recommend. Fourier interpolation generally has a ringing artifact. So we try to avoid using that. But that's something to be aware of there, OK. So we have some more examples there.
Now, if you have multiple runs of data sets, then you could align your second run to your first run, and maybe that's the sub-brick number four, the fifth sub-brick in your data set. And this is that you'd continue on through that. Now afni_proc will do all this for you. So you don't generally have to do this on your own anymore. But it's going to do something like this for you, OK?
And then you can plot the results. These parameters as fred2vrd file, you can plot that over here with 1DPlot. This takes this column of numbers and plots them out. And because you've said it came from 3DVolreg, it knows the labels. And it shows that this is your anterior-posterior translation, right-left translation, inferior-superior, and then the rotations about each of the axes. So y'all pitch and roll. This is from airplane terminology. And this is what those mean.
And so you can see by looking at the plots that you'll have peaks at times of motion. So here we have one around 160 seconds where the subject jerked their head up. And generally, you'll see this as a rotation on one of the axes. So around the right to the left is what we typically see, OK.
So we call it motion correction. But we don't actually fix anything. We're not fixing the motion. We're kind of mitigating the whole situation. We fix the data before and after the motion. And we tell you where the motion happened so that you can put that into your regression.
Because the data is gone during the time of the motion. Part of the subject has moved. They are not where they ought to be. We don't have the data that we want. So we just say, we're going to end up censoring that. And we're going to remove that from our data set.
And you have to consider this when you're designing your experiment, how your emotion will interact with your design of the study, the linear regression. So if you have people exercising, or signing, or gesturing in any way, with every stimulus, you have to consider that. Because that's going to be motion at the time that you're interested.
We did have one of our users, she had an experiment like that where three years later she decided she would finally get back to analyzing that. And with somebody signing at the time with each stimulus. And it was uninterpretable because you just have so much motion occurring for so long.
Now, if have a very, very short motion, and the bolt signal continues on for 15, 20 seconds, then you can extract the rest of it. You can say, I don't want to look at the beginning one, two, seconds, but I'll look at the rest. So it is possible.
But in this case, there was something like 10 seconds of motion. Difficult to do anything about it. It also says, look at your data as soon as you can get it. Try to analyze it as soon as possible. OK.
This is an example with a macaque where 3DVolreg didn't work. And so you can see here we have the original data set. And there is some motion going on. And it turns out, so the experiment was designed with a juice stimulus. And monkeys love juice.
So you give them juice. And they get out all excited. And of course, their jaws are moving too. And 3DVolreg doesn't know the difference between jaws and brain. It just looks at intensities.
And the jaws are very bright. And so it aligns, it takes into account the jaws. But if you mask off the jaws properly, then you can get a corrected image. So these are the curves at roughly the same boxes. So that's motion correction.
But if you want to align data from across two modalities-- so we'll consider anatomical data sets and EPI data sets, two different modalities, then you need another kind of cost function. So we have this program, 3DAllineate, which can compare data from different modalities. And it does include a least squares cost function, something like 3DVolreg and in 3DWarp Drive.
But it has a whole set of other ones. We have mutual information and normalized mutual information. These are the cost functions that a lot of other software programs use, this mutual information idea. And we have the Hellinger metric, which is the default in this program. We have correlation, various kinds of correlation.
And we found that these last two are the ones that we need the most often. So we used the local Pearson correlation, which is a simple correlation, but it's normalized by small neighborhoods around the brain. So we divide up our data set into 96 different volumes, and say each of these dodecahedrons inside the brain is correlated with the corresponding data set, the other data set, and calculate that correlation.
And what we do is we take the negative of the correlation. If they're negatively correlated, that's what we're looking for. And the reason we do that is that in EPI data, we are looking for bright CSF. So we're going to base our alignment on the CSF signal. So look for bright CSF matching dark CSF in the T1 weighted data set. So to get that opposite pattern, we're going to use that negative correlation.
And we wrap this into align EPI anat. This will call 3DAllineate for us. It creates a weighted EPI data set that has a maximum 90 percentile on your EPI data. It picks one particular sub-brick to use as your base and aligns the anatomical data set to that.
And if you ask for the reverse transformation, it will calculate the EPI to anatomical. And it will combine motion. It will call 3DVolreg for you and do your de-obliqing. It will match obliquities between the data set. And it will bring you to standard space if you provide it, an anatomical data set that's in standard space, OK.
So here, we've added an option called dash ed edge, which will show us the edges of our anatomical and our EPI data set. And we found that this was the best way to look for alignment between these two kinds of data sets, OK.
So this is the same data set we looked at yesterday in AFNI data 6 AFNI, the anatomical data set. And for years we looked at this, and we said they were acquired close together. We align to the closest EPI data set. And this was OK.
Visually, it looks about right. But when you start to look at the details, you see it's not quite right. So let's see if I can convince you of that, OK.
So this is our original data set. The EPI data has these cyan blue edges. And our anatomical data set has these purple edges. And where they overlap, the cyan and purple, we show it into a kind of magenta color.
And you can see that these edges, they are sort of aligned, not too great. But let's go down to the post image. And you can see, it's not perfect. But we have a lot more alignment. And you can see this even just with the overlay-underlay comparison that it's a lot sharper. So we found this is a good way to check for alignment is to look at that kind of contrast. Any questions?
AUDIENCE: Here, you're going to use say, affine transformations here. Is that necessarily relevant?
DANIEL GLEN: Well, these are affine transformations. So you're aligning between the EPI and that with an affine transformation. We don't use nonlinear between these two because they're roughly in the-- well, I'll get to-- there is a case where we do use nonlinear. But mostly we don't, OK.
Also besides the @ed_edge script, we have the @snapshot_volreg which will show you the edges as they are. It doesn't do an alignment for you. It just shows you the edges and creates a similar kind of display.
These are colored a little bit differently. But it creates a nice montage for you. And you can put this as part of your scripts as a kind of quality control. We have this little viewer program called AIV. So you can use that to what to look at your JPEGS and Pings and whatnot.
So yesterday in the AFNI interactive class, we looked at lots of different things for visualizing alignment. This is a really important part of alignment, trying to figure out if it is right. Because no matter what cost function we devise, your data may not agree with it. So you have to go in and look to see if it actually is aligned.
So for motion, looking at the graph across time, or even the image across time gives you a good idea of motion. We saw that you can have multiple controllers and look at the crosshairs for a very rough view, overlay opacity, the toggle of the overlay and the underlay.
I didn't show you one thing, these four, five, and six keys. And I think that's worth showing. I'm going to take a quick break out of this. You may want to do this too if you have your laptops ready. I'm in the AFNI data 6 AFNI directory. I'll start AFNI.
And I'll change the overlay to my EPI data, OK? And on any slider, on any image viewer click the four key. And you see that you get this kind of slider at the top. And you can drag that back and forth. OK?
So that's with the four key. The four key brings up that slider. If you hit any other number, I'll do the four again. You can turn that off. If you hit the five key, you can slide the image up and down. That's with the 5 key. OK? So turn that off with the 5 again.
Now the 6 key, this merges both the overlay and underlay into the underlay grayscale. So this is the six key. And so it's like a phantom EPI data set.
And you can do this, this is not just for EPI anatomical alignment. You can look at how well your data set matches a template. You can look at two days of scans against each other. So a lot of different uses for it, not just for EPI anatomical data sets.
Just to remind you, the E key, that E key, combine it with a lower opacity shows you a quick way to get edge, and edge display.
AUDIENCE: [INAUDIBLE]
DANIEL GLEN: That was the E key for edge enhancement. And that's a toggle. The O key turns on and off. And the U key is an underlay toggle.
AUDIENCE: My overlay is gray. The other one is still green. [INAUDIBLE]
DANIEL GLEN: So you may still have the six turned on. It's interesting how you can combine the six with the edges.
AUDIENCE: The first time I turned mine on, it also went green, [INAUDIBLE].
DANIEL GLEN: Yeah, so everything goes to just underlay when you use these 4, 5, 6, keys. Because it's a little bit confusing to add colors on top of this. It's made to just merge two images without color. Because they're supposed to be similar in some way. So this is with both the 6 and the E. That is allowable there.
AUDIENCE: Is there an edge for the overlay?
DANIEL GLEN: No, there's no edge for the overlay unless you use the @ed_edge script or the @snapshot_volreg. Those will show you edges in the overlay. But it's not built into the GUI. You have to run a script separately that creates edge data sets for you. All right.
AUDIENCE: With both overlay E, you should just get an edge, right? Only my 5 key did anything so. [INAUDIBLE]
DANIEL GLEN: There's also a checker board that you can get with the 3 in the number key. This is not great for EPI data sets. If you have two anatomical data sets, two data sets that you think should be exactly the same, then that's good. Otherwise, it's not. Across two data sets is just two different modalities. It's confusing.
So you can see we have a lot of different choices for how we can visualize two data sets and look for whether our data sets are aligned well or not.
Now, align EPI anat usually works between the EPI and data sets and the anatomical. And I say, usually, because it worked on 90% of the fcon 1000 data sets, which came from 25 sites around the world.
And it worked for 90% of them. It didn't work for a couple of them, This is where the data set contrast was not as good. And there is some reasons for that.
What we recommend is a few things, OK. Well, you can change your cost function. And actually, the one that works at best is not the original local Pearson correlation. Probably our most robust cost function is the LPC+ZZ cost function.
And what that does is it does a first pass with a combination of lots of cost functions, so adds a little bit of normalized mutual information, a little bit of least squares, and a lot of local Pearson correlation and finds a kind of combined cost function. And then from there, it does a final calculation on local Pearson correlation.
So LPC+ZZ seems to be a better option. But it's also good if you have worse contrast. If you have really terrible contrast-- because you remember that this is relying on CSF being different than the rest.
So CSF has to be bright on your EPI data set and distinguishable from the other tissues around it. If you don't have that, then LPC won't work well. So you need to try something else.
So you can try, normalized mutual information works well for that. LPA is a good cost function for that. So you can play with your cost functions if they start off far apart. So most of the time when you have anatomical and EPI data sets, the scanner will give it to you in correct coordinates.
If you get it from somebody else, it may have gone through a procedure. There are several pipelines someplace else that has lost that information. So you may not know, the coordinates may not be correct.
And so you may have to align centers first. Or for some reason they are far apart, or they are just plain wrong, we have big move, which does a 2 pass option. And then we have giant move which allows for a fairly large range of translation and rotation for up to a 45 degree rotation. And then we have ginormous move. Ginormous adds in a center alignment. So we can put the centers over each other before we start.
If you have a lot of edges that are much brighter than the rest, you can use the dash edge, which is a different procedure. And the LPA cost function works better there. If you have other questions like stroke or other kinds of lesions, you can ask us, we can give you scripts that have been developed specifically for that and some tips to what to do for that.
AUDIENCE: Why have a big giant in ginormous [INAUDIBLE].
DANIEL GLEN: Because most of the time the data doesn't need to be moved a lot. The coordinates are about right. There's just a very small amount of alignment that has to be done.
AUDIENCE: Time, it takes time to do all that.
DANIEL GLEN: It does make it slower. It does make it slower. But if you expand the range of parameters that you're looking at, it's more likely you can go wrong too. So most of the time, most people will not need to use any of these options.
When we introduced this, one of our users, Mike Beauchamp, he sent us this data set and said, your new alignment procedure doesn't work. Look at this. And we said, oh, well, we thought we tested out well. But it looks like there could have been something wrong. And he showed us this anatomical data set is over an EPI data set.
Now, as you have seen here, we generally do it the opposite way. So that was a clue that there may have been something, a different way of looking at it. You know, either we didn't take it into account or there's something wrong there.
So when you get this, this anatomical data set, it's been skull stripped. And it's sitting over our EPI data set. But if you look at the original anatomical data set, you can see that for this particular subject, there is a lot of CSF around the brain. And that's what we're seeing here.
So the alignment actually did work. Because we are aligning that CSF to that CSF. So in the end, the procedure was correct. And just by looking at the contrast of the CSF and the original data sets, you can see where things are, OK?
Some more special cases of alignment-- we have Helen Mayberg who used to be at Emory with Justin Rajendra who also used to be at Emory, and he's now working in our group. They were doing DBS, Deep Brain Stimulation, putting the electrodes into the brain. They needed to get these electrodes within a millimeter of their target location. And this is for treatment of depression.
They have pre-surgical MRI. They have CT. They have PET data. They have DTI data. And all these things had to be aligned together. So we used align EPI anat to do that with some tricks.
So to align CT to T1 data for instance, the T1 data, the skull in T1 data is just that thin black layer, a thin dark layer between the meninges and the skin. You can see a small layer that is the actual skull. And by reversing our T1 data set and then aligning it to the CT with a rigid body transformation, we could align these together. We used the same trick with ECoG data, which also has CT and T1 data sets. And we have this as part of the ALICE pipeline for dealing with ECoG data.
Some more unusual data sets-- this is manganese enhanced MRI. This is in rat brains. So we have-- manganese chloride is injected into the rat brain. We can see neuronal transfer over a day. So this is a kind of a day scan. And we have different hours that we take the scans. And it enhances as it goes through the tissue.
So in this case, it was done through the nose basically. And then it travels through the brain. And we can see it over the course of the time during the scan. And because there was so much enhancement, the regular procedures didn't work. And we used the dash edge option and then accounted for the small voxel sizes with the rat align option and LPA cost function.
So a small script using the same align EPI anat. So align EPI anat, although it's called align EPI anat, we don't just use it for EPI anatomical data sets. We can use it for all kinds of affine alignment.
We have a GUI for aligning EPI anat. It provides a very limited set of options, a couple cost functions. But it allows you to try that out. But most of you will be using AFNI proc, that py to do your alignment. And you won't have to deal with the small options in aligning EPI anat. And it will do it as part of your processing.
And these are the options you put in there. You use the align option. There are various options of which to align in your volreg. You can align to the last, or align to the first, or align to the minimum outlier. It doesn't really matter much which one you use, just use one that doesn't have motion in it. That would be a good idea, OK?
This is a typical series of transformations we're going to go into to align all of our data. So all of our data gets aligned. Our EPI data gets aligned to one particularly sub-brick of the EPI data set.
That gets aligned to an anatomical data set. That anatomical data set gets aligned to a template. And then we can use the atlases from various spaces that are associated with that template, and ones that we can get to from the space of that data set.
So we know, if it's an MNI template, we can use an MNI atlas directly. And we have transformations to go from teller Rackspace to MNI. And so we can use those too. And we'll do this across sessions.
Now, we have a script. If you want to remove bias from one session to another, you have two sessions of data and you don't want to have one with less motion than the other or less alignment done to it, you can create an in-between anatomical data set. And it's similar to the previous slide, except we've created this in-between anatomical.
And all the data will be aligned through this in-between. So maybe pre and post exercise, pre and post some other kind of treatment. And you can extend this idea across multiple sessions too.
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
LEARNING