02 - AFNI, FMRI and Dimon 2 of 2
Date Posted:
July 31, 2018
Date Recorded:
May 28, 2018
Speaker(s):
Rick Reynolds, NIMH
All Captioned Videos AFNI Training Bootcamp
Description:
Rick Reynolds, NIMH
Related documents:
For more information and course materials, please visit the workshop website: http://cbmm.mit.edu/afni
We recommend viewing the videos at 1920 x 1080 (Full HD) resolution for the best experience. A lot of code and type is displayed that may be hard to read otherwise.
PRESENTER: So I'll just-- maybe I'll show a couple more details on this stuff before moving on, rather than being too random with the 3dinfo command. Let me note, for example, for those of you that are not so comfortable with Unix, what that star .HEAD means. So if I type this 3dinfo-- and I'll shorten it, I'll get rid of the old 3-- this star .HEAD evaluated to match any file ending in .HEAD, in all caps. So all of these files, any of that-- it's like we had typed these things.
And you can actually stick an echo command, say, in front of this to just echo this text back to the terminal window. So we'd see what it's like we had typed with that wildcard expansion. And it's as if we had typed all this garbage-- 3dinfo dash prefix HEAD 4, and then [INAUDIBLE], and that tlrc.HEAD, dot, dot, dot, dot-- all these HEAD files. So it's as if we had typed all of that.
Let me note that it might be-- you can get info like the space of the data, as well. So you see mostly a ridge, and then one tlrc for standard space. Again, here, if you say space, it's not necessarily just tlrc for standard. Here, tlrc actually means Talairach space. You might see MNI. You might see MNI and NAT. If you define your own space, you could see that, I think.
What other options does 3dinfo have? If you've set up your system fully like the instructions showed, you'll be able to see this, but maybe not. If I did 3dinfo dash, and I hit tab, we've set it up where you get tab completion for options in the AFNI programs. So I hit tab after that dash, and I still see 3dinfo dash on the command line. But it showed me all the options. That's like file completion. If I type ls A and hit tab, it shows me all the files that start with A.
It's similar to that, except with AFNI programs, 3d-- 3d, say, Deconvolve dash-- I can hit Tab, it shows all the options. So that's a helpful thing. If you know the option starts with stim-something, you can hit Tab, and now you just see options-- now you-- 3dConvolve stim Tab, now you see options that start with stim. So that's very handy.
AUDIENCE: But it doesn't [INAUDIBLE].
RICK REYNOLDS: So you have to set up your-- set things up for this. So how many of you can see this? Just a couple of people. So if you want to set this up, you can ping us whenever, and we'll make sure that's working for you. But that is one extra step in the installation.
So we babbled about the dataset headers. NIfTI datasets, NIfTI was a standard that was written or defined by many groups in combination-- FSL, SPM, AFNI, I think BrainVoyage, I think-- I don't know. Anyway, a bunch of different groups, many people came together to argue and bicker for a while. And they eventually came up with this format that's based on the old ANALYZE format.
But ANALYZE-- if that evil keyword is up here, I don't see it-- ANALYZE format, like, didn't have spatial information. You'd have your dataset, but you don't know what is left, you don't know what is right. You don't know where it's located all the time. So it was a mess.
So NIfTI was created just to be specific mostly about that missing information-- where am I in space? So NIfTI is a data format. Do we have the word NIfTI here? Oh, on the top of the slide, excellent. So NIfTI is a format that-- you can pass data among software packages via NIfTI format. Because basically, everyone can read and write-- some scanners can write NIfTI directly. Phillips can, for example, I believe.
How do you create NIfTI datasets in AFNI? Basically any program can do it, any AFNI program. If you're using 3dcalc, 3d-- any program in AFNI that creates a dataset, you can say to make it call NIfTI. How? If you do 3dcalc, which is a three-dimensional calculator if you want to add two datasets or divide by three, whatever, if you take 3dcalc, and your input data set is EPI, and you do prefix "new," it's going to make new plus a ridge dot HEAD and BRIK, right? AFNI datasets but with the prefix "new."
If you use a, say, dash prefix "new" dot nii, it will make a NIfTI dataset, new.nii, or new.nii.gz-- it will gzip. It will compress it. So basically, any program you can say, write as NIfTI, just by specifying that in the output, OK?
Getting and installing AFNI. You've all suffered that-- those ordeal-- that ordeal already. I guess we don't really have to say anything about that. AFNI at the NIH scanners. I don't know how things are set up for you, but just it might be worth telling you what people do at the NIH.
So particularly on the GE scanners, this has been set up for, I don't know, 10 years or longer now. I don't-- I don't know-- a long time, since iFiles, before Dimon was-- DICOM files were used. When the scanner starts, they have a little script to start-- well, let me be most simple. We have a program called Dimon, for example, that can look for DICOM files that are-- as they show up in the file system in some directory. It will keep scanning the directory and see them as they come in.
Dimon will collect these images, these single slices, into volumes, and then ship those volumes to AFNI in real time. And you see them in AFNI. And you can-- and AFNI will do, like, motion correction. And you'll see a plot of motion regressors.
So even with this much, which doesn't take any setup, you can do this stuff-- in the olden days, people actually would type Dimon and type AFNI on a different computer, and Dimon would run on the scanner-- something that can see the file system from the scanner. And Dimon would send the AFNI images over TCP to a different computer, and you'd see them in AFNI. But that's terribly easy to set up as long as you can see the file systems.
And then you see the images as they're collected. That's tremendously useful, because, you know, the scanner itself that shows the image is great. But, you know, there's still a question of how they come out looking in a normal dataset.
So for example, some people mess up at the scanner, these fields are in millimeters, this one's in centimeters, I forget my field of view is not 240 centimeters-- millimeters. It's 240 centimeters. You're collecting images like this big, and the brain is this little area in the middle, and you get 3 voxels across in every direction. So that data is garbage. You'd like to see this at the scanner, and fix your protocol, and collect it properly.
Some people have gone through scanning and not seen this for months-- like, three months of collecting subject data with this problem, and not looking at the data. And that's just-- now you've just wasted all that time and resource and money. So you want to see the images right away. So it's nice to have some setup at the scanner that enables you to do this.
And the motion, seeing the motion right up front-- you're moving a little too much, just try to relax and breathe comfortably. Get your whole body as relaxed as possible and try not-- try to move less, OK? And just say-- of course, you'll say this anyway. You just assume they're moving too much. But it's nice to see the motion parameters so you get an idea of actually how much they are really moving in the scanner, at least according to the estimates of the software.
But so, beyond that, at the NIH they've automated this stuff. So they have scripts attached to the scanner so that when you hit Scan, it actually controls launching AFNI on a remote computer. It controls running Dimon locally on the scanner and sending the images over.
So at the NIH, the researcher doesn't have to think about this. They just say, OK, scan a new run of EPI-- collect more EPI data, and then they see this automatically, no effort involved. So it's nice to have some setup like this. So just some food for thought. I don't know how your setup is. You can even set up AFNI for real-time feedback and things like that.
So we've babbled about the command line programs, which we might call batch mode. Batch mode is like scripting or typing commands rather than clicking buttons in a GUI. Like I mentioned, there are quite a few programs in AFNI. And so there is an initial learning curve. But we have examples of how to do things.
We just recently replied to one of Tom Nichols' pre-prints of some analysis. And along with our reply- to that-- or pre-ply? I don't know-- we included examples of a full processing analysis-- everything. So we have-- we've recently redone an analysis with a complete set of scripts. So you can use something like that as a sample starting place.
But how do you learn all the individual programs? Just practice. Practice makes better, right? So do so. We have a lot of batch programs, a few examples of more than a hundred.
So this shows you-- this slide's a little dated. We have about 600 programs and plugins and models right now. But like, 3DDeconvolve does linear regression with ordinary least squares. 3dREMLfit does-- uses REML, an RML11 model for the generalized linear regression. 3dvolreg is an ancient program, but it's very fast for doing EPI registration.
This is what basically-- scanners were doing real-time registration using this software in like 1988. So it's built to be very fast. But of course, you don't care now, because everything's fast enough. But anyway, that's what it is.
Some of Gang's group analysis programs. Gang over there, most of his programs are written in R, but he has some-- he still puts 3d-something wrapper interfaces, so you're not running R directly. You just run them as if you ran any other program.
3dcalc, again, that's a calculator program. 3dresample, you can change the orientation how the data is stored on disk, change the grid voxel, the grid size, resampling, whatnot. Lots of programs.
SUMA-- AFNI, the little A-F-N-I, is the graphical interface for looking at volumetric data. SUMA is the graphical interface for looking at surface mapping data, so surfaces. SUMA, you can actually look at volumetric data, too. You can look at, like, correlation matrices, sparse matrix of nodes in a dataset that have correlations or whatever, some DTI measurement weights between them, whatever.
You've got-- there's basically a sparse matrix of data, you can show these in space, even connected to the surface in locations, and rotated, and all that. You can do wild stuff. Hopefully Daniel will have time to show you some of that at the end.
SUMA teaser movie. This is especially a teaser, because I can't show it to you. It was written-- this is a PowerPoint presentation, and I'm using Linux, so I can't show everything right on here. I think that's enough for command line stuff in here, too.
Let me show you a couple more things before I move onto Dimon a little bit. In this directory, just so you see, some of the other files that might be worth noting ahead of time, like this stim_av1_vis.txt. That's a stimulus timing file, just to give you a preview so it sink-- has a little time to sink in before we use it later on.
These are just-- this is a text file of numbers. It doesn't have to be integers. It doesn't have to be integral. It could be 0.716, whatever. But these are times of stimulus events from the beginning of each run.
So this has three rows, there are three runs. So the first event of this av1 vis class, a visual class that we'll-- visual stimulus event that we'll talk about, happened one minute into the first one, for example. So that's the most basic format of our stimulus timing files.
And also in here is motion.1d. 3dvolreg was run on this data to align each EPI volume to a base EPI volume, so taking this volume, aligning it, taking the next one, align it. And there are translations and rotations involved in that. And this motion.1d is just six columns of numbers that show the three translations and rotations around the axes.
You don't usually look at it like this, but it is just a simple text file that we can plot, so-- 1D plot. So I'll show-- that's a stupid [INAUDIBLE] let's add scaling in there. So I'll show this more later on, but so those are motion parameters, so just another couple of examples of what's in this directory.
So let me leap over to let's talk about Dimon briefly. How many-- is your data acquisition here for the most part automated? How many people actually analyze data? Do you have to handle DICOM files, or do you go straight to NIfTI or something like that? How many people handle the DICOMs?
OK, most of you. So this might be worth chatting about a bit. Bruker? What do we use to do that? Those are PAR/REC files?
AUDIENCE: No, no.
AUDIENCE: We [INAUDIBLE] write DICOM with their [INAUDIBLE] software. We did that with the-- [INAUDIBLE] monkey scanner.
RICK REYNOLDS: Mm-hmm. So do you get DICOMs out of there?
AUDIENCE: I take the raw Bruker file.
AUDIENCE: Yeah, but is there a export DICOM?
AUDIENCE: No, we did-- Bruker has this [INAUDIBLE] parameters use the--
RICK REYNOLDS: So that's PAR/REC files?
AUDIENCE: No, that's-- it's not-- PAR/REC is Phillips.
RICK REYNOLDS: Oh, Phillips, ah.
AUDIENCE: But they have their own structure, which is something a little bit like PAR/REC.
AUDIENCE: They have this raw data, and then we [INAUDIBLE] it. Like, we write it as [INAUDIBLE]
RICK REYNOLDS: Oh, good. Oh, from the scanner?
AUDIENCE: From a scanner. And then when we write it, and it automatically generated the header file and has all the [INAUDIBLE] information [INAUDIBLE]
RICK REYNOLDS: OK.
AUDIENCE: [INAUDIBLE]
RICK REYNOLDS: Yeah, this--
AUDIENCE: Or animal studies.
AUDIENCE: Yeah.
RICK REYNOLDS: Rats.
AUDIENCE: Yeah, looking at the rats and marmosets.
RICK REYNOLDS: Yeah. So in your case you may not have to worry about that. But if you're dealing with DICOM files, we do have a program that will collect them and turn them into datasets, AFNI or NIfTI, if you want to.
AUDIENCE: [INAUDIBLE] for Bruker files, you can read directly into AFNI, but you need to put all the parameters and headers in.
RICK REYNOLDS: Right, right. So to3, that's a pain. Yeah.
AUDIENCE: The way we have it set up at NIH for our monkey scanner, which is a Bruker scanner, is we've automated it with Dimon. And using the ParaVision software, we start a script that starts this Dimon software and automatically reads it in. Because it's a pain for every one of the researchers to figure out exactly what the parameter is.
And so Rick and my time is getting filled up with every variation of every experiment. And they actually write DICOM properly. So I highly recommend using the Dimon software. And if anyone needs the script that we wrote, we'd be happy to provide it.
AUDIENCE: So you can go Bruker to DICOM?
AUDIENCE: Bruker to DICOM. But you need this ParaVision thing, which I think might be an option.
RICK REYNOLDS: They weren't the ones writing NIfTI in real time?
AUDIENCE: They were not writing NIfTI in real time-- well, in the end they were writing--
RICK REYNOLDS: Because didn't they--
AUDIENCE: Well, they were writing--
RICK REYNOLDS: --they wrote NIfTI or AFNI, and then we converted it-- Dimon read that and then made--
AUDIENCE: No. No, we do it through the DICOM format. We're doing it Bruker's DICOM to AFNI format. Because what was happening was either people were writing their own Matlab scripts, or they were writing their own to3d commands, and everybody was doing it individually and getting themselves mixed up about orientations and, rats and mice and marmosets were upside down.
Yeah. And [INAUDIBLE] so I think-- I highly recommend this. I think it would be very interesting.
RICK REYNOLDS: Yeah. So this-- I'm up in the upper right corner here. So in AFNI handles AFNI 02 to3d. to3d is the oldest method. That was probably, possibly, the very first AFNI program? I don't know.
Because that program was used to create an AFNI dataset. So that's kind of a prerequisite for using an--
AUDIENCE: [INAUDIBLE] first.
RICK REYNOLDS: Yeah, but that predates AFNI.
AUDIENCE: Predates AFNI, but ccalc predate-- yeah, ccalc predates AFNI [INAUDIBLE]
RICK REYNOLDS: So yeah. This was sort of necessary for obviously doing anything with the AFNI format. But now, for the most part we don't recommend using to3d unless you have to. So you can run to3d, or if you have DICOM files or even GEI files, Dimon-- Dimon stands for DICOM monitor. It was written to monitor incoming DICOM files at the scanner and send them to AFNI in the real time.
But you don't have to use this at the scanner. You can just use it to assemble datasets-- or to3d. Even if you just have raw data with to3d, but then you have to tell to3d the voxel dimension-- the number of voxels, the orientation, the voxel size-- not just the number but how big each voxel is-- and stuff like that. So to3d is harder to use, but you can, of course, use it to take any raw data into a dataset.
But Dimon deals with DICOM files [INAUDIBLE] or in real time. So for example, we have some DICOM files here under AFNI_data6/EPI_run1. So I guess I will not make-- maximize this.
So if you recall, just to give you a little more practice at the terminal here, we're in the AFNI_data6/AFNI directory, because we cd'd into AFNI_data6 and cd AFNI. So let's go-- we want to go up one level, to the parent directory. And the parent is called dot dot.
The current directory is a dot, which you occasionally need. If you want to copy some file here, you might copy it to dot as the destination of what to put in this directory. Dot dot takes you up one level.
So now you can see the prompt shows at AFNI_data6, as does pwd. And if I type ls, now you see the AFNI directory that we used to be in. So this slide, for example, here is babbling about EPI_run1. So if I type ls on EPI-- and again, if I just type the E or the EP, I can hit Tab and get file completion there.
But if I look at the contents of this directory, it's got all these 8HRBRAIN blah, blah, something dot dcm file. So this has a bunch of DICOM files. So you could use Dimon, for example, to turn this into a dataset. I don't know if I'll follow all the examples like this. I guess this one's OK.
But so I can use Dimon dash infile-- you can say infile pattern, and you can give a wild card, or you can use infile prefix and say it's EPI_run1. It starts with 8. All the files are named something starting with that. So I can give this as a prefix. That's an alternate usage.
And I can just say GERT create dataset. There are a lot of goofy GERT options to Dimon. GERT stands for GE Real Time. So as with many things in AFNI, there's a historical background and why we use this stupid name for this stupid function. And you just have to-- you get used to that. It's entertaining to some degree.
So anyway, we tell it to create a dataset-- out of everything that looks like this. So I hit Enter, it finds a first volume that-- So it actually looks at these slices and looks at the coordinates in them until it finds something that repeats. And then it checks to see if the coordinates of these slices match the coordinates of the first volume that it found, and it's looking for missing slices.
It's doing a lot of error checking that's the purpose of it. But in this case, we just want to create a dataset. So after it goes through that that, it actually writes a little script that has a to3d command in it. So that you don't have to figure out to3d, it's doing that. Mm-hmm?
AUDIENCE: Can we see the Dimon, what's inside the script?
RICK REYNOLDS: Yeah. So the Dimon command itself is This and if I type ls now-- or actually, ls dash ltr-- ls, the ltr, the l is a long listing of the files. So it includes things like some permissions, or whether it's a directory, executable, read permissions, the owner of the group, the size of the object, the date that it was last modified, and then the name of the directory or file. So ltr-- l is a long listing, t sorts by time, an r is reverse sorting so that the newest stuff is on the bottom.
So ls dash ltr, we see the newest files on the bottom. So what did Dimon create? It created GERT_rico-- another goofy name, GE Real Time reconstruction. That's just the prefix. This is actually a script that has a to3d command in it. Dimon.files.run.003-- it found this-- it found that these files had run number three in them. So it was the third scanner acquisition.
But this is actually the list of the files that are this sorted list. Because sometimes, the alphabetical ordering of the files is not correct, you have to fix it, you may do so. And then OutBrick is the name of the dataset. If I look at the top 10 lines of this Dimon.files-- head shows the top 10 text lines-- these are the first 10 files that went into the dataset. But of course, there are thousands of them. There are 2,278 files that went into this run. It's 33 slices by 152 time points.
And lastly, that script-- if I look at that with less, or you can-- I guess you can't type "open" on your Mac, because open won't know that it's a text file, won't know what to do. Ah, if you-- since I don't know what editors you all have, we have an afni_open, abusing the Mac term, the Mac command open. We have an AFNI form of it that says afni_open dash e on some text file.
The e says to search my computer for some editor and use it to open-- look at this text file. So on mine, it's--
AUDIENCE: --know which editor [INAUDIBLE]
RICK REYNOLDS: Yeah, yeah. Use whatever you want to. But if you don't know, afni_open dash e will find something. Hopefully it'll find something. But that font sucks, so I'm going to stay on the terminal window.
So this is a tiny script-- basically just comments, setting a couple variables, and then a to3d command to create the dataset. So this to3d command has the name of the output datasets, the orient-- the sorting of the data, the binary data that it's going to convert is-- the z-coordinates are changing before time. That means you get slices varying over space first and then a new time point-- slices, new time point, new time point, point, slices. That's the zt.
But you don't have to memorize this, because that's Dimon's problem, not yours. I'm just showing you what it is. And then 34 slices, 67 time points, that corresponds to z is 34, t is 67. The tr is three seconds. The alt plus z is that interleaved slice timing pattern.
And then the names aren't on here. You could use a wild card or something like this, but this is just saying to read the names from this file. Anyway, so that's the script that Dimon eventually runs with the GERT create dataset option. And now you have that dataset. Questions about this?
AUDIENCE: So it may run [INAUDIBLE] own sequence, and then this [INAUDIBLE] is like a sequence is different, like it's not interleaved [INAUDIBLE] When we use the DICOM, do we have option to change that?
RICK REYNOLDS: Yeah, you can tell diamond what the slice time-- the slice pattern is. You can say alt plus z2, or sequential, or-- what is it when it's all at the same time? I forget. But you can tell it all these things. And you'd actually get the help for those things in to3d. It lists all the slice patterns. Yeah it's the same.
AUDIENCE: I think it's t0, which [INAUDIBLE]
RICK REYNOLDS: Yeah. Yeah, t0. Yeah, that-- or is that 3dTshift? I don't know. Anyway, so you can look it up and make sure of what you're doing. Any other questions about this?
So let's get slightly more complicated. Oh, we're out of time. So I'll be quickly more complicated-- quickly slightly more complicated. So here is-- there's also this DICOM_T1 directory. If we run Dimon-- maybe I'll just type myself. Infile pre DICOM_T1. They all start with I-- and try to create a dataset-- dataset.
It fails. Why? There are multiple reasons in this case. One is that if you look at the files in this-- what directory? DICOM directory-- well, if you look at the names, this is just a random set of names that Bob made up. But often, the names are not alphabetical. The alphabetical naming is just, oh, the scanner's choosing things in an asynchronous fashion and naming the files.
But internally, it knows the right order. But if you look at them alphabetically, it might not go like this. It might be all over the place. So the alphabetical ordering of the names won't necessarily match.
And you may have to tell Dimon to figure the ordering out. And there are many-- there are multiple ways it could do so. But there is a most common way that it will try first. And then hopefully that works for the most part. In this case, the header, the DICOM file, the DICOM headers inside these files-- in this case, it contains multiple information that disagrees with itself, multiple pieces of the same information that are different.
So you actually have to give it extra options in some case to say, work harder, in some sense. In this case, we have to tell it to use last element. So in the DICOM header, it might list some information twice, and the second to last occurrence is the correct one. So this is-- use last element is saying to do that.
And I can just say DICOM org. This DICOM org option is just telling Dimon to look at-- it looks at one particular DICOM field-- the relative instance number, I think, or something like that-- see the help-- and sorts on that and sees if it varies. So in this case, now yes, it found everything. 171 slices-- this is just an anatomical dataset, so it's not multiple volumes, but-- found 171 slices, but they were sequentially ordered, which is to say the z-coordinate was varying in a consistent manage-- manner. If it did not vary consistently, Dimon will whine about it. It will probably fail.
But with these options, it works. And then you're golden. So if you were to use Dimon in your setup, what options would you use? I don't know. That's the sort of thing you have to play with a little bit and figure out. I can help you do that. But the DICOM headers vary drastically, wildly. Even within one company, they can vary. And then across manufacturers and across decades, it's a little unpredictable.
So exactly what options you would need may depend. And some of your data doesn't come from here, right? You get it from elsewhere, and then the options will be different. So you have to think about this stuff. But this is one program you can use to create datasets.
We also have a DICOM to-- dcm2niix from Chris Rorden. We have an AFNI version of that, dicom2niix_afni, just so there's no-- so it doesn't conflate with the one he puts out. But so that's another option. Any questions about this?
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
LEARNING