24 - Surface Analysis: SUMA: Part 1 of 5
January 28, 2019
May 31, 2018
Daniel Glen, NIMH
All Captioned Videos AFNI Training Bootcamp
Gang Chen, NIMH
For more information and course materials, please visit the workshop website:
We recommend viewing the videos at 1920 x 1080 (Full HD) resolution for the best experience. A lot of code and type is displayed that may be hard to read otherwise.
So we're going to start our discussion now of SUMA. This is the surface mapping part of the AFNI package. So this is usually a lot of fun for everyone and we get really, really pretty pictures, so I think everyone would like that.
OK so we know about AFNI. We're able to look at functional activation inside AFNI. We can look at graphs there. We can even render inside AFNI, but that still looks inside the volume with a volume renderer with cuts through the volume and our rendering plug-in. And now this is the SUMA software where we can map things onto the surface of the same data mapped onto the surface.
We can look at that in a different way. We can also analyze our data on the surface. OK, so SUMA is a complement to AFNI. So AFNI does slice and volume rendering. SUMA does surfaces. Now SUMA also does slice and volume rendering, so you can look at a lot more data with SUMA by itself and in combination with AFNI.
And we'll show some of those things today and some of those things tomorrow. We support surfaces that we can get from other software packages. Freesurfer, namely, I think SureFit and Caret aren't supported anymore. And they've gone over to their Workbench software.
That's OK, because they also write Giftee. So instead of Nifty for volumes, we have Giftee for surfaces. So Giftee is available as a standard across software packages. And so we can look at Giftee data sets, too. Brain Voyager also produces these kinds of data sets. We can read and write lots of different surface formats, so getting data in isn't usually too big a deal.
AUDIENCE: So the Workbench [INAUDIBLE].
PRESENTER: The connect on Workbench, a lot of the things are in CIFTI. CIFTI, yes, another one, because we need more file formats, yes. So CIFTI is a combination format. I forget what CIFTI actually stands for.
No, it's with a C. CIFTI. Rick will know because he was part of the committee, but he left the room just in time. And we have some tools for looking at the header of CIFTI data sets, but manipulating them is a little bit more difficult. It's a wrapper for Nifty and Giftee, and functional Giftees, which we'll talk about, which are like our nimmel data sets.
So values that will assign to the nodes of a surface we put into a data set that we call nimmel.deset, which can also be in a CIFTI or Giftee format, the functional Giftee format. So now that Rick is back, we can ask him. Don't run away yet. What does CIFTI stand for? What's the acronym? You don't remember?
Yeah, but I just can't remember the connectivity. We could Google it, but that's too hard. OK, all right, so let me keep going. So AFNI-- the AFNI package, SUMA, we generally don't produce surfaces ourselves. So if you're going to look at surfaces, you will generally get them from somewhere else, because we don't make them, mostly.
We started. We started a little bit. So we do have a way to make some surfaces, and I will show that tomorrow. But most of the surfaces that you will get will probably be from Freesurfer from the Workbench, connect on Workbench or something like that. Oh, I can say Freesurfer, yeah.
Yeah, so a lot of our AFNI users use Freesurfer. We mostly recommend it and it works well for generating these surfaces. There are limitations. Maybe we'll look a little bit at those, too. The slide is a little bit old. Before you run Freesurfer you don't actually have to run the 3D uniform ais, which has now been replaced by 3D unifies, which is a good program to know about, but but Freesurfer does its own and three and four normalization, anyways. But it probably doesn't hurt to do it before Freesurfer, too. But I think we, in general, we don't actually do that.
So Freesurfer's gone through a lot of generations. SureFit is not available. It was wrapped into Caret. Caret is now replaced by the Workbench, and some people actually use Brain Voyager, too, which is a commercial package for volume and surface analysis.
OK, so let's assume you've got your surfaces out of Freesurfer. Then you can bring them into SUMA, and in SUMA we'll create what's called a standard mesh version of those surfaces, something that can be compared across subjects on to a common mesh design. And so node for node correspondence across subjects.
And then we can align our data. We align our surfaces with our own experimental data. Freesurfer, while it's a very good program, is very slow to generate a surface. So it can take a day or two days for creating a surface for any particular subject. And you may not want to do this for every session of every subject. So if you have multiple sessions, you can you can use the surface that was generated from one particular session and apply it to another by doing an alignment with us at SUMA, aligned to experiment. And we'll just transform that surface over into the experimental space.
We'll show you how to take the volumetric data and map it onto the surface. So the MRI data is volumetric. We're going to use part of that data in the surface domain, so we're going to look just at the surfaces. That has some advantages and some disadvantages. And we can also look at the time series on the surface.
And it's important to know and interesting, too, that we can do our analysis of surface data in almost exactly the same way that we analyzed diametric data. And the reason is that most of our 3D programs aren't really 3D in that they will just take any list of numbers and treat them as time series, as long as they're not spatially oriented, they will work on surface data sets.
So all of our things, 3 calc and temporal smoothing, well, not slice timing but slice timing is a spatial thing so that won't work. We have other programs, also, for clustering on the surface, too. The group analysis doesn't care where your data is. It will, as long as there is a correspondence across subjects, it works.
So even if you don't go through any of that process, you can still look at your data on the surface. So this is the simpler thing that you can do readily. And we provide surfaces for you for the end 27 data set. These are either in Telerec or MNI space. It doesn't really matter here. If your data is in one of those spaces you can just map it on, map the surface, map the data from your volume onto the surface. This is super easy to do. We'll show you how to do that.
Now it's a different subject, so different subjects, it won't map perfectly well. It's not as good as mapping to the surface of your subject, so you'd probably want to do this for your own subject, but if you just need something quick to just get an idea of where this activation is in some 3D, in a nice visualization, then you can do that here.
So we provide a script called @SUMA_Make_Spec. and for free surface, Mikse'" And so Freesurfer generates all these kinds of surfaces, Caret, also, had done the same thing. Workbench also provides these kinds of things. For Freesurfer and for Caret, we have scripts that will make a spec file. So a spec file is a specification file, a text file of how the surfaces are related to each other and the names of the surfaces.
PRESENTER: Yeah, so we don't make the expanded version. Freesurfer does that. And Caret and Workbench make that. Freesurfer has some tools for kind of going after the fact, too. There are also tools for making flat maps. We don't do that. We do provide the ones that we've done for the N27 data set.
We also have the MNI 2009 C template that we've converted to a surface. So even though it's a group data set, Paul Taylor recently made a surface out of that. I think Rick will be presenting that, so. That worked pretty well, too.
OK, so we've got SUMA_Make_Spec_FS. OK, so let's start SUMA. I'm not going to change to this directory. I'm going to start in a different directory. We're going to start in SUMA demo AFNI instead of surf data's similar. OK, so you should have a SUMA demo AFNI some place. Just start a terminal someplace. And let's see if I have it.
Let me you know, [INAUDIBLE] this over to this side, here. We're going to need a lot of space for all the things we're going to look at. OK, so on my system, this is where SUMA demo AFNI is. Can you see that? That's big enough. So CD to that directory.
And in that directory we can start AFNI. And we're going to start it with AFNI-Nimmel&. So the dash nimmel means it put it in a mode that it can talk to the world. It's the same neuroimaging markup language, but this time it's used over TCP/IP, and so SUMA will be able to talk to AFNI. We'll show you that.
The ampersand means put it in the background, and actually that's not needed anymore, because it automatically starts in the background. And then there's the SUMA command, but I'm going to run the run SUMA script instead with the TCSH run SUMA in that directory. And we can look to see what that is so that there's no black magic underneath.
OK, so it's really just one line. That's really doing everything, here. It's SUMA-spec. You give it the name of the spec file, the specification of your surfaces, and then an alignment volume. The volume that's aligned to your experiment. So Freesurfer provides a data set that's aligned to the surface and then you align that surface to-- align that volume to your experiment. And that's this one and it contains a transformation within its header that will move the surface if we need it.
In a lot of cases, it will be the identity. There isn't any transformation really at all. And we'll just keep it in its own space. All right, so I'm going to run TCSH. You should run the same. Type TCSH, run SUMA.
Did I type AFNI-Nimmel? So AFNI-Nimmel. And here's SUMA over here. So you've seen your first surface. I'm going to close some of these windows to give me some space here. So let's make this bigger.
OK, so you have a surface open, and it defaults to the smooth white matter surface. OK, and SUMA demo AFNI directory. Now if you're running Windows laptop, sometimes the OpenGL that this uses is not as pretty as others, but it will generally work. It might be a little bit flickery, but it will work.
I think windows needs to improve the open jail performance and their Linux subsystem, but I've found that mostly it will get us to where we need to get to. OK, so now once this is open you probably want to do something with it, and while the instructions say have it talk to AFNI. So we can do that.
First, what do we want? Yeah, I guess we can keep that there. OK, so let's do a T inside SUMA. Press the letter T over the SUMA window. And you'll see a lot of things are going on. And in AFNI-- I'll put this back on because it'll be easier to see.
You'll see that we've got these contours set up in the AFNI GUI in the image viewer. So we zoom in, and I can pan over. See, we have different surfaces. These are the contours of the surfaces SUMA has sent the surfaces over to AFNI, and AFNI in return has sent something back to SUMA.
There it is in the back of the brain. AFNI is sending over its volumetric data to SUMA from the overlay.
PRESENTER: The overlay here is the delay from a retina toppy experiment. It could be anything, really. It's whatever the data set happens to be, but by default, it has selected an overlay here for us, But you can turn on and off the overlay and it will remove that from SUMA.
PRESENTER: I have it as the demo subject ECC [INAUDIBLE] a veer .DL that DL plus a ridge. I guess I can just show you. You don't have that in your SUMA demo AFNI directory? You do have it, OK.
And it's set to automatically show the delay and the covariance for that delay. All right, so moving the surface around. OK, so we need to go down through this. And you'll notice that data sets, if they're done through Freesurfer, will automatically have the segmentation show up is inside SUMA. So here it shows us that there's a node clicked some place. That's the [? calprin ?] sockets.
OK, so we'll go and click in various places and you'll see. OK, so anyway, for now, we don't even we don't need to actually look at the colors on it. We can turn off the overlay for now. Rick will go through the volatile surf part. I just want to show you other aspects of the surface to start, although in the end, this is the part you're really interested in. But let's save that for a surprise later.
I turned off the overlay in AFNI. So the surfaces and SUMA are just great. They're colored, actually, by the convexity. So there is a two scale. There's a two-level grayscale on the surfaces. Plus there's lighting in the OpenGL system that gives it a shading.
All right. OK, so you can move the surfaces around if you haven't already by dragging with the left mouse button. OK, I'm sure we have this someplace here. OK, so the left the mouse button 1 will let you you drag this around and rotate the volume. And you can rotate it either direction.
And you can use the arrow keys. So the arrow keys are handy if you want to move in small increments and you want to record that. Like AFNI, SUMA has its own resource file. It's called dot SUMA RC. There's a long list of options there, variables that you can set if you want to customize SUMA to work the way you want it.
OK now this one is this one we called the walnut brain. OK, so mouse control button 1. OK so we're going to control and hold button, yeah, that same left button down. I think everyone will like this.
So you can have them applaud. Yeah. Control and the left mouse button. Yay.
PRESENTER: Did I skip that? Right. I think if, for some reason that's another order here. But oh, yeah, yeah, yeah. That's what you mean the shortcuts. We have the keyboard shortcuts-- where do we have those keyboard shortcuts? In the handouts directory. Let me put that in the background.
Yes, so we have a file here that's SUMA keystrokes. Let me open that up because that's that's a handy file to look at. I'm going to open it with my editor. OK, so, let's see if I could make that bigger.
So this is in your handouts directory. You can see the main keystrokes that are useful for SUMA there's a fair number of them. So we've talked about left, click, drag, and the control, left, drag. We have a bunch of other ones here. Let's go through some of these things.
OK, so if you are finished with the walnut brain, this is actually a useful thing despite being fun to look at, it's also useful. We'll look at it with the DTI data. Let's see. So control, left, double click will put that back together again. So if you do control, right and left and up and down, you can get two kind of cardinal views of control, left, right, up, down.
And zooming in is done through the scroll wheel. That's how I like to do it. I think you can also do the Z key, uppercase Z zooms in like in AFNI, and lowercase z zooms out or uses the scroll wheel. If you hold the middle button down you can move it to the side. You can zoom in with holding two buttons down, the middle and left also, but they never need to do that.
Now another thing you can do is you can click on the surface with the right mouse button. So in SUMA, the selection of a node is by right clicking. So let's do that. Right clicks on a surface. And you'll see that because it's talking to after AFNI, AFNI will bring it to that location.
So this we're looking at white matter, where is it in the brain? Let's move to a different spot. Where we click it takes us to a different spot. I clicked on a whole bunch of SUMA locations. So it sends each of them in succession to AFNI, and you can see that in SUMA, the current node is shown with this little crosshair, this three dimensional crosshair.
PRESENTER: Yeah, so you know, it depends on the version of the OS, too, which one. Usually it's the right click, but it depends on your mouse configuration. Yeah. And some of these things you have to either disable Mac shortcuts or even Linux can override some of these things, too.
There are a lot of keystrokes. If you've moved this off to the side someplace and you want to get it back, you can bring it home by hitting the home key. Function home. And the instructions say function left arrow on Macs, although I think it usually worked with a home key for me, but--
Now here's another fun feature. The M key. This turns on momentum. And when you turn on momentum, nothing happens. But it will. So if I take my cursor now and I rotate and I let go, it's kind of like, yeah, it's a little bit odyssey, 2001-- A Space Odyssey.
Now you can do it faster, right? It'll go as fast as you can move your--
PRESENTER: Yes, you can. In fact, this is the danger of momentum, because momentum will write out a very long movie if you're not paying attention, and I did this in a class and I talked a little too long after I did that. And it uses up all your memory and crashes your system, so that's not good. But it just crashed SUMA, so that wasn't so terrible. All right.
So yes, you can save a movie. Mostly this is for fun, but saving movies is the one place where it's good. But if you are saving a movie. I would say just do it with an arrow key. And tomorrow I think I'll show you how to draft so that it will do it for you and you will just have the script send an arrow key to SUMA, so it's very controlled.
And it will, but you can hit just an arrow key. But this-- you can control the speed of how big an arrow key moves, too inside SUMA environment variable. To turn momentum off, press M. OK, one thing you'll note here is how to get help, because everybody needs help once in a while. So here it is. You can do help by clicking on the Help button on the top there, or well, that's probably a good way to do it. And so help-- well, you can do control H to this usage, or just click on usage there and it'll show you a fairly long description of some of the things that SUMA can do.
And that's probably the way that you'll get most of your help, but you can also go to select web help. And so we have more-- in our documentation we have a fairly long description of SUMA. This is where this ought to point to, but we've moved over our [INAUDIBLE] recently, so it's in the wrong spot.
Anyways, you can see things like this with images. It's a very nice demonstration of how to use SUMA.
PRESENTER: You can. You can, and these are in the environment variables. Almost no one ever does this, but it is available. You can change the light from-- if you look for the light in it. We have some things that are available. We're probably just going around again and again through this.
Yeah, I think I've crossed through this. Let's go back to SUMA here. You have some things on light. So you can flip the light with an F key. That's one of the things you can do. Or you can go ahead and try any of those, capital F, so the light is in a different direction, so you get a darker shading that way.
Let's see. You can just hit Enter on here. It takes you to the next occurrence. It's XYZ coordinates, so you can move the light around. So open GL has a light location. In the default mode mode that SUMA runs in, there's one light, so you can move that light around by changing the coordinate here by hitting capital L.
This is a kind of pretty advanced feature. You don't have to do this. If you want, and there's a couple more things about-- control L changes the specularity and diffusivity of the light. You can also add in another light. So SUMA actually has the possibility to add in another light in space so you can get things better lit.
PRESENTER: Yes, that SUMA RC controls those things. So there's a bunch of settings for light. If you need more light, I can turn on a light for you. It's not just a switch, though. It's actually code. Oh yeah, yeah, yeah. Peel servers. Yeah, yes, so yeah, I'm going to get to that. I think it's next on the list.
Oh no, one other thing to do before this is the recording the pretty pictures. So let's see, you've got some pretty picture that you like, and then you can just to hit R to record into a recorder. And there we get another view. It looks just the same. And I can do another view over here, and then I get another one like that, another one like that, and this opens up this recording viewer, which is just like the viewer inside.
This has its own recording viewer. We didn't talk about it here, but this has a record button. This opens up the same kind of viewer. And you can see this lets you go through your recording, so just before the class I was recording something for tomorrow's class, something that takes a little bit longer to generate, and you can look at it.
This works like an AFNI image viewer, and you can turn V on. So right now it's just three images, so it kind of flips among all this things. But if it's a nice, smoother, video, you can record that here if we do if we turn momentum on this I'm going to do it with just [INAUDIBLE] and as slowly as I can.
So M and then right arrow. Oh, I'm sorry. Yeah, I didn't turn on the recording for keep recording. So if I want to keep recording, I have to do capital R. So far over the SUMA window. And now I'll do that again. Right arrow. OK.
And then I do uppercase R again. The momentum is still going but the recording has stopped. OK, so over here I've got my image viewer. I've got a recording of that. And I can save any particular image with the Save button, here.
So I could save that one as this is a surface view. And then set that, or it can right click on it and say I want to save it as an mpeg file, and then save the whole thing as a movie so I can put that on my webpage, I can put it into a presentation, Really nice videos that you'll be making with these things.
OK, you can have it record to the disk, too. You don't have to do this manually and type it out. You can do control R and do that. Or you can have a drive SUMA script. Drive SUMA to record for you.
OK, so AFNI and SUMA talk to each other with this neuroimaging markup language that goes over TCPIP. They can be on the same computer or they can be on computers on other sides of the world. AFNI and SUMA don't have to be in the same It used to be until a couple years ago that you could only run one pair of AFNI and SUMA. So one AFNI could talk. If there was a SUMA on a system, could only talk to one AFNI, and only one AFNI could talk to one SUMA.
But we found ways to work around that. And so you can have many AFNIs and many SUMAs talking to each other if you need need that. That's sometimes needed if you're running this on a server with other people running AF in SUMA. All right, what's the surface made of?
It's made out of nodes and the nodes are connected by edges. Now here we see our surface. If we want to look-- this is just our snapshot of our surfaces. This are kind of a doppelganger of a surface. I'm going to close that. And I'm going to zoom in onto this and I'll hit the letter P, the surface. So just do that.
And if you look closely, you'll see that the surface has a bunch of holes in it. Our surface is now this holey surface. And so you'll see this with nodes that are connected to each other by edges. And that's what a surface is. It's just a list of nodes and edges. And that's what's in our files that describe the contents of the surface, and the coloring that we shade the surface, and we're just filling it in with something, either convexity or just some shading. Or we may be coloring from AFNI, too. So it brings up the topic of the geometry of the surface.
So we know in any surface, one node is connected to another node and connects to another node, and they're connected in a certain order. The order that they're connected to each other is called the topology. The location in space, we'll call that the geometry of it. Now when we [INAUDIBLE] these surfaces, this family of surfaces from Freesurfer, they're all done with identical topology across their surfaces.
So you asked about pale surfaces-- we have the white gray, inflated, spherical, you can have flattened surfaces, also. So these would all have the same topology but they look different. How do we get to those surfaces? Let's do that. I'll zoom back out. And if you hit P again, you can go through the mesh.
And this is the point mode, then there's a hidden mode where you don't see it at all, and then back to the original, the field mode. So P cycles among the different modes. All right, if you want to see the other surfaces, you can cycle among those with the comma and the period keys, which, on the US keyboard, are below the less than, greater than. That's kind of an idea to use those as the arrow keys. So comma end period will take you among the different surface types.
OK so here we have the [? pial ?] surface. Here we have an inflated surface. We have a flat surface. Flat patch. We have the spherical surface. We have the regular white matter surface. I'm using the comma and the period keys. I'm actually just doing the period key to go one direction. And then back to the smooth white matter surface. Yes?
PRESENTER: If you have a hole, yes.
PRESENTER: We don't have a way to fix surfaces in-- we can show you where the surfaces are. We can even show you where are the problems, but we don't fix surfaces. You would have to go back to free server. We do have a way to generate surfaces from scratch, though, so that can be useful if you have good segmentation.
PRESENTER: Yeah. We have a program called SurfQual which will report where the problems are. You can also look into the surface, particularly with the mesh mode. And in combination with AFNI, so I had seen in this data there were some oddities in the server, so if I could find--
You'd probably find it before I can, but there's some places where the surfaces are a little strange. And you'll see that the surface is not exactly right for all the data. While they do a pretty good job, it's not perfect. And so for cases where you have lesions, some of these things aren't very good. And if you need to be precise, some things are very strange where it's twisted up in there.
So you can find some problems like that just by looking inside AFNI for the twistiness of the places like that. So we work with a group that does epilepsy lesions. They won't use Freesurfer for that because they need to see the surface pretty cleanly. Instead they've devised a whole method of their own using Isosurface inside SUMA. I'll show a little bit of that tomorrow.
AUDIENCE: So if you work [INAUDIBLE] space, you can make your own surface [INAUDIBLE]?
PRESENTER: You can. You can. So mostly people will be using Freesurfer, but if you have a really good segmentation inside of your volume, you can potentially make it with AFNI, SUMA's Isosurface program could be used to generate a surface. You need to have a segmentation, not necessarily from Freesurfer. For the group that's doing the epilepsy studies, they have done their own with a machine learning algorithm that works very well.
Yeah, so some of these surfaces, some of them are anatomical surfaces. Ones like this, the coordinates are real coordinates. This is the smooth white matter. This one's also a real coordinate. These inflated surfaces are not in their real-- there's no real space for it. They're in roughly the right place, but the coordinates of the-- they're XYZ coordinates and here are not really defined well.
The coordinates that they come from are defined well, so we can identify what anything is by looking at the same node across this family of surfaces and see the coordinates for that, even in the inflated surfaces, and even on the flat maps and in the and in the spheres. Yes.
So the way you look at one hemisphere is you click the left or right bracket. So if you hit the right bracket, that makes the right hemisphere disappear, and you have just the left. And if you hit it, you can bring that back by clicking it again, and vise versa.
Alternatively, we loaded the both hemispheres spec file. If you have a spec file that only has one of the hemispheres, it will only show that that one hemisphere. And generally when we make this in Freesurfer with @SUMA_Make_Spec at best, we generate a both hemisphere spec file and a left hemisphere and a right hemisphere spec file.
PRESENTER: Overlay two hemisphere. They're kind of mirror images, right, so that's it there is a procedure to do something like this, and it's complicated. But Hung Joon Joe had a kind of work where he looked at the [INAUDIBLE] lateral analog for every node, and the functional connectivity maximum on their [INAUDIBLE] side.
And he looked for the differences between that and an autism study. And it's pretty cool. So correlation actually happens pretty symmetrically. So if you look at the contralateral side, it happens almost exactly in the same place, as the same equivalent place, as on the one side. So you always have this symmetric thing going on. So so there are some ways to deal with it.
You can also apply transformations. So you can apply transformations to surfaces. So we have a program for doing that so you can take a surface and apply an affine transformation or a mirror image transformation and move it around as you want. So yeah, I have mirrored surfaces, so take one half. If you only have a left hemisphere you can make a right hemisphere from that. Yes?
PRESENTER: So someone here was asking about stroke lesions and doing that. So you could potentially do this volumetrically and then generate surfaces from that and then look at one surface at the same time over the other one. It's a bit tricky and a little confusing, but you can do it. Yes, I think that's probably a little bit more advanced topic that we'll have to talk about.
But yes, you can show pretty much as many surfaces as you want. And tomorrow we'll look at hundreds of surfaces all at the same time. We keep all this information. One important thing-- this is maybe one of the most important things about dealing with surfaces is that as opposed to volumes where we're looking at everything in a geometric position, though the big advantage of working with surfaces is that things that are more spatially in common are going to be treated more equivalently.
So this becomes important with things like smoothing. It is one of the reasons that you do surface analysis with the surface directly and not the end result of the volumetric analysis, because smoothing is a completely different operation. Smoothing will do things like here, if you're on two sides of [INAUDIBLE], smoothing in in a volume will mix those together.
But smoothing on a surface is a lot less likely to mix those together. You'll have to go all the way around and back down to the other side to get the contribution from the other surface-- from the other side of the [INAUDIBLE].
So dimensions that you work with in smoothing can be a lot larger, and it's a lot more robust to changes. That said, you still have to get the volume onto the surface in the first place, so you make decisions there of how you do that.
A lot of people don't really think about that process. That is a process that does move things from the volume onto the surface, and then if you're going to move it back to the volume you have to make choices there, too. And there are lots of other issues. This goes on and on just like I'm doing right now.