TERRY SOHL: There's always going to be that next tool. You know, when I first started 32 years ago, it was to help start the first NLCD. And so I've seen that progression of methodologies over time. There's always going to be that next best thing. AI is the next best thing now. Five years from now, I'm sure it's going to be AI-based, but it may be completely different. And so I think that's the thing that is always the case in science. Always staying on top of the game, looking at what's happening in the field and always adapting. TOM ADAMSON: Hello everyone, and welcome to another episode of Eyes on Earth, a podcast produced at the USGS EROS Center. Our podcast focuses on our ever changing planet and on the people here at EROS and across the globe, who use remote sensing to monitor and study the health of Earth. My name is Tom Adamson. Artificial intelligence, also known as AI, is quickly becoming a necessary part of geospatial work at EROS. I had a conversation with EROS Director Pete Doucette about AI and its uses here at EROS. I also spoke to others at EROS who are using AI in their work to help clarify what it all means and what its benefits and challenges are. Fair warning to listeners. We did have to use some of the jargon surrounding AI, like machine learning, deep learning, and convolutional neural networks. Now don't be intimidated. It's a new technology and a very big change we're going through. Pete encourages us to be open to the new technology and its great potential to helping us at EROS make the work we do better, more efficient, and more useful. And besides, at EROS, as the home of the Landsat archive, Landsat data's quality, consistency, and continuity are optimal for AI exploitation. So let's get to it. We'll start with EROS Director Pete Doucette. Last summer, we did an Eyes on Earth episode with a USGS scientist about mapping the California coastline. He described AI like this. You're teaching a computer to learn by example. Is this a fair way to describe AI, or is there more to it than that? PETE DOUCETTE: I think Sean's characterization was a useful way to think about AI in terms of a concise way to put it. I think to better understand the bigger picture of what AI is, because a lot of people are still wondering, What is it really? It's useful to understand the different phases of its evolution. So the term AI was initially coined back in 1956. At that time, what kind of dominated the AI methodologies, if you will, had more to do with rules-based systems, for example, creating a set of rules that an algorithm would just, would just carry out. And so that way of thinking of AI dominated for probably a couple of decades until we entered into the machine learning, which is kind of what Sean's alluding to here. Learn by example. ADAMSON: Up until that point, it's really just computers are doing things faster than humans can. DOUCETTE: Yes, in a way, depending on what the application was. So, oftentimes applications were very targeted and specific, not very flexible, but they were effective. ADAMSON: Okay. DOUCETTE: If we think about learning, say to, to ride a bike, okay. You can learn to ride a bike from a manual, a user's guide, on how to ride a bike. And so that's kind of a knowledge-based way of thinking about the early days of AI. A more effective way to learn to ride a bike is actually getting on it through trial and error, you know, balancing and falling down over and over again until you figure out how to balance, is more in line with the machine learning way in that you're taking in the information and learning from the information that you take in, in real time. And so that's kind of the, I guess, metaphor I would use to describe what AI was in the past, essentially a manual for learning how to do something versus a trial and error way of learning it. ADAMSON: So what is your background? How did you get interested in AI and promoting its use among geospatial professionals? DOUCETTE: So my first career job was with Defense Mapping Agency, which is now known as National Geospatial Intelligence Agency, or NGA. The mission of DMA, which hasn't changed, it's basically to map the planet, largely from satellite data, that part of the planet that the USGS doesn't map. So the USGS is charged with mapping the United States and NGA has the rest of the planet. So back in those early days, the mapping process was conducted through digitizing or tracing features from imagery, oftentimes from from satellite imagery. And the business of mapmaking transitioned from using analog film-based products, right, to digital products. That was a major transition for the mapmaking process. And that's roughly when I entered into that scene of mapmaking. And NGA was transitioning. With digital imaging, of course, that afforded you all kinds of computer vision based methods to assist in, you know, digitizing or tracing out features such as, you know, roads, buildings, rivers, these kinds of things. And so, it was a course that I took at Purdue in 1993, if I recall, that was called Engineering Aspects of Remote Sensing. It was taught by Dr. David Landgrebe, who was actually one of the pioneers on using Landsat data. And at that time, he was one of the forward thinkers on how to extract information from multispectral scanner, MSS, data back in the day. And he used that data in his course as part of the, as part of the, the labs and such. And so, that was my kind of introduction to multispectral data analysis through David Landgrebe's course. Curious that his last name was Landgrebe, right, using Landsat data. It was, you know, a heck of a coincidence there. But that's what first got me kind of intrigued about using computer vision methods to automate the process of digitizing features or characterizing features from multispectral data. ADAMSON: We're talking a lot about AI right now, but it really seems like there's a lot more buzz about it now than even five years ago. Why is that? DOUCETTE: There have been many pivotal moments through the history of AI. One of them occurred around 2010, which was the use of GPUs, graphical processing units, which was a hardware acceleration that was brought to bear on a method that's based on neural--artificial neural networks. And so they hadn't really demonstrated--neural networks hadn't--their value or potential up until that moment with using GPUs. And the realization was, well, we can just scale out the size of these neural network models, because GPUs allow us that additional compute power needed to train these models. It wasn't really recognized as a great breakthrough moment at the time. It was more incremental. But it crescendoed up into the mid-2010s where it really started to take off. So let me get to the more recent time that you're asking about, even just five years ago. So when I talk about artificial neural networks, and that today is really what identifies what I'll call modern AI, the foundation represented by modern AI is really the neural network methodology. ADAMSON: Okay. DOUCETTE: And there are other methods that have evolved through the decades that are more conventional. The way I would characterize the real difference with a neural network based modeling of learning is that it's more inspired by how humans learn. And so it's not as though it's simulating, you know, a human or mammalian brain. That would be kind of overstating it, because neural nets is essentially a very simple mathematical approach to learning in a way that biological neurons in a human brain, you know, don't do. It's much more complicated in a human brain. So in that sense, it's inspired, as I said. But the big difference is the way we train neural networks is inspired by the way humans learn through that trial and error process I was talking about before. So that's really the big breakthrough. And then just in the last few years, November of 2022, I believe, is when ChatGPT entered upon the cultural discourse of the day. That big breakthrough was led by the concept of the transformer. So that's the T in GPT. And again, transformer is another one of those terms where most people just say, well, you know, what does that mean? Why should I care? And how does it change the way I do business? The transformer is what allows the algorithm to make use of context. That was the big breakthrough. It's an absolutely pivotal concept in the world of machine learning in AI, the ability for the learning algorithm to make use of context. If you think about, for example, reading a novel. So any particular page that you happen to be reading, there's context, you know, of what's going on in that page in the story. But when you get to the end of the story, at the end of the book, what, you know, you read in the first chapter may factor in to understanding the plot of the story. And so that's context that's well beyond just the immediate surroundings, but the full story. So now you understand what happened, or you're able to factor in, what happened in chapter one to understand what happens at the end of the book. And so that's a much more expansive way of thinking about context. And so the word transformer itself, so one might ask, well, what's being transformed? And so what's being transformed is the input data is being made to be more context aware. That's the transformation process. ADAMSON: Okay. So we've covered some of the AI terms and learned about Pete's background and where he's coming from. I'm going to turn now to Rylie Fleckenstein, who works at EROS and is just about to finish his PhD in computer science at Dakota State University in South Dakota, and see how he would define AI. And we'll get into something called deep learning. He also has another interesting take on the context that Pete just talked about. Give me in a quick nutshell, how would you define artificial intelligence? And I know we call it AI. RYLIE FLECKENSTEIN: It's kind of the broad field that encompasses a lot of different subdomains. The two main primary subcomponents of that would be machine learning, which is really essentially using algorithms or statistical models or computers to help learn from data and learn to either make predictions or detect different patterns without explicitly instructing them or explicitly coding them to do that. And then a further subdomain even of machine learning then really is deep learning, which is actually my main passion and main focus, an extension of machine learning where you're using neural networks. So that's really, really been kind of the big explosion in the whole AI/ML boom is, you know, deep learning, these neural networks and the different architectures, and how they're able to be applied to these perception-based problems, which are traditionally extremely difficult to tackle with, you know, more traditional methods. Also, there's a lot more flexibility. When you're doing machine learning, there's like a certain kind of set of standard models and architectures that are primarily leveraged. And there isn't a lot of flexibility in like mutating the algorithm to fit your data or fit your problem. Deep learning, though, is a little bit more like, I would say, more like a box of Legos. And instead of being able to, or being forced to, you know, build, you know, a specific race car or something out of the box, it's really a box of Legos that you can construct in any way that you can think of that actually can better be better fit to your problem. They require a lot more compute, a lot more data, but they have been shown to be able to extract more meaningful information from data as, you know, data volumes increase and outperform traditional machine learning methods. ADAMSON: Okay. Can we talk more about Legos? FLECKENSTEIN: Sure. ADAMSON: This sounds like an interesting way to understand this a little bit better. You can buy a Lego set that does a race car. FLECKENSTEIN: Right. ADAMSON: And it comes with instructions. And you follow the instructions. You go step one, step two, and you put them together. And when you follow the instructions, you end up with a race car. FLECKENSTEIN: With machine learning, you're kind of given, here's a box of Legos. Build this race car because that's what it's supposed to build. Yeah. Where deep learning, it's like, here's Legos and you can kind of build whatever you want. So it's kind of the engineer has more flexibility, right? ADAMSON: Can you tell me more about convolutional neural networks? FLECKENSTEIN: Back to a bit of that Lego analogy. You know, all neural networks are really kind of made up of these little Legos. They're actually referred to as nodes. And you construct these nodes, which are essentially just like elementary tensor operations they call them, or mathematical operations, you know, just pretty elementary mathematical operations. And you construct them into layers and then the layers into modules and then the modules into architectures. And you can kind of construct or customize what this architecture looks like based off of your data. Well, convolutional neural networks are constructed specifically to deal with images to help extract that spatial information from the image. And essentially it's a neural network that's constructed, you know, specifically to be useful for images and utilizing the assumption that, in image data, you know, surrounding pixels have, you know, relevant information from next to each other, right? You know, spatial kind of orientation. ADAMSON: It's not just looking at the single pixel. FLECKENSTEIN: Right. ADAMSON: You know, this is a square, and what is it--if you look around it and you see agriculture-- FLECKENSTEIN: Right. ADAMSON: I mean, that makes sense to us, but you were able to get the algorithm to kind of figure that out, too. FLECKENSTEIN: Yeah. So that's what like the beauty of the, you know, CNNs, you know, the U-Net, which is not something I developed. That's been developed--Ronneberger 2015 is the paper, if you're curious. ADAMSON: Okay. FLECKENSTEIN: Ronneberger et al., they developed it. But yeah, this U-Net, you know, is a convolutional neural network that can pull that information out and that's spatial information, you know, right? So we knew that. And so we incorporated that into our algorithm, you know, to help pull that spatial information. ADAMSON: So we have some terminology taken care of. Machine learning. That's algorithms learning from data to detect patterns in the data. Deep learning, which uses neural networks, and neural networks is machine learning inspired by how humans learn. And then there's transformers. That's the algorithm making use of context. Now let's start talking about applications of AI. How is AI going to help science projects at EROS going forward? DOUCETTE: One of the areas where it will be particularly useful is in the area of data integration. So one way to get more information from remote sensing data is to combine from different sensor types. What we often refer to as multimodal data from different sensors, say, from the electro optical part of the, of the spectrum, the visible part, there's the thermal part. There's the radio part through synthetic aperture radar, for example. These are very different modes of information, which have been traditionally difficult to integrate. And when we do that, we come up with methods such as HLS, Harmonized Landsat Sentinel data, where we wind up resampling one data to look like the other dataset. So what transformers allow more naturally is to integrate multimodal data in a way that allows us to use those datasets in their native forms without having to realign and resample things. Kind of like what happens in, for example, a human or a mammalian brain. So we have our senses, right, that our brain integrates. So you're looking at me and you're listening to me. So there's a visual source of information and an auditory one. Two very different signal types. But the brain puts those together in a very almost seamless way. And we still don't understand quite how the brain does that, but it does it. And so can we take that same concept of integration of very different signal or observations and make sense of it in a more complete way? So that's where I think transformers, the modern AI concept, has the largest potential with where we're going to take our remote sensing science. ADAMSON: And in the future we're expecting Landsat Next, which is going to have even more of those spectral bands. DOUCETTE: Absolutely. ADAMSON: It feels like this will be even more important to be able to understand by the time we get to that. DOUCETTE: Yeah. So I think the future way to message the value of Landsat, if you will, is to demonstrate how we can combine it with other modes of remote sensing data that make it a more valuable product beyond just itself. That's what AI will provide us the ability to do. In theory, it's yet to be demonstrated in a very concrete way. But that's where the future research resides, is the integration aspect of multimodal data using transformers. We should be seeing, you know, an increased amount of research, at least I would encourage research in that particular direction. ADAMSON: Next we talk about another application of AI at EROS. For that I'd like to introduce Terry Sohl. SOHL: Hi, I'm Terry Sohl. I'm the science branch chief for the integrated science and applications branch here at EROS. I've been here about 32 years, and I'm glad to talk to you today. ADAMSON: All right. Thank you. SOHL: So I'm gonna look at the merger that we did recently of the National Land Cover Database and the Land Change Monitoring Assessment and Projection project. So two projects that did land cover, both used forms of deep learning. And I would still call that, you know, some form of AI, although a little bit more rudimentary. When we looked toward the future, we realized that we need to become more efficient. We needed to merge those projects, save some money, create a product faster, create a product with more accuracy. And that's where AI came in. And so we had a very daunting task a couple of years ago to take these very long-standing projects with a very long history and in two years completely revamp them. And I can't say enough about the team at EROS, both the contract and the government side, and how they did that, and that a completely new methodology was stood up, all AI based, linking three different AI models. We're faster, we're more efficient. We've merged those two projects. We've saved the government and the taxpayer money, and we're creating a superior product. So it's a win all the way around. ADAMSON: Now we'll go back to Rylie for some more detail on AI's use in NLCD. FLECKENSTEIN: For the last two years, the LCNext project, which is really the-- we have our legacy projects, the NLCD, the National Land Cover Database, and then the LCMAP, or the Land Change Monitoring Assessment and Projections project. So they essentially merged, it was about two years ago, into our LCNext, or Land Cover Next project. ADAMSON: The product that came out of that at the end of 2024 was Annual NLCD, Annual National Land Cover Database. We're using Landsat data to classify all these different kinds of land cover. FLECKENSTEIN: Yes, my primary job has been working on the classification algorithm for the Annual NLCD product. ADAMSON: What does classification mean real quick? FLECKENSTEIN: Yeah. So classification, you know, just for this particular problem, giving each particular pixel a label, right, a land cover label, given input data of Landsat imagery, essentially. ADAMSON: Classifying each pixel-- FLECKENSTEIN: Yeah, classifying. ADAMSON: --as wetland, as agriculture, or whatever the land cover is. FLECKENSTEIN: Or urban or trees, right. ADAMSON: Yeah, and there are several of them. FLECKENSTEIN: Yeah, yeah. ADAMSON: Okay. And there's trillions of pixels across the U.S. FLECKENSTEIN: Yes. ADAMSON: So you do need some kind of AI to help you do it faster? Otherwise, it would take forever. FLECKENSTEIN: Yeah. So, to looking at the legacy projects a little bit. Legacy NLCD did a phenomenal job of generating these similar maps across the conterminous United States, covering all of CONUS, the same classification legend, the same spatial resolution, but there was somewhat of a product generation latency. It usually took them 2 to 3 years to generate a new map because they relied pretty heavily on, you know, expert interpreters and scientists. They used some machine learning algorithms. But to support that, they also used a lot of like expert interpretation. So, they make an extremely high quality product. But it just took more time, right? It took a couple of years. We want to reduce that latency. We want to get the annual product out there and do it quickly and sufficiently. So that's kind of where, you know, we needed to rely on more advanced deep learning algorithms to try to replicate, you know, that high quality and retain that, you know, spatial cohesiveness and that temporal consistency and really keep that NLCD legacy feel that, you know, we know our customers love and maintain that high level of scientific consistency. And so, yeah, we really needed to leverage some more automated techniques to do it, be able to generate, you know, at an annual cadence. And now, you know, working on updating every year, right. You know, there's actually two components. There's, you know, the change detection. And then there's the classification. And the classification component is--we've coined as LCAMS, or land cover artificial mapping system. And yeah, I was the, the chief designer, you could say, or the lead designer and developer of that algorithm, leveraging, you know, deep learning. We actually, you know, using convolutional neural networks, specifically the U-Net, convolutional neural network and then also transformers as well for the temporal component. So introducing, you know, some of these advanced, spatial and temporal models and ideas into the algorithm to extract that spatial temporal information from the Landsat time series. ADAMSON: If this doesn't sound like a dumb question, why didn't we think of this a few years ago? SOHL: We didn't have the capability. There are definitely, you know, from a staffing perspective, but also from a computational and a resource perspective. It's very data hungry. It's very computationally hungry. And, you know, those resources continue to grow in terms of our access to them. And we need that. And so, to be frank, you know, some of the things that we are doing right now, five years ago, we couldn't. I mean, we just didn't have the computing resources or the technical skill to be able to pull it off. But that's part of the challenge, too. It's a combination of people, the IT resources, and the will to try something new. ADAMSON: It is changing fast. SOHL: It is changing fast. ADAMSON: Are there any other projects besides that one, that are going to benefit from that, too? SOHL: The there are multiple projects that are pursuing AI, and they're linking into some of the work that's been done on NLCD. I'm also looking forward to the next generation of AI in the building. And that's something that Neal Pastick is helping us lead. And that's, you know, right now, traditionally in the building, it's been project by project in terms of how we generate these products. What we're looking for right now is looking toward a new foundation model approach, transformer-based foundation model. That is something that could form the core of many projects. And so the thought is that by having this foundation model as a core, we can put different heads on that model and produce land cover, produce evapotranspiration, produce impervious surface and other datasets. And the advantage of that is from a cost perspective, but from a user perspective, too, it also ensures that all of those products that we generate are very consistent with each other. And that's a huge advantage compared to the every man for himself paradigm that we have right now for our projects. ADAMSON: Okay, it's kind of cutting across different research areas is what it comes down to? SOHL: Yeah, and it has the potential to-- less training data, less compute time, and leveraging work across multiple projects. So it should help us become more efficient. ADAMSON: Terry dropped the name of our fourth guest for this AI episode, Neal Pastick. Neal will tell us more about the foundation models that Terry mentioned. Will you go ahead and introduce yourself? Give us your background. NEAL PASTICK: Yeah, absolutely. Well, first off, Tom, thanks for having me on the show today. My name is Neal Pastick, and I'm a research physical scientist with EROS Data Center. I have been conducting data science research for environmental science applications for the past 15 plus years. So over the past half year or so, I've been developing a suite of geospatial or Earth foundation models. And you can think of these things as large-scale deep learning models pre-trained on huge corpuses of geospatial information. For example, imagery collected from Landsat, Sentinel-2, and the like. And the idea is akin to how large language models operate on text. But instead we're really just working with remote sensing data. So we're crawling the entire archive, trying to exploit patterns therein and using that information to better understand the landscape around us. So it's all about developing and leveraging artificial intelligence and more specifically, machine learning, deep learning, to help land use managers and scientists on the planet do more. ADAMSONS: What else can you say about foundational modeling? How can that be useful to someone who, downstream, is going to use the data that you're working on? PASTICK: Yeah. So there's two sides of that coin there. So if we develop this geospatial foundation model, which is, like I said, pre-trained on huge amounts of geospatial information, it's basically learning a compressed representation of what that data is. So it's a distilled knowledge hub, per se, that can be mined by others. So it's a compressed representation of the dataset. And these compressed weights can then be used for downstream tasks with a little bit of fine tuning and a limited amount of labels. The outputs of these models range from historical and future estimates of surface reflectance. So folks can use those outputs. Say, for example, we have this pesky issue of clouds within optical satellite imagery. These foundation models are really good at gap filling the clouds, making use of patterns that they've seen in other parts of the landscape. So we can gap fill imagery, which is, can be very pivotable for some applications. Another thing would be using those predictions of surface reflectance and turning those into snow abundance maps at a weekly and/or finer temporal resolution. So not only are the model weights useful, but the outputs from said model can be super useful as well. ADAMSON: Okay, so what kinds of things can that do? PASTICK: What's really exciting about these models is they're designed to be incredibly effective, even in the face of limited labeled data. And we are in a position as the land change monitoring mecca of the world, per se. We don't have a lot of good labeled data describing the landscape. So once we train these foundation models, they're really highly reusable tools that you can then use or repurpose for a bunch of different applications. So case in point, we've developed geospatial foundation models trained on Landsat and Sentinel-2 data. And we use those features inherent within the model for mapping burn scars and progression, forecasting invasive species cover in the western United States, monitoring snow depth abundance as well as snow water equivalent. And all that information is pivotal--pivotable, or pivotable, for land use managers and water resource managers. So even if they don't have the technical prowess to actually use the model themselves, the outputs are super important for them. But this also means once we stand these models up, researchers and users don't really have to start from scratch. They can leverage the weights within these models to kind of speed up deployment of solutions. In short, we're basically making a cost effective tool for gaining insights from geospatial information. ADAMSON: We're going to end part one of this episode of Eyes on Earth right there. We learned about machine learning, deep learning, neural networks, transformers, and now we can add foundational models to that list. These models are pre-trained on huge amounts of data, in our case geospatial data, and can be used like reusable tools for lots of applications. And users don't need to hash through all that data from scratch. And we talked about a few science applications of AI. In part two we'll discuss how AI could help us fly the Landsat satellites, and we'll explore the challenges and benefits of using AI. Check out our social media accounts to watch for that and all future episodes. You can also subscribe to us on Apple and YouTube Podcasts. VARIOUS VOICES: This podcast, this podcast, this podcast, this podcast, this podcast is a product of the U.S. Geological Survey, Department of Interior.