There's always going to be that next tool.
You know,
when I first started 32 years ago, it was to help start the first NLCD.
And so I've seen that progression of methodologies over time.
There's always going to be that next best thing.
AI is the next best thing now.
Five years from now, I'm sure it's
going to be AI-based, but it may be completely different.
And so I think that's the thing that is always the case in science.
Always staying on top of the game, looking at
what's happening in the field and always adapting.
Hello
everyone, and welcome to another episode of Eyes on Earth,
a podcast produced at the USGS EROS Center.
Our podcast focuses on our ever changing planet
and on the people here at EROS and across the globe, who use remote
sensing to monitor and study the health of Earth.
My name is Tom Adamson.
Artificial intelligence, also known as AI,
is quickly becoming a necessary part of geospatial work at EROS.
I had a conversation with EROS Director Pete
Doucette about AI and its uses here at EROS.
I also spoke to others at EROS who are using AI in their work
to help clarify what it all means and what its benefits and challenges are.
Fair warning to listeners.
We did have to use some of the jargon surrounding AI, like machine learning,
deep learning, and convolutional neural networks.
Now don't be intimidated.
It's a new technology and a very big change we're going through.
Pete encourages us to be open to the new technology
and its great potential
to helping us at EROS make the work we do better, more efficient, and more useful.
And besides, at EROS,
as the home of the Landsat archive, Landsat data's quality, consistency,
and continuity are optimal for AI exploitation.
So let's get to it.
We'll start with EROS Director Pete Doucette.
Last summer, we did an
Eyes on Earth episode with a USGS scientist about mapping the California coastline.
He described AI like this.
You're teaching a computer to learn by example.
Is this a fair way to describe AI, or is there more to it than that?
I think Sean's characterization was a useful way
to think about AI in terms of a concise way to put it.
I think to better understand the bigger picture
of what AI is, because a lot of people are still wondering,
What is it really?
It's useful to understand the different phases of its evolution.
So the term AI was initially coined back in 1956.
At that time, what kind of dominated
the AI methodologies, if you will, had more to do with
rules-based systems, for example, creating a set of rules
that an algorithm would just, would just carry out.
And so that way of thinking of AI dominated for probably
a couple of decades until we entered into the machine learning,
which is kind of what Sean's alluding to here.
Learn by example.
Up until that point, it's really just
computers are doing things faster than humans can.
Yes, in a way, depending on what the application was.
So, oftentimes applications were very targeted
and specific, not very flexible, but they were effective.
Okay.
If we think about learning, say to, to ride a bike, okay.
You can learn to ride a bike from a manual,
a user's guide, on how to ride a bike.
And so that's kind of a knowledge-based way of thinking about the early days
of AI.
A more effective way to learn to ride a bike is actually getting on it
through trial and error, you know, balancing and falling down
over and over again until you figure out how to balance, is more in line with
the machine learning way in that you're taking in the information
and learning from the information that you take in, in real time.
And so that's kind of the, I guess, metaphor I would use to describe
what AI was in the past, essentially a manual for learning
how to do something versus a trial and error way of learning it.
So what is your background?
How did you get interested in AI and
promoting its use among geospatial professionals?
So my first career job was with Defense Mapping Agency,
which is now known as National Geospatial Intelligence Agency, or NGA.
The mission of DMA, which hasn't changed,
it's basically to map the planet, largely from
satellite data, that part of the planet that the USGS doesn't map.
So the USGS is charged with mapping the United States
and NGA has the rest of the planet.
So back in those early days, the mapping process
was conducted through digitizing or tracing features
from imagery, oftentimes from from satellite imagery.
And the business of mapmaking transitioned from
using analog film-based products, right, to digital products.
That was a major transition for the mapmaking process.
And that's roughly when I entered into that scene of mapmaking.
And NGA was transitioning. With digital imaging,
of course, that afforded you all kinds of computer vision based methods
to assist in, you know, digitizing or tracing out features
such as, you know, roads, buildings, rivers, these kinds of things.
And so,
it was a course that I took at Purdue in 1993,
if I recall, that was called Engineering Aspects of Remote Sensing.
It was taught by Dr. David Landgrebe,
who was actually one of the pioneers on using Landsat data.
And at that time, he was one of the forward thinkers on
how to extract information from multispectral scanner, MSS, data
back in the day.
And he used that data in his course as part of the, as part of the,
the labs and such.
And so, that was my kind of introduction
to multispectral data analysis through David Landgrebe's course.
Curious that his last name was Landgrebe, right,
using Landsat data.
It was, you know, a heck of a coincidence there.
But that's what first got me kind of intrigued about
using computer vision methods to automate
the process of digitizing features
or characterizing features from multispectral data.
We're talking a lot about AI right now,
but it really seems like there's a lot more buzz about it
now than even five years ago. Why is that?
There have been many pivotal moments through the history of AI.
One of them occurred around 2010,
which was the use of GPUs, graphical processing units,
which was a hardware acceleration that was brought to bear
on a method that's based on neural-- artificial neural networks.
And so they hadn't really demonstrated-- neural networks hadn't--their value
or potential up until that moment with using GPUs.
And the realization was, well, we can just scale out
the size of these neural network models, because GPUs allow us
that additional compute power needed to train these models.
It wasn't really recognized as a great breakthrough moment at the time.
It was more incremental.
But it crescendoed up into the mid-2010s
where it really started to take off.
So let me get to the more recent time that you're asking about,
even just five years ago.
So when I talk about artificial neural networks,
and that today is really what identifies what I'll call modern AI, the foundation
represented by modern AI is really the neural network methodology.
Okay.
And there are other methods that have evolved through the decades
that are more conventional.
The way I would characterize the real difference
with a neural network based modeling of learning
is that it's more inspired by how humans learn.
And so it's not as though it's simulating, you know, a human or mammalian brain.
That would be kind of overstating it, because neural nets
is essentially a very simple mathematical approach to learning
in a way that biological neurons in a human brain, you know,
don't do. It's
much more complicated in a human brain.
So in that sense, it's inspired, as I said.
But the big difference is the way
we train neural networks is inspired by the way humans
learn through that trial and error process I was talking about before.
So that's really the big breakthrough.
And then just in the last few years, November of 2022, I believe, is
when ChatGPT entered upon
the cultural discourse of the day.
That big breakthrough was led by
the concept of the transformer. So that's the T in GPT.
And again, transformer is another one of those terms
where most people just say, well, you know, what does that mean?
Why should I care?
And how does it change the way I do business?
The transformer is what allows
the algorithm to make use of context.
That was the big breakthrough.
It's an absolutely pivotal concept in the world of machine learning in AI,
the ability for the learning algorithm to make use of context.
If you think about, for example, reading a novel.
So any particular page that you happen to be reading, there's context,
you know, of what's going on in that page in the story.
But when you get to the end of the story, at the end of the book, what, you know,
you read in the first chapter may factor in to understanding the plot of the story.
And so that's context
that's well beyond just the immediate surroundings, but the full story.
So now you understand what happened, or you're able to factor in,
what happened in chapter one to understand what happens at the end of the book.
And so that's a much more expansive way of thinking about context.
And so the word transformer itself,
so one might ask, well, what's being transformed?
And so what's being transformed is the input
data is being made to be more context aware.
That's the transformation process.
Okay.
So we've covered some of the AI terms
and learned about Pete's background and where he's coming from.
I'm going to turn now to Rylie Fleckenstein, who works at EROS
and is just about to finish his PhD in computer science
at Dakota State University in South Dakota, and see how he would define AI.
And we'll get into something called deep learning.
He also has another interesting take on the context that Pete just talked about.
Give me in
a quick nutshell, how would you define artificial intelligence?
And I know we call it AI. It's kind of the broad field
that encompasses a lot of different subdomains.
The two main primary subcomponents of that would be machine learning,
which is really essentially
using algorithms or statistical models or computers to help learn from data
and learn to either make predictions or detect different patterns
without explicitly instructing them or explicitly coding them to do that.
And then a further subdomain even of machine learning then really is
deep learning, which is actually my main passion and main
focus, an extension of machine learning where you're using neural networks.
So that's really, really been kind of the big explosion in the whole
AI/ML boom is, you know, deep learning, these neural networks
and the different architectures,
and how they're able to be applied to these perception-based problems,
which are traditionally extremely
difficult to tackle with, you know, more traditional methods.
Also, there's a lot more flexibility. when you're doing machine learning,
there's like a certain kind of set of standard
models and architectures that are primarily leveraged.
And there isn't a lot of flexibility
in like mutating the algorithm to fit your data or fit your problem.
Deep learning,
though, is a little bit more like, I would say, more like a box of Legos.
And instead of being able to, or being forced to, you know,
build, you know, a specific race car or something out of the box,
it's really a box of Legos that you can construct in any way
that you can think of that actually can better be better fit to your problem.
They require a lot more compute, a lot more data, but they have been shown
to be able to extract more meaningful information from data
as, you know, data volumes increase and outperform
traditional machine learning methods. Okay.
Can we talk more about Legos?
Sure. This sounds like an interesting way to understand this
a little bit better. You can buy a Lego set that does a race car. Right.
And it comes with instructions.
And you follow the instructions.
You go step one, step two, and you put them together.
And when you follow the instructions, you end up with a race car.
With machine learning, you're kind of given, here's a box of Legos.
Build this race car because that's what it's supposed to build.
Yeah. Where deep learning,
it's like, here's Legos and you can kind of build whatever you want.
So it's kind of the engineer has more flexibility, right?
Can you tell me more about convolutional neural networks?
Back to a bit of that Lego analogy.
You know, all neural networks are really kind of made up of these little Legos.
They're actually referred to as nodes.
And you construct these nodes, which are essentially just like
elementary tensor operations they call them,
or mathematical operations, you know, just pretty elementary
mathematical operations.
And you construct them into layers and then the layers into modules
and then the modules into architectures. And you can kind of construct
or customize what this architecture looks like based off of your data.
Well, convolutional neural networks are constructed specifically
to deal with images to help extract that spatial information from the image.
And essentially it's a neural network that's constructed, you know, specifically
to be useful for images and utilizing the assumption
that, in image data, you know, surrounding pixels
have, you know, relevant information from next to each other, right?
You know, spatial kind of orientation.
It's not just looking at the single pixel. Right.
You know, this is a square,
and what is it--if you look around it and you see agriculture-- Right.
I mean, that makes sense to us,
but you were able to get the algorithm to kind of figure that out, too.
Yeah.
So that's what like the beauty of the,
you know, CNNs, you know, the U-Net, which is not something I developed.
That's been developed--
Ronneberger 2015 is the paper, if you're curious.
Okay. Ronneberger et al.,
they developed it.
But yeah, this U-Net, you know, is a convolutional neural network
that can pull that information out and that's spatial information, you know,
right? So we knew that.
And so we incorporated that into our algorithm,
you know, to help pull that spatial information.
So we have some terminology taken care of.
Machine learning.
That's algorithms learning from data to detect patterns in the data.
Deep learning, which uses neural networks,
and neural networks is machine
learning inspired by how humans learn.
And then there's transformers.
That's the algorithm making use of context.
Now let's start talking about applications of AI.
How is AI going to help science projects at EROS going forward?
One of the areas where it will be particularly useful
is in the area of data integration.
So one way to get more information
from remote sensing data is to combine from different sensor types.
What we often refer to as multimodal data from
different sensors, say, from the electro optical part of the,
of the spectrum, the visible part, there's the thermal part.
There's the radio part through
synthetic aperture radar, for example.
These are very different modes of information,
which have been traditionally difficult to integrate.
And when we do that, we come up with methods
such as HLS, Harmonized Landsat Sentinel data, where we wind up
resampling one data to look like the other dataset.
So what transformers allow more naturally is to integrate
multimodal data in a way that allows us to use those datasets
in their native forms without having to realign and resample things.
Kind of like what happens in, for example, a human or a mammalian brain.
So we have our senses, right, that our brain integrates.
So you're looking at me and you're listening to me.
So there's a visual source of information and an auditory one.
Two very different signal types.
But the brain puts those together in a very almost seamless way.
And we still don't understand quite how the brain does that, but it does it.
And so can we take that same concept
of integration of very different signal or observations
and make sense of it in a more complete way?
So that's where I think transformers, the modern AI concept,
has the largest potential with where we're going to take
our remote sensing science. And in the future we're expecting
Landsat Next, which is going to have even more of those spectral bands.
Absolutely.
It feels like this will be even more important
to be able to understand by the time we get to that. Yeah.
So I think the future way to message the value of Landsat, if you will,
is to demonstrate how we can combine it
with other modes of remote sensing data
that make it a more valuable product beyond just itself.
That's what AI will provide us the ability to do.
In theory, it's yet to be demonstrated in a very concrete way.
But that's where the future research resides,
is the integration aspect
of multimodal data using transformers.
We should be seeing, you know, an increased amount of research,
at least I would encourage research in that particular direction.
Next we talk about another application of AI at EROS.
For that I'd like to introduce Terry Sohl.
Hi, I'm Terry Sohl.
I'm the science branch chief
for the integrated science and applications branch here at EROS.
I've been here about 32 years, and I'm glad to talk to you today.
All right. Thank you.
So I'm gonna look at
the merger that we did recently of the National Land Cover Database
and the Land Change Monitoring Assessment and Projection project.
So two projects that did land cover, both used forms of deep learning.
And I would still call that,
you know, some form of AI, although a little bit more rudimentary.
When we looked toward the future, we realized that we need to become more efficient.
We needed to merge those projects, save some money,
create a product faster, create a product with more accuracy.
And that's where AI came in.
And so we had a very daunting task
a couple of years ago to take these very long-standing projects
with a very long history and in two years completely revamp them.
And I can't say enough about the team at EROS,
both the contract and the government side, and how they did that, and that
a completely new methodology was stood up,
all AI based, linking three different AI models.
We're faster, we're more efficient.
We've merged those two projects.
We've saved the government
and the taxpayer money, and we're creating a superior product.
So it's a win all the way around.
Now we'll go back to Rylie for some more detail
on AI's use in NLCD. For the last two years,
the LCNext project,
which is really the-- we have our legacy projects,
the NLCD, the National Land Cover Database, and then the
LCMAP, or the Land Change Monitoring Assessment Projections project.
So they essentially merged, it was about two years ago, into our LCNext,
or Land Cover Next project. The product that came out of that at the end of 2024
was Annual NLCD, Annual National Land Cover Database.
We're using Landsat data to classify all these different kinds of land cover.
Yes, my primary job has been working on the classification algorithm
for the Annual NLCD product.
What does classification mean real quick?
Yeah.
So classification, you know,
just for this particular problem, giving each particular pixel a label, right,
a land cover label, given input data of Landsat imagery, essentially.
Classifying each pixel-- Yeah, classifying.
--as wetland, as agriculture, or whatever the land cover is. Or urban or trees, right.
Yeah, and there are several of them.
Yeah, yeah. Okay.
And there's trillions of pixels across the U.S. Yes.
So you do need some kind of AI to help you do it faster?
Otherwise, it would take forever.
Yeah.
So, to looking at the legacy projects a little bit. Legacy NLCD
did a phenomenal job of generating
these similar maps across the conterminous United States, covering all of CONUS,
the same classification legend, the same spatial resolution,
but there was somewhat of a product generation latency.
It usually took them 2 to 3 years to generate a new map
because they relied pretty heavily on, you know, expert interpreters
and scientists.
They used some machine learning algorithms.
But to support that, they also used a lot of like expert interpretation.
So, they make an extremely high quality product.
But it just took more time, right?
It took a couple of years. We want to reduce that latency.
We want to get the annual product out there
and do it quickly and sufficiently.
So that's kind of where, you know,
we needed to rely on more advanced deep learning algorithms
to try to replicate, you know, that high quality and retain
that, you know, spatial cohesiveness and that temporal consistency
and really keep that NLCD legacy feel that, you know, we know our customers love
and maintain that high level of scientific consistency.
And so, yeah, we really needed to leverage some more automated techniques to do it,
be able to generate, you know, at an annual cadence.
And now, you know, working on updating every year, right.
You know, there's actually two components.
There's, you know, the change detection.
And then there's the classification.
And the classification component is-- we've coined
as LCAMS, or land cover artificial mapping system.
And yeah, I was the, the chief designer,
you could say, or the lead
designer and developer of that algorithm, leveraging, you know, deep learning.
We actually, you know, using convolutional neural networks, specifically
the U-Net, convolutional neural network and then also
transformers as well for the temporal component.
So introducing, you know, some of these advanced, spatial
and temporal models and ideas into the algorithm to extract
that spatial temporal information from the Landsat time series.
If this doesn't
sound like a dumb question, why didn't we think of this a few years ago?
We didn't have the capability.
There are definitely, you know, from a staffing perspective,
but also from a computational and a resource perspective.
It's very data hungry.
It's very computationally hungry.
And, you know, those resources
continue to grow in terms of our access to them.
And we need that.
And so, to be frank, you know,
some of the things that we are doing right now, five years ago,
we couldn't. I mean, we just didn't have
the computing resources or the technical skill to be able to pull it off.
But that's part of the challenge, too.
It's a combination of people,
the IT resources, and the will to try something new.
It is changing fast. It is changing fast.
Are there any other projects besides that one, that are going to
benefit from that, too?
The there are multiple projects that are pursuing AI,
and they're linking into some of the work that's been done on NLCD.
I'm also looking forward to the next generation of AI in the building.
And that's something that Neal Pastick is helping us lead.
And that's, you know, right now, traditionally in the building, it's
been project by project in terms of how we generate these products.
What we're looking for right now is looking toward a new foundation
model approach, transformer-based foundation model.
That is something that could form the core of many projects.
And so the thought is that by having this foundation model as a core,
we can put different heads on that model and produce land cover, produce
evapotranspiration, produce impervious surface and other datasets.
And the advantage of that is from a cost perspective,
but from a user perspective, too, it also ensures that all of those products
that we generate are very consistent with each other.
And that's a huge advantage compared to the
every man for himself paradigm that we have right now for our projects.
Okay,
it's kind of cutting across different research areas is what it comes down to?
Yeah, and it has the potential to-- less training data, less compute
time, and leveraging work across multiple projects.
So it should help us become more efficient.
Terry dropped the name of our fourth guest for this
AI episode, Neal Pastick. Neal will tell us
more about the foundation models that Terry mentioned.
Will you go ahead and introduce yourself?
Give us your background.
Yeah, absolutely.
Well, first off, Tom, thanks for having me on the show today.
My name is Neal Pastick, and I'm a research physical scientist
with EROS Data Center.
I have been conducting data science research
for environmental science applications for the past 15 plus years.
So over the past half year or so, I've been developing a suite
of geospatial or Earth foundation models.
And you can think of these things as large-scale deep learning models
pre-trained on huge corpuses of geospatial information.
For example, imagery collected from Landsat, Sentinel-2, and the like.
And the idea is akin to how large language models operate on text.
But instead we're really just working with remote sensing data.
So we're crawling the entire archive, trying to exploit patterns therein
and using that information to better understand the landscape around us.
So it's all about developing and leveraging artificial intelligence
and more specifically, machine learning, deep learning,
to help land use managers and scientists on the planet do more.
What else can you say about foundational modeling?
How can that be useful to someone who, downstream, is going to use the data
that you're working on?
Yeah.
So there's two sides of that coin there.
So if we develop this geospatial foundation model, which is,
like I said, pre-trained on huge amounts of geospatial information,
it's basically learning a compressed representation of what that data is.
So it's a distilled knowledge hub, per se,
that can be mined by others.
So it's a compressed representation of the dataset.
And these compressed weights can then be used for downstream tasks
with a little bit of fine tuning and a limited amount of labels.
The outputs of these models range from historical
and future estimates of surface reflectance.
So folks can use those outputs.
Say, for example,
we have this pesky issue of clouds within optical satellite imagery.
These foundation models are really good at gap filling the clouds,
making use of patterns that they've seen in other parts of the landscape.
So we can gap fill imagery, which is, can be very pivotable
for some applications.
Another thing would be using those predictions of surface reflectance
and turning those into snow abundance maps at a weekly and/or
finer temporal resolution.
So not only are the model weights useful,
but the outputs from said model can be super useful as well.
Okay, so what kinds of things can that do?
What's really exciting about these models is they're designed
to be incredibly effective, even in the face of limited labeled data.
And we are in a position as the land change
monitoring mecca of the world, per se.
We don't have a lot of good labeled data
describing the landscape.
So once we train these foundation models,
they're really highly reusable tools
that you can then use or repurpose for a bunch of different applications.
So case in point, we've developed geospatial foundation
models trained on Landsat and Sentinel-2 data.
And we use those features inherent within the model
for mapping burn scars and progression, forecasting
invasive species cover in the western United States, monitoring
snow depth abundance as well as snow water equivalent.
And all that information is pivotal-- pivotable, or pivotable,
for land use managers and water resource managers.
So even if they don't have the technical prowess to actually use
the model themselves, the outputs are super important for them.
But this also means once we stand these models
up, researchers and users don't really have to start from scratch.
They can leverage the weights within these models to kind of speed up
deployment of solutions.
In short, we're basically making a cost effective tool
for gaining insights from geospatial information.
We're going to
end part one of this episode of Eyes on Earth right there.
We learned about machine learning, deep learning,
neural networks, transformers,
and now we can add foundational models to that list.
These models are pre-trained on huge amounts of data, in our case
geospatial data, and can be used like reusable tools for lots of applications.
And users don't need to hash through all that data from scratch.
And we talked about a few science applications of AI.
In part two we'll discuss how AI could help us fly the Landsat satellites,
and we'll explore the challenges and benefits of using AI.
Check out our social media accounts to watch for that and all future episodes.
You can also subscribe to us on Apple and YouTube
Podcasts.
This podcast, this podcast, this podcast, this podcast,
this podcast is a product of the U.S. Geological Survey, Department of Interior.