Astronomical data is the oldest data collected by humankind. The oldest known systematic astronomical observations came from the Babylonians in 1000 BCE. In the second century BCE, the first stellar catalog was compiled by Greek astronomer Hipparchus of Nicaea, which would eventually make it into the hands of Roman astronomer Ptolemy three centuries later.
Old data can be extremely valuable when studied with modern tools. And, when it comes to the cosmos, we have a whole lot of data collected in extensive sky surveys over the last few decades.
Google Cloud worked with researchers from The Asteroid Institute, a program of the non-profit B612 Foundation, to apply modern cloud computing tools and algorithms to old cosmological datasets to find and map asteroids in our solar system. The Asteroid Institute announced today that the new method has found and validated 104 new asteroids, opening the possibility of using this method to find hundreds or thousands of new asteroids hiding in old datasets.
“We’ve proven a new, computationally driven method of discovering asteroids,” said Dr. Ed Lu, Executive Director of the Asteroid Institute and former NASA astronaut, in a conversation with Vint Cerf, VP & Chief Internet Evangelist at Google about the new computation method for discovering asteroids. “The Minor Planet Center confirmed and added these newly discovered asteroids to its registry … opening the door for Asteroid Institute-supported researchers to submit thousands of additional new discoveries. This has never been done before, since previous methods have relied on specific telescopic survey operations. Instead, we’ve been able to identify and track asteroids using enormous computational power.”
ADAM and THOR: Hunting the sky together
The Asteroid Institute employs a cloud-based astrodynamics platform called the Asteroid Discovery Analysis and Mapping (ADAM) to hunt for asteroids in our solar system. ADAM is an open-source computational system that runs astrodynamics algorithms at massive scale in Google Cloud.
The novel algorithm used to discover these new asteroids is called Tracklet-less Heliocentric Orbit Recovery (THOR). The algorithm links points of light in different sky images that are consistent with asteroid orbits, trying to identify the orbits of asteroids that can be seen in old data. As such, THOR does not need a telescope to observe the sky to look for asteroids, but rather can hunt through old images to look for the familiar patterns. THOR can identify asteroids and calculate their orbits meeting criteria by the Minor Planet Center to recognize them as tracked asteroids.
For its initial demonstration, researchers from the Asteroid Institute and the University of Washington searched a 30-day window of images from the NOIRLab Source Catalog (NSC), a collection of nearly 68 billion observations taken by the National Optical Astronomy Observatory telescopes between 2012 and 2019.
Finding and tracking asteroids is a difficult problem for a few reasons. Space is huge, even when we know where we are supposed to look. The relative luminosity (the intrinsic brightness of a celestial object) is low in comparison to planets in our solar system or faraway stars. Also, the Earth is constantly moving through the solar system, as are the asteroids. Linking different images of the sky to previous signals in the data increases the variables and makes the entire endeavor extremely computationally expensive when dealing with large datasets.
“First, linking asteroid detections across multiple nights is difficult due to the sheer number of possible linkages, made even more challenging by the presence of false positives,” researchers wrote in a paper onthe introduction of THOR in May 2021. “Second, the motion of the observer makes the linking problem nonlinear as minor planets will exhibit higher order motion on the topocentric sky over the course of weeks.”
The way asteroid detection has been performed before was to use surveys from terrestrial telescopes and look for the tracks made by the asteroids that showed up in the data in multiple observations over time. This required time on expensive telescopes that often have other priorities, and a regular cadence of observations. It’s an inefficient and time consuming process. What THOR does is try to remove the need to rely on the asteroid tracklets in the data (thus “Tracklet-less”) and narrow the scope of the observation field to a dynamically selected series of “test orbits.” By focusing on a specific area through a portion of the NSC dataset (only 0.5%), the data becomes more linear and easier to manage, picking out potential asteroids with clustering and line-detection algorithms.
“This provides a path to scanning an otherwise voluminous 6D phase space with a finite number of test orbits and at feasible computational cost,” researchers wrote.
One of the primary benefits of the THOR approach is that it can be used on datasets beyond the scope of just one celestial survey, opening up the possibility of investigating a diverse array of datasets and making connections between them. Thus, THOR becomes an independent agent from the observer (the telescope survey data) and able to make connections between different datasets from different times.
One interesting note from THOR is that it is not based on machine learning or neural networks. The power of THOR is in taking data from the test orbits and applying statistics plus physics, through linear and clustering algorithms and then utilizing a massive amount of computational power through the cloud. One area where machine learning can be applied in the future will be in making the search through the infinite possible orbits smarter and faster, in the same way that DeepMind’s AlphaGo algorithm efficiently determines which move to make next on a Go board among a very large number of possibilities.
Scaling ADAM with Google Cloud
THOR runs on the ADAM, which is The Asteroid Institute’s astrodynamics-as-a-service platform powered by Google Cloud, including the scalable computational and storage capabilities in Compute Engine, Cloud Storage, and Google Kubernetes Engine.
In 2019 and 2020, Google Cloud’s Office of the CTO conducted architectural sessions and organized hackathons with solution architects at Google Cloud to find the best way to implement ADAM on Google Cloud at scale. Google Cloud also provided cloud credits and technical support for the ADAM platform’s current development and future work.
“The potential of this software ecosystem also stretches far beyond historical data – with additional development, ADAM::THOR will be able to perform real-time asteroid discovery on observations as they come in from telescopes around the globe,” said Joachim Moeyens, co-creator of THOR and an Asteroid Institute Fellow
While THOR makes digging through existing astronomical datasets computationally feasible, it is important to consider that the data consists of a massive amount of space, searching for continuous orbits. It’s still a very large number of candidate orbits that need to be searched and thus requires the type of storage capacity and computing resources that only the public cloud can provide. The raw astronomical data is stored in Google Cloud Storage, with refined data later abstracted into BigQuery. Google Cloud enables ADAM to scale by allowing it to run on thousands of machines simultaneously, which results in the ability to analyze the data in a reasonable amount of time.
One of the most powerful aspects of the partnership between B612 and Google Cloud is to make ADAM a “discovery-as-a-service” platform available for future researchers.
“With the arrival of the public cloud, researchers can now get access to large computational resources when they need them with much less overall cost and time penalty,” said Cerf. “And we’re seeing that large data sets can be applied to new problems.”
Cloud BlogRead More
