The Big Picture
Thanks to Caltech graduate students in Pietro Perona’s computer vision lab, the next wildlife photo you snap might set you on a path to helping map life on Earth.
Breakthroughs ensue when creative, talented researchers are given means and freedom in a culture of curiosity-driven exploration. At Caltech, breakthroughs aren’t just the be-all and end-all; they are also the begin-all.
Thanks to Caltech graduate students in Pietro Perona’s computer vision lab, the next wildlife photo you snap might set you on a path to helping map life on Earth.
Integral membrane proteins (IMPs): They’re unquestionably important and notoriously elusive.
Caltech biochemistry professor Bil Clemons is on a mission to take some of the guesswork out of how we study them.
At the interface of behavioral economics, neuroscience, psychology, and psychiatry, Caltech postdoc Caroline Charpentier is elucidating how we interpret the actions of others, arrive at decisions, and resolve uncertainties.
Heads nodding, hands going up, and eyes widening in wonder: For Caltech graduate student Arian Jadbabaie, these are indications that “science just may be blowing their minds.”
We asked members of the Caltech community to give examples of what can be achieved when you put more people on the task.
Six Planets in 48 Hours
“Through a citizen science project called Exoplanet Explorers, we asked for help sifting through data: thousands of potential signals from planets orbiting other stars captured by NASA’s Kepler Space Telescope. In April 2017, the project was featured on the television show Stargazing Live, and within 48 hours volunteers had identified a star hosting at least six planets between the size of Earth and Neptune. This was a fascinating discovery because we have no planets in our solar system like them!”
Jessie Christiansen, Scientific Research Associate, Infrared Processing and Analysis Center
Ask for Directions
“No matter who you are or where you are in your career, make connections with other people. Look up, down, all around you—and network. Having been involved in academic research, industry startups, and science policy, I have seen that there are many avenues to success. A community of professionals beyond your immediate circle can help you glimpse what is possible. And if you are a Caltech student or alumnus, join the Caltech Alumni Advisors Network (CAAN), an online mentoring platform where you can meet remarkable people who are accomplishing countless amazing things.”
Christie Canaria (PhD ’08), Member, Caltech Alumni Association Board of Directors
Positive Associations
“My mom has always told me that I have what it takes to do anything, and when I got to Caltech, all of a sudden I became part of a community that tells me the same thing. The scholarship I received—which was established not by one individual, but by many members of the Caltech Associates—is an extension of the encouragement I receive from those around me. The financial support means a lot, and so does knowing that there are people out there who believe in me. They believe that my education is a good investment, and through my hard work I will prove them right.”
Hana Keller (Class of 2019), Mechanical Engineering
Which Is a Glitch?
“Thousands of volunteers are helping to advance the work of the Laser Interferometer Gravitational-Wave Observatory (LIGO), which is led by Caltech. Through LIGO’s citizen science project Gravity Spy, funded by the National Science Foundation and headed by Scott Coughlin and a team at Northwestern University, citizen scientists are helping us improve LIGO’s astrophysical reach as it detects ripples in spacetime. Thus far, 13,000 participants have helped classify more than 600,000 ‘glitches,’ caused by a variety of sources, including ground vibrations, wind, and the electronics that are used to measure the signal itself. This work is helping us identify the telltale traces of gravitational waves.”
Jess McIver, Senior Postdoctoral Scholar in Physics, Caltech LIGO Laboratory
STEM on Stems
“Students from Pasadena and Los Angeles public schools lent a hand—actually, 150 hands—to help with a computer program I developed for Elliot Meyerowitz and his team to delineate plant stem cells from microscope images. As part of a larger lesson in plant biology, we taught the students how to manually guide the program with a mouse to create more accurate cell outlines. The students’ delineations enabled us to test the code we developed to segment images. Now, in Caltech’s general spirit of goodwill, other labs can benefit from our work. We are also using the students’ delineations to train computer programs to generate outlines better and faster.”
Alexandre Cunha, Director, Center for Advanced Methods in Biological Image Analysis, and Computational Scientist, Center for Data-Driven Discovery
Thanks to Caltech graduate students in Pietro Perona’s computer vision lab, the next wildlife photo you snap might set you on a path to helping map life on Earth.
“In the future,” says Pietro Perona, “you could point your phone at a rash or a mole on your skin and the phone would tell you, ‘Go see a doctor,’ or ‘Take another picture tomorrow and let’s see where this goes.’ You would have peace of mind.”
That’s just one of many possibilities that Perona, Caltech’s Allen E. Puckett Professor of Electrical Engineering, sees on the horizon. He and an alumnus of his lab, Serge Belongie (BS ’95), and their research groups are building Visipedia. A tool that blends human expertise, computer vision, and machine learning, Visipedia will help anyone learn about anything—just by uploading a picture of it. The project epitomizes the scale of innovation in Caltech’s CS+X initiative, which marries groundbreaking science with groundbreaking computing.
“The whole web, this huge repository of wonderful information, is indexed by words,” Perona says. “But when we have an image—a visual query—we don’t know what to do unless there is an expert next to us. We’ve gotten so numb to the idea that we’ll never find the answer out.”
Already, we can get some answers. With funding from Google and partners at iNaturalist, Tree People, Cornell Lab of Ornithology, and Cornell Tech, Perona has made it possible for people to identify plants and animals, be better birders, and catalog urban trees. He has even freed grad students from days of observing fly behavior.
“How far can we go?” Perona asks. “I’ll tell you. Since I am at Caltech and we get the super-best students in the world, my ambition is huge. I want my phone to become an expert in my pocket that will tell me about any object I might find.”
Caltech built photo recognition into the Merlin Bird ID and iNaturalist apps, which help people instantly identify wildlife and plants—and contribute to human knowledge about them. To do this, Grant Van Horn, who just completed his graduate studies in the Perona lab, worked closely with the groups that created these apps. He developed algorithms that identify birds and other species in uploaded photos. Then he drew on more photos and people’s expertise to improve accuracy. Van Horn sees birds as a perfect testbed for future progress in machine vision and learning. Why? Birds vary subtly in looks, songs, and behaviors—and they have a large human fan base that contributes tons of data.
Biologists worldwide set cameras outdoors to collect image-based data about animal populations and biodiversity. These cameras take pictures automatically when they detect motion or body heat—day or night, rain or shine. Researchers then toil to sort the resulting photos—many of them dark, blurry, or empty of animals. Now, Perona-lab grad student Sara Beery is training algorithms to identify animals in these pictures. (See how a model she trained to detect animals and mark them with red boxes performed with test images from Southern California and around the world.) Thanks to connections she made when alumni support enabled her to attend the Grace Hopper Celebration, Beery teamed up earlier this year with engineers at Microsoft AI for Earth to integrate her algorithms into forthcoming software for biologists.
Integral membrane proteins (IMPs): They’re unquestionably important and notoriously elusive.
Caltech biochemistry professor Bil Clemons is on a mission to take some of the guesswork out of how we study them.
The cell membrane, just a few nanometers thick, is what separates the interior of a cell from the cell’s environment. Integral membrane proteins, as their name suggests, are embedded in cell membranes. In contact with both the cell’s interior and everything around the cell, IMPs are gatekeepers, transporters, and conduits of information, and they enable cells to communicate with each other.
Because of their complicated habitat—partway in the cell, partway out, and partly in the cell membrane—researchers find it extremely difficult to successfully extract IMPs for study. Typically, scientists study an IMP by inserting its DNA into bacteria. Then they cross their fingers and hope that the bacteria will make the protein, a process analogous to farming. But the process is hit and miss—mostly miss. Eight or nine times out of 10, the bacteria simply don’t produce enough IMP to be useful.
About half of the drugs on the market today—from cancer treatments to blood pressure medications—target specific IMPs, since they’re the gateways for transferring material into and out of our cells. And the vast majority of IMPs in the human body, upwards of 99 percent, have yet to be fully characterized.
The trial and error involved in getting bacteria to cooperate is a waste of researchers’ time and resources. We need to up the odds of finding cooperative bacteria.
“At Caltech, a lot of us have interests bridging biology and computation and different aspects of engineering,” Clemons says. “It’s really an environment for solving this type of problem.”
Clemons applied for and received the National Institutes of Health (NIH) Director’s Pioneer Award.
With this multi-year grant, he promised to tackle the biomedical research challenge of studying IMPs in a highly unconventional way. It deviated from Clemons’s usual methodologies—in fact, it deviated from everyone’s methodologies.
“I think it’s fair to say that if I had been anywhere else, I would not have tried this,” he says. “At Caltech, you don’t get penalized for trying something that hasn’t been done.”
“I knew I couldn’t do it myself,” Clemons shares. “But I knew I could build a team to do it.
“Caltech’s a place where we get to interact in a way that you probably couldn’t on any other campus. Not that that’s impossible on other campuses, it’s just that, typically, we wouldn’t all be in the same place. Someone who’s in biochemistry would be in the med school. People doing machine learning would be on a different campus in the math department.”
Most of the intellectual resources Clemons needed were already at Caltech, including Yaser Abu-Mostafa, professor of electrical engineering and computer science; Yisong Yue, assistant professor of computing and mathematical sciences; Tom Miller, professor of chemistry; and Richard Murray, Caltech’s Thomas E. and Doris Everhart Professor of Control and Dynamical Systems and Bioengineering. But to do the work, Clemons also needed to recruit Shyam Saladi, a student from the University of Illinois at Urbana-Champaign.
Clemons and Saladi had met through Caltech’s Summer Undergraduate Research Fellowship (SURF) program, when Saladi joined Clemons’s lab in the summer of 2012. Caltech convinced Saladi to come for graduate school in 2014 with the incentives of the Institute’s Benjamin M. Rosen Fellowship and admittance to the Biochemistry and Molecular Biophysics option.
“We brought Shyam in and gave him the keys to the problem,” Clemons says. “That’s where it really went from idea to reality. We needed someone like Shyam, who’s got the computational chops and highly specialized background to figure it out.”
Clemons proposed using computers to design a statistical model that would predict how bacteria will react when asked to create a protein they don’t normally produce. “People said, ‘There’s no way you can capture a complex biological problem as a simple computational model,’” he says. “And I thought they were probably right. But I figured, ‘Why not try?’”
Saladi and Clemons talked about the problem and the improbable way they would go about solving it—and Saladi realized his background had been preparing him for a project exactly like this one. Equipped with undergraduate training in electrical engineering as well as in molecular and cellular biology, he turned to the raw chemistry of membrane proteins.
To collect data sets—outputs of experiments from years past—Saladi made cold calls to 19 universities and research centers worldwide. Of the researchers he contacted, 12 declined. It would have taken time to look for notes they’d most likely tossed, and Saladi’s reason for wanting the information probably sounded pretty far-fetched.
But seven researchers agreed to help. They went back through handwritten lab notebooks to scan and send PDFs of outputs from old experiments. Why did they agree to devote so much time to such a long-shot scheme? “I think the only reason anyone agreed to help me was that I was calling from Caltech,” Saladi says.
Saladi then set out to systematically convert the insights from dozens of experiments into new numbers.
“When Shyam started aggregating all of those data sets, we discovered that much of the data was either incomplete or mislabeled,” Clemons explains. “We had to go back to the primary sources and talk to the authors. It was a heroic effort on Shyam’s part.”
Half a million computing hours later, with assistance from a raft of undergraduates, Clemons and Saladi cracked the nut. With 7,819 lines of code, they devised a software tool they named IMProve. It can improve by two- or threefold the chances of selecting a gene sequence that will express protein in meaningful amounts.
“Frankly, the system worked better than we’d hoped,” Clemons says. “And we know we can do better.”
Clemons and his team are continuing to refine IMProve. “We have this first model that demonstrates it works. People are already using the software. We are continuing on this project because there is more data out there, and the more data points you have, the better your model is going to be.”
Saladi, who will earn his PhD in 2019, is contemplating what comes next. “We are in the early days of machine learning for biology: proving we can capture a complex biological problem with an algorithm that can do better than someone’s intuition or best guess,” he says. “After we turn knowledge into code and organize data into numbers, then we can turn those numbers into new tools, models, and insights. It’s really exciting to know that as we become more systematic in gathering all that we know in biology, we will change the way we do science.”
At the interface of behavioral economics, neuroscience, psychology, and psychiatry, Caltech postdoc Caroline Charpentier is elucidating how we interpret the actions of others, arrive at decisions, and resolve uncertainties.
Other times, it’s advantageous to go a step further and try to determine how a person’s actions relate to her expected outcomes or goals, and then emulate her behavior with these goals in mind rather than copying what she did.
Imitation and emulation often result in the same behavior, but they are distinct processes that recruit different parts of the brain and rely on different computations. And these computations show that we’re pretty optimal decision makers when it comes to which strategy to use.
What underpins that decision-making process? How much do we take into account risk, uncertainty, timing, and other variables when deciding whether to imitate or emulate? How much do the stability of our environment, our understanding of a given situation, and our self-confidence influence the ways we learn?
Charpentier works with psychology professor John O’Doherty in a lab that has received support from the National Institute of Mental Health through the Caltech Conte Center for Social Decision Making. The lab is also affiliated with the Caltech Brain Imaging Center (CBIC) and has benefited from several contributions to Break Through: The Caltech Campaign, including the gifts that created the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech and the Allen and Lenabelle Davis Leadership Chair for CBIC.
Founded in 2003 through a gift from the Gordon and Betty Moore Foundation, CBIC today is part of the Chen Institute at Caltech. The center brings together researchers across campus who are studying the powerful biological and chemical computing machine that is the human brain, and endowing CBIC is a high priority for the Break Through campaign.
Heads nodding, hands going up, and eyes widening in wonder: For Caltech graduate student Arian Jadbabaie, these are indications that “science just may be blowing their minds.”
Jadbabaie revels in the thought that looking up at the nighttime sky is an exercise in time travel. He also enjoys harnessing the power of quantum mechanics to demonstrate that levitation is possible. Through science outreach, he aims to show schoolchildren that nature is awesomely weird.
In the lab of Caltech assistant professor of physics Nick Hutzler, Jadbabaie uses lasers and cryogenic chambers that are as cold as outer space to conduct tabletop experiments designed to determine whether the laws of physics break down at the atomic and molecular levels at very low temperatures. In short, he’s in search of even more weirdness.
Even in his free time, Jadbabaie is science-minded. This past summer, he and Hutzler guided three high school students enrolled in Caltech’s Summer Research Connection program (SRC) as they built a laser in the Hutzler lab. Jadbabaie also takes every opportunity to help younger children understand science.
“There is this connection that happens when you demonstrate science to someone,” he says. “We’re both in a state of wonder at what we’re witnessing. It’s a very unifying and exhilarating event.”
Jadbabaie is one of a growing number of Caltech graduate and undergraduate students, postdoctoral scholars, and faculty who are committed to educating and inspiring Pasadena-area schoolchildren about science and engineering.
Recently, The Chuck Lorre Family Foundation—whose founder is the creative force behind television’s The Big Bang Theory—made a three-year commitment to Break Through: The Caltech Campaign to help Caltech reach even more children, especially those from underrepresented groups.
Funds from The Chuck Lorre Family Foundation already have been used to provide stipends for teachers from local schools so that they can participate in the SRC program as well as scholarships for students involved in the Community Science Academy @ Caltech. This support also gives CTLO flexibility to experiment and try out new ideas and programs.
“We are seeking to provide opportunities and really rich experiences for the schools, kids, and families we work with, and we recognize that doing that work is itself a part of our students’ education and a part of the professional development of our scientists and future scientists,” says Cassandra Horii, CTLO director. “It’s a symbiotic relationship.”
Hutzler (BS ’07), who is Jadbabaie’s faculty adviser, not only encourages his grad student’s work, but also is actively involved in outreach himself. When Hutzler returned to Caltech in 2017 as a member of the faculty, he approached CTLO and asked how he could help. He especially welcomes opportunities to bring people into his lab.
“I think a lot of people are interested in science, but so often it’s presented in a way that’s opaque,” Hutzler says. “It’s worthwhile not only to speak of the fundamental questions we’re trying to answer, but also to show the process. Scientists are thoughtful, precise, and thorough.”
Thanks to the Moore-Hufstedler Fund, an endowment designed to enrich student life at Caltech, Jadbabaie and fellow graduate student Nikita Klimovich were able to purchase a superconductivity demo kit that they will use to inspire new generations of schoolchildren to begin their own journeys into the weirdness of nature. Jadbabaie is grateful that CTLO connects him to outreach opportunities, and he also has a Facebook group of similarly
minded Caltech students who offer each other guidance and critiques about teaching science to young children.
“As a grad student, you have an obvious struggle to balance your life,” Jadbabaie says. “But 99 percent of the time, I’m happy that I am able to visit the schools. It reminds me of why I’m pursuing science.”
Receive The Caltech Effect and the Caltech magazine newsletter