Constructing the cosmos isn’t easy. There are a lot of moving parts to keep in mind, ranging from the astronomically large to the infinitesimally small. But in a plain campus building on the northern fringe of the University of Illinois, Robert Patterson hurtles through a galaxy he and his colleagues created, checking and re-checking his path to account for both physics and physiology. After all, he wants to create an authentic voyage through the universe, but he doesn’t want to make anyone’s stomach turn.
Click to launch the photo gallery
Patterson handles film direction and camera choreography at the Advanced Visualization Lab, where he and his colleagues have been constructing super high-quality, high-resolution, data-driven 3-D visualizations since 1985. While you may not have heard of the AVL, you likely know their work; they specialize in those large- format productions that are the bread and butter of museum dome cinemas. They played an instrumental role in creating some of the stunning data visualizations for the recent IMAX feature “Hubble 3D,” and they currently have work showing at New York’s American Natural History Museum and the California Academy of Sciences in San Francisco, among other places.
Those productions aren’t born simply of creativity but of complex computational science, and at that intersection the AVL has found a comfortable role. Tapping the supercomputing power at its disposal at the University of Illinois’ National Center for Supercomputing Applications (the mythical birthplace of HAL 9000), the AVL can make sense of massive data sets that others cannot, making it among the best in the world at turning complex data into science-driven cinematic art.
“Visualization is a supercomputing problem when you have terabytes of data,” says AVL Director Donna Cox. “A lot of places do not have the supercomputing power that we have, so we have focused on leveraging state of the art computer graphics tools and embedding them in a supercomputing environment where we can devote all these processors to the problems of visualization.”
Those problems are big—so big in fact that even skilled data visualizers aren’t always up to the task. Take the James Webb Space Telescope which, when it launches in 2014, will be the world’s most powerful. Computational models at NASA’s Goddard Space Flight Center already paint a pretty good picture of what Webb will see when it opens its eyes, and researchers there would like to share that. But even with the immense talent pool and huge computing resources available at NASA, the Goddard team realized the size and complexity of the data presented serious visualization challenges.
“We’re lucky here at NASA to have some of the world’s great data visualizers, so we start with the premise that we already have great people,” says Wade Sisler, executive producer for television and multimedia at Goddard. “But when it came time for visualizing the grander structure of time and space we realized even with the great talent we have here we didn’t have what we needed to do it justice.”
Sisler became familiar with the AVL several years prior – “they took my breath away with Cosmic Voyage,” he says of their work on the popular Morgan Freeman-narrated IMAX production that premiered in 1996. He was more recently impressed with their visualizations of black holes in the 2006 digital dome production Black Holes: The Other Side of Infinity, and his team tapped the AVL to turn the Webb models into cinematic shorts.
When Galaxies Collide: Using computational data from Goddard Space Flight Center models, the AVL created this animation depicting violent galaxy formation in the universe’s formative years for the James Webb Space Telescope. The longer film is available via NASA’s JWST page.
“We realized they were probably the best organization on the planet to visualize the structure of space and time,” Sisler says. “It requires a very special ability to control the flight of a camera through complex models.”
That ability is part intellectual talent and part technical prowess. Visualization teams at the AVL are a mash-up of scientists, visual artists, mathematicians, filmmakers, and software designers – among other things – and everyone learns from everyone else. Cox calls them renaissance teams, a term she coined to describe their interdisciplinary nature.
Then there’s the technology, the quarter-century of software development and technological toolmaking that has pushed the AVL’s capabilities beyond that of other data visualization houses. The team regularly writes custom software from scratch to solve problems for particular data sets or to create certain kinds of renders. They’ve written numerous plug-ins for off-the-shelf programs and have open-sourced much of their own code. Some of these tools have been in the making since 1986, Cox says, and, coupled with the supercomputing capabilities of the NCSA they make for a potent production suite.
Not least of these tools is Virtual Director, the program that lets Patterson sit down in front of a massive 3-D screen and travel at mind-numbing speeds through the data, zooming, panning, plotting paths, and otherwise engineering the cosmos. Using his laptop and a six-degree-of-freedom trackball controller, Virtual Director lets Patterson become his audience, gently guiding them through proto-planetary disks or the violent crush of a black hole with a deft flick of the fingertip.
Using the AVL’s homegrown tools, Patterson can plot a course through the data, examine his path in free space, and make adjustments to his production in 3-D. He can then share his work in progress with scientists intimately familiar with the data and its meaning. The AVL prides itself on its art, but it places a higher emphasis on staying true to the science.
The universe organized itself into a sort of “cosmic web” starting in the era of reionization (when it was just 20 million years old) and continuing to this day. This AVL visualization depicts how that may have happened, and is part of a longer film that is available via NASA’s JWST page.
“We collaborate with the scientists very intensively at the beginning to learn a lot,” Patterson says. “Then, once we’ve developed a plan, we’re sharing what we’ve done along the way, making sure we’re on track with the science. Later on, it might become more aesthetic; adjusting color, making sure we have the kind of depth we want, etc. There’s a science review, and then there’s a sort of cinematic review.”
Often, the science and the cinema must inform each other to make a visualization work. Many of the data sets the AVL deals with – particularly astrophysical data – contain dramatic ranges of scales. Moving from the very big to the very small seamlessly requires Patterson – and therefore the audience – to move very quickly through space and time while also growing or shrinking by several orders of magnitude.
“In Hubble 3-D, our eyes got about 50,000 light years wide so we could see the structure of our nearby univese, and then we would shrink way down,” Patterson says. “But we’ve done much more dramatic interocular distance animations where our eyeballs might be a few parsecs wide while we’re in the Milky Way, and then as we leave they just grow dramatically.”
But in a digital dome or IMAX environment that fills the viewers’ peripheral vision, all that zooming and scaling can actually create a physical feeling of motion, a sensation that Patterson wields to his advantage – to a measured extent.
“A lot of times when I’m creating camera moves for a large screen I tend to think of the camera as the theater,” Patterson says. “So if you tilt the camera down you can then imagine people actually tilting forward or tilting back or rolling, and it really can produce a physical impact. I like to give people an interesting, embodied, engaging experience – a voyage moving through the data – but I’m careful with that.”
The JWST will allow astronomers to peer deep into space where it will see the dense clouds in which young stars and planets are forming. This AVL visualization is part of a longer film that is available via NASA’s JWST page.
Those immersive, large-scale, 3-D digitial dome productions add flavor to a trip to the museum, but these visualizations go beyond simple entertainment. The AVL isn’t just about making films, Cox says, but also about making sense of information. Which is a fairly future-proof business model these days.
“Data is our biggest future challenge,” Cox says, referring not just to the AVL but the larger world. “Data is being gathered everywhere, not just in the science domain but in the humanities as well. We’re producing it everywhere from the Department of Homeland Security to telescopes that are capturing petabytes of data.”
The mountain of data that is piling up is meaningless if it can’t be represented in a way that makes sense, not just to scientists but to the public. While astrophysics remains the AVL’s forte, Cox’s renaissance teams are eager to tackle more diverse projects. Right now they are most thrilled about moving into the geosciences, and soon she hopes they will move more actively into visualizing the life sciences as well. Eventually, she wants to mesh the humanities and sciences into sweeping visualizations that take the whole world into account.
“In the future we want to take abstract relational databases on society and demographics – migration and migration of ecologies, for instance – and couple that with our large-scale geoscience data,” she says. “We’ll make meaning of all the different data that’s out there by fusing it into visuals that help us understand of the complexity of nature.”
In the more immediate future, the AVL continues to turn complex data into art for public outreach, directing scientific phenomena like actors on a stage. Sisler is already looking forward to another collaboration between the AVL and NASA, this time for a museum dome show slated for late 2011 or early 2012, though he’s not quite ready to reveal exactly what it’s about.
Given the AVL’s record of turning even the most complex data into art, it could be about anything in the universe.
