Using New Models and Big Data to Better Understand Financial Risk

The financial crisis of 2008, which saw the failure of major investment banks Bear Stearns and Lehman Brothers, and the subsequent government bailout of insurance giant American International Group (AIG), had a ripple effect around the globe. How did America’s housing collapse lead to the downfall of these institutions? And why did that, in turn, translate into a severe economic downturn?

Not having a clear picture of systemic risk in the financial system, an issue encapsulated in the “too big to fail” interventions, is widely cited as the reason for this financial contagion — the chain reaction of failures between connected parties. However, in the nearly eight years since the crisis, with additional upheavals from the sovereign debt crisis and flash crashes that have followed, researchers and regulators are still teasing out the nuances of risk in a globally connected market, while exploring new ways to manage a system that is evolving at an unprecedented pace.

Researchers at MIT's Institute for Data, Systems, and Society (IDSS) have been a big part of these efforts, looking deeply at the problem of systemic risk in finance through a multidisciplinary lens. By bringing together engineers, information theorists, mathematicians, economists, biologists, and policy experts, IDSS has the opportunity to reframe the way the system is viewed. The goal is to generate new questions, better models, and, ultimately, a more robust and resilient financial system.

The financial ecosystem

A central theme across IDSS research is the idea of using a systems approach to analysis. In the case of the financial system this means taking a wide view, accounting for linkages and their effects across the entire system, as opposed to focusing on individual banks or market subsections.

“When an ecologist is asked to help manage a particular ecology they think not just about the particular plants or animals they’re raising, they think about the bacteria in the soil and the sources of food in the system. I think that’s what we’re missing now when we think about financial regulation — we don’t think about the system as a system,” Andrew Lo, the Charles E. and Susan T. Harris Professor at the MIT Sloan School of Management, said in a recent interview for the Journal of Financial Planning.

Lo, one of the faculty leads of the finance efforts at IDSS, has been advocating this approach to financial risk analysis — termed the adaptive markets hypothesis (AMH) — for more than a decade. He and his colleagues use principles from evolutionary biology to draw parallels between the observed dynamics of financial systems and those of ecosystems. The idea is to move regulators and investors away from viewing markets as physical systems — rational, immutable, efficient, and mechanistic — towards a more complex model: a “highly adaptive organic system” that is directly impacted by human decisions and behavior. Regulators and and the regulated respond to each other, and their strategies co-evolve over time.

This approach has particular relevance in light of the 2008 crisis, and continues to gain legitimacy as more advances are made in understanding drivers of individual decision making. Work by IDSS faculty member Alex “Sandy” Pentland, the Toshiba Professor of Media Arts and Sciences, for instance, also uses biological concepts to inform his research about financial behaviors. Yielding a non-traditional set of risk measures, Pentland and his colleagues found that “financial outcomes for individuals are intricately linked with their spatio-temporal traits,” meaning that the frequency and location of a person’s spending has strong predictive value about their propensity to overspend or miss payments. This is analogous to the interconnections between animal foraging behavior and their life outcomes, and has powerful implications for making better financial decisions, on both the individual and institutional levels.

In an opinion piece for the Proceedings of the National Academy of Arts and Sciences, Lo and his collaborator Simon Levin of Princeton University “propose that the financial system has crossed a threshold of complexity where the system is evolving faster than regulators and regulations can keep pace,” necessitating a new, interdisciplinary paradigm for modeling and predicting system-wide risk.

New ways to measure risk

An instrumental feature of the ecosystem model is its capacity to detail complexity — both of the system’s components and their interactions. The network modeling of systems does the same, but from an engineering perspective.

“The issue of how individual level shocks can propagate, amplify, and create systemic risk is clearly a systems question,” says Asu Ozdaglar, director of the Laboratory for Information and Decision Systems (LIDS) and a faculty lead of the IDSS finance efforts. “Decades of research at LIDS and MIT School of Engineering, which has studied systems approaches and how these create stabilities or instabilities under different circumstances, is highly relevant to understanding systemic risk.”

In some of her most recent research, Ozdalgar, the Joseph F. and Nancy P. Keithley Professor in Electrical Engineering; Daron Acemoglu, the Elizabeth and James Killian Professor of Economics; and their colleague Alireza Tahbaz-Salehi of Columbia University, explore the relationship between network architectures and systemic risk. Their research demonstrates that network architecture as a whole, rather than an individual component’s number and quality of connections, can be a better indicator of when shocks to a system might propagate. By modeling the effect of small shocks (a few defaulted loans, say) compared with large shocks (such as multi-bank failures) on different types of networks, Ozdaglar, Acemoglu, and Tahbaz-Salehi show key features of financial contagion. Their results indicate that densely connected networks are well-equipped to deal with small shocks — diversification helps to absorb them — whereas, sparse, or less-connected networks are less able to do so. Interestingly, however, there is a “phase transition,” and when shock size crosses a certain threshold it is the dense networks that do poorly — the interconnections facilitate contagion — while the more isolated connections in sparse networks stop failures from spreading. Ozdaglar writes in an article for MIT’s EECS Connector, “Financial interconnections create stability in response to small shocks but become powerful dominoes when shocks are large.”

Using data to see the big picture

Equally important to understanding systemic risk are the data underpinning the models. In order to fully appreciate the forces driving market behavior it is essential to have data that are coherent, coordinated across sectors, and accessible. This was made clear in the wake of the 2008 crisis: At that time, even systemically important institutions, such as Lehman or AIG, were not required to share critical risk data with regulatory agencies, making it impossible to detect early signs of trouble or to implement rapid resolution plans. The Dodd-Frank Wall Street Reform and Consumer Protection Act, passed in 2010, changed this landscape by mandating central reporting of large swaths of data. However, in today’s financial markets this is just one new source of information for regulators to manage. There has also been exponential growth of data sets from other areas, like market intelligence and social media platforms, and Internet search tools.

With all of this newly available data, much of it highly granular, come the challenges of managing and and analyzing it: navigating its sheer size, ensuring its privacy and security for all stakeholders, and being able to derive models from it to inform policy and decision making. One key way IDSS researchers are addressing these challenges is in collaboration with the Consortium for Systemic Risk Analytics (CSRA). CSRA was founded in 2010 by a group of researchers from finance and academia — including Lo — who saw how badly the financial system was affected by the incomplete indicators of systemic risk available in 2008. IDSS, along with the Laboratory for Financial Engineering, the Center for Finance and Policy, and CSRA are collaborating on developing tools, such as open-source software and a public-access systemic risk dashboard, to deepen understanding of systemic risk and to develop new risk analytics that can serve as early warning systems.

A particularly unique challenge in managing financial data is privacy. Unlike many other industries, whose trade knowledge and ideas are patentable and therefore protected, the financial industry’s intellectual property is largely unpatentable, consisting of business processes that are trade secrets and therefore proprietary. This, combined with issues of consumer data privacy, can create a significant obstacle to accessible data. The tension between protecting trade secrets and providing regulators with systemic risk transparency is another topic addressed by IDSS researchers. Lo and Pentland have, for example, with different respective projects, worked on cryptographic computational methods called “secure multiparty computation tools” which allow aggregate risk exposures to be determined without compromising the privacy of any individual institutions; only encrypted information is used by the regulators.

“Big data and machine learning have completely transformed several industries,” says Lo. “I think the same thing is happening to the financial industry. We’re now seeing interconnections among different parts of the system that have never really been visible before. Thanks to the combination of large amounts of data and our ability to analyze that data to develop new narratives, we can now manage risk much more effectively and also identify new sources of value for investors and other financial market participants. It’s launched a whole new golden age of financial innovation and discovery.”

The multidisciplinary approach

IDSS, in drawing talent from many disciplines, allows the financial system to be viewed from multiple vantage points. “Different approaches bring different perspectives, which are always useful,” says Ozdaglar. For instance, “the approach to systemic risk developed originally in LIDS is not only strongly interdisciplinary, but takes a systems approach, which is well catered to the problems at hand.”

What makes the multidisciplinary work at IDSS stand out, though, from the many important and highly collaborative research projects happening at MIT, is its scope. “The way IDSS is organized is around big challenges,” says Lo. “And it’s that scope that makes the effort different from anything that’s ever been done before. We’re focusing on some of the most difficult problems facing society. Systemic risk is not just in the financial system, but it also affects the environment, through climate change, for example. Very large systems are often systems that everyone takes for granted and, therefore, nobody feels responsible for understanding or maintaining them. By focusing squarely on these systemic issues, we can make much more progress than before and have lasting impact, not just on our academic endeavors, but on society itself.”

This article is part of a series highlighting major areas of research and innovation at MIT’s new Institute for Data, Systems, and Society.