Big Data

Integrated Water Resources Planning in Southern California

On April 11, I presented a lunchtime keynote at SAWPA’s annual Santa Ana River Watershed 2013 conference in Costa Mesa, CA. It reflected on Integrated Resources Planning in Southern California, my experiences, and where we may be headed.

From where I stand today, I look back on twenty years of integrated water resources management in Southern California; and I look forward towards a world that demands the systems thinkers we’ve produced over that time.

What started as an analytical tool, borrowed from the power industry to help us select among alternate investment portfolios, has evolved to become an essential framework for decision-making in this complex “system of systems” that is our world today. Looking to the future, it offers a pathway to “real-time” management of our increasingly interconnected networks and infrastructure – both gray and green.

Frankly, it has arrived just in time, as a model of effective collaboration and partnership.

The need is created by the success of our economy and society to improve our standard of living – and our willingness to take on the challenge of extending those gains to the vast majority of the world’s population waiting their turn for the same.

Our prosperity has come at a price of course. Largely due, perhaps, to our predecessors inability of to foresee just how successful they would be in exploiting the resources of the world around them.

I often quote the Royal Charter of the Institution of Civil Engineers, secured by its then president Thomas Telford in London in 1828, because it reflects the confidence, boarding on arrogance perhaps, which drove our nineteenth century industrial revolution. It defines “the profession of a Civil Engineer” as: “being the art of directing the Great Sources of Power in Nature for the use and convenience of man.” What a bold and sweeping claim that is.

It implies that natural resources and our planetary assets are all essentially limitless – or at least that they could be treated as limitless without impairing our long-term growth and prosperity.

On November 5, 1913 a crowd of over 40,000 gathered to see water arriving through the Los Angeles Aqueduct.

On November 5, 1913, a crowd of over 40,000 gathered to see water arriving through the Los Angeles Aqueduct.

It reflects a worldview that can be heard in William Mulholland’s closing words at the commissioning of the Los Angeles Aqueduct in 1913 as water began cascading into the San Fernando Valley to the cheers of over 40,000 onlookers, “There it is – Take it!”

And it underpins the bold promise made by the Metropolitan Water District of Southern California in the 1952 Laguna Declaration, unambiguously guaranteeing to “provide its service area with adequate supplies of water to meet expanding and increasing needs.” Effectively chartering Metropolitan to continue to reach outside of its services area and secure the water needed to close any gap between the region’s growing water needs and its local supplies.

In the intervening years, our worldview has changed – for many of us, radically changed. What once only astronauts and visionaries could see as the small blue spaceship in a universe of darkness is now viewed, as such, by almost anyone who downloads the news, or reads a blog, or gets their information the old-fashioned way from television anchors. It’s a story of exponential population growth, huge increases in energy and resource consumption, shifting patterns of urbanization, and sometimes-violent competition for fresh water and food.

We are in an era of human dominance over natural systems so successful that it appears to approach the carrying capacity of our planet to provide for the very humanity that shapes it. This picture can be viewed as really challenging at best – possibly hopeless at worst. And dystopian visions of broken civilization are visible in all forms of media . . . a dark view that I emphatically reject.

We have a track record of accomplishing bold, audacious goals. We have surpassed the ambitions of our nineteenth and early-twentieth century forbearers who fought in the industrial revolution. And we will achieve the bold new goals of the sustainability revolution we are currently embroiled in.

With fundamentally different goals come profoundly different outcomes. When we revisit our definition of “the problem” and reframe our mission and purpose, radically new solutions and outcomes reveal themselves. You could call it “low hanging fruit.”

We will be much more efficient in the use of the resources we employ and the ecosystems that supply them – now we are see our world differently. The OWOW vision of one water, one watershed, one world that is interdependent on every actor in it for its health and well being will produce different results.

When we see ourselves as part of interconnected systems, we are fundamentally breaking down the boundaries that were one of the secrets of nineteenth century success – the divide and conquer strategy. We are establishing an entirely new model for systems integration in the built and natural environments.

This meeting today is, in part, the result of our success in promoting the wide adoption of integrated planning and systems thinking in Southern California. I see that evolution having occurred in three phases as we evolved from narrowly focused “portfolio planning” in the 1990’s; to multi-stakeholder/multi-interest systems planning after the year 2000; to today’s broadly accepted systems thinking, further enhanced by the emerging technologies of pervasive sensors and monitoring, together with real-time big data analytics.

Single-Purpose to Portfolio Planning

But to start, let me take you back to the first time I understood the importance of an “integrated systems perspective.” It’s a story I’ve told many times. It was October 1992. I was meeting with Dick Balcerzak, who was then Assistant General Manager at Met. Dick was a very practical and experienced civil engineer – a tunneling expert. I was the consultant project manager for Met’s 1993 Strategic Plan, and we were meeting to discuss water supply objectives.

Dick told the story of why he thought Met needed such a plan. One of his division heads had approached him about a reclamation project which Met was about to support financially. When Dick asked why Met was paying for such an expensive project, he was told that it was essential to meet Met’s goal of developing 400,000 acre feet of reclaimed water. Dick was shocked. “Four hundred thousand acre-feet, whose goal is that?” He thought it was more reclaimed water and more money than Southern California needed or could afford.

Then, he challenged us to answer the question: “How much reclaimed water should be developed in Southern California?” And how much water from transfers, from conservation, from groundwater recovery, and from new storage? They were great questions and not easy to answer – then or now. They led to the preparation of the first Integrated Water Resources Plan for Southern California, completed about three years later in March 1996. As an aside, that 1996 report targeted a total of 450,000 acre feet of recycled water by 2020, with 220,000 coming from then existing projects and 230,000 from new projects.

But what did the story illustrate? For Dick, it was an example of the shortcomings of an organization structured around its own internal functional responsibilities, with little communication and coordination among functions. Everyone seemed to be setting their own goals and doing their own thing. For the division head, it must have felt like Dick didn’t get the important long-term role that recycling would play in the delivery of water to Southern Californians.

Dick’s definition of “expensive” was surely based on a short-term comparison to the cost of Met’s existing imported supplies, which at the time were looking a little less reliable than anyone expected. For the division head, the decision to support the project may have been based on criteria that Dick wasn’t even considering. Maybe it contributed to solving a wastewater effluent disposal problem. Maybe it provided some security against interruptions in supply due to earthquakes.

From my perspective, it convinced me of the need for a better process to address the frustrations that Dick experienced in reaching agreement on the problem, and better tools to help the division head justify his recommendations to management and policy makers.

MWD IRP Cover Scan Dec 17, 2012-page1Ultimately, the 1996 Met IRP was the analytical process that justified a redefinition of Met’s relationship with its member agencies.  As Jack Foley and Woody Wodraska stated in the forward to its executive summary: “ The IRP represents . . . the recognition that meeting Southern California’s future water needs is a shared responsibility among many water providers.” And it coincided with a flurry of integrated water resources plans that followed. The process forced a more holistic view of the urban water cycle and explicitly championed the partnerships that were needed to manage within it.

Multi-Stakeholder, Multi-Interest Planning Processes and Tools

After 1996, we all made tremendous progress in developing both the process and tools of integrated systems planning. And the scope of our efforts embraced wider and more diverse interests. Expectations regarding the benefits that could result from integrated planning increased; and the range of creative options for achieving new objectives increased as well. Through these many projects and partnerships, we evolved from enlightened portfolio managers to broad systems thinkers – an evolution in perspective that prepares us for addressing the complexity of sustainability issues and climate change in the years ahead

I went on to work as a consultant for the Los Angeles Department of Public Works Bureau of Sanitation on an effort that began as their Integrated Plan for the Wastewater Program; and ended, in 2006, as the City of Los Angeles Integrated Resources Plan – a joint effort of the Bureau of Sanitation and the Los Angeles Department of Water and Power. It integrated water supply, water conservation, water recycling, and stormwater management into what had previously been a wastewater facilities planning process.

Just as important, it also relied upon broad public input in the development of planning level policies. The process opened up a community dialogue and way of doing business that continues to this day. An while the analytical tools live on, the stakeholder process and community networks altered decision-making and governance in a lasting way in Los Angeles. It continues to strengthen a broad commitment to a “systems perspective” in dealing with new, difficult, and complex problems.

Integrated Operations, Management and Adaptation in “Real Time”

After 2006, I went on to spend time in Singapore developing better simulation tools for modeling infrastructure and environmental systems. Those tools are being used by the Singapore Housing Development Board in the planning and design of a new eco-township for a population of over 200,000.  And I continue to promote that research at the University of South Florida Patel College of Global Sustainability.  It’s a strong belief of mine that supporting integrated planning processes with better tools to simulate the actual performance of our institutional and investment decisions will only enhance the quality of the stakeholder dialogue, strengthen commitments to partnership, and improve outcomes.

So where is this all heading? It’s obvious to me that the emerging breakthroughs in big data collection, storage, and analytics offer ever more exciting possibilities for those of us who are committed to resilience, adaption and sustainable practices in the years ahead.

For those organizations and agencies that have incorporated integrated systems thinking into their governance and decision-making, the future could not be more promising.

The power of big data analytics to shed light on the unexpected behavior of our natural and built systems is only beginning to be appreciated. Realizing its full potential, however, relies on our ability to reach higher levels of cooperation and understanding than we have ever achieved. The incentive to do so couldn’t be greater, and the examples of what can be accomplished are immediately surrounding us.

As planners, designers, and engineers that combine small-scale distributed technologies, functional green landscape, high-performance building technology, and our backbone centralized utility grids we benefit immensely from the advanced analytics applied to urban and natural systems data. Through simulation and analysis like that going on in Singapore, we can provide insights into the performance and resilience of sustainable solutions during the development process – helping insure that the investments communities are making live up to the expectations they have established.

But that’s only addressing the planning and design phase of transforming our urban landscape. Big data analytics will drive more institutional integration in real-time operations than we have ever imagined – so-called smart cities and smart networks are developing worldwide. Just look at the intelligent operations center that Rio de Janeiro has implemented in collaboration with IBM for a real-life example.

We should be increasing the investment in sensors and monitoring; and integrating all of the data available in our urban watersheds including both the man-made and the natural systems that supports us. If we don’t do it for ourselves, no doubt new entrants like IBM, Cisco, Google, and others will integrate it on their own.

The technologies for radical improvement in sustainable performance and resilience are emerging rapidly. What will they be used for? Who will decide? For me, success depends on the commitment of communities and their leaders, businesses and their employees, interest groups and their members to develop and insist upon new goals. To utilize the processes and tools of integrated management for the accomplishment of new purposes, ambitious and aspirational goals that are respectful of the capacity of our natural environment and able to reach a growing global population. This is what makes you, attending this event, the real heroes of this story.

We have come a long way from the belief in a strategy that aspired to command and control the great forces of nature. We have taken the useful analytic framework of integrated resources planning and used it to uncover new relationships among the infrastructure and the institutions we manage and lead. The insights, once learned, are not easily forgotten. And the collaboration produces success that leads to further collaboration. No more divide and conquer. From this point on, let us integrate, adapt, and thrive as partners in a sustainable future.

Big Data and the Water Industry

IWA Congress Dublin 2012Last May, at the inaugural IWA World Congress on Water, Climate and Energy held at the impressive new Convention Centre in Dublin, I presented a keynote address on big data analytics and its impact on the water industry. Since then, I have received a number of requests for a transcript of the presentation:

Over the last several years, I have been introduced to a realm of science and technology that many of us in the water industry don’t know much about – and really should pay closer attention to. In fact, it is a field of new science that barely has a name. Let’s just call it “big data analytics;” and let’s focus on how it can make our cities “smarter.”

New Science without a Name

It’s interesting that when The Data Warehousing Institute (TDWI) surveyed data management professionals on this topic in May 2011, 7% of respondents hadn’t “seen or heard of anything resembling big data analytics” – they knew nothing about it. Most respondents (65%) didn’t have a name for it but generally understood its meaning. The remaining 28% (roughly a quarter) both understood the concept and had named it – most calling it “big data analytics” but including names as well, like “advanced” analytics, “discovery” analytics, or “exploratory” analytics.


And while twenty years ago, I/T professionals used to struggle with the cost of data storage and management, today the storage of massive amounts of data is virtually free, providing for increasingly sophisticated approaches to mining it, analyzing it, discovering relationships within it, and ultimately utilizing it to predict the behavior of the complex systems (and systems of systems) it represents. This is an emerging technology which will no doubt touch every aspect of our lives (and already has if you shop with a credit card or over the internet). But for our purposes it’s the management of . . . [Click to view the full PDF]