The National Security Agency and Technological Change

This post builds on the one Lukas put up last week. Most commentaries on the Snowden Affair, PRISM, and the other NSA programs that have come to light have focused on whether these programs are constitutional, whether Snowden is a hero or villain or something else, and, now, what these programs will mean for US foreign relations. I have also heard people ask how any of us could be surprised by these programs, and for a few days, people spent a lot of time talking about Snowden's girlfriend's pole-dancing skills. In other words, the Snowden Affair has all the markings of a major American media event.

In this post, I'd like to exercise the historian's prerogative by exploring how these NSA programs fit into a longer historical trajectory, namely how government spending and procurement influence technological change.
The history and sociology of science and technology are full of well-known stories of how government funding affected the direction and growth of technological innovation. The best known stories in the United States have to do with technical advances made at MIT, Harvard, and Los Alamos during World War II and the wide variety of scientific breakthroughs and technologies that emerged from Cold War defense spending. (Mark Buchanan recently put up an entertaining post about the many technologies that ultimately have roots in government spending.) There are many earlier examples in the United States from WWI and even the 19th century. Of course, any comprehensive history of the military-science-technology relationship would have to go back much further; in the West, right through 18th century French science societies through da Vinci at least back to Archimedes.

We can assume that spending on intelligence and the technology that undergirds it exploded after 9/11. 9/11 was to the surveillance-industrial complex what Sputnik was for Cold War sci-tech funding. It would be interesting to know whether the programs that developed after that date were merely extensions—if massively scaled up extensions—of things that were already in the works. It would also be interesting to know how many new programs developed after that date (versus building on old programs).

But it would also be fascinating to learn how these programs have influenced technological change, if at all. Do fundamentally new and largely unknown computing technologies lay behind the NSA's capabilities. Are these capabilities mostly the result of hugely scaling up technologies that are already well known (server farms, data mining algorithms, etc.)? Or will we look back at the NSA's programs as greatly changing computing technologies? If so, which companies would have produced these technologies for the agency? Mostly defense contractors? Or mostly computing firms? Or might the government have its own internal R&D shops? 

Economists and historians often examine "spill over" to see how government, typically military, spending ends up influencing the broader economy. To the degree that new technologies, processes, and techniques are being developed through these programs, for several reasons, it will likely be very difficult down the road to determine how much these things have moved into the domestic sector.  First of all, the NSA can likely prevent the spread of new technological systems (if truly new, x-technologies, like quantum computing, are a part of the programs). But the agency cannot easily stem the dissemination of the experience and tacit knowledge that people will gain by working in these programs. People will move to other jobs and take their experiences in, say, developing data mining algorithms with them. Again, the movement of this knowledge will be very difficult to track. 

Second, contractors, like Snowden, do a significant portion of US intelligence work. Today, on Meet the Press, Rep. Nancy Pelosi said that the Obama Administration has done a great deal to decrease the role of contractors in classified projects. I don't know where Pelosi is getting her information, but my instinct says that she is overestimating the decline of contractors under Obama. As long as intelligence remains tied to the use of enormous computer networks, contractors will likely continue to play an essential role. Even in the midst of news about Snowden, we have learned more about Amazon's contract to build cloud computing infrastructure for the CIA. Booz Allen Hamilton employees and other such consultants and contractors will take the lessons they learn in working for the NSA and apply them elsewhere. I'm sure the opposite is also true: the contractors are bringing lessons learned from private industry and using them for the intelligence agency. Indeed, in terms of the movement and synthesis of knowledge, between the private and public sectors, these contracting firms are likely important nodes that historians and sociologists would do well to examine . . . if they ever can . . . all of this is veiled in such horrible secrecy. In this case, however, secrecy might also have dire implications for our ability to study the realities of US innovation policy, since the surveillance-industry complex will have an unknown relationship to technological change and economic growth.

What I keep wondering is how we will see these things in ten or twenty years. Will we see the NSA's influence on technology as we now see Cold War sci-tech funding, that is, as a hugely important source of technological change and knowledge production? Or will the NSA's programs seem like just seem like another (ultimately boring) application of "big data" and the "app economy"? 

If any readers—especially those readers with deep knowledge of computing and/or computing history—have thoughts about the relationship between the NSA's programs and technological change, I'd love to hear them.


The NSA programs, or more accurately the revelation of them, will push in ernest the development of subversive technologies.

In 1993 John Gilmore, one of the first employees at Sun Microsystems, a founder of the EFF and Cypherpunks mailinglist, famously said "The Net interprets censorship as damage and routes around it". Much of the intellectual-techno-intertia of the past 30 years was put toward this effort. From the building anonymous networks meant to protect dissident thought in despot countries to the protection of internet neutrality from privatised censorship (aka voluntary censorship by corporations). For much of this time the ideologies and idealists concerned with censorship courted at least minor support from their western host nations. The NSA revelations will mark a turning point in this history.

"Now that we have made the Internet work, the next task is to stop it
from being a platform for massive surveillance, and makes it work in a
way that respects human rights, including privacy."
-- Richard Stallman's on the occasion of his induction to the Internet Hall of Fame June 27 2013

The turning point is one that takes us back to the days when cypherpunks worked in-difference to the west in the effort to free strong crypto as a right for all (See cypherpunks and history of PGP). I'm reminded of the minor inertia put into subversive technologies such as distributed DNS in the past few years.

In the 90's the EU expressed concern that the entire domain infrastructure was controlled by a US institution, ICANN's. The response was a promise that this would not be used for gain and only to protect the basic functionality of the internet. Fast forward to a few years ago when ICANN teamed with DHS to take down sites that link to copyright content. Both the DHS (meant to protect borders) and ICANN skirted their charter. The effects of this, and the subversive technologies being built to "route around it" will be seen in the coming decades. But this simple tale does not compare to what could could come from more recent events.

I imagine that the question of trust resulting from the NSA revelations will extend past the institution of government and well into corporate environments as those that work at Google, Apple, Microsoft or the other institutions complicit in the surveillance apparatus question sternly their employers for answers.

What has become now clear in my mind however is that the cypherpunk of today will no longer be one contemplation over what effect their code will have on the system of governance alone. That is, the age old issue of decentralisation of power, or the "The Cathedral and the Bazaar" constructions Eric S. Raymond described in 1997. The question now relates to how your code will effect a society that may welcome a Post Privacy era. Can we write code to protect the definition of privacy that is so delicately entwined with a human intuition that technology changes so rapidly.

As a historian of White House ICT adoption and policy, crises most certainly influence technological trajectories. IARPA and DARPA funding for projects relating to social network analysis, "big data," and the hardware to facilitate such work including investments in quantum computing rise dramatically around 9/11. If WW2 facilitated the development of computers and the Cold War facilitated the development of distributed computing (fiber optics and the Internet), then the War on Terror has facilitated the development of the analysis of distributed computing/data.

We find ourselves in an odd Lord of the Rings moment with the NSA as an ambivalent gray Barud Dur with a burning green eye (though the NSA would like everyone to believe the tower has a white/blue color scheme, it doesn't provide sufficient information to make that assessment). The Eye searches the land for wily ring-bearers/terrorists. Much of the time it succeeds by pinning down the correct target in its gaze and calling for nazgul/special ops forces. Other times it correctly ignores non-ringbearers. Sometimes though, it fails to locate the ringbearer because the ringbearer doesn't match the profile it is looking for. Other times, it focuses on a hapless innocent who happens to match the profile but is not a ring bearer.

Yes. We live in the Great Age of Statistics where numerical models govern far more of our lives than we are conscious. As a Statistical Age however, we must always remember that there is a place reserved for uncertainty and hence resistance. Statisticians do not deal in certainty, they deal in likelihood. That's where we are.

John Laprise

"Unknown"'s reference to Tolkien can't go by without me pointing out that Palantir, one of the great government technology vendor success stories of recent years, selling big data tools to spy agencies, is named after the seeing stones of Lord of the Rings, those devices that ultimately corrupt those who use them. I always thought it a truly bizarre and telling choice of name for the company!

John, thanks for your comment. I downloaded your dissertation a few months back, but haven't had a chance to crack it yet. Is anyone writing the pre-history of "big data" yet? I'd love to see someone give a historical treatment of the topic as people have done with nanotechnology (discovering in the process that it is largely re-labeled material science). Another issue I've been thinking of is the Ensmenger "Here Come the Computer Boys" one: in part, the Snowden story is about how we've come to depend on a new class of technicians.

Unknown, Palantir is precisely the kind of company that I was talking about in the spill over paragraphs. A Palo Alto firm that works for industry and government--it seems as if data mining is leading to a technological convergence (a la Nathan Rosenberg).

Note: Only a member of this blog may post a comment.

back to top