Terra Analytica: The Automated Gaze of Corn and Soy

One could posit that the automated gaze that environmental analytics afford is based in how machine vision is entangled with the abstraction of labor. Lewis Mumford famously wrote that new colonists lacked labor power and were therefore propelled to invent labor-abstracting agricultural devices.⁵ Mumford overlooked the messiness of human laborers and their ability to organize in demand of basic requirements as a motivational factor to develop obedient mechanical replacements: cyber arms.

The politicized nature of labor mechanization is exemplified through the foundational marriage of agricultural engineering and plant breeding, which occurred between the cultivar VF-145 (a so-called square tomato) and the UC Blackwelder mechanical tomato harvester. Both are engineering marvels, a culinary tragedy and a testimony to the overlooked anti-unionist and xenophobic incentive to develop machinery and remote-controlled equipment for harvest.

In the mid-1950s plant breeder Gordie Hanna and agricultural engineer Coby Lorenzen, two scientists at the University of California, Davis, teamed up to invent a machine that could mechanically harvest tomatoes. Devices were optimized for monoculture production; however, more crucially, a tomato species was bred to withstand the rough touch and bulk handling of machine harvesting. It is often reported that Hanna literally threw tomatoes on the highway from a nearby bridge to test their resistance to impact.⁶

By 1962, the machine and the plant were market ready. Two years later, the United States Government refused to extend the provision of Public Law 78 allowing foreign nationals, who are often referred to as “otherwise inadmissible aliens,”⁷ into the US to help with crop production and harvesting.⁸ This is cleverly alluded to in Alex Rivera’s mock-marketing film Why Cybraceros? (2012) in which the ultimate American Dream is one where immigrants can labor in the US remotely, never having to live in, or become the responsibility of, American society.⁹ A short animated demonstration within the video depicts stereotyped Mexican workers controlling robotic harvesters in the US from across the border in Mexico. Rivera’s video sarcastically posits these caricatures as trouble-free, low-cost laborers for which the state is not accountable: the perfect immigrant.

The mechanization of labor ushered by monoculture engineering developments, such as the UC Blackwelder mechanical tomato harvester, also brought about carefully delineated fields that unintentionally catered to the eye in the sky. Fields serving mechanical harvesters were characteristically geometrical, for example in the shape of long rectangles to accommodate the machine’s preference for straight lines, and a width a minimum of twice that of the machine, allowing for the necessary 180-degree turn at the end of each lap. Checkered lines such as those pictured in the dry belt near Champion, Alberta, would inspire a mathematical reading of space through calculable parameters. Such neatly isolated variables could then inform narratives of economic growth and facilitate the application of the circular causality and regulatory logics of cybernetics to industrial agriculture. The prairie township fields reminiscent of Hanna’s square tomato became surfaces, which are easier to capture, measure, and valuate.

The “input” and “output” paradigms that dominate computational and financial operations could now easily be measured in land allocations such as the following:

Thus, the tools mechanical harvesting produced ushered in an aesthetics of machine-readability that also facilitated narratives of circular causality.

These aesthetics also facilitated crop analysis using photogrammetry based on views from above. Contemporaneous modes of capture, measure, and valuation, along with forecasting based on machine learning, are rooted in a tradition of methods earlier developed for interpreting aerial photography. Before these photographs were bulk processed like Californian tomatoes by trained algorithms, pattern recognition and analysis were termed “photographic interpretation.”¹⁰ In 1960, the preface of the American Society of Photogrammetry and Remote Sensing’s volume Photographic Interpretation described the practice as akin to “detectives who are involved in the extraction of valuable information from aerial photographs…obscured by nature’s camouflage.”¹¹ Massive fields dedicated to monoculture were one of the areas where such extraction was pioneered.

Photogrammetry was largely used to survey geologic structures, drainage patterns, and agricultural areas, which could indicate lithology and soil structure as well as crop yield forecasting. Various types of film and filters were employed to capture oblique and vertical views, such as Camouflage Detection and Ektachrome film, or Panchromatic 25 A and Infrared 89 A filters. Depending on the film or filter, certain wavelengths are largely absorbed while others are reflected, and these properties vary by crop type, or for example when a plant is diseased. In a diseased leaf, the spongy mesophyll tissue, which is usually highly reflective of infrared light, collapses while for a time retaining its green color. Thus, the infrared film can detect crop disease before it is observable to panchromatic photography.¹² As such, as a means of cross-referencing with a more complete spectrum, several images of an area were aggregated and compared for analysis, in combination with the use of vertical stereograms to understand topography. Stereograms were created by using two images of an area taken from the same height at different angles and collated with the help of a mirror or scanning stereoscope with a binocular attachment used to plot boundaries or terrain features on the aerial photographs. Photographs were aligned precisely with match lines and annotated with colored pencils according to land use.¹³ The analysis of these images was determined with pattern detection, usually a combination of geologic, drainage, or crop pattern features.

Crop patterns in aerial photography were largely determined through tone and texture. The tone had an array of values from nearly white to light gray, and the texture spanned such values as reticular, cordlike, striate, and linear. From these texture and pattern values, we can infer the future algorithms’ pseudocode.¹⁴ As an example, based on land use categories employed in Switzerland circa 1960,¹⁵ we can imagine the following instruction set:

In the 1960s, while automation was being researched, the analysis central to photogrammetry was being accomplished by an interpreter, invariably a human, and usually female-identified.¹⁶ It was also understood that the properties that the interpreter was trained to detect—texture, pattern, tone, or shape—were qualitative and therefore appraised subjectively.¹⁷ The complexity of this appraisal also lay in the necessary knowledge of the multifarious interrelations between soil, plants, and weathering processes, together with experience in recognizing their combined agencies. A number of questions were provided to interpreters during training to aid the assessment process, such as: “Is the pattern location related to: (a) land, (b) water, (c) continent, (d) subcontinent, (e) regional, (f) local?”; “Is the pattern[:] a) random or (b) systematic?”; “Is the tone of the pattern[:] (a) Consistent, (b) Lighter, or (c) Darker?”; “Is the pattern a unit in itself or subject to breakdown of identity?”; or “Is the pattern near or related to similar patterns?”¹⁸ However, the conclusion was ultimately understood to be a subjective process achieved through the interpreter’s deduction. As the task of interpreting has been largely delegated to computational assemblages, such as Descartes Lab, it has come to be considered an objective form of intelligence. Thus, the methods and conclusions inferred from the female-identified human, which were deemed subjective interpretation, became understood as objective when performed by the machinic.

Photo interpretation was also key in agricultural economics.¹⁹ In the US, yield was forecasted for commercially important crops by the Department of Agriculture. Thus, wartime photography analysis became integrated into Rural Sociology, a then-new field dedicated to the economics of farm production.²⁰ Perhaps unsurprisingly, software based on pattern recognition, which is used to predict crop yield contemporaneously, is an area of significant interest to military organizations, namely the US Defense Advanced Research Projects Agency (DARPA).²¹ In 2017, DARPA solicited proposals for its Geospatial Cloud Analytics (GCA) program to develop technology for accessing and analyzing geospatial data.²² The defense research agency has been making significant investment into machine learning–enhanced, pattern recognition–based software that processes satellite imagery to estimate the likelihood of social unrest resulting from low crop yield.²³ Inflation driven by rising agricultural commodity prices, or “agflation,” is now widely acknowledged as a key stressor for uprisings such as what is referred to by some as the Arab Spring.²⁴ According to these reports, in 2010 and 2011, a winter drought in China, a heat wave in Russia, and an outbreak of stem rust in East Africa all contributed to dramatically reduce grain production for export.²⁵ The climactic stress, leading to droughts and global wheat shortages, exacerbated the subsequent upheavals in the Middle East/North Africa region (MENA).²⁶ To this effect, Joseph Evans, the program manager in DARPA’s strategic technology office, states: “One of the things we’ve seen is that regional unrest has been linked to circumstances that seem detached from national security—like the price of bread.”²⁷ This introduces a new twist to Joachim Radkau’s statement: “When it comes to implementing ecological concerns, it is especially advantageous if they converge with military interests.”²⁸

One of the companies that has received a significant amount of funding through DARPA’s GCA program is the New Mexico–based Descartes Lab, a software development corporation focused on the provision of geospatial intelligence through the automation of analysis of satellite imagery and data. It is named after French mathematician René Descartes, to whom we owe the Cartesian system, which describes the position of an object through coordinates. He also cemented the separation of mind and body in his dualism theory, a division that still haunts us today. Descartes Lab was founded in 2014 as a spin-off from the Los Alamos National Laboratory (host of the Manhattan Project, the World War II nuclear bomb program and a former private ranch school for boys). In addition to providing so-called “business solutions,” Descartes Lab offers a platform that is publicly accessible through a Python client. A limited amount of imagery is available to experiment with and understand how the parameters work. Full access to the platform and data catalog requires a registered account and, more importantly, that one be granted access. Finis-terra was created with openly available data. At the time, 2018, this consisted of an array of approximately twenty images upon which certain functions could be applied, such as segmentation through measurement of infrared light reflected from the earth’s surface together with synthetic aperture radar (SAR). The accessibility of this data and these functions is obtained through the terminal (via the Python client) and requires some understanding of the command-line interface (CLI) as well as the Python programming language. This, together with the very limited satellite imagery made available to the general user, creates ambiguity around its open-access policy. Who is granted full access to this data, and at what cost, is relevant in considering the power dynamics at play in geospatial intelligence services.

The geological gaze of the Anthropocene has trained us to understand deep time as earthward vision through the succession of strata. In other words, to look from below is to look from the past. From this modernist epistemology, it follows that to look from above is to look from the future. As prediction modeling based on geospatial intelligence has implications reaching agriculture, insurance valuation, commodity market value, and military operations, guiding not only decision making and policy, but also value fluctuation in real time, the eye in the sky becomes an all-seeing oracle, performing the future through its predictions made truth. Similar to the bird’s-eye perspective stemming from ballooning, which had a profound epistemological effect on viewing the world,³⁰ the computational perspective of machine learning coupled with GIS data provides the contemporary material ordering of the globe, embodying what Adrian Mackenzie has referred to as a new “enunciative modality.”³¹ The latter generates operational statements by connecting various elements in the production of classifications and predictions. However, machine learning does not understand pattern per se. Pattern recognition, or data analysis, is rather based in the perception of difference. And while this may relate to earlier stereoscopic practices in which the difference between two images is used to infer topography, machine learning deduction is more akin to “approximation through variation.”³² How might these recursive and dynamic processes modify modes of knowledge production and of knowing? Is the future presenting itself from the vantage point of unsupervised learners creating new realities by iterative model generation?

In The Future of Futures: The Time of Money in Financing and Society (2011), Elena Esposito refers to the change in direction engendered by monetarization. As opposed to the industrial, capitalistic, or modern society that looked backward, as evidenced by their post formulations—post-industrial, post-capitalistic, and post-modern—“the risk society defines itself by looking forward.”³³ Technologies of capture such as geospatial intelligence produce forms of abstraction that facilitate the risk society, fueled by derivatives trading. Like derivatives, which abstract “from the specificity of the capital, making every form of capital convertible into any other,”³⁴ prediction modeling abstracts from the specificity of the crop, making the risk associated with lower or higher predicted yield a value that can be traded, irrespective of the property, thus facilitating the circulation of investments. If prediction is resolutely predicated upon a future looking toward the earth from above, Finis-terra allows one to contemplate the view from the future, to lie in its silk and woolen abstraction, creating a space to conjure subversive threads. At the same time, the woven textile grids and patterns point to the reification of this epistemology shaped by the futureabove, and consider the implications of algorithmic subjectivation. Rebecca Uliasz outlines how taxonomy is central to the training of algorithms to recognize patterns in digital images.³⁵ Taxonomies mobilize and calcify knowledge systems, in this case based on a set of parameters readable from the perspective, angle, and distance of a satellite lens and rooted in the predictive potential of the view from above. How might we then understand the specific co-individuation of crops being captured with the technologies they are being captured by? And the implications for the more-than-human being captured?

In reference to earlier conceptions of photogrammetry, pattern recognition software is the new “detective” extracting valuable information from aerial photographs. Software such as Descartes Lab, which described itself in 2022 as “global-scale predictive intelligence,”³⁶ presents an ontological shift from interpretation audited by female-identified computers to machinic intelligence. This realignment promises the exploitation of crops with unparalleled efficiency, where the earth’s surface is rendered a digital field that affords agricultural mapping, inspection, and surveillance (for crop pests as well as so-called illegal migrants)—a conception in which the digitally cultivated field has the ability to combine and navigate a seemingly endless variety of elements while simultaneously promising perfectly tailored specificity.

Based on methods developed in photogrammetry, satellite imagery is analyzed with algorithms trained to recognize several kinds of patterns, an iterative process of difference recognition. These readings are then used to price crop insurance, to estimate crop yield and to predict social unrest. Much like Kathryn Yusoff’s conception of the geological strata gaze as a worlding agent of meta-ontologies,³⁷ technologies of capture from above are in turn inscribing the earth, in a recursive iteration of READ-WRITE.

Such orderings of space and nature as that prescribed by photogrammetry and geospatial intelligence software are also manifest in Spring Carpets, often referring to ancient Persian carpets depicting pleasure gardens seen from above. They were used as floor coverings during wintertime, alluding to a symbolic control of the ruling elites over the succession and return of seasons.⁴⁰ As intimations of larger pleasure gardens, their design was organized around a central watercourse from which four streams flowed at geometrical angles. These then forked into a myriad of tributary canals, islands, and ponds populated by fish, flower plots, and trees.⁴¹ The rug produced a specific representation of nature, a human-organized space with differentiated entities. Furthermore, as a microcosm, the carpet had the startling ability to represent both the smallest part of the garden and its totality.⁴² These are stand-in qualities foundational to prediction modeling techniques, that is, the capacity to individuate elements and to generalize and make large-scale inferences based on those individuations.

Thinking with the Earth-as-body might help us to understand the processes at play in geospatial intelligence, whose changeable inscription of history can not only be read or traced, but also allegedly predicted. As space and its capture is now emerging as a lucrative domain for technical and financial systems, the figure of the hammock and the Spring Carpet are introduced as apparatuses of embodied-earth-thinking, as they bridge notions of the technological-capture frontier economy through remote sensing or viewing from above, with earlier notions of the colonial frontier of exploration.

An early European depiction of the hammock by Stradanus is famously used in postcolonial studies to critique the so called “Awakening” of America by Amerigo Vespucci.⁴⁴ In this image, an often-overlooked detail is that “America” was disturbed from History’s slumber from the comfort of a hammock’s net. Christopher Columbus, as well as other chroniclers such as Bartolomé de Las Casas, noticed the hammocks with great interest.⁴⁵ After crossing the westernmost regions of the Atlantic ocean, unknown to Europe and therefore dubbed finis terrae, Columbus noticed the low-hanging beds in the Bahamas in 1492 and stated in his diary that “people were sleeping in nets between the trees.”⁴⁶ “They lye on a coarse Rug or Matt, and those that have the most plentiful Estate or Fortunes, the better sort, use Net–work, knotted at the four corners in lieu of Beds, which the Inhabitants of the Island of Hispaniola, in their own proper Idiom, term Hammacks,”⁴⁷ wrote de las Casas in 1552.

The hammock was one of the many curiosities brought to Europe. Gonzalo Fernández de Oviedo y Valdes, one of the first chroniclers of “New World” goods, introduced Spanish, English, and French audiences to an enormous variety of previously unheard-of exotica including the pineapple, the canoe, smoking tobacco, the manatee, and hammocks.⁴⁸ To produce the textile-based exotica, the Spanish empire placed Indigenous women under forced labor as weavers.⁴⁹ Undoubtedly, the hammock wove a story of pain, loss of dignity, and loss of way of life. These tensions between the possibility to hold subversive threads of knowledge as well as world-restoring practices together with the knotted histories of colonial degradation, provide a critical antagonism.

According to archaeological records, hammock culture dates back to 400 BCE, from the Taíno and Arawak peoples who settled in South and Central regions of the South American continent from the Caribbean islands. The first hammocks were made using hammock tree bark, from which the term hamaca is derived.⁵⁰ “Amaca” is the Taíno word for fish nets, which also described the appearance of their cotton hammocks.⁵¹ The Wayuú of La Guajira argue that a spider named Waleker taught them how to weave hanging beds.⁵² Hammocks were also used for birth and during funerary rites to wrap the embalmed dead before burials.⁵³ Hamacas were fundamentally world-restoring threads.

Like Columbus, who marveled at the people of the new world in their hammocks, Finis-terra offers a glimpse of the entanglement of finance, imperialism, and colonialism afforded by the automatic gaze. Finis terrae, in addition to literally signifying the land’s end, is also a Latin term that designates a payment, settlement, fine, or tax—thus a coupling with the historic material process of the frontier thesis and its financial contemporary. The unresolved contradictions of uneven “patterns of accumulation”⁵⁴ woven from wool and silk begin to address the task of undertaking a “political economy of knowledge.”⁵⁵ For Cusicanqui, this process of decolonial thought requires the examination of economic and material mechanisms obfuscated by discourses that maintain colonial domination. Here, we can think of the discourse of efficiency put forward by geospatial intelligence companies such as Descartes Labs: “We create new sources of operational advantage for our customers and partners by combining multimodal data with machine intelligence to deliver solutions quicker, more efficiently and at planetary scale.”⁵⁶ According to Descartes Lab, their service enables “a better understanding of the physical world through the scalable analysis of geospatial data.”⁵⁷ Hannah Arendt warned of the seductive potential of abstracted tools of analysis in The Human Condition (1958). She explains that the disembodiment of eye and body, or eye and mind, creates a distance that is powerful because of its inherently reductionistic capacity.⁵⁸ This begs the question: what onto-epistemologies are modes of geospatial analytics mobilizing?⁵⁹

In some respects, Finis-terra also refers to the end of the earth as a habitable planet, not as an inevitable scenario, but rather a cautionary tale. Fatalism poisons the nourishment of practices capable of restoring the world and as such, this contribution refutes the inevitability of planetary crisis.

The financial sphere, “ever in search of new fields [and surfaces] to securitize,”⁶¹ has been apprehended as a pattern of accumulation. In considering how automated pattern recognition turns the Earth into a permanent yet uneven surface of accumulation, it is useful to remember that machine learning operates as an accumulation—of settings, data, and devices—rather than a knowledge shift.⁶² Embroidering databases into recognizable shapes, such as in the work Finis-terra, renders the subjection of terrestrial sense data and movements into mathematical formulae material. These patterns are rooted in the practice of photogrammetry and the interpretation of photographs, from which “regularities…appear through a series of related differences and similarities.”⁶³ The capacity to infer and draw conclusions from these generalizations are principles embedded in the field of geospatial intelligence. One of its main protagonists, the largely DARPA-funded Descartes Lab in partnership with several agribusinesses, thus compares weather patterns with grain-growing patterns, resulting in the emergence of potential sites for investment, or divestment, according to valuation in the calculations of finance capital. This could be thought of as the reduction of “nature” to patterns of accumulation that terraform the earth through analytical systems; the satellite camera lens and the algorithm, slowly scanning the earth, as a tool that both captures and erases. Similar to the condemning apparatus in Kafka’s Penal Colony,⁶⁴ the surveying apparatus is rewriting the earth into zones of lower or higher risk, which may warrant rising insurance premiums or the deployment of counter-revolutionary military forces.

Space conceived as areas of preemptive potential is alluring, like discourses of continued growth made possible with ever more efficient systems. This, however, is based in an ontological ordering of the world, namely the Cartesian reduction and translation of the world into algebraic formulae, which Arendt foresightedly disqualified as a possible indication of truth or objectivity. She also noted that Descartes’s mathematical analysis could not surpass sensorial perception.⁶⁵ The earth is, after all, the site of many incomputables, which can be confounded or erased by incompatibility with mathematical analysis or geospatial intelligence taxonomies, such as migrants, traditional and Indigenous knowledges, or complex species such as lichen.⁶⁶ What is not captured is potentially not simply excluded from the archival database, but being slowly overwritten. Reminiscent of the colonial frontiers of so-called discovery, claiming terra nullius and erasing people to the order of genocide, Cusicanqui’s words resonate. We must uncover subversive threads of knowledge and, with them, decolonial epistemologies.

A thought of counterintelligence on which to linger from Karen Barad: “[T]he ‘knower’ does not stand in a relation of absolute externality to the natural world being investigated—there is no such exterior observational point.”⁶⁷ They also expound Niels Bohr’s quantum theory, which insisted that human epistemology must consider its indivisibility with the environment that it seeks to observe and understand. On the one hand, Finis-terra invites us to stay with the trouble of the onto-epistemologies mobilized by geospatial intelligence,⁶⁸ and on the other, to move beyond modes of subjectivation these technologies formulate by inviting an embodied-earth-thinking. As such, by decentering the perspective of the gaze, it gestures toward a possible decolonial critical epistemology in media practice. As narratives of capture permeate environmental analysis, it becomes urgent to develop modes of sensing and knowing that stem from embodied cosmologies, ones in which humans are “of” the world and not external observers.