30:T1658,How does an algorithm see colors? What are the differences between computer and human ways of seeing color? How can critical making trouble these human-machine perceptions of color? Our installation, Algorithmic Sea, flows out of these questions. Water is at once translucent, opaque, and transparent. It also serves as a metaphor for fluidity, movement, and situatedness. We use water and its characteristics as a prism for understanding human and algorithmic seeing. Through our critical making process we ask the question “what is the color of water?” by collaboratively composing a sea of colors with visitors and algorithms. Our goal with this prompt is not to provide an answer, but to reflect on the inherent qualities of partiality, change, and bias through a fluid critical making practice. In our net art and site-specific installation, hexagons of color undulate as they drift out of the frame, highlighting the flowing quality of how humans and computers see and perceive the visual world and its chromatic qualities. The work of Abraham Gottlob Werner, a German geologist, inspired our inquiry. Over two hundred years ago, Werner created a color chart with 108 colors. His ambition was to make a color standard through a terminology that used corresponding minerals, allowing the comparison of colors within a systematized scheme. He conceived this system to support the scientific research of the time: his standardization would allow a control for the comparison of chromatic observations in scientific research. Most notoriously, the naturalist Charles Darwin used Werner’s color charts while embarking on his HMS Beagle voyage, in order to give his future readers a more precise sense of the colors he was seeing in animals and plants across the world (Syme, 2019). The color charts created by Werner are beautiful and useful, but also limited. In order to make a standard, Werner constrained colors to his particular observations, making them stationary and non-variable. As such, they cannot respond to or account for the many particular ways each person sees color, as well as its site-specific environmental situatedness (e.g. light angles) and cultural variations (see Schorr, 2020). Werner’s charts minimize the impermanence of color, the way that colors change in an environment and in our emotional context. In this sense, when Patrick Syme (2019) adapted Werner’s charts, he wrote that “the play of colour changeability is left out, as it is impossible to represent them.” The same way that minerals themselves change colors as they interact with the world, so are our human-emotional ways of seeing influenced by mental states, memories, and personal connections. That is, in his attempt to make a technical standard, Werner minimized how color is not static. While such a standard may have served scientific goals, what are the consequences of such systematic simplifications? Today, instead of Werner’s color charts, computational systems and algorithmic models are largely responsible for defining colors in images, videos, and data systems for a diversity of uses. Werner’s technological creation of a standard of color is not too distant from the limitations imposed by these contemporary systems, which turn color into mathematical values rather than mineral comparisons. Moreover, the analysis of these mathematical values is used for a variety of industrial and commercial uses, such as image segmentation, curation, and organization. Our work is a response to these algorithmic operations, an attempt at discussing how they are situated and impermanent, much like our own human ways of seeing and understanding color. In this article, we explore how algorithms see color, and what are the implications and politics of these ways of seeing. To grapple with these questions, we engage in a critical making process where we, inspired by “critical technical practice” (Agre, 1997) and “critical making” (Ratto, 2011), focus on practice as a way of creating knowledge on technological processes. This involves bridging research, arts, and programming, as well as working with different outputs (a net art piece, site-specific installations, and this interactive article). We discuss here how our processes of critical artistic making open up particular possibilities when engaging with technology, building on the potentials this methodology affords – openness and experimentation. We begin by discussing our approach to making, which centers on flow and impermanence. Following that, we discuss the background of algorithmic ways of seeing color, and how this theoretical and empirical literature offers the foundations that our critical making builds upon. Based on these methodological and conceptual undercurrents, we explore our practice in creating the Algorithmic Sea. We discuss what we have(n’t) done, backtracking our key decisions and considering how this process allowed us to see differently. Next, we discuss three questions our critical making practice raises about the limitations in algorithmic ways of seeing color – as well as their implications. Finally, we conclude by outlining the contributions this article makes to the field, especially how the metaphor of water (situated and impermanent) can be used to further understand both critical making and algorithmic seeing. We have structured this article in an interactive format, including several ways you (the reader) can engage with the content beyond reading. We see this as a continuation of our critical making practice, as these interactive moments extend and elucidate our conceptual and methodological arguments, while also allowing you to engage with flow in action.31:T1045,This article departs from the notion that we spark insights through making and creating media (see Ingold, 2013; Gauntlet, 2015), especially when engaging with algorithmic culture. To investigate the computational and human ways of seeing color we embraced a playful process, where “careful tinkering with materials, electronic components, and digital fabrication processes provided an embodied understanding of technologies and their inherent malleability” (Bogers & Chiappini, 2019, p. 11). We make sense of our critical practice by engaging with the concepts of “critical technical practice” (see Agre, 1997; Harwood, 2019) and “critical making” (Ratto, 2011), as well as the metaphor of fluidity. As a transdisciplinary investigation, we find critical technical practice (CTP) to be a useful approach because it centers on a “split identity––one foot planted in the craft work of design and the other foot planted in the reflexive work of critique” (Agre, 1997). Soon & Cox (2020), building upon CTP, suggest that it is important to further “queer” Agre’s split identity. They argue that computational culture does not have to be seen as a way to solve problems or fix things, but instead as a space we can occupy to “question existing technological paradigms and further create changes in the technical system.” As such, Soon & Cox propose “aesthetic programming” as a form of CTP based on “writing, coding, and thinking to imagine, create and propose alternatives.” Related to this, the practice of “critical making” (Ratto, 2011), adds a different facet in the relation between practice and research. While it may be argued that the main goal of CTP is the final result, for “critical making” the making is only “a means to an end.” As such, it faces the “constructive process as the site for analysis and its explicit connections to specific scholarly literature.” While we see our created artistic piece as an important outcome of the investigation which can be exhibited, we also value the process of making it as a powerful source of knowledge. Our process engaged with CTP and critical making while centering the need of a fluid collaborative practice. Rather than operating from pre-set goals, this meant we sought to let the materials we worked with (algorithms, images, paint, light, interfaces) spark insights, informing how our practice operated. This process was composed of many small experimentations. As Markham & Pereira (2019) describe, this means facing “continuous tinkering, careful tracking, and later reflecting on small elements, which in turn focused our attention on whether and how these seemed to make a difference in action” (p.19). Through our process, small changes and experiments generated important discussions that constantly redirected our goals in the making process. We explored paths that were originally unforeseen, drifting and flowing as our experimentation with the materiality of digital systems unfolded. The fluidity of our making process is also present in our collaboration. Our process was conducted virtually, due to the covid-19 pandemic, across the sea, between Brazil and Denmark. It was important for us to devise ways for collaborating that both engaged our different perspectives, but also the materialities of our respective practices. Each of us contributed with different experiments using materials such as code, projector, paint, recycled bottles, photography, video, and text. Our collaboration relied on open-source programming, which allows the sharing of code and software development. All of our code-based experiments were created on the OpenProcessing platform, where we were able to share programming sketches with eachother as well as a wider network of artist-programmers. Our approach to critical making finds its specificity in fluidity, both in our iterative relation with the materiality of digital systems and our modes of collaboration. We described how this experimentation flowed in the Process section, where we narrate our process, showing how decisions were taken by weighing different alternatives in practice. 32:T1b66,Our critical making took place in constant dialogue with scholarship around algorithms and color. This wider theoretical and conceptual background complements our intervention, especially by complicating the modes through which technologies of perception are sociotechnical formations that are always marked by “inbuilt aberrations,” biases, and limitations (Pasquinelli & Joler, 2020). These aberrations often have consequences for the way color is expressed and encoded. Photographic film technology, for example, has embedded in it many biases in its representation of color, which are still present in the current times. The reliance on Kodak’s “Shirley” card – named after a white-skinned model named Shirley – for film exposure calibration in photographic studios is an example of how photographic imaging has been constructed for the white skin (Roth, 2009; Benjamin, 2019b). Today, with the advent of digital cameras, smartphones, and networked images, photographic technology is entangled with the algorithm, bringing new forms of aberrations and biases. Color is now represented through digital systems, which also allow for different forms of algorithmic analysis through computer vision algorithms. These automated operations are now embedded in many facets of everyday life, such as in e-commerce (e.g. searching and categorizing clothes for color in a clothes website), social media image analysis, and Automated Number Plate Reader cameras which detect a passing car’s color. These, as well as other algorithms, are often presented as neutral and impartial technological solutions, “promoted and perceived as more objective or progressive than the discriminatory systems of a previous era” (Benjamin, 2019a, p. 11), but the reality is that they embody and reflect new and old inequities and partialities. Although the range of use of computer vision is broad, critical scholarly discussion on color has occurred mostly as related to facial recognition. Discussion around race and algorithms has been particularly important in critiquing the way computer vision is biased towards whiteness, while ignoring Black, Brown, and Indigenous people. The work of Joy Buolamwini & Timnit Gebru (2018), for example, has been extremely influential in critiquing the biases of facial recognition detection of humans, and outlining the particular impacts not being recognized has for Black people, especially women (see also Angwin et al, 2016, for other social impacts). However, little has been described about the societal consequences of computer vision’s capacity to perceive and analyze color more widely, especially the limitations around how color is understood computationally beyond as a characteristic of humans. Intervening in this field was a goal of our critical making practice. Our interest in critiquing how computer vision sees color also departs from how it exposes the limitations in algorithms – the oft-ignored inbuilt aberrations, biases, and limitations they have as technologies of perception. First, it is not possible to essentialize color, as it resides in "ways of seeing" (Berger, 2008) as much as what’s being seen (cf. Werner’s experiments in the introduction). As the historian and philosopher of technology Carolyn Kane (2014) proposes, the “fundamental polemics of color is that on one hand it inheres in objects in the external world, while on the other hand it is argued that color is a phenomenon of interior, subjective perception.” She further argues that > whether optical or algorithmic, color is not exclusively about vision. Rather, it is a system of control used to manage and discipline perception and thus reality. And perception … necessarily involves a field of forces and strategies–power and knowledge relations–that extend beyond any single viewer, physical technology, or image-artifact. This multifaceted condition of color—between things, people, and a wider network of actors—muddles the role of algorithmic seeing, exposing how its decision-making is partial and situated. Furthermore, within computer vision, color has always been present and inextricable from imaging technologies and their politics. If film cameras – as seen in the example of the Shirley cards – have relied on decisions and assumptions around how to create color, how can our Algorithmic Sea project probe the assumptions within algorithms? What are the decisions guiding algorithmic processes of color selection and analysis? Kane’s history of Chromatic Algorithms sees there is a “mathematical intensification and transformation of perception in the age of algorithmic optimization.” These shifts in perception marked by quantification and datafication have particular social, political, and commercial consequences, as the world is rendered “not for human eyes but to power automated monitoring, analysis, and decision-making” (p.5). This shift in how color is perceived in the algorithmic age is relevant to our project because it points to how algorithms change the way we experience colors, but also the power humans and algorithms have in these relations (Amoore, 2020). John Berger (2008) wrote that “when we ‘see’ a landscape, we situate ourselves in it.” In a similar fashion, the Australian posthuman and feminist phenomenologist Astrida Neimanis explores the complexity of how “we” see and experience water in Body of Water (2017), inspired by Audre Lorde (1984) and Adrienne Rich (1986). While Lorde asked what are the consequences of ignoring or minimizing differences, Rich’s Politics of Location emphasizes the individual information that comes from embodied materiality. Inspired by Neimanis, tracing how the personal and political converge in water-like fluidity as understood through and in the body informs our critical making decisions to combine human and computer seeing in all aspects of our project, especially in relation to perception. Our critical making practice aims to investigate color in computer vision as it stands precisely in the intersection between 1.) color as a situated and multifaceted perception that is both objective and subjective at the same time; and 2.) color as a social and historical construction in the algorithmic age, especially as it becomes mathematically-constituted and enmeshed with machine prediction-analysis. Thereby, our critical making had the objective of complicating and exposing these entanglements, attempting to visualize color by centering on multiplying entanglements of human-machine perception. We intentionally experimented with color seeing as a “flow” rather than a determinate data point, thus accepting the possibility of an uncertain, cloudy, doubtful, and partial algorithmic account (Amoore, 2020). Our process thus sought to use critical making to reframe power and control in algorithmic and human perception, while also interrogating what our expectations of seeing in the algorithmic age are (or could be).33:T6ee,In this section, we discuss the process behind the critical making of the Algorithmic Sea, engaging with the reflections it generated and connections to the scholarship on critical algorithm studies. We detail how, as described in our methodological and theoretical framing, our practice departs from the metaphor of water, focusing on movement, flexibility, mutability, flow, reflectivity, and opaqueness as core guiding notions. Our final result, the Algorithmic Sea, is composed of a net art (public online website) and a series of physical site-specific installations. The online interface allows users to submit an image of water. As they submit, they are invited to choose colors that relate to that image, first from a color picker and then by clicking on a pixelated version of the submitted image. Afterwards, the Google Cloud Vision machine learning algorithm is used to identify the two “most dominant” colors in the image. These colors are mixed in the form of metaballs, where the user can play with them before dropping them as a hexagon into the Algorithmic Sea, a space where these colors are shown as a pulsing hexagon alongside colors chosen by other visitors. The physical site-specific installations display the Algorithmic Sea through immersive projection-mapping, according to the possibilities afforded by each space. The Algorithmic Sea was built through the continuous experimentation with digital materialities which we discuss here as Opaqueness, Undulation, Constrainability, Fluidity, Liquidity, Crystallizability, and Dissolvability. Each of the subsections engages with one facet of our making process, and how our decisions lead to reflections on the situatedness of algorithms and color, as well as our own process as critical makers.34:T7f8,How can we address the opaqueness of how algorithms decide what are the dominant colors in an image? Our initial idea was to ask visitors to submit an image and represent how an algorithm saw the different colors. The challenge of our critical making process became how to elucidate and communicate such algorithmic operations, even when they are based on somewhat opaque (or boring) mathematical decisions. After experimenting with different algorithmic operations, we began exploring how they compared to what we perceived in the image. This led to our decision of, besides the algorithmic color selections, also having user color selections, allowing visitors to select a color from their memory and one color from a pixelated version of the image. This decision overlays the opaqueness and subjectivity of the algorithmic selection with that of human decisions, allowing people to actively partake in this process. Our result juxtaposes human and algorithmic perceptions together not to compare them, but to expose how such perceptions are particularly situated – not good or bad, incorrect or correct, but results of different ways of seeing. As Louise Amoore (2020) suggests, > machine learning is a composite figure in which humans learn collaboratively with algorithms, and algorithms with other algorithms (...) In contemporary machine learning, humans are lodged within algorithms, and algorithms within humans, so that the ethicopolitical questions are concerned less with asserting human control over algorithms and more with how features are extracted and recognized from a teeming data environment. Our reasoning for considering both the human and the computer in all phases of the process was to acknowledge and trouble the opacity of human and algorithmic ways of seeing. Rather than opting to represent algorithmic seeing as a black box, we show the ways algorithms are constantly being contested and acted on, and the pluralism that is possible from thinking of this as an agonistic collaboration (Crawford, 2015). 35:T632,To undulate means to move like a wave. Each hexagon in the sea fluctuates: they are never still, but shifting as they fade between two human-selected colors and two computer-selected colors. To arrive at such water-like undulation, we initially experimented with showing the results of human and algorithmic selections of color from an image as separate circles. They were exhibited side by side as a composition. However, we noticed, that this result came across as fixed and static. Instead of these separate shapes, we were struck by the impermanence of colors, and exhibiting these colors as flux, always in-between different points. For this, we were inspired by how the color of water itself changes from place to place, and time to time. The color of water can even serve as a signal of environmental destruction (e.g. during an algal bloom the sea turns red). These shifts and disruptions and alternating perspectives had to be represented in our way of visualizing human and algorithmic selections. We could then present color as in-between human and algorithmic seeing, never frozen, always shifting. In this way, each color is a partial account of the way both humans and computers see color. This serves to highlight the subjective nature of color (both in humans and machines), and our subtle acceptance of the pervasive digitization of our experience of color – which has become much more mathematical, as described by Kane (2014). Our final interface serves thus as a provocation, as it never stays really in any of the color selections of either humans or algorithms.36:T6c3,Each medium has a pacing – from a book with its inherent slowness of leafing through pages to the choices of how an exhibition is presented through the physical spacing of paintings on a wall. In the development of the Algorithmic Sea, we experimented with how to pace user interaction, and how such pacing poses questions to visitors about their perception of color. Ultimately, we saw the need to emphasize the fluid transitions between interfaces, in order to create a wave-like flow. This timing and rhythm was chosen to allow the user to make instinctual choices regarding their perception, as well as to mirror the flowing water movement that inspired the piece. Ultimately, we designed the experience to allow the user to reach a “flow” state (Csikszentmihalyi, 1990), a state of mind where the user loses the perception of time and is able to enter a different experience of reality while engaging with the interface. The flow of the interface became a crucial part of the narrative of the work. We begin by asking the visitor to select an image, and then ask them to pick a color from the color picker based on their memory. That color, rather than being hidden, is added to the hexagon on the bottom of the screen, which continues across the interaction. After user selections are made, the remaining colors on the screen are plotted into a 3D space before becoming the algorithmic selection – a subtle movement that forecasts the mathematical operation behind the algorithm. The movement of information across different interactions in the piece is made in a way that nothing disappears from the interface out of nowhere, thus guiding the user’s eyes and helping to understand the narrative flow of the work.37:T7bf,As we decided to ask users for color selections based on their memory of the image, a question emerged: which kind of interface should be created for this interaction? That is, the computer allows for a selection within the 16.777.216 colors of the RGB spectrum, but how can an interface facilitate a choice when there are so many options? As we experimented with different color pickers, we perceived that there are many different ways of selecting colors, and that each design brings particular sensitivities and affordances. Each color picker comes with an interface composed of sliders, buttons, textboxes (for color values), and/or direct manipulation. Moreover, because we know that humans understand color contextually, the juxtapositions within a color picker’s interface have direct consequences to our perception and selection. We had to depart from these limitations to consider how to acknowledge and work from a situated perspective. After experimenting with different color pickers and analyzing their different limitations, we ended up developing one that intentionally constrained user choices within pre-set colors. This mode of selection is particularly limiting as users cannot choose from all colors in the RGB spectrum, but from a very finite amount. However, we understood this restriction as beneficial for user creativity because it encouraged a quicker decision-making process. Even if the selection was an approximation of the color that the visitor remembered in the picture, they could select by clicking only once, while seeing all the options they had in an instinctual manner. We tested this result (Fig. 9) in a workshop, where a 10 year-old participant pointed to a limitation in the interface: the colors were too closely organized, which made it very difficult to make a particular selection. Incorporating this feedback reminded us of the iterative character of critical making, but also led to a pedagogical discussion within the workshop. 38:T553,How could we best represent user color selections as they make them? At first, colors would leave the screen, and users couldn’t see or interact with their choices before submitting them to the sea. We addressed this concern by returning to the idea of liquidity, and making the color selections not only visible, but tactile through metaballs – organic blobby shapes that meld together. Metaballs are special because of their capacity to smoothly connect, giving “shape to realistic, organic-looking creations, suitable for modelling human bodies, animals and other organic figures, which are very hard to model using traditional geometric methods” (Shen & Thalman, 1995). Through using shapes and visuals that originate from the natural world, the process of building the Algorithmic Sea echoes how seeing color extends through humans and computers, across the online-offline continuum. Ultimately, our use of the metaball design allowed us to create a space where the user could move the colors across the screen, seeing how they fluidly and organically mix and blend like paint. We found that this liquidity of movement, making colors tactile and malleable, was particularly useful for eliciting playful interaction with users. By defying the limitations of the computer screen, we could make the colors seem liquid even as part of a digital system.39:T5d7,Since the beginning of the process, we considered how the liveness of the website’s interactivity could unfold into a physical experience. Through our experiments, two important realizations occurred to us: we wanted the experience to be immersive, allowing people (and things) to penetrate the sea of colors; and we wanted the installation to be site-specific, mapped into the physicality of the space in which it was projected. These two aspects can reflect how digital materialities can dissolve into the physical space and allow for a particular form of interaction with colors. We experimentally decided to play with how the walls and the ceiling could allow visitors an expanded experience, beyond the confines of a single plane. To create this immersiveness, we created a tool inspired by the perspective of a cave, where the visitor's field of vision is completely occupied and the panoramic view eliminates external noise and facilitates the transition to this “color space.” This tool had to be capable of correcting the distortion of the projected image, dissolving the digital space into a physical environment. By manipulating the vertices of a virtual cube, one can fit the projection in the intersection of two walls with the ground (see Fig. 12). The three sides of the cube emulate the distortion of a person's perspective in three-dimensional space and thus make it possible to properly map a perspective from the virtual world to the site-specific physical environment.3a:Tc16,In making the Algorithmic Sea, we had to decide on a particular algorithm for analyzing color in an image. To explain this choice, it’s important to first describe how algorithms understand color in images. As explained by Smits & Wevers (2021), “Computers process digital images as three-dimensional matrices (n [height] x n [width] x n [depth/colours]) of numbers. Colour is expressed by stacking three layers of intensities, containing values of red, green and blue (RGB-values).” This means algorithms understand images as multi-dimensional points in a data space (see Fig. 14 and 15). A commonly used method for color analysis is the k-means clustering algorithm. What this algorithm does is cluster the colors in the image by first plotting them on the aforementioned 3-dimensional space and then iteratively finding the mean points of the graph. K-means clustering is an unsupervised machine learning method. Though the term unsupervised may suggest they act completely on their own, finding things by themselves without any oversight, that is a misconception. As Bechmann and Bowker (2019) suggest, unsupervised algorithms are controlled (or supervised!) in many different ways: the definition of the task and outcome variables, data, model selection, data preparation, and model training and deployment. For example, in the case of color selection, we tell the algorithm how many dominant colors we want it to identify in the data space. Interestingly, the k-means clustering algorithm is non-deterministic, so it won’t select the exact same result every time it is run – thus always leading to uncertainty and mystery. We chose to use a commercial computer vision system (Google Cloud Vision), as opposed to the k-means clustering algorithm, to foreground the experience of algorithmic opaqueness. That is to say, while Google's infrastructure of algorithmic analysis is omnipresent on the web, it is rarely fully understood. For example, Google does not explain the particular way its algorithm of color analysis works (i.e. the exact method for clustering the colors). We thought this opacity offered an interesting provocation: the Algorithmic Sea uses Google's "cloud" to analyze color, and yet our understanding of this process is only partial, even after interrogation and experimentation. Each algorithm, as any scientific apparatus, creates boundaries of what matters or doesn’t, enacting “agential cuts” (Barad, 2003, p. 815) in the data space it operates on. These cuts are important as they define, in a strict sense, how algorithms see color in an image. As suggested by Amoore (2020), it is the "condensation" of data traces that defines what matters and which actions are taken – in this case, transforming all possible colors that one sees in an image into a single “dominant color.” These decisions are based on the limitations of each way of cutting, which often are not fully described or explained. From this an important question emerges: what is getting privileged (or hidden) when seeing gets reduced to calculations in a data space? 3b:T9c7,While the choices around cutting the data space define what gets seen, the organization of the data space influences how such cutting takes shape. This means that the choice of a color standard also defines how an algorithm sees an image’s dominant color. In the Introduction, we spoke of Werner’s charts and how they created a standard for representing color in the 18th century, with particular limitations. While photography went on to assume other standards (such as the problematic Shirley cards discussed in the Background section), computers rely on mathematical systems to represent and record colors. These computational standards include RGB (Red, Green, Blue), CMYK (Cyan, Magenta, Yellow, and Key), HSV (Hue, Saturation, Value), and others. These mathematical standards compress and simplify the colors we see through our eyes into a colored computational image. For example, RGB uses three color values (red, green, and blue), each ranging only between 0 and 255. Such values are then added computationally to generate one of the 16.777.216 possible colors in the system. Historically speaking, the decision for representing color as R, G, and B emerges from the Young–Helmholtz’s theory of trichromacy of the human eye. Such theory proposed that the “human eye contains cells sensitive to the three primary colours of red, green, and violet blue” (Arnkill, 2013). Though still relevant today, and embedded in digital systems (including the Algorithmic Sea), this understanding of color is still disputed and interrogated. As seen in the experiment in Fig. 15, changing the color standard directly impacts how colors are organized in the three-dimensional space, and leads to different color selections (see Jurio et al, 2010 for a computer science proof of this). This is perhaps best exemplified by the work of digital humanities scholar Leonardo Impett, which suggests that computer vision could potentially be taught to see according to the the very different color schemes of the Italian Renaissance (rather than RGB, for example). Our decision to use RGB in the Algorithmic Sea gives continuity to particular histories and infrastructures of human/algorithmic seeing. Moreover, such a decision directly biases what colors are chosen in our piece and made visible to users. This leads to many questions for future critical practice: could other standards of color be used to reshape algorithmic ways of seeing? What colors are being ignored, omitted, or deprioritized?3c:T639,Algorithms "see" differently than humans. Human perception of color is intensely contextual, being constantly tuned according to the environment. For example, our perception of lime green changes when it is surrounded by blue (see Fig. 16). This is called color constancy, and it is a process not yet fully explained by scientific research – the most accepted explanation is that both our retina and our cortex are involved in this process somehow (Arnkill, 2013). Algorithms, in contrast, don’t possess this innate capability of compensating for different illumination conditions when looking at images. They see images as discrete and flat objects, rather than in relation to a wider environment. In the Algorithmic Sea, the algorithm for selecting the dominant color does not take into consideration the wider environment from which the submitted image was taken. However, we juxtapose this algorithmic selection with the human's remembered color. This comparison is meant to inspire contemplation about these different forms of perception and how context influences them. This difference between how algorithms and humans see serves to point how contextualization takes shape. The way humans see color is shaped by biological and emotional processes, including that of color constancy. Computers are also shaped by other forms of context – the algorithm, the color standard, the variables, etc. How could algorithms also be taught to see color in connection to its situated environment? And what are the creative potentials of adding enhanced forms of context in computer seeing? 3d:T892,In this interactive essay, we have discussed our critical making process for the Algorithmic Sea net art and site-specific installations. Our particular perspective on critical making builds on both notions of critical making as a reflective practice and a continuous connection/contribution to scholarly inquiry. The specificity in our process is its focus on a fluid practice of collaboration across researchers and digital materiality, as well as an ongoing experimental disposition. Our practice builds upon a wider background of historical and theoretical literature on color algorithms. Based on these threads, we established our critical making inquiry into how color is situated and multifaceted, but also historically and socially constituted. Our critical making experimented with algorithmic and human perceptions of color to better understand their particularities. Our process highlights the challenges we faced to represent this locatedness and contextuality, as well as the many issues that emerge from creating an interactive experience that seeks to spark reflection. The metaphor of water and its qualities helps center flux and impermanence in the making process. Finally, we discussed crucial limitations of the algorithmic ways of seeing present in the making of the Algorithmic Sea – cutting, standardizing, and contextualizing. Our objective in considering these limitations was highlighting the situatedness of all ways of seeing. Although algorithms are often presented as neutral and impartial, they have in their "lenses" intrinsic biases and limitations. If “new kinds of literacy [are] needed to both comprehend and intervene in the automated visual systems that are embedding themselves in our everyday lives” (McCosker & Wilken, 2020), we believe a fluid critical practice can offer a starting point. It both serves as a mode for critical inquiry that centers collaboration with different perspectives and materialities, and the movement from critique into evocative public intervention. Using the metaphor of water as a reflecting pool for critical making, algorithmic and human ways of seeing color are unsettled as situated, impermanent, and flowing. 3e:T1099,Agre, P. (1997). Toward a Critical Technical Practice: Lessons Learned in Trying to Reform ai. In G. Bowker, L. Gasser, L. Star, & B. Turner (Eds.), *Bridging the Great Divide: Social Science, Technical Systems, and Cooperative Work* (pp. 130–157). Erlbaum Hillsdale. Amoore, L. (2020). *Cloud Ethics: Algorithms and the Attributes of Ourselves and Others*. Duke University Press. Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias. *ProPublica*, *23*. Arnkil, H. (2013). *Colours in the Visual World*. Aalto University School of Arts, Design and Architecture. Barad, K. (2003). Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter. *Signs: Journal of Women in Culture and Society*, *28*(3), 801–831. https://doi.org/10.1086/345321 Bechmann, A., & Bowker, G. C. (2019). Unsupervised by any other name: Hidden layers of knowledge production in artificial intelligence on social media. *Big Data & Society*, *6*(1). https://doi.org/10.1177/2053951718819569 Benjamin, R. (2019a). *Captivating technology: Race, carceral technoscience, and liberatory imagination in everyday life*. Duke University Press. Benjamin, R. (2019b). *Race after technology: Abolitionist tools for the new Jim code*. Polity Press. Berger, J. (2008). *Ways of Seeing*. Penguin UK. Bogers, L., & Chiappini, L. (2019). *The Critical Makers Reader:(Un)learning Technology: Vol. (12)*. Institute of Network Cultures. Buolamwini, J., & Gebru, T. (2018). *Gender shades: Intersectional accuracy disparities in commercial gender classification*. *Conference on Fairness, Accountability and Transparency*, 77–91. Crawford, K. (2016). Can an Algorithm be Agonistic? Ten Scenes from Life in Calculated Publics. *Science, Technology, & Human Values*, *41*(1), 77–92. https://doi.org/10.1177/0162243915589635 Csikszentmihalyi, M. (2008). *Flow: The Psychology of Optimal Experience*. Harper Perennial Modern Classics. Harwood, G. (2019). Teaching Critical Technical Practice. In L. Bogers & L. Chiappini (Eds.), *The Critical Makers Reader:(Un) learning Technology*. Institute of Network Cultures. Ingold, T. (2013). *Making: Anthropology, Archaeology, Art and Architecture*. Routledge. Jurio, A., Pagola, M., Galar, M., Lopez-Molina, C., & Paternain, D. (2010). A Comparison Study of Different Color Spaces in Clustering Based Image Segmentation. In E. Hüllermeier, R. Kruse, & F. Hoffmann (Eds.), *Information Processing and Management of Uncertainty in Knowledge-Based Systems. Applications* (Vol. 81, pp. 532–541). Springer. https://doi.org/10.1007/978-3-642-14058-7\_55 Kane, C. L. (2014). *Chromatic algorithms: Synthetic color, computer art, and aesthetics after code*. University of Chicago Press. Markham, A. N., & Pereira, G. (2019). Analyzing public interventions through the lens of experimentalism: The case of the Museum of Random Memory. *Digital Creativity*, *30*(4), 235–256. https://doi.org/10.1080/14626268.2019.1688838 McCosker, A., & Wilken, R. (2020). *Automating Vision*. Routledge. Neimanis, A. (2017). *Bodies of Water*. Bloomsbury Publishing. Pasquinelli, M., & Joler, V. (2020). The Nooscope manifested: AI as instrument of knowledge extractivism. *AI & Society*. https://doi.org/10.1007/s00146-020-01097-6 Ratto, M. (2011). Critical Making: Conceptual and Material Studies in Technology and Social Life. *The Information Society*, *27*(4), 252–260. https://doi.org/10.1080/01972243.2011.583819 Roth, L. (2009). Looking at Shirley, the Ultimate Norm: Colour Balance, Image Te. *Canadian Journal of Communication*, *34*(26). Schorr, S. (2020). *Twilights: Ephemeral Metallics*. https://www.sarahschorr.com/Artist.asp?ArtistID=49550&Akey=48347V9L&ajx=1#!Group1_Pf193729 Shen, J., & Thalmann, D. (n.d.). *Interactive shape design using metaballs and splines*. Smits, T., & Wevers, M. (2021). The agency of computer vision models as optical instruments. *Visual Communication*. https://doi.org/10.1177/1470357221992097 Soon, W., & Cox, G. (2021). *Aesthetic Programming: A Handbook of Software Studies*. Open Humanities Press. Syme, P. (2018). *Werner’s Nomenclature of Colours: Adapted to Zoology, Botany, Chemistry, Mineralogy, Anatomy, and the Arts*. Smithsonian Books.3f:[{"id":"c44ffa9513cd41f23ea136b034d074f67de77609c6a78d96e0036d6e92d076479bcfcbd2e3fee0bd05f2ed4e61ca438de32dd65c8c9959a5858ae5a1411e9e13","href":"https://static.observableusercontent.com/files/c44ffa9513cd41f23ea136b034d074f67de77609c6a78d96e0036d6e92d076479bcfcbd2e3fee0bd05f2ed4e61ca438de32dd65c8c9959a5858ae5a1411e9e13","name":"algorithmic sea.jpg","type":"image/jpeg","lastModified":1619814917616.637,"size":47996},{"id":"e3aab054580f9a43aab9a238555d0d1f163331924bf0cac14a68bd2b0562d2091c71117bdb66083a26e84b59801b44adcb32d95520d49878366975e3ade49376","href":"https://static.observableusercontent.com/files/e3aab054580f9a43aab9a238555d0d1f163331924bf0cac14a68bd2b0562d2091c71117bdb66083a26e84b59801b44adcb32d95520d49878366975e3ade49376","name":"download.png","type":"image/png","lastModified":1619814710057.07,"size":1654339},{"id":"0c95deed3ecabd0bf9a03084197d8f5540c0f3a8205eed08d9059420e2034f006d0739d6062210e549012fba96857e2b84b2cf4e87bb56eb6ed0c7f8345aff6a","href":"https://static.observableusercontent.com/files/0c95deed3ecabd0bf9a03084197d8f5540c0f3a8205eed08d9059420e2034f006d0739d6062210e549012fba96857e2b84b2cf4e87bb56eb6ed0c7f8345aff6a","name":"IMG_0589-min.png","type":"image/png","lastModified":1626955339378.361,"size":4450646},{"id":"041cd1095fbc725afd075ff46a4de6d9e1cf17c8c54687eed28cc967c308ad23be746712d8413d3fe58618a1af1659580abd688c3525c5d179ba963cceae390b","href":"https://static.observableusercontent.com/files/041cd1095fbc725afd075ff46a4de6d9e1cf17c8c54687eed28cc967c308ad23be746712d8413d3fe58618a1af1659580abd688c3525c5d179ba963cceae390b","name":"PosterFINALSchorr.pdf","type":"application/pdf","lastModified":1626955413707.109,"size":1885577},{"id":"084c4b34f9e509ed00018fe43dec7b8376f4446af78725b1495937934a0508c851a3cea126da6b59cae93d88e6d3174a0b802a5ef7a6d5cd458bd5e130bb6178","href":"https://static.observableusercontent.com/files/084c4b34f9e509ed00018fe43dec7b8376f4446af78725b1495937934a0508c851a3cea126da6b59cae93d88e6d3174a0b802a5ef7a6d5cd458bd5e130bb6178","name":"YouCut_20210422_130243628.mp4","type":"video/mp4","lastModified":1626956061724.841,"size":12520848},{"id":"9ece6413a36c9828c2f9b24c02710b90485f787eedb6986c9f01bf4ed1d3f47b71bd0de8add0a8c4b02bec0ac6d1c1f9c0aa62803380a6deb7f6cc287b520ff8","href":"https://static.observableusercontent.com/files/9ece6413a36c9828c2f9b24c02710b90485f787eedb6986c9f01bf4ed1d3f47b71bd0de8add0a8c4b02bec0ac6d1c1f9c0aa62803380a6deb7f6cc287b520ff8","name":"algo-sea-cropped.mp4","type":"video/mp4","lastModified":1626970535472.742,"size":7462194},{"id":"bc107ce12904420c7570cbff02c3830bb2932f2bf77a3e3c8d1f9c6bc0d9acd4da9a74dff582565ebbc0e1a533cd7ccb5a3dc92c7180c1aa1ba05f336d5e62a1","href":"https://static.observableusercontent.com/files/bc107ce12904420c7570cbff02c3830bb2932f2bf77a3e3c8d1f9c6bc0d9acd4da9a74dff582565ebbc0e1a533cd7ccb5a3dc92c7180c1aa1ba05f336d5e62a1","name":"color-pallete.mp4","type":"video/mp4","lastModified":1619691397886.878,"size":4415486},{"id":"430693148254581249b3b289a1ad1bb10a6566790d3fe06ebe1dfb394d25196ca0dc62d15b7a6f23addb7ec092d4653333ccd5b6e95cdee8d089cbdfa66bf558","href":"https://static.observableusercontent.com/files/430693148254581249b3b289a1ad1bb10a6566790d3fe06ebe1dfb394d25196ca0dc62d15b7a6f23addb7ec092d4653333ccd5b6e95cdee8d089cbdfa66bf558","name":"Screenshot 2021-04-29 at 12.32.14.png","type":"image/png","lastModified":1619692342535.281,"size":640757},{"id":"1ede24992fbb6e5fc389a045bde325bc047cbf39bc3441118dd870c6caf6eb6d9cfd89cdb536dc7e1230840deb8db7556b3c55d67e06fb883f9e2ba5f4d3c256","href":"https://static.observableusercontent.com/files/1ede24992fbb6e5fc389a045bde325bc047cbf39bc3441118dd870c6caf6eb6d9cfd89cdb536dc7e1230840deb8db7556b3c55d67e06fb883f9e2ba5f4d3c256","name":"proj0.jpg","type":"image/jpeg","lastModified":1619787044374.822,"size":31974},{"id":"35f94e6c45ac65c587f1c0e62c3f8a616cb3ed33ecc76a8a0c11f218be89ff1a571628b720760755965c32c6cc3f29454cb33c91b419a261c5a05d9e0d215633","href":"https://static.observableusercontent.com/files/35f94e6c45ac65c587f1c0e62c3f8a616cb3ed33ecc76a8a0c11f218be89ff1a571628b720760755965c32c6cc3f29454cb33c91b419a261c5a05d9e0d215633","name":"proj1.jpg","type":"image/jpeg","lastModified":1619787044042.378,"size":36577},{"id":"a735e735f46a883c9824dfbbaee495c63f58f6fd2ca5b2500837ebd39896dac84656b5c17a47a567b8c7c3fde5ae83ffa536f6e5a3822d512ebd6a20e94f405e","href":"https://static.observableusercontent.com/files/a735e735f46a883c9824dfbbaee495c63f58f6fd2ca5b2500837ebd39896dac84656b5c17a47a567b8c7c3fde5ae83ffa536f6e5a3822d512ebd6a20e94f405e","name":"Screenshot 2021-04-30 at 14.43.37.png","type":"image/png","lastModified":1619786644476.956,"size":176819},{"id":"126a81386ede163650b69b5fbd3d2a866b9ac2730d5632f49844584918ef01b7b05cf75e12f20a2bc820e2c984f99434a86371ea80051d3373355eec413025da","href":"https://static.observableusercontent.com/files/126a81386ede163650b69b5fbd3d2a866b9ac2730d5632f49844584918ef01b7b05cf75e12f20a2bc820e2c984f99434a86371ea80051d3373355eec413025da","name":"Screen Recording 2020-08-11 at 20.15.47.mp4","type":"video/mp4","lastModified":1619829264672.951,"size":2252369},{"id":"8edb6743e28483d00beae04b1005d4c8df7a018438c236734d6da2b3cbd768d569088e576147965a952f7fee9cf7a1979475311f02061dcba967412840e33f00","href":"https://static.observableusercontent.com/files/8edb6743e28483d00beae04b1005d4c8df7a018438c236734d6da2b3cbd768d569088e576147965a952f7fee9cf7a1979475311f02061dcba967412840e33f00","name":"Screenshot_2020-04-02 11.18.49_QBGcoC.png","type":"image/png","lastModified":1619827807634.852,"size":525245}] 2e:{"title":"Algorithmic Sea:","cells":[{"id":0,"value":"# Algorithmic Sea: \n## The Fluid Critical Making and Seeing of Color","pinned":false,"hidden":false,"mode":"md"},{"id":5623,"value":"*Authors: Gabriel Pereira, Sarah Schorr, & Carlos de Oliveira*","pinned":false,"hidden":false,"mode":"md"},{"id":98,"value":"introVideo = html`

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":122,"value":"*In this interactive essay, we discuss the critical making process behind the Algorithmic Sea, available on [algorithmicsea.com](https://www.algorithmicsea.com/).*","pinned":false,"hidden":false,"mode":"md"},{"id":133,"value":"md``","pinned":false,"hidden":false,"mode":"ojs"},{"id":45,"value":"### Introduction\n","pinned":false,"hidden":false,"mode":"md"},{"id":48,"value":"

“And how important it is to know how
\nto mix on the palette those colours
\nwhich have no name, and yet are the
\nreal foundation of everything.”
\nVincent van Gogh\n

","pinned":false,"hidden":false,"mode":"html"},{"id":7,"value":"$30","pinned":false,"hidden":false,"mode":"md"},{"id":5063,"value":"FileAttachment(\"download.png\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":5067,"value":"*Fig. 1 – In this experiment, we show all the 16.777.216 colors of the RGB spectrum. This visualization, though mathematically complete, feels partial and insufficient.*","pinned":false,"hidden":false,"mode":"md"},{"id":5711,"value":"FileAttachment(\"IMG_0589-min.png\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":5684,"value":"*Fig. 2 – To make it even more interactive, [you can download](${await FileAttachment(\"PosterFINALSchorr.pdf\").url()}) this visualization and print it as an A3 poster. The poster includes blank spaces for you to paint any colors you feel are missing.*","pinned":false,"hidden":false,"mode":"md"},{"id":82,"value":"md`### Critical Making: A fluid practice of collaboration`","pinned":false,"hidden":false,"mode":"ojs"},{"id":21,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":87,"value":"FileAttachment(\"Screenshot 2021-04-29 at 12.32.14.png\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":116,"value":"_Fig. 3 – Some of our experiments, [saved on OpenProcessing.](https://openprocessing.org/curation/68898) These sketches, as well as [the artwork's code](https://github.com/The-Color-of-Water/algorithmicsea) and [this article's code](https://observablehq.com/@vamoss/algorithmic-sea), are available for all to fork, remix, and change._","pinned":false,"hidden":false,"mode":"md"},{"id":4826,"value":"### Background: How color algorithms connect to our practice","pinned":false,"hidden":false,"mode":"md"},{"id":4829,"value":"$32","pinned":false,"hidden":false,"mode":"md"},{"id":4814,"value":"allColors","pinned":false,"hidden":false,"mode":"ojs"},{"id":4990,"value":"*Fig. 4 – Live representation of all the colors submitted, each line representing an image and the color selections it generated. The first two columns are human selections, and the other are made by the AI (the amount of AI colors changes because it is not a fixed number).*","pinned":false,"hidden":false,"mode":"md"},{"id":4818,"value":"wheel3","pinned":false,"hidden":false,"mode":"ojs"},{"id":4995,"value":"*Fig. 5 – Live radial representation of the last submitted color selections. On the outer ring are the human selections, and the inner ring are AI selections (with the length of each line correlated to the “certainty” of the AI). We find measuring these results limiting, and thus present it here only as a way to aesthetically explore how they show differences between human and machine perceptions.*","pinned":false,"hidden":false,"mode":"md"},{"id":4833,"value":"### Process: Interactive flow as a critical practice","pinned":false,"hidden":false,"mode":"md"},{"id":4836,"value":"$33","pinned":false,"hidden":false,"mode":"md"},{"id":4839,"value":"#### Opaqueness: The algorithmic uncertainty","pinned":false,"hidden":false,"mode":"md"},{"id":5026,"value":"$34","pinned":false,"hidden":false,"mode":"md"},{"id":5402,"value":"#### Undulation: Conjuring impermanence","pinned":false,"hidden":false,"mode":"md"},{"id":5029,"value":"$35","pinned":false,"hidden":false,"mode":"md"},{"id":5007,"value":"#### Fluidity: Following the current","pinned":false,"hidden":false,"mode":"md"},{"id":5035,"value":"$36","pinned":false,"hidden":false,"mode":"md"},{"id":4859,"value":"FileAttachment(\"Screenshot 2021-04-30 at 14.43.37.png\").image()","pinned":false,"hidden":false,"mode":"ojs"},{"id":4863,"value":"*Fig. 6 – This scheme describes the functioning of the Algorithmic Sea's user interaction.*","pinned":false,"hidden":false,"mode":"md"},{"id":5725,"value":"walkthroughVid = html`

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5736,"value":"*Fig. 7 – A video showing an early experiment with the Algorithmic Sea at Schorr's studio. In it, we see a user selecting their color and submitting it to the projected sea.*","pinned":false,"hidden":false,"mode":"md"},{"id":5004,"value":"#### Constrainability: Color pickers as limitations","pinned":false,"hidden":false,"mode":"md"},{"id":5032,"value":"$37","pinned":false,"hidden":false,"mode":"md"},{"id":254,"value":"html`\n

\n\n\n`\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":5093,"value":"*Fig. 8 – Two different color pickers from the iro.js library. As shown above, each of the different color pickers affords a different bias for color selection: on the left, it's easier to adjust the color's saturation and value, and on the right the hue and saturation. Experiment with them to see how the same color shows up in a different way in the other color picker.*","pinned":false,"hidden":false,"mode":"md"},{"id":64,"value":"existingVideo = html`

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":5054,"value":"*Fig. 9 – The limited color picker we created.*","pinned":false,"hidden":false,"mode":"md"},{"id":5009,"value":"#### Liquidity: Making water movement tactile through metaballs","pinned":false,"hidden":false,"mode":"md"},{"id":5038,"value":"$38","pinned":false,"hidden":false,"mode":"md"},{"id":5558,"value":"p5(metaBallSketch, \"metaballContainer\")","pinned":false,"hidden":false,"mode":"ojs"},{"id":5152,"value":"*Fig. 10 – The movement of metaballs and their organic and water-like shape. In this visualization you can experiment with changing their color and other parameters.*","pinned":false,"hidden":false,"mode":"md"},{"id":5020,"value":"#### Crystallizability: Hexagons as connective shapes","pinned":false,"hidden":false,"mode":"md"},{"id":5041,"value":"Our first experiments with representing the color selections in the sea worked with circles. Our shift to hexagons was inspired by two aspects: 1. Hexagons are a common shape in nature, due to their geometrically-fitting characteristics (e.g. the structure of snowflakes, bee hives, or even the foam of the sea); and 2. Hexagons are directly related to the tradition and visuals of hexagon quilting. In quilting, hexagons fit together like puzzle pieces and the intricate stitching is both visible and obscured. At close inspection and through revealing the underside of the quilt, the construction and tedious architecture can be understood and admired. But, at first glance, the adjoining hexagons are mostly viewed as a continuous field (like a sea). Through looking at quilting and understanding how these shapes could fit together, we considered how a sea is composed of individual parts (crystals and water molecules), each with their own characteristics.","pinned":false,"hidden":false,"mode":"md"},{"id":5157,"value":"circleVideo = html`