The title that opens this reflection is a statement by computer scientist Sandra Avila, who is also a co-author of the lines that will flow here. Her account took place during the recording of a podcast on science and feminism.[1] In that situation, Avila recounted a non-trivial research experience. She may never have been able to talk about it with her colleagues in computer science, let alone with collaborators in dermatology who offered their “expertise advice.” It was in informal conversations with feminist researchers that Avila found a safe place to talk about a situation that had made her cry with anger because she felt cheated.
Avila was a member of a computer science research group that was investigating a variety of topics. When they decided to start research into skin cancer, they immediately recognized the need to establish partnerships and seek the technical opinion of researchers in the field of dermatology.[2] Her role in that collaborative project was to develop a tool based on Artificial Intelligence, more specifically Machine Learning,[3] to facilitate and optimize diagnosis, based on the identification of patterns in medical imaging exams.
Throughout her life and career, Avila was motivated to work on projects that could improve people’s lives. She was enthusiastic about science and its ability to democratize and provide answers to “social” problems. The idea of developing a tool that had a practical application in healthcare sounded like an excellent opportunity to put her ideology into practice.
Montage by Sandra Avila with images of skin lesions from ISIC Archive
Avila received academic and public recognition for her research and determination to improve algorithms that included the identification of cancer in black skin, which was neglected by the tools available.[4] This exclusion resulted in a higher mortality rate for this type of cancer in people with black skin, since late diagnosis decreases the chances of remission.[5]
On learning that the tools available did not include samples of people with black skin, meaning that they were not gauged to identify a variety of skins and thus neglected a section of the population, Avila decided to work on this “technical flaw”[6] and proudly declared her defiance and determination to only make the tool available when it was robust enough, but also non-exclusive.[7] Her little-big insubordination was to resist the processes of neoliberal acceleration, seeking to maintain as much as possible the commitment to a technology that is inclusive and anti-racist. The ambition cultivated is democratic access. Responsibility means making science a tool for reducing social inequality. There can be no good technique without ethical commitment and no responsibility that doesn’t start from citizen commitment.[8]
However, one day Avila realized that the recommendation she had received from the dermatologist to exclude samples of fingernails, palms, and feet did not simply exclude “confusing samples,” as the specialist stated, but the possibility of identifying the highest incidence of skin cancer in black populations, which occurs precisely in those parts of the body with the lowest melanin index.[9]
Her experience is not trivial, not because it is a complex and problematic collaboration outside the curve, but because she had to rely on expertise and was disappointed by a late discovery. She realized too late in the process that her work was helping to perpetuate exclusion.
Did the dermatologist intentionally recommend the exclusion of samples to deliberately harm the black population? We don’t know. What matters is highlighting the structural mechanisms of exclusion and the maintenance of privilege.
This event allows us to raise questions about the permanence of eugenic strategies at the heart of scientific endeavors, as well as about the challenges of interdisciplinarity. Interdependence and collaboration between disciplines, a possible definition for interdisciplinarity, is a structural dynamic of scientific practices.[10] It is not an eccentricity, but rather a product of specialization between areas, something that has intensified in the disciplinary trajectory of modern science. The modern scientific enterprise creates the conditions of existence for interdisciplinarity, characterizing itself as a scenario of broad collaboration between scientists who draw on the expertise of other areas.
The effort to collaborate is not based on pure curiosity or inventiveness, originality, or an attempt to “think outside the box.” It is common for areas of knowledge to depend on evidence, methodologies, or concepts from other areas to support their own, relying on propositions for which they themselves have no knowledge—something that can be characterized as an epistemic dependence that implies the need for testimony from other scientists (Jacksland, 2021).
There are many possible arrangements and forms of interdisciplinary dialogue, exchanges, and loans. Collaborations that can be characterized by the meeting of cognitively divergent disciplines (Jacksland, 2021), to varying degrees—some more familiar and close, others radically distant. This is why such borrowing, the way in which the findings of one discipline are lent to another, can take place under different conditions, both cognitively and politically. The prestige of certain areas and scientists is always at stake in these disciplinary transactions, as Avila’s situation highlights. She is a young black woman, dependent on medical testimony—which enjoys great authority in Brazilian society. This is a highly hierarchical and masculinist field, in which whiteness is normalized (Castro, 2022).
In addition to interpersonal power dynamics, a recurring feature of interdisciplinary exchanges is the lack of adequate multidisciplinary spaces and occasions in which collaboration can take place in a more horizontal or in-depth manner. There is a certain instrumentalization of these exchanges, justified by the false emergency that the sciences need to deal with, largely because they are part of techno-scientific arrangements that respond to the demands and temporality of the market.
There are situations of knowledge encounters in which the epistemic conditions result from such dependence on impenetrable knowledges whose representative experts must provide testimony.[11] Are there ways of ensuring that such expert translators can adhere to epistemic norms that guarantee reliability to those exposed to a relationship of opaque or translucent epistemic dependence (Jacksland, 2021)?
The consequences of this dependence are epistemological and political, as Avila’s case illustrates. Is it an exaggeration to consider that blind trust can lead to naivety and collaboration in eugenic projects? What does it mean to call eugenic such endeavors that nonetheless seek to distance themselves from the Nazi strategies and racist projects of the past?
Eugenics policies explicitly intertwined with race management in Brazil, which were characterized by hygiene strategies aimed at “progress” and that expressed concerns about the fate of the nation and its degeneration associated with racial mixing (Schwarcz, 1993). This agenda and vocabulary are criminal today, but there are ventures that address a certain improvement of the human being promoted by biotechnology (Marini) or those associated with a new rhetoric of difference associated with disability today (Lopes, 2015),[12] which highlight the “invisible” permanence of discrimination in the management of population.
As anthropologist Marko Monteiro (2012) points out, eugenics is central to understanding the processes driven by technology and biomedicine today, as it is a historical example of the most radical expression of a logic of politicizing life. However, it is not a question of accusing biotechnology of being eugenic, which would be simplistic, attributing only a nefarious intention to biotechnology. On the contrary, it’s about understanding its forms of management and social control.
Do genetic improvement strategies share social and political implications with algorithms that exclude the identification of skin cancer in black people? Is selecting and preventing birth also a necropolitics of letting die? These are rhetorical questions that we do not intend to address here. Rather, this essay’s purpose is simple: to narrate an event in which a well-meaning scientist discovered that her work perpetuated exclusion. We believe that scientists should talk more about their “failures,” because there is a lot to learn from them.
Notes
[1] MUNDARÉU – Podcast de Antropopologia: https://mundareu.labjor.unicamp.br/21-todo-laboratorio-e-sobre-pessoas/
[2] Dermatology specialists were consulted at different times. Various researchers were consulted, to help find databases with samples for analysis, or to assess the progress of the development of the tools. The results produced were presented for evaluations of the algorithms’ recognition capacity, in order to jointly raise hypotheses about the reasons for the algorithms’ recognition difficulties. Together they asked themselves: why are the algorithms getting easy recognitions wrong, but at the same time getting difficult recognitions right (from a medical perspective)?
[3] Machine learning is a field of artificial intelligence focused on building algorithms that can “learn” patterns from data about a certain phenomenon. The algorithms are usually applied to new data to make predictions or provide other useful results, such as identifying tumors in scans.
[4] The specialized literature that informed her project, as well as the databases available, was limited to data on lesions on white skin. It wasn’t Avila’s choice, but the only public data available was from white people.
[5] Regarding the challenge and delay in diagnosis, Avila warns: the difficulty of diagnosis may be related to the bias—on the part of doctors and the population, prior to the bias of the algorithms—that dark pigmentation is protective against skin cancer. However, if a person has skin, they can develop skin cancer. Even today, the majority of textbooks that serve as roadmaps for diagnosing skin diseases do not include images of skin diseases as they appear in black people, or when they do, it is no more than 10% [ref: https://www.nytimes.com/2020/08/30/health/skin-diseases-black-hispanic.html]. If dermatologists only learn to diagnose a lesion by its characteristics on white skin, they may not be able to classify it on black skin as the patterns may be different.
[6] Ruha Benjamin (2019) has already warned us about the discriminatory mechanisms present in scientific and technological development, which are not “side effects” but are part of a “discriminatory design” project.
[7] In an article published by Fapesp, the institution that funded the research, the researcher declares her commitment and the need to democratize the technology: “The ways in which tumors present themselves and the aggressiveness with which they evolve, however, are different in each of the six existing skin phototypes. Comparing the image of a skin lesion from a black person with a database made up of samples from white people could lead the system to make mistakes. ‘We need a database that mirrors Brazilian diversity.'” Article available at: https://revistapesquisa.fapesp.br/diagnosticos-digitais/
[8] There is no actual software yet, no tool to be implemented. So far, the research has produced a solution (and various models) that can correctly classify images of melanoma lesions and benign lesions with a high accuracy rate (+95%). The researcher pointed out that developing the software is the fastest part of the process, but there’s no point in developing it if the model doesn’t work for everyone yet. The software in this process is the interface for using the model. It’s worth noting that this solution was built by many hands (undergraduate, master’s, doctoral and post-doctoral researchers and professors). There are several works in the literature and applications for diagnosing skin lesions. But the tool was not necessarily inspired by others.
[9] Acral melanoma is not related to sun exposure, as it usually develops on the palms of the hands, soles of the feet and nails. Black-skinned patients therefore face a poor prognosis, with increased morbidity and mortality, which is usually the result of late diagnosis. In addition, the acral regions, especially the feet, are often overlooked by dermatologists in physical assessments (Kelly et al., 2009; 2015).
[10] It is important to differentiate interdisciplinarity from other disciplinary arrangements or encounters, such as transdisciplinarity, as these are projects with different purposes and implications. Nelson Maldonado-Torres (2016) suggests that transdisciplinarity should explicitly be based on an emancipatory and decolonizing orientation of interdisciplinary spaces. The transdisciplinarity proposed by Maldonado-Torres is closer (or is on parallel tracks) to pluriepistemic or interepistemic claims (Barbosa Neto & Goldman, 2022; Guimarães, 2022), aimed at recognizing neglected knowledges among hegemonic scientific practices. These ethnic studies and non-hegemonic knowledges claimed for the consolidation of transdisciplinary spaces are not committed to cultivating tolerance towards diversity, but rather to dismantling forms of power. The same does not apply to interdisciplinarity, which is characterized as collaboration between legitimized, albeit hierarchical, sciences.
[11] In the social sciences, the example presented by Jacksland refers to the work of Karen Barad, a theoretical physicist and feminist philosopher of science, who becomes a “privileged translator.” His point is to shed light on the use of quantum mechanics in critical theory, based on the work developed by Vicky Kirby—for whom the integration between quantum mechanics and critical theory is only possible with the help of translators. This is an example of broad interdisciplinary borrowing, where few translators have joint experience in both disciplines, as is the case with Karen Barad.
[12] In rescuing the historical developments associated with the categories of intelligence in order to understand their connection to the emergence and transformations of the category of disability in the present day, Lopes refers to the racist and eugenicist scene of the 19th century, highlighting the historical link between race and sexuality, thus constituting a biopolitical challenge in the management of individuals and populations.
References
Barbosa Neto, E. R., & Goldman, M. (2022). A maldição da tolerância e a arte do respeito nos encontros de saberes – 1a. Parte. Revista De Antropologia, 65(1), e192790. https://doi.org/10.11606/1678-9857.ra.2022.192790
Castro, Rosana. Pele negra, jalecos brancos: racismo, cor(po) e (est)ética no trabalho de campo antropológico. Revista de Antropologia, São Paulo, v. 65, n. 1, e192796, 2022. ISSN: 1678-9857. DOI: https://doi.org/10.11606/1678-9857.ra.2022.192796. Disponível em: https://www.revistas.usp.br/ra/article/view/192796. Acesso em: 13 de junho de 2022
Benjamin, Ruha. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge: Polity Press, 2019.
Guimarães, César. “Eu sou a força de puxar”: o encontro interepistêmico com Maria Luiza Marcelino, mestra quilombola e umbandista. Saúde e Sociedade. 2022, v. 31, n. 2.
Jaksland, Rasmus. Norms of Testimony in Broad Interdisciplinarity: The Case of Quantum Mechanics in Critical Theory. J Gen Philos Sci 52, 35–61 (2021). https://doi.org/10.1007/s10838-020-09523-5
Kelly, A. Paul, and Susan Taylor. Dermatology for skin of color. McGraw-Hill Education/Medical, 2009.
Kelly, A. Paul; Taylor, Susan C.; Lim, Henry C.; Serrano, Ana Maria A. (2015). Taylor and Kelly’s Dermatology for Skin of Color, 2nd edition. New York: McGraw-Hill Medical.
Lopes, Pedro. Negociando Deficiências: identidades e subjetividades entre pessoas com “deficiência intelectual”. Dissertação de Mestrado, Universidade de São Paulo, 2014.
Maldonado-Torres, Nelson. (2016). Transdisciplinaridade e decolonialidade. Sociedade E Estado, 31(1), 75–97. https://doi.org/10.1590/S0102-69922016000100005
Monteiro, Marko Synésio Alves. Os dilemas do humano – Reinventando o corpo numa era (bio)tecnológica. São Paulo: Anablume, 2012.
Schwarcz, Lilia. O espetáculo das raças: cientistas, instituições e questão racial no Brasil, 1870-1930. São Paulo: Companhia das Letras, 1993.
1 Trackback