Augmented cognition

Augmented cognition is an interdisciplinary area of psychology and engineering, attracting researchers from the more traditional fields of human-computer interaction, psychology, ergonomics and neuroscience.^[1]^[2] Augmented cognition research generally focuses on tasks and environments where human–computer interaction and interfaces already exist. Developers, leveraging the tools and findings of neuroscience, aim to develop applications which capture the human user's cognitive state in order to drive real-time computer systems.^[3]^[4] In doing so, these systems are able to provide operational data specifically targeted for the user in a given context.^[3] Three major areas of research in the field are: Cognitive State Assessment (CSA), Mitigation Strategies (MS), and Robust Controllers (RC).^[4] A subfield of the science, Augmented Social Cognition, endeavours to enhance the "ability of a group of people to remember, think, and reason."^[5]

History

In 1962 Douglas C. Engelbart released the report "Augmenting Human Intellect: A Conceptual Framework"^[6] which introduced, and laid the groundwork for, augmented cognition. In this paper, Engelbart defines "augmenting human intellect"^[6] as "increasing the capability of a man to approach a complex problem situation, to gain comprehension to suit his particular needs, and to derive solutions to problems."^[6]

Modern augmented cognition began to emerge in the early 2000s. Advances in cognitive, behavioral, and neurological sciences during the 1990s set the stage for the emerging field of augmented cognition – this period has been termed the "Decade of the Brain."^[3] Major advancements in functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) have been pivotal in the emergence of augmented cognition technologies which seek to monitor the user's cognitive abilities. As these tools were primarily used in controlled environments, their further development was essential to pragmatic augmented cognition applications.^[3]

Research

DARPA's Augmented Cognition Program

The Defense Advanced Research Projects Agency (DARPA) has been one of the primary funding agencies for augmented cognition investigators. A major focus of DARPA's augmented cognition program (AugCog) has been developing more robust tools for monitoring cognitive state and integrating them with computer systems. The program envisions "order of magnitude increases in available, net thinking power resulting from linked human-machine dyads [that] will provide such clear informational superiority that few rational individuals or organizations would challenge under the consequences of mortality."^[7]^[8]

The program began in 2001, and has since be renamed to Improving Warfighter Information Intake Under Stress Program.^[9] By leveraging such tools, the program seeks to provide warfighters with enhanced cognitive abilities, especially under complex or stressful war conditions.^[3] As of 2002, the program vision is divided into four phases:^[3]

Phase 1: Real-time cognitive state detection
Phase 2: Real-time cognitive state manipulation
Phase 3: Autonomous cognitive state manipulation
Phase 4: Operation demonstration and transition

Proof of concept was carried out in two phases: near real time monitoring of the user's cognitive activity, and subsequent manipulation of the user's cognitive state.^[8]

Augmented Cognition International (ACI) Society

The Augmented Cognition International (ACI) Society held its first conference in July 2005.^[4] At the society's first conference, attendees from a diverse background including academia, government, and industry came together to create an agenda for future research. The agenda focused on near-, medium-, and long-term research and development goals in key augmented cognition science and technology areas.^[4] The International Conference on Human Computer Interaction, where the society first established itself, continues to host the society's activities.^[10]

Translation engines

Thad Starner, and the American Sign Language (ASL) Research Group at Georgia Tech, have been researching systems for the recognition of ASL. Telesign,^[11] a one-way translation system from ASL to English, was shown to have a 94% accuracy rate on a vocabulary with 141 signs.^[12]

Augmentation Factor

Ron Fulbright proposed the augmentation factor (A⁺), as a measure of the degree a human is cognitively enhanced by working in collaborative partnership with an artificial cognitive system (cog).^[13]^[14]^[15] If W_H is the cognitive work performed by the human in a human-machine dyad, and W_C is the cognitive work done by the cog then A⁺ = W_C/W_H. In situations where a human is working alone without assistance, then W_C = 0 resulting in A⁺ = 0 meaning the human is not cognitively augmented at all. In situations where the human does more cognitive work than the cog, A⁺ < 1. In situations where the cog does more cognitive work than the human, A⁺ > 1. As cognitive systems continue to advance, A⁺ will increase. In situations where a cog performs all cognitive work without the assistance of a human, then W_H = 0 resulting in A⁺ = <undefined> meaning attempting to calculate the augmentation factor is nonsensical since there is no human involved to be augmented.

Human/Cog Ensembles

Whereas DARPA's AugCog program focuses on human/machine dyads, it is possible for there to be more than one human and more than one artificial element involved. Human/Cog Ensembles involve one or more humans working with one or more cognitive systems (cogs). In a human/cog ensemble, the total amount of cognitive work performed by the ensemble, W*, is the sum of the cognitive work performed by each of the N humans in the ensemble plus the sum of the cognitive work performed by each of the M cognitive systems in the ensemble:^[13]^[14]^[16]

W* = $\sum _{k=1}^{N}$ W^k_H + $\sum _{k=1}^{M}$ W^k_C

Controversy

Privacy concerns

The increasing sophistication of brain-reading technologies has led many to investigate their potential applications for lie detection. Legally required brain scans arguably violate “the guarantee against self-incrimination” because they differ from acceptable forms of bodily evidence, such as fingerprints or blood samples, in an important way: they are not simply physical, hard evidence, but evidence that is intimately linked to the defendant's mind. Under US law, brain-scanning technologies might also raise implications for the Fourth Amendment, calling into question whether they constitute an unreasonable search and seizure.^[17]

Human augmentation

Many of the same arguments in the debate around human enhancement can be analogized to augmented cognition. Economic inequality, for instance, may serve to exacerbate societal advantages and disadvantages due to the limited availability of such technologies.

Fearing the potential applications of devices like Google Glass, certain gambling establishments (such as Caesar's Palace in Las Vegas) banned its use even before it was commercially available.^[18]

References

^ Miller, Christopher A.; Dorneich, Michael C. (2006). "From Associate Systems to Augmented Cognition: 25 Years of User Adaptation in High Criticality Systems". Foundations of Augmented Cognition. 2: 344–353.
^ Stanney, Kay M.; Schmorrow, Dylan D.; Johnston, Mathew; Fuchs, Sven; Jones, David; Hale, Kelly S.; Ahmad, Ali; Young, Peter (2009). "Augmented cognition: An overview". Reviews of Human Factors and Ergonomics. 5 (1): 195–224. doi:10.1518/155723409X448062.
^ ^a ^b ^c ^d ^e ^f Schmorrow, D.D.; Kruse, A.A. (2002). "DARPA's Augmented Cognition Program-tomorrow's human computer interaction from vision to reality: Building cognitively aware computational systems". Proceedings of the IEEE 7th Conference on Human Factors and Power Plants. doi:10.1109/HFPP.2002.1042859. ISBN 0-7803-7450-9. S2CID 108991554.
^ ^a ^b ^c ^d Reeves, Leah M.; Schmorrow, Dylan D.; Stanney, Kay M. (2007). "Augmented Cognition and Cognitive State Assessment Technology – Near-Term, Mid-Term, and Long-Term Research Objectives". In Schmorrow, Dylan D. (ed.). Foundations of Augmented Cognition. Lecture Notes in Computer Science. Vol. 4565. Springer Berlin Heidelberg. pp. 220–228. doi:10.1007/978-3-540-73216-7_25. ISBN 978-3-540-73215-0. S2CID 33734841.
^ Chi, Ed H. (2009). "Augmented social cognition". Proceedings of the 2009 ACM SIGMOD International Conference on Management of data. pp. 973–984. doi:10.1145/1559845.1559959. ISBN 9781605585512. S2CID 12900355.
^ ^a ^b ^c Engelbart, Douglas C (October 1962). "Augmenting Human Intellect: A Conceptual Framework". SRI Summary Report AFOSR-3223 • Prepared for: Director of Information Sciences, Air Force Office of Scientific Research, Washington DC, Contract AF 49(638)-1024 • SRI Project No. 3578 (AUGMENT,3906,). The Doug Engelbart Institute. Archived from the original on 2011-05-04. Retrieved 2015-04-20.
^ Schmorrow, Dylan; McBride, Dennis (June 1, 2004). "Introduction". International Journal of Human-Computer Interaction. 17 (2): 127–130. doi:10.1207/s15327590ijhc1702_1. ISSN 1044-7318. S2CID 218498878.
^ ^a ^b Cummings, M. L. (2010). "VIEWS". Ergonomics in Design. 2: 5.
^ National Research Council (US) Committee on Opportunities in Neuroscience for Future Army Applications. Opportunities in Neuroscience for Future Army Applications. Washington (DC): National Academies Press (US); 2009. Appendix D, Research on Managing Information Overload in Soldiers Under Stress. Available from:https://www.ncbi.nlm.nih.gov/books/NBK207983/
^ "Virtual, Augmented and Mixed Reality". HCI International. 2014. Retrieved 19 April 2015.
^ "Telesign". Telesign. Georgia Tech Contextual Computing Group. 2015-04-19. Retrieved 2015-04-19.
^ McGuire, R.M.; Hernandez-Rebollar, J.; Starner, T.; Henderson, V.; Brashear, H.; Ross, D.S. (May 2004). "Towards a one-way American Sign Language translator". Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings. pp. 620–625. CiteSeerX 10.1.1.129.3571. doi:10.1109/AFGR.2004.1301602. hdl:1853/28993. ISBN 978-0-7695-2122-0. S2CID 7143693.
^ ^a ^b Fulbright, Ron (2020). Democratization of Expertise: How Cognitive Systems Will Revolutionize Your Life. Boca Raton, FL: CRC Press. ISBN 978-0367859459.
^ ^a ^b Fulbright, Ron (2017). "Cognitive Augmentation Metrics Using Representational Information Theory". Augmented Cognition. Enhancing Cognition and Behavior in Complex Human Environments. Lecture Notes in Computer Science. Vol. 10285. pp. 36–55. doi:10.1007/978-3-319-58625-0_3. ISBN 978-3-319-58624-3 – via Springer.
^ Fulbright, Ron (2018). "On Measuring Cognition and Cognitive Augmentation". Human Interface and the Management of Information. Information in Applications and Services. HIMI 2018. Lecture Notes in Computer Science. Vol. 10905. pp. 494–507. arXiv:2211.06477. doi:10.1007/978-3-319-92046-7_41. ISBN 978-3-319-92045-0. S2CID 51603737 – via Springer.
^ Fulbright, Ron; Walters, Grover (2020). "Synthetic Expertise". Augmented Cognition. Human Cognition and Behavior. Lecture Notes in Computer Science. Vol. 12197. pp. 27–48. arXiv:2212.03244. doi:10.1007/978-3-030-50439-7_3. ISBN 978-3-030-50438-0. S2CID 220519330 – via Springer.
^ Tennison, Michael; Moreno, Jonathon (March 2012). "Neuroscience, Ethics, and National Security: The State of the Art". PLOS Biology. 10 (3): e1001289. doi:10.1371/journal.pbio.1001289. PMC 3308927. PMID 22448146.
^ Clark, Matt (May 8, 2013). "Google Glass Violates Nevada Law, Says Caesars Palace". IGN.