References
Heaven, D. et al. Why deep-learning AIs are so easy to fool. Nature 574, 163–166 (2019).
Marcus, G. Deep learning: a critical appraisal. Preprint at https://arxiv.org/abs/1801.00631 (2018).
Moon, J., Kim, J., Shin, Y. & Hwang, S. Confidence-aware learning for deep neural networks. In Proc. 37th International Conference on Machine Learning (eds Daumé, H. & Singh, A.) 7034–7044 (PMLR, 2020).
Bulusu, S., Kailkhura, B., Li, B., Varshney, P. K. & Song, D. Anomalous example detection in deep learning: a survey. IEEE Access 8, 132330–132347 (2020).
Musliner, D. J. et al. OpenMIND: planning and adapting in domains with novelty. In Proc. Ninth Annual Conference on Advances in Cognitive Systems (Advances in Cognitive Systems, 2021).
Muhammad, F. et al. A novelty-centric agent architecture for changing worlds. In Proc. 20th International Conference on Autonomous Agents and MultiAgent Systems 925–933 (International Foundation for Autonomous Agents and Multiagent Systems, 2021).
Jafarzadeh, M. et al. Open-world learning without labels. Preprint at https://arxiv.org/abs/2011.12906 (2020).
Jafarzadeh, M. et al. A review of open-world learning and steps toward open-world learning without labels. Preprint at https://arxiv.org/abs/2011.12906 (2020).
Levesque, H. J. Common Sense, the Turing Test and the Quest for Real AI (MIT Press, 2017).
Kejriwal, M., Santos, H., Mulvehill, A. M. & McGuinness, D. L. Designing a strong test for measuring true common-sense reasoning. Nat. Mach. Intell. 4, 318–322 (2022).
Maher, M. L. Evaluating creativity in humans, computers and collectively intelligent systems. In Proc. 1st DESIRE Network Conference on Creativity and Innovation in Design (eds Christensen, B. T. et al.) 22–28 (Desire Network, 2010).
Brachman, R. J. & Levesque, H. J. Machines Like Us: Toward AI with Common Sense (MIT Press, 2022).
Mak, R., Walton, J., Keely, L., Heher, D. & Chan, L. Reliable service–oriented architecture for NASA’s Mars Exploration Rover mission. In Proc. 2005 IEEE Aerospace Conference 1006–1019 (IEEE, 2005).
Silver, D. et al. A general reinforcement learning algorithm that masters chess, shogi and go through self-play. Science 362, 1140–1144 (2018).
Tomašev, N., Paquet, U., Hassabis, D. & Kramnik, V. Reimagining chess with alphazero. Commun. ACM 65, 60–66 (2022).
Cully, A., Clune, J., Tarapore, D. & Mouret, J.-B. Robots that can adapt like animals. Nature 521, 503–507 (2015).
Cincotti, A., Iida, H. & Yoshimura, J. Refinement and complexity in the evolution of chess. In Information Sciences 2007, Proceedings of the 10th Joint Conference (ed. Wang, P. P.) 650–654 (World Scientific, 2007).
Berger, E. R. & Dubbs, A. Winning strategies in multimove chess (i, j). J. Inf. Process. 23, 272–275 (2015).
Naudé, W. Artificial intelligence vs COVID-19: limitations, constraints and pitfalls. AI Soc. 35, 761–765 (2020).
Tu, J. et al. Exploring adversarial robustness of multi-sensor perception systems in self driving. In Proc. 5th Conference on Robot Learning (eds Faust, A. et al.) 1013–1024 (PMLR, 2022).
Terryn, S., Brancart, J., Lefeber, D., Van Assche, G. & Vanderborght, B. Self-healing soft pneumatic robots. Sci. Robot. 2, eaan4268 (2017).
Bilodeau, R. A. & Kramer, R. K. Self-healing and damage resilience for soft robotics: a review. Front. Robot. AI 4, 48 (2017).
Metz, C. OpenAI unveils A.I. that instantly generates eye-popping videos. The New York Times (15 February 2024).
Kejriwal, M. Designing artificial intelligence for open worlds, 2023 AAAS annual meeting. AAAS https://aaas.confex.com/aaas/2023/meetingapp.cgi/Paper/30919 (2023).
Taleb, N. N. Antifragile: Things that Gain from Disorder Vol. 3 (Random House, 2014).
Marsland, S. Novelty detection in learning systems. Neural Comput. Surveys 3, 157–195 (2003).
Chandola, V., Banerjee, A. & Kumar, V. Anomaly detection: a survey. ACM Comput. Surveys 41, 1–58 (2009).
Aminikhanghahi, S. & Cook, D. J. A survey of methods for time series change point detection. Knowl. Inf. Syst. 51, 339–367 (2017).
Missikoff, M., Navigli, R. & Velardi, P. The usable ontology: an environment for building and assessing a domain ontology. In Proc. First International Semantic Web Conference, The Semantic Web - ISWC 2002 (eds Horrocks, I. & Hendler, J. A.) 39–53 (Springer, 2002).
Wang, D., Shelhamer, E., Liu, S., Olshausen, B. A. & Darrell, T. Tent: Fully test-time adaptation by entropy minimization. In Proc. 9th International Conference on Learning Representations (OpenReview.net, 2021).
Mitchell, T. et al. Never-ending learning. Commun. ACM 61, 103–115 (2018).
Bateni, P., Barber, J., van de Meent, J.-W. & Wood, F. Enhancing few-shot image classification with unlabelled examples. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 1597–1606 (IEEE, 2022).
Loyall, B. et al. An integrated architecture for online adaptation to novelty in open worlds using probabilistic programming and novelty-aware planning. In Proc. AAAI Spring Symposium on Designing AI for Open-World Novelty (AAAI, 2022).
Bonjour, T. et al. Decision making in monopoly using a hybrid deep reinforcement learning approach. IEEE Trans. Emerg. Topics Comput. Intell. 6, 1335–1344 (2022).
Di, X. & Shi, R. A survey on autonomous vehicle control in the era of mixed-autonomy: from physics-based to AI-guided driving policy learning. Transport. Res. C Emerg. Technol. 125, 103008 (2021).
Chernova, S. & Veloso, M. Interactive policy learning through confidence-based autonomy. J. Artif. Intell. Res. 34, 1–25 (2009).
Kejriwal, M. in Domain-Specific Knowledge Graph Construction 9–31 (Springer, 2019).
Santos, H., Mulvehill, A. M., Shen, K., Kejriwal, M. & McGuinness, D. L. TG-CSR: A human-labeled dataset grounded in nine formal commonsense categories. Data Brief 51, 109666 (2023).
Gao, R. et al. ObjectFolder 2.0: a multisensory object dataset for Sim2Real transfer. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10588–10598 (IEEE, 2022).
Fuad, A. & Al-Yahya, M. Recent developments in Arabic conversational AI: a literature review. IEEE Access 10, 23842–23859 (2022).
Shrivastava, A., Singh, S. & Gupta, A. Constrained semi-supervised learning using attributes and comparative attributes. In Proc. 12th European Conference on Computer Vision, Computer Vision – ECCV 2012 (eds Fitzgibbon, A. et al.) 369–383 (Springer, 2012).
Tseitlin, A. The antifragile organization. Commun. ACM 56, 40–44 (2013).
Russo, D. & Ciancarini, P. Towards antifragile software architectures. Proc. Comput. Sci. 109, 929–934 (2017).
Abid, A. et al. Toward antifragile cloud computing infrastructures. Proc. Comput. Sci. 32, 850–855 (2014).
Torralba, A. & Efros, A. A. Unbiased look at dataset bias. In Proc. 2011 IEEE Conference on Computer Vision and Pattern Recognition 1521–1528 (IEEE, 2011).
Samala, R. K., Chan, H.-P., Hadjiiski, L. & Koneru, S. Hazards of data leakage in machine learning: a study on classification of breast cancer using deep neural networks. In Medical Imaging 2020: Computer-Aided Diagnosis, Proceedings of SPIE, Volume 11314 (eds Hahn, H. K. & Mazurowski, M. A.) 1131416 (SPIE, 2020).
Gamage, C. et al. Novelty generation framework for AI agents in angry birds style physics games. In Proc. 2021 IEEE Conference on Games (CoG) (IEEE, 2021).
Kejriwal, M. & Thomas, S. A multi-agent simulator for generating novelty in monopoly. Simul. Model. Pract. Theory 112, 102364 (2021).
Höfer, S. et al. Sim2Real in robotics and automation: applications and challenges. IEEE Trans. Autom. Sci. Eng. 18, 398–400 (2021).
Lee, W. & Xiang, D. Information-theoretic measures for anomaly detection. In Proc. 2001 IEEE Symposium on Security and Privacy 130–143 (IEEE, 2000).
Killick, R. & Eckley, I. changepoint: an R package for changepoint analysis. J. Stat. Software 58, 1–19 (2014).
New, A., Baker, M., Nguyen, E. & Vallabha, G. Lifelong learning metrics. Preprint at https://arxiv.org/abs/2201.08278 (2022).
Chen, M. et al. Evaluating large language models trained on code. Preprint at https://arxiv.org/abs/2107.03374 (2021).
Goss, S. A. et al. Polycraft World AI Lab (PAL): an extensible platform for evaluating artificial intelligence agents. Preprint at https://arxiv.org/abs/2301.11891 (2023).
Acsintoae, A. et al. UBnormal: new benchmark for supervised open-set video anomaly detection. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 20111–20121 (IEEE, 2022).
Hamon, R., Junklewitz, H. & Sanchez Martin, J. I. Robustness and Explainability of Artificial Intelligence. Report No. JRC119336, EUR 30040 EN (Publications Office of the European Union, 2020).
Lakkaraju, H., Kamar, E., Caruana, R. & Horvitz, E. Identifying unknown unknowns in the open world: representations and policies for guided exploration. In Proc. Thirty-First AAAI Conference on Artificial Intelligence 2124–2132 (AAAI Press, 2017).
Nayak, A., Timmapathini, H., Ponnalagu, K. & Venkoparao, V. G. Domain adaptation challenges of BERT in tokenization and sub-word representations of out-of-vocabulary words. In Proc. First Workshop on Insights from Negative Results in NLP (eds Rogers, A. et al.) 1–5 (Association for Computational Linguistics, 2020).
Manning, C. & Schutze, H. Foundations of Statistical Natural Language Processing (MIT Press, 1999).
Lin, R. & Kraus, S. Can automated agents proficiently negotiate with humans? Commun. ACM 53, 78–88 (2010).
Meta Fundamental AI Research Diplomacy Team et al.Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science 378, 1067–1074 (2022).
Marcus, G. Horse rides astronaut. Substack garymarcus.substack.com/p/horse-rides-astronaut (2022).
Bang, Y. et al. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. In Proc. 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (eds Park, J. C. et al.) 675–718 (Association for Computational Linguistics, 2023).
Shen, Y. et al. ChatGPT and other large language models are double-edged swords. Radiology 307, 2 (2023).
Liu, X., Ospina, J. & Konstantinou, C. Deep reinforcement learning for cybersecurity assessment of wind integrated power systems. IEEE Access 8, 208378–208394 (2020).
Panesar, A. Machine Learning and AI for Healthcare (Springer, 2019).
Minn, S. AI- assisted knowledge assessment techniques for adaptive learning environments. Comput. Educ. Artif. Intell. 3, 100050 (2022).
Kumar, S. & Tomar, R. The role of artificial intelligence in space exploration. In Proc. 2018 International Conference on Communication, Computing and Internet of Things (IC3IoT) (eds Niranjan, S. K. et al.) 499–503 (IEEE, 2018).
Chantry, M., Christensen, H., Dueben, P. & Palmer, T. Opportunities and challenges for machine learning in weather and climate modelling: hard, medium and soft AI. Phil. Trans. R. Soc. A 379, 20200083 (2021).
Saba, L. et al. The present and future of deep learning in radiology. Eur. J. Radiol. 114, 14–24 (2019).
Ngo, R., Chan, L. & Mindermann, S. The alignment problem from a deep learning perspective. Preprint at https://arxiv.org/abs/2209.00626 (2022).
Wang, J. X. Meta-learning in natural and artificial intelligence. Curr. Opin. Behav. Sci. 38, 90–95 (2021).
Wu, Y. et al. Brain-inspired global-local learning incorporated with neuromorphic computing. Nat. Commun. 13, 65 (2022).
Chen, X., Shrivastava, A. & Gupta, A. NEIL: extracting visual knowledge from web data. In Proc. 2013 IEEE International Conference on Computer Vision (ICCV) 1409–1416 (IEEE, 2013).
Mitchell, M. Abstraction and analogy—making in artificial intelligence. Ann. N. Y. Acad. Sci. 1505, 79–101 (2021).
Chalapathy, R. & Chawla, S. Deep learning for anomaly detection: a survey. Preprint at https://arxiv.org/abs/1901.03407 (2019).
Salehi, M. et al. A unified survey on anomaly, novelty, open-set, and out of-distribution detection: solutions and future challenges. Transactions on Machine Learning Research https://openreview.net/forum?id=aRtjVZvbpK (2022).
Doorenbos, L., Sznitman, R. & Márquez-Neila, P. Data invariants to understand unsupervised out-of-distribution detection. In Proc. 17th European Conference, Part XXXI, Computer Vision – ECCV 2022 (eds Avidan, S. et al.) 133–150 (Springer, 2022).
Erdil, E., Chaitanya, K., Karani, N. & Konukoglu, E. Task-agnostic out-of-distribution detection using kernel density estimation. In Proc. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis: 3rd International Workshop, UNSURE 2021, and 6th International Workshop, PIPPI 2021 (eds Sudre, C. H. et al.) 91–101 (Springer, 2021).
Sastry, C. S. & Oore, S. Detecting out-of-distribution examples with gram matrices. In Proc. 37th International Conference on Machine Learning (eds Daumé, H. III & Singh, A.) 8491–8501 (PMLR, 2020).
Nassif, A. B., Talib, M. A., Nasir, Q. & Dakalbab, F. M. Machine learning for anomaly detection: a systematic review. IEEE Access 9, 78658–78700 (2021).
Zhang, Y. & Yang, Q. An overview of multi-task learning. Natl Sci. Rev. 5, 30–43 (2018).
Caruana, R. Multitask Learning (Springer, 1998).
Van Steenkiste, G., van Loon, G. & Crevecoeur, G. Transfer learning in ECG classification from human to horse using a novel parallel neural network architecture. Sci. Rep. 10, 186 (2020).
Zhang, M.-L. & Zhou, Z.-H. A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26, 1819–1837 (2013).
Thung, K.-H. & Wee, C.-Y. A brief review on multi-task learning. Multimedia Tools Appl. 77, 29705–29725 (2018).
Bi, J., Xiong, T., Yu, S., Dundar, M. & Rao, R. B. An improved multi-task learning approach with applications in medical diagnosis. In Proc. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Part I (eds Daelemans, W. et al.) 117–132 (Springer, 2008).
Wang, Y., Yao, Q., Kwok, J. T. & Ni, L. M. Generalizing from a few examples: a survey on few-shot learning. ACM Comput. Surveys 53, 1–34 (2020).
Pourpanah, F. et al. A review of generalized zero-shot learning methods. IEEE Trans. Pattern Anal. Mach. Intell 45, 4051–4070 (2022).
Boult, T. E. et al. Learning and the unknown: surveying steps toward open world recognition. In Proc. Thirty-Third AAAI Conference on Artificial Intelligence 9801–9807 (AAAI Press, 2019).
Song, Y., Wang, T., Cai, P., Mondal, S. K. & Sahoo, J. P. A comprehensive survey of few-shot learning: evolution, applications, challenges and opportunities. ACM Comput. Surveys 55, 1–40 (2023).
Ade, R. & Deshmukh, P. Methods for incremental learning: a survey. Int. J. Data Mining Knowl. Manag. Process 3, 119–125 (2013).
Zhang, M., Levine, S. & Finn, C. MEMO: test time robustness via adaptation and augmentation. In Proc. 36th International Conference on Neural Information Processing Systems (eds Koyejo, S. et al.) 38629–38642 (NeurIPS, 2022).
Zhang, M. et al. Adaptive risk minimization: learning to adapt to domain shift. Adv. Neural Inf. Process. Syst. 34, 23664–23678 (2021).
