The Unprecedented Results of Allowing a Neural Net to Self-Select Its Neurons

In the past, ownership of purebred dogs was considered a status symbol. People would proudly display their German Shepherds or Cocker Spaniels, often boasting about their impeccable pedigrees. This obsession with purity was exacerbated during the time of Hitler’s Nazi Germany, when “contaminated” lineages for German Shepherds were frowned upon. However, it was during the Victorian era of the 1800s that people engaged in aggressive inbreeding of dogs, resulting in the breeds we see today, including their health issues such as hip and joint problems.

Today, the focus has shifted from purity to genetic diversity, as we learn more about how inbreeding can lead to debilitating health conditions and unstable temperaments. In an unstable and rapidly changing environment due to climate change, diverse gene pools are highly sought after. This natural law has prompted the question: what if machines, like artificial intelligence (AI), could also thrive in a more diverse engineering setting?

A groundbreaking experiment conducted by a team of researchers from institutions including North Carolina State University and the Indian Institute of Science sought to explore whether advanced AIs, specifically neural nets, would choose diversity and whether this choice would enhance AI performance. The study was aimed at understanding the operational choices of AIs when left to their own devices.

The experiment, led by Professor William Ditto and his team, introduced a non-human intelligence, an artificial intelligence, to see if it would choose diversity over the lack of diversity and how this choice would impact its performance. The AI was given the ability to modify the composition of its neural network, essentially offering control over its own brain, a concept known as meta-learning for AI.

The results were astonishing. The AI exhibited a preference for diverse neural arrangements, and this learned diversity led to an unprecedented increase in processor speed and enhanced accuracy. The researchers found that the self-selecting, diverse AI outperformed the homogenous one by a significant margin in a standard numerical classifying exercise. This demonstrated that AI, when given the ability to learn how it learns, can improve its internal structure, embrace diversity, and enhance its learning and problem-solving abilities.

The implications of these findings are significant, particularly as AI becomes increasingly integrated into various applications, including critical functions such as aircraft and autonomous vehicles. The team believes that learned diversity can significantly improve the performance of physics-informed AIs, making them more robust and efficient.

In essence, the study highlights the importance of diversity in AI development, mirroring the evolutionary trajectory observed in the natural world. Just as it has contributed to the success of humans, animals, and plants, diversity seems to separate winners from losers in the machine world, marking a crucial factor in the future of AI.