AI Transparency and the Need for Open-Source Models

In order to protect people from the potential harms of AI, some regulators in the United States and European Union are increasingly advocating for controls and checks and balances on the power of open-source AI models. This is partially motivated by the desire of major corporations to control AI development and to shape the development of AI in a way that benefits them. Regulators are also concerned about the pace of AI development, as they worry that AI is developing too quickly and that there is not enough time to put in place safeguards to prevent it from being used for malicious purposes.

The AI Bill of Rights and the NIST AI Risk Management Framework in the U.S., along with the EU AI Act, support various principles such as accuracy, safety, non-discrimination, security, transparency, accountability, explainability, interpretability, and data privacy. Moreover, both the EU and the U.S. anticipate that standards organizations, whether governmental or international entities, will play a crucial role in establishing guidelines for AI.

In light of this situation, it is imperative to strive for a future that embraces transparency and the ability to inspect and monitor AI systems. This would enable developers worldwide to thoroughly examine, analyze, and improve AI, particularly focusing on training data and processes.

To successfully bring transparency to AI, we must understand the decision-making algorithms that underpin it, thereby unraveling AI’s “black box” approach. Open-source and inspectable models play an integral part in achieving this goal, as they provide access to the underlying code, system architecture, and training data for scrutiny and audit. This openness fosters collaboration, drives innovation, and safeguards against monopolization.

To witness the realization of this vision, it is essential to facilitate policy changes, grassroots initiatives, and encourage active participation from all stakeholders, including developers, corporations, governments, and the public.

Current State of AI: Concentration and Control

Presently, AI development, especially concerning large language models (LLMs), is primarily centralized and controlled by major corporations. This concentration of power raises concerns regarding the potential for misuse and prompts questions about equitable access and the fair distribution of benefits from advancements in AI.

In particular, popular models like LLMs lack open-source alternatives during the training process due to the extensive computing resources required, which are typically available only to large companies. Nevertheless, even if this situation remains unchanged, ensuring transparency regarding the training data and processes is crucial to facilitate scrutiny and accountability.

OpenAI’s recent introduction of a licensing system for certain AI types has generated apprehension and concerns about regulatory capture, as it could influence not only the trajectory of AI, but also broader social, economic, and political aspects.

The Need for Transparent AI

Imagine relying on a technology that makes impactful decisions on human/personal life, yet leaves no breadcrumb trail, no understanding of the rationale behind those conclusions. This is where transparency becomes indispensable.

First and foremost, transparency is crucial and builds trust. When AI models become observable, they instill confidence in  their reliability and accuracy. Moreover, such transparency would leave developers and organizations far more accountable for the outcomes of their algorithms.

Another critical aspect of transparency is the identification and mitigation of algorithmic bias. Bias can be injected into AI models in several ways.

• Human element: Data scientists are vulnerable to perpetuating their own biases into models.
• Machine learning: Even if scientists were to create purely objective AI, models are still highly susceptible to bias. Machine learning starts with a defined dataset, but is then set free to absorb new data and create new learning paths and new conclusions. These outcomes may be unintended, biased, or inaccurate, as the model attempts to evolve on its own in what’s called “data drift.”

It is important to be aware of these potential sources of bias so that they can be identified and mitigated. One way to identify bias is to audit the data used to train the model. This includes looking for patterns that may indicate discrimination or unfairness. Another way to mitigate bias is to use debiasing techniques. These techniques can help to remove or reduce bias from the model. By being transparent about the potential for bias and taking steps to mitigate it, we can help to ensure that AI is used in a fair and responsible way.

Transparent AI models enable researchers and users to examine the training data, identify biases, and take corrective action towards addressing them. By making the decision-making process visible, transparency helps us strive for fairness and prevent the propagation of discriminatory practices. Moreover, transparency is needed throughout the life of the model as explained above to prevent data drift, bias and AI hallucinations that produce false information. These hallucinations are particularly prevalent in Large Language Models, but also exist in all forms of AI products. AI observability also plays important roles in ensuring performance and accuracy of the models creating safer, more reliable AI that is less prone to errors or unintended consequences.

However, achieving transparency in AI is not without its challenges. Striking a careful balance is necessary to address concerns such as data privacy, security, and intellectual property. This entails implementing privacy-preserving techniques, anonymizing sensitive data, and establishing industry standards and regulations that promote responsible transparency practices.

Making Transparent AI a Reality

Developing tools and technologies that can enable inspectability in AI is crucial for promoting transparency and accountability in AI models.

In addition to developing tools and technologies that enable inspectability in AI, tech development can also promote transparency by creating a culture of it around AI. Encouraging businesses and organizations to be transparent about their use of AI can also help to build trust and confidence. By making it easier to inspect AI models and by creating a culture of transparency around AI, tech development can help to ensure that AI is used in a fair and responsible way.

However, tech development can also have the opposite effect. For example, if tech companies develop proprietary algorithms that are not open to public scrutiny, this can make it more difficult to understand how these algorithms work and to identify any potential biases or risks. Ensuring that AI benefits society as a whole rather than a select few requires a high level of collaboration.

Researchers, policymakers, and data scientists can establish regulations and standards that strike the right balance between openness, privacy, and security without stifling innovation. These regulations can create frameworks that encourage the sharing of knowledge while addressing potential risks and defining expectations for transparency and explainability in critical systems.

All parties related to AI development and deployment should prioritize transparency by documenting their decision-making processes, making source code available, and embracing transparency as a core principle in AI system development. This allows everyone the opportunity to play a vital role in exploring methods to make AI algorithms more interpretable and developing techniques that facilitate understanding and explanation of complex models.

Finally, public engagement is crucial in this process. By raising awareness and fostering public discussions around AI transparency, we can ensure that societal values are reflected in the development and deployment of AI systems.

Conclusion

As AI becomes increasingly integrated into various aspects of our lives, AI transparency and the use of open-source models become critical considerations. Embracing inspectable AI not only ensures fairness and accountability but also stimulates innovation, prevents the concentration of power, and promotes equitable access to AI advancements.

By prioritizing transparency, enabling scrutiny of AI models, and fostering collaboration, we can collectively shape an AI future that benefits everyone while addressing the ethical, social, and technical challenges associated with this transformative technology.