Motivation and Challenges
The current field of artificial intelligence is dominated by centralized, closed-source, and oligopolistic tech giants. A few companies control the highest-performing models, largely due to the extreme centralization of power that drives the development and inference of models.
Creating a machine learning model typically involves three main stages: pre-training, fine-tuning, and inference. These stages are crucial for developing a powerful and accurate model that can generalize well to new, unseen data.
Pre-training Stage
In the pre-training stage, the model is trained on a large, generic dataset. This dataset is unrelated to the final task the model will perform and is intended to help the model learn various features and patterns. For example, in the case of language models, this may involve learning language structures, grammar, and extensive vocabulary from a large corpus of text. The goal here is to develop a model that has a good understanding of the basic structure of the data it will be handling, whether it's text, images, or other forms of data.
There are some centralizing forces in the pre-training stage:
Data Collection and Curation - The key to the pre-training stage is aggregating large datasets from various sources, including literature, digital articles, and professional databases. Industry giants like Google have historically utilized user-generated content to build unparalleled efficient models, a practice that continues today, with entities like Microsoft and OpenAI obtaining top-tier data through exclusive alliances and proprietary platforms. The concentration of these capabilities within a few companies has led to significant centralization in the AI industry.
Additionally, relying on proprietary datasets for model training introduces important ethical considerations and risks perpetuating biases. AI algorithms fundamentally derive operational paradigms from foundational data, thereby generating inherent biases that are easily embedded and replicated. This situation underscores the necessity for thorough scrutiny and ethical oversight during the development process to ensure that models reflect fair and intentional patterns and correlations.
Resource Requirements - It has been established that the effectiveness of a model is logarithmically enhanced with the quantity of training data, indicating that models benefiting from the widest GPU computing cycles typically perform the best. This dynamic introduces a significant centralizing influence in the pre-training stage, driven by the economies of scale and productivity advantages possessed by major tech and data companies. This trend is evident in the dominance of industry giants like OpenAI, Google, Amazon, Microsoft, and Meta, which own and operate the majority of global data centers and have access to NVIDIA's latest, state-of-the-art hardware.
Fine-tuning Stage
After pre-training, the model undergoes fine-tuning. In this stage, the model is further trained on a smaller, specific task dataset. The purpose is to adjust the weights and features the model learned during pre-training to better suit the specific task at hand. This may involve teaching a language model to understand medical terms or training an image recognition model to differentiate between different bird species.
The fine-tuning stage enables the model to focus and improve its performance on tasks of interest to end users. Once again, there are some centralizing forces in the fine-tuning stage, with the most important being closed-source models and verifiability.
During the fine-tuning stage, the model's parameters are refined and set to shape its functionality and performance. The prevailing trend is towards proprietary AI models, such as OpenAI's GPT series and Google's Gemini, meaning that the internal operations and parameters of these models are not disclosed. Therefore, when users request inferences, they cannot verify whether the responses truly come from the model they believe they are interacting with.
This lack of transparency may have adverse effects on users, especially in cases where trust and verifiability are crucial. For example, in the medical field, AI models may assist in diagnosing diseases or recommending treatment methods, but doctors cannot confirm the source and accuracy of the model's inferences, potentially leading to mistrust or misdiagnosis. If healthcare professionals cannot ascertain whether AI recommendations are based on the most reliable and up-to-date models, the consequences could directly impact patient care and outcomes, emphasizing the importance of transparency and accountability in AI deployment.
Inference Stage
The inference stage is the phase where the trained model is actually applied. At this point, the model has been trained and fine-tuned, ready to make predictions on new data. In the case of AI models, this may involve answering questions, translating languages, or providing recommendations. This stage is where the trained model is put to practical use and is typically where the value of the model is realized.
In the inference stage, factors leading to centralization include:
Access: Centralized frontends for accessing AI models may pose risks, potentially preventing users from accessing APIs or inferences. When a few entities control these gateways, they can unilaterally decide to deny access to critical AI services for various reasons, including policy changes or disputes. This centralization highlights the need for decentralized approaches to ensure broader, more resilient access to AI technology, mitigating the risks of censorship and access inequality.
Resource Requirements: The cost of performing inference in artificial intelligence, especially for tasks requiring significant computational resources, has become a centralizing force within the tech industry. High inference costs mean that only large companies with substantial financial resources can afford to scale the operation of advanced AI models. This financial barrier limits the full potential of utilizing cutting-edge AI technology for smaller entities or individual developers.
As a result, this situation is increasingly dominated by a few powerful participants, stifling competition and innovation. This centralization not only affects the diversity of AI development but also restricts access to the benefits of AI, limited to a small portion of well-funded organizations, creating significant imbalances in the technological ecosystem.
In the centralized field of artificial intelligence, some recurring themes have emerged, particularly regarding the evolution of Web2 companies. These entities, initially established as open networks, often shift their focus towards maximizing shareholder value. This transition often leads to the closure of networks and adjustments to algorithms to block external links, which often contradicts the best interests of users.
These situations where companies incentivize actions that are inconsistent with user needs often occur as organizations mature and secure external funding. In fact, we have already seen this phenomenon with OpenAI, which was initially a non-profit organization aimed at democratizing the use of artificial intelligence, demonstrating how shifts in industry focus can manifest. While it is easy to attribute this to individual companies, we believe it reflects systemic issues triggered by the concentration of power within the tech industry, which often leads to inconsistencies between company incentives and broad user needs.
Potential Future: Artificial Intelligence and Web3
The crypto world provides a foundation for artificial intelligence, making the exchange of information and value seamless, open, and secure. Blockchain technology offers a clear and traceable system for managing transactions and recording data. At the intersection of cryptocurrency and artificial intelligence, many opportunities have emerged, where the two domains can mutually reinforce and benefit from each other's capabilities.
Incentive Alignment
Decentralized computing holds significant value in the pre-training and fine-tuning stages of model development. Base models typically require a large amount of GPU computing cycles, often running these processes in centralized data centers. The Decentralized Physical Infrastructure Network (DePIN) can provide decentralized, permissionless computing access. Through economic incentives from cryptocurrencies, software can autonomously compensate hardware usage without the need for central governance entities. This enables users of the network to control the network, adjust incentive mechanisms, and ensure that data and model providers receive adequate compensation.
Verifiability
Current artificial intelligence infrastructure predominantly leans towards proprietary models, requiring users to trust inference providers to execute queries through specified models and generate legitimate outputs. In this context, cryptographic proof systems have become a critical technology, providing a mechanism to verify model outputs on the blockchain. This process allows users to submit queries, with inference providers using agreed-upon models for processing, subsequently generating an output with an encrypted proof. This proof serves as verifiable evidence, demonstrating that the query was indeed processed through the specified model.
The primary goal of these measures is to shift the heavy computational tasks to off-chain environments while ensuring that results can be verified on-chain. This approach not only alleviates the computational burden on the blockchain but also introduces levels of transparency and trustworthiness by providing immutable evidence, proving the accuracy and completeness of off-chain computations.
The inclusion of these cryptographic proofs in the AI model verification process addresses several key issues related to closed-source AI systems. It mitigates the risk of opaque or unverified computations, enhances the integrity of the computing process, and fosters a trust-based relationship between users and inference providers. Additionally, this approach aligns with the broader trend towards decentralization and transparency, echoing the fundamental principles of blockchain technology.
Composability
One of the primary advantages of decentralized finance and blockchain networks is their composability. Composability allows for the widespread use of "money Legos" in DeFi, combining different protocols and outputs to create new applications. While this modular nature introduces new forms of risk into the system, it also simplifies application development for developers, increases innovation and development speed, and can provide a more user-friendly and convenient experience.
Similar to how cryptocurrencies provide composability for financial products, they can also create composability for AI networks and applications. As a permissionless and trustless layer, AI modules can be built and operate independently while remaining interconnected with other modules, forming a network capable of providing various services. Through the network effects of blockchain and cryptocurrencies, decentralized AI projects and applications can interconnect, completing the overall architecture of artificial intelligence.
For example, data preprocessed using Akash or Filecoin can be used to train models with Marlin, Gensyn, or Together. After fine-tuning, these trained models can respond to user queries (inference) through Bittensor. While it may seem more complex, end users only need to interact with a single frontend, while developers and applications can benefit from building on different stacks and applications.
Another important aspect of composability achieved through decentralized networks is data composability. As users become increasingly interested in owning their generated data and demand the ability to carry data between different AI protocols, they will require their data not to be confined within closed environments. Decentralized and open-source AI applications enable portable data.
Data Protection
Decentralized computing, combined with external data and privacy solutions, provides users with more autonomy over their data, making it a more attractive choice than centralized counterparts. In particular, methods like Fully Homomorphic Encryption (FHE) allow computations on encrypted data without the need to decrypt the data.
With FHE, machine learning models can be trained using encrypted datasets, maintaining the privacy and security of the data throughout the entire training process. This provides an end-to-end secure solution with strong encryption guarantees, enabling privacy-preserving model training in edge networks and allowing developers to build AI systems that protect user privacy and leverage advanced AI capabilities.
The role of FHE extends to securely running large language models on encrypted data in cloud environments. This not only protects user privacy and sensitive information but also enhances the ability to run models on applications with inherent privacy. As AI is integrated into various fields, especially sensitive areas like finance, the demand for technologies like Fully Homomorphic Encryption, which can prevent potential information leakage, becomes crucial.
Automatic Upgradability
AI can be used to maintain, update, and automatically upgrade smart contracts based on a series of changes and conditions. For example, AI can be used in protocols to adjust risk parameters based on changes in risk and other market conditions. A common example is the money market, which currently relies on external organizations or DAO decisions to adjust risk parameters for lending assets. AI agents can simplify the updating and upgrading of specific parameters, which would be a significant improvement compared to humans and DAO organizations, which may be slow and inefficient.
Challenges of Decentralized AI
Decentralized AI faces a range of challenges, particularly in balancing the openness of cryptography with the security issues of AI and the computational requirements of AI. In cryptography, openness is crucial for ensuring security, but in the field of AI, open models or their training data increase the risk of adversarial machine learning attacks. Developing applications using these two technologies presents significant challenges. Additionally, the application of AI in blockchain, such as AI-based arbitrage bots, prediction markets, and decision mechanisms, raises issues of fairness and manipulation. While AI has the potential to enhance the efficiency and decision-making capabilities in these areas, there is a risk that AI may not fully grasp the subtle dynamics of human-driven market movements, leading to unforeseen consequences.
Another area of concern is the use of AI as an interface for cryptographic applications. While AI can help users navigate the complex world of cryptocurrencies, it also carries risks, such as susceptibility to adversarial inputs or leading to overreliance on AI for critical decision-making. Additionally, integrating AI into the rules of blockchain applications, such as DAOs or smart contracts, carries risks. Adversarial machine learning could exploit weaknesses in AI models, leading to manipulated or incorrect results. Ensuring that AI models are accurate, user-friendly, and resistant to manipulation is a significant challenge.
Furthermore, combining AI with zero-knowledge proofs or multi-party computation is not only computationally intensive but also faces obstacles such as high computational costs, memory constraints, and model complexity. Tools and infrastructure for zero-knowledge machine learning (zkML) are currently underdeveloped, and there is a lack of skilled developers in this field. These factors result in a significant amount of work needed before zkML can be implemented at a consumer scale.
Conclusion
Balancing decentralization and trust is particularly important in maintaining the decentralized spirit of blockchain and ensuring the reliability of AI systems, especially when using trusted hardware or specific data governance models. In the next part of this article, we will delve into the technologies that support decentralized AI and the key role of the Marlin infrastructure in achieving this goal.
Part 2: Implementation Technology Overview
In the previous part of this article, we discussed the drawbacks of centralized AI and how Web3 mitigates these issues. However, running models on-chain is not feasible due to the extremely high gas fees. Attempting to increase the computational capacity of the underlying blockchain will increase the requirements for validator nodes, potentially reducing decentralization, as small home validators may face difficulties.
In the next section, we will introduce some popular tools and technologies necessary for further developing AI in Web3, namely zero-knowledge proofs (ZKPs), fully homomorphic encryption (FHE), and trusted execution environments (TEEs).
ZKP and ZKML
Zero-knowledge proofs (ZKPs) are particularly important for AI and Web3 as they can improve scalability and privacy protection. They allow for off-chain computation with on-chain verification (verifiable computation), which is much more efficient than re-executing computations on all nodes of the blockchain, reducing network load and supporting more complex operations. zkML can enable AI models to run on-chain. This ensures that the outputs of these off-chain computations are trustworthy and verified.
Additionally, zkML can verify specific aspects of the machine learning process, such as confirming that a specific model was used for predictions or that a model was trained on a specific dataset. zkML can also be used to verify the computation process. For example, it allows computation providers to prove through verifiable evidence that they have used the correct model to process data. This is particularly important for developers relying on decentralized computation providers (such as Akash) and wishing to ensure the accuracy and integrity of computations.
zkML is also useful for users who need to run models on their data while maintaining data privacy. They can execute models on their own data, generate proofs, and subsequently verify the correct usage of the model without compromising the privacy of the data.
FHE
As mentioned earlier, fully homomorphic encryption (FHE) allows direct computation on encrypted data without the need for decryption. This technology has important applications in the field of artificial intelligence, particularly in machine learning and sensitive data processing.
One of the primary applications of FHE is training machine learning models using encrypted datasets. This approach ensures that data remains encrypted and secure throughout the entire training process. As a result, FHE provides a comprehensive secure solution, maintaining the privacy of data from the beginning to the end of the machine learning process. This is particularly important in edge networks where data security and privacy are crucial, and computational resources are typically more limited compared to centralized data centers.
Using fully homomorphic encryption (FHE) technology, a system can be developed that protects user privacy while fully leveraging the advanced capabilities of artificial intelligence. By ensuring that data remains encrypted during storage and processing, FHE provides strong cryptographic guarantees, preventing unauthorized access and data leakage. This is particularly important in scenarios involving sensitive information, such as personal data in medical applications or confidential financial records.
FHE extends its utility to running large language models in cloud environments. By enabling these models to process encrypted data, FHE ensures the protection of user privacy and sensitive information. As more and more AI applications are deployed in cloud environments, data security becomes an important issue, making this capability increasingly crucial. In fields requiring strict confidentiality, such as law, healthcare, and finance, the ability to securely run models on encrypted data enhances the applicability of artificial intelligence.
FHE addresses the critical need to protect sensitive data from potential information leakage and unauthorized access. In fields where data privacy is not just a preference but a regulatory requirement, FHE provides a method of leveraging artificial intelligence without compromising data security and compliance standards.
TEE
Trusted Execution Environments (TEEs) offer significant advantages in training and executing AI inference, particularly in security assurance, isolation, data privacy, and protection. As secure isolation environments, TEEs provide strong security and integrity for data and computation.
The first major benefit is the assurance of improved security. TEEs are specifically designed to counter vulnerabilities in systems with a broad Trusted Computing Base (TCB), including the operating system kernel, device drivers, and libraries. These components are more susceptible to attacks due to their larger attack surface. By providing a secure execution environment, TEEs can protect critical applications, maintaining the integrity and confidentiality of software within the isolated area even if the host operating system is compromised.
Another key advantage is isolation. In the isolated area, code and data are securely stored and accessible only to code within the isolated area. This design prevents external access, including access from other virtual machines or hypervisors, protecting against physical attacks and threats from other virtual machines.
TEE (Trusted Execution Environment) facilitates remote attestation to verify that software is running inside a genuine TEE. This feature is crucial for ensuring the authenticity and integrity of software running within the isolated area. It establishes trust between remote entities and the trusted execution environment, ensuring the security of the software and its execution environment without tampering.
Finally, TEE excels in data protection. The secure features of TEE hardware protect the confidentiality and integrity of computations. This includes secure delivery of code and data (such as keys) to the isolated area. TEE also establishes trusted communication channels for retrieving computation results and outputs, ensuring the security of data throughout the entire lifecycle within the isolated area. These features make TEE an ideal environment for training and executing AI inference, especially in applications requiring high levels of security and data integrity.
Marlin Oyster
Marlin Oyster is an open platform for developers to deploy custom computing tasks or serve trusted execution environments. Similar to Intel's SGX and AWS's Nitro Enclaves. Through Oyster, developers can execute code in an isolated environment and ensure that the host or any other applications within it cannot alter the integrity of computations within the trusted execution environment. In addition to the computational integrity and confidentiality guarantees provided by Trusted Execution Environments (TEEs), the Oyster platform offers additional benefits:
Uptime: Oyster ensures the availability of applications through a monitoring protocol, penalizing downtime and reallocating tasks to nodes that are running normally. This mechanism ensures that developers deploying on Oyster provide continuous application functionality and vitality to end users.
Serverless: Similar to AWS Lambda, Oyster's serverless framework allows developers to deploy applications without specifically renting dedicated nodes. Developers save costs and reduce management overhead by only paying for the runtime of their applications.
Networking: Oyster's isolated area comes pre-equipped with networking capabilities, facilitating the establishment of secure TLS connections within the isolated area. This feature enables the execution of external API queries and manipulation of services with exposed endpoints, enhancing the integration of applications with the internet.
Relays: Oyster supports the offloading of compute-intensive tasks to off-chain environments through relay contracts. These smart contracts enable the execution of functions on Oyster, ensuring reliable results and event-based responses, optimizing the use of on-chain resources.
Benchmarks
In a benchmark comparison between the zkML framework and TEE ML (Oyster), performance metrics indicate that Oyster is more efficient. Specifically, the Oyster framework demonstrates significantly lower total computation time across all tested machine learning models.
For the ordinary least squares model on the Iris dataset, the zkML framework (RisQ) requires over 32 seconds for proof and verification, while Oyster completes the task in just 0.047 seconds. Similarly, on the same dataset, the neural network using zkML (EZKL framework) exceeds 212 seconds for a total time for 500 inputs, while Oyster only requires 0.045 seconds. The significant difference in processing time indicates the higher efficiency of Oyster in these scenarios.
On the MNIST dataset, the LeNet model further reinforces this observation. The zkML framework (EZKL) requires 60 seconds for verification and proof, while Oyster only needs 0.056 seconds. Even though DDKang's zkML framework performs better than EZKL, with a total time of approximately 3.33 seconds, it still falls short of Oyster's 0.056 seconds.
Overall, the data shows that Oyster provides a more efficient solution for machine learning tasks compared to the tested zkML frameworks. Its faster computation time indicates that, for the provided benchmark tests, Oyster can handle the same tasks in significantly less processing time, thus having an advantage in efficiency and speed.
To widely adopt verifiable, decentralized artificial intelligence, off-chain encrypted verification systems must go beyond performing simple tasks such as ordinary least squares computations. The key advancement needed is the ability to handle more complex tasks, specifically running popular Large Language Models (LLMs) efficiently. This requires increased computational power, algorithm efficiency, and scalability to handle the complex and resource-intensive requirements of modern LLMs, enabling more complex and diverse AI applications within decentralized frameworks. The zkML frameworks are still in their early stages, and at this stage, their ability to handle these prompts is severely impacted as generating zk proofs is a computationally intensive task.
While zkML protocols have not yet demonstrated handling prompts for LLMs, it is reasonable to assume that the time difference in processing between the trusted TEE of Oyster and these zkML frameworks is at least as significant as the examples discussed earlier. With Marlin's Oyster, benchmark results for various LLMs can be established:
GPT2-XL Benchmark Results within Oyster:
Isolated Area Configuration: 12 CPUs, 28GB Memory (c6a.4xlarge)
Prompt: Ethereum is a technology operated by the community
Result: "Ethereum is a technology operated by the community, enabling the internet to function properly. Just as Linux has impacted computers, Ethereum will empower the internet with functionality in the future."
Time taken to generate output: 22.091819524765015 seconds
Tokens per second: 1.6295624703815754 seconds
### Part 2: Conclusion
The development and distribution of artificial intelligence technology are increasingly dominated by a small number of large enterprises with advanced hardware and complex models. This level of centralization has raised concerns about regulatory scrutiny, inherent biases, and the verification of the integrity and fairness of AI systems. In contrast, the fundamental principles of cryptocurrency—namely, permissionless and resistance to censorship—provide a pathway to democratizing artificial intelligence technology.
The decentralization and open-source nature of blockchain technology enable decentralized artificial intelligence to compete with centralized counterparts. This is achieved through DePINs, cryptographic proofs, and the use of public-private key pairs, collectively ensuring the secure, transparent, and fair development and use of artificial intelligence. To realize the full potential of decentralized artificial intelligence, especially within the blockchain ecosystem, a robust off-chain computing infrastructure is needed. This is crucial for efficiently, accurately, and verifiably handling complex AI tasks.
Currently, Trusted Execution Environments (TEE) emerge as the most viable solution to meet this requirement. TEEs provide a secure and isolated execution space for code, protecting the confidentiality and integrity of the data being processed. This makes them the best choice for off-chain computing required for AI applications on the blockchain. Advancements in the field, such as zkML, FHE technology, and the enhancement of Trusted Execution Environments (TEE), are crucial for overcoming current limitations in the decentralized AI ecosystem. This progress will promote a more open, accessible, and secure field of artificial intelligence, aligning with the decentralized ideals of the crypto community.
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
