AI storage is booming, can Filecoin come out and pick up garbage? What are thermal guidance and cold storage?? Preface: Filecoin has not sought cooperation for many years, and Juan has not appeared in public. The reason for writing the theme of Filecoin is entirely because I have a neighbor named Kan Ge @ tktang88, who is a huge Filecoin investor, and many Filecoin miners who share their knowledge and future expectations about Filecoin. Especially, I am very interested in one point that Kan Ge mentioned this time. So this tweet is not a commercial advertisement, nor does it encourage people to buy FIL, but rather a new perspective on decentralized storage. Main text The day before yesterday, Micron's financial report expectations cast a shadow over the entire market, and yesterday, Micron's good and expected financial report drove the market to rise sharply in the short term, even surpassing Meta and Tesla in market value. The reason is that the storage demand in the AI era may exceed the imagination of many people. Because AI training and inference require high-speed read and write, vector databases, KV cache offloading, model parameters, and inference intermediate states all require stronger memory and storage capabilities. This logic is hardware level, with stronger certainty and more direct revenue. But AI storage requirements will not only stay at high-speed memory and SSD. With more and more model training, inference, agent and user generated content, there will inevitably be another type of more troublesome data in the future, which is a large amount of data that is of no value in the short term, has extremely low access frequency, may never be used again in the future, but enterprises dare not easily delete it. This is the focus of today's discussion, the storage of junk data! Data in the AI era will naturally be layered. At the forefront is hot data, which is being used for training and inference and requires high-speed access. This part is dominated by HBM, DRAM, NVMe SSD, and high-speed networks. The middle is warm data, which may be reused in the near future, such as model checkpoints, training shards, vector indexes, experimental logs, evaluation data, and datasets that are still being iterated. Finally, there is cold data, which refers to data that has already been trained and will not be called in the short term, but may be needed again in the future due to retraining, rollback, copyright, regulation, auditing, security incidents, and model replication. Especially, the demands dominated by Cold Data and Micron are not in the same position. Micron leads the high-speed storage, training, and inference of the data being used. This part of the data has the highest gold content, value, and cost, so the hardware used for storage is in short supply. But for cold data, the so-called cold data refers to data that is used very infrequently, such as raw data used for model training, cleaned data, deduplication records, annotation records, and user generated images and videos that are almost considered garbage. Most of these things are not usually opened again, and may not even be read once every few years, but deleting them directly is not feasible. Because in the future, it may be necessary to retrain, roll back the model, interpret a certain output, handle copyright disputes, face regulatory audits, or simply because after the new model appears, previously useless data suddenly becomes useful again. So the most troublesome part of the AI era is that there will be more and more data, and the risk of deleting data will also increase. In the early stages of many AI businesses, data management was relatively extensive, with hot data, warm data, and cold data not being separated. Especially for a large amount of low-frequency access data, if it continues to occupy high cost storage, it will definitely not be cost-effective in the long run. Greatly increasing storage costs, or using high-speed cloud storage is not cost-effective. Can we throw all these cold data into a hard drive "cold storage"? The answer is negative. If these AI data are just thrown into a cold storage without indexing, labeling, sources, model version mapping, or cleaning process records, then even if this batch of data is still there, it is almost as if it has been lost. What is needed is to keep metadata hot and data ontology cold. The data ontology can be stored in cold storage, but the directory, source, and hash of the data CID、 License, creation time, cleaning method, corresponding model, usage record, privacy label, retention period, and recovery test results need to be placed in a searchable, readable, and auditable hot index layer. This is why Filecoin and decentralized storage can be reconsidered. Especially for decentralized storage infrastructure that already has network storage capabilities. Filecoin has a large amount of network storage capacity, and although having multiple hard drives does not make much sense in itself, these hard drives on the blockchain already have the prototype of verifiable cold storage. Especially, the unique features of Filecoin compared to traditional cloud storage are content addressing, multi provider storage, and on chain proof. Speaking in person means that customers don't have to rely solely on one cloud provider saying 'the data has been saved', but can continuously verify that the data is still there, the content has not been changed, and can be retrieved through the same content identifier in the future. This ability is meaningful for AI cold data. From this perspective, the real opportunity for decentralized storage may be the AI cold data management layer. Responsible for migrating data from training clusters, cloud object storage, and enterprise local servers, first performing deduplication, compression, privacy scanning, copyright tagging, encryption, and sharding, and then throwing large files into cold storage while retaining hot indexes. In the future, the model will need to be retrained, and the system can retrieve data by source, time, label, and model version. Without this ability, Filecoin is just a warehouse. With this ability, decentralized storage can become a part of AI data infrastructure. Different decentralized storage projects should also be viewed separately. Filecoin is more suitable for discussing verifiable cold data warehouses, as its core is a storage market and data proof, suitable for large files, low-frequency access, fixed version dataset snapshots, model checkpoints, scientific research data, public training corpora, and privacy processed audit logs. Arweave is more suitable for permanently disclosing data, model specifications, data source records, and tamper proof public archives, but data involving privacy and deletion rights is difficult to directly put in because permanent storage itself brings compliance issues. Storj and Sia are closer to decentralized object storage, and if the user experience and price are good enough, they can compete for a portion of backup and archiving needs, but also prove availability, recovery speed, enterprise services, and long-term economic models. Of course, the most important thing is that it is cheap enough. AWS Glacier Deep Archive、Google Archive、Azure Archive、 Enterprise tape libraries, local object storage, hard drive manufacturers, and cloud vendors will all compete for AI cold data. Especially for data with extremely low access frequency, tape and deep archiving are still very competitive. If decentralized storage wants to win, the first priority is affordability, but in addition to affordability, it still needs to meet the capabilities of verifiability, multi provider, vendor neutrality, and content addressing. Cheap is just a knock on the door. With the continued development of AI, there will be more and more cold or junk data, and this part of data is likely to become one of the most headache inducing costs for AI companies in the future. That's also why I think decentralized storage, which already exists, can be discussed again for its affordability. The biggest problem with projects like Filecoin in the past was that the supply (mining machines) was available, but the demand was not completely non-existent. There are a large number of hard drives, storage providers, and decentralized narratives on the internet, but real customers and real payments are a mess. If AI cold data really becomes a big market now, and decentralized storage can achieve "hot indexing cold storage" and be cheaper than traditional storage, then these existing hard drives have the opportunity to be applied in real life. Of course, from the current investment perspective, it cannot be assumed that Filecoin should also rise just because Micron has risen. The business logic of the two is completely different. Micron sells hardware, while Filecoin depends on paid storage capacity, actual customer numbers, renewal rates, retrieval success rates, recovery costs, storage provider profits, and whether these business growth can ultimately be transmitted to FIL's demand, collateral, fees, or destruction. There is still a long way to go for decentralized storage, especially whether the system of "hot indexing and cold storage" can be implemented. This is where Filecoins should truly learn from. The demand for AI cold data is likely to emerge, but where this demand will ultimately flow depends on who can achieve enough affordability, stability, ease of retrieval, and auditability. If Filecoin can only prove that it has many hard drives, then it doesn't make much sense. If Filecoin can prove that these hard drives can handle real paid data and can be stably retrieved, fully recovered, and continuously renewed after a few years, then these seemingly unwanted junk data in the AI era may indeed bring a second chance to decentralized storage. end