Published jointly by Fujitsu and Twist Bioscience, both of which operate in the archival storage market, the report warns of an abysis opening up between available storage capacity and demand which will exceed 7.8 million petabytes by 2030 and only get worse.
This means that businesses will have to delete large swathes of old data to make room for the new which sounds obvious but if you are trying to build AI type machines is going to be your worst nightmare.
While hard disk drives (HDDs) and solid state drives (SSDs) do an excellent job of holding and supplying the quantities of data that servers and client devices need to function, neither are well-suited to storing information en masse and for long durations. In fact, the best thing for archival storage is still Linear Tape-Open (LTO) magnetic tape. The current generation of tape, LTO-9, has a native capacity of 18TB and can be purchased for as little as $150 (or roughly $8.30/TB).
The report said that large companies will have to invest heavily in tape and other archival media, as the volume of data produced by business operations continues to rise.
Report author Jon Monroe,said that while cost-effective, tape has its weaknesses too; data can only be accessed serially, making it hard to locate specific files, and companies also need to migrate to fresh tape on a semi-regular basis to avoid data loss.
The result is that there will have to be a new system of storeage developed and the most promising is DNA.
DNA comprises four molecular building blocks: adenine (A), guanine (G), cytosine (C), and thymine (T). These compounds connect in pairs (A-T & G-C) to form the rungs of the famous double helix ladder.
This structure can be utilised as an extremely dense and durable form of data storage, by converting binary 1s and 0s into the four-lettered genetic alphabet. A single gram of DNA has been found to be capable of storing 215 PB (220,000 TB) of data.
The technology is unusable at scale at the moment because it takes ages to data to DNA.
“The datacenters of the future will need everything the SSD, HDD, and tape industries can manufacture and deliver, as well as requiring new DNA and optical and perhaps other enterprise storage technologies, to cost-effectively and reliably preserve the priceless artefacts of our personal, corporate, and cultural history,” added Monroe.