Category Archives: Storage
Frank Frankovsky, who helped oversee Facebook’s custom server effort, has sold his optical startup to Sony. And in the process, given Sony an entry into the growing data center market. Optical Archive Inc., a startup created by a former Facebook infrastructure guru that is using Blu-ray disks in place of hard drives or tape to store files, has been purchased by Sony for an undisclosed sum. The deal, which involves a startup that didn’t raise outside funding and doesn’t have a stated price tag, is significant because it involves Frank Frankovsky, the former head of Facebook’s Open Compute server efforts, who left the social networking company in 2014.
Facebook has done a whole lot of work in order to keep your oldest and least popular photos neatly tucked away and ready to be served up whenever people want to look at them. Today Facebook is opening up about many of the ideas its infrastructure team has implemented in the two cold storage facilities it maintains at its data centers in Oregon and North Carolina. People share 2 billion photos on Facebook’s services every day, so Facebook needs to store them efficiently. And Facebook has made improvements on hardware and software to deal with that scale.
There are certain ways of doing things in hardware engineering, and engineers simply follow these rules because there’s no use fighting them even if they wanted to. Frankly, most don’t even think about it because it’s just a given, but during a recent tour of Facebook’s hardware lab, director of engineering Matt Corddry, says Facebook scale requires them to rethink the old rules and let engineers imagine outside industry standards.
Optical storage has also been used for long-term data retention and environmental stress tests indicate that the latest generation of Blu-ray optical media should have an expected life-time of at least several decades. At the Creative Storage Conference Bill Cubellis from Sony said that properly made archival grade optical discs should have a shelf life of 50 years. At the Open Compute Project Summit in January 2014 Facebook presented a 1 PB Blu-ray disc storage system prototype with 10,000 discs. Facebook estimated that this system would reduce the storage costs by 50% and the energy consumption by 80% compared to their current cold storage system (probably HDD based).
This was a busy week for digital storage meetings with the Facebook sponsored Open Compute Project Summit right across the street from the SNIA Winter Symposium and the Non-Volatile Memory Storage Industry Summit. I want to talk about both events and their implications to the future of digital storage. This piece will cover some observations about digital storage at the Open Compute event.
At last year’s Open Compute Summit, Facebook VP of Engineering Jay Parikh suggested the company might consider Blu-ray discs as a medium for improving durability and cutting costs for its long-term cold storage efforts. Now, Parikh said Tuesday during a keynote at this year’s event, the company is actually doing it. Facebook has built a prototype Blu-ray system capable of storing 10,000 discs and 1 petabyte of data in a single cabinet, with plans to scale it to 5 petabytes per cabinet. Blu-ray storage would save the company 50 percent in costs and 80 percent in energy usage over its existing hard-disk-based cold storage methods, Parikh said, and will provide 50 years worth of durability for data stored on the discs.
Facebook’s graph search feature requires finding and serving the right data fast, and from a database that currently houses more than a trillion posts and 700 terabytes of data overall. In a Thursday morning blog post, engineer Ashoat Tevosyan dove into some of the challenges of building infrastructure that can handle these demands.
Intelligent Storage: Learn about modularized, high-availability and high density storage servers built by Wiwynn with 30 individually hot pluggable SAS/SATA HDDs built with Intel Atom processor C2000 product family
Those who imagine that virtualization is an essential ingredient of cloud computing must have been mystified to discover from my post on Monday thatFacebook prefers not to use the technology. Instead of abstracting away the hardware layer as conventional wisdom suggests one should do in a cloud architecture, Facebook very much factors it in, with its insistence on designing its own custom-built database machine. What’s going on? In fact it’s replication not virtualization that counts in cloud computing — the ability to scale out across an elastic multitude of identical instances.
Who are the world’s biggest server sellers? No one really knows. Venerable research firms like IDC and Gartner will tell you that the server game is dominated by familiar names like IBM, Dell, HP, Cisco, and Fujitsu, but the truth is a bit more complicated. You see, the giants of the internet — the companies that need more servers than anyone — are buying massive amounts of custom-built gear straight from manufacturers in Asia, and they prefer to keep the specifics under wraps. This includes Google and Amazon and others.
TAIPEI – The pre-show conference for Taiwan’s Computex expo took place yesterday, with organisers forecasting Taiwan’s cloud computing market – the main focus of the trade show this year – will be worth NT$15 billion (S$630 million) by 2015. Featuring some 1,700 exhibitors and 5,000 booths, this year’s Computex will showcase the latest developments in cloud computing and information communications technology (ICT), and is expected to attract more than 36,000 international buyers.
Remember when “Other” was just a rounding error in market share reports? Now in the server market, it just might be the main event, as Facebook’s Open Compute project, cloud computing, and other trends drive buyers to no-name server vendors instead of IBM, HP, and Dell. Time to short the incumbents?
Digital storage is crucial to many business models, especially cost sensitive cloud based business models. These applications are often on the cutting edge of technology while at the same time facing enormous challenges in terms of data center energy consumption and operating costs, scaling up storage capacity to meet demand and protecting user data.
Many data centers sit on a lot of “cold storage” — servers containing terabytes of user data that must be retained but is rarely accessed, because users no longer need that data. While the servers are considered cold because they are rarely utilized, their hard drives are usually spinning at full speed although they are not serving data. The drives must keep rotating in case a user request actually requires retrieving data from disk, as spinning up a disk from sleep can take up to 30 seconds. In RAID configurations this time can be even longer if the HDDs in the RAID volume are staggered in their spin up to protect the power supply. Obviously, these latencies would translate into unacceptable wait times for a user who wishes to view a standard resolution photo or a spreadsheet.
Remember that third building that Facebook reportedly is building at its data center in Prineville, Oregon? Turns out that building isn’t a third regular data center, but a deep storage facility, according to Facebook execs on a tour of the Oregon data center. The building, which will potentially be 84,000 square feet, will be filled with disc or flash storage and will act as the “backup to the backup to the backup,” storage for the facility’s data, explained Facebook’s Ken Patchett.