Compute Cycles

Announcing ZeroCompute™: Eco-conscious, Instantaneous Utility Supercomputing

Today’s researchers are shackled by a lack of access to compute power. Novel advances in modern science and engineering demand higher performing systems to solve increasingly difficult problems.  

Cycle is pleased to announce our new intelligent orchestration software to meet these challenges, called "ZeroCompute™".

ZeroCompute solves access to remote high performance cloud computing environments with a patent pending technology designed to accelerate access to HPC and BigData systems. This approach can simultaneously manage science that might normally require brontobyte datasets, and five hundred billion concurrent jobs.

After years building Utility HPC & Utility Supercomputing software, Cycle's engineers have discovered that the fastest way to execute any algorithm is to just not run it at all.

Although seemingly obvious, by simply not running the science on any cores, we remove the computational challenges and data transfer bottlenecks of today’s BigCompute and BigData workloads. Because ZeroCompute "completes" the floating point in the nanosecond it takes the software to decide not to run anything, accordingly, the system exceeds a peak floating-point performance of over one billion petaflops, at a cost of $0.00 per flop.

By combining ZeroCompute with a nice hammock and an adult beverage, all researchers can now compute with ease. "This is earth shattering technology! For years we have been extremely worried about how can we possibly compute against such large datasets,” says Dr. James Cuff, Cycle’s Chief Technology Officer. “Now with ZeroCompute, we just don't have to worry. It’s a relief that we can kick back, and allow ZeroCompute to not run the workload for us. It’s so easy anyone can do this, simply by doing nothing!"

We have already seen wide adoption of ZeroCompute methodologies in the market, which will yield a huge slowdown in drug discovery pipelines, massive losses for some financial institutions due to lack of modeling, and engineering and manufacturing companies simply ceasing to create any new, competitive products. Thankfully, ZeroCompute will make this all very easy.

As a further benefit, ZeroCompute is exceptionally ecologically conscious. Using advanced mathematics, and not running any science, has resulted in the highest possible flops/W of any modern computing architecture.

Leaving unused compute capacity to go to waste may power a new economic collapse, increased use of streaming video, and a commodities market rise in stone wheels, caveman suits and wooden clubs; we advise going long on all cave options.  

Jason Stowe, CEO of Cycle Computing said in a statement, "We should have done this years ago! It has been a huge challenge to actually enable utility supercomputing at scale. We used to spend tremendous effort creating 50,000-core supercomputers at the push of a button; now you don't even have to push the button."

Unlike competing technology that also doesn’t make actual cloud HPC easy, ZeroCompute is fully available today, and for immediate download from file:///dev/null

Fortune 500s discuss Cloud HPC, Utility Supercomputing @ Cycle's re:Invent session

As many of you know, at Cycle we think that giving every researcher, scientist, engineer, and mathematician access to the compute power they need, exactly when they need it, will enable humanity to achieve what its truly capable of. 

So we organized five Fortune 500 Cycle customers of ours to talk at AWS re:Invent at 1pm Wednesday the 28th, about Cloud, HPC, and utility supercomputing.

Whether its building safer medical devices, managing risk, helping quantify genomes at scale, protect hard-earned retirement funds, or find medicines to cure disease, they'll be talking about how they use Cloud to do it!

At 1pm tomorrow (Wednesday) come to "Enterprise HPC in the Cloud: Fortune 500 use cases" in room 3401A to see:

• HartfordLIfe
• Johnson & Johnson
• Life Technologies 
• Novartis 
• PacificLife

If you can't make the session come to Cycle's Booth #220, and we can talk more there!

Fast and Cheap, pick two: Real data for Multi-threaded S3 Transfers

Gentleman start your uploads! They're free now but how fast can we do them? Lately we’ve been working with clients solving big scientific problems with Big Data (Next Generation Sequencing analysis is one example) so we’ve been working hard to transfer large files into and out of the cloud as efficiently as possible. We’re optimizing two costs here: money and time.

Lucky for us, Amazon Web Services continues to drive down the costs of data transfer. We were excited to see that all data transfer into AWS will be free as of July 1st! They’re also reducing the cost to transfer data out of AWS. Less money, more science, yes!

We still need to optimize for time, however. The scalability of the Elastic Compute Cloud (EC2) means we can throw as many cores at a scientific problem as we can afford in a very short time. But what if our input or result data is so large that the time to transfer it far outweighs the time to analyze it?

Our previous work has shown that file transfers often do not fill the pipe to capacity, and are often limited by disk I/O and other factors. Therefore, we can speed transfers by using multiple threads to fill the pipe.

 

As shown above, this work involved moving data directly to a file system using rsync. But since that time, we’ve begun to rely upon the Simple Storage Service (S3) as both a staging area and long-term storage solution for input and result data. S3’s availability and scalability are far superior than even striped Elastic Block Store volumes running on an instance. Besides, using S3 over an instance serving an EBS file system means we have a smaller support footprint. So how can we move data quickly using multiple streams into and out of S3?

Multi-stream Uploads

S3 supports multiple stream data transfers differently depending on the direction of the transfer. If you move data into S3, you can make use of the multi-part upload feature. This enables you to break up a large file into chunks and upload them separately. When you’ve uploaded each chunk, you issue another call to S3 to stitch the chunks together. So we can take a large file, chunk it up, and transfer the chunks simultaneously over many threads to fill our upload pipe.

Multi-stream Downloads

When downloading S3 objects, you can make use of the ranged GET feature of S3. The Hyper Text Transfer Protocol (HTTP) defines a Range header you can add to GET requests. This feature allows you to specify a byte range and only transfer that part of the resource. Because S3 supports the Range header, we can create a list of chunks - specified by byte ranges - and download them simultaneously. After all chunks are downloaded, we can concatenate them and checksum the resulting file to verify integrity.

Test Environment and Results

We tested a new transfer tool by repeatedly uploading and downloading a 2.2 GB NGS file using different numbers of threads. Tests were run on an m1.large EC2 instance. The time to upload the entire file and the time to download it from S3 (including time to reassemble and verify the checksum) were recorded and averaged. As you can see in this first graph, the total bandwidth used in both directions increased with an increasing number of threads.

By using many threads, the time to transfer decreased dramatically. This graph shows the percent increase by number of threads. In the best case, we were able to decrease the time to upload the file to S3 by nearly 2.5 times!

Multi-stream Challenges

Even with a multi-part transfer tool, you still run into the following issues:

• You need to track the transfers and retry any that fail
• You can be too efficient with your transfers and overload your data pipe
• Some Internet proxies drop the Range header from requests silently, resulting in many copies of the entire S3 object to be downloaded
• The time to concatenate downloaded chunks can outweigh some of the transfer gains
• Many S3 tools assume the etag header of the object contains the MD5 checksum of that object (this is not the case with mutipart uploaded objects!) causing integrity checks to fail
  
  

Coming soon: A JetS3t app update

We’ll be addressing these challenges in the coming weeks. Unfortunately, the most commonly used S3 transfer tools have limited support for parallelization. We are big fans of s3cmd, but it doesn’t support multiple threads, and it lacks support for multipart transfers. JetS3t is another option. It’s Synchronize application has support for multiple threads, but it’s focused on moving lots of smaller files in parallel rather than one big file in chunks. Because the JetS3t library has support for multi-threaded transfers, and because it’s open source, we used it as the basis for our own multipart data transfer tool. It’s this tool that generated the results you see above.

We’ll be releasing an application to do this automatically in the next week or so, to enable anyone to easily do parallel uploads.

Experience with Data Transfer into EC2

Maximizing data throughput: Multi-stream data transfer into Amazon EC2

It is common for cloud computing articles to talk at length about the abundant hardware resources the cloud can offer the modern researcher or analyst, but little is typically said about the back end data store available with cloud computing. Before any research in the cloud can take place, data must be staged in a manner that is accessible to your cloud-based compute resources. It becomes non-trivial to perform the staging portion of your cloud use if your data sets are large.

The Amazon EC2 cloud provides large quantities of hardware suitable for high speed, high throughput scientific computing. Coupled with the AWS storage and Amazon S3 system it makes a formidable platform for anyone looking to do large scale, scientific computing on quantities of file-based data. In this post we’ll explore data ingress to EC2 as download speeds out of EC2 are typically much higher, both from consumer grade and enterprise level Internet connections, as we typically see more upload than download in the practical use of AWS services. Large data sets get transferred in to EC2, working data stays on cloud-local storage, and summarized, compact results are brought back from the cloud.

Let’s look at a few common cases for moving a large data set into AWS-hosted storage and explore the transfer rates, benefits and drawbacks of each approach.

Case #1: Consumer Grade Internet Connection

It is trivial to saturate the upstream network pipe of a consumer grade cable or DSL Internet connection transferring using only a single stream to EC2. With strong encryption on the ingress stream, CPU demands are insignificant for modern personal computing hardware. Given a 512 kilobit upload speed, it is possible to upload 5.1 gigabytes in one day with no compression.

Case #2: Mid-Range Business-Class Internet Connection

Among our clients we see average business-class Internet connections with six to eight megabits of upload performance. Even with the more capable six megabit upload connection, it is still possible to saturate the line with a single, encrypted rsync into EC2. With no compression, it is possible to transmit 64 gigabytes a day from a mid-range, business-class Internet connection.

Case #3: Enterprise-Level Internet Connection

With our larger clients it is not uncommon to see Internet connections with sustained data-out capabilities of 100 megabits or more. In this class of Internet connection WAN principles such as frame size, packet order and retransmissions start to come in to effect and require consideration to optimize the ingress rate. With little to no optimization, we have seen upper bounds on a single Internet-bound network stream of around 18 megabits. Even without compression or multiple streams, an Internet connection operating at this outbound rate can transfer nearly 200 gigabytes in a 24 hour period. There is, of course, potential for the single-stream transfer to be further optimized but often enterprise network teams have a philosophical problem of a single application leveraging more than 50% of an Internet connection shared by potentially tens of thousands of people.

Special Case #1: Parallel S3 Streaming

If the inbound files are relatively homogeneous and reasonably large, we have seen instances where a parallel upload into S3 can be advantageous. S3 is globally distributed with massively scalable ingress. When file sizes are similar, it is possible to saturate an outgoing network pipe without worrying about one stream of data lagging behind the others because it is larger. It should be noted that once data is in S3, it can be presented via s3backer to a large number of EC2 instances at high speeds using the same principles of S3’s massive scalability.

Special Case #2: Parallel Throttled rsync

There are cases where a single-stream approach out of an enterprise-class connection is insufficient for the data transfer needs. On a massive, shared network connection two undesirable conditions can be observed:

1. A single upload will sometimes fall very short of the available network bandwidth of a particular site due to the network topology; and
2. Multiple uploads will sometimes overwhelm a shared network connection causing complaints from other users and potential business disruption.

We have seen very good results packaging rsync transfers into multiple parallel streams, where each stream is limited to a rigid amount of bandwidth consumption, in situations where sustained throughput needs to be consistent and relative peak-free, but overall throughput should not user more than the network administrators are comfortable with. This approach allows us to push the limits of a shared network connection without saturating it. We often come as close as 90% of the available network bandwidth, meaning that we can perform a significant amount of data transfer into the cloud in a minimal amount of time, with no perceivable disruption to the other users and applications on the network.

Final Thoughts

The EC2 ingress rates are quite capable of handling large data transfers from connections with significant upstream bandwidth available to them. Parallel transfers with capped parallel streams can help take maximum advantage of enterprise-class outbound connections without imparting any serious latency issues on other users within your organization.

If the data is compressible it is possible, using tools such as rsync, to greatly decrease the number of bits transmitted over the wire. In our experiments we were able to increase upload rates by a factor of 3 on heavily compressible ASCII text data used for genetic computations. We have pushed data set uploads of 250 gigabyte to EC2 in 22 hours by using a single encrypted and compressed rsync stream on a 8 megabit upload link.

It is unlikely that the ingress performance into Amazon for even a single EC2 instance will ever come into play unless you have an outbound Internet connection with sustained rates of a gigabit per second or more. Nevertheless, we recommend running at least an m1.large machine for data reception on the EC2 side of the transfer to handle the encryption demands. We have seen inbound transfer rates of encrypted data surpass the capabilities of the basic m1.small instance type.

When performing HTTPS based transfers a potentially severe impact on transfer rates is traversing corporate infrastructure for HTTP proxy and DNS filtering. These HTTP security architectures tend not to be designed with sustained data transfer performance characteristics in mind but rather control over interactive web browsing. If at all possible, avoid these pieces of infrastructure in your data transfer path.

Finally: if your ability to transfer exceeds your available Internet connection, fear not. There is a solution. Simply mail Amazon a USB disk drive or drive array with your data and they will promptly load it into an S3 bucket for you. As amazingly low tech as this approach sounds, for multi-terabyte data transfer it can be surprisingly quick. After all, you should never under-estimate the bandwidth of a truck full of drives...