We present Dithen, a novel computation-as-a-service (CaaS) cloud platform
specifically tailored to the parallel execution of large-scale multimedia
tasks. Dithen handles the upload/download of both multimedia data and
executable items, the assignment of compute units to multimedia workloads, and
the reactive control of the available compute units to minimize the cloud
infrastructure cost under deadline-abiding execution. Dithen combines three key
properties: (i) the reactive assignment of individual multimedia tasks to
available computing units according to availability and predetermined
time-to-completion constraints; (ii) optimal resource estimation based on
Kalman-filter estimates; (iii) the use of additive increase multiplicative
decrease (AIMD) algorithms (famous for being the resource management in the
transport control protocol) for the control of the number of units servicing
workloads. The deployment of Dithen over Amazon EC2 spot instances is shown to
be capable of processing more than 80,000 video transcoding, face detection and
image processing tasks (equivalent to the processing of more than 116 GB of
compressed data) for less than 1inbillingcostfromEC2.Moreover,theproposedAIMD−basedcontrolmechanism,inconjunctionwiththeKalmanestimates,isshowntoprovideformorethan27costagainstmethodsbasedonreactiveresourceestimation.Finally,Dithenisshowntooffera38state−of−the−artinCaaSplatformsonAmazonEC2(AmazonLambdaandAmazonAutoscale).AbaselineversionofDitheniscurrentlyavailableathttp://www.dithen.comunderthe"AutoScale"option.
