FeaturesOutsourcingSecurity ComplianceData managementCloud managementInfrastructure

Following the astounding failure of TVB's Miss Hong Kong 2012 voting exercise last Sunday night, when an unexpected volume of web traffic busted the voting application's preconfigured server load in a 10-minute voting slot, annoyed TV audience and critics bombarded online forums, and questioned the gaming mechanics and architecture design of the voting application.

In an exclusive interview with Asia Cloud Forum yesterday, Chin-Tang Chin (pictured below), Microsoft Hong Kong's director of developer and platform evangelism group addresses some of the most common concerns about the set up of the voting platform behind the Miss Hong Kong 2012 voting application, and the scale of the web traffic it received that night. Chin also explains the intricacies of building a truly scalable application, and why couldn't Microsoft auto-provision more server capacities to cater to the humongous internet requests that battered the voting application

How was the cloud platform set up for TVB Miss Hong Kong 2012 voting system different from that of TVB Fun?

Chin Tang Chin: Voting on TVB Fun is a three-step process: user log in, TV program selection and vote casting. As for TVB's Miss Hong Kong voting exercise held last Sunday, the application workflow was simplified for the audience, it required one to just input his/her Hong Kong identity card number and his/her choice of candidate.

"It involves a lot of business decisions and guidance to develop a scalable application that can handle huge traffic volume."   -- Chin-Tang Chin, Microsoft HK's director of developer and platform evangelism

[Editor's note: Voting for TVB's Miss Hong Kong 2012 could be done via a web application and a mobile application. Chin's reference to the "voting application" did not specify whether it was the web application or the mobile application, or both. In a brief interview with Asia Cloud Forum, Cherry Picks' CEO Jason Chiu clarified that his company was responsible for building the mobile application, but declined to name the party that built the web application. Users experienced "server error" on both voting applications.]

From IT's point of view, the voting system design was no different from that of TVB Fun -- Windows Azure was still used as the underlying cloud computing platform, and an application was written on top to collect the audience's voting information.

Of course, the exact business flow tailored for Miss Hong Kong was slightly different from TVB Fun's.

The Miss Hong Kong voting application was jointly developed by TVB and Cherry Picks, and Microsoft provided technical support with the underlying cloud platform (Windows Azure). While Microsoft provides full technical support, Microsoft may not have complete information on the project, because it was not involve in developing the voting application. Other than technology, a lot of other business decisions may be involved during the application design process.

Any application has to work with its underlying cloud computing platform. It involves a lot of business decisions and guidance to develop a scalable application that can handle huge traffic volume. For example, to be able to scale out, is the application designed to handle huge traffic volume by adding instances? During the process of designing the Miss Hong Kong voting application, Microsoft has provided Cherry Picks with some guidance including Microsoft's global technology documents MSDN (Microsoft Developer Network) on making the right decisions to develop a scalable application on Azure.

How much server capacity was provisioned for the TVB Miss Hong Kong 2012 voting exercise? Was it designed to cater to serve 50-100K voters?

Chin: It is not very convenient for me to discuss this, because this involves the customer's specifications, and this is confidential information. And as S K Cheong, general manager (broadcasting) of TVB said in a press conference yesterday, TVB will in 24 hours announce the appointment of a third-party consulting firm to investigate this incident. The report will provide a much detailed explanation for the incident with numbers and findings.

That said, the Miss Hong Kong voting application was designed to handle a certain level of traffic. On Sunday night, the web traffic that entered the application was way over the expected traffic -- well beyond the workload that the application was designed to cater to. This was why some audience could not access the Miss Hong Kong voting application.
 
I'd like to clarify that the voting system involves an application on top and Windows Azure cloud platform underneath. Throughout the entire process of Miss Hong Kong voting, Windows Azure did not suffer any impact, and Azure was functioning exactly the way it was supposed to do. In our entire data center, no other customer was affected due to the bursting of bandwidth [of the Miss Hong Kong voting application]. No customer reported performance slowdown, application unavailability, or server outage on the network computing level. In short, while Azure was functioning well throughout the whole period, it was the application on top that could not cater to the unexpected traffic.

How much more traffic are we talking about?

Chin: In our initial investigation of the log in Microsoft's data center, our engineers identified some unusual data traffic targeting at the TVB Fun [Miss Hong Kong 2012 voting] application. We have evidence to show that within 15 minutes just before the voting period started, the system has already recorded unusual data traffic that was many times higher than the original expected traffic. The traffic was substantially higher than the number of the audience of the TV show, and was also higher than the total population of Hong Kong.