Driving Innovation with Automation

MSA Leverages Control-M to Turn Big Data into Big Insights for Clients

About Management Science Associates

Automation: Scale or Fail

“We help customers extract powerful insights from their data and we use that intelligence to help them secure the best product positions in retail stores. It is not enough for us to tell customers ‘Here’s what the data says.’ We also want to say, ‘Here are the insights that are important to your business, and how you can capitalize on them’,” says Kevin Whitfield, Applications Group Director. “We want to empower customers to get those answers on their own through our systems, through embedded analytics, machine learning, and data science.”

To do that, MSA needed an efficient, modern infrastructure that supported its plan to transition to the Hadoop big data environment. Its legacy workload management system was impeding the company’s ability to scale out. Supporting the tens of thousands of daily jobs the IMS division needed for batch processing workloads and file transfers would further strain the system, increasing job failure rates to 20%. These issues impacted strict SLAs in MSA’s client contracts. To keep up, operations staff regularly worked weekends and overtime to meet daily and weekly deliverable commitments to customers.

“File transfers really drive our business. We deal with over 180,000 inbound, 8,500 internal, and 3,000 outbound file transfers on a weekly basis,” says Brandon Johnson, MSA’s Sr. Technical Manager for the IMS division. “We were dealing with hundreds of invalid file transfers weekly, just within our IMS division.”

The volumes, failure rates, and reliance on scripting in day-to-day operations were unsustainable if MSA wanted to take advantage of Hadoop to introduce new innovations and services.

Taking the Platform Approach

“MSA decided to significantly upgrade its workload automation and file transfer capabilities to support its development of new services on Hadoop. After an extensive investigation and piloting multiple workload automation products, MSA selected the Control-M digital business automation platform, including Control-M Managed File Transfer, because it has deep integration and support for Hadoop and databases, and robust self-service capabilities.

“About 75 percent of our internal users do their own work within Control-M. So the process that we have for how they build, develop, and promote through the environment is very important,” says Johnson. “Being able to use Control-M to enforce site standards and business rules to direct people to build their process flows correctly has been huge.”

Driving Innovation with Automation

Control-M does more than save steps for MSA. It saves time—time that has been reallocated to accelerating innovation and transitioning to Hadoop. The improved automation and control that Control-M provides cut 12 to 18 hours out of the time required to process key weekly client deliverables.

“Previously, the process would start on Wednesday and was finished on Sunday. Therefore, we had to pay a lot of people to work extra hours to make sure this process completed throughout the weekend. They’d done that for years until Control-M came into play,” says Johnson. “There’s absolutely no weekend time anymore. We basically reduced two days’ worth of work. Between that and having to spend less time on error resolution, we’ve repurposed 15 to 20 percent of our staff.”

What's next?

Following the successful migration to Control-M for its consumer packaged goods industry business, MSA is expanding its use across all divisions.

“Using Control-M allowed us to expand into new revenuegenerating opportunities. Combined with our Hadoop implementation, we can do more with our clients’ pointof- sale data and we have been able to take on new client work,” says Johnson. “It was very hard with the previous solution to be able to take on new projects, because the process was archaic and couldn’t address the needs of the client.”