Case Study:

How to backup 350 servers and 1 million files every night

TRES internet BV

TRES internet is a leading online marketing company in The Netherlands. With a specialization in online strategy, concepts, building, and marketing, TRES internet is successful at archiving marketing goals and gaining customer value for its clients. For more information see www.tres.nl

The Challenge

TRES internet BV hosts over 350 servers, with over 1 million files. All of these files are backed up every night. As the number of files continued to increase, they faced new challenges: "We were using an in-house developed FTP program to perform backups, but it could no longer complete the backup during the night," said Erwin Hoberg, Software Developer and a member of the team responsible for the implementation of the new backup system. "In addition, the old program wasn't handling various error conditions very well, so we needed a new software with robust error handling, so we could be 100% sure that all those million files were successfully backed up every night," he continued.

The requirements for the new backup software could be summarized like this: Fast speed, reliability, automation, and extensive reporting. The first two were needed to perform the full backup reliably every night. And the last two were needed to integrate the backup software with the existing intranet system that provides overall control, automation, and management for all of the servers.


The Solution

When the development team started to search for a solution, the first thing they needed to tackle was the speed: "We did some testing with BatchSync FTP and it reached an improvement of over 10x the existing system. We were amazed by BatchSync's performance," Erwin Hoberg said. This improvement was largely because of BatchSync FTP's multithreaded nature, which uses multiple concurrent data streams to transfer data. "We also liked BatchSync's ability to handle various errors, report them, and continue the file transfer process without interruption," Hoberg continued. "With BatchSync's really fast speed and good error handling, we were able to cover the first two points of our requirements."

After having reliable data transfer in place, the development team needed to integrate BatchSync with their operational software. TRES internet BV had developed an intranet system to monitor and control all of their servers. This intranet system contains information about all of servers, including the IP addresses, login names and passwords, and current system and backup statuses. BatchSync was integrated into intranet system by using its command line interface. "We query our intranet database for all servers that need to be backed up. Then we call BatchSync with the needed command line parameters to perform the backup for each server separately. And finally, we save the backup results to the database," Hoberg explained. "This way we can see the backup status for all of the servers at one glance," he continued.

"The integration process went quite easily, but there were times when we needed technical support from SiteDesigner Technologies. And it was top notch. They were helpful, capable, and quick to find fixes and solutions for us. So all in all, I would definitely recommend BatchSync," Erwin Hoberg summarized.


The Conclusion

"The new backup system is now running in the production, and it is a success. It performs the full backup in 1 hour and 20 minutes, and it's fully automated, controlled, and monitored via our intranet system. It also supports company growth, as we can add new servers to the system, and they are automatically fully backed up. So we started with our requirements for speed, reliability, automation, and full reporting, and now we have it all," Erwin Hoberg said in conclusion.


Learn More