Setting Up External Access To A Veeam SureBackup Virtual Lab

Hey y’all, happy Friday! One of the things that seems to still really fly under the radar in regards to Veeam Backup & Replication is its SureBackup feature. This feature is designed to allow for automated testing via scripts of groups of your backups. An example would be if you have a critical web application. You can create an application group that includes both the database server and the web server and when the SureBackup job is run Veeam will connect a section of its backup repository to a specified ESXi host as a datastore and, start the VMs within a NAT protected segment of your vSphere infrastructure, run either the role based scripts included or custom ones you specify to ensure that the VMs are connecting to the applications correctly, and then when done shut the lab down and fire off an e-mail.

That workflow is great an all but it only touches on the edge of the power of what SureBackup can do for you. In our environment not only do we have a mandate to provide backup tests that allow for end-user interaction, but we also use SureBackup for test bed type applications such as patch tests. An example of the latter here is when I was looking to upgrade our internal Windows-based CA to Server 2012 R2. I was able to launch the server in the lab, perform the upgrade and ensure that it behaved as expected WITHOUT ANY IMPACT ON PRODUCTION first and then tear down the lab and it was like it never happened. Allowing the VMs to stay up and running after the job starts requires nothing more than checking a box in your job setup.

By default access to a running lab is fairly limited. When you launch a lab from your Veeam server a route to the NAT’d network is injected to the Veeam server itself to allow access, but that doesn’t help you all that much if you are wanting others to be able to interact; we need to expand that access outwards. This post is going to walk you through the networking setup for a Virtual Lab that can be accessed from whatever level of access you are looking for, in my case from anywhere within my production network.

Setting Up the Virtual Lab

 

The first step if you haven’t setup SureBackup in your environment at all is to set up your Virtual Lab.  The first of two parts here that are critical to this task is setting up the Proxy IP, which is the equivalent to your outside NAT address if you’ve ever worked on a firewall. This IP is going to essentially be the production network side of the Lab VM that is created when you setup a Veeam Virtual Lab.

1-set-nat-host

Next we need to set up an isolated network for each production port group you need to support. While I use many VLANs in my datacenter I try to keep the application groups I need to test on the same VLAN to make this setup simple, but it doesn’t need to be, you can support as many as you need. Simply hit add, browse out and find the production network port group you need to support, give the isolated network a name and specify a VLAN.

2a-setup-vlans

The last step of setting up the Virtual Lab in this regard is creating a virtual NIC to map to each of your isolated networks. So where I see a lot of people get tripped up with this is always make the proxy appliance IP address here map to the default gateway of the production network it is reflecting. If you don’t do that the launched lab VMs will never be able to talk outside of the lab. Second, in regard to the Masquerade IP try to aim for some consistency. Notice that in my production network I am using a Class B private address space but with a class C mask. By default this will throw off the automatic generation of the Masquerade IP and I’ve found it isn’t always consistent across multiple Virtual NIC setups.  If you setup multiple isolated networks above you need to repeat this process for each network. Once you are done with this you can complete your Lab Setup and hit Finish to have it build or rebuild the appliance.

2-create-nat-network

Tweaking the SureBackup Job

For the sake of brevity I’m assuming at this point that you’ve got your Application Groups setup without issue and are ready to proceed to fixing your SureBackup job to stay up and running. To do so on the Application Group screen All you have to do is check the “Keep the application group running after the job completes” box. That’s it. Really. Once you do that this lab will stay up and running until you right click on the job in the Veeam Backup & Replication Console and choose stop. I’ve been lobbying for year for a “stop after X hours” option but still haven’t got very far with that one, but really the concern there is more performance impact from doubling a part of your load since you are essentially running 2 copies of a segment of your datacenter. If you have plenty to burn it isn’t an issue.

3-keep-lab-up

Fixing the Routing

Now the final step is to either talk to your network guy or go yourself to where your VLAN routing is taking place and add a static route to the IP range of your inside the lab into the routing table through the Proxy Appliance’s IP. For the example we’ve been working through in this post our Proxy appliance has an IP of 172.16.3.42 and all of our Lab networks are within the 172.31.0.0/16 network. If you are using a IOS based Cisco switch to handle your VLAN routing the command would be

After that is done, from anywhere that route is accessible from you should now be able to pass whatever traffic inbound to the lab network addresses. So sticking with our example, for a production VM with the IP address 172.16.3.10, you would interact with the IP 172.31.3.10 in whatever way needed. Keep in mind this is for lack of a better word one way traffic. You can connect in to any of the hosts within the lab network but you can’t really have them reach directly out and have them interact on the production network.

4a-testing

One More Thing…

One final tip that I can give you on this if you are going to let others in to play in your labs is to have at least one workstation grade VM that you include in each of your Applications Groups with the software needed to test with loaded. This way you can enable RDP on that VM and they user can just double-click an icon and connect into the lab, running their tests from there. Otherwise if you have locally installed applications that need to connect to hosts that are now inside the lab you are either going to need to reconfigure the application with the corrected address or modify the user’s hosts file temporarily so that they connect to the right place, neither of which is particularly easy to manage. The other nice thing about a modern RDP session is you can cut and paste files in and out of it, which is handy if the user wants to run reports and the like.

4-connecting-into-the-lab

As an aside I’m contemplating doing a video run through of the setting up a SureBackup environment to be added to the blog next week. Would you find such a thing helpful? If so please let me know on twitter @k00laidIT.

Lots of new stuff coming from Veeam

Veeam had what they called “THEIR BIGGEST EVENT EVER” and while it at times did seem to be really heavy on the sales for the sake of sales pitch, there was a lot of stuff to legitimately be excited about for those of us who use their products. From the features coming in Veeam Backup & Replication in version 9.5 in a couple of months through the first new feature of next year’s version 10 all in total there were 5 major announcements here today that those of us using the product can make use of. In this post I’m going to run briefly through these and in the coming months will provide some deeper insights when possible.

Veeam Backup & Replication / Veeam ONE 9.5 (October 2016)

  • Nimble Storage Integration- Nimble with be the next vendor after EMC, NetApp and HP storage systems that will allow Veeam to interact at the array level, allowing for backups from snapshot. If you are a Nimble customer (like me) this is going to be some good stuff
  • Advanced usage of Windows Server 2016 ReFS- This is the real gravy here for anybody who is having to work with any kind of synthetic operations with their backup files. Through an integration Veeam has with Microsoft when ReFS is used to back your Veeam repositories your weekly rollups are going to take a heck of a lot less time and as well as see less storage consumption for long terms “weekly fulls”.  This is due to ReFS’ basic mechanism where file copies and moves never actually move data, it just moves the pointers. An example I’ve seen is on a 10 GB change rate backup the weekly full went from 35 minutes on NTFS to 5 minutes on ReFS. Now move that out to a real production dataset and you are really talking about something. There will be a lot more of this in follow-up posts.
  • Direct Restore to Microsoft Azure – If you are resource constrained (which you usually are in a situation where you need a restore) Veeam now has the ability to restore a VM (even if it is vSphere based) directly to Azure. Pretty cool and I think probably the first of what we’ll see on this thread
  • vCloud Director Integration
  • VeeamONE 9.5 – If your organization needs to work with charge back this is something that is directly supported in VeeamONE. If you haven’t played with VeeamONE yet, please do so, I’ve yet to meet anyone who hasn’t found one problem with VeeamONE when first installed in their virtualization environment

Veeam Agents (November-December 2016)
agent versions

Expanding on the Veeam Endpoint for Windows (and now Linux) Veeam has come out with a Veeam Agents for Windows and Linux product. While Endpoint is and will still be available for standalone installations, we finally have an enterprise managed version we’ve been looking for and we truly can have one centrally managed Veeam installation for our virtual, physical and workstation backups. As you can see there’s still a lot to like about the Free version including the new ability to restore directly to Azure or Hyper-V, the paid versions give us server grade capabilities such as Application-aware processing and transaction log processing. Further one I’m excited about as part of my use case for this is for my mobile workforce is the ability for workstations and remote office servers to cache their backups locally when they aren’t connected to the Internet and then ship them back to the corporate office or Cloud Connect repository when once again connected. This is good stuff that has been a long time coming.

Veeam Availability Console (Q1 2017)

I truly want to believe this is the first edge of “one UI to rule them all”, but the Veeam Availability Console is a web-based console to let you monitor and manage all of your Veeam resources; VBR, Agent, Cloud Connect, etc. This is an evolution of the managed backup portal available to Service Providers for a bit now and allows it to be moved downstream to the Enterprise. Let me  reinforce the emphasis on the Enterprise, while included in licensing you are going to have to be so big of an organization/installation to be allowed access to it. Hopefully as subsequent versions are released that will trickle down more.

Veeam Availability Orchestrator (Q1 2017, beta soon)

Veeam for a DevOpsy world. VAO will allow you to automate many of the processes you need to do with Veeam based upon your disaster recovery plan. Let’s say your plan requires you do so many backups, so many replicas, regular testing and comply with documentation practices. Orchestrator is going to allow you to take all that on paper and define it in workflows so in theory you are always in compliance, and if you aren’t have the documentation to show you where you aren’t. I’ve seen quite a few things about this, things that are going to be available to everybody to test soon, and they are all very powerful things.

Veeam Office 365 E-mail Backup (Q4 2016)

Of the new products announce this is the biggie. For those of us who have already began or have done Exchange migrations to Office 365, Veeam now has the ability to backup those mailboxes to your local repositories so that you always know that data is there. I don’t know how those conversations have gone for you but this is a major pain point for us in going to the cloud. Pricing or even how it is going to be sold still isn’t set but what is known is that when released the end of this year it will be free for a year for all Veeam customers with an active support contract and for 3 years for those with Enterprise Plus licensing.

Again, while I have no knowledge that it will happen I have to believe this is the first baby step into a whole host of things to make our cloudy life better in the future with Sharepoint, OneDrive and anything else coming down the road.

Veeam Backup & Replication integration with IBM storage (????, preview May 2017)

Finally the last announcement was the first related to Veeam Backup version 10, in this case the next storage vendor integration. This integration is going to work with any IBM product based on their Spectrum Virtualize software and should work like any other of their integrations. With this we also go to learn that the first technical preview of v10 will coincide with VeeamON 2017 in New Orleans, so mid May 2017.

Presenting at VeeamON 2015: Design, Manage and Test Your Data Protection with Veeam Availabilty Suite

Last week I was presented with the honor of being invited to speak at Veeam Software‘s annual user conference, VeeamON. While this was not my first time doing so I was very happy with the end result this year, with 30-40 attendees and positive feedback both from people I knew beforehand as well as new acquaintances who attended.

My session is what I like to think of as the 1-1000 MPH with Veeam, specifically targeting the SMB space but with lots of general guidelines for how to get your DR system up and running fast and as error-free as possible. Some of the things I do with Veeam buck the Best Practices guide but we have been able to maintain high levels of protection over many years without much interruption. The session starts with the basics of designing your DR plan, then designing your Veeam infrastructure components to suit your needs, followed by tips for the actual implementation and other tricks and gotchas I’ve run into over the years.

Anyway due to the amount of information that was covered I promised attendee’s that I would put my slide deck out here for reference so here it is. If anybody has comments, questions or anything in between please feel free to reach out to me either through the comments here or on twitter. For attendees please keep an eye on your e-mail and the #VeeamON hashtag as the videos of all presentations should be made available in the coming weeks.

Proud to be a Veeam Vanguard

On July 27th Rick Vanover over on the Veeam Blog announced the inaugural class of what is known as the Veeam Vanguard of which I am honored to have been selected as a member. What the heck is a Veeam Vanguard? While best described in Rick’s announcement blog post, my take is that this group is composed of members of the IT and virtualization global community who are Veeam users and go above and beyond in sharing their knowledge of the ins and outs of the various Veeam products.  Frankly I am flabbergasted to be named and wish to thank them for the nomination.

Without getting too gushy or fanboyish, I have found over the years that Veeam’s products tend to solve problems we all deal with in a virtualized world. Backup & Replication especially had made my day in, day out life easier because I know my data is nice and protected and I can test just about anything I want to do without effecting the production environment.

In closing I just want to say congrats to all of the other nominees and that I look forward to seeing what you have to share. To say the group is geographically diverse is an understatement as Veeam was ever so nice to include the nationalities of all members, it’s very cool to see so many flags represented. Many included I’ve followed on twitter and the blogspace for quite some time, while are others are new to me but in the end I’m sure there will be some great knowledge shared and I look forward to getting to know you.

Setting Up Endpoint Backup Access to Backup & Replication 8 Update 2 Repositories

A part of the Veeam Backup & Replication 8 Update 2 Release is the ability to allow users to target repositories specified in your Backup Infrastructure as targets for Endpoint Backup. While this is just one of many, many fixes and upgrades (hello vSphere 6!) in Update 2 this one is important for those looking to use Endpoint Backup in the enterprise as it allows for centralized storage and management and equally important is you also get e-mail notifications on these jobs.

Once the update is installed you’ll have to decide what repository or repositories will be available to Endpoint Backup and provide permissions for users to access them. By default every Backup Repository Denies Endpoint Backup access to everyone. To change this for one or more repositories you’ll need to:

  1. Access the Backup Repositories section under Backup Infrastructure, then right click a repository and choose “Permissions.”
  2. Once there you have three options for each repository in regards to Endpoint permissions; Deny to everyone (default), Allow to everyone, and Allow to the following users or groups only. This last option is the most granular and what I use, even if just to select a large group. In the example shown I’ve provided access to the Domain Admins group.
  3. You will also notice that I’ve chosen to encrypt any backups stored in the repository, a nice feature as well of Veeam Backup & Replication 8.

Also of note is that no user will be able to select a repository until they have access to it. In setting up the Endpoint Backup job when the Veeam server is specified you are given the option to supply credentials there so you may choose to use alternate credentials so that the end users themselves don’t actually have to have access to the destination.

Getting Started with Veeam Endpoint Backup

This week Veeam Software officially released their new Endpoint Backup Free product introduced at VeeamON last October after a few months of beta testing. The target for this product is to allow image based backup of individual physical machines, namely workstations, allowing for Change Block Tracking much like users of their more mature Backup & Replication product have been used to in virtualized environments. Further Veeam has made a commitment that in the product is and should always be freely available making it possible for anybody to perform what is frankly enterprise level backup of their own computers with no cost other than possibly a external USB drive to store the backup data.  I’ve been using the product throughout the beta process and in this post I’ll outline some of the options and features and review how to get started with the product.

Also released this month by Veeam is the related Update 2 for Backup & Replication 8. This update in this case allows a Backup Repository to be selected as a target for your Endpoint Backup job after some configuration as shown here. Keep in mind if you are wanting to backup to local USB or a network share this isn’t necessary but if you are already a B&R user this will make managing these backups much better.

Getting Started with Installation

Your installation optionsI have to say Veeam did very well keeping the complexity under the water in this one. Once downloaded and run the installation choices consist completely of one checkbox and one button. That’s it. Veeam Endpoint Backup seems to rely on a local SQL Server Express installation to provide backend services just like the bigger Backup & Replication install but it is installed on the fly. I have found that if there is pending Windows Updates to complete the installer will prompt you to restart prior to continuing to configuring your backup.

Configuring the Job

Once the installation is complete the installer will take you directly into configuring the backup as long as you are backing up to an external storage device. If you plan to use a network share or Veeam Backup Repository you will need to skip the step and configure the job once in the application. Essentially you have the following options:

  • What you wantto backup
    • Entire computer; which is image based backup
    • Specific volumes
    • File level backup
  • Where you want to back it up to (each will generate another step or two in the wizard)
    • Local storage
    • A shared folder
    • Veeam Backup & Replication repository
  • Schedule or trigger for backups
    • Daily at a a specific time
    • Trigger a backup on a lock, log off or when the backup target is connected


Personally I use one of three setups depending on the scenario. For personal computers I use a external USB drive triggered on when the backup target is available but set so that it never backs up more than once every 24 hours. In the enterprise using Endpoint Backup to deal with those few remaining non-virtualized Windows servers these are configured to backup to a Veeam Backup Repository on a daily schedule. Finally I will soon begin rolling this out to key enterprise laptop users and there backup will be to a B&R Repository as well but triggered on the user locking the workstation with a 24 hour hold down. Keep in mind all of these options can be tweaked via the Configure backup button in the Veeam Endpoint Backup Control Panel.

media-createCreating the Recovery Media

The last step of installing/configuring Endpoint Backup is to create the restore media. This creates a handy disk or ISO that you can boot off of to allow you to do a Bare Metal (or Bare VM :)) recovery of the machine. From an enterprise standpoint if you are rolling Endpoint Backup out to a fieldful of like machines I really can’t find a good reason to create more than one of these per model of device. Personally I’ve been creating the ISOs for each model and using it in conjunction with a Zalman VE-300 based external hard drive to keep from having lots of discs/pen drives around. If you are using this to backup physical servers it would also be a first step to being able to quickly restore to a VM if that is part of your disaster recovery plan.

As a trick what I’ve found is I have installed the product on a VM for no other reason but to create the recovery media. This way I know I’ll have the drivers to boot to it if need be. Further once you boot to the recovery media you’ll find all kinds of little goodies that make it a good ISO to have available in your bag.

Conclusion

I’ve played with lots of options, both paid and free, over the years for backing up a physical computer on a regular basis and even setting the general Veeam fanboy type stuff aside, this is the slickest solution for this problem I’ve ever seen. The fact that it is free and integrates into my existing Enterprise solution are definitely major added bonuses, but even in a standalone, “I need to make backups of Grandma’s computer” situation it is a great choice. If you find you need a little help with getting started the Veeam has created a whole Endpoint Backup forum just for this product. My experience both here and with other products is that there is generally very quick response from very knowledgeable Veeam engineers, developers and end users happy to lend a hand.

Top New Features in Veeam Backup & Replication v8

We are now a couple of months out from the release of version 8 of Veeam Software’s flagship product Backup & Replication. Since then we’ve seen the first patch release a couple of weeks after, almost a Veeam tradition, and I’ve had it deployed and running for a while now. In that time I’ve found a lot to really like in the new version.

End to End Encryption

Backup & Replication now has the ability to encrypt your backup data from the moment it leaves your production storage system, through the LAN and WAN traffic and once it is at rest, either on disk or tape. This encryption is protected by password stored both with humans as well as within the Enterprise Manager database keeping you from losing backups. Finally the encryption does not change ratios for either compression or deduplication of the backup data.

Resource Conservation Improvements

Quite a few of the new Backup & Replication features are geared towards keeping your RPO goals from getting in the way of production efficiency. First and foremost is the availability of Backup I/O Control, a feature that will monitor the latency of your production storage system and if measured metrics climb above a user defined level will throttle backup operations to return systems to acceptable levels.

On the networking side if you have redundant or other none production WAN links you now have the ability to specify preferred networks for backup data, with failover to production if it isn’t available. Further the WAN Accelerator for site to site backup copy and replication has been improved to allow for up to 3x what was seen in v7.

Cloud Connect

Both of the above features make this one possible. With this new version brings a new partnership opportunity where VARs and other cloud storage service providers have the ability to directly act as a repository for your backup data. These providers can then allow you to spin these backups up as part of a second offering or as part of a package. With this the need to own, manage and maintain the hardware for a DR site becomes much lighter and I personally believe this will be a big deal for many in the SMB space.

New Veeam Explorers for Recovery

Veeam has been phasing out the use of the U-AIR wizards for item level restore for a while but with v8 we now have the release of the Explorers for Active Directory, Microsoft SQL Server and Exchange. The Active Directory one is particularly of note because it not only allows you to restore a deleted AD item but do so with the password intact.  Transaction log backup for SQL servers is also now supported allowing for point in time restore. The Exchange option has a few new features but I especially like the option of recovering hard-deleted items.

These are frankly just the tip of the iceberg when it comes to the new features. For more on what’s new I recommend you checkout the What’s New documents for both Backup & Replication as well as for VeeamONE, Veeam’s virtualization infrastructure monitoring package.

 

VeeamON 2014: Conference Season Veeam Style

I write this aboard about the coolest painted plane I’ve had the pleasure of flying on, en route to Las Vegas, NV to attend and speak at the inaugural VeeamON conference being held at the Cosmopolitan.  The conference is being held by Veeam Software, one of the leaders in virtualization backup, known best for its Veeam Backup & Replication product. The conference itself represents a pretty big milestone for a global company who in my opinion has done a very solid job of getting social right from the corporate standpoint. It is also going to time well due to the pending version 8 release of Backup & Replication.

I have been working with Veeam’s Backup & Replication software for a little over four years now and find it to be both powerful as well as easy to use, a nice combination when talking about the product responsible for the safety of your data. I will be speaking about my experiences with this software package from the small government organization standpoint and how it helps us deal with some of the particular challenges that come from being in that segment. My session will be on Wednesday at 8:30 AM.

This will be my first time speaking in this type of setting so we’ll see how it goes, but there will be no shortage of seasoned veterans providing sessions. Others speaking include a great deal of the staff from Veeam including Anton Gostev, Doug Hazelman, Rick Vanover, & Ben Milligan and those are just the ones that I’m personally familiar with. Further the virtualization industry will also be well represented by the likes of Chris Wahl, Symon Perriman, and Joep Piscaer.  Finally Alexis Ohanian of Reddit will serve as the celebrity speaker. All in all for a first time event they seem to have brought some very strong speakers to the event, we’ll see if I can hold up my part.

What To Look For
One of the things that I really like about this conference is the variety of options they are providing attendees to make the most of their time. Monday is Partner day, open only to their partners, but at the same time they will be having a variety of community driven Veeam User Group sessions for the rest of us attending. Also from the community side of things there will be a few vBrownbag sessions sprinkled through Tuesday and Wednesday. These are generally much shorter, 15-20 minutes and are great for people to share little tips and tricks of the industry. I myself will be providing a session on Physical Backup Strategies on Tuesday at 8:20 talking about how we use the open source software product Areca Backup to handle the role of backing up the few physical machines I have left in my environment.

One of the biggest draws and one that will be of great importance to both me and my employer is the ability to take the Veeam Certified Engineer (VMCE) course while attending. This course, a prerequisite to being able to sit the VMCE exam, is typically $3000 US and last 5 days. At the conference they will be condensing it into 2.5 days and conference attendees are able to take the course for only $650.

Also going on as an aside to the sessions are the Lab Warz game and offsite tour of a Modern Datacenter. Registrants for Lab Warz will compete against each other to create the ultimate data protection scenario for cash and prizes. The offsite tour will take a group of attendees to the Cobalt Cheyenne datacenter to see how datacenter is done on the large scale.

Keynotes
Even f you are unable to attend yourself the Keynotes on both Tuesday and Wednesday will be streamed live.  The big news most likely will be the announcement of the general release of version 8 of Veeam’s Availability Suite which includes the Backup & Replication product as well at the Veeam ONE virtualization infrastructure monitoring package.  Both of these products have been in beta for the past few months and from my own personal experiences with them Veeam has done a very good job of making great software better.  I wouldn’t be surprised if there weren’t a few surprises announcements there as well. It’s not everyday you get to host your own inaugural global event, might as well take advantage

Conclusion
I’m going to go ahead and sign out here for now. Be sure to check back later as I plan to update frequently through the week with news and information.