Living Rural and Tech Community (It Really Is Possible)

I had/have the honor today of presenting a couple vBrownBag sessions while here at VMworld. The first of these was about my journey from living in an area with little to know Tech Community resources available to becoming a part of the bigger global community and why that’s a good thing. As I feel this has really changed my career and enabled me to grow my skills as an IT professional in ways I never thought possible this is a subject I’m pretty passionate about.

So does that paragraph sound familiar to you? If so please consider watching the presentation below (it’s only 10 minutes) and start your own journey. If you need help along the way reach out to me on twitter @k00laidIT.

VMworld 2017 US: T -2

I write this while traveling to sunny and amazingly hot Las Vegas for the 2017 edition of VMworld US. I hope to provide feedback and news throughout the conference, highlighting not only the excellent content and programs but also the best the virtualization community has to offer.

Today will be a travel day as well as a day to meet up with friends, new and old. Tomorrow, the Sunday before the conference, is when the real fun begins with things like Opening Acts for me, TAM and partner content for others as well as a number of social events.

What We Know So Far

Yesterday was the day that Vmware went on a killing spree, announcing the depreciation of Windows based vCenter, the flash based vSphere web client and the vmkLinux APIs and its associated driver ecosystem. All of these enter the depreciated state with the next major version of vSphere and then will be gone for ever and ever in the revision after that. Each of these are significant steps towards the evolution of vSphere as we know it, and when coupled with the advances in PowerCLI in version 6.5 the management of our in house infrastructure has been changed for the better.

These announcements came rapid fire on the Friday before Vmworld with the death of the Windows based vCenter coming first. As we have had versions of varying success of the vCenter Server Appliances (VCSA) for over 5 years now it’s been a long time coming. I myself migrated two years ago and while it was good then with the latest 6.5 version, with its PhotonOS base, excellent migration wizard and in appliance vCenter Update Manager support it has show it is definitely the way forward.

The flash client was the next announcement to come and again, we are looking at an depreciation that needs to happen and is most definitely going to be a good thing but does come with some apprehension. With most things that have been depreciated by Vmware we’ve had at least 1 feature rich version of the replacement out and stable before they announced the predecessor’s demise. This isn’t the case with the flash based web client. While the latest builds are getting very, very good there are still major things that either are quirky or simply aren’t there yet. The good news to this is we have been given almost immediately assurances by everyone involved with the product management that we the vSphere admins will never be left without a GUI management ability for any given task we have today and I for one believe them. The last components of what is known as the HTML5 client in my opinion simply can’t come enough, I’m tired of having to hop through multiple GUIs and browsers to be able to perform basic tasks in my daily work life.

Finally the day was finished with the announced depreciation of the non-native Linux drivers. To be honest I didn’t know that these were even still a thing as every Linux VM I’ve rolled for the past many years have been able to work with the native drivers. I’m sure there are those that at this point may still need additional time but the as the removal is still a couple of versions off this should be something can be mitigated now that the end is known.

Conclusion

With all of these preconference announcements related to Vmware’s flagship product is this going to be the year where Vmworld is chocked full of improvements to vSphere. This will be my 3rd one in 4 years and each year I’ve felt their focus was elsewhere. While vSAN, NSX, and the like are definitely where the company’s seeing growth all of these things rely on vSphere as an underlay. I for one would be happy to see a little love shown here.

With that happy thought I’m going to shut it down and land. For those coming to Vmworld this weekend safe travels and for those at home look for more info as its known here on koolaid.info.

A VMworld US 2017 To Do List

If you work in the virtualization or datacenter field (are they really different anymore?) you probably know that VMworld US 2017 is next week, August 27-31. While VMware may not be the only option out there when it comes to virtualization anymore VMworld is still the defacto event for people in the field. This conference’s definition of community is unrivaled in scope with just as much if not more going on outside of the conference agenda as  in it.

As with all things worth doing conference attendance probably needs a checklist. Have you done yours? If not here are the high points of mine. I’m not going to bore you with “Jim will be attending session so and so”; well except for VMTN6699U and VMTN6700U you should totally join me at those sessions, but these are pretty general things I try to do each time.

  • Take Your Vitamins– I hate to say it but the Vegas Flu is a real thing. Between being in the recirculated air of a jumbo jet for any number of hours to bookend event and being in the recirculated air of a Vegas hotel/casino/conference center I always seem to get at least a mild head cold at some point during the week. Start about now taking whatever version of Vitamin C supplement you like and do so throughout the event to help head this issue off.
  • Bring Sharable Power- The average conference attendee has 3 devices on them at all times, phone, tablet and laptop. These things will start to get low on battery about midday and that just won’t do. In theory lots of places will have power outlets but with 25,000+ attendees they are still in short supply. I typically bring a big battery pack, a travel surge protector and USB power cables for everything under the sun so that I can plug in and share at sessions and keynotes.
  • Get There Early and Be Ready To Learn– While the conference doesn’t start in earnest until Monday the 27th I always try to arrive midday Saturday because there is so much going on before the conference starts. One of the highlights of the entire conference to me each year is Opening Acts, a series of panel sessions put on by VMunderground and vBrownBag on Sunday afternoon. These sessions always prove to be insightful and are traditionally more career-centric or more wide-ranging than your typical VMworld session. The fact that this is followed by the always awesome VMunderground party that night is not lost on me either. Also, if you are a VMware TAM customer there is exclusive content for you on Sunday afternoon.
  • Be Comfortable Being Yourself– So what do you wear? My friend Matt Crape covered this well in his recent post but I would like to add that go with what makes you most comfortable networking with your peers. If you are good with shorts and a t-shirt, go for it. Me personally I’m a golf shirt and jeans kind of guy so that’s most of what you’ll see from me. Your days at VMworld are most likely going to be between 15-20 hours so go with what feels good unless that’s naked. Nobody needs to see that. 😉
  • Get Out and Be Social– This is not a “Woo Hoo, It’s Vegas So Let’s Party” topic. Yes, you can do that if that’s your prerogative, but keep in mind some of the smartest minds in your chosen career are going to be here and out at both events in the evening as well as in the hang space during the day. Go meet people as they are typically pretty nice and cool. While the VMworld sessions are what’s being sold as the content of the conference I will book very few of those, choosing instead to spend my time learning from others how they are dealing with many of the same issues that I have and make connections that can prove helpful down the road.
    Where to go be social? During the day the HangSpace/ VM Village is the place to go. In the evenings there is a never-ending list of gatherings to find your way to. I personally will be making sure I attend the Veeam party and VMunderground as they are my 2 evening must do’s each year and are typically among the biggest. Past that I’ll just go with the flow.
  • Be Social Online Too– If you are a tweeter be sure to use not only the #VMworld hashtag but also that of whatever session or event you are currently in. If you look around it will typically be on a wall somewhere. This will help you extend the conversation during the session. If you aren’t on twitter yet you may want to consider that, often this is a great way to see what your colleagues are saying about announcements and such in real time. It also serves a great way to meet up with others at the conference.
  • Get Some Sleep When Possible– I know this sounds counter-intuitive to the previous topic but if you are a 40-year-old like me this week will catch up to you. It is definitely possible to do events and conference from 7:30 AM to after midnight each day and while that’s a lot of fun, by Wednesday there are so many zombies walking around Mandalay Bay it looks like an episode of the Walking Dead. If you’ve been working on the session builder already take a look at your schedule and make room for you to sleep in a morning sometime midweek. You can catch up on the sessions once you get back.

While there’s more than that for me those are the basics. If you are going please hit me up @k00laidIT on twitter, I’d love to have a coffee, a beer or just a conversation with you. Have a great time!

P.S. Wear comfortable shoes!

Tech Conferences in Las Vegas for Newbies

As June is here we are deep into tech conference season already so I find myself behind the curve somewhat with this post, but here we are. I am extremely fortunate to have an employer who understands the value of attending Tech Conferences for IT Professionals and I’ve been able to attend at least one each year since 2014; going back and forth between CiscoLive and VMworld with a sprinkling of VeeamON and more local events such as vBrisket and VMUGs for good measure. As a “Hyper-Converged Admin” my choice of which “biggie” conference is done each year by looking at where my projects land; last year was CiscoLive due to a lot of Voice and Security Projects, this year VMworld due to lots of updates coming down the pike there and a potential VDI project.

The problem when you have a conference with north of 25,000 attendees is that you are limited in where you put these on. While Cisco does tend to move around some, VMworld has typically either been in San Francisco or Las Vegas. With the Moscone Center closed again this year for renovation we find pretty much all of the big guys are back in Las Vegas, with both CiscoLive and VMworld at Mandalay Bay once again as well as AWS re:Invent and Dell/EMC World in town this year as well. If you haven’t been to one of these Tech Conferences before or to Las Vegas both can be both exciting and overwhelming, but with a little help from others and some decent tips neither are that big of a deal.

Las Vegas Basics

So for a small town guy like me Las Vegas is very cool town, but tiring. The common thread I feel and have heard others voice as well is that Las Vegas is deceptively large because all of the hotels on the strip are so massive. While you can see from your Mandalay Bay window that New York New York is just the next block, it is probably about a mile away if walking there. Why this is important is that if you look at the list of hotels on each conference’s list you’ll see lots of options, but getting to that 8AM session may require a 30+ minute walk or even longer shuttle ride if you chose to stay at the Cosmopolitan (my personal favorite of all Las Vegas hotels but prohibitively far away). Couple that with temperatures in the triple digits during summer and proximity becomes more important.

Hotel Choices

So the first tip for any of these conferences is get a hotel as close as possible. For CiscoLive and VMworld keep in mind that you can move freely between the Mandalay Bay, Delano the Luxor and the Conference Center without ever stepping foot outside.  I would highly recommend trying to be in one of these. If you are booking late and the conference is out of rooms it’s worth trying to book directly through the hotel as they don’t let the events have the whole place. That said you are still going to be in for a hike. For example I stayed in the Mandalay Bay last year and it was approximately 1800 steps from my room to the entrance to conference.

Many of the vendor types that seemingly live their lives at these types of events like to opt for either the nearby Marriott Courtyard Las Vegas South or the Holiday Inn at Desert Club Resort for those that like a kitchen. From either of these you’re a quick Uber or Lyft away from the Conference Center entrance but don’t have to deal with the hustle and bustle of staying on the Strip if you don’t want to.

Getting Around

Speaking of Uber and Lyft, getting around with out walking is a bit of a consideration as well, both for the daily commute as well as for the various events. Traffic in the afternoons into the early morning is pretty impressive on the actual strip so to be honest I’ve not heard good things about trying to rely on the conference shuttles when available. Further I’ve heard many complaints from those who are locals that drive in and try to find parking.

Where that leaves you is 1) ride sharing service, 2) using the monorails, or 3) walking. Uber is nice because they are pretty knowledgeable about routing you around traffic regardless of time of day. Keep in mind when it comes to this and Mandalay Bay there are actually two defined Uber pickup/drop off spots, one outside of the conference center and another around the valet area underneath the hotel drop off area. These are impressively far apart so be sure you know where you want picked up before you request a ride.

The monorails are also nice but short. For those of you going to CLUS this is a good way to get to the Customer Appreciation Event as it will drop you off close to the T-mobile Arena.

Finally walking is a decent option, especially after dark for the various vendor events, but I do recommend if you are going to do it find a buddy or 3 or 4. I’ve never personally seen violence on the strip but you hear about it and there are lots of “character buskers” dressed like everything from Michael Jackson to Spongebob that will harass you.

One final note, while first impressions are important there really isn’t any point to being that person in the fancy shoes unless you’ve got booth duty. I typically while go buy a new pair of good running shoes a week or two before the conference so I can break them in and then that’s what I wear. If you are a step tracker kind of person like me expect 20,000 and up each day so take care of your feet.

Things To Do

Seriously, there’s plenty to do even if you weren’t at a conference already providing lots to do. Regardless of your interest if the conference doesn’t have you jam-packed enough you can find something you like here.

If you are new to IT or are just starting to get your name out there the most important things to do outside of the sessions is to get out there and be social. Both of the conferences we are talking about here have a great community that surround it with some wonderful people in it. The first step if you aren’t already would be to get yourself on twitter and follow the hashtag stream for your event (#CLUS for CiscoLive US, #VMworld for VMworld) , not only while you are there but before especially as many outside events will be planned then. Be sure to find the social area for your given conference and go make friends. Outside of the standard conference hours you’ll find that many of the Vendors will have events planned for attendees. If you have partners or vendors you work heavily with its worth asking your SE if they are doing anything.

CiscoLive Basics

CiscoLive will be held this year June 25-29th and promises to be a great show once again. While I have really enjoyed all of the conferences I’ve attended CLUS  was my first and near to my heart. First off of all those I’ve been to this one feels more academic than others. There aren’t really as many softball sessions and the sessions are a bigger part of the focus for the event than other. That said, they do a very good job of supporting the social community by having a Social Media Hub right in the middle of it all with special events for the twitterazzi most days. I highly recommend showing up and if nothing else walking up and just introducing yourself, trust me, you’ll fit right in there somewhere especially if you bring a Kilt. 😉 If you can come in early on Sunday the annual Tweetup Sunday afternoon is always a good time to make friends.

If you are going to CiscoLive you should have at this point booked most of your sessions. A couple of points here. First do not overbook yourself on sessions. While the pressure is always there to make sure you are getting all the education out of it as possible every session these days is recorded and can be watched later. My decision on if I’m going to do a particular session is based on if the subject is directly related to something that’s got me stumped and I want the opportunity to touch base with the speaker. Past that I’ll watch most after the fact. A better use of your time is getting out and networking, soaking up some of the distributed information there and will in many cases serve as a resource after the fact. I’ve yet to leave an event and not come home to do some kind of redesign based on things I’ve learned from the community.

A highlight for anybody who’s been to CLUS is always the Customer Appreciation Event. This year Bruno Mars will take over the T-mobile Arena and I am legitimately bummed that I will be missing it. The celebrity keynotes are always very good as well and usually provide a different view on how technology interacts with the world. I truly enjoyed listening to Kevin Spacey last year and this year they’ve booked Bryan Cranston.

Regarding keynotes, I typically like watch these in the social areas rather than packing myself into the keynote halls. The seating is better, there’s fewer people and usually refreshments are close at hand, plus you can find a surface to put your computer/iPad on to take notes and/or live tweet the talk.

VMworld Basics

As much as the focus on CiscoLive is on the direct educational benefit the focus from VMworld is more on learning from the community. With the conference officially running from August 27-31 there just as many official conference sessions as there are at CiscoLive, but I find there to be more lower level, marketing style sessions at VMworld. What makes up for it though is any number of community learning opportunities surrounding it. If you can swing coming in either Saturday or very early Sunday the vBrownbag/VMunderground Opening Acts is always a great place to learn about what is coming next in virtualization and technology. Speaking of vBrownBag, these guys have a stage running concurrent to the conference with session about anything you can conceive of all week long. Historically the vBrownBag stage has been found in the Hang Space (VMworld for social media area) but this year is still to be determined.

Another thing you’ll find is the potential to have your evenings books is exceptionally high with multiple vendor events every single night, traditionally starting with vBeers on Saturday evening. At some point as we get closer to the conference VMworld will fill a website with information and registration links for many of the gatherings to make scheduling easy. The Veeam, VMunderground and vExpert/VCDX/VMUG parties are always the most talked about. There is also the annual VMworld Party with typically big name acts but at the time of this writing there really isn’t any information about this yet. Be sure to follow along online and on social media to find out soon enough.

Conclusion

With all that being said, just go enjoy yourself as you are meant to do. There’s a reason that Denise Fishburne refers to CiscoLive as “Geek Summer Camp” because it does feel that way, regardless of the conference you’re attending. Everybody does things their own way. As I’ll be attending VMworld this year if you are there and want to say hi feel free to reach out and find me on twitter @k00laidIT.

Vegas Baby! Heading to CiscoLive! 2016

As 2016 moves into April we find ourselves ready to go into the conference season once again. For the past couple of years I’ve been to VMworld because that is where my work has had me focused, but for the same reason I will be heading the Cisco Live in Las Vegas, NV this year. The event will be held at the Mandalay Bay Resort July 10-14. Yes it will be hot, but let’s be honest you are going to be inside most of the time. This is the 2nd time I’ve attended Cisco Live US (you may see it referred to as #CLUS quite a bit) and if this is anything like the last time it’s going to be great. I have been particularly impressed with the content they make available and the community that has grown around it.

What to do

The first and foremost thing you should check out at Cisco Live is the always excellent sessions throughout the conference. If you are new to conferences this is actually something to consider sooner than later; the session catalog is currently up and the scheduler will open on May 3. I recommend that if you have any particular sessions or focus you are looking at with this trip go ahead and have a list done early and then be ready on the 5/3, many popular sessions will fill up quickly and nobody wants to wait in the overflow line. 😉

To be honest if you just look at the scope of topics covered in the session list it is a bit overwhelming. While I’m no grizzled veteran of conferences by any means what I’ve found best is to pick a focus or two and then start there. For example this year we have a big focus on upgrading our edge security and our production datacenter to include Cisco UCS solutions. What sessions I pick will almost entirely be from either the Security and Datacenter & Virtualization tracks to support those goals. Keep in mind all of these sessions will be available to you online after the fact so keep in mind the people giving them as well.

cae

If you have never been to one of the major tech conferences (20k attendees and up) there is never really a shortage of things to do, ranging from the educational to the social to just straight fun. Cisco Live is in my opinion a great event with a better than most mix of content and social, the highlight of which is the Customer Appreciation Event. The CAE this year will be held at the T-Mobile Arena and features concerts with Maroon 5 and Elle King. I saw Maroon 5 in a very cold field  a couple of years ago and they are a pretty good show and I’ve really liked what I’ve heard from Elle King on the radio.

Besides the concert event there will be no shortage of things to do if you are socially inclined. Besides the mixers each evening there are a wide array of events from different vendors in the Cisco ecosystem each evening. Many of these are by invite only so now would be an appropriate time to be reaching out to Account Execs you have at the various vendors and see if they are doing anything there.

 

20130627_173819000_iOSGo forth and be social

This will be my 6 tech conference in 4 years and while the content of the sessions is great and extremely helpful like I mentioned above all of that content is available online, 24/7/365 after the conference. What is not is the ability to meet and have conversations with some of the best minds of our chosen field. My very first major conference was CLUS 2013 in Orlando, FL and as I got myself out of my shell and started to meet people I was frankly floored by the combined brain power in such a small area. The way I’ve often put this to people is that the entire state of West Virginia, where I am from, has a total of 3 CCIEs in it. While this is not a normal demographic, there are only 50,000 some worldwide. At one point that first year I found myself  sitting in a discussion where out of 20 people I was the only person NOT a CCIE and really it is amazing what you can absorb in the social settings at Cisco Live. If you are willing to put yourself out there and not be the cave-dwelling geek many of us are naturally drawn to be you will find a community of people who will readily accept you in.

So how do I find such social people and befriend them? Well fear not there are plenty of ways. To start with if you are just starting out in your tech career the very first advice is to get yourself on twitter if you haven’t already. I literally setup my twitter account walking down the main concourse of CLUS 4 years ago and it has presented no end of enjoyment, help and opportunity since. Once you have said account head on over to Tom Hollingsworth’s site and sign yourself up for the annual twitter list.

Now that you are in the social mood right off the bat one of the first places I will be locating is the Social Media Hub. This is pretty much the main congregation area for the socials types. At some point in the early evening Sunday there will be an opening Tweetup there, if you attend be sure to say hi!

If you are interested in going yourself but haven’t registered yet you can do so on the Cisco Live 2016 website.

A how-to on cold calling from the customer perspective

Now that I’m back from my second tech conference in less than two months I am fully into the cold call season and I am once again reminded why I keep meaning to buy a burner phone and setup a Gmail account before I register next year. It seems every time I get back I am destined to months of “I am so glad you expressed deep interest in our product and I’d love to tell you more about it” when the reality is “I am calling you because you weren’t nimble enough to lunge away from our team of booth people who are paid or retained based on as many scans they can get. Most often when I get these calls or e-mails I’ll give each company a courteous thanks but no thanks and after that the iDivert button gets worn out.

The genesis of this post is two-fold. First a cold call this morning that was actually destined for my boss but when informed he wasn’t here went into telling how glad the person was that I had personally expressed interest in their product, WTF? This first event reminded me of a second, where a few months ago I was at a mixer preceding a vendor supplied training when I was approached by a bevy of 20 something Inside Sales Engineers and asked “what can I do to actually get you to listen?” From this I thought that just in case a young Padawan Sales Rep/Engineer happens to come across this, here are those ways to make your job more efficient and to stop alienating your potential customers.

Google Voice is the Devil

I guess the first step for anybody on the calling end of a cold call scenario is to get me to answer the phone. My biggest gripe in this regard and the quickest way to earn the hang up achievement is the currently practice of many of startups out there to use Google Voice as their business phone system. In case you don’t know with Google Voice they do local exchange drop offs when you call outside of your local calling area, meaning that when you call my desk I get a call with no name and a local area code, leaving me with the quandary of “is this a customer needing support or is this a cold call?” I get very few of the former but on the off-chance it is I will almost always answer leaving me hearing your happy voices.

I HAVE AN END CALL BUTTON AND I AM NOT AFRAID TO USE IT, GOOD DAY TO YOU SIR/MADAM!

You want to know how to do this better? First don’t just call me. You’ve got all my contact info so let’s start with being a little more passive and send me an e-mail introducing yourself and asking if I have time to talk to you. Many companies do this already because it brings with it a good deal of benefits; I’ve now captured your contact info, we’re not really wasting a lot of time on each other if there is zero interest, I don’t have to drop what I am dealing to get your pitch. If this idea just absolutely flies in the face of all that your company holds dear and you really must cold call me then don’t hide behind an anonymous number, call me from your corporate (or even better your personal DID) with your company’s name plastered on the Caller ID screen so at least I have the option to decide if it’s a call I need to deal with.

A Trade Show Badge Scan List Does Not Mean I am (or anybody else is) Buying

I once again had an awesome time at VMworld this year but got to have an experience that I’m sure many other attendees have had variants of. There I was, happily walking my way through the show floor through a throng of people, when out of my peripheral vision a booth person for a vendor not to be named literally stepped through one person and was simultaneously reaching to scan my badge while asking “Hi, do you mind if I scan you?” Yes, Mr./Ms. Inside Sales person, this is the type of quality customer interaction that happened that resulted on me being put on your list. It really doesn’t signify that I have a true interest in your product so please see item one above regarding how to approach the cold call better.

I understand there is an entire industry built around having people capture attendee information as sales leads but this just doesn’t seem like a very effective way to do it. My likelihood of talking to you more about your product is much higher if someone with working knowledge of your product, say an SE, talks to me about your product either in the booth or at a social event and then the communication starts there. Once everybody is back home and doing their thing that’s the call I’m going to take.

Know Your Product Better Than I Do

That leads me to the next item,  if by chance you’ve managed to cold call me, get me to pick up and finally manage to keep me on the line long enough to actually talk about your product, ACTUALLY KNOW YOUR PRODUCT. I can’t tell you how many times I’ve received calls after a show and the person on the other end of the line is so blatantly doing the fake it until you make it thing it isn’t funny. Keep in mind you are in the tech industry, cold calling people who most likely are fairly tech savvy and capable of logical thought, so that isn’t going to work so well for you. Frankly, my time is a very, very finite resource and even if I am interested in your product, which is why I took your call, if I’m correcting the caller that is an instant turn off.

I get that the people manning the phones aren’t going to be Senior Solutions Architects for your organization but try this on for size; if you’ve got me talking to you and you get asked something you don’t know, don’t be afraid to say you don’t know. This is your opportunity to bump me up the chain or to loop in a more technical person to the call to get the discussion back on the right track. I will respect that far more than if you try to throw out a BS answer. Meanwhile get as much education as you can on what you’re selling. I don’t care if you are a natural sales person, you aren’t going to be able to sell me my own pen in this market.

Employees != Resources

So you’ve got yourself all the way through the gauntlet and you’ve got me talking and you know your product, please don’t tell me how you can get some resources arranged to help me with designing my quote so the deal can more forward. I was actually in a face to face meeting once where the sales person did this, referring to the technical people within the organization as resources and I think my internal response to this can best be summed up in GIF form:

obama_kicks_door

This absolutely drives me bonkers. A resource is an inanimate object which can be used repeatedly without consequence except in the inevitable end result where the resource breaks. What you are calling a resource is a living, breathing, most likely highly intelligent human being who has all kinds of responsibilities, not just to you but to his family, community and any other number things. By referring to them as this, and therefore showing that you think of them as something that can be used repeatedly without consequence, you are demeaning that person and the skill set he or she has, and trust me that person is most likely who we as technical professionals are going to connect with far more than we are with you.

So that’s it, Jim’s guide to getting me on the phone. I’m sure as soon as I post this many other techniques will come to my mind and I’ll have to update this. If you take this to heart, great, I think that is going to work out for you. If not, well, I still hope I’ll remember to buy that burner phone next May and the Gmail account is already setup. 😉

Veeam Backup Repository Best Practices Session Notes

After a couple days off I’m back to some promised VeeamON content. A nice problem that VeeamON had this year is the session choices were much more diverse and there were a lot more of them. Unfortunately this led to some overlap of some really great sessions. A friend of mine, Jaison Bailey of vBrisket fame and fortune, got tied up in another session and was unable to attend what I considered one of the best breakout sessions all week, Anton Gostev‘s Backup Repository Best Practices so he asked me to post my notes.

For those not too familiar with Veeam repos they can essentially be any manner of addressable disk space, whether local, DAS, NAS, SAN or even cloud, but when you start taking performance into account you have to get much more specific. Gostev, who is the Product Manager for Backup & Replication, lines out the way to do it right.

Anyway, here’s the notes including links to information when possible. Any notations I have are in bold and italicized.

Don’t underestimate the importance of Performance

  • Performance issues may impact RTOs

Five Factors of choosing Storage

  • Reliability
  • Fast backups
  • Fast restores
  • DR from complete storage loss
  • Lowest Cost

Ultimate backup Architecture

  • Fast, reliable primary storage for fastest backups, then backup copy to Secondary storage both onsite AND offsite
  • Limit number of RP on primary, leverage cheap secondary
  • Selectively create offsite copies to tape, dr site, or cloud

Best Repo: Low End

  • Any Windows or Linux Server
    • Can also serve as backup /backup proxy server
  • Physical server storage options
    • Local Storage
    • DAS (JBOD)
    • SAN LUN
  • Virtual
    • iSCSI LUN connected to in guest Volume

Best Backup Repo: High End

Backup Repos to Avoid

  • Low-end NAS  & appliances
    • If stuck with it, use iSCSI instead of other protocols * Ran into this myself with a Qnap array as my secondary storage, not really even feasible to run anything I/O heavy on it
  • SMB (CIFS) network shares
    • Lots of issues with existing SMB clients
    • If share is backed up by server, add actual server instead
  • VMDK on VMFS *Nothing wrong with running a repo from a virtual machine, but don’t store backups within, instead connect an iSCSI LUN directly to the VM and format NTFS
    • Extra logic on the data path- more chances for data corruption
    • Dependent on vSphere being functional
  • Windows Server 2012 Deduplication (scalability) *I get his rationale, but honestly I live and die by 2012 R2 deduplication, it just takes more care and feeding than other options. See my session’s slides for notes on how I implement it.

Immediate Future: Technologies to keep in mind

  • Server 2016 Deduplication
    • Same deduplication, far greater performance and scale (64 TB files) *This really will be a big deal in this space, there is a lot of upside to a simple dedupe ability rolled into a Windows server
  • ReFS 2.0
    • Great fit for backup repos because it has built in data corruption protection
    • Veeam is currently working on some things with it

Raw Disk

  • Raid10 whenever you can (2x write penalty, but capacity suffers)
  • Raid5 4x write penalty, greater risks)
  • Raid6 severe performance overhead (6x write penalty
  • Lookup Maximum performance per spindle
  • A single job can only keep about 6-8 spindles busy- use multiple jobs if you have them to saturate
  • RAID volume
    • Stripe Size
      • Typical I/O for Veeam is 25k-512KB
      • Windows Server 2012 defaults to 64KB
      • At least 128 KB stripe size is highly recommended
        • Huge change for things like Synthetics, etc
    • RAID array
      • Fill as many drives as possible from the start to avoid expansion
      • Low-end sorage systems have significant performance problems
    • File System
      • NTFS (Best Option)
        • Larger block size does not affect performance, but it helps avoid excessive fragmentation so 64KB block size recommend
        • Format with /L to enable larger file records
        • 16 TB max file size limit before 2012 (now 256)
        • * Full string of best practices for format NTFS partition from CLI: Format <drive:> /L /Q /FS:NTFS /A:8192
      • ReFS not ready for prime time yet
      • Other
    • Backup Job Settings
      • Always a performance vs disk space choice
      • Reverse incremental backup mode is 3x I/O per block
      • Consider forever incremental instead
      • Evaluate transform performance
      • Repository load
        • Limit concurrent jobs to a reasonable amount
        • Use ingest rate throttling for cross-SAN backups

Dedupe Storage: Pains and Gains

  • Gains
    • True global dedupe
    • Lowest cost/ TB
  • Do not use deduplicating storage as your primary backup repository!
  • But if you must leverage vendor-specific integrations, use backup modes without full backup transformation, us active fulls instead of synthetics
  • If backup performance is still bad, consider VTL
  • 16TB+ backup storage optimization for 4MB blocks (new)
  • Parallel processing may impact  dedupe ratios

Secondary Storage Best Practices

  • Vendor-specific integrations can make performance better
  • Test Backup Copy retention processing performance. If too slow consider Active Full option of backup copy jobs (new in v9)
  • If already invested and stuck
    • Use as primary storage and leverage native replication to copy backups to DR

Backup Job Settings BP

Built-In deduplication

  • Keep ON for best performance (except lowest end devices) even if it isn’t going to help you with Per VM backup files
  • Compression
    • Instead of disabling keep Optimal enabled in job and use “decompress before storing- even locally
    • Dedupe-friendly isn’t very friendly any more (new)
      • Will hinder faster recovery in v9
  • Vendor recommendations are sometimes self-serving  to achieve higher dedupe ratios but negatively effect performance

Disk-based Storage Gotchas

  • Gostev loves tape
    • Cheaper
    • Reliable
    • Read-only
    • Customer Success is the biggie
    • Tape is dead
      • Amazon, Google & 50% of Veeam customers disagree
  • Storage-level corruption
    • RAID Controllers are your worst enemies
    • Firmware and software bugs are common, too
    • VT402 Data Corruption tomorrow at 1:30 for more
  • Ransomware  possible

The 2 Part of the 3-2-1 Rule

  • 3 copies, 2 different medias, 1 offsite
  • Completely different storage type!

Storage based replication

  • Betting exclusively on storage-based replication will cost you your job
  • Pros:
    • Fantastic performance
    • Efficient bandwidth utilization
  • Cons:
    • Replicates bad data too
    • Backups remain in a single fault domain

Backup Copy vs. Storage-Based Copy

  • Pros:
    • Breaks the data loop (isolated source and target storage)
    • Implicitly validates all source data during its operation
    • Includes backup files health check
  • Cons:
    • Higher load on backup storage

Make Tape out of drives

  • Low End:
    • Use rotated drives
    • Supported for both primary & backup copy jobs
  • Mid-range:
    • Keep an off-site copy off-prem (cloud)
  • High End:
    • Use hardware-based WORM solutions

Virtualize your Repository (SOBR)

  • simplify backup storage and backup job management
  • Reduce storage hardware spending by allowing disks to be fully utilized
  • Improve backup storage performance and scalability

 

Presenting at VeeamON 2015: Design, Manage and Test Your Data Protection with Veeam Availabilty Suite

Last week I was presented with the honor of being invited to speak at Veeam Software‘s annual user conference, VeeamON. While this was not my first time doing so I was very happy with the end result this year, with 30-40 attendees and positive feedback both from people I knew beforehand as well as new acquaintances who attended.

My session is what I like to think of as the 1-1000 MPH with Veeam, specifically targeting the SMB space but with lots of general guidelines for how to get your DR system up and running fast and as error-free as possible. Some of the things I do with Veeam buck the Best Practices guide but we have been able to maintain high levels of protection over many years without much interruption. The session starts with the basics of designing your DR plan, then designing your Veeam infrastructure components to suit your needs, followed by tips for the actual implementation and other tricks and gotchas I’ve run into over the years.

Anyway due to the amount of information that was covered I promised attendee’s that I would put my slide deck out here for reference so here it is. If anybody has comments, questions or anything in between please feel free to reach out to me either through the comments here or on twitter. For attendees please keep an eye on your e-mail and the #VeeamON hashtag as the videos of all presentations should be made available in the coming weeks.

VMworld 2015: What We Know So Far

As the first general keynote is wrapping up here in San Francisco I’ve been trying to keep track of what’s been announced this morning both in the keynote but also by way of the blogsphere. Long story short my take is there isn’t any thing new for the traditional vSphere customer, but if you are ready to start moving some of those workloads to the cloud there is going to be plenty of new things to enable what VMware is calling the hybrid cloud (repeatedly); the ability to support both your legacy apps (you know what we’re actually using) as well the new, shiny cloud native apps your developers are deploying at the speed of light.

Please forgive the notes based format found below, but I wanted to get the information out there.

Announcements so far:

  • Picture2-1024x475EVO SDDC Manager
    • “Single Pain of Glass for managing all the hardware in your datacenter racks including
      • EVO:Rail for compute, storage
      • Partner networking devices for management, spine and top of rack
      • Rack power distribution
      • Covers vRealize Suite, NSX 6.2, VSAN 6.1, vSphere 6
    • Is this the EVO:Rack they hinted at last year?
    • http://www.vmware.com/radius/vmworld-2015-the-end-of-the-beginning-lets-go/
  • Vmware Integrated OpenStack 2
    • Updates to the Kilo release, enabling features including
      • Expanded language support
      • Multi-region, multi-hypervisor support
      • Load Balancing as a Service
      • Autoscaling
  • vSphere Integrated Containers & Photon Support
    • Enables the truly hybrid cloud, with Photon/Bonneville/ESXi handling life under vCenter and Photon Machine powering your Cloud Native Apps
  • Project SkyScraper; hybrid cloud capabilities for vSphere allowing for extending DC to public cloud while supporting on premises standard concerns like security and business continuity ideas
    • Cross Cloud vMotion & content sync between on-prem and vCloud Air
    • vCloud Air Hybrid Cloud Manager- free download behind fee based capability
  • NSX 6.2 update allowing for deeper integration with the physical devices below it
    • Allows for the microsegmentation of physical servers, big differentiator past when compared to Cisco ACI
    • Will need partners, not known at this point but I’m guessing not Cisco
    • Also now supports cross vCenter vMotion over VXLAN
    • Has a TraceFlow capability allowing visability to what data is passing through
    • Announced late last week that there are now over 700 NSX customers, about double what was announced at Vmworld last year
    • Greater reliability through support for a secondary NSX manager that will take over if the primary fails
    • http://www.crn.com/news/networking/300077934/vmware-gets-physical-with-latest-nsx-software-defined-networking-update.htm
  • VSAN 6.1
    • 3rd total release
    • VSAN Stretched Cluster support, can now have geographically diverse clusters with synchronous replication between sites
    • VSAN for ROBO- Seems interesting, can have large number of 2 node VSAN clusters at your Remote Offices that are then centrally managed through vCenter.
      • Does it make use of stretched cluster for for data protection per site?
    • Now supports native Windows and Oracle clustering methods, WSFC and RAC
    • New high performance hardware supportd in ULLtra DIMM SSDs and NVM interfaces
    • New management features such as a Web Client Health Check plugin for VSAN and a management pack for vROPS
  • SRM 6.1
    • Stretched Cluster as well, seems to be the theme this year
    • Storage Policy Protection Groups; uses tags 1. tag a VM; 2. tag a datastore; protect the datastore with SRM
    • http://www.viktorious.nl/2015/08/31/vmworld-2015-srm-6-1-whats-new-stretched-cluster-support-and-more/
  • Other:
    • vSphere Content Library will be able to sync content between on-prem and vCloud Air bidirectionally
    • Vmware identity services, VMW’s assault on Active Directory

Where In The VMworld Is Jim?

Two weeks from today the official start to VMworld 2015 begins and to say I am excited to head out is a bit of an understatement. VMworld is a great place to learn more about a different branch of virtualization, see the bleeding edge of this technology and meet new people or renew conversations with some of the brightest minds in the industry. I myself will be focusing on VDI, specifically in the case of Disaster Recovery, and looking at Hyper-Converged systems this year along with all the other fun stuff. While this is only my second year going I’ve found that one of the biggest challenges to VMworld is schedule management; if you are interested in the social/community side of things you can very well go 20 hours a day Sunday through Thursday. Even with that you are going to be challenged to hit all the things you want because there is a great deal of overlap. With a paid conference attendance you (or VMUG Advantage membership) you will be able to view the vast majority of sessions online after the fact, but that doesn’t help with the get out and meet people stuff.

Since by nature I am a schedule driven kind of guy I took some time this weekend to kind of map out the whats and the wheres to what I want to do and in hopes of getting to meet up with the 5 people who may read this blog I’ll throw a copy of it out here. The sessions are still in flux because there a few that are full that I still  hope to make it into, especially a couple of the Expert Led HOL Workshops, but I think I’ve got the rest of it worked out. Hope to see you there!

Sessions (So Far:)

8/31 10:00-11:30 Keynote General Keynote
9/1 10:30-12:00 ELW-MBL-1651 Horizon 6 Advanced Workshop
9/1 12:30-1:30 INF-4535 5 Functions of Software Defined Availability
9/1 3:30-4:30 INF-5211 Automating Everything VMware with PowerCLI
9/2 9:30-10:30 STO-6552-GD Meet the VVols Engineering Team with Patrick Dirks
9/2 11:00-12:00 INF-6396-GD Platform Security with Mike Foley
9/2 2:30-3:30 INF-5123 Managing vSphere 6 Deployments & Upgrades, Part 2
9/3 10:30-11:30 EUC-5067 NSX and Horizon Ref. Architecture
9/3 12:00-1:00 INF-4529 VMware Certificate Management for Mere Mortals

vBrownBag TechTalks in the VMworld Hangspace

8/31 1:15 Gina Minks- Why you need to #BackThatSaaSUp
8/31 2:00 Jaison Baily and the vBrisket Community
8/31 3:00 In Tech We Trust Podcast

Socially:

Saturday: Inbound
Sunday: Aug. 30
8-11 AM vBreakfast Mel’s Drive-In
1-4 PM Vmunderground/vBrownBag Opening Acts City View at Metreon
4-7 PM Welcome Reception VMworld Solution Exchange
8-11 PM Vmunderground WUpaaS City View at Metreon
Monday: Aug. 31
5-6 PM SolidFire vExpert VIP Dinner Thirsty Bear Brewing
6-9 PM SolidFire Pursuit of Hoppiness Thirsty Bear Brewing
7-11 PM vBrisket Party Southpaw BBQ
Tuesday: Sept. 1
2:30-4 PM VMworld Bloggers Meetup VMworld Hang Space
4:30-6 PM VMworld Hall Crawl Solutions Exchange
7-10 PM vExpert Party Julia Morgan Ballroom
8 PM-12-AM Veeam Party City View at the Metreon
Wednesday Sept. 2
5-7 PM Red Hat Happy Hour Restaurant LuLu
7:30-10:30 PM VMworld Party AT&T Park
Thursday: Sept. 3
3 PM- ? vFootball Pete’s Tavern