Archive

Uncategorized

In 1997, about a year after launch, Hotmail was growing exponentially, adding thousands of new users every day. We were on fire. And then one night, it all seemed to unravel. We had a program called the “janitor” that ran as an overnight batch process and it erased all of the email that users put in the “trash” folder. Except this night, a bug spawned an army of other janitors that cleaned out everyone’s inboxes, too. That’s right, deep-sixed their email. Here is what went through all our spinning heads: “We’re fucked, it’s over.”

It’s pronounced whiff-eee-o, that horrible, terrifying moment that nearly every entrepreneur goes through when they are certain that their company is dead. I’ve seen it happen in so many different ways: A legal ruling goes against you; Apple refuses to approve your app unless you change the feature that makes it special;  Google launches a competitive product, and aims right at you; A critical technology partner decides not to renew their contract. It’s that moment before you gather yourself to do battle, when all seems lost. I have struggled through that moment, first at Hotmail and again at IronPort. Coming out the other side, scars and all, there are a few critical things you take with you:

The Hotmail WFIO

Once we had pulled the plug on the extra “janitors”, it turned out that about 25% of our total users were affected. Not everyone, but holy shit, a quarter of our customers had lost everything. Understandably, they were pissed. CNET and ZDnet were both on the horn wanting to know what happened. Customer care was inundated with angry calls and (ironically) emails. We figured out how to restore a few thousand customers, but millions were completely unrecoverable. While the calls rolled in, we were trying to figure out how to fix things.

I remember Hotmail’s CEO Sabeer calling us all in a room. “Hey, Rex (the COO), how long will it take to restore the email from the tape backups?” And I’ll never forget his answer: “Um, those got really expensive, so we stopped doing them about a month ago.” Gulp. Long … painful … silence.

The higher-level problem was that people were just getting comfortable trusting us with hosting their email and now we had completely let them down. Their email was just gone. We did a lot of communicating to the users and promised them we would “grandfather” them in for some planned paid services for free. We apologized profusely and explained how the sun, the moon and the stars lined up against us for it to happen. We also clearly explained what steps we were going to take so that it would never happen again. Over time, they started receiving more email, their inboxes filled up, and we just rode it out.

The IronPort WFIO

At IronPort, we developed a super fast and scalable email gateway that ran roughly 10 times faster than any other alternatives. We were definitely in the right place and time when the spam flood came. Our gateway was the only one that could handle the load. Just as at Hotmail, we were adding customers as fast as we could get the company names jotted down, but we needed to add an anti-spam component to our offering.

We struck up a partnership with Brightmail, the leading anti-spam software company, and the joint product – an IronPort gateway with Brightmail – was unbeatable in the marketplace. The only problem was we were heavily dependent on each other with both companies scrambling to build what each other had. Over time we both knew there was going to be a day of reckoning.  We attempted to merge the two companies to solve the problem, but the VCs couldn’t agree on terms.  And then Symantec bought Brightmail.

We knew the clock was ticking and had most of the engineering team working on IronPort anti-spam. But it was way late and wasn’t working. Shortly after the Symantec acquisition, we started hearing reports from channel partners that they were planning to cancel our contract. Although we knew that day would eventually come, we were totally unprepared! WFIO!

As detailed in a prior post, my VP of engineering, Nawaf, “cracked the egg with a sledgehammer” and got our anti-spam product working just in time.  Instead of Symantec canceling the contract, we went on the offensive and faxed a letter to all of our customers cancelling the contract with THEM– a position of strength. We managed through the madness and got to the other side.

After the IronPort WFIO had receded in my rear-view, I realized that you can almost always get to the other side. You just need to keep in mind a few things, and have some emergency tools ready to pull out.

It’s never really as bad as it seems.

Companies are damn resilient. Although it certainly feels like death at the time, it rarely is and companies just keep on moving forward. Ingenuity and guts usually help you find your way out of the jam. In fact, much of a company’s value is usually created by figuring out a solution to the big obstacle. Certainly, that was the case at IronPort where our entire business was built on the back of our anti-spam product.

Get all the brains around the table.

Whenever we went through a WFIO, we’d get all of the smartest people in the room and work through every angle. This is a little counterintuitive because most leaders have the tendency to share very little with the extended team as they are worried about freaking them out. This is a mistake on a number of fronts – trusting your team in crisis brings the company together in amazing ways and their contributions may very well save your company. All of our serious issues resulted in an “Apollo 13” atmosphere where we’d bring together top engineers, architects, VPs – anybody that could materially contribute – and hash it out.  Fighter pilots, who are constantly in pursuit of perfection, have what they call a “rank-less” debrief after every mission where everyone involved, regardless of rank, speaks up to criticize what went wrong.

Lead from the front.

This is the time for leadership – you cannot punk out. I think about a ship captain sailing the Atlantic in the 1700’s and rolling into a huge storm: regardless of how fearful, doubtful, or just scared shitless you may be, there’s only one way to play it with your team – you are in total control. Think of how much it could affect the outcome?! The team needs you to lead them through the problem. Back to the ship captain, what if he grabbed a bottle of whiskey and holed-up under his bed? The ship would certainly be lost. However, if he calmly makes a pot of coffee, ties himself to the bridge, and starts shouting orders, then I believe the chances of the ship making it through go up dramatically. The leader needs to be the first one there, the last one to leave, and be willing to do anything it takes – like answer customer care calls or personally drive a replacement part to an irate customer. Nothing is beneath a leader in times of crisis.

In between Hotmail and Ironport, I started an ecommerce company that got up to about six people. I funded it myself. When I couldn’t get funding for it and I had burned through all of my Hotmail money, I shut it down.  Sometimes it really is over, and I believe that one of the clearest signs is when you are completely out of cash.  But right up until that point, when the music is still playing, when your team is still driving hard, you  have a shot – and usually a decent one.

One of the differences between being a CEO and a venture capitalist is that I obviously meet with many more CEOs now than I did then. As such, it has become more apparent that many of my struggles as a CEO are surprisingly common. One observation that stands out, probably because it is rarely discussed, is how many founder/CEOs have relationship struggles with their significant others and families. For me, the brightest years at IronPort were without a doubt the darkest years at home. While I was focused, motivating, articulate, and decisive at work, I was inconsiderate, preoccupied, self-centered, and lazy at home.

Now, having worked through that time with my family, I’m in a much better place to reflect on what happened, how I could have handled things differently, and offer some advice to other founders who may be caught up in a similar dynamic.

As a first time founder/CEO, I really had no idea what I was doing. Sure, I had gone to business school, worked at plenty of large companies and even other successful startups, but nothing prepared me for the incredible stress and overwhelming life focus of actually running a startup.

I did my best to move up the learning curve: I surrounded myself with great mentors, board members, coaches, and, most importantly, the challenging, wicked smart executive team members that worked with me everyday. We definitely made lots of mistakes, but we did many things right and IronPort grew to be a very large and successful company over the seven years before we ultimately sold to Cisco in 2007. All that said, I believe I could have been a much more effective leader if I had leaned in at home. As my relationship with my family deteriorated, so did my concentration at work as I was constantly trying to manage it in fits and starts. Here are some details of my personal struggle:

Part of the magic of a startup is the fear of death. You have only so much money in the bank, and if you don’t get to the right milestone before you run out, then you’re dead—company goes under, it’s over. There’s a way to cheat death when you are not going to make it—you sound the alarm and force everyone to code through the night and/or weekend. This is stereotypically the life one signs up for at an early stage tech startup. Get in early, kill yourself with a team making something great, and get a meaningful product out before you run out of money. And hopefully, make it up to that hardworking team with stock options later.

I didn’t code, but as the CEO, I felt it necessary to be there physically with the engineering team. I would sit through architecture discussions, product reviews, and wireframe layouts. Sometimes, I would just get everyone lunch or dinner. When we started pulling consistent coding weekends, we brought in the entire management team to serve the engineers: We brought them food, washed their cars, got oil changes, took in their dry cleaning, and arranged for childcare for their kids in the office. Lead by example, lead from the front, was the CEO approach I convinced myself was necessary.

Now contrast this with my home life.

One of the stated values at IronPort was “work/life balance,” but I clearly wasn’t living it. I was rarely home. And when I was home, well, let’s just say I wasn’t particularly helpful or cheery. My perspective at the time was: I’m killing myself at work, so when I get home, I just want to kick back with a cocktail and watch some TV. All I do is talk to people all day long and so at home, I’d really prefer not to talk much, just relax.

This posture was, of course, completely opposite to how my wife felt. After having left her VP role in a successful startup, she was now home speaking in monosyllabic words to kids all day and was starving for adult conversation when I got in the door. And that part about sitting on my ass in front of the TV with a cocktail? This ran counter to all of her efforts to teach the kids about pitching in as a family. The message of everyone helping to cook, clean, and be responsible for the household fell completely flat when daddy wouldn’t so much as take out the trash or change a light bulb. Nope, I was far too important for that and suggested she should hire someone to keep the house clean or even cook, if that was “stressing her out”.

Ugh. I was completely missing the point and talking past her… I was setting such a great example at work, but such a terrible one at home where I often acted like a self-important asshole.

As IronPort grew, I was constantly on the road with customers, press, analysts, and of course, recruiting and energizing employees. We ultimately did over 60% of our revenue outside of the U.S., and we all felt it very important to support all of our disparate offices from Europe to Asia to South America. There were times in a given month when I was gone 50-75% of the days. Even when I was home, I was usually in this brutal state of sleep deprivation and recovery from adjusting to yet another time zone. While I was gone, 100% of the daily burden fell on my spouse, usually resulting in a solid week of arguments upon my return. I started referring to the week after a long trip as “re-entry”, like John Glenn’s Friendship Seven fireball.

After years of working full-time with our first child, and part-time after our second, my Harvard MBA wife, who had had an amazing career in her own right, “decided” to become a full-time mom and take care of our children shortly after our third was born. I say “decided” because at the time, it was clear to both of us that I wasn’t willingly scrubbing in as a 50/50 partner at home. She endured the rocky years while I was running IronPort, but insisted that when it was over, we were going to re-evaluate and recalibrate.

I took about 18 months off in between IronPort and joining Andreessen Horowitz. During that time, I was packing lunches, driving carpools, and making dinners, and began doing my real part in the family. With the help of my wife and other role-model dads, I essentially got re-programmed and it has continued to work for us even though I’m working full-time again. Now one might say that being a partner at a VC firm, even a hard working one, isn’t the same as being a founder/CEO of a startup… I’ll admit that’s true. However, now that I’m on the other side, I believe that I could have coached my former CEO self to success as well. Here are the most critical things I needed to change:

Disconnect to Connect. Although it’s easy for me to see it now, at the time I clearly thought what I was doing at work was far more important and urgent than what was going on at home. It sounds weird now, but this required a real mindset change for me. My wife dropped a bunch of hints (e.g. “How did I suddenly land in a 1950’s relationship?!”), but I was undeterred in the thick of it. The shock of almost losing the relationship made me pay more attention, but I was only going through the motions with my mind still firmly attached to the business. I believe the change in attitude came from truly connecting and tuning in at home. This required disconnecting from work (e.g. turning off the computer and phone), and completely focusing all of my attention on the details of the home. Cooking a great meal. Helping with a science project. Discussing the future with my partner. I was often rightly accused of being physically present without being mentally present. If you find yourself sneaking into the bathroom to complete emails, then you’re certainly not in the moment… Getting some time physically out of the Silicon Valley pressure cooker was also helpful in changing my perspective.

Participate. It’s just not possible to be a real partner if you aren’t materially participating. I believe even the busiest CEOs must drive a carpool, pack a lunch, help with homework, make a breakfast or dinner, and consistently attend school events. Being involved every week is the only way to stay connected at home, and it cannot be outsourced. No matter how exhausted I am from traveling, I push myself to “not be lazy” at home—it’s just too important. When you are involved, there is a natural cadence to planning the week together and communication improves dramatically.

Communicate. Multiple, daily phone and text check-ins are the norm now, but not then. When I was traveling at IronPort, I would sometimes go for days without communicating at all. Now that I am completely tuned in to the weekly family schedule, we plan and calendar family meals (perhaps the single most important thing we do), pickups and drop-offs, and make adjustments on the fly. E.g. Did some time suddenly free up so I can complete an errand? Can I pick something up on the way home? Etc. My norm is to check in between meetings, but if I’m the “parent on duty”—i.e., if my wife is out of town—then I will start a meeting with, “You’ll have to excuse me, but I’m the parent in town so I need to keep my phone handy in case of an issue.” Communication was by far my biggest area for improvement.

Planning and Priorities. My wife and I have a weekly date night. My son and I are in a fantasy football league together. I cook with my daughters. Most times these have become immovable appointments on my calendar. There is a phrase—“truth in calendaring”—if something is important, then you must carve out time in your life to do it. When my calendar reflects that I can’t do a meeting on Wednesday and Friday mornings before 9am, because I cook breakfast and drive a carpool, then it’s amazing how meetings just don’t get scheduled. If at all possible, living physically close to the office is also a huge help to juggling the priorities. It means that I can cut out for a family dinner and then go back to the office or have a late meeting afterwards.

In retrospect, I believe that I could convince the hardest working CEOs that having some real life balance by investing in your important relationships will make you a better CEO. When you are out of balance, it affects your stress, judgment, and ultimately becomes another destabilizer just when you need to be the most put together. I also believe this change is actually a much better example of leadership than the one I was exuding. When a leader shows the way toward getting things done and balancing their life, it sets a much better example for everyone else in the company who struggle with it too.

As a firm, we strive to be unusually transparent in our thinking, where through our blogs, we all go into great detail on our reasoning for each investment. In addition, we are diligent in getting back to entrepreneurs after every meeting with us to give them a detailed explanation of our internal deliberations and outcome. It is through these constant public and private discussions that we want the founder community to better understand our evolving thinking and investment criteria.

In the vein of transparency, I wanted to clarify some of my recent comments. Late last week while in NYC, I stopped by the Wall Street Journal for an informal conversation to discuss what I was seeing that was new and interesting in venture capital.  The resulting article has been the subject of much debate and, in my view, over-interpretation.  As a result, I would like to offer up a more detailed explanation and context of what I said and meant…

One of the many things that I mentioned was how, as a partnership, we’ve had some internal discussions about the relative differences between the consumer series A and the enterprise series A. I was trying to make the point that by and large, we were raising the bar for most consumer series A rounds such that we would be looking for more traction and/or evidence that the entrepreneur’s idea were actually working as opposed to the enterprise A. This is not an absolute, just an observation based on our recent deliberations. Let me explain further.

Many of the seed and series A consumer companies, especially mobile applications, that I have seen recently are still in the throes of proving out product/market fit for their idea.  The entrepreneur is usually in the middle of A/B testing to try to get one or more important end user statistics working such as downloads, daily active users (DAU), monthly active users (MAU), and a compelling cohort analysis of usage over time.  This messy, but necessary, experimentation process where theories are rapidly tested and retested was the stage that I referred to as “Fruit fly experiments”.  Although it was not my intention, I see how this analogy could be offensive to entrepreneurs that are in the thick of this problem – I don’t mean to  make light of their struggle.  Having been in the thick of it myself multiple times, I have a deep appreciation for how hard and emotionally draining the product/market fit process is and apologize for the careless analogy.

Coming back to the central point, I was not speaking categorically about a policy change within Andreessen Horowitz – we do and will continue to invest in consumer and enterprise series As – full stop.  I was attempting to make a more nuanced point about the difference in criteria: we will often do an enterprise series A when it is at the idea stage or an unproven prototype. The reasoning is that the enterprise opportunities and landscape usually includes a unique technology advantage or business insight borne out of the entrepreneur’s deep understanding of the subject area.  This usually stands in contrast to many of the consumer ideas that are primarily business model innovations that are very interesting but still need proving out.

For consumer businesses that are in this ideation stage, we are more likely to become active seed investors or we will typically wait until it is clear that the business has caught “lightning in a bottle” which usually takes place at the B round. Now there are many exceptions: Anki, for instance, came in at the idea/prototype phase and we jumped all over doing the Series A – why? The entrepreneurs had a deep background in robotics from Carnegie Mellon and were taking that expertise into an entirely new category, toys.  The founders had a technological secret and an unusual background for prosecuting the opportunity.

Part of why I love my job is because we get to hear from entrepreneurs everyday about the problems they are trying to solve – both on the consumer front and in the enterprise. As we continue to evolve the way we think about our investments, we are committed to keeping an open dialogue with entrepreneurs at any stage and focus.