Friday, December 26, 2014

A two-speed IT architecture for the digital enterprise

It's tricky building a digital enterprise. Aivars Lode avantce

A two-speed IT architecture for the digital enterprise

Delivering an enriched customer experience requires a new digital architecture running alongside legacy systems.

December 2014 | byOliver Bossert, Chris Ip, and J├╝rgen Laartz

Digital business models have become essential for companies across a range of industries. With social networks and e-commerce websites setting new benchmarks for speed, agility, and user-friendliness, consumers expect similar online performance from banks, retailers, and telecommunications companies. Attackers born in the digital age give consumers what they want, but many older companies struggle to meet customer expectations. For them, going digital is now a prerequisite for surviving and thriving. Success requires strong capabilities in four areas.
First, because the digital business model allows the creation—and shorter time to market—of digital products and services, companies need to become skilled at digital-product innovation that meets changing customer expectations. One such new offering for consumers is car-insurance policies enabled by geolocation-tracking technology, where the price of the policy depends on how much and how aggressively a person actually drives.
Second, companies need to provide a seamless multichannel (digital and physical) experience so consumers can move effortlessly from one channel to another. For example, many shoppers use smartphones to reserve a product online and pick it up in a store.
Third, companies should use big data and advanced analytics to better understand customer behavior. For example, gaining insight into customers’ buying habits—with their consent, of course—can lead to an improved customer experience and increased sales through more effective cross-selling.
Fourth, companies need to improve their capabilities in automating operations and digitizing business processes. This is important because it enables quicker response times to customers while cutting operating waste and costs.
A two-speed IT architecture will help companies develop their customer-facing capabilities at high speed while decoupling legacy systems for which release cycles of new functionality stay at a slower pace.

Implications for enterprise architecture

Each of the four levers poses a substantial challenge for IT. For example, many banking-product lines—among them credit cards, investments, and checking and savings accounts—are managed in silos. This makes it difficult to get a comprehensive view of customers quickly, for example, to assess their loan applications. What’s more, channels are often managed and tracked independently, complicating matters for customers who wish to use multiple channels as they pursue a transaction. For instance, customers starting a loan application on their smartphones may find that they have to reenter data when they switch to desktop computers to fill in the more detailed information required. Weak systems integration and slow database-access times can prevent customers from enjoying a real-time shopping and purchasing experience. Analytics capabilities are especially difficult to integrate with operational process flows. Manual steps in these processes, such as rekeying and transferring information, present major obstacles to both analytics and automation of processes.
While a few players have overcome some of these hurdles, it is a big challenge for many IT executives to implement all four levers so customers can, for instance, purchase individually tailored products across multiple channels. One important reason is that the legacy IT architecture and organization, for example, which runs the supply-chain and operations systems responsible for executing online product orders, lacks the speed and flexibility needed in the digital marketplace.
Indeed, the ability to offer new products on a timely basis has become an important competitive factor; this might require weekly software releases for an e-commerce platform. That kind of speed can only be achieved with an inherently error-prone software-development approach of testing, failing, learning, adapting, and iterating rapidly. It’s hard to imagine that experimental approach applied to legacy systems. Nor would it be appropriate, because the demand for perfection is far higher in key back-end legacy systems. Quality, measured by the number of IT system errors, and resilience, measured by the availability and stability of IT infrastructure services, comes at slow speed but is critical for risk- and regulatory-compliance management and for core transactional activities such as finance and online sales. In contrast, lower IT-system quality and resilience can be acceptable in customer-facing areas, for instance, when users participate in the testing of new software. For these reasons, many companies need an IT architecture that can operate at different speeds.

The building blocks of digital-enterprise architecture

In our experience, digital-enterprise architecture needs to accommodate the following elements to deliver the functionality that the digital enterprise requires.
Two-speed architecture. This implies a fast-speed, customer-centric front end running alongside a slow-speed, transaction-focused legacy back end. For software-release cycles and deployment mechanisms, the customer-facing part should be modular, to enable quick deployment of new software by avoiding time-consuming integration work. In contrast, the transactional core systems of record must be designed for stability and high-quality data management, which leads to longer release cycles.
Instant cross-channel deployment of functionality. New microservices defining only a small amount of functionality, such as lookup of the next product a consumer would most likely purchase, should be deployable in an hour rather than in several weeks. Such microservices should also be available across all channels. Ideally, it should be possible to develop these services in multiple programming languages rather than being locked into a single development framework.
Zero downtime. In digital global operations, days-long maintenance windows are no longer an option. Upgrades of systems affecting the consumer’s experience should be seamless, using a concept that allows the deployment of a new software or service in parallel with the old version. First, only about 1 percent of the user traffic is routed to the new version. Only when the new version fulfills a set of key performance indicators will all traffic be routed to the new version. Moreover, in daily operations, there should be fallback mechanisms in place so that issues arising in one service do not harm overall operations more than necessary. If, for instance, a retailer’s personalized recommendation service is unavailable, a random recommendation in a relevant category would be displayed rather than an annoying web error page.
Real-time data analytics. Customers generate data with every move they make within an app. The ability to analyze that information in real time can make analytics an integral part of operational processes and not just a stand-alone capability. For example, one retailer analyzes customers’ purchases automatically when they pay with their credit cards; along with the receipt, the business provides a savings coupon for a product the customer will probably be interested in buying the next time he or she shops at the store.
Easy process configuration. Business users themselves should be able to change automated processes. This would allow them, for example, to eliminate unnecessary process steps without requiring time-consuming coding by an IT developer.
Product factory. Industries that provide digital products, such as banking and telecommunications, need to decouple the products from the processes. A bank, for example, would implement one sales process and reuse it for all products, such as accounts and cards.
Automated scaling of IT platforms. In a digital business, workloads expand and become harder to predict. Ideally, this load would be balanced across private- and public-cloud environments, with mechanisms in place to ensure that when one provider has an outage, others can take over the workload.
Secure architecture. In a digital business model, cybersecurity must be an integral part of the overall application. Not only does the company have more valuable data to protect, making it more attractive to hackers, but the digital strategy also opens new interfaces to customers, suppliers, and partners—interfaces that can be exploited by hackers.

Moving toward two-speed architecture

Unlike enterprises that are born digital, traditional companies don’t have the luxury of starting with a clean slate; they must build an architecture designed for the digital enterprise on a legacy foundation. What’s more, while most companies would have been comfortable in the past going through a three- to five-year transformation and not implementing new features in the meantime, today’s highly competitive markets no longer allow players to alter architecture and business models sequentially. It is therefore important to realize that the transformation toward digital is a continuous process of delivering new functionality (see sidebar, “A retailer begins the two-speed journey”). Successful digital transformations focus on the following aspects.
Manage a hybrid target architecture with very different platforms. Digital target architectures are heterogeneous, with transactional platforms managed for scalability and resilience coexisting alongside other systems optimized for customer experience. The transformation can be sustained only if a high-level target architecture and standards in critical areas such as cybersecurity are clearly described from the beginning. Without them, the transformation can be slowed down by the complexity of legacy and new hardware and application provisioning.
Plan for ongoing software delivery with blends of methodologies. There isn’t time to develop software by using a waterfall model and then separating the transformation into several long phases, as in traditional multi-year IT transformations. Nor is the solution to migrate all delivery to agile methodologies. The answer is to do both but blend the benefits of agile (iterative development, continuous delivery) into the waterfall model. Now, the software solution for each business challenge has to be constantly developed, tested, and implemented in an integrated fashion. This requires clear segregation of platforms into domains managed for fast iterative delivery (for example, for customer-experience applications) or for transactional integrity (for back-end transactional systems).
Develop the low-speed architecture, too. It’s important to establish a clear distinction between the two IT models from the beginning and not only focus on the fast-speed part but also develop the transactional back-end architecture. Those systems of record require rigorous development and testing methodologies and must be managed for resilience and scalability, with no compromises.
Build a new organization and governance model in parallel with the new technology. In the digital enterprise, business and IT work together in a new and integrated way, where boundaries between the two start to blur. This partnership has to be established during the transformation.1
Change mind-sets. By transforming the architecture, technology can become a key factor for a company’s competitiveness. Such a development requires increased management attention and usually a place on the board agenda. While IT efficiency clearly remains important, spending levels may well rise as companies transform IT from largely being a necessary expense to being a true business enabler. As such, expenses are managed as investments rather than just costs; this will often require a substantial mind-set shift for the organization.
Run waves of change in three parallel streams. In a two-speed transformation, it makes sense to have an implementation plan that runs in three parallel streams. The digital-transformation stream builds new functionality for the business, supported by the results of a short-term optimization stream that develops solutions that might not always be compliant with the target architecture (for example, using noncompliant interfaces). To ease the development of short-term measures and create a sustainable IT infrastructure, an architecture-transformation stream is the third necessary component.

WeWork: Now a $5 Billion Co-Working Startup

Collaboration is the key to a huge valuation. Aivars Lode avantce

WeWork: Now a $5 Billion Co-Working Startup

Rich Price for Four-Year-Old Office-Sharing Company With Big Ambitions

WeWork customers in the lobby of its Fulton Street offices in New York. The startup’s backers liken it to sharing-economy firms rather than co-working-space competitors like Regis and Rocketspace. 

By Lindsay Gellman and Eliot Brown
Updated Dec. 15, 2014 7:41 p.m. ET

WeWork Companies Inc., a provider of shared office space, believes it can be as transformational to its industry as upstarts like Airbnb Inc. and Uber Technologies Inc. are in travel and transportation.
The four-year-old company, which divvies up rented office space and sublets mostly to startups, said on Monday it closed a $355 million funding round. The deal values the company at about $5 billion, said people close to the matter.
The valuation puts the small New York-based company in the same league as social bookmarking site operator Pinterest Inc. and media and Internet companyIAC/InterActiveCorp .
Adam Neumann, WeWork’s 35-year-old co-founder, hopes to make the company a hothouse for new business formation—by bringing together entrepreneurs who share space, office services and, potentially, ideas.
Thirty-five-year-old Adam Nuemann co-founded WeWork, which is now valued at about $5 billion. Cassandra Giraldo for The Wall Street Journal
“We happen to need buildings just like Uber happens to need cars, just like Airbnb happens to need apartments,” Mr. Neumann said in an interview.
The latest financing was co-led by funds and accounts managed by T. Rowe Price Associates Inc., clients of Wellington Management, and Goldman Sachs Group, according to WeWork. Investors from prior rounds including J.P. Morgan Chase & Co., Harvard Management Co. and Benchmark also participated, according to WeWork.
“If I showed you their cash-flow statement, you would not compare it to a real-estate company,” said Henry Ellenbogen, a portfolio manager at T. Rowe Price. “You’d compare it to a brand or tech company—maybe Chipotle or Uber.”
Unlike its closest competitors, WeWork has a new-era sheen, its own mobile app, and deliberately chose to place properties in hot areas like Washington, D.C.’s Shaw neighborhood.
Viewed as a traditional real estate venture, WeWork’s valuation wouldn’t be nearly as rich, Mr. Neumann acknowledges.
Still, WeWork’s investors are “really not looking at the real estate,” he said. They see it as a force for upending small-business office rentals with a new model providing sleek furnishings and plenty of collaboration.
WeWork members socialize in the lobby of the office-sharing company’s New York headquarters this week. Cassandra Giraldo for The Wall Street Journal
Mr. Neumann said the company’s app, which serves as an internal directory and allows WeWork customers to communicate idea and applicants, also provides information on events like regular happy hours—touchstones of the work community it aims to foster.
WeWork said its December revenue puts it on an about $150 million annual revenue run rate. It also said the month’s annualized operating income puts its valuation at roughly 100 times income. It expects to grow significantly in years ahead, which would lower that ratio. Landlords typically trade between 18 times and 20 times earnings, according to Jed Reagan, an analyst at real estate consultants Green Street Advisors.
Regus PLC, the shared office space company that offers more traditional office space than WeWork, has a market capitalization of about $2.8 billion and 2013 operating profit of about $150 million. Regus went public in 2000, and its market value today is about half its dotcom-era peak at more than 360 pence a share.
WeWork said it has leased about 1.6 million square feet in New York, making it the fastest expanding company by footprint in the city since 2010. In all, it expects to have about 3.5 million square-feet globally by the end of 2015, total space larger than the Empire State Building. Also in the works is living space, akin to a high-end dorm for 20-something workers, people briefed on the company’s plans said.
Dozens of firms offer space to those willing to pay a premium for a desk or office nestled among startups and some mature companies.
Price depends on location. In the company’s Financial District headquarters, it charges $400 a month for a desk and $1,400 a month for a small two-person office, well above the area’s rate for such space.
News Corp , which owns The Wall Street Journal, is a customer.
Israeli-born Mr. Neumann looks more like one of his startup tenants than a traditional landlord. He has shoulder-length wavy hair, shuns a tie and keeps the top two buttons of his shirt undone. His office, in a WeWork hub in New York City’s financial district, stocks an array of liquor, and he sometimes implores guests to do tequila shots, visitors to his office say.
Unlike Airbnb and Uber, WeWork has a large fixed expense—rent paid to building owners—and the company may be as vulnerable in a downturn as any other co-working firm, said skeptics.
The business is a risky one in which its costs, fees paid to landlords, are fixed, but its revenues from startups and established businesses can fall quickly when the economy slows.
“The small-to-medium-size businesses, they get particularly impacted in a recession,” said Jon Halpern, who ran shared office space firm HQ Global Workplace in the early 2000s.
HQ grew rapidly in the run-up to the dot-com bust. But in the recession that followed, its shares plummeted. It ultimately sold itself to Regus.
As for his company’s next steps, Mr. Neumann said he is focused on growing the business. He plans to expand WeWork from its current 23 locations to 60 in the next year.
Landlords who have discussed the matter with WeWork executives say they have said it is planning an initial public offering sometime in the next two or three years.

H-P Moves to Retain Corporate Customers Ahead of Breakup

HP thinks it can retain it's enterprise with high margin customers. Aivars Lode avantce

H-P Moves to Retain Corporate Customers Ahead of Breakup

Company to Offer Versions of Integrity Server Lines that Use Intel’s Xeon Chips

Superdome and NonStop servers are still used by banks, telecommunications carriers and other companies particularly concerned with reliability. Getty Images
By Don Clark
Dec. 2, 2014 12:04 a.m. ET

Hewlett-Packard Co. , as it prepares to split in two, is unveiling on Tuesday a plan to help retain important customers by allowing them to leave behind a processor technology that has found few takers besides H-P.
The Palo Alto, Calif., company said it would offer versions of two computer server lines under H-P’s Integrity moniker—Superdome and NonStop—that will be powered by Intel Corp. ’s Xeon chips, which are widely used in other servers from H-P and other vendors. Its Integrity machines now use Intel’s Itanium chips, a specialized strain of technology that sprang from a joint venture between the companies two decades ago.
Revenue from these “business-critical” servers, as H-P calls them, declined 29% in the quarter ended in October over a year earlier. But Superdome and NonStop servers are still used by banks, telecommunications carriers and other companies particularly concerned with reliability.
Such systems accounted for only $929 million in revenue in the fiscal year ended October 31, dwarfed by the $12.5 billion generated from more popular x86 servers, but keeping good relations with customers that use them has other benefits. Such companies buy software, services and other hardware from H-P that hinges on the applications running on the Superdome and NonStop machines, said Patrick Moorhead, an analyst with Moor Insights & Strategy.
“It’s about keeping some very high-margin customers,” he said.
Holding on to these customers will be an important factor in H-P’s plans to split the company in half, with one entity serving enterprises and another handling personal computers and printers.

“These are a very conservative type of customers—they don’t want to take any risks,” said Antonio Neri, senior vice president and general manager of H-P’s enterprise group. “But eventually they have to move to a new architecture.”
He stressed H-P doesn’t plan to stop developing Itanium-based systems but said the benefits of moving to Intel’s mainstream Xeon technology are significant when combined with other enhancements H-P is offering.
Intel, which introduced its last Itanium model in late 2012, has disclosed plans for a successor, which is code-named Kittson. The chip maker hasn’t said when that product will arrive nor described models it may develop after that. Intel, based in Santa Clara, Calif., gets the bulk of its revenue from chips based on the x86 technology that evolved from personal computers. It announced plans with H-P to diverge from that approach in 1994, when it faced competition from a new breed of chips designed by companies likeInternational Business Machines Corp. and Sun Microsystems Inc.
Itanium chips finally went on sale in 2001 but failed to attract many customers besides H-P. Meanwhile, Intel kept rapidly improving its x86 Xeon chips.
H-P’s Itanium-based Superdome line, which runs the Unix operating system, is used by businesses for a variety of heavy-duty computing chores. The Nonstop line handles more sensitive jobs like ATM networks and stock exchanges.H-P inherited these crash-resistant machines from former operations of Tandem Computers Inc.

Where most servers these days include one to four processor chips, the two H-P systems compete with machines from IBM and Oracle Corp. that have many more. The new Superdome model to be announced Tuesday, for example, has sockets to plug in 16 Xeon chips and offers nine times the performance of a conventional H-P system with eight Xeon chips, the company said. H-P has developed accessory chips and software to speed up communications between chips and improve reliability.
For the Superdome system, H-P is encouraging customers to move to the Linux operating system or other software, Mr. Neri said. H-P is porting NonStop software to run on Xeon chips. The company is offering services to help customers migrate to the new technology in both cases.
H-P is announcing the new Xeon-based offerings at an event in Barcelona that includes software and data storage upgrades. H-P paid hundreds of millions of dollars to Intel to keep working on Itanium, according to documents disclosed during the litigation. A state court judge in California ruled in 2012 that Oracle was contractually obligated to keep supporting Itanium, a ruling that Oracle is appealing.
One factor pushing H-P away from Itanium has been reduced support for the technology by software companies. Oracle, for example, in March 2011 said it would stop developing new versions of its popular database software for Itanium-based computers, a decision that prompted H-P to sue the software maker.

Revenue drops at IT giant CSC... 'Good progress' says chief

Another legacy providers revenue shrinks, is that a trend? Aivars Lode avantce

Revenue drops at IT giant CSC... 'Good progress' says chief
By | Kat Hall 7th November 2014 16:31

Don't worry, 'transformation' efforts are underway
Revenue at CSC fell by 3.4 per cent to $3.08bn for the firm's second quarter as "transformation efforts" took hold at the IT giant.
Meanwhile, operating income fell 3.3 per cent compared to the same period last year – to $340m. Half-year net income fell by $112m to $307m, compared with the first six months of 2013.

Sales were down across CSC's three divisions for the quarter, but the firm's Global Infrastructure Services segment took the biggest hit - with revenue dropping 6.9 per cent to $1.036bn.
Growth in next-generation offerings of cloud and cyber helped to partially offset the impact of price-downs in the division, restructuring and contract conclusions, said the firm in a statement.
Revenue for CSC's Global Businesses Services segment fell 1.9 per cent to 1bn, while its North American Public Sector division fell 1 per cent to 1.04bn.
Chief executive Mike Lawrie said the results reflected the firm's "transformation efforts" and demonstrate "good progress" for the quarter.
He said: “In the commercial business, we delivered solid improvements in profitability and are showing good revenue growth in our next-generation offerings, which is helping to offset headwinds in the infrastructure business. Our public sector business has seen a normalization of revenue and continues to deliver strong margins, which is helping to drive overall earnings growth.”
CSC has approximately 74,000 employees and reported revenue of $12.9bn for the 12 months ended 3 October, 2014. ®

Software Firms Scramble to Jump Into Containers

More open source software that is disrupting the incumbents. Aivars Lode avantce

Software Firms Scramble to Jump Into Containers

Docker, a Leader in Container Technology, Is Hot Among Programmers

By Shira Ovide
Nov. 4, 2014 8:08 p.m. ET
A tiny company with little revenue to speak of has pushed the mighty Microsoft Corp. to up its game in computer operating software.
The maker of Windows, the software that undergirds most personal computers as well as corporate data centers, announced a partnership last month with a relatively unknown venture called Docker. The San Francisco company is a leader in container technology, which is suddenly all the rage among programmers. Containers run on the Linux operating system, and Microsoft has had to add them to its corporate-focused Windows Server product or risk irrelevance.
Mike Schutz, a general manager working on Microsoft’s cloud-software services, said the company has worked with container technologies for years and moved to incorporate the technology into Windows computer servers recently in response to interest from customers and software developers. “For us, it’s all about the customers,” Mr. Schutz said.
The fear also extends beyond Redmond: Containers challenge everyone, from upstarts like VMware Inc. and Inc. to incumbents including Dell Inc. and Hewlett-Packard Co.
What makes containers so compelling? The technology encloses a program (or a piece of one) in a layer of software that connects seamlessly to the operating system and other computing resources it depends on to run. Putting a program in a container has a number of benefits, but a crucial advantage is that it can be moved quickly and easily from one computer to another—say, from a programmer’s laptop to a test system to the cloud. Given the pace of Internet time, harried chief technology officers are desperate for anything that speeds up the process.
“Think about a cake,” said Scott Johnston, senior vice president of product at Docker, likening the cake part to a server and the icing to a program. “You want to be able to change frosting from chocolate to vanilla. If there’s paper between the two, you can lift up the frosting and replace it.”
Docker kicked off the container boom 18 months ago, when it released its technology (also called Docker) under a free-of-charge open-source license. The software sparked the kind of rapid uptake generally reserved for consumer gewgaws like FarmVille, clocking 43 million downloads as of early October. Users include Google Inc.,International Business Machines Corp. , Spotify, Yelp Inc. —and, yes, Microsoft—as well as nontech companies like the BBC and a handful of big banks, according to people familiar with the financial institutions’ operations.
“The interest level is off the charts,” says Dave Bartoletti, an analyst with technology research firm Forrester Research Inc.
Although containers are redefining the Internet’s state of the art, they’re not entirely new. The idea has been around since the early 2000s, and it has been part of the Linux operating system since 2007. Companies including Google and Oracle developed their own container systems, but they were difficult to use and failed to gain a wide following. The technology remained relatively obscure until Docker finally made using it easy. The company standardized the containers and organized a comprehensive set of software commands to manage them. Software developers, eager to take advantage of the power behind kingpins like Google, started adopting the technology in droves.
Beyond revving up the pace of online development, containers are an emerging alternative to the software construct known as a virtual machine, or VM, that is a foundation of today’s Internet. Containers aren’t only faster and easier to use than VMs; they also make far more efficient use of computing hardware. Engineers say they can run between two and six times as many containerized programs as VMs on the same hardware, dramatically reducing infrastructure costs.
The potential of container technology to unseat VMs threatens the market for software sold by Citrix Systems Inc. and VMware, as well as tech giants such as Dell, Hewlett-Packard, IBM and Microsoft. The market for virtualization software is estimated by the research firm IDC to be valued at $4 billion in 2013 and growing at a compound annual growth rate of 13%.
VMware has warned about container technology’s limits but also moved to embrace it as a complement to its software. The company announced in the summer that it is working with Docker and said the best way to use containers is with VMware’s virtualization software.
A spokeswoman for Citrix didn’t respond to a request for comment.
Google doesn’t use VMs at all internally, and Docker makes that approach attractive for less engineering-heavy companies. Pantheon, a Web host that is a heavy user of containers rather than VMs, employs two full-time employees to run more than 70,000 websites for organizations including the Boston Herald and AAA. Fifty technicians would be required to do the same work using VMs, according to Pantheon Chief Executive Zack Rosen.
“We would not be able to do what we do without containers,” Mr. Rosen said.
The shift toward containers also ratchets up competition between, which rents computing capacity through its Amazon Web Services division, and Google, whose Google Compute Engine offers similar services and has gone all in for containers. On Tuesday, the search giant announced a product that lets software developers run and manage Docker containers on Google’s servers. The aim is to make it easier for developers to deploy containerized applications on Google Compute Engine than on Amazon Web Services. Even if AWS customers run containers on Amazon’s servers, they’ll run more efficiently, reducing the amount of resources the users need to rent and thus curtailing Amazon’s revenue.
“AWS customers are increasingly taking advantage of the ease and simplicity of launching and managing Docker containers with Amazon,” a spokeswoman said.
Still, containers have a long way to go before they can be called mainstream. Many large companies are experimenting with them but not yet using them widely, according to executives at both corporations and tech companies. Some observers say the buzz surrounding containers outstrips their utility.
As for Docker itself, the company has positioned itself above the fray, guiding development of its free software for the common good. That means its business remains a work in progress. The company holds $55 million in funding from marquee investors including Benchmark, Greylock Partners, Sequoia Capital, Trinity Ventures and Yahoo Inc. co-founder Jerry Yang .
Questions loom as to how it will generate revenue. One possibility is by offering a software development platform as a service, technical support, and eventually commercial versions of its software. As a technology, though, it’s already a Silicon Valley fixture and promises to drive the next generation of software-based businesses. “This is the first inning of a new wave of applications that will move from development to deployment in one fluid motion,” Docker’s Mr. Johnston says. “We’re humbled that the market has given us the opportunity.”

How A 'Deviant' Philosopher Built Palantir, A CIA-Funded Data-Mining Juggernaut

How A 'Deviant' Philosopher Built Palantir, A CIA-Funded Data-Mining Juggernaut 

By Andy Greenberg and Ryan Mac
August 14, 2013
Since rumors began to spread that a startup called Palantir helped to kill Osama bin Laden, Alex Karp hasn’t had much time to himself.
On one sun-baked July morning in Silicon Valley Palantir’s lean 45-year-old chief executive, with a top-heavy mop of frazzled hair, hikes the grassy hills around Stanford University’s massive satellite antennae known as the Dish, a favorite meditative pastime. But his solitude is disturbed somewhat by “Mike,” an ex-Marine–silent, 6 foot 1, 270 pounds of mostly pectoral muscle–who trails him everywhere he goes. Even on the suburban streets of Palo Alto, steps from Palantir’s headquarters, the bodyguard lingers a few feet behind.
“It puts a massive cramp on your life,” Karp complains, his expression hidden behind large black sunglasses. “There’s nothing worse for reducing your ability to flirt with someone.”
Karp’s 24/7 security detail is meant to protect him from extremists who have sent him death threats and conspiracy theorists who have called Palantir to rant about the Illuminati. Schizophrenics have stalked Karp outside his office for days at a stretch. “It’s easy to be the focal point of fantasies,” he says, “if your company is involved in realities like ours.”
Palantir lives the realities of its customers: the NSA, the FBI and the CIA–an early investor through its In-Q-Tel venture fund–along with an alphabet soup of other U.S. counterterrorism and military agencies. In the last five years Palantir has become the go-to company for mining massive data sets for intelligence and law enforcement applications, with a slick software interface and coders who parachute into clients’ headquarters to customize its programs. Palantir turns messy swamps of information into intuitively visualized maps, histograms and link charts. Give its so-called “forward-deployed engineers” a few days to crawl, tag and integrate every scrap of a customer’s data, and Palantir can elucidate problems as disparate as terrorism, disaster response and human trafficking.
Palantir’s advisors include Condoleezza Rice and former CIA director George Tenet, who says in an interview that “I wish we had a tool of its power” before 9/11. General David Petraeus, the most recent former CIA chief, describes Palantir to FORBES as “a better mousetrap when a better mousetrap was needed” and calls Karp “sheer brilliant.”

And now Palantir is emerging from the shadow world of spies and special ops to take corporate America by storm. The same tools that can predict ambushes in Iraq are helping pharmaceutical firms analyze drug data. According to a former JPMorgan Chase staffer, they’ve saved the firm hundreds of millions of dollars by addressing issues from cyberfraud to distressed mortgages. A Palantir user at a bank can, in seconds, see connections between a Nigerian Internet protocol address, a proxy server somewhere within the U.S. and payments flowing out from a hijacked home equity line of credit, just as military customers piece together fingerprints on artillery shell fragments, location data, anonymous tips and social media to track down Afghani bombmakers.
Those tools have allowed Palantir’s T-shirted twentysomethings to woo customers away from the suits and ties of IBM, Booz Allen and Lockheed Martin with a product that deploys faster, offers cleaner results and often costs less than $1 million per installation–a fraction of the price its rivals can offer. Its commercial clients–whose identities it guards even more closely than those of its government customers–include Bank of America and News Corp. Private-sector deals now account for close to 60% of the company’s revenue, which FORBES estimates will hit $450 million this year, up from less than $300 million last year. Karp projects Palantir will sign a billion dollars in new, long-term contracts in 2014, a year that may also bring the company its first profits.
The bottom line: A CIA-funded firm run by an eccentric philosopher has become one of the most valuable private companies in tech, priced at between $5 billion and $8 billion in a round of funding the company is currently pursuing. Karp owns roughly a tenth of the firm–just less than its largest stakeholder, Peter Thiel, the PayPal and Facebook billionaire. (Other billionaire investors include Ken Langone and hedge fund titan Stanley Druckenmiller.) That puts Karp on course to become Silicon Valley’s latest billionaire–and Thiel could double his fortune–if the company goes public, a possibility Karp says Palantir is reluctantly considering.
The biggest problem for Palantir’s business may be just how well its software works: It helps its customers see too much. In the wake of NSA leaker Edward Snowden’s revelations of the agency’s mass surveillance, Palantir’s tools have come to represent privacy advocates’ greatest fears of data-mining technology — Google-level engineering applied directly to government spying. That combination of Big Brother and Big Data has come into focus just as Palantir is emerging as one of the fastest-growing startups in the Valley, threatening to contaminate its first public impressions and render the firm toxic in the eyes of customers and investors just when it needs them most.
“They’re in a scary business,” says Electronic Frontier Foundation attorney Lee Tien. ACLU analyst Jay Stanley has written that Palantir’s software could enable a “true totalitarian nightmare, monitoring the activities of innocent Americans on a mass scale.”
Karp, a social theory Ph.D., doesn’t dodge those concerns. He sees Palantir as the company that can rewrite the rules of the zero-sum game of privacy and security. “I didn’t sign up for the government to know when I smoke a joint or have an affair,” he acknowledges. In a company address he stated, “We have to find places that we protect away from government so that we can all be the unique and interesting and, in my case, somewhat deviant people we’d like to be.”
Palantir boasts of technical safeguards for privacy that go well beyond the legal requirements for most of its customers, as well as a team of “privacy and civil liberties engineers.” But it’s Karp himself who ultimately decides the company’s path. “He’s our conscience,” says senior engineer Ari Gesher.
The question looms, however, of whether business realities and competition will corrupt those warm and fuzzy ideals. When it comes to talking about industry rivals, Karp often sounds less like Palantir’s conscience than its id. He expressed his primary motivation in his July company address: to “kill or maim” competitors like IBM and Booz Allen. “I think of it like survival,” he said. “We beat the lame competition before they kill us.”
KARP SEEMS TO enjoy listing reasons he isn’t qualified for his job. “He doesn’t have a technical degree, he doesn’t have any cultural affiliation with the government or commercial areas, his parents are hippies,” he says, manically pacing around his office as he describes himself in the third person. “How could it be the case that this person is cofounder and CEO since 2005 and the company still exists?”
The answer dates back to Karp’s decades-long friendship with Peter Thiel, starting at Stanford Law School. The two both lived in the no-frills Crothers dorm and shared most of their classes during their first year, but held starkly opposite political views. Karp had grown up in Philadelphia, the son of an artist and a pediatrician who spent many of their weekends taking him to protests for labor rights and against “anything Reagan did,” he recalls. Thiel had already founded the staunchly libertarian Stanford Review during his time at the university as an undergrad.
“We would run into each other and go at it … like wild animals on the same path,” Karp says. “Basically I loved sparring with him.”
With no desire to practice law, Karp went on to study under Jurgen Habermas, one of the 20th century’s most prominent philosophers, at the University of Frankfurt. Not long after obtaining his doctorate, he received an inheritance from his grandfather, and began investing it in startups and stocks with surprising success. Some high-net-worth individuals heard that “this crazy dude was good at investing” and began to seek his services, he says. To manage their money he set up the London-based Caedmon Group, a reference to Karp’s middle name, the same as the first known English-language poet.

Back in Silicon Valley Thiel had cofounded PayPal and sold it to eBay in October 2002 for $1.5 billion. He went on to create a hedge fund called Clarium Capital but continued to found new companies: One would become Palantir, named by Thiel for the Palantiri seeing stones from J.R.R. Tolkien’s Lord of the Rings, orbs that allow the holder to gaze across vast distances to track friends and foes.
In a post-9/11 world Thiel wanted to sell those Palantiri-like powers to the growing national security complex: His concept for Palantir was to use the fraud-recognition software designed for PayPal to stop terrorist attacks. But from the beginning the libertarian saw Palantir as an antidote to–not a tool for–privacy violations in a society slipping into a vise of security. “It was a mission-oriented company,” says Thiel, who has personally invested $40 million in Palantir and today serves as its chairman. “I defined the problem as needing to reduce terrorism while preserving civil liberties.”
In 2004 Thiel teamed up with Joe Lonsdale and Stephen Cohen, two Stanford computer science grads, and PayPal engineer Nathan Gettings to code together a rough product. Initially they were bankrolled entirely by Thiel, and the young team struggled to get investors or potential customers to take them seriously. “How the hell do you get them to listen to 22-year-olds?” says Lonsdale. “We wanted someone to have a little more gray hair.”
Enter Karp, whose Krameresque brown curls, European wealth connections and Ph.D. masked his business inexperience. Despite his nonexistent tech background, the founders were struck by his ability to immediately grasp complex problems and translate them to nonengineers.
Lonsdale and Cohen quickly asked him to become acting CEO, and as they interviewed other candidates for the permanent job, none of the starched-collar Washington types or M.B.A.s they met impressed them. “They were asking questions about our diagnostic of the total available market,” says Karp, disdaining the B-school lingo. “We were talking about building the most important company in the world.”
While Karp attracted some early European angel investors, American venture capitalists seemed allergic to the company. According to Karp, Sequoia Chairman Michael Moritz doodled through an entire meeting. A Kleiner Perkins exec lectured the Palantir founders on the inevitable failure of their company for an hour and a half.
Palantir was rescued by a referral to In-Q-Tel, the CIA’s venture arm, which would make two rounds of investment totaling more than $2 million. (See our sidebar on In-Q-Tel’s greatest hits.) “They were clearly top-tier talent,” says former In-Q-Tel executive Harsh Patel. “The most impressive thing about the team was how focused they were on the problem … how humans would talk with data.”

That mission turned out to be vastly more difficult than any of the founders had imagined. PayPal had started with perfectly structured and organized information for its fraud analysis. Intelligence customers, by contrast, had mismatched collections of e-mails, recordings and spreadsheets.
To fulfill its privacy and security promises, Palantir needed to catalog and tag customers’ data to ensure that only users with the right credentials could access it. This need-to-know system meant classified information couldn’t be seen by those without proper clearances–and was also designed to prevent the misuse of sensitive personal data.
But Palantir’s central privacy and security protection would be what Karp calls, with his academic’s love of jargon, “the immutable log.” Everything a user does in Palantir creates a trail that can be audited. No Russian spy, jealous husband or Edward Snowden can use the tool’s abilities without leaving an indelible record of his or her actions.
From 2005 to 2008 the CIA was Palantir’s patron and only customer, alpha-testing and evaluating its software. But with Langley’s imprimatur, word of Palantir’s growing abilities spread, and the motley Californians began to bring in deals and recruits. The philosopher Karp turned out to have a unique ability to recognize and seduce star engineers. His colleagues were so flummoxed by his nose for technical talent that they once sent a pair of underwhelming applicants into a final interview with Karp as a test. He smelled both out immediately.
A unique Palantir culture began to form in Karp’s iconoclast image. Its Palo Alto headquarters, which it calls “the Shire” in reference to the homeland of Tolkien’s hobbits, features a conference room turned giant plastic ball pit and has floors littered with Nerf darts and dog hair. (Canines are welcome.) Staffers, most of whom choose to wear Palantir-branded apparel daily, spend so much time at the office that some leave their toothbrushes by the bathroom sinks.
Karp himself remains the most eccentric of Palantir’s eccentrics. The lifelong bachelor, who says that the notion of settling down and raising a family gives him “hives,” is known for his obsessive personality: He solves Rubik’s cubes in less than three minutes, swims and practices the meditative art of Qigong daily and has gone through aikido and jujitsu phases that involved putting cofounders in holds in the Shire’s hallways. A cabinet in his office is stocked with vitamins, 20 pairs of identical swimming goggles and hand sanitizer. And he addresses his staff using an internal video channel called KarpTube, speaking on wide-ranging subjects like greed, integrity and Marxism. “The only time I’m not thinking about Palantir,” he says, “is when I’m swimming, practicing Qigong or during sexual activity.”
In 2010 Palantir’s customers at the New York Police Department referred the company to JPMorgan, which would become its first commercial customer. A team of engineers rented a Tribeca loft, sleeping in bunk beds and working around the clock to help untangle the bank’s fraud problems. Soon they were given the task of unwinding its toxic mortgage portfolio. Today Palantir’s New York operation has expanded to a full, Batman-themed office known as Gotham, and its lucrative financial-services practice includes everything from predicting foreclosures to battling Chinese hackers.
As its customer base grew, however, cracks began to show in Palantir’s idealistic culture. In early 2011 e-mails emerged that showed a Palantir engineer had collaborated on a proposal to deal with a WikiLeaks threat to spill documents from Bank of America. The Palantir staffer had eagerly agreed in the e-mails to propose tracking and identifying the group’s donors, launching cyberattacks on WikiLeaks’ infrastructure and even threatening its sympathizers. When the scandal broke, Karp put the offending engineer on leave and issued a statement personally apologizing and pledging the company’s support of “progressive values and causes.” Outside counsel was retained to review the firm’s actions and policies and, after some deliberation, determined it was acceptable to rehire the offending employee, much to the scorn of the company’s critics.
Following the WikiLeaks incident, Palantir’s privacy and civil liberties team created an ethics hotline for engineers called the Batphone: Any engineer can use it to anonymously report to Palantir’s directors work on behalf of a customer they consider unethical. As the result of one Batphone communication, for instance, the company backed out of a job that involved analyzing information on public Facebook pages. Karp has also stated that Palantir turned down a chance to work with a tobacco firm, and overall the company walks away from as much as 20% of its possible revenue for ethical reasons. (It remains to be seen whether the company will be so picky if it becomes accountable to public shareholders and the demand for quarterly results.)
Still, according to former employees, Palantir has explored work in Saudi Arabia despite the staff’s misgivings about human rights abuses in the kingdom. And for all Karp’s emphasis on values, his apology for the WikiLeaks affair also doesn’t seem to have left much of an impression in his memory. In his address to Palantir engineers in July he sounded defiant: “We’ve never had a scandal that was really our fault.”
AT 4:07 P.M. ON NOV. 14, 2009 Michael Katz-Lacabe was parking his red Toyota Prius in the driveway of his home in the quiet Oakland suburb of San Leandro when a police car drove past. A license plate camera mounted on the squad car silently and routinely snapped a photo of the scene: his off-white, single-floor house, his wilted lawn and rosebushes, and his 5- and 8-year-old daughters jumping out of the car.
Katz-Lacabe, a gray-bearded and shaggy-haired member of the local school board, community activist and blogger, saw the photo only a year later: In 2010 he learned about the San Leandro Police Department’s automatic license plate readers, designed to constantly photograph and track the movements of every car in the city. He filed a public records request for any images that included either of his two cars. The police sent back 112 photos. He found the one of his children most disturbing.
“Who knows how many other people’s kids are captured in these images?” he asks. His concerns go beyond a mere sense of parental protection. “With this technology you can wind back the clock and see where everyone is, if they were parked at the house of someone other than their wife, a medical marijuana clinic, a Planned Parenthood center, a protest.”
As Katz-Lacabe dug deeper, he found that the millions of pictures collected by San Leandro’s license plate cameras are now passed on to the Northern California Regional Intelligence Center (NCRIC), one of 72 federally run intelligence fusion organizations set up after 9/11. That’s where the photos are analyzed using software built by a company just across San Francisco Bay: Palantir.
In the business proposal that Palantir sent NCRIC, it offered customer references that included the Los Angeles and New York police departments, boasting that it enabled searches of the NYPD’s 500 million plate photos in less than five seconds. Katz-Lacabe contacted Palantir about his privacy concerns, and the company responded by inviting him to its headquarters for a sit-down meeting. When he arrived at the Shire, a pair of employees gave him an hourlong presentation on Palantir’s vaunted safeguards: its access controls, immutable logs and the Batphone.
Katz-Lacabe wasn’t impressed. Palantir’s software, he points out, has no default time limits–all information remains searchable for as long as it’s stored on the customer’s servers. And its auditing function? “I don’t think it means a damn thing,” he says. “Logs aren’t useful unless someone is looking at them.”
When Karp hears Katz-Lacabe’s story, he quickly parries: Palantir’s software saves lives. “Here’s an actual use case,” he says and launches into the story of a pedophile driving a “beat-up Cadillac” who was arrested within an hour of assaulting a child, thanks to NYPD license plate cameras. “Because of the license-plate-reader data they gathered in our product, they pulled him off the street and saved human children lives.”
“If we as a democratic society believe that license plates in public trigger Fourth Amendment protections, our product can make sure you can’t cross that line,” he says, adding that there should be time limits on retaining such data. Until the law changes, though, Palantir will play within those rules. “In the real world where we work–which is never perfect–you have to have trade-offs.”
And what if Palantir’s audit logs–its central safeguard against abuse–are simply ignored? Karp responds that the logs are intended to be read by a third party. In the case of government agencies, he suggests an oversight body that reviews all surveillance–an institution that is purely theoretical at the moment. “Something like this will exist,” Karp insists. “Societies will build it, precisely because the alternative is letting terrorism happen or losing all our liberties.”
Palantir’s critics, unsurprisingly, aren’t reassured by Karp’s hypothetical court. Electronic Privacy Information Center activist Amie Stepanovich calls Palantir “naive” to expect the government to start policing its own use of technology. The Electronic Frontier Foundation’s Lee Tien derides Karp’s argument that privacy safeguards can be added to surveillance systems after the fact. “You should think about what to do with the toxic waste while you’re building the nuclear power plant,” he argues, “not some day in the future.”
Some former Palantir staffers say they felt equally concerned about the potential rights violations their work enabled. “You’re building something that could absolutely be used for malice. It would have been a nightmare if J. Edgar Hoover had these capabilities in his crusade against Martin Luther King,” says one former engineer. “One thing that really troubled me was the concern that something I contribute to could prevent an Arab Spring-style revolution.”
Despite Palantir’s lofty principles, says another former engineer, its day-to-day priorities are satisfying its police and intelligence customers: “Keeping good relations with law enforcement and ‘keeping the lights on’ bifurcate from the ideals.”
He goes on to argue that even Palantir’s founders don’t quite understand the Palantiri seeing stones in The Lord of the Rings . Tolkien’s orbs, he points out, didn’t actually give their holders honest insights. “The Palantiri distort the truth,” he says. And those who look into them, he adds, “only see what they want to see.”
DESPITE WHAT any critic says, it’s clear that Alex Karp does indeed value privacy–his own.
His office, decorated with cardboard effigies of himself built by Palantir staff and a Lego fortress on a coffee table, overlooks Palo Alto’s Alma Street through two-way mirrors. Each pane is fitted with a wired device resembling a white hockey puck. The gadgets, known as acoustic transducers, imperceptibly vibrate the glass with white noise to prevent eavesdropping techniques, such as bouncing lasers off windows to listen to conversations inside.
He’s reminiscing about a more carefree time in his life–years before Palantir–and has put down his Rubik’s cube to better gesticulate. “I had $40,000 in the bank, and no one knew who I was. I loved it. I loved it. I just loved it. I just loved it!” he says, his voice rising and his hands waving above his head. “I would walk around, go into skanky places in Berlin all night. I’d talk to whoever would talk to me, occasionally go home with people, as often as I could. I went to places where people were doing things, smoking things. I just loved it.”
“One of the things I find really hard and view as a massive drag … is that I’m losing my ability to be completely anonymous.”
It’s not easy for a man in Karp’s position to be a deviant in the modern world. And with tools like Palantir in the hands of the government, deviance may not be easy for the rest of us, either. With or without safeguards, the “complete anonymity” Karp savors may be a 20th-century luxury.
Karp lowers his arms, and the enthusiasm drains from his voice: “I have to get over this.”

What New Power Looks Like

What new power looks like. Aivars Lode avantce

Old power = Individually managing enterprise products and services

New power = Collectively managing source to consumer experiences